Learning to Improve: Early Evaluation of the First2 Network


  • Caitlin Howley ICF




Program evaluation, STEM persistence, Continuous improvement, Improvement science, Collective impact, Lived experience


The first two years of an undergraduate student’s STEM program of study are particularly treacherous, with roughly half of STEM majors switching to other fields or dropping out entirely after their sophomore year. Moreover, attending a rural high school that was unable to offer higher-level math courses and being a first-generation college student further increase the likelihood of STEM attrition.

   This presentation will feature evaluation results from the First2 Network’s first two years of operation. Funded by the National Science Foundation, First2 is a collective impact effort to improve STEM persistence among rural, first-generation college students in West Virginia. With students themselves as co-creators of solutions to problems of persistence, core Network activities include iterative testing of improvements to practice, development of a statewide network to support student success, and capacity building to ensure sustainability.

   Study methods include surveys and interviews, observation of Network activities, document review, social network analysis, and tracking of student progress. Key findings suggest rapid Network expansion, establishment of core processes to support workgroups, provision of Network-wide training in improvement science, and successful facilitation of summer immersive research experiences for targeted students (in-person in 2019 and virtually in 2020). Early outcomes include statistically significant improvements in students’ STEM efficacy and STEM identity following summer immersive experiences, and improved survey scores indicating progress through iterative testing cycles. Early persistence data from participating students is less promising. But since the purpose of the Network is to learn how to help students persist, members continue their efforts to improve.

Author Biography

Caitlin Howley, ICF

Dr. Caitlin Howley has nearly 25  of experience conducting program evaluation and education research. A Director at ICF, her specialities include rural education; science, technology, engineering, and math (STEM) education; youth engagement; and organizational and evaluation capacity building.





How to Cite

Howley, C. (2021). Learning to Improve: Early Evaluation of the First2 Network. Proceedings of the West Virginia Academy of Science, 93(1). https://doi.org/10.55632/pwvas.v93i1.837