Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Comprehensive Support and Student Success: Can Out Of School Time Make a Difference?
Year Published: 2023
This evaluation finds that when comparing students who gained admittance through a lottery to StudentU – a comprehensive afterschool and summer program for middle and high school students – to students who were not selected through the lottery, StudentU participants who entered the program with low baseline achievement accumulated more course credits, experienced the greatest gains in GPA, and were significantly less likely to be suspended than their peers who were not accepted through the lottery. The author reports that these outcomes “suggest that comprehensive services provided outside of the regular school may be a particularly effective strategy for improving outcomes of the most disadvantaged students.” Additionally, the author predicts that StudentU participants have an estimated 4 percentage point higher likelihood of graduating from high school than their non-participating peers.
Program Name: StudentU
Program Description:
StudentU is a nonprofit that provides wraparound supports, including academic programming, nutrition assistance, mentoring, and referrals for participants and their families to community services, to low-income or first-generation college students in Durham, North Carolina, free of charge. The services are delivered through afterschool and summer programming where public, charter, and private school students may apply as rising 6th graders and remain until high school graduation. Due to the high number of applicants, participants are chosen through a lottery system
Scope of the Evaluation: Local
Program Type: Afterschool
Community Type: Rural, Urban
Grade level: Middle School, High School
Program Website: https://studentudurham.org/
Evaluator: Komisarow, S., Duke University.
Evaluation Methods:
Data in this study comes from StudentU applications and lottery records, as well as data from the North Carolina Education Research Data Center (NCERDC). NCERDC data provided information including students’ course credits, grade point average, math and reading test scores, and disciplinary infractions. Using a matching procedure, StudentU application and lottery records were matched to existing NCERDC data in order to compare academic and behavioral outcomes for 324 lottery winners and losers pre-lottery (fifth grade) and post-lottery (ninth grade). The author used a two-stage least-squares approach to estimate the effect of participation in StudentU on student outcomes.
Evaluation Type: Experimental
Summary of Outcomes:
This evaluation finds that when comparing students who gained admittance through a lottery to StudentU – a comprehensive afterschool and summer program for middle and high school students – to students who were not accepted through its lottery, StudentU participants who entered the program with low baseline achievement upon entering accumulated more course credits, experienced the greatest gains in GPA, and were significantly less likely to be suspended than their peers who were not accepted through the lottery.
Regarding course credits, StudentU participants achieved around 0.45 more course credits by the end of ninth grade than those who did not participate in the afterschool and summer program–translating into a 6 percent increase compared to non-participants. These results were particularly significant for lottery winners who had a lower standardized math and reading test score average in fifth grade, or low baseline achievement, who accumulated around 0.822 more course credits by the end of ninth grade.
Similarly, regarding improvements in GPA at the end of ninth grade, students with low baseline achievement experienced a statistically significant gain of 0.370 weighted grade points. However, results for the full sample of lottery winners and lottery winners with high baseline achievement were statistically insignificant.
When assessing suspension rates, lottery winners were 10.3 percentage points (or 47 percent) less likely to be suspended during ninth grade than lottery losers. This rate was more significant for lottery winners with low baseline achievement, who were 17.1 percentage points less likely to be suspended in ninth grade.
The results suggest that students who entered StudentU with low baseline achievement experienced the most significant improvements in both academic performance and behavior as a result of the programming. When considering the intervening variables that attributed to these student outcomes, the author found that StudentU lottery winners were 2 percentage points more likely to make on-time grade progress, 8.6 percentage points less likely to fail a course in ninth grade, and 10.3 percentage points less likely to have disciplinary infractions in ninth grade.
Additionally, the author estimated the effect of participating in StudentU on a student's likelihood of graduating from high school, predicting that students who won the lottery to participate in StudentU would be 4 percentage points more likely to graduate from high school than their peers who were not accepted into StudentU.
