Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
21st Century Community Learning Centers: 2019-2020 Program Year Statewide Evaluation (Colorado)
Year Published: 2021
This evaluation of Colorado’s 21st Century Community Learning Center (21st CCLC) programs found that, based on teachers surveyed, students regularly participating in programs made academic and behavioral gains, with regular participants who attended both the fall and spring sessions even more likely to make improvements. Among regularly attending students in need of improvement, more than 7 in 10 improved their academic performance (73 percent) and class participation (73 percent), and approximately 2 in 3 students improved their motivation to learn (65 percent), class attentiveness (67 percent), and satisfactory homework completion (65 percent). Students who attended 21st CCLC programs in the fall and spring were more likely to make gains compared to students who attended for part of the year. For example, 74 percent of students who attended in both the fall and spring improved their academic performance, while 66 percent of students who attended in eithe
Program Name: Colorado 21st Century Community Learning Centers
Program Description:
Colorado’s 21st CCLC programming receives federal funding through the 21st CCLC initiative and provides afterschool and summer academic enrichment opportunities for children at high-poverty and low-performing schools throughout the state. This evaluation included data from grantees in the 2015-2020 cohort as well as the 2018-2021 cohort. In total, 61 grantees provided programming in 106 centers across the state, serving 19,401 students.
Scope of the Evaluation: Statewide
Program Type: Summer, Afterschool, Before school
Community Type: Rural, Urban, Suburban
Grade level: Elementary School, Middle School, High School
Program Demographics:
During the 2019-2020 program year, 49 percent of participants identified as female, and 51 percent identified as male. Regarding race and ethnicity, 59 percent identified as White, 56 percent of participants identified as Hispanic, 7 percent identified as Black or African American, 6 percent of participants identified as American Indian or Native Alaskan, 5 percent identified as Asian, 3 percent identified as Multi-Racial, and 20 percent identified as unknown or some other race.
Program Website: https://www.cde.state.co.us/21stcclc
Evaluator: Catherine Roller White Consulting
Evaluation Methods:
This evaluation conducted program and teacher surveys to collect data, as well as accessed EZReports. The online survey of program sites conducted at the end of the program year asked questions about family-school partnerships, progress toward reaching state performance measures, enrollment and participation rates, and program sustainability, while the teacher survey assessed changes in student behavior among students attending 21st CCLC programs 30 days or more (regular attendees). EZReports were used to collect data on student attendance data and demographics, activities offered, and staffing and partner details. Evaluators conducted analyses of variance (ANOVA) to compare outcomes between groups and accurately report results.
Evaluation Type: Non-experimental
Summary of Outcomes:
This evaluation of Colorado’s 21st CCLC programs during the 2019-2020 school year found during the coronavirus pandemic, students regularly participating in programs (attending the program 30 days or more) made academic and behavioral gains. Regular participants attending both the fall and spring sessions were most likely to make improvements. Additionally, the evaluation found that programs reported positively in the areas of family-school partnerships, progress toward reaching state performance measures, and implementing a quality improvement rubric that asked about areas such as personnel and leadership, process, and sustainability.
Related to student outcomes, among students with room for improvement, more than 7 in 10 showed growth in participating in class (73 percent) and academic performance (73 percent). A majority of students made gains in being attentive in class (67 percent), coming to school motivated to learn (65 percent), motivation (65 percent), satisfactorily completing homework (65 percent), and getting along with others (63 percent). Nearly half of attendees (48 percent) improved attending class regularly.
Students who attended programming in the fall and the spring made bigger improvements than students who attended either the fall or spring in 8 out of the 10 items included in the teacher survey. For example, 74 percent of students who attended in both the fall and spring saw gains in their academic performance, while 66 percent of students who attended in either the fall or spring saw gains in their academic performance.
Based on program sites’ end-of-year reporting, three-quarters of grantees (75 percent) reported engaging in effective communication with families, 66 percent shared that they frequently welcomed all families, and 48 percent frequently collaborated with their community. When measuring their progress toward meeting state goals, although the COVID-19 pandemic meant that many subgrantees did not have data available on these measures, those who did provide data shared that they were “making progress, meeting their goal, or exceeding their goal” on their performance measures. For example, most Cohort VII programs were making progress on their enrichment and parent/family activities performance measures, while most Cohort VIII programs were making progress on their school attendance and family engagement measures.
Additionally, subgrantees were required to report on how well they were implementing their programming and how they were working to improve in the following seven areas “personnel/leadership indicators, process indicators, evidence-based programs and practices, clear linkages, quality improvement feedback, congruency, and sustainability.” A majority of subgrantees reported that they were either meeting or exceeding expectations in all seven of these areas.
The evaluation also included anecdotes from 21st CCLC programs about the impacts the COVID-19 pandemic had on their programming. Programs described providing basic needs, such as meals and health and safety information about the pandemic, as well as sending families weekly supplies for online school or program activities, helping students access technology and the internet, connecting with students who were having a difficult time with remote learning, and some programs were able to offer socially distanced in-person programming to ensure students were able to receive the services they needed during the pandemic.
Date Added: February 12, 2024
