Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Denver Public Schools: Out-of-School Time Evaluation SY 2013-2014
Year Published: 2014
This evaluation examined 17 afterschool programs in the Denver Public School system that were funded in part by Colorado’s 21st Century Community Learning Center (21st CCLC) grant. It found that students who were new to the program and participated more than 30 days (first-year attendees) and students who attended more than 30 days multiple years in a row (multi-year attendees) saw improved school day attendance and believed that the program helped them perform better academically. Students regularly participating in the program also reported positively when asked about their social competency, their ability to plan for the future and their future expectations. The evaluation also found that multi-year attendees outperformed their non-participating peers when examining performance growth on the state’s academic assessments; however, there was little difference found when comparing students’ proficiency levels.
Program Description: Denver Public Schools provide afterschool school and summer learning programs to more than 5,000 kindergarten through 12th grade students each year. The programs provide academically enriching activities, arts and cultural activities, and physical activity; promote youth leadership, healthy eating, and parent engagement; and offer wrap-around family support services to the children and families they serve.
Scope of the Evaluation: Local
Program Type: Afterschool
Location: Denver, CO
Community Type: Urban
Grade level: Elementary School, Middle School, High School
Program Demographics: Among the regular afterschool program participants, 91 percent qualified for free or reduced price lunch, 12 percent had disabilities or special needs, and 42 percent were English language learners. Half of regular participants were female. Seventy-three percent of participants were Hispanic, 12 percent were African American, 8 percent were white, three percent were Asian, two percent were biracial, and one percent were American Indian/Alaska Native.
Evaluator: Read, D. M.
Evaluation Methods: In this comparison analysis, academic and youth development outcomes of 2,171 regular first-year attendees (participants who were new to the program and participated a minimum of 30 days) and 799 multi-year attendees (participants who attended more than 30 days multiple years in a row) were compared with a control group. Academic indicators gathered from Transitional Colorado Assessment Program (TCAP) data included school-day attendance and academic performance and growth in reading, writing and math. Outcomes related to youth development were also collected using the Survey of Academic and Youth Outcomes – Youth (SAYO-Y), measuring program attendees’ sense of competence, program experience, and future planning and expectations.
Evaluation Type: Quasi-experimental
Summary of Outcomes: The evaluation examined 17 afterschool programs in the Denver Public School system and found that students who were new to the program and participated more than 30 days (first-year attendees) and students who attended more than 30 days multiple years in a row (multi-year attendees) saw improved school day attendance and believed that the program helped them perform better academically. Students regularly participating in the program also reported positively when asked about their social competency, their ability to plan for the future and their future expectations. Multi-year attendees outperformed their non-participating peers when examining growth on the state’s academic assessments; however, there was little difference when comparing students’ proficiency levels.
In the 2013-2014 school year, both first-year students regularly attending a Denver Public School afterschool program and regularly attending multi-year students had statistically significantly higher average attendance rates than their non-participating peers. The evaluation calculated that the higher attendance rates translated to first-year attendees receiving approximately two additional days of instruction and multi-year attendees an additional 3.5 days compared to the control group. First-year attendees and the control group had the same average attendance rate in 2012-2013.
Looking at the growth in Transitional Colorado Assessment Program (TCAP) math, reading and writing, multi-year attendees performed better than both the control group and first-year attendees, which the author stated “shows a differential benefit for attendees who regularly participate in programming for at least two consecutive years,” although the finding was not statistically significant. The evaluation also found that multi-year attendees had statistically significantly higher scores in TCAP writing. However, there was little difference in proficiency on the TCAP math, reading and writing between first-year attendees, multi-year attendees and the control group.
When surveyed about their participation in the program as it related to academic support, approximately two-thirds of regularly attending program participants believed that the program helped them to do better in math (64 percent), reading (65 percent) and writing (64 percent). Both first-year and multi-year attendees on average mostly agreed that they were strong learners—which included rating themselves highly in problem solving, trying new things and persistence—although multi-year attendees rated themselves higher than first-year attendees.
Assessing social competency and the ability to plan for their future, students regularly attending the program mostly agreed that they were able to get along with their peers and take steps that would help them to reach future goals. Again, students regularly attending the program for multiple years rated themselves higher on the social competency questions, such as finding it easy to get along with other students and join new groups of students, and the future-oriented questions, such as goal setting and trying hard in school, than students regularly attending the program for the first time. Both groups of regularly attending student participants also reported high expectations for their future academic success—such as being successful in high school, graduating from high school and going to college, with multi-year attendees reporting slightly higher expectations than first-year attendees.
Associated Evaluation: https://www.denvergov.org/Portals/713/documents/2013-2014%20EVALUATION%20FINAL.pdf
