Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
High Quality Supplemental Educational Services (HQSES) and Afterschool Partnerships Demonstration Project
Year Published: 2010
An external analysis of student test scores, attendance and assessment records in 39 THINK Together afterschool sites in the Santa Ana Unified School District. The evaluation found that participating in the program had a positive impact on students’ math and reading skills, as measured through THINK Together assessments, as well as on students’ perceived math efficacy. It also found that frequent participation in the program had a positive effect on academics, where those students with higher levels of participation in the program saw greater gains on their California Standardized Test (CST) scores in reading and math and their California English Language Development Test (CELDT) scores compared to students who did not participate in the program.
Program Name: THINK Together
Program Description: Since 1994, THINK Together has been providing afterschool programming that creates opportunities for Southern California’s under-resourced elementary, middle, and high school children and youth to improve their academic outcomes, discover their passions, and reach their full potential. THINK Together is the primary provider for the Santa Ana Unified School District’s 21st Century Community Learning Centers. During the 2009-2010 school year, THINK Together provided afterschool programming at 225 public schools.
Scope of the Evaluation: Local
Program Type: Afterschool
Location: Santa Ana, CA
Community Type: Urban
Grade level: Elementary School, Middle School, High School
Program Demographics: Ninety-eight percent of participants were Hispanic, 62 percent were eligible for the federal Free or Reduce Price Lunch program, and 88 percent were classified as English language learners.
Program Website: https://thinktogether.org/
Evaluator: Vandell, D. L., O’Cadiz, P., & Hall, V. Department of Education, University of California, Irvine.
Evaluation Methods: In this study, four High Quality Supplemental Educational Services (HQSES) performance indicators were evaluated: “1) The number of students who enroll in THINK Together SES; 2) The number of eligible students who complete full programs; 3) The percentage of enrolled students, especially the lowest-achieving students, who improve their academic performance on CSTs in Language Arts or Math; and 4) the number of positive, supportive relationships students report having with adults and peers.” These four indicators, collected from 31 school sites, were measured using qualitative and quantitative data including: student program enrollment, attendance and completion data; standardized test scores; parent and student surveys; program observations; and interviews with parents, tutors, site coordinators, and program administrations.
Evaluation Type: Quasi-experimental
Summary of Outcomes: Pre- and post-surveys of students in the THINK program indicated that students’ math efficacy saw a significant increase due to programming. Looking at the program’s impact on students’ California Standardized Test (CST) scores, a strong majority of students in the SES program either achieved or maintained a CST proficiency level of “Basic” or above on their math (76 percent) and English language arts (ELA) (63 percent) tests. More than one-quarter of students in the program (26 percent) improved from “Below Basic” to “Proficient or Above” on the CST ELA and 13 percent improved on the math CST from “Below Basic” to “Proficient or Above.”
Additionally, the report found that increased rates of participation in the program had a significant positive impact on a variety of academic outcomes. Students who had high levels of participation in the program, or those students attending 25 to 26 sessions, saw greater gains in their reading efficacy and THINK math and reading test assessment scores than students with lower levels of program participation, those students attending the program less than 22 session. Students’ California English Language Development Test (CELDT) reading scores also increased as their participation in the program increased. The report also found that students who attended more SES sessions had a significant positive increase in their CELDT scores overall, as well as their CST ELA and math scores in comparison with the Santa Ana Unified School District population as a whole. Students who attended THINK for two academic years had consistently greater increases in all their CST and CELDT test scores than those who did not attend.
Associated Evaluation: http://faculty.sites.uci.edu/childcare/files/2013/07/HQSES-Year-Two-Report_2009-10.pdf
Date Added: September 27, 2016
