Evaluator: Huang, D. & Wang, J. University of California, Los Angeles, Center for Research on Evaluation, Standards, and Student Testing (CRESST).
Evaluation Methods: The evaluation’s multi-method approach combined qualitative and quantitative research methodologies, including longitudinal administrative data collected by the California Department of Education and school districts, as well as data collected by the evaluation team. Data, which were collected between 2006 and 2011, included student, parent and staff surveys; student and program staff focus groups; site observations; school day attendance; classroom behavior reports; and scores from standardized tests, such as the California High School Exit Examination (CAHSEE).
Evaluation Type: Quasi-experimental
Summary of Outcomes: The evaluation found that overall, students who participated in the After School Safety and Enrichment for Teens (ASSETs) programs performed better than their non-participating peers when looking at academics, school day attendance, behavior and physical fitness.
Examining academic outcomes, the evaluation reports that across the three school years that data were collected (2007-2008, 2008-2009, and 2009-2010), students who participated in the ASSETs programs frequently received higher passing rates and scores on the English language arts (ELA) and math sections of the California High School Exit Examination (CAHSEE) when compared to non-participants. Among students frequently participating in ASSETs programs, the difference was statistically significant on both CAHSEE sections across the three school years. For the purposes of this evaluation, “frequent participation” was defined as attending the program for at least 15 days per year.
A longitudinal analysis of the data found that ASSETs program participants were generally less likely than non-participants to drop out of school and were predicted to graduate at a higher rate than non-participants. The evaluation’s longitudinal findings also reported that while there was a small positive effect on program participants’ ELA California Standards Test (CST) scores compared to their non-participating peers, there was a small to no effect found on math CST scores and ELA and math CAHSEE pass rates.
The evaluation found that English language learner (ELL) students participating in the ASSETs programs outperformed their non-participating counterparts on the California English Language Development Test (CELDT), with ELL students who frequently participated in the programs receiving the highest average CELDT score. A longitudinal analysis found that ELL students participating in the program were slightly more likely than their non-participating peers to be reclassified as a fluent English speaker.
Regarding behavior, the evaluation found a statistically significant positive effect on ASSETs program frequent participants’ school day attendance in 2007-2008 and 2008-2009, and a small statistically significant positive effect in 2009-2010. Students who frequently attended the program were also less likely than non-participants to be suspended from school across all three observed school years. The evaluation estimated the effect of participation in the program as it related to school suspensions, finding that frequent participation in the ASSETs programs reduced students’ chances of suspension from school.
Students’ physical fitness was another area that the evaluation assessed, reporting that among ninth graders (the only grade level for which fitness information was collected), ASSETs program participants performed slightly better on average on the physical fitness assessment than non-participants. Additionally, frequent participants in the ASSETs programs had a higher passing rate than both non-participants and participants overall across the six benchmarks included in the physical fitness assessment (aerobic capacity, body composition, abdominal strength, trunk strength, upper body strength and flexibility). In 2009-2010, frequent participation was “associated with a statistically significant increase in the likelihood” of meeting the aerobic capacity, abdominal strength, trunk strength and upper body strength benchmarks when compared to non-participants.
A breakdown of outcomes based on ASSETs programs’ community type found that a majority of urban and rural sites reported meeting or making progress toward their goals associated with: behavioral change (95 percent of urban sites and 81 percent of rural sites), homework completion (91 percent of urban sites and 69 percent of rural sites), school day attendance (88 percent of urban sites and 71 percent of rural sites), and academic improvement (63 percent of urban sites and 61 percent of rural sites).