A project of the Afterschool Alliance.

Evidence of Program Quality and Youth Outcomes in the DYCD Out-of-School Time Initiative: Report on the Initiative’s First Three Years

Year Published: 2009

A multi-year evaluation of New York City’s Out-of-School Time (OST) initiative, which was launched in 2005 to provide free, quality afterschool and summer learning programs to New York City students and served 181,000 children between 2005 and 2008. The evaluation found that participants in the program reported high levels of academic motivation and moderate levels of academic benefits. Additional findings include that students in the program reported that they felt that they belonged and were safe in the program, and among parents surveyed, 74 percent agreed that the program made it easier for them to keep their jobs and 73 percent agreed that they missed less work than they had previously because their children attended the OST program.

Program Name: New York City’s Out-of-School Time (OST) initiative

Program Description: The New York City Department of Youth and Community Development (DYCD) launched the Out-of-School (OST) Time Programs for Youth Initiative in 2005. The goal of this initiative was to provide young people with free, high-quality out-of-school time opportunities after school, on holidays, and during the summer. The initiative served more than 181,000 youth throughout New York City between 2005 and 2008 and served more than 81,000 across 622 programs during the 2007-08 academic year alone.

Scope of the Evaluation: Local

Program Type: Afterschool

Location: New York City, NY

Community Type: Urban

Grade level: Elementary School, Middle School

Program Demographics: Fifty-one percent of participants were female, 39 percent were African-American, 38 percent were Hispanic, nine percent were Asian, seven percent were white, one percent were Native American, and six percent identified as “other.” Eighty-four percent were eligible for free or reduced-price lunch, 20 percent were English language learners, and 16 percent had special needs or disabilities.

Evaluator: Russell, C. A., Mielke, M. B. & Reisner, E. R. Policy Studies Associates, Inc.

Evaluation Methods: Researchers used parent and student surveys, site observations, and school attendance rate and academic performance measures to evaluate the impact of New York City’s Out-of-School Time (OST) initiative. This included academic motivation and academic benefits as reported by youth, performance on the state English language arts (ELA) and mathematics tests (for grades 3-8), and credits accrued and Regent exams passed for high school students. Academic impacts of the program were measured based on youth survey responses related to academic motivation and academic performance, school attendance rates, and standardized test scores on ELA and math tests (for elementary and middle school students) and Regents exams (for high school students).

Evaluation Type: Quasi-experimental

Summary of Outcomes: The study examined a number of youth outcomes of students participating in New York City’s Out-of-School Time (OST) initiative, including academics and safety, as well as parents’ attitudes about the program. Regarding academics, participants’ average score for the program helping them academically—which included asking students if the program helped them with reading and writing, math, and finishing homework—was 3.06 out of 4, with 4 representing that students strongly agreed with the statement. The academic benefit most reported by students was helping them to complete homework—more than half of participants “agreed a lot” that their program helped them with homework completion.

Participants overall reported high levels of academic motivation—as measured by questions about enjoying school, trying hard in school, paying attention in class, and doing well in school—with an average score of 3.34 out of 4, with 4 representing that students strongly agreed with the statement. Elementary school participants reported higher levels of academic motivation than middle or high school participants. Although students in the program saw small gains in their math and English language arts standardized test scores, there were no significant differences between gains made by participants and those made by matched non-participants. Among high school program participants, there was no significant difference in the number of credits they accrued each year as compared to students who did not participate in the program.

The study also found that based on student participant surveys, students felt that they belonged in the program and felt safe in the program. Overall, participants reported a strong sense of belonging in their program, with an average youth-survey scale score of 3.38 out of 4, and more than two thirds of participants “agreed a lot” that they felt safe in the program. High school students reported the strongest sense of program connection (3.48 out of 4). Looking at school day attendance, students participating in the OST initiative had better attendance than non-participants at the start of the evaluation and continued to have a higher school day attendance rate than non-participants throughout the study. Over the course of the study, there were not significant changes in attendance rates between participants and non-participants.

Parents of students in the program noted in surveys that the OST programs allowed them to work more or pursue more education. Across all responding parents, 74 percent agreed that the program made it easier for them to keep their jobs and 73 percent agreed that they missed less work than they had previously because their children attended the OST program. In addition, 71 percent of parents reported that they were able to work more hours.

Associated Evaluation: http://www.policystudies.com/_policystudies.com/files/OST_Evaluation_Report.pdf

Date Added: January 19, 2017