Policy Studies Associates

Afterschool programs and initiatives



New York City Department of Youth and Community Development

Since 2005, the Out-of-School Time (OST) Programs for Youth initiative (now COMPASS) of the New York City Department of Youth and Community Development (DYCD) has provided access to OST services through one of the largest afterschool systems in the country. For the first three years of the initiative (2005-06 through 2007-08), PSA’s evaluation examined the implementation and operations of these OST programs, and analyzed participants' educational and social development in light of the duration and intensity of their OST participation and the features of their OST programs. In later phases, PSA evaluated DYCD's efforts to improve OST program quality, including:

  • In-depth studies of strategies to support program quality in OST programs serving elementary- and middle-grades youth, including studies of the implementation of skill-based programming, social-emotional learning, literacy and STEM activities, and of the role of Education Specialists in continuous program improvement.

  • Evaluation of the Transition to High School programs, designed to facilitate the successful transition into high school for students entering the ninth grade.

  • The development of a program-quality measurement tool to be used by DYCD staff in monitoring the quality of its programs, and training of agency staff on the use of this tool for coaching and assessing programs.

  • Evaluation of a pilot initiative to build the capacity of OST provider organizations through intensive technical assistance focused on fiscal management, human resources management, and program quality.

  • A 10 year retrospective analysis of participation

Strategies for Social-Emotional Learning in COMPASS Programs
Longitudinal Analyses of the DYCD Transition to High School Program: Cohorts 1 and 2
Promising Practices in Planning and Implementing COMPASS Literacy Programming
Retrospective Analysis of COMPASS Participation, 2004-2015
Role of the Education Specialist in Supporting OST Program Quality
Lessons from the Field: DYCD OST Summer Programming
Supporting STEM Activities in Out-of-School Time Programs
Principal Perspectives on DYCD Out-of-School Time Programs
Building the Capacity of Nonprofits to Support High-Quality Youth Programs
Approaches for Integrating Skill-Based Activities in Out-of-School Time Programs
Achieving Program Quality in the DYCD Out-of-School Time Initiative: Strategies from 15 Programs
Evidence of Program Quality and Youth Outcomes in the DYCD Out-of-School Time Initiative

PSA OST Observation Instrument

PSA designed the Out-of-School Time (OST) Observation Instrument to collect consistent and objective observation data about the quality of afterschool activities. Grounded in a youth development framework, the PSA OST Observation Instrument is based on two core assumptions about the features of high-quality afterschool programs: (1) certain structural and institutional features (adequate qualified staff, adequate space and resources, and the availability of varied opportunities for academic and interpersonal skill-building) support the implementation of high-quality programs; and (2) instructional activities promote positive youth outcomes if they offer varied content, mastery-oriented instruction, and positive youth-to-youth and youth-to-staff relationships. PSA collected data on the psychometric properties of the observation instrument from studies in which the instrument has been used. Evidence from several of these studies has been reported in a report of the instrument's reliability and validity. The PSA OST Observation Instrument is available for use by researchers and practitioners. Please contact Christina Russell at crussell@policystudies.com or 202-939-5340 with any questions.

OST Observation Instrument: Report of the Validation Study
OST Observation Instrument-Procedures
OST Observation Instrument-Cover Sheet