My last post offered one perspective on the effects of charter schools in changing the lives of children in Harlem. Viewers of the video featured in the last post might, on the basis of the outcomes observed in the Harlem schools, believe Charter Schools are the solution to our education crisis. Today, I direct your attention to a new report from the National Center for Education Evaluation (NCEE). Readers of the NCEE report might believe that Charter Schools are generally ineffective at improving student outcomes. These two posts clearly demonstrate the danger of making important decisions based upon single perspectives. Neither is fictitious, neither is complete. Both offer valuable information for understanding what works and for whom it works. Both demonstrate the need to know more about this complex issue.

The Evaluation of Charter School Impacts, reports on a large empirical study of charter school outcomes. Here is an extract of key findings from the executive summary on the report:

• On average, charter middle schools that hold lotteries are neither more nor less successful than traditional public schools in improving student achievement, behavior, and school progress. Participating schools had no significant impacts on math or reading test scores either a year or two years after students applied, other measures of academic progress (such as attendance or grade promotion), or student conduct within or outside of school. Being admitted to a study charter school did significantly and consistently improve both students’ and parents’ satisfaction with school.

• The impact of charter middle schools on student achievement varies significantly across schools. Across 28 sites (covering 32 schools), the effects on reading scores after two years were estimated to be greater than zero in 11 sites and less than zero in 17 sites (with magnitudes ranging from -0.43 to +0.33 standard deviation units), with 4 of the individual site estimates statistically significant. The estimated effects on math scores were greater than zero in 10 sites and less than zero in 18 of the 28 sites (-0.78 to +0.65 standard deviation units), with 10 of the site estimates statistically significant.

• In our exploratory analysis, for example, we found that study charter schools serving more low income or low achieving students had statistically significant positive effects on math test scores, while charter schools serving more advantaged students—those with higher income and prior achievement—had significant negative effects on math test scores. Charter middle schools in large urban areas also had significant positive impacts on math achievement compared to negative impacts in other locales, although urbanicity was no longer an influential factor once such characteristics as students’ demographics and income levels were controlled for. There were also differential effects on reading achievement, with negative and significant impacts for study charter schools serving more advantaged students and no impacts for study charter schools serving fewer advantaged students.

• Some operational features of charter middle schools are associated with more positive (or less negative) impacts on achievement. These features include smaller enrollments and the use of ability grouping in math or English classes. Although impacts differed for study charter schools with longer- versus shorter- hours of operations or higher versus lower revenue per student, these features were no longer significant once other school and student characteristics were controlled for. We found no statistically significant relationships between achievement impacts and the charter schools’ policy environment, including the extent of its decision-making autonomy, the type of authorizer and how the authorizer held the school accountable, and whether it was operated by a private organization.

Gleason, P., Clark, M., Tuttle, C. C., and Dwoyer, E. (2010). The Evaluation of Charter School Impacts: Executive Summary (NCEE 2010-4030). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.