A recent analysis from Education Week carried a grim headline: “In Many Charter High Schools, Graduation Odds Are Slim.” Its core finding, “Charter high schools make up an outsized share of the number of public schools persistently graduating less than half of their students,” led the authors to question why charter high school graduation rates lag so far behind other public schools.
Unfortunately, the analysis represents a big step backward in the quality of research on charter schools. It compares graduation rates in charter schools, which are concentrated in urban areas where students in public schools have low graduation rates, with schools nationwide—including schools that serve more affluent neighborhoods, suburbs and towns, where students are much more likely to graduate.
Students attending charter schools flagged by the Education Week analysis might actually be more likely to graduate than if they attended an assigned neighborhood school. The analysis elides this possibility, because it does not compare apples with apples. That is the core principle of any assessment of school effectiveness.
A seminal dust-up about how to study charter schools occurred in 2004. That year, major newspapers gave front page coverage to a union-sponsored study saying charter school test scores were lower than the nationwide average, and drawing conclusions about effectiveness. A group of scholars led by Diane Ravitch attacked the study as invalid because its results were based on a biased method that novice education researchers are trained to avoid. As Education Week reported at the time, once researchers controlled their results for students’ racial and economic backgrounds, charter schools looked no worse than comparable public schools.
The Center on Reinventing Public Education then convened a consensus panel of researchers, which produced a book and reporters’ guide on how to validly study charter schools. These explained the importance of the counterfactual in research on school outcomes. A child can’t both attend and not attend a school at the same time, so researchers can’t observe effects directly. The consensus panel, and researchers in general, agreed that the most valid comparisons are with the district-run schools from whom charter students are drawn, or with the charter students’ own achievement levels before entering the school being studied—or best, with students who applied to charter school admissions lotteries but lost in a random drawing.
The aftermath of the dust-up was a major increase in the quality of charter school studies. Some researchers pioneered new techniques, like the “virtual twin” approach employed by Stanford University’s Center for Research on Education Outcomes (CREDO) to produce fairer statistical comparisons between charter school students and their peers in district-run public schools. Virtually nobody produces—and few believe—studies that compare the outcomes of charter schools and all the schools in the country, state, or metro area.
Some of the best research has been done on charter high schools, comparing charter attendees’ graduation rates, college attendance, and other outcomes with those of similar students in the district-run schools they would otherwise have attended. These studies, which rigorously compare charter and non-charter students who are alike when entering high school, find neutral to strong positive charter school effects on graduation and college attendance. A comprehensive review can be found here. The fact that these results are very different from what Education Week has just reported illustrates the importance of making the right comparison.
That said, one of the conclusions of the new report is rock solid. The graduation rates of schools, both charter and district-run, that serve high concentrations of low-income, black and Hispanic children are far too low. Kids need to leave elementary schools better prepared for high school, and high schools need to do a far better job supporting students and connecting them to adult life and the real economy. Today’s children and their communities will suffer if academic and job opportunities remain blocked.
Charter schools are making a positive difference, but it is not enough. Based on their performance, and recent improvements led by KIPP and other charter management organizations, charter schools might lead the way toward needed broader changes in public education. Studies that falsely paint charter high schools as failures because they serve students most in need of more effective schools are pointed in the wrong direction.
