I published a post called “This charter school bombshell was dropped last week.” There has been much debate about this report that claims urban charter schools perform better than their traditional public school peers. The National Education Policy Center released a rebuttal which asks some critical questions about the study. Here is the abstract:
Following up on a previous study, researchers sought to investigate whether the effect on reading and math scores of being in a charter school was different in urban areas compared with other areas and to explore what might contribute to such differences. Overall, the study finds a small positive effect of being in a charter school on both math and reading scores and finds that this effect is slightly stronger in urban environments. There are significant reasons to exercise caution, however. The study’s “virtual twin” technique is insufficiently documented, and it remains unclear and puzzling why the researchers use this approach rather than the more accepted approach of propensity score matching. Consequently, the study may not adequately control for the possibility that families selecting a charter school may be very different from those who do not. Other choices in the analysis and reporting, such as the apparent systematic exclusion of many lower-scoring students from the analyses, the estimation of growth, and the use of “days of learning” as a metric, are also insufficiently justified. Even setting aside such concerns over analytic methods, the actual effect sizes reported are very small, explaining well under a tenth of one percent of the variance in test scores. To call such an effect “substantial” strains credulity.
I am working on a theoretical piece that discusses why there are such drastic differences in findings about charter schools. We should all not be surprised that some scholars find good charter schools and some find bad. I think they are asking the wrong questions and these horse race studies are not as useful as they seem. I tend to think that the framing of these questions is off and local context is so important in anticipating the results of these charter schools. Some local contexts call for a massive overhaul of the educational system, while some do not need “competition” to help them continue to thrive. Often, schooling isn’t the main aspect of the problem, but that is a discussion for another day…