There has been a report and a review of this report that have been released recently about the quality of online learning in the Florida Virtual School. The actual report titled Virtual Schooling and Student Learning: Evidence from the Florida Virtual School.
The findings in presented in their abstract are: “We find that FLVS students are positively selected in terms of prior achievement and demographics, but perform about the same or somewhat better on state tests once their pre-high-school characteristics are taken into account. We find little evidence of treatment effect heterogeneity across a variety of student subgroups, and no consistent evidence of negative impacts for any subgroups. Differences in spending between the sectors suggest the possibility of a productivity advantage for FLVS.”
The review of it is found here. His criticisms are:
“A new report compares the performance of Florida Virtual School (FLVS) students with students in traditional brick-and-mortar schools and concludes the FLVS students perform about the same or somewhat better on state tests and at a lower cost. The report claims to be the first empirical study of K-12 student performance in virtual education. This is not correct, and the report in fact confirms the findings and repeats the methodological flaws and limitations of previous research. The report’s findings fail to account for the potential bias of student selectivity in the FLVS sample, the potential impact of regression effects, differential mortality in the two groups, and the fact that the virtual environment is simply a delivery medium. Given the limitations of research such as this new study, researchers have moved beyond simply investigating whether one medium is better than the other and begun—and need to continue—investigating under what conditions K-12 online and blended learning can be effectively designed, delivered, and supported.”
I agree with Barbour in that it is impossible to make statements about the entire delivery mode through this study, however the results do help us consider the efficacy of the Florida Virtual School itself. I also tend to agree with Barbour that selection bias is likely to impact the results in the regression analysis, but I don’t think they are damning the evidence here — especially considering the sensitivity checks and the number of controls the authors use (controlling for previous achievement and use value-added models is robust enough to tell me that these schools in these settings probably are providing courses that are equalling student achievement in these limited subjects with this group of students compared to traditional classrooms).
Overall, I think both pieces are useful and I think the authors of the Harvard paper did a really good job with their study. Barbour seems a bit harsh on their work, but ultimately both pieces help push the dialogue about online learning in K-12 settings further. Of course, that is only if your question is about the academic outcomes of one math and reading test. Schools obviously have other functions that are not capture by such limited measures…