An article claims and MIT study shows “online learning is just as effective as traditional education.”

The context is a MOOC course in mechanics. The author of the blog post seems to glamorize the results a bit.

Here is the abstract from the actual study:

“We studied student learning in the MOOC 8.MReV Mechanics ReView, run on the open source platform. We studied learning in two ways. We administered 13 conceptual questions both before and after instruction, analyzing the results using standard techniques for pre- and posttesting. We also analyzed each week’s homework and test questions in the MOOC, including the pre- and posttests, using item response theory (IRT). This determined both an average ability and a relative improvement in ability over the course. The pre- and posttesting showed substantial learning: The students had a normalized gain slightly higher than typical values for a traditional course, but significantly lower than typical values for courses using interactive engagement pedagogy. Importantly, both the normalized gain and the IRT analysis of pre- and posttests showed that learning was the same for different cohorts selected on various criteria: level of education, preparation in math and physics, and overall ability in the course. We found a small positive correlation between relative improvement and prior educational attainment. We also compared homework performance of MIT freshmen taking a reformed on-campus course with the 8.MReV students, finding them to be considerably less skillful than the 8.MReV students.”

My critiques are that pre-and-post tests for MOOCs are inherently biased towards completers only, which I am sure tend to be more motivated than even an average student in this course on a traditional course. In other words, MOOC students are self-selecting , no matter how one slices and dices the cohorts. The researchers do try to address this question, but I am not completely convinced.

The other issue I have with the title of the blog post is that it exaggerates the findings. This study is promising, but let’s not get too crazy yet. It is one course at one MOOC that uses a very specific pre-and-post test method that has its advantages and drawbacks.

With these caveats in mind, this study is interesting with intriguing findings! I am excited to see more as the field continues to grow!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s