I recently completed the Coursera course Data Analysis (Winter 2013) presented by Professor Jeff Leek. For those of you unfamiliar with the format, a MOOC (massive open online course) is a university-level course taught online to anyone who signs up and participates. Usually the lectures are in video format and there and quizzes and sometimes projects to score students’ performance.
A ton of work goes into putting together a MOOC and I have a lot of respect for all the staff involved. Additionally, these courses are 100% free. This makes it a bit problematic to criticize because I have invested nothing other than my time which I did so willingly. Nonetheless, the universities and companies who create these courses do so for a reason and I believe they are always looking to improve and better themselves. I felt that most salient problems in the class can be boiled down to two “challenges” with the MOOC format. These are not revolutionary criticisms, but I hope they add to the discussion on this new format of learning.
The phrase “Data Analysis” can mean hundreds of different things so I found it surprising that the class did not have a subtitle or specific focus. Yet in eight weeks, we covered a lot of material and were exposed to lots of high-end analysis techniques. In reading feedback on the class forums, the pacing of the course seemed to be contentious for many students. This brings me to the first challenge: a diverse student body. Diversity is almost always good as it fosters ideas and discussion. However, most university courses have prerequisites to make sure everyone in the class is on an even playing field. In a MOOC anyone with an Internet connection can participate. While I’m all for continued openness, I felt this class could have benefited by being more clear upfront about necessary knowledge. It seems as if many students were trying to learn R and/or basic statistics while taking this class. To me this seems like jumping into calculus hoping to pick up algebra on the way. I know Cousera wants to encourage participation, but they shouldn’t be afraid to be honest about course requirements.
The reason I don’t think MOOCs will ever replace college is the barrier between the professor and the students. Getting direct feedback on a paper or an answer to your specific question is the real value of in-person classes. That’s simply not possible in a MOOC. For this class, feedback was available from three places: quizzes, analysis papers, and the forums.
The quizzes are graded automatically and instantly upon submission and are great for quickly assessing your understanding of the lecture material. I like this format and I really think it works well. One suggestion is to have a “hint” button that offers a nudge in the right direction or a link back to the relevant lecture slides.
The data analysis projects gave us a chance to work with real data and apply techniques learned I the lectures to the data. This is a great idea, but in practice cannot succeed. The feedback on these projects came from other students in the class. While a peer grader can evaluate a statement like “Did the paper have sections X, Y, and Z?” how can I trust a peer grader to evaluate questions like “Were the correct statistical methods applied?” or “Did the author draw correct conclusions based on the analysis?” Even I felt a bit uneasy while grading my peers’ papers. Maybe I think the student did something wrong when really it was me who was wrong. Maybe we’re both wrong but I score the paper well because I did the same thing. I believe every paper gets scored by thee or four peers which should help to normalize errors, but that’s no guarantee of accuracy.
Finally the forums were the last place to get feedback. I found them useful when a quiz was worded poorly or even for reading through some ideas that other students were considering for their papers. Unfortunately, there is not really any way to trust the accuracy of the information. The professor occasionally commented, but usually more on course technicalities instead of content questions. There were a few community TAs, but they were fellow students and it seemed like they didn’t really have much access to the professor. It would have been better if a few grad students actually working for the professor were TAs and were able to give authoritative answers to questions. Honestly much of the communication between the TAs and students was arguing.
These three methods highlight the second challenge: feedback. Learning is difficult without feedback and I feel this course’s biggest downside was the lack of available feedback. I think a much better format would be like what I have seen in a few Udacity classes. The student enters information or code, or answers a question that is then instantly graded before moving on. Sometimes they are quick knowledge checks and other times they are longer application exercises. In either case, after answering (right or wrong) you get an immediate explanation detailing the correct answer/approach. This is much more constructive than getting a grade back from a peer who may or may not understand the material himself.
Professor Leek offered his thoughts about the class on the Simply Statistics Podcast and I was happy to see they brought up many of the same issues discussed here. I don’t think I agree with the necessity of a data analysis project in its current format. Certainly writing about and presenting conclusions is important, but the methods of doing so will be different for every different person depending on their field. If I gave a manager in the private sector a report in the style of an academic analysis, his eyes would glaze over. Especially in a course this short, I think communication skills should be left out in favor of focusing on the core content.
In the end, however, what is most important is if I learned something and I can definitely say I did learn from this course and I was exposed to several new and interesting techniques for analysis. I hope to see this course offered again with some improvements. Hopefully many of the students who were too discouraged to complete the course the first time around will consider revisiting it a second time. There is a lot of potential to MOOCs and I look forward to taking more in the future.