One aspect of working on MOOCs is that there is no clear way to measure it’s success. Do you use the stats and logs that indicate clicks and time-on-page, or look at the nature of the conversations and/or comments made?
That’s why this paper loaded to Academia.edu by George Veletsianos piqued my interest – is there something in here that can help me understand the metrics we need to use in order to measure the learning and/or success of a MOOC?
“Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption.”
Unsurprisingly the authors highlights the lack of literature around MOOCs that look into the metrics of MOOCs that are not captured on the MOOC platform (EdX, Coursera, FutureLearn, etc.), notably the social engagements, note-taking, and content consumption. Something I’d not considered before is the “availability of large-scale data sets appears to have shaped the research questions that are being asked about MOOCs.” It’s something I’ve wrestled with … are we asking the right questions about a course ‘success’, and do we have the right data to start with? I think not, on both counts. I would love to know more from learners on a MOOC, but the response rate on post-course surveys are typically low, typically completed by the ones who finished the course and enjoyed it. It’s the learners who signed up and didn’t visit the course, those who did visit the first step but then left, and those who dipped in and out that I really want to hear from. They have as much to say about the course, it’s content, it’s delivery, and it’s ‘merit’ as those who completed.
The paper concludes, rather disappointingly, by saying that “researchers need to dig deeper, and use an array of methodological tools to do so. Separately or together, each research method can lead to pragmatic suggestions to improve open teaching and learning through social, pedagogical, or technological approaches.” I shouldn’t be too surprised with the conclusion as there isn’t a good metric to define a MOOC or online courses’ success – it depends on what you define as the success (numbers of learners enrolled, numbers of learners completing, passed assessments, duration of study, post-course questionnaire, course reach, etc.)
Veletsianos, G., Collier, A., & Schneider, E. (2015). Digging Deeper into Learners’ Experiences in MOOCs: Participation in social networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational Technology 46 (3), 570-587