Here’s a question I’ve been battling for some time .. how do you measure the ‘success’ of a MOOC? The problem is that I haven’t been able to define what the ‘success’ is supposed to be, so to try and measure it seems, well, a pointless exercise.
So, here’s a few thoughts I’ve had based on my experiences as a learner on MOOCs (yes, plural), and as part of a team developing and delivering 4 FutureLearn MOOCs now (with a few more in the pipeline too!).
- Do you look for the headline figures of number of registered learners, or the number of registered learners that became learners (visited the course)?
- Do you look for the number at the number of learners who did something, that engaged on the course in some way .. as either a number (e.g. 4,000) or as a percentage of the learners who visited the course (e.g. 40%)?
- If you plan your MOOC to link to a paid-for course (degree, training, etc.) do you measure the success by the number of MOOC learners who enquire, or sign-up, to the linked course?
- Do you look to the quiz or test responses, to see who’s retained and regurgitated the information based on a ‘score’?
- Is it the final number of learners who make it through the length of the course to the end?
- Is the number of comments a worthy of a measurement of success? Do courses that have more comments (either in volume or as a percentage of active learners) indicate a greater success than those with fewer?
- Can you measure the success based on interactions on social media, through a defined hashtag? In which case do you measure the number of mentions on the hashtag or dig deeper and quantify the different sorts of engagements, ranging from “I’m on #such-and-such course” to enquiries or the detailed thought process involved in critical thinking along the lines of the MOOC subject?
- Is a successful course one that takes learners from the MOOC environment into a related course, be it a MOOC or other paid-for course? If so, are you capturing that data?