Category Archives: MOOC

How do you measure MOOCs?

How do you measure the ‘success’ of a MOOC?

Here’s a question I’ve been battling for some time .. how do you measure the ‘success’ of a MOOC? The problem is that I haven’t been able to define what the ‘success’ is supposed to be, so to try and measure it seems, well, a pointless exercise.

So, here’s a few thoughts I’ve had based on my experiences as a learner on MOOCs (yes, plural), and as part of a team developing and delivering 4 FutureLearn MOOCs now (with a few more in the pipeline too!).

  • Do you look for the headline figures of number of registered learners, or the number of registered learners that became learners (visited the course)?
  • Do you look for the number at the number of learners who did something, that engaged on the course in some way .. as either a number (e.g. 4,000) or as a percentage of the learners who visited the course (e.g. 40%)?
  • If you plan your MOOC to link to a paid-for course (degree, training, etc.) do you measure the success by the number of MOOC learners who enquire, or sign-up, to the linked course?
  • Do you look to the quiz or test responses, to see who’s retained and regurgitated the information based on a ‘score’?
  • Is it the final number of learners who make it through the length of the course to the end?
  • Is the number of comments a worthy of a measurement of success? Do courses that have more comments (either in volume or as a percentage of active learners) indicate a greater success than those with fewer?
  • Can you measure the success based on interactions on social media, through a defined hashtag? In which case do you measure the number of mentions on the hashtag or dig deeper and quantify the different sorts of engagements, ranging from “I’m on #such-and-such course” to enquiries or the detailed thought process involved in critical thinking along the lines of the MOOC subject?
  • Is a successful course one that takes learners from the MOOC environment into a related course, be it a MOOC or other paid-for course? If so, are you capturing that data?

Continue reading

Learning Online

Reading: Learner engagement in MOOCs

After attending a FutureLearn partners webinar about designing online courses, the age-old issue of encouraging and engaging learners in online communication came up. It made me reflect on my past posts about online learning, specifically this one: MOOCs – 9 points on what I like, and what I don’t. If you want to go and read it before carrying on, be my guest.

Hurry back!

Glad you came back. What annoys me about MOOCs, and some people who design online courses in general, is the assumption that everything you build will be used, and be used the way you want it to be used. VLEs are somewhat to blame for the apathy or lack of engagement in online activities, especially discursive or forums or comment sections – you’re locked into one specific tool for engagement. Continue reading

MOOCs

MOOCs – 9 points on what I like, and what I don’t

Over two years ago I wrote about a few experiences I’d had with some online courses / MOOCs, and why I ‘failed’ (according to the general headline figures of engagement, attendance, etc. that are used in mainstream press).

I want to revisit this, in light of more experience in both designing MOOCs and being a student on them.

Disclaimer: This is based on courses I’ve taken on the FutureLearn, Coursera, Cloudworks, EdX, and WordPress (OcTEL) platforms. I also highlight whether is was a student on the course, or part of the development team.

1. Comments and Engagement: For the most part I’ve been a silent students. This is both deliberate and accidental. Where it’s been a deliberate choice to not engage in the comments and discussion it’s been because I knew I didn’t have the time or inclination to trawl through the hundreds of fairly uninteresting posts to add my two-pennies worth or find the one nugget of insight that is worth anything. It’s also because, for some courses, I didn’t have enough interest to take my engagement further.

Continue reading

Badges - New Currency for Professional Credentials

Learning Providers (wk.4) #OpenBadgesMOOC

Badges - New Currency for Professional CredentialsHere we are at week four and challenge four. At the moment of writing I’ve earned three out of four badges that have been issued, with my response for the third challenge updated to take into account the feedback and comments about the lack of details I made about the competency framework and individual student ‘persona’.

You can read all my posts from this MOOC on the OpenBadgesMOOC tag, when I’ve written them!

Week 4 – Learning Providers
I can see the benefit of looking at badges for the soon-to-leave-education school leavers, and those already in work either actively looking for a new role or to up-skill themselves. But please don’t ignore higher education and both staff and students in it. Undergraduate degrees (and postgraduate degrees too) have as much to do with life skills as the specific subject of the degree award, and badges are a new and clear way to showcase these skills if, as I’ve said before, the criteria is selected and applied carefully and considerately.  Continue reading

Badges - New Currency for Professional Credentials

Employers (wk.3) #OpenBadgesMOOC

Badges - New Currency for Professional CredentialsHere we are, week three and challenge three. You can read all my posts from this MOOC on the OpenBadgesMOOC tag (when I’ve written them)!

Week 3 – Employers
The natural ‘end point for badges in any education setting is for those who have earned the badge to show them off for personal gain. This personal gain obviously will point to their employment prospects, at some time or another, and therefore employers will need to know what they are looking at, and why.

During the online session there was mention that “employers are in the early phase of early adopter stage for badges” (@sharonlflynn) but I have not seen evidence of this, and what type of employer (large, small, graduate employer, etc.)? Certainly Open Badges can only gain mainstream adoption in learning environments (MOOCs, degrees, adult education, social learning, etc.) if the employers recognise and value the output and badge criteria.  Continue reading

Anatomy of an Open Badge

Fundamentals (wk.2) #OpenBadgesMOOC

Badges - New Currency for Professional CredentialsHere we are, week two and challenge two. While we don’t have to keep to any timetable on the challenges, I decided I will – it’ll be a neater blogging experience this way.

MOOC: Badges – New currency for Professional credentials

You can read all my posts from this MOOC on the OpenBadgesMOOC tag, when I’ve written them!

Week 2 – Fundamentals
The context and concept for badges is being discussed and documented by those at Mozilla – Open Badges for Lifelong Learning – and those who, like me, see them as a tangible benefit for showing skills that are not assessed.

The paper by Antin and Churchill (2011) explores the gamification of social activity, through the rise and popularity of system like FourSquare, and more recently, although not covered in the paper, Get Glue (film & TV) and Lemon Tree (library game). This interaction with content and achievement has “popularized badges as a way of engaging and motivating users”, so why not as part of their learning? Why not indeed? Continue reading

Badges - New Currency for Professional Credentials

Openness (wk.1) #OpenBadgesMOOC

Badges - New Currency for Professional CredentialsWith the best will in the world I’ll be taking part in the Mozilla / BlackBoard MOOC called “New Currency for Professional Credentials”.

MOOC: Badges – New currency for Professional credentials

You can read all my posts from this MOOC on the OpenBadgesMOOC tag, when I’ve written them!

Why this MOOC?
Why is this MOOC interesting to me? I have posted about Open Badges a number of times, and how I feel the ability to demonstrate skills and knowledge obtained during a formal course of study (University) are not always easy to spot, and certainly not easy to show through the normal / formal certification received. If students can earn badges for skills, group work, ability, etc. rather than an overall ‘grade’ which is considers pass / fail then this is of use to prospective employers? Aren’t we always hearing how students are not being prepared for the modern workplace? With a Mozilla BackPack full of badges (here’s my Backpack: Mozilla Open Badges) the student able to show and demonstrate these skills will have an edge. Yes?

Now that the CourseSite has opened up the meatier sections we can see what the next six weeks is about. We’ll be looking at Openness, Badge Fundamentals, Employers and Learners, Providers, and Opportunities. There are a number of groups for us to join, no doubt to help us focus on the area(s) of most interest or importance to our own badge requirements (personal or professional). I joined the following groups:

  • Badges for Higher Education Instructor/Faculty Professional Development
  • Badges for MOOCs
  • Badges in Traditional Higher Ed Courses

Week 1 – “Openness” 
Week one is about ‘openness’ and the disparity between job descriptions and alignment of knowledge that formal courses have to the job specifications. This fresh approach (badges) could be viewed as a new ‘currency’ exchange “between job seekers, learning providers, and employers”. Continue reading

Badges - New Currency for Professional Credentials

Another MOOC – this time #OpenBadges and Professional Credentials

Badges - New Currency for Professional CredentialsAs if my MOOC failure rate isn’t bad enough, I’ve signed up for another MOOC in the vain hope that I’ll complete it (only 1 completion out of 6 so far). This one is run through the Blackboard CourseSites environment and is run available for self-enrollment now for a September 2013 start.

The MOOC aims to expand flexible learning opportunities and authentic evidence-based assessment with the use of the Mozilla Open Badge system for “accreditation and employer recognition”. The participating organisations plan to use the MOOC to

“… convene and moderate an international discussion on the role of badges as a new currency of exchange for high value, post-secondary credentials for the new workforce … [and] will explore the ecosystem for a new credential economy based on badges and surface aspects of what would be required to adopt such an approach.”

YouTube: Badges for Lifelong Learning: An Open Conversation

Starting on September 9th this will run for 6 weeks, so hopefully I’ve done what I need to before I start on my Masters degree with Grainne Conole at the University of Leicester. I do however see one very large downside to this MOOC – there are regular synchronous online sessions planned each week and, being on the other side of the Atlantic, means they will running at an awkward time for those in the UK or Europe.

MOOCs and Mobile Access

This is an excellent post  from Inge Ignatia de Waard on the impact of mobile access on learner interactions in a MOOC as part of her progress towards a PhD. I’ve added her slide deck below for you as this has some excellent findings and figures.

I am also (still) reading Inge’s book ‘MOOC Yourself‘ and hope to have a report/review of it posted up soon, but the short bio can be read here.

“Open Course in Technology Enhanced Learning” #ocTEL

Week 1: TEL Concepts and Approaches #ocTEL

  • Firstly … yes, I know the ‘error’ in the title! This one is called ‘Week 1 …’, and so was the last one: “Week 1: Induction #ocTEL”. I made a mistake, last week. While last week was technically the first week of the MOOC it was not assigned a numerical identity as it was the orientation / induction week. That’s why, if you’re reading this MOOC series back there are two ‘week 1′ posts!

I want to continue the style I started in my previous post by highlighting each activity as the ocTEL website/email introduces us to it …

… and I start with an confesison. I think I’m all read out. I’ve been reading so much recently, and with Inge Ignatia de Waard’s ‘MOOC Yourself’ (2013) book just added to the list, I’ve had enough. So this week I’ve taken some time ‘off’ and just done the bare minimum.

Activity 1.0: “If you only do one thing … “
Deciding on two of these resources to concentrate on was easy. Do I review Helen Keegan’s PELeCON keynote that I loved at last year’s pelc12 event because (a) I was in the audience during the recording and remember the gasps from the audience as we realised how risky and brave she’d been throughout the project, and (b) enjoyed the whole ARG-thing. Do I look into the ‘technology of touch‘ and the work of haptic technology that enables learning in a safe tactile environment? I want to stay away from something I’m familiar with (so that drops Sugata Mitra off the list, I’ve blogged about this work too), so that leave Eric Mazur and Stephen Downes / George Siemens references.

Eric Mazur, talking about peer instruction (three minutes from where the below video starts) is not familiar to me. Eric talks of the ‘ah ha’ moment that happens outside the classroom, and which is the hard part of ‘learning’ – is it the “information transfer” or the “assimilation of knowledge”? Continue reading