Making an existing course online-only and assessing by portfolio
6 August 2013
The Institute for Global Health’s Dr Jolene Skordis-Worrall and Dr Hassan Haghparast-Bidgoli explain the challenges and triumphs they encountered when turning one of their existing modules into an online-only, Moodle-based course.
Economic Evaluation, an optional three-week module for students on the MSc in Global Health and Development
How it works
Economic Evaluation trains students to evaluate the cost effectiveness of health interventions. Our students come from a variety of backgrounds – we have doctors, lawyers, physiotherapists and nurses from all over the world – so it’s a mixed bag of highly capable students to whom we are usually introducing an entirely new skillset.
The skills we want them to gain are primarily practical, as they need to emerge able to make complex calculations, and after teaching the course in a traditional classroom setting for some years we concluded that they would gain far more by learning independently and working through the exercises in their own time. We felt that students weren’t finishing the course with professional skills: they could explain the calculations extremely well, but many weren’t actually able to do them!
So, we applied for an E-Learning Development Grant (ELDG) and set up a Moodle-based course, delivering content via video lectures and a range of online exercises. We knew that we wanted to keep the lectures nice and short – ideally no longer than 20 minutes each – but we all had different ideas about what form the lectures should take: is it more important to see the lecturer or to have the slides larger and just hear the lecturer’s voice? How important is it to ‘interact’ with the slides? We decided to try a mixture of different styles in order to explore which the students preferred, but we actually found that they didn’t have strong feelings about it; all that mattered to them was the content. They didn’t want lecturers reading off their slides (whether or not they could see the lecturer) and they preferred lectures to contain examples of the concepts at work.
At the same time as moving the module online, we decided to change the mode of assessment from a project to a portfolio. This seemed the most effective way of evaluating students’ new practical skills and a good means of pulling together the smaller practical tasks into a single coherent document that demonstrated how the individual tasks built on one another to construct an advanced skillset. We also didn’t want the course to be any more stressful for students; it seemed more sensible to capture what they’d done throughout the course rather than adding a new task at the end. This enabled them (and us) to see how far they’d progressed in three weeks as they built a cost effectiveness analysis step by step. We’d never assessed using portfolios before so we approached Rosalind Duhs [a CALT Schools-Facing Teaching Fellow for SLMS] and she advised us on how to do it – she was incredibly helpful and with her support, we are planning to write up our experiences of using portfolio assessment for e-learning for a journal.
A key lesson we learned from the process of moving our module online is that setting up an online course may cost less than one might expect, but it takes significantly more time than one would imagine. The total budget for moving our course online was £900, provided by E-Learning Environments (ELE) in the form of an ELDG. This budget also needed to cover a rigorous evaluation of the student experience to ensure that we were achieving the didactic aims we had set for ourselves. To keep costs down, we used our own handy-cams for filming and YouTube for free video-editing. We did it all ourselves and this is where the high time cost was incurred. That said, we believe there was value to be gained in keeping control of the process during an ‘experiment’ such as this. We have a very clear idea of the level of skill required and the time commitment involved, and we believe the end product looks great and works well for the students whom we know so well.
To formally explore the student experience, we ran focus group discussions before, during and after the course. We had warned students taking on this course that it was something of an experiment and participation in these research activities was generally high. This approach enabled us to make minor changes at the outset, after speaking to the students before the course began.
One of the students’ biggest concerns was that they would not have enough contact with staff. With this in mind, we made sure that we were very accessible electronically. We created feedback deadlines and responded to what the students were doing almost immediately. After the course, most said they had not felt as isolated as they had expected but a minority still missed the interactivity of the classroom and we don’t believe we can ever entirely compensate for that lack. However, it’s important to note that our group of students is not a representative sample: for one thing, they had all come to London to do a residential Master’s degree, demonstrating their preference for classroom-based learning over online alternatives. In addition, our students come from all over the world and some were less comfortable working online as part of daily life.
The practical tasks we set were wide-ranging, from using Excel to set up cost spreadsheets and calculate formulae to critically evaluating articles and commenting on their course-mates’ critiques. We were keen to introduce a strong peer learning element to the course and to maximise peer support as another way to overcome that sense of isolation mentioned earlier – the idea that when you’re doing an e-learning course you’re sitting at home and if you don’t understand something, you’re alone and stuck. We gave them live chat functionality but they didn’t use it much and seemed to prefer emails. Half the class actually met up for ‘office hours’ each day to work together and problem-solve in person! Both these students and those that did the course from home really bonded during the three weeks; the peer support we witnessed was greater than we’d ever seen in our other courses and that’s not something you’d normally associate with e-learning.
While students found ways to cope with the complexity of the course and the range of activities, we got the very clear message that you can’t expect them to cover the same amount of work as they would in a didactic, classroom-based course. It takes time getting to grips with the technology and figuring things out for themselves rather than having it all explained to them. On the upside, once they have figured it out for themselves, it seems more likely to stick! We were really impressed by the depth of learning students demonstrated on this course.
We are going to keep offering this course as an e-learning module as we think the benefits outweigh the drawbacks in our context, but we will be making some changes. In the future we’re going to be even more careful about giving crystal-clear instructions to students at every stage of the process, especially in terms of creating a portfolio (which many had never done before) and also when setting tasks. We are even considering giving them rough estimations of how long each exercise should take, while making it clear that it doesn’t matter if it takes them longer.
While this course took a significant amount of time to put together, in future years the workload – at least prior to the course – will be far smaller because the bulk of the content is already there. We are planning to redo some of the video lectures but these are relatively minor amendments, which is very positive.
We absolutely believe that e-learning provides the ideal platform for a course that lends itself to ‘doing’ as much as ‘thinking’. The students gained far more practical skills from this year’s course than they ever did when it was delivered in the classroom. It’s an accepted fact that things you learn yourself stick with you for much longer than things other people have taught you, and that is a real benefit of e-learning. Despite their initial misgivings, all of the students said that they were glad they’d taken the course when we ran the final focus group and that given the choice again, they would still sign up! We’re excited about running it again next year.
Interview by Ele Cooper