How Dr Paul Bartlett’s Navy career inspired assessment innovation

Before joining UCL Physics and Astronomy in 2008, Dr Paul Bartlett spent five years working for the Royal Navy as an Engineer Officer.

It was while teaching engineering, maths and electronics to officers and ratings at HMS Collingwood that he was introduced to the idea of task books; a learning device familiar to many in the Armed Forces and which is now a fixture on some UCL courses.

In simple terms, a task book is a list of student competencies.

The books are referred to at regular intervals on the course, giving students the chance to sign off against their objectives and gives facilitators the opportunity to offer face-to-face feedback. Alongside a specially developed end-of-term test, Paul is keen to see how these methods can help markers deal with the mammoth task of assessing students’ experiments.

More students, more marking

“We’ve had a large increase in the number of students since I’ve been here and there are more demands on academic staff,” explained Dr Bartlett. Since 2008, he estimates the intake of students has increased by up to 50 per cent. That’s made the traditional form of assessment unsustainable.

Previously, students would complete a lab book, documenting their experiment, and then markers would retrospectively assess the lab book giving comments and feedback.

“That meant for every hour the student is in the lab, there’s an hour’s worth of marking material,” said Dr Bartlett. “That’s an enormous amount.”

Receiving feedback at the wrong time

And quantity wasn’t the only problem. The system also meant students would only receive feedback after they had completed their experiment rather than when the advice is most useful.

It was also too easy for students to ignore the written comments and focus on the mark, while the fact verbal feedback was unstructured meant reticent students could go long periods without requesting any face-to-face feedback.

Seeking solutions to these problems, Dr Bartlett worked out a way to bring two complementary ideas together.

How the system works

Data retrieval tests

The idea of a lab book is that anybody should be able to use the information contained in it to replicate the author’s work.

Up until this year, markers have assessed each page of students’ lab books to check that it is ‘fit for purpose’.

Now, however, the assessment takes the form of an end-of-term test that students complete using only their lab books. Simple questions check whether the student has followed the protocol and produced a clear record of their work.

This method incentivises students to clearly and accurately record their work and reduces the load on the marker while achieving the same learning objectives.

Dr Bartlett is also investigating whether the test could be taken on Moodle and assessed using automatic marking.

Task books

To supply regular formative feedback, demonstrators have begun to ask students to refer to their task books at the end of every second session.

This gives them the chance to talk through the students’ progress, agree which competencies they should check off and offer timely advice.

The structure ensures all students receive equal opportunities to gain face-to-face feedback.