Designing open book exams
This Toolkit provides evidence-based recommendations for designing open book exams and preparing students to take them.
17 February 2021
This guidance is informed by UCL’s COVID mitigation for assessment, our operating model for 2020-2021 centrally managed examinations and the introduction of the new digital assessment platform, AssessmentUCL.
Online open book exams provide a number of opportunities to promote high-quality student assessment. If they are well designed, they invite students to explore and demonstrate their insights in more complex ways, replicating access to sources of knowledge which they will have in the world of work.
However, as students will have unfettered access to textbooks and web-based resources, there is an increased risk, in the absence of invigilation, of:
- copying and pasting information;
- collusion with peers or friends; or
- seeking help from contract cheating companies.
The points below will help you to think through how you design your open book exam and support your students:
When designing the exam, consider which module/programme learning outcomes you are testing and how your questions reflect these. Structure questions that test students’ ability to apply analyse, evaluate, create, synthesise, interpret etc (table 1).
Structure your open book exam questions around particular problem-based scenarios or real-world cases. Provide background information on a given topic or area of study. This gives students an opportunity to apply their skills and knowledge.
Link to relevant qualitative or quantitative data and ask students to interpret and apply this in particular ways, e.g.:
What does the data show?
What relevance does this data or the scenario have?
What other factors could potentially affect this data?
How would you test for these? When devising questions, Socratic questions and questions reflective of levels and stages of learning may be useful (see table 2).
Devise coursework-type questions that probe student understanding, skills and knowledge rather than recall (see Aston, et al).
Remove ‘bookwork’ parts of questions: definitions; standard proofs etc, as answers can easily be found in books or online.
Adapt your assessment criteria to reflect the changed emphasis from recalled knowledge to applied knowledge and skills, and ensure that students are fully informed of them. Check assessment criteria with colleagues so they know how they have been adapted to the open book context.
If possible, before the exam, give students the opportunity to apply the criteria to their work in small groups to promote discussion and deepen understanding. Ensure they practise this new assessment format.
Create question papers with colleagues. Look upon this process as a supportive peer review.
Ensure you have good communication channels with students so they can get answers to their queries before and after the exams. Act on their suggestions if you can. You could set up a Moodle Forum for shared communication and create a set of FAQs based on students’ questions.
Reduce unnecessary stress through the judicious use of tone – address students directly using ‘you’ rather than the more impersonal ‘students’ or ‘they’.
Ensure that students have access to clear, detailed instructions on the open book exam(s) (see student guidance).
State a word count range and provide clear recommendations for how long a student should take to complete an exam. There is a risk that some students will spend 24 hours writing and over-produce. Explain in general terms what key qualities you are looking for in answers. The QAA notes that students require clear instructions and communications about how long they are supposed to spend on this form of exam and how their work will be marked (QAA 5 February 2021).
Table 1: Example questions relating to recall and application
|Type or level of question
|Students are asked to...
|Example questions and starters
|Knowing and remembering
|Recall knowledge of subject matter relevant to the discussion.
|Demonstrate understanding by constructing meaning from information
|Apply knowledge and understanding of a particular task or problem.
|Examine different concepts and make distinctions between them.
|Make judgements about concepts or ideas.
|Develop new ideas from what they know and understand.
|Type of Socratic question
|Example questions and starters
|Reason and evidence questions
|Origin or source questions
|Implications and consequence questions
Students will need to agree to a declaration of integrity before they submit their paper to AssessmentUCL.
Discuss the use of Turnitin similarity reports with your students – many students will be new to this as well as the dangers of using contract cheating sites, even in timed exams. See our student article advising against contract cheating.
Reduce the chances of cheating by making it very clear what kind of behaviour is unacceptable and detail what forms of material will be permitted in the context of the open book exam (Table 3).
|Strategies to minimise cheating
|Collusion: students help other students via text message or phone during exams
|Randomise the order of questions and answer options in multiple-choice question (MCQ) and short answer tests.
|Impersonation: someone else sits the exam for you
|Include assessment tasks which follow on from earlier work which the student has done as part of online discussions which preclude impersonation.
|Advice/support from parents and family
|This is difficult to provide in higher education as exams are likely to focus on relatively new research. Integration with previous work also helps here.
|Sending questions to contract cheating firms for help during exams
|Watermark exam questions to make it difficult to share them. Alert students to the implications for careers.
|Searching online for answers to questions
|Ensure that MCQ/short answer exams are fast-paced and do not re-use questions which could be found on the web.
|Copying and pasting in pre-prepared or earlier texts
|Turnitin will detect texts which have been submitted before but cannot guard against copying and pasting of newly-written texts. Questions built on new cases or scenarios or asking for fresh perspectives on an issue/problem can make it more difficult to use pre-prepared texts.
Marking, moderating and feedback
- Work with your faculty Exam Liaison Officer (ELO) and decide how markers/moderators and examiners will have access to scripts and how the marking process will work. AssessmentUCL will automatically capture candidate numbers on submission and in the filename itself. Submissions are anonymised and can be distributed as PDFs.
- Marking within the new platform is optional but there are benefits including using in-line comments, highlighting, stamps, drawing tools and easily accessed Turnitin similarity reports. The drawback of marking in the new platform is the time spent becoming familiar with it.
- Provide information to students on when results can be expected and how they can be accessed.
What is an online exam?
- Explain what a remote or online open book exam is (see for example our guidance on Supporting your students to complete an online open book exam or University of Oxford, 2021, p.3).
How to complete an online exam
- Highlight the importance of identifying and organising any materials you plan to refer to during the exam.
- Advise students to revise thoroughly because time constraints limit the extent of the open book resources you can refer to during an exam.
How to use the system
- Refer students to Examinations at UCL 2021 page concerning the technicalities of joining the exam, downloading question papers, starting and ending the exam.
- Encourage students to practise uploading their answers. If they’re using handwritten responses (calculations, graphs, etc) they will have to scan or take a photograph of their responses, convert these to a pdf document and upload these. Check the quality of these ‘mock’ documents is acceptable and provide advice if needed.
- Inform students fully of what to do if they encounter technical glitches during the exam; the Examinations at UCL 2021 explains the purpose of the Exam Query Form and how to report on any issues they have encountered.
- Timeframes may vary for different exams especially for timed exams within a 24-hour window. Your guidance should set out if there is an additional upload window allowed and students should be made aware of the reason for this time.
- Additional time allowances and rest times: make sure that students with disabilities know exactly how their additional requirements will be met during online exams.
- International times: provide guidance to students who are not based in the UK on when to start their exams and how long they have.
Supporting students' wellbeing
- Signpost to UCL's Student Support and Wellbeing – a team of expert wellbeing, disability and mental health staff providing a safe, confidential and non-judgemental space, in which students can discuss any issues that may be affecting their ability to study.
- Provide tips on creating ‘your own exam conditions’: quiet space, comfortable chair, suitable refreshments (Oxford Brookes advises safeguarding against spilling drinks), access to dependable time-keeping etc. See the 'Setting up for success - what you'll need on the day' section of our page on Supporting your students to complete an online open book exam.
Aston, P., Grove, M., Houston, K., & Sangwin, C. (2020). Alternative approaches to examinations in the mathematical sciences: Results from the survey.
Clay, J. (2020). Digital assessment in higher education.
Elsalem, L., Al-Azzam, N., Jum'ah, A. A., Obeidat, N., Sindiani, A. M., & Kheirallah, K. A. (2020). Stress and behavioral changes with remote E-exams during the Covid-19 pandemic: A cross-sectional study among undergraduates of medical sciences. Annals of Medicine and Surgery, 60, 271-279.
Emerge Education and JISC. (2020). Assessment rebooted: From 2020's quick fixes to future transformation.
Gopalan, A., & Chatley, R. (2020). Converting to online exams.
Imperial College London. (2021). Sitting a remote exam.
Lancaster, T., & Cotarlan, C. (2021). Contract cheating by STEM students through a file-sharing website: a Covid-19 pandemic perspective. International Journal for Educational Integrity, 17(1), 3. doi:10.1007/s40979-021-00070-0
Oxford Brookes University. (2020). Covid-19 and the 2020-21 examinations.
Quality Assurance Agency. (2021). How good practice in digital delivery and assessment has affected student engagement and success - an early exploration.
Redden, E. (2021, 5 February). A spike in cheating since the move to remote? Inside higher education.
Rees, A. (2020). What is it really like to take an online exam? Times Higher Education.
University of Newcastle Australia. (n.d.). A guide for academics - open book exams.
University of Oxford. (2021, 14 January 2021). Open-book exams: Guide for candidates Hilary Term 2021.
University of St Andrews. (2020). Online exam guidance: Remote ('open book') exams.
This guide has been produced by UCL Arena Centre for Research-based Education. Tables 1 and 2 are kindly Reproduced from University of Newcastle Australia. You are welcome to use this guide if you are from another educational facility, but you must credit the project.