Certainty-Based Marking (CBM)
Certainty-based or Confidence-based marking (in various forms) is many decades old. It has repeatedly been shown to stimulate
more careful thinking and learning than simple (right/wrong) marking, and to provide more reliable
assessments. LAPT aims to provide the best possible implementation of CBM as a learning tool, for open use.
What
is Certainty-Based Marking (CBM)?
- After each answer, you indicate your degree
of certainty that your answer will be marked as correct.
- In LAPT, this is on a 3-point scale: 1 (low), 2 (mid) or 3 (high)
- We deliberately do not use words like 'sure' or 'very sure'
because these mean different things to different people
- The mark scheme and the risk of penalties will guide your
choice of C level: see below.
Why use CBM?
- To make you think about how reliable your answer is
- To encourage you to try to understand the issues, not just to
react immediately to a question
- To encourage you to think laterally: other pieces of knowledge
may help to validate or question your answer
- To challenge you - if you won't risk losing marks if wrong, then
you don't really know the answer.
- If you are a careful thinker, but not very confident, you will
gain in confidence
- It is more fair - a thoughtful and confident correct answer
deserves more marks than a lucky hunch
- You need to pay attention if you make confident wrong answers -
to think, reflect and learn!
- Efficient study requires that you constantly question how your
ideas arise, and how reliable they are
How
does it work?
- Certainty levels 1, 2, 3 always give you marks 1, 2, or 3
when you are correct
- If you are wrong, then unless you opted for C=1 you will lose
marks: -2 at C=2 and -6 at C=3
When should I use the
different certainty levels?
- When you're sure, you obviously do best with C=3 . But you stand
to lose 6 (twice the gain), if you are actually wrong!
- If you are very unsure, you can avoid any risk of a penalty by
choosing C=1
- In between, you are best to use C=2: you stand to gain just
2 marks if correct, or lose 2 if you are wrong
- People are good at weighing up risks like this, but you will need
to think about whether you can justify your answer
- Look at the graph below, and you'll see how your average mark
depends on your choice of C level.
- suppose you think you only have a 50% chance of being right:
The highest graph for 50% on the bottom scale is black, for C=1. So you
will expect to boost your marks on average most by opting for low
certainty (C=1).
- if you think you can justify your answer well, with >80%
chance of being correct, then the red graph is highest, for C=3. Opt
for high certainty.
- Note that you are always best to be honest when indicating your
certainty: if you claim high certainty (C=3: red) when you don't
think it is justified, you will expect to do badly - with very likely a
negative mark on average. If you understand the topic well, and think
your answer is, say, 90% likely to be right, then you will lose out if
you opt for C=1 or C=2 rather than C=3. You will do best if you can
distinguish which answers are reliable and which uncertain.
"Confidence" or Certainty"?
I switched from talking about CBM as "Confidence-based marking" to "Certainty-based marking" in 2005.
"Certainty" seems to carry less inappropriate baggage than "Confidence" - which sometimes
suggests to people that confident or brash personalities are being rewarded. This is not so!
Those who are rewarded are those who can distinguish between reliable and unreliable answers.
Successful people acknowledge uncertainty when they encounter it, and this how they gain in self-confidence.
In the context of "How certain are you that this answer is right?" or "How confident are you that
this is right?", the two terms are equivalent. But "Certainty-Based Marking" is (I hope!)
less open to misinterpretation.
I have resurrected for the web an early classic review of some of the relevant concepts by
Andrew Ahlgren (1969)
Time-saving tips
- With a keyboard computer, you can enter T,F,Y or N for T/F Qs and 1,2 or 3 for certainty instead of using the mouse.
- Pressing ENTER after viewing Q feedback will normally take you on to the next question.
- To go back to earlier or skipped questions, use the INDEX. If you repeat attempts at Qs, these will be recorded as such.
- Don't use your browser's BACK or REFRESH facilities. These may abort your exercise.
Further Discussion:
Publications
Problems, queries, suggestions: mail to
cusplap@ucl.ac.uk
Go to
Top of Page