UCL News


Lower scores seen on postgraduate medical examination between 1997 and 2001

17 August 2005

The performance of doctors taking a postgraduate medical exam declined between 1997 and 2001 is revealed in a study carried out by the Royal Colleges of Physicians.

After a steady increase between 1985 and 1997, the overall performance of postgraduate students taking the Part 1 examination of the Membership of the Royal Colleges of Physicians of the United Kingdom, MRCP (UK), which is a part of higher specialist training, showed a decline from 1997 until 2001.

Performance on specific exam questions that were repeated across the years was 14.1% lower in 2001 than in 1996.

"There has been a decline in the performance of graduates from UK medical schools taking the MRCP (UK) Part 1 examination. The reasons for this are not clear, but the finding has implications for medical education, and further studies are needed of performance in other postgraduate and undergraduate examinations," wrote Professor Chris McManus [Centre for Health Informatics], and colleagues from the Royal Colleges of Physicians, the authors of the study.

"The role of postgraduate medical examinations is to set standards of practice and thus to assure the public and the medical profession that doctors have the knowledge and expertise required to diagnose and treat patients, and to progress in their medical careers," explained the authors, whose findings were published in the open-access journal BMC Medicine.

McManus et al assessed the performance of students on "marker questions" - questions that were repeated in different MRCP (UK) examinations held between 1985 and 2001, and which allow performance to be compared from one year to another.

Overall, the ability of examination candidates increased from 1985 to 1997, after which it declined sharply: on a graphical representation of the results, the curve has a clear "dog-leg" shape, which rises until 1997 and then drops sharply in the subsequent years.

In the second part of the study, the researchers compared the performance in an examination that took place in 1996 and another one that was held in 2001. The results show that fewer candidates who passed the examination in 2001 gave correct answers than had equivalents in 1996 and the candidates' mean score decreased from 69.4 in 1996 to 59.6 in 2001.

"The 'dog-leg' in performance [in] 1997 was not an artefact of changed Examination Regulations, mix of UK and overseas candidates, or time from qualification until taking the Examination," wrote the authors. "Study 2 confirmed that performance in 2001 was significantly worse than in 1996, that the poorer performance was found in graduates of UK medical schools, and that candidates passing the Examination in 2001 performed less well than those passing in 1996."

Reliance in the past on "norm-referencing" for setting the pass mark, when a fixed proportion of candidates was allowed to pass at each examination, meant that candidates passing the examination at the end of this period had a somewhat lower standard.

Norm referencing was abandoned by the MRCP (UK) in 2002 in favour of "criterion referencing," in which examiners determine the pass mark by assessing the desirable knowledge based on individual questions. Thus the standard required to pass all components of the examination is now fixed and independent of the standard of the candidates sitting at that diet

(McManus IC, Mollon J, Duke OL, et al., Changes in standard of candidates taking the MRCP(UK) Part 1 examination, 1985 to 2002: Analysis of marker questions. BMC Medicine, 2005;3:13).

'Medicine and Law Weekly'