Exam myths: Separating fact from fiction

From pass rates to examiner scoring, there are plenty of misconceptions about the exams. Here, our exams team set the record straight, tackling five common exam myths and explaining the facts behind them.

Pass rates are fixed or capped

It is a common misconception that exam boards decide that, for example ‘Only 70% of people can pass this exam’. In reality, awarding bodies are focused on maintaining the standard of the exam to ensure those who meet the required standard are able to pass.

At the RCR, we use GMC-approved standard-setting methods (Angoff and Hofstee) to determine the pass mark for each examination. These methods take into account examiner judgements about question difficulty alongside the performance of the candidate cohort, helping to ensure that outcomes are fair while maintaining the standards necessary for patient safety. As neither pass marks nor pass rates are predetermined, both may vary between sittings to reflect differences in exam difficulty and candidate performance.

Examiner scoring is subjective, with no mechanisms to ensure consistency

Marking is one of our most monitored processes featuring multiple layers of quality assurance, and we ensure that our examiners do not mark on instinct. Before examiners are permitted to mark, they undergo calibration, where they are trained on our marking requirements and guided by detailed mark schemes and scoring descriptors. We also run quality assurance activities such as discrepancy checks on double-marked papers, to identify marking differences that fall outside of the agreed threshold and require further review.

Removing questions from papers before results are awarded negatively impacts candidates' chances of success

A thorough post-exam review is conducted to ensure consistency of standards and fairness to candidates while upholding patient safety. This may result in the removal of individual questions from the score calculation. Decisions on question removal are taken by an expert panel, with consideration to the question performance and question content. If a decision is made to remove a question, there is a review to ensure that the remaining content still represents appropriate content coverage and exam metrics remain robust and fair. When a flawed question is removed from an exam, candidates are not penalised. This is because a question that doesn’t reliably distinguish between stronger and weaker performance introduces random error rather than meaningful assessment. Removing such a question helps ensure that everyone’s final score is a fairer and more accurate reflection of their true competence. If a question is removed following post-exam review, the pass mark is adjusted accordingly to reflect this change.

Appeals are never successful

The appeals process exists because procedural irregularities can occur and candidates should have a mechanism to raise concerns where they believe these may have affected the outcome of their exam. While it is true that most appeals do not result in a change to the outcome, appeals may be upheld when a procedural irregularity or bias is found to have affected the examination process. The outcome of successful appeals will depend on the circumstances of the case and the nature of the issues identified. For more information on what constitutes valid grounds for appeal and potential outcomes, please visit the Appeals Policy page on our website.

The purpose of exams is to make money for the College

The RCR does not make any money from UK resident doctor exams, nor does it ever intend to. In fact, these exams currently run at a loss, meaning they do not cover their own cost of delivery. This is not sustainable beyond the short term, and we would like to get to the point where they break even. There are several ways to achieve this, including making our internal processes more efficient and finding sensible ways to cut costs without diminishing output.