Peer Review- Using Double Reporting As A Tool For Revalidation
An audit of double reporting by peer review for the purposes of appraisal and revalidation.
The process of Medical Recertification requires a doctor’s performance to be assessed in as objective a way as possible. Double reporting can be used as one of the tools to provide evidence to support an individual’s recertification portfolio, by providing evidence of reflective practice. This audit suggests a framework whereby the individual radiologist can use double reporting in a fair, and objective manner.
There is significant debate about how the accuracy of diagnostic tests should be calculated and reported in the literature. (1,2) There are no objective benchmarks for acceptable levels of observation or interpretation and ambiguity errors. (2,3) Therefore standards relating to the process are suggested.
1. An exercise in double reporting should be performed annually as part of Appraisal/Revalidation.
2. Level 2 and 3 discrepancies should be referred to the local discrepancy meeting. (4)
3. A summary detailing modality type, number of examinations and the grading results should be produced. This report and supporting proformas should be then part of the individual’s consultant annual portfolio and should be presented for the purposes of appraisal/revalidation (5)
100% for each of the above
A possible grading system is proposed which parallels the discrepancy meeting findings. 0 = no disagreement, 1 = minor disagreement, 2 = moderate disagreement, 3 = major disagreement. Grading of discrepancy eg. inconsequential; likelihood of change of management and harm to patient will allow an assesssment of impact in addition to frequency.
-
The indicator
The percentage of discrepant reports
-
Data items to be collected
Two or more radiologists in the department with a similar case mix agree to double report a defined number of each other’s cases. These cases should come from routine radiology practice within a defined period of time and in a defined modality or set of modalities. The modalities should be restricted to those centring on image interpretation only i.e. CT, MR, radionuclide radiology and plain radiographs.The sample must reflect the reporting radiologist’s normal workload referral pattern. The radiologist evaluating the reports completes a form (resources) for each case reviewed. The two radiologists should discuss all cases with disagreement between them and come to a consensus view. In those cases where no agreement can be reached then the cases should be discussed at the discrepancy meeting.It may also be appropriate to review cases with moderate or major disagreement at the discrepancy meeting as there may be learning points to be made.
-
Suggested number
The sample size will depend a great deal on the facilities available in the department. A minimum of 30 cases a year in an individual modality would be reasonable but the principle of some rather than none applies.
Trusts should acknowledge the value of peer review through double reporting but be aware that it is inevitable that this process will result in loss of some clinical activity. For those radiologists taking part in double reporting, appropriate time must appear within their job plan as a clearly defined contribution to supporting professional activity.
All individuals involved in the process should have the same information available to them at the time of image interpretation. -This should include all the images performed for the original study viewed on an appropriate PACS workstation with the original referral clinical details.
The time taken will depend on the number of cases reviewed and the complexity of the modality chosen.
double reporting proforma
1.Peer review: Guidance on the use of double reporting RCR 2010
http://www.rcr.ac.uk/docs/radiology/pdf/BFCR(10)1_peer_review.pdf
2. Standards for Reporting of Diagnostic Accuracy (STARD)
http://bmj.com/cgi/reprint/326/7379/41.pdf
3. Soffa DJ, Lewis RS, Sunshine JH, Bhargavan M. Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging. J Am Coll Radiol 2004; 1: 212–217.
4. Standards for the Reporting and Interpretation of Imaging Investigations. London: The Royal College of Radiologists, 2006. http://www.rcr.ac.uk/docs/radiology/pdf/standardsforreportingandinetrpwebvers.pdf
5. Standards for Radiology Discrepancy Meetings. London: The Royal College of Radiologists, 2007http://www.rcr.ac.uk/docs/radiology/pdf/stand_radiol_discrepancy.pdf
6. Standards for Self Assessment of Performance. London: The Royal College of Radiologists,2007 http://www.rcr.ac.uk/docs/radiology/pdf/stand_self_assess.pdf
Audit of reports in the independent sector is common with some 10% of reports double read. This is useful for quality assurance, proof to referrers and of course for revalidation. Current time constraints within the NHS will not permit double reading of 10% of reports but as large a sample as practical should be used.
To overcome statistical concerns, estimates can be done to show what numbers are needed to show a difference of say less than 10% discrepancy (eg no significant discrepancy in a sample of 30 will satisfy a 95% confidence interval).If one or more major discrepancies are identified within a batch of 30 then a larger number of cases should be reviewed to determine the significance.
Some individuals/ trusts may prefer to do larger numbers in alternate years perhaps doing plain film reporting one year and eg CT the following year.
Dr Sue Barter
10 January 2010