Can routine data detect ‘rogue’ doctors?

Was Rodney Ledward a statistical outlier? Researchers have discovered that the rogue gynaecologist could have been detected through the analysis of routinely collected data, but they caution against over-interpreting apparently poor results from doctors saying a scientifically credible explanation for outliers is needed.

Ledward, who died in 2000, was struck off the medical register after being found guilty by the General Medical Council of bungling 13 operations. It was alleged that Ledward also damaged hundreds more women.

The research team from Birmingham University compared the performance of 142 gynaecology consultants with the performance of Ledward over a five year period, to determine if Ledward was a statistical outlier according to seven indicators from hospital episode statistics. The indicators were specifically chosen for their potential link with poor quality of service.

Their analysis, published in the British Medical Journal Online, identified Ledward as an outlier in three of the five years. Eight other consultants were also identified as outliers, but the researchers strongly caution against over-interpreting these consultants as having "poor" performance because valid reasons may exist that could credibly explain their results.

The proportion of consultants that were outliers in any one year varied from 9% to 20%.

The researchers say that cancer specialists, for example, may have high values for several indicators selected for the research such as surgical complications and long stays in hospital because they carry out difficult operations on very ill patients. The method therefore needs to be refined to deal with case mix variation.

The authors also warn of the potential limitations of statistics, including missing or poor quality data that can hamper all analyses, and they stress that the interpretation of outlier status is still as yet unclear. They recommend a structured approach to seeking explanations for outlier status.

Nevertheless, the researchers say their study has generated a robust new method for scanning multi-indicator, multi-year data from hospital episode statistics to identify outlier consultants in gynaecology. They state that, although such scanning methods could never be used with “complete diagnostic certainty”, they could be used to “identify signals from noise which need to be systematically and sensitively examined, perhaps confidentially, by peers.”