Caution urged in interpreting ‘outlier’ data

Ninety consultant teams in England are statistical outliers by the standards used in the Bristol Royal Infirmary inquiry, according to a data analysis conducted by Professor Sir Brian Jarman. However he says extreme caution should be applied in interpreting the results because there could well be scientifically credible explanations.

Professor Jarman, the inventor of the hospital standardised mortality ratio (HSMR) which is now used worldwide to measure quality of care, has looked at 10,000 tests of diagnoses, operations and procedures related to hospitals and consultant groups.

He told EHI Primary Care: “We found 270 hospital situations and 90 consultant teams that are potentially abnormal by the standard of Bristol.”

Professor Jarman was the only medical member of the public inquiry into the Bristol heart scandal which found the mortality rate between 1991 and 1995 was "probably double" the rate for England at that time for children under one.

Professor Jarman warned that great care was needed in interpreting the results of his latest analysis. He added: “An example of why you have to be extremely careful can be seen by looking at the Shipman inquiry where my colleagues showed that 12 doctors would have been abnormal by their calculations, only one of which was Shipman. The other 11 showed up for entirely legitimate reasons.”

Professor Jarman says one possible approach would be to notify the chief executives and medical directors of the hospitals concerned and allow them to investigate for technical or other explanations in the first instance.

The data analysis coincides with renewed calls from both England’s chief medical officer Sir Liam Donaldson and the chair of the Bristol heart scandal public inquiry, Sir Ian Kennedy, for more work to be done to eliminate variations in clinical practice and improve patient safety,.

In his annual report last week Sir Liam claimed that over-use and under-use of treatments were rife in the UK, as in other countries, and needed to be eliminated. His report focuses on four areas of medical practice where he claims variation in practice is widespread, including tonsillectomy and hysterectomy. He says changes in clinical practice and more use of drug treatment mean hysterectomy rates have fallen by 64 % in north and central London, but have dropped by only 15% in Northumberland, Tyne and Wear.

He states that the solution to the problem of variations in clinical practice have not been found and calls on commissioners of health services to reaffirm their commitment to the NHS principle of equity and says techniques should be developed further to facilitate benchmarking of provision.

Sir Liam’s report adds: “Inexplicable variability in the delivery of specific health interventions across the NHS suggests one of a number of problems: inappropriate resource allocation; poor spread of new knowledge to the consulting room, either through doctors, their patients or both; appraisal decisions and processes that

are insufficiently robust and less effective than they might be, either in their subject

matter, conduct, dissemination or implementation.”

In a speech last week to coincide with the fifth anniversary of the publication of the public inquiry report into the Bristol heart scandal, Sir Ian Kennedy said that even though progress had been made since Bristol the NHS still needed to improve patient safety.

He added: “The picture is one of pockets of activity but poor overall coordination and limited analysis and dissemination of any lessons. Every month that goes by in which bad, unsafe practice is not identified and rooted out and good practice shared, is a month in which more patients die or are harmed unnecessarily.”