Power to the people on testing times

So we can get our DNA tested with no questions asked. Boots will do it for you £129.

All sorts of tests are available at a price. If you want to know if your spouse, teenage son or direct reports are taking drugs, send off a hair sample and screen them to see what substances they’ve been consuming over the past 90 days.

Self-evidently the technology is running way ahead of the ethics here.   

But ethics can get messy can’t it? What with all that ‘on the one hand this, on the other hand that’ stuff. So let’s cut to the science because right there, at the heart of testing, there is a show stopper that should make us all pause before reaching for our credit cards to order that online test.

Big numbers mean a lot of hits and misses

Here is a topical if grim example – Ebola. Suppose that there is a test for the infection that takes seconds to perform and gives an answer that is correct 97% of the time.

And suppose you have 10,000 people per day arriving in the UK from at risk destinations, of whom 1% are infected with Ebola. Does it make sense to start testing these 10,000 people at the port of entry?

At 1% prevalence you will have 100 true positives out of the10,000 people who really do have the disease. Since your test is 97% accurate at detecting Ebola infections, it will detect 97 of these people. There will be three false negative results, meaning that three infected people get through by mistake.

But you’re still well ahead, so what’s not to like? Well, look at the false positive rate – that is the people who, in reality, don’t have Ebola but who still test positive. 

Of the 9,900 people who don’t have the disease, a random 3% are going to test positive in error – that’s 297 false positives compared to the 97 true positives.

So, for every one person in isolation who needs to be there, you have to lock up three who don’t for more testing. Result: isolation facilities overwhelmed, 297 more people each day ending up scared and angry – and your test rapidly getting a completely undeserved reputation for being inaccurate.

No going back

These figures are all made up; but the point is that interpreting test results is counter intuitive. ‘Common sense’ may tell us that it is better to be safe than sorry, that any test is better than none.

But while such sentiments may be common, they do not make sense. All tests, indeed all clinical information, has an error rate.

If you have ever sat in on a multi-disciplinary team meeting for cancer patients, you will know that radiologists sometimes disagree about MRI scans, histologists disagree about histology grading, and pathologists know that many of their tests have significant error rates.

In technical terms, the sensitivity and specificity of much clinical information is surprisingly low.

So what does this mean for an age in which people can finally side-step all those pesky doctors and get their mitts on any test they want because “prevention is better than cure”?

Or treat themselves to a whole body MRI scan (current London price ~£1,500) “just to be on the safe side”?

First off, it is unlikely that the powers that be will be able to regulate us out of this problem. The nature of the web means that if people want stuff – information, copywrited material, drugs or self-testing kits – there will always be a Silk Road that will purvey it to them. And usually a pretty broad one.

Secondly, like lots of clinicians I found the relationship between sensitivity, specificity and prevalence (which is the science at the root of all this) hard to grasp and even harder to get over to patients.

As a result, almost all clinicians tend to place undue certainty on the tests and information they have to hand. As for patients, they often end up mistakenly thinking they can’t have their chosen test for financial reasons rather than because it will not yield any useful information.

Data driven context needed  

What would help is to make sure that anyone ordering a test or a scan in the UK always gets the result via an app or an interface that has all this complex information already built into it: “Here is the result of your blood test. For someone of your age and sex this result means there is an X% chance that there may be something seriously wrong.”

Or, when ‘big data’ really gets into its stride: “Your test is positive and you should go and see your doctor.  But to put your results in context, it is worth comparing it against the last ten results from people like you (in your case 45 year old white men living in postcodes similar to yours).

“Out of these ten men like you who tested positive, seven turned out to have nothing wrong with them.” (See what I mean about it being counter-intuitive?).

This kind of thing would actually be really helpful for clinicians and patients in normal clinical practice; but for citizens venturing out into the Wild West of direct-to-consumer pathology testing it will be absolutely essential.  

Many kinds of data that used to be expensive are now cheap or free. This shift is beginning to apply to our biology. From Fitbit to direct OTC testing and imaging, our biological self is being becoming transparent and measurable.

Making sense of this will take good software as well as professional advice. But perhaps the most direct way that people will learn the pitfalls of over-eager testing will come from seeing friends and acquaintances exploring, regretting, celebrating, and scaring themselves witless accessing data they have never had before. Paternity test anyone?