ChatGP? The uncertain role of generative AI in NHS care

The pace at which generative AI has infiltrated product portfolios and media headlines in 2023 has brought excitement and consternation to those tasked with evaluating its implications for patient care. Owen Hughes reports

While AI and data technologies have already found a foothold in areas like diagnostics and drug discovery, the rise of large language models (LLMs) has sparked animated discussions around how, where, and if ChatGPT-like tools could find their way into clinicians’ digital toolkits.

According to a worldwide survey of 2,607 clinicians by Elsevier Health, 48% support the idea of using generative AI tools to support clinical decision-making, underscoring the desire of many clinicians to spend more time on patient care and less on burdensome admin work. view article here.

Yet issues with these tools fabricating or ‘hallucinating’ information means that generative AI’s suitability for clinical applications is far from certain.

Dom Cushnan, director of AI, imaging and deployment at NHS England, emphasises the importance of safely implementing AI tools by ensuring they are evidence-based and appropriate for clinical use. He will be speaking at Digital Health’s new AI and Data event at the end of this month.

While he acknowledges the excitement around generative AI and its potential in healthcare, Cushnan stresses the rigorous processes any digital tools must undergo before they can be integrated into clinical practice. This includes navigating the complex regulatory landscape stewarded by the likes of the ICO, MHRA, NICE, and the CQC.

“Getting your way through that is difficult, and also for us as commissioners, it can be hard for us to understand what the things are that we want to do to make sure that we methodologically test whether these products work,” says Cushnan.

“And that’s not just AI – that’s digital health as a whole.”

To help move AI tools from concept to reality, NHS England has set up the AI and Data Regulation Service , which acts as a guide detailing the required standards and the agencies involved in bringing AI products to the market.

Rather than trying to “boil the ocean”, Cushnan explains that efforts from NHS England and the NHS AI Lab are geared towards AI tools that are suitable for clinical environments and use more straightforward statistical models for their decision-making.

“When we established the AI Lab, it was to try and solve that problem statement around the safe, ethical adoption of AI. We’re still testing out some of those products that are in the market at the moment that aren’t generative AI [and] have more straightforward statistical approaches to how they come to the decision,” says Cushnan.

This isn’t to say that discussions around the potential for generative AI in the NHS aren’t happening at the centre. “There are a couple of things that we’re thinking about, and no decisions have been made on them,” Cushnan says.

“Obviously, [generative AI] is of interest to everybody. The question mark is, what is that area and what does that look like? We’re seeing some good use cases of where that could be applied, specifically in supporting things like dictation to notes or other kind of operational back-end activities.”

Democratising AI

Michael Measures is the director of technology at Answer Digital, an IT services and consulting firm and main digital delivery partner for the AI Centre for Value Based Healthcare.

Led by King’s College London and Guy’s and St Thomas Hospital NHS Foundation Trust, the AI Centre is a consortium of NHS trusts, universities and technology industries partners dedicated to researching and validating AI solutions for use in the NHS.

The collaboration has led to the development of platforms: AIDE – an AI deployment engine designed to streamline AI deployment through a centralised ‘app store’ – and FLIP, a federated learning interoperability platform that facilitates training models in a privacy-preserving manner.

Together, these platforms simplify the deployment of AI in clinical settings by lowering the barrier for integration into clinical pathways, especially for smaller AI companies that may lack the technical expertise or resources to deploy their innovations in a clinical environment.

“The idea is to be completely vendor agnostic, and to have that open community of people who are providing models onto the app on the app store,” says Measures.

“Obviously, we’re very excited about that, because it lowers the barrier of getting AI, in all of its forms, into clinical pathways.”

While the current focus is on DICOM imaging data, the aim is to evolve AIDE into a multi-modal system that can process diverse data types like electronic patient records and generate reports from unstructured data using generative AI.

“There’s no reason, really, why a generative AI model that’s been trained on a particular use case couldn’t be in that app store and pulled into a clinical workflow at the trusts; it’s just happened to be that we’ve started off with ones that look at imaging data,” says Measures.

“Once there is the emergence and evidence base for generative models, I’m sure they will start to be pulled in as well.”

Generative AI and the admin burden

Indeed, generative AI tools are already entering the market with this in mind. Nuance’s Dragon Ambient Experience (DAX) Copilot, which is currently being trialled in the US, combines ambient, conversational and generative AI – underpinned by Open AI’s GPT-4 model – to automatically generate reports from patient consultations.

Dr Simon Wallace, CCIO at Nuance, a Microsoft company, cites a survey of nearly 1,000 clinicians conducted by Nuance in 2022 that sets the scene of the challenge it’s attempting to tackle.

“We were trying to understand how much time clinicians spent creating their documents. We looked at nurses, doctors, allied health professionals, and it was about 13-and-a-half hours per week,” he says.

“We compared that figure to a similar survey we did seven years ago, and it’s gone up 25%. Not only is it roughly about a third of the week spent doing your documentation, but about over three hours of that is spent in out-of-hours-time.

A standout feature of DAX Copilot is its ability to identify ‘pertinent negatives’ that are essential medical documentation; for instance, if a patient reports a headache but doesn’t mention any sensitivity to light, this negative point becomes crucial in ruling out conditions like meningitis.

According to Wallace, data from US trials suggests DAX Copilot reduces documentation time by around 50%, while 75% of clinicians have reported better documentation quality and 79% have reported a better work-life balance.

While these are US figures, Wallace says the results are encouraging. “It’s AI providing a better patient experience, but also giving the clinicians a better experience with this admin chore and work-life balance,” he says.

“Doctors, nurses, physios…they need something that they can grab hold of that’s going to make their life easier and so that they can come in and enjoy the job they were trained to do.”

Data diversity and LLMs

The huge variety in the type, format and quality of data sitting across the NHS presents a considerable challenge to scaling generative AI tools, particularly those based on LLMs.

Phillip Matthews, director of product management, UK at Nuance, a Microsoft company, explains that while LLMs are useful for health-related tasks like finding a local GP or getting basic health advice, they need to be finely tuned for specialised clinical application.

“If you want to dive deeper into a clinical speciality, you take those foundational models and you hone them to a specific speciality – producing some type of surgical note or outpatient letter, for example,” he says.

This makes a Holy Grail scenario of using generative AI to understand a patient’s entire longitudinal history and offer precise treatments something of a pipe dream – at least for the time being.

Measures anticipates that generative AI will initially be adopted in peripheral areas like appointment scheduling and report generation. Only after extensive validation might LLMs then be approved for more direct interactions with patients.

He also cautions against getting carried away by the fanfare around AI, referencing a certain well-known digital health startup that made lofty claims about its AI prowess only to fall apart when held to scrutiny. “People are a little bit burned by some of that,” he says.

“I think we’ll see some of those things stewarded through which should build confidence over the longer term. But I think there’s always that worry that…  [we] don’t get caught up in the hype of it and go too fast, too quickly and lose sight of the evidence is needed.”

Figuring out the art of the possible

Indeed, while tools like ChatGPT have democratised generative AI and drummed up considerable excitement, there’s undoubtedly a need to exercise caution while the evidence base catches up – particularly in conversations concerning NHS care.

“I think the challenge that we all face is, how do we get to a point where we can realise the benefits of the technologies that we want to use?” says Cushnan.

“From the engagement that we’ve heard, most people would like some form of tooling that can help them get through work faster. And that’s on everything, whether you’re a policymaker and you’re thinking about what generative AI looks like, right through to a doctor on the ward or a nurse or an allied health professional.”

Cushnan concludes: “I grew up on Star Trek, I grew up on science fiction. If the art of the possible is around AI, I would love to figure out how we can get to a point where the evidence around using these technologies gives us confidence [to use] them in our day-to-day routine practice, whether it’s clinical or consumer-facing.”

Dom Cushnan, director of AI, imaging and deployment at NHS England will be speaking at AI and Data, 30-31 October 2023, a new event by Digital Health, the organisers of the market-leading Rewired and Summer Schools. The new event includes a wide-ranging programme on two stages: AI and Analytics, and Data and Research.

All sessions are CPD accredited. AI and Data is free for the NHS, public sector, start-ups, charities, education and research. Commercial tickets from £275+VAT. Find out more and register here