How chatbot technology can open up the world of published data

NHS Digital is trialling chatbot software on its website to help clinicians and researchers find published data more easily. Programme manager David West offers insight into the technology pilot and its anticipated rewards.

NHS Digital’s data is used and viewed by a diverse range of audiences from clinicians and researchers, to journalists and members of the public.

Our published data had more than 2 million views online and was downloaded almost a million times between July 2018 and July 2019. The NHS Digital website is a huge and complex repository of published data and it can be a daunting prospect for a researcher or data user to find what they are looking for.

In 2018/19 we published 287 statistical reports including a mixture of monthly, quarterly and annual reports on a diverse range of subjects covering the health and care sector.

Add the fact that we have data going back years for each topic and you have the maze of publications, methodologies, dates and data that even the most experienced person would find confusing.

So, can artificial intelligence (AI) improve the user experience of our website?

Until mid-October, visitors searching for data on the website can access a unique resource – a chatbot that has been designed to find and share our data in a much simpler way.

NHS Digital has been working in partnership with IPsoft to produce a version of the tech company’s Amelia application. Amelia is an intelligent virtual agent (IVA) with an innovative cognitive platform that supports human-like engagements. Amelia can understand context beyond simply identifying select keywords, register and react to emotional states, and switch contexts during conversations. Amelia is also able to learn from human interactions in order to improve over time.

The application has been rebranded as NHS Digital’s Virtual Data Assistant (ViDA). Users can ask ViDA questions about data and information held by NHS Digital including published reports and resources, statistics and clinical indicators and about the processes for accessing the data we hold that are not published online.

The benefits for users

We understand that NHS Digital website users searching for information want to see what information is available and be able to explore the relevant publications quickly and easily.

An interactive tool can quickly source relevant publications, facts and dashboards, without having to navigate through multiple pages of the NHS Digital website.

ViDA has a defined framework for the meta data stored within its database, which enables users to easily explore the data relevant to their enquiry.

ViDA also acts as a translator between clinical and colloquial terms. The meta data and publication names on the NHS Digital website use NHS language which means the current search functionality on the website may not return a relevant result if the same language isn’t used.

The meta data database contains a synonym library that will be continually developed, which has more detailed search capabilities so users can locate relevant information using public language and not clinical terms.

Learning more about our audiences

ViDA enable us to find out more about what users are searching for so we can continue to improve how we classify and tag our products, to make them easier to access.

Existing analytics show what people have clicked on and how they have got there but we don’t know whether users have found the information that they were looking for.

ViDA offers a solution to this by logging each individual conversation, as well as the ‘Emotional Satisfaction Score’ of the user, so we have a basis for analysis of user behaviours and satisfaction.

When users get their search results, ViDA asks if they found what they were looking for, and if no results are found, provides an appropriate response.

Being able to understand more about the user experience on the NHS Digital website means we can highlight topics that are of interest to users and allows us to improve how they search and access data on the website.

Improving access to other resources

As well as published data, we receive applications from researchers and clinicians for the unpublished data that we hold which is accessed through the data access request service (DARS).

There are cost, time and security restrictions associated with the DARS process that users are often unaware of when they submit a data access request. Users can also incorrectly assume that they require DARS when there are published data available.

ViDA is programmed with a series of DARS-related responses to inform users about the process. Information about DARS is provided through the programmed responses which lead to written and video content. This means users are informed of all their data access options, including costs, timelines and security implications.

The virtual data assistant is in beta until 18 October 2019, providing the opportunity to trial AI technology within NHS Digital so we can understand more about its capabilities and limitations.

We encourage regular users of our online data to test ViDA during the pilot and send us feedback about their experiences and how we can continue to improve access to our published data.

Please email any feedback to us at iuod@nhs.net

David West is the programme manager for innovative uses of data at NHS Digital