CSUS: big differences in usability of clinical software

The Clinical Software Usability Survey has found big differences in the reported usability of the software used by NHS staff.

Digital Health Intelligence launched the survey as a pilot project in November 2015, and closed it in March after more than 1,300 clinicians had provided detailed usability ratings for the clinical systems they use most often.

NHS users gave their software an overall average rating equivalent to ‘OK’, slightly lower than many other industries.

But contained within this aggregate score was an extremely wide spread of rating scores for types of software and individual software systems.

The comparative analysis revealed that, on average, GPs and other staff working in primary care gave the highest usability scores to their systems. 

The second highest average usability rating scores were given by staff working in mental health services, followed by staff working in hospital services.

However, among hospital electronic patient records, some of the most complex clinical systems in use, the study found a remarkable three-fold variation in the clinical usability scores between the highest and lowest rated systems.

Health professionals working in community health services gave the lowest overall scores to the systems they use; again with a three-fold variation in the usability score between the highest and lowest rated systems.

The cSUS project used two linked online surveys. The first was the Software Usability Survey, an established and widely used generic usability tool, chosen to enable comparisons between health and other sectors.

SUS already has a pedigree within the NHS, having been used by the Big Mental Health Survey, a national usability survey of mental health systems.

The second survey was the Clinical Software Usability Survey, developed through consultation with the CCIO Network, designed to measure the clinical usability of software.

Everyone taking part in the project was also asked to say what they like most about the system they were rating and what they would change about the system.

These narrative responses provided a 50,000-word treasure trove of information on the everyday experience of NHS staff using software to deliver care to patients.

Common themes that have emerged from the initial analysis include widespread frustration at the slow speed of networks and computers, and having to use clunky software that requires users to painstakingly navigate between multiple screens and applications.

Some respondents demonstrated a dark humor when asked what they liked about their software: ‘switching it off’ was one popular response.

Many more gave considered and insightful comments on how the software they use could better support them and their patients. A significant number also made detailed suggestions on the usability of their software could be improved.

Dr Marcus Baw, who has been one of the project leads, said: "I'm proud to be part of the team behind the first ever national study of NHS software usability, which will hopefully be the first step towards much more open discussion of system usability across the NHS.

“We are very pleased with the survey response and, in particular, the detailed, thoughtful and articulate free text information.

“These comments show that not only do NHS staff have opinions and views about the software they use, but that they want to share their experiences and help suppliers and local IT teams improve."

An initial cSUS results webinar was held on 8 May. The eight software companies that supported the pilot phase of the cSUS project will be provided with tailored cSUS reports related to their software.

They will also participate in a 15 June workshop, to help evaluate the results and inform the next stage of development.

A results workshop will also be run at the CCIO Summer School on 14 July, at which the comparative cSUS scores of individual suppliers will be revealed.

The evaluation of the cSUS pilot is already informing plans to create an ongoing ‘TripAdvisor’ like usability ratings platform covering software used by healthcare professionals.