Network outages at Cambridge lead to “resilience” review

Repeated outages at Cambridge University Hospitals NHS Foundation Trust have led to a “network resilience” review and weekly meetings with senior Hewlett Packard executives.

The trust signed a ten-year contract with HP to upgrade its network infrastructure in 2013. The deal made up about 70% of the trust’s £200 million eHospital programme that also included deploying the Epic electronic patient record.

However, four years into the HP contract, Cambridge has already gone to market with a new £140m tender for IT infrastructure services.

The trust told Digital Health News there have been three network outages this year, but this was not linked to the tender “in any way” and HP continues to meet its contractual obligations.

Board papers from the trust’s November meeting reveal that HP has been carrying out a review of “the network resilience in light of the latest data centre network outage”, and provided the trust with a stabilisation plan dated 24 October.

The issues have now been “escalated to HP’s European head and global chief executive” with weekly meetings and fortnightly phone calls between HP’s global chief executive and the trust’s chief executive, the board paper says.

Papers from the trust’s October meeting described one of the network outages on 22 September as a “critical internal incident”, “causing trust wide problems with printing and access to Epic”. A trust spokesperson told Digital Health News on 9 November that that “fault was rectified” and “network connectivity was restored”.

However, the trust has since confirmed that the network remedial work was not complete till 14 November. This involved changing the configuration of the wi-fi controllers and a firmware upgrade to the core network switch.

Afzal Chaudhry, chief medical information officer and consultant nephrologist at the trust, said no patients were affected by the outages, which have now been resolved.

“The change that has been made puts it in the position where the same problem cannot happen again.”

During the outages “a variable number of machines have not connected to the network properly” but this was mitigated by business continuity computer, found in every clinical area.

These computers were on a separate system and were updated every five minutes, Chaudhry said.

HP’s review of the network “ongoing” but was expected to be completed in early 2017.

Chaudhry said the review is looking at the “design of the network” as it was created four years ago and technology has changed.  The new £140m IT tender was also about “market testing” four years into the eHospital programme, as the trust had increased the use of its systems and seen a growing demand in the size and types of commodity IT services needed.

A spokesperson from Hewlett Packard Enterprise said: “HPE has provided a range of essential IT services to the Trust over the last three years and continues to be a strategic and valued partner. We will continue collaborating with [Cambridge] to support them in the delivery of a world-class hospital.”

Digital Health Intelligence: holds information on the clinical systems installed at trusts across the UK and uses this to calculate a Clinical Digital Maturity Index score. Cambridge University Hospitals NHS Foundation Trust ranks shared top of the table, with a score of 95 (requires log-in).
 

READ MORE:
* Cambridge university hospitals tenders £140m IT contract
* Cambridge signs with Epic
* Epic goes live at Cambridge