The holly and the IT
For most of us, Christmas and New Year is a welcome chance to take a few days off. But spare a thought for those who have to make sure that IT systems keep running while the people who usually run them are not around.
Only last year, ambulance callouts were seriously delayed on Christmas Day when London Ambulance Service NHS Trust suffered an IT outage that meant staff had to use pen and paper to record 999 calls.
Frontline crews were unable to receive phone calls from HQ, and ambulance crews summoned to provide emergency backup failed to arrive, after computer screens went blank on the evening of 25 December and in the early hours of Boxing Day.
Maintaining cover
“Ensuring systems don’t fall over always make me nervous,” says Jason Bradley, director of ICT at Barnsley Hospital NHS Foundation Trust. “It’s more likely to give me indigestion than my Christmas lunch.”
Susi Turner, communications manager at Walsall Healthcare NHS Trust, agrees and says forward planning is key to making sure this does not happen. “Given that the NHS is a 24x7x365 service, we maintain a rota for IT cover throughout the festive season,” he says.
“Our business continuity plan for this period is the same as for any other time of year. There is always a director on call at the hospital, though not necessary an IT director.”
Ensuring resilience
Peter Russell, interim director of health informatics at The Rotherham NHS Foundation Trust, believes that planning for Christmas begins with an organisation’s overall IT strategy and with making sure that resilient systems are in place all year round.
“It shouldn’t make any difference whether it’s Christmas Day or the middle of summer,” he says. “You must have systems that are able to soak up failure and continue to run, so as to buy you time to bring things back to full operating resilience when the IT team is once again at full strength.
“For example, in a storage area network, where you’ve got primary, secondary and redundant disc arrays, if one fails it shouldn’t bring the whole thing down and spoil your Christmas lunch.”
Continuity planning
“Healthcare IT systems now hold increasing amounts of clinical information, so it has become much more important for trusts to invest in resilient and highly available infrastructure,” Russell adds. “Sod's law guarantees that if something is going to go wrong, it will do so at the least convenient time.
“So it’s about ensuring your organisation has a good business continuity plan. For example, where do I find the right form to record patient activity manually? How do I request a diagnostic test when the system is down? How do I slip into the business continuity routine without degrading our clinical capability?
“These are processes that should be rehearsed on a regular basis as you never know when a system is going to fail. With the best will in the world and the most resilient technology in place, something will inevitably go down at some point; but it’s essential to guard against catastrophic failure.”
Bug-free New Year
Sometimes, though, an issue comes up that looks as if it will require more than continuity planning. Remember everyone’s apprehension as Y2K (the year 2000) approached?
Little in the IT world has ever caused more speculation than the so-called "Millennium Bug". For the benefit of younger readers, this was the name given to the threat posed by hardware and software that used only the last two digits of the year, rather than all four.
Critical systems risked being vulnerable when they ticked over from "99" to "00", making some interpret the year as 1900. Estimates of the damage that would ensue ranged from minor irritation to global meltdown. Indeed, some pessimists went as far as to take measures in preparation for the total collapse of civilisation.
“The Millennium New Year was a time when we were on call and therefore had to limit our celebrations,” says Colin Sweeney, now director of ICT at King’s College Hospital NHS Foundation Trust. “In fact, I attended a user group where they held their New Year party in October as a result.
“It was a bit strange to be singing Auld Lang Syne and behaving like it was not only a new year, but a new millennium.
As for the day itself, I remember watching TV clips of celebrations on the other side of the world in the afternoon of New Year's Eve. It soon became clear that the IT world wasn't going to come crashing to an end and that, thankfully, we were also unlikely to have problems that evening.”
Air-raising surprise
And sometimes, even things that go wrong can have a funny side. Reflecting on his time before joining the healthcare industry, Sweeney says: “The most amusing Christmas mishap I recall was when I was doing some consultancy work at a commodity brokers in the City.
“In those days, the computer system was a massive beast in its own room, which was air conditioned by fans firing cool air up through grills in the floor.
“One evening just before Christmas, when staff were on holiday, the cleaner went in to hoover the room and a large bag of pine needles from the departmental tree burst open on the floor. As the fans weren’t running at the time, she decided to sweep them all down the grills.
“You can imagine what happened when staff came back to work after the holiday and the air conditioning was fired up. Yup, absolute chaos – it was hysterical!”
Fingers crossed!
This year, the biggest challenge facing the NHS looks as if it will be a ‘real’ bug, rather than an IT one. The papers and broadcasters are already talking up fears of a ‘winter crisis’; knowing that ‘flu and Norovirus have barely made an impact on the nation’s A&Es and wards as yet.
NHS England medical director Sir Bruce Keogh has insisted that forward planning and the injection of extra funding should help the NHS cope better than in previous years. The government has released an extra £300m, bringing the total winter pot to £700m – 75% more than last year.
Let’s all hope he’s right. Also that good planning and investment in IT departments mean their managers are able to eat their Christmas lunch in peace this year.
