The recent case in Australia of a huge computer collapse at airline Virgin Blue has salutary lessons for all leaders, no matter what their business. The failure of a piece of hardware at the airline’s outsourced IT system supplier, Navitaire New Skies, a subsidiary of Accenture based in the USA, led to the crash of the reservations system, dozens of cancelled flights, disrupted flights for 25,000 passengers, a bill for the airline of over $20m and untold damage to its reputation.
The airline thought it had great backup systems from great vendors.
Plan B was to switch to different equipment. This was supposed to happen within 3 hours (seems an awfully long time to me). Only problem was that it didn’t happen for over a day!!
So over to Plan C, which involved manual processing. Sounds ok, except that the staff had not been fully trained, so they basically had to learn as they went along.
Reservations systems are at the very heart of an airline business – its lifeblood. Yet here we have a case where that system was faulty and the expected backups failed miserably. I can only hope the airline’s pilots are not as careless and trusting as its senior executives were.
Speaking of backup, all companies (at least I hope it is all), religiously back up their systems every night. Good. Only problem is that how many know if they will be able to recover from that backed up data in the event of some catastophe. I have heard of a couple of cases just recently where all sorts of problems were encountered and full recovery has not proved possible. Modern encryption techniques complicate things apparently.
An essential ingredient in your disaster recovery plans has to be regular checks that recovery can be achieved from backups.
{ 1 comment }
Great summary and so true. Leaders need to invest organisational resources (time and money) to ensure mechanisms are in place to protect the organisation. Unfortunately the problem that occurred with Virgin Blue is reflective of 70% to 80% of organisations. Organisations rarely put the effort in to set up, manage and then test recovery situations and therefore will never really know if they work.