The ABC’s excellent iview TV catch-up service suffered from an extended outage last night. Aside from slowing down our ability to catch those episodes of QI we missed, that experience also provides a few useful reminders for anyone managing IT infrastructure.
The ABC confirmed the issue in a post on Twitter this morning. It also offered a more detailed explanation on its Facebook page:
Last night we experienced technical issues with our scheduling system which caused a widespread outage. We sincerely apologise for the inconvenience caused and are working to ensure that this doesn’t happen again. Please note that iview is not staffed 24 hours a day. If you experience an issue like this outside of business hours please call the ABC Switchboard on 139 994 and report the issue to IT so that they can escalate it promptly. Thank you for your patience and understanding.
It’s worth saying this first in an era where attacks on the ABC from federal politicians and News Limited newspapers are a daily occurrence: the fact that iview was not available for a brief period does not represent a massive disaster for anyone. It’s a free service run on a highly limited budget. Despite those constraints, it does a much better job of offering on-demand content than any commercial network in Australia has managed.
That said, the outage also provides a few broader technology lessons:
Errors often come from unexpected interactions The cause of the issue here was a scheduling problem: presumably content that was scheduled to be made available following broadcast produced an unexpected error which eventually bought the system down. While testing should minimise these cases, it won’t catch them all. If a new problem emerges, add that to the testing scenario for future upgrades.
Acknowledge errors rather than trying to bury them When outages happen, you have to own up. Provide an explanation and investigate the causes, but also ensure the response is proportional to the issue. Choose appropriate channels. For a mass-market service like the ABC, Facebook and Twitter make sense; in many corporate environments, the combination of a brief email and a more detailed intranet post might be more appropriate.
Manage expectations about how to deal with outages As the Facebook post notes, iview doesn’t have 24-hour customer support. Make sure your staff know who to contact if there is a problem, and also provide guidelines for assessing the severity of a given issue. Some things aren’t so urgent that they require support staff to log on at 3am.
Comments
One response to “IT Pro Lessons From The ABC’s iview Outage”
I’d go so far as to say I watch more iView than normal free to air TV (across all channels). It’s just that good. So a little outage with a quick explanation is enough for me. No need to crucify the service that has done it first and still does it better than most in this content sparse climate called Australia.