How Tiny Mistakes Cause Huge Disasters

How Tiny Mistakes Cause Huge Disasters

“If I was setting up curriculum at a university,” says engineer Foone Turing, “I’d make an entire semester-long class on the Challenger disaster, and make it required for any remotely STEM-oriented major.”

Because, says Foone, the disaster was a lot less random or simple than people tend to think. In a thread of 102 tweets, which you can read in essay form here, Foone explains the real reason behind the Challenger disaster, the 2003 Columbia disaster, the sinking of the Titanic—and the last time you melted down from anxiety.

“The Challenger disaster wasn’t a single mistake or flaw or random chance that resulted in the death of seven people and the loss of a two billion dollar spaceship,” says Foone. “It was a whole series of mistakes and flaws and coincidences over a long time and at each step they figured they could get away with it because they figured the risks were minimal and they had plenty of engineering overhead. And they were right, most of the time. Then one day they weren’t.”

Normalisation of deviance

Foone describes the specific cascades of shortcuts and mistakes that led to these famous disasters, and the similar cascades that lead to horrific car crashes on supposedly safe roads. The common factor is that a system is built and stress-tested for one set of circumstances, then stretched a little further in regular use, then finally stretched so far, in a non-optimal situation, that something reaches a breaking point. And all the fudges and allowances fall apart in catastrophic failure. Foone calls this “Normalization of Deviance.”

A certain road, for example, is built for a 80km/h speed limit. But because drivers will always push it, the engineer makes sure the road is safe all the way up to 70 MPH under normal conditions. Inevitably, drivers end up taking it at 97km/h, since everyone knows that’s still safe. And then a few people figure it must be safe to drive a little faster than the “normal” 97km/h. And a few go even faster. And then it rains, and someone still goes 129km/h and they crash and die.

Everyone designing a system is constantly fighting against normalization of deviance. And the more things are designed to accommodate misuse, the more the users misuse it. We’re in an arms race. And not just designers and engineers, but all of us—we’re all designing our own lives with a normalization of deviance.

Personal disaster prep

We’ve all been there at some point: failing a test because we pulled too many all-nighters, or missing a deadline because we couldn’t find those extra hours we thought we’d have on the weekend, or having a huge fight because we piled up too many tiny grievances with our partner, or going bankrupt for a medical bill because we spent as much as we earned and tried to live inside a rapacious capitalist system.

Yes, the systems around us build in certain stressors and failure points, and until those systems are fixed, we have to try not to stack too many other stressors on top of them. Personally, as the father of a three-month-old baby, I’m quite freshly aware of this! Before, if the laundry or dishes or grocery shopping didn’t get done, enough other stuff was going smoothly that the household could function fine until the weekend. Raising a baby puts a lot more stress on the system, and it becomes crucial to run the dishwasher, or to load the laundry in time to get everything dry and folded before bedtime. A baby is a series of little catastrophes, so there’s less room for the others.

Catastrophe-aware empathy

The second personal lesson (another big one for a new parent): everyone around you is also managing a whole system of pressures and impending disasters, sometimes a lot more than you are, so you should be patient and empathetic with them.

In less personal terms, this is why asking someone for “just five minutes of their time” or “just a few bucks” is a bigger ask than you think. It’s why whenever you want something from someone else, you should think about every possible obstacle and sacrifice—travel time, opportunity cost, incidental expenses—and take them on yourself. Meet someone near their workplace, pay the tab, secretly pad out deadlines so a delay won’t be a catastrophe.

In close relationships, look for each other’s normalizations of deviance, and help them de-escalate. Take things off each other’s plate, swap errands or appointments, combine resources, cover for each other during stressful periods. And give yourself that same empathy.

And then, of course, get ready to adapt, because normalization of deviance comes for us all. The best we can do is give it fewer chances to cause a disaster.

Normalization of deviance thread | Foone on Twitter

Normalization of deviance: essay format | Foone on Thread reader


  • A certain road, for example, is built for a 80km/h speed limit.

    This example doesn’t make sense to me. If the speed limit is 80km/h, then designing the road to handle a limit of 100km/h is reasonable. This would only become an issue if the government increased the limit to 100km/h without improving the road to handle 120km/h. If a driver then travels at 129km/h in the rain, they’re a moron, and it has nothing to do with the “normalisation of deviance”.

    I would think that the “normalisation of deviance” occurs when the government increases the speed limit without improving the road to match it.

  • And the idiots who drive at 129km/h in the rain are the P-platers, driving utes, who think they’re the bomb after having their license for 6 months. I come across dangerous and simply stupid driving on the roads on a daily basis and, more often than not, it is a P-plater cutting across lanes with hardly any clearance or tailgating someone until they move across or driving way above the speed limit.

  • (One of) Murphy’s Law:

    It is impossible to make anything foolproof because fools are so ingenious.

  • Or, you could read the actual book that was written on this . . . . strangely called The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA by Diane Vaughan. Assuming anyone can be bothered to read books instead of tweets.

Show more comments

Comments are closed.

Log in to comment on this story!