Learning The Backup Lesson – The Hard Way

Today is a big day for me. It’s my last official day at Lifehacker. Over the last couple of years I’ve done my best to inform you all and help you solve various problems and challenges. So, I figured for my last feature article in this stint as a contributor at Lifehacker I’d share what I’ve learned from my most recent technology misadventure.

This is a lesson about overconfidence, backups, hardware failure and a beta versions. I hope you all learn something from it.

I’m generally pretty good at making sure my critical data is backed up. I have both a local external hard drive and NAS that I back up to as well as using a couple of different cloud sync services for my work files. I use Evernote for most of my text creation so that my docs are synced easily across multiple devices and use as many platform independent apps as possible so I can easily flip between Windows and macOS as needed.

It turns out, one piece of data that I thought was being backed up wasn’t; my photo library.

Errors are often cascades

My tale of woe has several pieces

  1. I ‘upgraded’ to the first developer beta of macOS Catalina
  2. I didn’t do what I usually did when playing with beta versions of taking a full disk image of my system
  3. I relied on my backups working
  4. My Mac had a significant hardware failure – a blown capacitor on the logic board

While the first developer beta is usually a bit of a crap shoot when it comes to reliability and performance, things were going pretty well. But then, while I was working the week before last, my MacBook Pro (a 13-inch model, not subject to any recalls at this stage) made a sound I hope no other computer user encounters. It went ‘pop’. There was no acrid smell or outpouring of blue smoke. But the computer was totally dead.

I didn’t panic. I waited 30 mins or so, plugged it back into power, waited a few minutes and tried to power it back on.

Nothing.

I was planning to fly to Sydney the next morning so I grabbed my Windows 10 laptop, loaded my critical apps on it (the machine had been wiped the night before as fate had it following my trip to China for CES Asia and being connected to a bunch of untrusted networks) and was ready for work. I made an appointment at the local Apple Store Genius Bar for my return.

When the machine was diagnosed, the blown capacitor was discovered and Apple undertook to repair the computer under warranty.

A note about Apple design

In the olden days, a blown logic board or motherboard was a huge pain in the butt but at least, because the hard drive was a seperate device, there was a good chance your data was safe even if the computer didn’t power up.

But Apple, along with many other computer makers, decided that soldering everything in place makes your computer a millimetre or two thinner and that’s more important than modularising construction.

When I got the Mac back

Once the machine was returned from the Genius Bar a couple of days later, all was good – I’m using the repaired system now.

But the device returned to me with macOS High Sierra. No matter I thought. I’ll connect it to my backup drive, recover my data and all will be good. However, Time Machine backups are more or less locked to specific macOS versions. So, in order to access my backups, I needed to update to the beta of macOS Catalina.

I did that, and tried to restore my photos. That’s when the eastern side of Melbourne may have heard me yell “Oh. Shit.”

My Time Machine backup didn’t have any of my photo library. Lots of other data, applications and settings were there but my photos were gone.

When macOS upgrades the Photo library to work with the most recent version of the app, it makes a backup of the file. For me, that file was zero bytes – empty.

Basically, my entire photo library was gone. And neither the backup on my NAS nor the hard drive had a copy of my photos. Somewhere along the line, my Time Machine backup broke.

Fortunately, I have copies of the last few years of photos on other devices. But pretty much anything before 2013 is gone. My first child was born in the 1990s so that’s a lot of childhood history gone. However, I downloaded all the photos I’ve had on social media accounts as I’ve closed them so I still have those archives. And there’s a possibility many of the photos could be recovered from a drive my ex-wife may still have.

Lessons

More than once, I’ve discussed the importance of the 3-2-1-0 backup regime.

That’s three copies of critical files on at least two different types of media with at least one offsite copy and no errors. Not taking a full image of my Mac before the upgrade and trusting my backup regime were the main causes of my undoing.

The hardware error might have been a case of ‘stuff happens’ but the timing was terrible. Coupled with my lack of vigilance in testing my backup strategy it’s lead to the loss of a lot of really important data.

Next time I test a beta version, I’ll be taking a full system image.

And I’ll put a regular check of my backups in my calendar as a repeating weekly entry. I’ll also add a regular full image of my system so, in the event of total hardware failure, I’ll have a bootable version of my system I can connect to another computer.

It’s also important to treat your backup software, whatever solution you use, with zero trust. Just because the software looks like it’s working it might not be performing as expected. Check your backups regularly – not just that they’re working but that you can recover data from them. When I was an IT manager, I made my team do this on corporate backups every week. I should have been doing the same at home.

So, that’s the end of this stint on Lifehacker for me. I’m sure I’ll pop up again in future when the team is short handed or someone takes a holiday.

Comments


4 responses to “Learning The Backup Lesson – The Hard Way”

Leave a Reply