Cisco Just Had An ‘Oh Sh*t’ Moment With Meraki [Updated]

Cisco Just Had An ‘Oh Sh*t’ Moment With Meraki [Updated]

It’s every sysadmin’s greatest nightmare. You do what you think will be a simple configuration change and accidentally lose a whole bunch of data. That’s exactly what happened over the weekend with Cisco’s Meraki service. A configuration change “caused certain data uploaded prior to 11:20AM Pacific time on August 3 to be deleted”. Whoops.

Cisco says the impact to users of their cloud-based networking system is limited. Configuration data pertaining to devices managed via Meraki is not affected.

However, user-uploaded data such as custom floor plans, company logos used to customise pages and device placement photos might have been lost. If you’ve been using Meraki to configure your phone system’s IVR then those sweet tunes you added to the hold music, amongst other items, might be gone as well.

It’s a good reminder that while cloud services are incredibly convenient and, generally, really reliable they are operated by human beings who can screw up just like the rest of us. And that keeping our own backups and records of configuration changes are a good idea.


Since publishing this article, Cisco have provided this statement.

“Cisco’s top priority is the satisfaction of our customers. On Saturday, August 5, 2017, Cisco notified customers that a configuration error had inadvertently deleted a limited number of Meraki customers’ user-uploaded data. The issue affected files in a North American data center storage service, and the incorrect setting was immediately fixed. We are taking measures to reduce the future risk of this type of configuration error. Network configuration data was not involved, so most affected customers should not experience an issue with network operations. No customer data was compromised. Cisco is working to restore the deleted files and will continue to update customers in real time via a Meraki Support Page. Cisco will continue to actively engage with our customers to provide whatever support is needed to remediate this issue.”