Companies Are Creating So Much Data, It Has To Be Shipped In Trucks

Companies Are Creating So Much Data, It Has To Be Shipped In Trucks

A terabyte isn’t cool, you know what’s cool? A petabyte. And companies like DigitalGlobe are creating more of them than they can upload to the cloud. That’s why Jeff Bezos has a service for shipping huge amounts of data via traditional roadways.

Photo: Getty

DigitalGlobe is a company that sells highly-detailed satellite imagery to clients ranging from journalists to oil companies to the US government. It’s been collecting these images of our world for 17 years and it needs to get 100 petabytes of data into the cloud. On May 9, Wired had the chance to be in the warehouse when an unmarked 18-wheeler pulled into an Amazon Web Services centre carrying five million times the bytes of the Library of Alexandria.

Amazon announced its Snowball data transfer devices in late 2015. This what one of them looks like:

Photo: Amazon

It’s basically just a desktop-sized hard drive that can withstand heavy damage and automatically encrypts/stores up to 80 terabytes of information. The client uploads their data to the Snowball and then it gets physically shipped to an Amazon data center. A few months ago, the program was supersized and a truck that’s basically a hard drive was introduced. It would take around 1250 Snowballs to handle DigitalGlobe’s archives but just a single snowball. And since that archive increases in size by 100 terabytes a day, many more Snowball-sized transactions will be needed in the near future.

Last year, Amazon Web Services made up just nine per cent of its total revenue, but it accounted for 56 per cent of its total operating income. It’s safe to say that the cloud computing arm of the company has been a big success. And with the Snowball program, Bezos recognised that there was a niche but lucrative market available. To upload all of DigitalGlobe’s data from your home computer, it would take 300 years. A Snowmobile transfer takes about 10 days. And with various companies trying to store more and more data for the purposes of machine learning, this market will only get bigger. Storage costs $US0.005/GB per month and downloading the data back from the cloud costs $US0.03/GB in the US. For storage alone, Amazon will rake in $US500k a month from DigitalGlobal, and every time the company needs one of its images, the cash register will ring.

It’s a testament to the cyclical nature of technological innovation that we’ve hit a point in which we have to do things the old-fashioned way if we ever want to get to the future.