Ask LH: Where Can I Store Massive Files Online?

Dear Lifehacker, We currently have 4TB of data spread across too many external hard drives. We were thinking of setting up a local storage RAID array, but we have this huge internet connection and thought it might be worth storing the data online as well. The problem is finding a solution that is cost effective, and allows large files (12GB and larger in some cases -- my partner is a graphics designer and video editor). What do you suggest? Thanks, Big Picture

Picture by kookalamanza

Dear BP,

It's an appealing thought, but I suspect in your case the home RAID option is ultimately going to be better for you. We've covered off online storage and backup services before, but very few options offer a cheap or simple way to deal with extremely large files. "Cheap" is the relevant term here; no-one is going to offer you a free service when you're dealing with 12GB files, and many of the paid services still restrict file size even if they offer "unlimited" storage. YouSendIt, for example, has a 2GB file size limit. Megaupload doesn't impose file size limits on its premium accounts, but you'd need to use FTP to automate the uploading of new files.

The other potential issue with this kind of storage approach is the need to get it into the cloud in the first place. Especially if you're regularly changing and altering files, you'll quickly consume a lot of bandwidth on your home internet connection. You'd certainly want a plan with a terabyte allowance (or an unlimited one) if you're going to go down this route, and the initial setup is still likely to take a very long time. All up, I'm not sure the game is worth the candle in this instance.

That said, if readers want to offer recommendations for specific services for maintaining large file collections online, we'd love to hear them in the comments.

Cheers Lifehacker

Got your own question you want to put to Lifehacker? Send an email to [email protected], and include 'Ask Lifehacker' in the subject line.


Comments

    Depending on your opinion of 'cheap', Amazon's S3 storage solution allows you to store large files with quite reasonable charges.

    For automating backups similar to Apple's Time Machine; there are several software packages available that will do the trick.

    At work we use a software package for Windows called Web Drive (webdrive.com) which we use for incremental daily backups to an internal SFTP server, but I know S3 is listed as an option as well.

    If you're on a Mac, I've heard good things about an app called Arq, which basically turns an S3 share into time machine.

    There are however a zillion apps for all operating systems for S3 - the best app will depend on your budget and requirements.

    Just got Drobo FS:
    http://www.drobo.com/products/drobo-fs.php
    from:
    http://www.kayellaustralia.com.au/static/drobofs/index.html?subsub=Drobo%20FS
    love it; expandable capacity, networked, plug in HDDs and go to work; better than RAID IMO

    If you've got such a huge internet connection, why not host it from your office? You just need:
    1) Raid 5 array, either in a desktop or a nas, shared on the network
    2) An always-on computer with some vpn/ftp server software installed
    3) Some port forwarding on the router.

    You'll lose the insurance of offsite backups, but it'll work out cheaper than cloud hosted in under 4 months.

    Crashplan has an option to backup to other PCs that you or your friends own for free.

    My Suggestion would be to do both.

    Have a local copy on a array of some variety (RAID5 would my suggestion).

    And backup your data to an outside service such as Amazon S3 or RackSpace Cloud.

    This will give you quick and easy access to your data as well as the security of off site storage.

    I'm pretty sure doing both was BP's plan all along...

    "thinking of setting up a local storage RAID array, but we have this huge internet connection and thought it might be worth storing the data online as well"

    A second vote from me for the Drobo. I have a DroboPro that accomodates eight drives. These can be SATA drives of any capacity and from any manufacturer. Pop in whatever drives you can afford (you don't have to install all eight drives at once) and you can still create 16TB volumes. Of course, you'll be limited by the physical space available across the drives you have. If you run close to capacity using 2TB drives, swap out the smallest drive (or oldest drive) and replace it with 3TB drives. Drobo will rebuild on the fly (NO DOWNTIME or lost connectivity) and you'll see the extra capacity. It's not a RAID-style shutdown, rebuild - it's all done on the fly.

    My Pro supports USB, Firewire 800 and iSCSI.

    I got my Drobo from iNeedStorage.com.au at Caringbah.

    Crashplan allows for unlimited everything

    (http://support.crashplan.com/doku.php/faq/mozy#are_crashplan_unlimited_plans_really_unlimited)

    I currently have somewhere near 1.5TB backed up for $3.95USD/month, and have already used the restore function to retrieve something I'd deleted a few weeks back which paid for the years hosting in itself.

    Worth every cent.

      Just to clarify, this is as a BACKUP solution. Its obviously way too slow if you need frequent access to these files.

      I'm using CrashPlan for more than 2 years. Very good option for keeping backups of both desktops and servers.

    Have you tried dropsend www.dropsend.com you can store large files indefinitely, or send them to someone else.

    One word.
    Backblaze.

    So basically the Lifehacker answer is the cloud is great for storing stuff, unless you have a real job with a real amount of data, and even if you do have an amazing pipe to the internet, it's still not practical.

    unraid by lime technology.

    It is a file-based bastardisation of raid5 that is ridiculously simple, fantastically elegant.

    You may want to consider : http://www.justattach.com
    It's a software solution that can be hosted in your office (since you have the hardware ready and big internet pipe) :-) It works well with your Outlook client if you're use the Outlook plug-in. My friend in New Zealand is using this for his business for transferring big files.

      Hello Bacon,
      We prefer something that can be hosted in our data center, rather than online storage.
      Is that what justattach do ? Can we control who to download the files and track it?

        Hi Lyn,
        It can be hosted in your data center. There are security settings that can allow users to set who can download files. Of course with tracking too. You may want to request a trial account from justattach.com to test out? That will give you a better picture.

    I personally have completely migrated to cloud storage because of its mere convenience factor. Nevertheless, because of the recent security compromise of Appleā€™s iCloud, I do have my doubts on the entire digital storage platform. Therefore, I do still remain paranoid and regularly back my data up into the old-fashioned external hard disks. They might be bulky in physical size but prevention is definitely better than cure.

    There are many digital storage platforms which are readily available these days. Not only are the sizes provided really massive, but they are conveniently available around the clock as well. Hence, I think it is the best medium yet if you are looking at a huge amount of data storage that you require. You will also be able to share the files inside the storage with anyone who has internet access in just one click. It is that simple.

Join the discussion!