How To Migrate From An Old NAS To A New One Overnight With rsync

How to Migrate From an Old NAS to a New One Overnight with rsync

A NAS, or network-attached storage device, is great for storing files and making them accessible from any computer in the house. But when you upgrade to a new one, you're stuck copying everything over by hand, swapping drives, and risking data loss. Here's a much more reliable method.

Title photo made using Denis Dubrovin

Not too long ago, I picked up a brand new NAS with way more space than my old model. It's great, but as soon as I set it up I was stuck with the task of transferring terabytes of data from to the new NAS. I didn't want to physically move drives, since the old drives were smaller than the new ones. Since I was moving from one Synology NAS to another, I could have used the Synology migration tool, but if I were moving from a Synology to a ReadyNAS, or from FreeNAS to Synology, I wouldn't have had that option.

I wanted a platform-agnostic method. After mulling over the dozen or so ways to get the job done, I settled on one of our old command line favourites: rsync.

Why rsync Is The Best Tool For The File Transfer Job

I can hear you now: "Why use the command line for this? I could use [X METHOD] to do the same thing," and you'd be right! There are a lot of ways to get the job done, but rsync has a few significant advantages. Here are the main ones:

  • rsync is probably already on your NAS. Whether your NAS is an off-the-shelf enclosure packed with drives or a DIY model running your preferred operating system, odds are it's running a variant of Linux. That means the command line — and rsync — are already installed. Sure, if you're moving from one brand of NAS to another of the same brand, there may be built-in apps to help, but the beauty of rsync is that it works anywhere, no matter the brand and no matter how big the drives or volumes are.
  • rsync retains metadata. That means details like file ownership, original creation date, modification dates, file permissions, and all that jazz are retained when your copy lands in its new destination. Users and permissions on the new system notwithstanding, if you don't want all of your files to get brand new creation dates, or revision history is important to you, this is a good method to make sure that information is preserved.
  • rsync eliminates the middleman. When you use your PC to copy files from one system to another, your computer is acting as a middleman between the two NASes. That creates another point of failure in the transfer, not to mention keeps your main computer unnecessarily busy. If you've ever done long, multi-file copy operations in Windows, you know this can be annoying, and even the smallest issue will make that copy crap out. When it does, you're stuck trying to figure out where the copy failed, why, and where to start again. Since rsync is a device-to-device copy, it runs whether your computer is on or not.
  • rsync supports resumed transfers, diffs and syncing. If for some reason your transfer dies, like a freak power outage, a system error, or any other problem, rsync is capable of picking up right where you left off without issue. Similarly, since rsync was designed to synchronise directories (and keep them in sync), if you have any need to keep your old NAS in sync with your new one, or you accidentally drop new files on your old NAS that should be on the new one, rsync can move them for you, and will only touch the changed or new files in the process.
  • rsync minimises network overhead. This may not matter to you to much if you do this overnight (which we advise), but one of the best things about rsync is that it doesn't carry the same kind of network load that other tools do. That means your other backups or downloads won't slow to a crawl just because you're using it, and other people on your network probably won't notice anything's going on at all.

As we mentioned, rsync is designed for synchronising files, not just copying them. We've shown you how to mirror systems with it before, and how to sync iTunes with any USB device, but it's also useful just for straight machine-to-machine copies because it's so flexible. It can even give you a handy progress bar so you can make sure everything goes smoothly.

Step One: Enable Remote Access And Choose Your Transfer Method

How to Migrate From an Old NAS to a New One Overnight with rsync

The other nice thing about rsync is that it's just a command line away. You can't just fire up a terminal window and start copying though — you'll need to do a little initial setup.

First, you'll need to make sure that SSH access is enabled on both your old NASes. SSH allows you to log in to your NAS from the command line, using a secure shell. On my Synology NAS, this was right under Control Panel, labelled "Terminal & SNMP." You'll find it in a similar place on your NAS; it's usually under System, Remote Access, or any other synonym for "remote management".

Once you have remote access enabled on both NASes, and you have the admin (or root) credentials for both systems, you're ready to go. Now we're ready to log in to our old NAS and push our files to the new one.

How To Use rsync To Transfer Files

How to Migrate From an Old NAS to a New One Overnight with rsync

Now it's time to get down to business. Use your favourite SSH tool (in Linux or Mac OS X, you can just open a terminal window; in Windows I like PuTTY for this) to log in to your old NAS. In OS X or Linux, just open a terminal window and type this:

slogin [email protected][IP ADDRESS OF YOUR OLD NAS]

You'll be prompted to log in, and once you do, you'll get dumped into a new command line representing your NAS. In Windows, open PuTTY. In the host name field, type in the IP address of your old NAS, make sure SSH is selected, and click Open. You'll be prompted for a username (admin or root) and a password. Once you're logged in, you'll be dropped at the command line for your NAS.

Now it's time to run rsync. The syntax is pretty simple. You'll need to tell rsync how to log in to the remote device, and where to put the files, all in one command. Here's how:

rsync -azP [SOURCE DIRECTORY] [email protected][IP ADDRESS OF YOUR NEW NAS]:[SOURCE DIRECTORY]

So, let's say I have a folder on my old NAS called "old_movies," and a folder on the new one called "new_movies". To copy everything inside "old_movies" to "new_movies", the command would look like this:

rsync -azP old_movies/ [email protected]:new_movies

where 192.168.1.X is the IP address of your new NAS. Once you run this, you'll be prompted to verify that the SSH key for the new NAS is indeed correct, and that you want to store it for future use. You may get a warning about the key the first time you connect, but that's OK. Type "yes" to continue. You'll be prompted to log in to the new NAS with the admin password. Go ahead and do that too. If everything else was successful, as soon as you do, your file sync will begin.

How to Migrate From an Old NAS to a New One Overnight with rsync

Before we go on, let's talk about those flags — the "-azP" — you see in the command above. They're important, and here's why:

  • The "-a" flag in there stands for "archive", and it's important to include. It makes sure that the sync command is recursive (meaning any sub-folders and files inside of old_movies are copied too) and it's important for preserving all of those modification dates, symbolic links, permissions, and other goodies we talked about earlier.
  • The "-P" flag combines two other useful flags into one here that you'll want to use. It combines the flags for "progress" and "partial", and when used you'll get a progress dialog at the command line that shows you which file is currently transferring, what percentage of that transfer is complete, and how many more files are left to check. As each file completes, you'll see an ever-growing list of completed file transfers, which is great for making sure everything transfers successfully. It also allows you to easily resume suspended or interrupted transfers. Combined, you can see how it will show you which file was the last one to go, where it failed, and if it failed, give you the option to resume. It's a pretty powerful combination.
  • The "-z" flag is a handy rsync tool that compresses the files when transferred, which gives you that whole "light on bandwidth" benefit we discussed. If the files are already compressed, you won't get much benefit here, but if they're not, this will get the job done without slowing down the rest of your home network.

You should double-check your command before you copy, just to make sure you're copying to the right directory and directory structure.

When it comes time to migrate your whole NAS, you can do this directory by directory, or you could do it like I did — in one swoop, overnight. Do one small directory as a test to make sure you have your syntax right, and when that works, move up to the parent directory, and copy the whole thing at once. It will take ages, but when you're finished, your new NAS will be set up exactly like your old one was, all ready to go.

If you're a stickler and prefer granular control, create your directories and shares on the new NAS, then just copy the contents of each directory from the old NAS to the new one. That way you've created the directories and granted them permissions using your NAS's interface, and the files inside will retain whatever permissions they had.

Troubleshooting Issues, Updating Directories And Additional Reading

How to Migrate From an Old NAS to a New One Overnight with rsync

If you run into issues with the copy, or with other odd, quirky problems, do some Googling for your error messages and the OS of your old or new NAS. You'll often be surprised how many people have experienced the same problem.

In my case, when I was migrating my Synology NAS, I ran into some quirky "rsync service is not running" errors when I tried to push or pull files, only to find out that I needed to make sure Synology's "Network Backup Service" and "Network Destination Service" were enabled on both the source and destination NAS — two checkboxes buried deep on the Service tab of "Info Center" in the Control Panel. I also had to make sure I logged in as root — logging in as admin wouldn't cut it. Luckily, Stefan came to my rescue with this helpful post on the error.

Similarly, Justin Ellingwood's rsync walkthrough on DigitalOcean's community forums is super-helpful if you want to learn the ins and outs of rsync for the purpose of moving files across systems, and helped me identify the best flags to use and when to use that trailing slash and when not to. If you accidentally write to the old NAS instead of the new one, and need to run a differential sync — that is, use rsync to just copy the changed or different files since your last sync — he has instructions on how to do that, too. If you're an old hat with rsync, this walkthrough at How-to Geek will help you take those skills to the next level.

Once you have all of your files migrated, you're free to do whatever you want with the old NAS. Turn it into a download box, a home theatre system, use it for unimportant files or backups, whatever you want. If you plan to sell or get rid of it though, make sure you wipe it properly and clean it up beforehand. With luck, you can get some cash for it.


Comments

    I've migrated from Netgear ReadyNAS to Synology Diskstation previously with infinite headaches - I wish I had an article like this back then! Very helpful and bookmarked for future use.

    I have a HPN40L running FreeNAS

    I recently used rsync to copy large video files over to a newly installed drive in the same unit

    I found that using the -z flag to trigger compression slowed down transfers and they went much faster without. Was achieving only about 7MB/s with -z, but without it reach up to 50MB/s

    tl;dr Use -z with caution as it might go way slower than without

    A little off subject, but I was wandering if any one knew of a way to set up Synology 214se's to sync with say DropBox or Drive without it becoming some comber some setup?

    I used rsync regularly to back up my nas. Is nice n fast and I used a gui to set it up. Gadmin-rsync make it quite easy

    This is now built into the synology os and runs through a GUI.

Join the discussion!

Trending Stories Right Now