Download The Entire Archive Of NASA's Astronomy Picture Of The Day With One Command

NASA's Astronomy Picture of the Day archive is a packed with awe-inspiring, high-resolution images of space that, incidentally, work great as desktop wallpapers. But NASA doesn't offer an easy way to grab everything. Command-line tool wget to the rescue!

A little backstory: My wife is a graphic designer working on a project related to the Griffith Park Observatory here in Los Angeles. She was looking for high-res images of space, and I knew Nasa's APOD archive was full of great images—but grabbing them all isn't easy, and clicking through the archive is tedious. So I turned to wget, a cross-platform command line tool.

Windows users, you can grab wget from here; Mac users, here's a good one for you. (Follow the installation instructions in the download.) Linux users should have it installed by default. Assuming you've got wget installed, open your command line, then cd to a folder you want to store them in (I've got mine in a folder on my capacious Media drive called wgot. The command for downloading the archives is simple:

wget -r -l2 -t1 -nd -N -np -w2 -A.jpg -erobots=off http://apod.nasa.gov/apod/archivepix.html

This command will crawl the archive page, follow any links, and eventually start downloading only the JPG files in the domain. The -w2 option adds a two-second wait between each ping—something I didn't include when I did it, but... I also don't want us to absolutely kill their servers, so hopefully that will help.

Everything downloaded takes up about 2.64GB on my drive, and you'll end up with a few low-quality dupes of the higher quality stuff, but you can easily sort by size and get rid of some of the lower resolution stuff. I'm a wget dabbler, so if you've got any improvements, let's hear them in the comments.

Astronomy Picture of the Day Archive [NASA]


Comments

    this looks to be a socially engineered ddos - what will happen if a large number of people decide to do this?

      how do you download them all at the same time, i tried using the command code they game me on Terminal but it doesnt work. cheers

    nice one ian! when are you coordinating one?

    Since the US comments link NEVER works, Adam Pash posted the torrent of the apod.

    http://adampash.com/downloads/apod.torrent

    (Also it seems the wget command doesn't work on the mentioned windows wget prog)

      the torrent
      http://adampash.com/downloads/apod.torrent

      It seems to be downloading only to around 99.5% and although many have got that amount, it seems that no body is still connected who has the last few file. Any one help please?

        I noticed the same thing, only have 4meg to go, there are 15seeds and no one handing out the last bit.

        Ditto here on only needing the last 4-5 meg.

        Adam - any chance you could seed it out for a bit to create a few more seeders?

        Thx!

    I'm also stuck at 99.5%, all when need are those last few megs. Somebody with them please seed.

    It's been stuck on 99.8% for the pass 3 days. Can someone please seed. I'll perma seed for the next several weeks if I have too.

    Days later I'm still stuck at 99.84%. 29 other peerss at the same point with 5 more loading happily away but soon to hit the wall...

    Any hope at all that someone can seed the archive? I'll seed for a week or two after I get that last bit...

    Whilst it's only 99.84% complete, 7-zip managed to extract almost all of the images, only failed on the last 8. All is not lost.

    Last time I checked there are 478 people scouring the internet for the last 0.16% of the torrent.

    Does anyone know a way to contact Mr. Pash other than to comment here? That 0.16% would take about a minute to hit, and once it did, the rest of us could seed it.

    25 days later, there are 235 people looking for that last, elusive 0.16% of the original file...

    In itself a pretty interesting phenomenon...

    Hi All, WGet does work within Windows, its just a bit fiddly. In a nutshell download from the above link in the main text of this article and install it in the windows or system 32 directoy. When you open command prompt it looks in here for the program so it should allow you to run it. Secondly and ill probably get shot for this remove the -w2 switch from the line as it takes ages to search otherwise. It took mine 20 minutes to start downloading on entering the following search.

    wget -r -l2 -t1 -nd -N -np -A.jpg -erobots=off http://apod.nasa.gov/apod/archivepix.html

    The it will likely download into the c:\doc's settings\'your username' root :)

    Have fun...

    99.8%!! this is maddening!

    Adam!! Could you please seed for like an hour? There are still 30 people who haven't given up on it!

    Question:

    Is it legal?
    What about the copyright issues?

    There's no such thing as a free lunch ...

    Guys, don't fall in the trap. Could cost more than you might imagine.

Join the discussion!

Trending Stories Right Now