If you're in need of an older version of an Android app, or are just curious about the free, non-Market apps and wallpapers out there, you can use a few custom Google searches to scavenge the web.
The basic search for an Android-install-ready APK file is this, as noted by the Tech-Recipes blog:
-inurl:htm -inurl:html intitle:"index of" apk
You can then modify and add on to that search to find "games", "live wallpaper" and the like. If you're going to install apps outside the Market, though, you'll definitely want some kind of security/scan solution, and there are free scanners available, as noted in our Android malware explainer.
Hit the link to brush up on more ways to find apps for your Google-powered phone through Google, including searches of popular file-hosting services.