Windows: Portable applications are handy for running applications from a USB stick or using an app without messing with your Windows installation. For the remainder, Zero Install can run software without installing it.
The app features a sort of makeshift app store for a number of included applications. From a single menu, you can download from a host of popular apps, including Audacity and Notepad++. Zero Install will then keep them up to date.
You can also set up Zero Install to sync installations between computers so that it's always up to date no matter which machine you're on.
Run Windows Programs Without Having to Install Them [Make Tech Easier]