Windows: Do you have iTunes installed? Are your sound drivers up to date? Chances are that's all you'll need to stream music between computers using SpeakerShare. The free, no-install app is easy to set up and get streaming.
SpeakerShare requires only Bonjour for Windows, which installs itself when you install iTunes, but can also be implemented separately. It also needs a sound card that has a DirectSound, "wave output", or other means of broadcasting itself — generally, the stock Windows sound drivers don't provide this, so you might have to track down your soundboard/card manufacturer. After that, though, using SpeakerShare is as simple as starting it on one computer, hitting a button, then starting SpeakerShare on the computer that's receiving and hitting "Connect". It worked between two Windows systems on my home network, and, unless you run into sound driver problems, it will probably work for your systems, too.
SpeakerShare is a free download for Windows systems only, requires the Bonjour software to run.