Firefox only (Win/Mac/Linux): Firefox extension Google Reader Watcher adds notifications for unread items right in your web browser, complete with a quick-access menu that takes you directly to the feed.
Once you've installed the extension, you'll need to take a trip to the add-on options panel to enter your Google account information, where you can also filter by label, force a secure connection, or choose whether or not to open Google Reader in a new tab. Hovering your mouse over the status bar will show a tooltip with the updated feeds, but right-clicking on the icon will bring up a menu that you can use to drill down directly into that feed in Reader—very useful for those of us with a massive number of feed items.
Google Reader Watcher is a free download, works anywhere Firefox does, and is a good replacement for the <Google Reader Notifier for Firefox, which has been turned into malware by a greedy extension developer. If you'd rather have your RSS notifications outside of your browser, you can check out the Google Reader Notifier for Windows.
Google Reader Watcher [Mozilla Add-ons]