Chrome only: Google Chrome extension Google Reader Notifier not only shows your current unread count and adds quick access with a popup dialog window, but it's also the official Google notifier for Reader.
Once you've installed the extension and made sure you are logged into Google Reader, you'll immediately see the unread count in the notification icon. If you head into the options page for the extension, you can have it open Google Reader instead of a popup dialog — if you already have Reader open in a tab, it will simply switch to that tab instead of opening a new one. There's plenty of other notification extensions out there, but since this is the official Google extension, it's probably the one you want.
Google Reader Notifier (by Google) [Google Chrome Extensions]