Amazon's new Amazon Prime Music service sounds astonishingly good, with ad-free music on and off-line for Prime subscribers. I wouldn't hold your breath for the service to launch in Australia, however.
The service relies on having an existing Amazon Prime service, and that's something that Amazon has never actually launched in Australia. Whenever Amazon has been queried about the kinds of rights that would be needed for Prime-style services in Australia, the only comments it has made on the record have related to rights situations in Australia being "difficult".
You could conceivably hook into Amazon Prime Music via a VPN, but you'd have to do more work than for, say, Netflix, because Amazon is significantly stricter when it comes to addresses and credit cards when it comes to Amazon Prime registration. An Australian credit card generally won't cut it.
Whether it would be worth the effort is also questionable; while it's true that no one music service has absolutely everything that everyone wants, there's a wealth of competing music services in Australia already. As such, if Amazon were to venture into local waters, it seems more likely it would launch with its exclusive TV content rather than try to take on the likes of Spotify, JB HiFi NOW and Rdio.