There's nothing wrong with cloudy ice cubes. Yes, what you're seeing are "impurities", but they're the same impurities in tap water, which is perfectly safe to drink. But if you prefer clear ice cubes, you can easily make them with boiled water.
Photo by David Smith
Over at About.com, chemist Anne Helmenstine explains that clear ice is formed from pure water that doesn't contain any dissolved gases. When you see cloudy ice, you're seeing crystalised "impurities," like calcium or fluoride. Clear cubes are usually made using water that's been purified through reverse osmosis or distillation. But you can make it from tap water, too. It just has to be boiled.
Helmenstine recommends boiling the tap water twice. Then, let it cool a bit for safety, and pour it into an ice cube tray and freeze it. It's pretty simple, but if you need more info, head to the link below.