Because big data implementations are often marketing-focused, they can emerge as "rogue" projects that aren't centrally managed by IT. Not only does that pose a security risk, it also means that the projects aren't likely to be drawing on a broad enough range of data to enable unusual insights.
Picture: Getty Images
A survey of 500 Australian business decision makers commissioned by Rackspace found that 36 per cent of businesses had implemented big data projects without consulting their IT department. "Whether we call it a rogue project or shadow IT, it doesn't seem like big data strategy is being driven by IT departments," said "IT has not demonstrated its willingness to move quickly.
To be fair, budget restrictions play a big part of that. "A lot of the IT leaders I speak to in businesses are very much stuck in the world of keeping the lights on," Dorney notes. But he argues that IT departments can still propose ideas even if they're not on the current schedule: "The IT centre needs to bring ideas to the business about how technology can apply to the businesses they're in. They need to show mastery of the technology itself."
When rogue projects happen, a big issue is that they're not necessarily going to draw on the full range of data available within the business. "If you've got these rogues running big data projects, you can have data that's quite dispersed. Part of the value of doing big data properly is relating all the parts. It can be very hard to generate optimal insights."