When a report on how many hours have been worked in your company over the last month takes 19 hours to run, it’s reasonable to assume something has gone wrong. But how do you find the cause?
Horrified picture from Shutterstock
Australian infrastructure delivery company Tenix was experiencing that problem with its monthly reports, which cover 1200 users across some 70 sites. IT manager Andrew Pamfleet eventually identified the cause as I/O bottlenecks on the storage area network in its Melbourne data centre, which relied on an ageing HP EVA storage array.
Six months ago, Tenix upgraded its storage to a Nimble Storage adaptive flash system (which uses SSD for cacheing and relies on processing power to rapidly write data to conventional hard drives). That has resulted in an immediate improvement, Pamfleet said at a media lunch to launch Nimble’s new CS700 array. The report now runs in a much more reasonable 2 hours and 23 minutes.
“Our JD Edwards ERP system had some reporting issues,” he said. “With the Nimble box, we were able to turn that around. We haven’t been able to get the box to break a sweat as yet.” The smaller size of the gear has also reduced per-rack charges, since the system takes up just three rack units, rather than 48.
Incidentally, that doesn’t mean the existing HP kit is being dumped. The system is being redeployed in a non-production environment outside the hosting centre.
The lesson? While it’s tempting to blame software performance problems on the software, the cause often lies elsewhere — and solving it may deliver additional cost benefits.
Comments
4 responses to “The Horror Of The 19-Hour Report”
But throwing money at the problem is only a bandaid solution, think about it, if the company increases in size that means more users to process and thus the time it takes to run the report would start blowing out again.
I would be asking a few more questions about the report and its design as well. Roughly 80% of all database performance improvements is optimizing your SQL code in the application or report.
JDE Reports taking a long time to run?? Say it ain’t so!
I concur with Marty. 1200 users, 19 sites, report of hours worked in a month… Let’s be generous and say users are logging tasks averaging 15 minutes each, which is extremely high resolution from what I’ve seen from most time logs. 32 logs a day, 5 days a week, 4 weeks a month, 1200 users, 768,000 records in total per month. Really seems like small fry numbers. They must be wrestling with some kind of unavoidable handicap, like poor database design or just generally bloated, awful enterprise systems, and have to throw raw power at the problem.
Sadly this is probably true, there is a lot of terrible enterprise software out there, and the CxOs are just sheep and install whatever the bigger one in front of them has, regardless of whether it is any good or not. And as much as we don’t think about it much, poorly performing software consumes more power, and thus can have a significant carbon footprint of their own. Fortunately tuning one poorly performing application can significantly reduce the power footprint of an application. No other industry can reduce their carbon footprint so easily as the enterprise software industry.