Servers

Five Simple Rules For Better Metrics

It’s easy to measure almost any aspect of IT performance if you make the effort, but how do you convert that raw data into meaningful metrics that will impress your boss and perhaps increase your budget? Follow these basic guidelines to generate metrics and reports that are meaningful and career-enhancing.

This list draws heavily on presentations given by experienced data centre managers John Parker and Donna Manley at Data Center World in Las Vegas, which I’m covering as part of our ongoing World Of Servers series. Despite the data centre focus, the same basic principles apply to any kind of metrics or reporting in the IT space.

Build detailed weekly metrics then compress

Whatever final metrics you present to others, presenting detailed performance information starts by collating as much detail as you can muster into a weekly spreadsheet. In a particularly intensive environment, you might opt for daily updates, but in most scenarios weekly data should suffice. You won’t necessarily share that with others unless it’s specifically requested, but weekly information makes it easy to create monthly, quarterly or annual reports as needed.

Parker advocates dividing metrics into two groups:

  • Tangible, based on easily defined and measured inputs. These will include tickets for resolution, system uptime and other performance-related numbers.
  • Intangible, based on calculated business figures such as cost or time savings. These will usually involve a degree of interpretation and will draw on the tangible figures.

The data you generate can be used both for operational reports (which give detailed, granular information) and for management reports (which summarise key trends). The latter are those you will consciously share with management. And on that point:

Be selective about what you share

While your manager may be clamouring for metrics, Parker recommends taking a measured approach. “Create your own reports at least three months before sending them out,” he said. “The worst thing you can do is start and then realise you’re not meeting the goals.”

“You really want to see what you have and how to fix it. Metrics tell you what you do and don’t do well. If you have to start reporting, start with what you do well, You can act a little stupid: ‘I didn’t know we needed service levels in there.’”

Metrics can serve as a measure of your job performance, but they also serve as a means of educating managers or clients. “I like to let my bosses know how many times I’ve been called and my team has been called, because if it’s a lot we need to do something about it,” Parker said.

Automate whenever possible, but choose useful over automated

In an ideal world, some sources of data will feed into your spreadsheets directly, but that won’t always be the case. If you need to collate data manually, so be it. “You gotta start slow and produce what you can do well,” Parker said. “It helps if you have tools, but you don’t always have them.”

Don’t be tempted into including numbers just because they’re easy to source from existing systems. “You want to be able to tell that story of how it has benefitted the organisation,” Manley said. “One of the metrics that we used to capture was around our managed storage systems utilisation, but we weren’t able to capture the detail behind them. The information that we were garnering form these particular metrics was not telling us a whole lot.”

“In the same way, the number of total changes is OK as a figure, but the number of changes just relates to a benchmark point,” Manley said. “We know we do a lot of changes — what’s important is how the changes relate to issues we have in operation.”

Focus on usefulness and regularity

“One thing you have to really work on is accuracy and consistency,” Parker advised. “If your reports look different every month, it’s going to confuse people.”

Changing format is risking. “They’re not going to want to read a five or ten page report or go looking for the stuff they want,” Parker said. Even with a consistent format, an email highlighting particularly notable figures (good or bad) can be worthwhile. “If everything’s good, they may not read the report but if things aren’t, they may.

“If you do anything with metrics make sure it’s smooth. It has to be easy to read and has to flow. Get other team members to look at it first. Does it make sense?” It’s also worth comparing each report to the previous period to make sure the figures remain consistent.

Recognise that needs change

“Don’t be offended when you put all this time and effort into creating metrics and graphs and you give it to management and they say ‘I don’t really like this’. Find out what they really like and do that. Every time I get a new CIO or a new VP, they don’t like it and they want it a different way,” Parker said.

“Those metrics can’t stay stagnant,” Manley agreed. Some of the older ones we had, when I look at them I go ‘what was I thinking when I asked for this metric?’ because they’re absolutely ridiculous. Don’t stay stuck in that same place.”

Done well, metrics can be a useful career boost. “In operations support, you’re always in the basement,” Parker said. “They don’t ever see what you do. Metrics and reports give you visibility.”

Lifehacker’s World Of Servers sees me travelling to conferences around Australia and around the globe in search of fresh insights into how server and infrastructure deployment is changing in the cloud era. This week, I’m in Las Vegas for Data Center World, looking at how the role of the data centre is changing and evolving.