Advice needed: IT department efficiency criteria

[byh.livejournal.com]

This is not a rant. Repeat - this is not a rant.

A few days ago our Very Big Boss decided IT department in the company needs more control and supervision. So here he comes and asks us to create some efficiency criteria we could report so higher management would estimate how well IT department works (if it actually works at all), is the department effective enough to address current issues, etc.

Any ideas?

Comments

[ohmyhead.livejournal.com]

Yes. Tell your Very Big Boss that he is paid absurd amounts of money to come up with this shit himself.

[wyrdrune.livejournal.com]

For $DEITY's sake don't let anyone come up with "fix time" or "number of outstanding job count" based metrics - they don't bloody work!

[byh.livejournal.com]

Well. As he also owns a large chunk of the company your suggestion is not a really good option. :)

[ulnagar.livejournal.com]

Look at SLA's with clients, internal and external.

Then perhaps look at what *might* happen if you were to move all internal IT work to a zero cost centre method. I.E. all work is billed internally meaning that there should be no budget directly allocated to IT per se, but all cost should be recovered via internal journalling.

You could then judge the efficiency of the department against the SLA (even if you made it up yourself) and the cost effectiveness against the zero cost centre.

[byh.livejournal.com]

We have a sort of helpdesk here. I was thinking about something like opened/closed tickets ratio within a specific time period.

As for fix time - in no way.

[wyrdrune.livejournal.com]

The problem with that is that it'll still encourage people to close tickets sooner to get the ratio up. You'll probably find a rush of tickets being closed towards the end of the time period.

[byh.livejournal.com]

Can't think of anything much better without getting it all too complicated (like this idea (http://community.livejournal.com/techsupport/1501431.html?replyto=14848247))... The original helpdesk idea was with users confirming closed tickets and only then ticket would be marked as "done". But real life shows that they never confirm.

[byh.livejournal.com]

Interesting but I am afraid it would be too complicated for us.

[wyrdrune.livejournal.com]

True, very true - we have problems here with users not signing off on things - I've been tempted to suggest to management that we publish a "name & shame" league table of how long it takes users to sign off on jobs. :-)

[forever-damned.livejournal.com]

User Satisfaction based on survey? Time taken to complete jobs? Average number of jobs completed in a day?

I mean really, this is your bosses job, not yours.

There's a whole bunch of metrics for my company but I'm SOHO onsite, so it works a bit differently.

[none-more-dead.livejournal.com]

http://crappygraphs.com/lets-have-a-meeting-to-discuss-the-implications-of-this-graph/

[gilmoure.livejournal.com]

Track user calls and time spent per user. Identify the ones that are the biggest time soak. Will likely be admin assistants, always asking for wires to be made invisible and how to cut and paste.

[mouser.livejournal.com]

I have SO saved that picture!

**************

They are trying to outsource your department. Which has seldom done any good for most support, and virtually ALWAYS costs more then keeping it inouse.

Group the basic "types of jobs" you do, with an estimated cost of outsourcing the job. There are certainly several companies in your area that do this sort of thing. Remember to include additional down time while you wait for them to show up.

[brotherflounder]

Number of days spent without hitting a $LUSER with a clue-by-four.

[lillyflowers.livejournal.com]

"They are trying to outsource your department. "

IAWTC. Many, many times I've seen bosses come and ask for things like this and every time somebody gets the axe. Polish up that resume and start looking — just in case.

[superbus.livejournal.com]

Forget that. That lets people cheat in a bad way. The ideal will be less towards helping the customer and more towards getting that ticket closed ASAP.

I recommend a survey system, if they're looking for metrics. Even though I have issues with survey systems, and ways to get around THAT, I do agree that it's overall the best way to get out of a sticky situation. My company does things like ask questions (how was the service? How was the professionalism? Would you recommend $COMPANY?), and has three possibilities or general categories: "Very pleased", "Pleased" and "Not pleased". I don't agree with that, and would prefer a graded system; put up some questions with a 10 point scale. This way, you can get an average as to how your techs are, can reward those at the top and work with those near the bottom, and handle any wild fluctuations.

[canray.livejournal.com]

Size of bar tab at end of day.

[the-s-guy.livejournal.com]

One problem is that most of the metrics don't boil down to anything more useful than "Here's how we were doing compared to last month." No techsupport team can be compared directly to any other TS team because the environments they operate in vary so much and have enormous impact on the resulting effectiveness. You can rate individual operators against the average of other people in the team with the same time in the trenches, but that does require a significant amount of techs in order to get data above random noise fluctuations. You can't even go back and compare against previous techs, because again, the working environment may have changed.

In order to compare the team as a whole versus other techsupport teams, you really have to compare the working environments, and the vast majority of the variables which affect this are controlled either by IT policy or corporate management. Everything from how often the incoming hold message or IVR voice clips are reviewed and updated, through to how much control there is over the user experience (inhouse has more than a specialist app team which has more than a public ISP), what the hiring and staff review policies are, HR policies in general, remuneration compared to other local employers, staff turnover etc.

If the employer is hiring experienced techs, providing quality TS tools, and has a total employment 'experience' which makes people want to stay longer than three months, then they're going to have a more efficient team. Especially if they let the lower levels of said team either set certain corporate policies or at least be able to have a good chance of improving them when they need improving.

And yes, someone is trying to outsource you. Depending on the politics at the upper levels and who's pushing for it, this will often mean that even if you can provide solid gold proof signed by the Pope and Joss Whedon that your team is more efficient, effective and cheaper than outsourcing, you will still be outsourced. Usually because the person pushing it thinks that they can use the resulting confusion to fiddle the numbers and get a promotion or bonus out of it.

Basically, find out whose agenda it's on. If it's the CEO or one of their favorites/relatives, you're boned. Get out now and make sure your name is on a lot of stuff so that you can sell yourself as a specialist consultant to whichever third party is handed the contract.

If the CEO isn't pre-sold on it, make a pre-emptive strike. Put some business-sounding numbers together (money, money, real costs and budget blowouts of other outsourcing attempts in the same industry, articles from Wall Street about re-insourcing, bottom lines, money moolah and cash. Bonus points if you can scrape up outsourcing disaster stories from their golfing buddies) and book ten minutes on their schedule so you can personally hand them the "project outcome projections". Tell them that there are some interesting results that you weren't expecting, and that perhaps they'd better look at it themselves. Don't label it 'good' or 'bad' - if you summarize it, they won't need to read it. And update your résumé anyway.

[mouser.livejournal.com]

IT does still need it's overhead budget.

That is a good point though. The problem isn't what IT spends, it's that IT isn't spending it on toys for itself but on services internal to the company.
"Before you can find out where you're going, you have to know where you are."

You could start by breaking down tickets by urgency vs. department/user as a measure of where all the IT resources are being spent. These are the kinds of metrics your CIO/CTO needs to make, but I'm assuming you don't HAVE one of those, nor a professionally trained department head, nor even someone whose primary job is "Boss of the department who deals with all the paperwork and not the techinical day-to-day stuff" except you.

Hmmm - Try

[Error: Irreparable invalid markup ('<a href-http://www.cio.com/topic/1406/metrics>') in entry. Owner must fix manually. Raw contents below.]

IT does still need it's overhead budget.

That is a good point though. The problem isn't what IT spends, it's that IT isn't spending it on toys for itself but on services internal to the company.
"Before you can find out where you're going, you have to know where you are."

You could start by breaking down tickets by urgency vs. department/user as a measure of where all the IT resources are being spent. These are the kinds of metrics your CIO/CTO needs to make, but I'm assuming you don't HAVE one of those, nor a professionally trained department head, nor even someone whose primary job is "Boss of the department who deals with all the paperwork and not the techinical day-to-day stuff" except you.

Hmmm - Try <a href-http://www.cio.com/topic/1406/Metrics> HERE </a> for a starter if you're going to try to do this. There are NO easy answers to this.

Also <a href=http://downloads.techrepublic.com.com/abstract.aspx?&q=metrics&docid=173305&promo=110000> THIS </a> is a free download about HelpDesk metrics.


Good Luck!

[mouser.livejournal.com]

Oh, and a LOT of metric monitoring costs money....

[byh.livejournal.com]

Thanks for the reply. I do not think outsourcing is a problem for us - here in Russia things are a bit different and outsourcing tech support is not that popular yet since local staff is pretty cheap and companies selling techsupport to corporate customers are relatively new and charge too much.

However you are right - when upper management comes with ideas like this then trouble is looming. In our case I think they are going to try and cut expenses on salaries in IT dept by means of fines and other sanctions. So far IT department is one of few that has stable income no matter what while the most other departments sometimes suffer fines and pay cuts when things get bad and they are unlucky enough to be blamed.

[byh.livejournal.com]

I used to attend weekly meetings with our CEO when I worked in a smaller company. Could not agree more with the picture.

...
Although it were the meetings which helped me to finally setup ICQ on my mobile - and when I did setup it meetings become much easier to attend except those moments when CEO felt like asking me a question and I never heard it.

[byh.livejournal.com]

Well, I do not think it is outsourcing they have in mind but they surely keep that axe beyond their back. Perhaps something like cutting our pay during 'inefficient' times.

[ianhess.livejournal.com]

Our team had some luck refocusing into subteams with 1 or 2 metrics each. Our tier 1 group was judged as a group (not individuals) about the number of cases that didnt get directly picked up and handled by a warm body, for instance. Judging a team as a group builds a desire to succeed together rather than gaming or sabotaging the metric system. Its not hard for a team lead or manager to figure out if a team member isnt pulling their weight.

Its not too hard, especially in large organizations to show how any metric involving ticket numbers is very bad. As anyone in IT knows, one ticket is not a unit of work, or even average unit of work. The closest we've been able to manage is reporting against ticket number by tier responsibility, because that roughly breaks tickets out into similar levels of time and difficulty. Ticket numbers are also incredibly susceptible to people gaming the system. Is a ticket 1 customer contact? Does each question from a user become a separate ticket? How do you keep the potentially time consuming tickets from being avoided by some team members?

I think the way to go is focusing on SLA for high severity tickets, backlog numbers (tickets open more than a week), customer surveys for satisfaction, and escalations due to customer temperature vs ticket difficulty.

[byh.livejournal.com]

Thanks for links - I'll study them and try to get some ideas, seems like it is much more complicated question than it initially seemed to be.

Yes, it is definitely not my job to get through all of that but our head of department does not have enough experience in this stuff either so he kind of asked for help. Too bad I can't simply direct him to these resources because he does not know bussness and technical English well enough. Plus a lot of adaptation to our specifics will be needed.

Although there is an advantage in the complexity. I am more than sure that the main point of raising such questions is an attempt to save some money cut from IT bugdet (pays mostly). So costs of implementing such monitoring may be impressive enough to give up the whole idea as it is. :)

[byh.livejournal.com]

Actually I was thinking about something as simple as inserting a simple survey right into helpdesk system so users would have to answer a question or two before they could submit or confirm tickets.

[sethb.livejournal.com]

Remember the fallacy of static analysis: you can't (correctly) assume that things will stay the same when you start measuring them.

Whatever you measure (and consider good) you'll get more of. The way you get more of it might not be to your advantage. (E.g. when a company started giving debuggers bonuses for each bug found and fixed, there were shortly a lot more bugs found and fixed. The code didn't get any better, rather there was a black market in bugs.)

Informal hard-to-measure criteria like "are the users happy" and "does the software help the company make money, or does it get in the way" work. Assume that the techs are going to game whatever your measure is.

[siliconshaman.livejournal.com]

There's only two metrics you need to worry about:
Jobs in per week
Jobs fixed [where user & tech say it's fixed] per week.

If the second is approximately the same as the first, it's a good week!

Although you might want to keep a tally of number of jobs called inper week by user, and how long they last. Because of the 80/20 rule... ie 80% of the work is caused by 20% of the users.

Allows you to identify which users cause the most work, and then you can do something about them. [either work to get rid of them or try to identify why.. although about 90% of the problems are not so much what, as whom.]

[byh.livejournal.com]

I was thinking about something like that as a quick solution. Perhaps we could arrange these reports into existing helpdesk system.

Identifying problem users would be especially useful.

And if they'll want more sophisticated analysis I'll give them something I figure out from the clever stuff from comments above.

[byh.livejournal.com]

Good points.

We do not have any metric monitoring yet but still we already suffer from gaming and techs tossing potentially problem tickets from one to another. Too bad we do not have enough of them to build teams but perhaps we'll figure something out.

[byh.livejournal.com]

Good point.
We know it too well.

Reminds me of a period when techs were substantially rewarded (could easily get 150% their monthly pay) when they were willing to resolve issues during weekends (while normally company does not work on weekends). And then they'd get a day off for each weekend.

All of sudden there were so much critical issues requiring immediate attention that almost every tech got double pay and they were missing half of working week taking their earned days off. Not that quality of service really improved though.

[siliconshaman.livejournal.com]

Also thought of something else. Keep track of who does which jobs, because the 80/20 rule also applies to the techs... ie 80% of the work gets done by 20% of the people. [well, if one is being cynical].

But you'll find that some people only take quick jobs and get a metric ton load of 5 minute jobs done.. while others prefer the meaty complex problems, so don't get as many done but generally have a higher rate of customer satisfaction [because 9 times out of 10, they'll have been through all the others first and will be quite fed up.]

And if you can't blind management by your brilliance, befuddle them with bullshit, MBA types like lots of pretty graphs.

[taleya.livejournal.com]

If you do anything relating to "How fast shit gets fixed" make sure you include next to that lovely chart (they love charts) a % of tickets raised that had useful information. And divide into "Low priority" (My mouse smells funny!) and "High Priority" (HOLY SHIT THE COMMS ROOM'S ON FIRE!)

You should be able to work out some rough guidelines to response times - Very Low = XX days up to High = X hours. SLAs are a good base for this, when you mix it with ticket priorities, you can get a good idea of whether or not people are pissing about with the stupid shit when the big shit needs doing...and cover your arse when they go "Why wasn't XXX's mouse replaced for a week???" "Uh sir, if you recall, the comms room had exploded at the time...."


And if anyone mentions "user feedback surveys" stamp on their heads, quickly.

[syberghost.livejournal.com]

The bottom line to everything meaningful you could measure is this:

The satisfaction of your customers. However, you can only meaningfully track that with a valid statistical sample of the work you've done for them. Naked user satisfaction surveys don't work, you get too much feedback from people who've had very little work done and too little from people who've had tons.

So the first thing you need is for your ticket system to be meaningful. EVERYTHING goes in it. If somebody sends you an email that you have to actually do some work to answer, you need to stick it into a ticket; and you need to make sure the customer sees that ticket, because it may come back to him later.

Then you do TARGETED customer satisfaction surveys. You pull a random sampling of the tickets, you contact the customer, and you find out if it was completed to his satisfaction.

Any measurement of the time tickets remain open isn't part of this; it's valuable internally to your manager to make sure people are closing their damn tickets, but what matters is that the customers think they're being helped. Whether or not they actually ARE getting good service matters only to the extent that it contributes to them thinking you are; the fact that they think they're helped or not is what keeps your job yours.

This is what your manager does anyway when he pays attention to what people tell him about your department; this is just a matter of formalizing the process, 'cause an anecdote ain't a metric.

[tecie.livejournal.com]

If they're looking to improve efficiency, they'll have to gain some sort of numbers, in the form of metrics. However, make sure your metrics are intelligent - Use a formal and complete ticketing system with appropriate severities and change control, where appropriate. Most ticketing systems are highly customizable in this regard.
At the same time, it's up to you in the IT department to commit to a series of process and procedures. That's not to say become procedure Nazi's -- but canned fixes for particular common problems can alleviate a number of problems with different techs having different methodologies.
That said, document everything and either hire, promote, or elect someone to represent your team to upper management. This person should be responsible, and should NOT -- under any circumstances -- be on either side of the fence. If the person is a corporate yes man then everyone is screwed, and if the person is 100% loyal to the IT group and not to management then the result is the same (however slightly more fun.)
Finally -- talk to your users. When you're creating or improving your processes, make sure the users are getting a good experience out of it. A lot of times, it helps to trim the fat off of the processes. However, don't bother with after call surveys. They're just worthless. Instead, offer regular opportunities for users to comment on IT in general.

[notthebuddha.livejournal.com]

Ticket issues should be measured by a bottleneck standard: how much work is issue X preventing from being done, or how much time is being spent working around it instead of doing something productive, or how much revenue is being held up, or how much of an unnecessary expense is at risk. Let the issue-submitter set this number themselves, and cc their supervisor each time. This will give everyone involved an immediate and fairly objective, quantitative idea how important the issue's resolution is, and an idea of how much could be spent to gainfully resolve it.

When the issue is resolved, likewise cc the original submitter their supervisor for short-term accountability, and periodically compile the empty bottles to justify IT's budget with verifcation from other departments built in.

[the-s-guy.livejournal.com]

Yup, it's a trouble warning all right.

[pat-barron.livejournal.com]

The problem is, if you're looking for a simple solution - there really isn't one. At least, not if you want to get any meaningful information out of the process.

If you're in an environment where work primarily comes in asynchronously via a ticket system, there are some particular metrics you want to track, for example: the amount of time between when a ticket comes in, and when a tech picks it up; the amount of effort (in terms of tech time) spent working on a ticket; the amount of time a ticket sits idle because nobody's working on it (while there are techs available); the amount of time a ticket sits idle because there are no techs available to work on it. To do this, you also have to keep track of how your techs spend their time - for instance: the amount of time spent working on tickets; the amount of time spent on "administrivia" (in meetings, in required training, etc.), time spent on professional development (taking certification classes, etc.), and other miscellaneous non-billable time. And of course, your techs will resist this, that's to be expected - it's a pain, and (especially if you have problems with gaming on the clock, and things like that) it puts them under a microscope that they'd probably rather not be under. But you need this information to draw meaningful conclusions about how much time tickets langish because there's no one available to work on them, and you can actually use this kind of info to lobby for more resources, headcount, etc., if you're chronically overbooked.

You also want to keep track of how much time it took to close tickets of particular severities, and how often tickets were closed beyond a reasonable/agreed amount of time for tickets of that severity. An SLA is really helpful here - not only to define your goals in how long it takes to close a ticket of a particular severity, but also what the severity levels actually mean. That way, when someone enters a "high severity" ticket because they want their keyboard cleaned or something, you say "Does this affect more than a hundred users? Or does this impact more than five digits or so worth of revenue? No? Then sorry, it's not high severity - end of discussion..." If you don't document your severity levels in an SLA or something, then the severity level means whatever any random user decides it means...

[pat-barron.livejournal.com]

Oh, and another useful stat that you'll really want to track, apropos of something someone said above: the amount of time a ticket sits idle, because you're waiting to get required information back from the user, or from a third-party (such as an outside vendor), and/or waiting for some particular event to occur. Even if you have tech time available, just because a ticket is sitting idle, doesn't necessarily mean it's because nobody has chosen to work on it - you may be waiting for someting else outside of your own control, that you need in order to make progress on the ticket.