Are your metrics and KPI’s telling the truth?
In technology, we are bombarded with stats, percentages, KPI’s and scorecards. Things like customer satisfaction rates, cost per transaction, RIO, response time, time to close, and system uptime are all part of our vernacular. Interestingly enough, we have blind faith in the accuracy of these numbers. Often our department’s success is determined by a few numbers that are trusted without proper scrutiny. We set goals and pay bonuses on these metrics. Have you ever stopped to review these numbers to determine if they are providing us with the truth?
The deluge of data and statistics is used to validate statements, sell products and sway the thoughts of the Executive. This can be a dangerous strategy without the data being put through some validation. Placing any information in such high regard without a proper review leaves an organization with the risk of potentially creating improper behaviours or generating unknown mistakes.
A key example of such a situation lies in the analysis of satisfaction surveys. These are commonly used by organizations to measure their IT operations. Assume conducting a satisfaction survey and achieving a score of 90%. It appears the organization is pleased with its IT service. You may be happy with this result, as you hit your KPI target and everyone gets their bonus. However, is it a true measurement of customer satisfaction? It all depends on the question and how it is asked. If the question was, “Are you very unhappy with the IT department?”, then the results may be suspect as it does not provide a strong representation of satisfaction, only of displeasure.
