For years, companies have battled advanced malware and sophisticated cybercriminals in a reactive mode: Purchase new technology to defend against emerging security challenges and regulatory requirements and hire more people and partners to manage it. Additionally, in many cases, companies will buy technologies they’re not ready or able to deploy simply to “check the box” and show action against the latest shiny-object threat.
What happens next? Security leadership may fail to implement key performance indicators (KPIs) that evaluate whether the security tool is actually working as intended and bringing value to the security program. For many companies, this negligence isn’t intentional; they just don’t know how to measure security effectiveness.
Security is notoriously difficult to quantify. For example, how do you calculate the value of stopping 99 percent of malware targeted at your enterprise? On the one hand, it might mean you prevented a catastrophic breach from happening. On the other hand, it might be that something in the 1 percent that did get through resulted in a catastrophic breach. Or, on yet another hand (yes, we have three hands), perhaps you think you have prevented a catastrophic breach, but as you’re reporting your great success upstairs, an adversary has already been on your network for several weeks and is in the process of completing a catastrophic breach. This type of scenario is why many security professionals get a headache when they’re asked to produce meaningful metrics on their operations.
However, there are business metrics that can be applied to security that can provide meaningful guidance on technology performance, operational excellence, and return on investment (ROI). They typically fall into three levels:
Tactical level:
These metrics relate to specific product performance, and they must be developed in the context of the greater security operational model. For example, having a tool that generates an enormous amount of alerts is not necessarily a metric indicating success. If those alerts are mostly false positives or redundant, the large number is actually counterproductive.
However, it is possible to define KPIs around “alerts leading to successful outcomes,” where the goal is to actually reduce the amount of analyst time wasted on useless alerts while enabling analysts to focus on alerts indicating actual events worth addressing. This also makes it possible to define the amount of time each analyst is spending on remediating actual problems vs. chasing false alarms, which is relatively easy to translate into ROI metrics.
[ Are you thinking enough about data integrity? See our related story: Sensitive data: Time to rethink your definition. ]
Leadership level:
These are KPIs focused largely on operational time, spend, and resources. These can help CISOs articulate ROI to the business (such as with the analyst/alert example used in the Tactical section), and also help them make operational decisions.
For example, “tools per analyst” is often a very useful KPI because it defines the optimal number of tools each analyst should be able to manage. This helps define optimal staffing levels and can also indicate tool issues if an analyst finds they simply do not have enough time to manage all the tools (again, the tactical alert example shows how a single misconfigured tool can cause operational waste).
Board level:
These metrics demonstrate, in clear and easily understood business terms, how security investments are strengthening corporate risk posture. Being able to show alignment between technology, operations, and business objectives can be an extremely effective way to prove value to the C-suite.
Unfortunately, many security organizations do not take this “inside-out” approach to security measurement, where they report on the things they can control and measure. Instead, they take an “outside-in” perspective, where they start with external threats and the tools they’re buying to combat them. This outside-in approach makes it impossible to prove value (buying a tool to stop a type of threat does not prove anything), and it drives the bloated infrastructure and operational problems that are perpetuating the breach epidemic.
Security optimization and measurable KPIs can help organizations make enterprise security radically stronger, simpler, less costly, and more accountable. It will also enable them to enhance their security posture and demonstrate security effectiveness and risk reduction in a way that everyone can understand – right up to the boardroom.
Want more wisdom like this, IT leaders? Sign up for our weekly email newsletter.
