GŲteborg, 23 Jan, 2012
ROI and SIEM
If youíre looking at this, youíre probably hoping I am going to tell you some magic metric or secret formula you can use to measure your return on investment (ROI) on a SIEM deployment. I know, I know, people have been talking about and wishing for this for years, but just like flying cars, one has yet to appear that works for the masses. Sorry.
But what we can do is close; we can come up with a measure of success (or failure) based on why you needed a SIEM in the first place, and on how you are using it now. If youíre taken a look at my previous posts, hopefully what youíve taken away from them is that you really need to plan for a SIEM and dedicate the resources to make it work. Based on that, weíre going to come up with a ruler by which you can see if you are getting the most bang for you buck, or if you flushed your money down the proverbial toilet.
So to start, we need to set up the stick that weíre going to measure against. This stick is going to include several things including the original reasons you wanted a SIEM in the first place, how much work itís generating, and in respect in that point, what are the results are that youíre getting out the other side.
Why did you need a SIEM in the first place? Did you get it for log aggregation and correlation? Did you get it for compliance reasons? Did you get it for real time alerting? So as step one, go back to your planning document and look at the original problem statement and what goals you hoped to achieve by getting a SIEM. To keep with the theme of our previous posts lets use as our example of PCI. You should have had the requirements put down that you needed to meet in order to be PCI compliant, so start going down that list and marking them off.
So are you meeting those requirements? Has your SIEM helped you reach your PCI compliance goals? Are their items missing, or is it doing all your had hoped and more?
The next thing weíre going to look at is the workload the SIEM is generating. If your SIEM isnít generating any work, then again go directly to fail. The whole idea of a SIEM is to show you things and get insights you couldnít see before. Since weíre doing this for compliance, unless all your reports are blank, there should be data being generated that require some type of action on your part. Donít try and fool yourself by saying you have a perfect network where nothing ever goes wrong - you know better than that right? New applications suddenly pop up running odd protocols, users get added and they start doing funny things, and devices come and go on your network. Thatís the nature of todays IT infrastructure - itís very fluid - and itís our job make sure itís done correctly. So whatís the work load? Take a look at your internal ticketing system and whatever project management software you use to get a better idea.
Lastly, what are the end results that youíre seeing? How do you quantify this? Well take the information from above, now compare your current load to the load before you had the SIEM Ė are the numbers of incidents going up or down compared to the number of hours being put into it? The idea here being that data is generated that you are able to now act on proactively to diffuse a problem before it becomes an incident. In plain terms, you want to get that unauthorized and unsecured application disabled from your credit card gateway before someone on the internet uses an attack to gain access to it.
And no, the end result isnít just equal to work-in-work-out, it also includes part of the goals we first measured. Is your SIEM helping you remain PCI compliant and maintain those goals? Iíve seen many customers initially meet their PCI requirements, but then fall out of compliance shortly afterwards because they donít stay on top of it. So do you have a successful process in place by which you can ensure the SIEM is showing you your compliance posture? As an example, are the correct reports being run that reflect your PCI requirements? Are they being reviewed every day? Are the correct devices logging to the SIEM to make your reports run? If your SIEM isnít showing you the issues and reflecting your PCI posture then again you have failed, which goes back to continuing to meet your originally stated goals.
So to sum this up, you need to measure yourself by three standards:
It seems simple enough, but if you actually do this exercise youíll see it actually gives you quite a bit of insight into what your SIEM is actually doing and if youíre getting the most bang for your buck. Itís ok if you find youíre not up to par; thatís the idea of this measurement. You are being proactive about your security by identifying issues and deficiencies so you can address them before anything bad happens. Thatís a good thing. Here at Secode we actually do this once a month with clients; we go over all the events that have happened, show how much work has gone into various incidents and show the customer their security posture so they can see if their goals are being met or not.
So how did you measure up?