If you’re looking at this, you’re probably hoping I am going to tell you some magic metric or secret formula you can use to measure your return on investment (ROI) on a SIEM deployment. I know, I know, people have been talking about and wishing for this for years, but just like flying cars, one has yet to appear that works for the masses. Sorry.

But what we can do is close; we can come up with a measure of success (or failure) based on why you needed a SIEM in the first place, and on how you are using it now. If you’re taken a look at my previous posts, hopefully what you’ve taken away from them is that you really need to plan for a SIEM and dedicate the resources to make it work. Based on that, we’re going to come up with a ruler by which you can see if you are getting the most bang for you buck, or if you flushed your money down the proverbial toilet.

So to start, we need to set up the stick that we’re going to measure against. This stick is going to include several things including the original reasons you wanted a SIEM in the first place, how much work it’s generating, and in respect in that point, what are the results are that you’re getting out the other side.

Why did you need a SIEM in the first place? Did you get it for log aggregation and correlation? Did you get it for compliance reasons? Did you get it for real time alerting? So as step one, go back to your planning document and look at the original problem statement and what goals you hoped to achieve by getting a SIEM. To keep with the theme of our previous posts lets use as our example of PCI. You should have had the requirements put down that you needed to meet in order to be PCI compliant, so start going down that list and marking them off.

So are you meeting those requirements? Has your SIEM helped you reach your PCI compliance goals? Are their items missing, or is it doing all your had hoped and more?

The next thing we’re going to look at is the workload the SIEM is generating. If your SIEM isn’t generating any work, then again go directly to fail. The whole idea of a SIEM is to show you things and get insights you couldn’t see before. Since we’re doing this for compliance, unless all your reports are blank, there should be data being generated that require some type of action on your part. Don’t try and fool yourself by saying you have a perfect network where nothing ever goes wrong - you know better than that right? New applications suddenly pop up running odd protocols, users get added and they start doing funny things, and devices come and go on your network. That’s the nature of todays IT infrastructure - it’s very fluid - and it’s our job make sure it’s done correctly. So what’s the work load? Take a look at your internal ticketing system and whatever project management software you use to get a better idea.

Lastly, what are the end results that you’re seeing? How do you quantify this? Well take the information from above, now compare your current load to the load before you had the SIEM – are the numbers of incidents going up or down compared to the number of hours being put into it? The idea here being that data is generated that you are able to now act on proactively to diffuse a problem before it becomes an incident. In plain terms, you want to get that unauthorized and unsecured application disabled from your credit card gateway before someone on the internet uses an attack to gain access to it.

And no, the end result isn’t just equal to work-in-work-out, it also includes part of the goals we first measured. Is your SIEM helping you remain PCI compliant and maintain those goals? I’ve seen many customers initially meet their PCI requirements, but then fall out of compliance shortly afterwards because they don’t stay on top of it. So do you have a successful process in place by which you can ensure the SIEM is showing you your compliance posture? As an example, are the correct reports being run that reflect your PCI requirements? Are they being reviewed every day? Are the correct devices logging to the SIEM to make your reports run? If your SIEM isn’t showing you the issues and reflecting your PCI posture then again you have failed, which goes back to continuing to meet your originally stated goals.

So to sum this up, you need to measure yourself by three standards:

Did we meet our initially stated goals and solve the problems we had originally identified which dictated we purchase a SIEM?
Is the workload being generated now higher then before you had your SIEM in place?
Are you getting a greater visibility into your network, resulting in serious incidents declining; and as a subset of that:

a. Are you continuously meeting your initially stated goals?

It seems simple enough, but if you actually do this exercise you’ll see it actually gives you quite a bit of insight into what your SIEM is actually doing and if you’re getting the most bang for your buck. It’s ok if you find you’re not up to par; that’s the idea of this measurement. You are being proactive about your security by identifying issues and deficiencies so you can address them before anything bad happens. That’s a good thing. Here at Secode we actually do this once a month with clients; we go over all the events that have happened, show how much work has gone into various incidents and show the customer their security posture so they can see if their goals are being met or not.

So how did you measure up?