As a security analyst in a growing company, it is often easy to get into the “set it and forget it” mentality. You create one alert after another. Then another. And another. With each alert comes a certain amount of work for an analyst. Analyst time costs money, and some alerts consume more time than others. If most of the alerts result in false positives, a large amount of resources are being spent unnecessarily. This underscores the importance of continuous tuning of alerts instead of iterating through useless noise.
But how can we know what to prioritize in order to free up our analysts’ valuable time? You should have a mechanism and strategy for collecting metrics describing the outcome of your security events. Whether you track alert triage activity in a ticketing system or not, you can start measuring today which events were true positive and actionable, or false positive and a waste of time.
True positive, or positively a waste of time?
What is a true positive anyway? Ideally, you are setting alerts tied to actionable use cases driven by a business objective. For example, your organization likely wants to prevent unauthorized access to sensitive internal resources. You can set alert and enrichment logic in your monitoring instrumentation to notify you when such activity occurs. You might even have a triage playbook that is enacted if the alert fires. An analyst picks up the ticket, looks at the data, and validates that the activity did in fact occur and then proceeds to triage the event according to the playbook.
A false positive is when the analyst determines that the use case the playbook was written for didn’t actually occur. As previously mentioned, you want to tune these as they occur instead of continuing to waste time on them.
This seems like a long-winded way to distinguish between true and false positives, but it is necessary to illustrate how valuable time can be spent.
Where is the valuable time being spent?
How many of your SOC analyst tickets are true positives? 100%? 50%? 10%? Or even less? I’d bet money that 10% or less of your company’s security incident tickets are true positive. That leaves around 90% or so false positives. If you don’t have a false positives rate on hand, I suggest adding a field within your ticketing software to include a drop down for false positives and true positives. Depending on how far you go back in the ticket, it could take some time to start this process. As a starting place, I’d go back at least one month from the current date. You can see in the diagram below that the first iteration of your new metric can be simple and high level.