Derek Arnold has spent the last 12 years securing large retail, medical device, and insurance companies. He has worked on large, diverse enterprises in the Fortune 500. His key specialties include security operations, threat intelligence, physical security and SIEM. As a principal consultant for Optiv, he helps organizations solve their unique security challenges using Splunk Enterprise.
Four Thoughts for SIEM Success
Security information and event management (SIEM) is a unique security tool in its ability to rapidly identify threats to an organization. Automatically sifting through terabytes of disparate data sources is an impressive feat of computer engineering. With that background, it is understandable that SIEM has a higher degree of system requirements, tuning and maintenance than average. After building, repairing and improving these systems this past year for our clients, I have noticed several emerging themes.
1. Exceed System Requirements
It pays to exceed the system requirements of the SIEM software vendor. Over the last few years, software vendors have been notorious for demanding excessive system resources. Simultaneously, with the rise of virtualization in the enterprise, systems administrators have become increasingly stingy with provisioning RAM, CPU, disk space and input/output operations per second (IOPS). While this may work for other enterprise apps, it can be detrimental to the success of a SIEM project. A resource-starved SIEM can lead to system errors, erratic behavior and seemingly endless troubleshooting. All these problems can translate into increased implementation cost, which could be avoided by exceeding system requirements. Don’t forget to build in a buffer for a double-digit percentage growth rate over the next year.
2. Develop Use Cases
It is important to develop use cases at the onset of the project and regularly after. This helps set the tone and priority of the project. Ask your security operations folks the top incidents to which they have been responding, and they likely will come up with where the visibility gaps are located. A simple use case could be, “I want to know when an employee logs in from more than two countries in a day.” Writing a clear use case like this increases the success rate of the project in two ways:
- Allows the prioritization of which log sources to onboard, parse and license.
- Gives the SIEM administrator a description of the visibility problem they are trying to solve.
3. Don’t Fear Agents
Modern SIEMs have some measure of remote log collection as an available option. However, this strategy can range from suboptimal to even unstable. System administrators can suffer from agent fatigue, especially at the endpoint. However, many SIEM agents are tested rigorously to provide for the most effective and reliable local collection methods. They often are tuned for limited use of system resources and bandwidth, and have some measure of fault tolerance built in. This is not always the case for remote collection.
However, if remote collection is the only option, find out the best practices from the SIEM vendor:
- How many remote devices can be collected from a node?
- What happens to the log in the event of a momentary interruption?
- What permissions, if any, does the collection agent require?
4. Focus on Ongoing Tuning
After the core deployment, plan on spending ample effort on tuning noise reduction and false positive removal. Too many organizations end up using the near-default SIEM settings, and then become complacent to the big red dashboards and endless list of incidents or offenses. Rather than becoming overwhelmed, plan on budgeting quarterly (or more often) tuning initiatives to get rid of faulty or irrelevant events. Open up frank communication between the incident handlers to find out which ones are false positives, or they will filter it out on their own. Get to the bottom of which events bear fruit and which ones need adjustments by working from large to small. Some techniques to consider are increasing the threshold, or chaining two or more log sources together to increase the fidelity.
SIEM can be a highly effective detective control to find and quickly react to suspicious activity. The complexity, level of implementation effort and system resources should not be overlooked. In order to maximize the investment, consider following the above four points.