Security information and event management

Initially, system logging was primarily used for troubleshooting and debugging. However, as operating systems and networks have grown more complex, so has the generation of system logs. The monitoring of system logs has also become increasingly common due to the rise of sophisticated cyberattacks and the need for compliance with regulatory frameworks, which mandate logging security controls within risk management frameworks (RMF). Starting in the late 1970s, working groups began establishing criteria for managing auditing and monitoring programs, laying the groundwork for modern cybersecurity practices, such as insider threat detection and incident response. A key publication during this period was NIST’s Special Publication 500-19. In 2005, the term "SIEM" (Security Information and Event Management) was introduced by Gartner analysts Mark Nicolett and Amrit Williams. SIEM systems provide a single interface for gathering security data from information systems and presenting it as actionable intelligence. The National Institute of Standards and Technology provides the following definition of SIEM: "Application that provides the ability to gather security data from information system components and present that data as actionable information via a single interface." This order was a response to an increase in ransomware attacks targeting critical infrastructure. By reinforcing information assurance controls within RMFs, the order aimed to drive compliance and secure funding for cybersecurity initiatives. == Information assurance ==

Published in September 2006, the NIST SP 800-92 Guide to Computer Security Log Management serves as a key document within the NIST Risk Management Framework to guide what should be auditable. As indicated by the absence of the term "SIEM", the document was released before the widespread adoption of SIEM technologies. Although the guide is not exhaustive due to rapid changes in technology since its publication, it remains relevant by anticipating industry growth. NIST is not the only source of guidance on regulatory mechanisms for auditing and monitoring, and many organizations are encouraged to adopt SIEM solutions rather than relying solely on host-based checks. Several regulations and standards reference NIST’s logging guidance, including the Federal Information Security Management Act (FISMA), Gramm-Leach-Bliley Act (GLBA), Health Insurance Portability and Accountability Act (HIPAA), Sarbanes-Oxley Act (SOX) of 2002, Payment Card Industry Data Security Standard (PCI DSS), and ISO 27001. Public and private organizations frequently reference NIST documents in their security policies. NIST SP 800-53 AU-2 Event Monitoring is a key security control that supports system auditing and ensures continuous monitoring for information assurance and cybersecurity operations. SIEM solutions are typically employed as central tools for these efforts. Federal systems categorized based on their impact on confidentiality, integrity, and availability (CIA) have five specific logging requirements (AU-2 a-e) that must be met. While logging every action is possible, it is generally not recommended due to the volume of logs and the need for actionable security data. AU-2 provides a foundation for organizations to build a logging strategy that aligns with other controls. NIST SP 800-53 SI-4 System Monitoring outlines the requirements for monitoring systems, including detecting unauthorized access and tracking anomalies, malware, and potential attacks. This security control specifies both the hardware and software requirements for detecting suspicious activities. Similarly, NIST SP 800-53 RA-10 Threat Hunting, added in Revision 5, emphasizes proactive network defense by identifying threats that evade traditional controls. SIEM solutions play a critical role in aggregating security information for threat hunting teams. Together, AU-2, SI-4, and RA-10 demonstrate how NIST controls integrate into a comprehensive security strategy. These controls, supported by SIEM solutions, help ensure continuous monitoring, risk assessments, and in-depth defense mechanisms across federal and private networks. == Terminology ==

The acronyms SEM, SIM and SIEM have sometimes been used interchangeably, but generally refer to the different primary focus of products: • Log management: Focus on simple collection and storage of log messages and audit trails. • Security event manager (SEM): Real-time monitoring, correlation of events, notifications and console views. • Security information and event management (SIEM): Combines SIM and SEM and provides real-time analysis of security alerts generated by network hardware and applications. Oftentimes commercial vendors provide different combinations of these functionalities which tend to improve SIEM overall. Log management alone doesn't provide real-time insights on network security, SEM on its own won't provide complete data for deep threat analysis. When SEM and log management are combined, more information is available for SIEM to monitor. A key focus is to monitor and help manage user and service privileges, directory services and other system-configuration changes; as well as providing log auditing and review and incident response. == Capabilities ==

• Data aggregation: Log management aggregates data from many sources, including networks, security, servers, databases, applications, providing the ability to consolidate monitored data to help avoid missing crucial events. • Correlation: Looks for common attributes and links events together into meaningful bundles. This technology provides the ability to perform a variety of correlation techniques to integrate different sources, in order to turn data into useful information. Correlation is typically a function of the Security Event Management portion of a full SIEM solution. • Alerting: The automated analysis of correlated events. • Dashboards: Tools can take event data and turn it into informational charts to assist in seeing patterns, or identifying activity that is not forming a standard pattern. • Compliance: Applications can be employed to automate the gathering of compliance data, producing reports that adapt to existing security, governance and auditing processes. • Retention: Employing long-term storage of historical data to facilitate correlation of data over time, and to provide the retention necessary for compliance requirements. The Long term log data retention is critical in forensic investigations as it is unlikely that the discovery of a network breach will be at the time of the breach occurring. • Forensic analysis: The ability to search across logs on different nodes and time periods based on specific criteria. This mitigates having to aggregate log information in your head or having to search through thousands and thousands of logs. == Components ==

SIEM architectures may vary by vendor; however, generally, essential components comprise the SIEM engine. The essential components of a SIEM are as follows: • A data collector forwards selected audit logs from a host (agent based or host based log streaming into index and aggregation point) • An ingest and indexing point aggregation point for parsing, correlation, and data normalization • A search node that is used for visualization, queries, reports, and alerts (analysis take place on a search node) A basic SIEM infrastructure is depicted in the image to the right. == Use cases ==

Computer security researcher Chris Kubecka identified the following SIEM use cases, presented at the hacking conference 28C3 (Chaos Communication Congress). • SIEM visibility and anomaly detection could help detect zero-days or polymorphic code. Primarily due to low rates of anti-virus detection against this type of rapidly changing malware. • Parsing, log normalization and categorization can occur automatically, regardless of the type of computer or network device, as long as it can send a log. • Visualization with a SIEM using security events and log failures can aid in pattern detection. • Protocol anomalies that can indicate a misconfiguration or a security issue can be identified with a SIEM using pattern detection, alerting, baseline and dashboards. • SIEMS can detect covert, malicious communications and encrypted channels. • Cyberwarfare can be detected by SIEMs with accuracy, discovering both attackers and victims. Modern SIEM platforms support not only detection, but response too. The response can be manual or automated including AI based response. == Regulatory compliance ==

SIEM systems play a significant role in helping organizations meet regulatory compliance requirements by centralizing log collection, enabling real-time monitoring, and generating audit-ready reports. Under the HIPAA Security Rule, covered entities must implement audit controls that record and examine activity in information systems containing electronic protected health information (ePHI). A proposed update to the HIPAA Security Rule (NPRM, December 2024) would require covered entities to deploy technology capable of detecting and responding to security incidents within 24 hours, a capability commonly provided by SIEM platforms. The Payment Card Industry Data Security Standard (PCI DSS) Requirement 10 mandates that organizations track and monitor all access to network resources and cardholder data, with SIEM systems commonly used to aggregate and analyze the required log data across distributed environments. Similarly, the Sarbanes-Oxley Act (SOX) requires organizations to maintain audit trails for financial system access, and SIEM platforms provide the centralized logging and retention capabilities necessary to demonstrate compliance during audits. == Correlation rules examples ==

SIEM systems can have hundreds and thousands of correlation rules. Some of these are simple, and some are more complex. Once a correlation rule is triggered the system can take appropriate steps to mitigate a cyber attack. Usually, this includes sending a notification to a user and then possibly limiting or even shutting down the system. Brute Force Detection Brute force detection is relatively straightforward. Brute forcing relates to continually trying to guess a variable. It most commonly refers to someone trying to constantly guess your password - either manually or with a tool. However, it can refer to trying to guess URLs or important file locations on your system. An automated brute force is easy to detect as someone trying to enter their password 60 times in a minute is impossible. Impossible Travel When a user logs in to a system, generally speaking, it creates a timestamp of the event. Alongside the time, the system may often record other useful information such as the device used, physical location, IP address, incorrect login attempts, etc. The more data is collected the more use can be gathered from it. For impossible travel, the system looks at the current and last login date/time and the difference between the recorded distances. If it deems it's not possible for this to happen, for example traveling hundreds of miles within a minute, then it will set off a warning. Many employees and users are now using VPN services which may obscure physical location. This should be taken into consideration when setting up such a rule. Excessive File Copying The average user does not typically copy or move files on the system repeatedly. Thus, any excessive file copying on a system could be attributed to an attacker wanting to cause harm to an organization. Unfortunately, it's not as simple as stating someone has gained access to your network illegally and wants to steal confidential information. It could also be an employee looking to sell company information, or they could just want to take home some files for the weekend. Network Anomaly Detection Monitoring network traffic against unusual patterns that includes any threats or attacks ranging from DDOS to network scans. Note SIEM can monitor data flow in the network and to detect and prevent potential data exfiltration efforts. In general dedicated Data loss prevention (DLP) take care about data loss prevention. DDoS Attack A DDoS (Distributed Denial of Service) Attack could cause significant damage to a company or organization. A DDoS attack can not only take a website offline, it can also make a system weaker. With suitable correlation rules in place, a SIEM should trigger an alert at the start of the attack so that the company can take the necessary precautionary measures to protect vital systems. File Integrity Change File Integrity and Change Monitoring (FIM) is the process of monitoring the files on your system. Unexpected changes in your system files will trigger an alert as it's a likely indication of a cyber attack. == Alerting examples ==

Some examples of customized rules to alert on event conditions involve user authentication rules, attacks detected and infections detected. == See also ==