How Security Teams Use SIEM Software for Threat Hunting

This makes security monitoring, by its nature mostly reactively based. At this point, a rule fires, an alert is generated and an analyst takes a look. What makes this model valuable is that it captures a non-negligible amount of attacker activity but it suffers from one key structural limitation: it only reveals what the detection rules will already catch. An attacker who understands the modes of monitoring systems can exploit the gaps between those rules, operating below detection thresholds long enough to complete their mission before any alert is raised.

This is exactly the limitation that threat hunting addresses. It is a forward-looking discipline where analysts interactively imagine what sorts of attacker behavior have not yet generated an alert, are currently invading the organization, or will simulate its functional patterns without ever triggering an alert under today's detection logic. Such work is operationally feasible on an enterprise scale due to SIEM software, which provides the access to historical data as well as query and analytical tools upon which threat hunters rely.

Security teams building or maturing a threat-hunting capability can evaluate SIEM software for security operations to see how new platforms and systems spans detection engineering as well as analyst-driven investigation workflows in a single environment.

Reactive Detection Is Not Enough

Detecting based on alerts assumes that the threat activity an organization is trying to detect will be equivalent to the alert activity for which detection rules were created. That assumption is becoming less and less tenable. Some advanced threat actors spend much of their effort researching the monitoring profiles of their targets, tuning their behavior to never trip a threshold, and leveraging legitimate tools and user credentials that are not suspicious on path in most automated detection methods.

That leads to a detection gap that just cannot be bridged by introducing more rules. Rules capture recognizable patterns they are efficient at managing threats they have been explicitly built to discover and inefficient against anything beyond that. Threat hunting is the way that an organization expands their detection capability into this unexplored frontier, leveraging analyst expertise and contextual reasoning to find the signs of malicious activity before those systems have been trained to detect them.

The Dwell Time Problem is the center of your universe

A further and more practical consequence of only using alert-driven detection is dwell time: the difference in time from when an attacker gets inside the environment until they are detected. This time range lasts from weeks to months in several documented breaches. In that duration, an attacker may be establishing persistence, leveraging reconnaissance, exfiltrating data or positioning to execute some sort of action destructive act while the security operations center looks at which alerts are firing and not activity that is not.

Threat hunting is specifically built for shorter dwell time providing analysts a repeatable methodology to search for attacker behavior that has yet to manifest in automated detections. The sooner a hunt finds a compromise, the less opportunity an attacker has to wreak havoc.

How does a SIEM help Threat Hunts?

Centralized Data Access

The results of a threat hunt depend only on the data that was available to the analyst conducting it. SIEM platforms provide the aggregation of logs across endpoints, network devices, identity systems, and cloud environments and applications into a single store query that enables comprehensive hunting. Having that centralization means hunters will not need to query each data source separately, too slow and fragmented for any investigation.

It matters, also the breadth and also the depth. An analyst who is looking for signs of lateral movement will require visibility into all authentication events (at least for Netlogon attacks), together with visibility into network connections and process execution data, as they can work together to provide a more robust body of evidence of an active attack. By ingesting all three, a SIEM allows a hunter to pursue activity across systems without tool-switching or losing the thread of an investigation.

Historical Log Retention

Threat hunting often looks backward. So a lot of hunt hypotheses kick off with questions like whether something was happening in the environment beforehand that is now known to be suspicious. To answer that question requires log data days, weeks or months old. SIEM platforms were designed to hold massive amounts of log data for very long retention times, providing hunters with the historical ownership that retrospective investigation demands.

This retro active ability will become important as new threat intelligence is discovered. At the instant that an analyst discovers a given technique or indicator has been linked to an active threat group, he/she can query historical data to assess if that activity already transpired in the environment. This transforms newly available intelligence into a retrospective hunt.

Query and Search Capabilities

While the type of SIEM affects the investigation workflow in a threat hunt: Hunters must pivot quickly from one datum to the next from a potential authentication event, to everything else involving the same account, to all network traffic associated with that endpoint, to monitoring all processes that executed at about the same time. This also means you can guide the logic behind it, and there are multiple platforms that allow for fast, flexible queries across large data sets enabling hunters to stay with that chain of reasoning at high speed. The effort required to create a slow or rigid query introduces friction, which essentially restricts how far down the rabbit hole a hunt can follow in a session.

Detection Engineering Feedback Loop

Threat hunting and detection engineering work in a productive cycle. So, if during a hunt we find that an attacker behavior was not covered by detection rules already in place then the discovery of such finding is used to create a new key rule. This is where those two activities take place in the SIEM; hunts point out gaps, new rules are written to address those and the platform’s detection capability increases as a result. As a result, this loop enhances the limit of what automated detection can capture, which in effect improves the quality of the threat hunting activity by minimizing how much time hunters will use investigating alerts already handled by rules on their own.

How Analysts Structure Threat Hunts

Hypothesis-Driven Hunting

The most formal method of threat hunting is based on a hypothesis: this is a precise, verifiable claim introducing some attacker behavior that might be in the environment. Hypotheses are based on threat intelligence, common attacker techniques like those in the MITRE ATT&CK framework, or patterns seen during past hunts or incidents. An analyst could then think that an attacker is using a compromised credential and that credential is being used to access systems outside the normal usage pattern, so they would query log data against that hypothesis.

Anomaly-Based Hunting

Not every hunt starts with a hypothesis. Anomaly-based hunting consists of querying for statistical outliers in the data accounts that are accessing an abnormal number of systems, processes communicating over ports they do not typically communicate on, authentication events occurring at times or from locations that appear to be anomalous and then investigating those outliers to determine whether they are benign anomalies or have been caused by bargain-hunting attackers. This kind of anomaly detection is much more efficient in SIEM platforms that support behavioral baselines and anomaly scoring–instead of showing analysts all outliers, it surfaces the most statistically significant deviations (or specific threats) back to them.

Intelligence-Led Hunting

Intelligence-led hunting starts with external indicators of compromise (IOCs), known attacker infrastructure, and documented techniques that form the basis for queries against SISEM data. If a threat intelligence feed identifies any command-and-control IP address or file hash tied to an existing threat group, hunters could query their historical log data for any signs of systems that have communicated with that infrastructure or executed that file. This methodology enables organizations to integrate threat intelligence directly into their security operations process.

The intersection of proactive investigation techniques and the shift from reactive to anticipatory defense is explored in depth in practitioner-level analysis of active defense security approach methodologies and how leading security teams are implementing them.

Catapult a Threat Hunting Program on SIEM Backdrops

Great threat hunting programs are more than just good tooling. They need analysts with the domain expertise to create impactful hypotheses and investigative prowess to follow the data wherever it leads you. That work is supported by SIEM platforms, which guarantee the availability, accessibility and query-ability of data but the quality of the hunt rests squarely on the analysts understanding both their environment and how attackers behave.

For organizations that are building out their threat hunting capability, it is critical to ensure that data from all relevant sources are being ingested into your SIEM prior to any hunting taking place. An hunt which couldn't see some systems or logs types will give a false sense of security as clean exit. Infrastructure-wide analysis of log coverage gaps is one of the most common reasons why threat hunts fail to surface active compromises that are subsequently discovered through alternative means, such as incident response or forensic investigations.

Guidance on how government agencies and security organizations approach structured threat hunting missions, including the operational model for proactive network defense, is available through the government threat hunting program resources published by the Cybersecurity and Infrastructure Security Agency.

Frequently Asked Questions

What Is Threat Hunting and Why It Is Free of Alert-Based Detection?

Threat hunting is the analyst-driven process of proactively searching for attacker activity that has not yet triggered automated alerts. If alert-based detection is reactive it detects the threats that fit a pre-defined rule. Automated systems help us by applying rules to the behavior, but those rules will only cover so much and people need to take up what they miss: that is where threat hunting comes in, looking for behavior outside of the rule sets using hypothesis testing, anomaly analysis and threat intelligence to surface activity not yet covered by automated safeguards.

SIEM software is at the heart of threat hunting because it helps organizations find and get from one tool to a final discovery very quickly.

SIEM gives threat hunting its centralized data repository, historical log retention and flexible query capabilities. Without a SIEM bringing all the data together in to one single queryable store, hunters would have had to access each data source on its own which in turn makes it unlikely that there will be sufficient time between activity observed and an investigation given how quickly they cross systems.

How does threat hunting enhance an overall detection capability?

The discoveries made during threat hunts have a direct impact on detection engineering. If an attacker conducts behavior that existing rules did not detect during a hunt, analysts can automatically compose new detection logic to capture the same pattern moving forward. This feedback loop subsequently reduces the hole in detection automated coverage, increasing the general quality of safety operations program with reference to time.