How We Validate Every Detection Before It Reaches Production
Inside CyberSift’s TDUC lifecycle for turning emerging threats into high-confidence detections.
A new attacker technique can become yesterday’s problem surprisingly quickly. Identity abuse, supply-chain compromise, AI-assisted attacks and techniques such as ClickFix are evolving faster than traditional detection-development cycles. MITRE’s 2026 ATT&CK updates reflect the same reality, with more frequent updates to capture emerging groups, software and attack activity.
For a security operations team, knowing that a technique exists is not enough. The harder question is whether you can detect it reliably in real environments without creating another stream of alerts that analysts learn to ignore.
At CyberSift, this is where our TDUC lifecycle comes in. Every new detection goes through a structured process before it becomes part of our production detection capability.
Every Detection Starts With a Hypothesis
Not every detection begins with a threat report.
Some start with a newly observed technique or campaign. Others come from threat intelligence, incident investigations, threat-hunting activity, or patterns we observe across monitored environments. And sometimes the starting point is simply a hypothesis:
What would an attacker doing this actually look like in the telemetry we have?
That distinction matters. Detection engineering should not be a race to translate every new indicator or headline into a rule. The objective is to identify observable attacker behaviour that can produce a useful security signal.
This is increasingly important as attackers use legitimate credentials, trusted applications and normal administrative functionality. A suspicious action may look completely benign when viewed in isolation. Context becomes the difference between an interesting event and a meaningful detection.
MITRE’s recent move toward Detection Strategies and Analytics reflects this broader shift toward understanding how behaviours can be detected rather than treating individual alerts as isolated events.
A detection is not valuable because it fires. It is valuable because it identifies meaningful behaviour.
We Build the Detection, Then Try to Break It
Once we have a hypothesis, we turn it into detection logic.
Our initial development happens in Sentio SIEM using Lucene queries. This gives the detection engineer a fast way to interrogate the available telemetry, test assumptions and understand what the underlying data actually looks like.
The first query is rarely the final query.
We test it. We inspect the results. We identify legitimate activity that looks suspicious. We adjust the logic. We add context where necessary. We remove conditions that make the rule unnecessarily fragile.
This is where detection engineering becomes much more than writing syntax.
A rule can be technically correct and still be operationally useless. It might depend on a field that is not consistently populated, trigger on common administrative activity, or be so restrictive that it only detects one very specific version of an attack.
The objective is to find the right balance between coverage, precision and resilience.
This is also where our threat-hunting capability feeds directly into detection development. A hunt can expose behaviour that was not previously covered, while an existing detection can provide the starting point for a deeper investigation.
We do not ask whether a query works. We ask whether it works well enough to trust.
Production Testing Means Testing Everywhere
A detection that looks excellent in one environment can behave very differently somewhere else.
Different customers have different operating systems, identity providers, applications, network architectures, administrative practices and levels of telemetry. A query that produces a high-confidence signal in one environment may generate hundreds of legitimate results in another.
That is why an important stage of the TDUC lifecycle is cross-environment validation.
Before deployment, we test the detection across the customer environments where the relevant telemetry exists. We look at what the rule actually finds, not what we expect it to find.
We evaluate factors such as:
True-positive versus false-positive behaviour
Alert volume and frequency
Coverage across different environments
Quality and consistency of the underlying telemetry
Whether legitimate administrative activity creates noise
Whether the detection identifies useful attacker behaviour or merely suspicious-looking events
Whether additional context could improve confidence
This process also exposes assumptions that are invisible during development.
A detection may perform well against a theoretical attack but fail when confronted with the messy reality of production data. Conversely, a seemingly simple rule may prove highly effective because the behaviour it targets is genuinely unusual.
The result is a more practical measure of detection quality: how useful is this alert to an analyst who has to investigate it?
Only Trusted Detections Become Production Controls
Once a detection has survived development and cross-environment testing, we convert the validated logic into the format required for production deployment.
The implementation details are intentionally secondary to the principle: detections are reviewed and deployed only to environments with the telemetry required to support them.
This creates an important separation between experimentation and production.
Our engineers can investigate new ideas, test hypotheses and iterate rapidly without turning every experiment into an alert for a customer. Production detections, on the other hand, have earned their place through evidence.
And the lifecycle does not end when a rule is deployed.
Detection quality has to be continuously reassessed. New applications appear. Infrastructure changes. Attackers modify their behaviour. Legitimate business processes evolve. A detection that was precise six months ago can become noisy as an environment changes.
That is why detection engineering sits alongside threat hunting and incident response rather than operating as a one-time configuration exercise. Incident investigations can reveal missed behaviours. Threat hunts can identify new opportunities for detection. Production alerts can show where existing logic needs refinement.
This creates a continuous feedback loop:
Threat intelligence creates hypotheses. Hunting finds behaviour. Detection engineering operationalises it. Incident response teaches us what to improve next.
Sentio provides the detection and investigation layer that makes this process operational, while CyberSift’s security analysts provide the expertise required to continually refine what the platform is looking for.
Detection Engineering Is a Discipline
The goal of a modern SOC should not be to accumulate thousands of detection rules.
It should be to maintain a smaller, more trusted set of detections that reliably identify meaningful attacker behaviour.
That requires a lifecycle.
At CyberSift, the TDUC process gives us a practical framework for moving from “this looks like something we should detect” to “we have evidence that this detection is useful in production.”
It also explains why detection engineering cannot be separated from the rest of security operations. The strongest detections are informed by current threats, tested against real telemetry, challenged by legitimate behaviour and improved through operational experience.
For organisations looking to strengthen this capability, the first step is often understanding where the current detection gaps and sources of alert noise actually are. CyberSift combines SIEM visibility with analyst-led threat hunting and incident response to help turn those gaps into actionable detection opportunities.
Good detection is not about generating more alerts. It is about generating alerts worth trusting.
If your SOC is collecting the right telemetry but still struggles with noise, coverage or confidence in its detections, that is a detection-engineering problem - and one worth measuring.
-Written by Stanislav Stoychev
