feat(sigma): ship the fifteen sequence detections as Sigma rules - #98
Merged
Conversation
The Sigma pack covered the recorder's own health: heartbeat silence, degraded coverage, MCP schema drift, denial bursts, untriaged criticals. All useful and none of it the product. The fifteen sequence detections, which are the thing a SOC would route on, had no Sigma representation at all, so a Splunk or Sentinel team wanting to alert on credential-exfil had to write the search themselves from a schema document. That is a strange gap for a project whose pitch is that your SIEM stays the console. If the console is theirs, the content has to be portable. Generated rather than written. Hand-writing fifteen would put rule ids, severities and MITRE ids in a second place that drifts from the first, which is the failure this repository keeps finding in its own documents. The generator replays the benchmark corpus through the real engine and reads the Detection objects it emits, so a severity that changes in rules.py changes here on the next run. Sixteen files for fifteen rule ids, and the extra one is a real finding. encoded-command-download emits two severities on purpose: critical for remote code fetched and executed, low for local content piped into an interpreter, with a comment in rules.py saying the low one exists "so it stops drowning the criticals". Keying the pack on rule_id alone produced a single rule at whichever severity the corpus happened to yield first, so it claimed critical while some firings are low. A test caught it. Each severity now gets its own rule with the selection pinned on action.outcome, because one static level would either page on the quiet variant or stay silent on the loud one. Rule UUIDs are derived from the rule id rather than generated, so rerunning is idempotent and a consumer who pinned an id keeps working. Severity joins the seed only for a split rule, which keeps the fourteen unsplit ids where they are. Five tests: every built-in rule has a Sigma rule, no Sigma rule outlives a deleted one, severities match what the engine emits, the documents are valid Sigma, and the ids do not churn on regeneration. Two of the fifteen have no corpus case and carry an explicit entry in the generator with the reason. host-subagent-swarm-burst needs several sessions on one host; off-hours-activity is opt-in behind an env var and a window. The table is checked against BUILTIN_RULE_IDS rather than trusted, so a sixteenth rule with neither a corpus case nor an entry fails the script instead of silently shipping a pack that claims more than it has. Tracked in #36. CI now lints tools/ as well, since the generator lives there and emits rules a SOC routes on. That surfaced one pre-existing S101 in mcp_audit_proxy.py, a type-narrowing assert after create_subprocess_exec, annotated rather than rewritten. 1125 tests, ruff clean, ruleset fingerprint unchanged at 56ad3de1 so the dogfood clock is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Sigma pack covered the recorder's own health — heartbeat silence, degraded coverage, MCP schema drift, denial bursts, untriaged criticals. All useful, none of it the product.
The fifteen sequence detections, which are what a SOC actually routes on, had no Sigma representation at all. A Splunk or Sentinel team wanting to alert on
credential-exfilhad to write the search themselves from a schema document.That is a strange gap for a project whose pitch is "your SIEM stays the console." If the console is theirs, the content has to be portable.
Generated, not written
Hand-writing fifteen would put rule ids, severities and MITRE ids in a second place that drifts from the first — the failure this repo keeps finding in its own documents.
tools/generate_sigma_pack.pyreplays the benchmark corpus through the real engine and reads theDetectionobjects it emits, so a severity that changes inrules.pychanges here on the next run.Sixteen files for fifteen rules, and the extra one is a real finding
encoded-command-downloademits two severities on purpose:criticalfor remote code fetched and executed,lowfor local content piped into an interpreter, with a comment inrules.pysaying the low one exists "so it stops drowning the criticals".Keying the pack on
rule_idalone produced a single rule at whichever severity the corpus yielded first — it claimed critical while some firings are low. A test caught it:Each severity now gets its own rule with the selection pinned on
action.outcome. One static level would either page on the quiet variant or stay silent on the loud one.Five tests
Every built-in rule has a Sigma rule; no Sigma rule outlives a deleted one; severities match what the engine emits; the documents are valid Sigma; and the UUIDs do not churn on regeneration (they are derived from the rule id, so a consumer who pinned one keeps working).
The two with no corpus case
host-subagent-swarm-burstneeds several sessions on one host.off-hours-activityis opt-in behind an env var and a window. Both carry an explicit entry in the generator with the reason, and that table is checked againstBUILTIN_RULE_IDSrather than trusted — a sixteenth rule with neither a corpus case nor an entry fails the script instead of silently shipping a pack that claims more than it has. Tracked in #36.Also
CI now lints
tools/, since the generator lives there and emits rules a SOC routes on. That surfaced one pre-existingS101inmcp_audit_proxy.py— a type-narrowing assert aftercreate_subprocess_exec— annotated rather than rewritten.56ad3de1, so the dogfood clock is untouched (the generator readsrules.py, never writes it)🤖 Generated with Claude Code