Repository navigation
P7b-1: match customer threat-intel lists and rank matched findings - #37
Merged
Merged
Conversation
Predictive security roadmap P7b, first slice (IOC matching) under the agreed defaults: AWS-native + customer-supplied lists, DynamoDB store, ranking factor, curated mapping and graph-only emulation deferred to P7b-2. - ThreatIndicators store (risk-engine): one row per indicator with a TTL — freshness is the TTL, so a dropped indicator expires by itself — batch writes and batched, bounded reads; list parsing (one indicator per line, '#' comments, IP v4/v6 and 12-digit accounts, dedupe, cap with counted invalid lines); and pathIndicators(), the fact join the runner uses (anomaly source_ip, trusted ExternalAccount account_id). - threat-intel-import (runtime command; CronJob suspended by default, rule-runner service account): imports plain S3 lists into the store. No internet: the lists are account-owned objects, and AWS's managed reputation lists need no import because GuardDuty applies them (their findings already feed RULE-018/RULE-029). - Ranking: a match adds a flat +10 threatIntelContribution to the risk score (additive like exploitability — a fact about the path, not a proportional strength), the issue records threatIntelMatches and the issue page shows a "Threat intelligence" card; refresh clears matches whose indicator is gone; lookup failures score without it and log. No auto-ticketing until /rules/quality shows measured precision. - Infra: Flash (Terraform) creates the ThreatIndicators table (TTL, deletion protection, CMK), grants the rule-runner role full access to it and s3:GetObject on threat_intel_list_bucket when set; the URIs, table name, and TTL flow through helm_values.config; chart gains threatIntelImport.*, config envs, schema entries, and the threat-intel-import egress component. Why this is safe: read-only for the engine (the importer only writes its own indicators table); no new outbound network path; matches are transparent (metadata + card) and cannot ticket; invalid/absent data degrades to "no match".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Predictive security roadmap P7b-1 (IOC matching) under the agreed defaults: AWS-native + customer-supplied lists, DynamoDB store, ranking factor, deferring TTP mapping + emulation (P7b-2). 35 files, +1299/−48.
Store & parsing (
packages/risk-engine/src/threat-intel.ts)ThreatIndicators— one row per indicator with a TTL (freshness is the TTL; a dropped indicator expires by itself), batch writes, bounded batch reads#comments, IPs v4/v6 + 12-digit accounts, dedupe, cap with counted invalid linespathIndicators()— the fact join the runner uses: anomalysource_ip, trustedExternalAccountaccount idImporter —
threat-intel-importruntime command, CronJob suspended by default on the rule-runner service account; imports plain S3 lists. No internet: lists are account-owned objects; AWS's managed reputation lists need no import (GuardDuty applies them — RULE-018/029 already consume the findings).Ranking — a match adds a flat +10
threatIntelContribution(additive like exploitability: a fact, not a proportional strength); issues recordthreatIntelMatches, the issue page shows a "Threat intelligence" card, refresh clears matches whose indicator is gone, lookup failures score without it and log. No auto-ticketing until/rules/qualityshows measured precision (P6 policy);risk:planis unaffected (no store to read, never claims a match).Infra — TF creates the table (TTL, deletion protection, CMK), grants the rule-runner role access +
s3:GetObjectonthreat_intel_list_bucketwhen set; URIs/table/TTL flow throughhelm_values.config; chart gainsthreatIntelImport.*, config envs, schema entries, and thethreat-intel-importegress component.Why this is safe
Test plan
npx jest --runInBand— 19 projects / 1775 tests passed (new: parsing/validation/join, store TTL/retry/dedupe, importer summary + failure, runner match + tolerant failure, scoring bonus, runtime validation/dispatch, UI card)npm run typecheck; full workspace lint;npm run format:checknpm run ci:build;npm run package:lambdas && npm run smoke:lambdasterraform fmt -check; testdata gainsthreatIntelImport.image; Deployment modules CI validates schema + render