security(feeds): reconcile DNSBL snapshot ownership on refresh

[seonghobae]

Verified defect

Fresh review of PR #167 found a shared threat-feed lifecycle defect outside the MISP parser itself. apply_threat_feed_import already treated ThreatIndicator material as snapshot-owned and reaped indicators withdrawn by a feed, while DNSBL material was only upserted. A feed could therefore withdraw a DNSBL address while Wardnet kept the stale block active indefinitely.

This is a shared Wardnet feed-admission/control-plane ownership problem, not a MISP-specific parsing rule. The repair belongs in the common feed reconciliation path so MISP and future feed adapters do not grow competing lifecycle implementations.

Hostile RED lineage

PR #167 remains the owning repair lane. Initial RED a639e626764e2caf593266dbf94d2b030626bbaf added withdrawal, overlapping-feed ownership, operator-ownership, and skipped-write-count cases. ba5dd624e50eec63efc453a54beb57cbe28295a9 corrected the withdrawal fixture so each refresh keeps unrelated valid material and reaches reconciliation instead of violating the established non-empty-snapshot contract. 46615df5caca8e6ef1738aba1995f681524b585d repaired the async harness. 6248be07657ada547a72b23b0ea50af3a88be20f added restart, predecessor-state serde, persistence-failure rollback, and retry coverage.

An intermediate schema-only attempt 6b3b0b07637d87fd65a4d2b0d2f8bf0bb835908f used raw IpAddr ownership fields and was intentionally reverted at 28d0ac12d37b4c97ea58b2d55831a6c1e7b9cf98; it is not the accepted production design. Exact predecessor CI 33983682746 compiled and reached the intended integration boundary, failing the stale-withdrawal and operator-owned-overlap write/count cases while the shared-other-feed and operator-withdrawal preservation cases passed.

Causal GREEN implemented

Causal source repair 7042aa19267886e3af9c378dddd879929837877b implements the accepted shared-boundary model:

• DnsblEntryKey uses the existing DnsblEntry.address / upsert_dnsbl stable identity;
• persisted feed dnsbl_keys and independent operator DNSBL ownership are serde-defaulted for predecessor-state compatibility;
• /api/dnsbl establishes operator ownership for the stable key;
• feed refresh reconciles threat and DNSBL ownership in the shared transaction;
• feed writes do not overwrite operator-owned DNSBL payloads, and upserted_dnsbl counts actual writes;
• a dropped feed key is removed only when neither another feed nor the operator still owns it;
• persistence failure rolls back DNSBL payload and ownership metadata together before retry.

Run 34000662730 executed the causal working tree before source commit/push. The locked workspace test suite passed, including all four DNSBL ownership hostile cases and all three restart/predecessor-state/persistence cases, and strict workspace Clippy passed. The temporary repair workflows were removed before the source commit. The previously valid CodeRabbit DNSBL-ownership review thread has since been verified against the committed implementation and resolved.

Exact-current integration state — 2026-09-06 KST

Protected/default main is a52ccd0a24a727d9349bb32def7713882d8cad1e. PR #167 non-destructively adopted that protected head and is currently Draft/mergeable at exact 0c83cd5956f512d79c6600e823fcfa6d6f32af4e; its base SHA is exactly the protected head.

The source defect is causally GREEN, but protected integration is not complete. Exact-current CI run 34000987921 remains queued; sole rust job 101399700528 has steps=[], labels=[ubuntu-24.04], runner_id=0, and no runner/group identity. Fuzz 34000988028 and the current security/SAST/CodeQL/required-workflow lanes are likewise non-passing. The exact runner/materialization specimen and acceptance criteria have been handed to canonical control-plane owner .github#712; Wardnet will not manufacture a no-op commit, churn selectors, or promote predecessor GREEN merely to obtain execution.

Completion gate

Keep this issue open until #167 or a verified successor reaches protected main with the complete valid RED/GREEN/test/persistence/evidence delta. Before merge, one unchanged exact head must obtain then-live repository/security/coverage/package/SBOM/provenance/review/thread/governance evidence plus fresh candidate-base compatibility. The live solo-maintainer approval/bypass ambiguity remains canonical .github#772 owner work and does not authorize self/model approval or routine administrator bypass.

Do not relax the non-empty feed-import contract, revive the reverted raw-address schema, move reconciliation into an adapter, infer ownership from source/TTL/audit text, introduce cross-service SQL, force-push/destructively rebase, weaken gates, reuse predecessor evidence, or close this issue merely because source GREEN exists.

[seonghobae]

Fresh exact-source reconciliation on #167@b6581d45d1b4eacb0ceeb37e2114303d94a0c5b0 narrows the GREEN implementation further.

• Stable DNSBL identity must be the address, because waf_ids_core::upsert_dnsbl currently replaces on item.address == entry.address; changing ownership to (address, source) or prefix semantics would diverge from the write key and is out of scope.
• AppData already has backward-compatible #[serde(default)] operator_threat_keys and threat_feed_ownership; the DNSBL analogue should follow the same persisted-state shape rather than introduce adapter-local memory.
• create_dnsbl currently calls upsert_dnsbl directly and records the audit entry. This is the operator-authority boundary that must mark the DNSBL key before future feed reconciliation.
• apply_threat_feed_import currently snapshots operator_threat_keys, calls replace_threat_feed_ownership only for threat keys, reaps only withdrawn threats, then blindly upserts every DNSBL row and reports feed.dnsbl.len(). The causal GREEN belongs here: replace per-feed DNSBL ownership, reap a dropped address only when no other feed or operator owns it, skip feed writes to operator-owned addresses, and count only actual feed writes.
• Persistence rollback is already provided by mutate_and_persist: it clones the previous AppData and restores it if snapshot persistence fails. The repair should keep all ownership/reaping/write changes inside that single mutation so a failed write cannot expose a partially reconciled state.

Current exact-head CI/Fuzz/Security/SAST/CodeQL runs 33972359920/33972359957/33972359918/33972359956/33972359997 are all queued and are non-passing evidence; the production DNSBL GREEN is still absent, so no rerun or thread resolution is appropriate yet.

[seonghobae]

RED coverage advanced on #167. 6248be07657ada547a72b23b0ea50af3a88be20f adds file-backed restart-before-withdrawal, predecessor-state serde, persistence-failure rollback, and retry cases on top of the existing withdrawal/shared/operator/skipped-write cases. A schema-only partial implementation (6b3b0b07637d87fd65a4d2b0d2f8bf0bb835908f) was deliberately reverted by 28d0ac12d37b4c97ea58b2d55831a6c1e7b9cf98 rather than leave the branch compile-incomplete before the common reconciler and operator-write path are wired atomically. Current #167 head is therefore a deliberate compile-clean RED; GREEN acceptance in this issue is unchanged.

[seonghobae]

Causal DNSBL ownership repair is now stable on parent PR #167 and has been non-force adopted onto current protected base. Source commit 7042aa19267886e3af9c378dddd879929837877b implements DnsblEntryKey(address), serde-default feed/operator ownership, atomic shared snapshot reconciliation, operator payload preservation, conditional stale reaping, actual-write accounting, predecessor-state compatibility and persistence rollback/retry. Rescue run 34000662730 executed that repaired working tree with full locked workspace tests + strict Clippy GREEN before committing only the two intended Rust files and self-removing the temporary repair workflows. The valid review thread is resolved.

Protected main@a52ccd0a24a727d9349bb32def7713882d8cad1e was then adopted by ordinary two-parent merge through successful restack run 34000892973; the run pinned both refs and re-ran full locked workspace tests + strict Clippy before non-force push d8b452cf1d609bb6e9c9a8a33f265c0a32dce7c9. Current exact #167 head is human-authored code-current doctoring commit 0c83cd5956f512d79c6600e823fcfa6d6f32af4e, mergeable with base SHA exactly a52ccd0...; its fresh required workflows are materialized but currently queued/pending, so this issue stays open until protected merge. No predecessor GREEN or queued state is being promoted to merge evidence.