M1 arming hardening: rehydrate retry queue (#194) + RPKI cross-check (#5) + disarm residuals

CHANGELOG.md

@@ -9,6 +9,11 @@ All notable changes to this project are documented here, following
 - Speedtest now runs providers sequentially, measures each download over the full window, and reports the fastest clean provider — fixing wildly variable, under-reporting results on fast links.
 
 ### Fixed
+- M1 arming hardening (control plane) — closes the three arming-safety follow-ups from the M1 interlock. (**#194**) the boot-time rehydrate path (RTBH/FlowSpec/XDP) that re-announces persisted mitigations on an armed restart no longer strands an entry when the re-announce fails: it now KEEPS the entry and retries on the next tick (a `pending_reapply` self-heal mirroring the journal `pending_mirror`), converging once the session recovers — and, on the mutable-rate planes (FlowSpec/XDP), the retry RE-DERIVES the controller's current rule/action rather than replaying the value captured at queue time, so a stale retry can never revert a fresher successful update; surfaced as `blackwall_{rtbh,flowspec,xdp}_reapply_pending` gauges. (**#193 residuals**) the SIGUSR1 disarm logged its `DISARMED` banner twice (once before the withdrawal, once after) — de-duped to the single accurate post-withdrawal banner; and the disarm withdraw-error tolerance is now tested for FlowSpec + XDP, not just RTBH.
+
+### Added
+- RPKI pre-announce cross-check (deployment #5) — an optional, **fail-open** advisory that warns an operator when blackwall's RTBH blackholes would be dropped by RPKI-validating upstreams. A new `rpki-validator=<url>` directive (absent ⇒ off, non-breaking) points at a Routinator HTTP API; at startup and every `rpki-check-interval` (default 1h) a background task checks each `eligible`/`protected` prefix's **host-length more-specific** (`/32`, `/128` — the shape an RTBH actually announces, because a ROA whose `maxLength` is shorter than the host length makes those blackholes RPKI-INVALID) against `AS<local-as>`, and WARNs on a state **transition** (not every interval) when a prefix is `invalid`/`not-found` or the validator becomes unreachable. Never blocks detection or arming; an unrecognized/malformed validity response is treated as validator-down (never a spurious `valid`). New pure `blackwall-rpki` crate (classifier/URL/more-specific logic, 100% covered); metrics `blackwall_rpki_validator_up` + `blackwall_rpki_uncovered_prefixes` (present only when the check runs). Enabling it needs Routinator's `http-listen` on the box (its `:8282` is RTR-only); until then the check is simply off.
+
 - `blackwall_flow_pop_last_seen_seconds` reported a nonsense ~epoch-sized value (so any "POP sensor down/stale" alert built on it was dead). Fallout from the monotonic-clock change: the collector stamps `AgentStat.last_seen_ms` from the monotonic clock, but the `/metrics` renderer computed the age against wall-clock epoch — mixing the two domains, the gauge always read ~current-epoch-seconds regardless of how recently the agent was seen. The renderer now uses the collector's monotonic clock (exposed as `blackwall_flow::monotonic_now_ms()`), so a freshly-observed agent reads ~0 s. Found validating the M0 sockpuppet fleet. Detection was never affected (it used the monotonic clock consistently); only the liveness gauge was wrong.
 - Pre-M0 detection-quality batch (makes the shadow observation's FP/FN signal trustworthy before the AS214806 detection-only deploy; from a full-codebase review + re-triage). **Detector correctness:** windowing/eviction/hold-down now use a **monotonic clock** (`Instant` baseline) so an NTP/wall-clock step can't freeze or force-evict the window; a new **minimum-sample gate** (`--min-samples`, default 8) stops sampling-variance false positives (2 packets at 1-in-65536 no longer extrapolate to a 131k-pps "attack"); the sampling-rate sanity clamp is now **direction-aware** — a low reported rate is clamped up (anti-suppression) while a legitimately-high adaptive-sampling rate is *trusted* up to `expected × --max-sampling-factor` (default 64, floored at 4) instead of being clamped down and hiding a real flood; sFlow decoding is **per-sample resilient** so one malformed sample no longer discards a whole datagram's valid observations (envelope-framing errors stay fatal); and mitigation selection treats destination **port 0** (fragmented / non-TCP-UDP flows) as "no port data" → RTBH, never emitting a useless `dst_port:0` FlowSpec rule (`blackwall-flow`). **Config/API:** the config lexer now treats `#` as a comment only at line-start or after whitespace, so a value like `md5=sec#ret` is no longer silently truncated (`blackwall-config`); `parse_and_resolve()` validates the policy at load for `blackwalld flow` (the M0 entry point) and `bird-config`, so a semantically-invalid config fails at container start instead of mis-behaving through the observation window; and the read API's `?limit=` is clamped to `[1, 1000]` so a negative or huge value can't 500 or dump a whole table (`blackwall-api`). **New observability** (`/metrics`, `blackwall_flow_*`): `min_sample_suppressed_total`, `sampling_near_ceiling_total{pop}`, `sample_decode_errors_total` (distinct from the per-datagram `decode_errors`), and `detections_opened_total` / `detections_cleared_total` — together they let an operator tell a quiet POP from one silently dropping samples. Non-breaking (new CLI flags default to prior behaviour; `--bps-threshold` is documented as L2/frame-length based).
 - Lab CI reliability (blackwall#88): a stuck in-scenario command could hang a gate silently until the job cap. Root cause was two-fold — the harness ran one-shot commands via an unbounded `Command::output()`, and gate drivers ran `cargo test` *inside* the netns, whose long-lived cargo process (holding the Cargo build lock + a subprocess tree) could wedge in CI where a `sudo`-spawned tree escapes GitHub's step timeout. Fixes: `netns::run` now bounds and kills every command (own process group, drained pipes, reader-join timeouts) so it can never block the caller; daemon teardown reaps with a bounded wait; `spawn_bird` redirects its stdio like the other daemons so a killed lab can't leak a pipe-holder; and gates now run the **pre-built** interop binary directly (`scripts/build-lab-tests.sh` → `target/debug/lab-tests/<pkg>-<test>`) instead of `cargo test`, removing the cargo lock/daemon from the scenario entirely.

bin/blackwalld/Cargo.toml

@@ -20,6 +20,7 @@ blackwall-metrics = { path = "../../crates/blackwall-metrics" }
 blackwall-speedtest = { path = "../../crates/blackwall-speedtest" }
 blackwall-bgp = { path = "../../crates/blackwall-bgp" }
 blackwall-rtbh = { path = "../../crates/blackwall-rtbh" }
+blackwall-rpki = { path = "../../crates/blackwall-rpki" }
 blackwall-xdp = { path = "../../crates/blackwall-xdp" }
 async-trait = { workspace = true }
 axum = { workspace = true }