diff --git a/CHANGELOG.md b/CHANGELOG.md index 0820f2a..4465da4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,10 @@ All notable changes to this project are documented here, following - nftables rendering bound the `prerouting` filter chain to the managed interface (`type filter hook prerouting … device`), which the kernel rejects — only ingress/egress chains may bind a device — so `blackwall-nft::apply` failed on any real ruleset. The chain is now unbound and classification is scoped per-rule with an `iifname` match, the correct pattern for a prerouting filter chain. Because an unbound chain runs for every interface, the closed posture (`default_state` closed) is now enforced by an explicit interface-scoped terminal `drop` rule instead of a chain-wide drop policy, so it no longer black-holes loopback or other-interface host traffic. Caught by the new deception↔scanner lab gate, the first end-to-end run of `apply` against a real `nft`. ### Added +- BIRD iBGP-snippet generator (deployment #3): `blackwalld bird-config --config ` generates BIRD's side of the blackwall↔BIRD iBGP session from blackwall's own config, so prefix lists and session params aren't hand-maintained in both `blackwall.conf` and `bird.conf`. A pure `blackwall_bgp::render_bird_ibgp` emits an `include` file: `OWN_V4/V6` prefix defines plus one MP-BGP `protocol bgp blackwall` session (ipv4/ipv6/flow4/flow6 channels; unicast import filters `net ~ [prefix+]`, flow filters `net.dst ~ [prefix+]`; MD5 → an `include "blackwall-secret.conf";` reference so the secret never lands in the generated file; GTSM → `ttl security on`). A new `rtbh local-addr=` sets blackwall's BGP source, emitted as BIRD's `neighbor` and bound by the speaker as its source so the two sides match by construction. Validated against real **BIRD 2.17.1** by a new `bird-gen` lab gate (the generated include establishes the session and imports both a `/32` blackhole and a FlowSpec rule) plus a `bird -p` parse-check. BIRD stays the fan-out point — blackwall injects once, BIRD re-advertises to every upstream via its existing per-peer filters. Non-breaking. +- Network-wide shadow mode (deployment #9): a global, opt-in `shadow` config directive that makes the mitigation plane log + record + meter every RTBH/FlowSpec/XDP mitigation the daemon *would* apply, **without executing it** — the interlock for running a detection-only deployment live on the security boundary. Shadow decorates the execution boundary only (a `ShadowBgpExecutor` that holds no BGP handle, so it's structurally incapable of announcing; no-op journals keep the mirror empty; the real iBGP session isn't spawned; the XDP map-apply is gated), while detection/selection/controller logic runs identically. Intended actions surface via INFO logs, a `blackwall_shadow_would_mitigate_total{plane,action}` counter, and `audit_log` rows (queryable through `/v1/audit`), with a startup `WARN: SHADOW MODE` banner. Non-breaking (absent `shadow` = live behavior). +- Anycast telemetry ingest (deployment #1/#2, sub-project D): the sFlow agent address (previously parsed then discarded) is now threaded into every `FlowObservation`, so the central `flow` daemon reads feeds from many anycast POPs as one logical view. A `pop agent= sampling=` config directive maps agents to POP names + expected rates; detections are tagged with the contributing POPs and a top-N attacker source-block (/24 v4, /48 v6) rollup; per-agent liveness and a sampling-sanity clamp (bounded to known agents, closing a spoofed-agent memory-DoS) guard the feed, exposed as `blackwall_flow_{pop_last_seen_seconds,agent_sampling_mismatch_total,unknown_agent_observations_total}`. `blackwalld sensor render-hsflowd` generates each POP's hsflowd.conf from the POP-map. Non-breaking (no `pop` block = per-victim detection as before, everything tagged `unknown`). +- Read-only control API (sub-project A·M4, Phase 1): a tenant-aware, bearer-authenticated (SHA-256, constant-time) read-only HTTP API (axum) with a code-first generated OpenAPI document at `/v1/openapi.json`, mounted as a task inside `blackwalld run` behind a new `api listen=… token-file=…` directive. Endpoints (all auth-guarded, under `/v1`): tenants + services + ip-assignments, active RTBH/FlowSpec/XDP mitigations, detections, sessions, and the audit log. A new `blackwall-api` crate holds the whole surface behind an `AppState` trait (unit-tested against an in-memory fake); the concrete store-backed impl + bind loop live in thin `blackwalld` glue. Mutation endpoints, daemon supervision, and a bench harness are later A·M4 phases. - AF_XDP zero-copy redirect foundation (sub-project B, B3.1 spike): the XDP program can now redirect a designated subset of packets to a userspace **AF_XDP** socket via a `BPF_MAP_TYPE_XSKMAP` (`XSKS`), and userspace actually receives them ahead of the kernel stack. The eBPF `xdp_filter` gained a small, fail-closed redirect fast path — an IPv4 UDP datagram whose destination port is in the userspace-populated `REDIRECT_PORT` set is handed to the AF_XDP socket bound at the frame's RX queue (`XskMap::redirect`); everything else keeps its current drop/rate-limit/SYN-cookie/pass behaviour, and an unconfigured box redirects nothing. Userspace adds `AfXdpReceiver` (in `blackwall-xdp`), a copy-mode AF_XDP receiver that owns the UMEM + fill/completion/RX rings and exposes the socket fd for registration into `XSKS` (`XdpDataplane::register_xsk` / `set_redirect_ports`); the AF_XDP socket/UMEM/ring layer is the pure-Rust **xdpilone** crate (no libbpf/libxdp-sys C dependency), since aya 0.13 ships only the `XSKMAP` binding. Proven end to end by a new `afxdp_redirect` veth lab gate (attach `xdp_filter`, bind an AF_XDP socket into `XSKS`, inject a UDP frame from the peer, assert it lands on the socket) plus a `BPF_PROG_TEST_RUN` assertion that the redirect branch fires. New stat reason `REASON_REDIRECT` (`XdpStats::redirected`). This is a feasibility spike + minimal foundation; the real deception-tier redirect condition and use-case wiring are the B3.2 follow-on (marked `// B3.2:` in the eBPF source). AF_XDP on veth is copy-mode only (no zero-copy); a real NIC can opt into zero-copy in B3.2 by dropping the copy bind flag. - In-kernel XDP SYN-cookie fast path (sub-project B, B2, issues #10/#9 follow-on): the XDP program now answers a TCP SYN **in-kernel** with a SipHash-cookie SYN-ACK via `XDP_TX`, at the driver level and ahead of nftables. It uses the **same cookie key as the userspace stateless responder** (shared via Postgres, get-or-create) and the same monotonic clock, so the two tiers' cookies interoperate: XDP absorbs the SYN flood, and a legitimate client's ACK falls through (`XDP_PASS`) to the userspace responder, which validates the byte-identical cookie and serves the port's banner. Enabled by `cookie-ports=` on the existing `xdp` config directive (`xdp interface=eth0 cookie-ports=8080,443`). Gated fail-closed: only SYNs to the box's own deception prefixes *and* a configured cookie-port are answered — everything else, including real services, passes through untouched. New metric: `blackwall_xdp_syn_cookies_sent_total`. - Stateless TCP SYN-cookie deception tier (sub-project A, issues #10/#9): closed ports opted into a new `stateless-tcp ports=…` config directive are routed to a keyed **SipHash-2-4** SYN-cookie responder over NFQUEUE instead of the interactive TPROXY honeypot. The responder answers a SYN with a stateless SYN-ACK cookie carrying no per-connection state; a client that completes the handshake gets the port's banner (PSH|ACK|FIN) and the connection closes — a spoofed-source SYN flood against these ports creates no state on the box. Dual-stack: IPv4 and IPv6 (v6 replies go via per-protocol raw sockets + `IPV6_PKTINFO`). A companion **reflection-safe UDP responder** answers deception UDP with a canned banner whose length is never greater than the request's (amplification factor ≤ 1), so it can never be abused as a UDP reflector. A new `DeceptionTransport` trait unifies the NFQUEUE (stateless) and TPROXY (interactive) transports behind one interface. New metrics on `blackwalld run`'s `/metrics`: `blackwall_stateless_syn_cookies_sent_total`, `blackwall_stateless_acks_validated_total`, `blackwall_stateless_acks_rejected_total`, `blackwall_stateless_udp_responses_total`. diff --git a/README.md b/README.md index 328246e..29e21dc 100644 --- a/README.md +++ b/README.md @@ -283,9 +283,12 @@ AF_XDP, rate limiting. **Deployment — Blackwall on an anycast network** *(next)* — turning the platform into a live deployment on an anycast ISP (centralized BGP brain + multi-POP telemetry). Staged, not flag-day: -- 🟡 **M0 — detection-only (shadow):** telemetry ingest (✅, above), plus a POP-sensor deploy - contract (hsflowd), a BIRD iBGP-snippet generator, network-wide shadow mode, and Incus/metrics - deploy glue. Run live, watch, tune — act on nothing. +- 🟡 **M0 — detection-only (shadow):** telemetry ingest ✅ (above), a POP-sensor deploy contract + (hsflowd) ✅, a **BIRD iBGP-snippet generator** ✅ (`blackwalld bird-config` emits BIRD's side of + the session + `OWN_V4/V6` defines from blackwall's config, validated against real BIRD2), and + **network-wide shadow mode** ✅ (a `shadow` directive logs+records every RTBH/FlowSpec/XDP + mitigation the daemon *would* apply, via `/v1/audit` + `blackwall_shadow_would_mitigate_total`, + without executing it). Remaining: Incus/metrics deploy glue. Run live, watch, tune — act on nothing. - ⏳ **M1 — arm with safety:** per-upstream blackhole communities, RPKI/ROA cross-check, a single audited disarm + blast-radius caps, and anycast self-protection (don't blackhole yourself). - ⏳ **M2 — deception + intel loop:** deception at home (tenant space only), coherent per-IP personas, diff --git a/docs/deployment.md b/docs/deployment.md index cda55b5..0035e43 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -66,9 +66,30 @@ sudo -E scripts/smoke-flow.sh # BGP mitigation end to end sudo -E scripts/smoke-deception.sh # routed deception → honeypot ``` +## Shadow mode (run detection-only, act on nothing) +For a first deployment on a live network, run the mitigation plane in **shadow +mode** before arming it. Add a bare `shadow` line to the config: +``` +shadow # log + record + meter every RTBH/FlowSpec/XDP mitigation, apply none +``` +With `shadow` set, `blackwalld flow` starts with a loud `WARN: SHADOW MODE — +mitigations are LOGGED, NOT APPLIED` banner. Detection, selection, and the +RTBH/FlowSpec/XDP controllers all run normally, but every mitigation the daemon +*would* apply is instead: +- logged at INFO (`shadow: would announce 203.0.113.7/32 …`), +- counted in `blackwall_shadow_would_mitigate_total{plane,action}` (Prometheus), and +- written to the audit log (visible via the read API's `/v1/audit` endpoint). + +No BGP session is opened, nothing is announced, and no XDP map is written — the +persistent mirror stays empty, so a later restart can't rehydrate never-vetted +entries. Run it for a day or a week, review the intended mitigations, and **arm +by removing the `shadow` line and re-applying**. Shadow is the mitigation-plane +interlock; the deception engine is unaffected. + ## Enable (staged) Bring the services up **deliberately**, watching `/metrics` and Postgres. Start -`flow` in detection-only mode (no rtbh/flowspec blocks in the config) first: +`flow` in detection-only mode (no rtbh/flowspec blocks in the config, or with +`shadow` set — see above) first: ```bash # systemd sudo systemctl enable --now blackwalld-flow @@ -223,13 +244,34 @@ POP's firewall/routes must permit outbound UDP to the collector over the WG interface, and the home box's config must accept it (`flow --listen 0.0.0.0:6343` or scoped to the mesh interface). +## Generate BIRD's side of the session (`bird-config`) +Blackwall's native speaker peers *into* your BIRD; the BIRD side of that iBGP +session (the `protocol bgp` stanza + the `OWN_V4/OWN_V6` prefix defines its +export filters reference) is generated from the same `blackwall.conf`, so you +don't hand-maintain the prefix/session lists twice: +```bash +blackwalld bird-config --config /etc/blackwall/blackwall.conf > /etc/bird/blackwall.conf +``` +Add `include "blackwall.conf";` to your `bird.conf` and `birdc configure`. This +requires `local-addr=` on the `rtbh` directive — blackwall's own BGP source +address, which the generator emits as BIRD's `neighbor` and the speaker binds as +its source so the session matches by construction (its family must match the +`peer=` address). If the `rtbh` block sets `md5=`, the generated file references +`include "blackwall-secret.conf";` instead of embedding the secret — keep that +one-line `password "…";` file `0600` alongside your other BIRD secrets. The +generated file is otherwise non-sensitive and safe to commit. BIRD remains the +fan-out point: blackwall injects a blackhole/FlowSpec once, and BIRD +re-advertises it to every upstream via your existing per-peer export filters — +so adding or retuning upstreams stays a pure BIRD-side operation. + ## Not yet implemented (know before you rely on it) -- **No AF_XDP zero-copy path yet.** B1 (XDP source-drop + rate-limit) and B2 - (stateless SYN cookies, both the userspace tier and the in-XDP fast path - above) are shipped; a zero-copy AF_XDP userspace path (B3) and xdpcap - observability + a DDoS-lab XDP gate (B4) are the remaining sub-project B - work. The nftables flowtable (`flowtable devices=…`) offloads established - forwarded real-service flows, but there is no kernel-bypass offload for the - deception/inspection path yet. Fine for moderate rates and on-box - source-drop/SYN-cookie absorption; line-rate volumetric attack traffic is - still best pushed to your router via BGP mitigation. +- **XDP data plane is complete but its scale ceiling is real.** Sub-project B is + shipped end to end: B1 (XDP source-drop + rate-limit), B2 (stateless SYN + cookies — userspace tier + in-XDP fast path), B3 (zero-copy AF_XDP UDP + responder), and B4 (xdpcap capture + the DDoS-lab XDP gates). Multi-queue + AF_XDP scaling is deferred (needs multi-queue-NIC hardware), and realistic + DDoS-scale stress testing is tracked separately (issue #67). The nftables + flowtable (`flowtable devices=…`) offloads established forwarded real-service + flows. Fine for moderate rates and on-box source-drop/SYN-cookie absorption; + line-rate volumetric attack traffic is still best pushed to your router via + BGP mitigation.