From 66901cf13f7cfb0b8233274be9454e08e9ab3599 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 24 Aug 2026 04:01:21 +0900 Subject: [PATCH 1/2] feat(store): HASH-partition security_event by tenant Convert unpartitioned security_event to PARTITION BY HASH (tenant_id) with eight children under pg_advisory_lock. Rows keep unmasked client IPs and paths. /healthz.event_partitions reports the child count. Logical restore still accepts schema 2 through the current version. --- CHANGELOG.md | 1 + README.md | 2 +- crates/waf-ids-core/src/lib.rs | 2 +- docs/architecture.md | 2 +- docs/doctoring/outbox-workers.md | 7 +- docs/doctoring/postgres-control-plane.md | 25 +- docs/product-technical-gap-baseline.md | 43 +-- docs/runbooks/operations.md | 1 + scripts/smoke.sh | 1 + src/control_plane.rs | 373 +++++++++++++++++++++-- src/lib.rs | 15 +- 11 files changed, 414 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ced0317..883bb7e4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Security +- PostgreSQL `security_event` is HASH-partitioned by `tenant_id` (8 children). Unpartitioned tables convert in place and keep unmasked client IPs and paths. `/healthz.event_partitions` reports the child count (0 on file/memory). Logical restore still accepts schema 2 through the current migration version; HASH does not change the snapshot shape. - PostgreSQL control-plane runtime is `wardnet_runtime` (NOSUPERUSER, NOBYPASSRLS, not the table owner). Migrations run as the login role, then `SET ROLE` so FORCE RLS binds even when the URL user is a superuser. Missing `wardnet.tenant_id` yields no rows. DDL (`DROP TABLE`, `DISABLE ROW LEVEL SECURITY`) is denied. Logical restore accepts schema 2 through the current migration version so a role-only upgrade cannot void the last pre-upgrade backup. - PostgreSQL control-plane logical backup and isolated restore drill (issue #80 remainder). `GET /api/backup` exports a hashed tenant snapshot (policy, events, outbox, receipts). `POST /api/backup` restores after schema and payload-hash checks. `POST /api/backup/drill` restores into an isolated tenant, compares unmasked invariants, and drops the drill tenant. Declared RPO is the last successful export; declared RTO is 60 seconds. File/memory adapters report `/healthz.backup=disabled`. Client IPs, paths, and actor names stay unmasked. - PostgreSQL control-plane mutations enqueue a transactional outbox row in the same transaction (issue #81). Security events append incrementally instead of rewriting the snapshot. A leased worker claims with `FOR UPDATE SKIP LOCKED`, retries with bounded backoff, dead-letters exhausted/permanent failures, and records unique receipts. Stdout SIEM export is at-least-once; the receipt is the exactly-once ack. `/healthz.outbox` and `GET /api/outbox` are operator-visible; `POST /api/outbox/{id}/replay` requeues dead letters with audit. File/memory adapters report `outbox=disabled`. `GET /api/outbox` is bounded to `EVENT_LIMIT` (dead letters and pending first). Processed `outbox_message` rows are pruned to that same cap on append, snapshot save, and worker ack; receipts stay as the exactly-once ack. diff --git a/README.md b/README.md index c01d5c89..35bd2d23 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,7 @@ Useful environment variables: - `ADMIN_TOKEN`: optional write token for management writes via `X-Admin-Token` - `DESTINATION_ALLOWLIST` / `DESTINATION_DENYLIST`: comma-separated hosts, `*.suffix`, or CIDRs for outbound `http`/`https`. Denylist wins. CIDR matches apply per resolved address and also authorize non-default ports. Loopback/private/metadata/site-local destinations are denied unless allowlisted (loopback development still permits loopback-class destinations). After a host is allowed, outbound HTTP connects only to those evaluated addresses (original Host/SNI). `/healthz.destination_mode` reports `production` or `development`. - `WAF_IDS_STATE_PATH`: optional JSON state path for loopback/community. When omitted, the service runs with seeded in-memory state. Production (non-loopback) binds require `CONTROL_PLANE_DATABASE_URL` instead. -- `CONTROL_PLANE_DATABASE_URL`: PostgreSQL URL for the production control plane (`postgres://…`). Secret; prefer `WAF_IDS_CREDENTIALS_PATH` key `control_plane_url`. `sslmode=require` / `verify-full` uses rustls with Mozilla roots (certificates always verified). `sslmode=disable` or omitted is plaintext. `allow`/`prefer` are rejected. After migrate, the session runs as `wardnet_runtime` (NOSUPERUSER, NOBYPASSRLS). `/healthz.persistence` reports `postgres` when connected. +- `CONTROL_PLANE_DATABASE_URL`: PostgreSQL URL for the production control plane (`postgres://…`). Secret; prefer `WAF_IDS_CREDENTIALS_PATH` key `control_plane_url`. `sslmode=require` / `verify-full` uses rustls with Mozilla roots (certificates always verified). `sslmode=disable` or omitted is plaintext. `allow`/`prefer` are rejected. After migrate, the session runs as `wardnet_runtime` (NOSUPERUSER, NOBYPASSRLS). `security_event` is HASH-partitioned by `tenant_id`. `/healthz.persistence` reports `postgres` when connected; `/healthz.event_partitions` reports the child count. - `DNSBL_ORIGIN`: DNSBL zone origin, default `dnsbl.local` - `EVENT_LIMIT`: retained event count, default `1000`; must be greater than zero. Also caps `GET /api/outbox` and processed outbox-row retention. - `CORAZA_LIB_PATH` / `CORAZA_RULES_PATH` / `CORAZA_DIRECTIVES`: optional in-process libcoraza. A missing library or empty ruleset fails startup. `/healthz.proven_engine` reports `coraza_in_process`. diff --git a/crates/waf-ids-core/src/lib.rs b/crates/waf-ids-core/src/lib.rs index 291a3900..b22c7d24 100644 --- a/crates/waf-ids-core/src/lib.rs +++ b/crates/waf-ids-core/src/lib.rs @@ -1177,7 +1177,7 @@ fn buyer_evidence_endpoints() -> Vec { "GET", "/healthz", "application/json", - "runtime health, persistence mode, DNSBL origin, and event retention limit", + "runtime health, persistence mode, DNSBL origin, event retention, and HASH event partitions", true, ), buyer_evidence_endpoint( diff --git a/docs/architecture.md b/docs/architecture.md index f020b2d6..6edc3e53 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -29,7 +29,7 @@ flowchart LR - `src/main.rs`: process startup and operator configuration from `BIND_ADDR`, `ADMIN_TOKEN`, `WAF_IDS_STATE_PATH`, `DNSBL_ORIGIN`, and `EVENT_LIMIT`. - `src/lib.rs`: Axum app, routing, management APIs, optional JSON persistence, gateway handler, upstream proxying, admin console, support bundle assembly, NDJSON event export, and in-crate HTTP tests. Persistence, destination-list, and sidecar settings validate before the readiness line is printed. -- `src/control_plane.rs`: PostgreSQL production authority (issue #80). Non-loopback binds require `CONTROL_PLANE_DATABASE_URL`. Tenant isolation is default-deny RLS under `wardnet_runtime` (not superuser/owner). `sslmode=require` uses rustls. The JSON file adapter remains loopback/community only. +- `src/control_plane.rs`: PostgreSQL production authority (issue #80). Non-loopback binds require `CONTROL_PLANE_DATABASE_URL`. Tenant isolation is default-deny RLS under `wardnet_runtime` (not superuser/owner). `sslmode=require` uses rustls. `security_event` is HASH-partitioned by `tenant_id` (`/healthz.event_partitions`). The JSON file adapter remains loopback/community only. - `src/outbox.rs`: transactional outbox + leased workers (issue #81). Security events append incrementally with an outbox row in the same transaction. Workers claim with `SKIP LOCKED`. `GET /api/outbox` is bounded to `EVENT_LIMIT` (processed rows pruned; receipts kept). `/healthz.outbox` is operator-visible. - `src/destination.rs`: fail-closed outbound URL policy (issue #79) for every `http`/`https` send. CIDR allowlist exceptions are per resolved address; blocking DNS is offloaded from Tokio workers. The outbound HTTP client DNS resolver returns only addresses that already passed policy (TCP peer pin / DNS-rebinding TOCTOU close). - `crates/waf-ids-core`: reusable domain models plus validation, upsert, scoring, DNSBL zone export, event retention, threat-feed freshness, KPI snapshot, and commercial readiness logic. diff --git a/docs/doctoring/outbox-workers.md b/docs/doctoring/outbox-workers.md index 120f9a18..cb6337a3 100644 --- a/docs/doctoring/outbox-workers.md +++ b/docs/doctoring/outbox-workers.md @@ -49,6 +49,7 @@ append, snapshot save, and worker ack. `outbox_receipt` rows stay; they are the exactly-once ack. Dead letters are never pruned. Loopback file/memory adapters keep in-process stdout SIEM and report -`outbox=disabled`. Remaining: HASH partitioning and additional consumers -(TAXII poll, Clearfolio, contextual-orchestrator) on the same -message/receipt contract. Backup/restore drill is on the PostgreSQL plane. +`outbox=disabled`. `security_event` HASH partitioning is on the PostgreSQL +plane. Remaining consumers: TAXII poll, Clearfolio, contextual-orchestrator +on the same message/receipt contract. Backup/restore drill is on the +PostgreSQL plane. diff --git a/docs/doctoring/postgres-control-plane.md b/docs/doctoring/postgres-control-plane.md index a7ac2ba9..7fff5bbf 100644 --- a/docs/doctoring/postgres-control-plane.md +++ b/docs/doctoring/postgres-control-plane.md @@ -26,6 +26,15 @@ Transaction isolation*. https://www.postgresql.org/docs/current/transaction-iso. commits in one transaction so a policy mutation cannot land without its audit records. +PostgreSQL Global Development Group. (2026). *PostgreSQL documentation: Table +partitioning*. https://www.postgresql.org/docs/current/ddl-partitioning.html + +- **Design impact:** `security_event` is `PARTITION BY HASH (tenant_id)` with + eight children so tenant-scoped SOC queries prune and high-volume appends do + not share one btree. The partition key is part of the primary key. Logical + backups stay a tenant snapshot; HASH is an on-disk layout, not a restore + schema bump that voids prior artifacts. + National Institute of Standards and Technology. (2022). *Secure Software Development Framework (SSDF) version 1.1* (NIST SP 800-218). https://doi.org/10.6028/NIST.SP.800-218 @@ -50,10 +59,10 @@ NOBYPASSRLS, not table owner) so FORCE RLS binds. Provision that role and `GET /api/backup` (admin read) exports a hashed logical snapshot stamped with the current `MIGRATION_VERSION`. `POST /api/backup` restores after schema-version -and payload-hash checks. Role-only migrations (v3 `wardnet_runtime`) do not -change table shape, so `verify()` accepts schema versions -`MIN_RESTORABLE_SCHEMA_VERSION` (2) through the current version rather than -rejecting pre-upgrade snapshots. `POST /api/backup/drill` restores into an +and payload-hash checks. Role-only migrations (v3 `wardnet_runtime`) and HASH +layout (v4) do not change the logical snapshot shape, so `verify()` accepts +schema versions `MIN_RESTORABLE_SCHEMA_VERSION` (2) through the current version +rather than rejecting pre-upgrade snapshots. `POST /api/backup/drill` restores into an isolated tenant, compares invariants, and drops the drill rows. Declared RPO: last successful export (`on-demand-logical-snapshot`). Declared RTO: 60 seconds. `/healthz.backup` is `ready` on PostgreSQL. @@ -68,5 +77,9 @@ https://doi.org/10.6028/NIST.SP.800-34r1 `pg_dump`) so RLS tenant context is preserved and secrets (admin tokens, database URL) are never copied. -Remaining: HASH partitioning for `security_event`, -optimistic concurrency. +`security_event` is HASH-partitioned by `tenant_id` (8 children) so tenant +queries prune and high-volume appends do not share one btree. Existing +unpartitioned tables convert under `pg_advisory_lock`; rows keep unmasked +client IPs and paths. `/healthz.event_partitions` reports the child count. + +Remaining: optimistic concurrency. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index a81bbef2..ec84f5fb 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1,6 +1,6 @@ # Product and technical gap baseline -Snapshot date: 2026-08-23T18:20Z (exact-head inventory of then-open GitHub PRs +Snapshot date: 2026-08-23T19:00Z (exact-head inventory of then-open GitHub PRs and Issues plus operator-perceptible gaps). Update this file on every hourly loop. Commercial contract and `/api/commercial/readiness` remain **2B KRW**. The @@ -25,7 +25,8 @@ not “waiting on review/CI time”. | PR | Title | Head | Checks | Reviews | Merge blocker | | --- | --- | --- | --- | --- | --- | -| [#103](https://github.com/ContextualWisdomLab/wardnet/pull/103) | feat(store): non-owner PostgreSQL runtime role after migrate | `feat/issue-80-runtime-role` stacked on #100 | local fmt/test/clippy + two `/healthz` smokes; live `postgres_runtime_role_is_not_superuser_and_rls_default_denies` | Author this pass | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 first. Do not `--admin`. Do not re-implement rustls, outbox, retention, or backup. | +| this pass | feat(store): HASH-partition security_event by tenant | `feat/issue-80-event-hash-partition` stacked on #103 | local fmt/test/clippy + two `scripts/smoke.sh` + live postgres `/healthz.event_partitions=8` | Author this pass | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 then #103 first. Do not `--admin`. Do not re-implement runtime role. | +| [#103](https://github.com/ContextualWisdomLab/wardnet/pull/103) | feat(store): non-owner PostgreSQL runtime role after migrate | `feat/issue-80-runtime-role` stacked on #100 | still-valid Devin restore-window finding fixed this pass (`MIN_RESTORABLE_SCHEMA_VERSION=2`); local fmt/test/clippy + smokes | Author this pass; Devin COMMENTED (v3 backup voiding addressed) | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 first. Do not `--admin`. Do not re-implement rustls, outbox, retention, backup, or HASH. | | [#102](https://github.com/ContextualWisdomLab/wardnet/pull/102) | feat(store): logical backup and isolated restore drill | squash-merged into #100 (`321e792`) | prior hour | Author prior hour | Folded into rustls stack. Do not re-implement. | | [#101](https://github.com/ContextualWisdomLab/wardnet/pull/101) | feat(store): bound outbox listing and prune processed rows | `feat/issue-81-outbox-retention` (`0c2167a`) stacked on #100 | still-valid Devin prune-cap finding fixed this pass (`EVENT_LIMIT` on save/ack) | Author; Devin COMMENTED (prune thread addressed) | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 first. Do not `--admin`. | | [#100](https://github.com/ContextualWisdomLab/wardnet/pull/100) | feat(store): rustls for production PostgreSQL `sslmode=require` | `feat/issue-80-postgres-rustls` stacked on #99 | local fmt/test/clippy + two `/healthz` smokes; live `sslmode=require` fails closed against plaintext postgres | Author this pass | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 first. Do not `--admin`. Do not re-implement the postgres gate or outbox. | @@ -59,7 +60,7 @@ by ruleset `18156473` (not by failing Checks). Do not `--admin` merge. | [#83](https://github.com/ContextualWisdomLab/wardnet/issues/83) | [P1] Add bounded distributed admission control, trusted client attribution, and overload behavior | high | | [#82](https://github.com/ContextualWisdomLab/wardnet/issues/82) | [P1] Integrate Keyverse identity, tenant authorization, consent, and human approval evidence | high (blocked) | | [#81](https://github.com/ContextualWisdomLab/wardnet/issues/81) | [P0] Add a transactional outbox and idempotent leased workers for external effects | **critical — first slice on #99; bounded list/retention on #101** | -| [#80](https://github.com/ContextualWisdomLab/wardnet/issues/80) | [P0] Add an authoritative PostgreSQL control plane with tenant isolation and recoverable migrations | **critical — gate on #98; rustls/backup on #100; non-owner role this pass** | +| [#80](https://github.com/ContextualWisdomLab/wardnet/issues/80) | [P0] Add an authoritative PostgreSQL control plane with tenant isolation and recoverable migrations | **critical — gate on #98; rustls/backup on #100; non-owner role on #103; HASH partitions this pass** | | [#79](https://github.com/ContextualWisdomLab/wardnet/issues/79) | [P0] Enforce a fail-closed destination policy for all outbound traffic | **critical — closed in runtime on #96** | | [#78](https://github.com/ContextualWisdomLab/wardnet/issues/78) | [P0] Fail closed when management credentials are absent | **critical — closed in runtime on #94** | | [#75](https://github.com/ContextualWisdomLab/wardnet/issues/75) | Rename Kubernetes manifest to wardnet.yaml after external-secret hardening lands | medium | @@ -95,7 +96,7 @@ Management auth is shared secrets (`X-Admin-Token`) plus optional multi-token RBAC. Keyverse (OIDC/SCIM/FIDO2) is not wired. Fail-closed (#78) is the prerequisite shipped on PR #94. -### Durable control plane (issue #80) — **production gate on #98; rustls on #100; backup this pass** +### Durable control plane (issue #80) — **production gate on #98; rustls on #100; backup on #102; runtime role on #103; HASH this pass** PostgreSQL is required for non-loopback binds (`CONTROL_PLANE_DATABASE_URL`). `src/control_plane.rs` migrates 3NF two-word tables with default-deny RLS @@ -110,8 +111,13 @@ into an isolated tenant, compares unmasked invariants, and drops the drill tenant. Declared RPO: last successful export. Declared RTO: 60s. `/healthz.backup` is `ready` on PostgreSQL, `disabled` on file/memory. Runtime is `wardnet_runtime` (NOSUPERUSER, NOBYPASSRLS) after migrate. -Remaining: event HASH partitioning, optimistic concurrency. Physical/PITR -backups stay a DBA concern. +Logical restore accepts schema 2 through the current migration version +(`MIN_RESTORABLE_SCHEMA_VERSION`); role-only and HASH-layout migrations do not +void pre-upgrade snapshots. `security_event` is `PARTITION BY HASH (tenant_id)` +with 8 children. Unpartitioned tables convert in place under `pg_advisory_lock`. +`/healthz.event_partitions` is 8 on PostgreSQL, 0 on file/memory. Client IPs +and paths stay unmasked across convert. Remaining: optimistic concurrency. +Physical/PITR backups stay a DBA concern. ### Transactional outbox (issue #81) — **first slice on #99; retention on #101** @@ -157,7 +163,8 @@ replays `security_event.recorded` as stdout SIEM with receipts. | Ten UI-UX areas | Inventoried in `docs/ui-ux/storybook-scene-inventory.md` | | Node Storybook | **Not hosted in `/admin`** (embedded-console architecture). File:// inventory is the scene/edge-case contract this pass. | | Outbox card | Embedded `/admin` Outbox section | -| Backup card | Embedded `/admin` Control-plane backup section this pass | +| Backup card | Embedded `/admin` Control-plane backup section | +| Event partitions KPI | Embedded `/admin` KPI tile from `/healthz.event_partitions` this pass | ### CSAP / SOC 2 vs PII unmasking @@ -169,9 +176,9 @@ future encryption-at-rest. ### Coverage / docstring bar Org 100% line/branch/docstring applies to **changed** surfaces this loop -(backup export/verify/restore/drill, health/API, admin card, EVENT_LIMIT -prune on save/ack). Remaining holes on untouched handlers stay listed -for later loops. +(HASH convert SQL, restorable schema window, `/healthz.event_partitions`, +admin KPI tile, probe upgrade preserving unmasked IPs/paths). Remaining +holes on untouched handlers stay listed for later loops. ### Ecosystem connectors (leverage order) @@ -185,17 +192,19 @@ for later loops. ## This loop’s shipped gap -Issue **#80** still-valid #98 finding: non-owner `wardnet_runtime` after -migrate so FORCE RLS binds (CI `POSTGRES_USER` is a superuser). Do not -re-implement #78, sidecar, pin, libcoraza, the postgres gate, outbox, rustls, -retention, or backup/restore. +Issue **#80** remaining: HASH-partition `security_event` by `tenant_id` (8 +children) stacked on #103. Still-valid #103 Devin finding: `verify()` now +accepts schema 2..=current so a role-only/HASH-layout upgrade cannot void +the last pre-upgrade logical backup. Do not re-implement #78, sidecar, pin, +libcoraza, the postgres gate, outbox, rustls, retention, backup/restore, or +the runtime role. ## Next hourly loop (do, do not report) 1. Second independent APPROVE on #91/#92. Do not `--admin`. -2. Keep #94/#95/#96/#97/#98/#99/#100 and this runtime-role PR merge-ready. +2. Keep #94/#95/#96/#97/#98/#99/#100/#103 and this HASH PR merge-ready. Merge order #94 independently; #95 then #96 then #97 then #98 then #99 - then #100 then this. + then #100 then #103 then this. 3. Next runtime gap if policy still blocks: extra #81 consumers (TAXII / - Clearfolio / orchestrator) or HASH partitioning. + Clearfolio / orchestrator) or optimistic concurrency. 4. Refresh this file’s PR/Issue tables from `gh pr list` / `gh issue list`. diff --git a/docs/runbooks/operations.md b/docs/runbooks/operations.md index 380311b9..a00c44dd 100644 --- a/docs/runbooks/operations.md +++ b/docs/runbooks/operations.md @@ -68,6 +68,7 @@ Expected fields: - `credentials_source`: `file`, `env`, or `none` - `admin_auth_configured`: whether any admin write token is configured - `backup`: `ready` on PostgreSQL (logical export/restore available) or `disabled` on file/memory +- `event_partitions`: HASH child count for `security_event` (8 on PostgreSQL, 0 on file/memory) ## Control-plane backup and restore drill diff --git a/scripts/smoke.sh b/scripts/smoke.sh index 9530cbc8..25b0b59c 100755 --- a/scripts/smoke.sh +++ b/scripts/smoke.sh @@ -84,6 +84,7 @@ assert_json_field "$health" 'data["proven_engine_fail_closed"] is False' assert_json_field "$health" 'data["destination_mode"] == "development"' assert_json_field "$health" 'data["outbox"] == "disabled"' assert_json_field "$health" 'data["outbox_pending"] == 0' +assert_json_field "$health" 'data["event_partitions"] == 0' engine_status="$(curl -fsS "$BASE_URL/api/waf/engine-status")" assert_json_field "$engine_status" 'data["mode"] == "ingest_hints_only"' diff --git a/src/control_plane.rs b/src/control_plane.rs index e4d65b5b..793ceef6 100644 --- a/src/control_plane.rs +++ b/src/control_plane.rs @@ -25,19 +25,23 @@ use waf_ids_core::{ /// Default tenant used until Keyverse supplies claims (#82). pub const DEFAULT_TENANT_ID: &str = "local-lab"; -const MIGRATION_VERSION: i32 = 3; +const MIGRATION_VERSION: i32 = 4; /// Oldest logical-backup schema that restores on this binary. /// -/// v3 only provisions `wardnet_runtime`; it does not change table shape. A -/// snapshot exported at schema 2 is restorable here so a role-only upgrade -/// cannot void the declared RPO. +/// v3 only provisions `wardnet_runtime`. v4 HASH-partitions `security_event` +/// without changing the logical snapshot shape. A snapshot exported at schema +/// 2 is restorable here so those upgrades cannot void the declared RPO. const MIN_RESTORABLE_SCHEMA_VERSION: i32 = 2; +/// HASH partitions for `security_event` (by `tenant_id`). +pub const EVENT_PARTITION_MODULUS: i32 = 8; /// Non-owner, non-superuser role used after migrations so FORCE RLS binds. pub const RUNTIME_ROLE: &str = "wardnet_runtime"; /// Declared RPO: last successful `GET /api/backup` (on-demand logical snapshot). pub const BACKUP_RPO: &str = "on-demand-logical-snapshot"; /// Declared RTO budget for an isolated restore drill. pub const BACKUP_RTO_BUDGET_MS: u64 = 60_000; +/// Session advisory lock for schema/partition DDL (cross-process). +const MIGRATION_LOCK_KEY: i64 = 80_201_680; /// Recoverable forward migration. Two-word snake_case names, 3NF, RLS. pub const MIGRATION_SQL: &str = r#" @@ -261,6 +265,119 @@ GRANT SELECT, INSERT, UPDATE, DELETE ON GRANT SELECT ON schema_migration TO wardnet_runtime; "#; +fn hash_partition_sql_for(parent: &str) -> Result { + if parent.is_empty() + || !parent.as_bytes()[0].is_ascii_lowercase() + || !parent + .bytes() + .all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'_') + { + return Err("hash partition parent must be a lowercase SQL identifier".to_string()); + } + let modulus = EVENT_PARTITION_MODULUS; + let unpartitioned = format!("{parent}_unpartitioned"); + Ok(format!( + r#" +DO $hash$ +DECLARE + kind "char"; + i integer; +BEGIN + SELECT c.relkind INTO kind + FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = current_schema() + AND c.relname = '{parent}'; + + IF kind = 'r' THEN + ALTER TABLE {parent} RENAME TO {unpartitioned}; + DROP INDEX IF EXISTS {parent}_tenant_event; + CREATE TABLE {parent} ( + tenant_id TEXT NOT NULL REFERENCES tenant_account (tenant_id), + event_id BIGINT NOT NULL, + timestamp_unix BIGINT NOT NULL, + client_address TEXT, + route_id TEXT, + action_name TEXT NOT NULL, + event_reason TEXT NOT NULL, + event_score INTEGER NOT NULL, + request_path TEXT NOT NULL, + PRIMARY KEY (tenant_id, event_id) + ) PARTITION BY HASH (tenant_id); + i := 0; + WHILE i < {modulus} LOOP + EXECUTE format( + 'CREATE TABLE {parent}_p%s PARTITION OF {parent} FOR VALUES WITH (MODULUS {modulus}, REMAINDER %s)', + i, i + ); + i := i + 1; + END LOOP; + INSERT INTO {parent} ( + tenant_id, event_id, timestamp_unix, client_address, route_id, + action_name, event_reason, event_score, request_path + ) + SELECT tenant_id, event_id, timestamp_unix, client_address, route_id, + action_name, event_reason, event_score, request_path + FROM {unpartitioned}; + DROP TABLE {unpartitioned}; + ALTER TABLE {parent} ENABLE ROW LEVEL SECURITY; + ALTER TABLE {parent} FORCE ROW LEVEL SECURITY; + DROP POLICY IF EXISTS tenant_isolation ON {parent}; + CREATE POLICY tenant_isolation ON {parent} + USING (tenant_id = current_setting('wardnet.tenant_id', true)) + WITH CHECK (tenant_id = current_setting('wardnet.tenant_id', true)); + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'wardnet_runtime') THEN + GRANT SELECT, INSERT, UPDATE, DELETE ON {parent} TO wardnet_runtime; + i := 0; + WHILE i < {modulus} LOOP + EXECUTE format( + 'GRANT SELECT, INSERT, UPDATE, DELETE ON {parent}_p%s TO wardnet_runtime', + i + ); + i := i + 1; + END LOOP; + END IF; + ELSIF kind = 'p' THEN + i := 0; + WHILE i < {modulus} LOOP + IF NOT EXISTS ( + SELECT 1 FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = current_schema() + AND c.relname = format('{parent}_p%s', i) + ) THEN + EXECUTE format( + 'CREATE TABLE {parent}_p%s PARTITION OF {parent} FOR VALUES WITH (MODULUS {modulus}, REMAINDER %s)', + i, i + ); + END IF; + i := i + 1; + END LOOP; + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'wardnet_runtime') THEN + GRANT SELECT, INSERT, UPDATE, DELETE ON {parent} TO wardnet_runtime; + i := 0; + WHILE i < {modulus} LOOP + IF EXISTS ( + SELECT 1 FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = current_schema() + AND c.relname = format('{parent}_p%s', i) + ) THEN + EXECUTE format( + 'GRANT SELECT, INSERT, UPDATE, DELETE ON {parent}_p%s TO wardnet_runtime', + i + ); + END IF; + i := i + 1; + END LOOP; + END IF; + END IF; +END +$hash$; +"#, + )) +} + /// Fail closed when a non-loopback bind has no control-plane URL. pub fn require_postgres_for_bind( bind_addr: &str, @@ -413,6 +530,51 @@ async fn assume_runtime_role(client: &Client) -> Result<(), String> { Ok(()) } +async fn apply_schema(client: &Client) -> Result<(), String> { + let applied = match client + .query_one( + "SELECT COALESCE(MAX(migration_version), 0) FROM schema_migration", + &[], + ) + .await + { + Ok(row) => row.get::<_, i32>(0), + Err(_) => 0, + }; + if applied < MIGRATION_VERSION { + client + .batch_execute(MIGRATION_SQL) + .await + .map_err(|error| format!("control plane migration failed: {error:?}"))?; + client + .execute( + "INSERT INTO schema_migration (migration_version) VALUES ($1) ON CONFLICT (migration_version) DO NOTHING", + &[&MIGRATION_VERSION], + ) + .await + .map_err(|error| format!("control plane migration version failed: {error}"))?; + } + let hash_sql = hash_partition_sql_for("security_event")?; + client + .batch_execute(&hash_sql) + .await + .map_err(|error| format!("control plane event hash partition failed: {error:?}"))?; + Ok(()) +} + +async fn event_partition_count(client: &Client) -> Result { + let row = client + .query_one( + "SELECT COUNT(*)::bigint + FROM pg_partition_tree('security_event'::regclass) + WHERE level = 1", + &[], + ) + .await + .map_err(|error| format!("control plane event partition count failed: {error}"))?; + Ok(row.get(0)) +} + /// Serializes schema application across connections (DROP/CREATE POLICY is not concurrent-safe). static MIGRATION_GATE: std::sync::LazyLock> = std::sync::LazyLock::new(|| Mutex::new(())); @@ -478,31 +640,15 @@ impl PostgresPlane { async fn migrate(&self) -> Result<(), String> { let _gate = MIGRATION_GATE.lock().await; let client = self.client.lock().await; - let applied = match client - .query_one( - "SELECT COALESCE(MAX(migration_version), 0) FROM schema_migration", - &[], - ) - .await - { - Ok(row) => row.get::<_, i32>(0), - Err(_) => 0, - }; - if applied >= MIGRATION_VERSION { - return Ok(()); - } - client - .batch_execute(MIGRATION_SQL) - .await - .map_err(|error| format!("control plane migration failed: {error:?}"))?; client - .execute( - "INSERT INTO schema_migration (migration_version) VALUES ($1) ON CONFLICT (migration_version) DO NOTHING", - &[&MIGRATION_VERSION], - ) + .execute("SELECT pg_advisory_lock($1)", &[&MIGRATION_LOCK_KEY]) .await - .map_err(|error| format!("control plane migration version failed: {error}"))?; - Ok(()) + .map_err(|error| format!("control plane migration lock failed: {error}"))?; + let result = apply_schema(&client).await; + let _ = client + .execute("SELECT pg_advisory_unlock($1)", &[&MIGRATION_LOCK_KEY]) + .await; + result } async fn assume_runtime_role(&self) -> Result<(), String> { @@ -539,6 +685,35 @@ impl PostgresPlane { Ok(count) } + pub async fn event_partition_count(&self) -> Result { + let client = self.client.lock().await; + event_partition_count(&client).await + } + + #[cfg(test)] + async fn security_event_tableoid(&self) -> Result { + let mut client = self.client.lock().await; + let tx = client + .transaction() + .await + .map_err(|error| error.to_string())?; + tx.execute( + "SELECT set_config('wardnet.tenant_id', $1, true)", + &[&self.tenant_id], + ) + .await + .map_err(|error| error.to_string())?; + let row = tx + .query_one( + "SELECT tableoid::regclass::text FROM security_event WHERE tenant_id = $1 LIMIT 1", + &[&self.tenant_id], + ) + .await + .map_err(|error| error.to_string())?; + tx.commit().await.map_err(|error| error.to_string())?; + Ok(row.get(0)) + } + #[cfg(test)] async fn runtime_ddl_is_denied(&self) -> Result { let client = self.client.lock().await; @@ -2753,4 +2928,148 @@ mod tests { .expect("tenant a rows"); assert!(!loaded_a.routes.is_empty()); } + + #[test] + fn hash_partition_parent_rejects_injection() { + assert!(hash_partition_sql_for("security_event").is_ok()); + assert!(hash_partition_sql_for("event_hash_probe").is_ok()); + assert!(hash_partition_sql_for("event;drop").is_err()); + assert!(hash_partition_sql_for("Event").is_err()); + assert!(hash_partition_sql_for("").is_err()); + let sql = hash_partition_sql_for("security_event").expect("sql"); + assert!(sql.contains("PARTITION BY HASH (tenant_id)")); + assert!(sql.contains(&format!("MODULUS {EVENT_PARTITION_MODULUS}"))); + } + + async fn owner_client(url: &str) -> tokio_postgres::Client { + let (client, connection) = tokio_postgres::connect(url, tokio_postgres::NoTls) + .await + .expect("owner connect"); + tokio::spawn(async move { + let _ = connection.await; + }); + client + } + + #[tokio::test] + async fn postgres_security_event_is_hash_partitioned() { + let Some(url) = test_database_url() else { + return; + }; + let tenant = unique_tenant("event-hash"); + let plane = PostgresPlane::connect_tenant(&url, &tenant) + .await + .expect("plane"); + assert_eq!( + plane + .event_partition_count() + .await + .expect("partition count"), + i64::from(EVENT_PARTITION_MODULUS) + ); + let mut seeded = AppData::seeded(); + seeded.events.push(sample_event(1, "/hash-path")); + seeded.next_event_id = 2; + plane.save(&seeded).await.expect("seed with unmasked event"); + let loaded = plane.load().await.expect("load").expect("snapshot"); + assert!( + loaded.events.iter().any(|event| event.path == "/hash-path" + && event.client_ip.map(|ip| ip.to_string()) == Some("198.51.100.20".into())), + "HASH partitions must keep client IPs and paths unmasked" + ); + let tableoid = plane + .security_event_tableoid() + .await + .expect("child tableoid"); + assert!( + tableoid.starts_with("security_event_p"), + "row must land in a HASH child, got {tableoid}" + ); + } + + #[tokio::test] + async fn postgres_hash_partition_convert_preserves_unmasked_probe_rows() { + let Some(url) = test_database_url() else { + return; + }; + let tenant = unique_tenant("hash-probe"); + let plane = PostgresPlane::connect_tenant(&url, &tenant) + .await + .expect("plane so tenant_account and role exist"); + plane.save(&AppData::seeded()).await.expect("seed tenant"); + + let client = owner_client(&url).await; + client + .batch_execute( + "DROP TABLE IF EXISTS event_hash_probe CASCADE; + CREATE TABLE event_hash_probe ( + tenant_id TEXT NOT NULL REFERENCES tenant_account (tenant_id), + event_id BIGINT NOT NULL, + timestamp_unix BIGINT NOT NULL, + client_address TEXT, + route_id TEXT, + action_name TEXT NOT NULL, + event_reason TEXT NOT NULL, + event_score INTEGER NOT NULL, + request_path TEXT NOT NULL, + PRIMARY KEY (tenant_id, event_id) + );", + ) + .await + .expect("unpartitioned probe"); + client + .execute( + "INSERT INTO event_hash_probe ( + tenant_id, event_id, timestamp_unix, client_address, route_id, + action_name, event_reason, event_score, request_path + ) VALUES ($1, 1, 1700000000, '198.51.100.20', 'demo', 'blocked', 'fixture', 80, '/probe-path')", + &[&tenant], + ) + .await + .expect("probe row"); + let sql = hash_partition_sql_for("event_hash_probe").expect("probe sql"); + client + .batch_execute(&sql) + .await + .expect("convert unpartitioned probe"); + let kind: String = client + .query_one( + "SELECT relkind::text FROM pg_class WHERE relname = 'event_hash_probe'", + &[], + ) + .await + .expect("kind") + .get(0); + assert_eq!(kind, "p"); + let children: i64 = client + .query_one( + "SELECT COUNT(*)::bigint FROM pg_partition_tree('event_hash_probe'::regclass) WHERE level = 1", + &[], + ) + .await + .expect("children") + .get(0); + assert_eq!(children, i64::from(EVENT_PARTITION_MODULUS)); + let row = client + .query_one( + "SELECT client_address, request_path, tableoid::regclass::text + FROM event_hash_probe WHERE event_id = 1", + &[], + ) + .await + .expect("converted row"); + let ip: String = row.get(0); + let path: String = row.get(1); + let tableoid: String = row.get(2); + assert_eq!(ip, "198.51.100.20"); + assert_eq!(path, "/probe-path"); + assert!( + tableoid.starts_with("event_hash_probe_p"), + "converted row must land in a HASH child, got {tableoid}" + ); + client + .batch_execute("DROP TABLE IF EXISTS event_hash_probe CASCADE") + .await + .expect("drop probe"); + } } diff --git a/src/lib.rs b/src/lib.rs index 1af222a1..6d7526a6 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -362,6 +362,7 @@ impl AppState { } else { "disabled".to_string() }, + event_partitions: 0, } } @@ -380,6 +381,9 @@ impl AppState { } Err(_) => health.outbox = "error".to_string(), } + if let Ok(count) = plane.event_partition_count().await { + health.event_partitions = count; + } health } } @@ -516,6 +520,8 @@ pub struct HealthStatus { pub outbox_oldest_age_seconds: Option, /// `ready` when PostgreSQL logical backup/restore is the authority; `disabled` on file/memory. pub backup: String, + /// HASH child count for `security_event` (0 on file/memory). + pub event_partitions: i64, } const PHISHING_DATABASE_DEFAULT_FEED_ID: &str = "phishing-database-active"; @@ -3323,8 +3329,8 @@ function table(capt,cols,rows){ } function toast(msg,ok){const d=document.createElement('div');d.className='toast '+(ok?'ok':'bad');d.textContent=msg;$('toast').appendChild(d);setTimeout(()=>d.remove(),4500);} async function guard(id,fn){try{await fn();}catch(e){$(id).innerHTML='

Error: '+esc(e.message)+'

';}} -async function loadKpis(){const k=await getJSON('/api/kpis'); - const t=[['Routes',k.route_count],['Threat indicators',k.threat_indicator_count],['DNSBL entries',k.dnsbl_entry_count],['Blocked events',k.blocked_event_count],['Monitor events',k.monitor_event_count],['Gateway mode',cap(k.gateway_mode)]]; +async function loadKpis(){const k=await getJSON('/api/kpis');const h=await getJSON('/healthz'); + const t=[['Routes',k.route_count],['Threat indicators',k.threat_indicator_count],['DNSBL entries',k.dnsbl_entry_count],['Blocked events',k.blocked_event_count],['Monitor events',k.monitor_event_count],['Gateway mode',cap(k.gateway_mode)],['Event partitions',h.event_partitions??0]]; $('kpis').innerHTML=t.map(([l,v])=>'
'+esc(l)+'
'+esc(v)+'
').join('');} async function loadRoutes(){const d=await getJSON('/api/routes'); $('routesBody').innerHTML=table('Configured routes',['Path prefix','Upstream','Mode','State'],d.map(r=>[esc(r.path_prefix),esc(r.upstream),modeBadge(r.mode),stateBadge(r.enabled)]));} @@ -4471,6 +4477,10 @@ mod tests { html.contains("id=\"backupDrillBtn\""), "restore drill button missing" ); + assert!( + html.contains("Event partitions"), + "HASH event-partition KPI tile missing" + ); assert!( html.contains(":focus-visible"), "focus-visible styling missing" @@ -7598,6 +7608,7 @@ mod tests { outbox_dead_letter: 0, outbox_oldest_age_seconds: None, backup: "disabled".to_string(), + event_partitions: 0, } ); From 8c30322d791c4d8df7dd432ac7ca388fe88ff2b5 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 24 Aug 2026 04:02:04 +0900 Subject: [PATCH 2/2] docs: record PR #104 in the product-technical gap baseline --- docs/product-technical-gap-baseline.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index ec84f5fb..677fe74d 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -25,7 +25,7 @@ not “waiting on review/CI time”. | PR | Title | Head | Checks | Reviews | Merge blocker | | --- | --- | --- | --- | --- | --- | -| this pass | feat(store): HASH-partition security_event by tenant | `feat/issue-80-event-hash-partition` stacked on #103 | local fmt/test/clippy + two `scripts/smoke.sh` + live postgres `/healthz.event_partitions=8` | Author this pass | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 then #103 first. Do not `--admin`. Do not re-implement runtime role. | +| [#104](https://github.com/ContextualWisdomLab/wardnet/pull/104) | feat(store): HASH-partition security_event by tenant | `feat/issue-80-event-hash-partition` stacked on #103 | local fmt/test/clippy + two `scripts/smoke.sh` + live postgres `/healthz.event_partitions=8` | Author this pass | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 then #103 first. Do not `--admin`. Do not re-implement runtime role. | | [#103](https://github.com/ContextualWisdomLab/wardnet/pull/103) | feat(store): non-owner PostgreSQL runtime role after migrate | `feat/issue-80-runtime-role` stacked on #100 | still-valid Devin restore-window finding fixed this pass (`MIN_RESTORABLE_SCHEMA_VERSION=2`); local fmt/test/clippy + smokes | Author this pass; Devin COMMENTED (v3 backup voiding addressed) | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 first. Do not `--admin`. Do not re-implement rustls, outbox, retention, backup, or HASH. | | [#102](https://github.com/ContextualWisdomLab/wardnet/pull/102) | feat(store): logical backup and isolated restore drill | squash-merged into #100 (`321e792`) | prior hour | Author prior hour | Folded into rustls stack. Do not re-implement. | | [#101](https://github.com/ContextualWisdomLab/wardnet/pull/101) | feat(store): bound outbox listing and prune processed rows | `feat/issue-81-outbox-retention` (`0c2167a`) stacked on #100 | still-valid Devin prune-cap finding fixed this pass (`EVENT_LIMIT` on save/ack) | Author; Devin COMMENTED (prune thread addressed) | Org 2-approval + self-author. Merge #95 then #96 then #97 then #98 then #99 then #100 first. Do not `--admin`. | @@ -202,9 +202,9 @@ the runtime role. ## Next hourly loop (do, do not report) 1. Second independent APPROVE on #91/#92. Do not `--admin`. -2. Keep #94/#95/#96/#97/#98/#99/#100/#103 and this HASH PR merge-ready. +2. Keep #94/#95/#96/#97/#98/#99/#100/#103 and #104 merge-ready. Merge order #94 independently; #95 then #96 then #97 then #98 then #99 - then #100 then #103 then this. + then #100 then #103 then #104. 3. Next runtime gap if policy still blocks: extra #81 consumers (TAXII / Clearfolio / orchestrator) or optimistic concurrency. 4. Refresh this file’s PR/Issue tables from `gh pr list` / `gh issue list`.