test: verify concurrent policy writers and restore

.github/workflows/ci.yml

@@ -45,6 +45,22 @@ jobs:
           PGDATABASE: policyweave_test
           PGPASSWORD: policyweave_ci_password
         run: sh db/tests/policy_revision_runtime.sh
+      - name: Verify PostgreSQL concurrent policy writers
+        env:
+          PGHOST: 127.0.0.1
+          PGPORT: 5432
+          PGUSER: policyweave_ci
+          PGDATABASE: policyweave_test
+          PGPASSWORD: policyweave_ci_password
+        run: sh db/tests/policy_revision_concurrency.sh
+      - name: Verify PostgreSQL policy revision restore
+        env:
+          PGHOST: 127.0.0.1
+          PGPORT: 5432
+          PGUSER: policyweave_ci
+          PGDATABASE: policyweave_test
+          PGPASSWORD: policyweave_ci_password
+        run: sh db/tests/policy_revision_restore.sh
       - run: npx playwright install --with-deps chromium
       - run: npm run test:e2e
       - name: Upload exact-head browser evidence

.gitignore

@@ -5,3 +5,4 @@ coverage
 playwright-report
 test-results/
 *.tsbuildinfo
+.codegraph/

AGENTS.md

@@ -12,7 +12,7 @@ PolicyWeave is a local-first privacy-policy fact-authoring workspace. It structu
 - Do not encode legal conclusions from memory. Every legal/rule/template decision needs an authoritative source, effective date, source revision, and implementation/test trace.
 - Do not commit identifying customer, individual, or real operational-organization data in tests, examples, fixtures, or product documentation. Publicly documented legal authorities, official document titles, standards bodies, source publishers, and the repository owner may be named when required for accurate provenance and citation. Production must not consume synthetic demo data.
 - Keep persistence objects semantically named with at least two words and `snake_case` unless a framework contract requires another convention. Avoid generic named persistence objects such as a standalone `id` table/collection.
-- Hosted persistence/publication must be introduced only behind explicit tenant, authorization, audit, encryption, immutable revision, and supersession contracts.
+- Hosted persistence/publication must be introduced only behind explicit tenant, authorization, audit, encryption, immutable revision, and supersession contracts. CI PostgreSQL restart and dump/restore evidence lives in `db/tests/policy_revision_restore.sh` and does not enable a hosted adapter. Seed `retention_status = applies` only in the same transaction as its `retention_rule`; autocommit fails the deferred fact contract.
 - GitHub Actions dependencies stay SHA pinned and checkout credentials must not persist.
 
 ## Verification

ARCHITECTURE.md

@@ -35,10 +35,10 @@ The active MVP is a React/Vite browser workspace. State is in memory and there i
 
 Authoring completeness is deliberately separate from legal sufficiency. Current readiness rules prove that product-defined fact responsibilities were explicitly addressed; they do not assert that a policy complies with law. Source/effective-date-bound legal validation belongs to the Legal Source Registry -> Review & Publication boundary.
 
-## Persistence boundary (Proposed schema; no runtime)
+## Persistence boundary (Proposed schema; CI-only runtime)
 ADR-0003 and `db/migrations/0001_policy_revision.sql` propose the first PostgreSQL contract. The 3NF write model uses `policy_revision` as aggregate root; `service_profile`, `collection_item`, `processing_purpose`, and `retention_rule` are revision-owned facts. `(tenant_account_id, revision_number)` identifies a version, while `(policy_revision_id, collection_item_key)` is the item-level UPSERT/idempotency key. Deferred database constraints lock the owning revision row, reject collection items under explicit no-collection, and reject retention-rule/status contradictions at transaction commit.
 
-This source contract is not an active datastore, released API, or migration-execution result. Hosted tenant authorization, immutable audit events, encryption, concurrent-lock evidence, migration rollback, and backup/restore remain open. Publication remains append-only/immutable with explicit supersession; writes across unrelated aggregates must not share a transaction merely for convenience. Named database/schema/persistence objects use at least two semantic words and `snake_case`, for example `policy_revision`, `collection_item`, `processing_purpose`, `review_finding`, `publication_revision`, and `legal_source_revision`.
+Exact-head CI executes that migration against digest-pinned PostgreSQL 18, including rollback, two-session locks with NULL-safe complete-value assertions, process restart, and custom-format dump/restore of complete collection-item plus independent collection and retention facts. This is not an active datastore, released API, or hosted adapter. Hosted tenant authorization, immutable audit events, encryption, deletion, operational backup/restore, and production-scale contention remain open. Publication remains append-only/immutable with explicit supersession; writes across unrelated aggregates must not share a transaction merely for convenience. Named database/schema/persistence objects use at least two semantic words and `snake_case`, for example `policy_revision`, `collection_item`, `processing_purpose`, `review_finding`, `publication_revision`, and `legal_source_revision`.
 
 Separate write-side draft commands from read-side rendered/review projections once hosted traffic justifies it. Account for revision hot spots and optimistic/constrained writes before adding collaborative editing. Keep source/customer integrations behind ACLs; do not form a shared kernel with unrelated ContextualWisdomLab products without demonstrated reuse.
 

CHANGELOG.md

@@ -5,6 +5,8 @@ All notable product changes are recorded here. PolicyWeave is pre-release; entri
 ## Unreleased
 
 ### Added
+- PostgreSQL restart and custom-format dump/restore evidence that preserves NULL-safe complete service/collection-item values, a collecting-without-retention cross-state fixture, and independent no-collection and applies-retention facts, then re-executes no-collection plus both retention-status/rule contradictions against the restored database.
+- PostgreSQL two-session concurrency evidence that observes real lock waits, rejects a collection-item writer racing with a no-collection update, and proves competing same-item UPSERTs converge to one row carrying the second writer's label, mode, and path with NULL-safe complete-value assertions and without timing-based transaction sleeps.
 - PostgreSQL 18 runtime contract coverage for migration apply/down/apply cycles, item-key UPSERT idempotency, and deferred rejection of no-collection, missing-retention-rule, and revision-owner contradictions. The database remains CI-only and is not a hosted product backend.
 - Proposed PostgreSQL `policy_revision` 3NF migration contract with tenant-scoped version identity, normalized revision-owned facts, owner-key immutability, parent-row-serialized deferred no-collection/retention consistency checks, and item-level natural-key UPSERT. It is source-validated only and does not claim a deployed database or hosted persistence.
 - Seven-step authoring workflow with distinct editing surfaces for service information, collection items, processing purposes, retention, third-party transfer, international transfer, and privacy contact.
@@ -53,6 +55,6 @@ All notable product changes are recorded here. PolicyWeave is pre-release; entri
 ### Not yet shipped
 - Authoritative legal-rule snapshots that can determine legal sufficiency beyond product-defined fact completeness.
 - Manual zoom and screen-reader evidence beyond the automated desktop/tablet/mobile Chromium accessibility and focus checks.
-- A product persistence adapter, durable hosted storage, tenant authorization, immutable audit history, concurrent-writer scheduling, restart, encryption, and backup/restore evidence. CI migration and rollback execution do not constitute a hosted runtime.
+- A product persistence adapter, durable hosted storage, tenant authorization, immutable audit history, encryption, operational backup/restore, and production-scale contention evidence. Bounded CI database execution, including process restart and dump/restore, does not constitute a hosted runtime.
 - Authenticated immutable publication revisions and public URL lifecycle.
 - Hosted tenant/security/operability evidence and endpoint load testing.

CLAUDE.md

@@ -5,3 +5,5 @@ Follow `AGENTS.md` as the repository-wide operating contract.
 Before changing code, read `docs/PRD.md`, `ARCHITECTURE.md`, the applicable ADRs, `docs/TRD.md`, `docs/research-traceability.md`, and `docs/product-technical-gap-baseline.md`. Preserve PolicyWeave's boundary: verified operator facts in, deterministic reviewable policy projection out; no legal-advice or compliance-guarantee claims.
 
 For behavior changes, add or strengthen tests before production code. Reconcile documentation and the product-gap ledger on the same branch. Use the exact current PR head for reviews, checks, and merge decisions, and never bypass governance to compensate for a failing or unassigned check.
+
+Deferred `policy_revision` fact triggers evaluate the final commit state. CI restore seeds `retention_status = applies` and `retention_rule` in one transaction; the owner runbook is `db/tests/policy_revision_restore.sh`.

db/tests/policy_revision_concurrency.sh

@@ -0,0 +1,195 @@
+#!/bin/sh
+set -eu
+
+: "${PGHOST:=127.0.0.1}"
+: "${PGPORT:=5432}"
+: "${PGUSER:=policyweave_ci}"
+: "${PGDATABASE:=policyweave_test}"
+: "${PGPASSWORD:=policyweave_ci_password}"
+export PGHOST PGPORT PGUSER PGDATABASE PGPASSWORD
+
+test_run_id=$$
+parent_release_fifo="/tmp/policyweave_parent_release_${test_run_id}"
+parent_ready_file="/tmp/policyweave_parent_ready_${test_run_id}"
+parent_writer_log="/tmp/policyweave_parent_writer_${test_run_id}.log"
+item_writer_log="/tmp/policyweave_item_writer_${test_run_id}.log"
+upsert_release_fifo="/tmp/policyweave_upsert_release_${test_run_id}"
+upsert_ready_file="/tmp/policyweave_upsert_ready_${test_run_id}"
+first_upsert_log="/tmp/policyweave_first_upsert_${test_run_id}.log"
+second_upsert_log="/tmp/policyweave_second_upsert_${test_run_id}.log"
+parent_writer_process_id=
+item_writer_process_id=
+first_upsert_process_id=
+second_upsert_process_id=
+
+psql_command() {
+  psql --no-psqlrc --set ON_ERROR_STOP=1 "$@"
+}
+
+cleanup_concurrency_test() {
+  for writer_process_id in \
+    ${parent_writer_process_id:-} \
+    ${item_writer_process_id:-} \
+    ${first_upsert_process_id:-} \
+    ${second_upsert_process_id:-}; do
+    kill "$writer_process_id" 2>/dev/null || true
+    wait "$writer_process_id" 2>/dev/null || true
+  done
+  rm -f \
+    "$parent_release_fifo" "$parent_ready_file" "$parent_writer_log" "$item_writer_log" \
+    "$upsert_release_fifo" "$upsert_ready_file" "$first_upsert_log" "$second_upsert_log"
+}
+
+wait_for_file() {
+  expected_file=$1
+  wait_name=$2
+  attempt_count=0
+  while [ ! -f "$expected_file" ]; do
+    attempt_count=$((attempt_count + 1))
+    if [ "$attempt_count" -ge 200 ]; then
+      echo "timed out waiting for ${wait_name}" >&2
+      return 1
+    fi
+    sleep 0.05
+  done
+}
+
+wait_for_lock() {
+  writer_application_name=$1
+  writer_log=$2
+  attempt_count=0
+  while :; do
+    waiting_count=$(psql_command --tuples-only --no-align --command \
+      "select count(*) from pg_stat_activity where application_name = '${writer_application_name}' and wait_event_type = 'Lock'")
+    if [ "$waiting_count" -ge 1 ]; then
+      return 0
+    fi
+    attempt_count=$((attempt_count + 1))
+    if [ "$attempt_count" -ge 200 ]; then
+      cat "$writer_log" >&2 || true
+      echo "timed out waiting for PostgreSQL lock: ${writer_application_name}" >&2
+      return 1
+    fi
+    sleep 0.05
+  done
+}
+
+trap cleanup_concurrency_test 0 1 2 15
+
+mkfifo "$parent_release_fifo" "$upsert_release_fifo"
+psql_command --file db/migrations/0001_policy_revision.sql
+
+psql_command <<'SQL'
+insert into policy_revision (policy_revision_id, tenant_account_id, revision_number)
+values ('30000000-0000-4000-8000-000000000001', '40000000-0000-4000-8000-000000000001', 1);
+SQL
+
+PGAPPNAME=policyweave_parent_writer PGOPTIONS='-c statement_timeout=15000' \
+  psql --no-psqlrc --set ON_ERROR_STOP=1 >"$parent_writer_log" 2>&1 <<SQL &
+begin;
+update policy_revision
+   set no_collection_confirmed = true
+ where policy_revision_id = '30000000-0000-4000-8000-000000000001';
+\! touch "$parent_ready_file"
+\! sh -c 'read release_signal < "$parent_release_fifo"'
+commit;
+SQL
+parent_writer_process_id=$!
+wait_for_file "$parent_ready_file" policyweave_parent_writer
+
+PGAPPNAME=policyweave_item_writer PGOPTIONS='-c statement_timeout=15000' \
+  psql --no-psqlrc --set ON_ERROR_STOP=1 >"$item_writer_log" 2>&1 <<'SQL' &
+begin;
+insert into collection_item (policy_revision_id, collection_item_key, collection_item_label)
+values ('30000000-0000-4000-8000-000000000001', 'contact_email', 'Contact email');
+commit;
+SQL
+item_writer_process_id=$!
+wait_for_lock policyweave_item_writer "$item_writer_log"
+printf 'release\n' >"$parent_release_fifo"
+wait "$parent_writer_process_id"
+parent_writer_process_id=
+
+set +e
+wait "$item_writer_process_id"
+item_writer_status=$?
+set -e
+item_writer_process_id=
+if [ "$item_writer_status" -eq 0 ]; then
+  echo 'competing collection item unexpectedly committed' >&2
+  exit 1
+fi
+grep -F -- 'no-collection confirmation conflicts with collection items' "$item_writer_log"
+
+psql_command <<'SQL'
+insert into policy_revision (policy_revision_id, tenant_account_id, revision_number)
+values ('30000000-0000-4000-8000-000000000002', '40000000-0000-4000-8000-000000000001', 2);
+SQL
+
+PGAPPNAME=policyweave_first_upsert_writer PGOPTIONS='-c statement_timeout=15000' \
+  psql --no-psqlrc --set ON_ERROR_STOP=1 >"$first_upsert_log" 2>&1 <<SQL &
+begin;
+select upsert_collection_item(
+  '30000000-0000-4000-8000-000000000002',
+  'contact_email',
+  'Initial contact email',
+  'required',
+  'Account registration form'
+);
+\! touch "$upsert_ready_file"
+\! sh -c 'read release_signal < "$upsert_release_fifo"'
+commit;
+SQL
+first_upsert_process_id=$!
+wait_for_file "$upsert_ready_file" policyweave_first_upsert_writer
+
+PGAPPNAME=policyweave_second_upsert_writer PGOPTIONS='-c statement_timeout=15000' \
+  psql --no-psqlrc --set ON_ERROR_STOP=1 >"$second_upsert_log" 2>&1 <<'SQL' &
+begin;
+select upsert_collection_item(
+  '30000000-0000-4000-8000-000000000002',
+  'contact_email',
+  'Concurrent contact email',
+  'optional',
+  'Account profile form'
+);
+commit;
+SQL
+second_upsert_process_id=$!
+wait_for_lock policyweave_second_upsert_writer "$second_upsert_log"
+printf 'release\n' >"$upsert_release_fifo"
+wait "$first_upsert_process_id"
+first_upsert_process_id=
+wait "$second_upsert_process_id"
+second_upsert_process_id=
+
+psql_command <<'SQL'
+do $concurrency_assertion$
+declare
+  stored_count integer;
+  stored_label text;
+  stored_mode text;
+  stored_path text;
+begin
+  select count(*),
+         max(collection_item_label),
+         max(collection_mode::text),
+         max(collection_path)
+    into stored_count, stored_label, stored_mode, stored_path
+    from collection_item
+   where policy_revision_id = '30000000-0000-4000-8000-000000000002'
+     and collection_item_key = 'contact_email';
+
+  if stored_count <> 1
+     or stored_label <> 'Concurrent contact email'
+     or stored_mode is distinct from 'optional'
+     or stored_path is distinct from 'Account profile form' then
+    raise exception 'concurrent collection-item UPSERTs did not converge';
+  end if;
+end;
+$concurrency_assertion$;
+SQL
+
+psql_command --file db/migrations/0001_policy_revision.down.sql
+cleanup_concurrency_test
+trap - 0 1 2 15