Phase 007E: financial durability research and cross-process verification

.github/workflows/research-ci.yml

@@ -0,0 +1,24 @@
+name: Research Conformance CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  research:
+    name: Independent research oracles and documentation contracts
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+      - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        with:
+          version: "0.13.6"
+      - name: Install Python 3.12
+        run: uv python install 3.12
+      - name: Run reproducible research verification
+        run: uv run --no-project --python 3.12 --with pytest pytest -v

DEBATES.md

@@ -1141,3 +1141,7 @@ and real-world prediction accuracy were not tested.
 ## Debate 32: Complete financial provenance versus implicit World causation (Phase 007D)
 
 **Thesis A:** force all expenses and receipts into World event traces. Rejected: an external authorized payment is not a World transition, and fabricating events destroys ownership truth. **Selected bounded A+B:** event sales must have verified actual execution witnesses, but externally authored advertising, other costs, settlements and writeoffs carry canonical author-asserted actor/authorization envelopes. The domain ledger remains the only money implementation; duplicate delivery is idempotent within supplied operations, conflicting identities reject. **Alternative C:** universal event scheduling, shared budget arbiter, persistent authenticated audit store; rejected absent experimental justification and approval. Distinct physical/real-world permission requires verification outside the model. Shared pool arbitration breaks agent independence and remains a separate STOP boundary. No Accepted ADR.
+
+## Phase 007E — Durable audit: integrity versus producer authority (2026-11-01)
+
+Thesis: canonical SHA-256 evidence supports deterministic, portable post-restart integrity checks. Antithesis: a writable data file and its unkeyed digest may be forged together; asserted authorization references are not cryptographic permissions. Resolution: implement narrowly scoped single-writer checkpoint export/import bound to re-executed World evidence, optionally check an externally anchored digest, and retain authentication/concurrency as explicit stops. No new ledger or distributed coordinator.

DECISIONS.md

@@ -547,3 +547,7 @@ On 2026-11-01, a bounded, explicitly opt-in Python `PopulationExecutor` adapter
 ### Phase 007D financial provenance checkpoint — proposed, NOT an Accepted ADR
 
 2026-11-01: Domain-owned `DeclaredFinancialAction` and `post_authorized_trial()` separate verified World-caused sales from author-asserted external paid actions, validate local operation identity and delegate exact cash/receivable conservation to `MarketingLedger`. Existing Phase 007C unprovenanced adapter remains available without being labeled authenticated. Under DEC-009, no economic rules migrated into compiler or World scheduler. Cryptographic authenticity, persistent replay, shared cash arbitration and empirical calibration require new explicit approval and are not implied. Evidence: `spec/financial-consequence-provenance-v0.md`, `evals/financial-consequence-provenance-v0.md`.
+
+## Phase 007E — Proposed, not Accepted ADR (2026-11-01)
+
+Bounded implementation record only: opt-in canonical financial audit v1 with strict trace-and-policy restoration and atomic single-writer checkpoint. No change to existing accepted DEC IDs or new Accepted ADR. Producer authentication, actual authorization and shared-state serializability remain undecided pending operator permission and separate threat-model evidence. See `spec/durable-financial-provenance-v1.md`.

OUT_OF_THE_BOX.md

@@ -539,3 +539,7 @@ The narrow Phase 007C **in-process** event-to-sale witness exists and is tested:
 ### Phase 007D frontier boundary — durable attestations and shared-resource serializability
 
 Verified source-to-sale World witnesses and explicit author-asserted external transaction envelopes still cannot authenticate an external principal, persist evidence across untrusted storage, or arbitrate a cross-agent shared budget. A possible *future* research program would define a threat model, signer trust root, durable delivery protocol, transaction scope and serializable shared resource scheduler, then falsify them via replay, reordering and crash recovery. None is implemented or authorized in 007D; do not confuse deterministic SHA-256 identifiers with an authenticated signature.
+
+## Phase 007E research frontier (not implemented)
+
+Potential future externally anchored digest catalogs, HMAC with operator-owned key lifecycle, append-only authenticated logs and stateful concurrency arbitration require independent operators/requirements. Hash chaining without an external trust root cannot turn mutable records into authenticated evidence. No such capabilities are claimed or authorized in this phase.

evals/durable-financial-provenance-measurements.json

@@ -0,0 +1,130 @@
+{
+  "python": "3.12.13 (main, Jul 18 2026, 16:55:18) [Clang 22.1.3 ]",
+  "os": "macOS-27.0.1-arm64-arm-64bit",
+  "source_sha256": "c64f1caaf28330152c4bc06c44f6a98b8041a8009f5dff7a8f76bed403b3bf46",
+  "provenance_version": "world-event-sale-v1",
+  "audit_schema": "funnelscript-financial-audit-v1",
+  "memory_kind": "parent tracemalloc traced allocations only",
+  "runs": [
+    {
+      "population": 10,
+      "segments": 2,
+      "seed": 11,
+      "population_seconds": 0.00337,
+      "financial_seconds": 0.001062,
+      "serialization_seconds": 0.001724,
+      "persistence_and_process_seconds": 0.23444,
+      "restoration_seconds_child": 0.001418,
+      "replay_seconds_child": 0.001045,
+      "total_seconds": 0.24117,
+      "tracemalloc_peak_bytes_parent": 126422,
+      "sales": 3,
+      "actions": 6,
+      "entries": 9,
+      "audit_sha256": "01d52cd9f847a78ed08a0b04304fb60d25b7611f224ae6676dbad4b12557458a",
+      "audit_bytes": 5565,
+      "cash": "105.00",
+      "replay_agrees": true
+    },
+    {
+      "population": 10,
+      "segments": 2,
+      "seed": 29,
+      "population_seconds": 0.003767,
+      "financial_seconds": 0.001278,
+      "serialization_seconds": 0.002171,
+      "persistence_and_process_seconds": 0.262823,
+      "restoration_seconds_child": 0.001012,
+      "replay_seconds_child": 0.000759,
+      "total_seconds": 0.270664,
+      "tracemalloc_peak_bytes_parent": 109771,
+      "sales": 3,
+      "actions": 6,
+      "entries": 9,
+      "audit_sha256": "fa261c007f4611fede17a99ec5864a4b57c42514cf7c0b0edd0795a9ca2f9664",
+      "audit_bytes": 5565,
+      "cash": "105.00",
+      "replay_agrees": true
+    },
+    {
+      "population": 100,
+      "segments": 2,
+      "seed": 11,
+      "population_seconds": 0.010621,
+      "financial_seconds": 0.006275,
+      "serialization_seconds": 0.011339,
+      "persistence_and_process_seconds": 0.238537,
+      "restoration_seconds_child": 0.004667,
+      "replay_seconds_child": 0.004675,
+      "total_seconds": 0.268821,
+      "tracemalloc_peak_bytes_parent": 401229,
+      "sales": 30,
+      "actions": 33,
+      "entries": 63,
+      "audit_sha256": "8bdaac8d169e813054303bd1a462105f4ea2b3a696d4105b9f1719d898a87e7c",
+      "audit_bytes": 42128,
+      "cash": "1050.00",
+      "replay_agrees": true
+    },
+    {
+      "population": 100,
+      "segments": 2,
+      "seed": 29,
+      "population_seconds": 0.011236,
+      "financial_seconds": 0.006783,
+      "serialization_seconds": 0.010436,
+      "persistence_and_process_seconds": 0.237354,
+      "restoration_seconds_child": 0.004795,
+      "replay_seconds_child": 0.004554,
+      "total_seconds": 0.267607,
+      "tracemalloc_peak_bytes_parent": 385125,
+      "sales": 30,
+      "actions": 33,
+      "entries": 63,
+      "audit_sha256": "fd1ba43b2d53c986f3609dde46120bf7d48affa465d593b13df2e27f2793613c",
+      "audit_bytes": 42128,
+      "cash": "1050.00",
+      "replay_agrees": true
+    },
+    {
+      "population": 1000,
+      "segments": 2,
+      "seed": 11,
+      "population_seconds": 0.084648,
+      "financial_seconds": 0.058016,
+      "serialization_seconds": 0.103763,
+      "persistence_and_process_seconds": 0.399554,
+      "restoration_seconds_child": 0.039629,
+      "replay_seconds_child": 0.041334,
+      "total_seconds": 0.663246,
+      "tracemalloc_peak_bytes_parent": 3946148,
+      "sales": 300,
+      "actions": 303,
+      "entries": 603,
+      "audit_sha256": "8dc0984352f736df426e12a250682c1e5c4f2f41f1344fb9316604444bca3f50",
+      "audit_bytes": 407713,
+      "cash": "10500.00",
+      "replay_agrees": true
+    },
+    {
+      "population": 1000,
+      "segments": 2,
+      "seed": 29,
+      "population_seconds": 0.113813,
+      "financial_seconds": 0.055502,
+      "serialization_seconds": 0.106322,
+      "persistence_and_process_seconds": 0.405889,
+      "restoration_seconds_child": 0.040259,
+      "replay_seconds_child": 0.04518,
+      "total_seconds": 0.695973,
+      "tracemalloc_peak_bytes_parent": 3825188,
+      "sales": 300,
+      "actions": 303,
+      "entries": 603,
+      "audit_sha256": "67a701966bab42d3490b8a0b47efe9f6081c475140a2aded1019ec606569d885",
+      "audit_bytes": 407713,
+      "cash": "10500.00",
+      "replay_agrees": true
+    }
+  ]
+}

evals/durable-financial-provenance-v1.md

@@ -0,0 +1,9 @@
+# Phase 007E — Executed independent evidence
+
+Reference scenario is hand-calculated in `spec/durable-financial-provenance-v1.md`. The experimental observations are preserved as raw JSON in `evals/durable-financial-provenance-measurements.json` (do not overwrite the Phase 007D receipt). Test the receipt with `uv run pytest -q tests/test_durable_financial_reference.py`; application tests live in `tests/test_durable_financial_audit.py` of `funnelscript`.
+
+Distinct Python subprocesses re-execute an actual World population, restore the audit from a single-writer checkpoint with an out-of-band hash anchor, replay every previously persisted action and compare canonical bytes/ledger snapshots. Two seeds each for cohorts 10, 100, and 1,000; conversion count is 30% by declared event fixture, not calibrated field conversion data. Timings distinguish parent population/integration/serialization and child restore/replay. Parent traced memory excludes subprocess allocations. Process startup and publishing overhead is reported separately. Do not compare against Phase 007D as a performance regression proof without matched conditions.
+
+Failure assumptions: honest original World execution witness, single writer, supported local filesystem atomic replacement, non-compromised independent anchor when supplied, complete checkpoint before acknowledgement. Altered records with recomputed hashes cannot be called authenticated. Shared budget/concurrent writes/physical delivery exactly-once and external authorization are explicitly unproven.
+
+Final schema/versioned ownership experiment receipt (2026-11-01): six trials run in fresh child processes; 10-agent runs 9 entries and 3 sales; 100-agent runs 63 entries and 30 sales; 1,000-agent runs 603 entries and 300 sales. All six persisted digests match child-process re-export and all 6 replays preserve cash. The child restore durations ranged from approximately 0.001 s to 0.040 s under this workstation's environment; figures are raw individual measurements, not benchmarks or extrapolation. Exact observations and source/environment fingerprints are in the adjacent JSON.

spec/durable-financial-provenance-v1.md

@@ -0,0 +1,41 @@
+# Phase 007E — Durable Financial Audit and Recovery Contract (proposed)
+
+Status: research-gated experimental opt-in domain contract, not an Accepted ADR. Anchors: discrete-event execution trace replay (Banks et al., *Discrete-Event System Simulation*), Fowler's idempotent message/transaction identity patterns (*Patterns of Enterprise Application Architecture*), NIST FIPS 180-4 SHA-256 integrity, POSIX rename and fsync semantics, and existing Phase 007B–D exact ledger conservation.
+
+## Falsifiable contracts
+
+- H-DPROV1: a validated trial and its complete sale/action witness produce identical domain-separated, schema-versioned canonical UTF-8 bytes regardless of input action order, hash randomization or process. Falsifier: same validated financial trial exports different bytes.
+- H-DPROV2: after a strict restoration using the original World trace and declared policy, replaying any previously committed action changes neither transaction count nor any accounting balance; conflicting tx/authorization reuse fails closed. Falsifier: a replay increases settled cash.
+- H-DPROV3: records must bind all source identifiers and financial details to a validated SHA-256 manifest, and optionally an independently held expected full-envelope digest. Falsifier: altered bytes are accepted without detection under the stated trusted digest assumption. SHA-256 alone is integrity, not a producer signature.
+- H-DPROV4: checkpoint publication via temporary file plus file sync, atomic same-filesystem rename and directory sync leaves either the old committed checkpoint or the new one on ordinary pre-rename failure. Falsifier: interrupted uncommitted write becomes a valid posting. No claim against compromised filesystem, malicious concurrent writer, or power-loss semantics without platform tests.
+- H-DPROV5: restored exact existing MarketingLedger yields cash = initial + receipts - ads - other paid cost; receivables = recognized - settled - writeoffs; profit = recognized - ads - costs - writeoffs. Falsifier: any recovery discrepancy against hand-calculated 10-agent oracle.
+- H-DPROV6: canonical audit bytes and IDs must agree across independent Python processes with differing PYTHONHASHSEED. Falsifier: mismatch.
+- H-DPROV7: single writer and independently replayed trial only; concurrent writers, shared budgets and global serializability are unsupported. Falsifier of any future claimed multiwriter safety: adversarial lost update.
+
+## Architecture alternatives and decision boundary
+
+A. Canonical export and strict trace-bound import: needed for portable exact replay, but alone cannot publish committed state.
+B. Add smallest single-file atomic checkpoint writer/reader, without a database or transaction engine: selected for the bounded single-writer file checkpoint. Persistent replay must explicitly carry forward the complete previously validated journal, not trust a process-local set.
+C. Message broker/transaction coordinator/distributed lock/event store: rejected absent failing concurrency requirements and operator authorization.
+
+## Trust and crash boundaries
+
+The caller owns the execution trace, policy and financial authorizations and must retain them for verified replay. A saved payload is not an independent proof that these authorities were legitimate. External manifest anchors are optional and must be stored outside the same writable trust domain. Without such an anchor, an attacker able to rewrite both content and checksum can forge otherwise schema-valid external financial actions. No source authentication, revocation check or real-world causal identification is asserted. Single-writer, same-filesystem atomic rename and no untrusted path are preconditions. Crash before rename retains previous committed image; after rename may yield the new committed image. The caller must explicitly handle ambiguous outcomes by rereading and replaying, never assuming exactly-once physical delivery. Recovered data is revalidated against source World events and the exact existing ledger.
+
+## Independent oracle
+
+10 agents (4/6), three World conversion sales of SAR 20, initial cash SAR 100, ads SAR 20, other paid costs SAR 5, three day-3 SAR 10 payments: day 1 cash 75 / receivables 60 / recognized 60 / receipts 0; day 3 cash 105 / receivables 30 / recognized 60 / receipts 30 / profit 35. Day-3 SAR 5 writeoff yields cash 105 / receivables 25 / recognized 60 / receipts 30 / profit 30. Calculated independently before implementation.
+
+## Executed proof boundaries and results (2026-11-01)
+
+Implemented in the domain-only `funnelscript.stdlib.marketing.audit` adapter with `export_audit`, `restore_audit`, `replay_audit`, `write_audit_checkpoint`, `read_audit_checkpoint`. Format `funnelscript-financial-audit-v1` has canonical sorted-key JSON, integer minor units, full trial metadata, event-to-sale provenance and source references, separately author-asserted external actor/authorization, ordered financial journal, SHA-256 digest with explicit domain-separation prefix `funnelscript:financial-audit:v1\0`, plus an optional independently pinned SHA-256 digest of the complete envelope. `restore_audit` requires an independently reproduced original World execution and sale policy and reproduces posting through the existing ledger. No key management or signed producer identity is implemented.
+
+An integrity-only manifest can be rewritten along with its payload; the independent pinned envelope hash detects such a rewrite only while the external anchor is trusted. The protocol does not authenticate external authorizations or prove external financial source transactions. Reader + restore must always run before a checkpoint is treated as a financial fact. Atomic file replacement does not provide global serializability; no automatic append, background queue or shared writer arbitration exists. The authoritative single-writer application must save the *new* complete audited result before acknowledging a financial operation. Receipt ambiguity after rename is resolved by rereading, not blind retry.
+
+**RED**: `uv run pytest -q tests/test_durable_financial_audit.py::test_phase007e_public_contract_is_available` failed (1 failed, exit 1), missing required public audit contract; not import/setup failure. **GREEN**: focused Phase 007E suite 26 passed (exit 0), after correcting test setup to provide an actual `Sequence` instead of a `reversed` iterator. This correction preserved the behavioral assertion and did not change production semantics. The Phase 007D oracle and cross-process separately seeded experiment are independently checked in `evals/durable-financial-provenance-measurements.json` and `tests/test_durable_financial_reference.py`.
+
+Multiwriter financial operations, shared resource arbitration, producer authentication and operational authorization remain **ARCHITECTURAL STOP — APPROVAL REQUIRED**, not phase-completion claims. Deletion, swapping or substitution of persisted financial records fails under the pinned complete-envelope digest; without the pin, a fully rewritten consistent journal can pass integrity and actor-assertion validation. Python/OS measurement values are environment-specific, not scaling forecasts.
+
+## GitHub verification remediation
+
+GitHub PR #3 initially had **no CI checks** because this repository had no workflow. Phase 007E adds the minimal independent `Research Conformance CI` workflow, pinned to the existing application repository's verified checkout/setup-uv actions and uv version. It runs the research pytest suite with an explicitly declared Python 3.12 interpreter and ephemeral pytest dependency. It does not install a database, runtime package, signing infrastructure, or application engine. No claim of CI success until GitHub reports the new check terminal-success.