Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/ci-theorem.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: Theorem Lane

# Machine-checked cache-stability theorems (#2149). Runs Z3 over proof/spec
# plus the grounding tests in tests/cache-theorem.test.ts. Ubuntu-only: the
# pinned z3-solver wheel targets linux x86_64; windows legs of ci.yml run the
# same test file with solver subtests skipped by design.
on:
pull_request:
branches: [master]
paths:
- "proof/**"
- "tests/cache-theorem.test.ts"
- "src/loop/**"
- "src/wire-body.ts"
- "src/system-anchor.ts"
- "src/chain-checkpoint.ts"
- "src/image-note.ts"
- "src/server.ts"
- "src/session.ts"
- "src/fake-completion.ts"
- "src/degenerate-retry.ts"
- "kernel/src/rebuild.ts"
- "kernel/src/render-refs.ts"
- "kernel/src/refs.ts"
- "kernel/src/segment.ts"
- "kernel/src/block-map.ts"
- "kernel/src/ccr.ts"
- ".github/workflows/ci-theorem.yml"
push:
branches: [master]
paths:
- "proof/**"
- "tests/cache-theorem.test.ts"
- "src/loop/**"
- "src/wire-body.ts"
- "src/system-anchor.ts"
- "src/chain-checkpoint.ts"
- "src/image-note.ts"
- "src/server.ts"
- "src/session.ts"
- "src/fake-completion.ts"
- "src/degenerate-retry.ts"
- "kernel/src/rebuild.ts"
- "kernel/src/render-refs.ts"
- "kernel/src/refs.ts"
- "kernel/src/segment.ts"
- "kernel/src/block-map.ts"
- "kernel/src/ccr.ts"
- ".github/workflows/ci-theorem.yml"

jobs:
theorem:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
# pin the solver so local iteration and CI verdicts match
- name: Install pinned z3
run: python3 -m pip install --quiet "z3-solver==4.9.1.0"
- name: Install dependencies
run: npm install --ignore-scripts --no-fund --no-audit
- name: Run theorem lane
run: node --import tsx --test tests/cache-theorem.test.ts
137 changes: 137 additions & 0 deletions proof/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Cache-Stability Theorems (#2149)

Machine-checked statements about the proxy's outbound-body assembly, proven in
Z3 over the byte-level model in `spec/*.smt2`. This is the **route-4 (model
specification)** lane of #2149: the theorems are proven over the *contract*,
and two mechanisms keep the contract honest against the code:

1. **Grounding tests** (`tests/cache-theorem.test.ts`, `[grounding:*]`): each
spec axiom is pinned to the real code it abstracts (below).
2. **The #2144 billing suite** (`tests/cache-proof.test.ts`): real proxy + real
kernel vs an ideal prefix-cache judge, byte-LCP assertions across six path
classes — the code↔spec sync evidence at whole-body granularity.

## Trust boundary (read this first)

A theorem here says: **any assembly function whose components satisfy the spec
axioms has the stated byte property.** It does NOT by itself say "the shipped
TS code satisfies the axioms" — that direction is covered by (1)+(2), not by
the solver. This is the honest strength of route 4: cheapest cost, guarantee
bounded to the spec surface. Routes 1–3 (compiling the real pipeline into a
symex front-end) would remove the grounding gap entirely and are tracked as a
separate issue after this lane lands.

## Theorems

### T1 — growth stability (`t1-s1*`, `t1-s2*`)

For history `H`, message `m`, pipeline `f`: `f(H+[m])` and `f(H)` share a
common byte prefix of depth at least `D`, where `D` = envelope head + stable
content of the merge-anchor element. The statement is encoded with an explicit
shared-prefix term `P` (`blen(P) = D`) as `substr(body, 0, blen(P)) = P` for
both bodies -- a decidable strengthening of the quantified "exists p >= D"
form, which Z3's string solver returns `unknown` on. Everything after `D` may differ only at
the existentially quantified declared carrier positions:

- **MERGE family** — nudge/hint text appended to the last-user turn's content
tail (`src/wire-body.ts appendTrailingUserText`; degenerate-retry;
fake-completion). Two scenarios are instantiated: merge anchor *flips* to the
new message (S1) and stays stationary (S2).
- **TAIL family** — trailing user element (imgNote conditional / chain
checkpoint stamp / fresh-turn fallback): optional presence + free content per
run; the checkpoint stamp carries the clock (`stampOutbound(nowMs)`) — the
x-at-interface impurity, modeled as a free function of a free integer
instantiated differently per run.

All other bytes (envelope, old elements, ref tags, separators, framing) are
shared subterms in both bodies ⇒ equal by congruence. Impure points (network,
session store, clock) are modeled as free functions/free inputs at the
interface per the owner stance — no pure extraction was needed; the surveyed
rebuild core has zero impure calls inside `kernel/src/rebuild.ts`.

### T2 — ref monotonicity / id-never-reused (`t2*`)

Bounded 4-step session over the real allocator rules
(`kernel/src/refs.ts assignRefs/allocateFreeRef`; archive op = #421
`applyCompactionArchive`): initial live map `{a→5, b→2, c→3}`, cursor 6; the
session HIGH (number 5) dies; archive prunes; two appends allocate.

- `t2-host.smt2` — **UNSAT**: with the persisted monotone cursor (grounded in
`src/session.ts`: `state.nextIndex` carried verbatim; `cursor := n+1`;
retire/archive never move it), no allocation reissues an ever-issued number,
even though the high died. This is the machine-checked form of the host duty
behind the kernel id-never-reused contract.
- `t2-mutation.smt2` — **SAT**: if the cursor is instead RECOMPUTED from the
pruned map after the archive (`highestUsedIndex+1`, the documented #421
residual), the solver produces the counterexample `n4 = 5` — the dead high
number reissued. The residual window is thereby characterized exactly: it
opens iff the cursor drops below a dead ever-issued number.

### Mutation discipline (acceptance criterion: proofs are not vacuous)

Each theorem has a mutation twin that injects a stable-region defect (a
same-length content rewrite strictly below `D` in T1 -- which byte changed is
existentially hidden at this abstraction level; the concrete byte surface is
#2144's job -- and the post-archive cursor reset in T2). The solver MUST find
each as a counterexample within the time budget; the T2 model exhibits exactly
`hui=3 creset=4 n3=4 n4=5` (the dead high reissued). A proof that cannot be
falsified by its own known-bad mutation is treated as broken, not merged.

T1 mutations are stated single-side (`substr(corruptedBody, 0, blen(P)) != P`):
sound because the uncorrupted side satisfies the property by its theorem file,
so one conjunct's failure IS the pair property's failure.

## Spec → code grounding map

| Spec axiom | Grounded by |
|---|---|
| `esc` E0/E1/E2/E3 (escaping algebra) | `[grounding:g3]` — homomorphism + injectivity of the inner JSON string encoder on 200 randomized BMP strings |
| `mktag` determinism/injectivity | `[grounding:g2]` — `indexToRef`/`refToIndex` round-trip + injectivity over 200k numbers |
| T2 allocation rules (floor, smallest-free-over-live, existing-untouched, BLOCKED-free, `cursor := n+1`) | `[grounding:g1]` — `assignRefs`/`allocateFreeRef`/`highestUsedIndex` on constructed maps, incl. the gap-below-cursor case |
| envelope/framing bytes (`hdr`/`ftr`/`fpre`/`fmid`/`fpost`/`SEP`) | `tests/cache-proof.test.ts` (#2144) — byte-LCP unexplainedDivergences=0 across four wires and six path classes |
| carrier anchor placement (merge-last-user / tail-append, optionality) | #2144 scenario matrix + call-site survey recorded in `spec/t1-*.smt2` headers |

## Running

```bash
# needs z3: CLI on PATH, or python3 with the z3 module (pip install z3-solver==4.9.1.0)
node --import tsx --test tests/cache-theorem.test.ts
```

Nine solver obligations (two context-sanity SAT guards per scenario family,
which catch vacuous proofs if the context ever becomes inconsistent; two
growth-stability UNSAT theorems; two T1 mutations; one T2 theorem; one T2
mutation) plus three grounding test groups. Without a solver installed, solver
subtests skip individually and grounding still runs, so the main suite stays
green; CI runs the `.github/workflows/ci-theorem.yml` lane (ubuntu, pinned
`z3-solver==4.9.1.0`, paths-filtered to the assembly surface) so every relevant
PR gets the verdicts. All nine resolve in well under a second total.

Solver compatibility notes (learned the hard way):

- Bytes are modeled as the SMT-LIB **String** theory (`str.++`/`str.len`/
`str.substr`), not sequences of bitvectors -- this parser rejects
`(_ Seq (_ BitVec 8))` sorts outright.
- No obligation context carries quantifiers or array terms. A quantified axiom
over a String-typed function (the old `mktag` injectivity) poisons SAT-mode
model building in this build (every mutation check went `unknown` with it
present, instant without); the `esc` algebra lives in
`spec/esc-axioms.smt2`, outside every obligation, since the proofs are
congruence arguments that never instantiate it.
- Verified on both Z3 4.9.1.0 (native, the CI pin) and 4.16.0.0 (WASM,
local iteration): identical verdicts and models.

## Known limitations (honest scope)

- Bounded instantiations: fixed role shapes (S1/S2), 4-step T2 session. The
general schema is the same equations at arbitrary N; extending N is mechanical
but not yet parameterized in SMT2.
- `esc` length bounds use worst-case ×6 (JSON `\uXXXX`); the real encoder is
tighter — the bound only constrains, never enables, a wrong conclusion here.
- Frame/envelope functions stay uninterpreted: their *placement* (content right
after `fpre`, separator discipline, carrier slots) is the load-bearing part
and IS asserted; their exact byte content is delegated to the #2144 suite.
- Stronger decompositional form of T1 (explicit `bodyA = P ++ CA ++ M1` split
witnesses with per-carrier offsets) is future work; the current explicit-P
prefix form already matches the issue's statement (prefix equality outside
the existentially quantified carrier positions).
10 changes: 10 additions & 0 deletions proof/spec/esc-axioms.smt2
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
; esc axioms E0-E3 (see preamble header for meanings). Kept separate from the
; theorem runs: they are quantified over String and are NOT needed for any
; obligation (proofs are congruence arguments; carrier merges are spelled out
; esc(c) ++ esc(C) BY CONSTRUCTION in the element definitions). This file
; exists so their mutual consistency stays machine-checked [esc-axioms-
; consistency] and so the grounding map in ../README.md has a concrete target.
(assert (= (esc bempty) bempty))
(assert (forall ((a BSeq) (b BSeq)) (= (esc (bconcat a b)) (bconcat (esc a) (esc b)))))
(assert (forall ((a BSeq) (b BSeq)) (=> (= (esc a) (esc b)) (= a b))))
(assert (forall ((s BSeq)) (and (<= (blen s) (blen (esc s))) (<= (blen (esc s)) (* 6 (blen s))))))
59 changes: 59 additions & 0 deletions proof/spec/preamble.smt2
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
; billion-context cache-stability theorems (#2149) -- shared preamble.
; Byte-level model of the outbound-body assembly contract. Conservative SMT2
; (uninterpreted functions + sequences + arrays + linear integer arithmetic);
; targets Z3 4.8+. Trust boundary and grounding map: see ../README.md.

(set-option :produce-models true)
(set-option :timeout 60000)

(define-sort BSeq () String)

(define-fun bempty () BSeq "")
(define-fun bconcat ((a BSeq) (b BSeq)) BSeq (str.++ a b))
(define-fun blen ((s BSeq)) Int (str.len s))
(define-fun bunit ((x Int)) BSeq (str.from_code x))
(define-fun bextract ((s BSeq) (i Int) (j Int)) BSeq (str.substr s i j))

; JSON array-element separator between serialized messages (fixed per wire).
(declare-const SEP BSeq)
(assert (= SEP (bunit 44)))

; esc: the char-wise string encoder inside the JSON document (V8 string
; escaping as used by JSON.stringify on each content/tools field).
; E0 empty maps to empty
; E1 homomorphism: esc(a ++ b) = esc(a) ++ esc(b) (per-char processing)
; E2 injectivity (escaping is deterministic and reversible)
; E3 length bounds: |s| <= |esc(s)| <= 6|s| (\uXXXX worst case)
; GROUNDING [g3] in tests/cache-theorem.test.ts pins E1/E2 against the real
; serializer on randomized inputs.
(declare-fun esc (BSeq) BSeq)
; NOTE: the esc AXIOMS (E0-E3) live in esc-axioms.smt2, deliberately
; OUTSIDE the theorem runs: the proofs are congruence arguments over shared
; subterms and need no quantified axioms, and keeping them out keeps every
; obligation quantifier-free and fast. See ../README.md.

; Per-message element frame: elem = fpre(r) ++ contentBytes ++ fmid(r)
; ++ toolsBytes ++ fpost(r, tagBytes).
; Content bytes sit immediately after fpre: this placement is what makes the
; guaranteed-stable depth D land right after the stable content of the
; merge-anchor element. Pure deterministic framing -- no axioms needed.
(declare-fun fpre (Int) BSeq)
(declare-fun fmid (Int) BSeq)
(declare-fun fpost (Int BSeq) BSeq)

; Envelope head/tail outside the message array (model field, stream flag,
; tool surface, ...): pure functions of (modelBytes, cfgBytes). Same inputs
; in both runs => same bytes by congruence. GROUNDING: #2144 billing suite
; pins these empirically (byte-LCP unexplainedDivergences=0 across wires).
(declare-fun hdr (BSeq BSeq) BSeq)
(declare-fun ftr (BSeq) BSeq)

; ACP tag bytes: deterministic function of the ref number.
; GROUNDING [g2] pins determinism/injectivity to indexToRef (kernel/src/refs.ts).
(declare-fun mktag (Int) BSeq)
; NOTE: no injectivity axiom here. The theorems never need it (tags appear as
; shared subterms on both sides of every equality), and a quantified axiom over
; a String-typed function poisons SAT-mode model building in this solver build
; (every mutation check went unknown with it present, instant without).
; Determinism/injectivity of the real tag renderer is grounded separately
; ([grounding:g2] in tests/cache-theorem.test.ts vs indexToRef).
18 changes: 18 additions & 0 deletions proof/spec/t1-s1-mutation.smt2
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
; MUTATION (S1): a regression that rewrites element 0's STABLE content in run
; B -- same length, different bytes (a stable-region corruption at the spec
; level; WHICH byte changed is existentially hidden at this abstraction, the
; concrete byte surface is pinned by the #2144 suite). Expectation SAT: the
; corrupted run fails the guaranteed-prefix property -- the proof is falsifiable.
(declare-const c0bad BSeq)
(assert (and (not (= c0bad (esc c0))) (= (blen c0bad) (blen (esc c0)))))
(define-fun elem0Bm () BSeq
(bconcat (bconcat (bconcat (fpre r0) c0bad) (fmid r0))
(bconcat (esc t0) (fpost r0 (mktag rho0)))))
(define-fun bodyBm () BSeq
(bconcat (bconcat (bconcat (hdr MB CFG) elem0Bm) SEP)
(bconcat (bconcat (bconcat (bconcat elem1 SEP) elem2) SEP)
(bconcat (bconcat elemM (ite hasTailB (bconcat SEP tailB) bempty))
(ftr CFG)))))
; single-side form: the uncorrupted side satisfies the property by the theorem
; file, so this conjunct's failure IS the pair property's failure.
(assert (not (= (bextract bodyBm 0 (blen P)) P)))
9 changes: 9 additions & 0 deletions proof/spec/t1-s1-theorem.smt2
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
; THEOREM 1 (S1): growth stability -- negated form; expectation UNSAT.
; P (defined in t1-s1.smt2) is the explicit common-prefix term: envelope head +
; stable content of the merge-anchor element; blen(P) = D. The statement says
; the first blen(P) bytes of BOTH bodies equal P -- the decidable form of
; "common byte prefix of depth at least D": Z3's string solver normalizes
; substr at boundaries aligned with the concat structure, whereas quantifying
; over a symbolic p is beyond its decision procedure (returns unknown).
(assert (not (and (= (bextract bodyA 0 (blen P)) P)
(= (bextract bodyB 0 (blen P)) P))))
Loading
Loading