Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,13 +101,21 @@ The evidence model is being revised. [`docs/design/evidence-v3.md`](docs/design/
its definition, permitted authority, execution provenance, target state, proof
artifacts and verifier identity all refer to the same thing.

**The model is implemented and activatable.** Before activation, records on disk
are `evidence/2`; beside each one the CLI writes an evidence/3 shadow, and
`compare` and `qualify` report how the v3 reading differs without changing the
v2 verdict. After `verify.py activate` writes `.verification/authority.json`,
**The model is implemented and activatable.** A project adopts it with
`evidence = "verification.ladder.evidence/3"` in its committed policy. Until a
checkout activates, records on disk are `evidence/2`; beside each one the CLI
writes an evidence/3 shadow, and `compare` and `qualify` report how the v3
reading differs, while `compose` refuses to report completion under the older
contract. After `verify.py activate` writes `.verification/authority.json`,
new `run`, `attest`, and `import-ci` records are `evidence/3`, and `compose`
applies the v3 admissibility rules as the authoritative predicate. Existing
`evidence/2` records remain readable for migration diagnostics, but cannot
satisfy activated v3 requirements.

**It has qualified against a real consumer.** Cerberus adopted evidence/3 on
0.3.0 and reached `READY FOR HUMAN GATE TRUE` from its own local gate, a real
GitHub Actions run and attested judgment. Six targeted mutations were each
refused for the reason that names them. The record is in
[`docs/qualification/cerberus-0.3.0.md`](docs/qualification/cerberus-0.3.0.md).

MIT licensed; see `LICENSE`.
148 changes: 148 additions & 0 deletions docs/qualification/cerberus-0.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Consumer qualification: Cerberus on 0.3.0 (`evidence-v3.2`)

The first run of the Ladder end to end outside its own repository, against a
real consumer rather than more self-tests. Recorded on 2026-09-23.

| | |
|---|---|
| Ladder installed | runtime `c86bf47` (0.3.0, `evidence-v3.2`), via `scripts/install-ladder.sh` from `main` at `6ea587e` |
| Consumer | [Deep-Sixed/Cerberus](https://github.com/Deep-Sixed/Cerberus) at `3956a48`, branch `claude/verification-ladder-adoption` |
| Consumer change | [Deep-Sixed/Cerberus#20](https://github.com/Deep-Sixed/Cerberus/pull/20): `verification.toml` and `.verification/` in `.gitignore`, and nothing else |
| CI evidence | push run [35847808939](https://github.com/Deep-Sixed/Cerberus/actions/runs/35847808939), job `validate` |
| Result | `READY FOR HUMAN GATE TRUE` under authoritative evidence/3; six targeted mutations each refused for the reason that names them |

Nothing in the Ladder was changed to make this pass. The consumer supplied its
own policy, and the verifier ran as installed.

## 1. Install

The documented path, into a clean `HOME`:

```sh
git clone https://github.com/Deep-Sixed/verification-ladder /tmp/ladder-bootstrap
sh /tmp/ladder-bootstrap/scripts/install-ladder.sh
~/.local/share/verification-ladder/check-install.sh
```

The checker reported `INSTALL CONFORMS`: the clone was detached at the pin with a
replacement-free tracked tree, the skill symlink resolved into the pinned clone,
and the invariant was present verbatim and checksum-verified (`282f2117…`). The
installed `verify.py` reports `evidence-v3.2` / `0.3.0`.

With no policy, the installed verifier refuses Cerberus: `BLOCKED`, exit 2,
"no verification policy found". Silence is not consent.

## 2. The consumer's policy

Cerberus requires every step of its `validate` job as a gate:

- `suite` (`scripts/ci.sh`), which also runs locally;
- `build`, `frontend-syntax`, `compose-config`, `container-build`,
`container-smoke` and `secret-scan`, which are established by CI only.

It adopts `verification.ladder.evidence/3` in the committed policy. It accepts CI
evidence only from `Deep-Sixed/Cerberus` → `.github/workflows/ci.yml` → job
`validate`, and only from push runs. Its six judgment rungs match this
repository's.

Unnamed `run:` steps are reported by the Actions API as `Run <command>`, and
that is the string each `[ci_steps]` entry names. All seven matched the real run
without adjustment.

## 3. Pre-activation evidence and qualification

All against state `3956a48 + sha256:d455437d…`:

| step | result |
|---|---|
| `baseline` | governs all 7 gates |
| `run` | `suite` PASS, 319 passed on managed Python 3.14.5, `drift: false`; evidence/2 record plus evidence/3 shadow, marked pre-activation |
| `import-ci` | all 7 gates PASS, each bound to its exact step; the provenance chain checked at import |
| `attest` ×6 | task, diff, self-review, requirements, architecture, fresh-review |
| `compose` | **refused**: the policy adopts evidence/3 and the checkout is not activated, so completion cannot be reported under evidence/2 |

`qualify` over the three records:

```
QUALIFIED outcome / kind / kind requirement / definition / authority
QUALIFIED projection binding / target projection
OUT OF SCOPE artifact chain (no gate with evidence_mode = "execution+attestation")
QUALIFIED admissibility (aggregate) / verifier contract / governance
QUALIFIED ci provenance 1 agree
QUALIFIED ci step 7 agree
QUALIFIED 12 DISAGREEMENT 0 NOT COMPARABLE 0 UNCOVERED 0 OUT OF SCOPE 1
M3 QUALIFIED TRUE
```

`activate` → `AUTHORITY SWITCH TRUE`. The declaration records the 12 qualified
predicates and `artifact chain` as out of scope, which composition enforces if
Cerberus later adds a behavioural gate.

## 4. Authoritative evidence/3

`run`, `import-ci` for the same push run, and the six `attest` rungs were
re-run. All three records are `evidence/3`, `evidence-v3.2`, authoritative, and
PASS.

```
VERIFICATION STATE: 3956a48c83a9 + sha256:d455437d… [evidence/3]
* build, compose-config, container-build, container-smoke,
frontend-syntax, secret-scan PASS execution ci
* suite PASS execution ci, local
six judgment rungs PASS attestation
BLOCKED REQUIRED GATES 0
STALE EVIDENCE 0
STATE MATCH TRUE
EVIDENCE ADMISSIBLE TRUE
DEFINITION CHANGE NONE
CLEAN CLIMB TRUE
READY FOR HUMAN GATE TRUE
```

## 5. Refusals

Each mutation changed one dimension, used copies of the records, and was undone
before the next:

| # | mutation | outcome |
|---|---|---|
| M1 | a tracked source file edited | `STALE EVIDENCE 3`, `STATE MATCH FALSE`, not ready |
| M2 | the CI record names another repository | "run belongs to 'someone-else/Cerberus', the repository under verification is 'Deep-Sixed/Cerberus'" |
| M3 | the CI job ran another commit | "job ran 000000000000, the run names 3956a48c83a9" |
| M4 | a record stamped with the pre-hardening contract | "records were produced under different verifier contracts (evidence-v3.1, evidence-v3.2)" |
| M5 | `main`'s green push run imported for this head | "run is for a0b39996bda0, expected 3956a48c83a9", and **no record written** |
| M6 | `.verification/authority.json` deleted | "this checkout carries no activation record … completion cannot be reported under verification.ladder.evidence/2" |

With the untouched evidence restored, `compose` read `READY FOR HUMAN GATE TRUE`
again, and Cerberus had no tracked changes.

## Observations

None of these is a defect that let bad evidence through. Each is recorded rather
than fixed here, so that qualifying a consumer doesn't turn into tailoring the
verifier until that consumer passes.

- **`compose` finds the baseline beside the first record it is given.** With the
first record outside `.verification/`, it reports `GOVERNING BASELINE MISSING`
even though the baseline exists beside the others. This fails closed, but the
message sends the reader looking for a baseline that is present.
- **The consumer's local gate needs the consumer's toolchain.** `scripts/ci.sh`
needs uv 0.11.29 and a uv-managed Python 3.14.5. The verifier runs whatever
`PATH` provides, and the record's `output_tail` carries the toolchain line
`scripts/ci.sh` prints. The Ladder itself needs only `git` and Python 3.11+.
- **The pinned checkout carries an older pair of scripts.** Since #7,
`scripts/` inside `~/src/verification-ladder` is the pin's own pair, which
expects an older pin. Run there, the checker reports `INSTALL DOES NOT CONFORM`
rather than passing. `INSTALL.md` says to run the bootstrap copies.
- **This session's git access could not push the `v0.3.0` tag** (HTTP 403, the
same restriction that refuses branch deletion). The installer pins by commit,
so qualification did not depend on the tag. Tagging `6ea587e` is left to the
maintainer.

## Not covered

- **Behavioural gates.** Cerberus declares none, so the artifact chain was out
of scope rather than exercised on real artifacts. It is covered by this
repository's own suite and by `tests/test_m4_activation.py`.
- **Gate timeouts and bounded execution**, which are deferred in the design
(§G3–§G4).
Loading