Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Changelog

Each entry names the verifier `VERSION` and its compatibility contract. The
contract is what decides composition: records carrying a contract other than the
verifier's own are refused (`docs/design/evidence-v3.md` §J). A release that
changes what admissible means increments it; a patch that does not, keeps it.

## 0.3.0 — contract `evidence-v3.2`

The same evidence/3 record shape as `evidence-v3.1`, judged under stricter
rules. Records written under `evidence-v3.1` no longer compose with this
verifier: re-run `run`, `attest` and `import-ci` to re-establish them.

What changed what admissible means:

- **Execution binds to the state it measured** (#3). `run` captures the target
before the gates execute and records any dimension that moved while they ran.
A drifted record, or one whose own verdict was BLOCKED or FAIL, is refused at
composition rather than rehabilitated by the passing rows inside it.
- **CI evidence binds the whole provenance chain** (#4). Repository, run,
attempt, workflow, job and commit are checked against expectations the policy
declares in `[ci]`, never against values read from the payload under
examination. `import-ci` refuses a chain it cannot bind before writing any
record, and composition checks the chain again.
- **The adopted contract is committed and fails closed** (#5). A project adopts
evidence/3 with `evidence = "verification.ladder.evidence/3"` in its policy
*and* an activation record in the checkout. A damaged activation record, an
unrecognized contract, or an activation the policy does not carry is refused
rather than read as evidence/2. Qualification is scoped to the predicates the
consumer's own policy puts in play.

Installation (#2): `scripts/install-ladder.sh` and `scripts/check-install.sh`
install and check a pinned checkout. **Their pin still names `760977f`**, which
predates everything above and still reports `evidence-v3.1`. Moving the pin to
this release is a separate change, because the installer refuses a pin that is
not already on `main`.

## 0.2.0 — contract `evidence-v3.1`

evidence/3 authority becomes available behind qualification and activation
(`docs/design/evidence-v3.md` §M4).
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "verification-ladder"
version = "0.2.0"
version = "0.3.0"
description = "A portable verification procedure and evidence machinery for coding agents"
readme = "README.md"
requires-python = ">=3.11"
Expand Down
10 changes: 8 additions & 2 deletions skills/verification-ladder/bin/verify.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,14 @@
# Provenance says which build produced a record; the contract says which records
# may compose. They are separate: a verifier can keep the v3 record shape and
# change what admissible means, and that must invalidate earlier evidence.
COMPATIBILITY = "evidence-v3.1"
VERSION = "0.2.0" # tracks pyproject's version; tests/test_schema_v3.py holds them equal
#
# evidence-v3.2 (0.3.0): drifted and refused records are rejected at composition,
# CI evidence binds repository -> run -> attempt -> workflow -> job -> commit
# against policy-declared expectations, and the adopted contract is committed
# and fails closed. evidence-v3.1 was the same record shape under weaker rules,
# so its records must not compose with these. CHANGELOG.md says what moved.
COMPATIBILITY = "evidence-v3.2"
VERSION = "0.3.0" # tracks pyproject's version; tests/test_schema_v3.py holds them equal
# A gate declares the target dimensions it depends on. Evidence must carry every
# dimension its gate declares, and dimensions it does not declare are never
# compared - so a moved runtime expires a behavioural gate while a lint gate,
Expand Down
20 changes: 20 additions & 0 deletions tests/test_v3_adoption.py
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,26 @@ def test_check_still_binds_a_record_to_the_runtime_it_names(tmp_path):
"only the runtime moved"


def test_composition_refuses_a_record_from_the_pre_hardening_contract(repo):
"""Through `compose`: a record stamped `evidence-v3.1` does not compose here.

That is the contract the previously pinned verifier writes. Nothing else
about this record is wrong - it is this checkout's own run, re-stamped - so
the refusal can only come from the contract.
"""
activate(repo)
local = repo / ".verification" / "v3.json"
assert cli(repo, "run", "--output", str(local))[0] == 0
record = json.loads(local.read_text())
record["verifier"]["compatibility"] = "evidence-v3.1"
local.write_text(json.dumps(record))

code, output = cli(repo, "compose", str(local))
assert code != 0, output
assert "produced under contract 'evidence-v3.1'" in output, output
assert "READY FOR HUMAN GATE TRUE" not in output


def test_check_refuses_a_shadow_record(repo):
"""A shadow record binds to nothing, so there is nothing to re-bind."""
pre_activation_evidence(repo)
Expand Down
32 changes: 32 additions & 0 deletions tests/test_verifier_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,38 @@ def test_a_contract_this_verifier_does_not_apply_is_refused():
assert "the rules in force" in reason


def test_records_from_the_pre_hardening_contract_are_refused():
"""evidence-v3.1 is the contract the pre-hardening verifier still stamps.

Its records have the same shape as these and were judged under weaker rules:
no drift rejection, no bound CI chain, an adoption that fell back to
evidence/2 on a parse failure. The contract was incremented so that they
stop composing here, not merely so that the number moved.
"""
assert verify.COMPATIBILITY != "evidence-v3.1"
status, reason = verify.verifier_status([_record(compatibility="evidence-v3.1")])
assert status == verify.INADMISSIBLE
assert "'evidence-v3.1'" in reason and "the rules in force" in reason

status, reason = verify.verifier_status([_record(), _record(compatibility="evidence-v3.1")])
assert status == verify.INADMISSIBLE
assert "different verifier contracts" in reason


def test_the_release_notes_name_the_contract_this_verifier_applies():
"""A release that changes the contract says so in its notes (§J).

Held mechanically, like VERSION against pyproject: the newest CHANGELOG entry
must name this verifier's VERSION and COMPATIBILITY, so neither can move
without the notes moving with it.
"""
changelog = (_P(__file__).resolve().parents[1] / "CHANGELOG.md").read_text()
newest = re.search(r"^## (\S+) \S+ contract `([^`]+)`", changelog, re.MULTILINE)
assert newest, "CHANGELOG.md has no '## <version> - contract `<contract>`' entry"
assert newest.group(1) == verify.VERSION
assert newest.group(2) == verify.COMPATIBILITY


def test_a_record_without_a_contract_establishes_nothing():
status, reason = verify.verifier_status([{"verifier": {"commit": "a" * 40}}])
assert status == verify.INADMISSIBLE
Expand Down
Loading