Skip to content

fix(release): the reference verifier refuses a commit or a digest that ends in a newline - #31

Merged
sinkog merged 1 commit into
develfrom
fix/verifier-full-match
Nov 2, 2026
Merged

sinkog merged 1 commit into
develfrom
fix/verifier-full-match

Conversation

@sinkog

@sinkog sinkog commented Nov 2, 2026

Copy link
Copy Markdown
Member

The defect

In Python the end anchor `$` also matches in front of a final newline. `DIGEST_RE.match` and `COMMIT_RE.match` in `tools/release_record_conformance.py` therefore accepted `<40 hex>\n` as a `source_commit` and `sha256:<64 hex>\n` as an artifact digest. The contract says 40 / 64 lowercase hex digits. Found by a test of the pre-check (#29), reproduced here.

Proof first (against the verifier of devel, before the fix; record re-authenticated so the refusal has to come from the contract check)

Vector Expected Verifier of d2d8ec7
`source-commit-trailing-newline` FAIL / `source_commit_form` PASS
`digest-trailing-newline` FAIL / `artifact_digest_form` PASS
`manifest-digest-trailing-newline` FAIL / `artifact_manifest_digest_form` FAIL, wrong reason (`artifact_manifest_mismatch`)
controls: two newlines, `\r`, leading newline, trailing space FAIL, same reason FAIL, same reason (unchanged)

The old verifier is read from git inside the test (`test_the_verifier_of_d2d8ec7_did_not_refuse_it_for_that_reason`), so the proof runs in CI; where the history is required its absence is a failure, not a skip.

The fix

Both patterns end with `\Z`. Two lines of the verifier.

The frozen yardstick: amended explicitly, not edited

  • `tests/fixtures/envelope3/FROZEN.sha256` is untouched and still checked against the blobs of d2d8ec7.
  • New `tests/fixtures/envelope3/AMENDMENTS.yaml` lists exactly two files: the verifier, and `tests/e3_mutants.py`. For each: the digest it had (must equal the frozen one), the digest it has (must equal the working tree), and the reason.
  • `tests/test_conformance_frozen.py` checks all three, that every amendment starts from the frozen digest and changes something, and that a release record can never be amended. Shown red both ways: an unlisted edit of `e3_vectors.py`, and an amendment with a wrong `from_sha256`.
  • `tests/e3_mutants.py`: nine mutants carry the text of the changed lines; the change is mechanical (`$` to `\Z` in them). None added, removed or retargeted. A first version of this PR missed the ninth and the planted suite failed with 79 errors; fixed and re-run.

Not changed, and why

  • The v0.2.1 record and every release. The released record still passes `record`, `blobs` and `crypto` with the fixed verifier (test pinned by commit id and SHA-256).
  • The compiler. `SOURCE_COMMIT_RE` has the same shape but reads the `.strip()`ped output of `git rev-parse`, where a final newline cannot occur.

Checks

  • Planted defects: the anchor put back at each place turns the targeted vectors red; the unmutated copy passes.
  • Full suite with `CIC_REQUIRE_RELEASE_HISTORY=1`: 815 passed, 0 skipped (794 on devel).

Merge is for the owner.

…t ends in a newline

In Python the end anchor $ also matches in front of a final newline, so DIGEST_RE.match and
COMMIT_RE.match accepted "<40 hex>\n" as a source_commit and "sha256:<64 hex>\n" as an artifact
digest, and gave artifact_manifest_mismatch instead of artifact_manifest_digest_form for a
manifest digest with a final newline. Both patterns now end with \Z.

Proof first: three new vectors fail against the verifier of d2d8ec7 (two of them were PASS on a
re-authenticated record), the test reads that verifier from git so the defect is reproduced in CI
and not remembered; four controls are refused before and after. A planted-defect test puts the
anchor back at each place and the vectors turn red.

The frozen yardstick is amended explicitly, not edited: FROZEN.sha256 is untouched and still
checked against the blobs of d2d8ec7; tests/fixtures/envelope3/AMENDMENTS.yaml lists the two
changed files with the digest they had, the digest they have and the reason, and
test_conformance_frozen.py checks all three. The second file is e3_mutants.py, where nine mutants
carry the text of the changed lines (mechanical, none added, removed or retargeted).

The released v0.2.1 record still passes record, blobs and crypto with the fixed verifier. The
compiler is not changed: its SOURCE_COMMIT_RE reads the stripped output of git rev-parse, where a
final newline cannot occur.

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEUCIQCLwIZ84N3bk20eWruUv8KTOH+KVNUBpjPb7SO0SGSBtgIgVcIKENjgFT2iutuCZ8IBLpvkWWgD79T7OZSG/I3+RTw=
hash-algorithm = sha256
digest = scaAfk0EzqHJCTcwf7/RcJQhlswbsqckmMpMQwyk8GU=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
@sinkog
sinkog merged commit 62ff98a into devel Nov 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant