Repository navigation
feat(release): the compiler writes envelope 3, held to the frozen conformance yardstick - #26
Merged
Merged
Conversation
added 6 commits
November 1, 2026 22:47
…ge commit The digests of the reference verifier, the golden fixtures, the vectors and the planted-defect proof are listed as computed from the blobs of d2d8ec7 (not regenerated), and checked against the working tree and, where the history is present, against that commit. A compiler change cannot move the yardstick it is measured against without a red test. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEYCIQCxVh9enWULFDjboyKsf7Z/wzJyh8U6BPaNMMzmQyeY0QIhAI4nowsZ0fD1R5TQ1MaAaiDmgt67ZgqlGwtlBCqk2dzt hash-algorithm = sha256 digest = iRXaqJUMYnIh58/E+QwG4aeD1UeohBCh1sKWG0/z1I8= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
Mandatory source_commit; the producer's explicit release_artifacts.paths, read from the source commit and resolved against its git tree only; per-artifact sha256: digests of the blob bytes, sorted by UTF-8 bytes, with the manifest digest; unique spec source paths; the declaration is refused before Vault is asked for anything. build_hash, release.sign and cic_countersign are the existing mechanism. verify-release reads envelope 1, 2 and 3 and leaves the artifact list to the conformance check; neither imports the other. The record the compiler writes is read back by the reference verifier and put through the golden coverage vectors; tests were written first and shown red. Writer-side tests of envelope 2 now assert envelope 3. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEUCIQCRK1t2AEnbQcM9C8c9eVH5NrKBOHJQKvzW4TcfJmQ1ygIgR0aZLGq4U9DKVYMXEgYENrXwdr3/lomezAz0xdFyzks= hash-algorithm = sha256 digest = i8VLtf9HEgyfzMnvnfzovdG3Q4PN5+22bHAb4+Wmh88= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
…ate timeout 20 minutes 58 source breaks of tools/compiler.py, each run against the tests that target it (50 must turn one red, 8 documented equivalent must survive; each break is compiled before it is run). The gate timeout goes from 10 to 20 minutes because the suite now carries both planted-defect proofs. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEQCIB10mVJKzVMoC+LZik+vOAwvy/4eKFmlXq4Lrbtub1r0AiA2WhqAj0tk1Fn+VIFr//EfN/TGOrCpDwnNyvreAgNwmQ== hash-algorithm = sha256 digest = qVOVUxxTou+M08uKqAzGK5YGnR4GYbwhw5HUMwxZdAc= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
…he source commit The artifacts come from the git tree, but the specs, the legacy provenance digests, the issuer and the validity are read from the working directory, and a clean git status does not make those the commit's bytes (ignored files, assume-unchanged, skip-worktree, line-end filters and sparse checkouts leave it clean). Before asking Vault for anything, run_release compares every such input with the blob of the source commit and the schema files it found with the selection made on the commit's tree; a difference stops it with worktree_differs_from_source_commit. Each case is a test with the clean status asserted as its premise, and the guard has its own planted defects. The residual (the validation gates read the working directory too) is documented. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEYCIQD8xFArc+eqlHff+PwlhMezml/FO6PPj0zPucWreV5LNAIhANMSL1gVK5XhtwYa6ph5DDqTgDEVRrCGN2NQabYELmfo hash-algorithm = sha256 digest = kmlxGot1yu1P5qXIBecwLqJlhAsJYiqutpafla+Hg3E= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
The gate with both planted-defect proofs takes under two minutes on the runner (41353832789, and the run of bb67999), so the raise to 20 minutes is not needed. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEUCICS3E2UVdOUCI4C5abqdISj99vGKKzoWygOjyXNMafqbAiEAqWlyV8H0E1KGq4j9wqRlhFF70Eww35lbfZAA1uyckHc= hash-algorithm = sha256 digest = 73RGsb/kOHJPszJwCVTMp8vu97nfWTCoVIB7I6XjQNQ= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
…published schema must be listed A required input of the release (project.yaml, commitment.yaml, the two grammar files) that is neither in the working directory nor in the source commit is no longer passed over by the consistency check; each is an end-to-end test with the real commitment loader. Every specs[].source_path must be among artifacts.files (spec_not_in_artifacts, before any Vault call). Two characterization tests pin the boundary that remains: the validation gates read the working directory (a composition that only the commit holds is not judged, and an ignored bad example that only the working directory holds stops the release). The test of the required inputs no longer reads its requirement from the constant it checks. Ten more planted defects. --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEQCIDS+LZ9cJGjMwAapjY768DYynDx3AvxOAcz4R6pUxkhIAiA9og5WE1PCWJZknuRNm+A4Qx5uCb0EHPOvkCnDAFjaGQ== hash-algorithm = sha256 digest = lMCF5xmp2gGvuxFvdQAdhPyDmgLK/wf+Kln4Kt8rAOg= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #23. The compiler writes envelope 3, measured against the contract (#24) and the frozen yardstick (#25). Merge is on hold until the owner releases it. No release is cut, nothing is signed or published, no real
project.yamldeclaration is added (separate PR, owner approves the list), the pre-check is not changed.Tests first
tests/test_compiler_envelope3.pywas written before the compiler change and run against the old compiler: 107 failed, 7 passed (the 7 are the ones that must hold before and after: the real envelope 1 and 2 records still verify, an unknown envelope is still refused, the compiler does not import the reference verifier, the old record shapes validate). Then the implementation.What changes (
tools/compiler.py)RELEASE_ENVELOPE_VERSION = 3;_release_hash_payload(bundle, 3)is the envelope 2 rule (no new canonicalization)._source_commit/_collect_provenance(commit):source_commitis mandatory, 40 lowercase hex, nevernull._git_tree,_git_blob(cat-file blob, no filter),_release_artifact_path_problem,_validate_release_artifact_declaration,_declaration_from_commit,_build_release_artifacts,_resolve_release_artifacts,_check_unique_source_paths.run_release: declaration resolved and validated before any Vault call;artifactsinside the bundle beforebuild_hash; envelope 3 in both places.run_verify_release: reads envelope 1, 2 and 3; for 3 it says the artifact list is not checked there. It does not import the conformance verifier, and the verifier does not call the new functions (tests).release.schema.yaml(envelope enum 3,artifacts, conditionalrequiredfor envelope 3),project.schema.yaml(release_artifacts.paths, nothing else under it).project.yaml, not the working directory (tested with a differing working copy, and with a declaration only in the working directory).Which vector proves which requirement
ls-tree; digest of the blob although the working copy has CRLF / other bytes / is missing;.gitattributes eol=crlf+core.autocrlfdeclaration_missing; untracked file =declared_path_absentartifactsblock equalsgolden.expected.json(digests recomputed withsha256sum); all 6 input orders give the same block; UTF-8 vs UTF-16 order; two runs give the samebuild_hashpartspatterns as on the golden record; controls (re-authenticated = validly signed)release/cic-primitives-v0.1.5.yaml(envelope 1) and the real v0.2.0 record from history (envelope 2, sha256 checked) verify with a fixed clock; synthetic v1/v2 tests unchanged; unknown envelope refusedrun_releasewith no Vault call and norelease/written;source_commitnot 40 lowercase hex (9 forms)build_hashof the written record equalsfx.independent_build_hash; AST/source checks both ways;tests/test_conformance_frozen.pyfreezes the verifier, vectors, fixtures and planted proof of #25 at d2d8ec7 (digests from the merge commit's blobs, not regenerated)Planted defects in the compiler
82 breaks of
tools/compiler.py(tests/e3_compiler_mutants.py), each run against the tests that target it: 72 must turn one red, 10 are documented equivalent and must survive (e.g.--full-tree, code-point vs UTF-8 order, submodule mode vs type test). Each break is compiled before it is run. The break "artifacts added after the hash" is killed by the independentbuild_hash; "artifacts resolved after Vault" by the Vault-call counter. (One of my own breaks was equivalent by coincidence of the fixture and was replaced - the harness reported it.)Source consistency (review findings, commits bb67999 and 83e30ea)
The review asked whether data read from the working directory can contradict the artifact evidence of the commit. It could:
specs(content andmeta_hash), the three legacy provenance digests, the issuer and validity (project.yaml,commitment.yaml) are read from the working directory, and thegit status --porcelainof the prerequisites stays clean for an ignored file, an assume-unchanged or skip-worktree file, a line-end filter and a sparse checkout.run_releasenow compares each of those inputs with the blob of the source commit, and the schema files it found with the selection made on the commit's tree, before any Vault call; a difference stops it withworktree_differs_from_source_commit(file and what differs). One test per case, each asserting first thatgit status --porcelainis empty; a test that the tree selection equals the glob selection.Absent on both sides (second review). An input that exists neither in the working directory nor in the commit is no longer passed over:
project.yaml,commitment.yamland the two grammar files are required inputs and the check names them itself;dependency.yamlis the one optional input (legacy digestnull, as before). End-to-end tests with the real commitment loader (commitment.yamlis named by the check; the others by the step that stops first:declaration_missing, the grammar gate), plus a test that the check names each of the four without relying on the constant it checks (a first version of the test did, and two of its planted defects survived; fixed).The boundary that remains, pinned by two characterization tests (second review). The validation gates read the working directory. (1) A composition that only the commit holds and the grammar rejects can be left out of the working directory (skip-worktree,
git statusclean): the gate passes and the release is cut; the test then puts the file back and shows the same gate rejects it. (2) The other direction: an ignored bad example that the commit does not hold stops a release of a good commit. Neither enters the record; both mean "the gate verdict is about the working directory, not about the commit". The tests are not wishes: if the boundary is ever closed they change on purpose. Documented indocs/*/RELEASE-COMPILER-ENVELOPE3.md. A release should be cut from a fresh clone.Every published schema is listed (owner requirement).
spec_not_in_artifacts: everyspecs[].source_pathmust be amongartifacts.files, checked before any Vault call (and again on the list actually written). A compiler-side refusal, not a rule of the record contract; a record-level check would be a contract revision and a change of the frozen yardstick, which is not made here.Existing tests changed
Only the writer side: the sandbox of
tests/test_compiler.pyis now a real git checkout (shared intests/e3_compiler_support.py), and the envelope 2 writer assertions assert envelope 3. The envelope 1 and 2 verification tests are unchanged.Read-only audit of the counter-signature service
cic-countersign(devel8ce2427, not modified):build_hashis an opaque digest "trusted as given, never recomputed"; the submitted document is carried for the audit log only (kind,versionlogged), not hashed or canonicalized. It reads nothing of the envelope or ofartifacts. No dependency found.Not in this PR / for the owner
project.yaml(the real release path staysdeclaration_missinguntil approved).artifacts.files, trust gates, signing, publication, production receipt.timeout-minutes10 -> 20: the suite now carries two planted-defect proofs; locally the whole gate takes ~4.5 min locally (782 tests, coverage 90.24 %, floor 88). Say so if you want it handled differently.