Skip to content

feat: conformance/drift verdict artifact in Rust and Go - #60

Merged
sinkog merged 2 commits into
mainfrom
feat/verdict-artifact
Oct 19, 2026
Merged

sinkog merged 2 commits into
mainfrom
feat/verdict-artifact

Conversation

@sinkog

@sinkog sinkog commented Oct 18, 2026

Copy link
Copy Markdown
Member

Implements the conformance/drift verdict artifact of docs/VERDICT-SCHEMA.md in Rust and Go. Scope is kept: Parse/Normalize/Resolve/Validate stay the environment's job; D-018 semantics, the materialization receipt and the Relay enforcement boundary are untouched.

What it adds

  • engine/src/verdict.rs: ConformanceVerdict with all five required fields (verdict_schema_version: 1, object, fields, conformance_plan_digest, observation_digest), build_verdict, verdict_value, verdict_canonical_bytes.
  • go/verdict: the Go peer (Build, Value, CanonicalBytes), plus an exported plan.Aggregate (Rust's aggregate is already public).
  • E_INCONSISTENT_VERDICT / R-VERDICT-CONSISTENT: the builder refuses an evaluate result that cannot be one (classified paths and consumed evidence that do not match one to one; a planned scalar missing from fields; an object that does not follow from fields). A real evaluate result always passes.
  • conformance/differential/verdict/: 7 artifact vectors (the plan/ inputs) and 4 refusal vectors (tamper edits), docs updated (VERDICT-SCHEMA.md status section, READMEs).

Where the values come from

  • fields: evaluate's own fields, so exactly the evaluated paths.
  • Both digests: the existing conformance_plan_digest / observation_digest (and Go peers), called, not recomputed; a test in each language pins this. The multi_key_collection_identity vector's plan digest is byte-identical to the pin of the existing digest/plan_digest_with_collection vector.
  • The pinned artifact bytes were produced by the Rust side first; the Go side was then run against those files without having seen them and reproduced all 7 byte-for-byte.

Tests

  • Rust: 10 unit tests in verdict.rs, verdict_vectors and verdict_vectors_detect_deliberate_divergence in engine/tests/differential.rs.
  • Go: 9 unit tests, TestDifferentialVerdictVectors, TestDifferentialVerdictVectorsDetectDeliberateDivergence.
  • Positive: 7 artifact vectors, equal canonical bytes in both languages and the same object/fields as the same-named plan/ pin. Negative: 4 refusals with the same code and path in both languages. Divergence: 8 deliberate divergences (version bump, changed object, changed/dropped/added field, altered or swapped digests) on each of the 7 vectors, 56 cases per language, each must be caught.

Local results

cargo fmt --check clean, cargo clippy --workspace --all-targets -- -D warnings clean, cargo test --workspace and --release all passing (71 lib, 7 differential, 11 properties, 8 value invariants, 3 conformance, 4 cli); gofmt, go vet, go test -race ./... all passing (7 packages).

Not done, as before

Signing the artifact and an intent-digest field stay "not decided" in the schema.

Not to be merged without review.

Gábor Z Sinkó added 2 commits October 18, 2026 23:41
… with a differential corpus

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEYCIQCaJU+Ywl9cqHVRvUUg1TydmrdkSvH8KC/yd+DagIipGQIhAJi+Pw1e3ul5M6wrOBh4PWCd3MDn/VNUW8tiTLxBt+Ec
hash-algorithm = sha256
digest = F0YbnUHqn8NXdvaZXkmXMGhp/AzNSHaNZsRM7W2FdCg=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
…ths the plan cannot produce, document the low-level builder's limits

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEQCIGZSBvHYHFyFSWbL8NRiiQVezQ31+1bSXYK4ETmdO8ciAiAEA2x9nFCGnk3n05DLh8NG27zGDx6I1T6igM98gUR1oQ==
hash-algorithm = sha256
digest = /7LnYrURshnUbEAx/j8JXxB4GWNcQCkpzRdqMyTJZVc=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
@sinkog

sinkog commented Oct 18, 2026

Copy link
Copy Markdown
Member Author

Both counterexamples reproduced on ba95174, in Rust and in Go, and are handled in d6fa504. The differential corpus (conformance/differential/) is unchanged.

Reproduction on ba95174 (the tests were written first and run against the old code)

  1. A path outside the plan, /ghost, present in both fields and consumed: build_verdict returned an artifact (Rust: expect_err panicked; Go: err == nil). The path sets agreed and object followed, so no check could object.
  2. A result evaluated under plan_a (/memory_gb, numeric) passed to build_verdict/Build with plan_b (same path, exact): accepted, and plan_b's digest written next to plan_a's verdicts. Both languages.

Change

  • Primary entry point: evaluate_verdict (Rust) / verdict.EvaluateAndBuild (Go) run evaluate and build the artifact in one operation. The plan digest in the artifact is the digest of the plan that was executed; the result cannot be swapped or edited between the two steps. This is the answer to 2.
  • Low-level builder kept, with its trust conditions written down in both module docs and in VERDICT-SCHEMA.md: it checks structure only; it does not know which plan produced the result (a same-path, different-comparator plan is accepted), does not re-evaluate (a self-consistent altered result is accepted), and cannot tell a real collection element path from one that only has the right shape. It is not described as rejecting every tampered result.
  • Structural check added (answer to 1): every fields path must be one the plan can produce, a planned scalar path or {collection}/{identity}/{element} with a non-empty identity. Order of checks unchanged, so the earlier refusal vectors keep naming the same path.
  • No change to the digest format, D-018, the receipt or the Relay boundary; no pipeline stage added.

New counterexample tests (Rust and Go, each)

  • counterexample_a_path_outside_the_plan_is_refused (fails on ba95174, passes now).
  • counterexample_the_low_level_builder_does_not_bind_the_plan: pins the documented limit (accepts, carries plan_b's digest).
  • counterexample_the_entry_point_binds_the_plan_to_its_result: each plan's digest sits next to the verdicts that plan produced, the comparator changes the field verdict (numeric vs exact on "16" against 16), and the artifact equals the low-level builder's output for the same plan's own result.
  • collection_element_paths_are_matched_by_shape_and_foreign_ones_refused (4 foreign shapes).
  • verdict_vectors_through_the_entry_point: all 7 artifact vectors, evaluated through the entry point, match the existing pins byte for byte.

Local results: cargo fmt --check, clippy -D warnings, cargo test --workspace and --release all passing (75 lib, 8 differential, 11 properties, 8 value invariants, 3 conformance, 4 cli); gofmt, go vet, go test -race ./... all passing. GitHub checks are in the next line.

@sinkog
sinkog merged commit d832ab2 into main Oct 19, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant