Repository navigation
test: property tests, Rust-vs-Go generated corpus, Go fuzz/-race and coverage in the gate - #59
Merged
Merged
Conversation
added 4 commits
October 17, 2026 13:40
…coverage in the gate (#54) The corpus-based suites pin cases someone thought of. This adds checks over inputs nobody wrote down. - engine/tests/properties.rs (proptest): canonical output is valid JSON with strictly ascending keys (checked by an independent serde_json visitor that, unlike serde_json::Value, does not silently drop a duplicate); output is independent of mapping insertion order; digest shape and sensitivity; exact compare is reflexive and order-independent; BigIntValue::try_new and Map::from_pairs accept exactly what they promise to; the reader never panics on arbitrary bytes or YAML-shaped text; the JSON text of a generated value reads back to the same canonical form. - Cross-language: the same suite writes a fixed-seed corpus; go/canonical /crosslang_test.go re-derives canonical bytes and digest for every document (50,000 agree). This found the float tie divergence (#57) at its first run. The gate fails if the corpus is set but missing or empty. - go/canonical/fuzz_test.go: native fuzz target (valid JSON out, fixed point of canonicalization, no panic); seeds run in go test. - gate.yml: cross-language step, go test -race, informational coverage summaries for both languages (Rust 94.8% lines, Go 82.2% statements at the time of writing; no floor yet). - fuzz.yml: weekly and on demand, PROPTEST_CASES=500000 and a 10 minute Go fuzz run; the failing Go input is kept as an artifact. Verified by mutation: try_new accepting anything, from_pairs allowing a repeat, write_map not sorting, and the float tie rule turned off each fail at least one of these checks (the last one only in the Go cross-language step, at line 8270 of the corpus, the original find). 500,000 cases per property and 2.1M Go fuzz executions found no further defect. Closes #54 --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEUCIDHuNBMCKBHDDusvOsloe/5XvpCQnDpClKvTPVzyScm7AiEAlZgHm1eofKc3MDopse2fRq7dlLAgoIFbMEb5tyXgsm4= hash-algorithm = sha256 digest = bOwGiFR+N9vHt7gKttK/2S+6LQOzGFjYKUpOq4NcZFM= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
--- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEQCIF53kkxYTqNfvydb+GLpUlQwpceDEJvXWd1PopvjZeG9AiBJnD1D/fkG58QqmCHEnd1vtwjFpR+mWp0yi+cbQbJMnQ== hash-algorithm = sha256 digest = T2TQXxQeqgZFlwlvO3/n0jlYOKLucPGs9UysjgLiIM4= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
--- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEUCIDYiszg1CSvnwQAWK0x5iwbOl7HBNQAdeDYZMYPrfC4+AiEAq+pC8bjq1aLytO3u08RyAdRRjrf+c4RfueONapSPvjo= hash-algorithm = sha256 digest = +GfoHDnzcI7qIQQPVZdkLoWwj8c+FQE/aAUvbkgU+gE= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
…e compiled out there) --- [signing-metadata] key = cic-my-sign-key signature = vault:v1:MEYCIQDUQiePChatttwlj6cw9IUBe7OL3NoQfP46hURoPhP61gIhAJsYqXp24ZeDNUj1a771FkVNr+bZ9UAnkZG77LVdL6fD hash-algorithm = sha256 digest = gSlGHqczeQlgDiQMNSotkb4NN0kLV7/QmcMFFB9xtpE= [certificate] -----BEGIN CERTIFICATE----- MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0 lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW -----END CERTIFICATE-----
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #54. Stacked: #55 ← #56 ← #58 (float fix) ← this; retarget to
mainas they merge. Depends on #58: without it the cross-language check below fails on the first divergence it found.Added
engine/tests/properties.rs(proptest, runs incargo test): valid sorted unique-key JSON (independent serde visitor), insertion-order independence, digest shape/sensitivity, exact-compare reflexivity, constructor contracts (BigIntValue::try_new,Map::from_pairs), reader never panics on arbitrary bytes / YAML-shaped text, JSON → reader → canonical round trip.go/canonical/crosslang_test.gore-derives canonical bytes + digest (50,000 documents agree). The Go test fails if the corpus is set but missing/empty.go/canonical/fuzz_test.go: native fuzz target (seeds run ingo test).gate.yml: cross-language step,go test -race, informational coverage summaries (Rust 94.8% lines, Go 82.2% statements; no floor yet — ratchet separately).fuzz.yml: weekly + on demand —PROPTEST_CASES=500000, 10 min Go fuzz, failing input kept as an artifact.What it found: the float tie divergence (#57, fixed in #58) on its first run (corpus line 8270). 500,000 cases per property and 2.1M Go fuzz execs found nothing else.
Verified locally (Rust via the
rust:latestcontainer, Go 1.26.9): fmt, clippy -D warnings, cargo test --workspace, go vet/test -race, cargo llvm-cov, go cover. Mutations — try_new accepts anything / from_pairs allows repeats / write_map not sorted → property tests fail; tie rule off → the Go cross-language step fails (the original find). The Actions workflow files themselves were only YAML-parsed, not run on a runner — the first CI run on this PR is their real test.Not done:
cargo-fuzztargets (proptest covers the same properties without a nightly toolchain); a coverage floor.🤖 Generated with Claude Code