feat(validation): strictly replay persisted conformance manifests by seonghobae · Pull Request #1097 · ContextualWisdomLab/fast-mlsirm

seonghobae · 2026-08-20T21:34:14Z

Fixes #1096. Advances #1077. Stacks directly on Draft #1095.

RED → GREEN

RED 6e5d5e483cc3c402a4caed1a1d4bec36f97e50ba establishes strict persisted-manifest replay acceptance for canonical identity, fingerprint tampering, exact root/nested key sets, hostile mapping/list/text subclasses, duplicate JSON keys, bounded JSON input, and normalization drift.
GREEN 2f75bd90f183793783093e0ca6ac4ff5e7de3779 adds ConformanceInventory.from_manifest() / from_json() through package-owned constructors with exact built-in container/text admission, exact nested schemas, fingerprint verification, canonical replay, duplicate-key/non-finite rejection, and a 1 MiB pre-parse JSON bound.
Coverage hardening c7f948f5642bbe804e6a158c553b4d46b730d329 exercises real nested engine/evidence/run-provenance round trips and every nested key-set boundary rather than only an empty-evidence shape.
Resource RED 5561fb9162d374b3f18eed80e4083f72824a476e proves unpaired-surrogate input escaped as UnicodeEncodeError and deeply nested bounded input could escape as RecursionError.
Resource GREEN ab735cc98dff17923356702227e6515b9c80c4bc maps both conditions into stable package-owned fail-closed ValueError contracts without weakening the 1 MiB bound.
Release record 7e7fcd8708b323b9e231dfb11511743310759efd records the governed buyer-visible replay contract.

Contract

Persisted cross-engine evidence can now be rehydrated without silently accepting schema drift, duplicate JSON members, caller-defined container/text callbacks, tampered fingerprints, normalization/order changes, invalid UTF-8 text identities, or parser-recursion escape. No jsonschema dependency is introduced.

Ownership boundary

Python remains validation/provenance/serialization only. No likelihood, fitting, scoring, alignment, discrepancy, bias/MAE/RMSE, uncertainty, Monte Carlo, or other psychometric/statistical arithmetic is introduced. Production numerical ownership remains Rust-first and Psychometrics Commons remains downstream.

Stack / acceptance

Base is the live feat/conformance-runtime-provenance-1094 branch from Draft #1095. Keep Draft until #1095 and predecessors integrate and this exact head has terminal hosted CI/security/package/coverage plus independent current-head review evidence. Predecessor-head checks and reviews are historical only.

coderabbitai · 2026-08-20T21:34:24Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: eefe04bb-d437-4adb-9f33-9592ec764856

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

seonghobae · 2026-08-20T21:38:04Z

@coderabbitai review

coderabbitai · 2026-08-20T21:38:12Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

seonghobae · 2026-08-20T21:41:44Z

@coderabbitai review

coderabbitai · 2026-08-20T21:41:49Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

seonghobae · 2026-08-20T21:59:27Z

Current exact HEAD is 1b5ddb4a21d6febd631ff4bdfda537b98dfe7733 (base c233b236f6dd6be48cc8ab7bd5c8f7fc4c9c247c). I preserved the concurrent replay implementation and added the minimal parsed-container nesting guard; the resource-boundary tests now pass.

Local evidence at this exact HEAD:

91 passed in the cross-engine replay/provenance suite
Ruff passed
Interrogate passed at 100%
git diff --check passed

One inherited parent-stack failure remains in tests/test_cross_engine_conformance_mutation_replay.py::test_inventory_manifest_rejects_mutated_container_before_iteration_callback: its fixture constructs executed evidence without the now-required run provenance. Per Issue #1096, I did not mutate #1095 source refs; parent compatibility needs a separate fix before this draft can advance.

seonghobae · 2026-08-21T01:51:37Z

Current-head review record for 1b5ddb4a21d6febd631ff4bdfda537b98dfe7733: strict persisted conformance replay is bounded before parsing (1 MiB/128 nesting), rejects duplicate keys and non-finite constants, admits only exact built-in containers/strings, enforces exact schemas, revalidates package contracts, and requires canonical reserialization plus matching inventory fingerprint. CodeGraph tracing confirms this remains provenance/serialization only; no psychometric arithmetic moved. Replay/boundary suites pass (82 passed), Ruff, production interrogate 100%, compileall, and git diff --check pass. This PR is stacked on the updated runtime-provenance parent; review exact current HEAD only and merge normally after its base, without bypass.

devin-ai-integration

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

seonghobae · 2026-08-21T02:14:20Z

Root cause fixed at current head 3ef35ca: the hosted full pytest run exposed one mutation-replay fixture that still constructed executed evidence without the runtime provenance required by the parent contract. Added the package-owned runtime provenance fixture (including output digests and redistribution metadata) to that test only. Focused conformance/manifest/provenance suites: 69 passed; Ruff, interrogate (production 100%), compileall, and diff checks passed. No production logic was changed.

* test(conformance): require runtime and redistribution provenance * feat(conformance): bind runtime and redistribution provenance * test(conformance): align inventory fixtures with runtime provenance * test(conformance): carry runtime identities in provenance replay fixture * test(conformance): carry runtime identities in execution fixture * docs(changelog): record runtime provenance contract * test(conformance): cover malformed runtime provenance controls * test(conformance): provide provenance for executed mutation fixture * feat(validation): strictly replay persisted conformance manifests (#1097) * test(conformance): require strict persisted manifest replay * feat(conformance): strictly replay persisted manifests * docs(changelog): record strict conformance manifest replay * test(conformance): cover nested manifest replay boundaries * test(conformance): require stable JSON resource failures * fix(conformance): stabilize bounded JSON replay failures * fix(conformance): bound parsed manifest nesting * test(conformance): provide provenance for executed mutation fixture

…1093) * test(conformance): require output provenance for executed evidence * fix(conformance): bind executed evidence to run outputs * test(conformance): align fixtures with executed provenance contract * docs(changelog): record executed conformance provenance gate * test(conformance): seal mutated output provenance before execution gate * test(conformance): provide provenance for executed mutation fixture * Revert "test(conformance): provide provenance for executed mutation fixture" This reverts commit b94cf2e. * test(conformance): provide legacy provenance fixture * feat(validation): complete conformance runtime provenance (#1095) * test(conformance): require runtime and redistribution provenance * feat(conformance): bind runtime and redistribution provenance * test(conformance): align inventory fixtures with runtime provenance * test(conformance): carry runtime identities in provenance replay fixture * test(conformance): carry runtime identities in execution fixture * docs(changelog): record runtime provenance contract * test(conformance): cover malformed runtime provenance controls * test(conformance): provide provenance for executed mutation fixture * feat(validation): strictly replay persisted conformance manifests (#1097) * test(conformance): require strict persisted manifest replay * feat(conformance): strictly replay persisted manifests * docs(changelog): record strict conformance manifest replay * test(conformance): cover nested manifest replay boundaries * test(conformance): require stable JSON resource failures * fix(conformance): stabilize bounded JSON replay failures * fix(conformance): bound parsed manifest nesting * test(conformance): provide provenance for executed mutation fixture

* feat(validation): add conformance run provenance * fix(changelog): restore fragment headings * test(validation): reproduce run provenance replay bypass * fix(validation): replay run provenance before serialization * docs(validation): record provenance replay integrity * fix(validation): bind executed conformance to run output provenance (#1093) * test(conformance): require output provenance for executed evidence * fix(conformance): bind executed evidence to run outputs * test(conformance): align fixtures with executed provenance contract * docs(changelog): record executed conformance provenance gate * test(conformance): seal mutated output provenance before execution gate * test(conformance): provide provenance for executed mutation fixture * Revert "test(conformance): provide provenance for executed mutation fixture" This reverts commit b94cf2e. * test(conformance): provide legacy provenance fixture * feat(validation): complete conformance runtime provenance (#1095) * test(conformance): require runtime and redistribution provenance * feat(conformance): bind runtime and redistribution provenance * test(conformance): align inventory fixtures with runtime provenance * test(conformance): carry runtime identities in provenance replay fixture * test(conformance): carry runtime identities in execution fixture * docs(changelog): record runtime provenance contract * test(conformance): cover malformed runtime provenance controls * test(conformance): provide provenance for executed mutation fixture * feat(validation): strictly replay persisted conformance manifests (#1097) * test(conformance): require strict persisted manifest replay * feat(conformance): strictly replay persisted manifests * docs(changelog): record strict conformance manifest replay * test(conformance): cover nested manifest replay boundaries * test(conformance): require stable JSON resource failures * fix(conformance): stabilize bounded JSON replay failures * fix(conformance): bound parsed manifest nesting * test(conformance): provide provenance for executed mutation fixture

) * test(validation): define cross-engine conformance inventory contract * feat(validation): add cross-engine conformance inventory * docs(changelog): record cross-engine conformance inventory * test(validation): cover conformance contract fail-closed edges * fix(validation): seal conformance package records * test(validation): seal conformance record subclasses * fix(validation): complete conformance commit provenance boundary * fix(validation): require executed conformance evidence * test(validation): reproduce unsupported conformance coverage claim * test(validation): pin execution-backed coverage invariant * test(validation): reproduce Unicode conformance identity drift * fix(validation): normalize conformance text before hashing * test(validation): make schema-version regex literal * test(conformance): expose post-init mutation replay gap * fix(conformance): revalidate records on manifest replay * docs(conformance): record replay integrity hardening * test(conformance): document replay fixtures * fix(changelog): restore authoritative fragment format * feat(validation): add conformance run provenance (#1085) * feat(validation): add conformance run provenance * fix(changelog): restore fragment headings * test(validation): reproduce run provenance replay bypass * fix(validation): replay run provenance before serialization * docs(validation): record provenance replay integrity * fix(validation): bind executed conformance to run output provenance (#1093) * test(conformance): require output provenance for executed evidence * fix(conformance): bind executed evidence to run outputs * test(conformance): align fixtures with executed provenance contract * docs(changelog): record executed conformance provenance gate * test(conformance): seal mutated output provenance before execution gate * test(conformance): provide provenance for executed mutation fixture * Revert "test(conformance): provide provenance for executed mutation fixture" This reverts commit b94cf2e. * test(conformance): provide legacy provenance fixture * feat(validation): complete conformance runtime provenance (#1095) * test(conformance): require runtime and redistribution provenance * feat(conformance): bind runtime and redistribution provenance * test(conformance): align inventory fixtures with runtime provenance * test(conformance): carry runtime identities in provenance replay fixture * test(conformance): carry runtime identities in execution fixture * docs(changelog): record runtime provenance contract * test(conformance): cover malformed runtime provenance controls * test(conformance): provide provenance for executed mutation fixture * feat(validation): strictly replay persisted conformance manifests (#1097) * test(conformance): require strict persisted manifest replay * feat(conformance): strictly replay persisted manifests * docs(changelog): record strict conformance manifest replay * test(conformance): cover nested manifest replay boundaries * test(conformance): require stable JSON resource failures * fix(conformance): stabilize bounded JSON replay failures * fix(conformance): bound parsed manifest nesting * test(conformance): provide provenance for executed mutation fixture * feat(validation): render accessible cross-engine conformance evidence (#1164) * test(validation): require accessible conformance evidence report * feat(validation): render accessible conformance evidence * docs(changelog): record conformance evidence report * test(validation): require downloadable long-form conformance rows * feat(validation): export long-form conformance evidence rows * docs(changelog): record long-form conformance export * fix(report): stop over-escaping the CSP meta-tag content in cross-engine report Strix flagged this exact pattern on this PR (CWE-693, CVSS 6.5): escape(_CSP, quote=True) converts the CSP's literal 'none' source expression to 'none', which browsers do not parse as valid CSP syntax, silently disabling the meta-delivered policy. _CSP is a fixed module constant with no user-controlled content, so interpolating it directly is safe. This mirrors the same fix already applied to every other report generator on main in #1230. Added a regression assertion that the CSP is embedded unescaped. Verified: pytest tests/test_cross_engine_conformance_report.py -- 8 passed. * Revert "fix(report): stop over-escaping the CSP meta-tag content in cross-engine report" This reverts commit 051bc1b.

seonghobae added 3 commits August 20, 2026 14:21

test(conformance): require strict persisted manifest replay

6e5d5e4

feat(conformance): strictly replay persisted manifests

2f75bd9

docs(changelog): record strict conformance manifest replay

7e7fcd8

test(conformance): cover nested manifest replay boundaries

c7f948f

seonghobae added 2 commits August 20, 2026 14:38

test(conformance): require stable JSON resource failures

5561fb9

fix(conformance): stabilize bounded JSON replay failures

ab735cc

fix(conformance): bound parsed manifest nesting

1b5ddb4

This was referenced Aug 20, 2026

test(conformance): align mutation fixture with runtime provenance #1099

Closed

feat(validation): complete conformance runtime provenance #1095

Merged

seonghobae marked this pull request as ready for review August 21, 2026 01:51

seonghobae enabled auto-merge (squash) August 21, 2026 01:51

devin-ai-integration Bot reviewed Aug 21, 2026

View reviewed changes

test(conformance): provide provenance for executed mutation fixture

3ef35ca

seonghobae merged commit 7c1d04a into feat/conformance-runtime-provenance-1094 Aug 21, 2026
35 checks passed

seonghobae deleted the feat/conformance-manifest-replay-1096 branch August 21, 2026 10:05

Navigation Menu

Repository navigation

feat(validation): strictly replay persisted conformance manifests - #1097

feat(validation): strictly replay persisted conformance manifests#1097
seonghobae merged 8 commits into
feat/conformance-runtime-provenance-1094from
feat/conformance-manifest-replay-1096

seonghobae commented Aug 20, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading

Review skipped

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

seonghobae commented Aug 21, 2026

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

seonghobae commented Aug 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

seonghobae commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

RED → GREEN

Contract

Ownership boundary

Stack / acceptance

Uh oh!

coderabbitai Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

coderabbitai Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

coderabbitai Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seonghobae commented Aug 20, 2026

Uh oh!

seonghobae commented Aug 21, 2026

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

✅ Devin Review: No Issues Found

Uh oh!

seonghobae commented Aug 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

seonghobae commented Aug 20, 2026 •

edited

Loading

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading

coderabbitai Bot commented Aug 20, 2026 •

edited

Loading