Skip to content

Diagnose v2 matrix run/verify compatibility failures with actionable evidence #63

Description

@MarcoPorcellato

External qualification report

A Latent-TRIZ R5 local qualification was stopped because a hash-bound receipt member changed after the authorized candidate commit. That invalidates the R5 evidence; no Latent-TRIZ correction, branch, PR, or evidence publication is requested here.

This issue tracks the CCP compatibility and diagnostic gaps exposed by the failed terminal run.

Reported environment and terminal facts

  • Candidate repository: MarcoPorcellato/Latent-TRIZ
  • Candidate commit: c7933ccbeb9a2857b543643a7e36e18aecba276d
  • Base: 188eb65b5e249923baddadeba52659f07fcd1609
  • Installed executable reported only commit-ci-preflight 0.1.0; its source commit was not bound or recorded.
  • The reporter inspected local origin/main at 866db18a571f55ed3d9b481d6c9c9c3bd5e98d55.
  • The run began from a clean candidate with resource admission allowed and no active/queued admission owner.
  • run --repository . --json reached terminal FAIL. Two repository-check jobs failed in distinct Python 3.11 and 3.12 images; the paired schema jobs passed. The report retains output digests only, not raw logs.

The generated outer document had receipt and receipt_id, but the reported inner receipt lacked schema_version. Immediate verification with the expected candidate commit returned receipt.parse_or_shape, with policy evaluation not run.

Verified CCP contract context

The locally available tracked origin/main ref above contains:

  • matrix configuration, matrix receipt, and matrix policy schema version 2.0;
  • MatrixReceiptV2 with a required inner schema_version;
  • policy-schema dispatch: a v1 policy selects the v1 receipt verifier, while a v2 policy selects the matrix receipt verifier.

Therefore the reported pair—matrix config v2 with policy v1—is not a supported matching pair in that source contract. The missing inner schema also does not match the current MatrixReceiptV2 shape.

This is not proof that current upstream emits malformed receipts: the executing binary has no recorded source provenance, and the failing receipt itself must remain invalid evidence. It is, however, a reproducible compatibility/diagnostic defect from an operator perspective: the current terminal message does not distinguish an unsupported config/policy pair, an outdated executable, and a malformed generated receipt.

Requested outcome

  1. Add a regression matrix covering both PASS and terminal FAIL receipts:
    • v1 run output verifies under its matching v1 policy;
    • matrix v2 run output verifies under its matching v2 policy;
    • a v2-config/v1-policy mismatch fails with an explicit compatibility finding rather than a generic receipt-shape error.
  2. Bind executable provenance (at minimum a source revision or explicit unknown) into the diagnostic surface used by run/verify.
  3. Provide a bounded, privacy-preserving failed-check classification sufficient to route a runtime-only failure without exposing raw logs, sources, secrets, or environment values.
  4. Document the supported migration path for repositories with historical matrix config v2 plus policy v1, or provide an automated compatibility diagnostic.
  5. Document a supported, non-destructive diagnostic/recovery path for an empty or incompatible admission-root child directory. It must remain fail-closed and must not require operators to delete or quarantine admission state manually.

Non-goals

  • Do not accept or repair the invalidated Latent-TRIZ R5 receipt.
  • Do not weaken receipt integrity, admission ownership, or log-redaction guarantees.
  • Do not make manual deletion/quarantine of admission locks, tickets, leases, or cache state a supported workaround.

Acceptance criteria

  • Exact automated tests cover the matching and mismatched schema/policy cases, including terminal FAIL output.
  • CLI output identifies the compatibility boundary and records bounded executable provenance.
  • Documentation states the migration/compatibility decision and the safe admission diagnostic path.
  • The tests prove that no raw check output is added to receipts or public diagnostics.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions