perf: attest exact product performance evidence before commercial acceptance

Owner-boundary defect

ContextualWisdomLab/Orgmetra#317 exposed a generic organization-level evidence-integrity gap while building buyer-path latency acceptance.

Exact Orgmetra head 108f1c0ee70eebfb0fa7d391aecb49a7927d912d already binds result/runtime/fixture bytes by SHA-256 and makes its benchmark runner emit the verified result digest. CodeRabbit nevertheless found a valid P1: a caller can replace both the caller-visible result bytes and a caller-supplied digest, construct matching runtime evidence, and still satisfy a leaf-local acceptance CLI. A mutable pathname or unauthenticated digest token is not a durable commercial evidence boundary.

This is not Orgmetra HR-domain truth and should not be solved by inventing a product-local signing key or mutable workflow. The organization control plane must provide a reusable, immutable, credential-separated performance-evidence attestation path.

Required trust boundary

Provide an organization-owned reusable workflow, invoked by an immutable full commit SHA, that produces and verifies an authenticated attestation for an exact performance result/evidence set.

The design should follow the existing exact-artifact attestation discipline in .github/.github/workflows/exact-artifact-sbom-attestation.yml and docs/doctoring/exact-artifact-sbom-attestation.md, while remaining performance-specific rather than overloading the wheel/sdist/SBOM contract.

At minimum:

bind caller repository, exact source SHA, workflow run identity, sealed evidence artifact ID/name/digest, result filename/SHA-256, runtime-evidence SHA-256, fixture SHA-256, selected performance profile, and predicate type;
validate same-run immutable artifact metadata through the GitHub API before credentialed attestation;
download the artifact only by immutable artifact ID and treat it strictly as inert data;
re-hash the exact result/runtime/fixture bytes inside the trusted reusable workflow and reject any mismatch, path traversal, symlink, extra-file/cardinality drift, malformed JSON, or unbounded artifact;
never execute product-provided evidence or scripts inside the id-token: write / attestations: write job;
attest the exact performance evidence with immutable actions/attest source and a versioned predicate such as https://contextualwisdomlab.org/attestations/product-performance/v1;
verify the generated attestation against the caller repository, central signer repository/workflow identity, exact source commit, subject digest, predicate type, and GitHub Actions OIDC issuer before returning success;
export the Sigstore bundle plus trusted-root/offline verification evidence, with a documented independent gh attestation verify command;
preserve the measurement/runtime distinction: an attestation proves evidence origin/integrity, not that p95 passed or that the service was production-equivalent;
keep self-hosted measurement jobs and credential-bearing service access outside the attestation signer job. If a self-hosted measurement runner is used, its identity/runtime must remain explicit evidence; the signer should attest only the sealed inert artifact set after exact digest checks.

Cross-repository prerequisite

.github#1228 already records a trust defect in the current reusable exact-artifact workflow: github.workflow_sha is not a sound way to identify the reusable workflow's own source for cross-repository callers. This new performance workflow must not repeat that pattern. Resolve reusable-workflow identity from an independently verifiable authority (for example the OIDC job_workflow_ref claim) before treating a cross-repo caller as GREEN.

RED evidence

Orgmetra CodeRabbit review on exact 108f1c0e...:

caller-supplied runnerResultSha256 only proves equality with the bytes the caller supplies to the CLI;
replacing both result bytes and digest can still pass that leaf-local handoff;
the digest must be bound to an independently authenticated runtime-evidence artifact or trusted orchestrator attestation.

Orgmetra will therefore keep commercial acceptance fail closed until this authenticated owner path exists; its local validators may validate structure and byte consistency but must not convert self-asserted evidence into a commercial acceptance receipt.

GREEN acceptance

Dedicated reusable workflow and verifier contracts land through normal protected review.
Cross-repo reusable-workflow source identity is cryptographically/authoritatively bound; exact-artifact-sbom-attestation.yml trusted-verifier checkout uses github.workflow_sha, breaks for cross-repo callers #1228's failure mode is covered by a deterministic RED fixture.
Exact artifact ID/name/digest and exact source repository/SHA are re-read from GitHub and verified before signing.
Credentialed attestation steps execute no caller/product code and hold only least-privilege contents/id-token/attestations/artifact-metadata permissions.
Result/runtime/fixture digests and selected profile are bound in a versioned predicate/receipt.
Wrong source SHA, wrong repository, predecessor run, wrong artifact ID/digest, substituted result+digest, malformed bundle, wrong signer workflow, wrong predicate, and unexpected self-hosted signer all fail closed.
Online verification and retained offline Sigstore bundle/trusted-root verification both succeed for an unchanged canary.
Orgmetra fix(review): validate model output on a copy so publish re-normalize succeeds #316/fix(review): give the publish gate the same evidence file as the model pool #317 consume the immutable released/pinned owner contract and only then restore a positive commercial acceptance receipt.

Primary references

GitHub. (2026). Using artifact attestations and reusable workflows to achieve SLSA v1 Build Level 3. https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/increase-security-rating

GitHub. (2026). Verifying attestations offline. https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/verify-attestations-offline

GitHub. (2026). actions/attest v4.1.0, commit 59d89421af93a897026c735860bf21b6eb4f7b26.

This issue owns only the authenticated cross-repository evidence/attestation boundary. Orgmetra owns HR domain truth, workload semantics, right-cleared fixture requirements, runtime/resource/cleanup evidence, and the p95 acceptance policy.

Consumer canary currentization: Orgmetra #317 is now exact b6968520ef191fe8a3fd84d3cd5d4dc23a778281 on People base c0ba482c57a77820128f259b4a43c06df643c0f3.

The leaf has implemented the fail-closed side of this owner contract. d2ab0b0... adds the substituted-result+digest RED requirement, 5b8b497... introduces the authenticated-evidence gate, 51da657... moves the commercial acceptance entry point behind that gate before any caller bytes/digests are trusted, and b696852... explicitly demotes the runner's emitted SHA-256 to structural evidence only. Thus the consumer no longer emits a positive commercial receipt while this owner capability is missing.

When #2162 produces a reviewed/released reusable attestation contract, Orgmetra should consume only that immutable owner version and add an exact canary proving: substituted result + substituted local digest still fails without a valid central attestation, while the unchanged attested result/runtime/fixture set passes signer-workflow/OIDC/source/subject verification before local p95/runtime acceptance is evaluated.

Authority update from #2166: exact current head is 5c09a98fd6bcca72f7cb0c3b70a0463590324006, stacked on #2164 874f44f9cf7047a18d9815d1677e625f5c7a8c67.

Two source-level false-GREEN paths found by exact-head review are now ordinary-forward repaired with regression coverage:

Retained signer manifest no longer emits ambiguous "result": "PASS"; it emits structural "verification_result": "VALID" and the legacy key is forbidden by tests (cfa040c6 → cc33ecdb → fb9e2751).
Both verifier and credentialed signer jobs now fail closed unless ${SOURCE_REPOSITORY%%/*} is exactly ContextualWisdomLab before artifact API access (183757b2 → 5c09a98f). This prevents an externally accessible reusable workflow from minting central attestations for non-CWL evidence merely because caller inputs match caller GITHUB_REPOSITORY.

The central predicate remains origin/integrity-only and explicitly does not prove latency threshold success, production equivalence, scientific fixture validity, or right clearance. Exact-head quality/security checks and re-review for 5c09a98f are still settling; no merge/consumer GREEN is claimed. #2164 remains the prerequisite owner for reusable-workflow OIDC source identity.

Current owner-path receipt (2026-09-13): implementation is active in Draft #2166 at exact 44445484181a5330ebaefcad37a9aadc8541672b, stacked on Draft prerequisite #2164 874f44f9cf7047a18d9815d1677e625f5c7a8c67.

Fresh repair this pass: hosted SAST on predecessor 9c4fc53... flagged the redundant final os.chmod(output, 0o700) in the bounded ZIP materializer. The suggested 0644 would be invalid/weaker for a directory, so 56d9be31... removes only the redundant final chmod while retaining mkdir(mode=0o700) and evidence-file 0400. 44445484... adds executable edge coverage for those permission invariants plus archive/output-parent symlinks, encrypted metadata, duplicate semantic filenames, streaming-failure cleanup, and successful CLI materialization. No suppression or gate weakening.

The central workflow still attests origin/integrity only. It does not prove p95, production equivalence, scientific validity, or right clearance. Current exact-head quality/SAST/Security/CodeQL runs and CodeRabbit review are settling and are not claimed GREEN.

Protected .github/main has advanced independently to 78393ea901efc0d57b35dd1e383e473425b7c1da; #2164 is mergeable but diverged from current protected truth. Therefore #2166 remains mutable/non-consumable. Normal prerequisite integration/reconciliation, exact-head gates/review, then an immutable consumer canary are still required before any Orgmetra commercial acceptance can use this authority.

Owner-path execution update: PR #2166 predecessor dac085903b9959f22faa068c34b6c18c896a56b5 reached a real exact-head RED in Product Performance Attestation Quality CI run 34765309704. All 68 tests passed, but the mandatory 100% production coverage gate failed because scripts/ci/materialize_product_performance_artifact.py lines 191–197 (_parser() construction) were uncovered (96% for that module; 98% overall). No coverage suppression or threshold weakening was used.

Ordinary-forward repair b48c88edc863259e751a1a981c10692357475eac adds focused execution of the real sealed-artifact CLI parser and asserts all five evidence roles remain explicit. Compare from dac085... is exactly one test-file commit (+26/−0). New exact-head Quality/SAST/Security/CodeQL runs are queued; no GREEN is claimed until they terminate successfully. #2166 remains Draft and stacked on mutable prerequisite #2164, so Orgmetra commercial acceptance must remain fail closed.

Owner-path progress on current authority:

fix(attestation): bind reusable workflow source to OIDC identity #2164 has been reconciled non-destructively with protected main@78393ea901efc0d57b35dd1e383e473425b7c1da by ordinary two-parent merge 73f5b9d3b2664521faf9033743d94d2746ddb12f; fresh compare is 8 ahead / 0 behind. The protected-main delta was disjoint from fix(attestation): bind reusable workflow source to OIDC identity #2164's six owned files.
feat(perf): attest exact product performance evidence #2166 was then non-force restacked by ordinary two-parent merge 6889a38eecb535968602e2f9b48d61be7edf8e30 onto exact parent fix(attestation): bind reusable workflow source to OIDC identity #2164 73f5b9d...; fresh compare is 29 ahead / 0 behind and the descendant delta remains only the performance-attestation files.
Before restack, b48c88... Product Performance Attestation Quality 34767420815, SAST 34767420826, and Security 34767421018 reached SUCCESS. Those statuses are predecessor-only and are not transferred.
New exact-head fix(attestation): bind reusable workflow source to OIDC identity #2164 and feat(perf): attest exact product performance evidence #2166 runs have been created and are queued/pending; exact-head CodeRabbit review has also been requested for both reconciled heads.

The issue remains open. Immutable protected authority and an Orgmetra canary are still required before leaf commercial acceptance can reopen.

Fresh owner-path finding on .github#2166: the central predicate carried performance_profile, but the verifier only validated that caller input syntactically and never bound it to the sealed result/runtime JSON. Orgmetra's current commercial evidence already carries selected_profile in both result and runtime, so a caller/profile substitution could otherwise produce an authentic origin/integrity attestation with the wrong profile label.

Ordinary-forward repair lineage on feat/product-performance-attestation:

e24e9b9c... / 09720f3f...: regression lane added and wired into Product Performance Attestation Quality CI. The hosted run for 09720f3f... was cancelled by subsequent ordinary-forward commits before a runner was assigned, so no hosted RED is claimed for that predecessor.
abfdf118...: verifier gains a fail-closed selected-profile binding gate.
4c2c4687...: both the uncredentialed verifier job and credentialed signer re-verification enable that gate, so each requires sealed result.selected_profile == runtime.selected_profile == performance_profile before emitting/signing the predicate.
df065815...: positive matching-profile coverage added so the new branch is exercised as well as mismatch/missing cases.

Current exact head is df065815212262d8201702c4e9b5cb9fd0a59274, still Draft. Exact-head Quality/SAST/Security/CodeQL are newly queued; predecessor GREEN does not transfer. Orgmetra commercial acceptance remains fail closed until this stack reaches protected immutable authority and a product canary succeeds.

Performance-attestation owner update: #2166 advanced ordinary-forward to cd737f4586d90828b22da2f439210d9265843b4e after a verified source-identity binding defect.

The central workflow already proves caller source_sha == GITHUB_SHA, but its inert verifier previously did not require sealed result/runtime candidate_sha to equal that authenticated source SHA. A same-run artifact for another candidate could therefore be signed under the caller run's source identity. 5ad9cc6... adds mismatch/missing regressions; 4c00f63... binds both sealed candidate declarations to the attested source; cd737f4... aligns the existing commercial identity contract. This is in addition to the prior selected-profile binding.

New exact-head Quality/SAST/Security/CodeQL runs are queued. #2166 remains Draft on #2164 73f5b9d3...; no immutable/released authority or product canary is claimed yet.

Authority receipt update: protected .github/main advanced to 828eaaefb0cc97bba4da63eb9270447476d26710 through Noema document-reader fixes after the prior attestation reconciliation. The main-only files had no overlap with #2164's six identity files or #2166's ten performance-attestation files.

#2164 was therefore ordinary-forward reconciled again with current protected main as 834d5163cb306a540a9064931a2694a730b9cd3f (0-behind; six identity files only). #2166 was then non-force restacked on that exact parent as ddd92ee491c8805c33bf5b2bfd608c2c492bcb26 (0-behind its parent; ten performance-attestation files only).

Both remain Draft. New exact-head checks have been regenerated and predecessor verdicts do not transfer. Orgmetra commercial acceptance must remain fail closed until the prerequisite and performance-attestation owner paths normally integrate and an immutable pinned product canary succeeds.

Caller-workflow trust-boundary repair advanced in #2166 to exact 6fb6676fe8c4b53d52d47e389628d1b2d3ef0af5. Fresh review found repository/SHA-only authentication did not distinguish the canonical product benchmark caller from another workflow at the same source commit. GitHub's reusable-workflow OIDC model exposes standard caller claims (repository, sha, workflow_ref, workflow_sha) separately from called-workflow job_workflow_ref/job_workflow_sha. Both verifier and signer jobs now authenticate those caller claims against the supplied source, and the signed predicate/manifest retain caller workflow ref/SHA for product-canary policy. Test-first b04ae6b... was superseded before hosted execution and is not claimed as hosted RED; current exact-head Quality/SAST/Security/CodeQL runs are regenerated and pending. #2166 remains Draft on #2164.

#2166 advanced again to exact fabae50166d0d6979bb71e8a93de2126528cb50d after revalidating the remaining CodeRabbit streaming-corruption finding on current source. 8238e293b8560158a8eb665559768ecc8e4c6073 now normalizes zipfile.BadZipFile, zlib.error, and EOFError after partial-output cleanup; fabae501... adds focused regressions for both decompressor exceptions. The inline finding was replied to and resolved only after repair. Fresh exact-head Quality/SAST/Security/CodeQL runs were regenerated; no predecessor verdict is transferred.

Current owner-path repair: exact #2166 head fabae50166d0d6979bb71e8a93de2126528cb50d produced Product Performance Attestation Quality CI run 34783209854, which failed only in Verify exact-head performance attestation contracts. The corrupt-stream regression tests/test_product_performance_corrupt_stream_errors.py already covered both zlib.error and EOFError, but the canonical quality workflow omitted that file from its pull-request path trigger, Python 3.10 compile set, and coverage/pytest execution while materialize_product_performance_artifact.py remained under 100% branch-coverage enforcement. This was CI manifest drift, not a reason to weaken coverage or the materialization boundary.

Ordinary-forward repair cc44ddb7bcb3a74538659a7c40e9f67170e28057 adds that regression to all three quality-manifest locations. Fresh exact-head #2166 runs have been regenerated (34966728387 quality, 34966728341 Security, 34966728421 SAST, 34966728418 CodeQL); they are non-terminal at this receipt, so no GREEN transfers from predecessors. #2162 remains open through protected/released owner authority and the Orgmetra canary.

Authority reconciliation after the CI-manifest repair: protected .github/main advanced to 91be6442906c7b6b4f600272c953699708394327. Fresh comparison showed no path overlap with #2164's six owned identity files, so prerequisite #2164 adopted current main by ordinary two-parent commit b5966eaa6237c9260823969cbae6f64000cb08d7 (0 behind). Descendant #2166 was then non-force advanced to 759c51f7544fc8176f53ed947370210d0f596815, with exact base b5966eaa... and only its 11 product-performance attestation files as delta. Fresh exact-head workflows were regenerated on both PRs; all are non-terminal, so no predecessor GREEN is transferred and #2162 remains open.

Hosted owner RED was reproduced and repaired without weakening the attestation trust boundary. #2166 predecessor exact 759c51f7544fc8176f53ed947370210d0f596815 ran Product Performance Attestation Quality CI 34967146861; Python 3.10 compile and hash-verified dependency installation passed, then the exact-head contract suite failed 1/87 (86 passed, 1 failed) in test_reusable_workflow_authenticates_and_records_caller_workflow_identity. RCA: the test oracle still required direct serialization from arguments.caller_workflow_ref / arguments.caller_workflow_sha, while production verifier intentionally validates those inputs through _require_authenticated_caller_workflow(arguments) and writes only the returned caller_ref / caller_sha to predicate and manifest. Current #2166 exact db4962ac1a39c5cd4e99b7ce95c153347886a9ed changes only that test oracle to require the validation call plus both validated predicate/manifest assignments. Production verifier/workflow, 100% coverage gate and security gates are unchanged. Fresh quality/security runs 34999775545 / 34999775739 / 34999775820 / 34999775651 are queued; GREEN is not claimed until terminal exact-head evidence exists.

OriginWeave downstream performance-attestation canary currentization.

Consumer PR ContextualWisdomLab/OriginWeave#327 is now exact 0d3911e558188db805a0affbe50e53ed02b481d0, 46 ahead / 0 behind on canonical HTTP parent #37. Its product-owned packager still emits the three inert root-level artifacts required for central intake (content-coding-result.json, content-coding-runtime.json, content-coding-fixture.json) with exact candidate identity and SHA-256 file identities; it does not consume mutable #2166 source.

The fixture artifact now also preserves an explicit product-side acceptance boundary: fixture_authority=deterministic_synthetic_no_external_dataset and fixture_acceptance_status=UNACCEPTED_SYNTHETIC_FIXTURE. The packager rejects a receipt that relabels that synthetic fixture as accepted. This is intentional: central attestation may authenticate exact bytes/origin/integrity, but it must not convert an OriginWeave synthetic regression fixture into a commercial buyer-path fixture or decide the p95 policy.

Acceptance canary after this owner normally integrates/releases remains: attest the exact product-owned result/runtime/fixture bytes under the released/pinned central contract; preserve exact candidate/profile/digests and fixture authority; then let an OriginWeave-owned post-attestation verifier decide product acceptance. Network freshness and realistic-fixture acceptance remain separate OriginWeave prerequisites. No mutable #2166 head is a downstream dependency.

Navigation Menu

Repository navigation

perf: attest exact product performance evidence before commercial acceptance #2162

Owner-boundary defect

Required trust boundary

Cross-repository prerequisite

RED evidence

GREEN acceptance

Primary references

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

perf: attest exact product performance evidence before commercial acceptance #2162

Description

Owner-boundary defect

Required trust boundary

Cross-repository prerequisite

RED evidence

GREEN acceptance

Primary references

Activity

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 13, 2026

seonghobae commented on Sep 15, 2026

seonghobae commented on Sep 15, 2026

seonghobae commented on Sep 15, 2026

seonghobae commented on Sep 24, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions