ci: run AgentMeasure conformance on the two urusilla vectors

[roy-tong]

This just turns the two fixtures you've already been running manually into a push/PR check — no new semantics.

What it does: on every push and PR, the AgentMeasure conformance Action runs against agentmeasure_urusilla_fixture_001.events.jsonl and _002, reporting PASS / FAIL / UNPROVABLE per invariant (execution grain, retry reconciliation, cost preservation, operation grain, evidence boundary). Vector 001 is the retry/cache boundary that caught #8/#9; vector 002 is the multi-operation shape that caught the grain-mixing defect (AM-U-007) — so this also pins both of those fixes permanently against your inputs.

If the checks aren't useful here, close freely — no hard feelings either way. And thank you again; a repeat external contributor who found three real defects is the best thing that has happened to this project.

[jaden3824]

Thank you for turning the two existing fixtures into a bounded CI proposal and for pinning AgentMeasure to a full commit SHA.

Before this workflow runs in the repository, please match the repository's existing workflow hardening:

1. Add top-level permissions: contents: read.
2. Replace actions/checkout@v4 with the repository-pinned actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.

I can verify the two fixture paths exist and that the AgentMeasure action is pinned, but there is currently no visible agentmeasure-conformance check result on this PR, so the fixture verdict remains unverified until the workflow actually runs. This review is about CI safety and observability only; it does not treat the PR as adoption, independent reproduction, or performance evidence.

[roy-tong]

Both hardening changes are in: permissions: contents: read at the workflow level, and actions/checkout pinned to a commit SHA (3a1661c).

One note since this workflow is newly introduced from a fork: GitHub may require maintainer approval before workflow runs execute on fork PRs. Once approved, the two fixture verdicts should become visible here — vector 001 is the retry/cache boundary that caught #8/#9, vector 002 is the multi-operation grain case (AM-U-007), so both should report PASS/UNPROVABLE per invariant rather than silently passing.

[jaden3824]

Thanks — the permissions scope and checkout pin are correct. However, commit 3a1661c shortened the AgentMeasure reference from the full SHA to @2cf476d. Please restore @2cf476d6f7d0fc45401db5a822e1f12de009ac74. I’ll approve the workflow after that.

[jaden3824]

One additional bounded audit before workflow approval: I ran the exact full AgentMeasure commit in a sanitized offline environment against both unchanged fixtures. Both returned exit 0 and PASS for the five currently supported invariants: execution-grain, retry-reconciliation, cost-preservation, operation-grain, and evidence-boundary.

Please make these final small changes:

1. Restore the full pin: roy-tong/AgentMeasure@2cf476d6f7d0fc45401db5a822e1f12de009ac74.
2. Add require: execution-grain,retry-reconciliation,cost-preservation,operation-grain,evidence-boundary to both AgentMeasure steps, so a future UNPROVABLE result cannot remain green.
3. As low-cost hardening, add persist-credentials: false to the checkout step.

Scope note: this action checks the generic AgentMeasure invariants only. token-subset and cache-distinction are currently NOT-SUPPORTED, and the action does not evaluate the x_urusilla sidecar or the fixture expected.json contracts. This PR therefore complements rather than replaces the existing fixture-specific validation. After the full pin and required-invariant settings are present, I can approve the workflow run.

[roy-tong]

All three items from the audit are in the branch head as of 9d85ecb:

• full pin restored: roy-tong/AgentMeasure@2cf476d6f7d0fc45401db5a822e1f12de009ac74
• require: execution-grain,retry-reconciliation,cost-preservation,operation-grain,evidence-boundary on both AgentMeasure steps, so a future UNPROVABLE cannot stay green
• persist-credentials: false on the pinned checkout (workflow-level permissions: contents: read unchanged)

Ready for re-review whenever convenient; no rush on the workflow approval.

[roy-tong]

Root cause found and fixed: the run was failing during action packaging, not in the conformance checks themselves. Our repo carried a dangling symlink (reference/adapters/deepseek-harness/node_modules/@deepseek-ai) that broke the staging step when the repo is consumed as an action — our own CI never hits that path because it checks out and runs scripts directly.

Upstream fix: roy-tong/AgentMeasure@423c999 (symlink removed). I've bumped the pin in this branch's workflow to the fixed SHA, which re-triggered both workflows — they now show as action_required waiting for maintainer approval to run from the fork branch. Could you approve the workflow runs? After that the conformance job should execute for real, and we can see the two urusilla vectors get checked in your CI.

Thanks for wiring this up — an external repo running the conformance pack in its own CI is exactly the adoption path the pack was built for.