feat(contracts): inject imageQaSample hook into oc_assert runtime#1452
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
fb7546d to
7a20a21
Compare
…1445) Adds a host-mediated image_qa leaf assertion to the outcome-contract DSL: validator (ReDoS-guarded at parse time), evaluator (optional ctx.imageQaSample hook), evaluate dispatch, and oc_assert expectedActualFor handling. No server-side LLM; deterministic passed:false fallback when the hook/screenshot is absent (SSOT #1359 P2/P4). Build + 114 contract tests pass. Runtime wiring + verdict=inconclusive translation + defense-in-depth ReDoS guard on the live path land in the follow-up #1452.
55af8f6 to
56161cd
Compare
Completes the runtime wire-up promised in the original #1432-B brief that was deferred from PR #1445. The DSL clause and evaluator landed in Part 2, but EvalContext.imageQaSample was never populated, so every image_qa contract evaluated to inconclusive in production. Changes: - Extract the sampling forwarding from the image_qa tool handler into a reusable `runImageQaSampling(ctx, params)` export. - oc_assert.buildEvalContext now accepts the calling ToolContext and injects an imageQaSample closure that delegates to runImageQaSampling, threading the host's sampling capability + requestClient bridge straight through. - oc_assert handler signature accepts the optional context param. - The image_qa evaluator now stamps `details.error` (in addition to `details.reason`) on infra-fault paths so oc_assert's isInconclusive check translates them to verdict='inconclusive' instead of 'fail'. Runtime wiring problems are not contract failures. SSOT (#1359) alignment: no server-side LLM ever — the closure only forwards via sampling/createMessage, and falls back to unsupported_by_host when the host lacks the capability. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
56161cd to
52863aa
Compare
Analysis & merge summary (automated review)Intent / direction. The runtime wire-up that activates the #1359 (SSOT) alignment — consistent. The only outbound path is host-mediated MCP sampling — no Fixes folded in during review (2 independent code-review passes):
Changes to reach merge-readiness. Rebased onto Verification. Merged to |
Summary
Stacked on #1445. Completes the runtime wire-up promised in the original #1432-B brief that was deferred from that PR.
image_qa.tsexports a reusablerunImageQaSampling(ctx, params)that does the sampling forwarding.oc_assert.buildEvalContextnow accepts the callingToolContextand injects animageQaSampleclosure delegating torunImageQaSampling.oc_asserthandler signature accepts the optional context parameter.image_qaevaluator stampsdetails.erroron infra-fault paths soisInconclusivetranslates them toverdict='inconclusive'(notfail). Runtime wiring problems are not contract failures.Why this is a fix not a feature
Without this, the image_qa DSL clause was dead code in production: every contract returned
inconclusivewith reasonhost_runtime_did_not_wire_imageQaSample. PR #1445 implicitly required this wire-up to keep its commitment.SSOT (#1359) alignment
sampling/createMessage, falls back tounsupported_by_host.Test plan
tests/contracts/oc-assert-image-qa-runtime.test.ts— 4/4 pass (pass on match, fail on mismatch, inconclusive without sampling cap, inconclusive without screenshot).tests/contracts/image-qa.test.ts— 9/9 still pass.tests/tools/image-qa.test.ts— 6/6 still pass.Stacked on #1445 → #1441.
Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com