Skip to content

fix(pixel): count a passing inspection of the snapshot awaiting its byte check - #6726

Open
Lightheartdevs wants to merge 3 commits into
mainfrom
claude/pixel-preview-receipt-binding
Open

Lightheartdevs wants to merge 3 commits into
mainfrom
claude/pixel-preview-receipt-binding

Conversation

@Lightheartdevs

@Lightheartdevs Lightheartdevs commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

What went wrong

Tower2 round 092 (main 04f0a835, qwen3-coder-next, OpenClaw 2026.6.33, session afc733bc), website-create. After its last publication (site-05f1589d…) the model:

  1. ran cd /workspace && grep -n "midnight-concert-card" …/index.html (read-only),
  2. ran a show/hide transition inspection of that same snapshot, which passed,
  3. ran three more read-only greps and gave its answer.

Nothing on disk changed, but the answer was replaced by "The requested show/hide interaction has not passed browser inspection."

Why

Any settled shell command after publishing takes the preview out of "current" until the host re-checks the published bytes at the end of the turn. The inspection receipt was matched only against the current preview. So in that window the passing receipt matched nothing and was dropped. At the end of the turn the byte check found no change and made the same snapshot current again, but the dropped pass was gone. The other order (inspect first, then grep) already worked, because the receipt arrived while the preview was still current.

Fix

While the preview is out of currency only because the end-of-turn byte check is pending, an inspection receipt now binds to that pending snapshot. The pending snapshot must belong to the same session, session key and workspace root. The final gate does not change: the proof counts only when that exact snapshot is current again (the host digest matched, or identical bytes were republished).

The fix is one helper, inspectionPreview(state), in tool-loop-guard.mjs. The proof, static-retention and page-error bindings now use it instead of state.workspacePreview.

Still not verified:

  • the files changed on disk, so the byte check fails and the preview stays stale
  • the receipt names another snapshot, fails, uses a static-only plan, or comes from a foreign session
  • the pending snapshot was dropped (an ineligible call such as sleep 1 &, or explicit bundle invalidation), or the session id, session key or workspace root changed. A receipt that arrives after that binds nothing, even if identical bytes are republished afterwards. (A receipt that bound before the drop is tied to that snapshot's hash, so it still counts if identical bytes are republished. That is correct: it tested those exact bytes.)

For the pending snapshot, a static re-check keeps an earlier transition proof and page errors select the repair step. This is what already happens for the current snapshot.

Evidence

  • Recorded fleet replay: tests/post-publication-tower2-round092.json holds the 23 recorded calls (exact arguments; results as the tools returned them, with the guard's appended guidance and host cwd removed). post_publication_currency.test.mjs replays them against a temporary workspace and the real host/workspace_preview.py. Both publications reproduce the recorded snapshot digests. On main the replay ends failed, the same outcome as the fleet run. On this branch it ends passed and delivers the model's answer with the preview link.
  • Unit tests: 22 R092: tests in preview_interaction_assurance.test.mjs. On main aa0623f4, 7 of them fail: every positive case (grep-then-inspect and inspect-across-grep, direct and Tool Search; static retention; page-error repair step; an inspection that settles after a parallel write of identical bytes). All 22 pass on this branch.
  • Review probes (added in 4ae235f8): an inspection that settles after a parallel write counts only when the byte check finds the published bytes unchanged. A late receipt for an earlier snapshot, settling after a newer publication and a grep, binds nothing. A pass on the pending snapshot does not verify a later publication of different bytes. The last two also pass on main; they guard against widening the binding.
  • Mutation checks: removing each of the session id, session key and workspace root checks fails exactly its own test. Binding to the last verified preview regardless of pending state fails 6 tests, including the existing "bundle output mutation invalidates publication…" test. Making the pending snapshot current as soon as a receipt binds (skipping the byte check) fails 15, including the late-receipt and changed-write probes. The bundle-invalidated case does not exercise the session checks, because invalidation drops the proof and the pending snapshot together; it catches the last-verified mutation.

Tests

  • Full plugin JS suite on tower2 (remote-js-test.sh): main aa0623f4: 3112 tests, 0 failures. This branch 4ae235f8: 3135 tests, 0 failures (23 new). This branch merged with current main 47b36328: 3144 tests, 0 failures.
  • Runtime integration tests (tests/runtime_*.integration.mjs) against the pinned OpenClaw 2026.6.33 copies on tower2 (read-only), run on a5f93846: main and this branch give identical per-test results with both the upstream copy and the ODS-repaired copy (64 tests each). That includes runtime_preview_delivery ("real harness recovers saved HTML without replaying work", 4/4 passing). The 12 failures are identical on main and the branch and come from this environment: the copied runtime lacks json5 (ERR_MODULE_NOT_FOUND). 4ae235f8 changes only a unit-test file.
  • CI: Scan for secrets fails, but not on anything in this PR. The scan also reads commits outside this branch. On 4ae235f8 its one finding is in dashboard/src/lib/applicationPins.js from commit 5977cb1b, which belongs to feat(extensions): Open, How to use and Applications for installed extensions #6731 and is on neither main nor this branch. On a5f93846 its 4 findings were all in dashboard-api/tests/test_host_agent_output_redaction.py, the same file main aa0623f4 failed on. This PR changes four pixel-agent files only, and the new fixture post-publication-tower2-round092.json has no host paths or personal data.

Replay harness changes (post_publication_currency.test.mjs)

(post_publication_currency.test.mjs)

These bring the harness closer to OpenClaw 2026.6.33, so the round 092 calls replay as they ran:

  • after_tool_call now receives the before-hook params merged over the model's arguments (mergeParamsWithApprovalOverrides in the pinned dist). This keeps the recorded exec workdir after the cancellation wrapper.
  • A bare /workspace (cd /workspace && …) maps to the temporary workspace, like /workspace/… already did.
  • A recorded failed edit changes nothing, and the replay checks that it still fails on the replayed bytes.
  • Round 092 replays with the production exec wrapper only. Production always wraps exec (index.js). Its execs pass workdir: "/workspace", and the guard models that merge only on the wrapper path. Rounds 061 and 069 still replay both ways.

Not in this PR

  • Finding (2) from the same round (coding turn with no final answer, caused by OpenClaw terminate:true on a byte-identical write) needs its own fix. This PR changes no runtime-repair JSON, hashes, installer, foreign-restore list, CI wiring or ODS-RUNTIME-REPAIRS.md, and it merges cleanly with fix(pixel): recover owner-turn context overflows by trimming tool output when that fits #6715's repair wiring.
  • After a grep, the tool-result coaching still tells the model to republish. That is unchanged. It is harmless, because republishing identical bytes also counts the earlier pass, but it costs one extra call.

Merge note for #6723 (required, whichever lands second)

This branch has a textual conflict with #6723 on the page-errors line of the inspection-receipt block. Resolving that line alone is not enough. Every place in #6723 that picks the snapshot an inspection belongs to must use the snapshot this PR binds receipts to:

  1. In the inspection-receipt block of afterToolCall, use inspectedPreview instead of state.workspacePreview in all three places: boundHiddenRequestedText(…), withHiddenRequestedText(…), and the siteId / sha256 of state.workspaceRequestedTextEvidence.
  2. In requestedTextsForInspection(params), replace runs.get(pending.runId)?.workspacePreview with inspectionPreview(…) of that run's state (skip it when the run is gone). This function merges without a conflict, so a resolver will not see it.

Why it matters: after a read-only grep there is no current preview. If step 2 is skipped, requestedTextsForInspection returns [], so the inspector is never asked whether the owner's requested text is visible at load, and this PR counts the pass. The turn is then delivered as passed although the requested footer is display:none at load. In fleet order, #6723 alone ends failed for that turn (the show/hide pass is lost after the grep, which is the defect this PR fixes), so the composition would turn a false negative into a false pass. If step 1 misses the evidence line, that line reads state.workspacePreview.siteId while the preview is undefined and throws a TypeError in after_tool_call.

Required test: whichever of #6723 or #6726 lands second adds the composed test below to tests/requested_literals.test.mjs. It publishes, runs a read-only grep, and inspects with the requested footer hidden at load; the turn must fail for the hidden text, and the passing show/hide steps must still count. It runs in fleet order (create run first, so the show/hide duty applies) and with the edit run alone, each with the grep before the inspection and with an inspection that starts before the grep and settles after it. It also makes the test helper return no requestedText block when no texts were sent, as the broker does (preview_inspection_capsule.py adds it only when the request has texts).

Composed on a scratch branch (not pushed): main 47b36328 + this branch + #6723 f424bdf8, with the composed test. Each row was run locally (node 24). "0 texts" means the inspector was not asked about any requested text.

Resolution Fleet order, grep then inspect Fleet order, inspect across grep Edit run only, grep then inspect Edit run only, inspect across grep
#6723 alone (no #6726) failed: show/hide not verified (0 texts) failed: show/hide not verified passed (0 texts) passed
Keep both sides of the conflict only passed (0 texts) passed passed (0 texts) passed
Old note: receipt block only passed (0 texts) failed: hidden text passed (0 texts) failed: hidden text
Evidence line left on state.workspacePreview TypeError TypeError TypeError TypeError
Full note (steps 1 and 2) failed: hidden text failed: hidden text failed: hidden text failed: hidden text

The composed test fails 4 of 4 cases on #6723 alone, 4 on "keep both sides", 2 on the old note and 4 on the TypeError resolution. It passes all 4 with the full note, and that composition passes the whole plugin JS suite on tower2: 3155 tests, 0 failures.

The "edit run only" row shows a gap in #6723 by itself: when a grep comes between publishing and inspecting, and the run has no show/hide duty, the hidden-text check does not run at all. With this PR and the full note, that case fails correctly.

The composition does not change tool-result coaching: while the snapshot awaits its byte check, the inspection result itself does not carry the hidden-text instruction (coaching reads the current preview). The gate still fails the turn, and finalization sends the one revision request.

Composed test (diff against #6723 f424bdf8)
diff --git a/ods/extensions/services/pixel-agent/tests/requested_literals.test.mjs b/ods/extensions/services/pixel-agent/tests/requested_literals.test.mjs
index 5e906fc00..a8625d95d 100644
--- a/ods/extensions/services/pixel-agent/tests/requested_literals.test.mjs
+++ b/ods/extensions/services/pixel-agent/tests/requested_literals.test.mjs
@@ -418,7 +418,7 @@ test('hidden requested text is a snapshot-bound miss with which text, which elem
 const PLAN = [{action: 'assert-hidden', locator: {selector: '#midnight-concert-card'}},
   {action: 'click', locator: {role: 'button', name: 'Show sold out', exact: true}},
   {action: 'assert-visible', locator: {selector: '#midnight-concert-card'}}];
-function inspection(guard, preview, id, texts, runContext) {
+function inspection(guard, preview, id, texts, runContext, beforeResult) {
   const params = {siteId: preview.siteId, sha256: preview.sha256, viewport: {width: 375, height: 812}, steps: PLAN};
   const {event, ctx} = call(guard, PREVIEW_INSPECTION_TOOL, params, id, undefined, runContext);
   const sent = guard.requestedTextsForInspection(structuredClone(params));
@@ -430,9 +430,12 @@ function inspection(guard, preview, id, texts, runContext) {
     steps: PLAN.map((step, index) => ({index, ...step, before: observed(index !== 0), stable: true, status: 'passed',
       ...(step.action === 'click' ? {after: observed(true)} : {})})),
     diagnostics: {renderedHiddenAttributeCount: 0, hiddenUntilFoundCount: 0}, blockedRequests: [],
-    requestedText: {viewport: {width: 1280, height: 720}, scrolled: true, texts: sent.map(text => texts[text] ?? {text, status: 'visible'})},
+    // Like the broker: a request without texts gets no requestedText block.
+    ...(sent.length ? {requestedText: {viewport: {width: 1280, height: 720}, scrolled: true,
+      texts: sent.map(text => texts[text] ?? {text, status: 'visible'})}} : {}),
     scope: INSPECTION_SCOPE};
   const result = {content: [{type: 'text', text: 'Preview inspection passed.'}], details};
+  beforeResult?.();
   guard.afterToolCall({...event, result}, ctx);
   return {sent, note: text(persist(guard, PREVIEW_INSPECTION_TOOL, id, result, runContext))};
 }
@@ -608,3 +611,60 @@ test('inspection evidence carried to a republish: hidden on the same bytes, unve
   assert.deepEqual(withHiddenRequestedText(moved, other, []).missing, []);
   assert.deepEqual(withHiddenRequestedText(moved, other, [HIDDEN_FOOTER]).missing, [{text: HIDDEN_FOOTER.text, hidden: HIDDEN_FOOTER}]);
 });
+
+// #6723 with #6726 (tower2 round 092 + fleet round 087): a settled read-only
+// grep after publishing leaves the snapshot awaiting its end-of-turn byte
+// check. An inspection of that snapshot must still be sent the requested texts,
+// and requested text hidden at load must still fail the turn once the byte
+// check restores the snapshot. In fleet order the create run leaves a
+// show/hide duty, so the same inspection must also count as interaction proof.
+for (const sequence of ['fleet order', 'edit run only'])
+for (const order of ['grep-then-inspect', 'inspect-across-grep'])
+test(`requested text hidden at load still fails after a read-only grep (${sequence}, ${order})`, async t => {
+  const revised = 'Night Garden FLEET-068aa1af7e Revised';
+  const original = {'index.html': eventPage({dawn: 'Dawn jazz'}), 'script.js': SCRIPT};
+  const root = workspace(original);
+  t.after(() => fs.rmSync(root, {recursive: true, force: true}));
+  let byteChecks = 0;
+  const guard = createToolLoopGuard({workspacePreviewInspectionAvailable: true,
+    verifyWorkspacePreview: async () => { byteChecks++; return true; }});
+  if (sequence === 'fleet order') {
+    guard.observeRun(context, 'pixel', {prompt: FLEET_CREATE}, {workspaceRoot: root});
+    inspection(guard, publishSite(guard, original, 'create-publish'), 'create-inspect', {});
+  }
+  const edit = {...context, runId: 'edit'};
+  guard.observeRun(edit, 'pixel', {prompt: FLEET_EDIT}, {workspaceRoot: root});
+  const indexPath = `${DIRECTORY}/index.html`;
+  const read = {content: [{type: 'text', text: original['index.html']}]};
+  call(guard, 'read', {path: indexPath}, 'edit-read', read, edit);
+  persist(guard, 'read', 'edit-read', read, edit);
+  const hiddenPage = eventPage({dawn: 'Dawn jazz', title: revised}).replace('<h3 class="event-title">Midnight Sold-Out Concert</h3>',
+    `<h3 class="event-title">Midnight Sold-Out Concert</h3><footer class="site-footer"><p>${FOOTER}</p></footer>`);
+  call(guard, 'write', {path: indexPath, content: hiddenPage}, 'edit-write', written, edit);
+  persist(guard, 'write', 'edit-write', written, edit);
+  const hidden = snapshot({...original, 'index.html': hiddenPage});
+  call(guard, 'pixel_ods_workspace_preview', {relativeDirectory: DIRECTORY}, 'edit-publish', {details: hidden}, edit);
+  persist(guard, 'pixel_ods_workspace_preview', 'edit-publish', {content: [{type: 'text', text: 'published'}], details: hidden}, edit);
+  const grepResult = {content: [{type: 'text', text: `1: ${FOOTER}`}], details: {status: 'completed', exitCode: 0}};
+  const grep = () => {
+    call(guard, 'exec', {command: `cd /workspace && grep -n "edited successfully" ${indexPath}`, workdir: '/workspace'},
+      'grep', grepResult, edit);
+    persist(guard, 'exec', 'grep', grepResult, edit);
+  };
+  const entry = {text: FOOTER, status: 'hidden', element: 'p', reason: 'display-none', culprit: 'article#midnight-concert-card'};
+  if (order === 'grep-then-inspect') grep();
+  // inspect-across-grep: the inspector runs while the snapshot is current and
+  // its result settles after the grep.
+  const {sent} = inspection(guard, hidden, 'edit-inspect', {[FOOTER]: entry}, edit, order === 'inspect-across-grep' ? grep : undefined);
+  assert.deepEqual(sent, [revised, FOOTER], 'the inspector is asked about the requested texts');
+  assert.notEqual(guard.verificationForRun('edit').status, 'passed');
+  assert.equal(await guard.revalidateWorkspacePreview({}, edit), true, 'the byte check restores the same snapshot');
+  assert.equal(byteChecks, 1);
+  const outcome = guard.verificationForRun('edit');
+  assert.equal(outcome.preview.sha256, hidden.sha256);
+  assert.equal(outcome.status, 'failed', 'the requested footer is hidden at load');
+  assert.match(outcome.text, /contains requested text that is not visible when it loads: "FLEET-068aa1af7e edited successfully" in p/);
+  assert.doesNotMatch(outcome.text, /show\/hide interaction has not passed/, 'the passing show/hide steps count');
+  assert.equal(guard.beforeAgentFinalize({}, edit)?.retry?.instruction,
+    REQUESTED_VISIBLE_TEXT_REVISION_INSTRUCTION.join(JSON.stringify([FOOTER])));
+});

Other open PRs

#6715, #6716, #6722, #6724 and #6698 merge cleanly with this branch. #6645 already conflicts with main.

🤖 Generated with Claude Code

Lightheartdevs and others added 2 commits September 25, 2026 15:46
…te check

Tower2 round 092 (main 04f0a83, qwen3-coder-next, session afc733bc):
website-create published site-05f1589d, ran a read-only grep, then ran a
transition inspection of that same snapshot that passed. The final answer was
replaced by "The requested show/hide interaction has not passed browser
inspection" although nothing on disk had changed.

Root cause: a settled later call (here the grep) takes the preview out of
currency until the end-of-turn host byte comparison. An inspection receipt
bound only to the current preview, so while the comparison was pending the
passing receipt bound to nothing and was discarded. The comparison then
restored the unchanged snapshot as current, but the discarded pass was not
recovered. Inspect-then-grep already passed; only grep-then-inspect failed.

Now, while the preview is out of currency only pending that comparison, an
inspection receipt binds to the pending snapshot of the same session, session
key and workspace root. Nothing else changes:
- The proof counts only when that exact snapshot is current again (the host
  digest matched at finalization, or identical bytes were republished).
- A receipt for another snapshot, a failed receipt, a static plan or a
  foreign session still binds nothing; changed bytes leave the preview stale.
- An ineligible call or explicit bundle invalidation drops the pending
  snapshot, so a later receipt binds nothing.
- Static checks retain an earlier transition proof and page errors select the
  repair step for the pending snapshot, as they already did for the current one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…host code

Adds the recorded round 092 calls (exact arguments, results as the tools
returned them) to the post-publication replay. Writes, edits, execs and both
publications run against a temporary workspace and host/workspace_preview.py,
and each publication reproduces the recorded snapshot digest. Before the fix
the replay ends "failed"; with it the model's answer is delivered as verified.

The replay harness now follows OpenClaw 2026.6.33 more closely:
- after_tool_call receives the before-hook params merged over the model's
  arguments (mergeParamsWithApprovalOverrides), so a recorded exec workdir
  survives the cancellation wrapper;
- a bare /workspace path (cd /workspace && ...) maps to the temporary
  workspace like /workspace/...;
- a recorded failed edit changes nothing and must still fail on the replayed
  bytes.
Round 092 replays with the production exec wrapper only: its execs pass
workdir "/workspace", which the guard models on that path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its byte check

Three probes from the #6726 review, next to the round 092 tests:
- An inspection that settles after a parallel write binds to the snapshot
  awaiting its byte check. It counts when the write left the published bytes
  unchanged (the byte check restores the snapshot) and not when it changed
  them. The identical-write case fails on main aa0623f and passes here.
- A late receipt for an earlier snapshot, settling after a newer publication
  and a grep, binds nothing; the newer snapshot stays unverified after its
  byte check.
- A pass on the pending snapshot does not verify a later publication of
  different bytes.
The last two pass on main as well; they guard the binding against widening.
The late-receipt and changed-write cases fail if binding a receipt were to
make the pending snapshot current without the byte check.

awaitingByteCheck() gains publish(), which writes and publishes new bytes with
both results settled, so the end-of-turn byte check can run afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant