Skip to content

fix(pixel): every non-passing preview inspection returns a ready, validated next call or a measured diagnosis - #6748

Open
Lightheartdevs wants to merge 4 commits into
mainfrom
claude/inspect-ready-next-call
Open

Lightheartdevs wants to merge 4 commits into
mainfrom
claude/inspect-ready-next-call

Conversation

@Lightheartdevs

@Lightheartdevs Lightheartdevs commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

OpenClaw 2026.6.33 drops before_agent_finalize revisions after a plugin tool call, so an inspection fix can reach the model only in the tool result. On main d4a61f3 (fleet round 107), website_create_artifact_preview_interaction failed on laptop, strixy and mac-mini, and the failure cascaded into website_followup_edit and conversation_reload_continuity. All three towers passed.

Every failing host used up the progress budget (4 consecutive or 12 total failures) on the inspector itself:

  • strixy r107: flattened steps were rejected with "call tool_describe ... then retry through tool_call". The model then called tool_call {id:"tool_describe"} twice and was stopped.
  • laptop r107:
    • Call 4 had no exact:true.
    • Call 5 used a state-qualified locator on a page that never hid the card.
    • The model then read the receipt's before as a pre-script snapshot, and later sent {}.
  • mac r106 ran into four locator failures on a page that works.
  • mac r107 said a broken click handler was "a timing issue".

Evidence is in tower2 ods-main-qualification-20260924-r049, round-10{5,6,7}.

Change

Every non-passing inspection result now names exactly one of these:

  • one ready, validated call on the same snapshot, when the site needs no change;
  • a measured diagnosis with one repair, plus the steps to run on the new snapshot;
  • or, when this failure ends the response's tool use, no call at all.

isError stays true. details stays the capsule receipt, or the invalid_request shape. The Evidence JSON, the palette line, page-error precedence and the INCOMPLETE (#6740) path are unchanged. Proof binding is unchanged.

Rejected arguments (invalid_request)

  • The text names the step, the key and the reason.
  • A lossless repair is offered as "call ... with exactly these args". It moves a flattened selector or role/name into locator, and adds exact:true to a role/name locator.
  • If the repaired steps do not test a change and an owner requirement is bound, the requirement plan is offered instead. For laptop call 4, that plan asserts .event-card.sold-out hidden, clicks, then asserts it visible.
  • Otherwise the text shows one placeholder shape example.
  • The tool_describe sentence is gone. The guard's coaching now says tool_call takes the tool's id directly, which is how OpenClaw 2026.6.33 tool-search findEntry works.

Locator failures (no_match, selector_not_unique, invalid_selector)

  • The owner-requirement plan is offered, and never reuses the failing locator.
  • Failure paths prefer a CSS target for the owner-named item over a role/name heading. That target never contains another heading or the control. It is measured whether or not the element is rendered, so a broken toggle gets a measurement instead of a no_match dead end (laptop call 13, site-c54d9d87).
  • A role/name click that matched nothing gets the published CSS locator of the control whose authored name matches ignoring case. The outline now records control roles and names. This covers laptop call 24, a visible text-transform: uppercase button.
  • fix(pixel): verify owner-requested exact control names after the page scripts run #6741's load-time names take precedence:

Measured failures (visibility_mismatch, click_failed, unstable)

  • The text states the step's measured display, visibility, opacity and rectCount, which clicks ran before it, and the inspector facts.
  • The next step depends on the owner's stated direction. That is statedVisibilityDirection: an initially-hidden phrase or a click that only reveals means hidden first, and a click that only hides means visible first. A toggle, both or neither leaves it unstated.
    • Stated, and the page contradicts it: repair.
    • Stated, and only the plan's order is reversed: the owner-order plan on the same snapshot, with "the site needs no change".
    • Not stated: both orders are offered, and the owner is never said to have asked for one.
  • After a click:
    • If passing earlier steps already measured one locator hidden, then clicked, then visible, and the failing step checks another element, those passing steps are offered. This is tower2 r107 call 20, a passing session that previously would have been told to repair a working site.
    • If the failing step is another element, the target plan is offered.
    • If the element changed where the plan expected no change, the corrected step is offered.
    • If the target did not change, repair.
  • After a repair, the steps for the new snapshot are the target plan without identifiers. They never reuse "the same steps". The target is an id, or the item's other classes when the class the repair may change is the locator (laptop call 40).

Final failure

  • The guard's previewInspectionGuidance(toolCallId, params) is bound to exactly the pending call, like previewInspectionTransition. It carries the stated direction and finalFailure from the read-only budget.failureEnds(lane).
  • On the failure that exhausts the budget or its lane, every next step is replaced by "do not call any tool; answer the owner now". The progress-finalization pending phase accepts that answer.

Review fixes (round 1, d775fbd)

Finding Fix
Before-click text treated the hidden-first default as the owner's words (P4, P5, P9) Stated direction only; neutral otherwise; the owner-order plan when only the plan is reversed
Post-click repair of a working site (tower2 r107 #20) Proven-prefix and off-target branches offer ready calls; repair only for an unchanged target
Laptop r107 not rescued Requirement plan for call 4; a CSS control for role/name click misses; CSS item targets; repair-stable steps
Heading dead end on a broken toggle CSS item target; the role/name text after "matched hidden before the click" names both causes
"The same steps" defeat themselves Target-plan steps with a repair-stable target
Ready args at the budget limit finalFailure replaces the next step
No ready call when hidden at load Owner-order or page-order plan on the same snapshot
Coaching still said "call tool_describe" Reworded

Replays of rounds 105 to 107 through the real guard and real Chromium:

  • laptop r107 (counterfactual, following each offered call): call 4 → requirement plan → "visible at load, owner has it hidden" repair with its steps → after the repair, the role/name click miss gets the #showSoldOutBtn plan → passes on the repaired page. Three failures, with a reset in between.
  • tower2 r107: call 20 → the passing steps 1–7 pass and verification passes.
  • strixy r107 and mac r106: unchanged ready calls. The final failure now says to answer.
  • mac r107: the repair steps (heading around .reveal-btn) pass once the handler toggles the class.

Review fixes (round 2, f67b0ab)

Finding Fix
A passing change of another element was offered in place of the owner's target. On a page whose click reveals an unrelated #note but leaves the requested #midnight-card hidden, the plan #note hidden, click, #note visible, #midnight-card visible failed at step 4 and was offered steps 1 to 3. Sending them passed, bound the visibility proof and set the run's verification to passed (reviewer probes P1 and P1c, direct and tool_call). The guard's own instruction says not to substitute an unrelated passing interaction. With a target named from the published source, the passing steps are offered only when they changed that target, and never when the failing step is on the target itself. That step then gets its measured repair. A failing step on another element gets the target plan, which measures the real defect.
  • Tower2 r107 call 20 keeps its ready call (steps 1 to 7). This was checked with the PR's fixture and also with the guard's outline, rebuilt from that session's own writes and edits (digest-verified against publication 19's sha256).
  • New test in inspection_recovery_replay.test.mjs: "another element's passing change is never offered in place of the owner's target". It fails on 447fd80 and passes here.

Composition notes (for whichever of these lands second)

Merge with main (#6741)

Main moved: #6741 added load-time control names. This branch merges 6cba572. The three conflicts are resolved as described in the merge commit:

The branch merges cleanly with current main 321f82a (#6746, playground paths only).

Not changed, and why

  • The proof accepts a transition in either direction (boundVisibilityInspection). This behavior predates this PR. Gating proof on direction would rest on the wording heuristic, and "Clicking shows it; clicking again hides it" already derives visible-first, so correct toggles would fail. This PR only removes the steering: it never offers a plan reversed against a stated direction.
  • Heading targets still appear when no CSS item target exists. After a heading matched hidden and then nothing rendered after the click, only a CSS target is offered. Otherwise the text names both causes and offers no call.

Deliberate test edits (owner sign-off)

  • inspection_transition_coverage.test.mjs: one assertion (from round 1). The tower2 r102 create-turn no_match text is kept byte for byte, with one ready call inserted. This is the same pattern as laptop call 5. Nothing else in that file changed.
  • preview_interaction_assurance.test.mjs: the finalize coaching assertion /tool_describe/ is inverted, and the new tool_call sentence is asserted.

Tests

  • Full Linux suite (remote-js-test.sh, tower2):

    Ref tests pass fail
    Round-2 head f67b0ab 3209 3171 0
    Round-2 head composed with main 0a63258 (local, not pushed) 3237 3199 0
    Merged head 447fd80 3208 3170 0
    Main control 6cba572 3177 3139 0
    Composed with current main 321f82a (local, not pushed) 3223 3185 0
    Round-1 head d775fbd 3168 3130 0
  • tower2 Linux:

    • node --test ods/tests/test-preview-inspection.mjs: 30/30.
    • python3 -m unittest tests/test_preview_inspection.py: OK (89 tests, 46 opt-in skipped).
  • Opt-in real Chromium (ODS_PREVIEW_BROWSER_TESTS=1): all 39 BrowserTests pass, including fix(pixel): verify owner-requested exact control names after the page scripts run #6741's. They ran locally on Windows, with the Playwright headless shell and a scratch shim for os.killpg. tower2 has no Python Playwright. The new tests cover:

    • the uppercase control needing its CSS locator;
    • the CSS item target measuring the broken toggle on site-c54d9d87;
    • call 24's id click reaching the measurement;
    • the mac r107 repair steps failing on the recorded page and passing after the repair.
  • New or updated JS tests:

    • inspection_guidance.test.mjs: stated direction for exactly the pending call, inheritance on a preserve turn, toggle, finalFailure, and outline control names.
    • inspection_recovery_replay.test.mjs: laptop calls 4, 5, 13, 24 and 26; strixy at the limit; tower2 r107 calls 20 and 21; mac r107 with and without the requirement; P4, P5 and P9; and load-time names with the published control.
    • run_progress_budget.test.mjs: failureEnds.
    • ods/tests/test-preview-inspection.mjs: direction-aware texts, and the final failure across every failing path.
    • New fixture inspection-recovery-tower2-round107.json, byte for byte from J/round-107/tower2/serving-logs/openclaw-pixel-a674292a-….jsonl.gz.
    • Laptop fixture: calls 13 and 24 added.

🤖 Generated with Claude Code

…ll or a measured diagnosis

OpenClaw 2026.6.33 drops finalize revisions after a plugin tool call, so the
inspection result is the only place a fix can reach the model. On main
d4a61f3 (fleet round 107) the website journey failed on laptop, strixy and
mac-mini while all three towers passed, and each failure ran out of the
four-consecutive-failure budget on the inspector itself:

- invalid_request named no step, key or args and told the model to "call
  tool_describe ... then retry through tool_call"; strixy obeyed it as
  tool_call {id:"tool_describe"} twice after flattened steps, laptop omitted
  exact:true and later sent {}.
- no_match / invalid_selector gave generic locator advice; mac round 106
  hit the fuse on a working page where the requirement-derived plan passes.
- visibility_mismatch gave one generic sentence; laptop and mac read the
  receipt's "before" as a pre-script snapshot and called a real click-handler
  bug a timing issue.

Now:

- invalid_request names the failing step and key. A lossless repair
  (flattened selector or role/name moved into locator; exact:true on a
  role/name locator) is validated and offered as exactly-these-args with the
  request's own identifiers. Otherwise, on a valid snapshot binding, the
  guard-bound owner requirement supplies the #6740 plan over the valid steps;
  else one placeholder shape example. The tool_describe/tool_call sentence is
  gone; the unsupported-role error lists the roles.
- A failed receipt whose failing step is a locator failure consults the
  bound requirement and appends a ready plan that never reuses the failing
  locator and never equals the failing plan (chooseTransitionTarget gains an
  avoid option). A role/name locator that matched nothing for assert-visible
  or click may be a hidden element and keeps its existing feedback.
  [class="..."] selectors get the whole-string explanation.
- visibility_mismatch, click_failed and unstable get the step's measured
  display, visibility, opacity and rectCount, the earlier clicks, fixed
  inspector facts (scripts run, real click, "before" is that step's
  measurement) and one repair-and-republish step; no args.

Details stay the capsule receipt (or the invalid_request shape) and isError
stays true; the INCOMPLETE path, page-error precedence, Evidence JSON and
palette line are unchanged. One assertion in
inspection_transition_coverage.test.mjs that required a tower2 round 102
locator failure to stay byte-identical now checks the recorded text is kept
with one ready call added.

Tests: unit tests for the argument repair and rejection text; replays of
strixy r107, laptop r107 (through createToolLoopGuard), mac r106 and mac
r107 from recorded bytes and receipts; opt-in real-Chromium replays in
ods/tests/test_preview_inspection.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lightheartdevs and others added 2 commits September 25, 2026 23:25
…nd the measured target

Review of PR #6748 found that the new failure texts could steer a model the
wrong way. Replays of rounds 105 to 107 through the real guard and real
Chromium showed where:

- The text before any click treated requestedVisibilityTransition's
  hidden-first default as the owner's words. It told hide-on-click owners to
  hide the element as the page loads, and said "the owner asked for it hidden
  initially" for plain toggles.
- After a click, any failing step got "repair the source". On tower2 round 107
  (a passing session) that included the model's own extra check of the renamed
  "Hide sold out" button, after steps 4, 6 and 7 had already measured the card
  hidden, then clicked, then visible.
- "Inspect the new snapshot with the same steps" kept state-qualified locators
  that the suggested repair removes; laptop round 107 call 40 failed that way.
- Laptop round 107 was not rescued. Its first ready args kept a step that could
  not pass. A visible button styled text-transform: uppercase was called hidden.
  A role/name heading target turned a broken toggle into a no_match dead end.
- A ready call was offered on the failure that exhausts the progress budget,
  where the next call is always refused.

Changes:

- statedVisibilityDirection (preview-interaction-assurance.mjs) returns
  'hidden' for an initially-hidden phrase or a click that only reveals, and
  'visible' for a click that only hides. A toggle, both or neither gives
  undefined. requestedVisibilityTransition is unchanged.
- The guard keeps the stated direction with the intent, and a preserve turn
  inherits it. The new previewInspectionGuidance(toolCallId, params) returns
  it for exactly the pending call, together with finalFailure from the
  read-only budget.failureEnds(lane). index.js passes it to the tool.
  Guidance is wording only: proof, the budget and the requirement are
  unchanged.
- Before any click:
  - Stated and the page is wrong: repair.
  - Stated and the plan is reversed: the owner-order plan on the same
    snapshot.
  - Not stated: both orders, and the owner is not named.
- After a click:
  - Passing earlier steps already measured one locator hidden, then clicked,
    then visible, and the failing step is another element: those steps are
    offered.
  - The failing step is another element: the target plan is offered.
  - The element changed where the plan expected no change: the corrected
    step is offered.
  - The target did not change: repair.
- Steps after a repair are the target plan (an id, or an item CSS locator
  that avoids the class the repair may change), never "the same steps".
- A lossless argument repair that tests no change gives way to the owner's
  requirement plan.
- A role/name click that matched no rendered control:
  - The text names text-transform and earlier renames as causes.
  - The control's published CSS locator is offered, from control names now
    recorded in the outline.
- Failure paths prefer a CSS item target over a role/name heading. The target
  is bounded below any element that holds another heading or the control. A
  heading matched hidden and then unrendered after the click is offered only
  a CSS target.
- On the failure that ends tool use, every next step is replaced by: do not
  call any tool, answer now.
- The coaching no longer says "call tool_describe"; tool_call takes the tool's
  id directly (OpenClaw 2026.6.33 tool-search findEntry).

Tests:

- Guard-level guidance tests: stated direction, preserve-turn inheritance,
  toggle, finalFailure, outline control names.
- Replays: laptop round 107 calls 4, 5, 13, 24 and 26, strixy round 107 at the
  limit, tower2 round 107 calls 20 and 21, and mac round 107 with and without
  the requirement.
- Reviewer probes P4, P5 and P9.
- failureEnds unit test.
- Opt-in Chromium BrowserTests: the uppercase control, the CSS item target on
  site-c54d9d87, the call-24 id click, and the mac round 107 repair steps.

inspection_transition_coverage.test.mjs is not modified further.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…call

PR #6741 added load-time control names to the capsule receipt and
locator feedback built from them (presentName, nameDiagnosis). It
conflicted with this branch in three files.

- workspace-preview-inspect.mjs:
  - locatorFeedback keeps #6741's texts, split into diagnosis and next step
    so a final failure can replace the next step.
  - When the load-time names show the requested name on a rendered control,
    the same steps are offered with that control's published CSS locator.
    #6741 already names that locator as the fix.
  - A name that differs keeps #6741's repair-or-rename text with no ready
    call.
  - Only an unexplained role/name click miss adds the rename causes
    (text-transform, an earlier click). The generic text #6741's tests pin
    is kept.
  - The evidence copy omits controls, as in #6741.
- tool-loop-guard.mjs: both import additions.
- test_preview_inspection.py: both BrowserTests blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the owner's target

measuredFeedback offered an earlier passing transition of any locator as
the ready call when a later step failed. On a page whose click reveals an
unrelated element but leaves the owner's requested card hidden, that call
passed, bound the visibility proof and verified the run (reviewer probe
P1/P1c of #6748). The guard's own instruction says not to substitute an
unrelated passing interaction.

With a target named from the published source, the passing prefix is
offered only when it changed that target, and never when the failing step
is on the target itself, which is then its measured defect. Tower2 round
107 call 20 (the prefix changed the target) keeps its ready call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant