Skip to content

fix(pixel): an empty search is an answer, not a failure - #7701

Merged
Lightheartdevs merged 4 commits into
mainfrom
fix/pixel-empty-search-not-a-failure
Oct 16, 2026
Merged

Lightheartdevs merged 4 commits into
mainfrom
fix/pixel-empty-search-not-a-failure

Conversation

@Lightheartdevs

Copy link
Copy Markdown
Collaborator

Problem

The run-progress budget charges every non-zero exit as a failure, and its 12-failure total never resets. grep, rg and git grep exit 1 when nothing matched. A coding run that searches for a name, learns "not here" and looks elsewhere spends that total on answers, and the guard then stops the run mid-task.

Seen on an RTX Spark laptop running Portal with Qwen3.8-Flash-Next, fixing bugs in a canvas browser game:

  • Round 1: 289 model calls, 21 good edits, stopped by the guard. The last three counted failures were greps for a function in the wrong file (exit 1, no output). Portal showed each one as "Failed".
  • Round 2 (fresh task): 268 calls, 12 edits, stopped again mid-diagnosis. Five of its counted failures were searches that found nothing; four of those were short read-only chains such as sed -n '860,900p' ui.js; echo ===; grep -n "function showScreen" -A 15 ui.js.
  • Round 3: 176 calls, 17 edits, stopped the same way.

Fix

lookupFoundNothing() decides whether an exec receipt is an empty search:

  • Single lookups: a grep, egrep, fgrep, rg, git grep or which (optionally after one literal cd DIR &&) that completed with exit 1. These exit 1 only when nothing matched (which: a name did not resolve); their errors exit 2 (git 128).
  • Chains: a list joined by ; or && in which every step is a lookup, an echo or sed -n 'N,Mp', optionally piped into head, tail, cat, wc, sort or uniq, and the last step is a bare lookup. None of those other steps can exit 1, so exit 1 still means a search found nothing.
  • Output: OpenClaw merges stdout and stderr; any error line disqualifies the command (GNU grep's warnings and binary-match notices are not errors). rg 13 prints some errors without a prefix, so rg may only be the last step, where its exit status decides.
  • Parsing: the command is scanned word by word with quotes removed, as dash would see it. ||, a lone &, redirection, grouping, substitution, comments, escapes outside quotes, $ or backticks inside double quotes and newlines all disqualify it. Options that run programs or write files are rejected however they are quoted: rg --pre/--hostname-bin, git grep -O/--open-files-in-pager, sort -o/-c, and any sed operand that starts with -.

The receipt must be bound to the exact pending call: run, transport (direct or Tool Search), session and deep-equal executed params. The model's own command comes from that call's record, never from the result. A match goes through the existing free-correction path as discovery: not charged, resets nothing, at most FREE_EMPTY_LOOKUPS_PER_RUN (12) per run, after which it is an ordinary failure again. Discovery never resets the rounds fuse, so a run that only searches still stops after roundsWithoutProgress. RUN_PROGRESS_LIMITS is unchanged.

Tests

  • tests/empty_lookup_progress.test.mjs (new) replays OpenClaw's hook sequence (before, after, persist) for each call, in both the direct and the Tool Search transport. It checks that:
    • empty searches and empty search chains between edits no longer stop a run;
    • the allowance is bounded;
    • real errors, chains that printed an error, and compound commands still trip the fuse;
    • an empty search never erases earlier failures;
    • a search-only run still stops on the rounds fuse;
    • a receipt that does not match the recorded call is charged.
  • tests/run_progress_budget.test.mjs has table tests over the chains from the Spark run plus dozens of rejected commands and receipts: rg 13's unprefixed errors, quoted -i to sed, sort -o, git grep -O'cmd', rg --pre, scanner edge cases and more.
  • Mutation checks: with the classifier reverted to each earlier version, the new tests fail.
  • The change went through two rounds of adversarial review: probing the scanner against dash quoting and the tools' option parsing and exit codes. Every finding is fixed in this PR.
  • Local run on Windows: the touched suites pass. The full plugin suite was compared against main on the same machine, and no test fails only on this branch. One timing test flaked once under load and passes in isolation on both branches. pixel_ingress.test.mjs hangs on Windows on both branches; CI runs it on Linux.

Follow-ups (not in this PR)

  • The task activity panel still labels an exit-1 search "Failed" even though it is no longer charged.
  • Reading a missing file (read ENOENT) is the other common "answer, not failure" case. It is worth the same treatment, with its own bound.
  • A JavaScript runtime in the agent sandbox. For web projects the model repeatedly tries to run or syntax-check what it wrote (node --check, node -e). With only python3 available those attempts fail and are charged, and it also tried to emulate JS through Python. Related: fix(pixel): tell sandboxed Pixel what its exec sandbox provides #6763. Changing the sandbox image needs a Pixel bump and a bundle repin (see fix(pixel): provide python alongside python3 in the agent sandbox #7640).
  • /workspace/.openclaw/tmp is not writable from the sandbox, but the model reaches for it to stage helper scripts.

🤖 Generated with Claude Code

Lightheartdevs and others added 4 commits October 15, 2026 21:19
The run-progress budget charged every non-zero exit as a failure, and its
12-failure total never resets. grep, rg and git grep exit 1 when nothing
matched, so a coding run that searches for a name, sees "not here" and looks
elsewhere spends that total on answers. On the RTX Spark, Portal fixing a
browser game made 21 good edits and was stopped by the guard on a no-match
grep.

lookupFoundNothing() recognizes one strict shape: a single grep, egrep,
fgrep, rg, git grep or which, optionally after one literal `cd DIR &&`,
that completed with exit 1 and printed nothing (OpenClaw merges stdout and
stderr, so no output means no error). The receipt must be bound to the exact
pending call (run, transport, session and executed wrapped params) and the
model's own command is read from that call's record. Such a lookup is
recorded as discovery through the existing free-correction path: not
charged, resets nothing, up to FREE_EMPTY_LOOKUPS_PER_RUN (12) per run.
Discovery never resets the round fuse, so a run that only searches still
stops after roundsWithoutProgress rounds. Pipelines, lists, redirections,
substitutions, any output and every other exit code remain failures. The
limits are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Models rarely run one bare grep. On the RTX Spark, four of the five empty
searches in the second Elf of Doom run were short read-only chains such as
`grep a f | head; echo ===; grep b g` or `sed -n '860,900p' ui.js; echo ===;
grep -n x ui.js`, and each was still charged as a failure.

lookupFoundNothing() now also accepts a list joined by `;` or `&&` when every
step is a grep-family lookup or `which`, a literal echo, or `sed -n 'N,Mp'`
over plain file arguments, optionally piped into head, tail, cat, wc, sort or
uniq, and the last step is a bare lookup. None of the other steps can exit 1,
so exit 1 still means a search found nothing. A single lookup must still print
nothing; a list must print no error line. Quotes are scanned, not stripped:
`$`, backticks and newlines inside double quotes, `||`, lone `&`, redirection,
grouping, comments and escapes outside quotes all disqualify the command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the chained-search check found commands it accepted that should
be charged. rg 13 prints regex, argument and file errors without its name
(`regex parse error:`, `error: ...`, `... (os error 2)`), and git grep and
which print `usage:` and `Illegal option`; an earlier `;` step that failed
that way went unseen. Options that write files or run programs passed as
read-only steps once quoted: `sed -n 1p f '-i'`, `sort -o FILE`,
`git grep -O CMD`, and `rg --pre CMD` even as a single lookup.

The scanner now returns each stage's words with quotes removed, and steps
are checked word by word: sed operands may not start with `-`, sort and
uniq take a fixed flag set, head/tail only a count, rg may not use --pre or
--hostname-bin, git grep may not use -O or --open-files-in-pager. Every
receipt, single or chained, must print no error line; grep's warnings and
binary-match notices are not errors, and errno phrases only count at the
end of a line. A single `grep -c` that prints 0 is now an empty answer too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rg 13 prints some errors with no recognisable prefix (`unrecognized file
type: jsx`, `the literal "\n" is not allowed in a regex`, `unknown encoding`),
so an rg that failed earlier in a `;` list, or before a pipe, went unseen.
rg may now only be the last step, where its own exit status (2 on error)
separates an error from an empty search. Also say what `which` actually
does: it exits 1 when any name does not resolve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Lightheartdevs
Lightheartdevs merged commit 29d01c6 into main Oct 16, 2026
77 checks passed
Lightheartdevs added a commit that referenced this pull request Oct 17, 2026
…xes on update

Node in the sandbox invites an edit and test loop, but every red node run
spent the run-progress failure budget (run-progress-budget.mjs:24-73): an
edit followed by a failing node --test stopped the response at the
12-failure total, and one failing node --check per file at the 4-failure
streak, before the fix.

- nodeReportedFailure (run-progress-budget.mjs) recognizes a command made
  only of node runs (--check and one script, --test and its files, or a
  script file, optionally behind timeout and 2>&1) that completed with exit
  1 and printed output. The guard records it as discovery, like an empty
  grep (#7701), up to FREE_NODE_RESULTS_PER_RUN per run, bound to the exact
  pending exec call in both transports. node not found, a timeout, a
  signal, no output, a shell error, a missing script or module, inline
  node -e code and pipes still count as failures; the round fuse and the
  direct same-command retry limit still apply.
- The verification guide drops the one-command loop workaround: node
  --check on each script, explicit node --test files, a harness script that
  exits 1 only when a check fails.
- Inline node, Python, perl or shell code that deletes a tree needs the
  owner like rm -rf. Only commands that run inline code are scanned for a
  delete in quotes, so grep -rn "rm -rf" is not refused.

A macOS native update never retired the running agent sandbox, and
OpenClaw keeps a container used in the last five minutes, so the old image
without node kept serving while the new prompt said node is there.
_retire_agent_sandboxes (pixel-macos-access-install.py) removes this
installation's pixel-sbx-agent-pixel-* containers as the owner while every
previous job is stopped and before the candidate starts, proves none
remain, and fails closed with cause, recovery and the Discord link.

Also, from the review:
- Linux/WSL: bootstrap's node proof is bounded (timeout 120), names its
  container and removes it on failure, and ends with a fixed message that
  the installer turns into the docker image rm recovery instead of the
  network message; a developer source without the sandbox Node.js label
  stops before apply. The Node.js download stops on a stalled connection.
- macOS update sandbox errors say to run the update again, not reinstall.
- The image workflow runs both install proofs against the real image,
  covers the guard and wrapper paths, and drops persisted credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lightheartdevs added a commit that referenced this pull request Oct 17, 2026
…xes on update

Node in the sandbox invites an edit and test loop, but every red node run
spent the run-progress failure budget (run-progress-budget.mjs:24-73): an
edit followed by a failing node --test stopped the response at the
12-failure total, and one failing node --check per file at the 4-failure
streak, before the fix.

- nodeReportedFailure (run-progress-budget.mjs) recognizes a command made
  only of node runs (--check and one script, --test and its files, or a
  script file, optionally behind timeout and 2>&1) that completed with exit
  1 and printed output. The guard records it as discovery, like an empty
  grep (#7701), up to FREE_NODE_RESULTS_PER_RUN per run, bound to the exact
  pending exec call in both transports. node not found, a timeout, a
  signal, no output, a shell error, a missing script or module, inline
  node -e code and pipes still count as failures; the round fuse and the
  direct same-command retry limit still apply.
- The verification guide drops the one-command loop workaround: node
  --check on each script, explicit node --test files, a harness script that
  exits 1 only when a check fails.
- Inline node, Python, perl or shell code that deletes a tree needs the
  owner like rm -rf. Only commands that run inline code are scanned for a
  delete in quotes, so grep -rn "rm -rf" is not refused.

A macOS native update never retired the running agent sandbox, and
OpenClaw keeps a container used in the last five minutes, so the old image
without node kept serving while the new prompt said node is there.
_retire_agent_sandboxes (pixel-macos-access-install.py) removes this
installation's pixel-sbx-agent-pixel-* containers as the owner while every
previous job is stopped and before the candidate starts, proves none
remain, and fails closed with cause, recovery and the Discord link.

Also, from the review:
- Linux/WSL: bootstrap's node proof is bounded (timeout 120), names its
  container and removes it on failure, and ends with a fixed message that
  the installer turns into the docker image rm recovery instead of the
  network message; a developer source without the sandbox Node.js label
  stops before apply. The Node.js download stops on a stalled connection.
- macOS update sandbox errors say to run the update again, not reinstall.
- The image workflow runs both install proofs against the real image,
  covers the guard and wrapper paths, and drops persisted credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lightheartdevs added a commit that referenced this pull request Oct 17, 2026
…xes on update

Node in the sandbox invites an edit and test loop, but every red node run
spent the run-progress failure budget (run-progress-budget.mjs:24-73): an
edit followed by a failing node --test stopped the response at the
12-failure total, and one failing node --check per file at the 4-failure
streak, before the fix.

- nodeReportedFailure (run-progress-budget.mjs) recognizes a command made
  only of node runs (--check and one script, --test and its files, or a
  script file, optionally behind timeout and 2>&1) that completed with exit
  1 and printed output. The guard records it as discovery, like an empty
  grep (#7701), up to FREE_NODE_RESULTS_PER_RUN per run, bound to the exact
  pending exec call in both transports. node not found, a timeout, a
  signal, no output, a shell error, a missing script or module, inline
  node -e code and pipes still count as failures; the round fuse and the
  direct same-command retry limit still apply.
- The verification guide drops the one-command loop workaround: node
  --check on each script, explicit node --test files, a harness script that
  exits 1 only when a check fails.
- Inline node, Python, perl or shell code that deletes a tree needs the
  owner like rm -rf. Only commands that run inline code are scanned for a
  delete in quotes, so grep -rn "rm -rf" is not refused.

A macOS native update never retired the running agent sandbox, and
OpenClaw keeps a container used in the last five minutes, so the old image
without node kept serving while the new prompt said node is there.
_retire_agent_sandboxes (pixel-macos-access-install.py) removes this
installation's pixel-sbx-agent-pixel-* containers as the owner while every
previous job is stopped and before the candidate starts, proves none
remain, and fails closed with cause, recovery and the Discord link.

Also, from the review:
- Linux/WSL: bootstrap's node proof is bounded (timeout 120), names its
  container and removes it on failure, and ends with a fixed message that
  the installer turns into the docker image rm recovery instead of the
  network message; a developer source without the sandbox Node.js label
  stops before apply. The Node.js download stops on a stalled connection.
- macOS update sandbox errors say to run the update again, not reinstall.
- The image workflow runs both install proofs against the real image,
  covers the guard and wrapper paths, and drops persisted credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lightheartdevs added a commit that referenced this pull request Oct 17, 2026
…xes on update

Node in the sandbox invites an edit and test loop, but every red node run
spent the run-progress failure budget (run-progress-budget.mjs:24-73): an
edit followed by a failing node --test stopped the response at the
12-failure total, and one failing node --check per file at the 4-failure
streak, before the fix.

- nodeReportedFailure (run-progress-budget.mjs) recognizes a command made
  only of node runs (--check and one script, --test and its files, or a
  script file, optionally behind timeout and 2>&1) that completed with exit
  1 and printed output. The guard records it as discovery, like an empty
  grep (#7701), up to FREE_NODE_RESULTS_PER_RUN per run, bound to the exact
  pending exec call in both transports. node not found, a timeout, a
  signal, no output, a shell error, a missing script or module, inline
  node -e code and pipes still count as failures; the round fuse and the
  direct same-command retry limit still apply.
- The verification guide drops the one-command loop workaround: node
  --check on each script, explicit node --test files, a harness script that
  exits 1 only when a check fails.
- Inline node, Python, perl or shell code that deletes a tree needs the
  owner like rm -rf. Only commands that run inline code are scanned for a
  delete in quotes, so grep -rn "rm -rf" is not refused.

A macOS native update never retired the running agent sandbox, and
OpenClaw keeps a container used in the last five minutes, so the old image
without node kept serving while the new prompt said node is there.
_retire_agent_sandboxes (pixel-macos-access-install.py) removes this
installation's pixel-sbx-agent-pixel-* containers as the owner while every
previous job is stopped and before the candidate starts, proves none
remain, and fails closed with cause, recovery and the Discord link.

Also, from the review:
- Linux/WSL: bootstrap's node proof is bounded (timeout 120), names its
  container and removes it on failure, and ends with a fixed message that
  the installer turns into the docker image rm recovery instead of the
  network message; a developer source without the sandbox Node.js label
  stops before apply. The Node.js download stops on a stalled connection.
- macOS update sandbox errors say to run the update again, not reinstall.
- The image workflow runs both install proofs against the real image,
  covers the guard and wrapper paths, and drops persisted credentials.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant