openclaw: site-skills injection yields little benefit — accuracy +2/20 at +16% tokens, no step reduction

[Julia-Lex]

Finding (2026-07-07 site-skills A/B, PR #95)

Same 20 LexBench-Browser tasks, same lexmount backend, judge gpt-5.4. Injecting matched site skills (--site-skills on) helps browser-use and codex substantially, but barely moves openclaw:

agent	judge pass (off→on)	steps	browser calls	tokens
browser-use	11/20 → 13/20	−43%	−53%	−15%
codex	12/20 → 11/20 (flat)	−57%	−57%	−58%
openclaw	9/20 → 11/20	+7%	+2%	+16%

Run dirs: experiments/LexBench-Browser/All/openclaw/gpt-5.4/{20260707_115337,20260707_120414}. Full per-case tables: docs/superpowers/specs/2026-07-07-site-skills-ab-report.md (PR #95).

What we verified

• The skill IS delivered and consumed: on-arm transcripts execute the skill's fallbacks verbatim (e.g. yelp via html.duckduckgo.com/html/ snippet queries — URLs match the skill doc word-for-word), flipping 4 anti-bot tasks (amazon/youtube/wenku/scholar) from give-up answers to correct ones.
• So the +2 accuracy is real; the problem is efficiency never materializes: skill-arm tokens are the highest of all arms, and openclaw+skills (11/20 @ 6.8M tokens) merely matches bare browser-use (11/20 @ 1.3M) at ~5x the cost.
• A native self-managed variant (skills installed as OpenClaw workspace skills, model reads on demand) was also measured and is WORSE: 8/20 pass @ 8.4M tokens, 35% of tasks never read the skill (implemented then reverted in PR feat(run): site-skills prompt injection for all agents — mechanism, fixes, and 4-agent A/B evidence #95, commit abac381).

Root cause (from per-task traces)

openclaw's step count is dominated by procedural overhead the skill cannot compress:

1. every task starts by reading its bundled browser-automation SKILL.md (1-2 turns) + browser status;
2. every page visit does snapshot + screenshot as a pair — screenshots are useless with media.image disabled (text-only model) yet still cost a turn each;
3. one tool call per assistant turn (no batching), so even a direct-URL path costs ~2x the calls browser-use spends;
4. when a wall is hit, the give-up policy fires before consulting injected knowledge unless the model happens to connect them (skills fixed exactly the cases where it did).

Suggested directions

• drop the screenshot half of the snapshot+screenshot ritual when image understanding is off;
• skip the startup skill-file read (inline the needed rules into the agent prompt);
• prompt nudge: on any 403/412/blocked page, re-read the injected Site knowledge section before falling back;
• longer term: allow multi-action turns if the OpenClaw CLI ever supports it.

If none of these land, the practical guidance stands: prefer --site-skills on for openclaw only when accuracy on anti-bot-heavy sites matters more than cost.