fix(transform): pass tool-search managed tools (defer_loading) through the tool rewrite#76
Merged
teamchong merged 3 commits intoJul 19, 2026
Conversation
byingyang
commented
Jul 7, 2026
byingyang
left a comment
byingyang
left a comment
Contributor
Author
There was a problem hiding this comment.
🤖 Self-review (deep): 1 CodeRabbit-missed finding(s) from a deep self-review.
Tools carrying defer_loading: true belong to Anthropic's tool-search beta: the server keeps them out of context (billed ~zero) until the model searches for them. Rewriting them — stub description, annotation-stripped schema, full docs rendered into the imaged Tool Reference — materializes documentation the API was deliberately keeping free, inflating every request. A Claude Code session with a large MCP surface ships hundreds of tools (~490k chars observed), so an unpatched proxy can image the entire set into every request. The tool_search_tool_regex/_bm25 server tools themselves must also pass through untouched: they are schema-less and server-defined, and stubbing their description breaks the beta's contract. Deferred tools now pass through byte-identical, are excluded from the imaged Tool Reference, and a deferred_tools_skipped counter lands in events.jsonl for observability.
…rred tools Self-review nit: no test exercised the below_min_chars early-return path (all tools deferred, tiny system prompt) where transformRequest returns the original body before req.tools = toolsRewritten ever runs. Locks in that this path stays byte-identical. Auto-fix-by: babysit-prs
teamchong
force-pushed
the
fix/defer-loading-passthrough
branch
from
July 19, 2026 03:45
a0bc870 to
bf4215b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Anthropic tool-search beta (
tool_search_tool_regex_20251119/_bm25_) lets callers mark tools withdefer_loading: true: the server keeps those tools out of the model's context — billed at ~zero — until the model searches for them.pxpipe's tool rewrite (
compressTools, on by default) currently treats every entry ofreq.toolsthe same: stub the description, strip schema annotations, and render the full docs into the imaged Tool Reference. For deferred tools this materializes documentation the API was deliberately keeping free — the imaged slab carries their full docs in every request, converting a ~free deferred tool into permanently-billed image tokens. With a large tool surface (a Claude Code session with many MCP servers ships hundreds of tools, ~490k chars of docs measured), the imaged slab is dominated by documentation that should never have entered context.Stubbing the search tool itself is also unsafe: it is schema-less and server-defined, and rewriting its description breaks the beta's contract.
Fix
Tools where
defer_loading === true, and tools whosetypestarts withtool_search_tool, now pass through the rewrite byte-identical: no stub, no schema strip, and no entry in the imaged Tool Reference. Everything else is rewritten exactly as before.A
deferred_tools_skippedcounter is added toTransformInfoandevents.jsonlfor observability.Verification
tests/defer-loading.test.ts(3 tests): deferred + search tools survive byte-identical while sibling tools are still rewritten; deferred docs are excluded fromtoolDocsCharsand absent from the outgoing body outside the tool's own definition; requests without deferred tools are byte-for-byte unaffected (no-op path).pnpm run buildclean.-psession through it (224 tools, nodefer_loading) — transforms and responses unchanged, confirming the no-op path on today's most common traffic.