Repository navigation
fix: account for deferred Responses tools in context budgets (#2391) - #2398
Conversation
…w nit ranxianglei#2398) The function returns a chars/4 TOKEN total of loaded tool records, not an item count; the old name misleads readers of a brand-new exported API. Rename only — no behavior change. All call sites are inside PR ranxianglei#2398's own diff.
[bot] 🏷 审查完成(方向关 → 回归关 → 代码关 → 测试关 → diff 整洁度),按结构化八问给结论。 方向关(原 issue #2391):#2391 要求排查「陈旧/重复的指令快照或工具 schema 序列化是否膨胀固定开销」。本 PR 解决其中工具 schema 侧机制——OpenAI tool-search( 八问:
行为变更披露(旧 → 新,均为有意,仅限 tool_search+defer_loading 会话):
审查发现(已处置):一个小命名问题——新导出的 测试有效性:7 个新测试断言均有牙——精确值断言钉住「usage 权威 / 校准顺序」语义,畸形形状保守处理逐例枚举,真代理 e2e 钉住 wire 保真与转发前拦截;端口 diff 整洁度:3 文件、单提交(+ 更名提交),每一行都与目的相关,无无关改动、无空白噪音。 判定:可合并 —— 待最后一个 CI job(windows-latest 22)完成及更名 commit 的重跑转绿后,请人工合并。方向正确、修在根因,wire 保真与双计费核查均通过,diff 干净。 |
[bot] 🏷 Closing status update for #2398 CI is now fully green on the current head Final state:
Nothing else outstanding from my side. Ready for human merge: #2398 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.153-master.a00b28f # this exact build
npm install -g billion-context@master # always the newest merged build
|
Summary
Fixes #2391.
Count Responses tool-search catalogs according to what is loaded into model context. Previously the estimator charged the entire JSON catalog, including function schemas marked
defer_loading: true. A large discoverable catalog could exceed the fixed input budget by itself and produce a preflight overflow with zero compressible ranges.tool_searchis present and a valid flat Responses function explicitly hasdefer_loading: true. Keep a small discovery stub, namespace descriptions, eager members, and unknown/malformed shapes.additional_tools,tool_search_call,tool_search_output, andmcp_list_toolsrecords in full, including definitions that retain the deferred flag.The forwarded catalog and input item order are preserved. No context-window increase, new configuration, dependency, version bump, or kernel change.
Regression coverage
Seven new tests cover deferred/eager catalogs, mixed namespaces without flag inheritance, conservative handling of unknown formats, loaded definitions, usage provenance/calibration, output headroom, and a real proxy forwarding to a local fake upstream.
The proxy fixture keeps a 272,000-token total window with a 68,000-token output allocation. Its 1.2-million-character deferred schema is forwarded intact; eager and loaded versions are rejected before reaching the upstream. Small loaded records retain their contents and positions after request rebuilding.
Verification
npm run build: passed; distribution import annotations passed.git diff --check: passed.ENOTFOUND|EAI_AGAIN, but this host returns a connect timeout; the same failure reproduces on unmodified basef2b84b6. The other failure was an intermittent Windows MCP child-process exit3221226505: observed during the full run and one isolated branch run, while the isolated baseline check and a subsequent branch check passed. Its cause remains unproven; canonical CI is still needed.The reporter also confirmed that the deployed compatibility patch resolved the original old-conversation failure while retaining the 272k limit. The exact original request was not captured, so this does not assert that every byte of the earlier overhead estimate has been attributed.
Related work
#2393 adds an actionable fixed-overhead diagnostic. This PR corrects deferred/loaded tool accounting and complements that diagnostic. Both touch
src/server/budget.ts, so merge order may require a small rebase.Reference: https://developers.openai.com/api/docs/guides/tools-tool-search