fix(openai): stop double-billing native GPT tool descriptions#19
Closed
rldyourmnd wants to merge 3 commits into
Closed
fix(openai): stop double-billing native GPT tool descriptions#19rldyourmnd wants to merge 3 commits into
rldyourmnd wants to merge 3 commits into
Conversation
renderToolDoc/renderFlatToolDoc rendered the tool description INTO the imaged slab, but rewriteToolsForGpt keeps function.description native (it only strips schema annotations). So on the GPT path each description was billed twice — native text AND image pixels — while gptBaselineImagedTokens credits only the stripped-schema delta (orig − stripped, in which the native-kept description cancels). The image paid for bytes nothing was saved on, and a big-description / tiny-schema tool falsely reported compressed=true. Design: keep native tools[] untouched (descriptions stay native, per the GPT path's design) and image only the heading + schema — the schema carries the annotations actually stripped from native, so every imaged byte maps to a real saving. gptBaselineImagedTokens is unchanged (its math was already correct); its comment was fixed — it described the delta as 'description tokens' when it is schema-annotation tokens. - renderToolDoc / renderFlatToolDoc no longer push the description - tests: tool-only imaging cases now use a realistic big SCHEMA (the profitable case under the dedupe); task-tool strip regressions get a system/instructions slab so compression still triggers; added an explicit test that a big description over a tiny schema is NOT imaged (no double-bill) and passes native through untouched
The dedupe (this PR) stopped imaging tool descriptions — they stay in the native JSON — but CHAT_HEADER / RESPONSES_HEADER still told the model the images hold "full tool/schema documentation". They don't: the image now holds the SCHEMA only; the description is native. - headers: "full tool/schema documentation" → "tool schema documentation", and spell out that native definitions carry each tool's name and description - pointers: "rendered tool docs are supplemental" → native JSON carries name + description; the rendered schema is supplemental - export the four framing constants (internal — not re-exported from index.ts) and add a regression test that none claim "full tool" and all keep native definitions authoritative Full suite green (633).
Contributor
Author
|
Pushed a follow-up. The |
teamchong
pushed a commit
that referenced
this pull request
Jul 18, 2026
…l imaging main already stopped double-billing native GPT tool descriptions: 4c78363 reworked the GPT path to strip parameter annotations from the native tools[] and image only those annotations (schemaAnnotationLines), keeping each tool's name and top-level description native. That superseded the original change in #19, but left the image framing stale — CHAT_HEADER/RESPONSES_HEADER and the pointers still told the model the images hold "full tool/schema documentation" and that native JSON is merely supplemental. Reword the four framing constants to match what is actually imaged: the native JSON tool definitions carry each tool's name and description and stay authoritative; the imaged parameter annotations are supplemental. Export the constants and add a regression test asserting the framing never claims "full tool" docs and always points at the native definitions. No behavior change to token accounting or imaging — framing text only. Salvaged from #19 (by rldyourmnd) and rebased onto current main.
Owner
|
Thanks for this @rldyourmnd, closing as I believe the core change already landed in |
teamchong
pushed a commit
that referenced
this pull request
Jul 19, 2026
…l imaging main already stopped double-billing native GPT tool descriptions: 4c78363 reworked the GPT path to strip parameter annotations from the native tools[] and image only those annotations (schemaAnnotationLines), keeping each tool's name and top-level description native. That superseded the original change in #19, but left the image framing stale — CHAT_HEADER/RESPONSES_HEADER and the pointers still told the model the images hold "full tool/schema documentation" and that native JSON is merely supplemental. Reword the four framing constants to match what is actually imaged: the native JSON tool definitions carry each tool's name and description and stay authoritative; the imaged parameter annotations are supplemental. Export the constants and add a regression test asserting the framing never claims "full tool" docs and always points at the native definitions. No behavior change to token accounting or imaging — framing text only. Salvaged from #19 (by rldyourmnd) and rebased onto current main.
teamchong
pushed a commit
that referenced
this pull request
Jul 19, 2026
…l imaging Align Chat and Responses framing with annotation-only tool imaging, preserving native tool definitions as authoritative. Includes output-level regressions adapted from #19.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #18 (Design B — conservative: native
tools[]untouched).renderToolDoc/renderFlatToolDocimaged the tool description, butrewriteToolsForGptkeepsfunction.descriptionnative (only schema annotations are stripped). So each description was billed twice — native text + image pixels — whilegptBaselineImagedTokenscredits only the stripped-schema delta (in which the native-kept description cancels).renderToolDoc/renderFlatToolDocno longer push the description; they image the heading + schema (which carries the stripped annotations), so every imaged byte maps to something actually removed from nativegptBaselineImagedTokensmath is unchanged (it was already correct); fixed its comment, which called the delta "description tokens" when it's schema-annotation tokensIf you'd rather move the description into the image and stub it native (Design A — more compression, but changes the outgoing tool text), I'm happy to redo it that way. typecheck/test/build green.