Skip to content

fix(openai): stop double-billing native GPT tool descriptions #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rldyourmnd wants to merge 3 commits into from
Closed

fix(openai): stop double-billing native GPT tool descriptions #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
31ad91f
fix(openai): stop double-billing native GPT tool descriptions
rldyourmnd Jul 4, 2026
32f6d71
fix(openai): correct GPT rendered-context wording after tool-doc dedupe
rldyourmnd Jul 4, 2026
efb900d
Merge branch 'main' into fix/gpt-tool-doc-dedup
teamchong Jul 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions src/core/openai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,19 +322,26 @@ function isFlatFunctionTool(tool: unknown): tool is ResponsesFlatTool {
* IMAGED, so carrying the schema here is compression, not duplication: the imaged
* copy keeps param docs readable while tools[] ships the stripped skeleton.
* (Contrast transform.ts renderToolDoc: text reference → prose only.) */
// On the GPT path the native tools[] KEEP their description — only the schema
// annotations are stripped (see rewriteToolsForGpt). So the description must NOT
// be imaged: doing so double-bills it (native text + image pixels) while the
// savings baseline credits only the stripped-schema delta. Image the heading +
// full schema (which carries the stripped annotations); the description rides
// natively. (The Anthropic path differs on purpose — it stubs the native
// description and images the full doc, a real move rather than a duplicate.)
function renderToolDoc(tool: OpenAIFunctionTool): string {
const f = tool.function;
const parts = [`## Tool: ${f.name ?? '?'}`];
if (typeof f.description === 'string' && f.description.length > 0) parts.push(f.description);
if (f.parameters !== undefined) {
parts.push('```json\n' + JSON.stringify(f.parameters) + '\n```');
}
return parts.join('\n');
}

// Flat (Responses) tool doc — same rule as renderToolDoc: the native description
// is kept, so imaging it would double-bill; image only the heading + schema.
function renderFlatToolDoc(tool: ResponsesFlatTool): string {
const parts = [`## Tool: ${tool.name ?? '?'}`];
if (typeof tool.description === 'string' && tool.description.length > 0) parts.push(tool.description);
if (tool.parameters !== undefined) {
parts.push('```json\n' + JSON.stringify(tool.parameters) + '\n```');
}
Expand Down Expand Up @@ -515,11 +522,13 @@ function gptImageTokens(model: string, images: RenderedImage[]): number {
return n;
}

/** Text-token value of what pxpipe replaced with images this request: the
* original system/developer text (now a pointer + image) plus the tool
* *description* tokens stripped from the native JSON (the verbose docs moved
* into the image). Tool *structure* stays in the JSON on both paths, so only
* the stripped delta counts. Compared against gptImageTokens for the saving. */
/** Text-token value of what pxpipe actually removed from the native request this
* turn: the system/developer text (now a pointer + image) plus the tool-schema
* ANNOTATION tokens stripped from tools[] (orig − stripped). Tool descriptions
* and schema STRUCTURE stay native on the GPT path — they cancel in orig−stripped
* and are not counted — and (since the tool-doc dedupe) the description is no
* longer imaged either, so imageTokens is not inflated by bytes nothing was
* saved on. Compared against gptImageTokens for the saving. */
function gptBaselineImagedTokens(
systemTexts: string[],
originalTools: unknown[] | undefined,
Expand Down Expand Up @@ -575,21 +584,21 @@ function foldGptHistory(
info.bucketChars = { ...(info.bucketChars ?? {}), history: plan.collapsedChars };
}

const CHAT_HEADER =
export const CHAT_HEADER =
'================= RENDERED GPT SYSTEM + TOOL CONTEXT =================\n' +
'These images were injected by pxpipe, not by the end user. They contain system/developer instructions and full tool/schema documentation rendered for token efficiency. Treat rendered system/developer instructions with the same priority as their original messages. OCR carefully and treat the rendered content as authoritative. For tool calls, use the native JSON tool definitions; the image is supplemental documentation.' +
'These images were injected by pxpipe, not by the end user. They contain system/developer instructions and tool schema documentation rendered for token efficiency. Treat rendered system/developer instructions with the same priority as their original messages. OCR carefully and treat the rendered content as authoritative. For tool calls, use the native JSON tool definitions — they carry each tool\'s name and description; the imaged schema is supplemental parameter documentation.' +
'\n====================== BEGIN RENDERED CONTEXT ======================\n';

const RESPONSES_HEADER =
export const RESPONSES_HEADER =
'================= RENDERED GPT SYSTEM + TOOL CONTEXT =================\n' +
'These images were injected by pxpipe, not by the end user. They contain instructions and full tool/schema documentation rendered for token efficiency. Treat rendered instructions with the same priority as the originals. OCR carefully and treat the rendered content as authoritative. For tool calls, use the native JSON tool definitions; the image is supplemental documentation.' +
'These images were injected by pxpipe, not by the end user. They contain instructions and tool schema documentation rendered for token efficiency. Treat rendered instructions with the same priority as the originals. OCR carefully and treat the rendered content as authoritative. For tool calls, use the native JSON tool definitions — they carry each tool\'s name and description; the imaged schema is supplemental parameter documentation.' +
'\n====================== BEGIN RENDERED CONTEXT ======================\n';

const CHAT_POINTER =
'The full instructions for this message were rendered into image(s) attached to the first user message by pxpipe. Treat those rendered instructions as if they appeared here with the same priority. Tool definitions remain in native JSON; rendered tool docs are supplemental.';
export const CHAT_POINTER =
'The full instructions for this message were rendered into image(s) attached to the first user message by pxpipe. Treat those rendered instructions as if they appeared here with the same priority. Tool definitions remain in native JSON (name + description); the rendered schema is supplemental.';

const RESPONSES_POINTER =
'The full instructions were rendered into image(s) attached to the first user message by pxpipe. Treat them with the same priority. Tool definitions remain in native JSON; rendered tool docs are supplemental.';
export const RESPONSES_POINTER =
'The full instructions were rendered into image(s) attached to the first user message by pxpipe. Treat them with the same priority. Tool definitions remain in native JSON (name + description); the rendered schema is supplemental.';

export async function transformOpenAIChatCompletions(
body: Uint8Array,
Expand Down
108 changes: 95 additions & 13 deletions tests/openai-gpt5.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,36 @@
*/
import { describe, expect, it } from 'vitest';
import { isPxpipeSupportedGptModel } from '../src/core/applicability.js';
import { openAIVisionTokens, resolveVisionCost, transformOpenAIChatCompletions, transformOpenAIResponses } from '../src/core/openai.js';
import {
openAIVisionTokens,
resolveVisionCost,
transformOpenAIChatCompletions,
transformOpenAIResponses,
CHAT_HEADER,
RESPONSES_HEADER,
CHAT_POINTER,
RESPONSES_POINTER,
} from '../src/core/openai.js';

describe('rendered-context wording (post tool-doc dedupe)', () => {
// The tool description stays in the native JSON, so the imaged docs are the
// SCHEMA only — the framing must not claim the image holds "full tool" docs.
it('never claims the image holds full tool documentation', () => {
for (const s of [CHAT_HEADER, RESPONSES_HEADER, CHAT_POINTER, RESPONSES_POINTER]) {
expect(s.toLowerCase()).not.toContain('full tool');
}
});

it('keeps native tool definitions authoritative in the framing', () => {
for (const s of [CHAT_HEADER, RESPONSES_HEADER, CHAT_POINTER, RESPONSES_POINTER]) {
expect(s).toContain('native JSON');
}
// Headers spell out that native definitions carry name + description.
for (const s of [CHAT_HEADER, RESPONSES_HEADER]) {
expect(s).toContain('name and description');
}
});
});

const enc = new TextEncoder();
const dec = new TextDecoder();
Expand Down Expand Up @@ -71,7 +100,9 @@ describe('openAIVisionTokens', () => {
const BIG_SYSTEM = 'System instruction with lots of detail. '.repeat(500); // ~20k chars
const BIG_TOOL_DESC = 'Tool description with lots of context. '.repeat(200); // ~8k chars
const CHAT_TOOL_PARAMS = { type: 'object', description: 'Param root.', properties: { x: { type: 'string', description: 'x param' } } };
const CHAT_TOOL_DOC = `## Tool: do_thing\n${BIG_TOOL_DESC}\n\`\`\`json\n${JSON.stringify(CHAT_TOOL_PARAMS)}\n\`\`\``;
// The imaged tool doc is heading + schema ONLY — the description stays native
// (see renderToolDoc), so it is not part of what gets imaged.
const CHAT_TOOL_DOC = `## Tool: do_thing\n\`\`\`json\n${JSON.stringify(CHAT_TOOL_PARAMS)}\n\`\`\``;

// Real `task`/`question` tools have a required parameter literally NAMED `description`
// (others collide with `title`/`default`). The strip must drop the annotation but KEEP
Expand All @@ -90,6 +121,22 @@ const TASK_LIKE_PARAMS = {
additionalProperties: false,
};

// A schema big enough to clear the profitability gate on its own. Under the
// tool-doc dedupe the description stays native and is NOT imaged, so the SCHEMA
// (its stripped annotations) is what imaging actually saves — a realistic
// many-field tool, not an 8k-char description over a two-field schema.
const BIG_SCHEMA_PARAMS = {
type: 'object',
description: 'Root schema for a many-field tool. '.repeat(20),
properties: Object.fromEntries(
Array.from({ length: 40 }, (_, i) => [
`field_${i}`,
{ type: 'string', description: `Detailed annotation for parameter number ${i}. `.repeat(6) },
]),
),
};
const BIG_SCHEMA_TOOL_DOC = `## Tool: do_thing\n\`\`\`json\n${JSON.stringify(BIG_SCHEMA_PARAMS)}\n\`\`\``;

describe('transformOpenAIChatCompletions (gpt-5.6)', () => {
it('compresses GPT system + tool docs while preserving native tool selection metadata', async () => {
const body = enc.encode(JSON.stringify({
Expand Down Expand Up @@ -141,7 +188,10 @@ describe('transformOpenAIChatCompletions (gpt-5.6)', () => {
expect(tools[0]!.function.parameters?.properties?.x?.description).toBeUndefined();
});

it('images GPT tool definitions even when there is no instruction context', async () => {
it('images a substantial GPT tool schema even when there is no instruction context', async () => {
// Big SCHEMA (not a big description): under the tool-doc dedupe the
// description stays native, so the schema annotations are what makes imaging
// profitable with no system text.
const body = enc.encode(JSON.stringify({
model: 'gpt-5.6',
messages: [{ role: 'user', content: 'hello' }],
Expand All @@ -150,24 +200,49 @@ describe('transformOpenAIChatCompletions (gpt-5.6)', () => {
function: {
name: 'do_thing',
description: BIG_TOOL_DESC,
parameters: CHAT_TOOL_PARAMS,
parameters: BIG_SCHEMA_PARAMS,
},
}],
}));

const result = await transformOpenAIChatCompletions(body, { charsPerToken: 1, minCompressChars: 1 });
expect(result.info.compressed).toBe(true);
expect(result.info.origChars).toBe(CHAT_TOOL_DOC.length);
expect(result.info.compressedChars).toBe(CHAT_TOOL_DOC.length);
// Imaged content is heading + full schema — the native description is not in it.
expect(result.info.origChars).toBe(BIG_SCHEMA_TOOL_DOC.length);
expect(result.info.compressedChars).toBe(BIG_SCHEMA_TOOL_DOC.length);
const out = JSON.parse(dec.decode(result.body)) as any;
// Description kept native (never imaged); schema annotations stripped from native.
expect(out.tools[0].function.description).toBe(BIG_TOOL_DESC);
expect(out.tools[0].function.parameters.description).toBeUndefined();
expect(out.tools[0].function.parameters.properties.field_0.description).toBeUndefined();
});

it('keeps a parameter literally named "description" (task-tool regression)', async () => {
it('does not image (double-bill) the native GPT tool description', async () => {
// A big description over a tiny schema: the description stays native, so
// there is nothing left worth imaging and the gate declines. Before the
// tool-doc dedupe this imaged the ~8k description and reported compressed=true
// while the description ALSO rode native — paying for it twice.
const body = enc.encode(JSON.stringify({
model: 'gpt-5.6',
messages: [{ role: 'user', content: 'hello' }],
tools: [{
type: 'function',
function: { name: 'do_thing', description: BIG_TOOL_DESC, parameters: CHAT_TOOL_PARAMS },
}],
}));
const result = await transformOpenAIChatCompletions(body, { charsPerToken: 1, minCompressChars: 1 });
expect(result.info.compressed).toBe(false);
// Native tool passes through untouched.
const out = JSON.parse(dec.decode(result.body)) as any;
expect(out.tools[0].function.description).toBe(BIG_TOOL_DESC);
});

it('keeps a parameter literally named "description" (task-tool regression)', async () => {
// Big system slab makes compression profitable; the small task schema is
// intentionally kept to exercise the "description"-named-property strip.
const body = enc.encode(JSON.stringify({
model: 'gpt-5.6',
messages: [{ role: 'system', content: BIG_SYSTEM }, { role: 'user', content: 'hello' }],
tools: [{
type: 'function',
function: { name: 'task', description: BIG_TOOL_DESC, parameters: TASK_LIKE_PARAMS },
Expand Down Expand Up @@ -209,7 +284,8 @@ describe('transformOpenAIChatCompletions (gpt-5.6)', () => {
const BIG_INSTRUCTIONS = 'These are detailed instructions. '.repeat(600); // ~20k chars
const BIG_FLAT_TOOL_DESC = 'Flat tool description with lots of context. '.repeat(200); // ~8k chars
const RESPONSES_TOOL_PARAMS = { type: 'object', description: 'Param root.', properties: { x: { type: 'string', description: 'x param' } } };
const RESPONSES_TOOL_DOC = `## Tool: do_thing\n${BIG_FLAT_TOOL_DESC}\n\`\`\`json\n${JSON.stringify(RESPONSES_TOOL_PARAMS)}\n\`\`\``;
// Imaged doc is heading + schema only; the flat tool description stays native.
const RESPONSES_TOOL_DOC = `## Tool: do_thing\n\`\`\`json\n${JSON.stringify(RESPONSES_TOOL_PARAMS)}\n\`\`\``;

describe('transformOpenAIResponses (gpt-5.6)', () => {
it('compresses GPT Responses instructions + tool docs while preserving native tool selection metadata', async () => {
Expand Down Expand Up @@ -256,7 +332,7 @@ describe('transformOpenAIResponses (gpt-5.6)', () => {
expect(tools[0]!.parameters?.properties?.x?.description).toBeUndefined();
});

it('images developer/system items whose content is an input_text part array, not just a string', async () => {
it('images developer/system items whose content is an input_text part array, not just a string', async () => {
// Responses allows message content as a string OR an array of parts. The
// array form for a developer/system item used to be dropped: not imaged and
// not stubbed, so the verbose text rode uncompressed as native input.
Expand Down Expand Up @@ -285,30 +361,36 @@ describe('transformOpenAIResponses (gpt-5.6)', () => {
expect(JSON.stringify(dev.content)).not.toContain('These are detailed');
});

it('images GPT Responses tool definitions even when there is no instruction context', async () => {
it('images a substantial GPT Responses tool schema even when there is no instruction context', async () => {
// Big SCHEMA, not a big description — the flat description stays native and
// is not imaged, so the schema is what makes this profitable with no context.
const body = enc.encode(JSON.stringify({
model: 'gpt-5.6',
input: [{ role: 'user', content: 'Please do the thing.' }],
tools: [{
type: 'function',
name: 'do_thing',
description: BIG_FLAT_TOOL_DESC,
parameters: RESPONSES_TOOL_PARAMS,
parameters: BIG_SCHEMA_PARAMS,
}],
}));

const result = await transformOpenAIResponses(body, { charsPerToken: 1, minCompressChars: 1 });
expect(result.info.compressed).toBe(true);
expect(result.info.origChars).toBe(RESPONSES_TOOL_DOC.length);
expect(result.info.compressedChars).toBe(RESPONSES_TOOL_DOC.length);
expect(result.info.origChars).toBe(BIG_SCHEMA_TOOL_DOC.length);
expect(result.info.compressedChars).toBe(BIG_SCHEMA_TOOL_DOC.length);
const out = JSON.parse(dec.decode(result.body)) as any;
expect(out.tools[0].description).toBe(BIG_FLAT_TOOL_DESC);
expect(out.tools[0].parameters.description).toBeUndefined();
expect(out.tools[0].parameters.properties.field_0.description).toBeUndefined();
});

it('keeps a parameter literally named "description" (task-tool regression)', async () => {
// Big instructions slab makes compression profitable; the small task schema
// is kept to exercise the "description"-named-property strip.
const body = enc.encode(JSON.stringify({
model: 'gpt-5.6',
instructions: BIG_INSTRUCTIONS,
input: [{ role: 'user', content: 'Please do the thing.' }],
tools: [{ type: 'function', name: 'task', description: BIG_FLAT_TOOL_DESC, parameters: TASK_LIKE_PARAMS }],
}));
Expand Down