Skip to content

Use Anthropic native output_config.format for structured outputs on supported models #385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
dpdiliberto wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Use Anthropic native output_config.format for structured outputs on supported models #385

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 47 additions & 26 deletions packages/proxy/src/proxy.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2609,34 +2609,55 @@ async function fetchAnthropicChatCompletions({
"Structured output is not supported with tools",
);
}
params.tools = [
{
name: "json",
description: "Output the result in JSON format",
input_schema:
parsed.data.type === "json_schema"
? parsed.data.json_schema.schema
: { type: "object" },
},
];

const thinkingParamsParsed = z
.object({
thinking: z.object({
type: z.literal("enabled"),
}),
})
.safeParse(params);

// Claude hack: if thinking is enabled, tool_choice cannot be specified. So
// we just omit tool_choice in that case.
if (
thinkingParamsParsed.success &&
thinkingParamsParsed.data.thinking.type === "enabled"
) {
delete params.tool_choice;
// Check if model supports native structured outputs (output_config.format)
// Models that support it: claude-haiku-4-5, claude-sonnet-4-5, claude-opus-4-5, claude-opus-4-6
const modelName = (oaiParams.model as string) || "";
const supportsNativeStructuredOutput =
modelName.includes("claude-haiku-4-5") ||
modelName.includes("claude-sonnet-4-5") ||
modelName.includes("claude-opus-4-5") ||
modelName.includes("claude-opus-4-6");

if (supportsNativeStructuredOutput && parsed.data.type === "json_schema") {
// Use Anthropic's native output_config.format API for proper strict validation
params.output_config = {
format: {
type: "json_schema",
schema: parsed.data.json_schema.schema,
Comment on lines +2623 to +2627
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve json_schema strict when building output_config

When response_format is json_schema and the model is in the native path, the proxy only forwards schema into output_config.format and drops json_schema.strict (and other metadata). That means callers who set strict: true will silently lose strict enforcement on these models, which is the core guarantee they asked for. This path is distinct from the other response_format mappings in this file that propagate strict, so the omission is likely unintended and will surface whenever clients rely on strict validation for required fields.

Useful? React with 👍 / 👎.

},
};
} else {
params.tool_choice = { type: "tool", name: "json" };
// Fall back to tool-based approach for older models or json_object format
params.tools = [
{
name: "json",
description: "Output the result in JSON format",
input_schema:
parsed.data.type === "json_schema"
? parsed.data.json_schema.schema
: { type: "object" },
},
];

const thinkingParamsParsed = z
.object({
thinking: z.object({
type: z.literal("enabled"),
}),
})
.safeParse(params);

// Claude hack: if thinking is enabled, tool_choice cannot be specified. So
// we just omit tool_choice in that case.
if (
thinkingParamsParsed.success &&
thinkingParamsParsed.data.thinking.type === "enabled"
) {
delete params.tool_choice;
} else {
params.tool_choice = { type: "tool", name: "json" };
}
}
}

Expand Down