Skip to content

fix: preserve Anthropic streaming overload errors in-band - #2698

Open
hustxiayang wants to merge 4 commits into
theagentrouter:mainfrom
hustxiayang:surface-529-streaming
Open

hustxiayang wants to merge 4 commits into
theagentrouter:mainfrom
hustxiayang:surface-529-streaming

Conversation

@hustxiayang

Copy link
Copy Markdown
Contributor

Description
Anthropic can return HTTP 200 for a streaming request and later report an overloaded_error in the Server-Sent Events (SSE) body. The current translator propagates that event as a Go error, so the client receives either a generic HTTP 500 before the response is committed or a terminated HTTP 200 stream after commitment. Neither result exposes the overload classification.

This change preserves HTTP 200 and translates an Anthropic streaming overload into a terminal Chat Completions SSE error:

data: {"error":{"type":"overloaded_error","message":"Overloaded"}}

The translator returns a typed AnthropicStreamError, and the response processor returns the serialized event as a normal body mutation only when its type is overloaded_error. The event is appended to any output already translated in the same callback. The processor then suppresses later provider chunks, omits [DONE], and records the request as failed. Other streaming error types and non-streaming errors retain their existing behavior.

Related issue

Fixes #2697

@hustxiayang
hustxiayang requested a review from a team as a code owner September 16, 2026 19:32
@netlify

netlify Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for theagentrouter ready!

Name Link
🔨 Latest commit 464b56f
🔍 Latest deploy log https://app.netlify.com/projects/theagentrouter/deploys/6ab6bd1ca5fb250008e6a13b
😎 Deploy Preview https://deploy-preview-2698--theagentrouter.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@hustxiayang hustxiayang changed the title fix: Preserve Anthropic streaming overload errors in-band fix: preserve Anthropic streaming overload errors in-band Sep 16, 2026
Signed-off-by: yxia216 <yxia216@bloomberg.net>
@codecov

codecov Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/extproc/processor_impl.go 90.32% 3 Missing ⚠️
internal/translator/anthropic_helper.go 91.30% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@missBerg missBerg added bug Something isn't working area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs) area/extproc ExtProc data-plane architecture, body limits, performance labels Sep 24, 2026
@hustxiayang

Copy link
Copy Markdown
Contributor Author

@yuzisun Could you please help to review it? Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/extproc ExtProc data-plane architecture, body limits, performance area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs) bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve Anthropic streaming overload errors for Chat Completions clients

2 participants