Skip to content

fix(translator): accept CRLF SSE framing in the Responses usage parser - #2679

Open
wangzhengzhuo05 wants to merge 1 commit into
theagentrouter:mainfrom
wangzhengzhuo05:fix/responses-sse-event-framing
Open

wangzhengzhuo05 wants to merge 1 commit into
theagentrouter:mainfrom
wangzhengzhuo05:fix/responses-sse-event-framing

Conversation

@wangzhengzhuo05

Copy link
Copy Markdown

Description

The Responses passthrough translator parses the upstream SSE stream to recover token usage, the resolved response model, and tracing chunks. Its parser only split events on LF blank lines, so a spec-compliant upstream that frames its stream with CRLF was never parsed at all: extractUsageFromBufferEvent kept the whole stream buffered and returned no usage.

internal/translator/openai_responses.go now resolves the event boundary through a helper that accepts both LF (\n\n) and CRLF (\r\n\r\n) blank lines, and each line of a CRLF-framed event has its trailing CR stripped before the data: field is read. The optional space after data: was already accepted by cutSSEDataPrefix.

Fixes #2463

Why

SSE permits CRLF line endings, so this is a framing bug rather than upstream non-compliance. The response bytes reach the client either way; what is lost is Gateway's own observability: gen_ai.usage.* metrics, access-log token fields, the resolved model, and tracing chunks were all missing for CRLF-framed streams.

How

  • internal/translator/util.go: added indexSSEEventBoundary, which returns the first blank-line event boundary and the length of that boundary, preferring whichever of the LF and CRLF forms appears first in the buffer.
  • internal/translator/openai_responses.go: extractUsageFromBufferEvent uses that helper and trims one trailing CR from each line inside an event, so a CRLF-framed event no longer leaves a CR on the data: payload or on the [DONE] sentinel.

Tests

  • TestIndexSSEEventBoundary in util_test.go: LF framing, CRLF framing, LF boundary before a later CRLF boundary, CRLF boundary before a later LF boundary, and a partial event that must stay buffered.
  • TestResponsesOpenAIToOpenAITranslator_ResponseBody: new subtests streaming response with CRLF framing and streaming response with data field without space, asserting the resolved model and input, output, total, cached and reasoning token usage.

Verification on the branch:

  • go test ./internal/translator/... -count=1 passes.
  • Mutation check: reverting only the parser change makes the CRLF subtest fail with empty token usage, and restoring it makes the suite pass again, so the new test guards the fix.
  • golangci-lint fmt --diff, golangci-lint run with the repository's test build tags, and misspell -error are clean on the touched files.

AI disclosure

The change was authored with AI assistance (OpenCode driving an open-source coding model) and reviewed and verified locally by the submitter for correctness, scope, and licensing (Apache-2.0) before submission, in line with the repository's generative AI policy.

The Responses passthrough parser only split events on LF blank lines, so a
compliant upstream that frames its stream with CRLF was never parsed: usage,
the resolved model, and tracing chunks were all dropped from the event stream.

Recognize both LF and CRLF event boundaries and strip the trailing CR from each
line of a CRLF-framed event.

Signed-off-by: wangzhengzhuo05 <175673456+wangzhengzhuo05@users.noreply.github.com>
@wangzhengzhuo05
wangzhengzhuo05 requested a review from a team as a code owner September 12, 2026 20:54
@netlify

netlify Bot commented Sep 12, 2026

Copy link
Copy Markdown

Deploy Preview for theagentrouter canceled.

Name Link
🔨 Latest commit dfbc222
🔍 Latest deploy log https://app.netlify.com/projects/theagentrouter/deploys/6aa5bbee2128bc000872e09e

@missBerg missBerg added bug Something isn't working area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs) area/observability Metrics, tracing, logging, GenAI observability labels Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/observability Metrics, tracing, logging, GenAI observability area/translation Provider/endpoint coverage and schema translation (incl. fidelity bugs) bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Responses API streaming parser ignores standard SSE framing

2 participants