fix(translator): accept CRLF SSE framing in the Responses usage parser - #2679
Open
wangzhengzhuo05 wants to merge 1 commit into
Open
wangzhengzhuo05 wants to merge 1 commit into
wangzhengzhuo05 wants to merge 1 commit into
Conversation
The Responses passthrough parser only split events on LF blank lines, so a compliant upstream that frames its stream with CRLF was never parsed: usage, the resolved model, and tracing chunks were all dropped from the event stream. Recognize both LF and CRLF event boundaries and strip the trailing CR from each line of a CRLF-framed event. Signed-off-by: wangzhengzhuo05 <175673456+wangzhengzhuo05@users.noreply.github.com>
✅ Deploy Preview for theagentrouter canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Responses passthrough translator parses the upstream SSE stream to recover token usage, the resolved response model, and tracing chunks. Its parser only split events on LF blank lines, so a spec-compliant upstream that frames its stream with CRLF was never parsed at all:
extractUsageFromBufferEventkept the whole stream buffered and returned no usage.internal/translator/openai_responses.gonow resolves the event boundary through a helper that accepts both LF (\n\n) and CRLF (\r\n\r\n) blank lines, and each line of a CRLF-framed event has its trailing CR stripped before thedata:field is read. The optional space afterdata:was already accepted bycutSSEDataPrefix.Fixes #2463
Why
SSE permits CRLF line endings, so this is a framing bug rather than upstream non-compliance. The response bytes reach the client either way; what is lost is Gateway's own observability:
gen_ai.usage.*metrics, access-log token fields, the resolved model, and tracing chunks were all missing for CRLF-framed streams.How
internal/translator/util.go: addedindexSSEEventBoundary, which returns the first blank-line event boundary and the length of that boundary, preferring whichever of the LF and CRLF forms appears first in the buffer.internal/translator/openai_responses.go:extractUsageFromBufferEventuses that helper and trims one trailing CR from each line inside an event, so a CRLF-framed event no longer leaves a CR on thedata:payload or on the[DONE]sentinel.Tests
TestIndexSSEEventBoundaryinutil_test.go: LF framing, CRLF framing, LF boundary before a later CRLF boundary, CRLF boundary before a later LF boundary, and a partial event that must stay buffered.TestResponsesOpenAIToOpenAITranslator_ResponseBody: new subtestsstreaming response with CRLF framingandstreaming response with data field without space, asserting the resolved model and input, output, total, cached and reasoning token usage.Verification on the branch:
go test ./internal/translator/... -count=1passes.golangci-lint fmt --diff,golangci-lint runwith the repository's test build tags, andmisspell -errorare clean on the touched files.AI disclosure
The change was authored with AI assistance (OpenCode driving an open-source coding model) and reviewed and verified locally by the submitter for correctness, scope, and licensing (Apache-2.0) before submission, in line with the repository's generative AI policy.