feat(openai): add WebSocket mode for Responses API#149
Open
taigrr wants to merge 103 commits intocharmbracelet:mainfrom
Open
feat(openai): add WebSocket mode for Responses API#149taigrr wants to merge 103 commits intocharmbracelet:mainfrom
taigrr wants to merge 103 commits intocharmbracelet:mainfrom
Conversation
Add WebSocket transport support for the OpenAI Responses API, enabling lower-latency persistent connections for tool-call-heavy workflows. Key features: - wsTransport manages WebSocket connection lifecycle with automatic reconnection before the 60-minute connection limit - previous_response_id auto-chaining for incremental continuation - generate:false warmup support via GenerateWarmup provider option - Falls back to HTTP transparently on WebSocket connection failure - One in-flight response at a time per connection (mutex-protected) New provider options: - WithWebSocket() enables WebSocket mode (requires WithUseResponsesAPI) - PreviousResponseID on ResponsesProviderOptions for explicit chaining - GenerateWarmup on ResponsesProviderOptions for prefill/warmup The WebSocket events use the same JSON structure as HTTP SSE events, so both Generate() and Stream() reuse existing event parsing logic. No changes to the LanguageModel interface or consumer-facing API.
…ponse_id When using WebSocket mode with previous_response_id chaining, the server already has the prior conversation context. Previously we sent the full input array every time, which was redundant and incorrect per the spec. Now wsTransport tracks lastInputLen (the number of input items sent in the last successful request). On subsequent calls with previous_response_id, only new items are sent. Additionally, function_call items are filtered out since the server generated those as part of its own response output. On chain breaks (previous_response_not_found errors), lastInputLen resets to 0 so the next call sends the full prompt.
…sponsesFinishReason - Handle response.failed events in both generateViaWebSocket and streamViaWebSocket (previously would return nil error with empty content) - Fix read goroutine leak on context cancellation by setting a read deadline when ctx is done, allowing ReadMessage to unblock - Use mapResponsesFinishReason in generateViaWebSocket to match the HTTP path's handling of incomplete_details reasons
2 tasks
andreynering
reviewed
Mar 6, 2026
5d6aef2 to
d6f5903
Compare
Gemini 3+ uses thinking_level (LOW/MEDIUM/HIGH/MINIMAL) instead of thinking_budget. The two are mutually exclusive per Google's API. Maps to genai.ThinkingConfig.ThinkingLevel already available in v1.45.0.
Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 4 updates: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk), [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [google.golang.org/genai](https://github.com/googleapis/go-genai). Updates `github.com/ardanlabs/kronk` from 1.19.6 to 1.20.1 - [Release notes](https://github.com/ardanlabs/kronk/releases) - [Commits](ardanlabs/kronk@v1.19.6...v1.20.1) Updates `github.com/aws/aws-sdk-go-v2` from 1.41.1 to 1.41.2 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@v1.41.1...v1.41.2) Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.9 to 1.32.10 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@config/v1.32.9...config/v1.32.10) Updates `google.golang.org/genai` from 1.47.0 to 1.48.0 - [Release notes](https://github.com/googleapis/go-genai/releases) - [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md) - [Commits](googleapis/go-genai@v1.47.0...v1.48.0) --- updated-dependencies: - dependency-name: github.com/ardanlabs/kronk dependency-version: 1.20.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all - dependency-name: github.com/aws/aws-sdk-go-v2 dependency-version: 1.41.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.10 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: google.golang.org/genai dependency-version: 1.48.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…racelet#153) Bumps the all group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk). Updates `github.com/ardanlabs/kronk` from 1.20.1 to 1.20.3 - [Release notes](https://github.com/ardanlabs/kronk/releases) - [Commits](ardanlabs/kronk@v1.20.1...v1.20.3) --- updated-dependencies: - dependency-name: github.com/ardanlabs/kronk dependency-version: 1.20.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the all group with 3 updates: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk), [github.com/aws/smithy-go](https://github.com/aws/smithy-go) and [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema). Updates `github.com/ardanlabs/kronk` from 1.20.3 to 1.20.8 - [Release notes](https://github.com/ardanlabs/kronk/releases) - [Commits](ardanlabs/kronk@v1.20.3...v1.20.8) Updates `github.com/aws/smithy-go` from 1.24.1 to 1.24.2 - [Release notes](https://github.com/aws/smithy-go/releases) - [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md) - [Commits](aws/smithy-go@v1.24.1...v1.24.2) Updates `github.com/kaptinlin/jsonschema` from 0.7.3 to 0.7.5 - [Commits](kaptinlin/jsonschema@v0.7.3...v0.7.5) --- updated-dependencies: - dependency-name: github.com/ardanlabs/kronk dependency-version: 1.20.8 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/aws/smithy-go dependency-version: 1.24.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/kaptinlin/jsonschema dependency-version: 0.7.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Note that the user agent defaults to Charm Fantasy/<version>, which means we need to maintain a const for the version number.
Allows overriding the default Bedrock endpoint URL, enabling use with custom or proxy endpoints. 🐙 Generated with Crush Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
bedrock.WithConfig internally calls option.WithBaseURL with the default regional endpoint, clobbering any user-provided base_url. Move the custom base URL append to after the bedrock config block so last-write wins. 🐨 Generated with Crush Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
When useBedrock is true, the DefaultURL ("https://api.anthropic.com") was
being set on providerOptions.baseURL in New(). This caused LanguageModel()
to append option.WithBaseURL("https://api.anthropic.com") after
bedrock.WithConfig had already set the correct bedrock-runtime URL,
overwriting it.
Only set the default base URL for non-bedrock providers. Bedrock gets its
URL from bedrock.WithConfig() in the SDK.
🐙 Generated with Crush
Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
…t#156) Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
…rmbracelet#113) Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 2 updates: [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema) and [google.golang.org/genai](https://github.com/googleapis/go-genai). Updates `github.com/kaptinlin/jsonschema` from 0.7.5 to 0.7.6 - [Commits](kaptinlin/jsonschema@v0.7.5...v0.7.6) Updates `google.golang.org/genai` from 1.50.0 to 1.51.0 - [Release notes](https://github.com/googleapis/go-genai/releases) - [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md) - [Commits](googleapis/go-genai@v1.50.0...v1.51.0) --- updated-dependencies: - dependency-name: github.com/kaptinlin/jsonschema dependency-version: 0.7.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: google.golang.org/genai dependency-version: 1.51.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…mbracelet#184) Bumps the kronk group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk). Updates `github.com/ardanlabs/kronk` from 1.21.3 to 1.21.4 - [Release notes](https://github.com/ardanlabs/kronk/releases) - [Commits](ardanlabs/kronk@v1.21.3...v1.21.4) --- updated-dependencies: - dependency-name: github.com/ardanlabs/kronk dependency-version: 1.21.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: kronk ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…annotations (charmbracelet#187) The Responses API streaming path was missing a handler for "response.output_text.annotation.added" events. This meant that url_citation and file_citation annotations—which carry source URLs and titles for web search results—were silently dropped during streaming. The non-streaming Generate path and the Chat Completions API streaming path both handled annotations correctly; only the Responses API Stream path was affected. Add a case for "response.output_text.annotation.added" that parses the annotation map and yields StreamPartTypeSource parts for url_citation and file_citation types, matching the behavior of the existing Generate path and the Anthropic provider. Update TestResponsesStream_WebSearchResponse to include annotation.added events in the mock stream and assert that source parts are emitted with the correct URL, title, and type.
…ort levels (charmbracelet#186) We were missing some "none", "minimal" and "xhigh" constants, for the `openai`, `openaicompat` and `openrouter` packages. Added the missing checks to ensure they work as well. Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 5 updates: | Package | From | To | | --- | --- | --- | | [cloud.google.com/go/auth](https://github.com/googleapis/google-cloud-go) | `0.18.2` | `0.19.0` | | [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.41.4` | `1.41.5` | | [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.32.12` | `1.32.13` | | [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema) | `0.7.6` | `0.7.7` | | [google.golang.org/genai](https://github.com/googleapis/go-genai) | `1.51.0` | `1.52.0` | Updates `cloud.google.com/go/auth` from 0.18.2 to 0.19.0 - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](googleapis/google-cloud-go@auth/v0.18.2...v0.19.0) Updates `github.com/aws/aws-sdk-go-v2` from 1.41.4 to 1.41.5 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@v1.41.4...v1.41.5) Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.12 to 1.32.13 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@config/v1.32.12...config/v1.32.13) Updates `github.com/kaptinlin/jsonschema` from 0.7.6 to 0.7.7 - [Commits](kaptinlin/jsonschema@v0.7.6...v0.7.7) Updates `google.golang.org/genai` from 1.51.0 to 1.52.0 - [Release notes](https://github.com/googleapis/go-genai/releases) - [Changelog](https://github.com/googleapis/go-genai/blob/v1.52.0/CHANGELOG.md) - [Commits](googleapis/go-genai@v1.51.0...v1.52.0) --- updated-dependencies: - dependency-name: cloud.google.com/go/auth dependency-version: 0.19.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all - dependency-name: github.com/aws/aws-sdk-go-v2 dependency-version: 1.41.5 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.13 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/kaptinlin/jsonschema dependency-version: 0.7.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: google.golang.org/genai dependency-version: 1.52.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the all group with 3 updates: [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/smithy-go](https://github.com/aws/smithy-go) and [google.golang.org/genai](https://github.com/googleapis/go-genai). Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.13 to 1.32.14 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](aws/aws-sdk-go-v2@config/v1.32.13...config/v1.32.14) Updates `github.com/aws/smithy-go` from 1.24.2 to 1.24.3 - [Release notes](https://github.com/aws/smithy-go/releases) - [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md) - [Commits](aws/smithy-go@v1.24.2...v1.24.3) Updates `google.golang.org/genai` from 1.52.0 to 1.52.1 - [Release notes](https://github.com/googleapis/go-genai/releases) - [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md) - [Commits](googleapis/go-genai@v1.52.0...v1.52.1) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.14 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: github.com/aws/smithy-go dependency-version: 1.24.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all - dependency-name: google.golang.org/genai dependency-version: 1.52.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: all ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…mbracelet#194) Bumps the kronk group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk). Updates `github.com/ardanlabs/kronk` from 1.21.4 to 1.22.0 - [Release notes](https://github.com/ardanlabs/kronk/releases) - [Commits](ardanlabs/kronk@v1.21.4...v1.22.0) --- updated-dependencies: - dependency-name: github.com/ardanlabs/kronk dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: kronk ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Wait until streaming has finished before dispatching tool calls to avoid getting tool results without a matching call: essentially orphans.
Add WebSocket transport support for the OpenAI Responses API, enabling lower-latency persistent connections for tool-call-heavy workflows. Key features: - wsTransport manages WebSocket connection lifecycle with automatic reconnection before the 60-minute connection limit - previous_response_id auto-chaining for incremental continuation - generate:false warmup support via GenerateWarmup provider option - Falls back to HTTP transparently on WebSocket connection failure - One in-flight response at a time per connection (mutex-protected) New provider options: - WithWebSocket() enables WebSocket mode (requires WithUseResponsesAPI) - PreviousResponseID on ResponsesProviderOptions for explicit chaining - GenerateWarmup on ResponsesProviderOptions for prefill/warmup The WebSocket events use the same JSON structure as HTTP SSE events, so both Generate() and Stream() reuse existing event parsing logic. No changes to the LanguageModel interface or consumer-facing API.
f9ec69c to
047cd5d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Add WebSocket transport support for the OpenAI Responses API, enabling lower-latency persistent connections for tool-call-heavy workflows.
Key features:
New provider options:
The WebSocket events use the same JSON structure as HTTP SSE events, so both Generate() and Stream() reuse existing event parsing logic.
No changes to the LanguageModel interface or consumer-facing API.
CONTRIBUTING.md.