Skip to content

feat(openai): add WebSocket mode for Responses API#149

Open
taigrr wants to merge 103 commits intocharmbracelet:mainfrom
taigrr:cd/websocket-mode
Open

feat(openai): add WebSocket mode for Responses API#149
taigrr wants to merge 103 commits intocharmbracelet:mainfrom
taigrr:cd/websocket-mode

Conversation

@taigrr
Copy link
Copy Markdown

@taigrr taigrr commented Feb 24, 2026

Add WebSocket transport support for the OpenAI Responses API, enabling lower-latency persistent connections for tool-call-heavy workflows.

Key features:

  • wsTransport manages WebSocket connection lifecycle with automatic reconnection before the 60-minute connection limit
  • previous_response_id auto-chaining for incremental continuation
  • generate:false warmup support via GenerateWarmup provider option
  • Falls back to HTTP transparently on WebSocket connection failure
  • One in-flight response at a time per connection (mutex-protected)

New provider options:

  • WithWebSocket() enables WebSocket mode (requires WithUseResponsesAPI)
  • PreviousResponseID on ResponsesProviderOptions for explicit chaining
  • GenerateWarmup on ResponsesProviderOptions for prefill/warmup

The WebSocket events use the same JSON structure as HTTP SSE events, so both Generate() and Stream() reuse existing event parsing logic.

No changes to the LanguageModel interface or consumer-facing API.

  • I have read CONTRIBUTING.md.
  • I have created a discussion that was approved by a maintainer (for new features).

Add WebSocket transport support for the OpenAI Responses API, enabling
lower-latency persistent connections for tool-call-heavy workflows.

Key features:
- wsTransport manages WebSocket connection lifecycle with automatic
  reconnection before the 60-minute connection limit
- previous_response_id auto-chaining for incremental continuation
- generate:false warmup support via GenerateWarmup provider option
- Falls back to HTTP transparently on WebSocket connection failure
- One in-flight response at a time per connection (mutex-protected)

New provider options:
- WithWebSocket() enables WebSocket mode (requires WithUseResponsesAPI)
- PreviousResponseID on ResponsesProviderOptions for explicit chaining
- GenerateWarmup on ResponsesProviderOptions for prefill/warmup

The WebSocket events use the same JSON structure as HTTP SSE events,
so both Generate() and Stream() reuse existing event parsing logic.

No changes to the LanguageModel interface or consumer-facing API.
…ponse_id

When using WebSocket mode with previous_response_id chaining, the server
already has the prior conversation context. Previously we sent the full
input array every time, which was redundant and incorrect per the spec.

Now wsTransport tracks lastInputLen (the number of input items sent in
the last successful request). On subsequent calls with previous_response_id,
only new items are sent. Additionally, function_call items are filtered
out since the server generated those as part of its own response output.

On chain breaks (previous_response_not_found errors), lastInputLen resets
to 0 so the next call sends the full prompt.
…sponsesFinishReason

- Handle response.failed events in both generateViaWebSocket and
  streamViaWebSocket (previously would return nil error with empty content)
- Fix read goroutine leak on context cancellation by setting a read
  deadline when ctx is done, allowing ReadMessage to unblock
- Use mapResponsesFinishReason in generateViaWebSocket to match the HTTP
  path's handling of incomplete_details reasons
@taigrr taigrr marked this pull request as ready for review February 24, 2026 23:45
Copy link
Copy Markdown
Member

@andreynering andreynering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just giving this a try. Some tool calls are failing. It always says:

response error:  (code:  )
Image

@taigrr taigrr force-pushed the cd/websocket-mode branch 2 times, most recently from 5d6aef2 to d6f5903 Compare April 20, 2026 14:09
mhpenta and others added 17 commits April 25, 2026 16:27
Gemini 3+ uses thinking_level (LOW/MEDIUM/HIGH/MINIMAL) instead of
thinking_budget. The two are mutually exclusive per Google's API.
Maps to genai.ThinkingConfig.ThinkingLevel already available in v1.45.0.
Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 4 updates: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk), [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [google.golang.org/genai](https://github.com/googleapis/go-genai).


Updates `github.com/ardanlabs/kronk` from 1.19.6 to 1.20.1
- [Release notes](https://github.com/ardanlabs/kronk/releases)
- [Commits](ardanlabs/kronk@v1.19.6...v1.20.1)

Updates `github.com/aws/aws-sdk-go-v2` from 1.41.1 to 1.41.2
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@v1.41.1...v1.41.2)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.9 to 1.32.10
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@config/v1.32.9...config/v1.32.10)

Updates `google.golang.org/genai` from 1.47.0 to 1.48.0
- [Release notes](https://github.com/googleapis/go-genai/releases)
- [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md)
- [Commits](googleapis/go-genai@v1.47.0...v1.48.0)

---
updated-dependencies:
- dependency-name: github.com/ardanlabs/kronk
  dependency-version: 1.20.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-version: 1.41.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.32.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: google.golang.org/genai
  dependency-version: 1.48.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…racelet#153)

Bumps the all group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk).


Updates `github.com/ardanlabs/kronk` from 1.20.1 to 1.20.3
- [Release notes](https://github.com/ardanlabs/kronk/releases)
- [Commits](ardanlabs/kronk@v1.20.1...v1.20.3)

---
updated-dependencies:
- dependency-name: github.com/ardanlabs/kronk
  dependency-version: 1.20.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the all group with 3 updates: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk), [github.com/aws/smithy-go](https://github.com/aws/smithy-go) and [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema).


Updates `github.com/ardanlabs/kronk` from 1.20.3 to 1.20.8
- [Release notes](https://github.com/ardanlabs/kronk/releases)
- [Commits](ardanlabs/kronk@v1.20.3...v1.20.8)

Updates `github.com/aws/smithy-go` from 1.24.1 to 1.24.2
- [Release notes](https://github.com/aws/smithy-go/releases)
- [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md)
- [Commits](aws/smithy-go@v1.24.1...v1.24.2)

Updates `github.com/kaptinlin/jsonschema` from 0.7.3 to 0.7.5
- [Commits](kaptinlin/jsonschema@v0.7.3...v0.7.5)

---
updated-dependencies:
- dependency-name: github.com/ardanlabs/kronk
  dependency-version: 1.20.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/aws/smithy-go
  dependency-version: 1.24.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/kaptinlin/jsonschema
  dependency-version: 0.7.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Note that the user agent defaults to Charm Fantasy/<version>, which
means we need to maintain a const for the version number.
aleksclark and others added 29 commits April 25, 2026 16:27
Allows overriding the default Bedrock endpoint URL, enabling use with
custom or proxy endpoints.

🐙 Generated with Crush

Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
bedrock.WithConfig internally calls option.WithBaseURL with the default
regional endpoint, clobbering any user-provided base_url. Move the
custom base URL append to after the bedrock config block so last-write
wins.

🐨 Generated with Crush

Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
When useBedrock is true, the DefaultURL ("https://api.anthropic.com") was
being set on providerOptions.baseURL in New(). This caused LanguageModel()
to append option.WithBaseURL("https://api.anthropic.com") after
bedrock.WithConfig had already set the correct bedrock-runtime URL,
overwriting it.

Only set the default base URL for non-bedrock providers. Bedrock gets its
URL from bedrock.WithConfig() in the SDK.

🐙 Generated with Crush

Assisted-by: AWS Claude Opus 4.6 via Crush <crush@charm.land>
…t#156)

Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
…rmbracelet#113)

Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 2 updates: [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema) and [google.golang.org/genai](https://github.com/googleapis/go-genai).


Updates `github.com/kaptinlin/jsonschema` from 0.7.5 to 0.7.6
- [Commits](kaptinlin/jsonschema@v0.7.5...v0.7.6)

Updates `google.golang.org/genai` from 1.50.0 to 1.51.0
- [Release notes](https://github.com/googleapis/go-genai/releases)
- [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md)
- [Commits](googleapis/go-genai@v1.50.0...v1.51.0)

---
updated-dependencies:
- dependency-name: github.com/kaptinlin/jsonschema
  dependency-version: 0.7.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: google.golang.org/genai
  dependency-version: 1.51.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…mbracelet#184)

Bumps the kronk group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk).


Updates `github.com/ardanlabs/kronk` from 1.21.3 to 1.21.4
- [Release notes](https://github.com/ardanlabs/kronk/releases)
- [Commits](ardanlabs/kronk@v1.21.3...v1.21.4)

---
updated-dependencies:
- dependency-name: github.com/ardanlabs/kronk
  dependency-version: 1.21.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: kronk
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…annotations (charmbracelet#187)

The Responses API streaming path was missing a handler for
"response.output_text.annotation.added" events. This meant that
url_citation and file_citation annotations—which carry source
URLs and titles for web search results—were silently dropped
during streaming.

The non-streaming Generate path and the Chat Completions API
streaming path both handled annotations correctly; only the
Responses API Stream path was affected.

Add a case for "response.output_text.annotation.added" that
parses the annotation map and yields StreamPartTypeSource parts
for url_citation and file_citation types, matching the behavior
of the existing Generate path and the Anthropic provider.

Update TestResponsesStream_WebSearchResponse to include
annotation.added events in the mock stream and assert that
source parts are emitted with the correct URL, title, and type.
…ort levels (charmbracelet#186)

We were missing some "none", "minimal" and "xhigh" constants, for the `openai`, `openaicompat` and `openrouter` packages.

Added the missing checks to ensure they work as well.

Co-authored-by: Andrey Nering <andreynering@users.noreply.github.com>
Bumps the all group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/auth](https://github.com/googleapis/google-cloud-go) | `0.18.2` | `0.19.0` |
| [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) | `1.41.4` | `1.41.5` |
| [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) | `1.32.12` | `1.32.13` |
| [github.com/kaptinlin/jsonschema](https://github.com/kaptinlin/jsonschema) | `0.7.6` | `0.7.7` |
| [google.golang.org/genai](https://github.com/googleapis/go-genai) | `1.51.0` | `1.52.0` |


Updates `cloud.google.com/go/auth` from 0.18.2 to 0.19.0
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](googleapis/google-cloud-go@auth/v0.18.2...v0.19.0)

Updates `github.com/aws/aws-sdk-go-v2` from 1.41.4 to 1.41.5
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@v1.41.4...v1.41.5)

Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.12 to 1.32.13
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@config/v1.32.12...config/v1.32.13)

Updates `github.com/kaptinlin/jsonschema` from 0.7.6 to 0.7.7
- [Commits](kaptinlin/jsonschema@v0.7.6...v0.7.7)

Updates `google.golang.org/genai` from 1.51.0 to 1.52.0
- [Release notes](https://github.com/googleapis/go-genai/releases)
- [Changelog](https://github.com/googleapis/go-genai/blob/v1.52.0/CHANGELOG.md)
- [Commits](googleapis/go-genai@v1.51.0...v1.52.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/auth
  dependency-version: 0.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-version: 1.41.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.32.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/kaptinlin/jsonschema
  dependency-version: 0.7.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: google.golang.org/genai
  dependency-version: 1.52.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps the all group with 3 updates: [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2), [github.com/aws/smithy-go](https://github.com/aws/smithy-go) and [google.golang.org/genai](https://github.com/googleapis/go-genai).


Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.13 to 1.32.14
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](aws/aws-sdk-go-v2@config/v1.32.13...config/v1.32.14)

Updates `github.com/aws/smithy-go` from 1.24.2 to 1.24.3
- [Release notes](https://github.com/aws/smithy-go/releases)
- [Changelog](https://github.com/aws/smithy-go/blob/main/CHANGELOG.md)
- [Commits](aws/smithy-go@v1.24.2...v1.24.3)

Updates `google.golang.org/genai` from 1.52.0 to 1.52.1
- [Release notes](https://github.com/googleapis/go-genai/releases)
- [Changelog](https://github.com/googleapis/go-genai/blob/main/CHANGELOG.md)
- [Commits](googleapis/go-genai@v1.52.0...v1.52.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-version: 1.32.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: github.com/aws/smithy-go
  dependency-version: 1.24.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
- dependency-name: google.golang.org/genai
  dependency-version: 1.52.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…mbracelet#194)

Bumps the kronk group with 1 update: [github.com/ardanlabs/kronk](https://github.com/ardanlabs/kronk).


Updates `github.com/ardanlabs/kronk` from 1.21.4 to 1.22.0
- [Release notes](https://github.com/ardanlabs/kronk/releases)
- [Commits](ardanlabs/kronk@v1.21.4...v1.22.0)

---
updated-dependencies:
- dependency-name: github.com/ardanlabs/kronk
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: kronk
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Wait until streaming has finished before dispatching tool calls to avoid
getting tool results without a matching call: essentially orphans.
Add WebSocket transport support for the OpenAI Responses API, enabling
lower-latency persistent connections for tool-call-heavy workflows.

Key features:
- wsTransport manages WebSocket connection lifecycle with automatic
  reconnection before the 60-minute connection limit
- previous_response_id auto-chaining for incremental continuation
- generate:false warmup support via GenerateWarmup provider option
- Falls back to HTTP transparently on WebSocket connection failure
- One in-flight response at a time per connection (mutex-protected)

New provider options:
- WithWebSocket() enables WebSocket mode (requires WithUseResponsesAPI)
- PreviousResponseID on ResponsesProviderOptions for explicit chaining
- GenerateWarmup on ResponsesProviderOptions for prefill/warmup

The WebSocket events use the same JSON structure as HTTP SSE events,
so both Generate() and Stream() reuse existing event parsing logic.

No changes to the LanguageModel interface or consumer-facing API.
@taigrr taigrr force-pushed the cd/websocket-mode branch from f9ec69c to 047cd5d Compare April 25, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.