feat(widget): add self-hosted orchestrator support via orchestrator-url attribute

[a1anfan]

Stacked on #921. Adds self-hosted orchestrator support to the Convai widget: setting the new orchestrator-url attribute connects the widget to a self-hosted orchestrator instead of the ElevenLabs cloud, and the optional orchestrator-agent-config attribute carries the exported agent configuration JSON (both the agent_config_dict/tools_config_list and agent_config/tools_config key spellings are accepted, plus an optional top-level bedrock_inference_profile). The parsed config feeds the client SDK's orchestrator session config from #921; the connection is forced to websocket, since self-hosted orchestrators only expose the conversation WebSocket.

Because self-hosted deployments have no HTTP API to serve a widget appearance config, orchestrator sessions skip the config fetch entirely and use a built-in default appearance (transcript and text input enabled), which the existing attributes and override-config can still override. The file upload button stays hidden by default, so no request leaves the customer network. Both changes are inert unless orchestrator-url is set; cloud behavior is untouched.

Testing

• unit tests
• build, lint, check-types across widget-core and widget-embed
• browser e2e and manual testing against a deployed orchestrator

---

Note

Medium Risk
Changes core session and widget-config wiring and connection selection; cloud paths are gated on orchestrator being disabled, but mis-parsed orchestrator JSON or attribute precedence could block sessions or surprise self-hosted users.

Overview
Adds experimental self-hosted orchestrator support to the Convai widget via orchestrator-url and optional orchestrator-agent-config (exported agent JSON mapped to the client SDK OrchestratorConfig shape). When set, the widget skips the cloud HTTP widget-config fetch, uses a built-in default appearance (still overridable via override-config), builds a websocket-only session with orchestrator on the client SDK, and warns if agent-id / signed-url are also present.

Orchestrator mode also disables cloud-only features: file upload and end-of-call feedback are turned off so traffic does not hit ElevenLabs APIs. Language handling exposes languageOverride for orchestrator session agent overrides when the user can pick a language.

^{Reviewed by Cursor Bugbot for commit e1345ec. Bugbot is set up for automated code reviews on this repo. Configure here.}

[a1anfan]

cursor review

[Copilot]

Pull request overview

Adds on-prem deployment support for the ConvAI widget by introducing new attributes that route sessions to a self-hosted orchestrator WebSocket and by providing a built-in default widget appearance when no HTTP config endpoint exists.

Changes:

• Add on-prem-url and on-prem-agent-config attributes and wire them into widget/session config providers.
• Introduce parseOnPremConfig (with unit tests) to map exported agent JSON into the client SDK’s onPremConfig.
• Add a changeset to release updated widget packages with the new on-prem functionality.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/convai-widget-core/src/utils/parseOnPremConfig.ts	New helper to parse on-prem agent JSON into OnPremConfig for the client SDK.
packages/convai-widget-core/src/utils/parseOnPremConfig.test.ts	Unit tests covering expected key mappings and invalid JSON handling.
packages/convai-widget-core/src/types/attributes.ts	Adds the new on-prem custom attributes to the allowed attribute list.
packages/convai-widget-core/src/contexts/widget-config.tsx	Skips HTTP widget config fetch in on-prem mode and uses a built-in default appearance config.
packages/convai-widget-core/src/contexts/session-config.tsx	Creates onPremConfig session configs and forces websocket connection type when on-prem-url is set.
.changeset/olive-poems-brake.md	Releases convai-widget-core and convai-widget-embed with on-prem support changes.

Suppressed comments (1)

packages/convai-widget-core/src/utils/parseOnPremConfig.ts:31

• on-prem-agent-config values are parsed from a string attribute and then forwarded into OnPremConfig fields (agentConfig, toolsConfigList, etc.). The current implementation forwards whatever types are present, which can produce invalid wire payloads (e.g., tools_config_list being an object instead of an array) and hard-to-debug orchestrator errors. Consider validating/coercing the expected shapes and dropping invalid fields instead of passing them through.

  try {
    const parsed = JSON.parse(agentConfigJSON);
    return {
      conversationUrl,
      agentConfig: parsed.agent_config_dict ?? parsed.agent_config ?? undefined,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[a1anfan]

cursor review

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/convai-widget-core/src/contexts/session-config.tsx:124

• languageAttribute.value is force-cast to the client SDK Language type. Since HTML attributes are free-form strings, this can pass invalid language codes through to the SDK/orchestrator and cause hard-to-diagnose session-start failures.

Prefer validating the attribute (e.g., using the existing isValidLanguage helper in src/types/languages) and only setting overrides.agent.language when it’s valid; otherwise omit the field (and optionally console.warn that the value was ignored).

          agent: {
            ...overrides.value?.agent,
            language: (languageAttribute.value as Language) || undefined,
          },

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 58c167c. Configure here.}

[Copilot]

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated no new comments.

Suppressed comments (1)

packages/convai-widget-core/src/utils/parseOrchestratorConfig.ts:32

• parseOrchestratorConfig normalizes http(s) to ws(s) but does not validate that the resulting url is actually a valid ws:///wss:// URL (e.g. ftp://... or a malformed value will pass through and fail later during WebSocket connection). Consider validating with new URL(...) and returning null with a clear error when the protocol is not ws/wss or the URL cannot be parsed.

  const url = rawUrl
    .replace(/^https:\/\//, "wss://")
    .replace(/^http:\/\//, "ws://");

  if (!agentConfigJSON) {
    return { url };
  }

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 2ad172b. Configure here.}

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 9e21fd1. Configure here.}

[kalvin807]

logic make sense. not a fan of this structure (session-config is already too long and complex). can we isolate orchestrator logic into a context?

import { computed, ReadonlySignal, useSignalEffect } from "@preact/signals";
import { ComponentChildren } from "preact";
import { createContext, useMemo } from "preact/compat";
import type { OrchestratorConfig } from "@elevenlabs/client";

import { useAttribute } from "./attributes";
import { useContextSafely } from "../utils/useContextSafely";
import { parseOrchestratorConfig } from "../utils/parseOrchestratorConfig";

const OrchestratorContext =
  createContext<ReadonlySignal<OrchestratorConfig | null> | null>(null);

export function OrchestratorProvider({ children }: { children: ComponentChildren }) {
  const url = useAttribute("orchestrator-url");
  const agentConfig = useAttribute("orchestrator-agent-config");
  const agentId = useAttribute("agent-id");
  const signedUrl = useAttribute("signed-url");

  const value = useMemo(
    () =>
      computed(() =>
        url.value ? parseOrchestratorConfig(url.value, agentConfig.value) : null
      ),
    []
  );

  useSignalEffect(() => {
    if (url.value && (agentId.value || signedUrl.value)) {
      console.warn(
        "[ConversationalAI] orchestrator-url takes precedence; agent-id and signed-url are ignored"
      );
    }
  });

  return (
    <OrchestratorContext.Provider value={value}>{children}</OrchestratorContext.Provider>
  );
}

export function useOrchestrator() {
  return useContextSafely(OrchestratorContext);
}

export function useIsOrchestratorSession() {
  const orchestrator = useOrchestrator();
  return useComputed(() => orchestrator.value !== null);
}

[kalvin807]

I think this can burn down to just this logic

const languageOverride = isValidLanguage(overrideLanguage.value)
  ? overrideLanguage.value
  : undefined;

[kalvin807]

also tagging @giannagerton for widget related changes

[a1anfan]

cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 266a924. Configure here.}

[Copilot]

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated no new comments.

Suppressed comments (2)

packages/convai-widget-core/src/utils/parseOrchestratorConfig.ts:55

• OrchestratorConfig.url is documented as a WebSocket URL, but parseOrchestratorConfig currently accepts any non-empty string (e.g. example.com, ftp://...) and will pass it through to the client. Validate that the normalized URL starts with ws:// or wss:// and fail early with a clear error.

  const url = rawUrl
    .replace(/^https:\/\//, "wss://")
    .replace(/^http:\/\//, "ws://");

packages/convai-widget-core/src/contexts/orchestrator-config.tsx:44

• orchestrator-url is treated as enabled for any truthy string, including whitespace. That can inadvertently switch the widget into orchestrator mode while passing an empty/invalid URL into parseOrchestratorConfig, resulting in a broken session config and skipped cloud config fetch. Trim the attribute and base enabled/config on the trimmed value.

  const value = useMemo(
    () => ({
      enabled: computed(() => !!url.value),
      config: computed(() =>
        url.value ? parseOrchestratorConfig(url.value, agentConfig.value) : null

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 732f553. Configure here.}