Skip to content

fix(runtime-sdk): align runtime.llm with the sandbox runtime facade contract - #665

Open
Byc4i wants to merge 1 commit into
chaitin:mainfrom
Byc4i:fix/runtime-llm-facade-contract
Open

fix(runtime-sdk): align runtime.llm with the sandbox runtime facade contract#665
Byc4i wants to merge 1 commit into
chaitin:mainfrom
Byc4i:fix/runtime-llm-facade-contract

Conversation

@Byc4i

@Byc4i Byc4i commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

PR: fix(runtime-sdk): align runtime.llm with the sandbox runtime facade contract

Closes #555

Problem

The guest runtime injects AGENT_COMPOSE_RUNTIME_BASE_URL and
AGENT_COMPOSE_SANDBOX_TOKEN for sandbox-scoped access to the daemon runtime
facade, but runtime.llm consumed neither. It always resolved the base URL
from BASE_URL / HTTP_URL (falling back to http://127.0.0.1:7410, which in
an isolated guest points at the guest itself) and sent no sandbox bearer
token, diverging from the managed runtime environment and from the
token-required runtime facade routes.

Solution

runtime.llm now resolves its endpoint in the contract order from #555:

  1. Explicit baseUrl option — always selects the legacy public
    agentcompose.v2.LLMService/Generate Connect JSON endpoint (backwards
    compatible, no token).
  2. Managed sandbox facade environment — the facade base URL comes from
    OPENAI_BASE_URL / ANTHROPIC_BASE_URL, or is derived from
    AGENT_COMPOSE_RUNTIME_BASE_URL + SANDBOX_ID; the call authenticates with
    AGENT_COMPOSE_SANDBOX_TOKEN (falling back to the family-specific
    OPENAI_API_KEY / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_API_KEY) sent as
    x-api-key. The wire protocol follows LLM_API_PROTOCOL: OpenAI Responses
    by default, Anthropic Messages for anthropic_messages (the
    chat_completions token is also valid for the shared OpenAI facade base).
  3. Legacy chain — without any facade environment, the previous
    BASE_URL / HTTP_URL / AGENT_COMPOSE_BASE_URL /
    AGENT_COMPOSE_HTTP_URL / 127.0.0.1:7410 chain is preserved unchanged.

Structured output maps to strict text.format: json_schema on the OpenAI
Responses facade and to prompt-guided JSON on the Anthropic Messages facade.
Response decoding handles Connect JSON, OpenAI Responses (output_text /
output[].content[].text), and Anthropic Messages (content[].text) shapes.

Secret handling

The facade token is used only to build the request credential header. Error
messages pass through a redactor for every credential environment variable, so
tokens never appear in thrown errors, stdout, stderr, or persisted artifacts.

Testing

  • Unit tests (runtime/agent-compose-runtime-sdk/test/llm-facade.test.ts,
    9 cases): managed URL resolution (family base URL and
    AGENT_COMPOSE_RUNTIME_BASE_URL + SANDBOX_ID derivation), authorization
    header, both wire protocols, outputSchema mapping for both protocols,
    explicit baseUrl override, legacy fallback, missing-model validation,
    timeout, non-2xx responses, and token redaction. Full SDK suite: 50/50 pass,
    tsc --noEmit clean.
  • Host/guest integration test
    (pkg/agentcompose/proxy/runtime_llm_guest_integration_test.go): shapes a
    request exactly as the SDK does inside a managed guest (x-api-key +
    Responses payload against the sandbox facade URL) and verifies the injected
    credential passes facade authorization and reaches the upstream; the same
    request without the credential is rejected with 401. Full
    pkg/agentcompose/proxy suite passes.

Documentation

  • runtime/agent-compose-runtime-sdk/README.md and
    docs/design/agent-compose-runtime_contract.md now describe the same URL
    resolution order, facade authentication, and wire-protocol behavior,
    including the secret-handling guarantee.

Notes for reviewers

  • The legacy LLMService.Generate path is deliberately kept and tested
    (contract item 5): explicit baseUrl and non-managed environments behave
    exactly as before, so existing out-of-sandbox consumers are unaffected.
  • chat_completions resolves to the OpenAI Responses facade endpoint because
    the daemon's runtime facade binds both OpenAI wire protocols to the same
    token and base URL; the handler validates token/protocol compatibility.

…ontract

runtime.llm previously ignored the managed sandbox facade environment and
always called the public LLMService Generate endpoint, which cannot work
from an isolated guest (127.0.0.1 resolves to the guest itself) and never
authenticates with the injected facade token.

Resolve the endpoint in contract order:
1. An explicit baseUrl keeps selecting the legacy LLMService Generate
   endpoint (backwards compatible, no token).
2. The managed facade environment (OPENAI_BASE_URL / ANTHROPIC_BASE_URL,
   or AGENT_COMPOSE_RUNTIME_BASE_URL + SANDBOX_ID) selects the sandbox
   runtime LLM facade, authenticated via x-api-key with
   AGENT_COMPOSE_SANDBOX_TOKEN (falling back to the family-specific
   OPENAI_API_KEY / ANTHROPIC_AUTH_TOKEN / ANTHROPIC_API_KEY). The wire
   protocol follows LLM_API_PROTOCOL: OpenAI Responses by default,
   Anthropic Messages for anthropic_messages.
3. Without facade configuration, the legacy BASE_URL/HTTP_URL chain is
   preserved.

Structured output maps to strict text.format json_schema on the Responses
facade and prompt-guided JSON on the Anthropic facade. Facade tokens are
redacted from thrown errors and never logged.

Cover managed URL resolution, authorization, explicit overrides, missing
configuration, timeout, and non-2xx responses with unit tests, and add a
host-side integration test proving the guest-injected credential reaches
the authenticated sandbox facade. README and the runtime contract document
now describe the same URL and authentication behavior.

Closes chaitin#555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(runtime-sdk): align runtime.llm with the sandbox runtime facade contract

1 participant