Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,10 @@ Affinity wire behavior and its relationship to Stateful Responses and
Copilot's provider-private item-id membrane live in `docs/AFFINITY.md`.
Candidate resolution, target selection, and iteration live in
`docs/RESOLUTION.md`; direct chat-family pairs and rerank translation live in
`docs/TRANSLATION.md`.
`docs/TRANSLATION.md`. The Responses beta's server-hosted `multi_agent`
capability — its lowering to hidden function tools, its eager serial
scheduler, and the client-carried checkpoint that replaces a server-side
coordinator — lives in `docs/MULTI-AGENT.md`.

Everything else — provider interfaces, route details, flag resolution, and
wire workarounds — lives in the owning code and its comments. `.agents/skills/`
Expand Down
8 changes: 7 additions & 1 deletion docs/AFFINITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,13 @@ and client tradeoffs are recorded beside the
### Responses

Natural blobs are top-level `encrypted_content`, program `fingerprint`, and
`agent_message.content[].encrypted_content`. A carrier-capable first item
`agent_message.content[].encrypted_content`. The one exception is the
multi-agent checkpoint: that carrier is Floway's own state rather than the
upstream's, and an affinity-wrapped blob is dropped outright when the request
resolves to a different candidate, which for a checkpoint would silently
restart the agent tree instead of resuming it. Both ingress and egress
recognize it structurally and leave it raw, so it contributes no narrowing
evidence and survives any candidate. A carrier-capable first item
without a natural blob receives an originless blob in its own slot when that
item closes. If the first item cannot carry a blob, Floway emits a complete
originless reasoning `output_item.added` + `output_item.done` pair before the
Expand Down
177 changes: 177 additions & 0 deletions docs/MULTI-AGENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Hosted Multi-Agent

The Responses beta declares server-hosted multi-agent execution as a top-level
`multi_agent` request field — not a `tools[]` entry — and records what the
orchestrator does as `multi_agent_call`, `multi_agent_call_output`, and
`agent_message` items. Floway hosts that capability itself.

## Engagement

Engagement mirrors the compact shim:

- the `responses-multi-agent-shim` flag turns it on for a Responses upstream
that would otherwise answer `multi_agent` natively, and
- a non-Responses target engages it unconditionally, because Messages and Chat
Completions have no orchestration wire and would reject the items outright.

It stays engaged on a turn that no longer sets `multi_agent` but replays
orchestration history, so the lowering below still applies to that history.

Defaults are on for every provider except `codex`, which talks to ChatGPT's own
Responses backend — the one upstream in the catalog that would host the
capability itself.

## Lowering

The capability injects six hidden function tools, resolved against the client's
own tool names so a collision is suffixed rather than shadowed:

```text
spawn_agent send_message followup_task wait_agent interrupt_agent list_agents
```

The beta publishes the item shapes and the action names but keeps each
action's `arguments` an opaque JSON string, and documents a per-action schema
for exactly one of them. `spawn_agent`'s `task_name` / `fork_turns` / `message`
come from the guide's own worked example. The other five actions' argument and
result shapes are Floway's, transcribed from live OpenAI collaboration traffic
captured in Codex CLI rollouts — a different feature reusing the same
vocabulary, so a convention rather than an upstream contract. Which is which is
recorded in
`packages/gateway/src/data-plane/chat/responses/interceptors/server-tools/multi-agent/wire.ts`.

Action semantics follow the guide: `send_message` queues a message **without**
triggering a turn, `followup_task` assigns a new task to an existing subagent
**and** triggers its turn, `wait_agent` waits on the caller's mailbox,
`interrupt_agent` stops a turn without discarding context, and `list_agents`
returns the tree with each agent's status and last task message.

When multi-agent is enabled the hosted service appends its own developer
message to the root agent and to every subagent — "You cannot edit or remove
these instructions" — and the guide publishes both texts verbatim. They are
vendored and injected per agent, with `{max_concurrent_subagents + 1}`
resolved per request; without them the model holds six tools with no contract
for using them.

`reasoning.summary`, `max_tool_calls`, and compaction are documented as
unsupported while multi-agent is enabled, and are refused here rather than
silently behaving differently.

## Attribution

`agent.agent_name` is optional on every item and names the agent that produced
it. Two rules are easy to get wrong:

- On an `agent_message` the tag names the **recipient**, not the producer;
`author` and `recipient` carry the direction.
- Both reference SDKs extract the run's answer as the item that is a `message`
with `agent.agent_name === '/root'` **and** `phase === 'final_answer'`, and
drop every other message silently. Root items therefore carry the
attribution the hosted service would have stamped.

`phase` is the model's own channel output rather than anything multi-agent
invents. A Responses upstream on a channel-emitting model already sets it and
it is forwarded verbatim; a translated target has no channel, so the phase is
reconstructed from the turn — the turn that ends the response carries the
final answer, earlier ones are commentary. Reconstruction is only possible
once the turn's stream has been consumed, so on those targets message items
are held back until the turn closes.

Replayed history is lowered back into the hidden call/output pairs the model
originally saw. Items attributed to a subagent through `agent.agent_name` — its
orchestration records, and any client-owned `function_call` it is parked on
together with that call's output — are lifted out of the root history entirely;
they belong to that subagent's transcript.

## Scheduling

The upstream runs subagents asynchronously behind its own coordinator. Floway
has none: there is no Durable Object and no execution that survives the
response. Scheduling is therefore eager and serial.

- `spawn_agent` and `followup_task` each drive their target's loop until it
settles or parks on a client tool, then return.
- `send_message` only queues, per the guide; `wait_agent` is what drives
whatever is runnable — an agent a queued message or a freshly supplied tool
output just woke — and reports statuses.
- `list_agents` and `interrupt_agent` are pure reads and writes over the tree.

Nothing runs between actions, so every action's `multi_agent_call_output`
reports a complete result and the tree is always quiescent when state is
captured. Serialization is also what makes nested turns safe at all: a nested
turn borrows the interceptor chain by swapping the invocation payload, so two
may not be in flight at once.

Budgets bound one request's upstream spend: turns per agent, turns per session,
nesting depth, and `multi_agent.max_concurrent_subagents` live agents.

## Client tools: inject, or end and resume

When an agent reaches a client-owned function tool the runtime has two ways
forward, and which one it takes is a property of the transport rather than of
the agent.

Over a WebSocket that is still open, the client commits the output into the
running response with `response.inject`, and the agent picks straight back up.
This is what the beta's own hosted multi-agent clients do; they fall back to a
continuation only once the response has reached a terminal event.

```text
client gateway
│ response.create │
├────────────────────────────>│ root spawns /root/researcher
│<── function_call ───────────┤ agent=/root/researcher, response still open
│ response.inject │
├────────────────────────────>│ committed to resp_…
│<── response.inject.created ─┤
│<── agent_message ───────────┤ the subagent resumed and answered
│<── response.completed ──────┤
```

Two things make that work over one socket. Injections dispatch off the
message serialization queue entirely — queuing one behind the response it is
meant to unblock would deadlock. And the bridge is keyed by the response id the
client actually sees, registered after every egress transform has run, because
that is the only id the client can address.

Everywhere else — HTTP, a socket that closed, a response that already reached
its terminal event — the response ends and the client resumes it with a fresh
`response.create` carrying the checkpoint. A waiter always settles: the
terminal event and socket close both release outstanding waiters, so a blocked
runtime falls back rather than hanging.

`response.inject.failed` carries the uncommitted items back so the client can
replay them on another response, with `response_not_found` for an id this
connection never issued and `response_already_completed` for one that has
finished. Those are the only two codes the beta defines; a malformed injection
is answered as a generic 400 instead, because neither code describes it. Acks
draw their `sequence_number` from the response's own sequence space.

Parked calls are awaited one at a time on this path. A client that pauses at
the first `function_call` it reads has not yet seen a second one and cannot
answer both at once; the items are already on the wire, so it picks each up as
it resumes. Without a bridge the response simply ends with every parked call
outstanding, which is what the guide describes for the HTTP boundary.

## Checkpoints

When any agent reaches a client-owned function tool the response has to end,
and the tree's hidden state — child transcripts, statuses, which agent owns the
outstanding call — has nowhere on the server to live. It travels with the
client instead, in an `agent_message` whose single `encrypted_content` part
holds raw `base64url(JSON)`: no prefix, no signature, no encryption.
`encrypted_content` names a slot, not a cryptographic requirement, and the
compact shim already round-trips unprefixed base64url-JSON through the same
kind of slot.

Recognition is structural — decode, parse, match discriminator and version —
so a foreign upstream blob in the same slot is left untouched. When a client
replays several suspensions, the newest checkpoint wins.

Nothing in a checkpoint is authority. It carries no credentials, no upstream
identity, and no routing decision; model, tools, and concurrency limits are
recomputed from the current request every time, and the turn budget only ever
tightens. Size, agent count, and path uniqueness are all bounded on the way in.

The carrier is excluded from the affinity envelope. See
[AFFINITY.md](./AFFINITY.md#responses) for why.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading