Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions PLUGIN.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,9 +202,9 @@ Lineage-only declaration (#2408): a host that cannot drive the full fork protoco

Incremental turns (#3248; not yet advertised in the plugin manifest — detect by trying the endpoint and handling the 503): a host that keeps its own full local history may stop resending it. `POST /__bili/plugin/turn` (same `/__bili/` gate; headers `x-bili-plugin` + optional `x-bili-plugin-model`, `x-bili-plugin-context-window`, `x-bili-plugin-max-output`) with `{ conversationId, sinceSeq, resync?, system?, path?, request, messages }`. The first turn (or any recovery) sends `sinceSeq: 0` + the full `messages[]` (`resync: true` is implied on seed — a wholesale replacement never 409s and works whether the canonical is empty or already live); every later turn sends ONLY what the client added since the last ack — including the previous assistant reply (client-owned). `request` carries anthropic params (model, max_tokens, temperature, top_p, top_k, stop_sequences, stream, tools, tool_choice, metadata, thinking, service_tier, parallel_tool_calls); `system` is sticky (omit to keep the last one); `path` defaults to `/v1/messages` (bare relative rides the lane's default upstream; the `/bili/<upstream>/…` form is explicit). The response is the upstream body streamed through, with `x-bili-applied-seq` = the canonical length after applying the delta — echo that value as the next turn's `sinceSeq`. `409 { error: "seq-mismatch" | "turn-in-flight", serverSeq }` means the client MUST recover with a resync turn (`sinceSeq: serverSeq` if it still has the history, else `resync: true` full seed). `503` means the lane runs without `storage.unified` (the canonical log lives in the unified messages table). The lane owns no session identity of its own: the assembled request carries the client's own `x-bili-plugin-conversation`, so incremental and normal full-resend traffic for one conversation share one session, one fold ledger and one canonical — switching modes needs no ceremony, and a conversation may interleave them (e.g. fall back to full resend while a resync is pending). Folding, nudges, tools and the plugin tool endpoints all work unchanged on the assembled body; the model never sees the difference. v1 caveats: the canonical log is the session's messages table — a turn is acked after the assembled body is persisted (best-effort immediate flush; a crash in between surfaces as 409 → resync), and the pipeline still walks the assembled prefix (O(full) CPU, cheap and zero-hit — the server-authored fast path is tracked on #3248). A `resync: true` that replaces a diverged canonical records a base-seq epoch (`session_ext` key `inc:base-seq`): the stale branch's rows stay in the append-only log for audit, but cold rebuilds slice past them — one resync heals, restarts included. On legacy-only storage (no ext surface) the marker degrades and cold rebuilds serve the full row union (409 → resync once, as before).

Marker divert (#3248 phase 2 — native extensions): instead of switching the provider URL to the turn endpoint, a host may send the incremental body on its ORDINARY model request. Stamp `x-bili-plugin-turn: 1` on the request (the lane requires `x-bili-plugin` too, as for every plugin signal); the body shape — top-level `conversationId` + `sinceSeq` + `messages`, no top-level `model` — is the final word: a marker'd body that is NOT incremental-shaped rides the normal pipeline byte-identically, so a host may stamp the marker whenever the feature is enabled and only convert turns it wants. The lane forwards the assembled request to the ORIGINAL request path (bare or `/bili/…` — no provider reconfiguration). Capability probe: `GET /__bili/plugin/turn` → `{ ok, protocol: 1, feature: "pluginIncrementalTurns", enabled, storage }` (loopback-gated like every `/__bili/` surface; a 404 = older server, stay full-resend). Gating: rollout feature `pluginIncrementalTurns` (default 0% = every install byte-identical to before) — enable with `rollout.features.pluginIncrementalTurns: true` (or a percent) in the bili config, or `BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1`; the feature additionally requires a live unified sidecar (shadow or full). When disabled mid-flight the lane answers 503 on incremental-shaped turns — fall back to a plain full-resend request. Reference host wiring: pi rides its `before_provider_headers` (marker) + `before_provider_request` (payload replacement) + `after_provider_response` (ack) extension events; dsh-native rides the fetch patch's `beforeSend` (body conversion) + `headersFor` (marker) + `onResponse` (ack). pi-native inherits the pi wiring unchanged (same extension factory — only proxy discovery differs). opencode rides its `http.request` hook (an `incremental` adapter callback appended after routing/stamping swaps the request body via `replaceRequestPayload` and stamps the marker on the replaced request; acks land on the fetch intercept's `onResponse`). omp has no header events at all — its native adapter reads the extension-stamped `prompt_cache_key` body field as the conversation id inside `beforeSend`, stamps the plugin headers directly on the request init (identity upgrades, never forks: wire mode already resolves the same value via `preferPromptCacheKeyIdentity`), and acks on `onResponse`. Non-Anthropic wire shapes (OpenAI `tool_calls`/`role:"tool"`) are detected by `wireMessagesLookAnthropic` and left on full resend — the lane's message ingestion is the anthropic wire. Planned turns are single-in-flight per URL and age out after 30s (a network-dead request must not block the URL forever). The shared client state machine lives in `src/agent/incremental-client.ts` (probe TTL cache, prefix-hash diffing, sticky system, per-conversation dead flag) — new native hosts should reuse it rather than re-implement the diffing.
Marker divert (#3248 phase 2 — native extensions): instead of switching the provider URL to the turn endpoint, a host may send the incremental body on its ORDINARY model request. Stamp `x-bili-plugin-turn: 1` on the request (the lane requires `x-bili-plugin` too, as for every plugin signal); the body shape — top-level `conversationId` + `sinceSeq` + `messages`, no top-level `model` — is the final word: a marker'd body that is NOT incremental-shaped rides the normal pipeline byte-identically, so a host may stamp the marker whenever the feature is enabled and only convert turns it wants. Incremental-SHAPED turns are lane traffic and carry the same trust class as the direct turn endpoint (#3343): non-loopback / tunnel / untrusted-origin callers get 403 on them (their capability probe already 403s, so a well-behaved host never sends deltas from afar), and management prefixes (`/__bili/`, `/__acp/`) never ride the divert. The lane forwards the assembled request to the ORIGINAL request path (bare or `/bili/…` — no provider reconfiguration). Capability probe: `GET /__bili/plugin/turn` → `{ ok, protocol: 1, feature: "pluginIncrementalTurns", enabled, storage }` (loopback-gated like every `/__bili/` surface; a 404 = older server, stay full-resend). Gating: rollout feature `pluginIncrementalTurns` (default 0% = every install byte-identical to before) — enable with `rollout.features.pluginIncrementalTurns: true` (or a percent) in the bili config, or `BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1`; the feature additionally requires a live unified sidecar (shadow or full). When disabled mid-flight the lane answers 503 on incremental-shaped turns — fall back to a plain full-resend request. Reference host wiring: pi rides its `before_provider_headers` (marker) + `before_provider_request` (payload replacement) + `after_provider_response` (ack) extension events; dsh-native rides the fetch patch's `beforeSend` (body conversion) + `headersFor` (marker) + `onResponse` (ack). pi-native inherits the pi wiring unchanged (same extension factory — only proxy discovery differs). opencode rides its `http.request` hook (an `incremental` adapter callback appended after routing/stamping swaps the request body via `replaceRequestPayload` and stamps the marker on the replaced request; acks land on the fetch intercept's `onResponse`). omp has no header events at all — its native adapter reads the extension-stamped `prompt_cache_key` body field as the conversation id inside `beforeSend`, stamps the plugin headers directly on the request init (identity upgrades, never forks: wire mode already resolves the same value via `preferPromptCacheKeyIdentity`), and acks on `onResponse`. Non-Anthropic wire shapes (OpenAI `tool_calls`/`role:"tool"`) are detected by `wireMessagesLookAnthropic` and left on full resend — the lane's message ingestion is the anthropic wire. Planned turns are single-in-flight per URL and age out after 30s (a network-dead request must not block the URL forever). The shared client state machine lives in `src/agent/incremental-client.ts` (probe TTL cache, prefix-hash diffing, sticky system, per-conversation dead flag) — new native hosts should reuse it rather than re-implement the diffing.

中文增量要点(#3248):`POST /__bili/plugin/turn`,首次/恢复用 `sinceSeq:0` 全量(种子隐含 `resync:true`,永不 409);此后每轮只发新增(含上一轮 assistant 回复,客户端自持)。响应即上游原文,`x-bili-applied-seq` 为应用后的长度,下一轮作 `sinceSeq` 回传。409 必须按 serverSeq 对齐或全量重同步;503 表示未开 storage.unified。会话身份沿用客户端自己的 conversation id——增量与全量重发同一会话可任意切换,折叠/工具/状态栏不变。阶段 2(标记分流):普通模型请求上盖 `x-bili-plugin-turn: 1` 头 + 增量形状 body(顶层 conversationId/sinceSeq/messages、无顶层 model)即可,lane 按原始 URL 转发;探针 `GET /__bili/plugin/turn`;开关 rollout `pluginIncrementalTurns`(默认 0%=零变化,`BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1` 或配置强制开)。客户端状态机在 src/agent/incremental-client.ts,新宿主复用勿重写。已接宿主:pi/pi-native(同一扩展工厂,后者零代码继承)、dsh-native(fetch 补丁 beforeSend/headersFor/onResponse)、opencode(http.request 钩子 incremental 适配回调换 body+盖章)、omp(无头事件通道,读 body 里 prompt_cache_key 当会话 id,直接在 init 上盖章);OpenAI 工具形状(tool_calls/role:tool)自动回退全量重发;每 URL 单飞行+30s 过期。
中文增量要点(#3248):`POST /__bili/plugin/turn`,首次/恢复用 `sinceSeq:0` 全量(种子隐含 `resync:true`,永不 409);此后每轮只发新增(含上一轮 assistant 回复,客户端自持)。响应即上游原文,`x-bili-applied-seq` 为应用后的长度,下一轮作 `sinceSeq` 回传。409 必须按 serverSeq 对齐或全量重同步;503 表示未开 storage.unified。会话身份沿用客户端自己的 conversation id——增量与全量重发同一会话可任意切换,折叠/工具/状态栏不变。阶段 2(标记分流):普通模型请求上盖 `x-bili-plugin-turn: 1` 头 + 增量形状 body(顶层 conversationId/sinceSeq/messages、无顶层 model)即可,增量形状 turn 属 lane 流量,与直连 turn 端点同信任级(#3343):非回环/隧道/不可信源一律 403(其探针本就 403,正常宿主不会远程发增量);管理前缀(`/__bili/`、`/__acp/`)永不走分流。lane 按原始 URL 转发;探针 `GET /__bili/plugin/turn`;开关 rollout `pluginIncrementalTurns`(默认 0%=零变化,`BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1` 或配置强制开)。客户端状态机在 src/agent/incremental-client.ts,新宿主复用勿重写。已接宿主:pi/pi-native(同一扩展工厂,后者零代码继承)、dsh-native(fetch 补丁 beforeSend/headersFor/onResponse)、opencode(http.request 钩子 incremental 适配回调换 body+盖章)、omp(无头事件通道,读 body 里 prompt_cache_key 当会话 id,直接在 init 上盖章);OpenAI 工具形状(tool_calls/role:tool)自动回退全量重发;每 URL 单飞行+30s 过期。

中文集成要点:fork 协议现为 2(服务端同时接受 1);先读取 manifest 能力及 snapshot,按原文、role、工具身份确认匹配前缀,仅提交该前缀的 orderedMessages,不猜测多模态分支点。被 `compress.protectedTools` 硬排除的消息在两个数组中均为 `"ref": null`(参与顺序与身份、无地址 ref),与其他前缀条目一样逐字提交;ref 合法值仅 `mNNNNN` 字符串或 `null`。父 revision 变化必须重新取快照并重新匹配。子会话使用独立稳定 id,首次请求可追加尚未出站的模型回复,不能重置继承的 refs/blocks。手动摘要使用 snapshot 的 refs 和 `expectedRevision: snapshot.parentRevision` 调用公开 tool;摘要期间历史发生变化时返回 409,不执行旧摘要。状态栏读取 `contextTokens`、`contextTokensSource`、`contextTokensAt`、`model`、`contextLimit`、`contextGeneration`、`sessionId` 与 `sessionRevision`,不得用累计账单 input/cache 冒充有效上下文。手动 compress/decompress 后为独立时间及 generation 的 estimate,不改累计账单或 last-usage 基线;下一次正数 usage 才恢复 usage,零值或缺失 usage 不得复活旧观测。未知来源返回 unavailable,tokens/time/generation 为 null。

Expand Down
7 changes: 6 additions & 1 deletion src/incremental-lane.ts
Original file line number Diff line number Diff line change
Expand Up @@ -431,7 +431,12 @@ export async function handleIncrementalTurn(
method: "POST",
url: path,
headers,
socket: { localPort: (req.socket as { localPort?: number } | undefined)?.localPort, remoteAddress: "127.0.0.1", destroy: (): void => { synthetic.destroy?.(); } },
// #3343: the REAL caller's socket, never a forged loopback one —
// every downstream gate (admin plane, tunnel admission, provenance)
// must judge the actual peer. Loopback topology is unchanged (real
// == 127.0.0.1); a non-loopback peer degrades to ordinary model
// traffic and can no longer ride onto the management plane.
socket: req.socket,
});

// Ack header: inject ahead of whatever status the pipeline writes.
Expand Down
12 changes: 11 additions & 1 deletion src/server/admin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,16 @@ function adminTrustedHostnames(bindHost: string): Set<string> {
return new Set(names);
}

/** #3343: the management-plane prefix test — the two internal prefixes the
* loopback + trusted-origin + anti-tunnel gate protects (handleAdminRoute).
* Shared with the incremental-lane marker divert in handle.ts, which must
* never accept these paths: a diverted turn re-enters the pipeline carrying
* the ORIGINAL request URL, so an unfiltered divert hands a non-loopback
* caller a request onto the management prefixes. */
export function isManagementPath(url: string | undefined): boolean {
return url !== undefined && (url === "/__bili/" || url.startsWith("/__bili/") || url === "/__acp/" || url.startsWith("/__acp/"));
}

// #2170 split-session canary: one log line per conversation base per process.
const splitWarnedBases = new Set<string>();

Expand All @@ -145,7 +155,7 @@ export async function handleAdminRoute(req: http.IncomingMessage, res: http.Serv
// #1073/#3245: self-addressed absolute-form requests arrive already
// rewritten to origin form by handle() (selfAddressedLocalPath), so the
// gates below always judge a path, never a full URL.
const isAdminPath = req.url === "/__bili/" || req.url?.startsWith("/__bili/") || req.url === "/__acp/" || req.url?.startsWith("/__acp/");
const isAdminPath = isManagementPath(req.url);
// #409: management must never be reachable THROUGH the bili tunnel, not
// even from a loopback client: the tunnel's inner connection originates
// from the proxy itself, so the remoteAddress gate alone is satisfied and
Expand Down
Loading
Loading