Skip to content

Port billion-context-pi's no-server mode: inline in-process compression lane for pi - #3284

Merged
ranxianglei merged 7 commits into
masterfrom
2026-10-15_pi-inline-mode
Oct 16, 2026
Merged

ranxianglei merged 7 commits into
masterfrom
2026-10-15_pi-inline-mode

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Closes #3278

What

Re-implements (not replicates) the legacy standalone package's no-server mode as an opt-in lane of the existing pi-native plugin: BILI_PI_INLINE=1 makes the pi extension compress entirely IN-PROCESS — no proxy process, no ephemeral port, no loopback hop.

How

  • refactor(server): the pre-traffic half of startServer() becomes an exported createEngineContext() (tee logger, kernel core, instance identity, watcher seed, session reload/lineage reconcile, GC timer, registry prefetch, admin ctx). startServer consumes it unchanged — pure move.
  • src/inline-engine.ts — boots the SAME EngineContext without a socket and dispatches each request through handle() directly. Upstream egress runs under withFetchTransport(rawFetch) so nested fetches bypass the patched chain (no self-loop).
  • src/agent/pi-inline.ts — a fetch-chain link installed BELOW the native intercept: serves the pseudo-origin http://billion-context.inline in-process; every other host passes through untouched. Model traffic AND the /__bili/+/__acp/ tool surface share one pipeline.
  • src/agent/pi-native.ts — planNativePi gains mode "inline"; precedence kill-switch > /bili/ rewrite launch > inline > attach > spawn. The pseudo-origin is frozen synchronously (no health probe can ever target a host that never listens). Degradation: if the engine cannot boot, model traffic passes direct (uncompressed) with one loud log; management calls reject until recovery (30s retry cooldown). WS intercept stays unpatched in inline mode (no proxy bridge exists).
  • Default off: no env and no pi.inline = today's spawn/attach behavior byte-for-byte. The bili pi launcher keeps passing BILLION_CONTEXT_PROXY as before; an EXPLICIT inline opt-in (env or file) still outranks that ambient attach target — in that case compression runs in the pi process and the launcher-spawned proxy sits idle until the pi process exits.

config surface

key type default category why the existing surface cannot express it
BILI_PI_INLINE / pi.inline boolean-ish (1/true) off host posture, env-only + config-file home under the existing top-level pi section (owner ask 2026-10-15) The question is WHERE compression runs (this process vs a spawned proxy). No providers-table field, compress-hierarchy key or launcher flag expresses "no origin at all"; BILLION_CONTEXT_PROXY attach targets a real origin by construction.
pi.inline (file) boolean false same knob, file resolution step Resolved env > file > default(off): BILI_PI_INLINE=1/true enables over the file, BILI_PI_INLINE=0/false explicitly disables over it; kill switches stay above both. Read through loadConfigFile() (same loader/validation surface as pi.subagents).

Migration: none — additive knob, default off. Seed entry added to website/config-reference/env-only.yaml; CONFIGURATION.md/zh regenerated (docs-sync guard green locally).

Behavior changes (disclosure)

  • New opt-in env var only; zero behavior change when unset.
  • startServer refactor is a pure extraction (admin-probe boot suite green).
  • Internal cost: dist/agent/pi-native.js grows ~2.3MB → 4.6MB (the server pipeline is now inlined into that bundle; sibling agent bundles already sit at 2–4.5MB).

Verification (local, targeted)

  • node --import tsx --test tests/pi-inline.test.ts tests/pi-native.test.ts tests/admin-probe.test.ts → 21/21 pass

  • npm run typecheck clean; npm run build success (dist-import-annotations OK)

  • tools/gen-config-docs.mjs check-coverage + scripts/config-surface-guard.mjs green

  • Acceptance probe (owner ask: 本地测试任务压缩能正常跑): drove a 10-turn growing task through the in-process engine against a mock upstream — 3 kernel compress cycles executed in-process, summary carrier [Compressed conversation section] present in 8/13 upstream-bound bodies, post-fold body shrank 51944→46380 bytes, 1 session tracked in-process:

    {"turnsDriven":10,"upstreamRequests":13,"compressCycles":3,"bodiesCarryingSummary":8,"maxBodyBytesBeforeFirstFold":51944,"minBodyBytesFromFirstFoldOn":46380,"shrankAfterFold":true,"inProcessSessions":1}

  • Full-suite verdict: remote CI on this head.

Follow-up on this branch (CI triage)

Follow-up commit 38aee3b (owner ask: JSON config home)

pi.inline added to the existing top-level pi section of billion-context.json (next to pi.subagents, #2230): { "pi": { "inline": true } }. planNativePi now resolves env > file > default(off) and stays pure over its inputs (the module wiring passes loadConfigFile().pi?.inline === true); docs seed + regenerated CONFIGURATION.md/zh; precedence pinned by a new test matrix (file-alone enable, explicit env off/on over the file, kill switches over the file). Local: pi-native 17/17, typecheck/build/docs guards green.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-15_pi-inline-mode (c1a64af)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3857-pr.3284.6bbdd06

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3284.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。open+closed 全量按「inline / no-server / 非 server / 进程内 / BILI_PI_INLINE」检索,同题条目仅 #3278(本 PR 的原始需求,PR 已 Closes 它),无并行实现或重复立项。

这是 [PR] issue(镜像 PR #3284),我不重做工作;进入 PR 审核流程:拉分支逐文件审 diff、审原 issue 方向与回归面、核对 CI。结论按结构化八问回复在本线程。

ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
Review of #3284 found the inline lane ignored the caller's AbortSignal.
Spawn mode cancels via socket close (res "close" -> relay clientAbort),
but InlineRes.destroy() marked writableEnded=true, so the pipeline read
the close as normal completion and ran the upstream round trip to
completion after the caller went away; the dispatch promise resolved
with a phantom response instead of rejecting.

- InlineRes mirrors http.ServerResponse socket-death semantics: destroy
  does not end the response; write/end are no-ops once dead
- makeDispatcher races the pipeline against init.signal and rejects with
  the abort reason like undici would
- pi-inline transport rethrows AbortError instead of degrading: a user
  cancel must not consume the one-shot engine-down warning
- e2e tests move to the real model lane (max_tokens>200 + slow marker)
  and assert the upstream round trip never completes post-abort
@ranxianglei
ranxianglei force-pushed the 2026-10-15_pi-inline-mode branch from f690d9a to fd0e5ff Compare October 15, 2026 13:18
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
Review of #3284 found the inline lane ignored the caller's AbortSignal.
Spawn mode cancels via socket close (res "close" -> relay clientAbort),
but InlineRes.destroy() marked writableEnded=true, so the pipeline read
the close as normal completion and ran the upstream round trip to
completion after the caller went away; the dispatch promise resolved
with a phantom response instead of rejecting.

- InlineRes mirrors http.ServerResponse socket-death semantics: destroy
  does not end the response; write/end are no-ops once dead
- makeDispatcher races the pipeline against init.signal and rejects with
  the abort reason like undici would
- pi-inline transport rethrows AbortError instead of degrading: a user
  cancel must not consume the one-shot engine-down warning
- e2e tests move to the real model lane (max_tokens>200 + slow marker)
  and assert the upstream round trip never completes post-abort
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR 审核完成。结论先行:方向正确、实现质量高;审核中发现并修复了 1 个真实缺陷(调用方取消不传播);CI 转绿即可合并。

结构化八问

  1. 是什么问题:把旧 billion-context-pi 的无服务模式移植为 bili 的 opt-in lane —— BILI_PI_INLINE=1 时 pi 扩展完全在进程内压缩,不起代理进程、不占临时端口、不走 loopback。默认关。
  2. 什么原因导致的(需求侧,移植 billion-context-pi 版本的非 server 模式 #3278):旧独立包有无服务器模式但双仓库维护要跨仓 npm 往返;要求重新实现(非复制)进 bili,接现有 API,默认关,移植后本地验证任务压缩能跑。
  3. 影响范围与严重程度:BILI_PI_INLINE 未设置时零影响(planNativePi 决策表测试逐分支钉住 kill-switch > rewrite-off > inline > attach > spawn);设置后模型流量与 /__bili/+/__acp/ 管理流量共用同一进程内管线。内部成本:dist/agent/pi-native.js 2.3MB→4.6MB(本地构建实测 4,794,914B,与兄弟 agent bundle 同量级)。
  4. 怎么修复的(机制层):
    • createEngineContext() 抽出 startServer 的流量前半场(逐行核对为纯移动,admin-probe 套件绿);
    • src/inline-engine.ts 无 socket 启动同一 EngineContext、直接经 handle() 派发,整个派发包在 withFetchTransport(rawFetch) 下,管线出站永不重入 patched fetch 链(防自环);
    • pi-inline.ts 在 native intercept 之下装一个 fetch 链链接服务伪源 http://billion-context.inline;伪源同步冻结,我验证了不存在任何能打到它的探测路径(readyOrigin 短路 + origin 重绑钩子仅 attach 模式存在)。
  5. 是否有回归问题:逐项审过——startServer 纯提取(行为保持);默认关路径 spawn/attach 逐字节不变;WS intercept 在 inline 不 patch(PR 已披露,合理——没有桥可打);admin DNS-rebinding Host 门未削弱(回环 host 头技巧,门逻辑本身未动);缓存/重试/游标/去重不受影响。审核中发现 1 个真实缺陷并已修复:inline lane 原本忽略调用方 AbortSignal——spawn 模式下客户端断开经 socket close 取消上游回合(relay 的 clientAbort),而 inline 的 InlineRes.destroy() 置了 writableEnded=true,管线把它当正常完成,调用方离开后上游仍跑到底(白烧 token),dispatch promise 还以幻影响应 resolve 而非 reject。行为变更披露:旧值 = 取消被忽略(上游跑完 + 幻影响应)→ 新值 = 与 undici/spawn 模式一致的中止(res.destroy 对齐 http.ServerResponse 的 socket-death 语义 + dispatch 对 signal 做 race 并以 AbortError reject + transport 层 AbortError 直接上抛、不消耗一次性降级告警)。修复 commit a4d537397(3 文件 +102/-17),新增 e2e 断言 abort 后上游回合绝不完成(slow 标记 + 完成断言,两层缺陷都兜得住)。本地定向 22/22 过 + typecheck/build 干净。
  6. 是否破坏缓存:不涉及——会话持久化、admin、缓存路径均未触碰。
  7. 是否改了设计方向:不涉及——纯新增 opt-in lane,bili pi launcher 仍照旧 spawn 代理。
  8. 是否引入新配置:BILI_PI_INLINE(env-only,host posture 类,与 BILI_NATIVE_PI 同族),默认 off,无迁移;CONFIGURATION.md/zh + env-only.yaml 种子已加,config surface 章节齐备。

其他审查记录

判定:CI 转绿后可合并(人工合并),无阻塞项。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

进度汇报(刚才进程被重启是因为我在跑一个过长的 CI 轮询循环,是我的调度错误,已改为一次性查询):

CI 已全绿:新头 a4d537397 上 43/43 check run 全部 success,0 失败、0 待定(含 ubuntu/windows × node 22/24 × 3 shard 全测试矩阵 + static-gates + 各 guard lane)。

当前状态汇总:

  • 审核结论不变(见上一条八问回复):方向正确,审核中发现的取消传播缺陷已修复并推送(a4d537397,即「fix(pi): propagate caller abort into the inline engine」commit)。
  • 本地定向验证 22/22 + typecheck + build 干净;远程全量 CI 现已确认转绿。
  • 分支与 master 无冲突(rebase 后干净合并树已验证)。

判定维持:可合并(人工合并)。PR 地址:#3284

awork added 7 commits October 16, 2026 00:15
The pre-traffic half of startServer (tee logger, kernel core, instance
identity, watcher seed, session reload/lineage reconcile, GC timer,
registry prefetch, admin ctx) becomes an exported EngineContext factory.
startServer consumes it unchanged — pure move, zero behavior change.
…off)

#3278: port of billion-context-pi's no-server mode, re-implemented rather
than replicated. With BILI_PI_INLINE=1 the pi extension compresses
IN-PROCESS: pi-inline.ts installs a fetch-chain link below the native
intercept that serves the pseudo-origin (http://billion-context.inline)
from inline-engine.ts, which boots the same engine context startServer
builds and dispatches each request through handle() directly — model
traffic AND the /__bili//__acp tool surface reuse one pipeline, no proxy
process, no ephemeral port, no loopback hop. Upstream egress inside the
engine runs under withFetchTransport(rawFetch) so nested fetches bypass
the patched chain (no self-loop). Degradation: if the engine cannot boot,
model traffic passes direct (uncompressed) with one loud log; management
calls reject until it recovers (30s retry cooldown).

planNativePi precedence: kill-switch > /bili/ rewrite launch > inline >
attach > spawn — inline is a deliberate choice about WHERE compression
runs, so it outranks ambient attach envs. WS intercept stays unpatched in
inline mode (no proxy bridge exists). Default off: unset = today's
spawn/attach behavior byte-for-byte.
tests/pi-inline.test.ts (hermetic XDG homes, mock upstream): transport
routing (pseudo-origin served in-process, other hosts untouched),
degradation (engine failure -> model traffic direct, mgmt rejects), and
an end-to-end in-process pipeline run (real engine boot, compressed
session tracked, admin health reachable via pseudo-origin, exactly one
upstream hit per request proving no fetch loop).
tests/pi-native.test.ts: plan gating matrix + armNativePi(inline) freezes
the pseudo-origin without spawning.
Seed entry in website/config-reference/env-only.yaml + regenerated
CONFIGURATION.md/zh (docs-sync guard). Host-posture category, default off,
values 1/true.
Review of #3284 found the inline lane ignored the caller's AbortSignal.
Spawn mode cancels via socket close (res "close" -> relay clientAbort),
but InlineRes.destroy() marked writableEnded=true, so the pipeline read
the close as normal completion and ran the upstream round trip to
completion after the caller went away; the dispatch promise resolved
with a phantom response instead of rejecting.

- InlineRes mirrors http.ServerResponse socket-death semantics: destroy
  does not end the response; write/end are no-ops once dead
- makeDispatcher races the pipeline against init.signal and rejects with
  the abort reason like undici would
- pi-inline transport rethrows AbortError instead of degrading: a user
  cancel must not consume the one-shot engine-down warning
- e2e tests move to the real model lane (max_tokens>200 + slow marker)
  and assert the upstream round trip never completes post-abort
Owner ask on #3278: make the no-server mode settable from the JSON
config file under the existing top-level "pi" section.

- FileConfig.pi gains inline?: boolean (#3278); resolved env > file >
  default(off) — BILI_PI_INLINE=1/true enables, BILI_PI_INLINE=0/false
  explicitly disables over the file value
- planNativePi takes the file value as a second argument (pure over its
  inputs, tests stay hermetic); module wiring passes
  loadConfigFile().pi?.inline === true
- docs seed (server.yaml) + regenerated CONFIGURATION.md/zh; coverage
  and config-surface guards green
@ranxianglei
ranxianglei force-pushed the 2026-10-15_pi-inline-mode branch from 38aee3b to c1a64af Compare October 15, 2026 16:21
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解决:分支已 rebase 到当前 master(6ee4a2ac9),新头 c1a64afe6 已推送,PR 恢复可合并状态(mergeable)。

结构化八问

  1. 是什么问题:本 PR 分支与目标分支冲突——master 在 PR 基点之后前进了约 40 个提交(含 image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 移除图片预压缩、[bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071 输出预算 floor、auto mode 状态丢失,会话累计输入了593.44M tokens #2740 auto-mode forge 等),与本 PR 触碰的同一批文件产生了冲突。
  2. 什么原因导致的:两侧改了同一批热文件。实际文本冲突只有一处:审核期间我加过一笔修正 [bug] 上游在会话中途不可用时,bili 把一次故障放大成四重代价:固定节奏 20 连击全部冷发、输出预算被静默钳制 83 次(最狠 65536→1360)、紧急收缩与 nudge 停摆均不可见 #3148 测试期望值的 commit(fd0e5ff1f,对齐 fix: floor the #2490 planning-clamp output at a 4096 capability minimum (#3071) #3101 的 4096 capability floor),而 master 随后自己落了同一个修复(ba456cfff,断言改动完全相同,仅注释措辞不同)——两边各修了一遍,撞上了。
  3. 影响范围与严重程度:无用户面影响。这是分支卫生操作;PR 内容本身未变,唯一差异是丢掉了那个已被 master 吸收的重复 commit——行为零差异(master 上该测试的断言值与本 PR 原 commit 完全一致)。
  4. 怎么修复的:rebase 到 master;唯一冲突点采用 master 版本并丢弃重复 commit(git rebase --skip);其余 5 个重叠文件(src/server.ts、src/config.ts、两份 CONFIGURATION 文档、config-reference yaml)为干净自动合并,逐一做了语义核查而非只看文本。
  5. 是否有回归问题:逐项核查过自动合并结果——
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及(BILI_PI_INLINE 的披露维持原 PR 描述不变,本次 rebase 未动配置面)。

透明记录

判定:冲突已清除,待远程 CI 转绿后可合并(人工合并)。

@ranxianglei
ranxianglei merged commit bf62c97 into master Oct 16, 2026
34 of 43 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.472-master.bf62c97 (rolling master tag).

npm install -g billion-context@0.0.472-master.bf62c97  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

移植 billion-context-pi 版本的非 server 模式

1 participant