Repository navigation
Port billion-context-pi's no-server mode: inline in-process compression lane for pi - #3284
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3857-pr.3284.6bbdd06This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3284.tgz
npm install -g packageInstall instructions are refreshed on each push. |
Review of #3284 found the inline lane ignored the caller's AbortSignal. Spawn mode cancels via socket close (res "close" -> relay clientAbort), but InlineRes.destroy() marked writableEnded=true, so the pipeline read the close as normal completion and ran the upstream round trip to completion after the caller went away; the dispatch promise resolved with a phantom response instead of rejecting. - InlineRes mirrors http.ServerResponse socket-death semantics: destroy does not end the response; write/end are no-ops once dead - makeDispatcher races the pipeline against init.signal and rejects with the abort reason like undici would - pi-inline transport rethrows AbortError instead of degrading: a user cancel must not consume the one-shot engine-down warning - e2e tests move to the real model lane (max_tokens>200 + slow marker) and assert the upstream round trip never completes post-abort
f690d9a to
fd0e5ff
Compare
Review of #3284 found the inline lane ignored the caller's AbortSignal. Spawn mode cancels via socket close (res "close" -> relay clientAbort), but InlineRes.destroy() marked writableEnded=true, so the pipeline read the close as normal completion and ran the upstream round trip to completion after the caller went away; the dispatch promise resolved with a phantom response instead of rejecting. - InlineRes mirrors http.ServerResponse socket-death semantics: destroy does not end the response; write/end are no-ops once dead - makeDispatcher races the pipeline against init.signal and rejects with the abort reason like undici would - pi-inline transport rethrows AbortError instead of degrading: a user cancel must not consume the one-shot engine-down warning - e2e tests move to the real model lane (max_tokens>200 + slow marker) and assert the upstream round trip never completes post-abort
(ai: vllm-qwen/qwen3.8-27b) PR 审核完成。结论先行:方向正确、实现质量高;审核中发现并修复了 1 个真实缺陷(调用方取消不传播);CI 转绿即可合并。 结构化八问
其他审查记录
判定:CI 转绿后可合并(人工合并),无阻塞项。 |
(ai: vllm-qwen/qwen3.8-27b) 进度汇报(刚才进程被重启是因为我在跑一个过长的 CI 轮询循环,是我的调度错误,已改为一次性查询): CI 已全绿:新头 当前状态汇总:
判定维持:可合并(人工合并)。PR 地址:#3284 |
The pre-traffic half of startServer (tee logger, kernel core, instance identity, watcher seed, session reload/lineage reconcile, GC timer, registry prefetch, admin ctx) becomes an exported EngineContext factory. startServer consumes it unchanged — pure move, zero behavior change.
…off) #3278: port of billion-context-pi's no-server mode, re-implemented rather than replicated. With BILI_PI_INLINE=1 the pi extension compresses IN-PROCESS: pi-inline.ts installs a fetch-chain link below the native intercept that serves the pseudo-origin (http://billion-context.inline) from inline-engine.ts, which boots the same engine context startServer builds and dispatches each request through handle() directly — model traffic AND the /__bili//__acp tool surface reuse one pipeline, no proxy process, no ephemeral port, no loopback hop. Upstream egress inside the engine runs under withFetchTransport(rawFetch) so nested fetches bypass the patched chain (no self-loop). Degradation: if the engine cannot boot, model traffic passes direct (uncompressed) with one loud log; management calls reject until it recovers (30s retry cooldown). planNativePi precedence: kill-switch > /bili/ rewrite launch > inline > attach > spawn — inline is a deliberate choice about WHERE compression runs, so it outranks ambient attach envs. WS intercept stays unpatched in inline mode (no proxy bridge exists). Default off: unset = today's spawn/attach behavior byte-for-byte.
tests/pi-inline.test.ts (hermetic XDG homes, mock upstream): transport routing (pseudo-origin served in-process, other hosts untouched), degradation (engine failure -> model traffic direct, mgmt rejects), and an end-to-end in-process pipeline run (real engine boot, compressed session tracked, admin health reachable via pseudo-origin, exactly one upstream hit per request proving no fetch loop). tests/pi-native.test.ts: plan gating matrix + armNativePi(inline) freezes the pseudo-origin without spawning.
Seed entry in website/config-reference/env-only.yaml + regenerated CONFIGURATION.md/zh (docs-sync guard). Host-posture category, default off, values 1/true.
Review of #3284 found the inline lane ignored the caller's AbortSignal. Spawn mode cancels via socket close (res "close" -> relay clientAbort), but InlineRes.destroy() marked writableEnded=true, so the pipeline read the close as normal completion and ran the upstream round trip to completion after the caller went away; the dispatch promise resolved with a phantom response instead of rejecting. - InlineRes mirrors http.ServerResponse socket-death semantics: destroy does not end the response; write/end are no-ops once dead - makeDispatcher races the pipeline against init.signal and rejects with the abort reason like undici would - pi-inline transport rethrows AbortError instead of degrading: a user cancel must not consume the one-shot engine-down warning - e2e tests move to the real model lane (max_tokens>200 + slow marker) and assert the upstream round trip never completes post-abort
Owner ask on #3278: make the no-server mode settable from the JSON config file under the existing top-level "pi" section. - FileConfig.pi gains inline?: boolean (#3278); resolved env > file > default(off) — BILI_PI_INLINE=1/true enables, BILI_PI_INLINE=0/false explicitly disables over the file value - planNativePi takes the file value as a second argument (pure over its inputs, tests stay hermetic); module wiring passes loadConfigFile().pi?.inline === true - docs seed (server.yaml) + regenerated CONFIGURATION.md/zh; coverage and config-surface guards green
38aee3b to
c1a64af
Compare
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决:分支已 rebase 到当前 master( 结构化八问
透明记录
判定:冲突已清除,待远程 CI 转绿后可合并(人工合并)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.472-master.bf62c97 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #3278
What
Re-implements (not replicates) the legacy standalone package's no-server mode as an opt-in lane of the existing pi-native plugin: BILI_PI_INLINE=1 makes the pi extension compress entirely IN-PROCESS — no proxy process, no ephemeral port, no loopback hop.
How
refactor(server): the pre-traffic half of startServer() becomes an exportedcreateEngineContext()(tee logger, kernel core, instance identity, watcher seed, session reload/lineage reconcile, GC timer, registry prefetch, admin ctx). startServer consumes it unchanged — pure move.src/inline-engine.ts— boots the SAME EngineContext without a socket and dispatches each request throughhandle()directly. Upstream egress runs underwithFetchTransport(rawFetch)so nested fetches bypass the patched chain (no self-loop).src/agent/pi-inline.ts— a fetch-chain link installed BELOW the native intercept: serves the pseudo-originhttp://billion-context.inlinein-process; every other host passes through untouched. Model traffic AND the/__bili/+/__acp/tool surface share one pipeline.src/agent/pi-native.ts—planNativePigains mode"inline"; precedence kill-switch > /bili/ rewrite launch > inline > attach > spawn. The pseudo-origin is frozen synchronously (no health probe can ever target a host that never listens). Degradation: if the engine cannot boot, model traffic passes direct (uncompressed) with one loud log; management calls reject until recovery (30s retry cooldown). WS intercept stays unpatched in inline mode (no proxy bridge exists).pi.inline= today's spawn/attach behavior byte-for-byte. Thebili pilauncher keeps passingBILLION_CONTEXT_PROXYas before; an EXPLICIT inline opt-in (env or file) still outranks that ambient attach target — in that case compression runs in the pi process and the launcher-spawned proxy sits idle until the pi process exits.config surface
pi.inline1/true)pisection (owner ask 2026-10-15)pi.inline(file)BILI_PI_INLINE=1/trueenables over the file,BILI_PI_INLINE=0/falseexplicitly disables over it; kill switches stay above both. Read throughloadConfigFile()(same loader/validation surface aspi.subagents).Migration: none — additive knob, default off. Seed entry added to website/config-reference/env-only.yaml; CONFIGURATION.md/zh regenerated (docs-sync guard green locally).
Behavior changes (disclosure)
Verification (local, targeted)
node --import tsx --test tests/pi-inline.test.ts tests/pi-native.test.ts tests/admin-probe.test.ts→ 21/21 passnpm run typecheckclean;npm run buildsuccess (dist-import-annotations OK)tools/gen-config-docs.mjs check-coverage+scripts/config-surface-guard.mjsgreenAcceptance probe (owner ask: 本地测试任务压缩能正常跑): drove a 10-turn growing task through the in-process engine against a mock upstream — 3 kernel compress cycles executed in-process, summary carrier
[Compressed conversation section]present in 8/13 upstream-bound bodies, post-fold body shrank 51944→46380 bytes, 1 session tracked in-process:{"turnsDriven":10,"upstreamRequests":13,"compressCycles":3,"bodiesCarryingSummary":8,"maxBodyBytesBeforeFirstFold":51944,"minBodyBytesFromFirstFoldOn":46380,"shrankAfterFold":true,"inProcessSessions":1}
Full-suite verdict: remote CI on this head.
Follow-up on this branch (CI triage)
test(#3148)): after the rebase all fourtestshard lanes failed on#3148: clampOutgoingOutput records the exact clamp event— expectedcapped: 1360, actual4096. Root cause: PR fix: floor the #2490 planning-clamp output at a 4096 capability minimum (#3071) #3101 ([bug] 0.1.195 输出预算三路饿死仍复现:无 output 上限模型 restore 无下限(1→252、fork 原样转发 1)、#2490 planning floor 钳到 1479–2623 → 3 会话 9 次 finish=length 空轮不重试(#546/#1665/#1840/#2904/#2756 残留) #3071) floored the output-clamp reduction target atmin(4096, window-input), but the [bug] 上游在会话中途不可用时,bili 把一次故障放大成四重代价:固定节奏 20 连击全部冷发、输出预算被静默钳制 83 次(最狠 65536→1360)、紧急收缩与 nudge 停摆均不可见 #3148 test (merged BEFORE fix: floor the #2490 planning-clamp output at a 4096 capability minimum (#3071) #3101) still pins the pre-floor raw cap. The identical failure reproduces on plain master (6b5b283's own CI, same test) — the red pre-dates this PR and is unrelated to it. Fixed here per the 几行级小修 policy as its own labeled commit: recorded-event AND wire assertions updated to 4096 plus the formula step documented; no separate issue filed because the one-line fix ships with this PR.Follow-up commit 38aee3b (owner ask: JSON config home)
pi.inlineadded to the existing top-levelpisection of billion-context.json (next topi.subagents, #2230):{ "pi": { "inline": true } }. planNativePi now resolves env > file > default(off) and stays pure over its inputs (the module wiring passesloadConfigFile().pi?.inline === true); docs seed + regenerated CONFIGURATION.md/zh; precedence pinned by a new test matrix (file-alone enable, explicit env off/on over the file, kill switches over the file). Local: pi-native 17/17, typecheck/build/docs guards green.