Repository navigation
[PR] fix: exempt the dsh plugin lane and compaction calls from the 32 MiB ingest band (#6908) - #6933
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11130-pr.6933.d17678bThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6933.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Reviewed PR #6933 (exempt the dsh plugin lane and compaction calls from the 32 MiB ingest band, closes #6908). I pulled the branch ( What I verified:
Minor observations (non-blocking):
CI: 0 failed, 1 pending (mysql-contract — unrelated to this PR, which doesn't touch storage). I'll confirm the final verdict when it's green. 8问 (中文):
判定: 代码层面可合(定向测试全绿、typecheck 干净、无回归、行为变更已披露)。CI 0 failed / 1 pending(mysql-contract,与本 PR 无关),全量判定待绿后确认。 |
…ingest band (#6908) The ingest band (network.ingestBodyBytes, default 32 MiB) measured the PRE-folding inbound body, but its premise ("any parseable 32 MiB+ text body is >= ~8M tokens, beyond every window") equates INBOUND BYTES with MODEL INPUT. That holds for the outbound/incremental lanes, not for the dsh plugin lane, which re-sends the host's ENTIRE raw ledger every round on the OpenAI wire (no incremental lane there, #3433): the body is the host's history, not the model input, and bili's job is to fold it (field: 32.0 MiB inbound vs 446.4 KiB outbound, 70x). A healthy session that crossed 32 MiB was 413'd at ingest forever, and the dsh compaction call (the ONLY action that shrinks the host's ledger via bili) is itself an over-band body, so the self-heal self-locked. Two dimensionally-motivated exemptions: (A) x-bili-plugin: dsh -> the body is fold substrate, exempt. (B) the dsh compaction marker present -> the self-heal must reach isDshCompactionCall (#3733 answers it locally with the folded snapshot); rc.2 replays the full shadowed region (#2193), so it crosses the band and must pass the gate. A 48-byte sniff decides; a false positive (a turn that merely quotes the marker) just pays the parse cost, identical to today's under-cliff behavior — the full marker-decisive check still runs after parse. Also fixes two observability gaps on the same line: - inboundBytesOf reported UTF-16 code units (raw.length) for strings; the host-json-parse inbound=...B field now reports UTF-8 bytes, matching the gate's byteLength (an 8%+ unit gap hid how close the body was to the wall). - the ingest 413 log now names the ACTUAL decoded byte count next to the cap (was limit-only), so operators can see the margin. Tests: 3 new e2e cases in issue5794-ingest-cap.test.ts — (A) a dsh plugin-lane over-band full-resend body is folded and forwarded (200 + 1 upstream call), (B) a compaction-marker body passes the gate and reaches the compaction guard (422), and the negative: a plain over-band body with no dsh header/marker is still refused 413 (the band is not widened).
1e9af39 to
9707b15
Compare
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved — the PR is re-based onto current master and mergeable again. What I did this run:
八问:
收工判定:冲突已解,分支已就绪,等 CI 全量结果即可。 |
#7426) The #6908 exemption was dsh-only (x-bili-plugin === "dsh"), so the opencode lane — which has the SAME full-resend structure (the host re-sends its ENTIRE raw ledger every round; bili's job is to fold it) — was still 413-locked at the 32 MiB band (field: 33.5 MiB inbound vs ~391K tokens of actual context, a healthy 390-block session). The criterion generalizes to ANY plugin lane: the x-bili-plugin header announces "my body is my ledger, fold it," which is dimensionally true for every cooperative plugin lane. Incremental-lane turns carry small deltas and never reach the band, so the wider net is harmless there. Both directions pinned: opencode lane passes the gate (folded + forwarded), non-plugin over-band bodies are still refused 413.
|
Superseded by #7443. Decision for #7426: lift the ingest band for ALL lanes (default 0 = unlimited), not just exempt the dsh/plugin lanes. With the band off by default there is nothing left to exempt, so this scoped approach is no longer needed. BILI_INGEST_BODY_BYTES remains as the opt-in re-arm path if a specific deployment hits the pre-fold parse CPU. |
) #5794's ingest band proxied 'beyond every model window' by raw decoded bytes, which is invalid on the re-send lanes (opencode/dsh re-send the whole raw ledger every turn and bili folds it) — it 413'd a 33.5 MB opencode turn whose folded context was tiny. Default -> 0 (unlimited, all lanes); the sibling receive (5 GiB) / decode (10 GiB) transport guards still bound pathological bodies. BILI_INGEST_BODY_BYTES re-enables the band. Supersedes #6933.
What this fixes
The 32 MiB ingest band (
network.ingestBodyBytes) measured the pre-folding inbound body, but its premise ("any parseable 32 MiB+ text body is ≥ ~8M tokens, beyond every window") equates INBOUND BYTES with MODEL INPUT. That holds for the outbound/incremental lanes, not for the plugin lanes, which re-send the host's ENTIRE raw ledger every round on the full-resend wire (dsh on the OpenAI wire, no incremental lane there, #3433; opencode V2 the same): the body is the host's history, not the model input, and bili's job is to fold it (dsh field: 32.0 MiB inbound vs 446.4 KiB outbound, 70x; opencode field #7426: 33.5 MiB inbound vs ~391K tokens of actual context).A healthy plugin-lane session that crossed 32 MiB was 413'd at ingest forever — and the dsh compaction call (the ONLY action that shrinks the host's ledger via bili, #3733) is itself an over-band body (rc.2 replays the full shadowed region, #2193), so the self-heal self-locked: the one action that could shrink the ledger was the one the gate refused.
The fix
Two dimensionally-motivated exemptions at the gate (
src/server/handle.ts):x-bili-pluginpresent (ANY plugin lane) → the body is fold substrate, exempt. [Bug][opencode][ingest/0.1.216] Long sessions on the opencode lane are permanently 413'd by the 32 MiB ingest hard limit | [Bug][opencode][ingest/0.1.216] opencode 车道的长会话被 32 MiB ingest 硬闸永久 413(且 #6908/#6933 的修复只豁免 dsh 车道,opencode 仍在外) #7426 generalized this from=== "dsh"to the whole cooperative plugin surface: the same full-resend structure (host ledger re-sent, bili folds it) is what the header announces, and the opencode lane was 413-locked by the dsh-only test. Incremental-lane turns carry small deltas and never reach the band, so the wider net is harmless there.isDshCompactionCall; a false positive (a turn that merely quotes the marker) just pays the parse cost, identical to today's under-cliff behavior — the full marker-decisive check still runs after parse.Plus two observability fixes on the same line:
inboundBytesOf(src/agent/dsh-native.ts) reported UTF-16 code units (raw.length) for strings; thehost-json-parse inbound=…Bfield now reports UTF-8 bytes, matching the gate'sbyteLength(an 8%+ unit gap hid how close the body was to the wall).network.ingestBodyBytes(default 32 MiB) was refused413 stage=ingestbeforeJSON.parse— even when folding would have shrunk it well under the window. (Originally dsh-only; [Bug][opencode][ingest/0.1.216] Long sessions on the opencode lane are permanently 413'd by the 32 MiB ingest hard limit | [Bug][opencode][ingest/0.1.216] opencode 车道的长会话被 32 MiB ingest 硬闸永久 413(且 #6908/#6933 的修复只豁免 dsh 车道,opencode 仍在外) #7426 generalized to all plugin lanes.)host-json-parsecost on over-band bodies. That cost is the lane's structural cost — it is already paid every round below the cliff, so the gate only ever added a discontinuity at 32 MiB, not a saving.Tests
5 new e2e cases in
tests/issue5794-ingest-cap.test.ts:All 8 cases in the file pass; typecheck clean.
Closes #6908
Closes #7426