Skip to content

feat: configurable request-body byte budgets under network.* (#3128) - #3202

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-14_limits-byte-budgets
Oct 16, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-14_limits-byte-budgets

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 14, 2026 •

Copy link
Copy Markdown
Owner

Part 2 of #3128 (part 1 — readable 413 instead of ECONNRESET — is #3138). Companion to #3138; both are needed to fully resolve #3128.

What

The three HTTP request-body caps were hardcoded constants (MAX_REQUEST_BYTES 100 MiB wire, MAX_DECODED_REQUEST_BYTES 200 MiB decoded, outbound 100 MiB). A single request over any of them was unusable. This makes each stage's budget tunable and raises the defaults per owner decision so multi-gigabyte single requests are accepted instead of reset.

Behavior change (old → new)

Stage Old (hardcoded) New default Configurable via
receive (inbound wire) 100 MiB 5 GiB network.receiveBodyBytes / BILI_RECEIVE_BODY_BYTES
decode (after content-encoding expansion) 200 MiB 10 GiB network.decodedBodyBytes / BILI_DECODED_BODY_BYTES
forward (rebuilt outbound) 100 MiB 5 GiB network.outboundBodyBytes / BILI_OUTBOUND_BODY_BYTES
  • Large-decode admission threshold now tracks the configured receive budget (was the old 100 MiB constant), so a multi-gigabyte decode no longer trips the 2-slot throttle at the old boundary.
  • WebSocket lanes are unchanged: they keep the fixed 100 MiB surface (MAX_REQUEST_BYTES) — this is an HTTP request-body surface, not a WS frame/message cap.
  • No new persistence format; no wire-shape change; the 413 response bodies are unchanged in shape (this PR only makes their limits configurable — the readable-413 payload enrichment is [PR] fix: over-limit request bodies now return a readable 413 instead of ECONNRESET (#3128) #3138's).

Config surface (per Configuration Surface Discipline)

  • Added: three number leaves under the existing network block: receiveBodyBytes, decodedBodyBytes, outboundBodyBytes. Env tier BILI_{RECEIVE,DECODED,OUTBOUND}_BODY_BYTES; file tier network.*; resolved env > file > default via the standard knobs.ts pattern. No new top-level config key.
  • Why network and not a new block: the original draft used a top-level limits block, but adding a new FileConfig top-level key trips the config-surface guard's red line (check-coverage enforces the top-level-key set). Per owner direction the three budgets moved under the existing network transport-knob namespace. network's documented scope is upstream-call timing, so these size caps are a slight scope extension — accepted because there is no dedicated request/proxy block and these are per-request transport byte budgets.
  • Why the existing surface cannot express it: there was NO existing knob for these caps — they were module-level constants in src/request-body-budget.ts with zero env/config/CLI reachability. The providers table, the three-level compress hierarchy, and the other network.* timing knobs govern other concerns; none expresses a per-stage byte budget.
  • Mechanisms considered: a new top-level limits block (rejected — tripped the config-surface red line); extending the compress hierarchy (rejected — compression token budgets ≠ wire byte caps); reusing plugin.snapshotCapBytes (rejected — different surface, fork-API snapshots). Nesting under network is the minimal home that clears the gate.
  • Migration: none required. Absent config resolves to the new defaults (5/10/5 GiB). Operators who relied on the old hard failure at 100 MiB can pin the legacy values explicitly (network.receiveBodyBytes: 104857600, etc.). Invalid/non-positive values fall back to defaults silently (consistent with sibling knobs).
  • Docs regenerated from the website/config-reference/process.yaml seeds (CONFIGURATION.md en+zh). All CI gates pass locally: docs-sync (idempotent regen), check-coverage, config-surface-guard, check-unused-exports.

Tests

  • New knobs-config-precedence case: defaults 5/10/5 GiB, file tier (network.*), env precedence, garbage/negative fallback.
  • Existing byte-budget suites (request-body-budget, image-request-budget, proxy-readbody, decode-fail-passthrough) pin the legacy 100/200/100 MiB values so their mechanism assertions stay independent of the shipped default — behavior on those suites is byte-identical to master.
  • Targeted-local green (typecheck + those suites); full suite = remote CI.

Rebased onto current master.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-14_limits-byte-budgets (76f5db9)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3903-pr.3202.5d7558d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3202.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重结论:不重复。 已检索仓库近 30 个 PR(open+closed)与 "byte budget" 全文搜索:没有其他 PR/issue 实现可配置字节预算。相关项仅三个:#3128(本 PR 的源 issue)、#3138(part 1:可读 413,姊妹 PR,仍 open)、#3197(CI 端口冲突修复,与本 PR 无功能交集)。无重复立项,进入审核。

审核对象:分支 2026-10-14_limits-byte-budgets @ 4c399ba5f(基于最新 master b2868ca68,落后 0),2 commits(feat + docs),16 文件 +165/-16。diff 全量通读(src 5 文件 + tests 5 + docs 5 + AGENTS.md 1)。

结构化八问

  1. 是什么问题:bili 的 HTTP 请求体上限是三个写死的常量(接收 100 MiB / 解码后 200 MiB / 转发 100 MiB)。单请求超过任何一条就直接不可用(连接被重置、用户端看到 ECONNRESET),且用户没有任何途径调大。典型触发场景:Codex/VS Code 带大图片的请求、pi 长会话。本 PR 是 [Bug/Enhancement] Codex 超过 100 MiB 的请求在 ACP 处理前断开:希望返回明确 413 并支持配置字节预算(#2606 后续) #3128 的 part 2:把三段上限做成可配置,并按 owner 决定抬高默认值。
  2. 什么原因导致的:不是代码 bug——三段上限是 src/request-body-budget.ts 里的模块级常量,零 env/config/CLI 可达性;数值定在单请求还到不了 GB 级的年代。本质是缺配置面 + 默认值过时。
  3. 影响范围与严重程度:修复前,凡单请求 >100 MiB 走 bili 的用户完全不可用(可用性级别,非体验瑕疵);修复后默认接受到 5 GiB。纯放宽 + 加配置面,不破坏任何既有小请求路径。
  4. 怎么修复的:新增顶层 limits 配置块(三个 number 叶子),走仓库标准 knobs.ts 解析链(env > file > default),非法/非正值静默回退默认(与兄弟旋钮一致)。三处执法点从常量改为旋钮取值:readBody 逐块检查(receive)、decodeRequestBody 增量检查(decode)、fetch 前出站检查(forward)。大解码 2 槽准入的阈值跟随 receive 预算——只调高 decoded 上限不会解除槽位保护。WebSocket lane 保持固定 100 MiB 表面(已核实 MAX_REQUEST_BYTES 在 responses-ws.ts 8 处 + ws-bridge.ts maxPayload 仍在用)。
  5. 是否有回归问题:逐项审过既有行为——
  6. 是否破坏缓存:否。未触碰缓存/重试/增量游标/去重键;配置热加载走既有 [perf] knobs re-read config file per call (~5-10/request) and forward() re-parses full wire body for outbound estimate #2078 缓存路径。
  7. 是否改了设计方向:没有转向。是既有配置层级上加一个顶层块,按 Configuration Surface Discipline 执行(PR 正文含必需映射:现有面为何表达不了、考虑过哪些替代并为何拒绝、迁移方式)。一处对 floor 2 原方案的简化值得点名:准入阈值不做独立字段、跟随 receiveBodyBytes——目标(调高上限不解除槽位保护)达成,表面更简。可接受。
  8. 是否引入新配置:是。limits.receiveBodyBytes(默认 5368709120 = 5 GiB)、limits.decodedBodyBytes(默认 10737418240 = 10 GiB)、limits.outboundBodyBytes(默认 5368709120 = 5 GiB);env 层 BILI_LIMITS_{RECEIVE,DECODED,OUTBOUND}_BYTES。无需迁移:缺省 = 新默认;想钉回旧行为的运营者显式配 104857600/209715200/104857600 即可。文档 en+zh + website 已再生成,CI 四个文档门禁全绿。

CI 红腿判定:既有已知基础设施缺陷,非本 PR 回归

12 个 test-shard job 中 6 个红,全部落在「端口族 live 套件」,失败形态与 #3197 引用的空 PR 实证(#3190)一一对应:

  • ubuntu s2:port-zone-live listen EADDRINUSE 127.0.0.1:18787(zone 基址被占)
  • ubuntu s3:claude-native port 45883 is held by a lifecycle-less bili proxy (pid 4096)(freePort 选位被另一测试文件的 daemon 抢走)
  • win s3:auto-restart replacement did not become ready within 0.8s(并发负载下替换进程启动超时)

这三种形态正是 #3197(在途,Closes #3182/#3184/#3188)定位的根因:#3145 分片把端口族 live 套件集中进同一 shard,#2816 的 concurrency=4 让多个 daemon 共享 zone 基址 18787 与临时端口池——确定性 shard 内争用,不是本 PR 引入。佐证:本 PR diff 不触碰这些路径;它自己的测试面(request-body-budget / image-request-budget / proxy-readbody / decode-fail-passthrough / knobs-config-precedence)在 CI 全绿;全部静态门禁(typecheck/build/docs-sync/config-surface-guard/unused-exports/lane-guard/kernel-guard/pi-subagents-guard 等)全绿。

已做动作:对 run 38600712419 触发了失败 job 重跑(API 返回 201 接受)。鉴于根因是 shard 内核争用,重跑大概率复现同样的红腿——真正的修法是 #3197 先落地。

判定

代码审查通过,无阻塞项。两条调度备注:

  1. CI:建议先合 ci: per-file zone-port bands for live suites + bounded TOCTOU retries (#3182/#3184/#3188) #3197(或对本 PR 豁免这 6 条已知红腿后合入);红腿属既有基础设施缺陷,与本 PR 无关。
  2. 与 [PR] fix: over-limit request bodies now return a readable 413 instead of ECONNRESET (#3128) #3138 的合并顺序:两个 PR 改的是 src/server.ts readBody 的相邻行——先合的那个会让另一个产生文本冲突需 rebase。part 1([PR] fix: over-limit request bodies now return a readable 413 instead of ECONNRESET (#3128) #3138)目前排队在前,若它先合,本 PR 需要一次快速 rebase 即可。

按 §7.4,本 PR 涉及配置面 + 默认值变更,属 human-only 合并类——确认无误请手动合并:#3202

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

重跑结果跟进(run 38600712419 attempt 2,已完成):

  • 同样的 6 条腿红、其余全绿——与 attempt 1 完全同构(ubuntu s2×2、ubuntu s3×2、windows s3×2)。
  • 抽了 ub22-s2 的完整日志核实失败签名:同一个子测试、同一行错误——not ok 1553 - zone live: marked upgrade-replacement fails fast on an occupied port instead of hopping (#3044) → listen EADDRINUSE: address already in use 127.0.0.1:18787。且该文件里相邻子测试(ok 1549–1552、1554)都过了,日志里能看到别的文件 spawn 的 daemon 正占着/跳过 18787——正是 ci: per-file zone-port bands for live suites + bounded TOCTOU retries (#3182/#3184/#3188) #3197 定位的「同 shard 内多个 live 套件共享固定 zone 基址」的确定性争用形态。

结论不变:红腿是既有 CI 基础设施缺陷(#3182/#3184/#3188,修复在途 #3197),重跑无法自愈。建议先合 #3197,再对本 PR 触发一次 CI(届时若绿即可直接走人工合并);或按已知红腿豁免处理。

代码审查判定维持:通过,无阻塞项;本 PR 属 human-only 合并类(配置面 + 默认值变更)。

@ranxianglei
ranxianglei force-pushed the 2026-10-14_limits-byte-budgets branch from 509649f to 79f871e Compare October 15, 2026 16:47
@ranxianglei ranxianglei changed the title feat: configurable per-request byte budgets with a 5 GiB default (#3128) feat: configurable request-body byte budgets under network.* (#3128) Oct 15, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

ci还是失败咋回事

awork added 3 commits October 16, 2026 08:58
The HTTP request-body caps were hardcoded constants (100 MiB wire / 200 MiB
decoded / 100 MiB outbound) with no surface to raise them. Add three knobs,
resolved env > file > default through src/knobs.ts:

  network.receiveBodyBytes   (BILI_RECEIVE_BODY_BYTES)   default 5 GiB
  network.decodedBodyBytes   (BILI_DECODED_BODY_BYTES)   default 10 GiB
  network.outboundBodyBytes  (BILI_OUTBOUND_BODY_BYTES)  default 5 GiB

The receive cap also drives the large-decode admission threshold. Defaults
raised per owner decision so multi-gigabyte single requests are accepted
instead of the socket being reset; WebSocket lanes keep the fixed 100 MiB
surface. Garbage / non-positive values fall back to the default. Nested under
the existing network block (no new top-level config key).
Owner rule (2026-10-14): when a defect makes a capability unavailable, fixing
availability outranks every other consideration (performance, memory,
elegance); ship the availability fix and track the performance cost separately.
Added as a leading bullet in AGENTS.md §7.3 Correctness Guardrails.
Regenerated CONFIGURATION.md / CONFIGURATION.zh-CN.md from the merged
config-reference seeds after rebasing onto current master. Adds the three
network.{receive,decoded,outbound}BodyBytes rows; website page assets are
unchanged (plain leaf entries carry no site_* block).
@ranxianglei
ranxianglei force-pushed the 2026-10-14_limits-byte-budgets branch from 79f871e to 76f5db9 Compare October 16, 2026 01:00
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

你看到的红是旧结果——那次运行基于的 master 上有两个 master 侧问题,今天都已修好,且都与本 PR 代码无关。 我已把分支 rebase 到最新 master 并推送,新一轮 CI 正在跑。

那次红(10-15 16:47 UTC 的运行)到底挂在哪

失败 根因 状态
4 条 storage-render golden 快照测试(每个平台 shard 1 都挂) #3295(#3266「移除 image_full」的合并 PR)收尾时删掉了持久化 state.stats 里 3 个死字段(imagesShrunk/imageBytesSaved/imageTokensSaved),但提交的金标快照忘了重新生成 → master 和所有在途 PR 的同名测试全红。已在 #3313 立项,修复 PR #3314 今早 08:52 已合入——晚于你的那次运行 ✅ 已修(#3314)
1 条 #1086 T5 liveness(ubuntu-22 shard 2,25 !== 60) T5 填充请求撞上输出侧 runaway guard,测试夹具问题,#3317 已合入 ✅ 已修(#3317)
(上上轮的端口占用 6 腿) 分片调度把端口族测试挤在一起,#3197 已修 ✅ 已修(你那轮 s3 三腿已绿)

为什么确定不是本 PR 的代码

我刚做的

  1. 分支 2026-10-14_limits-byte-budgets rebase 到当前 master(bf62c973f,含 test: rebaseline storage-render goldens after #3295 dropped dead image-stats fields from state.stats #3314/修 CI 无故变红:#1086 T5 用例的 'x'×360 填充触发 #3289 输出护栏,含 #3289 的每棵树必挂 #3317 两个修复;中途还吸收了一个无关的 pi-inline 合入)——零冲突;
  2. rebase 后本地验证:typecheck + build 干净,本 PR 自己的定向套件(knobs-config-precedence + 4 个 byte-budget 套件)30/30 通过;
  3. 已推送(新 head 76f5db973),CI 已于 01:00 UTC 启动。

八问(简版)

  1. 是什么问题:PR 的 CI 红,但红的是 master 侧过期测试夹具,不是本 PR 引入的缺陷。
  2. 什么原因:金标快照落后于 feat!: remove image pre-compression and the image_full tool (#3266) #3295 的有意 state 形状变更,漏了重生成;T5 那条是夹具撞 guard。两者都在 master 上发生。
  3. 影响范围:当时阻塞所有在途 PR 的合并判定(每 PR ≥5 个分片 job);用户侧零影响。现已随 test: rebaseline storage-render goldens after #3295 dropped dead image-stats fields from state.stats #3314/修 CI 无故变红:#1086 T5 用例的 'x'×360 填充触发 #3289 输出护栏,含 #3289 的每棵树必挂 #3317 解除。
  4. 怎么修:master 侧由 test: rebaseline storage-render goldens after #3295 dropped dead image-stats fields from state.stats #3314(重生成夹具)/修 CI 无故变红:#1086 T5 用例的 'x'×360 填充触发 #3289 输出护栏,含 #3289 的每棵树必挂 #3317 修掉;本 PR 只需 rebase 到新 master 重新裁决。
  5. 回归风险:rebase 零冲突,typecheck/build/定向 30 项全绿;本 PR 自身行为未变(仍是 network.* 下三个字节预算,默认 5/10/5 GiB)。
  6. 缓存:不涉及。
  7. 设计方向:不变。
  8. 新配置:无新增(rebase 只是换了基底)。

判定:等这轮 CI 出结果即可,预期转绿;合并仍按之前结论走人工(配置面 + 默认值变更类)。若这轮再红,我立刻深挖。

@ranxianglei
ranxianglei merged commit 21f90d9 into master Oct 16, 2026
43 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.476-master.21f90d9 (rolling master tag).

npm install -g billion-context@0.0.476-master.21f90d9  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

HaqimIskandar pushed a commit to HaqimIskandar/billion-context that referenced this pull request Oct 27, 2026
…anxianglei#4804)

The Responses WebSocket lanes (codex-responses-ws, responses-ws) measured
the RECONSTRUCTED full history against a hardcoded 100 MiB cap before
dispatch ran, so a long session whose processed outbound was far below the
cap died with 413 request_too_large on the next turn even when the client
sent only a few MiB of delta; checkpoint accounting shared the same cap and
cleared checkpoints near it, stranding continuations with
previous_response_not_found. ranxianglei#3202 made the HTTP budgets configurable but
explicitly left the WS surface fixed at 100 MiB.

- ws maxPayload, reconstructed-body admission, upstream frame/backpressure/
  output guards and checkpoint accounting all resolve limitsReceiveBytes()
  (BILI_RECEIVE_BODY_BYTES > network.receiveBodyBytes > 5 GiB) - zero new
  config fields
- one full serialization per turn: admission measures the same wire string
  the pipeline envelope pushes (was two)

Behavior changes (old -> new, defaults):
- WS wire frame cap 100 MiB -> 5 GiB
- WS reconstructed-body admission cap 100 MiB -> 5 GiB
- checkpoint retention cap 100 MiB -> 5 GiB (near-cap checkpoints survive)
- 413 message names the live budget instead of a constant "100 MiB"
Restore the old value with BILI_RECEIVE_BODY_BYTES=104857600 (this also sets
the HTTP receive budget, as it always did).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug/Enhancement] Codex 超过 100 MiB 的请求在 ACP 处理前断开:希望返回明确 413 并支持配置字节预算(#2606 后续)

1 participant