Skip to content

fix: resolve knobs from a content-keyed config cache and parse the outbound body once (#2078) - #2087

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-04_issue2078_knob-cache-parse-once
Oct 4, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-04_issue2078_knob-cache-parse-once

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2078

What

Two per-request hot-path costs introduced by #2030 (knobs) and the #1987/#2058 seam-forensics stamping:

  1. Every knob call re-read + re-parsed the config file. ~5–10 synchronous readFileSync + JSON.parse per request (server.ts alone has ~45 knob call sites).
  2. The seam block parsed the largest payload in flight three times — once for the outbound estimate's projection, once inside estimateWireOverhead, once inside the image-reserve raw scan.

How

Config cache (src/config.ts). loadConfigFile() now keeps one entry keyed by the file's RAW TEXT: the read happens per call (page-cache hit, ~µs), JSON.parse only when the content actually changed. Hot-reload stays exact — any external rewrite is visible on the next call; a missing file is never cached (a reappearing file shows immediately); corrupt-but-present content is cached as {} so one bad save does not re-log and re-fail-parse on every knob call until fixed. Every consumer of the returned object was audited read-only.

Parse-once estimate (src/server.ts, src/server/budget.ts). The seam block performs ONE JSON.parse(sentBody) and hands the result to outboundContextEstimate through a new optional third-state parameter: undefined = parse here (legacy behavior for any other caller), object = project from it, null = caller already tried and failed → keep the prepared projection and give the helpers "", which deterministically yields 0 exactly like today's failed-parse results. estimateWireOverhead accepts a pre-parsed object and skips its own parse; all existing string/Buffer callers are byte-identical.

Why not the issue's suggested mtime/size key

Measured: back-to-back rewrites can land on the SAME sub-ms mtimeMs on common filesystems (the new tests pin this), so a stat-keyed cache deterministically violates this repo's own hot-reload contract test — with a stat key, tests/knobs-config-precedence.test.ts "hot reload" went red locally (expected 222, got stale 111). Content keying keeps hot-reload exact while still removing the expensive half. Warm page-cache bench at a 29 KB config: read+parse 29.9 µs/call → read+compare 16.6 µs/call (stat-only would be 2.8 µs but is unsound); the saving fraction grows with config size.

Behavior changes (disclosed)

  • Malformed config file: the [acp-config] failed to parse … error log now fires once per distinct corrupt content instead of once per loadConfigFile() call (previously up to ~45× per request of duplicate log spam; ENOENT silence unchanged). Improvement, not regression.
  • The parsed config object is shared across calls until the content changes; all current consumers audited read-only.
  • Degenerate top-level array/scalar JSON request bodies (invalid under all four protocols — upstream rejects them): the overhead term reads 0 instead of ~1–2 tokens (JSON.stringify([])). Stats bookkeeping only, no gating impact. Every VALID body produces bit-identical numbers (same string → same parse result → same projection/overhead/image terms).

Not changed: the ENV tier stays live per call; no new config keys / env vars / CLI flags (config surface untouched); no persistence format, wire shape, or protocol changes. Both compression modes reasoned: plugin mode suppresses tool/nudge injection and shares this same estimate path with unchanged semantics; proxy mode semantics identical. outboundContextEstimate has exactly one production caller (the seam block) — nothing else migrated.

Tests

  • NEW tests/config-file-cache.test.ts (7): cache-hit reference equality; hot-reload visibility incl. same-size rewrite with restored mtime (content is truth); identical-content rewrite served from cache without re-parse; missing file → {}; corrupt file logs once per distinct corruption (via setLogCapture); BILI_CONFIG_FILE switch re-resolves.
  • NEW tests/issue2078-outbound-estimate.test.ts (3): pre-parsed ≡ legacy self-parse across all four protocols (with codec-live >0 assertions so the equivalence cannot hold vacuously); garbage body: null mode ≡ legacy fallback ≡ 0; size-diff liveness guard in both modes.
  • EXTENDED tests/wire-overhead.test.ts: estimateWireOverhead(proto, obj) ≡ estimateWireOverhead(proto, JSON.stringify(obj)) per protocol.
  • The owner-authored knob contract suite (tests/knobs-config-precedence.test.ts) passes UNCHANGED — it is what caught the stat-key design locally.

Verified locally: targeted suites green (28/28), npm run typecheck clean, npm run build success. Full regression delegated to CI.

@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-04_issue2078_knob-cache-parse-once (517ad4c)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2121-pr.2087.dd8bced

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2087.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,这是 PR #2078 的跟踪 issue(带 ework-agent-pr 标记)。按流程处理:拉取分支核对代码、审查方向与回归面、检查 CI,结论稍后按结构化七问回复。不重做实现。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 完成。PR #2087(单 commit 517ad4cc,分支 2026-10-04_issue2078_knob-cache-parse-once,base master)已拉取并逐项核验,结论:可合并。

结构化七问

1. 是什么问题
#2078 性能缺陷:knobs 每次调用都重新读盘+解析配置文件(每请求约 5–10 次);forward() 的 seam 块对同一份出站 body 重复 JSON.parse 三次(estimate 投影、wire-overhead 项、image-reserve),大 body 下是纯浪费的 CPU。

2. 什么原因导致的
loadConfigFile() 无缓存,每次调用走完整 readFileSync+JSON.parse;outboundContextEstimate 内部各助手各自独立解析同一个字符串。根因在「配置读取层」与「估计管线」两处各自为政,不是某个调用方写错。

3. 怎么修复的

  • 配置层:内容键控模块缓存——文本仍每次重读(热重载语义精确保持),仅当全文变化才重新 parse;ENOENT 静默、BOM 剥离、失败日志文案与旧 safeReadJson 逐字一致;损坏内容以 {} 缓存,日志按「不同损坏状态」各记一次。
  • 估计管线:seam 块单次 parse,结果经三态可选参数(undefined=旧行为自解析 / object=复用 / null=空串等价零值)传入;estimateWireOverhead 对 object 输入短路内部解析,string/Buffer 路径代码逐字节未动。选内容键而非 mtime/size 键是对的——新测试专门钉死了「同大小+mtime 复原」这一 stat 键失效模式。

4. 是否有回归问题
逐项审了 diff 触碰路径的既有行为:

  • loadConfigFile 语义(缺失→{}、BOM、解析失败日志、热重载可见性):保持——与旧实现逐字对照 + 7 个新测试钉住;
  • ENV 层优先级:保持(env 仍每次活读,只缓存 FILE 层);
  • web UI 改配置:绕过缓存直接重写文件 → 下次读取内容变 → 缓存 miss,热重载精确;
  • loadConfigFile 全部消费方审计为只读(无缓存对象变异路径;嵌套引用仅从 knobs 私有访问器返回);
  • estimateWireOverhead 其余调用方(outboundPayloadBreakdown、loop/core.ts)全传字符串,走未改动路径;
  • imageReserveFor 的 unknown body 参数是既有签名,PR 未触碰;object≡string 数值恒等(raw 路径本就是 probe+parse+委托同一 helper);
  • 两种压缩模式均推演:seam 块与模式无关,写入的 stats 字段不变,无门控决策依赖这些数字;
  • 高危模式清单过完:无 fail-fast 变吞错、无阈值/时序漂移、无输出格式变化。
    结论:无回归。

5. 是否破坏缓存
不涉及既有缓存/重试/游标策略——本 PR 的新缓存就是修复本身,失效键为文件全文内容,选择正确。前缀缓存、会话持久化、更新节流均不受影响。

6. 是否改了设计方向
不涉及。单模块性能修复,三态可选参数让其余调用方维持原路径,架构边界未动。

7. 是否引入了新的配置
不涉及。无新 config 字段、env var、CLI flag。PR 描述已如实披露唯一微行为差异:退化顶层 array/scalar body 的 overhead 项从 ~1–2 tokens 变为 0——纯统计记账,上游本来就会拒绝此类 body。

证据

本地定向验证:typecheck 干净;4 个相关测试文件 28/28 通过(config-file-cache 7、issue2078-outbound-estimate 3、wire-overhead 含新增、knobs-config-precedence);build 成功(含 import-annotations 检查)。CI:head commit 上 16/16 全绿。分支落后 master 4 个提交,但那些提交只碰 workflows/CONFIGURATION.md,与本 PR 零文件重叠,rebase 无冲突风险。diff 整洁:7 文件 +303/-29,每一行都服务于 #2078,无跑题改动。

判定:可合并。 无阻塞项,review 中未发现需要顺手修或立项的小问题。

@ranxianglei
ranxianglei merged commit 0a5f35b into master Oct 4, 2026
16 checks passed
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.15-master.0a5f35b (rolling master tag).

npm install -g billion-context@0.0.15-master.0a5f35b  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@ranxianglei
ranxianglei deleted the 2026-10-04_issue2078_knob-cache-parse-once branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[perf] knobs re-read config file per call (~5-10/request) and forward() re-parses full wire body for outbound estimate

1 participant