Skip to content

[PR] fix: collect expired witness rings during record-only traffic - #6084

Closed
Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-10_expired-tool-witnesses
Closed

Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-10_expired-tool-witnesses

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

Native-id tool routing need not call lookupToolWitness, but response streams still record hash witnesses. The record path only sweeps the current session, so expired rings from finished sessions accumulate indefinitely.

Changes

Amortize a global TTL sweep across record calls, at most once per TTL window (or after a backward clock jump). Reuse the existing sweep/index cleanup and reset the sweep timestamp in the existing test seam. Fresh witnesses and routing semantics remain intact; no new timer or configuration.

Validation

  • Before: tests/tool-ring.test.ts — 24 passed, 2 failed.
  • After: 26 passed, 0 failed, including duplicate-hash, capacity, TTL-boundary and routing cases.
  • npm run typecheck and npm run build passed.
  • Fork tree fa3fe489bfe7840c1ac8648a3469d00ee181d00f exactly matches tested local commit ac2ed7de6b512ec043e23f905c9d669a17f82e6c.

Full remote-suite verdict belongs to CI. Screened all 3,221 historical PRs; #1692 and #4978 cover earlier ring/routing behavior, while this fixes abandoned-session residency on record-only paths. Independent of #6046–#6055.

Closes #6083

@ranxianglei ranxianglei changed the title fix: collect expired witness rings during record-only traffic [PR] fix: collect expired witness rings during record-only traffic Nov 10, 2026

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

APPROVE — correct fix at the right layer; verified locally in both directions.

Direction: #6083 is real. recordToolWitness only sweeps the writing session (src/tool-ring.ts:129), while the full sweep lives in lookupToolWitness — but handlePluginTool (src/plugin.ts:2079) routes by host-stamped tool_use id first, and sessions whose tool calls are executed natively by the host never POST to /__bili/plugin/tool at all, so their expired rings and hashIndex memberships accumulate forever. The PR fixes it where the leak is produced: reclaim on the record path, amortized to at most one full sweep per TTL window (plus a forced sweep after a backward clock jump). Exactly what the issue asked for ("amortized, not repeated on every tool call"); no new timer, no new config.

Regression audit — pre-existing behaviors on the touched paths, each checked:

  • Lookup semantics (lookupToolWitness): unchanged. The record-side sweep drops exactly the entries the next lookup sweep would drop (same strict > WITNESS_TTL_MS predicate) → identical routing outcomes, only earlier memory release.
  • Duplicate-hash index safety (#1685 review tests): preserved — the global sweep reuses sweep()/dropFromIndexIfAbsent, so a surviving duplicate copy keeps its index entry (existing test "sweeping the older copy of a duplicated hash keeps the index" still passes).
  • Native-id ring (toolUseIds, #4580): untouched.
  • Map-iteration safety: sweep(id) deletes only its own key, JS Map iterators tolerate current-key deletion, and the path is synchronous single-threaded — iterating rings.keys() without copying is safe.
  • Worst-case residency is now bounded (~2×TTL per session) instead of unbounded.
  • Test seam: resetToolRingForTest() resets lastRecordSweepAt — required for test isolation, done.

Tests: both new tests are genuine regressions. I ran them against base code locally: 24 passed / 2 failed (exactly the two new tests — matches the PR body's "Before" claim); on head: 26/0. Deterministic fake clock, existing seams, and the size assertions run before any lookup, so they cannot be masked by the lookup-side sweep. Existing suite in the file all still passes.

Diff/house conventions: 34 added lines across 2 files, every line relevant to the purpose; fix: commit style, branch naming, Closes #6083 all clean; no version/config/kernel/wire changes. CI green on head 30bbd3d (mirror cache).

已尝试顺手修:hashIndex 的 doc comment 写的是 "populated on lookup sweep",但实际是在 record 时加入、sweep 时清理(且本 PR 之后 record 也会触发 sweep),注释与代码事实不符。一行修正已在本地提交为 499aa9ad,但向 fork 分支推送被拒(permission denied),需作者或有 fork 权限的人应用这一行(src/tool-ring.ts:35 整行替换为):
/** witness hash → set of sessionIds holding it (added on record, pruned on sweep). */


八问(中文,供人工 reviewer):

  1. 是什么问题:只走 native-id 路由、从不触发 hash lookup 的会话,其过期的 tool witness ring 和反向索引成员永不被回收,内存无界增长。
  2. 什么原因导致的:回收逻辑只挂在两条路径上——record 时只扫当前会话、lookup 时全量扫;而 record 路径每轮流量都走、lookup 路径在 record-only 场景下根本不发生,死会话的 ring 因此永久滞留。
  3. 影响范围与严重程度:影响长期运行的代理 + 存在原生工具执行车道(不回 POST)的部署;严重度为跨天/周的无界内存增长(「只增不减的失控增长」类),非即时不可用。
  4. 怎么修复的:在 record 路径加摊还的全局 TTL 扫——每个 TTL 窗口至多一次(时钟回拨强制一次),复用既有 sweep/索引清理,不动路由语义,不加定时器/配置。
  5. 是否有回归问题:审过 lookup 语义、duplicate-hash 索引安全、native-id ring、测试隔离四块既有行为,均保持不变(见上方 Regression audit);无阈值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及上游 prefix cache 或任何缓存失效策略;只是提前释放本会到点释放的内存条目。
  7. 是否改了设计方向:不涉及;单模块内补齐缺失的回收时机。
  8. 是否引入了新的配置:无新 env/config/默认值。

判定:可合(APPROVE)。唯一未落地项为上述一行注释修正(fork 推送权限所限,补丁已给出)。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Review 完成:APPROVE(https://github.com/ranxianglei/billion-context/pull/6084#pullrequestreview-5837191428)。根因定位正确、修法为 record 路径摊还全局 TTL 扫,本地双向验证(base 24/2 → head 26/0),CI 绿。一行注释修正因 fork 推送权限未落地,精确补丁已附在 review 里。

@Afloat16

Afloat16 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Applied the requested hashIndex comment correction in e1706ab: entries are added on record and pruned on sweep. This is the only line changed in the follow-up commit.

CI on this head now passes, but GitHub still expects the Windows Node 24 checks. The current workflow selects the core matrix for this PR; those checks are only generated for full scope. Could a maintainer add ci:full to run the required full matrix? The workflow on this head is identical to current master, so updating the base alone would not create those jobs.

Could you also apply ci:full to #6086 (4bead210) and #6089 (6f2a546), which are approved and whose current-head CI runs have succeeded with the core matrix, so the missing Windows Node 24 contexts can be generated for all three PRs? GitHub also reports these branches as behind master; updating the base alone does not generate the full matrix.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Verified e1706ab8fb locally: it sits directly on the original head 30bbd3dff, touches only src/tool-ring.ts, and is byte-for-byte the requested one-line comment replacement ("added on record, pruned on sweep") — nothing else changed. The APPROVE stands; CI's re-run on the new head is the last gate before merge.

@ranxianglei

Copy link
Copy Markdown
Owner

Landed via batch PR #6439 (merged) — closing as superseded.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Noted — confirmed #6439 is merged (head 65f036c). Review is complete on this thread: APPROVE posted, and the follow-up comment fix was verified byte-for-byte at e1706ab. Closing as superseded looks right; nothing further from my side.

@ranxianglei ranxianglei reopened this Nov 14, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: not a duplicate — this thread IS the fix PR for #6083; no other open PR targets the same mechanism (record-path TTL sweep for witness rings).

Labels: bug, scope/kernel, p2 (session-core tool-witness routing; slow memory residency, no usability break).

Verdict: already landed — no action needed

Live-verified today (2026-11-14):

八问

  1. 是什么问题:record-only 流量(原生 id 路由、不调 lookupToolWitness)下,已结束会话的过期 witness ring 只增不清,无限累积。
  2. 什么原因导致的:record 路径只清扫当前会话,缺少跨会话的全局 TTL 清扫;record-only 流量永远不会触发 lookup 侧的清扫。
  3. 影响范围与严重程度:仅 record-only 路径;每会话至多 32 条 hash 的小结构,慢速内存增长,不影响可用性 → p2。
  4. 怎么修复的:在 record 调用上摊还一次全局 TTL 清扫(每个 TTL 窗口至多一次,时钟回跳后重置),复用既有清扫/索引清理逻辑,无新定时器、无新配置。
  5. 是否有回归问题:无——只新增清扫时机,不触碰新 witness 记录与路由语义;TTL 边界、重复 hash、容量、路由用例全绿(26/26),CI 在合入批次上绿。
  6. 是否破坏缓存:不涉及(纯内存路由索引,不触碰 prefix cache/存储)。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

收工判定:无需任何操作 —— 修复已随 #6439 合入 master,本线程可关。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expired tool witness rings accumulate indefinitely on record-only routing paths

2 participants