Repository navigation
Conversation
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
APPROVE — correct fix at the right layer; verified locally in both directions.
Direction: #6083 is real. recordToolWitness only sweeps the writing session (src/tool-ring.ts:129), while the full sweep lives in lookupToolWitness — but handlePluginTool (src/plugin.ts:2079) routes by host-stamped tool_use id first, and sessions whose tool calls are executed natively by the host never POST to /__bili/plugin/tool at all, so their expired rings and hashIndex memberships accumulate forever. The PR fixes it where the leak is produced: reclaim on the record path, amortized to at most one full sweep per TTL window (plus a forced sweep after a backward clock jump). Exactly what the issue asked for ("amortized, not repeated on every tool call"); no new timer, no new config.
Regression audit — pre-existing behaviors on the touched paths, each checked:
- Lookup semantics (
lookupToolWitness): unchanged. The record-side sweep drops exactly the entries the next lookup sweep would drop (same strict> WITNESS_TTL_MSpredicate) → identical routing outcomes, only earlier memory release. - Duplicate-hash index safety (#1685 review tests): preserved — the global sweep reuses
sweep()/dropFromIndexIfAbsent, so a surviving duplicate copy keeps its index entry (existing test "sweeping the older copy of a duplicated hash keeps the index" still passes). - Native-id ring (
toolUseIds, #4580): untouched. - Map-iteration safety:
sweep(id)deletes only its own key, JS Map iterators tolerate current-key deletion, and the path is synchronous single-threaded — iteratingrings.keys()without copying is safe. - Worst-case residency is now bounded (~2×TTL per session) instead of unbounded.
- Test seam:
resetToolRingForTest()resetslastRecordSweepAt— required for test isolation, done.
Tests: both new tests are genuine regressions. I ran them against base code locally: 24 passed / 2 failed (exactly the two new tests — matches the PR body's "Before" claim); on head: 26/0. Deterministic fake clock, existing seams, and the size assertions run before any lookup, so they cannot be masked by the lookup-side sweep. Existing suite in the file all still passes.
Diff/house conventions: 34 added lines across 2 files, every line relevant to the purpose; fix: commit style, branch naming, Closes #6083 all clean; no version/config/kernel/wire changes. CI green on head 30bbd3d (mirror cache).
已尝试顺手修:hashIndex 的 doc comment 写的是 "populated on lookup sweep",但实际是在 record 时加入、sweep 时清理(且本 PR 之后 record 也会触发 sweep),注释与代码事实不符。一行修正已在本地提交为 499aa9ad,但向 fork 分支推送被拒(permission denied),需作者或有 fork 权限的人应用这一行(src/tool-ring.ts:35 整行替换为):
/** witness hash → set of sessionIds holding it (added on record, pruned on sweep). */
八问(中文,供人工 reviewer):
- 是什么问题:只走 native-id 路由、从不触发 hash lookup 的会话,其过期的 tool witness ring 和反向索引成员永不被回收,内存无界增长。
- 什么原因导致的:回收逻辑只挂在两条路径上——record 时只扫当前会话、lookup 时全量扫;而 record 路径每轮流量都走、lookup 路径在 record-only 场景下根本不发生,死会话的 ring 因此永久滞留。
- 影响范围与严重程度:影响长期运行的代理 + 存在原生工具执行车道(不回 POST)的部署;严重度为跨天/周的无界内存增长(「只增不减的失控增长」类),非即时不可用。
- 怎么修复的:在 record 路径加摊还的全局 TTL 扫——每个 TTL 窗口至多一次(时钟回拨强制一次),复用既有 sweep/索引清理,不动路由语义,不加定时器/配置。
- 是否有回归问题:审过 lookup 语义、duplicate-hash 索引安全、native-id ring、测试隔离四块既有行为,均保持不变(见上方 Regression audit);无阈值/时序/输出格式漂移。
- 是否破坏缓存:不涉及上游 prefix cache 或任何缓存失效策略;只是提前释放本会到点释放的内存条目。
- 是否改了设计方向:不涉及;单模块内补齐缺失的回收时机。
- 是否引入了新的配置:无新 env/config/默认值。
判定:可合(APPROVE)。唯一未落地项为上述一行注释修正(fork 推送权限所限,补丁已给出)。
Review 完成:APPROVE(https://github.com/ranxianglei/billion-context/pull/6084#pullrequestreview-5837191428)。根因定位正确、修法为 record 路径摊还全局 TTL 扫,本地双向验证(base 24/2 → head 26/0),CI 绿。一行注释修正因 fork 推送权限未落地,精确补丁已附在 review 里。 |
|
Applied the requested CI on this head now passes, but GitHub still expects the Windows Node 24 checks. The current workflow selects the core matrix for this PR; those checks are only generated for full scope. Could a maintainer add Could you also apply |
(ai: vllm-qwen/qwen3.8-27b) Verified |
|
Landed via batch PR #6439 (merged) — closing as superseded. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Duplicate screening: not a duplicate — this thread IS the fix PR for #6083; no other open PR targets the same mechanism (record-path TTL sweep for witness rings). Labels: bug, scope/kernel, p2 (session-core tool-witness routing; slow memory residency, no usability break). Verdict: already landed — no action neededLive-verified today (2026-11-14):
八问
收工判定:无需任何操作 —— 修复已随 #6439 合入 master,本线程可关。 |
Problem
Native-id tool routing need not call lookupToolWitness, but response streams still record hash witnesses. The record path only sweeps the current session, so expired rings from finished sessions accumulate indefinitely.
Changes
Amortize a global TTL sweep across record calls, at most once per TTL window (or after a backward clock jump). Reuse the existing sweep/index cleanup and reset the sweep timestamp in the existing test seam. Fresh witnesses and routing semantics remain intact; no new timer or configuration.
Validation
Full remote-suite verdict belongs to CI. Screened all 3,221 historical PRs; #1692 and #4978 cover earlier ring/routing behavior, while this fixes abandoned-session residency on record-only paths. Independent of #6046–#6055.
Closes #6083