Repository navigation
[PR] fix: bound init-snapshot memory/time with a top-100 min-heap scan - #6667
ranxianglei wants to merge 1 commit into
Conversation
snapshotInit allocated one InitMessageEntry per message and sorted the whole array before retaining 100, adding O(n) memory and O(n log n) synchronous work to the first proxied request of a long session even though the visible sample is capped at 100. Replace it with a bounded top-K min-heap (worst entry at the root) that keeps only the INIT_PER_MSG_CAP largest entries in memory while scanning. The output is byte-identical to the old whole-array stable sort: descending estTokens, earlier-message precedence for equal sizes, aggregate counts over every message, and thinking tokens still ranked. Closes #6655
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9902-pr.6667.28ea99bThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6667.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重 (duplicate screening): #6667 duplicates #6656. Two open PRs both Technical review of #6667 — the change is correct and clean.
One process note (not a blocker): the PR base is 17 commits behind current 8问 (结构化审核):
收工判定: #6667 本身可合(正确、干净、测试有牙、perf 属实)。唯一待办是重复 PR 的取舍: 🔶 需要决策 — #6656 与 #6667 同修 #6655,请二选一合并、关闭另一个(我不代关)。
|
(ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — #6667 的冲突不是机械冲突,是「同一修复的两个等价实现,其一已合入 master」,需要你拍板留哪个。我已打 现状(本轮 live 核实,非凭记忆):
为什么我不直接解: 两边都是正确且等价的实现,「取哪边」= 选内部算法 + 选保留哪个贡献,是设计/归属决策,不是「取一边」的机械冲突。floor 3 的 reviewer 已把「#6656/#6667 二选一」标成 🔶 需要决策、你还没拍板;且当时 #6656 未合,现在它已合入并 closed,情况变了。我不擅自替你选。 选项:
推荐 A;不决定的默认 = C(master 已修好,#6667 挂着无副作用,你随时可关或改选 B)。你回一句 A/B/C 我就执行(A=我留关单说明、由你关;B=我 rebase 取 min-heap 侧 force-push 后看 CI)。 标签: 8问(结构化审核 — 本轮 CI 车道):
收工判定: 本轮 CI 车道唯一 blocker(语义冲突)已定位并升级为 🔶 需要决策,已打 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending, no change this run Re-verified live (not from memory):
Nothing has changed since my floor-6 escalation and the owner has not chosen A/B/C, so there is no mechanical action to take this run. The full options + recommendation are in floor 6 — I'm not re-listing them. I did not force-push, modify the PR, or close it; the Holding for your A/B/C. Reply with the letter and I'll execute it. 8问(紧凑版,同 floor 6,无变化):
收工判定: 无新变化,决策仍待 owner;本轮仅复核 + 维持升级状态,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (4th re-trigger), no change Re-verified live this run (not from memory):
The owner has not chosen A/B/C, so there is still no mechanical action to take this run. The full options + recommendation are in floor 6 — I'm not re-listing them. I did not force-push, modify, or close the PR. Holding for your A/B/C. Reply with the letter and I'll execute it. 8问(紧凑版,同 floor 6/11,无变化):
收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (5th re-trigger), no change Re-verified live this run: master 8问: 同 floor 6/11/15,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (6th re-trigger), no change Re-verified live: master advanced to 8问: 同 floor 6/11/15/16,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
2 similar comments
[ework] CI fix verified green ✓ |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (7th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (8th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
3 similar comments
[ework] CI fix verified green ✓ |
[ework] CI fix verified green ✓ |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (9th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (10th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (11th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (12th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (13th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (14th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47/51,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (15th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47/51/55,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
1 similar comment
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (16th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47/51/55/59,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (17th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47/51/55/59/63,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (18th re-trigger), no change Re-verified live: master 8问: 同 floor 6/11/15/16/20/27/29/35/38/43/47/51/55/59/63/67,无变化——语义冲突(同一修复 #6655 的两个等价实现)待 owner 拍板;本轮未改代码,无回归/缓存/设计方向/配置影响。 收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 🔶 需要决策 — still pending (19th re-trigger), no change Re-verified live this run (not from memory):
Same situation as floor 6 — the conflict is not mechanical, it's two byte-identical-equivalent implementations of the same fix (#6655) colliding in Options (full detail in floor 6):
Owner has not chosen A/B/C. Holding; no action taken this run. I did not force-push, modify, or close the PR. Reply with the letter and I'll execute it. 8问 (紧凑版,同 floor 6,无变化):
收工判定: 无新变化,决策仍待 owner;本轮仅复核,未做任何改动。 |
[ework] CI fix verified green ✓ |
Summary
snapshotInit()insrc/context-analysis.ts(the strictly-advisory per-session observability module) allocated oneInitMessageEntryper message and sorted the complete array before retaining only 100. On a long first-request history that added O(n) temporary memory and O(n log n) synchronous work to the first proxied request, even though the visible sample is capped atINIT_PER_MSG_CAP(100).This replaces the allocate-all + sort-all approach with a bounded top-K min-heap (worst entry at the root) that keeps only the 100 largest entries in memory while scanning. Aggregate counts (
totalChars/estTokens) still cover every message.Behavior
No behavior change. The emitted
perMessagesample is byte-identical to the old whole-array stable sort:estTokenscountMessageTokensunchanged)Only the memory footprint and synchronous work of the snapshot drop (1M synthetic messages, local run):
The residual is the unavoidable per-message token count; the retained live memory is now 100 entries regardless of history length.
Tests
tests/context-analysis.test.ts:Existing init-profile tests still pass unchanged. No config surface change. No kernel change. No wire/persistence change.
Closes #6655