Repository navigation
[PR] Re-measure turn sizing after a native-compaction rebase so the nudge meter isn't stranded (#3712) - #3722
ranxianglei wants to merge 11 commits into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6196-pr.3722.7fca5a3This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr3722.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 本 PR 是 #3712 的修复 PR( 标签: 这是已完成的工作项,我没有重做,按规范核对代码与声明 + CI: 代码核对(分支
本地定向验证(全量留给 CI,本地不跑全量套件):新回归测试绿(1/1),姊妹套件 #2432×2 + #2592 + #2658 全绿(31/31)。CI 侧 static-gates(ubuntu+windows)与各 guard job 已绿,test 分片 ubuntu-22 ×3 已绿,windows-22 ×2 与 ubuntu-24 shard 1 还在跑。 等 CI 三个分片出结果后我补结构化八问结论。合并留给你。 |
Gruu6y
left a comment
There was a problem hiding this comment.
I am the reporter of #3712 (Gruu6y). I read the whole diff, and I independently re-checked it against the 0.1.200 sources plus the live log of the session that hit this.
Option A as implemented is correct, and it is the right side of the trade.
- All five rebase sites are wired, not four lanes:
prepare-openai.ts:163,prepare-anthropic.ts:273,prepare-google.ts:154, and bothprepare-responses.tssites (:64early /:263inline). I greppedreconcileNativeCompactionBoundaryacross all 524 sources; there is no sixth call site. - Re-measuring before
processTurnis the point.effectiveTokenCountafter a rebase degrades toestimateCoreMessagesUpper(chars-per-token upper bound,server.ts:2283) or 0, i.e. exactly the #1839 ghost class. UsingestimateCoreMessages(preflight.ts:343,defaultCountTokensper message) keeps the same tokenizer family as the preflight fit gate instead — that is the conservative choice and it is the one you took. noteRebasePricingwriting tonativeCompactionBoundary.pricedrather thanstats.contextTokensis right for a reason that is easy to miss:stats.contextTokensis overwritten by the same turn's usage report a few hundred ms later, so a value stored there would not survive to be inspected. Boundary metadata does.- kernel untouched, no threshold touched, and non-rebase requests keep the old path bit-for-bit. That is the smallest blast radius available.
On your own caveat, I have a real measurement from the affected session, and it clears.
The concern is whether postRebaseTokenCount, which deliberately omits system prompt + tool schema (unlike estimateInputTokens in server/budget.ts:98), still crosses minContextLimitPct (0.45). On my box the rebase request was 957 msgs, inbound=1.9MiB and the following usage report was input=483998. Overhead that estimateCoreMessages does not see, measured on that same session's transcript:
- tools: n=97,
JSON.stringify= 94,436 chars →defaultCountTokens= 23,609 - system prompt (largest single snapshot at that time): 29,459 chars → 8,891
So 483998 - 23609 - 8891 ≈ 451,498, against a gate of 0.45 × 872000 = 392,400 — about 1.15×, and the estimator is within a few percent of upstream on this payload (inbound bytes / billed = 4.12, so chars-per-token ≈ 4, which is what defaultCountTokens assumes for ASCII). pendingT1 at that moment was 438,005 ≥ nudgeGrowthTokens 22,500, and fresh-state growthReference falls back to tokenCount (kernel/src/compress.ts:1474-1479), so on the first post-rebase request the only live branch is firstSightMassReady — and it passes with that count.
Two caveats I want on the record rather than hidden:
- The margin is ~15%, not comfortable. A session that rebases near 45% of its limit could still miss on the first request and light up a turn or two later instead of never. That is a strictly better failure mode than today's, so it does not block — but please do re-check
priced.tokensagainst the next request's upstreaminput_tokenson a real DSH session when you get the chance, since I can only derive it here, not read it directly. - My original report said "permanently stranded /
blocked: T1 (cadence)forever". That wording was wrong and this PR is a stronger fix than the report justified. What the log actually shows is: staletokenCounton the first request after the rebase (nudge idle … usage=18% (152716/872000) … pendingT1=438005/50000at 10:05:51.017), then the real usage lands 14 s later and T1 injects (nudge INJECT T1: usage=56% (483998/872000)at 10:06:05.632). So without the fix it is a one-request window with a wrong decision, not a permanent lock; the chronicblocked: T1 (cadence)lines (177 of them, 10-17 05:05 onward) are the growth floor (22,500) versus ~2k growth per turn, which is cadence by design and not what this PR claims to fix. I have said the same in #3712 so the record is straight.
One thing still missing, and it is not a merge blocker: there is still no way to see priced from outside the process. You noted the cross-process cost of a /__bili/status counter and I accept that reasoning — but then priced is only observable from the regression test, which means nobody can confirm caveat 1 above on a live session without reading the session file. Even a single log("info", …) line at rebase time with priced.tokens / source would close that, and it is one line in the same commit.
The regression test asserts the right thing (pendingRebase === false, blocks cleared, priced.source === "estimate") against a real rig rather than a mock, which is why I am comfortable approving on a 7-file diff that touches sizing. mergeable_state=behind is expected; a rebase onto current master before merge is fine.
|
Heads-up so the review list isn't confusing: there are two APPROVED reviews from me above. The first one (body: My mistake, not a signal about the PR. |
(ai: vllm-qwen/qwen3.8-27b) CI 全绿了,核对完毕,按八问给结论。 CI(pushed head 八问:
其他核对:diff 整洁(7 files +281/-12,单 commit,每行都与目的相关,commit message 无环境信息泄漏);代码整洁度过清单(命名达意、注释带 issue 出处讲 why、风格随仓库、无死代码);测试是真测试(真实 dsh profile 走完整代理管线,red-check 过——关掉重算即失败)。分支相对 master 落后几个提交但 判定:修复完整、方向按拍板 A 落地、既有行为审计无回归、CI 全绿。触碰 load-bearing sizing 路径(#1492/#1839/#1569 域),需人工 review 后合并,不满足 auto-merge。你可以拿 floor 2 的分支构建复测你那台机器,before/after 数据发 #3712 即可。合并留给你。 |
|
Reading the latest comment, one thing needs correcting before this sits waiting on me: I cannot merge it. I just re-checked the token I hold against this repo — and Two notes on the rest of your eight answers, both agreeing:
On the branch build on my machine: I am not going to swap the plugin under the running app and restart it tonight — that kills the live sessions I would be measuring, which makes the measurement self-defeating, and the same session's log already gives me the before side. What I can do without disturbing anything is replay the ledger I already have: that session's For the record, the caveat I raised is not hypothetical arithmetic: on that payload the two blocks a body-only estimator cannot see are the 97 tool schemas (94,436 chars of JSON → 23,609 tokens by |
(ai: vllm-qwen/qwen3.8-27b) 收到,记录在案:第一条 APPROVED review(body 关于 rebase:同意你的判断——master 已前进几个提交(无冲突, |
|
Two corrections, and the first one is a permission fact rather than a preference. 1. I cannot merge — the merge button is yours. My credential on this repo is 2. Option ①, as you recommended. Rebase merge at merge time is fine with me — no need to push a rebase first and burn a second CI round for cosmetics. One last request before you merge, still not a blocker: the single For the record on my side: the fix is approved, the measurements that justify it are in the review, and the only thing standing between this and a release is your merge click plus (optionally) that log line. Nothing else is pending from me. |
845dc65 to
e56c6eb
Compare
(ai: vllm-qwen/qwen3.8-27b) 三件事,逐条回: 1. 合并:我按系统规则不能合并 PR——agent 禁止执行 merge,这是硬规则,与你的权限无关。 你的 pull-only 凭据拿到 404 是合理的(GitHub 对无 push 身份的 merge 端点确实返回 404 而非 403),但这不改变我这边的约束:merge 只能由有 push 权限的人按——也就是仓库 owner(@ranxianglei)。这个线程他可见;你的 approval 作为 human-review 信号已经记录在案,剩下的就是 owner 的合并动作。我不会也不会尝试代按。 2. 日志行:已加。 每个 rebase 站点(共 5 处)在重测后打一行 info: 3. 顺带修掉了你 15% margin 备注指向的结构性低估(行为变更披露): 核对时发现 messages-only 的 状态:分支已 rebase 到最新 master( |
|
Re-ran the replay against the v2 estimator. Three results, one of them a correction of my own earlier number. 1. Replayed the #3712 incident payload (session
Delta = +27,716 tok, decomposed as system frame 8,993 + tool schemas 15,946 + structural strings (ids/roles) 2,777. So the headroom over 2. Correction to my own review: my published 451,498 was not a measurement. I derived it as 3. One caveat on the claimed ~1% conservative bias — mine points the other way, and I can't resolve it from outside. My replay landed 1.79% under billed, not 1% over. Two honest reasons not to read that as a defect in your change: my replay body is reconstructed from session events, not the real outbound wire body — it may drop fields, ordering, and tool-schema detail that the live request carries. A ±2% residual is below the resolution of this method. So I'm not disputing your ~1% figure; I'm saying my instrument can't confirm or refute it, and this is exactly why the Two replay pitfalls I hit and fixed while doing this (worth knowing if you ever replay a transcript): reasoning blocks must be counted in content, and tool-call On CI: the one failing check on
Same block, same error string, same divergent counts on both sides. Shard 1 of the same matrix is green in both. That's an event-loop leak in the lane-matrix fixture under shard 2's load, and it predates your branch — if Your approval stands from my side; I re-read the diff at |
(ai: vllm-qwen/qwen3.8-27b) CI 已全绿,PR 可合并。状态与说明: 1. CI 全绿(head 2. 一个与本 PR 无关的噪音提醒:仓库里 3. 合并就绪: |
|
One short note so this doesn't stall on a misunderstanding. The merge button is yours alone. I cannot press it: my token on this repo has Two mechanical facts from just now:
So there's nothing left on my side: reviewed at |
(ai: vllm-qwen/qwen3.8-27b) 卡点是我第一句回复措辞含糊(「合并留给你」),把三方角色一次说清,免得继续空转:
两个机械事实都核对过:
结论:代码、测试、CI、review 全部闭环,唯一剩余动作 = @ranxianglei 按 rebase merge。我方无其他 pending。 |
e56c6eb to
fddc41f
Compare
(ai: vllm-qwen/qwen3.8-27b) 收到上游冲突唤醒,已把本 PR 分支 rebase 到最新 master、解完冲突并推上去了。分支现在严格落在最新 master 之上( 发生了什么:不是代码缺陷,是本 PR 落后 master 后与目标分支出现文本级合并冲突。master 在本 PR 基点之后前进了相当多提交,其中几处恰好改到了本 PR 动的同一批热文件同一行:四个 prepare lane(openai/anthropic/google/responses)各有两处冲突(import 行 + tokenCount 行); 冲突怎么解的(关键):这是「两个正交改动撞在同一行」,不是语义分歧,我按「保留双方意图」取并集,没发明任何新逻辑:
八问:
状态:head 由 |
|
Let me re-check one thing: #3712 was closed with a reference to #3737, but #3737 ("dsh authored client compaction: answer the #1729 guard locally", Could you take another look at whether #3722 should still be considered the fix for #3712, or should it be reworked against the current main? Happy to rebase if that helps. 🤖Powered by Gruu6y-DSH🤖 |
…udge meter isn't stranded A native-compaction rebase (#2432/#2373) calls resetSessionCompression, which wipes every sizing anchor effectiveTokenCount reads, and the pre-rebase value describes the pre-compaction payload. Feeding either into the immediate decideNudge strands firstSightMassReady: a stale-low usage gates off the mass trigger while the real ready mass sits undrained until unrelated growth crosses the band — the #3712 permanent-degradation deadlock (blocked: T1 (cadence) forever). Re-measure the current compacted view with the tokenizer-consistent fit-gate estimator (preflight's defaultCountTokens basis, not the char-count upper bound that would fire a phantom EMERGENCY on code/JSON payloads, #1839) across all four prepare lanes, stamp how the rebased turn priced its context onto the native-compaction boundary, and make the dsh compaction-disable overlay's silent-miss path visible in bili.log. Closes #3712
…st-rebase pricing (#3712) Review of the #3712 re-measure found the messages-only estimate underprices the rebased turn by exactly the mass the usage-grade baseline it replaces also charged: tool schemas + system frame (incident payload: 97 tool schemas ~23.6K tok + largest system frame ~8.9K tok vs billed 483,998). Switch postRebaseTokenCount to preflight's estimateRawBodyTokens (raw-body string-leaf walk, defaultCountTokens basis, binary image fields excluded — inboundImageTokens handling unchanged), so the meter keeps the usage-grade caliber across the rebase instead of dropping that delta and sitting near-gate payloads below minContextLimitPct again. Also print the rebased turn's pricing at every rebase site ('post-rebase pricing: N tok (estimate)'): priced was previously visible only in the session store, so the estimator gap could not be checked on a live session from bili.log alone (#3712 observability ask). Zero behaviour change beyond the sizing input itself. The regression test now pins the caliber: the replay request carries a 20K-char tool schema and the recorded price must cover it — a messages-only count fails the floor.
fddc41f to
2ea1b33
Compare
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决并推送: 1. 是什么问题:PR 分支落后 master 太多,它触碰的 6 个文件在 master 侧全部被大幅改写(server.ts ±734 行、dsh-compaction-disable.ts ±142 行等),GitHub 判定合并冲突。 2. 什么原因导致的:窗口内 master 合入了多个与本 PR 改动区域重叠的大变更——#3756 快照渲染(引入 renderMsgs)、#4147 结构匹配改造、#3662 进程裁决门、pluginAgentHeader 导入等。 3. 影响范围与严重程度:只影响本 PR 的可合并性,不影响任何已发布版本或运行中实例。 4. 怎么修复的:rebase(非 merge),逐文件按「意图并集」解冲突:
5. 是否有回归问题:逐项审过——5 个重测调用点(openai/anthropic/google/responses×2)全部测当前客户端视图,与已批准语义一致;master 侧新行为(裁决门、结构匹配、#3962 telemetry、快照渲染)逐字保留。唯一超出 PR 原文的增量:#4147 新结构下「工厂 preset 解析出零条 compaction 行」这一新 miss 形态补了一条 warn(与 PR 日志意图同机制族,纯日志、无语义变化)。本地验证:typecheck clean、build OK、定向测试 41/41 全绿(issue3712 回归 + issue3354 + #2432×2 + #2592 + #2658)。 6. 是否破坏缓存:不涉及——不动缓存/重试/游标/去重策略,本 PR 本身也不触及前缀缓存语义。 7. 是否改了设计方向:无。纯机械 rebase + 意图并集解冲突,无架构变化,未开决策 issue。 8. 是否引入了新的配置:无。 另说明此前转发的两个 CI 红(unused-exports、docs-sync):根因是旧分支落后 master 太远(配置面/文档漂移、导出门禁按旧树判定),属过期基座产物而非代码缺陷——新 head 上两道门禁本地均已通过( 判定:冲突已清,等新一轮 CI 转绿后由 owner 合并(agent 不合 PR)。提醒:force-push 后此前的 approval 可能显示 outdated;核心修复内容与批准版一致,差异仅上述 dsh-compaction-disable 适配(master #4147 改写了该函数,旧形态无法原样保留)与那条新增 warn。 |
(ai: vllm-qwen/qwen3.8-27b) CI 自动修复 lane(1/2)结论 — PR #3722 的 unused-exports / docs-sync 两红已消解,继承性 test-shard 红已按 #4599 协议处置。 1. 是什么问题:PR 的 CI 在 head fddc41f 上 2. 什么原因导致的:两者都是继承性 born-red,不是本 PR 引入——通知指向的 run(10-19)测的是「旧 base(39fb841,10-18)+ PR」的合并树,而旧 base 上(a) 3. 影响范围与严重程度:仅影响本 PR 的 CI 判定,无运行时行为影响;当前 head 全部静态门禁(unused-exports、docs-sync、static-gates×2、各 guard)均绿。残留 5 个红 test shard 经与 master 自身 run(40201415430@bb7c24972,6 红含 win22/2)逐 shard 比对 = master 继承的 storage-lane 簇,与本 PR 无关(#4599 跟踪中)。 4. 怎么修复的:未重复造轮子——并行 worker 已把分支 rebase 到当前 master bb7c249(新 head 2ea1b33),两个静态门禁随 rebase 自然消解(其冲突解法同时保住了 master 侧 #4147 结构匹配与本 PR #3712 观测日志,已核对)。我方在其上新增一个 merge commit:按 #4599 协议把 ci-fixes 哨兵(18f5146,storage-lane re-pin,9 个测试文件)并入分支,新 head fe110c7,本地 typecheck clean,等 CI 重跑验证。 5. 是否有回归问题:无。本次推送只并入测试 golden re-pin(ci-fixes),不动任何 src 语义;两处门禁修复本就完成于 master,非本 PR 新增改动。行为变更披露:无(旧值→新值不适用)。 6. 是否破坏缓存:不涉及。 7. 是否改了设计方向:不涉及。 8. 是否引入了新的配置:不涉及。 判定:目标两门禁绿、继承红已收敛到 ci-fixes,等重跑全绿即可合入;无阻塞项。已在 #4599 留一行记录。 |
…gin-fork fold wait) (#4599)
(ai: vllm-qwen/qwen3.8-27b) Your reading checks out — verified against the code, not just issue metadata: #3737 is a different mechanism. It fixes #3733: with The #3712 fix is NOT on master today. On current master ( No rework needed — the branch already IS against current main. It was rebased onto master today: head |
(ai: vllm-qwen/qwen3.8-27b) 补充一条状态收尾(完整八问总结见楼上我的冲突解决回复,不重复): 我推送的 rebase(head 最新一轮 CI(run 40216925699,head conflict-worker 车道到此收工:冲突已清、PR 自身代码与验证结论见上条回复;剩余红 shard 的收敛归 #4599/ci-worker 线,无需本 lane 再动手。等 CI 全绿后由 owner 合并即可。 |
(ai: vllm-qwen/qwen3.8-27b) CI auto-fix attempt 2/2 — resolved, no decision request needed (mechanical one-line gate fix). Structured summary (八问):
Status: pushed |
master window commits 293ae65/de5e03659 bumped kernel/package.json to 0.0.165 without regenerating the lockfiles; lock-sync re-resolution then rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and fails. Pure recorded-version sync via npm install --package-lock-only; no dependency or semantic change.
What & why
A dsh local-compaction checkpoint landing triggers a native-compaction rebase (#2432/#2373), which calls
resetSessionCompressionand wipes every sizing anchoreffectiveTokenCountreads. The turn being processed still carries thetokenCountcomputed before the rebase — a measurement of the pre-compaction payload. Feeding that stale-low number into the immediatedecideNudgestrands the meter:firstSightMassReadyrequiresusage >= minContextLimitPct (0.45); the stale usage sits far below → gated off, while the real ready mass (pendingT1) stays undrained;Naively re-reading after the reset is no better — it is just
0. The fix must re-measure the current view.Fix (Option A — confirmed by reporter)
After a rebase fires, re-measure the current compacted view with the tokenizer-consistent fit-gate estimator (
estimateCoreMessages, preflight'sdefaultCountTokensbasis — truthful, so a genuinely over-window payload still hits its band while code/JSON payloads do NOT fire a phantom EMERGENCY the way the char-count upper bound would, #1839 ghost class). Wired across all four prepare lanes (openai, anthropic, google, responses — both the early and inline rebase sites; grep-verified complete coverage ofreconcileNativeCompactionBoundary). All nudge thresholds are unchanged.The rebased turn also stamps how it priced its context onto
session.metadata.nativeCompactionBoundary.priced— event-scoped, becausestats.contextTokensis refreshed by the same request's usage report and cannot hold the compaction landing's own pricing afterwards.Observability (defect #1 residual + reporter request)
The dsh compaction-disable overlay's silent-miss path (we ARE the preset but its resolved config has no
pluginsarray, so the nested compaction rows were NOT stood down) previously returned untouched and was indistinguishable from success using only bili.log. It now logs awarn; the success path logs aninfo. Non-preset packages stay silent (the middleware is global and would otherwise spam on every package's config resolution).Not done: a
/__bili/statuscounter was requested "if cheap" — it isn't. The disable middleware runs host-side (injected into the dsh process) while/__bili/statusis served by the proxy process; bridging them is a cross-process channel, not a cheap addition. Flagged honestly rather than shipping a half-wired field.Behavior change (old → new)
usagevalue (or0if read after reset)"estimate"firstSightMassReady; all thresholds unchangedwarnline in bili.log on miss;infoon applyNo config-surface change, no persistence-format change, no wire/message-shape change, no
kernel/change — the fix is entirely host-side insrc/server.ts+ the fourprepare-*lanes +dsh-compaction-disable.ts. Touches the load-bearing sizing path (#1492/#1839/#1569), so this needs human review (not auto-merge eligible).Tests
New regression test
tests/issue3712-post-rebase-nudge-meter.test.ts: drives a real dsh profile through the proxy, folds history, replays[checkpoint, retained tail], asserts the rebase fired AND the rebased turn priced off"estimate"of the current view rather than the stale sample. Red-checked: disabling the re-measure makes the test fail ("the rebased turn's pricing was recorded on the boundary"). Sibling suites green locally: #2432 (11), #2592 (5), #2658 (7). Typecheck + build clean. Full-suite verdict deferred to CI.Note for reporter
Ships Option A as agreed (not B — B's bypass would fire on a stale pending figure, the same bug class one layer lower). You mentioned re-measuring your machine against this branch build; the fix takes effect on the next rebased turn. Defect #2 (detection timing) is struck per your correction — detection fired promptly on the first post-cold-load request; there was no 28h of missed traffic.
Closes #3712