Skip to content

[PR] fix(preflight): scope the usage baseline and k̂ estimate to the model that measured them - #6467

Open
ranxianglei wants to merge 6 commits into
masterfrom
2026-11-13_preflight-baseline-model
Open

ranxianglei wants to merge 6 commits into
masterfrom
2026-11-13_preflight-baseline-model

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

The preflight size model (local estimate × k̂ + usage baseline) diverged from the measured usage in both directions, firing spurious folds (#6430):

  1. Cross-model baseline (incident case 2, 303s preflight): a usage baseline measured under a prior model on the same route (282K under opus) carried no model provenance, so it survived the DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 F2 route demotion and drove the trigger while the current payload's own estimate read far below the target (~55K on a fable lane).
  2. Stale k̂ over-correction (incident case 1, 13.7s stall): a fresh same-route+same-model usage baseline (167K, below the 200K target) was overridden by a stale k̂=1.47 that inflated the local estimate (raw 151K → 223K) past the target.

Both are one mechanism: the baseline floor and the k̂-scaled estimate must be scoped to the model that measured them, and a fresh measurement must win over a stale calibration.

Fix

Fix A — model-scoped baseline demotion. Record the model that measured the usage baseline (lastInputTokensModel, set in settleUsageReport and both overflow arms). In the trigger, after the #1933 F2 origin demotion, demote the baseline floor to 0 when the baseline's model differs from the current request's model (both known). Mirrors the k̂ read gate's model dimension (currentCalibrationFactor).

Fix B — fresh-measurement cap on the calibrated estimate. When a fresh (same route+model, not demoted) baseline is present and the payload's raw estimate is not bigger than it, cap the calibrated estimate at the baseline. Rationale: the raw estimate is a direct measurement of the current payload; if it is not bigger than what upstream just billed on this exact route+model, the payload has not genuinely grown — the k̂-scaled excess is over-correction. Genuine growth (raw > baseline) leaves the estimate untouched and still folds. A demoted baseline floors at 0 so the cap is a no-op.

Tests

  • tests/issue6430-preflight-baseline-model.test.ts (new): Fix A (cross-model baseline demoted → no spurious preflight), Fix B (fresh same-model baseline caps a stale k̂ over-correction → no spurious preflight), control (same-model baseline above the window still drives preflight — no regression).
  • tests/issue2129-restart-measurement-state.test.ts: added lastInputTokensModel to the persist round-trip assertion.
  • Verified no regression: issue3645 (calibrated futility), issue1933 (preflight calibration), issue1987 (baseline anchor), issue1841 (futility skip) all green locally.

Behavior change

  • Old: the usage baseline carried only a route (origin) provenance; a cross-model (same-route) baseline drove the trigger. A stale k̂>1 could inflate the estimate past a fresh below-target baseline.
  • New: the baseline is also scoped by model; a cross-model baseline is demoted (the payload's own estimate judges). A fresh same-model baseline caps the calibrated estimate when the raw estimate is not bigger than it.
  • Risk: a session that switches models mid-lane will no longer inherit the prior model's baseline (it re-learns from the new model's first usage report). This is the intended behavior — the prior model's billing is not a valid size model for the new model.

Config surface

None. lastInputTokensModel is an internal stats field (persisted alongside lastInputTokensOrigin), not a config knob.

Closes #6430

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-13_preflight-baseline-model (b885591)

Latest build — install this exact version

npm install -g billion-context@0.0.10982-pr.6467.9fb168b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
b885591 billion-context@0.0.10982-pr.6467.9fb168b build
710a36f billion-context@0.0.10879-pr.6467.d2294ff build
74305b7 billion-context@0.0.10632-pr.6467.e3d25d8 build
e1a7fbc billion-context@0.0.10631-pr.6467.191aaa6 build
33822b5 billion-context@0.0.10174-pr.6467.c21feb3 build
ad1ff55 billion-context@0.0.10150-pr.6467.ff14874 build
2e64cc3 billion-context@0.0.9562-pr.6467.32b4f54 build
9fcff4e billion-context@0.0.9528-pr.6467.6eed51d build
19ab036 billion-context@0.0.9522-pr.6467.8b30e2a build
5289944 billion-context@0.0.9375-pr.6467.0b94456 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6467.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(preflight): scope the usage baseline and k̂ estimate to the model that measured them [PR] fix(preflight): scope the usage baseline and k̂ estimate to the model that measured them Nov 12, 2026
@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 12, 2026
@ranxianglei
ranxianglei force-pushed the 2026-11-13_preflight-baseline-model branch from 5289944 to 19ab036 Compare November 13, 2026 09:14
ranxianglei pushed a commit that referenced this pull request Nov 13, 2026
…switch e2e

The #6430 model-scoped baseline demotion is a third path into the
baselineFloor===0 state the #2313 forward-once escape anticipates: after a
mid-lane model switch the cross-model baseline no longer drives the
trigger, so the first switched-model request is forwarded to acquire the
new model's usage evidence (in the real world the over-window forward
400s and arms the overflow baseline) and the second request folds. The
test now asserts both legs: req2 forwards without summarization and
settles a fresh same-model baseline; req3 folds before forwarding.

Found in PR #6467 review: the old test asserted a fold on the first
switched-model request, which the intended #6430 behavior (stated in the
PR body) no longer does.
@ranxianglei
ranxianglei force-pushed the 2026-11-13_preflight-baseline-model branch from 2e64cc3 to ad1ff55 Compare November 15, 2026 09:55
ranxianglei pushed a commit that referenced this pull request Nov 15, 2026
…switch e2e

The #6430 model-scoped baseline demotion is a third path into the
baselineFloor===0 state the #2313 forward-once escape anticipates: after a
mid-lane model switch the cross-model baseline no longer drives the
trigger, so the first switched-model request is forwarded to acquire the
new model's usage evidence (in the real world the over-window forward
400s and arms the overflow baseline) and the second request folds. The
test now asserts both legs: req2 forwards without summarization and
settles a fresh same-model baseline; req3 folds before forwarding.

Found in PR #6467 review: the old test asserted a fold on the first
switched-model request, which the intended #6430 behavior (stated in the
PR body) no longer does.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 冲突修复完成(PR #6467,分支 2026-11-13_preflight-baseline-model)。

七问:

  1. 是什么问题:PR 分支落后 master 228 个提交(领先 4 个),与 base 分支产生合并冲突(merge state dirty),CI 无法合并。

  2. 什么原因:PR 的 4 个提交里有 1 个(2e64cc305a 重新生成 storage-render golden)与 master 同期对同一文件 tests/golden/storage-render/chat.json 的改动撞车——两边都从同一 base 值改了 stateDigest/stateDigestAligned/stats,master 还额外改了 metadata。golden 是持久化会话状态的 SHA-256 摘要,无法手工合并,必须按「master + PR」合并后的代码重新生成。

  3. 怎么处理的:把 PR 分支 rebase 到最新 master(26d9bcc649)。前 3 个提交干净落地;第 4 个提交在 chat.json 冲突。我装了依赖(npm ci)+ 构建 kernel/pi-subagents,跑 scripts/update-storage-render-goldens.ts 按合并后代码重新生成全部 4 个 golden,用生成结果解决冲突,git rebase --continue 完成。

  4. 冲突怎么解的(关键):chat.json 现在是合并态摘要——stateDigest=d3903d5c…(新值,非冲突任一侧)、metadata=0c7d18…(保留 master 的值,因 PR 的 lastInputTokensModel 是 stats 字段、不动 metadata)、stats=99ecef87…(新值,叠加了 master 与 PR 两边改动)。另外 3 个 golden(anthropic/google/responses)master 没动过,重生成后与 PR 原值一致、未变。PR 的逻辑代码一行未改,只解决了 golden 摘要冲突。

  5. 验证了什么:typecheck 干净(exit 0);golden-storage-render.test.ts 5/5 通过(证明重生成正确);preflight 修复测试 + 控制测试(issue6430/2129/3645/1933/1987/1841 + preflight-compress)25/25 通过。全量回归交给远程 CI。

  6. 是否有回归:无。本次只做了 rebase + golden 重生成,PR 的 preflight 逻辑(跨模型 baseline 降级 + 新鲜测量封顶)原样保留;golden 是按合并后代码重新生成的真值,由 storage-render 测试钉住。PR 本身的行为变更(跨模型 baseline 降级、mid-lane 切模型不再继承旧模型 baseline)是作者在 PR 描述里声明过的,非本次引入。

  7. 现在的状态 / 下一步:已 git push --force-with-lease 推回 PR 分支,head 从 2e64cc305a → ad1ff55155。分支现为 4 ahead / 0 behind master,合并冲突已消,等待 CI 在新 head 上重跑。注:镜像 CI 缓存是 2026-11-13 的旧快照(当时多为 skipped/queued),无法据此判断旧 CI 红绿;本次只处理了「合并冲突」这一硬阻塞,CI 结果以新 head 重跑为准。

判定:冲突已解决、分支已同步 master,可进入 CI 重跑。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

七问 summary — PR #6467 red-CI fix (attempt 1/6)

  1. What failed? 5 test shards failed on run 43701606055 (ubuntu 22/24 shards 1,3; windows 22 shard 2). All 4 failing tests were in tests/issue2189-preflight-billing-block.test.ts (anthropic billing-attribution block, legacy string system, bare-Error 429 fails-fast, genuine 429 replay).

  2. Root cause? The PR's Fix A (cross-model baseline demotion) correctly demotes a usage baseline measured under a different model (tokenizers bill differently across models). The [bug] 订阅登录下 preflight 摘要请求恒 429 rate_limit_error "Error":摘要 system 缺 Claude Code 归属块 #2189 test's setup incidentally relied on a cross-model baseline (300K settled under claude-big, small payload under claude-small) to fire preflight on r2. Under the new behavior that cross-model baseline is correctly demoted, so preflight no longer fires and the 4 billing/429 tests fail. Confirmed PR-caused (not inherited red): the test passes 6/6 on clean master, fails 4/6 on the PR branch.

  3. What did I do? Changed r1's model from claude-big to claude-small (same as r2) in all 4 tests of tests/issue2189-preflight-billing-block.test.ts, so the 300K baseline is same-model and not demoted. Preflight still fires; the test's real purpose (billing-attribution block + 429 handling) is preserved. 4-line test-only change, no production code touched.

  4. How did I verify? The [bug] 订阅登录下 preflight 摘要请求恒 429 rate_limit_error "Error":摘要 system 缺 Claude Code 归属块 #2189 test now passes 6/6 (was 4 fail / 2 pass). The PR's own new test (issue6430-preflight-baseline-model) still passes 3/3. npm run typecheck passes.

  5. Impact (behavior change)? No production behavior change. The PR's cross-model demotion is correct and intended (pinned by the PR's own test, issue6430 Fix A: a cross-model baseline must NOT arm preflight). The fix only aligns the [bug] 订阅登录下 preflight 摘要请求恒 429 rate_limit_error "Error":摘要 system 缺 Claude Code 归属块 #2189 test's setup with that new behavior.

  6. Regressions? None. Test-only change (4 lines). No other test or source file touched.

  7. Decisions needed? None — small mechanical fix consistent with the PR's intended behavior. New CI run started on head 33822b5228 (e2e-native-pi already green; test shards queued).

@ranxianglei
ranxianglei force-pushed the 2026-11-13_preflight-baseline-model branch from 33822b5 to e1a7fbc Compare November 17, 2026 11:30
ranxianglei pushed a commit that referenced this pull request Nov 17, 2026
…switch e2e

The #6430 model-scoped baseline demotion is a third path into the
baselineFloor===0 state the #2313 forward-once escape anticipates: after a
mid-lane model switch the cross-model baseline no longer drives the
trigger, so the first switched-model request is forwarded to acquire the
new model's usage evidence (in the real world the over-window forward
400s and arms the overflow baseline) and the second request folds. The
test now asserts both legs: req2 forwards without summarization and
settles a fresh same-model baseline; req3 folds before forwarding.

Found in PR #6467 review: the old test asserted a fold on the first
switched-model request, which the intended #6430 behavior (stated in the
PR body) no longer does.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI fix: aligned the #663 model-scoping test with the PR's #6430 cross-model demotion — pushed 74305b7050 (single commit, final head, the branch's one full CI pass is now running; its result is not yet verified).

What I did this run:

  1. Re-verified live check state (mirror cache at wake): 3 failing shards on head e1a7fbc1ca — test (ubuntu-latest, 22, 3), test (windows-latest, 22, 2), test (ubuntu-latest, 24, 3); everything else green.
  2. Fetched all 3 job logs: each shard fails on exactly ONE deterministic test — e2e #663 (model scoping): rejection learned for model A keeps model B's 32768 cap at tests/preflight-max-output-tokens.test.ts:387, assertion "second request triggered preflight summaries" (expected ≥1 summary call, got 0). Same failure on all three OS/node shards → not flake.
  3. Root cause: the PR's Fix A demotes a usage baseline measured under a different model. In this test, r2 (same session, model B gpt-6-standard, payload grown over-window) now lands in exactly the state the billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 forward-once escape is designed for: the trigger fires on the payload's own estimate while the cross-model baseline is demoted (baselineFloor === 0 with nonzero lastInputTokens), so the proxy forwards once to acquire model B's evidence instead of folding first (src/server.ts:4067 probeForEvidence). The mock upstream accepts model B, settles a fresh model-B usage baseline, and no summary call happens on r2. The test's r2 expectation predates the PR and assumed the old cross-model baseline inheritance.
  4. Fix (mechanical test alignment — same class as the head commit's [bug] 订阅登录下 preflight 摘要请求恒 429 rate_limit_error "Error":摘要 system 缺 Claude Code 归属块 #2189 alignment): extended the flow by one request. r2 now asserts the evidence forward (reaches upstream, zero summary calls); r3 (same model, still over-window) folds under the fresh trusted model-B baseline and its summaries keep the 32768 cap. The test's real purpose — model A's learned rejection must not strip the cap from model B's summaries — is preserved and still pinned.
  5. Verified locally (targeted): tests/preflight-max-output-tokens.test.ts 6/6 (was 5/6), the PR's own tests/issue6430-preflight-baseline-model.test.ts + tests/issue2189-preflight-billing-block.test.ts + tests/issue2129-restart-measurement-state.test.ts 13/13, tests/preflight-compress.test.ts 4/4, tsc --noEmit clean.
  6. Push discipline: fetched + rebased onto the remote head before pushing (remote was up to date, no concurrent changes), no force-push, no [ework:ci-held] marker on this final head.

结构化八问(中文)

  1. 是什么问题:CI 三个红 shard 实为同一个确定性测试失败——fix: Codex preflight summaries fail with unsupported max_output_tokens #663 model-scoping e2e 测试的第二请求(model B)不再触发 preflight summary(断言期望 ≥1 次 summary 调用,实际 0 次)。
  2. 什么原因导致的:PR 的 Fix A 把「不同模型测得的 usage baseline」降级为不可信([Bug][preflight] 0.1.213: local estimate × k̂ diverges from measured usage both ways → spurious preflight / wrong fold depth #6430 cross-model demotion)。第二请求(model B)的 baseline 被降级后(baselineFloor=0 但 lastInputTokens>0),恰好落入 billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 forward-once 逃生路径的设计场景:trigger 按 payload 自身估计触发,但代理先转发一次获取 model B 的新证据,不先折叠。测试的 r2 断言写于 PR 之前,基于旧的跨模型 baseline 继承行为。
  3. 影响范围与严重程度:仅影响该测试的期望值,无生产行为影响——forward-once 是 billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 的设计自愈路径(成功→settle 新 baseline;4xx→arm overflow),PR 自己的测试(issue6430)已钉住小 payload 的降级场景。
  4. 怎么修复的:把测试流程对齐到新行为(与 head commit 对 [bug] 订阅登录下 preflight 摘要请求恒 429 rate_limit_error "Error":摘要 system 缺 Claude Code 归属块 #2189 测试的对齐同类):r2 断言证据转发(到达上游、零 summary 调用),r3(同模型、仍超窗)在新的可信 model-B baseline 下折叠,且其 summary 保留 32768 cap——测试的真实目的(model A 学到的拒绝不得剥掉 model B summary 的 cap)保持不变且仍被钉住。
  5. 是否有回归问题:无。改动仅测试文件,无 src 变更;同文件 6 个测试 + PR 自有 3 个测试文件 13 个测试 + preflight-compress 4 个测试本地全过;typecheck 干净。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向:不涉及——未改 PR 的设计,测试只是对齐到已设计的 forward-once 行为。
  8. 是否引入了新的配置:不涉及。

收工判定:修复已推送(e1a7fbc1ca..74305b7050,单 commit,最终 head),CI 全量 pass 正在新 head 上运行,结果以 CI test lane 为准。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

No action taken — the reported conflict is stale; verified live this run.

  • Conflict check (live): GitHub API reports mergeable: true; the branch's merge base is exactly the current master head 77fe0b1d63 (the 19:25 [PR] docs(clients): dsh-go host | docs(clients): dsh-go 宿主 #6592 merge) — git merge-base --is-ancestor master branch confirms the branch already sits on the latest master, and git merge-tree against it produces a clean tree with zero conflicting files. An earlier run had already rebased onto this master (this worktree is the conflict-rebase one); the "dirty" flag in the event predates that state.
  • Check state (live, mirror cache @ 12:03:40Z, head 74305b7050): 0 failing — 14 in progress, 1 success, 15 skipped. That is the full CI pass triggered by the test-alignment fix pushed in the previous run; its verdict is still pending.
  • Pushes this run: none (no rebase needed, so nothing to push; no force-push).

Stopping here per the re-verify-first rule. If the in-progress CI pass comes back red, the next wake will handle it.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

The fix correctly scopes the preflight usage baseline and k̂ estimate to the model that measured them. Both incident cases (#6430: cross-model baseline driving spurious preflight, stale k̂ over-correction) are addressed by a single coherent mechanism: model-scoped baseline demotion (Fix A) and a fresh-measurement cap on the calibrated estimate (Fix B). The tests pin both directions plus the control (no regression). The behavior changes are disclosed in the PR body.

8问

  1. 是什么问题: 预检大小模型(本地估计 × k̂ + 使用基线)在两个方向上偏离了实测使用量,导致虚假折叠。
  2. 什么原因导致的: 使用基线只携带路由(origin)来源,没有模型来源;跨模型基线(同路由)在模型切换后仍然驱动触发器。过时的 k̂>1 会膨胀本地估计,超过新鲜的目标以下基线。
  3. 影响范围与严重程度: 受影响的用户面: 在会话中途切换模型的会话。触发条件: 基线模型与当前请求模型不同,或过时的 k̂>1 膨胀估计。严重程度: 特定条件下有问题(虚假预检折叠, 13.7s-303s 延迟)。
  4. 怎么修复的: Fix A: 记录测量基线的模型,在触发器中,当基线模型与当前请求模型不同时,将基线降级为 0。Fix B: 当新鲜(同路由+同模型)基线存在且负载的原始估计不大于它时,将校准估计限制在基线以内。
  5. 是否有回归问题: 审过的既有行为: #663 模型范围流程(已更新测试以反映新的两请求自愈行为), preflight-compress e2e 测试(已更新), golden 文件(已重新生成)。没有发现意外的回归。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。基线现在也按模型范围化,这与 k̂ 读取门的模型维度一致。
  8. 是否引入了新的配置: 不涉及。lastInputTokensModel 是一个内部统计字段(与 lastInputTokensOrigin 一起持久化),不是配置旋钮。

合并判定: 可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review complete: APPROVE posted on PR #6467 (comment-type review, self-approval blocked by GitHub). 8问 in Chinese included. CI green (30/30 checks, 0 failed). No small findings to fix on the spot. No kernel/ changes. Verdict: mergeable.

@ranxianglei
ranxianglei force-pushed the 2026-11-13_preflight-baseline-model branch from 74305b7 to 710a36f Compare November 18, 2026 19:29
ranxianglei pushed a commit that referenced this pull request Nov 18, 2026
…switch e2e

The #6430 model-scoped baseline demotion is a third path into the
baselineFloor===0 state the #2313 forward-once escape anticipates: after a
mid-lane model switch the cross-model baseline no longer drives the
trigger, so the first switched-model request is forwarded to acquire the
new model's usage evidence (in the real world the over-window forward
400s and arms the overflow baseline) and the second request folds. The
test now asserts both legs: req2 forwards without summarization and
settles a fresh same-model baseline; req3 folds before forwarding.

Found in PR #6467 review: the old test asserted a fold on the first
switched-model request, which the intended #6430 behavior (stated in the
PR body) no longer does.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and rebased — new head 710a36f2, CI pass now running.

This wake's conflict was real (unlike the previous one): master moved to a446a3c1 (the #6827 batch11-wave11 merge), whose golden-regen commit 75a45380 (#6775 persisted fields) collided with this PR's own golden-regen commit d8127c60 (persisted lastInputTokensModel field) in the 4 tests/golden/storage-render/*.json files.

  1. What was wrong: base-branch conflict on the 4 storage-render goldens — both sides had regenerated the same digest fields from different code trees (master: [PR] fix(persist): keep k̂ model provenance and the emergency-shrink marker across restarts #6775 fields; PR: lastInputTokensModel).
  2. How resolved: rebased all 6 PR commits onto origin/master; for the goldens, took master's side as placeholder and regenerated from the merged code with the repo's own mechanism (node --import tsx scripts/update-storage-render-goldens.ts) — the same "take master, regenerate below" precedent master itself used in 307337c7.
  3. Resolution correctness: the regenerated stateDigest/stateDigestAligned/sectionDigests.stats are a genuine third value, distinct from both sides (they hash the merged persisted state carrying BOTH [PR] fix(persist): keep k̂ model provenance and the emergency-shrink marker across restarts #6775's fields and lastInputTokensModel); all per-turn wire digests unchanged — outbound wire untouched, matching both sides' stated intent.
  4. Verification (local, targeted): tests/golden-storage-render.test.ts 5/5 on the final tree; the PR's own suites (preflight-max-output-tokens incl. the fix: Codex preflight summaries fail with unsupported max_output_tokens #663 model-scoping flow, issue6430, issue2189, issue2129) 11/11; tsc --noEmit clean. Full-suite verdict = the CI pass now running.
  5. Push: one force-push 74305b705 → 710a36f2 (explicitly authorized for this conflict-rebase; remote head verified unchanged before push; no CI-hold marker — this is the branch's one full CI pass).
  6. Live state after push: GitHub API confirms head 710a36f2a, mergeable: true, state open; checks pending on the new head.
  7. Notes: the workdir had been wiped between runs — remotes/checkout rebuilt from origin before starting. No design decisions needed, no gates tripped, no other files touched.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

No action taken — this conflict wake is stale; the rebase from the previous run already cleared it.

Verified live this run:

  • No conflict: branch head is 710a36f2a (the rebased head pushed in the previous run); master is still at a446a3c1c, the branch sits directly on it (merge-base --is-ancestor = yes), and git merge-tree against master is clean — zero conflicting files. GitHub API confirms mergeable: true on head 710a36f2a.
  • CI: the mirror cache still reports the pre-rebase head 74305b705 (its poller hasn't picked up the new head yet), so I can't yet state the verdict for 710a36f2a — the full CI pass triggered by the rebase push is in flight. mergeable_state: blocked right now reflects pending checks, not a conflict.

Stopping here per the re-verify-first rule; if the in-flight pass on 710a36f2a comes back red, the next wake will handle it.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

Re-review after rebase: content is identical to the previously reviewed head (same 6 commits, same 14-file diff, +356/-32). The fix correctly scopes the preflight usage baseline and k̂ estimate to the model that measured them. Both incident cases (#6430: cross-model baseline driving spurious preflight, stale k̂ over-correction) are addressed by a single coherent mechanism: model-scoped baseline demotion (Fix A) and a fresh-measurement cap on the calibrated estimate (Fix B). The tests pin both directions plus the control (no regression). The behavior changes are disclosed in the PR body.

8问

  1. 是什么问题: 预检大小模型(本地估计 × k̂ + 使用基线)在两个方向上偏离了实测使用量,导致虚假折叠。
  2. 什么原因导致的: 使用基线只携带路由(origin)来源,没有模型来源;跨模型基线(同路由)在模型切换后仍然驱动触发器。过时的 k̂>1 会膨胀本地估计,超过新鲜的目标以下基线。
  3. 影响范围与严重程度: 受影响的用户面: 在会话中途切换模型的会话。触发条件: 基线模型与当前请求模型不同,或过时的 k̂>1 膨胀估计。严重程度: 特定条件下有问题(虚假预检折叠, 13.7s-303s 延迟)。
  4. 怎么修复的: Fix A: 记录测量基线的模型,在触发器中,当基线模型与当前请求模型不同时,将基线降级为 0。Fix B: 当新鲜(同路由+同模型)基线存在且负载的原始估计不大于它时,将校准估计限制在基线以内。
  5. 是否有回归问题: 审过的既有行为: #663 模型范围流程(已更新测试以反映新的两请求自愈行为), preflight-compress e2e 测试(已更新), golden 文件(已重新生成)。没有发现意外的回归。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。基线现在也按模型范围化,这与 k̂ 读取门的模型维度一致。
  8. 是否引入了新的配置: 不涉及。lastInputTokensModel 是一个内部统计字段(与 lastInputTokensOrigin 一起持久化),不是配置旋钮。

合并判定: 可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Re-review complete on rebased head 710a36f: content identical to previously reviewed head (same 6 commits, same diff). APPROVE posted (comment-type, self-approval blocked by GitHub). 8问 in Chinese included. CI green (30 checks, 0 failed). No small findings. No kernel/ changes. Verdict: mergeable.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

1 similar comment
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

@ranxianglei ranxianglei mentioned this pull request Nov 19, 2026
… that measured them (#6430)

The preflight size model diverged from the measured usage in both
directions, firing spurious folds:

1. Cross-model baseline: a usage baseline measured under a prior
   model on the same route carried no model provenance, so it
   survived the #1933 F2 route demotion and drove the trigger while
   the current payload's own estimate read far below the target.

2. Stale k̂ over-correction: a fresh same-route+same-model usage
   baseline (below the target) was overridden by a stale k̂>1 that
   inflated the local estimate past the target.

Both are one mechanism: the baseline floor and the k̂-scaled estimate
must be scoped to the model that measured them, and a fresh
measurement must win over a stale calibration.

Fix A: record the model that measured the usage baseline
(lastInputTokensModel) and demote the baseline floor on model
mismatch, mirroring the k̂ read gate's model dimension.

Fix B: when a fresh (same route+model) baseline is present and the
payload's raw estimate is not bigger than it, cap the calibrated
estimate at the baseline — the k̂-scaled excess is over-correction.

Fixes #6430
ci-check added 5 commits November 19, 2026 19:50
… (session.ts)

The #6430 change shifted the existing lastInputTokensOrigin field and the
#1933 F1 comment line one space right (8→9 / 9→10), leaving the #1933 F1
comment internally inconsistent. Restore the original indentation and
indent the new field/comment to the block's field level (8/9), matching
the neighboring #1933 F2 comment. Type-only change; typecheck clean.
…switch e2e

The #6430 model-scoped baseline demotion is a third path into the
baselineFloor===0 state the #2313 forward-once escape anticipates: after a
mid-lane model switch the cross-model baseline no longer drives the
trigger, so the first switched-model request is forwarded to acquire the
new model's usage evidence (in the real world the over-window forward
400s and arms the overflow baseline) and the second request folds. The
test now asserts both legs: req2 forwards without summarization and
settles a fresh same-model baseline; req3 folds before forwarding.

Found in PR #6467 review: the old test asserted a fold on the first
switched-model request, which the intended #6430 behavior (stated in the
PR body) no longer does.
…tInputTokensModel field

The new stats field changes the persisted session state, so the
storage-render goldens' state digests drift. Regenerated with
scripts/update-storage-render-goldens.ts (intentional persist contract
change — the field is the PR's own). Diff is surgical: per wire only
stateDigest, stateDigestAligned and sectionDigests.stats changed; all
per-turn wire digests are unchanged, so the outbound wire is untouched.
The PR's Fix A demotes a usage baseline measured under a different model
(cross-model baselines are no longer inherited — tokenizers bill
differently across models). The #2189 test's setup incidentally relied on
a cross-model baseline (300K settled under claude-big, small payload under
claude-small) to fire preflight on r2; under the new behavior that
cross-model baseline is correctly demoted, so preflight no longer fires and
the 4 billing/429 tests fail.

Change r1's model from claude-big to claude-small (same as r2) so the 300K
baseline is same-model and not demoted. Preflight still fires; the test's
real purpose (billing-attribution block + 429 handling) is preserved.
The PR's Fix A demotes a usage baseline measured under a different
model. In the #663 model-scoping test, r2 (same session, model B,
grown over-window) now hits exactly the state the #2313 forward-once
escape is designed for: the trigger fires on the payload's own
estimate while the cross-model baseline is demoted to untrusted
(baselineFloor === 0 with a nonzero lastInputTokens), so the proxy
forwards once to acquire model B's evidence instead of folding first.
The mock upstream accepts model B, settles a fresh model-B usage
baseline, and no summary call is made on r2 — the old expectation
('second request triggered preflight summaries') no longer holds.

Extend the flow by one request: r2 asserts the evidence forward (no
summary call), r3 (same model, still over-window) folds under the
fresh trusted model-B baseline and its summaries keep the 32768 cap —
the test's real purpose (model A's learned rejection must not strip
the cap from model B's summaries) is preserved and still pinned.
@ranxianglei
ranxianglei force-pushed the 2026-11-13_preflight-baseline-model branch from 710a36f to b885591 Compare November 19, 2026 11:52
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Re-rebased onto the new master — new head b8855918, conflict cleared, CI pass now running.

This wake's conflict was real: master moved to 6453feeeb (#7151 hash-cleanup-batch1, plus #7191/#7281) and the same 4 tests/golden/storage-render/*.json files collided again — master regenerated those goldens on its side while this PR's golden-regen commit carries the lastInputTokensModel field.

  1. What was wrong: base-branch conflict on the 4 storage-render goldens (second occurrence — the golden files regenerate on every batch merge, so they re-collide each time master moves).
  2. How resolved: identical mechanism to the previous rebase — rebased all 6 PR commits onto origin/master, took master's goldens as placeholder, regenerated from the merged code with node --import tsx scripts/update-storage-render-goldens.ts.
  3. Resolution correctness: diff is again digest-fields-only (stateDigest/stateDigestAligned/sectionDigests.stats, 6 lines per file); all per-turn wire digests unchanged — outbound wire untouched.
  4. Verification (local, targeted): tests/golden-storage-render.test.ts green on the final tree; preflight-max-output-tokens (incl. fix: Codex preflight summaries fail with unsupported max_output_tokens #663 model-scoping flow), issue6430, issue2189, issue2129 — 11/11; tsc --noEmit clean. (Workdir had been wiped between runs — deps + kernel/pi-subagents/dist rebuilt before regenerating.)
  5. Push: one force-push 710a36f2a → b8855918e (remote head verified unchanged first; no CI-hold marker — the branch's one full CI pass).
  6. Live state after push: head b8855918e up, mergeable: true; checks pending on the new head.
  7. Note: the pre-rebase head 710a36f2a had a fully green CI pass (14 success / 16 skipped / 0 fail per the mirror cache) — the rebase only re-derived the 4 golden digests, so the new pass should track it. No design decisions needed, no gates tripped.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

Third re-review: content identical to the two previously reviewed heads (same 6 commits, same 14-file diff, +356/-32). Rebase onto newer master only — no code changes. Fix A (model-scoped baseline demotion) and Fix B (fresh-measurement cap) verified correct. Tests pin both directions plus control. Behavior changes disclosed. CI green (30/30).

8问

  1. 是什么问题: 预检大小模型(本地估计 × k̂ + 使用基线)在两个方向上偏离了实测使用量,导致虚假折叠。
  2. 什么原因导致的: 使用基线只携带路由(origin)来源,没有模型来源;跨模型基线(同路由)在模型切换后仍然驱动触发器。过时的 k̂>1 会膨胀本地估计,超过新鲜的目标以下基线。
  3. 影响范围与严重程度: 受影响的用户面: 在会话中途切换模型的会话。触发条件: 基线模型与当前请求模型不同,或过时的 k̂>1 膨胀估计。严重程度: 特定条件下有问题(虚假预检折叠, 13.7s-303s 延迟)。
  4. 怎么修复的: Fix A: 记录测量基线的模型,在触发器中,当基线模型与当前请求模型不同时,将基线降级为 0。Fix B: 当新鲜(同路由+同模型)基线存在且负载的原始估计不大于它时,将校准估计限制在基线以内。
  5. 是否有回归问题: 审过的既有行为: #663 模型范围流程, preflight-compress e2e, golden 文件。没有发现意外的回归。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。

合并判定: 可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Third re-review on rebased head b885591: content identical to prior heads (rebase only, no code changes). APPROVE posted. CI green (30/30, 0 failed, 0 pending). No findings. Verdict: mergeable.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][preflight] 0.1.213: local estimate × k̂ diverges from measured usage both ways → spurious preflight / wrong fold depth

1 participant