Skip to content

[PR] fix: name the oversized range in the manual compress lane's refusal (#6945) - #6951

Open
ranxianglei wants to merge 7 commits into
masterfrom
2026-11-16_compress-oversized-range-diagnostics
Open

ranxianglei wants to merge 7 commits into
masterfrom
2026-11-16_compress-oversized-range-diagnostics

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

⚠️ MODEL-FACING TEXT CHANGE: the all-candidates-failed refusal now appends an [oversized: ...] diagnostic naming the offending range, its measured input tokens, and a suggested split count. The existing refusal text is unchanged; the diagnostic is additive.

What this fixes (#6945, proposal 3)

The manual compress lane had no size sharding and no legacy fallback — one oversized range exhausted every external-summary target and the refusal was a generic exhausted with no hint of which range to split. This adds the minimum-viable diagnostics:

  • Extract the summary system prompt + JSON payload + input-token estimate into shared helpers (external-summary.ts) so the manual lane's pre-validation matches the HTTP guard's byte-for-byte.
  • Add ConfiguredSummaryPlan.tightestInputBudget — the smallest (contextWindow - outputTokens - 256) across the chain.
  • On an all-candidates-failed refusal, pre-validate each nominated range against the tightest budget and name the oversized ones with a suggested split count.

What is NOT in this PR

Proposals 1 (size sharding / half-span retry) and 2 (legacy fallback opt-in config flag) of #6945 need owner decision and are tracked in the issue thread. This PR is the no-decision diagnostics part only.

Regression safety

  • The HTTP guard's behavior is unchanged: byte-identical system/content strings, same budget formula (inputTokens > contextWindow - outputTokens - 256).
  • The diagnostic is additive — it only appears on an all-candidates-failed refusal, and only when a range exceeds the tightest budget.
  • Regression test: tests/external-summary-runtime.test.ts — oversized range names the offending range and suggested split in the refusal (#6945).

@ranxianglei ranxianglei added bug Something isn't working external-summary p2 labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-16_compress-oversized-range-diagnostics (b798110)

Latest build — install this exact version

npm install -g billion-context@0.0.10496-pr.6951.2faa7c9

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
b798110 billion-context@0.0.10496-pr.6951.2faa7c9 build
2dc86d0 billion-context@0.0.10492-pr.6951.477e9dd build
2e46b09 billion-context@0.0.10442-pr.6951.70a777d build
98ea2e1 billion-context@0.0.10424-pr.6951.75375c8 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6951.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Verification + review of PR #6951 (#6945 proposal 3 — the no-decision diagnostics part). I fetched the branch (98ea2e1) and reviewed the diff against master. It's a fix: scoped to the external-summary module and a model-facing text change (flagged correctly in the PR body). The PR body's regression-safety claims check out.

Code — verified correct:

Model-facing text (highest priority): the appended [oversized: m…-m… ≈Ntok, split ≥K …] is honest (≈/≥ labels), actionable, and introduces no phantom values — the range refs are the model's own nominations and the budget is a real computed value. Consumption chain is clean.

New test: a real regression test (asserts the diagnostic fires; would fail pre-fix). Determinism margin is comfortable — the fixture range is ~2013 tokens vs the 1664 budget (~21% headroom). Minor: no explicit negative assertion (a non-oversized range should not get the diagnostic); the existing test covers that case implicitly but doesn't assert the absence.

Status (live, mirror cache 13:45Z):

  • CI in-flight — 10 test jobs queued, 1 in_progress, 3 success (static-gates), 16 skipped, no failures. Authoritative green/red pending.
  • Branch is 53 commits behind master (mergeable_state: behind) — needs a rebase to current master before merge (§7.2). Not a code defect; a mergeability gate.
  • Labels already set: bug / p2 / external-summary (correct for this change).

Verdict: the code is correct, well-scoped, and regression-safe. It is not yet mergeable — it needs (1) a rebase to current master and (2) CI green on the rebased head. Per the [PR] verification workflow I have not rebased proactively (no CI failure, no explicit request). Say the word and I'll rebase + push; I'll also pick it up if CI comes back red.


结构化八问(中文,供人工 reviewer):

  1. 是什么问题:手动 compress 车道在"所有外部摘要候选全失败"时,拒绝信息只有一句泛化的 exhausted,不告诉用户/模型是哪个 range 超尺寸、该拆成几段。
  2. 什么原因导致的:手动车道对一个 range 一对一组装摘要任务,没有尺寸上限/分片/折半;超尺寸 range 在 fetch 前就被 context-budget 守卫全部拒绝,而拒绝文本是写死的泛化串,不携带"哪个 range 超了"的诊断。
  3. 影响范围与严重程度:仅影响 externalSummary 启用 + 手动 compress + 提名了超尺寸 range 的场景;表现为"能用但体验差"(拒绝不可操作,用户得自己猜拆哪个),需绕行(手动拆小 range)。非可用性缺陷。
  4. 怎么修复的:把摘要 system prompt + JSON payload + input-token 估算抽成共享 helper,让手动车道预校验与 HTTP 守卫逐字节一致;新增 tightestInputBudget(全链最小的 contextWindow-outputTokens-256);全失败拒绝时对每个提名 range 预校验,超预算的在拒绝文本里点名 range + 实测 token + 建议拆分数。纯增量诊断,不改既有拒绝文本。
  5. 是否有回归问题:HTTP 守卫行为逐字节不变(代数等价 + 字符串逐字节一致);诊断只在"全失败 + 有 range 超最紧预算"时出现,是增量;未触碰 config schema / 持久化 / wire 协议 / kernel。既有"全候选失败"测试不受影响(其 range 小、预算大不触发;且不断言精确文本)。
  6. 是否破坏缓存:不涉及——不触碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向:不涉及——仍是"手动车道全失败 → 拒绝"的既有路径,只是拒绝文本更具体;分片/legacy 回退(proposals 1/2)明确不在本 PR,留待 owner 决策。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值;tightestInputBudget 是既有 targets 的派生计算,不是新配置面。

收工判定:代码正确、范围干净、回归安全;但分支落后 master 53 个提交且 CI 仍在跑,暂不可合并——需 rebase 到最新 master + rebase 后 CI 绿。按 [PR] 核验流程我未主动 rebase(无 CI 失败、无明确要求);owner 发话或 CI 因 rebase 相关原因变红,我再 rebase + push。

ci-check and others added 2 commits November 16, 2026 22:49
…6945)

⚠️ MODEL-FACING TEXT CHANGE: the all-candidates-failed refusal now appends
an [oversized: ...] diagnostic naming the offending range, its measured
input tokens, and a suggested split count. The existing refusal text is
unchanged; the diagnostic is additive.

The manual compress lane had no size sharding and no legacy fallback — one
oversized range exhausted every external-summary target and the refusal was
a generic 'exhausted' with no hint of which range to split. This adds the
minimum-viable diagnostics (proposal 3 of #6945):

- Extract the summary system prompt + JSON payload + input-token estimate
  into shared helpers (external-summary.ts) so the manual lane's
  pre-validation matches the HTTP guard's byte-for-byte.
- Add ConfiguredSummaryPlan.tightestInputBudget — the smallest
  (contextWindow - outputTokens - 256) across the chain.
- On an all-candidates-failed refusal, pre-validate each nominated range
  against the tightest budget and name the oversized ones with a suggested
  split count.

The HTTP guard's behavior is unchanged (byte-identical strings, same
budget formula) — only the refusal gains the diagnostic. Proposals 1
(sharding) and 2 (legacy fallback) of #6945 need owner decision and are
tracked separately.
The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for #4521
(schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now
lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying
request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round
mechanics under test are unchanged.

Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
@ranxianglei
ranxianglei force-pushed the 2026-11-16_compress-oversized-range-diagnostics branch from 98ea2e1 to 2e46b09 Compare November 16, 2026 14:52
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Update — CI came back red; I investigated the logs and rebased the branch.

The red was not this PR's code. I pulled the failure logs:

  • architecture-docs: FAIL: reference/architecture.md missing: src/responses-system-sections.ts. That file was added on master in 1494b55150 ([Bug] Inconsistent system prompt separator in Responses internal re-request breaks cache prefix | [Bug] Responses 内部重请求的系统提示分隔符不一致,破坏缓存前缀 #6922) — after this PR's base. The gate runs on the PR+master merge tree, so it saw the master file but the PR's stale reference/architecture.md didn't list it. Pure stale-base.
  • The 3 red test shards each had exactly 1 failing test, all in subsystems this PR does not touch (e.g. #2929: nudge + proxy-tool-round fold keeps re-request/next-turn byte-stable in tests/issue2929-codex-fold-double-miss.test.ts — the nudge/fold lane). This PR only modifies external-summary-*.ts; the nudge/fold behavior moved on master in the 53 commits since the PR's base, so the PR's old tree no longer satisfies those tests. Same stale-base signature.

So all 5 failures were stale-base (the PR was 53 commits behind master), not regressions from this change. I then rebased the branch to current master (301e1a5):

  • Clean rebase — master did not touch any of the PR's 5 files, so the diff is byte-identical (same 5 files, 104 insertions / 7 deletions).
  • New head: 2e46b09535. mergeable_state is now blocked (waiting on the fresh CI run), no longer behind.

CI is re-running on the rebased head. The code review in my previous comment stands — the change itself is correct and regression-safe. I'll confirm once the new run is green.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

CI failure is a master-level issue, not caused by this PR.

Investigated the failing required test shards on the rebased head (2e46b09). The failing test is #2929: nudge + proxy-tool-round fold keeps re-request/next-turn byte-stable (tests/issue2929-codex-fold-double-miss.test.ts) — the nudge/fold lane, which this PR does not touch (the PR only modifies the external-summary module).

Evidence this is a master-level issue:

  1. The same test shard (test (windows-latest, 22, 4)) is also failing on PR [PR] fix(side-request): arm mid-band output-budget collapse rescue on the pi plugin lane (#6955) #6962 (a different PR on the same master base 301e1a5).
  2. The test file is byte-identical between this PR's original base (de895aa) and current master (301e1a5).
  3. The nudge/fold source changed on master after this PR's base: 1494b55 ([Bug] Inconsistent system prompt separator in Responses internal re-request breaks cache prefix | [Bug] Responses 内部重请求的系统提示分隔符不一致,破坏缓存前缀 #6922) and 4ccccab (Per-turn '[ACP status: …]' line keeps ordering the model to fold leftover small ranges right after a big fold — the opposite of what the compress receipt says #6928).
  4. PRs [PR] fix(side-request): arm mid-band output-budget collapse rescue on the pi plugin lane (#6955) #6962, [PR] fix: take an advisor reply's context from its last call, not the summed usage #6961, [PR] fix: read Claude Code's deferred-tools list to name and reach bili's compress tool #6960 are all in blocked merge state on the same master base — the failure is fleet-wide, not PR-specific.

The architecture-docs failure (missing src/responses-system-sections.ts in reference/architecture.md) is also a master-level gap (the file was added in #6922 without updating the docs). It is not a required check and does not block this PR.

Conclusion: This PR's code is clean and the CI failure is inherited from master. The PR is ready to merge once the master-level nudge/fold test issue is resolved. No further changes needed on this PR's branch.

ework-agent and others added 2 commits November 17, 2026 01:36
The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke
every time the nudge gate's schedule shifted. The last recalibration
(245ecb1) was measured on a direct local run (timestamps ON by default
since #4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0"
(timestamps OFF) — a different wire shape, a different schedule (no nudge at
index 5) — so it passed locally and failed on every CI shard.

Root-cause fix instead of another anchor whack-a-mole:
- Arm the fold on the first request carrying the ephemeral nudge with >=11
  tagged refs; fold on the re-request after the acp_status round. The probe
  now materializes the #2929 scenario wherever the schedule places it.
- Pin compress.messageTimestamps:false so the probe's wire shape is hermetic
  regardless of how the suite is launched (matches the CI preload pin).

Supersedes the 245ecb1 recalibration. Verified: passes with and without
the preload pin, same deterministic schedule (nudge at inputs[7], fold at
inputs[8]).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI reds fixed — all 4 were inherited master failures, none from this PR's diff.

Live re-verification first (mirror cache, run 43890454281 on head 2e46b095): 4 failing checks — test (ubuntu-latest, 24, 3), test (windows-latest, 22, 4), test (ubuntu-latest, 22, 3), architecture-docs.

Job-log forensics:

Action (per CI-failure coordination policy): git merge origin/ci-fixes into the PR branch — clean merge, no conflicts — then targeted local verification on the merged head: architecture-docs guard green, #2929 test green, this PR's own external-summary suites 28/28 green. Pushed final head 2dc86d0377 (unmarked — the branch's one full CI pass). One-line status posted on #4599. No PR content was touched; no design decisions needed.


结构化八问

  1. 是什么问题:PR 分支 CI 4 项红(3 个 test 分片 + architecture-docs),均为继承性失败,与 PR 自身改动无关。
  2. 什么原因导致的:master 自身 born-red——[PR] fix(responses): share developer segment concatenation between compressed re-request and steady state, fix prefix re-bill | fix(responses): 压缩重请求与稳态共用 developer 段拼接,修复每次折叠后的前缀重计费 (#6922) #6944 合入时漏补 reference/architecture.md 模块映射;codex 命中率问题分析 #2929 fold probe 对调度敏感,在 CI 时序下 pre-fold 请求未出现 ephemeral nudge。两处修复均已收敛在 ci-fixes 哨兵上。
  3. 影响范围与严重程度:仅 CI 门禁层面——每个 rebase 到 master 的 PR 都继承这 4 红,阻塞合入;不影响任何运行时行为。
  4. 怎么修复的:按协调政策 merge origin/ci-fixes 进 PR 分支(哨兵已含全部修复),不在本分支重复修 master 级失败;merge 后本地定向验证三个面全绿再推最终 head。
  5. 是否有回归问题:无。merge 只带入 master 的 docs 映射行 + codex 命中率问题分析 #2929 测试自身修订;本 PR 的 5 个文件一行未动,其测试 28/28 通过。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:修复完成,最终 head 2dc86d0377 已推送,等待 CI 全量确认。

The integration test asserted retryAt sat in a tight 30..95ms wall-clock
window after the 3rd failure, but a loaded CI runner can stall the event
loop between the arming and the read (2026-11-17: ~260ms, delta=-180ms),
and the 20ms recovery timer can advance the streak past 3 and re-arm
retryAt before the read. Assert the stall-proof invariant instead: the
failure that set failStreak=S armed retryAt = T_S + backoff(S) with
T_S <= now, so the arm is at most one full ladder step ahead of now, and
it is armed at all. The exact ladder values are pinned by the unit test.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI fix run — #3380 timing flake fixed, pushed as final head.

Re-verified the live check state first (mirror cache was stale, so one single GitHub API fetch of check-runs for head 2dc86d0377): exactly one failing check — test (ubuntu-latest, 24, 3) (job 133549111245) — everything else success/skipped (35 success / 12 skipped). The earlier inherited master reds (#2929 probe, architecture-docs map) are already green via the ci-fixes merge.

The failure: not ok 1421 - #3380 P2 integration: failures escalate on the ladder, then heal clean at tests/issue3380-dsh-rereg-selflock.test.ts:285 — retryAt sits on the 80ms ladder step (got -180ms).

Root cause (flake, not a PR regression): the test asserted retryAt sat in a tight 30..95ms wall-clock window after the 3rd failure armed it (retryAt = T_3 + 80ms). Two uncontrolled things break that window on a loaded CI runner: (1) the event loop stalled ~260ms between the arming and the read, so delta = 80 − 260 = −180; (2) the 20ms recovery timer keeps driving failures while the manifest is down, so the streak can advance past 3 and re-arm retryAt before the read. The PR diff (external-summary-*) is unrelated to dsh-native.ts — this is a pre-existing timing-sensitive test that flaked on this run.

Fix (test-only, no source change): replaced the tight window with the stall-proof invariant — the failure that set failStreak=S armed retryAt = T_S + backoff(S) with T_S <= now, so the arm is at most one full ladder step ahead of now (delta <= step, mathematically always true → no false-positive risk) and it is armed at all (retryAt > 0). The exact ladder values (20/40/80/5min-cap) remain pinned by the unit test in the same file.

Local verify: 5× clean, plus 3× under simulated CPU load (loadavg 3.09 on 7 cores) — all pass. Pushed b79811022d unmarked as the final head (full CI pass running). One-line status posted to #4599.


八问

  1. 是什么问题:Plugin tool registration self-lock: after a failed proxy respawn, re-registration is blocked by its own prior registrations — identity headers lost forever (wire mode) → 413 preflight_compress_failed on large sessions | [Bug] 插件工具注册状态自锁:proxy respawn 失败后重注册被自己挡死,永久失去身份头(wire 模式)→ 大会话 413 preflight_compress_failed #3380 集成测试在负载较高的 CI runner 上偶发失败——对退避定时器武装时刻的墙钟断言(got -180ms)超出预期窗口。
  2. 什么原因导致的:测试断言 retryAt 落在第 3 次失败后 30..95ms 的狭窄墙钟窗口内;但 (a) 加载的 runner 在「武装」与「读取」之间让事件循环停顿了约 260ms,(b) 20ms 恢复定时器在 manifest 持续失败期间会推动 streak 超过 3 并重新武装 retryAt。二者都不可控。
  3. 影响范围与严重程度:仅 CI 测试偶发(timing-sensitive 测试在负载 runner 上 flake),非产品回归、无用户影响。严重度:CI 红(阻塞本 PR),无外部影响。
  4. 怎么修复的:把狭窄墙钟断言换成抗停顿的不变量——设 failStreak=S 的那次失败武装了 retryAt = T_S + backoff(S) 且 T_S <= now,故武装值至多领先当前一个完整阶梯步(delta <= step,恒真、无误报风险),且确实已武装(retryAt > 0)。精确阶梯值仍由同文件单测钉死。
  5. 是否有回归问题:无——纯测试改动(未动源码);新断言数学上恒真(无误报风险);heal 阶段逻辑未变。
  6. 是否破坏缓存:不涉及——未触碰缓存/重试游标/去重键等失效策略。
  7. 是否改了设计方向:不涉及——无架构/边界/数据流变化。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

判定:本 PR 唯一残留红已修复并推送(b79811022d,未标记 = 全量 CI 一次通过);等待 CI 结果,若全绿即可合并。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working external-summary p2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant