Repository navigation
[PR] fix(overflow): bili's own oversize-gate refusal now triggers self-heal instead of a permanent 400 loop (#6122) - #6546
ranxianglei wants to merge 19 commits into
Conversation
…s a parseable overflow marker (#6122) The relay saw bili's OWN oversize refusal (bili_gate_oversize_refusal_*) come back through the client as a 400 and logged the #4452 CANNOT-engage warning on its own body — self-heal (same-request refold / emergency shrink) never armed and the oversized history was re-sent every turn (400 forever). The gate code is bili's definitive over-window proof: it now reads as an overflow with no learnable window (the 1,000,000 in the prose is a pricing floor), so the no-window arm (min of judging window and payload estimate) engages. The compaction refusal is a different class and stays dark.
…ine (#6122) A fork that inherited 1 of 5 active blocks logged only the adopted count — the 4 straddling blocks were invisible, forcing manual ledger archaeology in field triage. The inherited line now appends the straddled count when non-zero (the lineage-reconcile line already does, #2807); byte-identical when nothing straddles.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9947-pr.6546.f628cc9This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6546.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…e-overflow-recognition
…e default (#6480) The suite-wide baseline (tests/preload-storage-baseline.mjs) pins BILI_STORAGE_UNIFIED ??= "legacy" for the legacy-written corpus, but this suite asserts the fresh-install default posture — the exact #6480 pure-sqlite flip the storage-sqlite-default notice advertises. Under the CI preload chain the pin hid the notice and two subtests failed deterministically (red in every shard leg containing the file). Drop the inherited pin so the production default applies, per the preload's own pin-yourself contract.
The suite baseline preload (preload-storage-baseline.mjs) pins BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After #6480 flipped the storage default to pure-sqlite, the notice-registry suite's visibleNotices assertions expect the pure-sqlite default notice, which is gated on unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's "legacy" made the notice silent, so the suite was born-red on master. Pin the selection explicitly (#4011 lesson: a test that depends on the production default passes locally but fails in CI).
…-fixes form The branch's own duplicate fix (cfa8ee4: delete BILI_STORAGE_UNIFIED so the production default applies) and the sentinel's ci-fixes fix (c6369fb: pin BILI_STORAGE_UNIFIED="sqlite") both resolve the #6480 visibleNotices born-red and both satisfy unifiedSelectionIsSqlite. Per the CI coordination policy (2026-10-24) master-level fixes converge on ci-fixes, so keep the ci-fixes form and drop the branch-side duplicate — this also removes the future merge conflict with the ci-fixes->master PR. Verified: tests/notice-registry.test.ts 5/5 with the CI preload chain and 5/5 in a clean env.
(ai: vllm-qwen/qwen3.8-27b) CI fix run (attempt 1/3) — both reds diagnosed and resolved; branch pushed to Live re-verification first. The mirror cache was stale, so I did a single check-runs fetch for the old head Diagnosis (logs fetched for both jobs): every failure is What I did:
Status now: new CI run 八问(中文结构化结论):
判定:两个红均为继承性 Windows teardown flake(非本 PR 引入),已按政策 merge ci-fixes 并推送;新 run 排队中,若 flake 在新 runner 上复现,重跑失败 job 即绿(哨兵 #4599 floor 1324 既定处置)。 |
(ai: vllm-qwen/qwen3.8-27b) CI fix run (2nd wake) — same 2 reds re-verified, confirmed as the known master-level Windows EBUSY family; rerun triggered, then the run was externally cancelled; stopping here per fleet coordination. Live re-verification. Mirror cache was stale (frozen at 22:22Z), so a single check-runs fetch for head Log analysis (both jobs fetched): identical signature to the previous run — every failure is Fleet context (#4599, floors 1325–1344): this is no longer a one-PR flake — 4+ PRs (#6546, #4883, #6301, #6551, #5855, #4629) now share the identical signature, and a new mechanism claim is under the sentinel's verification (floor 1327: deterministic on Windows since What I did this run:
State of the PR: all 35 non-Windows-EBUSY checks green (all ubuntu shards, win22-2/3/5, every static gate, all e2e lanes, build artifact). The only remaining red is the 2 inherited Windows EBUSY shards — a master-level issue owned by the #4599 sentinel. Once the ci-fixes convergence lands, merging it into this branch (or a re-trigger) will close the PR. No code on this branch needs any further change from my side. 八问(中文结构化结论):
判定:本 PR 侧工作已完成(代码 + ci-fixes 收敛 + 验证全绿);剩余 2 红为 #4599 哨兵在管的 master 级 Windows EBUSY 问题,等其 ci-fixes 收敛落地后随重触发即绿。若 owner 刚才的取消是有意为之(例如准备直接落 ci-fixes 修复),本分支无需任何进一步动作。 |
…test The dsh plugin lane (ci-e2e-dsh.yml) runs node --import tsx --test WITHOUT the preload-storage-baseline.mjs preload, so BILI_STORAGE_UNIFIED is unset and the storage mode defaults to 'full'. Under the pure-sqlite default flip (5a80e2b, #6480), 'full' mode turns the JSON envelope writer OFF — so the .content-store.json sidecar is never written, and the test's readdirSync(namespace) hits ENOENT. Pin BILI_STORAGE_JSON_WRITER=on in the test so the content-store sidecar is written regardless of the storage mode. This keeps the corrupted-store detection path exercised in both full and legacy modes. Closes: dsh plugin lane red on master (run 43532980164, job 132255459726)
|
(ai: vllm-qwen/qwen3.8-27b) APPROVE — F4+F5 slice of #6122 is correct, well-tested, and green. What I verifiedCorrectness (F5 — the main fix).
Correctness (F4 — the log line). Tests. All three suites are well-written and would catch regressions:
Regressions. None. The overflow-pattern addition is purely additive (one new entry at the end of the array). The fork-adoption log line is additive (new clause only when CI. Green on the pushed head (0a565fa): 25 success, 3 cancelled (normal meta-cancels), 2 skipped (gated e2e lanes — expected). No failures. House conventions. No One minor scope note (non-blocking)The notice-registry test-isolation pin ( 8问审核清单
判定:可合。 修复正确、测试充分、CI 绿、无回归、无设计方向变更、无新配置。notice-registry 的 drive-by 测试隔离修复是唯一的范围瑕疵,非阻塞。 |
…er-r1 # Conflicts: # src/util.ts # tests/issue4452-markerless-reject.test.ts
The pure-sqlite default flip (5a80e2b, #6480) turns the JSON envelope writer OFF, so the .json session file this lane asserts on is never written and 'proxy must persist the session' / 'no persisted session carried compressed blocks' fail after the 20s wait. Pin the writer ON in the spawned client's env (the bootstrapped proxy inherits it) so the envelope this test was written for is still produced. Closes: native opencode lane red on master (run 43547593803, job 132302438610)
Same pure-sqlite default flip (5a80e2b, #6480) as the opencode lane: the JSON envelope writer is OFF, so 'proxy must persist the session within 20000ms' fails — the on-disk .json session file is never written. Pin the writer ON in the spawned dsh client's env (the bootstrapped proxy inherits it). Closes: native dsh lane red on master (run 43547497015, job 132302138843)
…e2e lane Same pure-sqlite default flip (5a80e2b, #6480) as the opencode/dsh lanes: the JSON envelope writer is OFF, so 'session persisted' fails — the on-disk .json session file is never written. Pin the writer ON in the spawned pi client's env (the bootstrapped proxy inherits it). Closes: pi codex-responses WS lane red on master (run 43547626291, job 132302540960)
(ai: vllm-qwen/qwen3.8-27b) Merge conflict resolved — pushed Master moved
Semantic compatibility check (the one that mattered): master's Local verification on the merged tree: targeted suites 58/58 green under the exact CI preload chain ( Live state after push: PR Two environment notes (not PR issues): the local Gitea mirror's git remote started redirecting to login this run, so fetch/push went directly to GitHub; the mirror's checks cache is also stale (fetched 02:47Z) — CI state above was confirmed via single direct API fetches. 八问
判定:冲突已全部解决并推送( |
…lane 4th lane of the pure-sqlite default flip (5a80e2b, #6480) family: the JSON envelope writer is OFF, so 'proxy must persist the session' and 'at least one persisted session must carry compressed blocks' fail — the on-disk .json session file is never written. Pin the writer ON in the spawned pi client's env (the bootstrapped proxy inherits it). Closes: pi package-native lane red on ci-fixes (run 43558179054, job 132336647955)
Same pure-sqlite default flip (5a80e2b, #6480) family: the JSON writer is OFF, so the per-session CCR content-store sidecar (.content-store.json) that 'child persists its own companion and survives a proxy restart' asserts on is never written (0 !== 1). The earlier b1bc17b pin only covered tests/plugin-fork.test.ts, not this file. Pin the writer ON at module load (before any SessionStore is constructed). Closes: dsh plugin lane red on ci-fixes (run 43558179102, job 132336519539)
|
Field report: this PR covers the symptom for unattributed traffic, but the lane still can never be learned I hit exactly the same 422 from a different entry point and spent a day on it, so here's my field data. This comment is not asking for anything in #6546 to change — the fix there is correct and I want it. I'm adding the part it doesn't reach, because after #6546 lands the request will stop 400-ing, but it will still never be compressed. The case I hit
Overshoot by 12,524 bytes. The two earlier requests show the real cost: ~7 MB re-sent uncompressed every turn, growing. Why attribution is structurally
|
| OS | Windows 11, MINGW64_NT-10.0-22631 |
| DSH | 0.2.1-rc.3 |
| billion-context | 0.1.213, commit 133f3525c |
| provider plugin | dsh-connect-workbuddy 5.21.3 (upstream copilot.tencent.com) |
| admin port | 127.0.0.1:18787 |
/__bili/stats |
blindTunnels.total: 0, unrecognizedPaths.total: 0 — bili sees the plaintext fine, this is purely the attribution gate |
Happy to produce a PR for the registration API if the shape works for you — I've already got the local reproduction and a small harness for it. 🙏
# Conflicts: # tests/e2e/e2e-native-dsh-persona.test.ts # tests/e2e/e2e-native-dsh.test.ts # tests/e2e/e2e-native-opencode.test.ts # tests/e2e/e2e-native-pi.test.ts # tests/e2e/e2e-pi-codex-ws.test.ts # tests/e2e/e2e-subagent-sessions.test.ts # tests/notice-registry.test.ts # tests/plugin-fork.test.ts
…atest master The sentinel confirmed the deterministic handle-leak hypothesis and landed the fix in ci-fixes: persist-zstd + persist-metadata-fold-gate now close their SQLite sidecars (closeQuietly) before the tmp-dir teardown, so Windows can unlink index.db. This is the exact family red on win22-5 (EBUSY unlink .../index.db, #5506 x3 + persist-zstd x10). Also brings master e5e1647 -> 9495c76 (release v0.1.214 + fixes) and the sentinel's storage-open-busy-lock tripwire (acc7944). Conflict in tests/notice-registry.test.ts resolved to ci-fixes's form (HEAD was byte-identical to the merge base; ci-fixes adds the BILI_STORAGE_JSON_WRITER unpin required by the dual-write baseline pins).
(ai: vllm-qwen/qwen3.8-27b) CI fix (win22-5) — resolved, pushed Live re-verification first. The mirror cache was stale (frozen at 04:17Z and didn't even list the test shards), so I fetched the failing job directly: Diagnosis + the sentinel's fix. This is the #4347 Windows-EBUSY family from the prior reds (floors 7/9). The #4599 sentinel has since confirmed the deterministic handle-leak hypothesis and landed the fix in What I did:
Status now: new CI run on head 八问(中文结构化结论):
判定:win22-5 红为继承性 Windows EBUSY teardown 问题(非本 PR 引入),已按政策 merge 哨兵 ci-fixes(含其确定性句柄泄漏修复)并推送;新 run 在 head 6b03788 上排队,EBUSY 分片将随修复转绿。本分支侧工作完成,无进一步代码改动。 On the field report (attribution bootstrap deadlock) Thanks — this is a well-researched report and the key claims check out against the code:
So the bootstrap deadlock you describe is real: with Two notes on routing:
Suggested next step: file an issue carrying your repro + harness (you offered the local reproduction), state the proposed shape, and let the owner decide on the API surface. Your second trap (byte-gate refusal driving a token-target refold that no-ops when threshold unit ≠ fold target unit) is a good independent sub-point to include there. I won't file it or build it myself — it's your proposal and it needs the owner's sign-off on the shape; happy to help route it if the owner wants. |
|
(ai: vllm-qwen/qwen3.8-27b) Re-review on new head 6b03788 — APPROVE still holds. The head moved from 0a565fa → 6b03788 via a ci-fixes merge (latest master + Windows EBUSY teardown fix). I verified the PR's own F4+F5 changes are byte-identical in the new head:
The 132-file / +5974 diff between the two heads is entirely from the ci-fixes merge (test-isolation pins, storage-writer pins, Windows teardown fix, release v0.1.214, etc.) — none of it touches the PR's F4+F5 logic. CI note: the ci-fixes merge re-triggered the full suite. As of this review, 14 checks are still in progress, 0 failures, 3 success, 13 skipped (gated e2e). No failures so far. The previous head (0a565fa) was fully green (25 success / 3 cancelled / 2 skipped). The PR's own changes are unchanged, so the green verdict carries over — but the new head's CI is still settling. Verdict: APPROVE (unchanged from the first review). The F4+F5 slice is correct, well-tested, and regression-free. The ci-fixes rebase introduces no changes to the PR's logic. 8问审核清单(增量 — 新 head 6b03788)
判定:可合(APPROVE 维持)。 PR 自身 F4+F5 变更在新 head 上逐字节验证不变,CI 重跑中无 failure。 |
[bot] 🏷 Re-review on new head 6b03788 — APPROVE still holds. Head moved from 0a565fa → 6b03788 via a ci-fixes merge (latest master + Windows EBUSY teardown fix). Verified the PR's own F4+F5 changes are byte-identical in the new head (pattern at CI: the ci-fixes merge re-triggered the full suite. As of this review, 14 checks in progress, 0 failures. The previous head was fully green. The PR's own changes are unchanged, so the green verdict carries over. Full re-review with 8问: #6546 (comment) |
What was broken
In the #6122 field case (DSH fork child inheriting a 1.13M-token parent history), one of the rejection bodies flying through the relay was bili's own oversize-gate refusal —
code: bili_gate_oversize_refusal_unattributedfromsrc/agent/native-intercept.ts. The relay treated it as an opaque gateway rewrite: no parseable overflow marker, the #4452CANNOT engagewarning printed on bili's own body, and the overflow self-heal (same-request refold / emergency shrink) never armed. Every retry re-sent the identical oversized history → 400 forever.This is not fork-specific: any oversized request that gets refused by the gate and surfaces back at the relay (fork children, oversized tool outputs, giant pastes) hits the same dead end.
Changes
src/util.ts—CONTEXT_OVERFLOW_PATTERNSgains/bili_gate_oversize_refusal/i, covering all five oversize variants (bare /_fallback/_no_live_state/_unattributed/_unattributed_bytes). The compaction refusal (bili_gate_dsh_compaction_refused) is a different class — no proxy available, body fits — and is deliberately not matched. No window number is learnable from the refusal prose ("1,000,000 tokens" is a pricing floor, not the model's window), so the no-window arm engages:min(judging window, payload estimate), then preflight folds under it and re-sends (使用pi 压缩后就提示prompt is too long #1195 same-request refold where available).src/fork-adoption.ts— the[resume-inheritance] … inherited …line now names the straddling active blocks when non-zero (, N active block(s) straddled the fork point (adopted once fully present, #6122)). The field triage had to rebuild the ledger by hand to see that 4 of 5 active blocks were inherited only implicitly; the lineage-reconcile line already reported this (fix(sessions): wire-neutral affinity hashing for cross-protocol continuity (#2791) #2807). Byte-identical when nothing straddles.Scope note
This is the F4 + F5 slice of #6122. F2/F3 (straddled-block coverage gate, parent-missing prefix fallback) are intentionally deferred: the DSH fork path (server side) is still in development, so those design decisions should be made against its final shape, not the current one.
Pre-flight
npm run typecheck— cleancontext-overflow+issue4452-markerless-reject+issue6122-fork-straddled-log→ 50/50;resume-miss-timestamp-seed+fork-adoption+fork-adoption-plan→ 20/20 (incl. the pre-# Cache hit collapses to 1–12% for a whole generation, then the very next request returns to 100% — repeated 15× in one day #5608 wording regression guard);native-intercept+ direct-fallback/window-guard trio → 116/116tests/derived-fork-block-inheritance.test.tshas one failure on pristine master in this environment (reproduced with the fix stashed) — unrelated, not introduced here