Repository navigation
[PR] fix(#4516): degraded direct fallback — add the missing oversize refusal, self-heal retry, and transition logging - #4546
Conversation
…ersize refusal, self-heal retry, transition logging The gateless native lanes (pi/omp/opencode-V1) had none of the safeguards their siblings carry when the proxy goes away: a provably-oversized raw history rode straight to the provider (1.06M tokens vs a 500K window in the field report), a failed bootstrap meant direct-for-process-lifetime, and the only trace was a single warn line. All three contracts now live in the shared intercept so every lane gets them: - size arm on directSendOrRefuse: the doom math is hoisted out of dsh into the intercept (media-aware two-stage pricing #4111-direction, #4079 floored bar, fail-open on missing data); provably-oversized bodies get a local 422 (x-bili-gate: oversize-refusal-fallback, retryable, wire-shaped per dialect) instead of a guaranteed upstream rejection. - self-heal: a request finding no live origin fires the owner's singleFlight respawn under a 15s cooldown and then holds out for the in-flight attempt under state.spawnWaitMs ?? readyTimeoutMs ?? 15s — the pi spawn lane sets SPAWN_BUDGET_MS+10s so a slow cold start (#4515) is waited out instead of refused at the generic cap. Recovery needs no host restart. - announce: every proxy/direct/refused TRANSITION logs exactly one line (the old one-shot warned flag is gone); state.lastUnavailableReason carries bootstrap-failed vs dead into the refusal wording. dsh keeps its gate-facing signature (a wrapper decodes init.body); the hoisted functions are re-exported so existing importers and tests are untouched. TECHNICAL-NOTES.md gains the degraded-fallback contract section. Behavior changes vs past releases: 1. Gateless lanes now REFUSE locally (422) what provably cannot fit the window while degraded — previously such bodies were sent direct and died at the provider. Bodies that fit, or where the window/body is unknown, still go direct exactly as before. 2. A failed bootstrap is retried per request under a 15s cooldown instead of staying down until the host process restarts. 3. The pi spawn lane waits SPAWN_BUDGET_MS+10s (70s) for readiness on the retry path instead of the generic 15s cap.
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 open+closed 全量按症状/机制检索(direct fallback / oversize refusal / respawn self-heal / transition logging):只有本 PR 引用 #4516,无并行工作。相邻但不重叠:#4539(修 #4515 boot 侧——pre-listen 语料全量扫描撞 SPAWN_BUDGET_MS,即「spawn 为什么死」)与本 PR(#4516 fallback 侧——「死了之后怎么办」)互补,合起来才是该现场事故的完整闭环;#4158/#4121(opencode direct-refusal 的 media 排除)是本 PR doom math 的上游来源,不是重复。 标签: bug, pi, scope/wiring, p2(与源 issue #4516 triage 对齐:缺陷类、pi 车道、协议接线域;p2 = 特定条件触发[proxy 不可用 × 历史超窗]且有绕行,非核心路径停摆)。注:本条 [PR] 镜像条目在 Gitea labels API 上返回 404(仅 状态说明:这是 agent-authored PR(正文自报 vllm-qwen/qwen3.8-27b),按 [PR] 流程审核中——已拉分支逐段读完 diff 并对照 master 验证关键机制。一个需要 owner 知情的现状:分支当前 |
11538bf to
bc50f1a
Compare
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.5995-pr.4546.2560f04This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4546.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 冲突已解决并 rebase 到最新 master( 1. 是什么问题 2. 什么原因导致的
3. 影响范围与严重程度
4. 怎么修复的
5. 是否有回归问题 6. 是否破坏缓存 7. 是否改了设计方向 8. 是否引入了新的配置 收工判定:分支已就绪可审,等 CI 全量结果;若 owner 认为「give-up 后也允许 self-heal」这个广度需要收窄(仅限 bootstrap 失败场景),那是 PR 自身逻辑的一行级调整,可在 review 中拍板。 |
…send Review nit from the #4546 review: the recovery-retry path built `restamped` but dispatched with `stamped.init` (and read outgoing headers off `stamped.init`). `withHeaders` returns an `.init` that is a pure function of only (`init`, `extra`) - never of its first argument - and both calls pass the identical `init` + `routedExtra`, so `stamped.init` == `restamped.init` in every branch. Provably no behavior change; this just makes both dispatch arguments come from the same `restamped` binding for readability.
(ai: vllm-qwen/qwen3.8-27b) #4546 review 完成。结论:代码扎实、无阻断项,建议新 CI 波次转绿后合并。 一个行为决策项留给 owner(不影响可合并性)。 我顺手做的(小问题直接修): 结构化八问:
CI 与可合并性:
判定:无阻断项,建议新 CI 波次转绿后由 owner 合并。 唯一决策项(自愈范围是否收窄到仅 bootstrap-fail)可在合并后定,不影响可合并性。我不关闭、不合并,留待 owner。 |
…ment #4316 PR3/3 (restampRefsToLedger) re-homes citation numbers once on every full-mode cold load; goldens pinning exact ref numbers are born-red. Re-pin the 7 storage-lane tests (engine-proofs x6 + orphan-recovery x1) modulo that renumbering — zero production code touched. - golden-storage-drive.ts: per-turn citation-normalized digest (shaCiteNorm) and state digest modulo stripOneTimeAlignment (stateDigestAligned); the deep-sort helper is hoisted into golden-canonical.ts for sharing. - corrupt-index-child.ts: new stripOneTimeAlignment normalizer — strips the refLedgerAligned stamp and re-homes refs onto canonical raw-id order so a clean full-mode read and the legacy fallback load compare equal. - storage-engine-proofs.test.ts: wire legs assert shaCiteNorm equality vs the legacy restart leg and the committed goldens; state legs assert stateDigestAligned; rollback leg now spells BILI_STORAGE_UNIFIED=legacy explicitly (#3818 flip: absent resolves to full, deleting the env would vacate the legacy path). - storage-orphan-recovery.test.ts: pins the content-addressable byRaw->byRef round-trip instead of a counter-era m00003 literal. - golden/storage-render/*.json regenerated with shaCiteNorm/stateDigestAligned. Converges the master-red storage-lane family (#4599 plan A, owner directive).
…dger parity masks The #4316 one-time renumbering restamped messageRefs but left state.tokenSnapshot keyed by the OLD refs, so post-restart bodies carried stale per-message token counts (1.3K vs 1.4K) and the full-mode-restart wire drifted off the legacy baseline (#4599). persist.ts: two-pass remap (stage moves, delete old keys, then write new keys) so entries whose new ref collides with another move's old ref survive; fires only when superseded refs exist. Aligned normalizer (corrupt-index-child.ts): the cacheLedger's segment bookkeeping (be/bl/bs, unfinalized tail line), the derived agg block, and per-line miss-attribution (prov/seam) are restart experience, not content — no-restart and restart legs legitimately differ there. Masked in the aligned view only; raw digests keep full accounting. Goldens regenerated (stateDigest + aligned). storage-engine-proofs 12/12, golden-storage-render 5/5 locally.
…persisted state (#4599) The fork-survival test asserted immediately after the compress tool response, but the fold lands in the live session state asynchronously (#3573 replay semantics). Slow CI runners persisted the pre-fold state ('0 block(s) (compressions=0)' cold load) and the deepEqual diverged. Bounded 5s wait on the in-memory fold; fast machines exit on the first iteration. Product semantics unchanged.
The #4316 cold-load alignment reads the unified sidecar; its ingest of just-flushed records completes asynchronously. Slow runners compared the pre-ingest state and saw refLedgerAligned missing while the fold itself had landed. Poll (bounded 5s) until the loaded state carries the flag; fast machines exit on the first probe.
Git-for-Windows autocrlf checks the bundled .md posts out with CRLF; the exact-match '---' fences then failed to parse and every post lost its title/excerpt/date frontmatter (Windows CI: 'bundled meet-bili post found' assertion). Normalize CRLF to LF inside parseFrontmatter when the fence is CRLF-shaped; LF content is byte-identical as before. Regression test with a CRLF fixture.
… storage mode (#4599) The #4316 cold-load alignment sets refLedgerAligned only when the unified store is available; config-clean machines (CI runners) run legacy mode where it is an honest skip. The test asserted the flag unconditionally, so every shard carrying this file was born-red on CI while unified-config machines passed. Expectation now mirrors store.replicasAvailable(); verified 49/49 in BOTH modes locally.
…mspec wrap on win32 (#4599)
The ci-check lane's expectation change (c496cfb) exposed a real Windows bug: stripInheritedProxy builds inlineEnv as a spread copy of process.env, which keeps the original key casing (ComSpec) — Windows env vars are case-insensitive but plain objects are not. planClientSpawn's exact env.COMSPEC probe then missed it and silently degraded every wrapped spawn to the bare-name 'cmd.exe' fallback (verified by repro on a real win32 box: childEnv.COMSPEC=undefined while process.env.COMSPEC was set). Fix: probe COMSPEC/ComSpec/comspec in planClientSpawn and planDshSpawn. Verified on real Windows (Node 22): launcher-opencode-inline + fix-679 + dsh-refresh 37/37; host tsc clean, 35 pass / 2 win32-skips.
…OMSPEC case-probe c496cfb re-derived the win32 expectation from ambient process.env, which still mismatches production: Node's process.env member access is case-insensitive on Windows while the child env handed to the planner is a plain object whose key keeps the OS casing (ComSpec), so the exact env.COMSPEC probe fell through to the bare-name fallback. The assertion now derives from the child's OWN captured env/options and probes COMSPEC/ComSpec/comspec exactly like planClientSpawn does post-dfeea8a11 — self-contained under any concurrent-test env churn in single-process CI shards — and additionally pins windowsVerbatimArguments=true on the wrap.
… headerless fast path, respawn rejection hardening (#4546)
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.845-master.c0e60cb # this exact build
npm install -g billion-context@master # always the newest merged build
|
(ai: vllm-qwen/qwen3.8-27b)
Closes #4516
What this changes
The gateless native lanes (pi/omp/opencode-V1) had none of the safeguards their siblings carry when the proxy goes away. The field report: a 1,063,850-token raw history (window 500,000) rode straight to the provider after a failed cold start, every retry 502'd, and the only trace was one warn line in bili.log. All three contracts now live in the SHARED intercept (
src/agent/native-intercept.ts) so every lane gets them:directSendOrRefusenow prices every body before a degraded send, using the doom math hoisted out of dsh (media-aware two-stage pricing, DSH's unattributed over-limit gate prices base64 images as text (bytes/4) — media-heavy requests are falsely rejected wi | dsh 无归属超限门把 base64 图片当文本计价(bytes/4)——媒体重的请求被假拒 422,上游本会正常接受 #4111 direction; Context Window Overflow Issue Summary: Raw Requests Sent Directly to Upstream Rejected + Local Intercept Cooldown Lockin | 上下文超窗问题汇总:请求裸冲上游被拒 + 本地拦截冷却锁死会话(统一分析入口) #4079 floored bar; fail-open on missing data). Provably-oversized bodies get a local 422 (x-bili-gate: oversize-refusal-fallback, retryable, wire-shaped per dialect) instead of a guaranteed upstream rejection.respawnunder a 15s cooldown, then holds out for the in-flight attempt understate.spawnWaitMs ?? readyTimeoutMs ?? 15_000. The pi spawn lane setsSPAWN_BUDGET_MS + 10s, so a slow cold start (Boot full-scan of historical corpus: 3.9 GB store cold start at 34 s+ → hits the 60 s hard budget on native spawn → inev | 启动 boot 全量扫描历史语料:3.9GB store 冷启动 34s+,撞上 native spawn 的 60s 硬预算 → 必然降级直连(随后以未压缩历史撞上游窗口) #4515) is waited out instead of refused at the generic cap. A failed bootstrap no longer means direct-for-process-lifetime; recovery needs no host restart.warnedflag is gone);state.lastUnavailableReasoncarriesbootstrap-failedvsdeadinto the refusal wording so users can tell "still starting" from "died".dsh keeps its gate-facing signature (a wrapper decodes
init.body); the hoisted functions are re-exported so existing importers/tests are untouched.TECHNICAL-NOTES.mdgains the degraded-fallback contract section.Behavior changes (old → new)
prompt is too longreadyTimeoutMshad no assigner repo-wide) while spawn budget is 60sSPAWN_BUDGET_MS + 10s(70s) on the retry path — slow cold starts are waited out, not refusedNote on ①: the local bar never assumes a window below 1M tokens (#4079 policy, unchanged from dsh's gate): declared sub-1M windows are printed in the message but the floored bar governs, since declared windows can be wrong-small and an undersized byte guard false-blocks. Bodies between a sub-1M declared window and 1M are left to preflight, same as dsh today.
Tests
New
tests/issue4516-pi-direct-fallback.test.ts(11 tests, hermetic XDG_STATE_HOME like the selfheal suite): both wire shapes of the refusal, fits-passes-direct, no-header fail-open, media core-fits vs core-over (#4111 direction), self-heal recovery with transition logs, cooldown single-attempt under concurrency, slow-cold-start waited-out (pins the #4515 shape), once-per-transition logging, dead-vs-bootstrap wording split.Targeted local runs all green: new suite 11/11; native-intercept-selfheal + native-intercept + dsh-native + pi-native 136/136; remaining intercept-touching suites 241/241; typecheck clean. Full-suite verdict left to CI on this head.
Deferred (decision item, out of scope here)
Extending the panel attribution surface (
src/server/admin.ts/__bili/plugin/attribution) beyond dsh to pi/opencode/omp lanes — tracked as a follow-up decision per the issue discussion.