Skip to content

[PR] fix: #5907 colon-prefixed gateway model ids resolve their window; unknown-window dsh oversize arm falls back to the 1M floor - #5910

Merged
ranxianglei merged 6 commits into
masterfrom
2026-11-08_custom-provider-window-fallback
Nov 9, 2026
Merged

ranxianglei merged 6 commits into
masterfrom
2026-11-08_custom-provider-window-fallback

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Closes #5907

Two independent defects behind one field symptom (unattributed dsh chains stranded at a 167232-token window while a 1M model sits behind the gateway):

1. vendor/region:model ids miss the context-limit table (src/config.ts).
modelRoots only stripped the slash-prefixed provider segment, so liuyue/cn:deepseek-v4.1-flash yielded roots starting with the namespace prefix and matched no ^-anchored family rule → env default 2e5 − 32768 output reservation = 167232 for a 1M model. Every root now also yields its tail after the LAST colon as a candidate, tested strictly AFTER the un-stripped root (a tag-shaped suffix like gpt-5:o3-preview → o3-preview must not false-match an unrelated family rule before the real name is tried). Ids without a colon produce identical roots to before.

2. Unknown window disarmed the dsh gate's oversize refusal (src/agent/dsh-native.ts).
Until the host resolves model info, effectiveContextWindow(ctx) is undefined — which custom providers often never do. The shared predicate then returns undefined by construction and both gate arms fell through to a direct send: a provably oversized UNATTRIBUTED body rode straight into a guaranteed upstream rejection (field shape from the issue: 1.45M-token sends 400ing or hanging 15s per turn). Both arms now pass declared ?? MIN_ASSUMED_WINDOW_TOKENS, keeping the #4079 never-assumes-below-1M doctrine; the durable refusal line says "no window declared; assuming the 1M floor" instead of echoing a fake declaration.

Complementary to #5875/#5613 (merged earlier today), which added the calibrated estimate tier on top of the optimistic floor but kept missing-window fail-open by construction — this PR closes exactly that remaining hole. The shared predicate's contract (undefined on missing data) is unchanged; the fallback lives at the two dsh call sites only, so every other lane (omp/opencode direct-send guards, #5032's pending per-lane arming decisions) is untouched.

Behavior changes (old → new):

  • Model ids containing : now reach family rules via the colon tail (previously: table miss → env default 2e5). No-colon ids: identical behavior. Affects every lane using lookupContextLimit as its fallback tier, plus tierGatedStandardWindow (claude tier cap) consistently.
  • dsh takeover gate, attribution-failed requests only: previously an unknown window fully disarmed the oversize arm (direct send); now the bar is the 1M floor, so a body whose optimistic core-text token count exceeds 1M (or whose ×1.14 calibrated estimate does, per [Bug][dsh] Model requests silently bypass the compression proxy (fail-open gate) — 927K-token prompt sent raw, 1-token reply, session bricked with 400 #5613) is refused locally with 422 instead of being forwarded into a guaranteed upstream rejection. Under-floor bodies, attributed traffic, and all other lanes: unchanged.

Config surface: none — no new config fields, env vars, or defaults; the existing modelContextLimit escape hatch keeps working as before.

Tests (both directions per #3617):


Follow-up commits (2026-11-08): the two minor log-surface items from the same report, each its own labeled commit (added pre-merge per the repo standing rule for small direct fixes)

  • fix: name the expected duplicate-instance outcome in the terminal EADDRINUSE diag — the ALREADY-SERVING branch of the holder diagnostic now states explicitly that a duplicate-instance start is the expected outcome of a host relaunch and cross-references a nearby [split-brain] notice naming this exiting pid as the same race settling, not data corruption. Purely additive wording; the wedged/dead diagnostic branch is untouched.
  • fix: slow-session warn fires once per session with size picture and cleanup hint — behavior change (old → new): the [slow-session] warn fired on EVERY turn past the 1s wall budget; it now fires once per sessionId per process (bounded map, earliest-warned evicted past 512). The first occurrence gains (~N est tok) (bytes/4 optimistic floor, labeled est because JSON overhead skews it) and a bili sessions rm <id> cleanup command (verified existing CLI surface). The passthrough pseudo-session shares the once-per-key semantics.

Item-b direction (once-per-session over hard-cap silence, full size line) follows the affected reporter's choice recorded in the issue thread. Both items are log-surface only — no control-flow change.

Follow-up commits (2026-11-09): rebase + second defect facet of #5907 (turns that never reach ACP at all)

  • Rebased onto current master (82e2d53c6e); conflict with the [PR] release v0.1.213 #5898 review batch resolved as the union of both intents — master's locale-readable numbers in the gate refusal lines + this PR's unknown-window windowNote fallback. All prior commits re-verified on the new head (targeted suites 216/216, typecheck, build).
  • fix: name the no-guard fetch postures in bili.log; document the direct /bili/ fallback (#5907) — the reporter's second observation: some turns produce ZERO bili.log lines because the host-side globalThis.fetch takeover never saw them. Findings verified against the code: assignment-based displacement by third parties is NOT silent since 方式一安装的dsh 原生插件:llm-pi-ai 传输(OpenAI SDK + options.fetch 注入)绕过 globalThis.fetch 拦截,模型请求不进代理、压缩完全不生效 #1158 (logged + re-chained as downstream), so the silent class is the no-guard posture — non-configurable host property or BILI_RECLAIM_FETCH_PATCH=0. That path now logs a ONE-TIME warn naming the degraded posture, its consequence (silent un-routing, zero log lines) and the /bili/ remedy; previously it was silent at install time. Behavior change (old → new): two new possible warn lines in bili.log under those postures — additive logging only, no wire/config/control-flow change. Tests pin both branches (reversible env-hatch case + a last-in-file case that locks the slot to simulate the non-configurable host). Docs (en+zh technical-notes) gain a section documenting that /bili/{upstream-URL} is directly addressable without any hook — the supported robust fallback when a lane's hook misses turns (captured fetch refs, non-fetch transports, other realms bypass the patch by construction and are invisible to the proxy).

Follow-up commit (2026-11-09): third facet of #5907 — naming the unattributed class in the zero-progress fits-forward warn

@ranxianglei ranxianglei added bug Something isn't working dsh p1 labels Nov 7, 2026
@github-actions

github-actions Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-08_custom-provider-window-fallback (cee765f)

Latest build — install this exact version

npm install -g billion-context@0.0.8802-pr.5910.a16617c

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
cee765f billion-context@0.0.8802-pr.5910.a16617c build
7b09b2f billion-context@0.0.8675-pr.5910.772356d build
a3f01bc billion-context@0.0.8579-pr.5910.dd82bf0 build
665163e billion-context@0.0.8570-pr.5910.ef5b747 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5910.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: #5907 colon-prefixed gateway model ids resolve their window; unknown-window dsh oversize arm falls back to the 1M floor [PR] fix: #5907 colon-prefixed gateway model ids resolve their window; unknown-window dsh oversize arm falls back to the 1M floor Nov 7, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 这是修 #5907 的 PR(#5910),两个独立缺陷各一个 commit;open/closed 无同机制在途 PR。与今日已合的 #5875/#5613 是互补不是重复——那两个加了校准估算层但构造上保留了缺窗 fail-open,本 PR 正是补那个洞。

我独立核对了 diff(base=当前 master,merge-base==origin/master,无陈旧基线;干净 4 文件 +77/-15,两 commit),并逐条 trace 了逻辑、比对 CONTEXT_LIMIT_TABLE 实际值、读了共享谓词实现、确认作用域。结论如下(结构化八问):

1. 是什么问题:自定义 OpenAI 兼容网关里模型 id 形如 vendor/region:model(冒号带命名空间前缀)时,bili 窗口查表命中不了家族规则,落到 2e5 默认(有效窗 167232),把 1M 模型当 1/6 预算;叠加 dsh 接管闸门在「归因失败 + 宿主从未解析出窗口」时超窗保险从构造失效,1.45M token 请求被照发进必然的上游 400 / 每轮干等 15s。一个字段症状,两个独立缺陷。

2. 什么原因:(a) modelRoots() 只剥斜杠前 provider 段,冒号后命名空间前缀(cn:)留着,候选根都不以家族名开头,永远匹配不到 ^-锚定规则;(b) bodyOverWindowDetail() 首行 if (bodyBytes===undefined || declaredWindow===undefined) return undefined——窗口未知直接返回 undefined,两个 dsh 调用点拿到 undefined,超窗臂整体失活。

3. 影响范围与严重度:仅 dsh 车道「无归因 + 宿主未声明窗口」子集(自定义 provider 典型场景);有归因流量、已声明窗口请求、omp/opencode 等其它车道完全不受影响。受影响用户该链每轮要么上游 400、要么挂 15s,体验毁灭级(用了就想卸载)→ p1(PR 已打 p1,合理)。有绕行(配 modelContextLimit)。

4. 怎么修的:(a) modelRoots 对每个根再额外产出「最后一个冒号之后」的尾部作候选,严格排在未剥根之后测试(防 gpt-5:o3-preview→o3-preview 这类标签状后缀先误配无关家族);无冒号 id 产出与之前逐字节一致。(b) 两个 dsh 调用点改传 declared ?? MIN_ASSUMED_WINDOW_TOKENS(1M floor),守 #4079「绝不假设低于 1M」教义;durable 日志行区分「未声明窗口、按 1M floor 假设」与「已声明 X」,不再回显假声明。共享谓词契约(缺数据→undefined)不动,fallback 只落在 dsh 两处调用点。

5. 是否有回归:审过既有行为逐项——

  • 无冒号 id:我逐一 trace 了 slash/no-slash × colon/no-colon 四种形状,产出与旧版逐字节一致(routing 控制行 + 既有 routing 测试不变)。
  • 已声明窗口:dsh 两臂 x ?? y === x(x 有值时),与旧路径逐字节相同;有归因流量不受影响。
  • 未尺寸化/流式 body:谓词首行先查 bodyBytes===undefined,即便传了 floor 仍走 「BUG」native-intercept 进程级 globalThis.fetch 补丁按 URL 形状接管所有模型流量,无关调用者(如 MemOS)被卷入宿主压缩流程 #1117 direct(测试钉住)。
  • 唯一有意变更=未知窗口的无归因超窗 body 从「照发」变「本地 422」(Behavior changes 已披露 old→new+原因)。
  • 高危模式逐项:无语义漂移(fallback 只在 undefined 分支)、无 fail-fast 变吞错(方向是更保守的本地拒绝)、无阈值/时序/输出格式漂移(仅 durable 日志文案更诚实)。未发现无意回归。

6. 是否破坏缓存:不涉及。窗口估算影响压缩预算,不进缓存键/身份;modelRoots 改动只改变用于预算的窗口取值,不触碰前缀缓存或消息身份。

7. 是否改了设计方向:不涉及。沿用既有 fallback-tier 架构,只补未知窗口这个洞;反而刻意保持共享谓词「缺数据→undefined」契约不变、把 fallback 收敛到 dsh 两处调用点——边界更干净。

8. 是否引入新配置:不涉及。无新字段/env/默认值;既有 modelContextLimit 逃生口照旧。

非阻塞观察(不拦):同一「未知窗口失活超窗臂」的洞也存在于通用直连兜底车道(native-intercept.ts:1506,#4516,omp/opencode 等)。本 PR 有意只修 dsh,其余车道留给 #5032 的分车道 arming 决策——合理分轨,dsh 字段症状已由本 PR 闭环;记录在此供后续,不作本 PR 阻塞项。

CI(head 665163e)全绿:全部 test 分片(ubuntu/windows × node22/24)+ 全部 guard(config-surface/lock-sync/version/lane/kernel/pi-subagents/docs-sync/unused-exports/lifecycle-ledger/test-import-graph)+ mysql-contract 均 success,无 failure(cancelled/skipped 为不适用/被取代的矩阵腿)。

测试有效性(#3617 双向):routing 正例含 issue 现场 id(liuyue/cn:deepseek-v4.1-flash→1M)+ 控制行钉住所有旧形状不变 + 真未知 id 仍 undefined;我对每条断言比对了 CONTEXT_LIMIT_TABLE 实际值(^gpt-5400K / ^glm-51M / ^deepseek-(r1|v3|ocr)128K 排在 ^deepseek1M 之前,specific 胜 broad),断言有牙、能兜住回归。dsh 两臂各加「未知窗+确证超窗→本地拒(新日志文案)」正例 + 「floor 之下仍 direct」负例 + 既有 declared-window 行不动。

判定:可合。 需人工 merge——涉及 fallback/default 语义变更,按 §7.4 属人工门禁(非 auto-merge 范畴)。labels(bug/p1/dsh)与本 PR diff 相符,无需调整。

ci-check added 5 commits November 8, 2026 23:59
…mit table (#5907)

modelRoots only stripped the slash-prefixed provider segment, so gateway ids like 'liuyue/cn:deepseek-v4.1-flash' yielded roots that start with the namespace prefix and missed every ^-anchored family rule, falling to the 2e5 env default (167232 after output reservation) for a 1M model. Each root now also yields its tail after the last colon as a candidate, tested strictly after the un-stripped root so tag-shaped suffixes cannot hijack an unrelated family rule; ids without a colon produce identical roots.
…no window is declared (#5907)

effectiveContextWindow answers undefined until the host resolves model info, which custom providers often never do; bodyProvablyOverWindow then disarms by construction and a provably oversized unattributed body rode DIRECT into a guaranteed upstream rejection (field shape: 1.45M-token sends 400ing or hanging 15s per turn). Both gate arms now size the bar at max(declared ?? 1M floor), keeping the never-assumes-below-1M doctrine of #4079; the refusal log distinguishes 'no window declared; assuming the 1M floor' from a real declaration.
…DRINUSE diag (#5907 minor item a)

A host relaunch racing the still-up previous instance lands the loser's
EADDRINUSE error block ~40ms beside its [split-brain] notice, which reads
like a fatal error although the loser exits cleanly and the serving
instance keeps everything. The serving branch of holderHintFragment now
states the expectedness explicitly and cross-references a nearby
[split-brain] notice naming this exiting pid as the same duplicate start
settling — purely additive wording on the ALREADY-SERVING branch only; the
wedged/dead diagnostic is untouched.
…leanup hint (#5907 minor item b)

A 3k-msg session exceeds the 1s wall budget on EVERY turn, so the warn
flooded the log one line per turn while adding nothing after the first.
It now fires once per sessionId per process (bounded map, earliest-warned
evicted past 512); the first occurrence carries the full size picture —
msgs + inbound bytes + bytes/4 est tokens (the repo-wide optimistic floor,
labeled est) — plus the 'bili sessions rm <id>' cleanup command (verified
existing CLI surface, src/cli.ts). Behavior change: repeat slow turns of
the same session are silent; the 'passthrough' pseudo-session shares the
same once-per-key semantics.
@ranxianglei
ranxianglei force-pushed the 2026-11-08_custom-provider-window-fallback branch from a3f01bc to 7b09b2f Compare November 8, 2026 16:13
@ranxianglei
ranxianglei merged commit 8626aeb into master Nov 9, 2026
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p1

Projects

None yet

1 participant