Skip to content

model window problem(I'm using opencode2 and 9router) #969

Closed

Description

@walsykerem

Error: context ~19344 tokens exceeds the model window 18320 (model=main), with 1 compressible range(s) still visible and preflight compression could not bring it under: no range could be compressed across 1 viable range (each was below minCompressRange, had an unusable summary, or failed to apply) Last unusable summary: the upstream returned a non-SSE body with no summary text (first 200 bytes: {"id":"chatcmpl-1789752825981","object":"chat.completion","created":1789752825,"model":"muse-spark-1.3-contributor","choices":[{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"s).. The over-window payload was NOT forwarded. Preflight will not call the upstream again for the next 5m for this identical request. Change the request or wait for the cooldown before retrying.

Error: context ~18985 tokens exceeds the model window 16161 (model=main), with 1 compressible range(s) still visible and preflight compression could not bring it under: no range could be compressed across 1 viable range (each was below minCompressRange, had an unusable summary, or failed to apply) Last unusable summary: the upstream returned a non-SSE body with no summary text (first 200 bytes: {"id":"chatcmpl-1789752921605","object":"chat.completion","created":1789752921,"model":"muse-spark-1.3-contributor","choices":[{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"s).. The over-window payload was NOT forwarded. Preflight will not call the upstream again for the next 5m for this identical request. Change the request or wait for the cooldown before retrying.

Activity

  1. ranxianglei commented on Sep 18, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    Traced this end-to-end against current master (v0.1.120). Short version: this may not be a real over-window situation — the window bili resolved for model=main (18320) is far below what this model family actually advertises, so preflight fires and then trips on an empty summary. Two layers, in order:

    What I verified (against src, not dist)

    • The message is assembled in src/server.ts:3020-3035 (failFast) from a preflightCompress result; the Last unusable summary: … non-SSE body … clause comes from src/preflight.ts:547 (diagnoseEmptySummary). Chain: context ≥ resolved window → preflight walks the 1 viable range → the summarization call returns a JSON body with empty content → unusable → fail-fast 502 + 5-min dead-end cooldown (src/preflight.ts:1006, src/server.ts:3161-3167).
    • The summary budget is already generous since preflight: raise summary output cap to 32k, clamp via models.dev #874: MAX_SUMMARY_OUTPUT_TOKENS = 32768, clamped down only if models.dev lists a smaller output cap (src/preflight.ts:41,274-276).
    • The smoking gun: your upstream model is muse-spark-1.3-contributor. The bundled models.dev snapshot lists the muse-spark-1.3 family at context = 1,048,576 (1M), output = 131,072. At ~19k tokens there is no overflow against a 1M window — yet bili resolved 18320. That means a source above the registry in the resolution chain overrode it. Order is anthropic-beta > plugin-report > launcher > configured(models.*.context) > registry > static-table (src/server.ts:1005-1010), so the registry's 1M never got a chance.
      • Caveat (honesty): if 9router's -contributor variant genuinely serves a much smaller window than the base family, then 18320 could be correct and the problem is purely the empty-summary path. Your log line below settles this.

    Layer judgment (root cause vs symptom)

    Why the remedy holds

    compress.modelContextLimit is the single highest-priority window source — it "outranks everything" incl. the plugin stamp, launcher, configured and registry (src/server.ts:997-998; CONFIGURATION.md §modelContextLimit). Pinning it to the model's true window removes the spurious trigger entirely; the ~19k payload then forwards without preflight.

    Do these now (unblocks immediately)

    1. Grab the decisive log line — it names the winning source. In ~/.local/state/billion-context/bili.log (Windows: %USERPROFILE%\.local\state\billion-context\bili.log) search [window] model=main. It prints source=… native=… effective=… launcher=… configured=… peek=… fallback=…. Also grep for weak overflow / learned conservative window (confirms a self-heal shrinkage).
    2. Pin the real window in your bili config (beats the opencode stamp): { "compress": { "modelContextLimit": <real window tokens> } }, then restart bili. (A per-model entry under the route also works, but the global pin is the sure thing here.) — or set the correct limit.context on the 9router model in opencode's config so the plugin stamps the right value.
    3. Start a fresh opencode session — the current one carries the stuck state + the 5-min dead-end cooldown keyed to that exact request.

    To pin the exact cause, please share

    • bili version (startup log line or bili --version);
    • the full [window] model=main … line;
    • any weak overflow … learned conservative window … lines;
    • the full [preflight] summary too short (…) : … lines untruncated — I want the complete finish_reason and whether a reasoning_content field is present;
    • how you declared the 9router model in opencode (its limit.context, if any).

    On duplicates

    Related to #853/#874 (same "preflight summary unusable" signature) but not a duplicate: #853 was finish_reason:"length" (thinking ate the shared output budget), fixed by raising the cap to 32k; yours is finish_reason:"stop" with empty content, and the window-resolution question wasn't in #853's scope. Keeping this open.

    Separate known gap (won't fix silently): the "non-SSE body with no summary text" wording actively misleads toward "upstream/MITM is broken". The accurate-diagnosis fix approved in #853 (floor 4) was dropped when #856 closed and never landed. Happy to ship that as a tiny follow-up PR referencing #853 + #969 if you want it.

    —— 中文一句话:这大概率不是真超窗——bili 给 model=main 解析出的窗口只有 18320,而 models.dev 里 muse-spark-1.3 系列标的是 1M,说明有更高优先级来源(opencode 的每模型 limit.context,或 #570 类自愈收缩)把窗口压小了,preflight 才误触发、又在空摘要(finish_reason=stop,与 #853 的 length 不同)上卡死;先用 compress.modelContextLimit 把真实窗口钉住并开新会话即可解困,日志里 [window] model=main 那一行能一锤定音。

  2. ranxianglei commented on Sep 18, 2026

    @ranxianglei
    Owner

    参考相关 native 几个 issue 另外你是用的启动器模式还是 url 还是 native

  3. ranxianglei commented on Sep 18, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    @Dog 问得关键——接法模式直接决定窗口从哪来,也就决定修法。三种我都在 src 里核过了:

    三种接法 → 各自的窗口来源

    模式 怎么接 窗口来源(代码位置)
    launcher bili opencode(bili 自己拉起 opencode) opencode 的 models.<id>.limit.context → BILI_LAUNCHER_MODEL_WINDOWS → launcherWindow(src/server.ts:1002)
    native opencode 里装 bili 原生插件(#809/#810 那种 plugins:["billion-context"]) 插件盖 x-bili-plugin-context-window ← opencode 该模型 limit.context(V1 extractV1Windows src/agent/opencode-native.ts:415;V2 catalog refreshWindows src/agent/opencode-v2.ts:135)→ pluginWindow,链上排第 1(仅 beta 之后)
    url 手动把 opencode 的 provider baseURL 指到 bili 的 /bili/ 端点 无任何盖章 → configured(models.*.context) ?? registry ?? table

    一个能缩范围的推论: 报错里请求模型是别名 main(上游 echo 出来的才是 muse-spark-1.3-contributor)。main 在 models.dev / 内置表里都查不到,而正常兜底是 ~100–200k——所以 18320 只可能来自 opencode 侧盖章(native/launcher),或一次自愈收缩。纯 url 且没在 bili 里配窗口的话,基本产不出 18320。⇒ 大概率是 native 或 launcher 模式,opencode 把这个 9router 模型的 limit.context 报成了 ~18320。

    参考的 native 相关 issue(按你的提示):

    对应修法:

    • native / launcher:把 opencode 里那个 9router 模型的 limit.context 改成真实窗口(盖出来的头就对了);或在 bili 配置里钉死 "compress": { "modelContextLimit": <真实窗口> } —— 这个来源优先级最高,压过插件盖章(src/server.ts:997-998:operator tuning outranks everything inside resolveRequestConfig)。
    • 若其实是自愈收缩(url 或学到的保守窗口):开新会话 + 同样用 modelContextLimit 钉住即可。

    @walsycommunity 麻烦确认三件事(定最终修法):

    1. 你是哪种接法?跑了 bili opencode(launcher)、还是把 baseURL 指向 /bili/(url)、还是装了 native 插件?
    2. 日志 ~/.local/state/billion-context/bili.log 里的 [window] model=main … 那一行(会打 source=? native=? effective=? launcher=? configured=? peek=? fallback=?)。看 source=:plugin=native、launcher=launcher、configured=你在 bili 配的;若这里 native 是大值、但实际生效却是 18320,那就是自愈收缩在下游压的。
    3. opencode 配置里那个 9router 模型有没有写 limit.context(写的多少)?

    —— 中文一句话:三种接法里,native/launcher 会把 opencode 声明的 limit.context 当权威窗口,url 则回落到 registry/表;因请求模型是别名 main、registry/表查不到且兜底偏大,18320 大概率是 opencode 把 9router 模型的 context 报小了——先确认接法 + 给那行 [window] 日志,再改 opencode 的 limit.context 或用 compress.modelContextLimit 钉住即可解困。

  4. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    link #955

  5. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    确认过 #955 和它的实现 PR #956,说下它俩跟 #969 的确切关系(含一个刚用 git 核到的状态):

    状态: #956(runtime-info 协议)已 merge 进 master(b6c2ef3 Merge pull request #956,在 v0.1.120=def3910 之上),但 v0.1.120 之后没有新的 release 提交 ⇒ 当前 npm 发布的构建里还没有它。@walsycommunity 若跑的是现发布版(v0.1.120),就还没有下面的可见性,要等下一个 release。

    #955 与 #969 的关系(核心):

    建议:

    想确认下你贴 #955 的意图:是要我把 #969 归到 #955 这条线跟踪(native 窗口来源可见性的用户案例),还是仅提示相关性?我按你说的办(不另开新 issue)。

    —— 中文一句话:#956 已入 master(b6c2ef3)但还没随 release 发布,所以现网 v0.1.120 尚无其"窗口来源可见性";#969 的 18320 本质是 opencode 经 native 插件上报的 limit.context,#955/#956 让它可查但不自动纠正,真正修法仍是客户端把该模型 context 报对或用 compress.modelContextLimit 钉住。

  6. added a commit that references this issue on Sep 19, 2026
  7. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    #976 ework你review这个改动

  8. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    Review of #976 — verified ✅ (note: it's already merged)

    First, a status correction worth stating plainly: #976 has already been merged into master — merge commit 1f463c8, fix commit 9703643; current master is ad0c73f (#975 sits on top). So this is a post-merge independent re-verification, not a pending gate. I did not re-open or merge anything (I never do merges). It is also not yet in a published release (latest published = v0.1.120) — reachable today via npm i -g billion-context@pr-976 or the next release.

    Verdict: approve — correct, complete, cleanly scoped, all gates green, genuine regression coverage. No rollback needed.

    What I checked (on master head ad0c73f)

    • Diff clean: single commit touching only src/server.ts, src/weak-overflow.ts + 4 test files (e2e-overflow-selfheal, issue496-forward-once-learn, plugin-truncation-weak-overflow, weak-overflow). No lock/generated/whitespace/unrelated churn. Not stale — its commit is on current master.
    • Gates green: typecheck clean · build OK · full suite 1916 pass / 0 fail / 2 skip (the 2 skips are the always-gated ACP_TEST_E2E codex e2e).
    • Regression tests present & passing — three dedicated #969 e2e cases mapping 1:1 to the contract:
      • overflow WITHOUT a window number → nothing learned (+ recovery without a learned window)
      • overflow with a STATED window → confirmed learning re-centers the limit and preflight recovers
      • an operator-declared window is never overridden by a confirmed learned window
      • plus #570 (failed-turn arms above true window don't retract) and #604 recovery.
    • Code walkthrough: both guess-persistence paths removed — forward()'s 400-without-number no longer writes the estimated payload size into confirmedContextLimits, and noteWeakOverflow no longer writes any learned* field; resolveSpeculativeLimit deleted and resolveLearnedLimit is confirmed-only; new operatorOwnsWindow guard (= compress.modelContextLimit set OR per-route models.<id>.context) blocks even a confirmed window from overriding an explicit operator declaration; the emergency shrink is retained as the unblock safety net. Grep confirms no dangling references to the deleted symbols.

    Why this is the right root cause

    Per the #976 analysis, the session shrank 18320 → 16161 across turns. A static opencode/native stamp would be constant; only the weak-overflow learner persists-and-shrinks. So removing guess-learning removes exactly the mechanism behind #969 (this matches my earlier triage hypothesis H2 — self-heal shrinkage).

    Notes (informational, non-blocking)

    —— 中文一句话:修了弱溢出学习把「猜测值」当窗口持久化、导致会话越缩越小(#969 的 18320→16161)的问题;改为只有上游在 overflow 里明确给出的窗口才允许学习,猜测一律不学、仅触发应急压缩,且不再覆盖操作员声明的窗口;该 PR 已并入 master(1f463c8),我独立复验 typecheck/build/全量测试(1916 过 0 挂)全绿、三条 #969 回归用例通过、diff 干净无无关改动,无需回滚。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions