Skip to content

context window problem continues #987

Closed

Description

@walsykerem
Image Image Image Image Image

like Context (session accounting, host footer scale): 103% (34k / 33k)
Context (session accounting, host footer scale): 0% (0 / 250k)
Context (session accounting, host footer scale): 14% (35k / 250k)
Context (session accounting, host footer scale): 113% (41k / 36k) The context window size keeps changing—starting with the value I initially selected (I tried various figures like 50,000, 250,000, and 1,000,000)—but then it automatically sets the limit based on the context size generated at that moment.So, while the context window I initially set (250k) is active, something happens where it essentially decides—on its own—that your limit is, say, 18k if you happen to be at 18k usage at that moment.

like:⚠ Retrying in 4s · attempt 7 · context ~18231 tokens exceeds the model window 16898 (model=subagent), with 1 compressible range(s) still visible and preflight compression could not bring it under: no range could be compressed across 1 viable range (each was below minCompressRange, had an unusable summary, or failed to apply) Last unusable summary: the upstream returned a non-SSE body with no summary text (first 200 bytes: {"id":"chatcmpl-1789806312367","object":"chat.completion","created":1789806312,"model":"muse-spark-1.3-contributor","choices":[{"index":0,"message":{"role":"assistant","content":""},"finish_reason":"s).. The over-window payload was NOT forwarded. Preflight will not call the upstream again for the next 5m for this identical request. Change the request or wait for the cooldown before retrying. ım using 9router+opencode2 .

By the way, you can get support for OpenCode v2 from "rorshopping/opencode-acp-v2" this repo; it just has a few major issues—for instance, the plan agent never stops when you press Esc and keeps consuming tokens in the background—but aside from that, feel free to pull features.

Activity

  1. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 Thanks for the detailed report — I've reproduced the mechanics of both symptoms from the code and am fixing what's fixable on the proxy side. Quick status while I finish:

    Why the window "keeps changing" — this is bili's overflow self-heal, misfiring on your setup. Your subagent requests carry model=subagent (a 9router alias). No window source matches that name (your configured 250k is keyed to real model names), so bili starts with a generic default. The upstream (muse-spark-1.3-contributor behind the alias) rejects the oversized payload with a real context-overflow error — I verified the detector only fires on genuine overflow markers, not random 400s. bili then learns a conservative window from the rejected payload size (~18k → your 16898) and pins it for all subagent requests in that session ("self-healed context window"). Since every later turn is bigger than that learned number, you see the limit "chasing" your usage. The retraction path only triggers after a successful turn above the learned value — which can't happen while every turn fails, so it deadlocks.

    Why compression couldn't save it — found two concrete bugs in the preflight summarization path, which I'm fixing now:

    1. The summary call requests max_tokens=32768 by default regardless of the model's window. On a ~17k-window model, input+output exceeds the window, and upstreams like yours answer with an empty completion instead of an error → every summary attempt is unusable → preflight gives up. Fix: clamp the summary call's max_tokens to the window headroom.
    2. The diagnostic for that case says "non-SSE body", but the body was valid JSON — misleading. Fix: name it precisely (empty completion + finish_reason + the model id the upstream answered as, which exposes the alias mismatch).

    The deeper gap (alias model names never get an authoritative window — the response echoes muse-spark-1.3-contributor but the request said subagent) needs a design decision from you, so I'll lay out options in my next comment rather than guess. In the meantime a workaround: set limit.context for the subagent model entry in your opencode config (the v2 plugin stamps it per-request when present).

    PR coming shortly.

  2. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 Follow-up with the fix — PR #988: #988

    What the PR fixes (the part still broken on current master):

    1. Compression dead-end. Preflight always asked the upstream for max_tokens = 32768 on summary calls, regardless of the model's window. Your upstream (muse-spark behind 9router) enforces input + output ≤ window, so it answered those calls with an empty completion (content: "") instead of an error — every summary attempt came back unusable, "no range could be compressed", and the retry loop never made progress. The output request is now clamped to the window headroom (window − input estimate, floor 64). Ample/unknown windows behave byte-identically to before.
    2. Misleading diagnosis. The "upstream returned a non-SSE body" message was wrong — your body was valid JSON, just an empty completion. It now reports precisely: shape, finish_reason, and the model id the upstream answered as. That last bit is what exposed the real oddity in your setup: you send model=subagent, the upstream answers as muse-spark-1.3-contributor.

    Important update on the "window keeps changing" half: while rebase-testing I found this was already fixed on master today by PR #976 (#969): a rejected payload size is no longer learned as a window (that's exactly how you got 16898 ≈ your usage at that moment), only windows the upstream explicitly states are trusted, and a window you declare yourself now beats any learned value forever. You're running 0.1.120 (visible in your panel screenshots) — this lands in the next release, so upgrading will stop the limit-chasing on its own. Until then (and as belt-and-braces afterwards): declare the window for that model — set limit.context on the subagent's model entry in your opencode config (the v2 plugin stamps it per-request) or add it under the route's models.<name>.context in bili config.

    One remaining design question for @walsycommunity: the root gap is that model=subagent is a 9router alias matching no authoritative window source. Options: (a) rely on client-stamped/route-configured windows as above — works today, zero code risk; (b) have bili learn an alias→real-model mapping from response echoes (subagent → muse-spark-1.3-contributor) and look up windows under the real name — nicer long-term but touches identity/window plumbing, so I'd rather get your sign-off before building it. Tell me which way you'd like it to go.

    Thanks also for the pointer to rorshopping/opencode-acp-v2 — bili already ships its own native OpenCode v2 plugin (src/agent/opencode-v2.ts, which is what stamps those headers), so nothing needs pulling from there; the plan-agent-doesn't-stop-on-Esc bug is on the opencode side.

    中文摘要:已提 PR #988——修复预检压缩在小型窗口下因 max_tokens=32768 超出窗口、上游返回空补全导致的压缩死锁(现按窗口余量收紧输出上限),并把误导性的"非 SSE 响应体"诊断改为精确的空补全报告(含 finish_reason 与上游实际应答的模型名);另外"窗口追着用量变"的问题已被同日合入主线的 #976/#969 修复(你装的 0.1.120 早于该修复,下个版本升级即可,期间可在配置中为该模型显式声明窗口)。可以合并。

  3. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    上下文大小变化是不好的设计 是哪一次引入的?直接把这个移除吧

  4. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 查了 git 历史,"运行时改窗口"这套机制的引入链如下:

    按你的指示,我直接把整套移除(并入 PR #988):不再从任何溢出事件学习或修改窗口,有效窗口完全由声明决定(配置 / 客户端上报 / beta 协商)。溢出发生时仍会触发一次性的紧急压缩(下一轮自动压到失败体积以下),但窗口数字本身永不再变。取舍说明:对没声明窗口且真实窗口小于兜底值的模型,bili 不再自动纠正——需要你在配置里显式声明(models.<id>.context),首次溢出的错误会原样透传给客户端,随后一轮自动压缩恢复。

    改动面涉及 server.ts、weak-overflow.ts、side-request.ts 和一批相关测试,跑绿后我再回复并更新 PR。

  5. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    Removal PR: #992

    The window is now purely a deployment property — declarations only. The learned-window channel (confirmedContextLimit(s), retraction guards, weak-overflow ladder) is deleted wholesale (~1,700 lines); the one-shot emergency shrink survives, armed from the upstream rejection itself (stated window, or min(declared, payload-estimate) when no number is parseable) so #496/#488 recovery behavior is unchanged.

  6. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 已按你的指示把"运行时改窗口"的机制整个移除了 → PR #988(commit 932ffc6,已推分支):

    移除内容

    引入历史(回答"哪一次引入的"):#172 引入向下自愈 → #393 变双向 → #281 加入按载荷大小猜测 → #504/#572 弱学习+撤回 → 今天 #976(issue #969)先移除了猜测部分,本 PR 清掉其余全部。

    保留的机制:一次性紧急收缩(真实溢出后,下一轮预检把载荷折到失败尺寸以下再发)+ 纯声明式解析链(插件头 → runtime-info → launcher → 路由配置 → registry → 内置表 → 默认值)。窗口从此只由声明决定,运行期不动。

    代价提醒:小窗口模型(比如 subagent 这类别名路由)现在必须显式声明——bili 配置里加 models.<id>.context,或 opencode 侧配 limit.context(v2 插件会打成 x-bili-plugin-context-window 头);不声明就走默认值,靠溢出后的紧急收缩兜底。

    顺带修了原报告里的另一个死锁(commit 0e3ff5a,同 PR):预检摘要请求固定带 max_tokens=32768,对窗口只有 ~17k 的上游直接超限,muse-spark 就返回空补全(content:"")→ 每次摘要都不可用 → "no range could be compressed" 重试循环。现在按窗口余量收紧输出上限;诊断信息也精确化了——会显示上游实际应答的模型名,能直接看出 subagent ↔ muse-spark-1.3-contributor 的别名错位。

    验证:typecheck ✅ · 全量测试 1987 pass / 0 fail / 2 skip(e2e 门控)✅ · build ✅

    PR: #988 — 等人工合并。

  7. ranxianglei commented on Sep 19, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    [bot] 🏷 收到你的 #992 —— 移除这半以它为准:更彻底(整个 weak-overflow.ts、session 里的 confirmedContextLimit(s) 字段、side-request guard 的 learned 参数全删,~1,700 行)。我此前在 #988 里的那个移除 commit(932ffc6)已经撤下,避免重复劳动。

    #988 已收窄为只剩与 #992 零文件重叠的正交修复(针对原报告里的另一个死锁):

    • 预检摘要请求的 max_tokens 现在按窗口余量收紧。之前固定发 max_tokens=32768;像 muse-spark(经 9router)这类执行 input+output ≤ window 的上游,对这种请求不报错而是返回空补全(content:"")——于是每次摘要尝试都不可用 → "no range could be compressed" → 客户端重试循环。现收紧为 window − input 估算(下限 64,留 256 token 脚手架余量),余量充足或未知时行为不变。
    • 诊断精确化:diagnoseEmptySummary 不再把纯 JSON 空补全报成 "non-SSE body",会带出 finish_reason/stop_reason/status 和上游实际应答的模型名——你的 case 里就会直接显示请求是 subagent、上游应答的是 muse-spark-1.3-contributor,别名错位一眼可见。

    状态:rebase 到当前 master(ea2b33f),head df840a6;typecheck ✅ · 全量测试 2000 pass / 0 fail / 2 skip ✅ · build ✅。

    两个 PR 无依赖、任意顺序可合并;#992 合并后本修复仍然必要——窗口变成纯声明式之后,一个正确声明的小窗口(比如 models.subagent.context)如果没有这个 clamp,预检照样会被 32k 的 max_tokens 请求打死锁。

    🔗 #988 (等你人工合并)

  8. added 2 commits that reference this issue on Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions