Skip to content

PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496

Closed

Description

@ranxianglei

Closing as fixed by PR #500 (forward-once-then-learn), merged and publishing in v0.1.82 (PR #510).

Summary of the fix: when image tokens make estimateCoreMessages + imageTokens exceed the window but the text-only estimate still fits, and there is no prior overflow evidence (lastInputTokens < limit && learnedLimit === undefined), preflight now forwards the request exactly once and lets the upstream arbitrate real billing; the relayed 400 (if any) is a strong signal that immediately learns the true window (per-model learnedContextLimits). Subsequent image-heavy requests fail fast without ever looping.

e2e coverage in tests/issue496-forward-once-learn.test.ts pins exactly ONE forward across 3 requests (no amplification loop) and the learned limit persisting.

Activity

  1. ranxianglei commented on Sep 3, 2026

    @ranxianglei
    OwnerAuthor

    🤖 ework agent · qwen3.8-27b

    [bot] 🏷 收到,已按 Triage 先审核再动手。

    核实(代码路径):preflightCompressIfNeeded(src/server.ts)里 payloadEstimate = estimateCoreMessages(...) + imageTokens,tokenCount = max(lastInputTokens, payloadEstimate)。一旦进入折叠,currentTokens = max(lastInputTokens, foldedText + imageFloor);而折叠只删文本、imageFloor 恒定 → 当 imageFloor ≥ limit 时该式恒 ≥ limit,循环耗尽 → 硬 502 preflight_compress_failed。确认成立,且是回归向量(master 不计数图片,同场景正常转发)。

    方案评估:同意 forward-once-then-learn,它打在根因上——把"纯估算驱动"的拦发改成"无实证时才放行一次交给上游裁决"。三种情形都收敛:tile 计费首转成功零变化;字节计费 relay 首转 400 → 现有 self-heal(含 rejectedImageTokens)学窗 → 之后 fail-fast,#488 的无限 400 循环仍被打破(严格优于 master)。

    实现要点(会写成测试固化):仅当三者同时满足才放行一次——① 折叠后文本已 fit(max(lastInputTokens, foldedText) < limit,即超窗唯一成分是图片);② lastInputTokens < limit;③ 该 model 无 learnedContextLimit。任一不满足维持现状 fail-fast。

    基于 PR #489 tip(f7deccc)开发,完成后附 PR 链接。

  2. ranxianglei commented on Sep 3, 2026

    @ranxianglei
    OwnerAuthor

    🤖 ework agent · qwen3.8-27b

    已完成,PR:#500(基于 PR #489 tip f7deccc 的 stacked PR,会把 #489 一并带入 master)。

    审核结论(动手前三步)

    1. 复现 / 查证 —— 问题成立
    确认了硬 502 的代码路径:当 imageFloor(图片估算)单项 ≥ 窗口时,preflight fit 门走的是硬 fail-fast(502 preflight_compress_failed,绝不转发),不是提前压缩。issue 里的场景(官方 Anthropic/OpenAI 贴 2 张 400KB JPEG、窗口 200K → 估算 ~267K ≥ 200K → 硬 502,真实成本仅 ~3K;同场景 master 正常转发)是真实可触发的,官方 API 多模态用户的回归向量成立。

    2. 层次判断
    PR review 里「高估只是提前触发压缩、方向安全」的论断,只在 imageFloor < 窗口 时成立;一旦图片估算单项撑爆窗口,性质就从「早压缩」变成「永不转发的误杀」。所以根因不在「估算偏高」本身,而在**「仅凭估算就硬拦」这个决策层**。改估算上限(比如默认加 cap)只治标,而且对字节计费 relay(#488)会反向低估——一个全局默认值无法同时适配两种计费模型。正确的层面是:让上游来裁决实际计费。

    3. 方案评估 —— 采纳 forward-once-then-learn
    实现为三条件门(全满足才放行一次):

    • 图片是唯一超窗成分(textEstimate < limit,文本折叠后已 fit);
    • 无上游实证(session.stats.lastInputTokens < limit 且该 model 无 learned limit);
    • 有图(imageTokens > 0)。

    满足则转发一次让上游裁决:tile 上游接受(usage 回报小值 → 零误杀、零行为变化);字节 relay 拒一次(400)→ 现有 self-heal 学窗(PR #489 已含 rejectedImageTokens)→ 后续请求带准确提示 fail-fast。恰好一次被拒转发,严格优于 master 的无限 400 loop,无需任何旋钮。 任一实证信号存在时,回落原有 fold / fail-fast 逻辑,行为不变。

    改动

    预检

    typecheck ✓;test 986/987(唯一失败是 launcher.test.ts 的 codex PATH 解析环境性失败,已在干净 base f7deccc 上复现同样失败,与本改动无关);相关测试 8/8;build ✓。

    未纳入(仍跟踪于本 issue,非阻塞)

    请 review #500。

  3. added a commit that references this issue on Sep 4, 2026
  4. ranxianglei commented on Sep 4, 2026

    @ranxianglei
    OwnerAuthor

    Closing — fixed by PR #500 (forward-once-then-learn), merged on master and publishing in v0.1.82.

    When images are the sole over-window component (text fits) and the session holds no upstream overflow evidence, the payload is forwarded ONCE so the upstream arbitrates billing: pixel-tile upstreams (official Anthropic/OpenAI) accept it at real cost; byte-counting relays reject → the existing self-heal learns the window from the rejected payload → subsequent requests fail fast. Pinned by an e2e test: exactly ONE forward across 3 requests (no #488 400-loop regression). Independently reviewed by two sessions (see PR thread).

  5. added 6 commits that reference this issue on Sep 5, 2026
    f78a63d
    bfe9e4c
    5783e03
    7bdcc73
    dce7502
    19f0fbb
  6. 5 remaining items

  7. added a commit that references this issue on Oct 1, 2026
  8. added a commit that references this issue on Oct 1, 2026
  9. added a commit that references this issue on Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingp2

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions