Skip to content

fix: retry MALFORMED_RESPONSE on DSH built-in DeepSeek routes (#2605) - #2610

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-10_dsh-retry-malformed-response
Oct 10, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-10_dsh-retry-malformed-response

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2605

What & why

DSH's default retry whitelist (DEFAULT_RETRYABLE_CODES in packages/llm/llm/src/retry-policy.ts) omits MALFORMED_RESPONSE. The DeepSeek adapter finalizes every streamed tool call with a JSON.parse(content.arguments) check (llm-deepseek/src/translate.ts); when the model occasionally emits argument JSON with the opening quote of the first value missing (a sampling flake — the issue shows 17/18 calls legal, longest 10,957 chars fine), that validation raises MALFORMED_RESPONSE, which is not in the whitelist ⇒ zero retries ⇒ the whole turn dies and the answer is never delivered.

Fix

Both bili-owned DSH patch carriers now add MALFORMED_RESPONSE to the built-in DeepSeek routes' retry whitelist:

  • dsh.bundle.patch.yml — the profile-bundle lane (dsh plugin --profile <name> add billion-context)
  • the launcher overlay .bili-acp.patch.yml written by writeDshAcpPatch (src/launcher.ts) — the bili dsh lane; without this, launcher-only profiles without the bundle would keep the bug

Each row targets llm-deepseek and llm-deepseek-account (the two built-in route entries from @deepseek-ai/dsh-base; ids are stable across versions) with:

retryPolicy:
  mode: normal
  retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE]

Key mechanics:

  • A cordis patch replaces the targeted row's whole config rather than deep-merging it (applyEntryPatches, vendor/include), so each row restates every default code plus MALFORMED_RESPONSE — dropping any default code would silently remove its retries. The comment in both files pins this for future editors.
  • Rows carry mode: normal only and omit maxRetries/backoff on purpose, so DSH's defaults flow through unchanged (5 retries, 500ms→10s, jitter 0.1).
  • Rows absent on another DSH version or custom composition are skipped with a warning, never a boot failure. retry-policy.ts is byte-identical from dsh v0.1.1-rc.2 through master (verified by tag-tree diff), so the static rows are safe on all realistic DSH versions.
  • Users can override: the profile's own cordis.patch.yml applies after the bundle layer (last write wins per row); the launcher overlay applies last, consistent with how compaction-basic → auto: false already behaves there.

Behavior change (intended, disclosed)

For all bili dsh-lane users, on the built-in llm-deepseek / llm-deepseek-account routes only:

before after
MALFORMED_RESPONSE retries 0 (turn killed immediately) up to 5 (default backoff 500ms→10s, jitter 0.1)
other codes / budgets / timings defaults unchanged (resolved-policy delta vs default = +MALFORMED_RESPONSE -[])

Extra provider requests happen only on the rare malformed event; each retry re-runs the failed step inside the same open turn over the same durable history (failed partial output never reaches the model).

Config surface

N/A — no new bili config fields, env vars, or CLI flags. The change sets values on DSH-side entries through bili's existing patch carriers; the override path is DSH's own profile cordis.patch.yml.

Verification (real dsh 0.2.0-rc.2, hermetic prefix install)

  • tests/e2e/e2e-native-dsh.test.ts 2/2 pass and tests/e2e/e2e-native-dsh-persona.test.ts 1 pass + 1 pre-existing env skip (headless Auto-permission surface, unrelated to this change) — boots real dsh 0.2.0-rc.2 with the patched layer mounted; an invalid row would fail schema validation at boot
  • Real cordis patch engine (applyEntryPatches from the installed packages, real base→headless→bili layering): both rows resolve to exactly the intended config, zero "entry not found" warnings
  • Real RetryPolicySchema standard-schema validation: OK; negative control (empty list) rejected as expected
  • Real resolveRetryPolicy: resolved = {mode: normal, maxRetries: 5, retryableCodes: [5 defaults + MALFORMED_RESPONSE], initialDelayMs: 500, maxDelayMs: 10000, jitterRatio: 0.1}
  • typecheck clean, build clean; writeDshAcpPatch tests updated to pin the exact code list (teeth kept) and pass

Both bili-owned dsh patch carriers (dsh.bundle.patch.yml and the launcher .bili-acp.patch.yml overlay) now add MALFORMED_RESPONSE to the built-in DeepSeek routes' retry whitelist. Each row restates every default code because a cordis patch replaces the targeted row's whole config rather than deep-merging it. Rows carry mode: normal only, so DSH's default budget/backoff (5 retries, 500ms->10s, jitter 0.1) flows through unchanged. Rows absent on other DSH versions are skipped with a warning, never a boot failure.
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-10_dsh-retry-malformed-response (eb0e4d2)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2823-pr.2610.d3b410a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2610.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始审 PR #2610(fix #2605)。已拉取分支核对:

  • 分支 2026-10-10_dsh-retry-malformed-response(head ba70cc1),相对 master 仅 4 文件 +29/-4:dsh.bundle.patch.yml(两条重试行)、src/launcher.ts(overlay 内联串同步追加)、tests/launcher.test.ts(断言钉住全量码表)、CLIENTS.md(两车道文档)。
  • CI:24 个检查里 20 个已过(含 real dsh e2e、kernel/lane/version guard、docs-sync),4 个 test 矩阵还在跑。
  • 正在做定向验证(DSH 侧默认值核对 + 本地定向测试 + rebase 干净度),完成后按八问体例给完整结论。

ework-agent added 2 commits October 10, 2026 15:14
The runLaunch-dsh overlay test still anchored 'auto: false' at end-of-file ($ in multiline mode), which the #2605 retry rows below it broke: the assertion failed on the new .bili-acp.patch.yml content. Mirror the update already made at the writeDshAcpPatch test (drop the EOF anchor, pin both llm-deepseek / llm-deepseek-account retryableCodes rows, keep $ on the final row).
dsh.bundle.patch.yml ships in the npm package but had no unit teeth: an absent or drifted retry row is warn+skip at dsh boot, so the real-dsh e2e stays green either way (the overlay lane was pinned by launcher.test.ts). Pin both rows' exact code lists in the bundle file and assert the launcher overlay carries byte-identical rows, so a future drift on either carrier fails fast locally.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #2610 review 完成(含两个直接修掉的小问题)— 结构化八问

① 是什么问题:dsh 桌面端走内置 DeepSeek 路由(llm-deepseek / llm-deepseek-account)时,模型采样偶发产出畸形 tool-call 参数 JSON(缺一个引号即可),DSH 适配器 finalize 时 JSON.parse 抛 MALFORMED_RESPONSE,而 DSH 默认重试白名单不含该码 → 零重试、整个 turn 死掉且不可恢复(#2605,用户 LightX001 实报,dsh 0.2.0-rc.2 + bili 0.1.189)。

② 什么原因导致的:DSH resolveRetryPolicy 的 DEFAULT_RETRYABLE_CODES 只有 [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT];而 cordis patch 对命中的行是整行 config 整体替换(无深合并),所以修复必须在 bili 拥有的两个 patch 载体里把全部默认码重述一遍再加 MALFORMED_RESPONSE——这正是本 PR 的做法。我对上游 retry-policy.ts 源码逐条核对过:五个默认码与 PR 重述列表逐字一致;省略 maxRetries/backoff 时 DSH 默认值(5 次、500ms→10s、jitter 0.1)自然流入;6 码列表通过其校验(非空、无重复)。

③ 影响范围与严重程度:仅 bili dsh 通道(bili dsh launcher overlay + profile-bundle 安装)且命中内置 DeepSeek 两路由的用户;触发条件是采样 flake(概率性),一旦发生该 turn 完全无法使用(答案未交付)。其他线路/提供商/重试预算均不受影响。

④ 怎么修复的:两个载体(dsh.bundle.patch.yml + writeDshAcpPatch 的 .bili-acp.patch.yml)各加两行 retryPolicy: {mode: normal, retryableCodes: [默认五码 + MALFORMED_RESPONSE]};目标行不存在的 DSH 版本 warn+skip、不炸启动;用户可用自己 profile 的 cordis.patch.yml(后应用、last-write-wins)覆盖。

⑤ 是否有回归问题:逐项审过既有行为——compaction-basic auto:false 行未动;整行替换的安全性(triage floor 3 已核 base bundle 这两行无内联 config,不会丢东西);wire/persistence/config surface/kernel 均未触碰(src/launcher.ts 改动只是字符串常量);fail-fast 语义不变。未发现回归。 发现并直接在 PR 分支修掉两处测试问题:

  • 252039c44:tests/launcher.test.ts:3983(runLaunch dsh shim-lane 测试)仍用 EOF 锚定断言 auto: false\n$,新内容下必然失败——本地复现确认(pre-fix 红、post-fix 绿),已按作者 3585 行的同款改法更新;
  • eb0e4d2db:新增 tests/issue2605-dsh-retry-patch.test.ts——bundle 文件随 npm 发布但此前零单测牙口(缺行在 real-dsh e2e 里只是 warn+skip,删了也绿),现钉死两行精确码表 + 断言两个载体逐字节一致。
    本地定向验证:上述 5 个相关测试全绿、tsc --noEmit 干净。

⑥ 是否破坏缓存:不涉及——bili 侧缓存/重试/游标/去重键零改动;DSH 侧仅新增 MALFORMED_RESPONSE 的重试资格,prompt_cache_key 机制不受影响。

⑦ 是否改了设计方向:不涉及——完全复用现有 patch 载体机制,与 compaction-basic auto:false 同一模式。

⑧ 是否引入了新的配置:不涉及(config surface N/A,无新字段/env/flag)。

行为变更披露(有意):内置 DeepSeek 两路由上 MALFORMED_RESPONSE 重试次数 0 → 至多 5(DSH 默认退避),PR 正文表格已列明。
已知局限(监控项,不另立项——同属本修复机制族):重述码表复制了 DSH 的 DEFAULT_RETRYABLE_CODES;若未来 DSH 版本新增默认码,bili 的列表会静默漏掉直到同步(retry-policy.ts 自 v0.1.1-rc.2..master 逐字节未变,e2e 钉在 0.2.0-rc.2,风险有界)。

判定:代码与行为审查通过,小问题已就地修完;head eb0e4d2db 的 CI 全矩阵正在跑(此前旧 head 上唯一会红的就是那个过期断言,现已修),全绿即可合。

@ranxianglei
ranxianglei merged commit 98fb91c into master Oct 10, 2026
23 of 25 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.216-master.98fb91c (rolling master tag).

npm install -g billion-context@0.0.216-master.98fb91c  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@YunzhouLi-hub

Copy link
Copy Markdown
Contributor

Heads-up: master (98fb91c) is red on both Windows test combos — run 38034641775, test (windows-latest, 22) and test (windows-latest, 24) fail with:

✖ #2605: dsh.bundle.patch.yml restates every default code + MALFORMED_RESPONSE on both DeepSeek routes

Ubuntu is green. First seen on my PR #2619 CI; verified it reproduces on master own CI run of the same commit, so it came in with this merge rather than that PR. Repro at a glance: test (windows-latest, 24) job log, ~07:36Z.

@YunzhouLi-hub

Copy link
Copy Markdown
Contributor

Root cause of the Windows-only failure — confirmed locally with core.autocrlf=true (same as the windows runners):

dsh.bundle.patch.yml has no eol attribute, so Git-for-Windows checks it out with CRLF (32 CR bytes in my working copy). The new test does byte-exact includes() against \n-joined rows:

tests/issue2605-dsh-retry-patch.test.ts:26-29
fs.readFileSync("dsh.bundle.patch.yml", "utf8") then bundle.includes("- id: llm-deepseek\n config:\n retryPolicy:\n mode: normal\n retryableCodes: [...]\n")

With CRLF every \n becomes \r\n → all three includes fail on Windows; LF checkouts (ubuntu) pass. The second test passes on both because writeDshAcpPatch() generates the overlay itself with \n.

The repo already fixed this exact class of bug for the golden snapshots in .gitattributes (#1305): tests/golden/wire-contract/*.json text eol=lf. Same one-line treatment applies here: add dsh.bundle.patch.yml text eol=lf to .gitattributes. Happy to send that as a tiny PR if you want.

ranxianglei pushed a commit that referenced this pull request Oct 10, 2026
…2605)

tests/issue2605-dsh-retry-patch.test.ts reads the patch file raw and
byte-matches multi-line rows; Git-for-Windows autocrlf=true converts it to
CRLF at checkout, failing both windows CI shards on every run since #2610
landed (master red at 98fb91c, run 38034641775). Same mechanism as the
#1305 golden pin above. In-passing fix carried on #2625.
YunzhouLi-hub pushed a commit to YunzhouLi-hub/billion-context that referenced this pull request Oct 10, 2026
…t includes (ranxianglei#2631)

ranxianglei#2605's bundle test asserts byte-exact multi-line literals against the
shipped dsh.bundle.patch.yml; Git-for-Windows checks it out CRLF
(autocrlf) and the includes fail on every Windows leg since ranxianglei#2610
merged — master's test (windows-latest, 22|24) legs have been red
since 07:30Z, blocking every PR's required checks (ranxianglei#2631).

Same failure class as ranxianglei#1305 (golden JSONs): pin eol=lf in
.gitattributes, and belt-and-suspenders normalize CRLF after read in
the test so checkouts that bypass gitattributes still pass.
@ranxianglei
ranxianglei deleted the 2026-10-10_dsh-retry-malformed-response branch October 15, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

一次坏工具参数就让整轮作废:希望把 MALFORMED_RESPONSE 纳入重试名单

2 participants