Skip to content

fix(dsh): stale verifyAttachAndRecover landings must never clobber the self-healed register (#1783) - #1787

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-01_dsh-stale-landing-guard
Oct 1, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-01_dsh-stale-landing-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Unblocks the v0.1.176 release PR #1783 (windows-22 test deadlock).

Root cause

tests/dsh-native.test.ts #983 maybeRetry self-heals a base-less register after a failed respawn timed out on windows-latest/node 22 only (both attempts of run 36740908901, timeout waiting for plugin headers stamped, 5109ms vs ~62ms locally).

The apply() attach chain is a long-lived async landing: loopback probe + evidence grace + process spawn are all slow on the windows runner. While it is in flight, maybeRetry can already have self-healed the register onto a NEW origin. When the stale landing finally resolved it:

  • wiped register.base / register.toolsReady unconditionally (clobbering the heal), or
  • registerTools' catch armed a global retryAt = now + 10s against a base the register had already moved past, freezing every later heal behind the wall.

Local repro: wrapping globalThis.fetch with a 40–200ms delay on the dead port-1 preset turns the fast green path into the exact CI deadlock; 0ms always passes.

Fix — one contract, three changes

A landing may only write the register while it still owns it (base === undefined or base === staleOrigin):

  1. landingOwnsRegister() guards both wipe sites + the fallback state.origin reset in verifyAttachAndRecover
  2. the unfreeze wipe also clears retryAt — a back-off armed against the stale base dies with it
  3. registerTools scopes its back-off to the base that actually failed (if (register.base === base))

Regression tests

Two deterministic interleavings added to tests/dsh-native.test.ts:

  • slow preset probe (60ms) — late landing vs in-flight manifest fetch
  • slow spawn seam (120ms) — back-off armed before the landing's unfreeze wipe

Both fail on master, pass with the fix (verified via stash-revert). Full file: 41/41. Typecheck clean. Product semantics unchanged — self-heal stays request-driven with the same back-off; retries are simply no longer poisoned by stale landings.

…e self-healed register

windows-22 CI on the v0.1.176 release PR (#1783) deadlocked the #983
self-heal test: the apply() attach chain resolves long after it started
(loopback probe + evidence grace + process spawn are all slow there), and
in the meantime maybeRetry can already have healed the register onto a
new origin — the late landing then wiped register.base/toolsReady
unconditionally, or registerTools' catch armed a global 10s retryAt
against a base the register had already moved past, freezing every later
heal attempt behind the wall.

Three changes, one contract: a landing may only write the register while
it still owns it (base undefined, or still the stale origin the chain set
out to replace):

- landingOwnsRegister() guards both wipe sites and the fallback origin
  reset in verifyAttachAndRecover
- the unfreeze wipe also clears retryAt — a back-off armed against the
  stale base dies with it
- registerTools scopes its back-off to the base that actually failed

Two deterministic regression tests pin the interleavings (slow preset
probe; slow spawn seam) — both fail on master, pass with the fix.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-01_dsh-stale-landing-guard (8665088)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1764-pr.1787.643b6ef

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1787.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

收到。这是已完成工作的 PR(#1787 → 解锁 release #1783),我按 PR Review 规范走一遍,不重做实现:

  • 拉取分支核对代码(landing ownership 契约的三处改动)
  • 本地 typecheck + dsh-native 全量测试 + stash-revert 验证回归测试在 master 上确实失败
  • 等 CI 矩阵跑完(当前 e2e 全绿,4 个 test job 还在跑,windows-latest/node 22 是原始死锁现场,重点盯)

结论稍后按结构化七问给出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review 完成 — 结论:可合并,解除 v0.1.176(#1783)的 windows-22 死锁。

结构化七问

1. 是什么问题
dsh native 插件里 verifyAttachAndRecover 是一条长生命周期异步落链(loopback probe + 证据宽限 + spawn)。它在飞行途中时,maybeRetry 可能已把 register 自愈到新 origin;晚到的 stale landing 随后无条件抹掉 register.base/toolsReady,或对着 register 已经离开的 base 挂起一个全局 retryAt = now+10s 墙——后续所有自愈被冻在墙后。windows runner 上 probe/spawn 慢(5109ms vs 本地 ~62ms)把竞态窗口撑大到必现;生产上表现为 bili 工具 ≥10s 黑屏且可被重复 landing 反复续墙。

2. 什么原因导致的
根因是 register 写入缺少所有权检查:异步 landing 不检查 register 是否已移动就无条件写;retryAt 是不绑定任何 base 的全局标量。层次上,CI 超时是表面现象,病因是「stale 落点 + 无作用域退避」的组合,不是 windows 本身。

3. 怎么修复的
单一契约、三处落地:landing 只有仍拥有 register(base === undefined || base === staleOrigin)时才允许写。(a) landingOwnsRegister() 守卫两个 wipe 点 + fallback state.origin 重置;(b) unfreeze wipe 同时清 retryAt——挂在 stale base 上的退避随其死亡;(c) registerTools 的退避收窄到实际失败的 base(if (register.base === base)),register 已移走则不再挂墙。修的是产生缺陷的机制(写入点+退避作用域),不是症状遮蔽。

4. 是否有回归问题 — 审过以下既有行为,逐项结论:

  • 单链常态路径:apply() attach 模式在 arm 链之前同步设 register.base = attachOrigin(src/agent/dsh-native.ts:614),landing 时 guard 恒真 → 字节级不变;
  • healthy-attach 短路、pinned-evidence 拒判日志/文案、unfreeze 的 env/state 变更、真实失败时的重试节奏(RETRY_INTERVAL_MS=10s 未动、base 未移走时墙照常挂)→ 不变;
  • 错误日志行保留,无 fail-fast 变吞错;
  • 有意变更(均已披露):① unfreeze wipe 现在会清 retryAt(旧:死 preset 的 10s 墙活过 unfreeze);② registerTools 在 manifest fetch 期间 register 已移走时不再挂全局墙(旧:无条件挂);③ landing wipe / state.origin 重置变为条件式(register 未移走时是无操作)。

5. 是否破坏缓存 — 不涉及。改动全部限于 dsh 插件进程内的 register/backoff 状态;proxy 侧前缀缓存、会话持久化、去重键均未触碰。

6. 是否改了设计方向 — 没有转向。扩展既有 #983/#1130/#1365 恢复设计,加一条所有权不变量;自愈仍是 request-driven、同一 10s 退避,只是不再被 stale landing 投毒。一个非阻塞观察:同函数内 probe-healthy(:285-286)与 pinned-recovered(:293-294)两处写入仍未纳入契约——分析认为当前安全(飞行中 respawn() join 同一 single-flight;routed evidence 使 home 收敛到同一值;最坏情形一次请求内自纠),但触发条件属于多生命周期的奇异交错。已按规则立 #1788(来源标记指向本 PR,标注低危/预存/非阻塞),处理方式由 owner 定。

7. 是否引入了新的配置 — 不涉及。无新 env/config/默认值。

验证证据

  • CI(head 8665088):13/13 全绿,含原死锁现场 test (windows-latest, 22)、windows-24、ubuntu-22/24、e2e、build-artifact、version-guard、hermetic registry e2e、四个 real-client lane;mergeable_state: clean。
  • 本地:typecheck 干净;tests/dsh-native.test.ts 41/41。
  • stash-revert 证明:src 回退到 PR base(master)、保留新测试 → 两个新测试均失败于 timeout waiting for self-healed tool registration(正是 CI 死锁症状);恢复修复后通过。固定延迟(60ms/120ms)对 5s waitFor deadline vs 10s 墙,时序确定,非环境运气。
  • diff 整洁:2 文件、单 commit、每行都与本 PR 目的相关,无空行/重排噪音。

后续动作

合并 #1787 后请对 release PR #1783 重新触发 CI(它的 windows-latest/node-22 job 就是被这个 bug 卡住的)。按规矩我不合并 PR,请人工合入:#1787

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant