Skip to content

[PR] fix(pi-subagents): watchdog timeout no longer falsely reports completed; empty reply diagnostics include upstream termin | fix(pi-subagents): watchdog 超时不再误报 completed;空回复诊断带上游终态 - #4411

Open
ranxianglei wants to merge 11 commits into
masterfrom
2026-10-23_delegate-timeout-status

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4359

修了什么

1. watchdog 超时被误报为 completed(状态误判)

旧行为:异步子进程被 watchdog 杀掉(code===null、run.timedOut 置位)时,只要还有任何非空进度文本或 stderr,effectiveExitCode 就会强制为 0,随后 run.status = effectiveCode === 0 && !noOutput ? "completed" : "failed" → 部分输出 alone 就能把 run 标成 completed,父 agent 收到成功通知但没有任何交付物。

新行为:新增纯函数 classifyDelegateOutcome(code, timedOut, agentSettled, output, stderr)。completed 要求有效退出码成功且有正向完成证据:真实非 null 退出码且无 watchdog 超时,或观察到 agent_settled。watchdog kill 只在 agent 未 settle 时记 timedOut(onKill/onEofGrace 由 agentSettled 门控),因此 timeout + 未 settle = 一律 FAILED,与有无部分文本/stderr 无关。

settled 例外保留:已交付输出但进程在 teardown/EOF 挂起(code===null、agentSettled=true、timedOut 未置位)仍算 completed。agentSettled=false 时该规则与同步路径三元式(timedOut → failed)逐行投影一致,async/sync 语义对齐。

行为变更披露:未 settle 的 run 上,watchdog kill / 外部 null-code 关闭 / spawn 错误且带有部分文本或 stderr 的情形,现在报告 FAILED ⚠️(旧值:completed)。结果文件与活动日志不变,仍可人工检查。这正是 issue 报告的误判场景。

2. 空回复诊断区分上游 error/incomplete 与 completed-empty

事件 applier 捕获最后一条 assistant message_end 的终态:stopReason / rawStopReason / errorMessage(含 error 字段回退)+ text/thinking 块计数 + 该轮 usage。失败正文、结果文件回填、warn 日志各追加一行 upstream terminal state: stopReason=…[, rawStopReason=…][, errorMessage=…] , text_blocks=N, thinking_blocks=N[, , last_turn_usage(in=A, out=B)];未观测字段渲染 unknown,print 模式(无事件流)渲染 unavailable。NO_FINAL_OUTPUT_BODY 不再断言 "likely a thinking-only stop"(那只是可能原因之一),原因判断交给诊断行。响亮失败的行为本身不变。

测试

  • tests/delegate-no-output.test.ts:11 行终态矩阵(含两行此前误判)+ async/sync 一致性投影断言。
  • tests/events.test.ts:issue 中 A/B/C 三种 message_end 形状提取、terminal-only 事件、非 assistant 跳过;顺手把该文件 4 处既有坏类型断言换成收窄断言。
  • tests/delegate-event-applier.test.ts:applier last-wins 捕获、terminal-only 事件不触发 onUsage。
  • tests/delegate-tool.test.ts:formatter 渲染 + 基础文案回归。
  • 本地定向运行:改动模块相关测试全部通过;src typecheck 干净;三个改动的测试文件 strict tsc 均退出 0。全量判定以 CI 为准。

第二个 commit(顺手小修,独立标注)

做 #4359 时发现 tests/delegate-tool.test.ts 存在 6 处既有 strict-tsc 错误(测试文件从不被 CI strict typecheck,所以一直不可见):injectedWaitMessage 参数 result?: { file: string } 过窄(运行时按 truthy 安全读取,测试名就叫 "tolerates a missing file")→ 放宽为 file?: string(仅类型、向后兼容);4 个本地 run 字面量缺 usageReported → 补 false;wait-timeout 集成测试以 2 参调用 cancelTool.execute → 对齐兄弟调用补 (undefined, undefined, ctx)。该文件 strict tsc 现退出 0,108 个 delegate-tool 测试全过。

版本

pi-subagents 0.2.2 → 0.2.3(Pi-Subagents Boundary 要求);根 package-lock.json 连带一行。

范围外(另行处理)

  • 内外预算冲突(case D):README 已加提示(嵌套 sync delegate 自带 timeoutMinutes 超过外层剩余预算会被外层先杀);向子进程传递剩余 deadline 属独立变更。

@ranxianglei ranxianglei changed the title fix(pi-subagents): watchdog 超时不再误报 completed;空回复诊断带上游终态 [PR] fix(pi-subagents): watchdog timeout no longer falsely reports completed; empty reply diagnostics include upstream termin | fix(pi-subagents): watchdog 超时不再误报 completed;空回复诊断带上游终态 Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_delegate-timeout-status (fd6993a)

Latest build — install this exact version

npm install -g billion-context@0.0.9708-pr.4411.363d8cf

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
fd6993a billion-context@0.0.9708-pr.4411.363d8cf build
270ae59 billion-context@0.0.7200-pr.4411.5ea79e8 build
ff7cd62 billion-context@0.0.6185-pr.4411.169ce07 build
84088f2 billion-context@0.0.6098-pr.4411.898bfb4 build
3819b3c billion-context@0.0.6002-pr.4411.e571439 build
0572473 billion-context@0.0.5310-pr.4411.70138cb build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4411.tgz
npm install -g package

Install instructions are refreshed on each push.

awork added 2 commits October 23, 2026 23:08
…; empty-reply diagnostics carry upstream terminal state (#4359)

Two defects in the async delegate lifecycle:

1. Status misclassification. When the watchdog killed a non-settled child
   (code===null, run.timedOut set), partial progress text or stderr alone
   coerced effectiveExitCode to 0 and the run was marked "completed" — the
   parent got a success notification with no deliverable. New pure rule
   classifyDelegateOutcome: "completed" requires a successful effective exit
   AND positive completion evidence — a real non-null exit code without a
   watchdog timeout, OR agent_settled observed. The settled exception is kept
   (delivered output whose process hung in teardown/EOF still completes); with
   agentSettled=false the rule projects exactly onto the sync ternary, so
   async/sync semantics are aligned. Behavior change: watchdog kills, external
   null-code closes, and spawn errors of non-settled runs now report FAILED
   (artifacts kept for inspection) instead of completed when partial output or
   stderr exists.

2. Empty-reply diagnostics. Exit-0 empty replies previously got one generic
   "likely a thinking-only stop" message. The event applier now captures the
   last assistant message_end terminal state (stopReason/rawStopReason/
   errorMessage + text/thinking block census + that turn's usage); the failure
   body, result-file backfill, and warn log carry an "upstream terminal state:"
   line separating upstream error/incomplete from completed-but-empty.
   Unobserved renders as unknown; print mode says unavailable. NO_FINAL_OUTPUT_BODY
   no longer asserts a cause itself.

pi-subagents 0.2.2 -> 0.2.3 (+ root lockfile line). Regression coverage:
terminal-state matrix incl. the two misclassified rows, async/sync parity
projection, message_end extraction on issue A/B/C shapes, applier last-wins
capture, formatter rendering. README: timeout-is-failure semantics plus a
nested inner/outer budget-conflict hint.
…-tool.test.ts (found while working #4359)

Six latent type errors the test file carried before this PR (test files are
not strict-typechecked by CI, so they stayed invisible):

- injectedWaitMessage's run parameter typed result?: { file: string } while
  the runtime reads run.result?.file truthy-safely and the test named
  "tolerates a missing file" deliberately passes result: {} — loosen to
  result?: { file?: string } (type-only, backward compatible).
- Four local run literals in the buildWaitResult/buildCancelResult usage
  tests lacked usageReported, which the tests then assert flips to true —
  seed usageReported: false.
- The wait-timeout integration test called cancelTool.execute with 2 args
  against the 5-arg Tool.execute signature — pass the same trailing
  (undefined, undefined, ctx) its sibling wait call passes.

Strict tsc on the file now exits 0; all 108 delegate-tool tests pass.
@ranxianglei
ranxianglei force-pushed the 2026-10-23_delegate-timeout-status branch from ddc9d57 to 0572473 Compare October 23, 2026 15:14
@ranxianglei ranxianglei added bug Something isn't working p2 pi scope/wiring labels Oct 30, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 watchdog 误报 completed / 空回复诊断两条签名在 open+closed issue、PR 中均无同机制条目;最接近的 #2832(已关,preflight 摘要探针路径)与 #3745(acp_delegate_wait 展示层)子系统不同。本 PR 即 #4359 的修复 PR(floor 2 有完整分析记录)。

标签: bug / pi / scope/wiring / p2——pi 车道子代理缺陷,协议接线层,有绕行(结果文件仍可人工检查)。

现状核查(本 PR 为此前 agent 工作,未重做,只核分支与 CI):

  • 分支净变更 9 文件 389+/52-,全部在 pi-subagents/(delegate-events.ts、delegate-tool.ts + 4 个测试)+ 根 package-lock.json 一行,零 kernel/ 文件。
  • CI(head ff7cd62):40 项检查 = 23 绿 + 16 跳过 + 1 红:kernel-guard。

kernel-guard 红 = 陈旧基线假阳性,不是本 PR 改了内核:取证(job 日志 2026-10-26T08:17Z)显示触发时 master 还落后于分支经 #4599 ci-fixes sweep 带进的三处内核变更(kernel/package.json、kernel/src/compress.ts、kernel/src/sync.ts),merge ref 的 diff 非空 → 守卫索要 kernel-change: 声明。这三处现已全部落在当前 master;分支对当前 master 的净 diff 里 kernel/ 为零。因此不补声明(补了就是虚假声明),正确处置是把分支基线同步到当前 master 后重跑。

正在做:把最新 master + ci-fixes 合进分支推送重触发 CI(master 自身目前在若干 test 分片上红的在档继承家族见 #4599;合入 ci-fixes 会带入 #5140 存储模式钉,对应分片预期转绿,剩余 Windows 家族项与本 PR 无关)。完成后回帖八问结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收工报告(八问)

  1. 是什么问题:本 PR 修 pi 子代理两个缺陷——①watchdog 杀掉异步子进程后,只要有部分文本/stderr 就把 run 误报 completed(父 agent 收到成功通知但没有任何交付物);②空回复诊断分不清上游是 error/incomplete 还是 completed-empty。本轮我的任务不是重做修复(此为既有 agent 工作,[Bug] pi-subagents: Incorrectly reports "completed" when watchdog times out with only partial output (empty-reply termin | [Bug] pi-subagents:watchdog 超时且仅有部分输出时误报 completed(附空回复终态对照) #4359 floor 2 有完整分析),而是核分支、查 CI、处理唯一红项。
  2. 什么原因导致的:CI 唯一红项 kernel-guard 的根因是陈旧基线假阳性,不是本 PR 改内核——触发时(10-26 08:17Z)master 还落后于分支经 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 ci-fixes sweep 带进的三处内核变更(kernel/package.json、kernel/src/compress.ts、kernel/src/sync.ts),merge ref diff 非空 → 守卫索要 kernel-change: 声明。job 日志与本地 tree 级比对均已取证。
  3. 影响范围与严重程度:原缺陷影响 pi 车道所有异步 delegate 超时场景(p2,有绕行:结果文件与活动日志仍可人工检查)。本轮基线同步对 PR 内容本身零行为变更。
  4. 怎么修复的:PR 内容不动。处置 = 把分支基线同步到当前 master:merge origin/master@44067a5bc + origin/ci-fixes@67f8e8940(零冲突;ci-fixes 带入 [Bug][fork] Cold restore cannot use an intact unified raw tail for a non-retained parent (#4266 item 4) #5140 存储模式钉,tests/plugin-fork.test.ts +9/-1,舰队惯例,[PR] ci-fixes: master-CI convergence — docs surface registration + #4666 v9 stamp #5208 落地后自动退出 PR diff),推送新 head sha=270ae5941。未补 kernel-change: 声明——本 PR 对当前 master 净 diff 里 kernel/ 为零,补了就是虚假声明。
  5. 是否有回归问题:审过的既有行为——①settled 例外保留(已交付输出但 teardown/EOF 挂起仍算 completed);②agentSettled=false 时与同步路径三元式逐行投影一致;③NO_FINAL_OUTPUT_BODY 只删了一句推测性断言("likely a thinking-only stop"),响亮失败行为不变;④结果文件/活动日志写入路径未动;⑤第二个 commit 纯类型放宽 + 测试字面量补齐,无运行时变化。本地定向验证全绿:pi-subagents strict tsc 干净、改动 4 个测试文件 162/162、组件全量套件 279/279、根 typecheck(含 kernel+pi-subagents 构建)干净;kernel-guard 规则本地模拟 PASS(净 diff 零 kernel/ 文件)。
  6. 是否破坏缓存:不涉及(不触碰缓存/重试/游标/去重路径)。
  7. 是否改了设计方向:否。状态判定收敛为纯函数 classifyDelegateOutcome + 诊断信息增强,均在 pi-subagents 层内,无架构/边界转向。
  8. 是否引入新配置:否(无新 env/config/默认值)。

行为变更披露(PR 正文已载,复核确认):未 settle 的 run 上,watchdog kill / 外部 null-code 关闭 / spawn 错误且带部分文本或 stderr 的情形,现在报 FAILED ⚠️(旧值:completed)。这正是 issue 报告的误判场景,属有意变更。

CI 状态:新 head 已触发(90 checks,其中 47 queued——舰队 runner 队列积压在档见 #4599,43 cancelled 为被取代的旧 run)。预期开跑后静态门禁全绿;test 分片若命中在档继承家族(#5140 已被带入的钉覆盖;Windows 家族项待 owner 裁决,与本 PR 无关)按 #4599 协议处置。已在 #4599 留一行同步记录。

判定:分支侧就绪(kernel-guard 假阳性已消除、本地两层测试全绿),待 CI 矩阵消化完队列确认绿后即可合并——合并动作在你手里:#4411

ework-agent and others added 4 commits November 14, 2026 04:35
The suite baseline preload (preload-storage-baseline.mjs) pins
BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After #6480 flipped the
storage default to pure-sqlite, the notice-registry suite's visibleNotices
assertions expect the pure-sqlite default notice, which is gated on
unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's
"legacy" made the notice silent, so the suite was born-red on master. Pin
the selection explicitly (#4011 lesson: a test that depends on the production
default passes locally but fails in CI).
master independently consumed 0.2.3 in #5637 (stand acp_delegate down for
user-scope installs); the PR's original 0.2.2 -> 0.2.3 bump now nets to
zero against master, which would trip pi-subagents-guard (src changed,
no version change in the diff). Take the next patch version instead.

Also syncs the component's own package-lock.json, which the original bump
left stale at 0.2.2 (two-line surgical edit; a full npm regen would churn
44 lines of npm-version libc-field drift).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and pushed — PR head is now fd6993a637, live-verified mergeable: true (state blocked = checks pending, no conflict).

What this run did:

  1. Base sync (the blocker). Branch was 1517 commits behind master (last synced at 44067a5bc on 10-30; the 10-31 merge-guard attempt was interrupted by a daemon restart and never followed up). Merged origin/master@e1f31ad9d5 — zero conflicts: master's pi-subagents changes since the last sync only touched src/index.ts / src/pi-subagents-detect.ts / READMEs, no overlap with this PR's delegate-events.ts / delegate-tool.ts.
  2. Version double-booking fix (mechanical). The PR's original bump was pi-subagents 0.2.2 → 0.2.3, but master independently consumed 0.2.3 in [Pi] acp_delegate stands down only for project-scope pi-subagents — a user-scope install duplicates the sub-agent surface in every project #5637. After the base sync the net diff had no version change at all, which would have tripped pi-subagents-guard ("src changed without bump"). Re-bumped to 0.2.4: pi-subagents/package.json + root package-lock.json (1 line each) + the component's own package-lock.json (surgical 2-line edit — it had been stale at 0.2.2 since the original bump; a full npm regen would have churned 44 lines of npm-version libc-field drift).
  3. ci-fixes merge (inherited-red preemption). Per CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 latest state, master e1f31ad9d5 is itself red: notice-registry born-red (fixed in ci-fixes c6369fb6a1, not yet on master) + the Windows EBUSY flake family (sentinel tracking). Merged origin/ci-fixes (1 commit, tests/notice-registry.test.ts +5) so this PR's CI doesn't re-hit the known inherited red. One line logged in CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599.
  4. Push discipline. Plain fast-forward push, no force-push, no rebase — branch history and authorship intact. Infra note: the mirror git proxy (origin) started redirecting to its login page mid-run; the push went through the direct GitHub remote instead.

Local targeted verification (full-suite verdict = CI on the pushed head):

  • pi-subagents strict tsc: clean
  • pi-subagents component suite: 279/279 pass
  • Guard simulations on the final tree: pi-subagents-guard PASS (src changed + version bumped), kernel net diff = 0, root package.json untouched, lockfile idempotent (lock-sync PASS)

八问:

  1. 是什么问题: PR 与 master 冲突(dirty)——分支基线停在 10-30 的 44067a5bc,落后 master 1517 个提交;基线同步后又暴露 pi-subagents 版本双占([Pi] acp_delegate stands down only for project-scope pi-subagents — a user-scope install duplicates the sub-agent surface in every project #5637 已在 master 消费 0.2.3)。
  2. 什么原因导致的: 分支长期未同步(10-31 merge-guard 被 daemon 重启打断后无人接手);版本双占是 [Pi] acp_delegate stands down only for project-scope pi-subagents — a user-scope install duplicates the sub-agent surface in every project #5637 与本 PR 各自独立把 0.2.2 提到 0.2.3 的必然结果,基线一同步即显形,不修必红 pi-subagents-guard。
  3. 影响范围与严重程度: 仅本 PR 的合并阻塞 + 一个下轮 CI 必红的守卫项;对已发布功能零影响。
  4. 怎么修复的: merge origin/master(零冲突)+ 机械重 bump 0.2.4(3 文件 4 行)+ merge origin/ci-fixes(带入 notice-registry 存储模式钉,预防在档继承红);fast-forward push,无 force-push、无 rebase。
  5. 是否有回归问题: 无。净 diff 相对 master = PR 原有 7 个 pi-subagents 文件 + 版本/lockfile 3 行 + ci-fixes 测试钉 5 行;PR 行为(未 settle 的超时 run 改报 FAILED)不变;master 侧 pi-subagents 变更与本 PR 文件零交集。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及(ci-fixes 带入的是测试内 env 钉值,非新配置面)。

CI 预期: 新 head 已触发。静态门禁预期全绿(本地已逐项模拟);test 分片若命中在档继承家族(Windows EBUSY flake,#4347 家族,重跑即绿)按 #4599 协议处置,我继续盯。合并动作在你手里: #4411

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Status update (full report in my previous comment above):

Done & verified:

Live CI right now (mirror cache, fetched 23:59Z): 24 checks on fd6993a637 — 1 green (plan), 23 queued (GitHub queue congestion), 0 red.

Remains / risk: nothing actionable yet. If a test shard hits the documented inherited Windows EBUSY flake family (#4347 family, green on re-run), I'll handle it per the #4599 protocol. No design decisions needed. Merge action stays with you: #4411

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Conflict resolution is already reported in my comment above (23:44Z) — head fd6993a637, mergeable: true. This run's fresh evidence:

Live CI on fd6993a637 (mirror cache, 23:59Z): 24 checks — 1 green (plan), 23 pending (queued/running), 0 red. Nothing to fix yet.

Verified last run: pi-subagents tsc clean, component suite 279/279, guard simulations all green (pi-subagents-guard with the 0.2.4 re-bump, kernel net-diff 0, lock-sync idempotent).

Remaining risk: if a test shard hits the documented inherited Windows EBUSY flake family (#4347 family, green on re-run), I'll handle it per the #4599 protocol — re-run, no code change. No design decisions needed. Merge action stays with you: #4411

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Verdict: both red shards are the documented inherited Windows EBUSY flake family — not a regression from this PR. Re-run triggered, no code change (there is nothing to fix in code).

What this run did:

  1. Re-verified live first. The mirror check cache was stale (frozen at 00:16:40Z, ~3h before the failure event, and it doesn't list the test shards), so per the task's log-fetch allowance I pulled the two failed job logs one-shot.
  2. Signature confirmed on both shards — identical to the windows-22-1 lane randomly red: during test cleanup, unlink index.db hit an EBUSY external lock, couldn't survive tmp-rm | windows-22-1 车道随机红:测试清理期 unlink index.db 遇 EBUSY 外部锁,熬不过 tmp-rm 的 9.6 秒重试梯子(#2799/#2908 族复发) #4347 family documented in CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 (same signature on master's own run 43469650189 @ e1f31ad9d5):
    • shard 3: tests/persist-zstd.test.ts — 10/10 store-zstd tests, all EBUSY: resource busy or locked, unlink '...Temp\bili-zstd-XXXX\index.db' (rimraf cleanup vs Windows file lock), 2735/2759 pass.
    • shard 1: same 10 persist-zstd + 3 tests/metadata-fold.test.ts, same EBUSY-unlink-index.db error, 1615/1628 pass.
  3. Regression check: the PR's net diff touches only pi-subagents delegate files + version/lockfile lines + the ci-fixes test pin — zero overlap with persist-zstd / metadata-fold / the storage layer. All static gates on the head are green (pi-subagents-guard, lane-guard, config-surface-guard, architecture-docs, kernel-guard, lock-sync, …); the only failures in this run are these two shards.
  4. Action: triggered rerun-failed-jobs on run 43528702534 (HTTP 201). One line logged in CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599. No push needed — no code change.

八问:

  1. 是什么问题: 新 head fd6993a637 上两个 Windows node-22 test 分片(win22 shard 1 与 shard 3)红。
  2. 什么原因导致的: Windows 文件锁 flake——测试清理时 rimraf unlink 临时目录 index.db 被 OS/杀软瞬时占用报 EBUSY;属在档 windows-22-1 lane randomly red: during test cleanup, unlink index.db hit an EBUSY external lock, couldn't survive tmp-rm | windows-22-1 车道随机红:测试清理期 unlink index.db 遇 EBUSY 外部锁,熬不过 tmp-rm 的 9.6 秒重试梯子(#2799/#2908 族复发) #4347 家族,master 自己的 run(43469650189 @ e1f31ad)同签名,是继承性失败,非本 PR 引入。
  3. 影响范围与严重程度: 仅本 PR 的 CI 门禁(2 个分片);产品零影响;同一 flake 也影响 master CI(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 在档,sentinel 跟踪 flake-vs-real 确认中)。
  4. 怎么修复的: 无代码改动(代码无可修)——job 日志一次拉取核对签名,确认与在档继承家族一致后触发 rerun-failed-jobs(run 43528702534,201),按 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 协议「重跑即绿」处置。
  5. 是否有回归问题: 无。PR 净 diff 只碰 pi-subagents delegate 文件 + 版本/lockfile + ci-fixes 测试钉,与 persist-zstd/metadata-fold/storage 零交集;失败测试全在存储临时目录清理路径,与本 PR 无关。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及(flake-vs-real 根因确认由 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 sentinel 跟踪,不在本 PR 处理)。
  8. 是否引入了新的配置: 不涉及。

状态: 重跑已触发,等结果。若重跑转绿,本 PR 唯一剩余动作是你的合并: #4411 ——若重跑仍红(即 flake-vs-real 判定为真 bug),按 #4599 协议升级处理,我继续盯。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

🔶 需要决策

Re-run #1 (run_attempt 2) is still red — same two win22 shards, same mechanism, shifted distribution. This is the documented inherited Windows EBUSY family, not a regression from this PR. Per the #4599 protocol the fix belongs in ci-fixes (sentinel lane), so I stopped fixing and am requesting your call on how to unblock the merge.

Verified facts (this run, one-shot log fetches):

Options:

Recommendation: A. Default if undecided: A — nothing needs doing but waiting; the sentinel is already tracking it, and I will merge ci-fixes into this branch + re-verify the moment the fix lands.

八问:

  1. 是什么问题: 本 PR 两个 win22 test 分片重跑后仍红——继承性 Windows EBUSY 文件锁 flake 家族(windows-22-1 lane randomly red: during test cleanup, unlink index.db hit an EBUSY external lock, couldn't survive tmp-rm | windows-22-1 车道随机红:测试清理期 unlink index.db 遇 EBUSY 外部锁,熬不过 tmp-rm 的 9.6 秒重试梯子(#2799/#2908 族复发) #4347),非本 PR 回归;修复属 ci-fixes(sentinel 车道),尚未落地。
  2. 什么原因导致的: Windows runner 文件锁(OS/杀软瞬时占用临时目录 sqlite index.db)→ rimraf 清理 unlink 报 EBUSY;master 自己的 run(43469650189 @ e1f31ad)同签名,CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 在档,sentinel 跟踪 flake-vs-real 确认中。
  3. 影响范围与严重程度(按选项): A = 仅本 PR 合并延迟(时间成本),无产品影响;B = 每次约 15 分钟 CI 资源且可能仍红,无产品影响;C = 合并记录带 2 个红分片(继承性,与本 PR 内容无关),无产品影响但破「全绿合并」惯例。
  4. 怎么修复的(推荐 A): 等 sentinel 把 EBUSY 清理修复(unlink 重试/fs.rm maxRetries 一类)收敛进 ci-fixes,我再把 ci-fixes merge 进本分支,CI 重跑自然转绿;本 PR 分支不改任何代码(协议:master 级修复必须收敛 ci-fixes,禁止在自己分支重复修)。
  5. 回归风险(按选项): A = 无,无新增变更;B = 无代码变更,仅重跑;C = 无代码回归(红与本 PR diff 无关),但若 EBUSY 家族后被判定为真 bug,本次合并记录缺全绿基线。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。
选项 动作 成本 确定性 协议符合
A(推荐/默认) 等 ci-fixes 落地 EBUSY 修复 → 我 merge 进本分支 时间 确定转绿 ✅
B 再重跑失败分片 ~15 分钟/次 赌 ✅(但无根因进展)
C 带 2 个继承红分片直接合并 无 立即 ⚠️ 破全绿惯例

等你拍板;选 A 的话我这边零动作,ci-fixes 一落地我就接。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant