Skip to content

[PR] fix: validate Claude watchdog health payloads - #7220

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_validate-claude-watchdog-health
Nov 20, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-18_validate-claude-watchdog-health

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Fixes #7218.

Problem

The Claude MCP watchdog accepts any successful HTTP status as a healthy proxy. An HTML page or a JSON {"ok":false} reply with status 200 resets its consecutive-failure counter, so recovery never runs and MCP tools remain pinned to the unusable listener.

Change

Require a successfully decoded health payload with ok === true. The existing 2,000 ms AbortSignal covers both fetching the response and reading its body. Non-success statuses, invalid payloads and incomplete body reads return false. This keeps the current consecutive-failure threshold and existing recovery/settings behavior, without adding a TCP pre-probe or a second timeout budget.

Validation

  • 15 new real-HTTP cases: 11 fail / 4 pass on unmodified master; all 15 pass with the change. Invalid responses include HTML, false/missing/non-boolean flags, null/primitive/malformed JSON and an empty 204 response.
  • 36 related tests pass, including the complete existing Claude MCP entry suite covering posture gating, failure/reset behavior, origin drift and managed settings.
  • Recovery cases use the default health probe against actual local HTTP listeners. After the third failure, the existing settings writer repins an isolated settings file and an actual forwardTool request reaches the replacement listener with the expected arguments and conversation ID. Only the launcher result is injected; no real Claude process is started.
  • Healthy JSON with either JSON or plain-text content type stays on the original listener. HTTP 503 with an ok payload remains unhealthy. A valid health response resets the consecutive-failure counter. Separate stalled-header and stalled-body cases exercise the original timeout through real sockets.
  • npm run typecheck, npm run build, npm run check:test-import-graph and git diff --check pass. The CI process-test classifier reports no unregistered spawners across 1,358 test files.
  • E2E_CHECK=1 finds the Codex CLI and built dist, but the configured upstream at 127.0.0.1:8199 is unreachable. Real-model Codex phases have not run. Remote full-suite CI and the real-model merge gate remain outstanding.

Prior work

Compared all 274 live fork branches and 89 upstream history commits for src/claude-mcp-entry.ts (five complete file versions). The existing watchdog probe remains status-only. Actual patches in #1874, #3673, #6049, #3825 and #5242 were checked; shared three-state liveness/body-timeout changes do not update this private watchdog probe.

@ranxianglei ranxianglei changed the title fix: validate Claude watchdog health payloads [PR] fix: validate Claude watchdog health payloads Nov 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复. The PR independently compared 274 live fork branches + 89 upstream history commits for src/claude-mcp-entry.ts (all five complete file versions); the shared three-state liveness / body-timeout changes (#1874, #3673, #6049, #3825, #5242) do not touch this private watchdog probe. No prior fix for this specific defect.

Verdict: APPROVE (pending full-suite CI green). I independently re-verified the fix, the tests, and the regression surface at PR head 7e69d5b0 (worktree) — not just the PR's claims.

What I verified:

  • Fix is correct against the real contract. The bili health endpoint (src/server/admin.ts:299-316) returns 200 {"ok": true, ...} when ready and 503 {"ok": false, reason:"starting"} while booting. Requiring a decoded payload?.ok === true matches that contract exactly; the old res.ok alone let a foreign HTTP-200 listener (dev SPA HTML, {"ok":false} JSON) read as healthy and reset the failure counter, so recovery never ran.
  • Pure tightening, no regression. The change only turns previously-"healthy" 200s into failures for the invalid-payload cases (the bug). It never turns a failure into healthy: healthy bili (200 ok:true)→true, booting (503)→false, dead (refused)→false, timeout/busy→false (unchanged).
  • Tests are real, not tautological. 15 real-HTTP cases (local http.Server, real settings writes, real forwardTool calls). With the fix: all 15 pass. With the master version of claude-mcp-entry.ts swapped in: exactly 11 fail / 4 pass — matching the PR's claim. The 4 that pass on master are the 2 valid-payload cases, the 503 case (res.ok already false), and the headers-stall case (fetch itself times out); the body-stall case is the distinguishing one (master reads res.ok true once headers arrive; the fix's res.json() times out on the incomplete body).
  • Existing suites green: claude-mcp-entry.test.ts 21/21; watchdog-health + watchdog-health-daemon + zcode-mcp-entry 11/11. npm run typecheck exit 0. git diff --check clean.
  • CI: static gates all passed; full test shards (ubuntu/windows × node22/24) still in the runner queue (PR ~8h old). mergeable_state=blocked is normal while CI runs.

One non-blocking observation (DRY, not a blocker). This is now the 3rd inline copy of the res.ok && payload?.ok === true check — the other two are probeLiveness (src/liveness.ts:101, three-state with a TCP pre-probe) and fetchHealthInfoDefault (src/launcher.ts:4046, full HealthInfo). The PR deliberately did NOT reuse probeLiveness: the watchdog's probe seam is a boolean (not the three-state verdict), and it avoids adding a TCP pre-probe / second timeout budget to the 30s tick. That's a sound call — the recovery path (ensureProxyRunning → probeLiveInstances, src/launcher.ts:4243) already carries the #3654/#4811 busy≠dead handling, so a busy proxy is kept as a candidate and not double-spawned. A future pass could extract a small shared healthyPayload(res) helper, but the differing contracts (boolean / three-state / full-info) make it a separate refactor, not part of this fix. Noted, not blocking.

Suggested labels (judged on this PR's own diff, not inherited from #7218): bug + p2. It's a real defect (the watchdog's recovery is defeated), but the trigger is narrow — the proxy must die mid-session AND its old port must be reused by a foreign 200-serving service — and the impact is session-scoped with a workaround (restart Claude re-runs the SessionStart hook). #7218 suggested p1; I read it p2 on the narrow trigger, but the owner can re-judge.

Auto-merge gate (§7.4): meets all eight (single-module, regression test reproduces the bug, no config/wire/kernel/persistence change, pure fix:, clean diff, Fixes #7218, no load-bearing infra) — pending full CI green. Merge stays human-only.


结构化八问

  1. 是什么问题:Claude MCP 看门狗的默认健康探测只判 res.ok(HTTP 200),把"HTTP 2xx"误当"bili 代理健康"。旧端口被外部服务(开发 SPA 的 HTML、{"ok":false} JSON)占上后,探测读成"健康",连续失败计数每 tick 归零,恢复/重绑永不触发,该会话的 MCP 工具(acp_status/compress)被钉死在不可用监听器上直到会话结束。
  2. 什么原因导致的:根因是探测函数 probeProxyHealth(src/claude-mcp-entry.ts:106)只返回 res.ok,没有校验健康载荷。bili 的健康契约是 200 配 {"ok":true}、503 配 {"ok":false},单看 200 不足以证明是健康的 bili 代理——任何外部监听器都能对任意路径回 200。探测把"HTTP 2xx"和"bili 健康"混为一谈。
  3. 影响范围与严重程度:仅 Claude Code 原生 MCP 车道。触发条件窄:代理必须在会话中途死掉,且旧端口随后被一个回 200 的外部服务占用。一旦触发,该会话的 MCP 工具在重启 Claude 前不可用(看门狗是唯一的会话中途恢复机制,模型通道走静态烘焙的 base URL,无其他重绑路径)。属"特定条件下有问题"。
  4. 怎么修复的:要求健康探测拿到成功解码的载荷且 ok === true(严格布尔),复用既有的单个 2000ms AbortSignal(同时覆盖取响应头与读 body)。非 2xx、HTML、malformed JSON、空 204、body 读不完,一律返回 false(失败),让既有的连续失败阈值与恢复/重绑路径按设计运行。最小改动:不加 TCP 预探测、不加第二份超时预算、不动阈值与恢复行为。
  5. 是否有回归问题:无。改动只收紧"健康"判定——只把原本误判为健康的 200(无效载荷场景)转为失败,绝不把失败判成健康。已逐 case 核对:健康 bili(200 ok:true)→true、启动中(503)→false、死掉(连接被拒)→false、超时/busy→false(与改前一致)。既有 21 项 claude-mcp-entry 套件 + 相关 watchdog 套件全绿。
  6. 是否破坏缓存:不涉及。这是 Claude MCP 车道的代理存活看门狗,与 LLM 前缀缓存/重试/游标/去重键无关。
  7. 是否改了设计方向:无。同一 boolean 探测缝、同一 30s tick × 3 次失败阈值、同一恢复/重绑路径,只有"健康"谓词按真实契约收紧。
  8. 是否引入了新的配置:无新 env/config/默认值。

收工判定:可合(待全量 CI 绿)。 干净、正确、最小、测试扎实的根因修复。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

CI update: full suite now green. All test shards completed success (ubuntu-latest 22/24 × shards 1-3, windows-latest 22 × shards 1-5), plus static-gates (ubuntu+windows), test-import-graph, and every static gate (gate, config-surface-guard, lock-sync, architecture-docs, lane-guard, unused-exports, docs-coverage, lifecycle-ledger-guard, docs-sync, pi-subagents-guard, version-guard, kernel-guard, build-artifact, mysql-contract, plan). The real-model e2e lanes (codex/pi/dsh/opencode/OpenCode-V2-WS) and the hermetic registry e2e are skipped in this run — they're gated behind their env vars and run in the separate real-model merge gate.

The PR now meets the full §7.4 auto-merge gate (including "full-suite green on the pushed head"). Review verdict stands: APPROVE — ready for human merge. Merge stays human-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude watchdog accepts HTTP-200 HTML or not-ok health replies and skips recovery

2 participants