Skip to content

[PR] fix: verif solver-less entry guard must return — clean skip instead of skip-with-error - #4272

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-22_verif-guard-skip-return
Oct 23, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-22_verif-guard-skip-return

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4270. Part of the #4140 instance-3 family (solver-less runs emitting malformed TAP lines).

Problem

node:test's t.skip() only marks the test skipped — it does not stop the test body. The two verif harnesses that used a bare t.skip(...) in their entry guard therefore kept executing after the skip and threw at the first real solveSmt2 call:

The throw is not itself a path bug: it is simply the first consequence of the body continuing to run (locally: missing build artifact; on windows CI: doubled-drive ENOENT per #4195; in this repro: no solver available).

Fix

2 lines: add the missing return at both entry guards, matching the idiom already used by the other seven solver-gated verif tests (verif-bug3691-oscillation:36, verif-symbolic-smoke:42/58/81, verif-bug3676-wire-reorder:106, verif-induction-append-stability:51, verif-bug3769-orphan-window:53):

  • tests/verif-generic-append-stability.test.ts:50
  • tests/verif-responses-ingress.test.ts:52

Behavior changes (old → new)

  • Solver-present environments: none. Guard evaluates true; execution path byte-identical (verified: both tests pass against a real cvc5 before and after).
  • Solver-less environments: these two tests now emit a clean skip line only (ok … # SKIP no SMT solver available, # fail 0 / # skipped 2) instead of a hybrid skip+error entry. Intended log-shape fix in the degraded path only; effective coverage is unchanged either way (the bodies were never actually running there).

Relationship to sibling tickets

Verification (local, targeted)

  • Reproduced pre-fix with the solver ladder pointed at nonexistent paths: both tests rendered not ok … # SKIP plus an uncaught no solver available stack at tests/verif-generic-append-stability.test.ts:141 and tests/verif-responses-ingress.test.ts:157.
  • Post-fix, same environment: ok … # SKIP no SMT solver available ×2, no error block, # fail 0 / # skipped 2.
  • Solver-present: both tests pass against a real cvc5, identical to pre-fix behavior.
  • npm run typecheck green. Full-suite verdict = remote CI.

No new config/env/CLI surface. Test-harness-only change.

@ranxianglei ranxianglei changed the title fix: verif solver-less entry guard must return — clean skip instead of skip-with-error [PR] fix: verif solver-less entry guard must return — clean skip instead of skip-with-error Oct 22, 2026
@Zhishulo

Copy link
Copy Markdown
Contributor

去重提示:同一处 2 行改动已先由 #4271 提出(384bc12,base 0a9c639,内容与 #4272 逐字节相同),#4272 是 #4270 自动分诊分支。两份等价,合一个即可;若合本 PR,#4271 我这边可关。hunk 与 #4269 不重叠(generic :50/#52 vs :139+),可独立合入。

…e body run into skip-with-error

node:test's t.skip() only marks the test skipped; it does not stop the
test body. The two verif harnesses that used a bare t.skip() in their
entry guard therefore kept executing after the skip and threw at the
first real solveSmt2 call, emitting a hybrid 'not ok … # SKIP' TAP line
with an uncaught error stack (#4140 instance-3 malformed line) even
though the tally counted them as skipped.

Add the missing return at both guards, matching the idiom already used
by the other seven solver-gated verif tests. Solver-present runs are
byte-identical (guard is true, body path untouched).

Closes #4270
@ranxianglei
ranxianglei force-pushed the 2026-10-22_verif-guard-skip-return branch from 072a9ee to 1b8e651 Compare October 22, 2026 06:24
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 本 PR 修 #4270(#4140 instance-3 家族的入口守卫面)。与兄弟单机制不同、互补:#4195(win32 双盘路径 join,df5e82412 已合入 master)、#4269(测试中段 solver 丢失加固,open PR,hunk 与本 PR 不重叠)。近期无其他 PR 触碰同一对 guard 行。

标签: bug + scope/ci + p3(测试脚手架日志形状缺陷:无求解器时产出 skip-with-error 混合条目;仅影响降级路径,有效覆盖两种环境下都不变。)

独立复审结论(非复读 issue):

验证过什么

  1. 根因成立,本地独立复现。 在 master(0a9c639)上把 solver ladder 指向不存在的路径跑这两个测试:两者都输出 not ok … # SKIP no SMT solver available … + no solver available 错误块——正是混合条目(本地 spec reporter 的 ✖ failing tests: 段 / CI TAP 里掉出所有统计桶的那行)。修复后同环境:干净 ok … # SKIP ×2,# fail 0 / # skipped 2,无错误块。
  2. 惯用式一致性。 修后全仓 9 处 no SMT solver available 入口守卫统一为 return t.skip(...);其余 7 处原本就正确(verif-bug3691-oscillation:36、verif-symbolic-smoke:42/58/81、verif-bug3676-wire-reorder:106、verif-induction-append-stability:51、verif-bug3769-orphan-window:53)。
  3. 回归清点。 diff 仅两个测试文件的入口守卫(2 文件 × 1 行,t.skip → return t.skip)。有求解器路径:guard 求值 true,执行路径逐字节相同——本地对真实 cvc5 1.4.2 实测两测试通过(pass 2 / fail 0)。未触生产代码、wire、配置面、kernel/。
  4. stale base 已处理。 审时发现分支落后 master 9 个提交,其中含 Windows: verif symbolic driver path gets a doubled drive letter (D:\D:\a\...) — solver-less CI renders skip-with-error #4195 的 fileURLToPath 修复(改的是这两个文件里的 REPO 路径行,与 guard 行不重叠)。已 rebase 到当前 master(0a9c639)并推送,新 head 1b8e651ca;diff 仍恰好 2 行,rebase 后本地双环境复验(有 solver pass 2 / 无 solver skipped 2,均无错误块)。
  5. 测试有效性。 本变更是 harness 修复而非新断言;测试自身的断言牙(Q1/Q2/Q3 SAT/UNSAT + 真代码回放)不变,有求解器时仍完整生效。

行为变更披露(旧→新):仅无求解器环境——混合 skip+error 条目(本地 failing-tests 段 / CI not ok … # SKIP 畸形行)→ 纯 skip 行(ok … # SKIP,# fail 0 / # skipped 2)。有意为之的降级路径日志形状修复;有效覆盖不变(那些环境下 body 本来就没真正跑过)。有求解器环境零变化。

八问

  1. 是什么问题:无 SMT 求解器时,这两个 verif 测试「标记了 skip 但继续执行」,产出 skip+error 混合条目(CI 上即 Windows CI test shards go red randomly: two flake signatures (cleanup rmdir race in #1952 lease test, egress-fuse e2e hang in #4079) #4140 instance-3 的畸形 TAP 行)。
  2. 什么原因导致:node:test 的 t.skip() 只标记状态、不停止 body;入口守卫漏了 return,body 继续跑到第一个真实 solver 调用处抛错。
  3. 影响范围与严重程度:仅无 solver 环境(多数开发机、部分 CI runner);纯日志形状缺陷,测试结论本身不受影响;p3。
  4. 怎么修复:两处入口守卫补上缺失的 return,对齐其余 7 个 solver-gated 测试的既有惯用式。
  5. 回归问题:无。有 solver 路径逐字节相同(实测);未触任何生产行为。
  6. 缓存破坏:不涉及。
  7. 设计方向:未变。
  8. 新配置:无(env/config/CLI 面均未动)。

与 #4269 的关系确认:两者 hunk 不重叠(本 PR:generic 入口 :50 / ingress 入口 :52;#4269:generic 中段 ~:141+),合并顺序相互独立,#4195 已在 master 就位——三个凑齐即该家族闭环。

状态:rebased head 1b8e651ca 的 CI 正在跑(test shards 进行中)。判定:可合并(待 CI 绿确认后跟进一条)。

@ranxianglei ranxianglei added bug Something isn't working p3 scope/ci labels Oct 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-22_verif-guard-skip-return (1b8e651)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5079-pr.4272.d3e1fdd

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4272.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Oct 22, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

来源: PR #4272 (#4272) 分析 CI 失败时发现

新实例 — issue1452 socket-hardening 的 kat-reap 子测试墙钟截止被高负载击穿

现象:run 39735646915(PR #4272 head 1b8e651ca,ci:full 全矩阵)job test (windows-latest, 22, 2)(job id 119681172008):2456 tests → pass 2442 / fail 1 / skipped 13。失败用例:lifecycle ledger + kat knob: idle keep-alive close classifies reason=idle-timeout as clean FIN (#1452),tests/issue1452-socket-hardening.test.ts(超时点 :281);error: kat reaper did not FIN the idle socket within 6000ms (kat=300ms)。

证据链(负载误报,非 reap 缺失):

  1. 失败后紧邻的服务端账本行:tcp closed reason=idle-timeout age=6269ms reqs=1 —— reap 确实触发了,只是落在 6269ms,超 6000ms 截止 269ms(约 4.5%);
  2. 测试自述(:278):正常落点 ≈kat+1s ≈1300ms ±40ms —— 本次 ≈4.8 倍慢,典型 runner 负载拖慢;
  3. 同一日志多条 another bili instance is running(无单实例守卫:多个 bili 代理进程并存共写 sessions 目录,伴随孤儿 ref、有注册表无 transcript、计数自相矛盾的污染簇 #394 共享运行器并发实例告警),佐证该时刻运行器高负载;
  4. 同分片重跑(attempt 2)全绿 → 偶发确认。

家族归属:与 #3420/#3449/#1607/#1697/#1770 同族(issue1452 文件内墙钟假设在高负载 Windows runner 上被击穿)。#3421+#3435 已把先中招的 retired-stall-guard 子测试(:411+,首字节等待)改为事件循环 tick 计数(对「runner 冻结时墙钟预算与 round-trip 一起冻结」免疫),但本文件的其余墙钟预算仍未转换——本次命中的 :281(kat-reap 6000ms setTimeout)即其一;同族还有 :118/:169/:218/:319/:468/:476/:519/:532 等。

与本 PR 无关性:PR #4272 仅给两个 verif 测试入口守卫补 return(test-harness-only),不存在影响 socket 时序的机制路径;失败纯属负载偶发落在其头上。

建议修复方向(排期由 owner 定):按 #3435 同款做法把 kat-reap 子测试(:265–283)的等待从墙钟 setTimeout 改为 tick 计数/相对进度截止(或至少放宽预算并注释负载余量依据);其余未转换点位可一并清点。不建议在本 PR 顺手改——socket 加固时序面是多次迭代过的敏感区(#1562→…→#3435 一长串),且本 PR 应保持单 scope。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 终审跟进(上条 review 承诺的绿确认):

  • 全矩阵(run 39735646915,head 1b8e651ca,ci:full):30 jobs = 29 success + 1 skipped(red-signal,设计内跳过)= 全绿。 12 条 test 腿(ubuntu/windows × 22/24 × 3 分片)+ 全部 guard/sync/static-gates 均 success。
  • e2e 车道全绿(同一 head):local-registry、native-dsh、native-pi、native-opencode、pi-codex-ws、codex-persona、fake-codex、windows codex overlay。
  • 唯一波折:test (windows-latest, 22, 2) 首跑红在 与本 PR 无关的 issue1452 socket-hardening kat-reap 时序用例(负载误报,reap 实际落点 6269ms vs 6000ms 截止;证据链与处置见本线程上一条及已记入 Windows CI test shards go red randomly: two flake signatures (cleanup rmdir race in #1952 lease test, egress-fuse e2e hang in #4079) #4140 清单的新实例);同分片重跑 attempt 2 绿,偶发确认。

最终判定:可合并。 代码面 2 files × 1 line(回归清点见首条 review);行为变更仅降级路径日志形态(solver-less 环境由 skip+error 混合条目 → 干净 skip 行,solver-present 字节级不变);无新配置/缓存/设计方向变化。合并按规则留给人工:#4272

@ranxianglei
ranxianglei merged commit d5fc17b into master Oct 23, 2026
154 of 192 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.748-master.d5fc17b (rolling master tag).

npm install -g billion-context@0.0.748-master.d5fc17b  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR p3 scope/ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verif: guard-level solver skip never returns — solver-less runs still emit a failing-test entry instead of a clean skip

2 participants