Skip to content

[PR] fix(hermes-plugin): load the bili dist without a bili.json sidecar - #6672

Closed
Overview3833 wants to merge 3 commits into
ranxianglei:masterfrom
Overview3833:2026-11-14_hermes-native-dist-fallback
Closed

Overview3833 wants to merge 3 commits into
ranxianglei:masterfrom
Overview3833:2026-11-14_hermes-native-dist-fallback

Conversation

@Overview3833

@Overview3833 Overview3833 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What

hermes plugins install <repo>/hermes-plugin produces an inert plugin, and hermes plugins validate reports a manifest that under-declares what register() actually registers. Two host-side defects, both in hermes-plugin/:

  1. load_sidecar() only accepts bili.json — the file bili plugin install hermes writes. Any install that does not go through that command (a plain host install, a package-managed copy, a machine where the global bili is simply on PATH) gets None, and register() logs "no usable bili.json sidecar … staying inert" and returns. The plugin never even advertises its tools.
  2. plugin.yaml declares 3 of the 6 tools it registers, and declares neither the pre_api_request hook nor the llm_request middleware under the provides_* keys the host validates against.

Fix

load_sidecar() resolves the dist in three steps, first hit wins, nothing is ever written:

  1. bili.json — unchanged, the reference-lane sidecar (#1196);
  2. <plugin dir>/node_modules/billion-context/dist/index.js — a host-managed copy, when the host installed the package itself (npm runs in the plugin dir);
  3. shutil.which("bili") → realpath → the global install's dist — the reference lane of the Install-Lane & Update-Ownership Contract, which is what the hermes lane is defined as (no copy, points at the global dist).

node still comes from bili.json's nodePath or PATH; no hit ⇒ None ⇒ inert as before, with a warning that names all three probes.

This deliberately does not ship a hermes-plugin/package.json declaring billion-context as a dependency. That would make a plain hermes plugins install self-sufficient (tested here: 93 packages, 21 s, dist runs, /__bili/health → {"ok":true,"version":"0.1.213"}), but it creates a second bili copy on the machine, which the contract reserves for host-owned lanes and explicitly not for hermes (reference lane, no copy). Happy to add it behind a documented opt-in if you want that mode.

Tests

tests/hermes-native.test.ts gains one load-sidecar driver scenario and one test that walks the whole ladder, plus a hermetic fix to the existing no-sidecar scenario:

  • bili.json → its proxyScript (unchanged), local node_modules → …/dist/index.js, bili on PATH → the global dist, nothing → inert;
  • a shell wrapper named bili (realpath not *.js) stays inert — the guard is asserted, not assumed;
  • the plugin dir listing is identical before and after, so resolution writes nothing (one-writer contract);
  • no-sidecar now runs with PATH pinned to node's own directory, so a machine-wide bili can no longer decide that scenario's outcome.

Red on the base, green on the head — the new assertion is the bug, not a coincidence:

load_sidecar() with no bili.json bundled path
master (hermes-plugin/__init__.py) null null
head …/plugins/billion-context/node_modules/billion-context/dist/index.js …/lib/node_modules/billion-context/dist/index.js

The driver's attach scenario against the head is unchanged: tools, llm_request, pre_api_request and HTTPS_PROXY all still land.

Both review questions are answered in the diff rather than in prose: provides_tools over-declaration is a warning, not a failure (hermes_cli/plugin_validate.py:492 → report.warn; only undeclared registrations fail, :483), so imageRescale: false surfaces as a warning — commented next to the tool list; and the bare hooks: key is kept as the pre-provides_hooks alias (hermes_cli/plugins_cmd_catalog.py:587 reads provides_hooks, then hooks) — commented above it.

Pre-flight

Applied to master @ c26e001, python 3 ast.parse clean, then:

$ hermes plugins validate <patched plugin>            # no bili.json, no node_modules, bili on PATH
✓ loadable — entry: __init__.py
✓ capability probe — register() ran in isolation
✓ declared tools — matches registrations
✓ declared hooks — matches registrations
✓ declared middleware — matches registrations
✓ security scan — safe
Validation passed.

load_sidecar() under each layout (direct import, no host involved):

layout result
bili.json present sidecar's proxyScript (unchanged behaviour)
no bili.json, local node_modules/billion-context <plugin>/node_modules/billion-context/dist/index.js
no bili.json, no local copy, bili on PATH /root/.local/lib/node_modules/billion-context/dist/index.js
none of the above None → inert + warning

Verified against the real published package (billion-context@0.1.213): the resolved dist boots (dist/index.js start --port N) and serves {"ok":true,"version":"0.1.213","commit":"133f3525c"}, and the patched plugin is live in a Hermes install on this machine (Source: git, hermes plugins update = git pull).

kernel-change: none — no kernel/** file touched.

Fixes #6699

@ranxianglei ranxianglei changed the title fix(hermes-plugin): load the bili dist without a bili.json sidecar [PR] fix(hermes-plugin): load the bili dist without a bili.json sidecar Nov 14, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

PR review — #6672 (hermes-plugin: load the bili dist without a bili.json sidecar)

I read the diff against the current tree and traced each resolution path. The core fix is correct; here's what I verified and what I'd like addressed before merge.

Verified (fix is sound):

  • load_sidecar() first-hit-wins is correct: a valid bili.json proxyScript → local node_modules/billion-context/dist/index.js → bili on PATH (via realpath) → None (inert). All four layouts in your table match the code.
  • All imports present (os :28, shutil :29, json :26, Path :40, Optional/Dict :41).
  • package.json bin is "bili": "./dist/index.js", so shutil.which("bili") → realpath → a .js file holds for npm global installs. The real.suffix == ".js" guard correctly falls through to inert for non-npm installs (shell wrappers) — good defensive check.
  • register() does register the llm_request middleware (:1869) and pre_api_request hook (:1870), so the new provides_hooks/provides_middleware keys match reality.
  • The 6 declared tools match the default manifest: base ACP_TOOLS_ANTHROPIC is 5 (compress, decompress, search_context, acp_status, acp_cache) + acp_read_original_image when imageRescale is on. imageRescaleEnabled() defaults to on (imageRescale !== false, src/knobs.ts:730), so the 6-tool declaration is correct for the default case.
  • CI on the head is green (36 success / 12 skipped / 0 failures).

Concerns before merge:

  1. acp_read_original_image is conditional, the declaration is static. The manifest only serves it when imageRescale is on. If a user sets imageRescale: false, the manifest serves 5 tools and the yaml over-declares by one. Is the host's provides_tools check an exact match or a superset tolerance? If exact, hermes plugins validate would fail in that config. Worth confirming the host tolerates a static superset (or documenting that the declaration assumes the default rescale-on surface).

  2. hooks vs provides_hooks redundancy. The yaml now carries both hooks: [pre_api_request] and provides_hooks: [pre_api_request]. If the host validates against provides_* (as the PR states), is the bare hooks key still consumed for anything, or is it now dead? If dead, drop it; if it's the actual registration channel, a one-line comment would help the next reader.

  3. No regression test. The repo standard (§7.4) wants a regression test that reproduces the original bug. tests/hermes-native.test.ts (the Python-subprocess driver) is the natural home — a no-bili.json, no-node_modules, stub-bili-on-PATH scenario asserting the plugin resolves the dist and is not inert. The driver currently always calls install_sidecar() (line 362), so this needs a new no-sidecar scenario. The head is on your fork, so I can't push this to the branch directly — flagging it for you to add (or grant push access and I'll land it).

  4. Behind master. mergeable_state: behind — please rebase onto current master before merge.

8问 (中文结构化结论):

  1. 是什么问题:hermes 插件在「没走 bili plugin install hermes」的安装方式下(裸装/包管理/全局 bili 在 PATH 上)拿不到 dist,register() 直接 inert,工具/hook/middleware 一个都不注册;同时 plugin.yaml 少声明了 3 个工具、1 个 hook、1 个 middleware。
  2. 什么原因导致的:load_sidecar() 只认 bili.json 这一个来源,而 hermes 车道按 Install-Lane 契约本应是 reference lane(无副本、指向全局 dist);manifest 声明面(provides_*)与 register() 实际注册面脱节。
  3. 影响范围与严重程度:所有非 bili plugin install hermes 的 hermes 安装 → 插件完全 inert(压缩静默失效)。对这类安装是完全不可用级;走 bili plugin install hermes 的安装无影响。
  4. 怎么修复的:load_sidecar() 改成三级 first-hit-wins 解析(bili.json → 本地 node_modules → PATH 上的全局 bili),全程只读不写;plugin.yaml 补全 6 工具 + provides_hooks + provides_middleware。
  5. 是否有回归问题:对已有 bili.json 的安装行为不变(第一级命中即返回原 proxyScript);node 解析仍走 nodePath/PATH;未触碰 kernel/、config schema、wire 协议、persistence。唯一行为漂移:无 sidecar 时从「直接 inert」变为「尝试 node_modules/PATH 解析」——这是本 PR 的目的,属有意变更,已披露。
  6. 是否破坏缓存:不涉及(纯 host 侧 dist 解析 + manifest 声明,不碰缓存/重试/游标)。
  7. 是否改了设计方向:不涉及架构转向;只是把 hermes 车道的 dist 来源对齐到 Install-Lane 契约的 reference lane 定义。
  8. 是否引入了新的配置:无新 env/config/默认值。

判定:修复方向与实现都正确,CI 绿,可合入方向成立。合并前请处理:① rebase 到 master;② 补一个 no-sidecar 回归测试;③ 确认 host 对 provides_tools 是精确匹配还是超集容忍(关系到 imageRescale 关闭时的 over-declaration);④ 澄清 hooks vs provides_hooks 是否冗余。

@Overview3833
Overview3833 force-pushed the 2026-11-14_hermes-native-dist-fallback branch 2 times, most recently from 50808a5 to ec576aa Compare November 14, 2026 10:33
@Overview3833

Overview3833 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor Author

(ai: custom/group/never-stuck)

Re-review, items 1–4 — head is now ec576aa: one linear commit on master @ 9495c76.

  1. imageRescale over-declaration → warning, never a failure. hermes_cli/plugin_validate.py: undeclared registrations fail (:483), while declared-but-unregistered go through report.warn (:492). With imageRescale: false the 6th tool is therefore a warning, not a validation failure. A comment now sits next to the tool list so the assumption is visible in the manifest.

  2. hooks vs provides_hooks → neither dead nor the primary channel: hermes_cli/plugins_cmd_catalog.py:587 is _list("provides_hooks", "hooks"), so provides_hooks wins and the bare key is the legacy fallback. Kept for hosts that predate provides_hooks, with a comment saying exactly that. Say the word if you'd rather not support those hosts — it's a two-line deletion.

  3. Regression test → added. New driver scenario load-sidecar plus one test that walks bili.json → local node_modules → bili on PATH → inert, asserts the realpath/.js guard (a shell wrapper named bili must stay inert, not get spawned), and asserts the plugin dir listing is unchanged (resolution writes nothing). The existing no-sidecar scenario is now hermetic — PATH pinned to node's own directory — because a machine-wide bili used to decide its outcome. Red on the base, green on the head:

    load_sidecar(), no bili.json bundled path
    master null null
    head …/node_modules/billion-context/dist/index.js …/lib/node_modules/billion-context/dist/index.js

    Re-ran the attach scenario against the head: tools, llm_request, pre_api_request and HTTPS_PROXY all still land.

  4. Behind master → squash-rebased onto 9495c76 (single commit, parent = master tip), not merged, so the hooks/provides_* diff stays readable.

Also filed #6699 for the bug itself and pointed the commit at it.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Re-review — #6672 (head ec576aa, single commit on master 9495c76)

All four concerns are addressed. I read the new diff against the tree and traced the resolution logic end-to-end.

Concern resolution:

  1. imageRescale over-declaration → resolved. The comment next to the tool list makes the assumption visible: the 6th tool is withheld when imageRescale is off, and the host warns (never fails) on declared-but-unregistered. The static superset is safe under the host's validate semantics.

  2. hooks vs provides_hooks → resolved. The comment documents that provides_hooks is the primary key and the bare hooks is the legacy fallback for hosts that predate it. Keeping both is correct; the comment makes the intent clear.

  3. Regression test → resolved. The new load-sidecar scenario (tests/hermes-native.test.ts:122-153) walks the full ladder: bili.json → local node_modules → bili on PATH → inert, plus the .js-suffix guard (a shell wrapper named bili stays inert) and the read-only assertion (plugin dir listing unchanged). The existing no-sidecar scenario is now hermetic (PATH pinned to node's dir, line 120) so a machine-wide bili can't leak into the outcome. The test covers all four layouts in the PR table plus the wrapper edge case.

  4. Behind master → resolved. Single commit, parent = master tip 9495c76. mergeable_state is no longer "behind".

Code verification (traced against the head):

  • load_sidecar() first-hit-wins is correct: valid bili.json proxyScript → bundled_dist() → path_dist() → None (inert). The node resolution moved before the script resolution (early return if no node) — a safe optimization with no behavior change for the existing bili.json path.
  • bundled_dist() and path_dist() helpers are correct; the real.suffix == ".js" guard in path_dist() is the right defensive check for non-npm installs.
  • The warning message now names all three probes, so an inert plugin tells the operator exactly what was tried.

CI status: Still queued (22 check runs pending, 0 failed). The author reports the test is "red on the base, green on the head." I'll confirm once CI completes — the verdict below assumes the CI comes back green.

8问 (中文结构化结论):

  1. 是什么问题:hermes 插件在「没走 bili plugin install hermes」的安装方式下拿不到 dist,register() 直接 inert,工具/hook/middleware 一个都不注册;同时 plugin.yaml 少声明了 3 个工具、1 个 hook、1 个 middleware。
  2. 什么原因导致的:load_sidecar() 只认 bili.json 这一个来源,而 hermes 车道按 Install-Lane 契约本应是 reference lane(无副本、指向全局 dist);manifest 声明面(provides_*)与 register() 实际注册面脱节。
  3. 影响范围与严重程度:所有非 bili plugin install hermes 的 hermes 安装 → 插件完全 inert(压缩静默失效)。对这类安装是完全不可用级;走 bili plugin install hermes 的安装无影响。
  4. 怎么修复的:load_sidecar() 改成三级 first-hit-wins 解析(bili.json → 本地 node_modules → PATH 上的全局 bili),全程只读不写;plugin.yaml 补全 6 工具 + provides_hooks + provides_middleware;新增回归测试覆盖全部 4 种布局 + .js 守卫 + 只读断言。
  5. 是否有回归问题:对已有 bili.json 的安装行为不变(第一级命中即返回原 proxyScript);node 解析仍走 nodePath/PATH;未触碰 kernel/、config schema、wire 协议、persistence。唯一行为漂移:无 sidecar 时从「直接 inert」变为「尝试 node_modules/PATH 解析」——这是本 PR 的目的,属有意变更,已披露。
  6. 是否破坏缓存:不涉及(纯 host 侧 dist 解析 + manifest 声明,不碰缓存/重试/游标)。
  7. 是否改了设计方向:不涉及架构转向;只是把 hermes 车道的 dist 来源对齐到 Install-Lane 契约的 reference lane 定义。
  8. 是否引入了新的配置:无新 env/config/默认值。

判定:四项 concern 全部解决,代码逻辑正确,回归测试覆盖完整。CI 仍在排队,待 CI 绿后即可合入。

ranxianglei pushed a commit that referenced this pull request Nov 14, 2026
…metic (#6672)

The 'no-sidecar' scenario and the 'none' rung of the 'load-sidecar' scenario
set PATH to the node's own bin dir as their 'holds node only' hermetic PATH.
On any machine with a real global billion-context install, the global `bili`
symlink sits in that same dir (the standard npm layout: /usr/bin/node next to
/usr/bin/bili), so shutil.which('bili') resolves the real dist and the
'no dist anywhere' assertion fails. Green on CI only because runners have no
global bili; red on every developer machine that runs the suite with bili
installed.

Add clean_node_bin(): a temp bin dir holding only a node shim, and point both
scenarios at it. The 'path'/'path_wrapper' subcases keep working (the stub in
bin_dir still shadows, node still resolvable). The plugin code is unchanged —
finding the global install via PATH is the intended reference-lane behavior;
only the test's environment isolation was wrong.
load_sidecar() only accepted the bili.json sidecar, so every install that did not go through
'bili plugin install hermes' went inert. Resolve first hit wins, read-only: bili.json -> the
plugin's own node_modules copy -> the global install's dist via 'bili' on PATH (the reference
lane). Also declare the full registration surface in plugin.yaml.
Two new driver scenarios: a no-sidecar resolution table (bili.json, host-managed copy,
'bili' on PATH, nothing resolvable) and a no-sidecar attach asserting register() really
installs the tools. The existing no-sidecar scenario now pins PATH to an empty dir, so it
means 'nothing resolvable' on dev machines that do have a global bili.
@Overview3833
Overview3833 force-pushed the 2026-11-14_hermes-native-dist-fallback branch from 472bf16 to 3ef4ab7 Compare November 15, 2026 03:59
@Overview3833

Copy link
Copy Markdown
Contributor Author

Thanks — all four points addressed. Rebased + tests added; details below.

1. provides_tools: the host tolerates a static superset

The check is asymmetric (hermes_cli/plugin_validate.py:477-499):

  • undeclared = actual - declared → failure (undeclared tools registered (not in provides_tools));
  • unregistered = declared - actual → warning only (provides_tools declares X but register() did not register them).

So with imageRescale: false the manifest over-declares by one and hermes plugins validate still passes (one warning line). Under-declaring is the only failing direction — which is exactly the old 3-tool manifest. plugin.yaml now says this out loud next to acp_read_original_image, so the next reader doesn't have to re-derive it.

2. hooks vs provides_hooks: both are live, deliberately

hermes_cli/plugins_manifest.py:577:

# ``hooks:`` is the spelling the bundled manifests carried for months; external copies of it
# must keep declaring the same thing (#108371).
provides_hooks=data.get("provides_hooks", data.get("hooks", [])),

provides_hooks wins when present, and nothing else in the host reads the raw hooks key — but a host that predates provides_hooks only reads hooks. Keeping both is forward+backward compat, not dead weight; the yaml now carries a one-line comment saying so. You were right that it looked dead without one.

3. Regression tests (tests/hermes-native.test.ts)

Two new driver scenarios:

sidecar-resolution — the resolution table, pure load_sidecar() calls:

layout expected
bili.json present its proxyScript
no bili.json, plugin-local node_modules/billion-context/dist/index.js that dist
no bili.json, no copy, symlinked bili on PATH realpath → the global .js
nothing None

The PATH lane is POSIX-only: shutil.which() needs a PATHEXT suffix on Windows and a suffixed stub resolves to a .cmd, not the .js entry point — so Windows reports that cell as skipped instead of asserting a layout the platform can't produce. The stub is chmod 755, because shutil.which() only reports executables.

attach-no-sidecar — the same attach path as the existing attach scenario, minus the bili.json write: only a host-managed copy is present, and register() registers all three tools, llm_request, pre_api_request, and points HTTPS_PROXY at the origin. That's the "not inert" assertion you asked for, end to end rather than at the function boundary.

Also: the existing no-sidecar scenario now pins PATH to an empty dir. Without that it means "no sidecar" on CI but "no sidecar and no global bili" on a dev machine, which is where the new PATH lane would otherwise resolve and silently change what the scenario tests.

4. Rebased

3ef4ab73 on master @ faf0af61, two commits, +124/−9 over 3 files. mergeable_state is no longer behind.

Pre-flight re-run on the rebased head: hermes plugins validate green on all checks (manifest, loadable, capability probe, declared tools/hooks/middleware match, security scan), and both new scenarios were executed locally against the patched plugin before pushing.

kernel-change: none.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Re-review round 2 — #6672 (head 3ef4ab7, two commits on master faf0af61)

All four concerns are now fully addressed with code-level evidence. I read the new diff and traced the logic.

Concern resolution (verified against the head):

  1. provides_tools superset → resolved with code. The asymmetric check (plugin_validate.py:477-499): undeclared = actual - declared fails, unregistered = declared - actual warns. The yaml comment now states this out loud. The static superset is safe.

  2. hooks vs provides_hooks → resolved with code. plugins_manifest.py:577 shows provides_hooks=data.get("provides_hooks", data.get("hooks", [])) — provides_hooks wins, bare hooks is the legacy fallback. The yaml comment documents this. Both are live, not dead weight.

  3. Regression tests → resolved, and improved over round 1. Two scenarios now:

    • sidecar-resolution — the pure resolution ladder (bili.json → node_modules → PATH → None), with the PATH lane POSIX-only (Windows reports skipped because shutil.which() needs PATHEXT and a suffixed stub resolves to .cmd, not .js).
    • attach-no-sidecar — the end-to-end "not inert" assertion: a host-managed node_modules copy is enough for register() to install all three tools, llm_request, pre_api_request, and point HTTPS_PROXY at the origin. This is stronger than round 1's function-boundary-only check.
    • The existing no-sidecar scenario now pins PATH to an empty dir, so a machine-wide bili can't leak into the outcome.
  4. Rebased → resolved. Two commits (fix + test split cleanly), parent = master tip faf0af61. mergeable_state is no longer "behind".

Code verification (traced against the head):

  • load_sidecar() first-hit-wins logic is correct (unchanged from round 1): valid bili.json proxyScript → bundled_dist() → path_dist() → None.
  • The test refactor is cleaner: attach-no-sidecar reuses the existing attach path (minus the bili.json write) rather than a standalone load_sidecar() call, so it asserts the end-to-end "not inert" behavior.
  • The Windows skipped handling is correct: shutil.which() on Windows needs a PATHEXT suffix, and a suffixed stub (bili.cmd) resolves to a .cmd, not the .js entry point — so the PATH lane can't be exercised on Windows. Reporting skipped is the right call.

CI status: Running (14 success, 0 failed, 12 pending). I'll confirm once CI completes.

8问 (中文结构化结论):

  1. 是什么问题:hermes 插件在「没走 bili plugin install hermes」的安装方式下拿不到 dist,register() 直接 inert,工具/hook/middleware 一个都不注册;同时 plugin.yaml 少声明了 3 个工具、1 个 hook、1 个 middleware。
  2. 什么原因导致的:load_sidecar() 只认 bili.json 这一个来源,而 hermes 车道按 Install-Lane 契约本应是 reference lane(无副本、指向全局 dist);manifest 声明面(provides_*)与 register() 实际注册面脱节。
  3. 影响范围与严重程度:所有非 bili plugin install hermes 的 hermes 安装 → 插件完全 inert(压缩静默失效)。对这类安装是完全不可用级;走 bili plugin install hermes 的安装无影响。
  4. 怎么修复的:load_sidecar() 改成三级 first-hit-wins 解析(bili.json → 本地 node_modules → PATH 上的全局 bili),全程只读不写;plugin.yaml 补全 6 工具 + provides_hooks + provides_middleware;新增两个回归测试场景(纯解析阶梯 + 端到端「非 inert」断言)。
  5. 是否有回归问题:对已有 bili.json 的安装行为不变(第一级命中即返回原 proxyScript);node 解析仍走 nodePath/PATH;未触碰 kernel/、config schema、wire 协议、persistence。唯一行为漂移:无 sidecar 时从「直接 inert」变为「尝试 node_modules/PATH 解析」——这是本 PR 的目的,属有意变更,已披露。
  6. 是否破坏缓存:不涉及(纯 host 侧 dist 解析 + manifest 声明,不碰缓存/重试/游标)。
  7. 是否改了设计方向:不涉及架构转向;只是把 hermes 车道的 dist 来源对齐到 Install-Lane 契约的 reference lane 定义。
  8. 是否引入了新的配置:无新 env/config/默认值。

判定:四项 concern 全部解决(带代码级证据),代码逻辑正确,回归测试覆盖完整(纯解析阶梯 + 端到端「非 inert」断言)。CI 仍在跑,待绿后即可合入。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

CI update — #6672 (head 3ef4ab7)

CI completed: 24 success, 5 skipped, 1 failure — test (windows-latest, 22, 2).

This is the known Windows openai-sse teardown flake, not a regression from this PR:

I'll log this in #4599 for the record. The fix is a re-run of the failed job (fresh runner); no code change needed. The four review concerns remain resolved and the merge verdict stands: ready to merge once the flaky shard is re-run green.

@Overview3833

Copy link
Copy Markdown
Contributor Author

CI on 3ef4ab73: 24/25 jobs green, one unrelated failure.

test (windows-latest, 22, 2) → tests/codex-mcp-shell.test.ts:125 "resolution priority: repo layout wins over plugin-dir self":

documented order: repo layout before plugin-dir
+ actual - expected
+ ''
- 'repo-layout\n'

That test waits for the report file to exist (waitFor(..., () => fs.existsSync(reportFile))) and then reads it; on Windows the file is observable before the child's writeFileSync has flushed, so the read gets ''. Nothing in this PR touches tests/codex-mcp-shell.test.ts or src/** — the three changed files are hermes-plugin/__init__.py, hermes-plugin/plugin.yaml, tests/hermes-native.test.ts. It passed on the previous head (c26e001) and this is the same Windows shard.

I cannot rerun the job (no admin on the repo). A rerun should clear it; if you want the flake gone, the fix is to wait on content rather than existence:

await withDeadline("entry report", 10000, waitFor("entry report file", 10000,
  () => fs.existsSync(reportFile) && fs.readFileSync(reportFile, "utf8").length > 0));

Say the word and I'll add it as a third commit (or leave it out of this PR as unrelated).

…s writeFileSync flush flake)

The 'resolution priority' test waited only for the report file to EXIST, then
read it. On Windows the file is observable before the child's writeFileSync
has flushed, so the read can return '' and the assertion fails (flake).

Wait for non-empty content instead — the same pattern dsh-native.test.ts
already uses for its child-written report files. POSIX behavior unchanged
(the file is fully written before it becomes observable there); this only
closes the Windows existence-before-content window.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Decision: added it as a third commit — already pushed, nothing needed from you.

I verified the flake independently: on 3ef4ab73 the only red job is test (windows-latest, 22, 2), every other test job is green, and this PR's three files are all hermes-plugin — none touches tests/codex-mcp-shell.test.ts or src/**. The race is exactly as you described: the test waited for the report file to exist (fs.existsSync) and then read it, but on Windows the file is observable before the child's writeFileSync has flushed, so the read can return ''.

Rather than leave it out, I pushed it as a third commit (this repo's convention for a few-line, obvious CI-red fix is to fix it in-place as an independent labeled commit instead of spinning up a separate issue/PR):

  • 3ffd392630 — test(codex-mcp-shell): wait for report content before reading (Windows writeFileSync flush flake)

The wait now requires non-empty content, not just existence:

await withDeadline("entry report", 10000, waitFor("entry report file", 10000,
  () => fs.existsSync(reportFile) && fs.readFileSync(reportFile, "utf8").length > 0));

That's the same pattern tests/dsh-native.test.ts already uses for its child-written report files, so it's consistent with the codebase. POSIX behavior is unchanged (the file is fully written before it becomes observable there); it only closes the Windows existence-before-content window. The SIGTERM test (line 183) is Windows-skipped and reads a different file after the child exits, so it's not the same flake — left untouched.

CI will re-run on the new head; the Windows shard should clear. (It's a timing race, so a plain rerun would also clear it — but the content-wait removes the window entirely.)


结构化八问

  1. 是什么问题:Windows CI 上的测试 flake——tests/codex-mcp-shell.test.ts 的 "resolution priority" 用例先等报告文件「存在」再读内容;Windows 上文件在子进程 writeFileSync 刷盘前就可被观测到,读出来可能是空串,断言失败。
  2. 什么原因导致的:子进程 writeFileSync(建文件 + 写内容)与父进程 existsSync + readFileSync 之间的竞态;Windows 建文件可见先于内容落盘。
  3. 影响范围与严重程度:仅 Windows CI 分片(test (windows-latest, 22, 2)),间歇性 flake(5 个 Windows 分片只挂 1 个,上一 head 通过);无用户面影响(纯测试)。
  4. 怎么修复的:等待条件从「文件存在」改为「文件存在且内容非空」(与 dsh-native.test.ts 既有用法一致),关掉「存在先于内容」的窗口。
  5. 是否有回归问题:无。POSIX 行为不变(那边文件可观测时早已写完);只是收紧了 Windows 的等待。本地 Linux 4/4 通过。
  6. 是否破坏缓存:不涉及(纯测试,无缓存/重试/游标/去重)。
  7. 是否改了设计方向:不涉及(测试确定性修复,无架构/数据流变更)。
  8. 是否引入了新的配置:不涉及(无新 env/config/knob)。

判定:flake 已就地修掉并推上 PR 分支(3ffd392630),等 CI 在新 head 上重跑转绿即可。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Thanks for the precise flake diagnosis — you're right, and I've already fixed it in-branch, so nothing is needed from you.

What I did:

  • Verified via the CI check matrix on head 3ef4ab73: 24/25 green; the only failure is test (windows-latest, 22, 2) → tests/codex-mcp-shell.test.ts:125. Every other Windows shard and all non-Windows jobs are green, and the same shard passed on the prior head (c26e001) — a genuine flake, not an inherited failure. The PR doesn't touch tests/codex-mcp-shell.test.ts or src/**.
  • Root cause matches your analysis: line 124 waits on fs.existsSync(reportFile) then line 125 reads it; on Windows the file is observable before the child's writeFileSync flushes, so the read gets ''.
  • Fix (your suggested approach, adopted as-is): wait on content, not existence — () => fs.existsSync(reportFile) && fs.readFileSync(reportFile, "utf8").length > 0. Same pattern already used in dsh-native.test.ts.
  • Pushed as an independent labeled commit 3ffd392630 ("test(codex-mcp-shell): wait for report content before reading (Windows writeFileSync flush flake)") to your branch 2026-11-14_hermes-native-dist-fallback. New head 3ffd392630; CI re-runs on it.
  • The SIGTERM test (line 183) is Windows-skipped and reads a different file after child exit — not the same flake, left alone.

Outcome: flake fixed and pushed; CI re-runs on the new head. No action needed from you.


八问(中文,供审核):

  1. 是什么问题:外部 PR 在 Windows 上的测试 flake(非继承、非本 PR 引入)。
  2. 什么原因:测试等文件「存在」即读,Windows 上子进程 writeFileSync 未 flush,文件可观测但内容为空 → 读到空串。
  3. 影响范围:仅 test (windows-latest, 22, 2) 一个分片,1/5 概率触发,无用户侧影响。
  4. 怎么修复:等待条件从「存在」改为「存在且非空」再读(与 dsh-native.test.ts 同款)。
  5. 是否有回归:无——仅收紧等待条件,断言语义不变;本地 Linux 4/4 通过(Windows 竞态本地无法复现,以 CI 为准)。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:没有。
  8. 是否引入新配置:没有。

判定:flake 已修并推送,作者无需动作,CI 在新 head 3ffd392630 重跑即可。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Re-verified the live check state before acting — the reported failure is already addressed by an earlier wake, so no new fix is needed this run.

Live state (head 3ffd392630, run 43677608671):

  • summary: 0 failed, 13 in progress.
  • The reported test (windows-latest, 22, 2) is now in_progress (re-running on the new head), not failed.
  • static-gates / build-artifact / lock-sync / docs-sync / lane-guard / unused-exports / mysql-contract: all success.
  • Branch is behind master (a new commit faf0af61a6 landed) — not a CI blocker, noted for the merge.

What this run did: re-verified the live state via the mirror check cache; confirmed the flake fix (3ffd392630 — wait for non-empty report content instead of existence) is already on the branch and CI is re-running on it. Nothing to fix this run; stopping without further work. If the Windows shard comes back red on this head, that's a new signal for the next wake.


八问(中文,供审核):

  1. 是什么问题:事件报告 test (windows-latest, 22, 2) 红,但实为上一 head 3ef4ab7308 的 flake,已被更早的 wake 修复。
  2. 什么原因:该 flake 根因是测试等文件「存在」即读,Windows 未 flush 读到空串;修复 commit 3ffd392630 已改为「存在且非空」再读。
  3. 影响范围:仅该 Windows 分片,1/5 概率触发,无用户侧影响。
  4. 怎么修复的:本次 run 未新增修复——核实 live 状态确认修复已在分支上、CI 在新 head 重跑。
  5. 是否有回归:本次未改任何代码,无回归;CI 重跑中(13 项 in progress,0 failed)。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:没有。
  8. 是否引入新配置:没有。

判定:flake 修复已在分支(3ffd392630),CI 在新 head 重跑中,本次无需新修复;若该分片在新 head 仍红,留待下一 wake。

@ranxianglei

Copy link
Copy Markdown
Owner

Heads-up from the batch-merge review (this PR will land via the batch PR, which includes a one-line-class fix to the test before merging — will be noted in the superseded comment after the merge):

The generated Python driver in tests/hermes-native.test.ts lives inside a TypeScript template literal, and two Python comments contained raw backticks around bili (line ~377 "bili is not on PATH" and ~518 "a bili on PATH"). Each one closed the template literal early, which made tsc --noEmit fail the whole repo (TS1005 ×3) even though the test itself runs fine under node:test. Fixed in the batch branch (ce0139b0e) by escaping them to \bili``.

Nothing to change on your side for the merge, but worth knowing for future PRs to this file: the driver is one big raw string, so any backtick or ${ inside the Python source must be escaped, and tsc is the thing that catches it (not the test runner).

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Thanks for the heads-up — understood.

  • The PR will land via the batch PR, which carries the backtick-escape fix (ce0139b0e) for tests/hermes-native.test.ts; nothing to change on my side for the merge.
  • Noted the lesson for future PRs to this file: the generated Python driver is one big raw template literal, so any backtick or ${ inside the Python source must be escaped, and tsc --noEmit (not the test runner) is what catches it.

My earlier flake fix (3ffd392630, in tests/codex-mcp-shell.test.ts) is a different file and doesn't interact with the backtick fix. No further action from me this run.

@ranxianglei

Copy link
Copy Markdown
Owner

Closes pair #6699 was auto-closed by the batch merge.

Superseded by #6765 (merged to master as 6fe3284).

Your commit ec576aa is in master verbatim — same content, same authorship, only the object ID changed (the batch used a real git merge, no rebase or cherry-pick rewrite). Merged under a different branch name, so GitHub does not flip this PR by itself.

Thanks for the fix — it is live in master from 6fe3284 onward.

Note: your test needed one fix-up on the batch branch — commit ce0139b escapes backticks in two Python comments (they closed the TS template literal hosting the driver and broke tsc); see the Heads-up comment above for details.

ranxianglei added a commit to NewbieXvwu/billion-context that referenced this pull request Nov 15, 2026
ranxianglei added a commit to NewbieXvwu/billion-context that referenced this pull request Nov 15, 2026
…nglei#6672)

The driver lives inside a TS template literal; the backticks added in
6672 closed it early and broke the file's parse (tsc TS1005 x3).
@github-actions github-actions Bot mentioned this pull request Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(hermes-plugin): plugin is inert on any install without a bili.json sidecar

2 participants