fix(runtime): add bounded retry to managed node version probe - #771
Merged
Conversation
added 2 commits
August 4, 2026 14:10
A single transient external-process failure during managed Node validation (process fails to start, a non-zero exit such as npm exit code 7, or non-semver output) was latched into a permanent bundled_resource_invalid failure event, surfacing a misleading "installation incomplete / reinstall / check antivirus" alert even though the bundled runtime was healthy and activated moments later. Wrap command_version's single probe (command_version_once) in a bounded retry helper (probe_command_version_with_retry): 3 attempts with 250/500/1000ms backoff (~1.75s worst case), aligned with activate_copy_with_retry. All three transient error classes are retried; real failures (node major too low) are decided in validate_runtime, outside the probe, so they are never retried. A warn is logged only when a retry will follow; the budget-exhausted final failure is left to the existing caller warn to avoid duplicate logging. command_version keeps its signature, so node/npm/npx call sites are unchanged. classify_error and AionUi are intentionally untouched.
The four version_probe_* tests all inject a closure into
probe_command_version_with_retry, so command_version_once never runs and
the retry is never exercised through the call chain. If the retry wrapper
were detached from command_version (calling command_version_once
directly), all four would still pass — the regression that matters is
invisible to them. AC1's second half ("does not report phase=Failed via
the reporter") had no coverage at all, even though that event is what
makes AionUi show the installation-integrity alert.
Add two #[cfg(unix)] tests driving the real chain — spawn a process, get a
real exit code 7, through command_version -> validate_runtime ->
validate_managed_runtime -> the bundled activation path — sharing one
fixture: a minimal bundled managed-Node tree whose bin/npm exits 7 for
the first N invocations and then prints a version. The attempt counter
lives in a file because every probe attempt is a fresh process, as in
production.
- transient_npm_version_failure_is_absorbed_without_failed_report:
2 failures inside the budget -> Ok, version 24.11.0, source Bundled,
exactly 3 npm invocations, and zero Failed phases on the reporter.
Verified as a real guard: with VERSION_PROBE_ATTEMPTS forced to 1 it
fails with this issue's own error, "bundled Node runtime failed
validation ...: npm exited with exit status: 7".
- persistent_npm_version_failure_still_reports_bundled_resource_invalid:
failure outlasting the budget -> spends exactly VERSION_PROBE_ATTEMPTS,
still errors, and still reports bundled_resource_invalid, so the fix
cannot swallow a genuinely broken install.
Unix-only because the fixture needs executable shell scripts as
node/npm/npx stand-ins; the retry logic is platform-independent and the
version_probe_* tests cover it everywhere. Product code is untouched.
kaizhou-lab
pushed a commit
that referenced
this pull request
Aug 4, 2026
🤖 I have created a release *beep* *boop* --- ## [0.1.58](v0.1.57...v0.1.58) (2026-08-04) ### Features * **conversation:** agent-driven session auto-naming (ACP session_info_update + claude generate_session_title) ([#768](#768)) ([c551c66](c551c66)) * **conversation:** fork a conversation into a new one at a chosen message ([#772](#772)) ([bb7c14e](bb7c14e)) * **fs:** add ChatFileRef content endpoints ([#757](#757)) ([e8b6f4c](e8b6f4c)) * **fs:** pdf stream endpoint + office ChatFileRef resolve + retire fs/resolve & WS fs/read ([#762](#762)) ([e4e991a](e4e991a)) ### Bug Fixes * **agent:** keep the thought-level picker on a resumed conversation ([#763](#763)) ([dcf015e](dcf015e)) * **runtime:** add bounded retry to managed node version probe ([#771](#771)) ([1dee9f1](1dee9f1)) * **session:** keep claude session cost cumulative across process respawns ([#767](#767)) ([f498e9d](f498e9d)) * **session:** settle cards through teardown and resume so no stored row spins forever ([#766](#766)) ([3f61cc4](3f61cc4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a false "AionUi installation incomplete / reinstall / check antivirus" alert triggered by a single transient external-process failure during managed Node validation. In the reported incident,
npm --versionexited with code 7 once; the bundled runtime was actually healthy and activated ~38.8s later, so the alert was misleading and the reinstall/antivirus guidance was useless to the user.Root cause:
command_versionran the managed Node--versionprobe once with no retry, so a transient exit was latched into a permanentbundled_resource_invalidfailure event that AionUi surfaced to the user and reported to Sentry.Fix: Bounded retry on the
--versionprobe, scoped entirely to AionCore. On retry success,install_and_validate_with_reporterreturnsOkand never broadcastsphase=Failed, so the UI never shows the alert and never reports.classify_errorand AionUi are intentionally untouched — this is Option 1 from the approved fix spec.Changes
crates/aionui-runtime/src/node_runtime/mod.rs: splitcommand_versioninto a thin wrapper plusprobe_command_version_with_retry(generic, closure-injected, unit-testable, no process spawn) andcommand_version_once(the original body, verbatim). The public signature ofcommand_versionis unchanged, so all node/npm/npx call sites are untouched. New private constantsVERSION_PROBE_ATTEMPTS = 3andVERSION_PROBE_BACKOFFS = [250, 500, 1000]ms(~1.75s worst case), independent of the existing copy-activation retry budget. Adds 4 inline unit tests that inject a closure and never exercisecommand_version_once.crates/aionui-runtime/src/node_runtime/managed/tests.rs: adds two#[cfg(unix)]tests that spawn real shell-script stand-ins for node/npm/npx (npm exits 7 for the first N invocations, then prints a version) and drive the real call chaincommand_version -> validate_runtime -> validate_managed_runtime -> bundled activation path. One covers a transient failure inside the retry budget (expects success, zeroFailedphases reported); the other covers a persistent failure beyond the budget (expectsbundled_resource_invalidstill reported). Test-only, no product code changes.Scope boundaries (intentional)
classify_error(managed.rs) and all AionUi files are deliberately unchanged. Persistent failures (probe failing for the full ~1.75s budget) still classify asbundled_resource_invalidand still alert/report, to avoid under-reporting real failures.activate_copy_with_retry; a persistent failure spends the full ~1.75s before the verdict. Only transient error classes (process fails to start, non-zero exit, non-semver output) reach the retry loop — "node major too low" is decided invalidate_runtime, outside the probe, and is never retried.warnonly when a retry will follow (fieldslabel/attempt/max_attempts/error); the budget-exhausted final failure is left to the existing caller warn inmanaged.rsto avoid duplicate logging; noinfoon retry-success; no sensitive payloads.readywithin AionUi's 15s reconcile window will still alert/report. Further reduction is out of scope for this fix.Related repositories
RuntimeFailureKindtype are intentionally untouched.Test plan
cargo test -p aionui-runtime version_probe— 4 new closure-injected unit tests pass (RED before implementation, GREEN after).cargo test -p aionui-runtime— full crate regression, 94 passed / 0 failed, including unchanged classification tests (classify_error_still_detects_bundled_invalid_by_message,classify_error_detects_bundled_node_runtime_missing,bundled_runtime_missing_reports_bundled_resource_missing).cargo fmt --all -- --check— clean.cargo clippy -p aionui-runtime -- -D warnings— clean.just push— full pre-push gate (migration check, lint, format, workspace test suite: 8119 tests run, 8119 passed) — clean.#[cfg(unix)]subprocess-driven tests (added in the second commit) are not yet covered by a logged CI run against this exact HEAD in the fix archive; they ran as part of thejust pushworkspace suite above but are worth a second look in CI given they are Unix-only and won't run on Windows, where the original incident occurred. The retry logic itself is platform-independent and is covered on every platform by theversion_probe_*unit tests.runtime-installation-integrity-failurereport no longer fire on a transient-but-recoverable probe failure in a real/packaged environment is out of automated scope for this fix and has not been executed.