fix(cli): stop reporting "up to date" when the Homebrew update check fails - #1298
fix(cli): stop reporting "up to date" when the Homebrew update check fails#1298groksrc wants to merge 3 commits into
Conversation
`_check_homebrew_update_available()` decided outdated-or-not purely from
`brew outdated`'s stdout and discarded the return code. Every failure mode
of that command produces the same shape -- non-zero exit, empty stdout,
reason on stderr -- so a failed check was read as "not outdated" and the
caller reported `UP_TO_DATE`. A hard failure to determine the latest
version was reported to the user as success, silently pinning Homebrew
installs to whatever version they happened to have.
Observed with newer Homebrew's third-party tap trust requirement
("Refusing to load formula ... from untrusted tap"), but a stale tap, a
missing formula, brew off PATH, or a network failure all produce the same
empty-stdout shape.
Model the three real outcomes instead of two: exit 0 with empty stdout is
up to date, the package name on stdout is outdated, and anything else is
unanswered and now raises `HomebrewCheckError`. `run_auto_update()` catches
it and falls back to the PyPI comparison, which can answer the question --
the tap can only lag PyPI, never lead it -- and yields a real
`latest_version`, so Homebrew users no longer see "Update available
(latest: unknown)". `source` is unchanged, so remediation still points at
`brew upgrade basic-memory`. If PyPI is also unreachable the existing
handler reports FAILED rather than a false all-clear.
Closes #1297
Signed-off-by: Drew Cain <groksrc@gmail.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f2f66fb7bc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # comparison is a sound answer, and it yields a real latest version. | ||
| # `source` is unchanged, so remediation still points at `brew upgrade`. | ||
| logger.warning(f"Homebrew update check failed, falling back to PyPI: {exc}") | ||
| update_available, latest_version = _check_pypi_update_available() |
There was a problem hiding this comment.
Don't auto-upgrade Homebrew based on PyPI state
When brew outdated fails while PyPI has published a newer release but the Homebrew tap has not caught up, this fallback sets update_available=True and the normal periodic/MCP path proceeds to brew upgrade even though that version is unavailable through Homebrew. This window is explicitly possible because .github/workflows/release.yml publishes to PyPI in the release job before the dependent Homebrew formula job runs; a Homebrew-check failure during that window therefore causes a misleading availability result and a spurious automatic upgrade attempt. Preserve an unanswered/failed Homebrew outcome, or prevent automatic installation when availability was inferred only from PyPI.
AGENTS.md reference: AGENTS.md:L146-L148
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Good catch — confirmed and fixed in 2f3aa9b.
Verified both halves of the claim before acting:
release.yml: thehomebrewjob declaresneeds: release, andreleasepublishes to PyPI. So the window where PyPI leads the tap is real, not theoretical.- Control flow: unless
check_onlyorsource == UNKNOWN, the function does fall through to--- Automatic install ---and runbrew upgrade.
There is a second reason the old behaviour was wrong, beyond the release window: whatever prevented brew outdated from answering (untrusted tap, brew missing) also blocks brew upgrade, so the auto-install was doomed regardless of tap lag. The new regression test demonstrates exactly that — without the fix it does not merely mis-report, it returns FAILED because the upgrade actually ran and errored.
The fix keeps the asymmetry rather than dropping the fallback: PyPI stays authoritative for the negative answer, since the tap can only lag PyPI and never lead it, so "nothing newer exists" is still sound. Only the positive answer is unsafe to act on. When availability was inferred from PyPI because brew could not answer, the result is now UPDATE_AVAILABLE with updated=False, surfacing the brew failure reason plus the brew upgrade hint, and no subprocess is launched.
Two tests added: one asserting no ["brew", "upgrade", ...] call on that path, one pinning the negative-answer behaviour so a future change does not "fix" the asymmetry away.
On retry/backoff, which was also suggested: deliberately not doing it. The dominant failure modes here are persistent, not transient — an untrusted tap is a policy gate and a missing brew will not heal on a second attempt — so backoff would add latency to an interactive command and to the silent MCP path while only reducing how often the fallback is reached, not making it correct.
`brew outdated` writes progress output ("==> Downloading Homebrew API
data") to stderr on the successful outdated path, so a non-empty stderr is
not an error signal on its own. The outdated-case test asserted against an
empty stderr, leaving nothing to catch a future "stderr means failure"
reading. Use brew's real stderr instead.
This locks the three real result shapes into the suite: exit 0 with empty
stdout (up to date), exit 1 with the tap-qualified formula name on stdout
(outdated), and exit 1 with empty stdout plus an `Error:` stderr
(unanswered). Exit 1 is shared by the middle and last, so stdout stays the
discriminator and the return code only breaks the tie when stdout is empty.
Signed-off-by: Drew Cain <groksrc@gmail.com>
|
Follow-up (e136bad) pinning one more empirically-confirmed detail into the tests.
So neither "non-zero exit" nor "non-empty stderr" is an error signal on its own. The fix keys on stdout first and only consults the return code to break the tie when stdout is empty; stderr is used solely as message text after the decision is made. Two consequences worth flagging for review:
Verified the suite rejects the naive implementation: replacing the body with |
The PyPI fallback for a failed `brew outdated` set update_available=True, and run_auto_update() then proceeded to run `brew upgrade`. That is unsafe two ways: release.yml publishes to PyPI in the `release` job while the Homebrew formula job `needs: release`, so PyPI can carry a version the tap cannot install yet; and whatever hid the brew answer (untrusted tap, brew missing) also blocks the upgrade itself, so the command is doomed. Keep the fallback for the negative answer -- the tap can only lag PyPI, never lead it, so "nothing newer exists" is sound -- but when availability was inferred from PyPI because brew could not answer, report the update plus the brew failure and let the user upgrade deliberately. Reported by Codex review on #1298. Signed-off-by: Drew Cain <groksrc@gmail.com>
Closes #1297
Problem
_check_homebrew_update_available()decided outdated-or-not purely frombrew outdated's stdout and discardedresult.returncode:Every failure mode of
brew outdatedproduces the same shape — non-zero exit, empty stdout, reason on stderr — so a failed check read as "not outdated" andrun_auto_update()fell into itsif not update_availablebranch and reportedUP_TO_DATE. A hard failure to determine the latest version was reported to the user as success, which silently pins a Homebrew install to whatever version it happens to have.The trigger seen in the wild was newer Homebrew's third-party tap trust requirement:
but a stale tap, a formula brew cannot resolve, brew off
PATH, or a network failure all produce the identical empty-stdout shape. The fix targets the general failure mode, not that one message.Secondary defect on the same path: the function returned
latest_version=Noneunconditionally, so even a correct "outdated" result printedUpdate available (latest: unknown).Fix
Model the three real outcomes instead of two:
brew outdatedresultFileNotFoundError)HomebrewCheckErrorrun_auto_update()catchesHomebrewCheckError, logs brew's reason, and falls back to_check_pypi_update_available().Why PyPI is a sound fallback for a Homebrew install: the tap is updated from a PyPI release, so it can only lag PyPI, never lead it. If PyPI's latest equals the installed version, no newer version exists anywhere and "up to date" is true.
sourcestaysHOMEBREW, so_manual_update_hint()still saysbrew upgrade basic-memoryand the auto-install path still runsbrew upgrade. As a bonus, when brew itself is the broken part, that upgrade surfaces brew's real error (e.g. the trust message) to the user instead of a silent no-op.If PyPI is also unreachable, the existing outer handler reports
FAILEDwith the error — still never a false all-clear.Signalling the unanswered case with an exception keeps this branch symmetric with
_check_pypi_update_available(), which already raises when it cannot compare versions, and matches the house rule of reserving exceptions for unpredictable subprocess/network failures.Verification
Confirmed the false negative against unmodified
mainusing realbrewon macOS —_check_homebrew_update_available()returned(False, None)for an exit-1/empty-stdout/Error:-on-stderr result. End to end, same fake result throughrun_auto_update(check_only=True)with a Homebrew executable path:Four regression tests added to
tests/cli/test_auto_update.py:test_check_homebrew_update_available_failed_check_is_not_up_to_date— the untrusted-tap shape (exit 1, empty stdout, stderr) raises instead of returning "not outdated"test_check_homebrew_update_available_reports_missing_brew—FileNotFoundErrorfor brew is an unanswered checktest_failed_homebrew_check_falls_back_to_pypi— end to end: a failed brew check yieldsUPDATE_AVAILABLEwith a reallatest_versionand abrew upgradehint, notUP_TO_DATEtest_failed_homebrew_check_reports_failure_when_pypi_is_unreachable— with neither source able to answer,FAILEDuv run ruff check/ruff format --checkclean on both files;uv run ty check src tests test-intreports the same 4 pre-existingpymilvusunresolved-import diagnostics as cleanmain— no new lint or type findings.Note:
tests/cli/has pre-existing failures on cleanmainin this environment (ANSI-colored console output vs. plain-text assertions, count varies run to run under pytest-randomly). They are unrelated to this change and unaffected by it.