Harden the update flow

[david-hummingbot]

Closes 32 risk scenarios in the update flow, plus 39 more — 14 found by installing the result and running an update through it, and 25 across six rounds of review. Draft for review of the approach.

Grouped by what they touch: C Condor, H hummingbot-api, S the shared git machinery (fires once per checkout), P cross-platform. V is the later pass — see Group V, which supersedes S2 and corrects what the original Also covers section claimed.

Rebased onto 30dd1443; two conflicts needed care rather than a side picked, both noted below.

Group C — Condor

#	Scenario	Fix
C1	npm run build emptied frontend/dist while uvicorn served from it. A reload mid-build returned 500, not 404 — a missing index.html raises inside FileResponse. A build that failed left no dashboard at all, while the run said the previous bundle would come back. It had already been deleted.	Builds to dist.new and swaps, so exposure is one rename and dist is untouched unless the build wins. dist.old becomes a free rollback. The SPA fallback answers 503 with an explanation, and the failure message is now true. .gitignore covers both scratch dirs. make build-frontend swaps the same way (V36), and the 503 has a test (V37).
C2.1	A Restart Now button rendered directly beneath the engine's own "dependencies failed — fix it before restarting". Pressing it booted new code against the old venv.	The run.state == "failed" arm is gone.
C2.2	make run turns remain-on-exit off once boot is confirmed, so a restart handed over with the pane set to close — a successor that failed to boot took its traceback with it.	remain-on-exit is re-enabled across the handover and turned off again once the successor serves. A failed os.execv now logs and writes restart-failure.log.
C3	A Local admin could discover, preflight, resolve and apply an update entirely in the browser, then had to leave it and run make restart — the one step that makes any of it take effect.	POST /updates/relaunch, and the dashboard applies it itself: short cancellable countdown, restart, reconnect, one reload. Telegram keeps its manual button. Never on a failed run. Every open tab reloads, not only the one that pressed the button, and only once the successor is answering — V30, V31.
C4	startup() migrates stored data on every boot and the moves are one-way, so a rollback leaves Condor looking for directories that no longer exist. Nothing said so.	migration-incoming warning when the incoming commits touch condor/migrations.py.
C5	Nothing verified the update worked. python -m condor.doctor already existed, read-only and exiting non-zero on real failures — wired only into make install.	A final non-fatal doctor step whose output lands in the journal, with a new WARNED state so "the update succeeded" cannot swallow "and here is what is still wrong". Pinned in V35.
C6	The default chat agent wrote its routines into the tracked repo-root routines/ — 13 files upstream actively maintains. Improving a shipped routine edited a file upstream also edits.	Writes go to the gitignored local root; the shipped root stays as the read fallback, so the catalog is as full as it ever was. Migration lifts what the product already wrote there (untracked only).
C7	requires-python had a floor and no ceiling, so uv picked the newest interpreter and a fresh install died inside a transitive llvmlite build naming neither Condor nor the real constraint.	>=3.12,<3.14 plus a .python-version.
C8	Nothing warned that the dashboard being read was the one about to be rebuilt.	frontend-will-rebuild warning.
C9	The interrupted-run message described the state of the disk accurately and omitted that the process reading it had booted on the partial result.	Names the steps that finished and says to re-run before anything else.
C10	A local user with role USER gets no Updates tab and no explanation.	Names the role and the knob, not the tab.
C11.1	A customized playbook silently stopped receiving upstream changes, for ever. forked_from was written and never read back — content_digest was called in exactly two places, both at fork time.	stale_forks() supplies the missing comparison, reported before the update ("this update changes 2 shipped files you have your own version of") and after ("upstream has since changed these"). Local still wins — this reports, it does not take anything back.
C11.2	The dashboard's save erased the stamp on the first edit, so there was nothing to compare even in principle.	Writes carry the stamp across, as the three stores already did.
C11.3	A playbook upstream retires survives as a local-only phantom and keeps running.	Reported as retired, pointing at the mute set.
C12	A hand-edited shipped file blocked the update, and discard destroyed the work while reporting "Restored 1 tracked file(s)" — a destructive act in the vocabulary of a repair.	New keep-mine: moves the edit into .condor/agents where the product would have put it, resets the tracked file, and leaves the install in the supported state. Offered only where every conflicting path is forkable. Discard now names what was lost.
C13	Killed mid-swap leaves a good dist.new and dist.old and no dist — and the static mount and SPA fallback both sit inside one dist.is_dir() evaluated at startup, so Condor came up serving a bare 404 at /.	Adopted at boot, preferring dist.new.
C14	Nothing checked free disk space, and an update writes a venv, a node_modules, a bundle and an image.	low-disk warning naming the figure.
C15	npm ci deletes node_modules before installing, so a failure left no dependency tree and the build could not run either.	Its own failure state, with the actual remedy.
C16	The run guard was asyncio.Lock and an in-process flag — a second Condor on the same checkout could fast-forward the same repo concurrently.	Advisory lock file naming the holding pid.
C17	Both the engine and the banner explained the missing relaunch control as a safety trade: that re-execing races whatever started Condor into a second copy on the same port. It does not — request_restart() SIGTERMs, teardown() runs, and main() execs once the loop is gone, replacing the image in place.	Comments corrected, and the accurate reading is what C3 is built on. Three places were missed and still told the old story — see V39.

Group H — hummingbot-api

#	Scenario	Fix
H1.1	CI publishes only on merges to the default branch, tagging only latest and the version. So on a feature branch the update silently offered, and applied, main's code, while the status card truthfully reported the checkout as up to date.	If the published tag cannot correspond to this checkout, the image update is not offered and the reason is stated. Branch name is not the test: a fork has a main too, --abbrev-ref answers the literal string HEAD when detached, and unpushed commits mean the branch is main and the code is not.
H1.2	make build is the documented way to run your own code, and Condor offered to overwrite the result with the registry's.	An image whose digest the registry does not hold is reported as built locally, not behind. A probe that cannot answer is neither — see V32.
H1.3	"source" mode was unreachable on every real install — no shipped compose file has a build: key.	Removed. The git fast-forward stays, and the step now says what it is for: the shared files the container reads.
H2	A docker step killed at the 1800s ceiling said nothing about whether a half-rebuilt stack was safe to touch.	Both compose steps are idempotent; the message now says so.

Group S — shared git machinery

#	Scenario	Fix
S1	rev-parse --abbrev-ref HEAD returns the literal string HEAD when detached, so origin/HEAD resolved to the default branch, nothing reported being ahead, and merge --ff-only quietly fast-forwarded a detached checkout onto main and left it detached. Nothing anywhere called symbolic-ref.	A detached-head block that names the commit, before anything reads a branch name or fetches.
S2	The stash escape was one-way: the work survived, but recovering it needed a shell the Local user was never asked to open.	Replayed after the fast-forward. The conflicting case turned out not to work as written — see V1, which supersedes this row.
S3	A crashed git leaves .git/index.lock behind and every later subcommand fails on it, with an error that sends the reader hunting a process that is not there.	One line naming the file.

Group P — cross-platform

Targets Linux, macOS and WSL2. Native Windows is explicitly out of scope: os.execv, tmux, make and the nvm.sh sourcing are POSIX, and .venv/Scripts vs bin is unhandled throughout.

#	Scenario	Fix
P1	content_digest hashed raw bytes, so the same file digested differently on a CRLF checkout (sha256:6d9b716cc24c vs sha256:ee60f3851972) — which would make C11.1 cry wolf about trading-rule files on Windows.	Newlines normalized before hashing, with the pre-normalization form still accepted so existing stamps do not all fire at once.
P2	No .gitattributes, so line endings were whatever each host produced — and an untouched tracked file could reach C12's dead end without anyone having edited it.	* text=auto, with eol=lf pinned for the files that are hashed or executed.
P3	Timeouts are tuned for a Linux dev box; a /mnt/c checkout on WSL2 or Docker Desktop on macOS runs several times slower.	Failures name the step and say the retry is idempotent.
P4	A vite build over WSL2's or Docker Desktop's memory cap is SIGKILLed — rc 137, no stderr worth reading, indistinguishable from any other build failure.	Named, with the knob to raise.
P5	macOS (APFS) and Windows (NTFS) are case-insensitive, so two tracked paths differing only in case are one file there and a clone silently loses one.	CI guard against case-collisions and stray CRLF.

Also covers

Two cases found while checking what happens to files an agent creates rather than edits, both the same shape as C11 and neither detectable by the stamp:

• A shipped routine edited locally. A stamp lives in frontmatter and _stamp() deliberately refuses to invent it for a file that has none — but the library ships 19 .py routines, so an agent editing one shadowed it permanently with nothing able to notice.
• A file the agent creates under a name upstream later ships. Local wins per item, upstream's version is unreachable, and no stamp ever existed to compare.

Both are reported for files under an agent's own slug. Two trees were missed entirely and are fixed in V6 and V7 — including, awkwardly, the shipped-routine case named just above, whose stock copy lives at the repo root rather than under the agent.

An identical unstamped copy is not reported — copying is not diverging — and a file with no shipped counterpart at all is the ordinary "the agent wrote something new" case, which was never a fork of anything.

---

Rebased onto main

Branched before #241–#244; rebased onto 30dd1443 (177 commits), and again onto f15e401d once #258 merged. Three conflicts are worth naming, because resolving any of them the obvious way loses something:

• condor/web/routes/agents.py — main renamed strategies/strategy.md → loops/loop.md and added the SEC-693 server-pin gate to the same write C11.2 had changed to write_preserving_stamp. Main's side carries the security gate, so taking it is the instinct — and that drops C11.2 silently. All three kept.
• condor/migrations.py — both branches added a v5 behind the marker .migrated-v5: main's FEAT-128 rename, and this branch's chat-routine lift. Whichever wrote the marker first would stop the other ever running. The lift is now v6.
• frontend/src/components/RelaunchBanner.tsx (fix(web): reload open tabs after relaunch (CORR-434) #258) — main grew useReloadAfterRelaunch, which is the same true → false watcher V30 had added here privately, and edited the banner C3 had rewritten. Three hunks, all resolving toward this branch: main's side still describes the passive banner C3 replaces. The private watcher goes and the shared hook stays, so one reload path serves both. reloadOnce stays with it — the hook rides a 30s query, which is right for a tab that is only watching, while the tab that pressed the button polls every second.

Main's stock controllers (agents/<slug>/controllers/**, FEAT-126) need no new handling: they sit under an agent slug, so the C11 machinery already reaches them, .py and .yml included through the unstamped arm.

Group V — found by installing and running it

The 32 above were found by reading. V1–V14 were found by doing the documented install into an empty directory — Condor's make install + make run, then hummingbot-api's make setup + make deploy — pointing the two at each other, pushing a synthetic upstream commit, and applying the update from the dashboard.

V15 onward are review, in six rounds: of this branch, and then of those fixes in turn. Each row says which, because the pattern is worth seeing — a fix that covers one path and not its siblings accounts for most of them.

#	Scenario	Fix
V1	Supersedes S2. stash is offered as a way out of dirty-conflict, and that block means the incoming commits touch the file being parked — so the replay conflicting is the main line, not a corner. git stash pop handles that badly unattended: conflict markers written into the tracked file, index left unmerged, stash kept, exit non-zero. The run still reported succeeded, over a checkout holding a half-merged agent playbook. The remedy it printed — "resolve it by hand with git stash pop" — fails with needs merge when followed. The docstring said a conflict aborts; nothing aborted.	apply plus an explicit drop, because apply can be undone and pop cannot. The undo is deliberately narrow: reset --hard is the obvious abort and is wrong here, since these checkouts are runtime working directories and unrelated uncommitted work is expected — the preflight gates on the intersection of dirty and incoming for exactly that reason. Only the stash's own paths go back, tracked ones to HEAD and untracked ones removed, and the abort is claimed only once the unmerged set is actually empty. Files stashed with -u live in a third parent commit stash show ignores, so they are asked for separately. A new stashed-work warning closes the other half: git status says nothing about stashes, so parked work used to sit there with every surface reporting a healthy install.
V2	H1.2 never fired. It rests on registry_has_digest — ask the registry whether it holds this digest — and that needs a digest to ask about. RepoDigests is populated for a build only under the containerd image store; under the classic overlay2 store, still the default on Docker Engine, docker build leaves it []. So local_image_digest returned None, _image_facet took the "never been pulled" arm, and preflight turned it into a hard block over the whole update — Condor's half included — coded registry-unreachable while the registry answered fine in the same call. No test covered local is None.	"No registry digest" and "no image" were one answer to two questions. local_image_identity splits them: docker image inspect fails outright when the image is absent, so the id answers is it here and RepoDigests answers does the registry know it. Present-without-a-digest now reports as built here — the conclusion the containerd store reaches by the other route. Error codes also move onto the facet; preflight had been deriving them with "compose" in image.error, which is how this came to name a cause that had not happened.
V3	git fetch origin <branch> exits 128 both when the remote is unreachable and when the remote answered fine and has no branch of that name. Every failure read as the first, so an unpushed feature branch — the ordinary state of anyone mid-change — produced "the remote is unreachable", and the status card said "Failed to fetch from remote". Both send the reader to debug a network that was never the problem. incoming-unknown is the wrong shape besides: nothing is incoming, there is no upstream.	no-upstream-branch names the branch, says it exists only here, and gives the two ways out — push it, or switch to one that tracks origin, naming that branch when the remote's default can be read. The block stays, because a branch with no upstream genuinely cannot be fast-forwarded; what changes is that the reason is true.
V4	The advisory lock never named its holder. _acquire_run_lock opened the file "w", which truncates at open — before flock is even attempted — so the process losing the race wiped the pid written by the one holding it. start then read an empty file and dropped the (pid N) the refusal exists to carry, and anyone who went and looked at update.lock found it empty too.	Open "a+", and empty the file only once the lock is held — the point at which this process is entitled to write it.
V5	C5's new step cried wolf on every macOS update. _listening_binds falls back to lsof, and -sTCP:LISTEN makes lsof print the state as its own trailing token: NAME is TCP 127.0.0.1:8088 (LISTEN), so the last column is (LISTEN), not an address. Split on ( it yielded the empty string — and the empty string was in _is_public_bind's wildcard set. Any listener on the dashboard port therefore read as bound to all interfaces; in local mode that is a FAIL whose remedy is "unset WEB_HOST", a variable nobody set, on a port that was loopback all along. On a healthy install it was the only thing the new doctor step reported.	Drop the state token before reading the address. The empty string also stops counting as a wildcard: a bind this code could not parse is unknown, and unknown is not evidence of exposure — otherwise the next parsing slip becomes a security warning again.
V6	Three whole trees were never scanned for stale forks. all_stale_forks walks iter_agent_slugs(), which is built on _is_agent_dir — and that rejects a leading underscore. So _shared, the library every agent reads under its own, was never passed in: published skills, routines and (since FEAT-126) controllers could all diverge from the shipped copies in silence.	One entry added to the enumeration. The comparison itself always worked — stale_forks("_shared") returned the right answer with nothing calling it. Worth naming the asymmetry this leaves: locally_overridden walks raw paths, so the pre-update warning always covered _shared. An operator was told once, at the update that changed the file, and never again — including when the customization came after the change had landed.
V7	C6's own tree had no coverage at either end. FEAT-033 keeps the shipped routine library at the repo-root routines/ and FEAT-115 moved only the write target under the agent, so the chat's pair is (<local>/condor/routines, <repo>/routines) — the one layer whose stock root is not <stock>/<slug>/. stale_forks asked _homes for the pair, got <stock>/condor/routines, found nothing there on any install, and filed every chat routine under "an ordinary file the agent authored". locally_overridden missed it from the other end: it skipped any incoming path whose first segment was not agents/. So improving a shipped routine — the thing the local layer exists to allow — shadowed it in silence, before the update and after.	Both ends ask the loader where stock actually is. The per-file comparison moves into _compare_layer so the second root reuses it rather than growing a copy that can drift, and rel_prefix keeps the reported path the one the reader would go looking for.
V8	StaleFork.rel is relative to one library's root, so every forked playbook rendered as a bare AGENT.md and the warning read (AGENT.md, AGENT.md, AGENT.md) — three files it could not tell apart. Nine agents ship, so an install with more than one customized is the ordinary case; and main retiring three agents makes a retired entry collide with a stale one under the same name.	rel keeps its meaning, because the stores and the tests read it as library-relative. The qualified form is a new label, which is what the surfaces show.
V9	test_three_slow_servers_resolve_in_one_timeout_not_three asserted elapsed < DELAY * 1.5 — a wall clock, which measures the machine as much as the code. It passed alone and failed under a loaded full-suite run: red that says nothing about the property and trains people to re-run rather than look.	Rendezvous instead of clock. Every fetch waits at a barrier until all three arrive, so a concurrent resolver releases the moment the last one gets there however loaded the host, and a serial one cannot reach the assertion at all — it fails on the rendezvous timeout rather than hanging. The deterministic assertion (max_concurrent == 3) was already sitting next to the flaky one and was always the thing carrying the test. Checked both ways: green as written, and assert 1 == 3 when the route's gather is replaced with a sequential loop.
V10	Reported live: a local-only install with a pending update where clicking the "Updates available" notification did nothing at all. Two files have to agree about one number and nothing checked that they did. .env decides who the dashboard is — local mode signs in as ADMIN_USER_ID, no password — and config.yml decides what that id may do. On the install in question .env resolved to 1883786161 (a duplicated key; all three parsers that read that file take the last assignment) while config.yml gave that id the role user, with admin sitting on id 1. Every admin route answered 403, the frontend dropped the Updates tab, and the notification's own ?tab=updates link fell back to Servers without a word. The update engine was working the whole time — nothing could reach it. And nothing was wrong enough: make doctor printed a green ✓ ADMIN_USER_ID 1883786161, a tick on the exact value that had no rights.	Three parts. Doctor: Admin access names the id, the role it actually has and the key to change; ADMIN_USER_ID (duplicate) fires separately on a second assignment, because a duplicate is silently authoritative while the first line is the one people read. Dashboard: the fallback stays — the tab genuinely is not theirs — but it now says so instead of redirecting in silence, naming the role, the knob and make doctor. Message: get_user_role returns the enum, so C10's 403 rendered 'UserRole.USER' — a string that appears nowhere in config.yml, so the reader could not search for it.
V11	make install runs setup-chrome, which was python -c "import kaleido; kaleido.get_chrome_sync()" with stderr sent to /dev/null. Three things went wrong at once when it stalled: no timeout, so it waited for ever; no progress, so nothing separated slow from wedged; and the only line on screen was Setting up Chrome for chart rendering..., indefinitely. Measured in that state: 256 KB of a ~150 MB archive after thirty minutes, connection open and idle. The install could not finish and nothing said why — reached by following the documented setup, on the path where every update step in this branch got a timeout and a named failure.	condor/setup_chrome.py, following condor.setup_llm's convention. The stall detector is the part that matters: a cold download on a slow link can legitimately run for minutes, so a ceiling generous enough not to cut that off is also generous enough to sit on a dead connection just as long — bytes arriving is what separates them. A stall names the proxy or firewall that a held-open idle connection usually means; the ceiling names the knob to raise (CONDOR_CHROME_TIMEOUT) and says the link was merely slow. Progress is printed as it goes. An already-installed browser now costs nothing instead of being re-fetched every install, and the real error is shown instead of discarded. Always exits zero: charts are optional and an optional renderer must not fail an install.
V12	compose_up shells docker compose up -d into HUMMINGBOT_API_DIR, so it never passes the guard make deploy has. Same exposure: container_name: is fixed for every service, so two checkouts are one stack to Docker — Compose matches by name, prints Recreate, and the update takes over the other checkout's containers and its postgres volume, silently. Condor points at a directory; Compose acts on a project, and those differ whenever two checkouts share a basename — which they do by default, both being hummingbot-api.	Compose records the project's working directory on every container, so the question is answerable. A foreign-stack block before the update, and the same question again immediately before compose_up — the confirm screen can be minutes old and what is at stake is a database. Nothing running, an unlabelled container, or an unreachable Docker are all "no conflict".
V13	A live dashboard reported "No local digest for hummingbot/hummingbot-api:latest — it has never been pulled by tag". The image had been on disk for days and the container was running; Docker Desktop was restarting. Only one probe in that path needs the daemon, which is why nothing else gave it away — measured with the socket gone: compose config rc=0 (parses locally), buildx imagetools inspect rc=0 (asks the registry), image inspect rc≠0. So a service definition and an available digest both resolve, and the single failure lands on the branch that concludes the image was never pulled. V2 did not help: splitting "no registry digest" from "no image" gives two answers about the image, and this is not one.	Ask whether the daemon is there before concluding anything about what it holds. One extra command, only on the error path — a test pins that the happy path never runs it — and the available digest the registry already gave us is kept rather than discarded.
V14	remote_default_branch turned refs/remotes/origin/HEAD into a name with rsplit("/", 1)[-1] — right for main, wrong for every branch with a slash: fix/x came back as x. image_tracks_this_checkout compares against it, so on a repo with a slashed default the comparison never matched and the image was skipped with a reason that read like a deliberate finding. V3 prints the name as advice, so it was also telling people to git switch to a branch that does not exist.	Strip the known prefix, and return None when the ref is not under it rather than inventing a name from what is left.
V15	Review, P1. FEAT-115 moved the chat's routine write target to <local>/condor/routines, but assistant_routines sends the chat through discover_routines, which reads the root library and the shared one and never that. The agent wrote routines it could not then see — and the v6 migration, which lifts previously untracked routines out of routines/, moved existing visible ones into the same unread directory.	The chat's own layer shadows the general library, exactly as an agent's does. FEAT-033's carve-out is kept: the shipped catalog is still read beside it.
V16	Review, P1. has_update_stash looked for our message anywhere in the stash list while apply/drop default to stash@{0}, and the stack reorders on every push. With an operator stash made after ours, theirs was applied and dropped — their work restored into the tree unasked, ours left parked, and the step reported "Restored the work that was stashed before the update."	Resolve our entry by name to a stash@{n} and address every command at it. Nothing between the apply and the drop can renumber it.
V17	Review, P1. Edited through the product and in the checkout is two customizations. keep-mine moved the checkout's copy onto the product's with shutil.move, which replaces it — one destroyed, while the message said the version was kept.	Refused, naming both and where they are. Only the operator can say which they meant.
V18	Review, P1. The update lock is taken before check() and released only by the task created after it, so a git or docker that cannot start left it held by a process that was not updating — locking out every other process on the checkout until it exited.	The setup between the lock and the task is wrapped; any failure releases before it propagates.
V19	Review, P1. The relaunch banner treated a 403 like a dropped connection — reasonable for a restart, since the socket usually closes before the reply. For a non-admin seat the restart never happens, so the poll finds the same server answering, reloads, and the countdown starts again: a reload loop that seat can never end.	A 403 is a refusal, not silence: the banner stops and says the seat lacks the admin role. The half of V10 the Updates tab already had.
V20	Review, P2. keep-mine is offered for an untracked-conflict too, and checkout HEAD -- <path> fails on a path HEAD never had — so the resolution reported failure after moving the file.	Only tracked paths are reset. Moving an untracked one was the resolution.
V21	Review, P2. The CRLF guard read the working tree, where * text=auto promises nothing — it governs what git stores. A checkout with core.autocrlf=true (the Windows default, inherited by a WSL2 clone under /mnt/c) materializes CRLF for exactly the files this is meant to pass, failing every ordinary .py and .tsx.	Reads the index via git ls-files --eol, which is what text=auto actually governs. Checked both ways: passes on a clean checkout, and still catches a blob forced in as i/crlf.
V22	Review, P2. keep-mine moves a batch, and the "destination already exists" check ran inside the move loop — so a clash found half way through returned "nothing was moved" with the earlier files already gone. The checkout was left showing a deletion nobody made, and the update still blocked.	Every destination is validated before anything moves, and the refusal names only the paths that actually clash.
V23	Review, P1. A staged addition kept its index entry. Resetting only paths present in HEAD is right for an untracked file and wrong for a staged one: checkout HEAD -- cannot touch a path HEAD never had, so the entry survived the move, and the incoming commit adding that same path still had something to collide with — reported as a success.	Staged additions get rm --cached; tracked ones still reset to HEAD.
V24	Review, P1. V16 fixed the lookup and left the advice. The abort message still said stash@{0} and a bare git stash pop. Ours is not necessarily on top, so following the printed remedy restored somebody else's work — the exact mistake V16 had just stopped the code from making.	The message carries the ref that was actually resolved.
V25	Self-review. V15 stamped the chat's routines as an agent's. Discovering its local layer with agent_slug=CHAT_SLUG sets source="agent:condor", and that stamp is load-bearing: the MCP tool renames an agent: routine to <slug>/<name> and labels it agent-scoped with the specialist slug — which is empty for the chat.	The chat's local layer is the writable half of the general library, so it discovers without a slug and keeps global: the bare name and scope the rest of that catalog has.
V26	Self-review. V12 refused the operator's own update through a symlink. Compose records the resolved working directory and os.path.normpath does not resolve symlinks, so reaching the same directory through one — macOS /tmp, any symlinked project root — compared unequal to itself. foreign-stack fired, naming that same directory back at them.	realpath on both sides.
V27	Self-review. V11's stall detector killed healthy downloads. With no resolvable download root the byte counter is pinned at 0, so last_change never moves and the stall branch aborts a transfer that is progressing — then blames a proxy for it.	A stall verdict needs a size to watch; without one, only the ceiling applies.
V28	Review, P1. V15 made chat routines listable but not runnable. The layer went into assistant_routines alone, while RoutineStore._resolve_routine starts a run through get_routine → discover_routines, which did not know about it. A newly authored routine failed to start, and a local override of a shipped name showed its own description in the catalog while running the shipped code — you read one thing and ran another.	The layer moves into discover_routines, where the catalog, get_routine and the resolver behind a run all see the same thing; the special case is gone with it. Tests now pin resolution and not only the catalog, which is the gap that let this through.
V29	Found while fixing V28. discover_routines' mtime shortcut restores an unchanged root file by stem out of the previous merged dict. Sound while root shadowed shared; unsound the moment a local layer could shadow root, because the entry restored for a root stem may be the override — so a deleted override kept being served and the shipped routine stayed unreachable. It surfaced as cross-test pollution: test_primitives_catalog failed in the suite and passed alone.	The root library gets a cache of its own. fresh_start still honours _routines_cache = None, which is the invalidation every existing caller uses.
V30	A tab that did not press the button never reloaded. C3's reload lived inside the countdown, so only the tab that started the relaunch picked up the rebuilt bundle. Every other one — a second admin's, one that pressed Cancel, one whose seat cannot restart at all — watched required go false, unmounted the banner, and carried on serving the bundle it booted with, with nothing left on screen to say so. The same went for every tab when the relaunch came from Telegram or make restart, since neither goes near the browser.	The watcher sits above the early return, because the transition it waits for is the one that stops the strip being rendered. Now main's own useReloadAfterRelaunch — see Rebased onto main.
V31	And the tab that did press it reloaded too early. It reloaded on the first response it could get, but SIGTERM only starts the shutdown: through the drain and teardown() the dying process answers 200 for seconds. So the reload landed the new bundle — already swapped onto disk by C1's build step — against the old API, which is the one thing this was supposed to avoid.	required is the flag that cannot lie: set once per process and never cleared, so only the successor answers false. Both reload paths share one guard, since either may get there first.
V32	Review. H1.2 guessed when it could not tell. registry_has_digest returns None for "could not ask" — typically a Docker Hub rate limit — and that fell through to behind=1. The two wrong answers are not symmetric: call it a pull and Condor offers to overwrite a make build whose only copy is on that host; call it a build and a real update hides behind a silent "up to date".	Neither. The probe failing is a fact about the probe: image-origin-unknown blocks with the reason, which is what an error on the image facet already does, and retrying is the remedy.
V33	Review. The facet tests reached the network. _facet stubbed three updater calls and left registry_has_digest real, so it shelled out to docker buildx imagetools inspect — 13.4s for a single test. It agreed with its own assertion only because the registry answered one of the two ways that reach the same branch; had it ever answered not found, the test would have failed.	Stubbed, with all three branches pinned separately. The file went from 13.4s to 1.0s.
V34	Review. POST /updates/relaunch was missing from the gate table. That table exists to be exhaustive — "every one of them must answer 403" — and the route it omitted is the one that restarts the process. The gate itself was never wrong. _stub_pipeline had the same shape of gap: run_doctor and stash_pop ran for real, the latter against this very checkout.	The route joins the table. Both pipeline calls are stubbed; that file went from 6.6s to 1.6s.
V35	Review. C5's WARNED arm had no test. run_doctor appeared in no test file at all, so nothing held the three properties that make it safe.	Pinned: warned, still succeeded, still owed the relaunch. Failing the run on the doctor's verdict would report code already on disk as never having landed, and withhold the one thing that reaches the fix for whatever it is complaining about.
V36	Review. C1 was only half fixed. The atomic swap went into the updater; make build-frontend still ran a bare npm run build, and vite's emptyOutDir empties the served directory in place. run and restart stop Condor first, but the target is public and a Condor started any other way — run-fg, a supervisor, a second checkout — is still serving out of dist.	The Makefile swaps through dist.new exactly as the updater does, with a test reading the target so it cannot drift back.
V37	Review. C1's 503 had no HTTP test. TestClient had been sitting imported and unused in the bundle-build tests ever since the 503 was written.	A test that takes index.html away, asks for a page, and asserts 503 with the explanation and no-cache — using that import.
V38	Review. V19's denied banner still offered "Restart now". It posts the same request from the same seat and is refused the same way, underneath a sentence explaining that it will be.	The button is gone in denied. What that seat has to do instead is in the message, and neither half of it is a button.
V39	Review. C17 was over-claimed. Three places still told the old story — that exec'ing races a second Condor onto the port: the engine's own module docstring, the Telegram handler, and the restart-flow tests. This branch's relaunch route refutes it in as many words, so the codebase argued both sides of the same question.	All three now say what the code does, and why the run stops short anyway: the moment to interrupt a running bot is not the pipeline's to pick.

How V10 and V11 were verified

V11 was checked against the real download on all three paths: a successful fetch (131 MB in 37s, with progress), an already-installed browser (instant skip, nothing re-fetched), and a hard failure forced through a dead proxy — which now prints Connection refused instead of discarding it, and still exits 0 so the install continues. The stall path is covered by tests, being the one that cannot be forced on demand.

For V10, the live install was left untouched. The state was reproduced in a throwaway install on a spare port: as a non-admin the tab is replaced by "Updates is admin-only … Give this user role: admin in config.yml", and make doctor fails with the id, the role and the key. Granting the role brings the tab back and GET /updates answers 200.

Not fixed here

One thing the clean-room install surfaced that is not this branch's to fix:

• Two hummingbot-api checkouts share one compose project. The directory basename is the project name and container_name is hardcoded, so make deploy in a second checkout adopts the first's containers and its postgres volume — it printed Recreate, not Create. Belongs in hummingbot-api, and is fixed there in fix: connector config visibility and XRPL market pricing hummingbot-api#227; V12 covers the path an update takes, which does not go through make deploy.

[greptile-apps]

Confidence Score: 4/5

[High risk] Hardens the update flow with schema changes and migration logic.

The PR does not appear safe to merge until the two frontend build paths cannot interfere with each other's bundle swap.

Findings

1.  Concurrent builds can remove bundle ▶
2.  Test disrupts live dashboard ▶
3.  Fast reconnect poll cannot authenticate ▶

Summary

This PR hardens Condor's update preflight, conflict recovery, image handling, dashboard builds, health checks, and relaunch flow. Since the previous review, it also:

• Makes the public Makefile frontend build swap bundles instead of emptying the served directory.
• Handles inconclusive image-origin checks and expands relaunch and update-flow tests.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[In-process update] --> C[frontend/dist.new and dist.old]
  B[make build-frontend] --> C
  C --> D[frontend/dist served by Condor]

Loading

_{Reviews (9) · Last reviewed commit: "Fix seven findings from review"}

[greptile-apps]

Fast reconnect poll cannot authenticate

The new successor check requires a successful response body, but this poll calls /meta/relaunch without a bearer token. That endpoint requires authentication, so the one-second poll cannot receive required: false. The initiating tab must instead wait for the authenticated shared query’s next 30-second poll to reload.

Knowledge Base Used: Frontend application

[greptile-apps]

Concurrent builds can remove bundle

If make build-frontend runs while the in-process updater is building, both processes delete and reuse the same dist.new and dist.old directories. The Makefile target does not take the update lock, so one build can remove the other's output or interleave the renames, leaving the running dashboard without a complete bundle. The in-process build uses the same directories in utils/updater.py.

[greptile-apps]

Test disrupts live dashboard

If Condor is serving the dashboard from this checkout while the backend tests run, this test renames the real frontend/dist/index.html. Browser requests then receive 503 until the test restores it. A concurrent build can also interfere with cleanup. Testing against a temporary bundle would avoid changing the served files.