fix(runtime): stop probing the local llama backend while a cloud provider is active

[plombeer31]

Fixes #112

A cloud-backed session opened with /health + /props against http://127.0.0.1:8080, warned that nothing answered, kept a 3-second footer poller running against it, and refreshed a local ModelProfileManager on every turn — for a backend the session never talks to. The warnings read as an active-backend failure on a run whose real provider was healthy the whole time.

(The permalinks in the issue point at 9d525ef, months stale; every call site here was re-located on origin/main @ 0ed39a7.)

The gating predicate, and why it is safe to call this early

activeTextProviderIsLlamaServer(llm) moves out of src/tui/local-turn-gate.ts into src/llm/provider/registry/active-text-provider.ts, beside resolveLlmConfig, and is re-exported from its old home so its existing callers and tests are untouched. src/runtime/ and src/sidecar/ now gate on the same predicate without importing from src/tui/.

It is safe at the very top of buildRuntime because resolveLlmConfig(config) is a pure function of config with no I/O — it either returns the llm block or synthesizes the single local-llama default from localModels.*. Nothing about "is the active text provider local?" needs ProviderRegistry.fromConfig, which is what made the previous triage passes read this as a bootstrap-hoisting job.

Detection stays KIND-based (any llama-server entry, so a renamed entry is still gated) and keeps a deliberate conservative default: an active id resolving to no entry counts as local. The direction matters — a wrong "local" costs one wasted probe, a wrong "cloud" runs inference on an unprobed profile. A sibling providerIdIsLlamaServer(llm, id) answers the same question for an arbitrary chain link.

What is gated

Surface	Change
src/runtime/bootstrap.ts	The boot /health line, the /props probe inside resolveModelProfile, and the minUsableContextWindow warning (local-only advice — its hints name --ctx-size / localModels.managed.contextSize).
src/sidecar/main.ts	The start_session health probe and its llm_unavailable event. Config is re-read per session, not closed over, since the shell can rewrite it between sessions.
src/agent/agent-loop.ts	The turn-start refresh() and the between-steps refreshIfStale(), behind one new optional localBackend dep. Absent (test/legacy wiring) reads as "always local", so pre-existing loop tests are byte-identical in behaviour.
src/tui/llm-health/llm-health-poller.ts	tick(), refreshModelLabel() and updateUrl()'s emit return early on a cloud route.
src/tui/select-context-usage.ts	The poller's n_ctx is only consulted while a local backend is the active route.

The ModelProfileManager is not deleted — only its probes are deferred. Construction is pure field assignment with no I/O (verified: the constructor only assigns). Deleting it on a cloud boot would leave a mid-turn fallover to a llama-server link running on a frozen plain-instruct profile with no route back to the real one.

The TUI poller: not polling, not polling-and-hiding

The poller re-reads config on its own tick rather than being started/stopped by each switch path. There are three of those (ProvidersOrchestrator.setActiveText, the composer switch, save-provider-wizard), and a poller that had to be told about each would miss whichever is added next; a self-reading tick resumes within one interval after any of them, with no wiring. On a cloud route it emits nothing, so llmHealth.status stays unknown rather than latching down.

updateUrl() is gated too, and separately from tick(): it made no request (the tick() it schedules returns early), but it emitted llm_model_updated {model: null, contextWindow: null} unconditionally, so /llama <url> on a cloud session would blank the tray label the active provider had put there. The bookkeeping reset still runs unconditionally, so a later switch back to local re-discovers the model.

Presentation was already partly correct — selectComposerBackendMeta returns cloud → healthy and gates local status behind localConfigured — but resolveWindow in select-context-usage.ts read llmHealth.contextWindow unconditionally, so after a local→cloud switch the cloud model's gauge would be drawn against a llama-server n_ctx. That stale reading is reachable by design, not only by a lost race: agent-event-reducer.ts deliberately preserves contextWindow when an llm_model_updated omits it, which is exactly the shape notifyCatalogModel emits. It is now guarded, and the guard has its own test.

Overlap with the open TUI PRs: #318/#319 (context-chip token accounting + live recalc) and #320/#321 (context-gauge persistence, per-session model) all live in the context/model area. The only line here in their neighbourhood is the resolveWindow guard in select-context-usage.ts (source #2 of four). If one of those lands first this is a two-line rebase; flagging rather than coordinating, as asked.

Lazy restore — including the cloud→local fallover

New src/llm/local-backend-gate.ts: DeferredLocalBackendProbes replays exactly once the probes boot skipped — /health logging, the /props profile + grammar + slot-pool refresh, and the context-window advice. ensureProbed() returns true only for the call that performed them, so a caller whose next act is its own /props refresh skips it: one just landed. Concurrent callers await the same restore and report false. It latches even on a throw — synchronous or asynchronous; the restore() call sits inside the try precisely so a sync throw cannot leave the probes re-arming on every call — and the bootstrap restore additionally swallows its own errors so a diagnostic cannot fail a link.

Two paths reach local inference after a cloud boot, and both go through it:

(a) The operator switches back to a local provider. isActive() re-reads live config, so the loop's turn-start gate opens on the next turn and calls ensureProbed() before anything is sent.

(b) The fallback chain falls over from a cloud link to a llama-server link mid-turn. This is a gap this PR opens, not one main had — worth stating precisely, because an earlier revision of this description had it backwards.

On main the fallover was already safe. Verified on a clean origin/main @ 0ed39a7, booted runtime, cloud 429 → llama-server fallback:

boot local urls: [".../health", ".../props"]
FALLOVER: props=1 | urls: [/props, cloud×3, /completion, /completion, cloud]

Boot ran /health + /props unconditionally, and main's turn-start refresh() was unconditional too, so a second /props landed before the cloud attempts. The local link served with a real profile and a /props-sized slot pool. The only main scenario that matches "a plain-instruct profile, a one-slot pool and no /props" is a local server that is down at boot and at turn start — in which case the fallover completion fails a moment later anyway.

The gap is created by this PR: boot skips both probes, and the loop's turn-start refresh is gated on the active provider, which is still cloud at the moment a fallover happens. So the link the chain picks would otherwise be the first thing in the process to touch llama-server, on deferred state. prepareLink is necessary because of the gating, not because main was broken.

Traced: createFallbackCompleter / createFallbackStreamer (src/runtime/llm-fallback-seam.ts) drive runWithFallback, which calls chain.advanceFrom(...) and re-runs the same attempt body against the next link; the attempt body resolved the link's provider via deps.resolveSlice(providerId) and sent the completion immediately. Nothing in src/llm/fallback/ (resolveFallbackChain, run-with-fallback.ts, provider-fallback-chain.ts) touches profile, health or slot state — the chain is pure switching policy.

The fix is a new optional prepareLink?: (providerId) => Promise<void> on FallbackSeamDeps, awaited at the top of each attempt in both seams — the last point at which the chosen link is known and the completion has not been sent. resolveSlice could not carry it (synchronous), and folding it into the attempt body would put bootstrap's knowledge of link kinds into the seam. Bootstrap wires it to createLocalLinkPreparer(...); on every other attempt it is one predicate call.

Sustained fallover: the local link keeps serving, so it keeps being refreshed

fallback.appendLocal defaults to true, so a rate-limited or down cloud primary falls over to the llama-server link on every turn under the default config shape — this is not an opt-in path. A one-shot restore is not enough there. The operator can still swap the model behind llama-server mid-outage, and the active text provider stays cloud the whole time, which is what the loop's turn-start gate reads: after the first fallover latched ensureProbed(), nothing would refresh the profile again and the prompt would keep being built with the first model's chat template and GBNF grammar. observeCompletionModelId would keep flagging the manager stale with no consumer. Measured over three turns with the model hot-swapped between turns 2 and 3:

without the arm below: turn1 props=1 | turn2 props=0 | turn3 props=0   (profile pinned)
main:                  turn1 props=1 | turn2 props=1 | turn3 props=1
this PR:               turn1 props=1 | turn2 props=1 | turn3 props=1

createLocalLinkPreparer (in local-backend-gate.ts, so the three decisions are unit-testable rather than an inline closure in buildRuntime) does, for a llama-server link only:

1. noteLinkServed() — the loop's turn-start refresh gains a second arm that fires on takeLinkServed() even while the active provider is cloud. Take-and-clear, so a recovered primary buys exactly one trailing refresh and then the local probes go quiet again.
2. ensureProbed() — true means the restore just ran and a fresh /props already landed; nothing further for this attempt.
3. otherwise refreshIfStale() — on a cloud-active turn the loop's own between-steps refresh is gated off, which leaves this the only consumer of the staleness flag, and it sits closer to the request than the line it replaces.

Every non-local link returns on step 0, so the zero-request criterion is unchanged: a cloud bootstrap, a cloud turn and a sidecar cloud start still make zero local requests. Re-verified after this change with a counting fetchImpl over a real createAgentRuntime — cloud bootstrap plus three cloud turns served by a working cloud provider produced three https://cloud.invalid/v1/chat/completions requests and nothing else at all: 127.0.0.1:8080 and 127.0.0.1:19092 both zero.

Local embeddings are untouched — they hang off memory.embeddings.enabled + localModels.embeddings.enabled and their own port (19092), and are still probed under a cloud text provider.

Acceptance criteria

Criterion	Covered by
Zero local /health + /props at CLI bootstrap on a cloud config	src/runtime/local-probe-gating.test.ts — "makes zero local text requests…", exact counts 0 on 127.0.0.1:8080, with a local-route control asserting 1 + 1 and a KIND control (llama-server under a custom id still probes)
Zero at sidecar session start	src/sidecar/local-probe-gating.test.ts — boots the real bootstrapSidecar, pushes an NDJSON start_session at stdin, reads the response off stdout; asserts [] local URLs, plus a local-route control asserting the 2 /health calls
Zero on cloud turns	src/agent/agent-loop-local-gate.test.ts — counting fetchProps stub behind a real ModelProfileManager: 0 on a cloud turn, 1 on a local turn, 1 with no gate wired (legacy parity). Plus local-probe-gating.test.ts — "…across three cloud TURNS, not just at boot", through the booted runtime on the default fallover chain shape
No local-backend failure/warning on a healthy cloud run	bootstrap test asserts no warn/error record matching /llama|context window/
TUI does not present inactive local health as the active provider's	llm-health-poller.test.ts — "probes nothing at TUI startup…": 0 checkLlamaServer calls, 0 /props fetches, no actions emitted at all over several tick windows; "emits nothing from updateUrl…" for the ungated emit, with a local control; and select-context-usage.test.ts — a cloud row plus a leftover llmHealth.contextWindow of 4096 must resolve to null, with a local-route control
Cloud turns use a non-local/plain profile, never refresh the manager	bootstrap boots on PLAIN_INSTRUCT_PROFILE with no probe; loop test asserts the profile is unchanged after a cloud turn
Selecting local-llama lazily restores health/profile/context/slots	local-probe-gating.test.ts — "lazily restores the local backend when the operator switches to it": boot cloud (0 requests), providerRegistry.setActive("local-llama") + config flip, run a turn, assert /health = 1 and /props = 1 before inference. Plus the loop-level switch test and the DeferredLocalBackendProbes unit tests
Local embeddings still probed independently	bootstrap test — /health on :19092 = 1 while :8080 = 0
cloud→local hot switching / fallover	now end-to-end: local-probe-gating.test.ts — "a cloud->local FALLOVER warms the link before it serves, turn after turn" boots the real runtime, flips the cloud provider to 429, and asserts /health = 1, /props = 1 with /props landing before the local /completion, then /props = 2 and 3 over the next two turns with /health still 1. Plus llm-fallback-seam.test.ts ordering (["prepare:cloud","serve:cloud","prepare:local","serve:local"], unary and streaming, through the real factories), the createLocalLinkPreparer unit tests, and the loop-level three-turn hot-swap test
Sustained fallover does not freeze the profile / grammar	agent-loop-local-gate.test.ts — real seam + real gate + real preparer + real AgentLoop, cloud 429 on every turn, model swapped behind the fake llama-server between turns 2 and 3: cumulative /props [1, 2, 3], /health replayed once, and the profile each local completion was actually built with is ["gemma4-think", "gemma4-think", "qwen-think"]
Poller resumes after a switch	llm-health-poller.test.ts — "resumes within one tick…", config flipped mid-run with no call on the poller

Fully covered: all of them.

Test evidence

• npm run lint (tsc --noEmit): clean.
• Targeted: npx vitest run src/runtime src/sidecar src/agent src/tui/local-turn-gate.test.ts src/llm/fallback src/llm/provider/registry src/llm/local-backend-gate.test.ts src/llm/model-profile-manager.test.ts src/llm/llama-server-health.test.ts src/tui/llm-health src/tui/select-context-usage.test.ts src/tui/components/context-chip.test.tsx src/tui/components/context-panel.test.tsx src/tui/usage-at-pairs.test.ts src/tui/local-backend-readiness.test.ts → 606 passed, 1 failed (48 files).
• The one failure is src/sidecar/send-message-concurrency.test.ts > serialises two rapid send_message calls FIFO without crossing state. Pre-existing: it fails identically on a clean origin/main (0ed39a7) checkout in the same environment.
• Full npx vitest run src/tui (this change alters the poller and a context selector): 244 files, 2642 tests, all passed. (One unhandled EACCES spawning a fake llama-server binary in local-models-orchestrator-auto-update.test.ts — pre-existing environment noise, no test failed.)
• Fails without the fix: with the six behaviour files (bootstrap.ts, sidecar/main.ts, agent-loop.ts, llm-health-poller.ts, select-context-usage.ts, llm-fallback-seam.ts) reverted to origin/main and the new tests kept, 15 of the new tests fail — both bootstrap cloud tests, the three-cloud-turns test, the embeddings one, the lazy-restore one, the end-to-end fallover one, the sidecar cloud test, three loop tests (cloud turn, hot switch, sustained fallover), both prepareLink ordering tests, three poller gating tests and the resolveWindow guard test. The ones that still pass with the fix reverted are exactly the intended controls (local route still probes, legacy-deps parity) and the unit tests for the new local-backend-gate.ts module itself.
• Mutation battery, 15 mutations on the final tree — the boot /health gate, the boot /props gate, both loop gates, the served-link arm, the prepareLink await (unary and streaming), bootstrap's wiring of it, all three decisions inside createLocalLinkPreparer, the gate's failure latch, the poller's tick and updateUrl gates, the resolveWindow guard, and the predicate's conservative default — all 15 killed.

Left out

Nothing from the issue's scope. Three adjacent things deliberately untouched, stated as limitations rather than as equivalences:

• LocalModelsOrchestrator.autoStartIfReady uses a different predicate. It also gates on the active provider, but by id (activeTextProvider !== "local-llama"), where everything in this PR is KIND-based. The two are not equivalent: for a llama-server entry under a custom id this PR's predicate says local and the orchestrator's says not-local, so the managed daemon is not auto-started for it. That is pre-existing, errs toward less local activity, and leaves no acceptance criterion here unmet — so it is left alone and recorded as a divergence, not folded into this diff.
• prepareLink warms the configured local URL, not the link's own URL. The providerId says which link is about to serve, not where it lives; bootstrap warms the one local backend the runtime owns — the ModelProfileManager over the shared LlamaServerClient, which reads localModels.url per request. A second llama-server entry pointed at a different host is announced through the hook but probed against the configured URL. Pre-existing ModelProfileManager limitation (a singleton over one client, not a per-link cache); multi-endpoint local links would need a manager per link before the hook could mean more. Now stated in the hook's docstring.
• notifyCatalogModel still emits an optimistic label from the Models tab without probing. It costs no request, and the Models tab is an explicit local surface.