Repository navigation
Local AI: allow setup on busy GPUs and handle model-load OOM at use time #1391
Description
Problem
A supported GPU that is temporarily occupied by another application can block Local AI installation before downloads begin. Setup tells the user to close GPU applications and retry setup. Download and installation should depend on hardware capabilities and total usable GPU capacity; temporary memory pressure should be handled when the user actually runs Local AI.
Release context: requested ahead of the upcoming RTX Spark drop.
Confirmed code path
Reviewed commit: 621f4100f8b40ac84e844bcc39b01fe03a2238bb. The reporter's exact app version and a live reproduction of their machine are not established.
- LocalInferenceSelector already selects the model/profile using total GPU capacity and CUDA compatibility. Its memory estimate includes weights, main/draft KV caches, and runtime workspace. Free memory does not downgrade the recommendation.
- LocalInferenceEligibility explicitly allows
EligibleButBusythroughCanInstall. - CapabilitiesPage instead requires strictly
Eligible, disables Continue with Local AI enabled, and displays "Close GPU applications and retry setup." The wizard caches the hardware snapshot; its Recheck control is only shown for unknown hardware, so freeing VRAM does not automatically unblock this page. - PreflightLocalAiHardwareStep takes a fresh reading and independently returns a terminal failure for
EligibleButBusy, before runtime/model downloads. - SetupPipeline later starts the empty router, captures a GPU baseline, forces a test inference/model load, and verifies GPU placement. Removing only the two early gates would still allow temporary memory pressure to fail setup after downloads.
Current post-setup failure UX
- Normal inference loads the model on demand. The Companion has no dedicated load-time OOM recovery UX.
- A gateway lifecycle error becomes a generic red Chat status message using the upstream summary/message, or "Agent error". The turn ends and Send becomes available, but the error has no Retry action. See mapping and rendering.
- If the router survives and reports the model as unloaded, the Local AI page can still show Running / Downloaded and verified. Router health does not describe the failed model load.
- Router process crashes have bounded automatic router restart; this does not replay the failed inference. The detailed OOM log capture/reset currently discussed in setup belongs to setup's verification path.
- Exact gateway OOM wording and successful subsequent model reload have not been verified end to end.
Expected behavior and acceptance criteria
- Supported hardware with low free VRAM retains the same recommended model/profile and can select Local AI, continue, download, and finish setup.
- Remove the free-VRAM rejection from setup UI/preflight. Keep total-capacity, CUDA/runtime compatibility, required hardware facts, download integrity, and configuration checks.
- Setup completes without requesting model inference or requiring successful GPU model loading. Remove
CaptureLocalAiGpuBaselineStep,VerifyLocalAiInferenceStep, andVerifyLocalAiGpuLoadStepfrom the mandatory installation path. Preserve their useful coverage as explicit diagnostics/developer or release validation. - Keep empty-router health, registered model/path checks, WSL reachability, gateway configuration, and ownership checks. Completion copy accurately describes installation/router readiness rather than claiming successful inference.
- On actual use, a confirmed GPU OOM produces clear guidance such as "Not enough GPU memory is available to load this model. Close other GPU applications and try again." Keep installed artifacts and the selected model/profile intact; no setup restart or redownload is required.
- Provide an explicit recovery action for the failed load/request. It must target the correct session/model, avoid duplicate submissions, and avoid automatically replaying completed agent/tool work. Validate the gateway retry contract before choosing direct retry versus restoring the failed prompt for user-controlled resubmission.
- Track recoverable model-load failure separately from router health so a healthy empty router cannot erase the failure. Clear it on an appropriate successful load/recovery.
- Classify OOM only from evidence belonging to the current managed model/load attempt. Stale logs, generic HTTP 500 responses, host-RAM allocation failures, and unrelated CUDA/driver faults must not be mislabeled as GPU OOM. Preserve an actionable generic failure when evidence is inconclusive.
- Freeing GPU memory allows successful inference with the same installed model and profile. Memory becoming constrained after setup also follows this recovery path.
Validation and proof plan
Current evidence: the existing filtered LocalInferenceQualificationTests suite passed, 51 tests, 0 failures, 0 skips. No implementation changes, full build, or live OOM/UI reproduction were performed during this review.
For implementation, add focused regression tests alongside each change for busy/idle selection parity, UI/preflight admission, setup finishing with an unloaded model, retained integrity checks, load-error classification, stale-error isolation, and retry behavior. Run the required full build plus Shared and Tray tests, and affected SetupEngine and Connection tests. Run the required MXC setup/connect validation when exercising the changed gateway setup/connect E2E path.
Collect current-head isolated UI and real Gateway-to-local-model proof: install while GPU memory is occupied; finish with the model unloaded; trigger a reviewed load failure; show recovery guidance; free memory; successfully retry without reinstalling. A fake error proves presentation/classification only, not actual CUDA OOM recovery. Include rubber-duck review before PR publication.
Applicable proof pools: windows-wsl-dgx-blackwell, windows-winui-interactive, and windows-wsl-gateway-e2e; include windows-11-arm64 for native RTX Spark ARM64 release proof, and windows-wsl-mxc for the required setup/connect MXC E2E path. Record unavailable hardware/dependencies as Not verified / blocked.
Related
- Local AI setup intermittently fails (~30%): llama-server CUDA crash (cudaErrorSharedObjectInitFailed in flash-attn) on RTX Spark N1X during warm-up #1242 tracks an intermittent Spark CUDA warm-up crash and distinguishes GPU OOM, host-memory allocation failure, and driver/runtime faults. This issue changes installation policy and recovery UX; it does not establish a fix for those underlying faults.
- Local AI setup failure leaves tray app running with half-installed/unverified local inference #1276 records the earlier problem of failed setup verification leaving confusing installed/runtime state. Preserve the distinction between downloaded-and-verified artifacts, router health, and successful model loading.
Activity
clawsweeper commentedon Sep 10, 2026
Codex review: this still needs some work. Reviewed October 3, 2026, 4:39 PM ET / 20:39 UTC.
Summary
Busy-GPU installation is fixed and shipped, but the requested attempt-specific OOM guidance and recovery flow remain missing. The related loading-feedback issue overlaps without covering those requirements.
Reproducibility: no. high-confidence current-main GPU OOM reproduction is established. Source proves the missing failure-state and recovery presentation, while the merged setup PR records pressure-induced failure followed by successful inference.
Root-cause cluster
Relationship:
canonicalCanonical: #1391
Summary: This remains the distinct request for GPU OOM recovery; installation is implemented, and related loading feedback and CUDA faults do not cover all remaining requirements.
Members:
partial_overlap: fix(setup): allow Local AI setup on busy GPUs and current Gateway #1399 - Merged implementation covers busy-GPU installation and explicitly excludes post-setup load-error UX.partial_overlap: No loading indicator during local AI model load — first prompt shows "Agent error" #1445 - Tracks first-prompt initialization feedback without establishing OOM classification or safe failed-request recovery.adjacent_distinct: Local AI setup intermittently fails (~30%): llama-server CUDA crash (cudaErrorSharedObjectInitFailed in flash-attn) on RTX Spark N1X during warm-up #1242 - Tracks an intermittent CUDA kernel initialization fault and distinguishes it from GPU memory exhaustion.adjacent_distinct: fix(local-ai): add non-destructive recovery #1384 - Merged non-destructive installation recovery does not implement chat model-load recovery.Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.
Maintainer decision needed
Why: Installation policy is already accepted; the remaining choice determines session targeting and whether previously completed agent or tool work could run again.
Next step
Confirm the failed-request recovery interaction, then scope runtime classification and Chat presentation together; the accepted installation contract does not need another decision.
Review details
Best possible solution:
Preserve deferred loading and installed artifacts, retain attempt-specific load failures separately from router health, and offer an inline user-controlled recovery action without replaying completed agent work.
Do we have a high-confidence way to reproduce the issue?
No high-confidence current-main GPU OOM reproduction is established. Source proves the missing failure-state and recovery presentation, while the merged setup PR records pressure-induced failure followed by successful inference.
Is this the best way to solve the issue?
Partly: deferred loading is already implemented. The remaining recovery should use current-attempt evidence and user-controlled resubmission unless the Gateway provides verified safe retry semantics.
AGENTS.md: found and applied where relevant.
Remaining risk / open question:
Codex review notes: model internal, reasoning medium; reviewed against 7127c16d537c.
Label changes
Label changes:
P2: The installation blocker is shipped; the remaining work improves recovery from temporary model-load pressure without an established current unrecoverable first-use failure.impact:ux-friction: Generic chat errors omit the cause and explicit recovery guidance, while prior related proof records successful inference after GPU pressure was released.issue-rating: 🐚 platinum hermit: Current issue advisory state selects this label.clawsweeper:needs-live-repro: Current issue advisory state selects this label.issue-rating: 🦞 diamond lobster: Current issue advisory state no longer selects this label.clawsweeper:source-repro: Current issue advisory state no longer selects this label.P0: Current review triage priority is P2, so this older priority label is no longer current.impact:ux-release-blocker: Current review impact labels areimpact:ux-friction.Label justifications:
P2: The installation blocker is shipped; the remaining work improves recovery from temporary model-load pressure without an established current unrecoverable first-use failure.impact:ux-friction: Generic chat errors omit the cause and explicit recovery guidance, while prior related proof records successful inference after GPU pressure was released.Evidence reviewed
What I checked:
src/OpenClaw.SetupEngine/SetupPipeline.cs:123, 7127c16d537c)src/OpenClaw.SetupEngine/SetupPipeline.cs, 3c43751b2bac)src/OpenClaw.Tray.WinUI/Chat/ChatEventMapper.cs:217, 7127c16d537c)src/OpenClaw.Connection/LocalAi/LlamaServerClient.cs:105, 7127c16d537c)Likely related people:
How this review workflow works
@clawsweeper re-reviewcomments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.@clawsweeper re-reviewor@clawsweeper re-runon an open PR or issue to request a fresh review only.@clawsweeper reviewto request a fresh review only.@clawsweeper autofix,@clawsweeper automerge,@clawsweeper fix ci, or@clawsweeper address review.@clawsweeper explainto ask for more context, or@clawsweeper stopto stop active automation.fix(setup): allow Local AI setup on busy GPUs and current Gateway (#1399
github-actions commentedon Sep 29, 2026
This issue has been automatically marked as stale due to inactivity.
Please add updated OpenClaw Windows details or it will be closed.