From 12a4485d226d6fab385387bd014b2b93dcd3af8e Mon Sep 17 00:00:00 2001 From: Thomas Date: Thu, 8 Oct 2026 20:39:31 -0400 Subject: [PATCH 1/7] added plan --- docs/directory.md | 1 + docs/system-path-backend-plan.md | 337 +++++++++++++++++++++++++++++++ 2 files changed, 338 insertions(+) create mode 100644 docs/system-path-backend-plan.md diff --git a/docs/directory.md b/docs/directory.md index 62fa536..5b7a4a4 100644 --- a/docs/directory.md +++ b/docs/directory.md @@ -1372,6 +1372,7 @@ Prefer `rg` for local search. On Windows/PowerShell, use patterns like `rg -n "p | `CONTRIBUTING.md` | Developer quickstart: setup, run, dev loop, tests, and PR checklist | | `docs/directory.md` | This file — project structure and feature reference | | `docs/tests.md` | Test suite layout, commands, and what each test covers | +| `docs/system-path-backend-plan.md` | Proposed staged implementation for using system-managed llama.cpp tools from PATH (issue #399) | | `docs/gpu-monitoring.md` | User setup guide for NVIDIA SMI, AMD SMI, and the optional cross-vendor all-smi collector | | `docs/maintenance.md` | Release, dependency, compatibility, and repository maintenance guidance | | `docs/security.md` | Security model, trust boundaries, and reporting guidance | diff --git a/docs/system-path-backend-plan.md b/docs/system-path-backend-plan.md new file mode 100644 index 0000000..9ca06cd --- /dev/null +++ b/docs/system-path-backend-plan.md @@ -0,0 +1,337 @@ +# Implementation plan: System (PATH) llama.cpp backend + +Source: [issue #399](https://github.com/thomas9120/LLama-GUI/issues/399) +Prepared: 2026-10-08 +Repository baseline: `6543038` +Status: proposed implementation; no runtime changes have been made. + +## Goal + +Let users activate the llama.cpp tools already available on the GUI process's +`PATH`. Llama GUI will construct launch arguments, start and stop its own child +processes, and provide the existing Chat, API, Benchmarking, and Monitor features. +The user's system package manager will own llama.cpp installation and updates. + +This replaces the issue author's workaround of symlinking system executables into +`llama/custom/bin/`. It is particularly useful for NixOS packages and wrappers +that supply their own runtime dependencies, and also applies to other system +installations. + +## Proposed behavior and scope + +- Add backend ID `system`, displayed as **System (PATH)**, on supported platforms. + Keep it selectable when tools are missing so activation can explain the problem. +- Require an executable `llama-server` for activation. Treat `llama-cli`, + `llama-bench`, `llama-perplexity`, utility tools, and `llama-fit-params` as + independently available features. Existing official and Custom backend + requirements remain unchanged. +- Use the PATH inherited by the Python GUI process. Resolve each tool separately; + tools need not share one directory. Display the paths used for discovery. +- Launch the discovered absolute path with the normal GUI-generated arguments. + Preserve symlink and wrapper entry points rather than replacing them with their + final targets. Use an argument list and the existing subprocess implementation. +- Resolve again before a launch or probe. Package updates can replace a symlink + target while the GUI is open. A running process continues until the normal stop + action; a later launch uses the then-current tool. +- Preserve the inherited environment for System, then merge the already-supported + per-launch `LLAMA_`/`GGML_` overrides. Do not prepend repository binary or library + directories for this backend. +- Disable release selection, GUI binary updates, and Repair for System. A failed + activation leaves the current backend active. Missing System tools never fall + back to official or Custom binaries. +- Keep the existing model root, presets, bundled chat templates, subprocess + working directory, authentication, and process lifecycle. Grammar files remain + user-selected paths; there is no package-data-directory discovery in this scope. + +This feature does not install system packages, modify PATH, source shell startup +files, configure Nix/CUDA, or connect to an already-running server. The existing +external-server feature continues to handle that last case. No new runtime +dependency or general provider framework is needed. + +## Existing foundations and changes needed + +| Area | Current behavior | Required change | +|------|------------------|-----------------| +| `backend/services/llama_manager.py` | Backend specs, folder selection, Custom activation, packaged runtime checks, official-install metadata | System classification, discovery/probes, activation, and correct preservation of the official installation | +| `backend/app.py` and `backend/context.py` | `find_tool_executable()` returns a path inside one selected installation folder | Delegate to shared discovery; represent an unavailable System tool explicitly; register the activation route | +| `backend/services/process_manager.py` | Preflight and launch share validation; environment prepends a selected bin directory; the estimator locates `llama-fit-params` separately | Use System discovery/environment for every subprocess and validate the executable actually being invoked | +| `backend/routes/status.py` | Installation readiness requires CLI and server; tool availability is reported as booleans | Apply System's server requirement and add discovered paths/build information | +| `backend/routes/install.py` | Custom activation has install/process interlocks; other actions assume a downloadable backend | Add System activation and reject inappropriate install/update/release operations | +| `backend/routes/lifecycle.py` | Open Folder creates an application-owned installation directory | Give System a read-only folder-opening policy | +| `ui/js/manager/manager-backends.js`, `ui/js/manager/manager-install.js`, and `ui/js/manager/manager-status.js` | Backend selection, release fetching, activation, and build-tag propagation | System controls, status, switching, and build information | +| `ui/index.html`, `ui/js/quick-launch-ui.js`, and `ui/js/flag-core.js` | Setup markup, launch readiness, and shared argument generation | System guidance, per-tool readiness, and correct build gates | + +Use the existing service/context injection and focused frontend namespaces. +Keep shared flag state authoritative across Configure, Quick Launch, Chat, and +command preview. Touch other consumers only where a demonstrated assumption about +folders, availability, or build tags requires it. + +## Stage 1: Backend classification and executable discovery + +**Outcome:** one resolver finds the selected backend's tools without changing +existing official/Custom behavior. + +1. Add System to `build_backend_specs()` in `backend/services/llama_manager.py`. + Keep `CUSTOM_BACKEND_SPECS` limited to its two real folder slots; startup must + not create a System bin/grammar directory. +2. Add narrow helpers for System detection and for identifying installations + managed outside the GUI. Use them where existing code assumes that every + non-Custom backend is an official installation. Keep Custom folder behavior + separate from System PATH behavior. +3. Introduce shared tool resolution in the owning backend service. For System, + use `shutil.which()` on a known tool filename, make the returned path absolute + without dereferencing its symlinks, and check file/executable availability. + Search the inherited PATH only, including its normal precedence and relative + entries; use platform-aware executable naming. Do not expand the search into + other repository installations or launch a shell to perform discovery. +4. Restrict discovery inputs to known llama.cpp tools. Include `llama-fit-params` + as an internal discovery target without adding it to the public launch-tool + allowlist. Do not permit arbitrary executable names through the API. +5. Make `find_tool_executable()` delegate to that resolver. Use `None` for an + unavailable System tool, update the `BackendServices` callable annotation, + and explicitly guard all consumers before path operations. Preserve the + existing folder-path results for official and Custom installations. +6. Remove System calls into folder-only helpers. System has no single bin + directory: runtime enumeration and setup should not scan or create PATH + directories. A guard in the folder helper can catch accidental misuse. + +**Verification:** add focused discovery cases under `tests/backend/`, covering +PATH precedence, tools in different directories, missing tools despite local +official binaries, spaces in paths, executable permissions, symlinks/wrappers, +Windows `.exe` lookup, and existing official/Custom resolution. Use fixtures for +platform-specific lookup rather than relying on the host's installed llama.cpp. + +**Exit criterion:** a missing System tool is explicit, and choosing System cannot +select a tool from a different backend by fallback. + +## Stage 2: Runtime probes, environment, and process integration + +**Depends on:** Stage 1. +**Outcome:** the existing launch pipeline can run system tools and wrappers. + +1. Add a System branch to `_build_process_env()` that copies the inherited + environment without injecting local binary/library paths. Apply existing + validated per-launch overrides afterward. Keep official/Custom environment + construction unchanged. +2. Validate System executables with a bounded, model-free `--version` subprocess + using the discovered entry point, inherited runtime environment, existing + window-hiding conventions, and a short timeout such as five seconds. Parse + stdout and stderr for recognized build formats; successful execution with an + unknown build format is distinct from a failure to execute. +3. Use that execution check for System runtime health instead of requiring nearby + DLLs, `.so` files, or `@rpath` libraries. Do not run `ldd`/`otool` against a Nix + wrapper and interpret a script-inspection failure as a broken installation. + A successful version probe establishes that the entry point starts; it does + not establish model loading or GPU compatibility. +4. Cache probe results under the existing runtime-health lock. Include the + executable path and file/symlink identity in the System cache key, retain a + bounded TTL, and invalidate on activation or discovery changes. Ordinary + status polls should reuse results rather than spawn probes on every request. + Avoid holding process/config locks during subprocess execution. +5. Integrate missing-tool errors and System probes into preflight and launch. + Resolve and validate the same entry point that is passed to `Popen`; if it + disappears between validation and execution, return the normal logged, + sanitized launch failure. Preserve install/process interlocks, runtime + generations, output capture, health polling, and Windows process groups. +6. Route `_fit_params_executable()` through shared discovery. In memory + estimation, validate `llama-fit-params` itself for System; the current code + locates the estimator separately but validates the requested CLI/server tool. + Missing estimation support must not block server launches. +7. Review `get_buffer_types()`, which currently chooses CLI. For System, prefer + an available supported CLI/server probe target, using server when CLI is + absent. Preserve the existing unavailable/error behavior if the build does + not support the probe rather than implying that GPU capability was verified. + +**Verification:** cover preserved PATH/library variables, wrapper execution, +known/unknown builds, probe failures/timeouts, cache invalidation, missing +estimator, server-only buffer discovery, unchanged argument/environment payloads, +and existing start/stop behavior. Extend +`tests/backend/test_process_environment.py` and relevant service tests. + +**Exit criterion:** server preflight and launch use the system entry point and +environment, with recoverable errors and no packaged-library requirement. + +## Stage 3: Activation, persistence, status, and switching + +**Depends on:** Stages 1-2. +**Outcome:** System can be activated, restored after restart, and switched safely. + +1. Add proposed route `POST /api/activate-system` in `backend/routes/install.py` + and register it in `backend/app.py`. It accepts no client-supplied executable + path. Reuse the Custom activation install-slot claim/release pattern to reject + activation during an install or while a GUI-managed process is running. +2. Discover tools and require the System server execution probe to succeed before + changing configuration. Return discovered paths, found/missing tool lists, + normalized build tags, and actionable failure information. Validate request + shape and log unexpected errors before returning `sanitize_error()` output. +3. Persist the selection under `config_lock`, merging with the latest config so + model-root/external-target settings survive. A simple marker such as + `backend: "system"`, `tag: "system"`, `version: "system"` is sufficient for + selection; discovered paths and build tags come from current discovery/cache, + not persisted Nix-store paths. Configuration is unchanged on failed probing. +4. Preserve `official_install` when leaving an official build. Update both Custom + activation and `get_official_install_status()` so System is never recorded as + the official installation. Switching System -> Custom -> official must retain + the correct official backend/tag. Reuse Activate Existing for the return path. +5. Extend `/api/status` additively. Retain `executables` booleans and existing + fields. Mark the available-backend entry with `system_path: true`, expose + per-tool discovered paths/build tags and probe state, and calculate System + readiness from its server rather than both CLI and server. Keep selected + backend configuration separate from discovered version information. +6. Report a removed/broken System server as unavailable/stale with PATH guidance. + Do not clear the user's selection or silently select another backend. On GUI + restart, rediscover with that process's inherited environment. +7. Explicitly reject System in install, repair-via-install, update, and + activate-existing official requests before downloads or config mutation. + Return an empty release list for System without a release lookup. Verify + official catalog refresh does not turn System into a download target. +8. Keep cleanup limited to the existing application-owned directories. Preserve + the active System selection while clearing removed official-install metadata. + For System Open Folder, open the discovered server's containing directory + without creating it; report unavailable if the server cannot be found. +9. Update the Route Modules table and endpoint count in `docs/directory.md` in + the same implementation change. This plan adds one exact route; the current + baseline's 50 endpoints would become 51 if no other routes change meanwhile. + +**Verification:** successful/failed activation, running/install rejection, +restart restoration, server-only status, direct misuse of install/update routes, +official/System/Custom round trips, config merge preservation, cleanup, and +read-only folder opening. Extend `tests/backend/test_custom_slots.py` and +`tests/backend/test_extracted_routes.py`; put System-specific cases in a focused +`test_system_backend.py` file under the existing backend tests directory. + +**Exit criterion:** a System selection survives restart and cleanup, and users +can return to their previous installation without downloading it again. + +## Stage 4: Frontend controls, readiness, and build compatibility + +**Depends on:** Stage 3. +**Outcome:** the user can discover, activate, and understand System from the GUI. + +1. Render the backend from backend-provided metadata in the existing Manager + package. Selecting it shows **Activate System**, hides release selection, + and explains that the OS package manager owns installation and updates. + Add minimal setup markup in `ui/index.html`; keep the current script order. +2. Display each discovered tool/path and availability using `textContent`. + Show server as required for System and the other tools as feature-specific. + Retain both CLI/server requirements in existing official/Custom views. +3. Distinguish the pending dropdown choice from the active backend. Route System + activation to its endpoint, retain duplicate-action guards, refresh status + afterward, and show errors near the activation controls. Polling must not + reset the user's pending selection. +4. Disable System update/repair and suppress release fetching, including forced + refresh paths. Selecting an official target while System is active should + offer Activate Existing where appropriate. Derive this from accepted status. +5. Adjust Quick Launch and other affected readiness checks to consider the + requested executable. A missing CLI must leave server/Chat usable. Missing + benchmark/perplexity tools or the estimator should produce feature-specific + guidance, with backend launch validation remaining authoritative. +6. Feed detected build tags into the existing shared flag-core compatibility + gates. PATH can resolve CLI and server from different builds; use the tag + for the tool whose arguments are being generated, including tool changes + and benchmark source configurations. Extend the existing tag mechanism + narrowly rather than creating per-tab flag state. Unknown build formats + retain the existing conservative gate behavior and an explicit unknown + version display; do not invent a release tag from a package version. +7. Keep command preview on the existing shared argument builder. The System + details show the actual executable path; a new command builder is unnecessary. + Update folder/remove action labels and explanations to match Stage 3. +8. Explain discovery failures concretely: **llama-server was not found on the + PATH inherited by Llama GUI. Start the GUI from the environment where it is + available, or restart the GUI after changing its launch environment.** + Avoid displaying the entire environment or offering automatic shell edits. + +**Verification:** extend `tests/frontend/manager_releases_unit.cjs`, relevant +flag-core/launch/benchmark tests, and `tests/frontend/flag_sync_smoke.cjs`. +Check activation, pending selection, disabled actions, server-only readiness, +tool-specific build gates, safe path rendering, and official return activation. +Run `node --check` on every changed JS file and `npm run test:frontend` for DOM +wiring/shared-state changes. + +**Exit criterion:** a server-only System installation is usable through the +normal GUI, and unavailable optional features have clear, localized errors. + +## Stage 5: Regression coverage and release checks + +**Depends on:** Stages 1-4; add focused tests alongside their owning stages. +**Outcome:** the whole feature and existing backends pass the release checks. + +- Exercise three representative System layouts: server only, a complete set of + tools, and wrapper/symlink entry points. Include a broken server, different + CLI/server builds, an unknown version format, and a changed package target. +- Keep host llama.cpp installations out of deterministic unit tests. Use + temporary files, mocked subprocess responses, and environment fixtures. +- Update `tests/frontend/llama_flags_supported_unit.cjs` so selecting System + checks PATH tools rather than local downloaded binaries first. Cover selection + in `tests/frontend/llama_flags_runner_unit.cjs`. Preserve explicit binary + directory precedence and the required pinned-build CI checks. The full flag + comparison still needs both CLI and server; server-only runtime support must + not weaken that test's existing contract. +- Recheck official and both Custom activation, launch environment, missing-tool + errors, cleanup, and switching. Reuse the existing suites rather than adding + another process manager or browser harness. +- Run the full backend suite in the project venv, the frontend suite, and docs + link/route-sync checks once the staged work is integrated. + +```powershell +.venv/Scripts/python.exe -m unittest discover tests -v +npm test +.venv/Scripts/python.exe -m unittest tests.backend.test_docs_links tests.backend.test_docs_sync -v +``` + +On Unix, use `.venv/bin/python` for the Python commands. Follow [the test +reference](tests.md) for focused commands during each stage. Keep CI on its +existing Linux and Windows runners; cover macOS path/probe behavior with fixtures. + +**Exit criterion:** automated checks pass without requiring a system llama.cpp +installation, GPU, or native macOS runner, and CI's pinned binary check remains +strict. + +## Stage 6: User documentation and native validation + +**Depends on:** Stage 5. +**Outcome:** the feature is documented and its real environment behavior is known. + +1. Update `docs/directory.md` with System selection/discovery, runtime validation, + status metadata, and the final route contract. Update `docs/tests.md` for new + coverage, and `docs/troubleshooting.md` for inherited PATH, restart, unknown + build tags, and package-manager-owned updates. +2. Document terminal-versus-desktop/Pinokio environment differences. Keep startup + entry points and runtime dependencies unchanged. If implementation requires + launcher/startup changes, inspect the companion launcher documented in the + [project reference](directory.md#companion-repositories) before making them. +3. Validate on NixOS with an actual system package/wrapper: activate, load a model, + send a Chat message, inspect Monitor, stop, restart, and switch back. Test a + package/profile update to confirm later launches rediscover the entry point. +4. Smoke-test a conventional Linux installation, Windows PATH executables, and + macOS package-manager/desktop startup when those environments are available. + Check optional tools only where installed; a missing estimator is an expected + unavailable feature. Record which native checks were actually run. + +**Exit criterion:** docs describe the implemented behavior, and native NixOS +validation establishes that wrappers and real model/GPU loading work. If that +environment is unavailable, explicitly record the remaining native check rather +than claiming fixtures establish NixOS/CUDA compatibility. + +## Effort estimate and completion criteria + +This is a small-to-medium implementation. Discovery is straightforward; runtime +environment assumptions, optional tools, build gates, and switching account for +most of the integration work. + +- Stages 1-2: approximately one developer day. +- Stage 3: approximately half to one day. +- Stage 4: approximately one day. +- Stages 5-6: approximately half to two days, depending on native environments. + +Allow roughly **3-5 developer days** for a reviewed implementation by someone +familiar with this repo. Native NixOS/macOS availability can extend elapsed time. +Expect around **10-14 production files** plus focused tests and documentation; +keep the final diff driven by the concrete consumers listed above. A minimal +server-launch prototype could take 1-2 days but would not complete this plan. + +The feature is complete when System activates without copied/symlinked repository +binaries, normal server/Chat flows work with its inherited environment, missing +optional tools affect only their features, official/Custom switching remains +correct, system files are never modified by maintenance actions, automated checks +pass, and native validation results or remaining limitations are documented. From 59cc9e008d2fc244b98cb09c8feaefd7e5b3d64d Mon Sep 17 00:00:00 2001 From: Thomas Date: Thu, 8 Oct 2026 20:54:14 -0400 Subject: [PATCH 2/7] Add initial System (PATH) backend discovery - Classify System as externally managed and resolve known tools from PATH - Handle unavailable PATH tools across status, runtime, and launch checks - Add isolated discovery coverage for platform lookup and path precedence --- backend/app.py | 8 +- backend/context.py | 2 +- backend/routes/status.py | 2 +- backend/services/llama_manager.py | 150 +++++- backend/services/process_manager.py | 15 +- tests/backend/test_services.py | 12 +- tests/backend/test_system_path_discovery.py | 482 ++++++++++++++++++++ 7 files changed, 654 insertions(+), 17 deletions(-) create mode 100644 tests/backend/test_system_path_discovery.py diff --git a/backend/app.py b/backend/app.py index a8d646b..e9fd356 100644 --- a/backend/app.py +++ b/backend/app.py @@ -194,11 +194,17 @@ def get_tool_filename(tool): def find_tool_executable(tool): cfg = load_config() - return llama_manager_service.get_backend_bin_dir(APP_CONTEXT, cfg.get("backend")) / get_tool_filename(tool) + return llama_manager_service.resolve_backend_tool_executable( + APP_CONTEXT, cfg.get("backend"), tool + ) def get_runtime_files(): cfg = load_config() + if llama_manager_service.is_system_backend(cfg.get("backend")): + # System tools live across PATH directories; there is no single + # repository bin directory to enumerate. + return [] search_dir = llama_manager_service.get_backend_bin_dir(APP_CONTEXT, cfg.get("backend")) if not search_dir.exists(): return [] diff --git a/backend/context.py b/backend/context.py index df07372..c973dd2 100644 --- a/backend/context.py +++ b/backend/context.py @@ -56,7 +56,7 @@ class BackendServices: binary_suffix: str = "" current_arch: str = "unknown" current_platform: str = "unknown" - find_tool_executable: Callable[[str], Path] = _missing_service + find_tool_executable: Callable[[str], Optional[Path]] = _missing_service get_local_llama_metrics: Callable[[str, str, str], Tuple[Optional[str], str]] = _missing_service get_local_llama_props: Callable[[str, str, str], Tuple[Optional[str], str]] = _missing_service get_local_llama_slots: Callable[[str, str, str], Tuple[Optional[str], str]] = _missing_service diff --git a/backend/routes/status.py b/backend/routes/status.py index 5400032..24766ed 100644 --- a/backend/routes/status.py +++ b/backend/routes/status.py @@ -20,7 +20,7 @@ def get_status(request, response, ctx): for tool in services.llama_tools: name = services.get_tool_filename(tool) executable = services.find_tool_executable(tool) - exes[name] = executable.is_file() and ( + exes[name] = executable is not None and executable.is_file() and ( services.current_platform == "win32" or os.access(executable, os.X_OK) ) diff --git a/backend/services/llama_manager.py b/backend/services/llama_manager.py index 780d8d9..83af60e 100644 --- a/backend/services/llama_manager.py +++ b/backend/services/llama_manager.py @@ -47,12 +47,43 @@ "custom-02": {"label": "Custom 02", "folder": "custom-02"}, } +# Backend ID for llama.cpp tools resolved from the GUI process's inherited +# PATH. The OS package manager owns installation and updates; Llama GUI only +# discovers, launches, and stops its own child processes. Stage 1 covers +# classification and discovery; activation/probes/launch arrive in later stages. +SYSTEM_BACKEND_ID = "system" +SYSTEM_BACKEND_LABEL = "System (PATH)" + +# Internal PATH discovery target that is never a public launch tool. Memory +# estimation locates it separately (Stage 2); Stage 1 only resolves it. +SYSTEM_INTERNAL_TOOLS = ("llama-fit-params",) + def is_custom_backend(backend: Any) -> bool: return isinstance(backend, str) and backend in CUSTOM_BACKEND_SPECS +def is_system_backend(backend: Any) -> bool: + return isinstance(backend, str) and backend == SYSTEM_BACKEND_ID + + +def is_externally_managed_backend(backend: Any) -> bool: + """Installations owned outside the GUI (package manager, not llama/).""" + return is_system_backend(backend) + + +def is_official_backend(backend: Any) -> bool: + """Downloadable llama.cpp builds managed under llama/bin/.""" + return ( + isinstance(backend, str) + and not is_custom_backend(backend) + and not is_system_backend(backend) + ) + + def get_backend_bin_dir(ctx: AppContext, backend: Any) -> pathlib.Path: + if is_system_backend(backend): + raise ValueError("System (PATH) backend has no single bin directory.") if backend == "custom": return ctx.paths.llama_custom_bin if is_custom_backend(backend): @@ -61,6 +92,8 @@ def get_backend_bin_dir(ctx: AppContext, backend: Any) -> pathlib.Path: def get_backend_grammars_dir(ctx: AppContext, backend: Any) -> pathlib.Path: + if is_system_backend(backend): + raise ValueError("System (PATH) backend has no single grammars directory.") if backend == "custom": return ctx.paths.llama_custom_grammars if is_custom_backend(backend): @@ -71,6 +104,93 @@ def get_backend_grammars_dir(ctx: AppContext, backend: Any) -> pathlib.Path: def custom_backend_bin_label(backend: str) -> str: return f"llama/{CUSTOM_BACKEND_SPECS[backend]['folder']}/bin/" + +def is_system_discovery_tool(ctx: AppContext, tool: Any) -> bool: + """Restrict PATH discovery to known llama.cpp tools. + + Public launch tools come from ``ctx.services.llama_tools``; the memory + estimator is an internal target that must never become a launch option. + Anything else returns False so arbitrary executable names cannot pass + through the API into ``shutil.which()``. + """ + if not isinstance(tool, str) or not tool: + return False + try: + allowed = tuple(ctx.services.llama_tools or ()) + except Exception: + allowed = () + return tool in allowed or tool in SYSTEM_INTERNAL_TOOLS + + +def resolve_system_tool_executable( + ctx: AppContext, tool: str +) -> Optional[pathlib.Path]: + """Resolve one known tool from the inherited PATH, or None if unavailable. + + Uses ``shutil.which()`` on the platform-aware filename so normal PATH + precedence (including relative entries) applies. The result is made + absolute with ``os.path.abspath()`` without dereferencing symlinks, so + Nix/store wrappers keep their entry point. No shell is launched and no + repository installation directory is consulted: tools may live in + different directories and a missing tool never falls back elsewhere. + """ + if not is_system_discovery_tool(ctx, tool): + return None + try: + filename = ctx.services.get_tool_filename(tool) + except Exception: + return None + if not isinstance(filename, str) or not filename: + return None + if "/" in filename or "\\" in filename or "\x00" in filename: + return None + if ".." in pathlib.PurePosixPath(filename).parts: + return None + try: + found = shutil.which(filename) + except Exception: + return None + if not found: + return None + path = pathlib.Path(os.path.abspath(found)) + try: + if not path.is_file(): + return None + except OSError: + return None + current_platform = getattr(ctx.services, "current_platform", None) or sys.platform + if current_platform == "unknown": + current_platform = sys.platform + if current_platform != "win32": + try: + if not os.access(path, os.X_OK): + return None + except OSError: + return None + return path + + +def resolve_backend_tool_executable( + ctx: AppContext, backend: Any, tool: str +) -> Optional[pathlib.Path]: + """Shared resolver for the selected backend's tools. + + System backends resolve each known tool independently from PATH and + return None when it is missing or undiscoverable. Official and Custom + backends preserve the existing folder-path result (which callers check + for existence) so their behavior is unchanged. + """ + if is_system_backend(backend): + return resolve_system_tool_executable(ctx, tool) + try: + filename = ctx.services.get_tool_filename(tool) + except Exception: + return None + try: + return get_backend_bin_dir(ctx, backend) / filename + except Exception: + return None + # (gpu_target, family label) for every target upstream publishes. LEMONADE_ROCM_TARGETS = [ ("gfx103X", "AMD RDNA2"), @@ -114,7 +234,7 @@ def build_backend_specs(current_platform: str, current_arch: str) -> dict[str, A def with_custom(specs: dict[str, Any]) -> dict[str, Any]: return {**specs, **{ key: {"label": spec["label"]} for key, spec in CUSTOM_BACKEND_SPECS.items() - }} + }, SYSTEM_BACKEND_ID: {"label": SYSTEM_BACKEND_LABEL}} if current_platform == "win32": if current_arch == "arm64": @@ -489,11 +609,33 @@ def _validate_runtime_dependencies_uncached( current_platform = ctx.services.current_platform or sys.platform if current_platform == "unknown": current_platform = sys.platform + if is_system_backend(cfg.get("backend")): + # Stage 1 has no packaged-library requirement for PATH tools; the + # Stage 2 version probe will establish that an entry point starts. + # Report availability explicitly without scanning PATH directories. + for tool in tool_names: + try: + exe_path = ctx.services.find_tool_executable(tool) + except Exception: + exe_path = None + if exe_path is None: + missing_executables.append(ctx.services.get_tool_filename(tool)) + return { + "ok": True, + "checked": False, + "checked_tools": checked_tools, + "unchecked_tools": unchecked_tools, + "checked_runtime_files": checked_runtime_files, + "unchecked_runtime_files": unchecked_runtime_files, + "required_runtime_files": [], + "missing_runtime_files": [], + "missing_executables": missing_executables, + } runtime_dir = get_backend_bin_dir(ctx, cfg.get("backend")) for tool in tool_names: exe_path = ctx.services.find_tool_executable(tool) - if not exe_path.exists(): + if exe_path is None or not exe_path.exists(): missing_executables.append(ctx.services.get_tool_filename(tool)) continue try: @@ -594,7 +736,7 @@ def activate_custom_backend(ctx: AppContext, backend: str = "custom") -> dict[st with ctx.state.config_lock: cfg = dict(ctx.services.load_config()) - if cfg.get("backend") and not is_custom_backend(cfg.get("backend")) and cfg.get("tag"): + if cfg.get("backend") and is_official_backend(cfg.get("backend")) and cfg.get("tag"): cfg["official_install"] = { "backend": cfg["backend"], "tag": cfg["tag"], @@ -628,7 +770,7 @@ def get_official_install_status( backend = stored.get("backend") tag = stored.get("tag") version = stored.get("version") - if cfg.get("backend") and not is_custom_backend(cfg.get("backend")) and cfg.get("tag"): + if cfg.get("backend") and is_official_backend(cfg.get("backend")) and cfg.get("tag"): backend = cfg.get("backend") tag = cfg.get("tag") version = cfg.get("version") or tag diff --git a/backend/services/process_manager.py b/backend/services/process_manager.py index 09d650c..9b1d2ab 100644 --- a/backend/services/process_manager.py +++ b/backend/services/process_manager.py @@ -491,8 +491,10 @@ def _load_config_safe(ctx: AppContext) -> dict[str, Any]: def _fit_params_executable(ctx: AppContext) -> Any: - suffix = getattr(ctx.services, "binary_suffix", "") or "" cfg = _load_config_safe(ctx) + if llama_manager.is_system_backend(cfg.get("backend")): + return llama_manager.resolve_system_tool_executable(ctx, "llama-fit-params") + suffix = getattr(ctx.services, "binary_suffix", "") or "" bin_dir = llama_manager.get_backend_bin_dir(ctx, cfg.get("backend")) return bin_dir / f"llama-fit-params{suffix}" @@ -575,7 +577,7 @@ def get_buffer_types(ctx: AppContext) -> dict[str, Any]: tool = "llama-cli" if "llama-cli" in allowed_tools else (allowed_tools[0] if allowed_tools else "llama-cli") exe_name = ctx.services.get_tool_filename(tool) exe_path = ctx.services.find_tool_executable(tool) - if not exe_path.exists(): + if exe_path is None or not exe_path.exists(): return { "buffers": ["CPU"], "default": "CPU", @@ -719,7 +721,7 @@ def estimate_memory(ctx: AppContext, tool: str, args_list: Optional[Iterable[Any return {"error": f"Unknown tool: {tool!r}"} exe_path = _fit_params_executable(ctx) - if not exe_path.exists(): + if exe_path is None or not exe_path.exists(): return {"error": "llama-fit-params not found. Install or repair llama.cpp first."} runtime_health = dict(ctx.services.validate_runtime_dependencies([tool])) @@ -1063,6 +1065,11 @@ def normalize_process_env(value: Any) -> dict[str, str]: def _build_process_env(ctx: AppContext) -> dict[str, str]: env = os.environ.copy() cfg = _load_config_safe(ctx) + if llama_manager.is_system_backend(cfg.get("backend")): + # System tools use the inherited environment as-is; no repository + # binary/library directory is prepended. Per-launch LLAMA_/GGML_ + # overrides are merged by the caller. + return env bin_dir = llama_manager.get_backend_bin_dir(ctx, cfg.get("backend")) runtime_paths = [str(bin_dir)] existing_path = env.get("PATH", "") @@ -1145,7 +1152,7 @@ def _validate_launch_environment( ) -> tuple[Optional[Path], Optional[str]]: exe_name = ctx.services.get_tool_filename(tool) exe_path = ctx.services.find_tool_executable(tool) - if not exe_path.is_file(): + if exe_path is None or not exe_path.is_file(): return None, f"{exe_name} not found. Install llama.cpp first." current_platform = ctx.services.current_platform diff --git a/tests/backend/test_services.py b/tests/backend/test_services.py index 2284083..8196b00 100644 --- a/tests/backend/test_services.py +++ b/tests/backend/test_services.py @@ -406,13 +406,13 @@ def test_darwin_arm64_returns_metal_backends(self): def test_darwin_x64_returns_cpu_only(self): specs = llama_manager.build_backend_specs("darwin", "x64") - self.assertEqual(list(specs.keys()), ["cpu", "custom", "custom-02"]) + self.assertEqual(list(specs.keys()), ["cpu", "custom", "custom-02", "system"]) self.assertIn("macos-x64", specs["cpu"]["asset"]) def test_darwin_unknown_arch_returns_empty(self): specs = llama_manager.build_backend_specs("darwin", "ppc64") - self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}}) + self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}, "system": {"label": "System (PATH)"}}) def test_linux_x64_returns_cpu_vulkan_rocm_openvino(self): specs = llama_manager.build_backend_specs("linux", "x64") @@ -458,23 +458,23 @@ def test_linux_arm64_returns_cpu_and_vulkan(self): def test_linux_s390x_returns_cpu_only(self): specs = llama_manager.build_backend_specs("linux", "s390x") - self.assertEqual(list(specs.keys()), ["cpu", "custom", "custom-02"]) + self.assertEqual(list(specs.keys()), ["cpu", "custom", "custom-02", "system"]) self.assertIn("s390x", specs["cpu"]["asset"]) def test_linux_unknown_arch_returns_empty(self): specs = llama_manager.build_backend_specs("linux", "riscv64") - self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}}) + self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}, "system": {"label": "System (PATH)"}}) def test_unknown_platform_returns_empty(self): specs = llama_manager.build_backend_specs("freebsd", "x64") - self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}}) + self.assertEqual(specs, {"custom": {"label": "Custom"}, "custom-02": {"label": "Custom 02"}, "system": {"label": "System (PATH)"}}) def test_custom_backend_available_without_prebuilt_backend(self): specs = llama_manager.build_backend_specs("plan9", "weird64") - self.assertEqual(list(specs.keys()), ["custom", "custom-02"]) + self.assertEqual(list(specs.keys()), ["custom", "custom-02", "system"]) def test_asset_patterns_contain_tag_placeholder(self): for platform_name, arch in [("win32", "x64"), ("darwin", "arm64"), ("linux", "x64")]: diff --git a/tests/backend/test_system_path_discovery.py b/tests/backend/test_system_path_discovery.py new file mode 100644 index 0000000..7f37c95 --- /dev/null +++ b/tests/backend/test_system_path_discovery.py @@ -0,0 +1,482 @@ +"""Stage 1: System (PATH) backend classification and executable discovery. + +Covers the plan's verification list: PATH precedence, tools in different +directories, missing tools despite local official binaries, spaces in paths, +executable permissions, symlinks/wrappers, Windows .exe lookup, and existing +official/Custom resolution. All lookups use temporary fixture directories with +an isolated PATH, never the host's installed llama.cpp. + +Host independence: Windows' ``shutil.which()`` only matches executable +extensions, so extensionless Unix tool names cannot be discovered with the +real lookup on a Windows host. Tests that need Unix semantics use an explicit +exact-match ``which`` fixture (the plan's "fixtures for platform-specific +lookup"); cross-platform precedence/directory/space cases use ``.exe`` names +with the real lookup so stdlib PATH ordering is exercised on every host. +""" + +import contextlib +import os +import pathlib +import stat +import tempfile +import unittest +from unittest import mock + +from backend import app +from backend.http import Request +from backend.routes import status as status_routes +from backend.services import llama_manager, process_manager +from tests.backend.test_extracted_routes import DummyResponse +from tests.backend.test_services import make_service_context + + +def _make_ctx(root, platform="linux", suffix=""): + ctx = make_service_context(root) + services = ctx.services + services.current_platform = platform + services.current_arch = "x64" + services.binary_suffix = suffix + services.llama_tools = [ + "llama-cli", + "llama-server", + "llama-bench", + "llama-perplexity", + ] + services.get_tool_filename = lambda tool: tool + services.binary_suffix + services.backend_specs = llama_manager.build_backend_specs(platform, "x64") + store = {} + + def load_config(): + return dict(store) + + def save_config(cfg): + store.clear() + store.update(cfg) + + services.load_config = load_config + services.save_config = save_config + services.find_tool_executable = lambda tool: ( + llama_manager.resolve_backend_tool_executable( + ctx, services.load_config().get("backend"), tool + ) + ) + services.get_runtime_files = lambda: [] + services.get_platform_label = lambda: "Test" + services.get_llama_api_target = lambda: {"host": "127.0.0.1", "port": 8080} + services.validate_runtime_dependencies = lambda tools=None: ( + llama_manager.validate_runtime_dependencies(ctx, tools) + ) + return ctx + + +@contextlib.contextmanager +def _exact_which_fixture(): + """Unix-style exact-name lookup over the current PATH, any host. + + Matches the first PATH entry containing the exact filename, without + PATHEXT extension probing. Executability is left to the resolver under + test so permission cases stay deterministic. + """ + + def fake_which(filename, *args, **kwargs): + if not isinstance(filename, str) or not filename: + return None + if "/" in filename or "\\" in filename: + return None + for entry in os.environ.get("PATH", "").split(os.pathsep): + if not entry: + continue + candidate = pathlib.Path(entry) / filename + try: + if candidate.is_file(): + return str(candidate) + except OSError: + continue + return None + + with mock.patch.object(llama_manager.shutil, "which", side_effect=fake_which): + yield + + +def _write_executable(path, content="tool"): + path = pathlib.Path(path) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content) + try: + mode = path.stat().st_mode + path.chmod(mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + except OSError: + pass + return path + + +class SystemBackendSpecsTests(unittest.TestCase): + def test_system_listed_with_path_label_and_no_download_asset(self): + for platform, arch in [("win32", "x64"), ("darwin", "arm64"), ("linux", "x64")]: + with self.subTest(platform=platform, arch=arch): + specs = llama_manager.build_backend_specs(platform, arch) + self.assertIn("system", specs) + self.assertEqual(specs["system"]["label"], "System (PATH)") + self.assertNotIn("asset", specs["system"]) + + def test_custom_specs_unchanged_and_startup_creates_no_system_dirs(self): + self.assertEqual( + set(llama_manager.CUSTOM_BACKEND_SPECS), + {"custom", "custom-02"}, + ) + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp) + # Startup creates official + custom directories only; there is no + # System folder to create and none may appear. + for backend in list(llama_manager.CUSTOM_BACKEND_SPECS) + ["cpu"]: + llama_manager.get_backend_bin_dir(ctx, backend).mkdir( + parents=True, exist_ok=True + ) + self.assertFalse((ctx.paths.llama / "system").exists()) + + def test_helpers_classify_backends(self): + self.assertTrue(llama_manager.is_system_backend("system")) + self.assertFalse(llama_manager.is_system_backend("cpu")) + self.assertFalse(llama_manager.is_system_backend("custom")) + self.assertFalse(llama_manager.is_system_backend(None)) + self.assertTrue(llama_manager.is_externally_managed_backend("system")) + self.assertFalse(llama_manager.is_externally_managed_backend("cpu")) + self.assertTrue(llama_manager.is_official_backend("cpu")) + self.assertFalse(llama_manager.is_official_backend("custom")) + self.assertFalse(llama_manager.is_official_backend("custom-02")) + self.assertFalse(llama_manager.is_official_backend("system")) + self.assertFalse(llama_manager.is_official_backend(None)) + + def test_folder_helpers_reject_system(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp) + with self.assertRaises(ValueError): + llama_manager.get_backend_bin_dir(ctx, "system") + with self.assertRaises(ValueError): + llama_manager.get_backend_grammars_dir(ctx, "system") + + def test_system_never_recorded_as_official_install(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp) + ctx.services.save_config( + {"backend": "system", "tag": "system", "version": "system"} + ) + official = llama_manager.get_official_install_status(ctx) + self.assertIsNone(official["backend"]) + self.assertIsNone(official["tag"]) + + +class SystemPathDiscoveryTests(unittest.TestCase): + def test_path_precedence_first_entry_wins(self): + # .exe names use the real stdlib lookup on every host. + with tempfile.TemporaryDirectory() as tmp: + root = pathlib.Path(tmp) + first = root / "first" + second = root / "second" + _write_executable(first / "llama-server.exe", "first") + _write_executable(second / "llama-server.exe", "second") + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + ctx.services.save_config({"backend": "system"}) + with mock.patch.dict( + os.environ, {"PATH": os.pathsep.join([str(first), str(second)])} + ): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, first / "llama-server.exe") + with mock.patch.dict( + os.environ, {"PATH": os.pathsep.join([str(second), str(first)])} + ): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, second / "llama-server.exe") + + def test_tools_in_different_directories(self): + with tempfile.TemporaryDirectory() as tmp: + root = pathlib.Path(tmp) + server_dir = root / "server-dir" + cli_dir = root / "cli-dir" + _write_executable(server_dir / "llama-server.exe", "server") + _write_executable(cli_dir / "llama-cli.exe", "cli") + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + with mock.patch.dict( + os.environ, + {"PATH": os.pathsep.join([str(server_dir), str(cli_dir)])}, + ): + server = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + cli = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-cli" + ) + self.assertEqual(server, server_dir / "llama-server.exe") + self.assertEqual(cli, cli_dir / "llama-cli.exe") + + def test_missing_tool_despite_official_binaries_has_no_fallback(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + official = ctx.paths.llama_bin + official.mkdir(parents=True, exist_ok=True) + _write_executable(official / "llama-server.exe", "official") + empty = pathlib.Path(tmp) / "empty-path" + empty.mkdir() + with mock.patch.dict(os.environ, {"PATH": str(empty)}): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertIsNone(resolved) + + def test_spaces_in_paths(self): + with tempfile.TemporaryDirectory() as tmp: + spaced = pathlib.Path(tmp) / "dir with spaces" / "tools" + _write_executable(spaced / "llama-server.exe", "spaced") + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + with mock.patch.dict(os.environ, {"PATH": str(spaced)}): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, spaced / "llama-server.exe") + + def test_executable_permission_required_on_unix_not_windows(self): + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + bindir.mkdir(parents=True) + candidate = bindir / "llama-server" + candidate.write_text("not-executable") + unix_ctx = _make_ctx(tmp, platform="linux") + win_ctx = _make_ctx(tmp, platform="win32") + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=False + ): + self.assertIsNone( + llama_manager.resolve_backend_tool_executable( + unix_ctx, "system", "llama-server" + ) + ) + # Windows never requires the executable bit. + self.assertEqual( + llama_manager.resolve_backend_tool_executable( + win_ctx, "system", "llama-server" + ), + bindir / "llama-server", + ) + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=True + ): + self.assertEqual( + llama_manager.resolve_backend_tool_executable( + unix_ctx, "system", "llama-server" + ), + bindir / "llama-server", + ) + + def test_symlink_entry_point_preserved(self): + with tempfile.TemporaryDirectory() as tmp: + root = pathlib.Path(tmp) + store = root / "store" + linkdir = root / "links" + store.mkdir() + linkdir.mkdir() + target = _write_executable(store / "llama-server-v1", "real") + link = linkdir / "llama-server" + try: + if link.exists() or link.is_symlink(): + link.unlink() + link.symlink_to(target) + except (OSError, NotImplementedError) as exc: + self.skipTest(f"symlinks unavailable: {exc}") + ctx = _make_ctx(tmp, platform="linux") + with mock.patch.dict(os.environ, {"PATH": str(linkdir)}): + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=True + ): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, linkdir / "llama-server") + # abspath() preserves the link; resolve() would collapse it. + self.assertNotEqual(resolved, target) + self.assertEqual(resolved.resolve(), target.resolve()) + + def test_wrapper_script_resolves(self): + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + wrapper = bindir / "llama-server" + bindir.mkdir(parents=True) + wrapper.write_text("#!/bin/sh\nexec /opt/llama/llama-server \"$@\"\n") + ctx = _make_ctx(tmp, platform="linux") + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=True + ): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, wrapper) + + def test_entry_point_never_resolved_through_symlinks(self): + # Proves the resolver uses abspath(), not resolve()/realpath(), so + # Nix/store wrappers keep their entry point. No symlink privilege + # needed: any resolve() call fails the test. + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + _write_executable(bindir / "llama-server", "entry") + ctx = _make_ctx(tmp, platform="linux") + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=True + ), mock.patch.object( + pathlib.Path, + "resolve", + side_effect=AssertionError("must not resolve"), + ): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, bindir / "llama-server") + + def test_windows_exe_lookup(self): + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + _write_executable(bindir / "llama-server.exe", "windows") + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + resolved = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + self.assertEqual(resolved, bindir / "llama-server.exe") + # An extensionless file is not a fallback for the .exe name. + (bindir / "llama-server.exe").unlink() + _write_executable(bindir / "llama-server", "extensionless") + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + self.assertIsNone( + llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + ) + + def test_unknown_tools_rejected_and_fit_params_internal(self): + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + _write_executable(bindir / "evil-tool", "evil") + _write_executable(bindir / "llama-fit-params", "estimator") + ctx = _make_ctx(tmp, platform="linux") + self.assertNotIn("llama-fit-params", list(ctx.services.llama_tools)) + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + with _exact_which_fixture(), mock.patch.object( + llama_manager.os, "access", return_value=True + ): + self.assertIsNone( + llama_manager.resolve_backend_tool_executable( + ctx, "system", "evil-tool" + ) + ) + self.assertIsNone( + llama_manager.resolve_backend_tool_executable( + ctx, "system", "../bin/llama-server" + ) + ) + fit_params = llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-fit-params" + ) + self.assertEqual(fit_params, bindir / "llama-fit-params") + + def test_official_and_custom_resolution_preserved(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp) + official = _write_executable(ctx.paths.llama_bin / "llama-server", "cpu") + custom_dir = llama_manager.get_backend_bin_dir(ctx, "custom") + custom_tool = _write_executable(custom_dir / "llama-server", "custom") + self.assertEqual( + llama_manager.resolve_backend_tool_executable( + ctx, "cpu", "llama-server" + ), + official, + ) + self.assertEqual( + llama_manager.resolve_backend_tool_executable( + ctx, "custom", "llama-server" + ), + custom_tool, + ) + # Folder backends keep their path result even when the file is + # absent; only System reports None explicitly. + (ctx.paths.llama_bin / "llama-cli").unlink(missing_ok=True) + self.assertEqual( + llama_manager.resolve_backend_tool_executable(ctx, "cpu", "llama-cli"), + ctx.paths.llama_bin / "llama-cli", + ) + + +class SystemMissingToolConsumersTests(unittest.TestCase): + def test_status_launch_and_runtime_handle_missing_system_tool(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + ctx.services.save_config({"backend": "system", "tag": "system"}) + empty = pathlib.Path(tmp) / "empty" + empty.mkdir() + with mock.patch.dict(os.environ, {"PATH": str(empty)}): + self.assertIsNone( + llama_manager.resolve_backend_tool_executable( + ctx, "system", "llama-server" + ) + ) + response = DummyResponse() + with mock.patch( + "backend.services.model_dir.get_models_dir_info", + return_value={"models_dir": str(ctx.paths.models)}, + ), mock.patch( + "backend.services.official_backends.get_backend_specs", + return_value=dict(ctx.services.backend_specs), + ): + status_routes.get_status( + Request("GET", "/api/status", "", {}), response, ctx + ) + self.assertFalse( + response.payload["executables"][ + ctx.services.get_tool_filename("llama-server") + ] + ) + exe_path, error = process_manager._validate_launch_environment( + ctx, "llama-server" + ) + self.assertIsNone(exe_path) + self.assertIn("not found", error) + buffers = process_manager.get_buffer_types(ctx) + self.assertEqual(buffers["buffers"], ["CPU"]) + self.assertIn("not found", buffers["error"]) + health = llama_manager.validate_runtime_dependencies(ctx) + self.assertTrue(health["ok"]) + self.assertFalse(health["checked"]) + + def test_app_find_tool_delegates_and_runtime_files_empty_for_system(self): + with tempfile.TemporaryDirectory() as tmp: + bindir = pathlib.Path(tmp) / "bin" + expected = _write_executable(bindir / "llama-server.exe", "system") + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + ctx.services.save_config({"backend": "system"}) + with mock.patch.dict(os.environ, {"PATH": str(bindir)}): + with mock.patch.object( + app, "APP_CONTEXT", ctx + ), mock.patch.object( + app, "load_config", ctx.services.load_config + ), mock.patch.object( + app, "CONFIG_FILE", ctx.paths.config_file + ): + self.assertEqual( + app.find_tool_executable("llama-server"), expected + ) + self.assertEqual(app.get_runtime_files(), []) + empty = pathlib.Path(tmp) / "empty" + empty.mkdir() + with mock.patch.dict(os.environ, {"PATH": str(empty)}): + with mock.patch.object( + app, "APP_CONTEXT", ctx + ), mock.patch.object(app, "load_config", ctx.services.load_config): + self.assertIsNone(app.find_tool_executable("llama-server")) + + +if __name__ == "__main__": + unittest.main() From a6e41c8904fa63e605e63e33da0a5f942e2f8af2 Mon Sep 17 00:00:00 2001 From: Thomas Date: Thu, 8 Oct 2026 21:09:59 -0400 Subject: [PATCH 3/7] Probe System backend tools before launch - Check PATH executables with bounded `--version` probes and cache results by file identity - Use the discovered System tools for launch and memory-estimator validation --- backend/services/llama_manager.py | 233 ++++++-- backend/services/process_manager.py | 127 ++++- tests/backend/test_process_environment.py | 36 ++ tests/backend/test_system_runtime_probes.py | 564 ++++++++++++++++++++ 4 files changed, 907 insertions(+), 53 deletions(-) create mode 100644 tests/backend/test_system_runtime_probes.py diff --git a/backend/services/llama_manager.py b/backend/services/llama_manager.py index 83af60e..97aba0d 100644 --- a/backend/services/llama_manager.py +++ b/backend/services/llama_manager.py @@ -22,11 +22,13 @@ from ..context import AppContext from ..http import sanitize_error from . import official_backends +from .subprocess_utils import get_no_window_creationflags RPATH_LIBRARY_RE = re.compile(r"^\s*@rpath/([^\s(]+)") LDD_NOT_FOUND_RE = re.compile(r"^\s*([^\s]+)\s+=>\s+not found\s*$") SHA256_DIGEST_RE = re.compile(r"^sha256:([0-9a-fA-F]{64})$") +BUILD_TAG_RE = re.compile(r"\bbuild\s+(\d+)\b", re.I) # Runtime dependency validation shells out to otool on macOS and ldd on Linux, # so results are cached briefly. Config-changing operations (install, cleanup, @@ -55,9 +57,18 @@ SYSTEM_BACKEND_LABEL = "System (PATH)" # Internal PATH discovery target that is never a public launch tool. Memory -# estimation locates it separately (Stage 2); Stage 1 only resolves it. +# estimation locates it separately; Stage 1 only resolves it. SYSTEM_INTERNAL_TOOLS = ("llama-fit-params",) +# Bounded, model-free `--version` execution check for System entry points. +SYSTEM_VERSION_PROBE_TIMEOUT_SECONDS = 5 + + +def parse_llama_build_tag(output: str) -> Optional[str]: + """Return the normalized ``bNNNNN`` tag from `--version` output, if present.""" + match = BUILD_TAG_RE.search(output or "") + return f"b{match.group(1)}" if match else None + def is_custom_backend(backend: Any) -> bool: return isinstance(backend, str) and backend in CUSTOM_BACKEND_SPECS @@ -191,6 +202,164 @@ def resolve_backend_tool_executable( except Exception: return None + +def probe_system_tool_executable( + ctx: AppContext, tool: str, exe_path: pathlib.Path +) -> dict[str, Any]: + """Run a bounded, model-free `--version` check of one System entry point. + + Uses the discovered entry point itself (never a resolved symlink target), + the inherited runtime environment, and the shared window-hiding flags. + A successful run with an unrecognized build format reports ``ok=True`` + with ``build_tag=None``; that is distinct from a failure to execute. + """ + try: + completed = subprocess.run( + [str(exe_path), "--version"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + env=os.environ.copy(), + cwd=str(ctx.paths.root), + timeout=SYSTEM_VERSION_PROBE_TIMEOUT_SECONDS, + check=False, + creationflags=get_no_window_creationflags(), + ) + except subprocess.TimeoutExpired: + print( + f"[llama_manager] system version probe timed out for {tool}: {exe_path}", + file=sys.stderr, + ) + return { + "ok": False, + "build_tag": None, + "error": f"timed out after {SYSTEM_VERSION_PROBE_TIMEOUT_SECONDS} seconds", + "executable": str(exe_path), + } + except OSError as exc: + print( + f"[llama_manager] system version probe failed for {tool}: {exc}", + file=sys.stderr, + ) + return { + "ok": False, + "build_tag": None, + "error": f"could not start ({exc})", + "executable": str(exe_path), + } + if completed.returncode != 0: + print( + f"[llama_manager] system version probe exited with status " + f"{completed.returncode} for {tool}: {exe_path}", + file=sys.stderr, + ) + return { + "ok": False, + "build_tag": None, + "error": f"exited with status {completed.returncode}", + "executable": str(exe_path), + } + combined = "\n".join( + part for part in (completed.stdout, completed.stderr) if part + ) + return { + "ok": True, + "build_tag": parse_llama_build_tag(combined), + "error": None, + "executable": str(exe_path), + } + + +def _system_executable_identity(exe_path: pathlib.Path) -> Optional[tuple[Any, ...]]: + """File/symlink identity for cache invalidation across package updates. + + ``os.stat()`` follows the entry point, so replacing a symlink target + while the GUI is open changes the identity and stale probe results are + never reused. Returns None when the file cannot be stated. + """ + try: + stat_result = os.stat(exe_path) + except OSError: + return None + return ( + stat_result.st_dev, + stat_result.st_ino, + stat_result.st_size, + stat_result.st_mtime_ns, + ) + + +def _system_runtime_cache_key( + ctx: AppContext, backend: Any, tool_names: tuple[str, ...] +) -> tuple[Any, ...]: + """Cache key including each tool's path and file identity. + + Discovery itself is cheap, so resolve on every status poll and let the + identity detect replaced/removed tools; only cache misses spawn probes. + """ + parts = [] + for tool in tool_names: + try: + exe_path = resolve_backend_tool_executable(ctx, backend, tool) + except Exception: + exe_path = None + parts.append( + ( + tool, + str(exe_path) if exe_path is not None else None, + _system_executable_identity(exe_path) if exe_path is not None else None, + ) + ) + return (backend, tool_names, tuple(parts)) + + +def _validate_system_runtime_dependencies( + ctx: AppContext, tool_names: tuple[str, ...] +) -> dict[str, Any]: + """Execution-check health for System tools; never requires packaged libs. + + A successful probe establishes that the entry point starts. It does not + establish model loading or GPU compatibility. Missing tools are reported + explicitly and never fail the check on their own; only a present tool + that fails to execute does. + """ + checked_tools: list[str] = [] + probe_failures: list[str] = [] + missing_executables: list[str] = [] + build_tags: dict[str, str] = {} + probes: dict[str, dict[str, Any]] = {} + for tool in tool_names: + try: + exe_path = resolve_backend_tool_executable( + ctx, SYSTEM_BACKEND_ID, tool + ) + except Exception: + exe_path = None + if exe_path is None: + missing_executables.append(ctx.services.get_tool_filename(tool)) + continue + probe = probe_system_tool_executable(ctx, tool, exe_path) + probes[tool] = probe + if probe["ok"]: + checked_tools.append(tool) + if probe["build_tag"]: + build_tags[tool] = probe["build_tag"] + else: + probe_failures.append(tool) + return { + "ok": not probe_failures, + "checked": bool(checked_tools), + "checked_tools": checked_tools, + "unchecked_tools": sorted(probe_failures), + "checked_runtime_files": [], + "unchecked_runtime_files": [], + "required_runtime_files": [], + "missing_runtime_files": [], + "missing_executables": missing_executables, + "build_tags": build_tags, + "system_probes": probes, + } + # (gpu_target, family label) for every target upstream publishes. LEMONADE_ROCM_TARGETS = [ ("gfx103X", "AMD RDNA2"), @@ -563,16 +732,6 @@ def validate_runtime_dependencies( current_platform = ctx.services.current_platform or sys.platform if current_platform == "unknown": current_platform = sys.platform - skipped = _linux_runtime_validation_opt_out(current_platform) - if skipped is not None: - return skipped - if current_platform != "darwin" and not current_platform.startswith("linux"): - return { - "ok": True, - "checked": False, - "required_runtime_files": [], - "missing_runtime_files": [], - } tool_names = tuple(tools) if tools is not None else () if not tool_names: @@ -584,6 +743,33 @@ def validate_runtime_dependencies( print(f"[llama_manager] load_config failed during runtime validation: {e}", file=sys.stderr) cfg = {} + if is_system_backend(cfg.get("backend")): + # System health comes from bounded `--version` execution probes, so it + # runs on every platform. Neither the Linux-only ldd opt-out nor the + # Unix-only packaged-library gate applies here. + cache_key = _system_runtime_cache_key(ctx, cfg.get("backend"), tool_names) + now = time.monotonic() + with ctx.state.runtime_health_lock: + cached = ctx.state.runtime_health_cache.get(cache_key) + if cached is not None and now - cached[0] < RUNTIME_HEALTH_CACHE_TTL_SECONDS: + return copy.deepcopy(cached[1]) + + result = _validate_system_runtime_dependencies(ctx, tool_names) + with ctx.state.runtime_health_lock: + ctx.state.runtime_health_cache[cache_key] = (time.monotonic(), result) + return copy.deepcopy(result) + + skipped = _linux_runtime_validation_opt_out(current_platform) + if skipped is not None: + return skipped + if current_platform != "darwin" and not current_platform.startswith("linux"): + return { + "ok": True, + "checked": False, + "required_runtime_files": [], + "missing_runtime_files": [], + } + cache_key = (cfg.get("backend"), tool_names) now = time.monotonic() with ctx.state.runtime_health_lock: @@ -610,27 +796,7 @@ def _validate_runtime_dependencies_uncached( if current_platform == "unknown": current_platform = sys.platform if is_system_backend(cfg.get("backend")): - # Stage 1 has no packaged-library requirement for PATH tools; the - # Stage 2 version probe will establish that an entry point starts. - # Report availability explicitly without scanning PATH directories. - for tool in tool_names: - try: - exe_path = ctx.services.find_tool_executable(tool) - except Exception: - exe_path = None - if exe_path is None: - missing_executables.append(ctx.services.get_tool_filename(tool)) - return { - "ok": True, - "checked": False, - "checked_tools": checked_tools, - "unchecked_tools": unchecked_tools, - "checked_runtime_files": checked_runtime_files, - "unchecked_runtime_files": unchecked_runtime_files, - "required_runtime_files": [], - "missing_runtime_files": [], - "missing_executables": missing_executables, - } + return _validate_system_runtime_dependencies(ctx, tool_names) runtime_dir = get_backend_bin_dir(ctx, cfg.get("backend")) for tool in tool_names: @@ -811,8 +977,7 @@ def _probe_official_build(ctx: AppContext) -> tuple[bool, Optional[str]]: return False, None if result.returncode != 0: return False, None - match = re.search(r"\bbuild\s+(\d+)\b", result.stdout + "\n" + result.stderr, re.I) - return True, f"b{match.group(1)}" if match else None + return True, parse_llama_build_tag(result.stdout + "\n" + result.stderr) def activate_official_backend(ctx: AppContext, backend: str) -> dict[str, Any]: diff --git a/backend/services/process_manager.py b/backend/services/process_manager.py index 9b1d2ab..f2f0934 100644 --- a/backend/services/process_manager.py +++ b/backend/services/process_manager.py @@ -574,7 +574,19 @@ def parse_list_devices_output(output: str) -> list[str]: def get_buffer_types(ctx: AppContext) -> dict[str, Any]: allowed_tools = list(ctx.services.llama_tools or []) - tool = "llama-cli" if "llama-cli" in allowed_tools else (allowed_tools[0] if allowed_tools else "llama-cli") + if llama_manager.is_system_backend(_load_config_safe(ctx).get("backend")): + # PATH tools need not share one directory or build: prefer an + # available CLI probe target and use the server when CLI is absent. + ordered = [tool for tool in ("llama-cli", "llama-server") if tool in allowed_tools] + if not ordered: + ordered = ["llama-cli"] + tool = next( + (candidate for candidate in ordered + if ctx.services.find_tool_executable(candidate) is not None), + ordered[0], + ) + else: + tool = "llama-cli" if "llama-cli" in allowed_tools else (allowed_tools[0] if allowed_tools else "llama-cli") exe_name = ctx.services.get_tool_filename(tool) exe_path = ctx.services.find_tool_executable(tool) if exe_path is None or not exe_path.exists(): @@ -711,26 +723,33 @@ def _memory_estimate_args(args: list[str]) -> list[str]: return filtered_args -def estimate_memory(ctx: AppContext, tool: str, args_list: Optional[Iterable[Any]], env: Any = None) -> dict[str, Any]: +def _validate_system_estimator(ctx: AppContext, exe_path: Path) -> Optional[str]: + """Confirm the System estimator starts; estimation never blocks launches.""" try: - environment = normalize_process_env(env) - except ValueError as exc: - return {"error": str(exc)} - allowed_tools = ctx.services.llama_tools or [] - if tool not in allowed_tools: - return {"error": f"Unknown tool: {tool!r}"} - - exe_path = _fit_params_executable(ctx) - if exe_path is None or not exe_path.exists(): - return {"error": "llama-fit-params not found. Install or repair llama.cpp first."} + health = dict(ctx.services.validate_runtime_dependencies(["llama-fit-params"])) + except Exception as exc: + print(f"[process] system estimator probe lookup failed: {exc}", file=sys.stderr) + health = {} + probe = (health.get("system_probes") or {}).get("llama-fit-params") + if probe is None or probe.get("executable") != str(exe_path): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-fit-params", exe_path + ) + if probe.get("ok"): + return None + detail = probe.get("error") or "could not be started" + return ( + f"llama-fit-params {detail}. " + "Memory estimation is unavailable for this installation." + ) - runtime_health = dict(ctx.services.validate_runtime_dependencies([tool])) - missing_runtime_files = runtime_health.get("missing_runtime_files") or [] - if missing_runtime_files: - missing = ", ".join(str(name) for name in missing_runtime_files) - plural = "libraries" if len(missing_runtime_files) != 1 else "library" - return {"error": f"Missing llama.cpp runtime {plural}: {missing}."} +def _run_memory_estimate_command( + ctx: AppContext, + exe_path: Path, + args_list: Optional[Iterable[Any]], + environment: Mapping[str, str], +) -> dict[str, Any]: args = flatten_launch_args(args_list) filtered_args = _memory_estimate_args(args) @@ -776,6 +795,41 @@ def estimate_memory(ctx: AppContext, tool: str, args_list: Optional[Iterable[Any } +def estimate_memory(ctx: AppContext, tool: str, args_list: Optional[Iterable[Any]], env: Any = None) -> dict[str, Any]: + try: + environment = normalize_process_env(env) + except ValueError as exc: + return {"error": str(exc)} + allowed_tools = ctx.services.llama_tools or [] + if tool not in allowed_tools: + return {"error": f"Unknown tool: {tool!r}"} + + exe_path = _fit_params_executable(ctx) + backend = _load_config_safe(ctx).get("backend") + if exe_path is None or not exe_path.exists(): + if llama_manager.is_system_backend(backend): + return {"error": ( + "llama-fit-params was not found on the PATH inherited by Llama GUI. " + "Memory estimation is unavailable for this installation." + )} + return {"error": "llama-fit-params not found. Install or repair llama.cpp first."} + + if llama_manager.is_system_backend(backend): + # Validate the estimator itself rather than the requested tool. + estimator_error = _validate_system_estimator(ctx, exe_path) + if estimator_error is not None: + return {"error": estimator_error} + else: + runtime_health = dict(ctx.services.validate_runtime_dependencies([tool])) + missing_runtime_files = runtime_health.get("missing_runtime_files") or [] + if missing_runtime_files: + missing = ", ".join(str(name) for name in missing_runtime_files) + plural = "libraries" if len(missing_runtime_files) != 1 else "library" + return {"error": f"Missing llama.cpp runtime {plural}: {missing}."} + + return _run_memory_estimate_command(ctx, exe_path, args_list, environment) + + def _launch_api_target_values( ctx: AppContext, args_list: Optional[Iterable[Any]] ) -> tuple[Any, Any]: @@ -1147,19 +1201,51 @@ def redact_sensitive_text(text: Any, args: Iterable[Any]) -> str: return result +def _validate_system_launch_probe( + ctx: AppContext, tool: str, exe_path: Path, exe_name: str +) -> tuple[Optional[Path], Optional[str]]: + """Confirm a System entry point starts, using the cached version probe. + + Falls back to a direct probe when discovery changed since the cached + result, so the validated entry point is always the one passed to Popen. + """ + try: + health = dict(ctx.services.validate_runtime_dependencies([tool])) + except Exception as exc: + print(f"[process] system runtime probe lookup failed: {exc}", file=sys.stderr) + health = {} + probe = (health.get("system_probes") or {}).get(tool) + if probe is None or probe.get("executable") != str(exe_path): + probe = llama_manager.probe_system_tool_executable(ctx, tool, exe_path) + if probe.get("ok"): + return exe_path, None + detail = probe.get("error") or "could not be started" + return None, ( + f"{exe_name} {detail}. " + f"Check that it runs with `{exe_name} --version`." + ) + + def _validate_launch_environment( ctx: AppContext, tool: str ) -> tuple[Optional[Path], Optional[str]]: exe_name = ctx.services.get_tool_filename(tool) exe_path = ctx.services.find_tool_executable(tool) + backend = _load_config_safe(ctx).get("backend") if exe_path is None or not exe_path.is_file(): + if llama_manager.is_system_backend(backend): + return None, ( + f"{exe_name} was not found on the PATH inherited by Llama GUI. " + "Install it with your system package manager, then restart the GUI." + ) return None, f"{exe_name} not found. Install llama.cpp first." current_platform = ctx.services.current_platform if current_platform == "unknown": current_platform = sys.platform if current_platform != "win32" and not os.access(exe_path, os.X_OK): - backend = _load_config_safe(ctx).get("backend") + if llama_manager.is_system_backend(backend): + return None, f"{exe_name} is not executable. Run chmod +x on {exe_path}." recovery = ( f"Run chmod +x on {llama_manager.custom_backend_bin_label(backend)}{exe_name}." if llama_manager.is_custom_backend(backend) @@ -1167,6 +1253,9 @@ def _validate_launch_environment( ) return None, f"{exe_name} is not executable. {recovery}" + if llama_manager.is_system_backend(backend): + return _validate_system_launch_probe(ctx, tool, exe_path, exe_name) + runtime_health = dict(ctx.services.validate_runtime_dependencies([tool])) missing_runtime_files = runtime_health.get("missing_runtime_files") or [] if not missing_runtime_files: diff --git a/tests/backend/test_process_environment.py b/tests/backend/test_process_environment.py index ed1f811..8d72e83 100644 --- a/tests/backend/test_process_environment.py +++ b/tests/backend/test_process_environment.py @@ -51,3 +51,39 @@ def test_literal_values_and_empty_values_are_preserved(self): value = {"GGML_TEST": "a=b; $HOME 'quoted'", "GGML_EMPTY": ""} self.assertEqual(process_manager.normalize_process_env(value), value) + +class SystemProcessEnvironmentTests(unittest.TestCase): + def _system_ctx(self): + ctx = AppContext() + ctx.services.load_config = lambda: {"backend": "system", "tag": "system"} + ctx.services.llama_tools = ["llama-cli", "llama-server"] + ctx.services.get_tool_filename = lambda tool: tool + ctx.services.current_platform = "linux" + return ctx + + def test_system_env_preserves_inherited_paths_without_repo_prepend(self): + ctx = self._system_ctx() + with mock.patch.dict( + os.environ, + { + "PATH": "inherited-path", + "LD_LIBRARY_PATH": "inherited-libs", + "DYLD_LIBRARY_PATH": "inherited-dyld", + }, + ): + env = process_manager._build_process_env(ctx) + self.assertEqual(env["PATH"], "inherited-path") + self.assertEqual(env["LD_LIBRARY_PATH"], "inherited-libs") + self.assertEqual(env["DYLD_LIBRARY_PATH"], "inherited-dyld") + self.assertNotIn(str(ctx.paths.llama_bin), env["PATH"]) + self.assertNotIn(str(ctx.paths.llama_bin), env["LD_LIBRARY_PATH"]) + + def test_official_env_still_prepends_repo_bin_dir(self): + ctx = self._system_ctx() + ctx.services.load_config = lambda: {"backend": "cpu", "tag": "b1"} + with mock.patch.dict(os.environ, {"PATH": "inherited-path"}): + env = process_manager._build_process_env(ctx) + self.assertTrue( + env["PATH"].startswith(str(ctx.paths.llama_bin) + os.pathsep) + ) + diff --git a/tests/backend/test_system_runtime_probes.py b/tests/backend/test_system_runtime_probes.py new file mode 100644 index 0000000..3547d55 --- /dev/null +++ b/tests/backend/test_system_runtime_probes.py @@ -0,0 +1,564 @@ +"""Stage 2: System (PATH) runtime probes, environment, and process integration. + +Uses temporary fixture executables, mocked ``subprocess.run`` responses, and +an isolated PATH. No host llama.cpp installation, model, or GPU is required. +""" + +import os +import pathlib +import subprocess +import sys +import tempfile +import unittest +from unittest import mock + +from backend.services import llama_manager, process_manager +from backend.services.subprocess_utils import get_no_window_creationflags +from tests.backend.test_system_path_discovery import ( + _exact_which_fixture, + _make_ctx, + _write_executable, +) + + +def _completed(args, returncode=0, stdout="", stderr=""): + return subprocess.CompletedProcess(args, returncode, stdout, stderr) + + +def _probe_ok(tool, exe_path, tag="b1234"): + return { + "ok": True, + "build_tag": tag, + "error": None, + "executable": str(exe_path), + } + + +def _health_with_probes(probed, missing=(), ok=True): + """Canned ``validate_runtime_dependencies`` result for integration tests.""" + build_tags = { + tool: probe["build_tag"] + for tool, probe in probed.items() + if probe.get("build_tag") + } + return { + "ok": ok, + "checked": bool(probed), + "checked_tools": sorted(probed), + "unchecked_tools": [], + "checked_runtime_files": [], + "unchecked_runtime_files": [], + "required_runtime_files": [], + "missing_runtime_files": [], + "missing_executables": list(missing), + "build_tags": build_tags, + "system_probes": dict(probed), + } + + +class SystemVersionProbeTests(unittest.TestCase): + def test_known_build_tag_parsed_from_stdout_and_stderr(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + exe = pathlib.Path(tmp) / "llama-server" + with mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed( + [str(exe), "--version"], stdout="version: b1234 (build 1234)" + ), + ) as run: + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertTrue(probe["ok"]) + self.assertEqual(probe["build_tag"], "b1234") + self.assertIsNone(probe["error"]) + self.assertEqual(probe["executable"], str(exe)) + run.assert_called_once() + with mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed( + [str(exe), "--version"], stdout="noise", stderr="Build 99" + ), + ): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertTrue(probe["ok"]) + self.assertEqual(probe["build_tag"], "b99") + + def test_unknown_build_format_is_successful_execution(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + exe = pathlib.Path(tmp) / "llama-server" + with mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed( + [str(exe), "--version"], stdout="llama-pack 2.0-custom" + ), + ): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertTrue(probe["ok"]) + self.assertIsNone(probe["build_tag"]) + + def test_nonzero_exit_timeout_and_spawn_failure(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + exe = pathlib.Path(tmp) / "llama-server" + with mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed([str(exe), "--version"], returncode=1), + ): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertFalse(probe["ok"]) + self.assertIn("status 1", probe["error"]) + with mock.patch.object( + llama_manager.subprocess, + "run", + side_effect=subprocess.TimeoutExpired([str(exe)], 5), + ): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertFalse(probe["ok"]) + self.assertIn("timed out", probe["error"]) + with mock.patch.object( + llama_manager.subprocess, + "run", + side_effect=OSError("loader missing"), + ): + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", exe + ) + self.assertFalse(probe["ok"]) + self.assertIn("could not start", probe["error"]) + + def test_probe_uses_entry_point_inherited_env_and_hiding(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + exe = pathlib.Path(tmp) / "llama-server" + with mock.patch.dict(os.environ, {"PATH": "probe-path"}), mock.patch.object( + llama_manager.subprocess, "run", return_value=_completed([], stdout="b7") + ) as run: + llama_manager.probe_system_tool_executable(ctx, "llama-server", exe) + kwargs = run.call_args + self.assertEqual(kwargs.args[0], [str(exe), "--version"]) + self.assertNotIn("shell", kwargs.kwargs) + self.assertEqual(kwargs.kwargs["timeout"], 5) + self.assertEqual(kwargs.kwargs["cwd"], str(ctx.paths.root)) + self.assertEqual( + kwargs.kwargs["creationflags"], get_no_window_creationflags() + ) + # Inherited environment: PATH preserved, not rewritten. + self.assertEqual(kwargs.kwargs["env"]["PATH"], "probe-path") + self.assertIsNot(kwargs.kwargs["env"], os.environ) + + def test_real_execution_reports_unknown_build(self): + # The host Python always exists; `--version` exits 0 with an + # unrecognized format, proving the execution path runs entry points + # (including wrappers) without a shell. + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", pathlib.Path(sys.executable) + ) + self.assertTrue(probe["ok"]) + self.assertIsNone(probe["build_tag"]) + + def test_symlinked_entry_point_executes(self): + with tempfile.TemporaryDirectory() as tmp: + link = pathlib.Path(tmp) / "llama-server" + try: + link.symlink_to(pathlib.Path(sys.executable)) + except (OSError, NotImplementedError) as exc: + self.skipTest(f"symlinks unavailable: {exc}") + ctx = _make_ctx(tmp, platform="linux") + probe = llama_manager.probe_system_tool_executable( + ctx, "llama-server", link + ) + self.assertTrue(probe["ok"]) + self.assertEqual(probe["executable"], str(link)) + + +class SystemRuntimeHealthTests(unittest.TestCase): + def _win_ctx(self, tmp): + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + ctx.services.save_config({"backend": "system", "tag": "system"}) + return ctx + + def test_broken_tool_marks_not_ok_and_missing_is_explicit(self): + with tempfile.TemporaryDirectory() as tmp: + server = _write_executable(pathlib.Path(tmp) / "bin" / "llama-server.exe") + ctx = self._win_ctx(tmp) + calls = [] + + def fake_run(args, **kwargs): + calls.append(args) + if str(server) in args[0]: + return _completed(args, returncode=3) + raise AssertionError(f"unexpected probe: {args}") + + with mock.patch.dict(os.environ, {"PATH": str(server.parent)}), mock.patch.object( + llama_manager.subprocess, "run", side_effect=fake_run + ): + health = llama_manager.validate_runtime_dependencies(ctx) + self.assertFalse(health["ok"]) + self.assertIn("llama-server", health["unchecked_tools"]) + self.assertIn( + "llama-cli.exe", health["missing_executables"] + ) + probe = health["system_probes"]["llama-server"] + self.assertFalse(probe["ok"]) + self.assertEqual(probe["executable"], str(server)) + + def test_server_only_reports_build_tag_and_stays_ok(self): + with tempfile.TemporaryDirectory() as tmp: + server = _write_executable(pathlib.Path(tmp) / "bin" / "llama-server.exe") + ctx = self._win_ctx(tmp) + with mock.patch.dict(os.environ, {"PATH": str(server.parent)}), mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed([], stdout="version: 5678 (build 5678)"), + ): + health = llama_manager.validate_runtime_dependencies(ctx) + self.assertTrue(health["ok"]) + self.assertTrue(health["checked"]) + self.assertEqual(health["checked_tools"], ["llama-server"]) + self.assertEqual(health["build_tags"], {"llama-server": "b5678"}) + self.assertIn("llama-cli.exe", health["missing_executables"]) + + def test_probes_cached_and_invalidated_by_identity_or_clear(self): + with tempfile.TemporaryDirectory() as tmp: + server = _write_executable( + pathlib.Path(tmp) / "bin" / "llama-server.exe", "v1" + ) + ctx = self._win_ctx(tmp) + with mock.patch.dict(os.environ, {"PATH": str(server.parent)}), mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed([], stdout="build 100"), + ) as run: + first = llama_manager.validate_runtime_dependencies(ctx, ["llama-server"]) + second = llama_manager.validate_runtime_dependencies(ctx, ["llama-server"]) + self.assertEqual(first, second) + run.assert_called_once() + # A package update replacing the file changes its identity. + server.write_text("v1 with more content so size and mtime change") + with mock.patch.dict(os.environ, {"PATH": str(server.parent)}), mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed([], stdout="build 101"), + ) as run: + third = llama_manager.validate_runtime_dependencies(ctx, ["llama-server"]) + self.assertEqual(run.call_count, 1) + self.assertEqual(third["build_tags"], {"llama-server": "b101"}) + ctx.state.clear_runtime_health_cache() + with mock.patch.dict(os.environ, {"PATH": str(server.parent)}), mock.patch.object( + llama_manager.subprocess, + "run", + return_value=_completed([], stdout="build 101"), + ) as run: + llama_manager.validate_runtime_dependencies(ctx, ["llama-server"]) + self.assertEqual(run.call_count, 1) + + +class SystemLaunchIntegrationTests(unittest.TestCase): + def _ctx_with_server(self, tmp): + ctx = _make_ctx(tmp, platform="linux") + ctx.services.save_config({"backend": "system", "tag": "system"}) + server = _write_executable(pathlib.Path(tmp) / "bin" / "llama-server") + ctx.services.find_tool_executable = lambda tool: ( + server if tool == "llama-server" else None + ) + model = pathlib.Path(tmp) / "model.gguf" + model.write_text("model") + return ctx, server, model + + def test_preflight_uses_system_entry_point(self): + with tempfile.TemporaryDirectory() as tmp: + ctx, server, model = self._ctx_with_server(tmp) + health = _health_with_probes( + {"llama-server": _probe_ok(server, server, "b200")} + ) + with mock.patch.object( + ctx.services, + "validate_runtime_dependencies", + return_value=health, + ): + result = process_manager.preflight_launch( + ctx, "llama-server", ["-m", str(model)], {} + ) + self.assertTrue(result["ok"]) + self.assertEqual(result["executable"], "llama-server") + + def test_unknown_build_probe_still_passes_preflight(self): + with tempfile.TemporaryDirectory() as tmp: + ctx, server, model = self._ctx_with_server(tmp) + health = _health_with_probes( + {"llama-server": _probe_ok(server, server, None)} + ) + with mock.patch.object( + ctx.services, + "validate_runtime_dependencies", + return_value=health, + ): + result = process_manager.preflight_launch( + ctx, "llama-server", ["-m", str(model)], {} + ) + self.assertTrue(result["ok"]) + + def test_broken_server_fails_preflight_and_launch(self): + with tempfile.TemporaryDirectory() as tmp: + ctx, server, model = self._ctx_with_server(tmp) + health = _health_with_probes( + {}, + ok=False, + ) + health["system_probes"] = { + "llama-server": { + "ok": False, + "build_tag": None, + "error": "exited with status 1", + "executable": str(server), + } + } + with mock.patch.object( + ctx.services, + "validate_runtime_dependencies", + return_value=health, + ): + preflight = process_manager.preflight_launch( + ctx, "llama-server", ["-m", str(model)], {} + ) + self.assertIn("error", preflight) + self.assertIn("status 1", preflight["error"]) + launched = process_manager.launch_process( + ctx, "llama-server", ["-m", str(model)] + ) + self.assertIn("error", launched) + self.assertIn("status 1", launched["error"]) + + def test_missing_server_reports_path_guidance(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = _make_ctx(tmp, platform="linux") + ctx.services.save_config({"backend": "system", "tag": "system"}) + ctx.services.find_tool_executable = lambda tool: None + exe_path, error = process_manager._validate_launch_environment( + ctx, "llama-server" + ) + self.assertIsNone(exe_path) + self.assertIn("PATH", error) + self.assertNotIn("Repair Install", error) + + def test_launch_passes_entry_point_and_inherited_env(self): + with tempfile.TemporaryDirectory() as tmp: + ctx, server, model = self._ctx_with_server(tmp) + ctx.services.normalize_llama_api_target = ( + lambda host, port: {"host": host, "port": int(port)} + ) + health = _health_with_probes( + {"llama-server": _probe_ok(server, server, "b200")} + ) + fake_process = mock.Mock(pid=1234) + fake_process.poll.return_value = None + with mock.patch.dict( + os.environ, + {"PATH": "inherited-path", "LD_LIBRARY_PATH": "inherited-libs"}, + ), mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object( + process_manager.subprocess, "Popen", return_value=fake_process + ) as popen, mock.patch.object( + process_manager.threading, "Thread" + ): + result = process_manager.launch_process( + ctx, + "llama-server", + ["-m", str(model)], + env={"LLAMA_TEST_VAR": "override"}, + ) + self.assertEqual(result["pid"], 1234) + child_argv = popen.call_args.args[0] + self.assertEqual(child_argv[0], str(server)) + self.assertIn(str(model), child_argv) + child_env = popen.call_args.kwargs["env"] + self.assertEqual(child_env["PATH"], "inherited-path") + self.assertEqual(child_env["LD_LIBRARY_PATH"], "inherited-libs") + self.assertEqual(child_env["LLAMA_TEST_VAR"], "override") + self.assertNotIn(str(ctx.paths.llama_bin), child_env["PATH"]) + + def test_launch_reports_sanitized_error_when_binary_disappears(self): + with tempfile.TemporaryDirectory() as tmp: + ctx, server, model = self._ctx_with_server(tmp) + health = _health_with_probes( + {"llama-server": _probe_ok(server, server, "b200")} + ) + with mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object( + process_manager.subprocess, + "Popen", + side_effect=FileNotFoundError(str(server)), + ): + result = process_manager.launch_process( + ctx, "llama-server", ["-m", str(model)] + ) + self.assertIn("error", result) + self.assertIsNone(ctx.state.process) + + +class SystemBufferTypesTests(unittest.TestCase): + def _ctx(self, tmp, found): + ctx = _make_ctx(tmp, platform="linux") + ctx.services.save_config({"backend": "system", "tag": "system"}) + ctx.services.find_tool_executable = lambda tool: found.get(tool) + return ctx + + def _buffer_run(self, exe, test_case): + def fake_run(args, **kwargs): + test_case.assertEqual(args[0], str(exe)) + return _completed( + args, + stdout="Available buffer types:\nCUDA\n", + stderr="", + ) + + return fake_run + + def test_server_used_when_cli_absent(self): + with tempfile.TemporaryDirectory() as tmp: + server = _write_executable(pathlib.Path(tmp) / "llama-server") + ctx = self._ctx(tmp, {"llama-server": server}) + health = _health_with_probes({"llama-server": _probe_ok(server, server)}) + with mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object( + process_manager.subprocess, "run", side_effect=self._buffer_run(server, self) + ): + result = process_manager.get_buffer_types(ctx) + self.assertEqual(result["buffers"], ["CPU", "CUDA"]) + self.assertEqual(result["default"], "CUDA") + + def test_cli_preferred_when_present(self): + with tempfile.TemporaryDirectory() as tmp: + cli = _write_executable(pathlib.Path(tmp) / "llama-cli") + server = _write_executable(pathlib.Path(tmp) / "llama-server") + ctx = self._ctx(tmp, {"llama-cli": cli, "llama-server": server}) + health = _health_with_probes( + { + "llama-cli": _probe_ok(cli, cli), + "llama-server": _probe_ok(server, server), + } + ) + with mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object( + process_manager.subprocess, "run", side_effect=self._buffer_run(cli, self) + ): + result = process_manager.get_buffer_types(ctx) + self.assertIn("CUDA", result["buffers"]) + + def test_neither_available_returns_not_found_error(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = self._ctx(tmp, {}) + result = process_manager.get_buffer_types(ctx) + self.assertEqual(result["buffers"], ["CPU"]) + self.assertIn("not found", result["error"]) + + +class SystemEstimateTests(unittest.TestCase): + def _ctx(self, tmp): + ctx = _make_ctx(tmp, platform="win32", suffix=".exe") + ctx.services.save_config({"backend": "system", "tag": "system"}) + return ctx + + def test_missing_estimator_reports_unavailable(self): + with tempfile.TemporaryDirectory() as tmp: + ctx = self._ctx(tmp) + empty = pathlib.Path(tmp) / "empty" + empty.mkdir() + with mock.patch.dict(os.environ, {"PATH": str(empty)}), mock.patch.object( + process_manager.subprocess, "run" + ) as run: + result = process_manager.estimate_memory(ctx, "llama-server", []) + self.assertIn("llama-fit-params", result["error"]) + self.assertIn("unavailable", result["error"]) + run.assert_not_called() + + def test_broken_estimator_blocks_estimate_but_not_launch(self): + with tempfile.TemporaryDirectory() as tmp: + estimator = _write_executable( + pathlib.Path(tmp) / "bin" / "llama-fit-params.exe" + ) + server = _write_executable( + pathlib.Path(tmp) / "bin" / "llama-server.exe" + ) + ctx = self._ctx(tmp) + bad_probe = { + "ok": False, + "build_tag": None, + "error": "exited with status 1", + "executable": str(estimator), + } + health = _health_with_probes({"llama-fit-params": bad_probe}, ok=False) + model = pathlib.Path(tmp) / "model.gguf" + model.write_text("model") + with mock.patch.dict(os.environ, {"PATH": str(estimator.parent)}), mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object(process_manager.subprocess, "run") as run: + estimated = process_manager.estimate_memory( + ctx, "llama-server", ["-m", str(model)] + ) + self.assertIn("error", estimated) + self.assertIn("unavailable", estimated["error"]) + run.assert_not_called() + # Server launches do not require the estimator. + server_health = _health_with_probes( + {"llama-server": _probe_ok(server, server)} + ) + ctx.services.find_tool_executable = lambda tool: ( + server if tool == "llama-server" else None + ) + with mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=server_health + ): + exe_path, error = process_manager._validate_launch_environment( + ctx, "llama-server" + ) + self.assertIsNone(error) + self.assertEqual(exe_path, server) + + def test_working_estimator_runs_through_shared_command(self): + with tempfile.TemporaryDirectory() as tmp: + estimator = _write_executable( + pathlib.Path(tmp) / "bin" / "llama-fit-params.exe" + ) + ctx = self._ctx(tmp) + health = _health_with_probes( + {"llama-fit-params": _probe_ok(estimator, estimator)} + ) + with mock.patch.dict(os.environ, {"PATH": str(estimator.parent)}), mock.patch.object( + ctx.services, "validate_runtime_dependencies", return_value=health + ), mock.patch.object( + process_manager.subprocess, + "run", + return_value=_completed([], stdout="host 100 200 300\n"), + ) as run: + result = process_manager.estimate_memory( + ctx, "llama-server", ["-m", "model.gguf"] + ) + self.assertEqual(result["rows"][0]["total_mib"], 600) + self.assertEqual(run.call_args.args[0][0], str(estimator)) + self.assertIn("-fitp", run.call_args.args[0]) + + +if __name__ == "__main__": + unittest.main() From cf87a617a62e3159b51d0a26959481ca8e1d9a3b Mon Sep 17 00:00:00 2001 From: Thomas Date: Thu, 8 Oct 2026 21:30:51 -0400 Subject: [PATCH 4/7] Activate System backend from PATH - Probe llama-server before switching and rediscover System tools on startup - Report tool availability and PATH guidance in status - Keep System tools out of install and update flows --- backend/app.py | 1 + backend/routes/install.py | 44 ++- backend/routes/lifecycle.py | 21 +- backend/routes/status.py | 56 +++- backend/services/llama_manager.py | 115 +++++++ backend/services/process_manager.py | 5 +- docs/directory.md | 4 +- tests/backend/test_system_backend.py | 434 +++++++++++++++++++++++++++ 8 files changed, 669 insertions(+), 11 deletions(-) create mode 100644 tests/backend/test_system_backend.py diff --git a/backend/app.py b/backend/app.py index e9fd356..971cf83 100644 --- a/backend/app.py +++ b/backend/app.py @@ -979,6 +979,7 @@ def do_DELETE(self): .add("POST", "/api/install", install_routes.start_install) .add("POST", "/api/update", install_routes.start_update) .add("POST", "/api/activate-custom", install_routes.activate_custom) + .add("POST", "/api/activate-system", install_routes.activate_system) .add("POST", "/api/launch/preflight", process_routes.preflight_launch) .add("POST", "/api/presets/fingerprint", process_routes.fingerprint_preset) .add("POST", "/api/launch", process_routes.launch) diff --git a/backend/routes/install.py b/backend/routes/install.py index bf47e5e..13538ad 100644 --- a/backend/routes/install.py +++ b/backend/routes/install.py @@ -37,7 +37,7 @@ def get_releases(request, response, ctx): spec = None query = urllib.parse.parse_qs(request.query or "") backend = (query.get("backend") or [""])[0].strip() - if llama_manager.is_custom_backend(backend): + if llama_manager.is_custom_backend(backend) or llama_manager.is_system_backend(backend): response.json([]) return if backend: @@ -89,6 +89,15 @@ def start_install(request, response, ctx): if llama_manager.is_custom_backend(backend): response.error("Use /api/activate-custom to set up the custom backend", 400) return + if llama_manager.is_system_backend(backend): + if activate_existing: + response.error("Use /api/activate-system to activate System (PATH) tools", 400) + else: + response.error( + "System (PATH) tools are installed by the OS package manager; " + "use /api/activate-system to use them", 400 + ) + return if backend not in backend_specs: response.error(f"Unsupported backend: {backend}", 400) return @@ -142,6 +151,11 @@ def start_update(request, response, ctx): if llama_manager.is_custom_backend(backend): response.error("Cannot auto-update a custom backend installation", 400) return + if llama_manager.is_system_backend(backend): + response.error( + "System (PATH) tools are updated by the OS package manager", 400 + ) + return if backend not in backend_specs: response.error(f"Unsupported configured backend: {backend}", 400) return @@ -216,6 +230,34 @@ def _update(latest_tag, backend_name): response.json({"status": "started", "from": tag, "to": latest}) +def activate_system(request, response, ctx): + body = request.body or {} + if not isinstance(body, dict): + response.error("Invalid request body", 400) + return + for key in ("path", "executable", "executable_path", "bin_dir", "directory"): + if key in body: + response.error( + "System activation uses the PATH inherited by Llama GUI; " + "client-supplied paths are not accepted", + 400, + ) + return + try: + claim_error = _claim_install_slot(ctx) + if claim_error is not None: + response.error(*claim_error) + return + try: + result = llama_manager.activate_system_backend(ctx) + response.json(result) + finally: + process_manager.release_install_slot(ctx) + except Exception as e: + print(f"[install] activate system backend failed: {e}", file=sys.stderr) + response.error(sanitize_error(e, 500), 500) + + def activate_custom(request, response, ctx): backend = (request.body or {}).get("backend", "custom") if not llama_manager.is_custom_backend(backend): diff --git a/backend/routes/lifecycle.py b/backend/routes/lifecycle.py index 173ea73..811758e 100644 --- a/backend/routes/lifecycle.py +++ b/backend/routes/lifecycle.py @@ -27,9 +27,24 @@ def post_open_folder(request, response, ctx): try: if folder == "llama": backend = ctx.services.load_config().get("backend") - target = (llama_manager.get_backend_bin_dir(ctx, backend).parent - if llama_manager.is_custom_backend(backend) else ctx.paths.llama) - target.mkdir(parents=True, exist_ok=True) + if llama_manager.is_system_backend(backend): + # Read-only policy: reveal the discovered server's directory + # without creating anything. A missing server reports PATH + # guidance instead of falling back to an application folder. + try: + server_exe = ctx.services.find_tool_executable("llama-server") + except Exception: + server_exe = None + if server_exe is None or not server_exe.is_file(): + response.error(llama_manager.SYSTEM_PATH_GUIDANCE, 409) + return + target = server_exe.parent + elif llama_manager.is_custom_backend(backend): + target = llama_manager.get_backend_bin_dir(ctx, backend).parent + target.mkdir(parents=True, exist_ok=True) + else: + target = ctx.paths.llama + target.mkdir(parents=True, exist_ok=True) else: target = model_dir.get_models_dir(ctx) lifecycle_service.open_folder_in_file_manager(target) diff --git a/backend/routes/status.py b/backend/routes/status.py index 24766ed..d8f7f80 100644 --- a/backend/routes/status.py +++ b/backend/routes/status.py @@ -27,10 +27,18 @@ def get_status(request, response, ctx): runtime_files = services.get_runtime_files() runtime_health = dict(services.validate_runtime_dependencies()) has_config = bool(cfg.get("tag")) - core_tools_present = all( - exes.get(services.get_tool_filename(tool), False) - for tool in ("llama-cli", "llama-server") - ) + is_system = llama_manager.is_system_backend(cfg.get("backend")) + if is_system: + # System readiness needs the required server entry point, not the + # optional CLI. A missing estimator or benchmark tool never blocks it. + core_tools_present = bool( + exes.get(services.get_tool_filename("llama-server"), False) + ) + else: + core_tools_present = all( + exes.get(services.get_tool_filename(tool), False) + for tool in ("llama-cli", "llama-server") + ) installed = has_config and core_tools_present and runtime_health.get("ok", True) config_stale = has_config and not installed process_status = process_manager.get_process_status_snapshot(ctx) @@ -43,6 +51,42 @@ def get_status(request, response, ctx): print(f"[status] failed to read llama API target: {exc}", file=sys.stderr) api_target = {"host": LLAMA_HOST, "port": LLAMA_PORT} + # PATH discovery for display: per-tool entry points, availability, and + # probe state. Paths resolve regardless of the active backend so the + # GUI can preview System before activation; build/probe details come + # from runtime health and are only present while System is active. + system_probes = runtime_health.get("system_probes") or {} + system_build_tags = runtime_health.get("build_tags") or {} + system_tools = {} + for tool in services.llama_tools: + try: + discovered = services.find_tool_executable(tool) + except Exception: + discovered = None + probe = system_probes.get(tool) or {} + system_tools[tool] = { + "path": str(discovered) if discovered is not None else None, + "available": bool( + exes.get(services.get_tool_filename(tool), False) + ), + "build_tag": system_build_tags.get(tool), + "probe_ok": probe.get("ok"), + "probe_error": probe.get("error"), + } + + system_server_error = None + if is_system: + server_probe = system_probes.get("llama-server") or {} + if not system_tools.get("llama-server", {}).get("available"): + system_server_error = llama_manager.SYSTEM_PATH_GUIDANCE + elif server_probe.get("ok") is False: + detail = server_probe.get("error") or "could not be started" + server_name = services.get_tool_filename("llama-server") + system_server_error = ( + f"{server_name} {detail}. " + f"Check that it runs with `{server_name} --version`." + ) + response.json( { "installed": installed, @@ -55,6 +99,8 @@ def get_status(request, response, ctx): "runtime_files_label": "Runtime libraries", "runtime_health": runtime_health, "missing_runtime_files": runtime_health.get("missing_runtime_files", []), + "system_tools": system_tools, + "system_server_error": system_server_error, **model_dir_info, "running": process_status["running"], "active_process_tool": process_status["active_process_tool"], @@ -73,6 +119,8 @@ def get_status(request, response, ctx): "id": key, "label": spec["label"], **({"custom": True, "bin_dir": llama_manager.custom_backend_bin_label(key)} if llama_manager.is_custom_backend(key) else {}), + **({"system_path": True} + if llama_manager.is_system_backend(key) else {}), } for key, spec in backend_specs.items() ], diff --git a/backend/services/llama_manager.py b/backend/services/llama_manager.py index 97aba0d..372c34d 100644 --- a/backend/services/llama_manager.py +++ b/backend/services/llama_manager.py @@ -56,6 +56,14 @@ SYSTEM_BACKEND_ID = "system" SYSTEM_BACKEND_LABEL = "System (PATH)" +# Actionable guidance when the required System server entry point cannot be +# found on the inherited PATH. Never dump the environment or offer shell edits. +SYSTEM_PATH_GUIDANCE = ( + "llama-server was not found on the PATH inherited by Llama GUI. " + "Start the GUI from the environment where it is available, or restart " + "the GUI after changing its launch environment." +) + # Internal PATH discovery target that is never a public launch tool. Memory # estimation locates it separately; Stage 1 only resolves it. SYSTEM_INTERNAL_TOOLS = ("llama-fit-params",) @@ -926,12 +934,119 @@ def activate_custom_backend(ctx: AppContext, backend: str = "custom") -> dict[st return {"ok": False, "error": sanitize_error(e, 500)} +def activate_system_backend(ctx: AppContext) -> dict[str, Any]: + """Activate System (PATH) tools after a successful server execution probe. + + Discovery uses the inherited PATH only; no client-supplied path is + accepted (the route rejects those before calling here). Only + ``llama-server`` is required; every other known tool is reported as an + independently available feature. Configuration is unchanged unless the + server probe succeeds. Discovered paths and build tags are returned for + display but never persisted: restart rediscovers them. + """ + try: + try: + tools = list(ctx.services.llama_tools or []) + except Exception: + tools = [] + try: + server_filename = ctx.services.get_tool_filename("llama-server") + except Exception: + server_filename = "llama-server" + + paths: dict[str, str] = {} + found: list[str] = [] + missing: list[str] = [] + for tool in tools: + try: + exe_path = resolve_system_tool_executable(ctx, tool) + except Exception: + exe_path = None + try: + exe_name = ctx.services.get_tool_filename(tool) + except Exception: + exe_name = str(tool) + if exe_path is None: + missing.append(exe_name) + else: + found.append(exe_name) + paths[tool] = str(exe_path) + + try: + server_exe = resolve_system_tool_executable(ctx, "llama-server") + except Exception: + server_exe = None + if server_exe is None: + return { + "ok": False, + "found": sorted(found), + "missing": sorted(missing), + "missing_required": [server_filename], + "paths": paths, + "build_tags": {}, + "error": SYSTEM_PATH_GUIDANCE, + } + + probe = probe_system_tool_executable(ctx, "llama-server", server_exe) + if not probe.get("ok"): + detail = probe.get("error") or "could not be started" + return { + "ok": False, + "found": sorted(found), + "missing": sorted(missing), + "missing_required": [], + "paths": paths, + "build_tags": {}, + "probe_error": detail, + "error": ( + f"{server_filename} {detail}. " + f"Check that it runs with `{server_filename} --version`." + ), + } + + build_tags: dict[str, str] = {} + if probe.get("build_tag"): + build_tags["llama-server"] = probe["build_tag"] + + with ctx.state.config_lock: + cfg = dict(ctx.services.load_config()) + if cfg.get("backend") and is_official_backend(cfg.get("backend")) and cfg.get("tag"): + cfg["official_install"] = { + "backend": cfg["backend"], + "tag": cfg["tag"], + "version": cfg.get("version") or cfg["tag"], + } + cfg["version"] = SYSTEM_BACKEND_ID + cfg["backend"] = SYSTEM_BACKEND_ID + cfg["tag"] = SYSTEM_BACKEND_ID + ctx.services.save_config(cfg) + ctx.state.clear_runtime_health_cache() + return { + "ok": True, + "backend": SYSTEM_BACKEND_ID, + "tag": SYSTEM_BACKEND_ID, + "version": SYSTEM_BACKEND_ID, + "found": sorted(found), + "missing": sorted(missing), + "missing_required": [], + "paths": paths, + "build_tags": build_tags, + "server_path": str(server_exe), + } + except Exception as e: + print(f"[llama_manager] activate_system_backend failed: {e}", file=sys.stderr) + return {"ok": False, "error": sanitize_error(e, 500)} + + def get_official_install_status( ctx: AppContext, cfg: Optional[Mapping[str, Any]] = None ) -> dict[str, Any]: cfg = cfg or ctx.services.load_config() stored = cfg.get("official_install") stored = stored if isinstance(stored, Mapping) else {} + if is_system_backend(stored.get("backend")): + # System is never an official installation; ignore a stale record. + stored = {} backend = stored.get("backend") tag = stored.get("tag") diff --git a/backend/services/process_manager.py b/backend/services/process_manager.py index f2f0934..41ca394 100644 --- a/backend/services/process_manager.py +++ b/backend/services/process_manager.py @@ -1715,7 +1715,10 @@ def remove_llama_files(ctx: AppContext) -> int: with ctx.state.config_lock: config_data = _load_config_safe(ctx) - if not llama_manager.is_custom_backend(config_data.get("backend")): + if not ( + llama_manager.is_custom_backend(config_data.get("backend")) + or llama_manager.is_system_backend(config_data.get("backend")) + ): config_data.update({"version": None, "backend": None, "tag": None}) config_data.pop("official_install", None) ctx.services.save_config(config_data) diff --git a/docs/directory.md b/docs/directory.md index 5b7a4a4..dda2147 100644 --- a/docs/directory.md +++ b/docs/directory.md @@ -100,7 +100,7 @@ ### Route Modules (`backend/routes/`) -`API_ROUTER` at the bottom of `backend/app.py` is the authoritative registry: 49 exact routes plus one prefix route, 50 endpoints total. Keep this table in sync with it — a route that is registered but undocumented here is the drift that is hardest to notice. +`API_ROUTER` at the bottom of `backend/app.py` is the authoritative registry: 50 exact routes plus one prefix route, 51 endpoints total. Keep this table in sync with it — a route that is registered but undocumented here is the drift that is hardest to notice. | Route | Endpoints | |-------|-----------| @@ -108,7 +108,7 @@ | `external_server.py` | `GET /api/chat/target` (read the live and remembered target), `POST /api/chat/target` (register an externally started llama-server as the proxy target; `POST {"restore": true}` re-registers the address saved by an earlier session), `DELETE /api/chat/target` (clear it) | | `benchmarks.py` | `POST /api/benchmark/wikitext2` — ensure WikiText-2 raw test file exists | | `process.py` | `POST /api/launch`, `POST /api/launch/preflight`, `POST /api/presets/fingerprint`, `POST /api/estimate-memory`, generation-bound `POST /api/stop`, `POST /api/send-input`, `POST /api/cleanup-llama`, `GET /api/output`, `GET /api/llama/health`, `GET /api/llama/buffer-types` | -| `install.py` | `GET /api/backends` — refresh the cached official CUDA/ROCm catalog (`?refresh=1` requests a manual refresh; returns a warning on fallback), `GET /api/releases`, `GET /api/download-progress`, `POST /api/install`, `POST /api/update`, `POST /api/activate-custom` (optional `backend`: `custom` or `custom-02`; omitted defaults to `custom`) | +| `install.py` | `GET /api/backends` — refresh the cached official CUDA/ROCm catalog (`?refresh=1` requests a manual refresh; returns a warning on fallback), `GET /api/releases`, `GET /api/download-progress`, `POST /api/install`, `POST /api/update`, `POST /api/activate-custom` (optional `backend`: `custom` or `custom-02`; omitted defaults to `custom`), `POST /api/activate-system` (System PATH activation; accepts no client-supplied path) | | `metrics.py` | `GET /api/llama/metrics`, `GET /api/llama/slots`, `GET /api/llama/props` — Prometheus proxy and template-capability props | | `models.py` | `GET /api/models` — list GGUF files recursively as names relative to the active model root | | `model_dir.py` | `POST /api/models-dir` — set or reset the active model root | diff --git a/tests/backend/test_system_backend.py b/tests/backend/test_system_backend.py new file mode 100644 index 0000000..bd1864a --- /dev/null +++ b/tests/backend/test_system_backend.py @@ -0,0 +1,434 @@ +"""Stage 3: System (PATH) activation, persistence, status, and switching. + +Uses temporary fixture executables, mocked ``subprocess.run`` responses, and +an isolated PATH. No host llama.cpp installation, model, or GPU is required. +""" + +import json +import os +import pathlib +import subprocess +import tempfile +import unittest +from unittest import mock + +from backend.http import Request +from backend.routes import install, lifecycle, status +from backend.services import llama_manager, official_backends, process_manager +from tests.backend.test_extracted_routes import DummyResponse +from tests.backend.test_services import make_service_context + + +def _completed(args, returncode=0, stdout="", stderr=""): + return subprocess.CompletedProcess(args, returncode, stdout, stderr) + + +def _version_run(stdout="version: b1234 (build 1234)", returncode=0): + def fake_run(args, **kwargs): + return _completed(args, returncode, stdout, "") + return fake_run + + +class SystemBackendTests(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.addCleanup(self.tmp.cleanup) + self.ctx = make_service_context(self.tmp.name) + services = self.ctx.services + services.current_platform = "win32" + services.current_arch = "x64" + services.binary_suffix = ".exe" + services.llama_tools = ["llama-cli", "llama-server", "llama-bench"] + services.get_tool_filename = lambda tool: tool + services.binary_suffix + services.backend_specs = llama_manager.build_backend_specs("win32", "x64") + services.load_config = lambda: json.loads(self.ctx.paths.config_file.read_text()) + services.save_config = lambda cfg: self.ctx.paths.config_file.write_text(json.dumps(cfg)) + services.find_tool_executable = lambda tool: ( + llama_manager.resolve_backend_tool_executable( + self.ctx, services.load_config().get("backend"), tool + ) + ) + services.get_runtime_files = lambda: [] + services.get_platform_label = lambda: "Windows" + services.get_llama_api_target = lambda: {"host": "127.0.0.1", "port": 8080} + services.validate_runtime_dependencies = lambda tools=None: ( + llama_manager.validate_runtime_dependencies(self.ctx, tools) + ) + services.save_config({ + "backend": "cpu", "tag": "b123", "version": "b123", + "models_dir": "saved-model-root", + }) + # Official build on disk so the return path never needs a download. + official = self.ctx.paths.llama_bin + official.mkdir(parents=True, exist_ok=True) + for tool in ("llama-cli", "llama-server"): + (official / services.get_tool_filename(tool)).write_text("cpu") + # PATH fixture dir for System tools. + self.fixture_bin = os.path.join(self.tmp.name, "system-bin") + os.makedirs(self.fixture_bin, exist_ok=True) + + def write_system_tool(self, tool): + path = os.path.join( + self.fixture_bin, self.ctx.services.get_tool_filename(tool) + ) + with open(path, "w") as handle: + handle.write(tool) + return path + + def path_env(self): + return mock.patch.dict(os.environ, {"PATH": self.fixture_bin}) + + def activate_system(self, body=None): + response = DummyResponse() + install.activate_system( + Request("POST", "/api/activate-system", "", {}, body=body or {}), + response, + self.ctx, + ) + return response + + def get_status(self): + response = DummyResponse() + status.get_status(Request("GET", "/api/status", "", {}), response, self.ctx) + return response.payload + + # Activation success / failure. + + def test_successful_activation_persists_marker_and_preserves_official(self): + server = self.write_system_tool("llama-server") + cli = self.write_system_tool("llama-cli") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + response = self.activate_system() + self.assertEqual(response.status, 200) + payload = response.payload + self.assertTrue(payload["ok"]) + self.assertEqual(payload["server_path"], server) + self.assertEqual(payload["paths"]["llama-server"], server) + self.assertEqual(payload["paths"]["llama-cli"], cli) + self.assertEqual(payload["build_tags"], {"llama-server": "b1234"}) + self.assertEqual(payload["missing_required"], []) + cfg = self.ctx.services.load_config() + self.assertEqual(cfg["backend"], "system") + self.assertEqual(cfg["tag"], "system") + self.assertEqual(cfg["version"], "system") + self.assertEqual(cfg["models_dir"], "saved-model-root") + self.assertEqual( + cfg["official_install"], + {"backend": "cpu", "tag": "b123", "version": "b123"}, + ) + # Discovered paths are display-only, never persisted. + self.assertNotIn("server_path", cfg) + self.assertNotIn("paths", cfg) + + def test_unknown_build_format_still_activates(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", + side_effect=_version_run("llama-pack 2.0-custom"), + ): + response = self.activate_system() + self.assertTrue(response.payload["ok"]) + self.assertEqual(response.payload["build_tags"], {}) + self.assertEqual(self.ctx.services.load_config()["backend"], "system") + + def test_missing_server_fails_with_path_guidance_and_keeps_config(self): + previous = self.ctx.paths.config_file.read_bytes() + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ) as probe: + response = self.activate_system() + payload = response.payload + self.assertFalse(payload["ok"]) + self.assertEqual(payload["missing_required"], ["llama-server.exe"]) + self.assertIn("PATH inherited by Llama GUI", payload["error"]) + probe.assert_not_called() + self.assertEqual(self.ctx.paths.config_file.read_bytes(), previous) + + def test_broken_server_fails_with_check_guidance_and_keeps_config(self): + self.write_system_tool("llama-server") + previous = self.ctx.paths.config_file.read_bytes() + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", + side_effect=_version_run("", returncode=1), + ): + response = self.activate_system() + payload = response.payload + self.assertFalse(payload["ok"]) + self.assertIn("status 1", payload["error"]) + self.assertIn("--version", payload["error"]) + self.assertEqual(self.ctx.paths.config_file.read_bytes(), previous) + + def test_activation_rejects_client_supplied_paths(self): + self.write_system_tool("llama-server") + for body in ({"path": "C:\\evil"}, {"executable": "x"}, {"bin_dir": "y"}): + with self.subTest(body=body): + previous = self.ctx.paths.config_file.read_bytes() + with self.path_env(): + response = self.activate_system(body) + self.assertEqual(response.status, 400) + self.assertEqual(self.ctx.paths.config_file.read_bytes(), previous) + + def test_activation_rejected_while_running_or_installing(self): + self.write_system_tool("llama-server") + self.ctx.state.install_in_progress = True + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ) as probe: + response = self.activate_system() + self.assertEqual(response.status, 409) + probe.assert_not_called() + self.ctx.state.install_in_progress = False + with self.path_env(), mock.patch.object( + process_manager, "is_process_running", return_value=True + ), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ) as probe: + response = self.activate_system() + self.assertEqual(response.status, 400) + self.assertIn("Stop running process", response.payload["error"]) + probe.assert_not_called() + self.assertEqual(self.ctx.services.load_config()["backend"], "cpu") + + # Restart restoration. + + def test_selection_survives_restart_with_new_environment(self): + server = self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + # A fresh process inherits the same PATH and rediscovers the entry. + fresh = make_service_context(self.tmp.name) + fresh.services.load_config = lambda: json.loads( + self.ctx.paths.config_file.read_text() + ) + fresh.services.llama_tools = list(self.ctx.services.llama_tools) + fresh.services.get_tool_filename = self.ctx.services.get_tool_filename + with self.path_env(): + found = llama_manager.resolve_backend_tool_executable( + fresh, "system", "llama-server" + ) + self.assertEqual(str(found), server) + self.assertNotIn("paths", fresh.services.load_config()) + + # Status. + + def test_server_only_status_is_installed_with_tool_details(self): + server = self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + payload = self.get_status() + self.assertTrue(payload["installed"]) + self.assertFalse(payload["config_stale"]) + self.assertEqual(payload["backend"], "system") + # Selected marker stays separate from discovered version info. + self.assertEqual(payload["version"], "system") + self.assertEqual(payload["runtime_health"]["build_tags"], {"llama-server": "b1234"}) + server_entry = payload["system_tools"]["llama-server"] + self.assertEqual(server_entry["path"], server) + self.assertTrue(server_entry["available"]) + self.assertEqual(server_entry["build_tag"], "b1234") + self.assertTrue(server_entry["probe_ok"]) + cli_entry = payload["system_tools"]["llama-cli"] + self.assertIsNone(cli_entry["path"]) + self.assertFalse(cli_entry["available"]) + self.assertIsNone(payload["system_server_error"]) + system_backends = [ + item for item in payload["available_backends"] if item["id"] == "system" + ] + self.assertEqual( + system_backends, + [{"id": "system", "label": "System (PATH)", "system_path": True}], + ) + + def test_removed_server_reports_stale_with_guidance_and_keeps_selection(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + empty = os.path.join(self.tmp.name, "empty-bin") + os.makedirs(empty, exist_ok=True) + with mock.patch.dict(os.environ, {"PATH": empty}): + payload = self.get_status() + self.assertFalse(payload["installed"]) + self.assertTrue(payload["config_stale"]) + self.assertEqual(payload["backend"], "system") + self.assertIn("PATH inherited by Llama GUI", payload["system_server_error"]) + self.assertIsNone(payload["system_tools"]["llama-server"]["path"]) + + def test_broken_server_status_reports_check_guidance(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", + side_effect=_version_run("", returncode=3), + ): + payload = self.get_status() + self.assertFalse(payload["installed"]) + self.assertIn("status 3", payload["system_server_error"]) + self.assertIn("--version", payload["system_server_error"]) + + # Install/update/release misuse. + + def test_system_excluded_from_download_and_update_routes(self): + with mock.patch.object( + llama_manager, "get_releases" + ) as lookup, mock.patch.object(llama_manager, "install_release") as download: + response = DummyResponse() + install.get_releases( + Request("GET", "/api/releases", "backend=system", {}), response, self.ctx + ) + self.assertEqual(response.payload, []) + response = DummyResponse() + install.start_install( + Request( + "POST", "/api/install", "", {}, + body={"backend": "system", "tag": "b999"}, + ), + response, + self.ctx, + ) + self.assertEqual(response.status, 400) + response = DummyResponse() + install.start_install( + Request( + "POST", "/api/install", "", {}, + body={"backend": "system", "activate_existing": True}, + ), + response, + self.ctx, + ) + self.assertEqual(response.status, 400) + self.assertIn("activate-system", response.payload["error"]) + lookup.assert_not_called() + download.assert_not_called() + self.assertFalse(self.ctx.state.install_in_progress) + self.ctx.services.save_config({"backend": "system", "tag": "system"}) + response = DummyResponse() + with mock.patch.object( + llama_manager, "get_releases" + ) as lookup, mock.patch.object(process_manager, "is_process_running", + return_value=False): + install.start_update(Request("POST", "/api/update", "", {}, body={}), + response, self.ctx) + self.assertEqual(response.status, 400) + lookup.assert_not_called() + self.assertFalse(self.ctx.state.install_in_progress) + + def test_official_catalog_refresh_never_targets_system(self): + self.ctx.services.save_config({"backend": "system", "tag": "system"}) + with mock.patch.object( + llama_manager, "get_releases", return_value=[] + ), mock.patch.object( + llama_manager, "get_release_by_tag" + ) as by_tag: + official_backends.refresh(self.ctx, force=True) + by_tag.assert_not_called() + + def test_stored_system_is_never_reported_as_official(self): + cfg = {"backend": "system", "tag": "system", + "official_install": {"backend": "system", "tag": "system"}} + reported = llama_manager.get_official_install_status(self.ctx, cfg) + self.assertIsNone(reported["backend"]) + self.assertIsNone(reported["tag"]) + + # Round trips and maintenance. + + def test_official_system_custom_round_trip_needs_no_download(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + self.assertEqual( + self.ctx.services.load_config()["official_install"]["backend"], "cpu" + ) + custom_bin = llama_manager.get_backend_bin_dir(self.ctx, "custom") + custom_bin.mkdir(parents=True, exist_ok=True) + for tool in ("llama-cli", "llama-server"): + (custom_bin / self.ctx.services.get_tool_filename(tool)).write_text("custom") + custom_response = DummyResponse() + install.activate_custom( + Request("POST", "/api/activate-custom", "", {}, body={"backend": "custom"}), + custom_response, + self.ctx, + ) + # The stock custom fixture lacks llama-bench, but activation succeeds + # with optional tools missing; the official record must survive. + self.assertTrue(custom_response.payload["ok"]) + cfg = self.ctx.services.load_config() + self.assertEqual(cfg["backend"], "custom") + self.assertEqual(cfg["official_install"]["backend"], "cpu") + with mock.patch.object( + llama_manager, "_probe_official_build", return_value=(True, "b123") + ): + result = llama_manager.activate_official_backend(self.ctx, "cpu") + self.assertTrue(result["ok"]) + self.assertEqual(self.ctx.services.load_config()["tag"], "b123") + self.assertEqual(self.ctx.services.load_config()["models_dir"], "saved-model-root") + + def test_cleanup_preserves_system_selection_but_clears_official_record(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + removed = process_manager.remove_llama_files(self.ctx) + self.assertGreater(removed, 0) + cfg = self.ctx.services.load_config() + self.assertEqual(cfg["backend"], "system") + self.assertEqual(cfg["tag"], "system") + self.assertNotIn("official_install", cfg) + self.assertTrue(self.ctx.paths.llama_bin.exists()) + + def test_open_folder_reveals_server_dir_without_creating(self): + server = self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + with self.path_env(), mock.patch.object( + lifecycle.lifecycle_service, "open_folder_in_file_manager" + ) as opener: + response = DummyResponse() + lifecycle.post_open_folder( + Request("POST", "/api/open-folder", "", {}, body={"folder": "llama"}), + response, + self.ctx, + ) + self.assertEqual(response.payload, {"opened": True}) + opener.assert_called_once_with(pathlib.Path(self.fixture_bin)) + + def test_open_folder_reports_unavailable_without_creating(self): + self.write_system_tool("llama-server") + with self.path_env(), mock.patch.object( + llama_manager.subprocess, "run", side_effect=_version_run() + ): + self.assertTrue(self.activate_system().payload["ok"]) + empty = os.path.join(self.tmp.name, "empty-bin") + os.makedirs(empty, exist_ok=True) + with mock.patch.dict(os.environ, {"PATH": empty}), mock.patch.object( + lifecycle.lifecycle_service, "open_folder_in_file_manager" + ) as opener: + response = DummyResponse() + lifecycle.post_open_folder( + Request("POST", "/api/open-folder", "", {}, body={"folder": "llama"}), + response, + self.ctx, + ) + self.assertEqual(response.status, 409) + self.assertIn("PATH inherited by Llama GUI", response.payload["error"]) + opener.assert_not_called() + self.assertFalse(os.path.exists(empty + "-created")) + + +if __name__ == "__main__": + unittest.main() From 2ad35bb5c75cc250a8a68e1fa4e993e8eb2e1dad Mon Sep 17 00:00:00 2001 From: Thomas Date: Fri, 9 Oct 2026 15:16:25 -0400 Subject: [PATCH 5/7] Support System PATH tools across frontend workflows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add frontend coverage for System backend activation and per-tool availability - Gate launch and benchmark behavior using each tool’s detected build tag --- package.json | 2 +- tests/frontend/module_namespace_unit.cjs | 2 + tests/frontend/system_path_frontend_unit.cjs | 549 +++++++++++++++++++ ui/index.html | 13 + ui/js/benchmark-ui.js | 44 +- ui/js/flag-core.js | 59 +- ui/js/manager/manager-backends.js | 297 +++++++++- ui/js/manager/manager-install.js | 52 +- ui/js/manager/manager-main.js | 1 + ui/js/manager/manager-status.js | 9 +- ui/js/quick-launch-ui.js | 47 +- 11 files changed, 1030 insertions(+), 45 deletions(-) create mode 100644 tests/frontend/system_path_frontend_unit.cjs diff --git a/package.json b/package.json index 7dc3aad..832f2e0 100644 --- a/package.json +++ b/package.json @@ -3,7 +3,7 @@ "private": true, "scripts": { "test": "npm run test:unit && npm run test:flags && npm run test:frontend", - "test:unit": "node tests/frontend/custom_launch_args_unit.cjs && node tests/frontend/launch_args_unit.cjs && node tests/frontend/chat_template_selection_unit.cjs && node tests/frontend/output_cursor_unit.cjs && node tests/frontend/inference_stats_unit.cjs && node --test tests/frontend/memory_estimate_ui_unit.cjs tests/frontend/inference_polling_unit.cjs tests/frontend/process_output_unit.cjs && node --test tests/frontend/monitor_package_unit.cjs tests/frontend/monitor_system_cards_unit.cjs tests/frontend/monitor_gpu_cards_unit.cjs tests/frontend/monitor_inference_rendering_unit.cjs tests/frontend/monitor_terminal_runtime_unit.cjs tests/frontend/monitor_polling_unit.cjs tests/frontend/monitor_card_preferences_unit.cjs && node tests/frontend/process_lifecycle_unit.cjs && node tests/frontend/benchmark_args_unit.cjs && node tests/frontend/chat_rendering_unit.cjs && node tests/frontend/chat_ui_unit.cjs && node --test tests/frontend/chat_window_unit.cjs && node tests/frontend/chat_tools_unit.cjs && node tests/frontend/chat_compaction_unit.cjs && node tests/frontend/character_cards_unit.cjs && node tests/frontend/sampler_presets_unit.cjs && node tests/frontend/presets_unit.cjs && node tests/frontend/preset_roving_focus_unit.cjs && node tests/frontend/model_switch_ui_unit.cjs && node tests/frontend/hf_download_ui_unit.cjs && node tests/frontend/remote_tunnel_ui_unit.cjs && node tests/frontend/api_tab_unit.cjs && node tests/frontend/external_server_ui_unit.cjs && node tests/frontend/shared_services_unit.cjs && node tests/frontend/manager_releases_unit.cjs && node tests/frontend/manager_lifecycle_unit.cjs && node tests/frontend/manager_model_cache_unit.cjs && node tests/frontend/manager_model_dir_unit.cjs && node tests/frontend/theme_ui_unit.cjs && npm run test:frontend:modules && npm run test:flag-definitions", + "test:unit": "node tests/frontend/custom_launch_args_unit.cjs && node tests/frontend/launch_args_unit.cjs && node tests/frontend/chat_template_selection_unit.cjs && node tests/frontend/output_cursor_unit.cjs && node tests/frontend/inference_stats_unit.cjs && node --test tests/frontend/memory_estimate_ui_unit.cjs tests/frontend/inference_polling_unit.cjs tests/frontend/process_output_unit.cjs && node --test tests/frontend/monitor_package_unit.cjs tests/frontend/monitor_system_cards_unit.cjs tests/frontend/monitor_gpu_cards_unit.cjs tests/frontend/monitor_inference_rendering_unit.cjs tests/frontend/monitor_terminal_runtime_unit.cjs tests/frontend/monitor_polling_unit.cjs tests/frontend/monitor_card_preferences_unit.cjs && node tests/frontend/process_lifecycle_unit.cjs && node tests/frontend/benchmark_args_unit.cjs && node tests/frontend/chat_rendering_unit.cjs && node tests/frontend/chat_ui_unit.cjs && node --test tests/frontend/chat_window_unit.cjs && node tests/frontend/chat_tools_unit.cjs && node tests/frontend/chat_compaction_unit.cjs && node tests/frontend/character_cards_unit.cjs && node tests/frontend/sampler_presets_unit.cjs && node tests/frontend/presets_unit.cjs && node tests/frontend/preset_roving_focus_unit.cjs && node tests/frontend/model_switch_ui_unit.cjs && node tests/frontend/hf_download_ui_unit.cjs && node tests/frontend/remote_tunnel_ui_unit.cjs && node tests/frontend/api_tab_unit.cjs && node tests/frontend/external_server_ui_unit.cjs && node tests/frontend/shared_services_unit.cjs && node tests/frontend/manager_releases_unit.cjs && node tests/frontend/system_path_frontend_unit.cjs && node tests/frontend/manager_lifecycle_unit.cjs && node tests/frontend/manager_model_cache_unit.cjs && node tests/frontend/manager_model_dir_unit.cjs && node tests/frontend/theme_ui_unit.cjs && npm run test:frontend:modules && npm run test:flag-definitions", "test:frontend:modules": "node tests/frontend/module_namespace_unit.cjs", "test:flag-definitions": "node tests/frontend/flag_definitions_unit.cjs", "test:flags": "node tests/frontend/llama_flags_runner_unit.cjs && node tests/frontend/llama_flags_supported_unit.cjs", diff --git a/tests/frontend/module_namespace_unit.cjs b/tests/frontend/module_namespace_unit.cjs index 7e0c3ed..ac104f6 100644 --- a/tests/frontend/module_namespace_unit.cjs +++ b/tests/frontend/module_namespace_unit.cjs @@ -308,6 +308,7 @@ const facadeKeyContracts = { "captureLaunchSettings", "compareLaunchSettings", "configure", + "getBinaryTagForTool", "getCombinedSpeculativeType", "getCurrentTool", "getFlagValues", @@ -328,6 +329,7 @@ const facadeKeyContracts = { "registerApi", "replaceFlagValues", "setBinaryTag", + "setBinaryTags", "setCurrentTool", "setCurrentToolValue", "setFlagValue", diff --git a/tests/frontend/system_path_frontend_unit.cjs b/tests/frontend/system_path_frontend_unit.cjs new file mode 100644 index 0000000..e9eae4f --- /dev/null +++ b/tests/frontend/system_path_frontend_unit.cjs @@ -0,0 +1,549 @@ +const assert = require("node:assert/strict"); +const fs = require("node:fs"); +const path = require("node:path"); +const vm = require("node:vm"); + +const ROOT = path.resolve(__dirname, "..", ".."); +const { getScriptPaths } = require("./script_order.cjs"); + +// --------------------------------------------------------------------------- +// Shared DOM stub (extends the manager_releases_unit harness with System IDs +// and innerHTML tracking for the safe-path-rendering check). +// --------------------------------------------------------------------------- + +function makeElement(track) { + let html = ""; + const classNames = new Set(); + const listeners = {}; + const el = { + children: [], + value: "", + textContent: "", + className: "", + style: {}, + disabled: false, + checked: false, + open: false, + title: "", + dataset: {}, + classList: { + add(...names) { + names.forEach((name) => classNames.add(name)); + el.className = Array.from(classNames).join(" "); + }, + remove(...names) { + names.forEach((name) => classNames.delete(name)); + el.className = Array.from(classNames).join(" "); + }, + toggle(name, force) { + const shouldAdd = force === undefined ? !classNames.has(name) : Boolean(force); + if (shouldAdd) classNames.add(name); + else classNames.delete(name); + el.className = Array.from(classNames).join(" "); + return shouldAdd; + }, + contains(name) { + return classNames.has(name); + }, + }, + appendChild(child) { + this.children.push(child); + return child; + }, + append(...nodes) { + for (const node of nodes) this.children.push(node); + return this; + }, + replaceChildren(...nodes) { + this.children = nodes; + return this; + }, + listeners, + addEventListener(type, handler) { (listeners[type] ||= []).push(handler); }, + removeEventListener() {}, + setAttribute(name, value) { el.dataset[String(name)] = String(value); }, + getAttribute(name) { return el.dataset[String(name)] ?? null; }, + querySelectorAll() { return []; }, + querySelector() { return null; }, + cloneNode() { const c = makeElement(track); c.value = el.value; c.textContent = el.textContent; return c; }, + setCustomValidity() {}, + focus() {}, + }; + Object.defineProperty(el, "options", { + get() { return this.children; }, + configurable: true, + }); + Object.defineProperty(el, "innerHTML", { + get() { return html; }, + set(value) { + html = String(value || ""); + if (typeof track === "function") track(html); + else if (track && typeof track.push === "function") track.push(html); + this.children = []; + this.value = ""; + }, + configurable: true, + }); + return el; +} + +const SYSTEM_PATH_GUIDANCE = "llama-server was not found on the PATH inherited by Llama GUI. Start the GUI from the environment where it is available, or restart the GUI after changing its launch environment."; + +function systemStatus(overrides = {}) { + return { + installed: true, + config_stale: false, + version: "system", + backend: "system", + tag: "system", + running: false, + platform: "linux", + platform_label: "Linux", + arch: "x64", + executable_suffix: "", + available_backends: [ + { id: "cpu", label: "CPU" }, + { id: "system", label: "System (PATH)", system_path: true }, + ], + executables: { + "llama-server": true, + "llama-cli": false, + "llama-bench": false, + }, + official_install: { backend: "cpu", tag: "b123", version: "b123", files_present: true }, + runtime_health: { + ok: true, + checked: true, + build_tags: { "llama-server": "b1234" }, + system_probes: { + "llama-server": { ok: true, build_tag: "b1234", error: null, executable: "/usr/bin/llama-server" }, + }, + }, + system_tools: { + "llama-server": { path: "/usr/bin/llama-server", available: true, build_tag: "b1234", probe_ok: true, probe_error: null }, + "llama-cli": { path: null, available: false, build_tag: null, probe_ok: null, probe_error: null }, + "llama-bench": { path: null, available: false, build_tag: null, probe_ok: null, probe_error: null }, + }, + system_server_error: null, + ...overrides, + }; +} + +// --------------------------------------------------------------------------- +// 1. Manager: System selection, activation, update/repair, pending, rendering. +// --------------------------------------------------------------------------- + +async function testManager() { + const innerHtmlWrites = []; + const track = (html) => innerHtmlWrites.push(html); + const elements = new Map(); + [ + "release-select", "refresh-releases", "backend-discovery-status", "backend-select", + "installed-backend-summary", "version-badge", "installed-info", "btn-repair", + "btn-install", "btn-update", "release-group", "custom-backend-info", + "custom-backend-folder", "custom-backend-title", "system-backend-info", + "system-backend-tools", "system-backend-title", "install-status", + "download-progress", "progress-fill", "progress-text", + ].forEach((id) => elements.set(id, makeElement(track))); + + const fetchCalls = []; + let latestStatus = systemStatus({ installed: false, config_stale: false, version: null, backend: null }); + // Preview before activation: PATH discovery visible, server present. + latestStatus.system_tools["llama-server"] = { path: "/usr/bin/llama-server", available: true, build_tag: "b1234", probe_ok: true, probe_error: null }; + + const context = { + window: { addEventListener() {}, LlamaGui: {} }, + document: { + createElement: () => makeElement(track), + createTextNode: (text) => ({ textContent: String(text || "") }), + getElementById: (id) => elements.get(id) || null, + }, + console, + fetch: async (url, options) => { + fetchCalls.push({ url, options }); + if (url === "/api/status") return { ok: true, json: async () => latestStatus }; + if (url === "/api/activate-system") { + return { ok: true, json: async () => ({ ok: true, found: ["llama-server"], missing: ["llama-cli"], server_path: "/usr/bin/llama-server", paths: { "llama-server": "/usr/bin/llama-server" }, build_tags: { "llama-server": "b1234" } }) }; + } + if (url.startsWith("/api/releases")) return { ok: true, json: async () => [] }; + if (url.startsWith("/api/backends")) return { ok: true, json: async () => ({ warning: "" }) }; + return { ok: true, json: async () => ({}) }; + }, + }; + context.window.window = context.window; + vm.createContext(context); + context.window.__LLAMA_GUI_TEST_HOOKS__ = true; + for (const src of getScriptPaths().filter((s) => s === "js/api-client.js" || s.startsWith("js/manager/"))) { + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui", src), "utf8"), context, { filename: `ui/${src}` }); + } + const manager = context.window.LlamaGui.manager; + const test = manager._test; + assert.equal(typeof test.isSystemBackend, "function", "isSystemBackend must be exposed for tests"); + assert.equal(test.isSystemBackend("system", latestStatus), true); + assert.equal(test.isSystemBackend("cpu", latestStatus), false); + + // Selecting System shows Activate System, hides releases, shows setup text, no fetch. + const backendSelect = elements.get("backend-select"); + await manager.checkStatus(); + const releaseCallsBefore = fetchCalls.filter((c) => c.url.startsWith("/api/releases")).length; + backendSelect.value = "system"; + test.onBackendChange(); + assert.equal(elements.get("btn-install").textContent, "Activate System"); + assert.equal(elements.get("release-group").style.display, "none"); + assert.equal(elements.get("system-backend-info").style.display, ""); + assert.equal(elements.get("custom-backend-info").style.display, "none"); + assert.match(elements.get("system-backend-tools").textContent, /llama-server.*\/usr\/bin\/llama-server/); + assert.equal(elements.get("btn-update").disabled, true); + assert.equal(elements.get("btn-repair").disabled, true); + assert.equal( + elements.get("btn-repair").classList.contains("hidden"), + false, + "repair button should be visible but disabled when system is selected as install target" + ); + assert.equal( + fetchCalls.filter((c) => c.url.startsWith("/api/releases")).length, + releaseCallsBefore, + "selecting System must not fetch releases" + ); + + // Activation routes to /api/activate-system with duplicate guard. + let releaseGate; + const pendingCalls = []; + context.fetch = async (url, options) => { + fetchCalls.push({ url, options }); + if (url === "/api/status") return { ok: true, json: async () => latestStatus }; + if (url === "/api/activate-system") { + pendingCalls.push(JSON.parse(options.body)); + await new Promise((resolve) => { releaseGate = resolve; }); + return { ok: true, json: async () => ({ ok: true, found: ["llama-server"], missing: ["llama-cli"], server_path: "/usr/bin/llama-server", paths: {}, build_tags: {} }) }; + } + return { ok: true, json: async () => [] }; + }; + const first = manager._test.installRelease(); + const second = manager._test.installRelease(); + await Promise.resolve(); + await Promise.resolve(); + assert.deepEqual(pendingCalls, [{}], "duplicate System activation must be ignored"); + assert.equal(elements.get("btn-install").disabled, true, "polling must not unlock activation in flight"); + latestStatus = systemStatus(); + releaseGate(); + await first; + await second; + assert.match(elements.get("install-status").textContent, /System.*activated.*llama-server/); + assert.equal(elements.get("installed-backend-summary").textContent, "Installed backend: System (PATH)"); + assert.match(elements.get("version-badge").textContent, /System \(PATH\).*b1234/); + + // Installed rendering: server required, CLI feature-specific, paths as text. + // Stub DOM nodes do not aggregate descendant text like real DOM, so walk + // the tree the way textContent would read. + const collectText = (node) => { + let out = String(node.textContent || ""); + for (const child of node.children || []) { + if (typeof child === "string") out += child; + else if (child && typeof child === "object") out += " " + collectText(child); + } + return out; + }; + const info = elements.get("installed-info"); + const dump = collectText(info); + assert.match(dump, /Required tool/); + assert.match(dump, /Feature tools/); + assert.match(dump, /\/usr\/bin\/llama-server/); + assert.match(dump, /unknown|b1234/); + for (const html of innerHtmlWrites) { + assert.ok(!html.includes("/usr/bin"), "discovered paths must render via textContent, never innerHTML"); + } + + // Unknown build shows explicit unknown display, gates stay conservative. + latestStatus = systemStatus({ + runtime_health: { ok: true, checked: true, build_tags: {}, system_probes: { "llama-server": { ok: true, build_tag: null, error: null, executable: "/usr/bin/llama-server" } } }, + system_tools: { + "llama-server": { path: "/usr/bin/llama-server", available: true, build_tag: null, probe_ok: true, probe_error: null }, + "llama-cli": { path: null, available: false, build_tag: null, probe_ok: null, probe_error: null }, + }, + }); + await manager.checkStatus(); + assert.match(elements.get("version-badge").textContent, /unknown build/); + + // Stale (removed server) shows PATH guidance and keeps selection. + latestStatus = systemStatus({ + installed: false, config_stale: true, + runtime_health: { ok: true, checked: false, build_tags: {}, system_probes: {} }, + system_tools: { + "llama-server": { path: null, available: false, build_tag: null, probe_ok: null, probe_error: null }, + "llama-cli": { path: null, available: false, build_tag: null, probe_ok: null, probe_error: null }, + }, + system_server_error: SYSTEM_PATH_GUIDANCE, + }); + await manager.checkStatus(); + assert.match(collectText(elements.get("installed-info")), /PATH inherited by Llama GUI/); + assert.equal(elements.get("installed-backend-summary").textContent, "Configured backend: System (PATH) (incomplete)"); + assert.equal(backendSelect.value, "system", "stale selection must not reset"); + + // Failed activation surfaces the backend error near the controls. + context.fetch = async (url) => { + if (url === "/api/status") return { ok: true, json: async () => latestStatus }; + if (url === "/api/activate-system") return { ok: true, json: async () => ({ ok: false, error: SYSTEM_PATH_GUIDANCE }) }; + return { ok: true, json: async () => ({}) }; + }; + backendSelect.value = "system"; + test.onBackendChange(); + await manager._test.installRelease(); + assert.match(elements.get("install-status").textContent, /PATH inherited by Llama GUI/); + + // Official return: selecting an official target while System is active + // offers Activate Existing derived from accepted status. + latestStatus = systemStatus(); + await manager.checkStatus(); + assert.equal(test.canActivateOfficialBackend(latestStatus, "cpu"), true); + assert.equal(test.canActivateOfficialBackend(latestStatus, "system"), false); + backendSelect.value = "cpu"; + test.onBackendChange(); + await manager.checkStatus(); + assert.equal(elements.get("btn-install").textContent, "Activate Existing"); + assert.equal(backendSelect.value, "cpu", "pending official target survives status polls"); + + // Forced backend refresh must not fetch releases for System. + const releasesBefore = fetchCalls.filter((c) => c.url.startsWith("/api/releases")).length; + backendSelect.value = "system"; + test.onBackendChange(); + await manager.refreshBackends(true); + assert.equal( + fetchCalls.filter((c) => c.url.startsWith("/api/releases")).length, + releasesBefore, + "forced refresh must suppress release fetching for System" + ); + + console.log("system manager unit tests passed"); +} + +// --------------------------------------------------------------------------- +// 2. Flag core: per-tool build tags. +// --------------------------------------------------------------------------- + +function testFlagCore() { + const context = { window: { LlamaGui: {} }, console, shouldOmitLegacyLoadFlag: () => false }; + context.window.window = context.window; + vm.createContext(context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/flag-core.js"), "utf8"), context, { filename: "ui/js/flag-core.js" }); + const core = context.window.LlamaGui.flagCore; + + core.setBinaryTag("system"); + core.setBinaryTags({ "llama-server": "b11000", "llama-cli": "custom" }); + assert.equal(core.getBinaryTagForTool("llama-server"), "b11000"); + assert.equal(core.getBinaryTagForTool("llama-cli"), "custom"); + assert.equal(core.supportsLoadModeOnly("llama-server"), true); + assert.equal(core.supportsLoadModeOnly("llama-cli"), false); + assert.equal(core.supportsNativeReasoningEffort("llama-server"), true); + assert.equal(core.supportsNativeReasoningEffort("llama-cli"), false); + + // Unknown build formats stay conservative. + core.setBinaryTags({ "llama-server": "" }); + assert.equal(core.supportsLoadModeOnly("llama-server"), false); + assert.equal(core.supportsNativeReasoningEffort("llama-server"), false); + + // buildLaunchArgs uses the tag for the tool being generated, not the + // currently selected tool: server from a new build emits the native flag + // while CLI from an old build keeps the legacy kwargs path. + vm.runInContext(` + window.LlamaGui.flagCore.configure({ + getDefaultFlagValues: () => ({}), + getFlags: () => [ + { id: "chat_template_reasoning_effort", flag: "--reasoning-effort", tool: "server", type: "text" }, + { id: "preserve_thinking", flag: "--preserve-thinking", tool: "server", type: "bool" }, + ], + }); + window.LlamaGui.flagCore.setCurrentToolValue("llama-cli"); + window.LlamaGui.flagCore.setBinaryTag("system"); + window.LlamaGui.flagCore.setBinaryTags({ "llama-server": "b11000", "llama-cli": "b10000" }); + `, context); + const serverArgs = vm.runInContext( + `window.LlamaGui.flagCore.buildLaunchArgs({ tool: "llama-server", model: "", flags: { chat_template_reasoning_effort: "high" } }).args.flat()`, + context + ); + assert.ok(serverArgs.includes("--reasoning-effort"), "new server build must emit the native flag"); + const cliArgs = vm.runInContext( + `window.LlamaGui.flagCore.buildLaunchArgs({ tool: "llama-cli", model: "", flags: {} }).args`, + context + ); + assert.equal(cliArgs.length, 0, "per-tool gates must not leak across tools"); + + console.log("system flag-core unit tests passed"); +} + +// --------------------------------------------------------------------------- +// 3. Quick Launch: server-only readiness, CLI-specific guidance. +// --------------------------------------------------------------------------- + +function testQuickLaunch() { + const ids = [ + "quick-command-preview", "command-preview-text", "quick-context-preset", "quick-context-custom", + "quick-gpu-mode", "quick-gpu-custom", "quick-fit-toggle", "quick-fit-target", "quick-fit-ctx", + "quick-fit-summary", "quick-template-pack", "quick-template-summary", "quick-temperature", + "quick-top-k", "quick-top-p", "quick-min-p", "quick-repeat-penalty", "quick-presence-penalty", + "quick-temperature-input", "quick-top-p-input", "quick-port", "quick-profile-select", + "quick-profile-summary", "quick-metrics-toggle", "quick-api-key-control", "quick-auth-section", + "quick-server-fields", "quick-server-summary", "model-select", "quick-model-select", + "quick-chip-model", "quick-chip-profile", "quick-chip-context", "quick-chip-gpu", "quick-chip-api", + "quick-api-protected-badge", "btn-quick-launch", "btn-quick-stop", "btn-launch", "btn-stop", + "btn-sidebar-launch", "btn-sidebar-stop", "btn-quick-launch-label", "btn-quick-stop-label", + "btn-sidebar-launch-label", "btn-sidebar-stop-label", "quick-launch-readiness", + "sidebar-launch-reason", "quick-launch-status", "quick-runtime-state", "quick-runtime-model", + "quick-runtime-build", "quick-server-url", "quick-server-webui", "quick-saved-presets", + "quick-presets-status", + ]; + const elements = new Map(ids.map((id) => [id, makeElement()])); + for (const id of ["btn-launch", "btn-stop", "btn-quick-launch", "btn-quick-stop", "btn-sidebar-launch", "btn-sidebar-stop"]) { + elements.get(id).classList.remove("hidden"); + } + elements.get("model-select").value = "smoke.gguf"; + elements.get("model-select").options = [{ value: "smoke.gguf" }]; + elements.get("command-preview-text").textContent = "llama-server -m models/smoke.gguf"; + elements.get("quick-template-pack").selectedOptions = [{ textContent: "Auto" }]; + + let currentTool = "llama-server"; + let latestStatus = systemStatus(); + const context = { + window: { LlamaGui: { searchableSelect: { enhance() {} } } }, + document: { + getElementById: (id) => elements.get(id) || null, + createElement: () => makeElement(), + createTextNode: (text) => ({ textContent: String(text || "") }), + querySelector: () => ({ textContent: "", classList: { toggle() {}, add() {}, remove() {} } }), + querySelectorAll: () => [], + }, + console: { ...console, debug: () => {} }, + FLAGS: [], + QUICK_PROFILES: {}, + QUICK_CONTEXT_PRESETS: ["8192"], + setTimeout, clearTimeout, + }; + context.window.window = context.window; + vm.createContext(context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/quick-launch-ui.js"), "utf8"), context, { filename: "ui/js/quick-launch-ui.js" }); + const ui = context.window.LlamaGui.quickLaunchUi; + const flagCore = { + getCurrentTool: () => currentTool, + getFlagValues: () => ({}), + getSelectedModel: () => "smoke.gguf", + setSelectedModelValue: () => {}, + getLaunchArgs: () => ({ args: [["-m", "models/smoke.gguf"]], error: null }), + normalizeGpuLayersValue: (v) => v, + updateCommandPreview: () => {}, + }; + ui.configure({ + flagCore, + getLifecycleSnapshot: () => ({ activeRuntime: null, busy: false, phase: "idle" }), + getLatestStatus: () => latestStatus, + configFlagsUi: { ensureChatTemplateOption() {}, syncSensitiveTextInput() {}, createSensitiveTextInput: () => makeElement() }, + hasLaunchModelArg: (args) => (args || []).flat().includes("-m"), + updateQuickServerAddressPreview: () => {}, + getSelectedChatTemplateDropdownValue: () => "", + getQuickTemplateSummaryText: () => "", + getAllSamplerPresets: () => [], + }); + + ui.refresh(); + assert.equal(elements.get("btn-quick-launch").disabled, false, "server-only System must leave server launches usable"); + assert.equal(elements.get("btn-quick-launch").title, ""); + + currentTool = "llama-cli"; + ui.refresh(); + assert.equal(elements.get("btn-quick-launch").disabled, true, "missing CLI must block Terminal launches"); + assert.match(elements.get("btn-quick-launch").title, /llama-cli was not found/); + assert.match(elements.get("btn-quick-launch").title, /server.*usable|Chat.*usable/i); + + latestStatus = systemStatus({ + installed: false, config_stale: true, + system_tools: { + "llama-server": { path: null, available: false }, + "llama-cli": { path: null, available: false }, + }, + system_server_error: SYSTEM_PATH_GUIDANCE, + }); + currentTool = "llama-server"; + ui.refresh(); + assert.equal(elements.get("btn-quick-launch").disabled, true); + assert.match(elements.get("btn-quick-launch").title, /PATH inherited by Llama GUI/); + + console.log("system quick-launch unit tests passed"); +} + +// --------------------------------------------------------------------------- +// 4. Benchmark: missing-tool guidance and per-tool load-mode gate. +// --------------------------------------------------------------------------- + +function testBenchmark() { + const context = { + window: { LlamaGui: {}, addEventListener: () => {}, __LLAMA_GUI_TEST_HOOKS__: true }, + document: { getElementById: () => null, createElement: () => ({ appendChild: () => {}, classList: { toggle: () => {} } }) }, + console: { ...console, debug: () => {} }, + setInterval, clearInterval, + }; + context.window.window = context.window; + vm.createContext(context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/output-cursor.js"), "utf8"), context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/flags/helpers.js"), "utf8"), context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/flag-core.js"), "utf8"), context); + vm.runInContext(`window.LlamaGui.flagCore.setModelDirInfo({ + models_dir: "models", models_arg_root: "models", + models_dir_is_default: true, models_dir_available: true, models_dir_error: "", + })`, context); + vm.runInContext(fs.readFileSync(path.join(ROOT, "ui/js/benchmark-ui.js"), "utf8"), context); + const bench = context.window.LlamaGui.benchmarkUi; + const core = context.window.LlamaGui.flagCore; + + // Per-tool gate: bench from a new build translates mmap even when the + // server tag is old. + core.setBinaryTag("system"); + core.setBinaryTags({ "llama-bench": "b10900", "llama-server": "b10000" }); + const flags = [ + { id: "mmap", flag: "--mmap", type: "bool", label: "Mmap" }, + { id: "ctx_size", flag: "-c", type: "int", label: "Context" }, + ]; + const translated = bench.buildBenchmarkArgs({ + benchmarkType: "bench", flags, defaultFlags: {}, + source: { sourceType: "manual", model: "m.gguf", flags: { mmap: true } }, + }); + assert.ok(translated.args.some((entry) => entry[0] === "--load-mode"), "new bench build must translate legacy mmap"); + core.setBinaryTags({ "llama-bench": "b10000" }); + const legacy = bench.buildBenchmarkArgs({ + benchmarkType: "bench", flags, defaultFlags: {}, + source: { sourceType: "manual", model: "m.gguf", flags: { mmap: true } }, + }); + assert.ok(legacy.args.some((entry) => entry[0] === "-mmp"), "old bench build must keep the legacy flag"); + + // Missing bench tool produces feature-specific guidance via renderCommand. + const nodes = {}; + for (const id of ["benchmark-command-preview", "benchmark-status", "btn-run-benchmark", "benchmark-source-summary", "benchmark-applied-list", "benchmark-excluded-list", "benchmark-type", "benchmark-source", "benchmark-manual-model", "benchmark-preset-select"]) { + const el = makeElement(); + el.value = id === "benchmark-type" ? "bench" : id === "benchmark-source" ? "manual" : "m.gguf"; + nodes[id] = el; + } + context.window.LlamaGui.benchmarkUi.configure({ + flagCore: core, + getFlags: () => [], + getDefaultFlagValues: () => ({}), + getLatestStatus: () => systemStatus(), + }); + const documentBackup = context.document; + context.document.getElementById = (id) => nodes[id] || null; + context.document.createElement = () => ({ textContent: "", className: "", appendChild() {} }); + const result = bench.renderCommand(); + assert.match(result.error || "", /llama-bench was not found/); + assert.equal(nodes["btn-run-benchmark"].disabled, true); + void documentBackup; + + console.log("system benchmark unit tests passed"); +} + +(async () => { + await testManager(); + testFlagCore(); + testQuickLaunch(); + testBenchmark(); + console.log("system path frontend unit tests passed"); +})().catch((err) => { + console.error(err); + process.exit(1); +}); diff --git a/ui/index.html b/ui/index.html index c487d50..e68a057 100644 --- a/ui/index.html +++ b/ui/index.html @@ -261,6 +261,19 @@

Install

+ +
diff --git a/ui/js/benchmark-ui.js b/ui/js/benchmark-ui.js index 63af36e..6e21de3 100644 --- a/ui/js/benchmark-ui.js +++ b/ui/js/benchmark-ui.js @@ -185,9 +185,17 @@ // llama.cpp b10875 removed --mmap/--no-mmap, --mlock, and the -dio family // from every tool — llama-bench and llama-perplexity only advertise // --load-mode on such builds — so legacy load toggles must translate. - function isLoadModeOnlyBuild() { + // System (PATH) can resolve each tool from a different build, so the + // gate uses the tag for the benchmark tool being configured. + function isLoadModeOnlyBuild(tool) { const core = root.flagCore; - return Boolean(core && typeof core.supportsLoadModeOnly === "function" && core.supportsLoadModeOnly()); + if (!core || typeof core.supportsLoadModeOnly !== "function") return false; + try { + return Boolean(core.supportsLoadModeOnly(tool || "llama-bench")); + } catch (e) { + console.debug("Load-mode gate check failed", e); + return false; + } } function getLoadModeArg(args) { @@ -210,7 +218,7 @@ // On b10875+ builds incompatible legacy toggles cannot share the single // --load-mode slot, so name the winner instead of a generic exclusion. function loadModeConflictReason(flag, args) { - if (!isLoadModeOnlyBuild() || !hasLoadModeArg(args)) return ""; + if (!isLoadModeOnlyBuild("llama-bench") || !hasLoadModeArg(args)) return ""; if (flag.id !== "mmap" && flag.id !== "mlock" && flag.id !== "direct_io") return ""; const winner = getLoadModeArgValue(args); return winner @@ -235,7 +243,7 @@ } } if (flag.id === "mmap") { - if (isLoadModeOnlyBuild()) { + if (isLoadModeOnlyBuild(tool)) { const loadModeArg = getLoadModeArg(args); if (loadModeArg) { if (value && loadModeArg[1] === "mlock") { @@ -250,7 +258,7 @@ args.push(["-mmp", value ? "1" : "0"]); return true; } - if (flag.id === "mlock" && isLoadModeOnlyBuild()) { + if (flag.id === "mlock" && isLoadModeOnlyBuild(tool)) { // mlock is opt-in, so only the enabled state maps onto // --load-mode; off matches the new-build default behavior. if (!value) return false; @@ -266,7 +274,7 @@ return true; } if (flag.id === "direct_io") { - if (isLoadModeOnlyBuild()) { + if (isLoadModeOnlyBuild(tool)) { // "dio off" is the default load behavior on new builds, so // only the enabled state maps onto --load-mode. if (!value || hasLoadModeArg(args)) return false; @@ -424,7 +432,7 @@ if (flashAttention) args.push(["-fa", flashAttention]); if (cacheTypeK) args.push(["-ctk", cacheTypeK]); if (cacheTypeV) args.push(["-ctv", cacheTypeV]); - const mmapOffArg = isLoadModeOnlyBuild() ? ["--load-mode", "none"] : ["--no-mmap"]; + const mmapOffArg = isLoadModeOnlyBuild("llama-perplexity") ? ["--load-mode", "none"] : ["--no-mmap"]; if (options.pplMmap === false) args.push(mmapOffArg); args.push(["-f", String(options.promptFile)]); if (options.chunks !== undefined && options.chunks !== "") args.push(["--chunks", String(options.chunks)]); @@ -647,8 +655,28 @@ }; } + function benchmarkToolAvailability(tool) { + const status = typeof getLatestStatus === "function" ? getLatestStatus() : null; + if (!status) return null; + const systemTools = status.system_tools; + if (systemTools && typeof systemTools === "object" && systemTools[tool]) { + return Boolean(systemTools[tool].available); + } + const exes = status.executables; + if (exes && typeof exes === "object") { + const suffix = typeof status.executable_suffix === "string" ? status.executable_suffix : ""; + return Boolean(exes[tool + suffix]); + } + return null; + } + function renderCommand() { - const result = buildBenchmarkArgs(getBuildOptions()); + let result = buildBenchmarkArgs(getBuildOptions()); + const unavailable = benchmarkToolAvailability(result.tool); + if (!result.error && unavailable === false) { + const feature = result.tool === "llama-bench" ? "Throughput benchmarks" : "Perplexity runs"; + result = { ...result, error: `${result.tool} was not found on the PATH inherited by Llama GUI. ${feature} need it; other features remain usable.`, command: "" }; + } const command = byId("benchmark-command-preview"); const status = byId("benchmark-status"); const runBtn = byId("btn-run-benchmark"); diff --git a/ui/js/flag-core.js b/ui/js/flag-core.js index 3bddcab..2b824af 100644 --- a/ui/js/flag-core.js +++ b/ui/js/flag-core.js @@ -536,14 +536,48 @@ // Gate for llama.cpp b10434+, where --reasoning-effort became a native // launch flag (upstream PR 26941). The tag comes from /api/status // (config.json "version"): official releases use "bNNNNN"; both custom - // slots use "custom". Anything unrecognized stays on the legacy - // --chat-template-kwargs path, which every build accepts. + // slots use "custom". System (PATH) backends report per-tool build tags + // from bounded --version probes; PATH can resolve CLI and server from + // different builds, so the gate uses the tag for the tool being launched. + // Anything unrecognized stays on the legacy --chat-template-kwargs path, + // which every build accepts. function setBinaryTag(tag) { binaryTag = String(tag || ""); } - function supportsNativeReasoningEffort() { - const match = /^b(\d+)/.exec(binaryTag); + // Per-tool build tags for System (PATH) installations, keyed by full + // tool name ("llama-server", "llama-cli", ...). Official and Custom + // backends share one build, so they keep using the single binaryTag. + let binaryTags = {}; + + function normalizeBinaryToolKey(tool) { + const name = String(tool || "").trim(); + if (!name) return ""; + if (name.startsWith("llama-")) return name; + if (name === "server" || name === "cli" || name === "bench" || name === "perplexity") { + return `llama-${name}`; + } + return name; + } + + function setBinaryTags(tags) { + binaryTags = {}; + if (tags && typeof tags === "object" && !Array.isArray(tags)) { + for (const [tool, tag] of Object.entries(tags)) { + const key = normalizeBinaryToolKey(tool); + if (key) binaryTags[key] = String(tag || ""); + } + } + } + + function getBinaryTagForTool(tool) { + const key = normalizeBinaryToolKey(tool || currentTool); + if (key && binaryTags[key]) return binaryTags[key]; + return binaryTag; + } + + function supportsNativeReasoningEffort(tool) { + const match = /^b(\d+)/.exec(getBinaryTagForTool(tool || currentTool)); return Boolean(match) && Number(match[1]) >= 10434; } @@ -552,10 +586,11 @@ // llama-cli in favor of --load-mode (upstream PR 28334). The tag comes // from /api/status like the reasoning-effort gate above: official // releases use "bNNNNN" (trailing build suffixes such as "-cuda" are - // accepted); custom slots use "custom". Anything unrecognized stays - // silent, keeping older and custom builds working. - function supportsLoadModeOnly() { - const match = /^b(\d+)/.exec(binaryTag); + // accepted); custom slots use "custom"; System uses the per-tool tag. + // Anything unrecognized stays silent, keeping older and custom builds + // working. + function supportsLoadModeOnly(tool) { + const match = /^b(\d+)/.exec(getBinaryTagForTool(tool || currentTool)); return Boolean(match) && Number(match[1]) >= 10875; } @@ -592,7 +627,7 @@ if (f.id === "chat_template_reasoning_effort") { if (val && val !== "auto") { - if (supportsNativeReasoningEffort()) { + if (supportsNativeReasoningEffort(tool)) { args.push([f.flag, val]); } else { const kwargs = {}; @@ -614,7 +649,7 @@ // On the legacy path with a non-auto effort, preserve_thinking // is merged into the single kwargs object emitted above. const mergedIntoEffortKwargs = toolBase === "server" - && !supportsNativeReasoningEffort() + && !supportsNativeReasoningEffort(tool) && values.chat_template_reasoning_effort && values.chat_template_reasoning_effort !== "auto"; if (val === true && !mergedIntoEffortKwargs) { @@ -711,7 +746,7 @@ args.push(["-m", localModel.path]); } - if (supportsLoadModeOnly()) { + if (supportsLoadModeOnly(tool)) { const removedLoadFlags = new Set(["--mmap", "--no-mmap", "--mlock", "-dio", "-ndio", "--direct-io", "--no-direct-io"]); const hasRemovedLoadFlag = args.some((entry) => { if (Array.isArray(entry)) return removedLoadFlags.has(String(entry[0])); @@ -880,6 +915,8 @@ captureLaunchSettings, compareLaunchSettings, setBinaryTag, + setBinaryTags, + getBinaryTagForTool, supportsNativeReasoningEffort, supportsLoadModeOnly, updateCommandPreview, diff --git a/ui/js/manager/manager-backends.js b/ui/js/manager/manager-backends.js index 7195f26..43c4500 100644 --- a/ui/js/manager/manager-backends.js +++ b/ui/js/manager/manager-backends.js @@ -5,6 +5,7 @@ let lastInstalledInfoRenderKey = ""; let pendingInstallBackendId = null; let customActivationInProgress = false; + let systemActivationInProgress = false; function normalizeBackendId(value) { return String(value || "").trim(); @@ -31,6 +32,12 @@ .some((backend) => backend.id === backendId && backend.custom === true); } + function isSystemBackend(backendId, status = I.status.getLatestStatus()) { + if (normalizeBackendId(backendId) === "system") return true; + return backendOptionsFromStatus(status) + .some((backend) => backend && backend.id === backendId && backend.system_path === true); + } + function customBackendFolder(backendId, status = I.status.getLatestStatus()) { const backend = backendOptionsFromStatus(status).find((entry) => entry.id === backendId); return backend && backend.bin_dir ? backend.bin_dir : "llama/custom/bin/"; @@ -44,17 +51,159 @@ const target = normalizeBackendId(backendId); const official = status && status.official_install; const recordedBackend = normalizeBackendId(official && official.backend); + const sourceIsSwitchable = isCustomBackend(status && status.backend, status) + || isSystemBackend(status && status.backend, status); return Boolean( status - && isCustomBackend(status.backend, status) + && sourceIsSwitchable && target && !isCustomBackend(target, status) + && !isSystemBackend(target, status) && official && official.files_present && (!recordedBackend || recordedBackend === target) ); } + function systemBuildTagForTool(status, tool) { + const tags = status && status.runtime_health && status.runtime_health.build_tags; + const tag = tags && typeof tags === "object" ? tags[tool] : null; + return typeof tag === "string" && tag ? tag : null; + } + + function systemToolEntries(status) { + const tools = status && status.system_tools && typeof status.system_tools === "object" + ? status.system_tools + : {}; + return Object.entries(tools); + } + + function describeSystemToolsForSetup(status) { + const entries = systemToolEntries(status); + if (!entries.length) return "Checking PATH for llama.cpp tools..."; + return entries.map(([tool, info]) => { + const path = info && info.path ? info.path : "not found on PATH"; + return `${tool}: ${path}`; + }).join(" · "); + } + + function renderSystemToolRows(status, container) { + const entries = systemToolEntries(status); + for (const [tool, info] of entries) { + const available = Boolean(info && info.available); + const required = tool === "llama-server"; + const row = document.createElement("div"); + row.className = "installed-tool-row"; + const label = document.createElement("code"); + label.textContent = tool; + const state = document.createElement("span"); + state.className = available ? "exe-ok" : required ? "exe-missing" : "exe-optional"; + state.textContent = available ? "Available" : required ? "Missing · required" : "Not installed"; + row.appendChild(label); + row.appendChild(state); + container.appendChild(row); + const pathLine = document.createElement("div"); + pathLine.className = "installed-info-hint"; + const toolPath = info && info.path ? info.path : "not found on PATH"; + const tag = info && info.build_tag ? ` · build ${info.build_tag}` : ""; + pathLine.textContent = toolPath + tag; + container.appendChild(pathLine); + } + } + + function renderInstalledSystemStatus(status, appendRow, info, optionalToolsOpen) { + appendRow("Backend", backendLabelFromStatus(status, status.backend)); + const serverInfo = (status.system_tools || {})["llama-server"] || {}; + if (serverInfo.path) appendRow("Server", serverInfo.path); + appendRow("Server build", serverInfo.build_tag || "unknown build"); + + const requiredBox = document.createElement("div"); + requiredBox.className = "installed-tools"; + const requiredTitle = document.createElement("h4"); + requiredTitle.textContent = "Required tool"; + requiredBox.appendChild(requiredTitle); + const serverRow = document.createElement("div"); + serverRow.className = "installed-tool-row"; + const serverLabel = document.createElement("code"); + serverLabel.textContent = "llama-server"; + const serverState = document.createElement("span"); + serverState.className = serverInfo.available ? "exe-ok" : "exe-missing"; + serverState.textContent = serverInfo.available ? "Available" : "Missing · required"; + serverRow.appendChild(serverLabel); + serverRow.appendChild(serverState); + requiredBox.appendChild(serverRow); + if (serverInfo.path) { + const pathLine = document.createElement("div"); + pathLine.className = "installed-info-hint"; + pathLine.textContent = serverInfo.path; + requiredBox.appendChild(pathLine); + } + info.appendChild(requiredBox); + + const others = systemToolEntries(status).filter(([tool]) => tool !== "llama-server"); + if (others.length) { + const optionalBox = document.createElement("details"); + optionalBox.id = "installed-optional-tools"; + optionalBox.className = "installed-tools"; + optionalBox.open = Boolean(optionalToolsOpen); + const availableCount = others.filter(([, entry]) => entry && entry.available).length; + const optionalTitle = document.createElement("summary"); + optionalTitle.textContent = `Feature tools · ${availableCount} of ${others.length} available`; + optionalBox.appendChild(optionalTitle); + const hint = document.createElement("p"); + hint.className = "installed-info-hint"; + hint.textContent = "Each tool is optional for its own feature: Terminal needs llama-cli; Benchmarking needs llama-bench or llama-perplexity."; + optionalBox.appendChild(hint); + for (const [tool, entry] of others) { + const row = document.createElement("div"); + row.className = "installed-tool-row"; + const label = document.createElement("code"); + label.textContent = tool; + const state = document.createElement("span"); + state.className = entry && entry.available ? "exe-ok" : "exe-optional"; + state.textContent = entry && entry.available ? "Available" : "Not installed"; + row.appendChild(label); + row.appendChild(state); + optionalBox.appendChild(row); + const pathLine = document.createElement("div"); + pathLine.className = "installed-info-hint"; + const toolPath = entry && entry.path ? entry.path : "not found on PATH"; + const tag = entry && entry.build_tag ? ` · build ${entry.build_tag}` : ""; + pathLine.textContent = toolPath + tag; + optionalBox.appendChild(pathLine); + } + info.appendChild(optionalBox); + } + + const note = document.createElement("div"); + note.className = "installed-info-hint"; + note.textContent = "Installation and updates are managed by the OS package manager."; + info.appendChild(note); + } + + function renderStaleSystemStatus(status, appendRow, info) { + const warning = document.createElement("div"); + warning.className = "installed-info-warning"; + warning.textContent = status.system_server_error + || "Configuration exists, but the required llama-server was not found."; + info.appendChild(warning); + + const hint = document.createElement("div"); + hint.className = "installed-info-hint"; + hint.textContent = "System tools are discovered from the PATH inherited by Llama GUI. " + + "Start the GUI from the environment where llama-server is available, " + + "or restart the GUI after changing its launch environment. " + + "Updates are managed by the OS package manager."; + info.appendChild(hint); + + appendRow("Version (config)", String(status.version)); + appendRow("Backend (config)", backendLabelFromStatus(status, status.backend)); + const box = document.createElement("div"); + box.className = "installed-tools"; + renderSystemToolRows(status, box); + info.appendChild(box); + } + function renderBackendOptions(status) { const backendSelect = document.getElementById("backend-select"); if (!backendSelect) return; @@ -128,9 +277,15 @@ const hasInstalledBackend = Boolean(status && status.installed && installedBackend); const hasStaleBackendConfig = Boolean(status && status.config_stale && installedBackend); const customTargetSelected = isCustomBackend(installTarget, status); + const systemTargetSelected = isSystemBackend(installTarget, status); + const systemInstalled = isSystemBackend(installedBackend, status); + const customInstalled = isCustomBackend(installedBackend, status); const canActivateExisting = canActivateOfficialBackend(status, installTarget); - if (installBtn && !customTargetSelected) { + if (installBtn && systemTargetSelected) { + installBtn.textContent = "Activate System"; + installBtn.title = "Use the llama.cpp tools on the PATH inherited by Llama GUI"; + } else if (installBtn && !customTargetSelected) { installBtn.textContent = canActivateExisting ? "Activate Existing" : "Install"; installBtn.title = canActivateExisting ? "Use the official llama.cpp files already installed in llama/bin" @@ -138,24 +293,28 @@ } if (updateBtn) { - const canUpdate = !customTargetSelected && hasInstalledBackend && !isCustomBackend(installedBackend, status); + const canUpdate = !customTargetSelected && !systemTargetSelected && hasInstalledBackend && !customInstalled && !systemInstalled; updateBtn.disabled = !canUpdate; updateBtn.title = canUpdate ? "Check the installed backend for updates" - : customTargetSelected || isCustomBackend(installedBackend, status) + : customTargetSelected || customInstalled ? "Custom backend installations are managed manually" - : "Install llama.cpp before checking for updates"; + : systemTargetSelected || systemInstalled + ? "System (PATH) tools are updated by the OS package manager" + : "Install llama.cpp before checking for updates"; } if (repairBtn) { - const canRepair = !customTargetSelected && hasStaleBackendConfig && !isCustomBackend(installedBackend, status); - repairBtn.classList.toggle("hidden", !canRepair && !customTargetSelected); + const canRepair = !customTargetSelected && !systemTargetSelected && hasStaleBackendConfig && !customInstalled && !systemInstalled; + repairBtn.classList.toggle("hidden", !canRepair && !customTargetSelected && !systemTargetSelected); repairBtn.disabled = !canRepair; - repairBtn.title = customTargetSelected + repairBtn.title = customTargetSelected || customInstalled ? "Custom backend files are managed manually" - : canRepair - ? "Reinstall the configured backend files" - : "Repair is available only for incomplete default backend installs"; + : systemTargetSelected || systemInstalled + ? "System (PATH) tools are managed by the OS package manager" + : canRepair + ? "Reinstall the configured backend files" + : "Repair is available only for incomplete default backend installs"; } } @@ -180,6 +339,8 @@ if (releaseGroup) releaseGroup.style.display = "none"; const customInfo = document.getElementById("custom-backend-info"); if (customInfo) customInfo.style.display = ""; + const systemInfo = document.getElementById("system-backend-info"); + if (systemInfo) systemInfo.style.display = "none"; const folder = document.getElementById("custom-backend-folder"); if (folder) folder.textContent = customBackendFolder(backend, status); const title = document.getElementById("custom-backend-title"); @@ -194,11 +355,45 @@ if (repairBtn) repairBtn.classList.add("hidden"); } + function showSystemBackendControls(backend = selectedBackendId(), status = I.status.getLatestStatus()) { + I.install.resetReleasesForBackend(backend); + + const sel = document.getElementById("release-select"); + if (sel) { + sel.textContent = ""; + const option = document.createElement("option"); + option.value = backend; + option.textContent = backendLabelFromStatus(status, backend); + sel.appendChild(option); + sel.disabled = true; + } + const releaseGroup = document.getElementById("release-group"); + if (releaseGroup) releaseGroup.style.display = "none"; + const customInfo = document.getElementById("custom-backend-info"); + if (customInfo) customInfo.style.display = "none"; + const systemInfo = document.getElementById("system-backend-info"); + if (systemInfo) systemInfo.style.display = ""; + const tools = document.getElementById("system-backend-tools"); + if (tools) tools.textContent = describeSystemToolsForSetup(status); + const title = document.getElementById("system-backend-title"); + if (title) title.textContent = backendLabelFromStatus(status, backend) + " Setup:"; + const installBtn = document.getElementById("btn-install"); + if (installBtn) { + installBtn.textContent = "Activate System"; + } + const updateBtn = document.getElementById("btn-update"); + if (updateBtn) updateBtn.disabled = true; + const repairBtn = document.getElementById("btn-repair"); + if (repairBtn) repairBtn.classList.add("hidden"); + } + function showOfficialBackendControls() { const releaseGroup = document.getElementById("release-group"); if (releaseGroup) releaseGroup.style.display = ""; const customInfo = document.getElementById("custom-backend-info"); if (customInfo) customInfo.style.display = "none"; + const systemInfo = document.getElementById("system-backend-info"); + if (systemInfo) systemInfo.style.display = "none"; const sel = document.getElementById("release-select"); if (sel) sel.disabled = false; const installBtn = document.getElementById("btn-install"); @@ -215,6 +410,11 @@ const backend = selectedBackendId(); const installedBackend = installedBackendIdFromStatus(I.status.getLatestStatus()); pendingInstallBackendId = backend && backend !== installedBackend ? backend : null; + if (isSystemBackend(backend)) { + showSystemBackendControls(); + syncInstallActionButtons(I.status.getLatestStatus(), backend); + return; + } if (isCustomBackend(backend)) { showCustomBackendControls(); syncInstallActionButtons(I.status.getLatestStatus(), backend); @@ -270,6 +470,42 @@ } } + async function activateSystemBackend() { + if (systemActivationInProgress) return; + systemActivationInProgress = true; + const backend = selectedBackendId(); + const label = backendLabelFromStatus(I.status.getLatestStatus(), backend); + I.install.setInstallButtonsDisabled(true); + I.install.showStatus("info", `Checking ${label} tools on PATH...`); + try { + const result = await I.dependencies.fetchJson("/api/activate-system", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({}), + }); + if (result.ok) { + const foundList = (result.found || []).join(", "); + const missingList = (result.missing || []).join(", "); + const serverPath = result.server_path || (result.paths && result.paths["llama-server"]) || ""; + let msg = `${label} activated. Server: ${serverPath || "found on PATH"}.`; + if (foundList) msg += ` Found: ${foundList}.`; + if (missingList) msg += ` Unavailable: ${missingList}.`; + I.install.showStatus("success", msg); + await I.status.checkStatus(); + } else if (result.error) { + I.install.showStatus("error", result.error); + } else { + I.install.showStatus("error", `Failed to activate ${label}.`); + } + } catch (e) { + I.install.showStatus("error", `Failed to activate ${label}: ${e.message}`); + } finally { + systemActivationInProgress = false; + I.install.setInstallButtonsDisabled(false); + syncInstallActionButtons(I.status.getLatestStatus(), selectedBackendId()); + } + } + async function activateOfficialBackend(backend) { I.install.setInstallButtonsDisabled(true); I.install.showStatus("info", `Activating existing ${backend} backend...`); @@ -312,17 +548,21 @@ installBtn.disabled = !status.available_backends || status.available_backends.length === 0; const activeBackend = backendSelect ? backendSelect.value || "" : ""; - if (isCustomBackend(activeBackend, status)) { + if (isSystemBackend(activeBackend, status)) { + showSystemBackendControls(activeBackend, status); + } else if (isCustomBackend(activeBackend, status)) { showCustomBackendControls(activeBackend, status); } else { showOfficialBackendControls(); } syncInstallActionButtons(status, activeBackend); - if (customActivationInProgress) I.install.setInstallButtonsDisabled(true); + if (customActivationInProgress || systemActivationInProgress) I.install.setInstallButtonsDisabled(true); if (backendSelect) { const targetBackend = activeBackend; - I.install.ensureReleasesForBackend(targetBackend); + if (!isCustomBackend(targetBackend, status) && !isSystemBackend(targetBackend, status)) { + I.install.ensureReleasesForBackend(targetBackend); + } } if ((status.installed || status.config_stale) && status.tag && releaseSelect) { @@ -333,9 +573,16 @@ } if (status.installed) { - badge.textContent = isCustomBackend(status.backend, status) - ? backendLabelFromStatus(status, status.backend) - : status.version + " (" + status.backend + ")"; + if (isSystemBackend(status.backend, status)) { + const serverTag = systemBuildTagForTool(status, "llama-server"); + badge.textContent = serverTag + ? `${backendLabelFromStatus(status, status.backend)} ${serverTag}` + : `${backendLabelFromStatus(status, status.backend)} (unknown build)`; + } else { + badge.textContent = isCustomBackend(status.backend, status) + ? backendLabelFromStatus(status, status.backend) + : status.version + " (" + status.backend + ")"; + } badge.className = "badge badge-green"; } else if (status.config_stale) { badge.textContent = "Install Incomplete"; @@ -350,6 +597,8 @@ status.installed, status.config_stale, status.version, status.backend, status.executables, status.runtime_files, status.runtime_files_label, status.missing_runtime_files, status.platform, status.platform_label, status.arch, status.available_backends, + status.system_tools, status.system_server_error, + status.runtime_health && status.runtime_health.build_tags, ]); if (installedInfoRenderKey === lastInstalledInfoRenderKey) return; lastInstalledInfoRenderKey = installedInfoRenderKey; @@ -366,8 +615,11 @@ }; if (status.installed) { - appendRow("Version", String(status.version)); - appendRow("Backend", backendLabelFromStatus(status, status.backend)); + if (isSystemBackend(status.backend, status)) { + renderInstalledSystemStatus(status, appendRow, info, optionalToolsOpen); + } else { + appendRow("Version", String(status.version)); + appendRow("Backend", backendLabelFromStatus(status, status.backend)); if (isCustomBackend(status.backend, status)) { appendRow("Folder", customBackendFolder(status.backend, status)); } @@ -412,7 +664,11 @@ if (status.runtime_files && status.runtime_files.length > 0) { appendRow(status.runtime_files_label || "Runtime libraries", `${status.runtime_files.length} file(s)`); } + } } else if (status.config_stale) { + if (isSystemBackend(status.backend, status)) { + renderStaleSystemStatus(status, appendRow, info); + } else { const missingRuntimeFiles = Array.isArray(status.missing_runtime_files) ? status.missing_runtime_files.filter(Boolean) : []; @@ -446,6 +702,7 @@ if (isCustomBackend(status.backend, status)) { appendRow("Folder", customBackendFolder(status.backend, status)); } + } } else { const empty = document.createElement("div"); empty.className = "empty-state"; @@ -468,10 +725,12 @@ I.backends = { isCustomBackend, + isSystemBackend, canActivateOfficialBackend, selectedBackendId, onBackendChange, activateCustomBackend, + activateSystemBackend, activateOfficialBackend, updateStatusUI, }; diff --git a/ui/js/manager/manager-install.js b/ui/js/manager/manager-install.js index c9b1ccf..072b21b 100644 --- a/ui/js/manager/manager-install.js +++ b/ui/js/manager/manager-install.js @@ -15,6 +15,17 @@ const INSTALL_POLL_TIMEOUT_MS = 10 * 60 * 1000; const INSTALL_POLL_MAX_FAILS = 5; + function isSystemSelected(backend) { + if (I.backends && typeof I.backends.isSystemBackend === "function") { + try { + return I.backends.isSystemBackend(backend); + } catch (e) { + console.debug("System backend check failed", e); + } + } + return String(backend || "").trim() === "system"; + } + function refreshBackends(force = false) { if (backendRefreshPromise) { if (force) backendRefreshQueued = true; @@ -31,7 +42,7 @@ const result = await I.dependencies.fetchJson(force ? "/api/backends?refresh=1" : "/api/backends"); await I.status.checkStatus(); const backend = I.backends.selectedBackendId(); - if (force && !I.backends.isCustomBackend(backend)) await fetchReleases(backend); + if (force && !I.backends.isCustomBackend(backend) && !isSystemSelected(backend)) await fetchReleases(backend); if (hint) hint.textContent = result.warning || "Official CUDA/ROCm versions checked. Choose a backend to switch versions."; } catch (error) { console.warn("Failed to refresh official backend versions", error); @@ -39,7 +50,7 @@ // The discovery service is optional; release refresh still works // for built-in and third-party backends if it is unavailable. const backend = I.backends.selectedBackendId(); - if (force && !I.backends.isCustomBackend(backend)) await fetchReleases(backend); + if (force && !I.backends.isCustomBackend(backend) && !isSystemSelected(backend)) await fetchReleases(backend); } // A manual request during an automatic check gets one forced pass. // Requests during an already-forced pass share that pass. @@ -59,6 +70,21 @@ const sel = document.getElementById("release-select"); if (!sel) return; const backendParam = typeof backend === "string" ? backend.trim() : ""; + if (backendParam && I.backends && typeof I.backends.isSystemBackend === "function" + && I.backends.isSystemBackend(backendParam)) { + releaseFetchRequestId += 1; + cachedReleases = []; + releasesBackend = backendParam; + releasesBackendInFlight = null; + sel.innerHTML = ""; + const opt = document.createElement("option"); + opt.value = backendParam; + opt.textContent = "Managed by the OS package manager"; + sel.appendChild(opt); + sel.value = backendParam; + sel.disabled = true; + return []; + } const url = backendParam ? `/api/releases?backend=${encodeURIComponent(backendParam)}` : "/api/releases"; @@ -102,6 +128,9 @@ if (backendEl && I.backends.isCustomBackend(backendEl.value)) { return I.backends.activateCustomBackend(); } + if (backendEl && isSystemSelected(backendEl.value)) { + return I.backends.activateSystemBackend(); + } const backend = backendEl ? backendEl.value : ""; if (I.backends.canActivateOfficialBackend(I.status.getLatestStatus(), backend)) { return I.backends.activateOfficialBackend(backend); @@ -121,6 +150,10 @@ showStatus("error", "No saved installation config found to repair."); return; } + if (isSystemSelected(status.backend)) { + showStatus("error", "System (PATH) tools are managed by the OS package manager; Repair is not available."); + return; + } const ok = await I.dependencies.confirmAction( "Repair Install", @@ -143,9 +176,12 @@ return; } + const systemActive = status && isSystemSelected(status.backend); const ok = await I.dependencies.confirmAction( "Remove llama.cpp Files", - "Delete all files under llama/bin, llama/dll, and llama/grammars, and clear official install metadata? Both Custom slots, models, and presets will be kept.", + systemActive + ? "Delete all files under llama/bin, llama/dll, and llama/grammars, and clear official install metadata? The System (PATH) selection, both Custom slots, models, and presets will be kept." + : "Delete all files under llama/bin, llama/dll, and llama/grammars, and clear official install metadata? Both Custom slots, models, and presets will be kept.", "Remove" ); if (!ok) return; @@ -196,6 +232,11 @@ } async function checkForUpdates() { + const status = I.status.getLatestStatus(); + if (status && isSystemSelected(status.backend)) { + showStatus("error", "System (PATH) tools are updated by the OS package manager."); + return; + } showStatus("info", "Checking for updates..."); try { const result = await I.dependencies.fetchJson("/api/update", { method: "POST" }); @@ -314,7 +355,10 @@ body: JSON.stringify({ folder }), }) .then(() => { - const label = folder === "llama" ? "llama.cpp" : "models"; + const status = I.status.getLatestStatus(); + const label = folder === "llama" + ? (status && isSystemSelected(status.backend) ? "System tool" : "llama.cpp") + : "models"; showStatus("info", `Opened ${label} folder.`); }) .catch((e) => { diff --git a/ui/js/manager/manager-main.js b/ui/js/manager/manager-main.js index ff6f44c..104cb95 100644 --- a/ui/js/manager/manager-main.js +++ b/ui/js/manager/manager-main.js @@ -60,6 +60,7 @@ onBackendChange: I.backends.onBackendChange, updateStatusUI: I.backends.updateStatusUI, canActivateOfficialBackend: I.backends.canActivateOfficialBackend, + isSystemBackend: I.backends.isSystemBackend, installRelease: I.install.installRelease, waitForServerReady: I.lifecycle.waitForServerReady, applyModelDirInfo: I.modelDir.applyModelDirInfo, diff --git a/ui/js/manager/manager-status.js b/ui/js/manager/manager-status.js index f7d1138..bea1590 100644 --- a/ui/js/manager/manager-status.js +++ b/ui/js/manager/manager-status.js @@ -18,10 +18,17 @@ latestStatus = status; // Feeds the launch-arg gate for build-dependent flags such as native // --reasoning-effort (llama.cpp b10434+); custom backends and older - // installs stay on their compatible fallback path. + // installs stay on their compatible fallback path. System (PATH) + // reports per-tool tags because PATH can resolve CLI and server + // from different builds; the gate uses the tag for the tool whose + // arguments are being generated. Unknown formats stay conservative. if (window.LlamaGui.flagCore && typeof window.LlamaGui.flagCore.setBinaryTag === "function") { window.LlamaGui.flagCore.setBinaryTag(status.version); } + if (window.LlamaGui.flagCore && typeof window.LlamaGui.flagCore.setBinaryTags === "function") { + const buildTags = status && status.runtime_health && status.runtime_health.build_tags; + window.LlamaGui.flagCore.setBinaryTags(buildTags && typeof buildTags === "object" ? buildTags : {}); + } I.modelDir.applyModelDirInfo(status); I.backends.updateStatusUI(status); await notifyAcceptedStatusObserver(status); diff --git a/ui/js/quick-launch-ui.js b/ui/js/quick-launch-ui.js index 180e4a0..76005f7 100644 --- a/ui/js/quick-launch-ui.js +++ b/ui/js/quick-launch-ui.js @@ -503,8 +503,28 @@ } } + function getToolAvailability(status, tool) { + if (!status || !tool) return null; + const systemTools = status.system_tools; + if (systemTools && typeof systemTools === "object" && systemTools[tool]) { + return Boolean(systemTools[tool].available); + } + const exes = status.executables; + if (exes && typeof exes === "object") { + const suffix = typeof status.executable_suffix === "string" ? status.executable_suffix : ""; + return Boolean(exes[tool + suffix]); + } + return null; + } + function getQuickLaunchReadiness() { - if (getLatestStatus()?.installed === false) return { ok: false, type: "warning", message: "Install llama.cpp in Install and Update before launching." }; + const status = getLatestStatus ? getLatestStatus() : null; + if (status && status.installed === false) { + if (status.backend === "system" && status.system_server_error) { + return { ok: false, type: "warning", message: status.system_server_error }; + } + return { ok: false, type: "warning", message: "Install llama.cpp in Install and Update before launching." }; + } const result = flagCore.getLaunchArgs(); if (result.error) { return { ok: false, type: "error", message: result.error }; @@ -516,6 +536,31 @@ message: "Select a model or provide a remote model source before launching.", }; } + const tool = flagCore.getCurrentTool(); + const available = getToolAvailability(status, tool); + if (available === false) { + if (tool === "llama-cli") { + return { + ok: false, + type: "warning", + message: "llama-cli was not found on the PATH inherited by Llama GUI. Web / API server and Chat remain usable; install llama-cli with your system package manager for Terminal launches.", + }; + } + if (tool === "llama-server") { + return { + ok: false, + type: "warning", + message: status && status.system_server_error + ? status.system_server_error + : "llama-server was not found. Check Install and Update before launching.", + }; + } + return { + ok: false, + type: "warning", + message: `${tool} was not found. Check Install and Update before launching.`, + }; + } return { ok: true, type: "", message: "" }; } From 65337971777d24a87917b9a7ae963b62a5165fa7 Mon Sep 17 00:00:00 2001 From: Thomas Date: Fri, 9 Oct 2026 16:01:59 -0400 Subject: [PATCH 6/7] Document System PATH backend validation - Add representative System layout coverage for server-only, complete, symlink, wrapper, broken, and updated tools - Extend flag runner fixtures for PATH selection without repository fallback --- docs/directory.md | 58 ++++ docs/tests.md | 32 +- docs/troubleshooting.md | 28 ++ tests/backend/test_system_layouts.py | 327 ++++++++++++++++++ tests/frontend/llama_flags_runner_unit.cjs | 18 + tests/frontend/llama_flags_supported_unit.cjs | 16 + 6 files changed, 477 insertions(+), 2 deletions(-) create mode 100644 tests/backend/test_system_layouts.py diff --git a/docs/directory.md b/docs/directory.md index dda2147..d4c488b 100644 --- a/docs/directory.md +++ b/docs/directory.md @@ -1049,6 +1049,64 @@ Both slots retain `tag` / `version: "custom"` for existing flag-compatibility be **Open llama.cpp** opens the active Custom slot's root (containing `bin` and `grammars`), or the main `llama/` directory for official builds. Official update and repair paths exclude both slots. **Remove llama.cpp Files** only removes official runtime directories and metadata; it preserves both custom directories and an active custom selection. The GitHub release importer remains deferred. +## System (PATH) Backend + +Install & Update also offers **System (PATH)**, which uses the llama.cpp tools +already on the GUI process's inherited `PATH` instead of a downloaded build. +The OS package manager owns installation and updates; Llama GUI only +discovers, launches, and stops its own child processes. This replaces the old +workaround of symlinking system executables into `llama/custom/bin/` and fits +NixOS packages and wrappers that supply their own runtime dependencies. + +Selection and discovery live in `backend/services/llama_manager.py`. +`build_backend_specs()` appends the `system` spec (no download asset) and +`GET /api/status` marks it with `system_path: true` in `available_backends`. +Each known tool resolves independently with `shutil.which()` on the +platform-aware filename: inherited `PATH` only (normal precedence, relative +entries included), made absolute without dereferencing symlinks so +Nix/store wrappers keep their entry point, with no shell and no fallback to +repository installations. Discovery accepts only known llama.cpp tools; +`llama-fit-params` is an internal memory-estimation target, never a launch +option. A missing System tool resolves to `None` explicitly. + +**Activate System** sends `POST /api/activate-system` with an empty body — +client-supplied paths are rejected. The route holds the install/launch +interlock and requires a bounded, model-free `--version` probe of the +discovered `llama-server` before saving config; only `llama-server` is +required, every other tool is an independently available feature. Success +persists just the `backend`/`tag`/`version: "system"` marker (merging with the +latest config so model-root and other settings survive) and preserves the +remembered official installation for the return path. Failed probing leaves +configuration unchanged, and discovered paths/build tags are never persisted — +restart rediscovers them from the new process's environment. + +Runtime validation executes the discovered entry point with a 5-second +`--version` probe using the inherited environment and shared window-hiding +flags. Exit 0 with an unrecognized format is a successful execution with no +build tag (conservative flag gates apply); only failure to execute fails +health. Results are cached under the runtime-health lock keyed by executable +path plus file identity, so a replaced package target invalidates stale +probes. `ldd`/`otool` never run against System entries. Launches `exec` the +re-resolved entry point with the inherited environment plus validated +per-launch `LLAMA_`/`GGML_` overrides; no repository binary/library directory +is prepended. Memory estimation validates `llama-fit-params` itself and a +missing/broken estimator never blocks server launches. + +`GET /api/status` reports `system_tools` (per-tool discovered path, +availability, build tag, probe state), `system_server_error` (`None` when +healthy), and server-only readiness — a missing CLI leaves server/Chat +usable. The selected `version: "system"` marker stays separate from +discovered build tags, and a removed/broken server reports stale with `PATH` +guidance without clearing the selection. Release listing returns empty for +System, install/update/activate-existing reject it before any download or +mutation, cleanup preserves an active System selection while clearing removed +official-install metadata, and Open Folder reveals the discovered server's +directory read-only. The frontend (`ui/js/manager/manager-backends.js`, +`ui/js/flag-core.js`) renders per-tool paths, feeds per-tool build tags into +the shared flag-compatibility gates (unknown builds show `(unknown build)`), +and adjusts Quick Launch and Benchmarking readiness per requested tool, with +backend launch validation remaining authoritative. + ## Auto-Update System ### How It Works diff --git a/docs/tests.md b/docs/tests.md index aef205d..a8281df 100644 --- a/docs/tests.md +++ b/docs/tests.md @@ -181,6 +181,7 @@ Fast Node tests: - `manager_lifecycle_unit.cjs`: canonical Manager-package evaluation/configuration without DOM access, requests, or timers; dependency replacement after initialization; quit cancellation and sidebar wiring; restart/readiness/cache-busted reload; failure recovery for quit/restart controls; and successful app-update handoff to GUI lifecycle while retaining the app-update status renderer. Uses only injected API fixtures and recorded timers, with production test hooks disabled. - `shared_services_unit.cjs`: inert service loading; API cache defaults/overrides, method/body/header/signal forwarding, literal JSON null, invalid JSON and HTTP/network failures; confirmation/prompt keyboard, backdrop and cancel behavior, safe text assignment, focus placement, empty-versus-cancel results, and listener cleanup across repeated use. - `manager_releases_unit.cjs`: backend selection, backend-aware release fetching, `fetchJson` cache bypass, and installed-backend summary rendering. Custom slot cases cover backend-provided labels/paths, explicit activation payloads, duplicate/polling guards, failed activation retaining the current build, and update/repair restrictions. The suite uses the public Manager facade and gated rendering/action hooks, with injected confirmations rather than private-state mutation. It also checks inert configuration, repeated initialization, one action per click, and polling cleanup on install error/unload. The browser suite switches Custom → Custom 02 → official → Custom, including a missing-tool failure and unchanged shared flags. +- `system_path_frontend_unit.cjs`: System (PATH) Manager controls, shared build gates, and readiness. Covers selecting System (Activate System label, hidden release selection, setup text, no release fetch), activation routing with duplicate guards and PATH-guidance failures, disabled update/repair, official-target Activate Existing while System is active, pending-selection survival across polls, server-required versus feature-tool rendering with text-only paths, unknown-build display, stale-server guidance, per-tool flag-core tags with conservative unknown handling, server-only Quick Launch readiness with CLI-specific guidance, and missing-tool benchmark errors with per-tool load-mode translation. - `theme_ui_unit.cjs`: theme preference storage, `data-theme` root attribute application, unknown-theme normalization, and registry-driven color-scheme hints (asserted for every entry in `THEMES`, so a new theme with the wrong `scheme` fails here). Also covers the sidebar theme menu against a DOM stub: rendering one row per registry entry, `aria-checked`/roving `tabindex`, arrow-key wrapping, Home/End, Escape returning focus to the trigger, and outside-click dismissal. Asserts every `THEMES` entry has a matching palette block in `tokens.css`, so a theme cannot be offered in the menu while rendering as the fallback. Also enforces contrast floors for every theme, which is what makes adding a theme safe rather than merely cheap: - AA (4.5:1) for `--fg`, `--fg-muted`, the six semantic text colors (`--accent-text`, `--green`, `--red`, `--yellow`, `--favorite`, `--cyan`) and `--red-fg`. @@ -191,8 +192,8 @@ Fast Node tests: - `module_namespace_unit.cjs`: frontend script load order and exported namespaces, plus the Tier 2 Session 0 guardrail contracts described under [Common Commands](#common-commands). - `script_order.cjs`: shared test helper, not a suite. Parses active `