Skip to content

Latest commit

 

History

History
107 lines (81 loc) · 5.28 KB

README.md

File metadata and controls

107 lines (81 loc) · 5.28 KB

llama.cpp Instance Manager

A Windows installer and LAN dashboard for independent LLama-GUI installations.

Choose how many llama.cpp instances to install, assign available control-panel and inference ports, and optionally share a models folder. Each instance has its own upstream checkout, runtime, environment, backend files, and updates.

The manager provides saved-preset startup, local and LAN start/stop/restart, automatic LAN discovery, and a combined inference monitor. Detailed configuration, backend/model installation, and chat remain in each original instance.

Install on Windows

Download the .exe from Releases. Run setup, then follow the first-run wizard to name your server, select an instance count, choose folders, and review ports. Setup includes portable Python, Git, and Node. Internet access is used to download upstream instances and their dependencies.

The manager opens in its own Windows window. Closing the window keeps it running in the system tray; click the tray icon or launch the shortcut again to reopen it. The tray menu provides Open manager, Start at Windows login, and Quit manager. Quitting the manager leaves running instances alone.

The manager starts quietly in the tray at Windows login by default. Disable this in Settings or the tray menu if desired. The dashboard remains available at the manager's HTTP address for LAN browsers. Each Open instance link opens the original LLama-GUI in your browser.

Setup installs Microsoft's WebView2 Runtime if it is missing. Desktop themes and menu preferences are saved separately from your regular browser's preferences.

Windows Firewall setup is optional and asks Windows for approval to allow the selected TCP ports and discovery UDP 5238 on private networks.

  1. Choose Open instance to select a backend and model in the original GUI.
  2. Save a full server preset in that instance.
  3. Open Startup & installation → Startup settings in the manager, choose the saved preset, and optionally enable auto-start.

Start starts the control panel and selected saved model/server preset. Stop stops both. Restart stops and starts both using the selected preset. Open panel only is available while configuring an instance. Closing the manager browser tab does not stop the manager or its instances. Closing its desktop window has the same effect and leaves the tray icon available.

Control panels bind to 0.0.0.0. New instances include a normal Manager defaults preset containing their assigned inference port; Open instance loads that preset until another startup preset is selected. Saved presets retain their own ports. If a preset's port is occupied, the manager reports the conflict instead of changing it.

Use LAN servers to discover and connect other running managers, or enter their address manually. Connected servers expose the same lifecycle controls and loaded preset/model display. Detailed controls open in the original remote instance. Connections use plain HTTP without authentication or encryption.

The Monitor adds per-instance values directly for its combined and per-server cards. Reset session counters resets the displayed baselines, without interrupting inference. Monitor card order/visibility, themes, and navigation preferences are saved in the browser.

Updates and removal

Settings → Check for updates links to this fork's latest Windows installer. Run it to update the manager. Each independent LLama-GUI instance still updates through its original upstream installer and app-update controls.

Manager settings are stored under %LOCALAPPDATA%\llama.cpp Instance Manager. Instances default to its instances subfolder but can be installed elsewhere. Removing an instance from the manager leaves its files on disk. Uninstalling the manager preserves its settings, instances, and models. Each instance directory includes an Open instance.cmd launcher alongside its upstream checkout.

Build the Windows installer

.venv\Scripts\python -m pip install pyinstaller==6.22.3
.\scripts\build.ps1

The build requires Inno Setup 6 (-Iscc <path> overrides its location), verifies the pinned runtime checksums, and produces dist/*-setup.exe plus SHA256SUMS.txt. GitHub Actions builds and publishes tagged v* releases.

Development

python -m venv .venv
.venv\Scripts\python -m pip install -r requirements.txt
.venv\Scripts\python run_manager.py

On Windows, normal launches open the desktop window and --background starts in the tray. Use --headless to run only the HTTP server for automated tests or browser-only development. Linux source runs retain browser access without desktop dependencies. --quit --data-dir <path> waits for that manager to exit, preserving its instances.

Source runs also need Git and Node on PATH. Windows releases bundle their runtimes. See architecture and verification.

Attribution

Based on the menus and monitoring of Thomas9120's LLama-GUI. Theme tokens, inference-stat calculations, and logo assets are reused under GPL-3.0. The original application's source remains available in this repository's history. This manager is also GPL-3.0; see LICENSE.