diff --git a/docs/README.md b/docs/README.md index df70dcc5b..695271779 100644 --- a/docs/README.md +++ b/docs/README.md @@ -2,8 +2,8 @@ Stability Matrix is a multi-platform package manager for Stable Diffusion and related AI image/video generation tools. This documentation covers all major features and sections of the application. -This docuemtation is intended to provide a detailed guide and explaination of the many functions of Stability Matrix, its installation and use for both new and current users, and also more detailed and technical material for advanced users. -While it contains information on a vast majority of application specific functions, It also contains information that applies to AI image, video, and related generation aspects that can be useful both inside and outside of Stability Matrix. +This documentation is intended to provide a detailed guide and explanation of the many functions of Stability Matrix, its installation and use for both new and current users, and also more detailed and technical material for advanced users. +While it contains information on a vast majority of application specific functions, it also contains information that applies to AI image, video, and related generation aspects that can be useful both inside and outside of Stability Matrix. While not all encompassing on every minute detail, it is intended to be updated as new features and changes are released to the project as well as new ecosystem/model/usage information as-needed. Current Status: In-progress - Structure is in-place and planned docs are currently being progressively created and added. @@ -20,57 +20,61 @@ Current Status: In-progress - Structure is in-place and planned docs are current - [Overview](package-manager/overview.md) — Managing AI packages in Stability Matrix - [Supported Packages](package-manager/supported-packages.md) — Full list of supported inference and training packages - [Installing Packages](package-manager/installing-packages.md) — One-click install, hardware selection, GPU backends -- [Managing Packages](package-manager/managing-packages.md) — Launching, monitoring, updating, and deleting installed packages -- [Launch Arguments](package-manager/launch-arguments.md) — Configuring launch arguments per package -- [Extensions](package-manager/extensions.md) — Browsing and managing package plugins and extensions +- Managing Packages *(planned)* — Launching, monitoring, updating, and deleting installed packages +- Launch Arguments *(planned)* — Configuring launch arguments per package +- Extensions *(planned)* — Browsing and managing package plugins and extensions ### Inference - [Overview](inference/overview.md) — The Inference UI, panel layout, and project files -- [Text to Image](inference/text-to-image.md) — Generating images from text prompts -- [Image to Image](inference/image-to-image.md) — Using an image as a generation starting point -- [Image Upscale](inference/image-upscale.md) — Upscaling images with AI upscaler models -- [Video Generation](inference/video-generation.md) — Generating video with WAN and SVD models -- [Advanced Controls](inference/advanced-controls.md) — ControlNet, FaceDetailer, FreeU, LayerDiffuse, and more +- Text to Image *(planned)* — Generating images from text prompts +- Image to Image *(planned)* — Using an image as a generation starting point +- Image Upscale *(planned)* — Upscaling images with AI upscaler models +- Video Generation *(planned)* — Generating video with WAN and SVD models +- Advanced Controls *(planned)* — ControlNet, FaceDetailer, FreeU, LayerDiffuse, and more ### Checkpoint Manager -- [Overview](checkpoint-manager/overview.md) — Centralized model storage shared across all packages -- [Model Categories](checkpoint-manager/model-categories.md) — All supported model folder types explained -- [Metadata Editing](checkpoint-manager/metadata-editing.md) — Importing CivitAI metadata and editing model info +- Overview *(planned)* — Centralized model storage shared across all packages +- Model Categories *(planned)* — All supported model folder types explained +- Metadata Editing *(planned)* — Importing CivitAI metadata and editing model info ### Model Browser -- [Overview](model-browser/overview.md) — Multi-source model browser and download queue -- [CivitAI](model-browser/civitai.md) — Browsing and downloading from CivitAI -- [HuggingFace](model-browser/huggingface.md) — Browsing and downloading from HuggingFace -- [OpenModelDB](model-browser/openmodeldb.md) — Browsing upscaler models from OpenModelDB +- Overview *(planned)* — Multi-source model browser and download queue +- CivitAI *(planned)* — Browsing and downloading from CivitAI +- HuggingFace *(planned)* — Browsing and downloading from HuggingFace +- OpenModelDB *(planned)* — Browsing upscaler models from OpenModelDB ### Outputs -- [Overview](outputs/overview.md) — Image gallery, sorting, filtering, and batch operations -- [Image Metadata](outputs/image-metadata.md) — Reading embedded generation parameters and ComfyUI node data +- Overview *(planned)* — Image gallery, sorting, filtering, and batch operations +- Image Metadata *(planned)* — Reading embedded generation parameters and ComfyUI node data ### Workflows -- [Overview](workflows/overview.md) — Browsing and managing ComfyUI workflows -- [Community Workflows](workflows/community-workflows.md) — Browsing community workflows via OpenArt +- Overview *(planned)* — Browsing and managing ComfyUI workflows +- Community Workflows *(planned)* — Browsing community workflows via OpenArt ### Settings -- [Overview](settings/overview.md) — Navigating the settings hub -- [General](settings/general.md) — Theme, language, data directory, and shared folder settings -- [Accounts](settings/accounts.md) — Lykos account, OAuth login, and API tokens -- [Inference Settings](settings/inference-settings.md) — Inference UI behavior and defaults -- [Updates](settings/updates.md) — Auto-update channel and frequency settings +- Overview *(planned)* — Navigating the settings hub +- General *(planned)* — Theme, language, data directory, and shared folder settings +- Accounts *(planned)* — Lykos account, OAuth login, and API tokens +- Inference Settings *(planned)* — Inference UI behavior and defaults +- Updates *(planned)* — Auto-update channel and frequency settings ### Advanced -- [Building from Source and Contributing](advanced/building-from-source.md) — Local builds, runtime targets, and where to start for code or docs contributions -- [Shared Folders](advanced/shared-folders.md) — Folder structure, symlinks, and cross-package model sharing +- [Overview](advanced/overview.md) — Advanced configuration and technical reference +- Building from Source and Contributing *(planned)* — Local builds, runtime targets, and where to start for code or docs contributions +- Shared Folders *(planned)* — Folder structure, symlinks, and cross-package model sharing - [Hardware Support](advanced/hardware-support.md) — CUDA, ROCm, DirectML, MPS, ZLUDA, IPEX, and CPU backends -- [Python Environment](advanced/python-environment.md) — Virtual environments, uv, pip, and Python version management +- Python Environment *(planned)* — Virtual environments, uv, pip, and Python version management - [ComfyUI Integration](advanced/comfyui-integration.md) — ComfyUI node API, WebSocket protocol, and custom nodes - [Environment Variables](advanced/environment-variables.md) — Per-package environment variable configuration ### Tips and Tricks - [Overview](tips/overview.md) — Tips and Tricks index - [Terminology](tips/terminology.md) — Common image generation terms and what they mean -- [Inference UI Tips](tips/inference-ui.md) — Effective use of the built-in Inference UI -- [Per-Package Tips](tips/per-package.md) — Package-specific tips and links to upstream documentation -- [AMD GPU Workflow](tips/amd-gpu-workflow.md) — Getting image and video generation working on AMD hardware -- [Model Dependencies](tips/model-dependencies.md) — Required secondary files for modern models (text encoders, VAEs, etc.) -- [VRAM Optimization](tips/vram-optimization.md) — Reducing VRAM usage without sacrificing too much quality or speed +- Inference UI Tips *(planned)* — Effective use of the built-in Inference UI +- Per-Package Tips *(planned)* — Package-specific tips and links to upstream documentation +- AMD GPU Workflow *(planned)* — Getting image and video generation working on AMD hardware +- Model Dependencies *(planned)* — Required secondary files for modern models (text encoders, VAEs, etc.) +- VRAM Optimization *(planned)* — Reducing VRAM usage without sacrificing too much quality or speed + +### Troubleshooting +- [Common Issues](troubleshooting/common-issues.md) — Symptom-first fixes for install, launch, GPU, and Inference problems diff --git a/docs/advanced/environment-variables.md b/docs/advanced/environment-variables.md index 8f0e7ea02..2f8744bb6 100644 --- a/docs/advanced/environment-variables.md +++ b/docs/advanced/environment-variables.md @@ -100,7 +100,7 @@ For ordinary Stability Matrix usage, the most practical variables here are `PYTO ## HuggingFace Cache Variables -These variables are useful when a package downloads models, tokenizers, datasets, or other assets from the Hugging Face ecosystem. In Stability Matrix, the most common reasons to set them are moving caches off the system drive, forcing offline operation, or making Hub requests more reliable on slow connections. These are mainly to modify HuggingFace operations within Packages themselves (HF features built into WebUI's, HF download capable extensions/custom nodes) +These variables are useful when a package downloads models, tokenizers, datasets, or other assets from the Hugging Face ecosystem. In Stability Matrix, the most common reasons to set them are moving caches off the system drive, forcing offline operation, or making Hub requests more reliable on slow connections. These mainly modify HuggingFace operations within packages themselves, such as HF features built into WebUIs or HF-download-capable extensions and custom nodes. Because Stability Matrix injects environment variables globally, remember that authentication or offline-mode settings here will affect every launched package that uses `huggingface_hub`, `transformers`, `datasets`, or a library built on top of them. @@ -140,26 +140,32 @@ Most users should leave these alone unless they are troubleshooting a specific R | `MIOPEN_LOG_LEVEL` | `5` | Sets MIOpen log verbosity. Higher values provide more detailed internal logging and are useful when debugging solver selection, kernel compilation, or runtime failures. | | `MIOPEN_CHECK_NUMERICS` | `0x02` or `0x04` | Checks tensors for NaNs, infinities, and related numerical problems. This is useful when a ROCm workflow produces corrupted outputs or starts failing only on certain models or resolutions. | | `MIOPEN_GEMM_ENFORCE_BACKEND` | `5` | Overrides MIOpen's GEMM backend selection. This is an advanced tuning variable that can be useful when comparing rocBLAS and hipBLASLt behavior or isolating backend-specific regressions. | -| `COMFYUI_ENABLE_MIOPEN` | `1` | Tells ComfyUI to keep the MIOpen-backed path enabled on ROCm builds where it may otherwise be disabled by default. Without this enabled, ComfyUI disables the `cudnn` backend path in its backend calls for RDNA 3, RDNA 4, and newer AMD GPUs, which in turn disables the MIOpen-backed functions that rely on that path. This variable is needed for MIOpen to function properly in those setups. | +| `COMFYUI_ENABLE_MIOPEN` | `1` | Tells ComfyUI to keep the MIOpen-backed path enabled on ROCm builds where it may otherwise be disabled by default. Without this enabled, ComfyUI disables the `cudnn` backend path in its backend calls for RDNA3, RDNA3.5, and RDNA4 AMD GPUs, which in turn disables the MIOpen-backed functions that rely on that path. This variable is needed for MIOpen to function properly in those setups. | | `TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL` | `1` | Enables the experimental ROCm AOTriton path in compatible PyTorch builds. In Stability Matrix's Windows ROCm ComfyUI integration, this is used for TheRock technical-preview PyTorch builds to enable AOTriton-provided built-in Flash Attention and PyTorch SDPA memory-efficient attention paths. | For some Windows ROCm-based ComfyUI launches, Stability Matrix already applies several of these optimizations automatically in package code, including: `MIOPEN_FIND_MODE=2` -`MIOPEN_SEARCH_CUTOFF=2` +`MIOPEN_SEARCH_CUTOFF=1` -`TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` (RDNA3 and newer only) +`MIOPEN_FIND_ENFORCE=1` + +`TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1` (RDNA3 / RDNA3.5 / RDNA4 only, and additionally excluded on the gfx1152/gfx1153 APU architectures where AOTriton isn't yet supported) `FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE` -`COMFYUI_ENABLE_MIOPEN=1` (RDNA3 and newer only) +`COMFYUI_ENABLE_MIOPEN=1` (RDNA3 / RDNA3.5 / RDNA4 only, no gfx1152/gfx1153 exclusion) `PYTORCH_ALLOC_CONF=max_split_size_mb:512,garbage_collection_threshold:0.8` Linux installs do not currently get the same automatic overrides, so they will need to be enabled by the user. +If you're using the ComfyUI-Zluda package specifically, it also sets its own environment variables at launch on top of the above: `FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE`, `MIOPEN_FIND_MODE=2`, `MIOPEN_LOG_LEVEL=3`, and `ZLUDA_COMGR_LOG_LEVEL=1`. If you're wondering why those already appear to be set for a ZLUDA install, this is why. + +Whatever you set in Stability Matrix's own environment-variable editor is applied last, so it always overrides these auto-applied defaults if the same variable name is used. + For a broader reference, see the [official ROCm environment variable documentation](https://rocm.docs.amd.com/en/latest/reference/env-variables.html) and the [official MIOpen environment variable documentation](https://rocm.docs.amd.com/projects/MIOpen/en/latest/reference/env_variables.html). diff --git a/docs/advanced/hardware-support.md b/docs/advanced/hardware-support.md new file mode 100644 index 000000000..de17a74ce --- /dev/null +++ b/docs/advanced/hardware-support.md @@ -0,0 +1,140 @@ +# Hardware Support + +Stability Matrix runs image and video generation packages on top of PyTorch, and PyTorch needs a compute backend that matches your GPU. This page breaks down which GPUs and platforms each backend targets, what Stability Matrix does automatically when it detects your hardware, the known caveats, and which packages expose each backend. + +[`Section Overview`](overview.md) | [`Home`](../README.md) + +## Table of Contents + +- [How Backends Are Chosen](#how-backends-are-chosen) +- [NVIDIA (CUDA)](#nvidia-cuda) +- [AMD on Windows](#amd-on-windows) +- [AMD on Linux — AMD (ROCm)](#amd-rocm) +- [Intel (IPEX)](#intel-ipex) +- [Apple Silicon (MPS)](#apple-silicon-mps) +- [CPU Fallback](#cpu-fallback) +- [What's Next](#whats-next) + +--- + +## How Backends Are Chosen + +Each package declares the set of PyTorch backends it supports, and Stability Matrix pre-selects a recommended one from your detected hardware. The general order of preference is CUDA for NVIDIA, then ZLUDA (Windows AMD), then IPEX (Intel), then native ROCm (Linux AMD, or supported Windows AMD), then DirectML (Windows AMD), and finally CPU as a last resort. If a package does not support your detected GPU, the recommended default falls back to CPU. + +The backend is chosen at install time from the **PyTorch Index** dropdown, and can be changed afterward from the package's **Python Packages** dialog. See [Selecting a Hardware Backend](../package-manager/installing-packages.md#selecting-a-hardware-backend) for where these options live in the UI. + +The lists below describe what the code checks for. Because hardware detection works off GPU names and compute capability, treat any GPU model boundaries as guidance rather than a hard guarantee: some GPUs work with manual configuration even when a badge is not shown, and some edge-case cards may need extra setup. + +## NVIDIA (CUDA) + +- **Platforms:** Windows and Linux. +- **GPUs:** NVIDIA GPUs are detected by name (including Tesla-branded cards). Compute-capability thresholds the code reasons about include legacy GPUs (compute capability below 7.5, roughly pre-Turing), Ampere-or-newer (8.6 and up), and Blackwell (12.0 and up). +- **What Stability Matrix does automatically:** + - Installs PyTorch from the CUDA index. The current default is the CUDA 13.0 wheel index (`cu130`); GPUs flagged as legacy NVIDIA fall back to an older CUDA 12.6 index (`cu126`) for ComfyUI installs. + - For Windows systems with an Ampere-or-newer GPU, ComfyUI exposes an optional **Install Triton and SageAttention** command for faster attention. + - `xformers` is added on CUDA (and ZLUDA) installs when a package requests it. +- **Caveats:** + - The `cu130` wheels require an NVIDIA driver of version 580 or newer. ComfyUI checks the installed driver on launch and warns if it is older than 580.x while `cu130` torch is installed, suggesting either a driver update or manually downgrading to an older torch index such as `cu128`. + - Turing (RTX 2000-series) or newer is the practical recommendation; older cards may still work but are treated as legacy. +- **Packages:** CUDA is the most broadly supported backend. Every inference package that lists a GPU backend supports CUDA, and CUDA-only packages include Fooocus, SimpleSDXL, ForgeClassic, FramePack, and the training tools (Kohya's GUI, OneTrainer, FluxGym, AI Toolkit). + +## AMD on Windows + +AMD support on Windows is the most involved case, because there are three different paths depending on your GPU and package: native ROCm (technical preview), ZLUDA, and DirectML. + +### Native ROCm (TheRock technical preview) + +Stability Matrix can install AMD's native ROCm PyTorch on Windows using AMD's TheRock multi-architecture wheels. This path is gated to a specific set of GPU architectures. The code recognizes the following `gfx` architectures as supported on Windows: + +- **RDNA4** — `gfx120x` (e.g. RX 9070, RX 9060 families). +- **RDNA3 / RDNA3.5** — `gfx110x` (RDNA3 desktop and mobile) and `gfx115x` (RDNA3.5 APUs such as the 890M / 8060S / Z2 Extreme families). +- **Older architectures** — `gfx101x` (RDNA1) and `gfx103x` (RDNA2), plus Vega/GCN5 (`gfx900`, `gfx906`). + +Architectures in the `gfx110x`, `gfx115x`, and `gfx120x` ranges are treated as "modern"; the rest are treated as "legacy" and use a more conservative attention path. + +**What Stability Matrix does automatically:** + +- When a supported AMD GPU is present on Windows, ROCm becomes the recommended backend for ROCm-capable packages (currently ComfyUI and Comfy-based flows via SwarmUI). +- Torch is installed from AMD's ROCm multi-arch index (`repo.amd.com/rocm/whl-multi-arch/`) as device-specific wheels (`torch[device-gfxNNNN]`). Vega parts (`gfx900` / `gfx906`) pull from the nightly multi-arch feed instead, since TheRock currently only publishes their device packages there. +- On modern architectures it applies a set of ROCm performance and attention environment variables at launch (MIOpen find-mode tuning, AOTriton experimental flash attention, `COMFYUI_ENABLE_MIOPEN`, and an allocator tuning string). AOTriton is excluded on the `gfx1152` / `gfx1153` APU architectures, which it does not yet support. Legacy architectures instead force a math SDP fallback. The full variable list and exactly which ones are auto-applied are documented in [Environment Variables](environment-variables.md#amd-and-rocm-variables). +- ComfyUI offers optional extra commands for supported AMD GPUs, including **Install Triton and SageAttention (ROCm)**, **Install Flash Attention (ROCm)** (legacy architectures), an **Install ROCm Development SDK** step, and an **Install bitsandbytes (ROCm)** step for Python 3.12 environments. + +**Caveats:** + +- Windows AMD ROCm is explicitly experimental. Stability Matrix prints a notice asking you to report issues to Stability Matrix first, since the setup may not be officially supported by the upstream package developers. +- Only the architectures listed above are eligible. If your AMD GPU is not on the list, the recommended default becomes ZLUDA or DirectML instead. + +### ZLUDA + +ZLUDA is a CUDA-to-AMD translation layer used by dedicated AMD-on-Windows packages. It is recommended on Windows AMD systems that are not covered by native ROCm. + +- **Packages:** ComfyUI-Zluda, Stable Diffusion WebUI AMDGPU Forge, and SD.Next (which lists ZLUDA among its backends). +- **What Stability Matrix does automatically:** + - Installs the ZLUDA runtime along with the required HIP SDK prerequisite (HIP SDK 6.4) and, for ComfyUI-Zluda, Visual Studio Build Tools for C++. Torch itself is installed from the CUDA index, since ZLUDA translates CUDA calls. + - ComfyUI-Zluda sets its own launch-time environment variables (`FLASH_ATTENTION_TRITON_AMD_ENABLE`, `MIOPEN_FIND_MODE`, `MIOPEN_LOG_LEVEL`, `ZLUDA_COMGR_LOG_LEVEL`, and a `TRITON_OVERRIDE_ARCH` derived from your GPU's `gfx` arch). See [Environment Variables](environment-variables.md#amd-and-rocm-variables) for details. +- **Caveats:** + - Installing the HIP SDK and Build Tools may require administrator privileges and a reboot. + - AMD GPUs below the RX 6800 may require additional manual setup (both ComfyUI-Zluda and AMDGPU Forge carry this disclaimer). + - ZLUDA is generally faster than DirectML for supported operations but remains an experimental translation layer. + +### DirectML + +DirectML is Microsoft's cross-vendor GPU acceleration API and acts as the broadest-compatibility fallback on Windows. + +- **GPUs:** AMD, Intel, and some NVIDIA GPUs on Windows. +- **What Stability Matrix does automatically:** Installs the `torch-directml` package instead of a CUDA/ROCm torch build. On a Windows AMD system with no ROCm-supported GPU, DirectML/ZLUDA is the fallback recommendation. +- **Caveats:** Broad compatibility, but generally slower than CUDA or native ROCm, and upstream DirectML development has largely stagnated. Where possible, native ROCm or ZLUDA is preferable for AMD GPUs. +- **Packages:** ComfyUI, SD.Next, SwarmUI, SDFX, Stable Diffusion WebUI DirectML, and Fooocus-MRE list DirectML support. + +## AMD (ROCm) + +On Linux, AMD GPUs use native ROCm directly, which is the mature AMD path. + +- **Platform:** Linux only. +- **GPUs:** Native ROCm-capable AMD GPUs. Stability Matrix recommends ROCm when the system has an AMD GPU, no NVIDIA GPU, and is running Linux. +- **What Stability Matrix does automatically:** + - Installs PyTorch from a ROCm wheel index. The default torch index is ROCm 6.4 (`rocm6.4`), and ComfyUI installs use a ROCm 7.2 index (`rocm7.2`). + - Selects ROCm as the recommended backend automatically for ROCm-capable packages. +- **Caveats:** + - Native ROCm on Linux depends on a system-level ROCm installation and a compatible kernel/driver stack, which Stability Matrix does not install for you. + - The Windows-only ROCm performance environment overrides described above are not auto-applied on Linux, so if you want them you can set them yourself via the [Environment Variables](environment-variables.md) editor. +- **Packages:** ComfyUI, Stable Diffusion WebUI, SD.Next, Stable Diffusion WebUI Forge, InvokeAI, SwarmUI, SDFX, OneTrainer, and Wan2GP list ROCm support. + +## Intel (IPEX) + +IPEX is the Intel Extension for PyTorch, targeting Intel's discrete and integrated Arc graphics via the XPU backend. + +- **Platforms:** Windows and Linux. +- **GPUs:** Intel Arc graphics. Note that detection is name-based and matches GPUs whose name contains "Arc", so Arc A-series and B-series discrete cards and Core Ultra parts with integrated Arc graphics are the intended targets. +- **What Stability Matrix does automatically:** + - Installs PyTorch from Intel's XPU index (`xpu`). + - Recommends IPEX when an Intel Arc GPU is detected and the package supports it. + - For SD.Next, the Intel path runs the package's own `--use-ipex` install/launch flow. +- **Caveats:** Because detection keys off the "Arc" name, older non-Arc Intel integrated graphics are not recognized as IPEX-capable. +- **Packages:** ComfyUI and SD.Next list IPEX support. + +## Apple Silicon (MPS) + +MPS is Apple's Metal Performance Shaders backend, used for GPU acceleration on Apple Silicon Macs. + +- **Platform:** macOS on Apple Silicon (arm64). M1 and newer. +- **What Stability Matrix does automatically:** + - On macOS ARM, hardware compatibility always passes during first-launch setup, so the MPS path is offered without a discrete GPU check. + - MPS is included with PyTorch on macOS and needs no separate compute-library download; the torch install uses the CPU wheel index, and PyTorch provides the Metal-backed device at runtime. +- **Caveats:** Support is specific to Apple Silicon; Intel Macs are not covered by this path. As with any backend, individual model or node compatibility can still vary. +- **Packages:** ComfyUI, Stable Diffusion WebUI, SD.Next, Stable Diffusion WebUI Forge, InvokeAI, SwarmUI, and SDFX list MPS support. + +## CPU Fallback + +CPU is the universal fallback that runs entirely on the processor with no GPU acceleration. + +- **Platforms:** All. +- **What Stability Matrix does automatically:** Installs PyTorch from the CPU wheel index (`cpu`). When a package supports no backend that matches your detected hardware, the recommended default falls back to CPU. +- **Caveats:** CPU execution is dramatically slower than any GPU backend and is generally only suitable for testing, or for systems without a compatible GPU. +- **Packages:** Most inference WebUIs that support multiple backends include CPU (for example ComfyUI, Stable Diffusion WebUI, SD.Next, Stable Diffusion WebUI Forge, InvokeAI, SwarmUI, SDFX, and Fooocus-MRE). + +## What's Next + +- [Installing Packages](../package-manager/installing-packages.md) — Where the PyTorch backend is chosen during install +- [Environment Variables](environment-variables.md) — The full list of ROCm/HIP and PyTorch variables, and which ones Stability Matrix auto-applies +- [Supported Packages](../package-manager/supported-packages.md) — The full package list and their hardware badges diff --git a/docs/advanced/overview.md b/docs/advanced/overview.md new file mode 100644 index 000000000..6619adab0 --- /dev/null +++ b/docs/advanced/overview.md @@ -0,0 +1,18 @@ +# Overview + +The Advanced section covers the technical details behind how Stability Matrix works: how packages are built and run, how shared model storage and symlinks are organized, how hardware backends are selected, how the Python environment is managed, and how the app integrates with ComfyUI. These pages are aimed at users who want to understand or fine-tune the internals rather than just use the defaults. + +[`Home`](../README.md) + +--- + +Most users never need to touch these topics, since Stability Matrix handles environments, dependencies, and shared folders automatically. They are here for troubleshooting, customization, and for anyone building from source or contributing to the project. + +## In This Section + +- [Environment Variables](environment-variables.md) — Per-package environment variable configuration +- [ComfyUI Integration](comfyui-integration.md) — ComfyUI node API, WebSocket protocol, and custom nodes +- Building from Source and Contributing *(planned)* — Local builds, runtime targets, and where to start for code or docs contributions +- Shared Folders *(planned)* — Folder structure, symlinks, and cross-package model sharing +- [Hardware Support](hardware-support.md) — CUDA, ROCm, DirectML, MPS, ZLUDA, IPEX, and CPU backends +- Python Environment *(planned)* — Virtual environments, uv, pip, and Python version management diff --git a/docs/getting-started/data-directory.md b/docs/getting-started/data-directory.md new file mode 100644 index 000000000..7c95750a2 --- /dev/null +++ b/docs/getting-started/data-directory.md @@ -0,0 +1,75 @@ +# Data Directory + +The data directory (also called the library) is the single folder where Stability Matrix keeps everything it manages: installed packages, shared model storage, generated images, downloaded tools, and its own settings file. This page explains what lives in that folder, where it goes by default, and how to choose or relocate it. + +[`Section Overview`](overview.md) | [`Home`](../README.md) + +## Table of Contents + +- [What the Data Directory Is](#what-the-data-directory-is) +- [What Lives Inside It](#what-lives-inside-it) +- [Default Location](#default-location) +- [Portable Mode](#portable-mode) +- [Changing the Data Directory Later](#changing-the-data-directory-later) +- [A Note on Disk Space](#a-note-on-disk-space) +- [What's Next](#whats-next) + +--- + +## What the Data Directory Is + +Stability Matrix stores all of the data it manages under one root folder, referred to internally as the library. The location is chosen during [first launch](first-launch.md) and can be changed later. Whenever the app needs to install a package, share a model, or save an output, it works relative to this one directory. + +The data directory always contains a `settings.json` file at its root. When the **Select Data Directory** dialog validates a folder, it treats the folder as an existing Stability Matrix library if a readable `settings.json` is present, and otherwise accepts the folder only if it is empty. + +## What Lives Inside It + +Stability Matrix creates and manages several subfolders inside the data directory. The main ones are: + +- **`Packages/`** — Each installed package is cloned into its own subfolder here, named after its display name (for example `Packages/ComfyUI`). This includes the package's own files and its Python virtual environment. +- **`Models/`** — The shared model library. Rather than every package keeping its own copy of large files, models are stored once here and shared. This folder is organized into type-based subfolders such as `StableDiffusion` (checkpoints), `Lora`, `VAE`, `ControlNet`, `ESRGAN` (upscalers), `Embeddings`, and others. The location of the `Models` folder can be pointed elsewhere with a model directory override in settings. +- **`Images/`** — The shared outputs folder. When output sharing is enabled for a package, its generated images are saved here instead of inside the package folder. Inference UI outputs are kept under `Images/Inference`. +- **`Assets/`** — Portable tooling that Stability Matrix downloads and manages for you, such as the `uv` utility, bundled Python installations, 7-Zip, and (for packages that need it) Node.js. Keeping these here means no system-wide Python or Git install is required. +- **`Workflows/`** — Saved ComfyUI workflows managed through the app. +- **`Tags/`** — Tag autocomplete data used by the Inference UI. +- **`.downloads/`** — A working folder for in-progress downloads. + +The root also holds the `settings.json` configuration file. This layout is why the data directory can grow large, and why it is treated as a single portable unit. + +## Default Location + +The default location depends on your operating system and on whether Portable Mode is enabled. + +For a non-portable install, the default library path is: + +| Platform | Default location | +|---|---| +| Windows | `%AppData%\StabilityMatrix` (the Roaming AppData folder) | +| Linux | `StabilityMatrix` inside your home directory (`~/StabilityMatrix`) | +| macOS | The application data directory, which resolves to `~/.config/StabilityMatrix` | + +For non-portable installs, Stability Matrix records the chosen library path in a `library.json` file kept in its AppData home folder, and reads that on startup to find your data directory. + +## Portable Mode + +Portable Mode keeps the data directory next to the application instead of sending it to one of the default locations above. When Portable Mode is used, the library is a folder named `Data` alongside the Stability Matrix executable, marked by a `.sm-portable` file inside it. + +Portable Mode is enabled by default in the **Select Data Directory** dialog, and it is the recommended option for most users. Because the app and its `Data` folder stay bundled together, the whole setup is easier to move to another folder, drive, or computer later. On startup, Stability Matrix checks for the portable marker first, so a portable `Data` folder always takes precedence over any saved non-portable path. + +## Changing the Data Directory Later + +The data directory can be changed after setup from the app's settings, which reopens the same **Select Data Directory** dialog used during first launch. You can either pick a new custom folder or switch to Portable Mode. Applying the change requires restarting Stability Matrix. + +Changing the setting only updates where Stability Matrix *looks* for its library — it points the app at the new location (and creates that folder if needed) but does not move your existing packages, models, or images for you. If you want to keep your current data, move or copy the contents of the old data directory to the new location yourself before or after switching, then confirm the app finds a valid `settings.json` there. + +## A Note on Disk Space + +Choose the data directory location with disk space in mind. The bulk of the space usage comes from the `Packages` and `Models` folders. A single package install is commonly in the multi-gigabyte range once its PyTorch dependencies are downloaded, and individual model checkpoints are frequently several gigabytes to tens of gigabytes each. Over time a model library can easily reach hundreds of gigabytes. + +Prefer a drive with plenty of free space, and ideally a fast one. FAT32 and exFAT drives are not supported, so pick a drive formatted with a modern filesystem (such as NTFS on Windows); the **Select Data Directory** dialog shows a warning if the chosen drive uses a FAT format. Placing the library inside a synced cloud folder such as OneDrive is also discouraged, and the dialog shows a warning when it detects a OneDrive path. + +## What's Next + +- [First Launch](first-launch.md) — Where the data directory is first chosen +- [Installing Packages](../package-manager/installing-packages.md) — What gets written into `Packages/` +- Shared Folders *(planned)* — How the `Models/` library is shared across packages diff --git a/docs/getting-started/first-launch.md b/docs/getting-started/first-launch.md index e5f2678db..3c02b9c9d 100644 --- a/docs/getting-started/first-launch.md +++ b/docs/getting-started/first-launch.md @@ -68,5 +68,5 @@ Once the license agreement is accepted and the data directory is configured, Sta From there, the usual next steps are: - [Install your first package](../package-manager/installing-packages.md) -- [Browse or import models](../model-browser/overview.md) +- Browse or import models with the Model Browser *(planned)* - If the user installed ComfyUI and downloaded a starter model during setup, they can [go straight to generating with the built-in Inference UI](../inference/overview.md) diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 8018d9890..7652fef6b 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -36,7 +36,7 @@ Windows releases are distributed as a `.zip` archive rather than a traditional i 2. Extract the archive to a folder where Stability Matrix should live. 3. Open the extracted folder and run `StabilityMatrix.exe`. -The Microsoft Visual C++ Redistributable for x64 is required on Windows. On many systems it is already present, but if a package fails to start because the required Microsoft C/C++ runtime is missing (e.g. missing c10.dll error loading PyTorch), install the latest [Visual C++ Redistributable x64 package](https://aka.ms/vc14/vc_redist.x64.exe) or see Microsoft's [Visual C++ Redistributable downloads page](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170). +The Microsoft Visual C++ Redistributable for x64 is required on Windows. Stability Matrix checks for it automatically during package installation and silently installs the required version as part of the normal prerequisite setup, so most users never need to do anything here. If a package still fails to start because the required Microsoft C/C++ runtime is missing (e.g. missing c10.dll error loading PyTorch), that's a sign the automatic install didn't complete successfully — as a fallback, install the latest [Visual C++ Redistributable x64 package](https://aka.ms/vc14/vc_redist.x64.exe) manually, or see Microsoft's [Visual C++ Redistributable downloads page](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170). On first launch, Windows may show a SmartScreen warning because the app was downloaded from the internet. If that happens, select **More info** and then **Run anyway** to continue, provided the download came from the official Downloads page or the project's GitHub Releases page. @@ -53,7 +53,7 @@ Official macOS releases are published for Apple Silicon as a `.dmg`. If Gatekeeper blocks the first launch, open the app once with **Open** from the context menu, or allow it from **System Settings > Privacy & Security** if macOS shows an override prompt there. -For platform support details and hardware expectations on Apple Silicon, see [Apple Silicon (MPS)](../advanced/hardware-support.md#apple-silicon-mps). +Platform support details and hardware expectations on Apple Silicon (MPS) will be covered in a planned Hardware Support page. ## Linux diff --git a/docs/getting-started/overview.md b/docs/getting-started/overview.md index 1109326bc..6b45dff2b 100644 --- a/docs/getting-started/overview.md +++ b/docs/getting-started/overview.md @@ -46,22 +46,22 @@ Stability Matrix is cross-platform, but the exact release formats and hardware t | Linux | Modern x86-64 desktop distributions | x64 | Official Linux releases are published for `linux-x64`, primarily as an AppImage, with an AUR package also available for Arch-based systems. Depending on the distribution, AppImage/runtime support packages such as `libfuse2`, `libappimage`, or `libxcrypt-compat` may be needed if they are not already provided by the system. | | macOS | Apple Silicon Macs, with macOS 12.3 or later recommended for AI workflows | arm64 | Official macOS releases are published for Apple Silicon (`osx-arm64`) as a `.dmg`. The app's AI workflows rely on the MPS backend on Apple Silicon. | -In other words, the practical supported release targets are Windows x64, Linux x64, and Apple Silicon macOS. Some project files include additional runtime identifiers, but the documented source-build support and the release pipeline currently focus on `win-x64`, `linux-x64`, and `osx-arm64`. For work from a local checkout instead of a packaged release, see [Building from Source and Contributing](../advanced/building-from-source.md) for the documentation entry point and links to the repository's contributor guide. +In other words, the practical supported release targets are Windows x64, Linux x64, and Apple Silicon macOS. Some project files include additional runtime identifiers, but the documented source-build support and the release pipeline currently focus on `win-x64`, `linux-x64`, and `osx-arm64`. For work from a local checkout instead of a packaged release, a planned Building from Source and Contributing page will serve as the documentation entry point and link to the repository's contributor guide. ## System Requirements -Stability Matrix itself is distributed as a portable, self-contained desktop app, so separate installation of Python, Git, or packags is not usually required. In practice, the real hardware requirements come from the packages, models, and workflows a user wants to run. +Stability Matrix itself is distributed as a portable, self-contained desktop app, so separately installing Python, Git, or other packages is not usually required. In practice, the real hardware requirements come from the packages, models, and workflows a user wants to run. - **Operating system and architecture**: Use one of the supported desktop targets listed above: Windows x64, Linux x64, or Apple Silicon macOS. - **GPU**: A dedicated GPU is strongly recommended for image and video generation. NVIDIA CUDA is the broadest and most mature path in current Stability Matrix workflows, with 900-series cards as a practical minimum and 2000-series or newer recommended for better compatibility and speed. AMD ROCm, AMD ZLUDA, Intel Arc (IPEX), and Apple Silicon (MPS) are also supported depending on platform. -- **VRAM**: Around 4 GB of VRAM is a practical minimum for older and lighter image-generation setups (Stable Diffusion 1.5), but 12+ GB is a better minimal target for most current basic models and workflows (e.g. SDXL, Z-Image Turbo). Large modern models such as unquantized FLUX variants, and many video-generation workflows, can push that much higher. Lower-VRAM video variants may work in the 6-8 GB range, while larger video models can require 16+ GB. As a general recommendation, a 16 GB VRAM GPU is a comfortable target for most commonly used workflows. -- **System RAM**: 16GB recommended minimum. Requirements vary by backend and model size, but more system RAM becomes important when workloads spill out of VRAM. 32+ GB of RAM can help avoid hard out-of-memory crashes on constrained VRAM setups, even though performance will still slow down when offloading occurs. On memory-constrained systems, it also helps to make sure the page file on Windows or the swap file or swap partition on Linux is configured with enough space to act as a last-resort buffer when both VRAM and system RAM are exhausted. +- **VRAM**: These figures are rough community rules of thumb, not values enforced by the app, and they'll keep shifting as new models arrive. As of mid-2026, roughly 4 GB of VRAM is a practical minimum for older and lighter image-generation setups (Stable Diffusion 1.5), but roughly 12+ GB is a better minimal target for most current basic models and workflows (e.g. SDXL, Z-Image Turbo). Large modern models such as unquantized FLUX variants, and many video-generation workflows, can push that much higher. Lower-VRAM video variants may work in roughly the 6-8 GB range, while larger video models can require roughly 16+ GB. As a general recommendation, a 16 GB VRAM GPU is a comfortable target for most commonly used workflows today. +- **System RAM**: 16 GB recommended minimum. Requirements vary by backend and model size, but more system RAM becomes important when workloads spill out of VRAM. 32+ GB of RAM can help avoid hard out-of-memory crashes on constrained VRAM setups, even though performance will still slow down when offloading occurs. On memory-constrained systems, it also helps to make sure the page file on Windows or the swap file or swap partition on Linux is configured with enough space to act as a last-resort buffer when both VRAM and system RAM are exhausted. - **Storage**: Plan for significant disk usage in the data directory. A single package install is typically in the 2-10 GB range, checkpoint models are often 2-20 GB or more each, and LoRAs or other adapters commonly range from hundreds of megabytes to around 1 GB each. An SSD is recommended for packages and active workflows, while slower bulk storage (HDD) can still be reasonable for large model libraries at the cost of initial model loading speed. - **CPU-only fallback**: CPU-only operation is possible, but it is mainly useful for testing or very light use. For real generation workloads, it is much slower than any supported GPU backend. If the intended hardware target is unclear, the safest general recommendation is a supported OS, a modern dedicated GPU, at least enough VRAM for the intended model family, and a storage drive with plenty of free space for packages, models, and outputs. -For a deeper breakdown of supported GPU backends, platform-specific acceleration paths, and hardware caveats, see [Hardware Support](../advanced/hardware-support.md). +A deeper breakdown of supported GPU backends, platform-specific acceleration paths, and hardware caveats is planned for a future Hardware Support page in the Advanced section. ## What's Next diff --git a/docs/inference/overview.md b/docs/inference/overview.md index 8abd0e1f4..22417624b 100644 --- a/docs/inference/overview.md +++ b/docs/inference/overview.md @@ -2,7 +2,7 @@ The Inference page is Stability Matrix's built-in image and video generation interface, powered by ComfyUI under the hood. It provides a structured, panel-based UI as an alternative to using a web browser to control ComfyUI directly. -[`Section Overview`](overview.md) | [`Home`](../README.md) +[`Home`](../README.md) ## Table of Contents @@ -38,12 +38,14 @@ The Inference UI can reopen a previously saved project tab on startup, provided ## Generation Modes -- [Text to Image](text-to-image.md): Creates images from prompts without a required source image. This is the default mode and the main entry point for most image-generation workflows. -- [Image to Image](image-to-image.md): Uses an input image together with prompt and sampler settings to guide edits, restyling, or controlled variation. -- [Image Upscale](image-upscale.md): Starts from an existing image and applies upscale methods exposed by the connected backend, including latent and model-based upscalers when available. -- [Video Generation](video-generation.md): Covers the video-oriented tabs exposed in the UI, including Wan Text to Video, Wan Image to Video, and SVD-style image-to-video generation. +- **Text to Image** *(planned page)*: Creates images from prompts without a required source image. This is the default mode and the main entry point for most image-generation workflows. +- **Image to Image** *(planned page)*: Uses an input image together with prompt and sampler settings to guide edits, restyling, or controlled variation. +- **Image Upscale** *(planned page)*: Starts from an existing image and applies upscale methods exposed by the connected backend, including latent and model-based upscalers when available. +- **Wan Text to Video** *(planned page)*: Generates video from a text prompt using Wan video models. +- **Wan Image to Video** *(planned page)*: Generates video from a source image using Wan video models. +- **SVD Image to Video** *(planned page)*: Generates video from a source image using Stable Video Diffusion. -All of these modes are implemented as separate tab view models, which is why different tabs can expose different cards, input requirements, and prompt behavior while still sharing the same backend connection. +Video generation is not a single mode — it's split across three independent tabs, each its own project type, so Wan Text to Video, Wan Image to Video, and SVD Image to Video can be opened side by side with their own settings. All of these modes are implemented as separate tab view models, which is why different tabs can expose different cards, input requirements, and prompt behavior while still sharing the same backend connection. ## Panel Layout @@ -81,14 +83,14 @@ The Inference page supports standard project-style actions for these files, incl Generated images can also carry Stability Matrix project metadata. When a saved output includes embedded Stability Matrix project data, dropping that image back onto a compatible Inference tab can restore the serialized state directly from the image metadata. -`.smproj` files are distinct from ComfyUI workflow JSON files. Project files capture the state of Stability Matrix's native Inference tabs, while the [Workflows Browser](../workflows/overview.md) is for browsing and managing ComfyUI workflow files. +`.smproj` files are distinct from ComfyUI workflow JSON files. Project files capture the state of Stability Matrix's native Inference tabs, while the Workflows Browser *(planned page)* is for browsing and managing ComfyUI workflow files. ## Related Pages -- [Text to Image](text-to-image.md) -- [Image to Image](image-to-image.md) -- [Image Upscale](image-upscale.md) -- [Video Generation](video-generation.md) -- [Advanced Controls](advanced-controls.md) -- [Outputs Overview](../outputs/overview.md) -- [ComfyUI Intergration](../advanced/comfyui-integration.md) +- Text to Image *(planned)* +- Image to Image *(planned)* +- Image Upscale *(planned)* +- Video Generation *(planned)* +- Advanced Controls *(planned)* +- Outputs Overview *(planned)* +- [ComfyUI Integration](../advanced/comfyui-integration.md) diff --git a/docs/package-manager/installing-packages.md b/docs/package-manager/installing-packages.md index 094a978ca..94bf9f12a 100644 --- a/docs/package-manager/installing-packages.md +++ b/docs/package-manager/installing-packages.md @@ -1,6 +1,6 @@ # Installing Packages -This page walks through installing an WebUI package in Stability Matrix using the **Add Package** screen. +This page walks through installing a WebUI package in Stability Matrix using the **Add Package** screen. [`Section Overview`](overview.md) | [`Home`](../README.md) @@ -25,7 +25,7 @@ Packages are displayed as a scrollable list of cards organized into three tabs a - **Training**: Model fine-tuning and training tools such as Kohya's GUI and OneTrainer. - **Legacy**: Older packages that are maintained for existing users but not recommended for new installations. May be stale and no longer receiving updates. -Each package card shows the package name, author, a short description, and a row of **hardware compatibility badges** indicating which PyTorch backends the package supports from the following types CUDA (NVIDIA), ROCm (AMD-native), DirectML, macOS (MPS), ZLUDA (AMD), IPEX (Intel), or CPU. Note that the absence of a particular hardware badge does not necessarily mean the package is incompatible, some packages may still be usable with manual configuration or community-provided workarounds. Within each tab, beginner-friendly packages appear first, followed by advanced tools in alphabetical order. +Each package card shows the package name, author, a short description, and a row of **hardware compatibility badges** indicating which PyTorch backends the package supports from the following types: CUDA (NVIDIA), ROCm (AMD-native), DirectML, macOS (MPS), ZLUDA (AMD), IPEX (Intel), or CPU. Note that the absence of a particular hardware badge does not necessarily mean the package is incompatible; some packages may still be usable with manual configuration or community-provided workarounds. Within each tab, beginner-friendly packages appear first, followed by advanced tools in alphabetical order. Use the tabs to switch between package types, or type in the search bar to filter the list by name in real time. Incompatible packages are hidden by default: enable *Show All Packages* to see packages that do not officially support your current hardware (e.g., CUDA-only packages on an AMD system). @@ -48,7 +48,7 @@ The **Advanced Options** section is a collapsible panel containing settings that - **Model Sharing**: Controls how model directories are linked to the shared `Models/` library. Options include **Symlink** (recommended for most users), **Configuration** (uses the package's own config files to point to shared paths), and **None** (isolated model folders). - **PyTorch Index**: Choose the PyTorch compute backend for your GPU. See [Selecting a Hardware Backend](#selecting-a-hardware-backend). - **Output Sharing**: Enabled by default. When enabled, generated outputs are saved to the shared `Images/` directory rather than inside the package folder. -- **Python Version**: Select the Python version for the package's virtual environment from the versions available via Stability Matrix's internal `uv` utility. A green checkmark indicates versions already downloaded and cached locally. Typically it is recommended to leave set to what the package is configured in SM as default for compatibility/upstream recommendation and, recommended only to change if specifically needed before installing. +- **Python Version**: Select the Python version for the package's virtual environment from the versions available via Stability Matrix's internal `uv` utility. A green checkmark indicates versions already downloaded and cached locally. It's typically best to leave this at the default Stability Matrix configures for the package, since that reflects compatibility testing and upstream recommendations. Only change it before installing if a specific version is actually needed. ### Python Dependencies Override @@ -96,7 +96,7 @@ The **PyTorch backend** determines which GPU acceleration library your package u | Backend | Platform | GPU | Notes | |---------|----------|-----|-------| | **CUDA** | Windows, Linux | NVIDIA (GTX 900-series and newer) | Best performance and broadest compatibility. CUDA toolkit is bundled with PyTorch; no separate driver installation beyond standard NVIDIA drivers. Turing (RTX 2000-series) or newer recommended. | -| **ROCm** | Windows, Linux | AMD (select GPUs per platform) | Native AMD GPU acceleration. On Linux, requires system-level ROCm installation. On Windows, uses AMD's TheRock technical preview builds. See [Hardware Support](../advanced/hardware-support.md#amd-rocm) for per-chip compatibility. | +| **ROCm** | Windows, Linux | AMD (select GPUs per platform) | Native AMD GPU acceleration. On Linux, requires system-level ROCm installation. On Windows, uses AMD's TheRock technical preview builds. Per-chip compatibility details are planned for a future Hardware Support page. | | **DirectML** | Windows | AMD, Intel, some NVIDIA | Microsoft's DirectML API. Broad compatibility but slower performance than CUDA or ROCm. Development is largely stagnant; consider native ROCm, or ZLUDA if need be, as an alternative for AMD GPUs. | | **ZLUDA** | Windows | AMD (via CUDA translation layer) | Experimental CUDA-to-AMD translation layer. Used by the ComfyUI-Zluda, SD.Next, and AMDGPU Forge packages. Generally faster than DirectML for supported operations. | | **IPEX** | Windows, Linux | Intel Arc (discrete and integrated) | Intel Extension for PyTorch. Requires Intel Arc GPU (A-series, B-series) or modern Intel Core Ultra with integrated Arc graphics. | @@ -105,9 +105,9 @@ The **PyTorch backend** determines which GPU acceleration library your package u The pre-selected backend is determined by Default GPU selected at First-Launch or in Default GPU setting, along with internal recommended Torch checks Stability Matrix determines based on detected hardware. If a package does not support your detected GPU, the recommended default will fall back to CPU. -> **Note:** The PyTorch backend is selected at install time, but can be changed afterward via the **Python Packages** dialog — accessible from the package's three-dot menu on the Packages screen. See [Python Environment Management](../advanced/python-environment.md#viewing-installed-python-packages). +> **Note:** The PyTorch backend is selected at install time, but can be changed afterward via the **Python Packages** dialog — accessible from the package's three-dot menu on the Packages screen. A planned Python Environment Management page will cover this in more detail. -For in-depth platform-specific guidance, including driver requirements and known caveats, see [Hardware Support](../advanced/hardware-support.md). +In-depth platform-specific guidance, including driver requirements and known caveats, is planned for a future Hardware Support page. ## Installation Progress @@ -134,7 +134,7 @@ The progress dialog shows a real-time log of each step. If any step fails, the d | First install (no cached wheels) | 5–15 minutes | | Slow connection or CPU-only install | 10–25 minutes | -> **Note:** PyTorch wheels are large and the multiple needed WHL files needed can accumulate to several GB's or more in total download size depending on backend used. The first installation on a fresh system downloads these wheels. Subsequent installs reuse cached wheels, making them significantly faster. +> **Note:** PyTorch wheels are large and the multiple WHL files needed can accumulate to several GBs or more in total download size depending on the backend used. The first installation on a fresh system downloads these wheels. Subsequent installs reuse cached wheels, making them significantly faster. ## One-Click Install @@ -148,7 +148,7 @@ For new users, Stability Matrix offers a streamlined **one-click install** exper - The **latest release version** (or latest commit, for packages without releases). - The **recommended PyTorch backend** detected from your hardware. - The **recommended shared folder method** (symlinks for most packages). - - The **package's recommended default Python version** + - The **package's recommended default Python version**. 3. **Installation**: clicking Install runs the same step pipeline described in [Installation Progress](#installation-progress). A progress bar shows the current status, and status text updates as each step completes. @@ -164,4 +164,4 @@ The one-click install dialog is a first-launch experience only. Once dismissed o ## Next Steps -Once a package is installed, you can launch it, monitor its console output, configure launch arguments, run multiple packages simultaneously, update to newer versions, or remove it entirely. See [Managing Packages](managing-packages.md) for details on all of these workflows. +Once a package is installed, you can launch it, monitor its console output, configure launch arguments, run multiple packages simultaneously, update to newer versions, or remove it entirely. A planned Managing Packages page will cover all of these workflows in detail. diff --git a/docs/package-manager/overview.md b/docs/package-manager/overview.md new file mode 100644 index 000000000..a4aeb742a --- /dev/null +++ b/docs/package-manager/overview.md @@ -0,0 +1,22 @@ +# Overview + +The Package Manager is where Stability Matrix installs, updates, launches, and manages the AI generation packages you use, such as ComfyUI, Stable Diffusion WebUI variants, Forge, InvokeAI, and various training tools. This section of the documentation covers how those workflows work. + +[`Home`](../README.md) + +--- + +From the **Packages** screen you can install new packages through a guided flow, keep multiple packages side by side as isolated installations, launch them with live console output, update or roll back versions, and configure per-package options such as launch arguments, environment variables, shared model folders, and extensions. Each package keeps its own Python environment while sharing common resources like the model library, so the same checkpoints and LoRAs do not need to be duplicated for every tool. + +## In This Section + +- [Supported Packages](supported-packages.md) — Full list of supported inference and training packages +- [Installing Packages](installing-packages.md) — One-click install, hardware selection, and GPU backends +- Managing Packages *(planned)* — Launching, monitoring, updating, and deleting installed packages +- Launch Arguments *(planned)* — Configuring launch arguments per package +- Extensions *(planned)* — Browsing and managing package plugins and extensions + +## What's Next + +- [Installing Packages](installing-packages.md) — Install your first package +- [Supported Packages](supported-packages.md) — See what you can install diff --git a/docs/package-manager/supported-packages.md b/docs/package-manager/supported-packages.md index 99d2a6f24..04391d283 100644 --- a/docs/package-manager/supported-packages.md +++ b/docs/package-manager/supported-packages.md @@ -22,7 +22,7 @@ Inference packages are used for generating images and video. They provide their | **Stable Diffusion WebUI reForge** | A fast-moving Forge fork that tracks new functionality and newer model architectures quickly. Beyond Stable Diffusion, it supports a range of newer families such as FLUX, SD3, PixArt, Hunyuan, WAN video models, and other recent transformer-led pipelines. | | **Stable Diffusion WebUI Forge - Neo** | An NVIDIA-focused Forge fork in rapid development, aimed at newer functionality, current model architectures, and a streamlined high-performance workflow. | | **ComfyUI** | A powerful, node-based graph UI for building custom inference pipelines across a wide range of modern image and video models. It has grown into one of the most popular local AI frontends, and Stability Matrix's Inference UI is built to work alongside it through ComfyUI's API and workflow backend. | -| **ComfyUI-Zluda** | A Windows-only ComfyUI variant using ZLUDA as an alternative AMD path when ROCm is not the preferred option, including on some modern Radeon GPUs and older GPUs without practical ROCm support. Like standard ComfyUI, it remains compatible with Stability Matrix's Inference UI through the same ComfyUI backend approach. HIP 6.4 SDK only, Radeon GPUs below RX 6800/6900 may require manual intervention post-install. | +| **ComfyUI-Zluda** | A Windows-only ComfyUI variant using ZLUDA as an alternative AMD path when ROCm is not the preferred option, including on some modern Radeon GPUs and older GPUs without practical ROCm support. Like standard ComfyUI, it remains compatible with Stability Matrix's Inference UI through the same ComfyUI backend approach. HIP 6.4 SDK only, Radeon GPUs below RX 6800 may require manual intervention post-install. | | **InvokeAI** | A professional-grade tool with a polished UI, canvas editor, and a comprehensive workflow system. | | **SD.Next** | An all-in-one WebUI supporting a broad range of SD models, backends, and video generation. | | **SwarmUI** | A dial-and-input-driven frontend for the ComfyUI backend installed in Stability Matrix, designed to make advanced workflows more accessible without requiring constant node-graph editing. Formerly known as StableSwarm, it was originally developed in-house at Stability AI and now continues as an independent project. It includes many built-in power-user features, broad support for current and newer model families, and direct access to ComfyUI's own graph web UI from within the SwarmUI interface when you want to drop down to backend-level workflow editing. | @@ -30,7 +30,7 @@ Inference packages are used for generating images and video. They provide their | **Stable Diffusion Web UI (DirectML)** | A fork of the AUTOMATIC1111 WebUI with DirectML support for running on Windows without CUDA. | | **FramePack** | An advanced next-frame-prediction neural network for progressively generating video content. | | **FramePack Studio** | A full-featured video generation application built on top of the FramePack architecture. | -| **Wan2GP** | A highly optimized Gradio UI for AI video creation using WAN-based models, with performance-focused features and worfklows aimed at making modern and newer video-generation models more practical on lower-VRAM systems. | +| **Wan2GP** | A highly optimized Gradio UI for AI video creation using WAN-based models, with performance-focused features and workflows aimed at making modern and newer video-generation models more practical on lower-VRAM systems. | ## Training Packages diff --git a/docs/tips/overview.md b/docs/tips/overview.md new file mode 100644 index 000000000..953f80c5d --- /dev/null +++ b/docs/tips/overview.md @@ -0,0 +1,18 @@ +# Overview + +The Tips and Tricks section collects practical guidance for getting more out of Stability Matrix and local AI generation in general. It covers common terminology, effective use of the built-in Inference UI, package-specific advice, hardware-specific workflows, and ways to work within limited VRAM. + +[`Home`](../README.md) + +--- + +These pages are meant to be dipped into as needed rather than read front to back. Some of the material applies broadly to AI image and video generation and is useful whether or not you are working inside Stability Matrix. + +## In This Section + +- [Terminology](terminology.md) — Common image generation terms and what they mean +- Inference UI Tips *(planned)* — Effective use of the built-in Inference UI +- Per-Package Tips *(planned)* — Package-specific tips and links to upstream documentation +- AMD GPU Workflow *(planned)* — Getting image and video generation working on AMD hardware +- Model Dependencies *(planned)* — Required secondary files for modern models (text encoders, VAEs, etc.) +- VRAM Optimization *(planned)* — Reducing VRAM usage without sacrificing too much quality or speed diff --git a/docs/tips/terminology.md b/docs/tips/terminology.md index 49e5ab386..15652486d 100644 --- a/docs/tips/terminology.md +++ b/docs/tips/terminology.md @@ -26,7 +26,7 @@ Most image-generation workflows start with a model, a prompt, a seed, and a set A checkpoint or model is the trained weight file or model bundle used for generation or editing. In older Stable Diffusion ecosystems this is often a single `.safetensors` file. In newer families such as FLUX.2, Qwen Image Edit, Z-Image, and WAN 2.x, the usable model may instead be split into multiple files or distributed as a diffusers-style bundle. -From a practical user perspective, the model is the main thing that determines what kinds of outputs and workflows are possible. It affects whether a setup leans toward realism, illustration, text rendering, editing, or video, and it often determines which secondary files or add-ons are compatible. +For most users, the model is the main thing that determines what kinds of outputs and workflows are possible. It affects whether a setup leans toward realism, illustration, text rendering, editing, or video, and it often determines which secondary files or add-ons are compatible. **Prompt** @@ -38,7 +38,7 @@ Different ecosystems respond differently to prompt style. Some older families re A negative prompt describes what you do not want in the result. It is still very important in SDXL-based families such as SDXL 1.0, Pony, Illustrious, and NoobAI. It is usually less dominant in newer instruction-led families such as FLUX Kontext, Qwen Image Edit, Anima, and some WAN editing or video workflows. -In practice, negative prompts are commonly used to suppress artifacts, anatomy problems, unwanted text, watermarks, muddy detail, or style traits you do not want carried into the final output. +They are commonly used to suppress artifacts, anatomy problems, unwanted text, watermarks, muddy detail, or style traits you do not want carried into the final output. **Seed** @@ -56,19 +56,19 @@ Too few steps can leave the output muddy or undercooked. Too many can waste time The sampler is the algorithm that decides how each denoising step is performed. Common examples include Euler, Euler Ancestral, DPM++ 2M, and UniPC. -In practice, sampler choice can change the feel of an image even when the prompt and seed stay the same. Different samplers can affect sharpness, smoothness, contrast, painterliness, stability, and how "creative" or literal the result feels. +Sampler choice can change the feel of an image even when the prompt and seed stay the same. Different samplers can affect sharpness, smoothness, contrast, painterliness, stability, and how "creative" or literal the result feels. **Scheduler** The scheduler is the noise schedule used by the sampler across the denoising process. Common examples include Normal, Karras, Exponential, and SGM Uniform. -The simplest mental model is that the sampler is the solver, while the scheduler controls how noise levels are spaced through the run. That is why some guides recommend not just a sampler, but a sampler and scheduler pairing. +The simplest way to picture it is that the sampler is the solver, while the scheduler controls how noise levels are spaced through the run. That is why some guides recommend not just a sampler, but a sampler and scheduler pairing. **CFG / Guidance Scale** CFG means Classifier-Free Guidance. It controls how strongly the output follows the prompt. Lower CFG usually gives looser, more flexible, or more creative output. Higher CFG usually pushes the model to obey the prompt more strictly, but it can also introduce artifacts or make the image feel forced. -Practical ranges vary by family. SDXL-style models often live around 4.5 to 8, FLUX dev-style models often work around 3 to 5, and turbo or distilled models such as Z-Image Turbo may work better much closer to 1.0 to 3.0. If CFG is too high, images can become brittle, oversaturated, distorted, or unnatural. If it is too low, the image may drift away from the prompt, fall back toward the model's built-in composition biases, or come out washed out and lacking detail. +Useful ranges vary by family, and the numbers below are community rules of thumb that shift with each model and fine-tune rather than fixed rules. SDXL-style models often live around 4.5 to 8, FLUX dev-style models often work around 3 to 5, and turbo or distilled models such as Z-Image Turbo may work better much closer to 1.0 to 3.0. If CFG is too high, images can become brittle, oversaturated, distorted, or unnatural. If it is too low, the image may drift away from the prompt, fall back toward the model's built-in composition biases, or come out washed out and lacking detail. **Denoise Strength** @@ -78,7 +78,7 @@ This is one of the most important edit-workflow settings because it determines w ## Model Components -**How these parts fit together** +**What each component does in the pipeline** In a typical diffusion pipeline, your prompt is first turned into machine-readable vectors by a text encoder. The generator then starts from random noise in a compressed space called the latent, repeatedly denoises that latent, and finally converts the latent back into pixels. Older Stable Diffusion families usually do this with a UNet-style denoiser; newer families often use a DiT-style denoiser instead. @@ -88,19 +88,19 @@ The UNet is the denoising network used in traditional Stable Diffusion architect In image generation, the UNet does not directly paint pixels from scratch. Instead, it looks at a noisy latent and predicts how to move that latent toward a cleaner image representation step by step. Each denoising step uses the prompt conditioning plus the current noise level to decide what should be kept, changed, or clarified. -In practical terms, SD 1.5, SDXL 1.0, and most SDXL fine-tune ecosystems such as Pony, Illustrious, and NoobAI are still UNet-based. Many surrounding tools such as ControlNet and IP-Adapter were also built first around UNet-style diffusion pipelines, which is why those ecosystems often feel especially mature. +SD 1.5, SDXL 1.0, and most SDXL fine-tune ecosystems such as Pony, Illustrious, and NoobAI are still UNet-based. Many surrounding tools such as ControlNet and IP-Adapter were also built first around UNet-style diffusion pipelines, which is why those ecosystems often feel especially mature. **DiT** DiT stands for Diffusion Transformer. It fills the same broad role as a UNet, but uses transformer-style attention blocks instead of the classic UNet layout as the core denoiser. -The practical idea is still the same: start from noise, then repeatedly predict a cleaner version. The difference is architectural. A DiT-based model is using transformer machinery to reason over the latent representation, which can improve scaling behavior and make it easier to build newer large-model families around attention-heavy designs. +The idea is still the same: start from noise, then repeatedly predict a cleaner version. The difference is architectural. A DiT-based model is using transformer machinery to reason over the latent representation, which can improve scaling behavior and make it easier to build newer large-model families around attention-heavy designs. When a guide says a model is "DiT-based," it usually means the main denoising engine is not a classic Stable Diffusion UNet. FLUX.1, FLUX.2, Qwen Image and Qwen Image Edit, Z-Image, and several newer video families fall into this broader transformer-led direction. **VAE** -VAE stands for Variational Autoencoder, or sometimes referred to as Variable Auto Encoder. In image-generation workflows, the VAE is the component that converts between normal image pixels and the model's numerical, sometimes compressed if using a UNet workflow, representative latent space. +VAE stands for Variational Autoencoder. In image-generation workflows, the VAE is the component that converts between normal image pixels and the model's numerical, sometimes compressed if using a UNet workflow, representative latent space. You can think of it as a translator between two worlds: @@ -146,7 +146,7 @@ CLIP is also used more broadly outside the text encoder slot itself, including i CLIP Vision is the image-encoder side of the CLIP family. Instead of reading text, it reads an image and converts that image into a feature representation the rest of the pipeline can compare against or condition on. -In practical workflows, CLIP Vision is most often mentioned with tools like IP-Adapter. A reference image is run through CLIP Vision, useful visual features are extracted, and those features are then used to guide generation. Depending on the tool, that guidance may lean more toward style, composition, subject identity, or overall visual similarity. +CLIP Vision is most often mentioned alongside tools like IP-Adapter. A reference image is run through CLIP Vision, useful visual features are extracted, and those features are then used to guide generation. Depending on the tool, that guidance may lean more toward style, composition, subject identity, or overall visual similarity. If a workflow asks for a separate CLIP Vision model file, it usually means the feature extractor for reference-image conditioning is not bundled into the main checkpoint. @@ -154,7 +154,7 @@ If a workflow asks for a separate CLIP Vision model file, it usually means the f T5 and UMT5 are transformer-based text encoders from the broader language-model world. In image-generation pipelines, they are used as prompt encoders for newer architectures that want stronger language understanding than older CLIP-only setups typically provided. -The practical difference users notice is often prompt behavior. Models using T5- or UMT5-style encoders may respond better to plain-language instructions, longer semantic prompts, editing instructions, or more natural phrasing. That does not automatically make them "better" in every case, but it often makes them feel less tied to old keyword-stack prompting habits. +The difference users notice is often prompt behavior. Models using T5- or UMT5-style encoders may respond better to plain-language instructions, longer semantic prompts, editing instructions, or more natural phrasing. That does not automatically make them "better" in every case, but it often makes them feel less tied to old keyword-stack prompting habits. These encoders are also large. In many workflows they are distributed as separate files and can consume a meaningful amount of VRAM and RAM. That is why FLUX-family, Qwen Image Edit, and WAN workflows often involve more moving parts than a single older-style checkpoint file. @@ -175,7 +175,7 @@ Examples of conditioning include: - reference-image features from IP-Adapter - LoRAs or other adapters that alter the model's behavior -If you want a practical mental model, think of conditioning as "what information the model is being asked to obey." +If you want a one-line summary, think of conditioning as "what information the model is being asked to obey." **How conditioning changes the result** @@ -187,7 +187,7 @@ That is why different conditioning types can cooperate or fight each other. Prom ControlNet is an add-on network that lets a diffusion model follow an external structural guide such as edges, depth, pose, lineart, segmentation, or similar control signals. It was designed so the original base model could stay mostly intact while a separate control branch learns how to inject that extra guidance. -In practical use, ControlNet is what you reach for when you want the model to preserve layout or structure while still generating a new image. For example: +ControlNet is what you reach for when you want the model to preserve layout or structure while still generating a new image. For example: - use canny or lineart when you want the output to follow major outlines - use depth when you want stronger scene geometry and spatial consistency @@ -216,7 +216,7 @@ IP-Adapter is a lightweight image-prompt adapter that uses features from a refer Technically, IP-Adapter works by extracting image features with an image encoder and injecting those features into added attention pathways, while leaving the original base model mostly frozen. From a user perspective, the important part is simpler: it lets you guide generation with image-based cues without replacing the whole checkpoint. -In practice, IP-Adapter is commonly used for: +IP-Adapter is commonly used for: - borrowing overall style or color feel from a reference image - keeping composition or layout closer to a reference @@ -233,7 +233,7 @@ From a user's perspective, a LoRA is usually an add-on file that teaches the bas LoRAs are popular because they are small, easy to share, and stackable. They are often far smaller than full checkpoints, which makes experimentation much easier. They also preserve the base model's broad capabilities better than swapping to a totally different checkpoint for every idea. -In practical terms: +A few rules of thumb: - a low weight usually gives a lighter influence - a high weight pushes the result harder toward the LoRA's learned behavior @@ -255,7 +255,7 @@ An embedding, often called Textual Inversion in Stable Diffusion communities, is The important difference from a LoRA is scope. A textual inversion embedding modifies prompt-space behavior by teaching the text encoder and model to associate a learned token with a concept. A LoRA usually changes the model more directly through added weights. -In practical use, an embedding often behaves like this: +In typical use, an embedding often behaves like this: - you load the embedding file - you place its special token in the prompt @@ -288,7 +288,7 @@ The main distinction is scope: Image to image, usually shortened to img2img, starts from an existing image instead of pure random noise. The input image is encoded into latent space, noise is added to it, and then the model denoises from that partially noised starting point while following the prompt. -The important practical result is that img2img tends to preserve some relationship to the source image. Depending on settings, that relationship may be loose or strong. Low denoise strength keeps more of the original composition, shapes, colors, and lighting. High denoise strength gives the model more freedom to reinterpret the image and can approach a near-regeneration. +The key result is that img2img tends to preserve some relationship to the source image. Depending on settings, that relationship may be loose or strong. Low denoise strength keeps more of the original composition, shapes, colors, and lighting. High denoise strength gives the model more freedom to reinterpret the image and can approach a near-regeneration. This is why img2img is commonly used for: @@ -314,7 +314,7 @@ The masked area is where the model is allowed to invent new content. The surroun **Outpainting** -Outpainting extends an image beyond its original borders. In practical terms, you enlarge the canvas, create empty or masked space around the existing image, and generate into that new area. +Outpainting extends an image beyond its original borders. You enlarge the canvas, create empty or masked space around the existing image, and generate into that new area. It is often used when you want to: @@ -329,7 +329,7 @@ Outpainting is basically a special case of inpainting where the masked region is A mask is the region that tells the model where edits should happen. In most inpainting workflows, the masked area is the editable area and the unmasked area is meant to stay unchanged or mostly unchanged. -In common inpainting interfaces, this is usually presented as a white painted mask layer drawn over the image. In practical terms, you mark the area you want changed, and everything outside that painted region is treated as preserved context. +In common inpainting interfaces, this is usually presented as a white painted mask layer drawn over the image. You mark the area you want changed, and everything outside that painted region is treated as preserved context. Some interfaces and workflows also let you import a separate black-and-white mask image and place it on top of the base image as the edit mask instead of painting it by hand. @@ -349,7 +349,7 @@ The usual pattern is: This matters because many models are more stable at moderate resolutions than at very large native resolutions. A direct high-resolution generation can be slower, heavier on VRAM, and sometimes structurally worse. Hires Fix gets around that by first solving composition at a smaller size and then improving detail in a second pass. -In practice, it is often used to reduce muddy detail, improve textures, and make large outputs feel more finished. But if the second denoise pass is too strong, it can also alter composition or introduce new mistakes. +It is often used to reduce muddy detail, improve textures, and make large outputs feel more finished. But if the second denoise pass is too strong, it can also alter composition or introduce new mistakes. **Refiner / Refining** @@ -382,7 +382,7 @@ It is worth remembering that upscalers do not recover hidden real detail. They h ## Model Add-Ons and Variants -**How these terms relate** +**Lineage versus packaging** This section is about lineage and packaging: what model you start from, how it was specialized, how it is distributed, and what larger ecosystem it belongs to. @@ -429,7 +429,7 @@ That specialization can target: - better text rendering or editing behavior - a narrower domain such as anime, fashion, portraits, or concept art -In practical usage, most of the models people browse on sites like Hugging Face or CivitAI are not pure base models. They are fine-tunes, merges, or other derivatives built on top of a broader base family. +Most of the models people browse on sites like Hugging Face or CivitAI are not pure base models. They are fine-tunes, merges, or other derivatives built on top of a broader base family. **Merge** @@ -437,19 +437,19 @@ A merge is a model created by mathematically combining two or more checkpoints o Merges are especially common in SDXL-derived communities because that ecosystem produced huge numbers of stylistically different checkpoints. A merge might try to combine, for example, one model's anatomy, another model's color handling, and another model's illustration style. -From a user perspective, a merge can be very good, but it can also be less predictable than a cleaner base or fine-tune lineage. If a model feels powerful but a little "mystery meat" in behavior, it is often a heavily merged release. +A merge can be very good, but it can also be less predictable than a cleaner base or fine-tune lineage. If a model feels powerful but a little "mystery meat" in behavior, it is often a heavily merged release. **VAE-baked / AiO** VAE-baked means the checkpoint already includes its VAE inside the model file, so you do not usually need to load a separate external VAE. -This term is most common in older Stable Diffusion checkpoint ecosystems, where releases could ship in several different ways. It also still comes up in SDXL discussions, but in practice most SDXL-derived checkpoints are already VAE-baked: +This term is most common in older Stable Diffusion checkpoint ecosystems, where releases could ship in several different ways. It also still comes up in SDXL discussions, where whether a checkpoint bakes in its VAE varies from release to release. Plenty of community SDXL checkpoints ship without a baked VAE, or bake in the notoriously broken fp16 VAE, so a matching external VAE was a near-mandatory download for much of the SDXL era. It is worth checking the model page rather than assuming. The common shipping options are: - model only, requiring a matching external VAE - model plus separate VAE - model with the VAE already baked in -In newer DiT-based ecosystems, you may also see AiO, short for all-in-one. In practice, AiO usually means the full generation stack is packaged together as one coordinated model release, often including the transformer or denoiser, text encoders, and VAE in the same bundled file or tightly coupled package. +In newer DiT-based ecosystems, you may also see AiO, short for all-in-one. AiO usually means the full generation stack is packaged together as one coordinated model release, often including the transformer or denoiser, text encoders, and VAE in the same bundled file or tightly coupled package. In many AiO releases, that really does mean a single bundled model file with the text encoder and or VAE included. The important nuance is that this is still not universal. Some modern DiT releases remain split into separate internal components, but are distributed and loaded as one complete package instead of expecting the user to assemble mismatched pieces manually. @@ -475,13 +475,13 @@ Common quantized releases and formats include fp8 and int8 checkpoints, as well What matters in practice is that quantization is both a precision choice and a release-format choice. Some quantized models are still distributed as ordinary checkpoint files in a lower precision such as fp8 or int8. Others are repackaged into formats such as GGUF that are designed around quantized inference workflows. -Quantized releases are especially relevant in newer heavy model ecosystems, where full-size versions may be too large for many local users. In practical terms, quantization is often the reason a model becomes runnable at all on smaller GPUs. +Quantized releases are especially relevant in newer heavy model ecosystems, where full-size versions may be too large for many local users. Often it is the reason a model becomes runnable at all on smaller GPUs. **GGUF** GGUF is a model file format commonly used for quantized transformer-style models. In image-generation contexts, it shows up most often with newer transformer-heavy families where full-size releases may be too heavy for many local systems. -The practical reason people care about GGUF is not the container format by itself. It is that GGUF releases are often paired with quantization levels that make otherwise large models more runnable on limited hardware, especially in workflows aimed at lower VRAM usage. +The reason people care about GGUF is not the container format by itself. It is that GGUF releases are often paired with quantization levels that make otherwise large models more runnable on limited hardware, especially in workflows aimed at lower VRAM usage. **Model Family / Base Family** @@ -494,7 +494,7 @@ Common modern families and ecosystems include: - **SDXL 1.0**: the major open Stable Diffusion XL base family, still foundational for a huge amount of community work - **Pony**: a large SDXL-derived ecosystem known for stylized, character-heavy, and expressive prompt behavior - **Illustrious / illustrative SDXL families**: SDXL derivatives centered on polished illustration and anime-adjacent output -- **NoobAI**: a newer, growing anime and illustration ecosystem derived from Illustrious. Many Illustrious LoRAs still work well with it, though the broader community content base is still larger around Illustrious. Workflows may use either v-prediction or EPS depending on the specific release and setup +- **NoobAI**: a newer, growing anime and illustration ecosystem derived from Illustrious. Many Illustrious LoRAs still work well with it, though the broader community content base is still larger around Illustrious. Workflows may use either v-prediction or EPS depending on the specific release and setup. - **Anima**: a 2B anime and illustration-focused base model family made by CircleStone Labs in collaboration with Comfy Org, built for stylized character art, illustration-heavy workflows, and strong anime-oriented visual behavior - **FLUX Kontext**: FLUX-family releases focused on instruction-following, contextual edits, and image-aware generation behavior - **FLUX Klein**: smaller FLUX.2-oriented variants designed to be lighter and faster than the heavier full-dev style releases @@ -532,7 +532,7 @@ Image to video starts from a still image and animates it into a clip. Instead of This usually gives the user more control than pure text-to-video, because the first frame already locks in much of the composition, subject appearance, and visual style. The model is still generating new frames, but it is doing so from a stronger visual anchor. -In practice, I2V is often used for: +I2V is often used for: - animating illustrations or portraits - adding camera motion to a still scene @@ -557,7 +557,7 @@ FPS means frames per second in the saved output video. It controls playback spee That distinction matters. If you keep the same frames but change the FPS, you are mostly changing how quickly those frames are shown, not asking the model to invent different motion. -In practical terms: +So: - higher FPS makes the clip play faster or look smoother if enough frames exist - lower FPS makes the clip play slower or feel more choppy @@ -586,13 +586,13 @@ These are reference frames used to guide the video across time. - an end frame anchors how the clip should finish - a keyframe is a more general term for any frame used as a visual reference at a particular point in time -The practical idea is that the model is not generating every frame with equal freedom. It is being told that certain points in the clip should stay closer to specific reference images or target states. +The idea is that the model is not generating every frame with equal freedom. It is being told that certain points in the clip should stay closer to specific reference images or target states. This can be useful when you want to control transitions, preserve a character, move from one scene state to another, or create a more directed animation path instead of fully unconstrained motion. ## Performance and Precision Terms -**How these terms relate** +**The hardware and runtime side** This section is about the hardware and runtime side of generation: which backend is doing the work, what precision the model is stored or computed in, what memory-saving tricks are enabled, and why one setup may be faster or more compatible than another. @@ -602,7 +602,7 @@ In practice, many generation problems that look like "the model is bad" are real CUDA is NVIDIA's GPU compute platform and the main acceleration path used by most PyTorch-based image and video generation software on NVIDIA GPUs. -In practical terms, CUDA is what lets tensor operations run on an NVIDIA GPU instead of the CPU. It is also the ecosystem many surrounding optimizations are built around, including cuDNN, TensorRT, xFormers, Flash Attention, and a large amount of custom inference code. That is why NVIDIA workflows usually have the widest software support and the most mature optimized kernels. +At its core, CUDA is what lets tensor operations run on an NVIDIA GPU instead of the CPU. It is also the ecosystem many surrounding optimizations are built around, including cuDNN, TensorRT, xFormers, Flash Attention, and a large amount of custom inference code. That is why NVIDIA workflows usually have the widest software support and the most mature optimized kernels. You will often still see names like `torch.cuda`, `device="cuda"`, or `cuda:0` even in projects that also support AMD, Intel, or Apple hardware. That does not always mean the whole project is NVIDIA-only. It often means the codebase grew up in a CUDA-first ecosystem and kept CUDA-shaped API names as the common GPU interface. @@ -617,17 +617,17 @@ The simple mental model is: - ROCm = the full AMD compute platform - HIP = the CUDA-like interface layer inside that platform -In practical usage, ROCm support can vary more by GPU generation, OS, wheel availability, and kernel support than CUDA support often does. But for supported Radeon and Instinct hardware, ROCm is the main native AMD path for local model inference. +ROCm support can vary more by GPU generation, OS, wheel availability, and kernel support than CUDA support often does. But for supported Radeon and Instinct hardware, ROCm is the main native AMD path for local model inference. **ZLUDA** ZLUDA is a compatibility layer that lets some CUDA-targeted software run on non-NVIDIA hardware by translating enough of the CUDA-facing behavior for those applications to work. -At a practical level, you can think of it as taking software that expects CUDA-style code and CUDA API calls, then bridging or translating enough of that behavior into HIP and ROCm-compatible behavior for AMD hardware to execute it, using tooling provided by the HIP SDK such as `hipify`. +You can think of it as taking software that expects CUDA-style code and CUDA API calls, then bridging or translating enough of that behavior into HIP and ROCm-compatible behavior for AMD hardware to execute it, using tooling provided by the HIP SDK such as `hipify`. -In practical local image-generation use, ZLUDA most often comes up as an alternative AMD path on Windows when native ROCm support is unavailable, incomplete, or simply not the preferred setup for a particular GPU or package. It is not the same thing as ROCm, and it should not be thought of as AMD's native compute stack. +For local image generation, ZLUDA most often comes up as an alternative AMD path on Windows when native ROCm support is unavailable, incomplete, or simply not the preferred setup for a particular GPU or package. It is not the same thing as ROCm, and it should not be thought of as AMD's native compute stack. -The practical mental model is: +Put simply: - ROCm = AMD's native compute platform - ZLUDA = a compatibility path for some CUDA-oriented software on other hardware @@ -642,7 +642,7 @@ In image-generation communities, IPEX usually comes up when discussing Intel-nat **MPS** -MPS means the Apple Metal Performance Shaders backend as exposed through PyTorch on macOS. In practical local-AI discussion, it is the Apple Silicon GPU acceleration path used on M-series Macs. +MPS means the Apple Metal Performance Shaders backend as exposed through PyTorch on macOS. For local AI work, it is the Apple Silicon GPU acceleration path used on M-series Macs. It allows model operations to run on the integrated Apple GPU instead of only on the CPU. That can make local inference much more usable on Mac hardware, but MPS is still its own backend with its own operator coverage, performance limits, and occasional compatibility gaps compared with CUDA. @@ -672,7 +672,7 @@ In practice, fp8 usually matters most for newer transformer-heavy models where f int8 is an 8-bit integer precision format used in quantized inference workflows. Unlike fp8, which is still a floating-point format, int8 stores values as integers and usually relies on extra scaling logic during inference. -From a user perspective, int8 mostly means a more aggressively compressed model that can fit on weaker hardware than its fp16, bf16, or fp32 equivalent. The tradeoff is that int8 models are more dependent on runtime support, and depending on the implementation they may lose more quality or flexibility than lighter quantization approaches. +For most users, int8 mostly means a more aggressively compressed model that can fit on weaker hardware than its fp16, bf16, or fp32 equivalent. The tradeoff is that int8 models are more dependent on runtime support, and depending on the implementation they may lose more quality or flexibility than lighter quantization approaches. **xFormers** @@ -684,7 +684,7 @@ Users usually encounter it as a toggle, install dependency, or troubleshooting d Flash Attention is a highly optimized attention implementation designed to reduce memory traffic and make attention layers faster and more memory efficient. -In practice, this matters because attention is one of the more expensive parts of modern image and video models, especially in larger transformer-led architectures. Better attention kernels can noticeably improve performance or make a workflow fit into available memory when it otherwise would not. +This matters because attention is one of the more expensive parts of modern image and video models, especially in larger transformer-led architectures. Better attention kernels can noticeably improve performance or make a workflow fit into available memory when it otherwise would not. Flash Attention is strongly associated with NVIDIA CUDA workflows, but supported ROCm paths also exist through AMD-backed kernel implementations and integrations. The important user-facing point is not the exact kernel internals. It is that Flash Attention is one of the main "fast path" optimizations users may see mentioned in setup guides for heavy models. @@ -708,7 +708,7 @@ Tiled VAE encode/decode means running the VAE in smaller image chunks instead of By breaking the image into tiles, the VAE only has to process one region at a time, which makes larger images possible on weaker hardware. The tradeoff is that tiled VAE workflows can sometimes introduce seams, slight inconsistency between regions, or slower total processing time if the implementation is not good. -In practice, tiled VAE encode/decode is often the difference between successfully handling a large image and hitting an out-of-memory error during latent conversion. +Tiled VAE encode/decode is often the difference between successfully handling a large image and hitting an out-of-memory error during latent conversion. **OOM / Out of Memory** diff --git a/docs/troubleshooting/common-issues.md b/docs/troubleshooting/common-issues.md new file mode 100644 index 000000000..6d95f3e0b --- /dev/null +++ b/docs/troubleshooting/common-issues.md @@ -0,0 +1,132 @@ +# Common Issues + +This page collects the problems that come up most often in Stability Matrix, organized by symptom: what you see, the likely cause, and what to try. It focuses on general fixes and safe first steps rather than deep per-package debugging. + +[`Home`](../README.md) + +## Table of Contents + +- [Before You Start](#before-you-start) +- [Install Failures](#install-failures) +- [Launch and Update Failures](#launch-and-update-failures) +- [GPU and Backend Problems](#gpu-and-backend-problems) +- [Linux and macOS](#linux-and-macos) +- [Model Browser and CivitAI](#model-browser-and-civitai) +- [Inference Connection and Workflow Errors](#inference-connection-and-workflow-errors) +- [Finding Logs and Reporting Bugs](#finding-logs-and-reporting-bugs) +- [What's Next](#whats-next) + +--- + +## Before You Start + +Many issues clear up with a few quick steps before deeper troubleshooting: + +- **Restart Stability Matrix**, and if a package is misbehaving, close and relaunch it. +- **Update Stability Matrix**, then update the affected package. A large share of reported problems are already fixed in a newer build. +- **Check free disk space** in your data directory. Package installs and model downloads need room to unpack, and low disk space is a common cause of failed installs. +- **Check antivirus quarantine.** Some antivirus suites quarantine or truncate files inside a package's virtual environment, which can break launches in ways that look like install corruption. If you suspect this, allow-list your data directory and reinstall. + +If the problem persists, find the matching symptom below. + +## Install Failures + +**"Unable to install any package," or an install that fails partway through.** +This is usually an environment problem rather than a specific package bug: interrupted downloads, low disk space, a network timeout, or antivirus interference during the Python environment setup. Confirm you have free disk space, try again on a stable connection, and check antivirus quarantine as described above. If downloads are timing out, the pip and uv network variables (`PIP_TIMEOUT`, `UV_HTTP_TIMEOUT`, and the retry variables) documented in [Environment Variables](../advanced/environment-variables.md#common-environment-variables) can make installs more resilient on slow connections. The console output on the install page is the best place to see the underlying error. + +**A package fails to start with a missing C/C++ runtime, for example an error loading PyTorch or a missing `c10.dll`.** +This is the Visual C++ Redistributable prerequisite. Stability Matrix normally installs it automatically, so this means the automatic step did not complete. The fallback (installing the redistributable manually) is covered in [Installation → Windows](../getting-started/installation.md#windows). + +**Kohya's GUI fails with `No module named 'pkg_resources'` or `No module named 'packaging'`.** +These modules come from Python packaging tools (`setuptools` / `packaging`) that a training environment expects to be present before its own dependencies install. Stability Matrix pre-installs `packaging` and `setuptools` for Kohya's GUI, so if you hit this, first make sure Stability Matrix and the package are up to date, then reinstall the package so the environment is rebuilt cleanly. + +**A ComfyUI install fails with `File not found: venv/uv-build-constraints.txt`.** +This is a known, reported class of issue tied to a build-constraints file that only resolves when the working directory is the install directory. Recent Stability Matrix builds explicitly avoid leaking that setting into the running server. If you see it, update Stability Matrix and reinstall or update ComfyUI; if it persists on the latest build, report it (see [Finding Logs and Reporting Bugs](#finding-logs-and-reporting-bugs)). + +**Forge Neo fails to install or reinstall.** +Forge-based packages track fast-moving upstream repositories, and install failures here are frequently upstream dependency-resolution problems rather than a Stability Matrix bug. On Linux in particular, newer Python versions can make Torch resolution fail. As a first step, update Stability Matrix and retry the install; if it still fails, capture the console output and report it, since the exact cause tends to shift with upstream changes. + +**You installed the CUDA backend but have an AMD GPU (or vice versa).** +The PyTorch backend is chosen at install time from the **PyTorch Index** dropdown and can be changed afterward. If the wrong one was selected, you generally do not need to reinstall from scratch: open the package's **Python Packages** dialog (from the package's three-dot menu on the Packages screen) and switch the PyTorch Index there. See [Selecting a Hardware Backend](../package-manager/installing-packages.md#selecting-a-hardware-backend) and [Hardware Support](../advanced/hardware-support.md) for which backend matches your hardware. + +## Launch and Update Failures + +**A package stops launching after updating Stability Matrix or Windows.** +Environment changes can leave a package's virtual environment in a stale state. Update the package so its environment is refreshed, and if that does not help, reinstalling the package rebuilds it cleanly while leaving your shared models and outputs intact. + +**ComfyUI won't update or launch, is stuck on an old version, or reports "no update available."** +This is a recurring class of report rather than a single bug, and the cause varies (a pinned branch, a detached checkout, or a partially applied update). First confirm you are on the latest Stability Matrix build, then try updating the package again. If it stays stuck, reinstalling the package is the most reliable reset. + +**`xformers` errors after an update.** +`xformers` is tightly coupled to specific PyTorch and CUDA versions, so an update on one side can break the pairing. Updating the package (which realigns the versions) usually resolves it. `xformers` is only added on CUDA and ZLUDA installs when a package requests it, as noted in [Hardware Support](../advanced/hardware-support.md). + +**A package fails to launch with a `sitecustomize.py` `__main__.__file__` AttributeError on Windows.** +Stability Matrix writes a `sitecustomize.py` helper into each virtual environment, and this error is a known/reported class of issue in that area. Because that file loads on every interpreter startup, external software (some antivirus suites) truncating or corrupting it can also trigger startup failures. Update Stability Matrix so the current helper is written, check antivirus quarantine, and reinstall the package if the file remains damaged. + +**Stability Matrix itself is slow to launch or does not launch after updating.** +First give it a moment on the first launch after an update, since some one-time setup runs then. If it still does not start, check antivirus quarantine of the application folder, and consult the application log described in [Finding Logs and Reporting Bugs](#finding-logs-and-reporting-bugs) for a startup error to report. + +## GPU and Backend Problems + +**Older NVIDIA cards (Pascal / GTX 10-series) fail with a PyTorch or CUDA error.** +Older GPUs may need an older PyTorch build than the current default. Stability Matrix treats legacy NVIDIA GPUs specially and can fall back to an older CUDA index for them, but if the auto-selected variant does not work, you can change the PyTorch Index from the **Python Packages** dialog. See [NVIDIA (CUDA)](../advanced/hardware-support.md#nvidia-cuda) for how legacy cards are handled. + +**A new NVIDIA card errors on the newest CUDA build.** +The current default CUDA wheels require a recent NVIDIA driver. ComfyUI checks your driver on launch and warns if it is older than the required version, suggesting either a driver update or an older Torch index. Updating your NVIDIA driver is the usual fix; the driver-version detail is covered in [NVIDIA (CUDA)](../advanced/hardware-support.md#nvidia-cuda). + +**AMD GPU on Windows: which backend should I use?** +AMD on Windows has three paths (native ROCm preview, ZLUDA, and DirectML), and the right one depends on your specific GPU. Rather than duplicate that here, see [AMD on Windows](../advanced/hardware-support.md#amd-on-windows) for the full breakdown and caveats. + +**Your GPU is not recognized (for example the newest APUs).** +Hardware detection works from GPU names and compute capability, so very new or unusual parts can be missed. As noted in [Hardware Support](../advanced/hardware-support.md#how-backends-are-chosen), some GPUs still work with manual configuration even when no badge appears; you can set the PyTorch Index manually and test. + +**`torch.cuda.OutOfMemoryError` or other out-of-memory errors during generation.** +This means the workload exceeded your available VRAM, not a Stability Matrix bug. Try a smaller image or batch size, a lower-VRAM model, or a VRAM-optimization flag if your package offers one. Tuning PyTorch's allocator via `PYTORCH_ALLOC_CONF` can help with fragmentation-related OOM; see [PyTorch and CUDA Variables](../advanced/environment-variables.md#pytorch-and-cuda-variables). The VRAM guidance in the [Overview](../getting-started/overview.md#system-requirements) is a useful sanity check for what a given model family needs. + +## Linux and macOS + +**The Linux AppImage will not run on some distributions.** +The AppImage may need runtime support packages such as `libfuse2` (and, depending on the distribution, ICU or related libraries) that are not installed by default. This and the `.desktop` and AUR quirks are covered in [Installation → Linux](../getting-started/installation.md#linux). + +**macOS Gatekeeper blocks the first launch.** +This is expected for a downloaded app. The steps to allow it are in [Installation → macOS](../getting-started/installation.md#macos). + +## Model Browser and CivitAI + +**Search results are missing, or model metadata does not update.** +This is a known ecosystem issue rather than a fault in Stability Matrix. The CivitAI API has changed over time, and some content (notably certain NSFW material) has moved off the public API, so results can differ from the CivitAI website. There is often nothing to fix on the Stability Matrix side beyond keeping it updated; if a specific search or download consistently fails on the latest build, report it with details. + +**Selecting a model card feels laggy.** +Large browse results with many previews can make selection feel slow. Narrowing your search or letting the current page finish loading generally helps. + +## Inference Connection and Workflow Errors + +**A generation is rejected with a 400 error such as a node not found.** +This means the workflow needs a ComfyUI custom node or extension that is not installed. When a workflow's required extensions are missing or out of date, Stability Matrix shows an **Install Required Extensions?** dialog listing them; accepting it installs the missing extensions and restarts the package before generating. If you cancel that prompt, the generation cannot run until the extensions are installed. + +**Inference cannot connect, or connection is refused.** +Inference talks to a local ComfyUI server, so this usually means the backend is not running, or it is running on a non-default host or port. Stability Matrix reads the package's launch arguments and connects to the host and port there, falling back to `127.0.0.1:8188` when none are set. If you set a custom `--port` or `--listen`, make sure the package is actually launched and that those launch arguments are configured on the same package Inference is pointed at. + +## Finding Logs and Reporting Bugs + +Two kinds of logs are useful when something goes wrong: + +- **The Stability Matrix application log.** This is written to `app.log` in the `Logs` folder under Stability Matrix's application data directory (on Windows, `%AppData%\StabilityMatrix\Logs`). The quickest way to open it is the **Logs** shortcut under the directory shortcuts in `Settings`, which opens that folder directly. +- **Package console output.** Each package's own startup and runtime output appears in its console view, reached with the **Console** action on the package. This is where install and launch errors from the underlying tool are shown, and it is the best place to copy an error message from. + +When reporting a bug on the [GitHub issue tracker](https://github.com/LykosAI/StabilityMatrix/issues), including the following makes it far easier to help: + +- Your Stability Matrix version and operating system. +- Your GPU (and, on AMD/Intel, which backend you selected). +- The affected package and its version. +- A relevant excerpt from the application log or the package console output. + +For community help and quick questions, the project also has a Discord server, linked from the [project README](https://github.com/LykosAI/StabilityMatrix#readme). + +> **Note on known bugs:** Some issues above are known/reported classes of problem without a guaranteed user-side fix. For those, the most reliable steps are to update Stability Matrix, update or reinstall the affected package, and — if it persists on the latest build — report it on the issue tracker with logs so it can be investigated. + +## What's Next + +- [Hardware Support](../advanced/hardware-support.md) — Which GPU backend matches your hardware, and how backends are chosen +- [Environment Variables](../advanced/environment-variables.md) — Network, cache, and PyTorch variables useful when troubleshooting installs and GPU behavior +- [Installing Packages](../package-manager/installing-packages.md) — Where the PyTorch backend and Python Packages dialog live