Documentation: English | 한국어 | 日本語 | 繁體中文 | Українська
LLiMa is SiMa.ai's runtime and Model Compiler toolkit for generative AI models on Modalix DevKits, including LLMs, VLMs, and ASR models.
It provides a C++ runtime, Python-backed command-line orchestration, and Model Compiler tooling for running compiled GenAI model directories on SiMa hardware. The Python layer supports the CLI; it is not a separate public runtime API.
MoLE evaluates text and image-plus-text accuracy from a host or directly on a Modalix DevKit. Image tasks reuse upstream lmms-eval scoring; see MoLE image-plus-text accuracy for setup and supported tasks.
The Modalix DevKit runtime is delivered as three Debian packages:
sima-lmm-core: C++ runtime library, includinglibsima_lmm_runtime.so.sima-lmm-dev: public C++ headers andSimaLMMCMake package metadata.sima-lmm-cli: Python CLI modules, internal nanobind bridge, andllimacommand-line entry point.
The Model Compiler tooling is delivered as a Python wheel:
sima-lmm[sdk]: compiler SDK dependencies, including internal SiMa packages.sima-lmm[sdk_ext]: external model, MoLE, benchmark, and evaluation dependencies for use alongside an installed Model Compiler environment.
Install the latest LLiMa runtime on a Modalix 3.0.0 DevKit with sima-cli:
sima-cli neat install llimaTo install a specific release, branch, or artifact reference, include it in the target:
sima-cli neat install llima@<version-or-ref>The root artifact contains install_llima.sh, an explicit install manifest,
and the three LLiMa Debian packages. The installer verifies the Modalix 3.0.0
platform before APT resolves the MLA-RT and SiMa DMA-heap runtime dependencies.
Build consumers that only need to download the three LLiMa Debian packages can use the download-only subpackage:
sima-cli neat install llima/debs@<version-or-ref>This preserves the previous LLiMa artifact behavior and does not install the
downloaded packages. Use sima-cli neat install --help for the full target
syntax and environment options.
Build from source inside the Neat SDK. The normal full build handles required setup, including submodules:
./build.sh --all --cleanCommon build modes:
./build.sh --all --clean # build all runtime debs and artifact layouts
./build.sh --clean --core # package only sima-lmm-core
./build.sh --clean --core --dev # package core and development files
./build.sh --no-dist # build debs without publication layoutsBuild output is generated under build-deb/. The installable bundle is written
to dist/; its download-only three-package profile and archive are written to
dist/debs/.
The runtime links directly to the platform simaai-mlart-modalix and
simaai-heap libraries. Building LLiMa does not fetch or install NEAT Internals
packages.
Install and validate those packages on Modalix; do not use the DevKit as the source-build environment.
LLiMa expects a compiled runtime model directory produced by Model Compiler. The directory must contain runtime configuration and compiled ELF files required by the DevKit runtime.
The LLiMa model zoo is hosted on Hugging Face:
Run the interactive CLI:
llima run <model_dir> --mode cliRun the web server:
llima run <model_dir> --mode webModel files load in parallel by default. When starting LLiMa alongside an
active MLA workload, set SIMA_LLIMA_RUN_DISABLE_PARALLEL_LOAD=1 to load models
one at a time so other inference requests can run between model loads.
LLiMa connects through MLA-RT's /dev/mla backend and requests scheduler group
priority 10, leaving the platform default priority 0 available to
latency-sensitive vision processes. Set MLA_CTX_PRIORITY before startup to
override this value; lower numbers have higher priority, and negative values
require CAP_SYS_NICE.
For EAGLE3 or Gemma4 MTP speculative decoding, pass the parent compiled-model
directory containing the target and draft model subdirectories. llima run
reads their serialized configurations and automatically loads the pair.
Model resolution order:
- Use the provided path if it exists.
- Check
/media/nvme/llima/models/<model>. - Check
$LLIMA_MODELS_PATH/<model>if set.
Useful model-management commands:
llima search <term>
llima pull <model>
llima list
llima rm <model>Inside llima run --mode cli:
add image <file>: add an image.set system <prompt>: set the system prompt.clear system: clear system prompt, chat history, and images.clear history: clear submitted prompts, responses, and all images while preserving the system prompt.print history: print chat history.set audio <file>: set an audio file for transcription.set language <lang>: set transcription language.set lora <name>: load LoRA weights if present in the model package.unset lora: clear LoRA weights.enable-thinking: enable thinking mode and clear chat history.disable-thinking: disable thinking mode and clear chat history.help: print available commands.
Set SIMA_LLIMA_ENABLE_DRAFT_HIGHLIGHT=1 to highlight tokens accepted from the
draft model when running speculative decoding in CLI mode.
For active compiler development, use the Python 3.12 virtual environment
installed by Model Compiler; do not create a separate LLiMa virtual
environment. The Model Compiler installer creates a model-compiler virtual
environment and installation directory in the first writable location:
/sdk-extensions/model-compiler/sdk-add-on/model-compiler(legacy fallback)$HOME/sdk-extensions/model-compiler
Activate the installation used by your system, then install LLiMa with the
sdk_ext dependencies. The installed Model Compiler environment already
provides the internal SiMa compiler packages.
source <model-compiler-venv>/bin/activate
python -m pip install -e '.[sdk_ext]'
llima-compile --helpAlternatively, use a self-managed Python 3.12 development environment that already has a compatible Model Compiler package set installed.
Build and validate the pure-Python compiler wheel and the independently installable MoLE package:
./build_compiler_wheel.sh
./build_mole_package.shThe commands are independent; build_mole_package.sh does not require
build_compiler_wheel.sh to run first. Both packaging scripts create or reuse the shared
build/wheel-tools-venv Python 3.12 environment. The helper installs the
build package there when needed and does not modify the system Python.
Wheel versions follow the Debian package policy. Exact release tags produce the
tag version; branch builds use the base version from deps/manifest.json plus
the normalized branch and 12-character commit, for example
0.3.0+develop.0123456789ab. Set LLIMA_WHEEL_VERSION only when an explicit
version override is required.
The root of dist/ is the installable DevKit bundle. The compiler wheel,
guarded Model Compiler installer, and package metadata are written to
dist/compiler/. The same verified wheel is staged with the MoLE installer and
package metadata in dist/mole/. As in Core packages, artifact checksums are
recorded in each profile's metadata.json; the legacy Debian archive keeps its
separate checksum file under dist/debs/.
dist/compiler/metadata-wheel.json is a download-only profile for consumers
that need the compiler wheel as an input to another package bundle. It contains
only the wheel and its checksum. Its installation command is the POSIX no-op
:—the same download-only convention as Core. For example:
sima-cli neat install --type wheel --install-dir <wheel-dir> llima/compiler@developdist/
├── sima-lmm-<version>-Linux-{core,dev,cli}.deb
├── install_llima.sh
├── llima-install-manifest.txt
├── llima-package-manifest.json
├── metadata.json
├── debs/
│ ├── sima-lmm-<version>-Linux-{core,dev,cli}.deb
│ ├── sima-llima-<ref>.tar.gz
│ ├── sima-llima-<ref>.tar.gz.sha256
│ └── metadata.json
├── compiler/
│ ├── sima_lmm-<version>-py3-none-any.whl
│ ├── install_compiler.sh
│ ├── metadata.json
│ └── metadata-wheel.json
└── mole/
├── sima_lmm-<version>-py3-none-any.whl
├── install_mole.sh
└── metadata.json
Activate an existing Model Compiler environment, then install the latest published compiler artifact from the default branch:
source <model-compiler-venv>/bin/activate
sima-cli neat install llima/compilerTo install the latest artifact published for a specific branch:
source <model-compiler-venv>/bin/activate
sima-cli neat install llima/compiler@<branch>For a persistent MoLE installation with a dedicated virtual environment, run:
sima-cli neat install llima/moleLLiMa documentation is maintained in this repository:
| Topic | Description |
|---|---|
| LLiMa | Overview of LLiMa, supported models, model manager commands, and capabilities. |
| System Requirements | Prerequisites and system requirements for model compilation. |
| Model Compilation | Complete guide to compiling LLM and VLM models for Modalix. |
| Model Deployment | Deploy compiled models to Modalix devices. |
| LLiMa CLI | Manage, run, and serve GenAI models on Modalix. |
| MoLE - Modalix Language Model Evaluator | Benchmark LLM accuracy and performance on Modalix. |
| Contributor Guide | Develop, test, document, and review changes to LLiMa. |
