Skip to content
sima-neatPublic

About

Runtime and compile time tooling for LLM, VLM models

Resources

Code of conduct

Contributing

Security policy

Stars

6 stars

Watchers

0 watching

Forks

SiMa.ai

LLiMa

Documentation: English | 한국어 | 日本語 | 繁體中文 | Українська

Vulcan CI Runtime Language Python

LLiMa is SiMa.ai's runtime and Model Compiler toolkit for generative AI models on Modalix DevKits, including LLMs, VLMs, and ASR models.

It provides a C++ runtime, Python-backed command-line orchestration, and Model Compiler tooling for running compiled GenAI model directories on SiMa hardware. The Python layer supports the CLI; it is not a separate public runtime API.

MoLE evaluates text and image-plus-text accuracy from a host or directly on a Modalix DevKit. Image tasks reuse upstream lmms-eval scoring; see MoLE image-plus-text accuracy for setup and supported tasks.

Packages

The Modalix DevKit runtime is delivered as three Debian packages:

  • sima-lmm-core: C++ runtime library, including libsima_lmm_runtime.so.
  • sima-lmm-dev: public C++ headers and SimaLMM CMake package metadata.
  • sima-lmm-cli: Python CLI modules, internal nanobind bridge, and llima command-line entry point.

The Model Compiler tooling is delivered as a Python wheel:

  • sima-lmm[sdk]: compiler SDK dependencies, including internal SiMa packages.
  • sima-lmm[sdk_ext]: external model, MoLE, benchmark, and evaluation dependencies for use alongside an installed Model Compiler environment.

Install Runtime Packages

Install the latest LLiMa runtime on a Modalix 3.0.0 DevKit with sima-cli:

sima-cli neat install llima

To install a specific release, branch, or artifact reference, include it in the target:

sima-cli neat install llima@<version-or-ref>

The root artifact contains install_llima.sh, an explicit install manifest, and the three LLiMa Debian packages. The installer verifies the Modalix 3.0.0 platform before APT resolves the MLA-RT and SiMa DMA-heap runtime dependencies.

Build consumers that only need to download the three LLiMa Debian packages can use the download-only subpackage:

sima-cli neat install llima/debs@<version-or-ref>

This preserves the previous LLiMa artifact behavior and does not install the downloaded packages. Use sima-cli neat install --help for the full target syntax and environment options.

Build LLiMa

Build from source inside the Neat SDK. The normal full build handles required setup, including submodules:

./build.sh --all --clean

Common build modes:

./build.sh --all --clean         # build all runtime debs and artifact layouts
./build.sh --clean --core        # package only sima-lmm-core
./build.sh --clean --core --dev  # package core and development files
./build.sh --no-dist             # build debs without publication layouts

Build output is generated under build-deb/. The installable bundle is written to dist/; its download-only three-package profile and archive are written to dist/debs/.

The runtime links directly to the platform simaai-mlart-modalix and simaai-heap libraries. Building LLiMa does not fetch or install NEAT Internals packages.

Install and validate those packages on Modalix; do not use the DevKit as the source-build environment.

Run

LLiMa expects a compiled runtime model directory produced by Model Compiler. The directory must contain runtime configuration and compiled ELF files required by the DevKit runtime.

The LLiMa model zoo is hosted on Hugging Face:

https://huggingface.co/simaai

Run the interactive CLI:

llima run <model_dir> --mode cli

Run the web server:

llima run <model_dir> --mode web

Model files load in parallel by default. When starting LLiMa alongside an active MLA workload, set SIMA_LLIMA_RUN_DISABLE_PARALLEL_LOAD=1 to load models one at a time so other inference requests can run between model loads.

LLiMa connects through MLA-RT's /dev/mla backend and requests scheduler group priority 10, leaving the platform default priority 0 available to latency-sensitive vision processes. Set MLA_CTX_PRIORITY before startup to override this value; lower numbers have higher priority, and negative values require CAP_SYS_NICE.

For EAGLE3 or Gemma4 MTP speculative decoding, pass the parent compiled-model directory containing the target and draft model subdirectories. llima run reads their serialized configurations and automatically loads the pair.

Model resolution order:

  1. Use the provided path if it exists.
  2. Check /media/nvme/llima/models/<model>.
  3. Check $LLIMA_MODELS_PATH/<model> if set.

Useful model-management commands:

llima search <term>
llima pull <model>
llima list
llima rm <model>

Interactive CLI

Inside llima run --mode cli:

  • add image <file>: add an image.
  • set system <prompt>: set the system prompt.
  • clear system: clear system prompt, chat history, and images.
  • clear history: clear submitted prompts, responses, and all images while preserving the system prompt.
  • print history: print chat history.
  • set audio <file>: set an audio file for transcription.
  • set language <lang>: set transcription language.
  • set lora <name>: load LoRA weights if present in the model package.
  • unset lora: clear LoRA weights.
  • enable-thinking: enable thinking mode and clear chat history.
  • disable-thinking: disable thinking mode and clear chat history.
  • help: print available commands.

Set SIMA_LLIMA_ENABLE_DRAFT_HIGHLIGHT=1 to highlight tokens accepted from the draft model when running speculative decoding in CLI mode.

Model Compiler Python Package

Install the Model Compiler tooling in editable mode

For active compiler development, use the Python 3.12 virtual environment installed by Model Compiler; do not create a separate LLiMa virtual environment. The Model Compiler installer creates a model-compiler virtual environment and installation directory in the first writable location:

  1. /sdk-extensions/model-compiler
  2. /sdk-add-on/model-compiler (legacy fallback)
  3. $HOME/sdk-extensions/model-compiler

Activate the installation used by your system, then install LLiMa with the sdk_ext dependencies. The installed Model Compiler environment already provides the internal SiMa compiler packages.

source <model-compiler-venv>/bin/activate
python -m pip install -e '.[sdk_ext]'
llima-compile --help

Alternatively, use a self-managed Python 3.12 development environment that already has a compatible Model Compiler package set installed.

Build wheel artifacts

Build and validate the pure-Python compiler wheel and the independently installable MoLE package:

./build_compiler_wheel.sh
./build_mole_package.sh

The commands are independent; build_mole_package.sh does not require build_compiler_wheel.sh to run first. Both packaging scripts create or reuse the shared build/wheel-tools-venv Python 3.12 environment. The helper installs the build package there when needed and does not modify the system Python.

Wheel versions follow the Debian package policy. Exact release tags produce the tag version; branch builds use the base version from deps/manifest.json plus the normalized branch and 12-character commit, for example 0.3.0+develop.0123456789ab. Set LLIMA_WHEEL_VERSION only when an explicit version override is required.

The root of dist/ is the installable DevKit bundle. The compiler wheel, guarded Model Compiler installer, and package metadata are written to dist/compiler/. The same verified wheel is staged with the MoLE installer and package metadata in dist/mole/. As in Core packages, artifact checksums are recorded in each profile's metadata.json; the legacy Debian archive keeps its separate checksum file under dist/debs/.

dist/compiler/metadata-wheel.json is a download-only profile for consumers that need the compiler wheel as an input to another package bundle. It contains only the wheel and its checksum. Its installation command is the POSIX no-op :—the same download-only convention as Core. For example:

sima-cli neat install --type wheel --install-dir <wheel-dir> llima/compiler@develop
dist/
├── sima-lmm-<version>-Linux-{core,dev,cli}.deb
├── install_llima.sh
├── llima-install-manifest.txt
├── llima-package-manifest.json
├── metadata.json
├── debs/
│   ├── sima-lmm-<version>-Linux-{core,dev,cli}.deb
│   ├── sima-llima-<ref>.tar.gz
│   ├── sima-llima-<ref>.tar.gz.sha256
│   └── metadata.json
├── compiler/
│   ├── sima_lmm-<version>-py3-none-any.whl
│   ├── install_compiler.sh
│   ├── metadata.json
│   └── metadata-wheel.json
└── mole/
    ├── sima_lmm-<version>-py3-none-any.whl
    ├── install_mole.sh
    └── metadata.json

Install a published Model Compiler artifact

Activate an existing Model Compiler environment, then install the latest published compiler artifact from the default branch:

source <model-compiler-venv>/bin/activate
sima-cli neat install llima/compiler

To install the latest artifact published for a specific branch:

source <model-compiler-venv>/bin/activate
sima-cli neat install llima/compiler@<branch>

Install MoLE

For a persistent MoLE installation with a dedicated virtual environment, run:

sima-cli neat install llima/mole

Documentation

LLiMa documentation is maintained in this repository:

Topic Description
LLiMa Overview of LLiMa, supported models, model manager commands, and capabilities.
System Requirements Prerequisites and system requirements for model compilation.
Model Compilation Complete guide to compiling LLM and VLM models for Modalix.
Model Deployment Deploy compiled models to Modalix devices.
LLiMa CLI Manage, run, and serve GenAI models on Modalix.
MoLE - Modalix Language Model Evaluator Benchmark LLM accuracy and performance on Modalix.
Contributor Guide Develop, test, document, and review changes to LLiMa.

About

Runtime and compile time tooling for LLM, VLM models

Resources

Code of conduct

Contributing

Security policy

Stars

6 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages