Documentation: English | 한국어 | 日本語 | 繁體中文 | Українська
This repository builds a distributable Model Compiler bundle for sima-cli.
The bundle contains:
- curated Python wheels for the Model Compiler package set
- direct internal wheel dependencies required by those packages
- binary package artifacts such as the MLA toolchain
- an installer script for the target host
- generated
metadata.jsonforsima-cli
Use this repository to build a repeatable Model Compiler installation for the Neat SDK or an Ubuntu host, with a predictable package set and a self-contained extension layout.
Install and authenticate sima-cli first. See the
sima-cli documentation for setup
instructions.
Then install Model Compiler inside the Neat SDK or on an Ubuntu 22.04/24.04 host. Select the package for the host architecture and the artifact ref you want to use:
# Current develop compiler
sima-cli neat install model-compiler/amd64@develop
sima-cli neat install model-compiler/arm64@develop
# Tagged release; replace v2.1.3.1 with the desired release tag
sima-cli neat install model-compiler/amd64@v2.1.3.1
sima-cli neat install model-compiler/arm64@v2.1.3.1The @develop target resolves to the latest successfully published build from
the develop branch and can briefly lag the branch head while CI is running.
- scripts/source.json: bundle manifest
- scripts/build_modelsdk_bundle.sh: end-to-end bundle builder
- scripts/download_modelsdk_wheels.sh: downloads Python and binary artifacts
- scripts/install_modelsdk_wheels.sh: installs the bundle on a target host
- scripts/generate_metadata.py: generates
metadata.json - docs/generated/index.md: generated Model Compiler API reference entrypoint
dist/: default output directory for built bundles
Generated API reference docs live under docs/generated. Start with docs/generated/index.md, which links to the generated AFE API pages.
scripts/source.json defines the bundle contents.
Example:
{
"sdk_version": "2.0.0",
"python_version": "3.10",
"system_dependencies": {
"ubuntu": [
"build-essential",
"curl",
"libllvm18",
"libopenblas0-pthread"
]
},
"dependency_overrides": {
"onnx": "1.17.0",
"onnxruntime": "1.21.1",
"protobuf": "4.25.7"
},
"python-packages": [
{ "name": "sima-frontend", "version": "2.0.0.dev0+master.371" }
],
"binary-packages": [
{
"name": "mla/toolchain/mla-toolchain",
"version": "v2.1.3560-develop.409",
"extension": ".zip"
}
]
}Fields:
sdk_version: used when constructing the bundle version stringpython_version: target interpreter version for the installed virtual environmentsystem_dependencies.ubuntu: apt packages installed on the target host before Python and virtual environment setupdependency_overrides: exact versions to rewrite into downloaded wheel metadata when neededpython-packages: top-level Python packages to include in the bundle; entries may includefileto download a wheel fromsima-pypi/<package-name>/, orurlfor a full direct wheel URL, when the wheel is not exposed by the configured Python indexbinary-packages: non-wheel artifacts fetched from Artifactory and installed into the Model Compiler virtual environment; the MLA toolchain derives itsx86oraarch64archive suffix from the build targetaarch64: optional architecture-specific overrides for fields that genuinely differ on ARM64;dependency_overridesentries are merged over the global map, so list only packages whose ARM64 pins differ
Native source builds triggered during installation, such as
llama_cpp_python, use the build backend's default parallelism. Set
MODELSDK_BUILD_PARALLEL_LEVEL to limit or override build parallelism on
resource-constrained machines.
Daily Component Update checks daily every four hours (00:17, 04:17, 08:17,
12:17, 16:17, and 20:17 UTC). Daily Master Component Update checks and builds
daily-master every six hours (00:17, 06:17, 12:17, and 18:17 UTC).
Both use develop's scripts/source.json.
A small Daily Component Update wrapper on the default branch calls the
updater workflow on develop; updater scripts and the manifest are checked out
from the same resolved develop commit. GitHub may delay scheduled runs.
Exact versions remain the build inputs. The optional
component-updates block explicitly lists packages managed by automation:
"component-updates": {
"python-packages": {
"sima-frontend": {"version-prefix": "3.0.0.dev0+develop."}
},
"binary-packages": {
"mla/toolchain/mla-toolchain": {
"version-prefix": "v3.0.0-",
"channel": "develop"
}
}
}The scanner chooses the greatest numeric build suffix in that exact prefix.
A prefix may differ from the current pin, explicitly authorizing the initial
base/channel transition. Subsequent runs advance only within that prefix.
Editing an exact pin does not change the configured prefix. Different components
may use different base versions. Package names are normalized, and matching
pins in dependency_overrides and python-packages update together.
URL/file-pinned packages are excluded. Conflicting duplicate pins are rejected.
Only listed components are managed when the block is present; an empty block
manages none. Older manifests without the block retain pin-derived discovery.
The supplied policy manages ten Python packages and the MLA toolchain. Moving
LLiMa snap references are not included. MLA uses version-prefix for the release
and channel for the branch: v3.0.0- plus develop matches
v3.0.0-3609-develop.453. Discovery orders the numeric revision first, then the
numeric channel build, allowing both to advance within 3.0.0 develop. Other
releases, PRs, and commit-only archives are excluded. An explicit policy allows
migration from a legacy 2.1 pin. Legacy binary prefixes ending in . retain
their existing suffix-only behavior. Architecture scans select only matching
x86 or aarch64 Ubuntu ZIPs; merging two scans requires a common version.
The private macOS/ARM64 runner checks Python 3.12 ARM64 or universal wheels.
A changed candidate refreshes daily from the scanned develop commit using
an explicit force-with-lease. Identical existing candidates
leave the branch untouched. A new develop commit refreshes daily even if no
package pins changed, so code and bundle-version changes are included. GitHub App credentials trigger the ordinary Build,
which packages, installs, and smoke-tests both architectures. These tests gate
cross-component compatibility; individual artifact availability does not.
scripts/source.json pins the verified master artifacts, so ordinary branch and
PR builds exercise the 3.0 master components. daily explicitly applies
scripts/component-policy-develop.json; daily-master applies
scripts/component-policy-master.json. Both policies are checked in and reviewed.
Both branches use build scripts from develop. Master selects master artifacts
for all ten managed Python components and the MLA toolchain; third-party pins
and the existing LLiMa snap policy remain unchanged. Missing compatible master
artifacts fail the update instead of publishing mixed develop/master pins.
Both channel policies must cover the same managed components as the manifest.
Both daily branches are disposable generated build channels, refreshed with an
explicit lease and independent concurrency groups. Do not edit them directly.
They publish branch-scoped packages and containers through the same Build gates.
Neither opens a PR back to develop; promote desired pins through a separately
reviewed change when needed. New develop code reaches both branches on the next
scan, so merging daily pins is unnecessary. Build notifications identify the
channel. An unchanged daily candidate does not trigger another build.
The six-hour master workflow explicitly dispatches Build for an unchanged
daily-master candidate; changed candidates trigger Build through their push.
Slack uses the saved upstream report when available. Without it, the notification
compares the exact candidate and its parent manifests, reports version changes
or verified no changes, and identifies missing upstream commit evidence.
Manual dry runs are available through workflow_dispatch. The optional
source_ref input can select a feature-branch manifest only with dry_run=true;
normal updates always read develop. Artifactory access
uses the private runner's existing netrc credentials. Pushes use
NEAT_RELEASES_APP_ID and NEAT_RELEASES_APP_PRIVATE_KEY.
GitHub executes scheduled and workflow_run workflows from the default branch:
merge the implementation PR into develop, then sync update-components.yml, update-master-components.yml,
daily-build-notify.yml, and container-build.yml to main to activate both
scheduled jobs, completion notifications, and the container duplicate-build
exclusion. The existing default-branch PR wrapper calls the now
disabled worker on develop, so daily-to-develop PR creation stops immediately
after the implementation merges. Existing open PRs are not closed automatically.
No daily branch needs to be created manually; the first non-dry run creates it.
Run each workflow with dry_run=true to exercise its policy without pushing or building.
Before you build, configure ~/.netrc with credentials for
artifacts.eng.sima.ai. The build downloads wheels and binary artifacts from
Artifactory, so it requires valid access tokens.
Example:
machine artifacts.eng.sima.ai
login <your-username-or-token-name>
password <your-artifactory-access-token>
Restrict the file permissions when needed:
chmod 600 ~/.netrcDefault build:
./scripts/build_modelsdk_bundle.shTypical explicit build:
./scripts/build_modelsdk_bundle.sh \
--source-json ./scripts/source.json \
--output-dir ./dist \
--index-url https://artifacts.eng.sima.ai/artifactory/api/pypi/sima-pypi-group/simple \
--extra-index-url https://pypi.org/simpleBuild a bundle for a specific architecture:
./scripts/build_modelsdk_bundle.sh --target-arch aarch64By default, the metadata version comes from the exact git checkout. If HEAD
has a release tag such as v1.0.0, the generated metadata.json uses
1.0.0. Otherwise, it falls back to
sdk_version.neat+branch.git-short-hash. Pass --bundle-version to override
this behavior.
The LLiMa Vulcan entry uses policy: snap for development builds. It resolves
the latest artifact from the matching Model Compiler branch. A feature branch
without a matching LLiMa artifact falls back to develop; develop, main,
and release branches fail instead of crossing channels.
The Release workflow requires the intended LLiMa version and replaces snap
policy on the Model Compiler release branch with a commit-qualified ref such as
v0.4.0:<commit> before creating the Model Compiler tag. Tag builds reject
snap policy and non-commit-qualified refs. Generated metadata records the
requested ref, resolved commit, wheel version, and wheel filename.
The build creates a self-contained archive by default and performs these steps:
- Read the package manifest from
source.json. - Download every wheel in the target architecture's dependency closure.
- Download binary package archives such as the MLA toolchain.
- Copy the installer and source manifest into the output directory.
- Generate
manifest.txtwith the bundled wheel filenames. - Generate the ZIP archive plus
metadata.jsonandmetadata-offline.json.
The release workflow builds the full dependency closure into one archive.
metadata.json downloads that archive, extracts it into a temporary directory,
runs the installer locally, and removes the extracted directory afterward.
metadata-offline.json references the same archive but provides manual
distribution instructions for transferring the ZIP to another environment and
running its included installer there. It is available from Linux, macOS, and
Windows hosts so the archive can be downloaded before transfer; the installer
inside the archive remains Linux-only.
Output files in dist/ typically include:
model-compiler-<arch>.zipmetadata.jsonmetadata-offline.json
Test a freshly built bundle from your local machine before you publish it.
From this repository:
cd dist
python3 -m http.serverThen install the bundle from an Ubuntu host or a Neat SDK environment:
sima-cli install -m http://<ip>:8000/metadata.jsonReplace <ip> with the IP address of the machine that serves the dist/
directory.
This validates the Model Compiler bundle against a local metadata source.
The scripts download internal Python packages and binary artifacts from SiMa Artifactory.
Python wheels:
- primary index: Artifactory
- fallback index: public PyPI via
--extra-index-url
Binary packages:
- fetched from
https://artifacts.eng.sima.ai/artifactory/...
If your environment requires authentication, configure Artifactory credentials
with .netrc or your shell environment before you run the scripts.
After you build the bundle, copy the architecture-specific ZIP to the target Linux machine, extract it, and run the included installer. For example, for an ARM64 target:
unzip -q model-compiler-arm64.zip -d model-compiler-arm64
cd model-compiler-arm64
bash ./install_modelsdk_wheels.shUse model-compiler-amd64.zip for an amd64 target. Alternatively, install
through metadata.json with sima-cli, which extracts the same archive into a
temporary directory and runs this installer automatically.
Native dependency builds (including llama_cpp_python) use host gcc/g++
and their default matching C++ headers. The installer clears inherited SDK
compiler flags and compiler include/library search overrides for these builds;
it does not select a version from /usr/include/c++.
The installer performs these steps:
- Read
source.json. - Install required Ubuntu system packages from
system_dependencies.ubuntu. - Find or install the required Python version, using
pyenvwhen needed. - Read
manifest.txtto identify the bundled Model Compiler wheels. - Create a Model Compiler virtual environment.
- Install bundled binary packages into that virtual environment.
- Install top-level package specs from
python-packages, including extras such assima_lmm[sdk]. It uses manifest-listed wheels as local--find-linksinputs with--no-index. - Update shell startup files with the Model Compiler virtual environment
PATH. - Leave the extracted archive available for reuse; remove it manually if it is no longer needed.
The installer creates the Model Compiler virtual environment in one of these locations:
/sdk-extensions/model-compilerif/sdk-extensionsexists and is writable/sdk-add-on/model-compileras a backward-compatible fallback~/sdk-extensions/model-compilerotherwise
Binary package contents, such as the MLA toolchain, are installed into the same virtual environment under:
bin/include/lib/
The downloaded MLA toolchain zip is sanitized during bundle creation so only
its bin/ payload is preserved.
The installer also restores executable permissions for binaries copied into model-compiler/bin.
The installer adds Model Compiler environment setup to one of these files:
~/.bashrcwhen it exists, otherwise~/.bash_profile
It appends an idempotent block that exports:
PATH=<venv>/bin:$PATHAfter installation, reload your shell:
source ~/.bashrcor:
source ~/.bash_profileYou can also log out and back in.
After installing and activating the Model Compiler extension, run the fast smoke test:
activate-model-compiler
python /path/to/model-sdk/scripts/smoke_test_modelsdk.py --tier basicThe basic tier is intended for every CI/CD extension-install job. It checks:
- the active Python is the Model Compiler venv
- Model Compiler
bin/is onPATH - core MLA tools such as
mla-nm,mla-size,mla-readelf, andmla-isimare runnable afe-replay-compile,onnxsim, andllima-compileentry points are runnable- required Python modules including
afe,onnx,torch,sima_lmm,gguf,llama_cpp, andsafetensorsare importable
Heavier tiers are available for scheduled or pre-release jobs:
# Export a synthetic ResNet50 ONNX model with torchvision.
python scripts/smoke_test_modelsdk.py --tier resnet-export
# Export or reuse ResNet50, audit it, simplify it, and run quantize-only.
python scripts/smoke_test_modelsdk.py --tier resnet-quantize
# Same as resnet-quantize, but also runs the compile step.
python scripts/smoke_test_modelsdk.py --tier resnet-compile
# Download Qwen3-0.6B and run native graph generation, quantization, and compilation.
python scripts/smoke_test_modelsdk.py --tier llima-qwen3-compile
# Download YOLOv8n ONNX, simplify/audit it, and run quantize+compile.
python scripts/smoke_test_modelsdk.py --tier yolo
# Run all long-form smoke cases and print a final result summary.
python scripts/smoke_test_modelsdk.py --tier all
# Reuse a cached YOLO ONNX model and optionally verify model-to-pipeline references.
python scripts/smoke_test_modelsdk.py \
--tier yolo \
--yolo-model /path/to/yolo.onnx \
--model-to-pipeline-dir /path/to/tool-model-to-pipelineThe ONNX operator audit is informational by default. Add --strict-audit when
you want the smoke test to fail on unknown or unsupported operators in the
bundled support database.
When --work-dir is omitted, model tiers create temporary work directories
under ~/tmp. If --work-dir is supplied, model tiers create a fresh per-run
subdirectory under that path for intermediate and compiled artifacts. This
avoids collisions with stale files from previous smoke-test runs.
If an explicit --work-dir exists but is not writable, the runner falls back
to a per-user sibling such as ~/tmp/modelsdk-smoke-$USER.
The all, resnet-compile, llima-qwen3-compile, and yolo tiers also check
their compiled artifacts. The LLiMa tier requires floating-point and quantized
Model SDK files plus MPK archives with MLA ELF files. The other compile tiers
report package counts and sizes and run tools such as mla-size and
mla-readelf on packaged ELF files.
Use activate-model-compiler to enter the installed environment and
deactivate-model-compiler to leave it.
After creating a bundle, install it into an Ubuntu 24.04 container with:
# x86_64 host and bundle
./scripts/build_modelsdk_container.sh \
--bundle-dir dist/amd64/package \
--target-arch amd64 \
--image model-compiler:local \
--smoke-test
# ARM64 host and bundle
./scripts/build_modelsdk_container.sh \
--bundle-dir dist/arm64/package \
--target-arch arm64 \
--image model-compiler:arm64-local \
--smoke-testThe bundle directory must already contain install_modelsdk_wheels.sh,
source.json, manifest.txt, and the downloaded wheel and binary artifacts.
The container helper does not build or download the bundle. It passes the local
directory to BuildKit as a temporary build context, so package credentials and
the multi-gigabyte bundle are not copied into an image layer.
For example, on macstudio, download and extract the official ARM64 bundle
before invoking the container helper:
mkdir -p model-compiler-arm64 && cd model-compiler-arm64
sima-cli neat download model-compiler/arm64
unzip -q model-compiler-arm64.zip -d bundleOpen an interactive shell:
docker run --rm -it -v "$PWD:/workspace" model-compiler:localThe container starts Bash as a login shell. pyenv is initialized automatically,
and the Model Compiler virtual environment is active without running
activate-model-compiler manually. The same environment is applied to direct
container commands. The prompt includes the image version, for example
[model-compiler feature/container:deadbee]. Development builds use
branch:short-git-hash; an exact official release tag such as v2.1.3 is used
on a tagged commit. /etc/sdk-release records that version, the SDK and Python
versions, the target architecture, the source branch and commit, the UTC image
build time, and component versions resolved from the bundle's source.json.
The source manifest is also preserved at
/usr/local/share/model-compiler/source.json for detailed inspection.
Run a compile smoke test and keep its artifacts on the host:
mkdir -p modelsdk-smoke
docker run --rm \
-v "$PWD/modelsdk-smoke:/workspace/modelsdk-smoke" \
model-compiler:local \
python /opt/model-compiler-tests/scripts/smoke_test_modelsdk.py \
--tier resnet-compile \
--work-dir /workspace/modelsdk-smokeBuild natively on a matching host when possible: linux/amd64 on ll2 and
linux/arm64 on macstudio.
For daily, the Build workflow directly builds and smoke-tests amd64 and
arm64 container images after both package installation tests pass. It reuses
the same run’s package artifacts and publishes the multi-architecture GHCR
image before the Build run completes, so daily success notifications and PR
validation include container publication. This direct path does not require
a container completion listener on main.
After the Build workflow succeeds for a pushed branch, GitHub Actions builds
the amd64 and arm64 containers from that run's package artifacts and publishes
a multi-architecture image to a branch-scoped GHCR package. Branch names are
lowercased and runs of non-alphanumeric characters (including /, _, and
.) become a single -. Leading and trailing separators are removed, and
the branch suffix is limited to 180 characters to keep Docker names valid.
This follows the Neat SDK repo-and-branch naming convention. The package name is model-compiler-<branch>, except that
main uses model-compiler without a branch suffix:
main ghcr.io/sima-neat/model-compiler:latest
daily ghcr.io/sima-neat/model-compiler-daily:latest
develop ghcr.io/sima-neat/model-compiler-develop:latest
fix/container-build ghcr.io/sima-neat/model-compiler-fix-container-build:latest
The full source commit is also published as an immutable image tag. Branches
that normalize or truncate to the same name (for example, fix/foo and fix-foo) share a
package; use distinct normalized branch names when separate images are needed.
Deleting a branch deletes its package only when no live branch maps to it.
The canonical model-compiler release package is always retained. A daily
reconciliation run handles missed cleanup events and supports a manual dry run.
Previously published packages with a branch-hash suffix are retained while their source branch exists, but new builds publish only to the names above. Update pull commands and integrations to the new names to receive new builds. Legacy packages become eligible for cleanup after their branch is deleted.
Container builds use architecture-specific Buildx registry caches stored as
buildcache-amd64 and buildcache-arm64 tags in the branch package. A branch
also imports the matching develop cache when available. Deleting the branch
package therefore removes both its images and its build caches.
On ARM systems, activation enables the JAX compilation path with the NEON CPU
ISA by default. Use --no-jax as a compatibility or debugging fallback:
activate-model-compiler --no-jaxThis explicitly disables the JAX compilation path for the activation while
preserving unrelated XLA_FLAGS. Deactivation restores the environment values
that were present before activation.
After a successful install, the installer removes downloaded bundle resources from the bundle directory, including:
- manifest-listed wheel files
- binary package archives such as the MLA toolchain zip
- extracted binary package directories if present
It keeps installer and metadata files such as:
install_modelsdk_wheels.shsource.jsonmanifest.txtmetadata.json
- If a sample or test script fails with missing Python modules, confirm that it
uses the installed Model Compiler virtual environment and not a separate local
.env. - If a compiled package fails at runtime with missing shared libraries, check that:
- the required Ubuntu packages from
system_dependencies.ubuntuwere installed
- the required Ubuntu packages from
- If GitHub push fails from an automated environment, verify that the git remote has usable credentials.
- If a wheel is missing from Artifactory, provide
--extra-index-urlso the build can fall back to public PyPI for Python packages.
The archive contains the installer and every dependency, so regenerate it and
its matching metadata together with build_modelsdk_bundle.sh rather than
editing metadata independently.
This repository currently focuses on:
- curated host-side Model Compiler installation
- binary package inclusion for the MLA toolchain
sima-cli-style bundle metadata generation
If you add more extensions later, use the existing extension-style install root
under sdk-extensions/.
Every architecture's Build summary lists the actual selected SiMa component
versions, including MLA, and an expandable table of all other bundled packages.
The inventory is derived from downloaded artifact filenames, not just requested
pins. component-versions.json and component-versions.md are included in the
build artifacts; the same inventory is stored in metadata.json and
metadata-offline.json under component-versions. LLiMa's resolved commit is
reported separately.
Completed daily Build runs report success or failure (including cancellation)
to neat-vulcan-events, using the organization's SLACK_BOT_TOKEN secret and
SLACK_VULCAN_EVENT_CHANNEL_ID variable. Notifications link to the build and its
component tables. The Slack bot must have chat:write to post the result and
files:write to attach the private upstream-change report. Scope changes require
reinstalling the Slack app and updating SLACK_BOT_TOKEN if Slack issues a new
bot token. The completion listener must exist on default branch main;
it calls the protected develop worker and executes trusted notification code
from develop, never code from a build artifact. Feature-branch and develop builds do not send these notifications.
The periodic updater only rebuilds when the resolved candidate or develop
commit changes. To repeat validation of an unchanged candidate, manually run
Build on daily. The test-only codex/daily-component-updates-e2e branch is no
longer the publication target.
The three default-branch entry points (update-components.yml,
daily-build-notify.yml, and open-component-update-pr.yml) are kept in one
isolated commit: merge into develop first, then cherry-pick that commit onto
main. Each entry point calls its corresponding *-worker.yml@develop. Worker
logic and helper scripts are maintained only on develop for this rollout.
Package-install tests upload model-compiler-smoke-evidence-<arch>-<attempt>
on both success and failure, retained for 14 days. The artifact contains host CPU
and memory information, installed package versions, package and simulator hashes,
installation/activation logs, and complete smoke output. Each MLA simulator
invocation includes separate stdout/stderr, its original command and exit status,
and copies of the ELF/data/check inputs taken before SDK temporary-file cleanup.
A supervisor stops the simulator process group and reaps the simulator if its
wrapper is terminated, including when a caller timeout kills the wrapper with
SIGKILL. A failed invocation also gets a GDB replay with a 120-second limit. A successful
replay does not change the original failure. Job failure additionally uploads
model-compiler-smoke-work-<arch>-<attempt> for 7 days, containing the generated
smoke models and compiler outputs. Abrupt runner loss can still prevent uploading.
After extracting the evidence on a host with the matching compiler installed:
# replay.sh uses the captured absolute simulator path by default.
MODELSDK_SIMULATOR="$(command -v mla-msim)" sh path/to/failed-invocation/replay.shChoose the invocation directory whose command.json reports the failure (the
artifact also includes successful invocations and preflight --help calls).
The wrapper is enabled only in CI or when explicitly requested; normal compiler
activation and compilation are unchanged. To collect the same evidence locally:
python scripts/smoke_evidence.py run --output ./debug-evidence/resnet -- \
python scripts/smoke_test_modelsdk.py --tier resnet-compile-precisions --verbose \
--work-dir ./debug-results/resnet