1bit engine

Documentation: 1bit.gg · Measured results: wiki · Community: Discord

Site: github.com/1bit-MONSTER/site (builds 1bit.gg from this repository's README.md and docs/)

The 1bit engine is AMD's HRX runtime, our own HIP kernels on the Strix Halo GPU, and our NPU engine, all inside Lemonade. On a Ryzen AI Max+ 395, every graph node of every model and mode we test runs on the GPU: 43 of 43 configurations in the census, with 0 nodes on the CPU (#476, #480, #481). Qwen3.8-27B (UD-Q4_K_XL) with MTP speculative decoding decodes at 36.0 / 26.8 / 29.9 tok/s on code / prose / short chats (#487).

Lemonade stays the server you talk to (its catalog, downloads, router and UI). For the models it hands to 1bit, it runs the engine as one of its backends, behind an OpenAI-compatible API:

• HRX on the Radeon iGPU (docs/hrx.md): AMD's ggml-hrx with its Loom kernels and our HIP kernels, the engine's one GPU route. For a GGUF, Lemonade starts the engine's llama-server directly, one HTTP hop (#475, docs/serve.md). Placement is strict by default: the whole graph runs on HRX, or the load stops with the node that would not (#482, docs/serve.md). --device cpu runs the same build on the CPU.
• The NPU engine on the XDNA 2 NPU, in progress: correctness first, speed next (docs/npu.md).
• ONNX Runtime GenAI (Lemonade's ONNX format) on the CPU, and on the NPU through the user's own install of AMD's Ryzen AI Software, on Linux (#386, docs/onnx.md).
• DwarfStar, for DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next in its own GGUFs, with SSD expert streaming (docs/dwarfstar.md).
• Laya, opt-in, which picks where each request runs (docs/laya.md), and the Project Manager, ZAYA1-8B delegating to expert models (docs/pm.md).
• Every Hugging Face model architecture, mapped to a backend by a registry and kept current by a daily census (docs/registry.md).

Vulkan, ROCm, CUDA, ZINC, ComfyUI, Metal/MLX and Windows were cut from the engine's core (#306) and are not routes of the engine. Lemonade serves them with its own backends and builds (docs/lemonade.md).

Checked on the hardware. Every HRX pin change and every PR labelled hw-gate passes the hardware gate on Strix Halo before merge: it builds our HIP kernels into the server as the release does, runs kernel tests, known answers, KLD against the release pin and long-prompt needles, and checks that every graph of every gate model is on the GPU (#478, #484, docs/hw-gate.md). The runtime keeps one memory budget for every buffer, and after a GPU fault the server refuses new work and exits cleanly, so Lemonade can restart the model (#486). Serve numbers in these docs come from tools/bench.py, which A/B-measures configurations in the same run (docs/bench.md).

Install: through Lemonade. The packages are Lemonade with the onebit recipe and the engine (unpack it and run lemond-onebit), and 1bit OS, a Lemonade appliance on a bootable USB image. On Ubuntu 26.04 the stock lemond from the Lemonade PPA runs the engine through a systemd drop-in, with no patch to Lemonade (#483, docs/lemonade.md). The engine has no standalone package of its own. The first release goes out when every item on the release checklist is done (docs/releases.md).

Direction: HRX with our HIP kernels, plus the NPU engine, embedded into Lemonade rather than Lemonade embedded into the engine.¹ HRX met the decode gates in RFC #213, which removed the engine's Vulkan and ROCm builds. HRX is pinned to AMD's hrx-system and ggml-hrx with our commits on top, kept current by bump-hrx (docs/hrx.md). ZAYA1-8B (Zyphra) runs from our llama.cpp on HRX at about 90 tok/s decode in Q4_K_M, matching transformers. Zyphra's Zamba, Zamba2 and BlackMamba ran only on the removed Vulkan build and are to be ported to HRX. The working engine is being ported from our private development repository; see docs/PORTING.md. Measured results are on the wiki. This repository holds the verified code without the development history.

License

The source code in this repository is Apache-2.0. See LICENSE.

The release binaries (the Linux package and 1bit OS) also contain two proprietary components, compiled in: the 1bit NPU runtime and the 1bit HIP kernels for HRX. They are not in this repository, their source is not published, and they are not under Apache-2.0: they ship under the 1bit Binary License (free to use for any purpose and to pass on unmodified; third-party parts in NOTICE-BINARY). A build from this repository without them has no NPU engine and runs HRX on the public kernels only (NOTICE, docs/releases.md).

Thank you

Osmantic / ODS comes first. ODS introduced me to vibecoding, and that is where all of this started. Without it, this engine would not exist.

The Lemonade team and AMD's developers. Lemonade is where this engine runs. AMD's developers left breadcrumbs all over the place: the XDNA driver and XRT, IRON and Peano, HRX, their tested llama.cpp integration, their issues, their examples. This engine is what following those breadcrumbs built.

The repositories this engine is built on, in order of importance:

#	Repository	What it gives this engine	License
1	amd/xdna-driver	The XDNA 2 NPU kernel driver (amdxdna) the NPU engine runs on	GPL-2.0 (driver)
2	Xilinx/XRT	AMD's NPU runtime and its API, where the engine's NPU work started	Apache-2.0 (userspace)
3	Xilinx/mlir-aie	IRON and aiecc: how our own NPU kernels are written and compiled	Apache-2.0 WITH LLVM-exception
4	Xilinx/llvm-aie	Peano, the C++ compiler for the NPU's AI Engine cores	Apache-2.0 WITH LLVM-exception
5	ggml-org/llama.cpp	GGUF inference on the Radeon iGPU through HRX (AMD's tested pair), and on the CPU	MIT
6	ROCm/hrx-system (our fork 1bit-MONSTER/hrx-system carries two commits)	HRX, AMD's HIP Runtime Extended, behind the HRX0 device	Apache-2.0
7	torvalds/linux	The kernel, with amdxdna and amdgpu in-tree	GPL-2.0 WITH Linux-syscall-note
8	huggingface/tokenizers	Every model's tokenizer.json, byte-exact, behind our C ABI	Apache-2.0
9	NandhaKishorM/laya	The router that decides where each request runs	Apache-2.0
10	ROCm/FastFlowLM	The Q4NX NPU model format and its models on Hugging Face (FastFlowLM/*-NPU2), which the engine's NPU route runs on its own kernels	MIT
11	antirez/ds4 (DwarfStar)	DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next on its own kernels (ROCm on Strix Halo, CUDA, Metal)	MIT

Also built on nlohmann/json (MIT) and yhirose/cpp-httplib (MIT).

Every third-party copyright and license is listed in NOTICE. Each project keeps its own license; nothing here relicenses anyone's work.

Footnotes

1. The HRX + Loom + NPU direction, and embedding the engine into Lemonade rather than Lemonade into the engine, were proposed by geramyL, a moderator on Lemonade's Discord. Thank you. ↩