OpenBonsai

Open reverse-engineering research for extreme low-bit LLM conversion

---

OpenBonsai is an open research program for reproducing the conversion method behind PrismML's Bonsai models: transforming an ordinary pretrained language model into an end-to-end groupwise binary or ternary model without destroying its useful behavior.

This repository is not a reimplementation of PrismML's proprietary method and does not claim access to it. It contains:

What you get	Where
Evidence ledger & ranked hypotheses	docs/
Released-weight forensics	src/openbonsai/forensics.py
Reference quantizers & matrix lab	src/openbonsai/quantize.py, matrix_lab.py
Progressive QAT / distillation scaffold	src/openbonsai/qat.py, scripts/train_hf.py
1.7B forensic report	results/1.7b_forensics.md

The most important finding

The released weights rule out ordinary round-to-nearest post-training quantization.

Our 1.7B comparison against Qwen3-1.7B finds that Bonsai:

• changes a large fraction of discrete signs
• deliberately moves farther from the base in Euclidean weight space than naïve binarization
• also moves some higher-precision norm parameters
• yet retains much more task capability than naïve 1-bit PTQ

The likely optimization target is therefore function space (teacher logits, hidden states, and/or task loss) — not weight reconstruction.

Leading reconstruction hypothesis

1. Activation / Hessian-aware discrete initialization
2. A gradual FP → ternary → binary curriculum with hard-forward / smooth-backward QAT
3. Global teacher-logit + intermediate-state distillation
4. Learned group-scale compensation, folded into one FP16 scale at export
5. Explicit activation / KV-noise robustness training
6. A curvature-aware optimizer or preconditioner to stabilize the extreme discrete transition

Before spending compute, read the executive brief, hypothesis matrix, forensic methodology, and concrete recipe.

Repository map

OpenBonsai/
├── docs/                  evidence, hypotheses, plan, eval, recipe
├── src/openbonsai/
│   ├── formats.py         standalone GGUF / safetensors readers
│   ├── forensics.py       released-weight comparison pipeline
│   ├── quantize.py        binary / ternary + curvature-aware methods
│   ├── matrix_lab.py      curvature-aware matrix laboratory
│   └── qat.py             progressive-QAT building blocks (optional torch)
├── scripts/train_hf.py    small-scale HF distillation runner
├── configs/               phase-0 experiment configs
├── tests/                 dependency-light unit tests
└── results/               generated 1.7B forensic report + JSON

Quick start

Reproduce the artifact forensics

The comparison downloads about 4.2 GB. The second Qwen safetensors shard is a duplicate tied LM head and is intentionally not needed.

python -m venv .venv
source .venv/bin/activate
pip install -e .
./scripts/fetch_reference_artifacts.sh
./scripts/run_forensics.sh

Then inspect results/1.7b_forensics.md.

Run the tests

PYTHONPATH=src python -m unittest discover -s tests -v

Run a pilot QAT experiment

Install optional dependencies and provide a JSONL calibration/training corpus with a text field:

pip install -e '.[training]'
accelerate launch scripts/train_hf.py configs/phase0_ternary.json

The included runner is a hypothesis-discrimination harness, not a claim of production-scale training. A real 1.7B+ run should shard the student, cache teacher targets or isolate the teacher on separate workers, and checkpoint optimizer state.

Project rules

• Compare against exact same-base RTN, GPTQ/AWQ, and full-precision checkpoints
• Report both theoretical information rate and actual shipped / resident bytes
• Quantize embeddings and the LM head; disclose every exception
• Evaluate the hard exported checkpoint, never only fake-quant shadow weights
• Use identical prompts, decoding, context, and scoring across precision variants
• Treat every PrismML claim as a claim until independently reproduced

Documentation index

Doc	Topic
00 — Executive brief	High-level findings
01 — Evidence ledger	Sourced claims & observations
02 — Hypotheses	Ranked, falsifiable matrix
03 — Research plan	Experiment roadmap
04 — Evaluation protocol	Controls & metrics
05 — Implementation design	System design
06 — Source notes	External references
07 — Forensic methodology	Weight analysis method
08 — Training recipe v0	Concrete conversion recipe
09 — Roadmap	Near-term priorities

---

_{Apache-2.0 · Third-party model licenses and dataset terms still apply}