Small local judges for LLM systems. Give Decima a situation, a question and your options; it returns calibrated probabilities in milliseconds, on a CPU, with nothing leaving your machine.
pip install decima-aiOr try it first in your browser, nothing to install: Playground · Guard page.
$ decima "Your API has been returning 500 errors for an hour and our checkout is down!" --preset triage
decima-base · triage
category technical ███████████████████▌ 0.97
urgency 2 Now: the writer or their busi… ██████████████████▌ 0.92
sentiment negative ███████████████████▍ 0.97
$ decima "rm -rf ~/projects/app" --preset agent-gate
decima-agent · agent-gate
gate deny ███████████████████▋ 0.98
read_only no ▎ 0.01 p(yes)
$ decima "I was charged twice for my subscription" --question "Which team?" --choices billing,technical,sales
decima-base · 81 ms
answer billing ██████████████████▉ 0.94
technical ▉ 0.05
sales ▏ 0.01(Outputs shortened to the top answer per question; the CLI also prints the runners-up.)
New in 2.3: decima-guard, a guard for LLM apps that covers the OWASP LLM Top 10 and common web attacks, in English, Arabic and Persian (Finglish too). Unlike English-first guards, its injection check doesn't block ordinary Arabic and Persian chat: it lets 99.5% of 589 harmless Arabic messages through, where ProtectAI's detector lets 25.1% through. decima-guard 1.2 is much better at injections hidden in documents and tool results: on the 10 public sets of the Horizon-Labs prompt-injection suite its mean goes from 0.650 to 0.827 (BIPIA 0.346 → 0.694 F1, PIArena 0.310 → 0.770, LLMail-Inject 21% → 80% caught). Part of that is in-distribution (it trained on other splits of BIPIA, LLMail-Inject and OR-Bench); it is not a leaderboard rank, and it gave up a little on chat-message injections (0.910 → 0.902). 1.1 halved the wrong blocks on everyday spoken dialect (39.6% → 19.5% of ordinary lines, Casablanca test set) and 1.2 keeps that (20.0%). Try it in your browser, or see examples/guard_chatbot.py.
New: decima-pii v0.2 finds personal data (names, phones, emails,
national IDs, IBANs, cards, addresses ...) in Arabic, Persian, Urdu, Turkish and English, including numbers in Persian
and Arabic-Indic digits. 140M, int8 ONNX, about 13 ms per message on a CPU. On an outside clinical test set it never
trained on (MultiGraSCCo), v0.2 scores 0.910 F1 against 0.647 for OpenAI's privacy filter: Arabic 0.913 vs 0.581,
Persian 0.849 vs 0.508, Turkish 0.928 vs 0.626, English 0.957 vs 0.901 (v0.1: 0.870). On an Arabic test set chosen and
frozen before v0.2 was trained, and run once (Sitr), it scores 0.875 against v0.1's 0.856. Preview; limits on the
model card. It ships inside decima-ai (find and redact), with a
LiteLLM guardrail and a Presidio recognizer. New in 2.5: the
package loads decima-pii v0.2 and pins that model tag, so a later model update never changes your results silently;
revision="v0.1" gives the previous model.
Try it in your browser: the model runs in the tab.
Decima is the judge, not the writer. Put it next to your LLM, agent or extraction pipeline to decide what is safe, what
is true, where a request should go, and when the expensive model is really needed. It speaks TypeSafe's
/v1/systemone format (choice, noul, score), so code written for Jev works after a one-line change, and it ships
in Ollaya.
- Local and cheap: int8 ONNX, about 20–50 ms per decision on one CPU core. No GPU, no API key, no data leaves.
- Calibrated: 0.8 means right about 80 % of the time, so you can threshold, abstain, or escalate to a bigger model.
- Order-proof: shuffling the options never changes the answer (0 %; other open models in our tests: 10–27 %).
- Your options, at call time: any labels, any number. Multilingual backbone; evaluated in 20 languages.
Command line
decima "I was charged twice for my subscription" --question "Which team?" --choices billing,technical,sales
decima --file ticket.txt --preset triage --json
decima presets # triage, routing, moderation, pii, guard, guard-rag, guard-output, guard-web, agent-gate, kg-judge, …Python
from decima import Router
router = Router() # downloads the int8 model on first use
answer = router.predict(
{"message": "Someone logged into my account from another country and changed my password."},
{"team": {"type": "choice", "instructions": "Which team should handle this?",
"criteria": {"billing": "charges and refunds", "tech": "bugs and errors", "security": "account takeover"}},
"urgent": {"type": "noul", "instructions": "The customer needs help within the hour."}},
)
print(answer["answers"]["team"]["choice"], answer["answers"]["urgent"]["noul"]) # security 0.72Personal data (decima-pii): find and redact names, phones, emails, national IDs, IBANs, cards, passports, addresses, dates of birth and IP addresses in Arabic, Persian, Urdu, Turkish and English.
pip install decima-aifrom decima import PII # also importable as DecimaPII
pii = PII.from_pretrained() # amyrmahdy/decima-pii v0.2, int8 ONNX, 148 MB, downloaded once
print(pii.redact("اسمي محمود الشمري ورقم جوالي ٠٥٥١٢٣٤٥٦٧")) # اسمي [PERSON_NAME] ورقم جوالي [PHONE]decima pii "سلام، من سارا احمدی هستم. کد ملی من ۰۰۱۲۳۴۵۶۷۸ است." # spans, then the redacted text
cat letter.txt | decima pii --redact-only # or --json for {"spans": [...], "redacted": "..."}pii.find(text) returns [{"start", "end", "label", "text"}]. This is not the pii preset: the preset asks
decima-base a yes/no question ("does this text contain personal data, and which kind?"); decima.pii returns the exact
spans so you can mask them. The model is pinned to tag v0.2.1 (decoder v2.1: a number right after a phone word such as جوالي counts as a phone); PII.from_pretrained(revision="v0.1") or
decima pii --revision v0.1 gives the previous model with its exact output. In front of an LLM:
LiteLLM guardrail · Presidio recognizer. pip install decima-pii
installs the same package with a decima-pii command.
HTTP server (TypeSafe-compatible, binds to 127.0.0.1)
decima serve --model decima-agent --port 11436The official typesafe-sdk works against it unchanged: TypeSafeClient(api_key="local", base_url="http://127.0.0.1:11436").
Integrations: LiteLLM proxy guardrail (decima-guard behind LiteLLM's decision_model guardrail, no code) ·
LiteLLM PII masking (decima-pii masks Arabic/Persian/Urdu/Turkish/English personal data before the model sees it) ·
Presidio recognizer (decima-pii under Presidio's entity names) · Claude Code hooks (secret and command gates, rules first, fail closed) ·
MCP server (decima mcp) · LangChain / LangGraph ·
Docker · examples · Colab quickstart
| Preset | Decides | Model |
|---|---|---|
triage |
team, urgency, sentiment of a ticket or message | decima-base |
routing |
cheapest LLM tier that can answer well: small / medium / frontier | decima-base |
moderation |
which policy a post breaks, and how badly | decima-base |
pii |
is there personal data, and of which kind (yes/no; to find and mask the values, use decima-pii) | decima-base |
guard |
prompt injection or jailbreak, harmful request, toxicity, which OWASP LLM risk | decima-guard |
guard-rag |
indirect injection in a document or tool result; answer grounded in the source; tool call beyond the request | decima-guard |
guard-output |
the reply leaks the system prompt or secrets; output unsafe for HTML / SQL / shell; harmful reply | decima-guard |
guard-web |
is this HTTP input an attack, and which kind (SQL injection, XSS, path traversal, …) | decima-guard |
guardrails |
prompt injection or jailbreak, personal data, toxicity before a prompt reaches an LLM | decima-agent |
agent-gate |
allow / ask / deny a shell command, secret in an edit, model tier for a task | decima-agent |
kg-judge |
is a fact asserted, hypothetical, denied or planned; are two records one entity; how two things relate | decima-agent |
Presets are JSON files in decima/presets; write your own with any questions.
| Model | Size | For |
|---|---|---|
| decima-agent 2.3 | 321M | agent gates and knowledge-graph judgments; states up to 8,192 tokens |
| decima-guard 1.2 preview | 321M | OWASP LLM Top 10 and web-attack checks, injections in documents and tool results; English, Arabic, Persian |
| decima-pii 0.2 preview | 140M | personal-data detection and redaction; Arabic, Persian, Urdu, Turkish, English |
| decima-guard-small preview | 140M | the same checks, faster, a few points lower |
| decima-base 2.0 | 321M | general decisions, multilingual |
| decima-base-clean | 321M | no non-commercial training data; truly zero-shot on BTZSC |
| decima-small 1.1 | 122M | the smallest CPUs and the browser |
Honest numbers, on data the models never trained on unless stated.
Against the same-size open models, all re-run by us on one harness:
| decima-base | decima-small | Laya | Laya-multilingual | Kev-0.8B | |
|---|---|---|---|---|---|
| Parameters | 321M | 122M | 421M | 322M | 753M |
| jabr/classifier-benchmark v2, 49 tasks (zero-shot) | 0.673 | 0.616 | 0.583 | — | — |
| Kev's published protocol, 8 suites | 0.790 | 0.762 | 0.681 | 0.580 | 0.794 |
| Answer changes when options are shuffled | 0 % | 0 % | 27 % | 21 % | 10 % |
Decima trained on the train splits of some suites in Kev's protocol (test rows differ); jabr is zero-shot. The frontier is much higher: Jev scores 0.966 on jabr v2, and 4B–27B open models (Kev, SemIf-OpenJev) report Jev-level accuracy on their own tests. Decima wins on cost, speed and locality, not on hard open-ended judgment.
As a guard (decima-guard, int8, CPU), on frozen held-out sets that are public with every prediction (decima-guard-bench):
| decima-guard | ProtectAI prompt-injection-v2 | |
|---|---|---|
| Prompt injection / jailbreak, 6,532 examples: accuracy | 0.902 | 0.784 |
| Attacks missed, of 2,290 | 515 | 431 |
| False alarms on 4,242 harmless messages | 125 | 979 |
| Ordinary Arabic chat messages, injection check: let through (589) | 99.7% | 25.1% |
| Ordinary Persian chat messages, injection check: let through (205) | 100% | 20.5% |
Against Qwen3Guard-Gen-4B on the same rows: injection 0.902 vs 0.767, harmful requests 0.878 vs 0.911. On injections hidden in documents (Horizon-Labs suite, 10 public sets) it catches 53% of BIPIA's attacks and 63% of PIArena's, up from about 1 in 5 in 1.1. It still misses a fifth of real attacks translated into Arabic dialects, and on everyday spoken dialect (Casablanca test set) the full guard still blocks 20.0% of ordinary lines (1.0: 39.6%), mostly on the toxic question. Keep hard rules first, and see the limitations.
As a local stand-in for Jev in a published knowledge-graph pipeline (William Lyon's notebooks, same questions):
| Jev | decima-agent 2.2 (CPU) | |
|---|---|---|
| Planted "maybe / denied / planned" facts caught | 8/11 | 11/11 |
| Entity matching (Beer, 450 pairs), AUC | 0.992 | 0.972 |
| Assertion gate over extracted edges, F1 | 0.404 | 0.368 |
| Extraction ∪ relation selection, gated, F1 | 0.343 | 0.364 |
As a gate for a coding agent, on 40 cases written after training with no copies in any training data: decima-base 0.60 → decima-agent 2.2 0.85. Keep hard rules first; the hooks do.
It also plays games from text sensors, one decision per move: Snake, 42 apples a game (6 of 10 games alive at the 600-move limit); a side-scrolling platformer, 9 of 10 levels.
- Fuzzy, knowledge-heavy or multi-step decisions belong to a bigger model. Use Decima's confidence to know when.
- States are limited to 512 tokens (decima-base, decima-small), 2,048 (decima-guard) or 8,192 (decima-agent); longer input is refused, never silently cut.
- A model can be confidently wrong. For safety gates, keep rules first and Decima second.
- decima-base trained on some non-commercial datasets; see release/LICENSING.md before selling a product built on it.
Every system's predictions, including Decima's and the competitors', are downloadable in decima-bench-predictions, with the scores and a text-free manifest of every item; its card has the commands to re-score every table without running a model. Decima's predictions can also be regenerated from the released weights and the frozen item builders:
uv sync
# item sets (calibration items per suite: kev/laya 300, decima 500, typed 1000, others none)
uv run python -m bench.items --suites 'kev/*' --calib 300 --out runs/items/kev.jsonl
uv run python -m bench.items --suites 'laya/*' --calib 300 --out runs/items/laya.jsonl
uv run python -m bench.items --suites 'decima/*' --calib 500 --out runs/items/decima.jsonl
uv run python -m bench.items --suites 'jevbench/*' 'typed/*' --calib 1000 --out runs/items/jevtyped.jsonl
uv run python -m bench.items --suites 'btzsc/*' --out runs/items/btzsc.jsonl
# predictions (int8 ONNX, 1 thread) and scores, one set at a time
for s in kev laya decima jevtyped btzsc; do
uv run python -m bench.predict --items runs/items/$s.jsonl --system onnx --checkpoint export/v1i-int8 \
--threads 1 --out runs/preds/$s--decima.jsonl
M=acc; [ $s = btzsc ] && M=macro_f1
uv run python -m bench.score --items runs/items/$s.jsonl --metric $M --preds decima=runs/preds/$s--decima.jsonl
done
# audit numbers: overlap, confidence intervals, headline calibration
uv run python scripts/audit/overlap.py && uv run python scripts/audit/bootstrap.py && uv run python scripts/audit/calibration.pyexport/v1i-int8 is the int8 ONNX export; the same files are onnx/int8/ in the
model repository
(hf download amyrmahdy/decima-small --include 'onnx/int8/*' --local-dir decima-small, then pass
--checkpoint decima-small/onnx/int8). Parameter counts for every system:
uv run python scripts/audit/params.py. Competitor predictors (Laya, Kev) run in their own environments: bench/external/. Training recipe,
data builders and every experiment — including the ones that failed — are in the
technical report.
decima/ |
ONNX runtime, Router, presets, CLI, server, MCP, LangChain helpers; model, export, int8 quantisation |
integrations/ |
Claude Code hooks, MCP, LangChain |
examples/ |
runnable examples and a Colab quickstart |
train/ |
training (distillation + gold labels, crash-safe checkpoints) |
teacher/ |
teacher data generation and gold-label builders |
bench/ |
evaluation harness: one item format for every system, scorer, speed |
docs/ |
technical report, evaluation audit, benchmarks |
Code and model: Apache-2.0. Synthetic dataset: CC BY 4.0. Training-data licences
(some non-commercial) are disclosed in the model card and release/LICENSING.md. Predictions dataset:
CC BY 4.0.
If you use Decima, please cite it (GitHub's "Cite this repository" reads CITATION.cff):
@misc{madani2026decima,
title = {Decima: A Small, Calibrated, Permutation-Invariant Decision Model via Late-Interaction Distillation},
author = {Madani, A. M.},
year = {2026},
howpublished = {\url{https://github.com/amyrmahdy/decima}},
note = {Models: \url{https://huggingface.co/amyrmahdy}}
}Questions and problems: GitHub issues.