Yes. I would tighten the definition substantially before handing it to an implementation agent.
Your phrase “ultra-fast, sophisticated and intelligent, deterministic-first decision tree classifier” captures the intent, but decision tree is too narrow technically. It implies a conventional ML tree such as CART/Random Forest, whereas what we're actually designing is more powerful:
OpenCodifier is a local-first decision runtime that combines deterministic rules, hierarchical decision graphs, lightweight NLP/semantic models, probabilistic classification, candidate narrowing, verification, caching, and calibrated confidence to turn unstructured input into small, machine-actionable decisions—without generating natural-language answers.
Or, more product-oriented:
OpenCodifier is an ultra-fast, deterministic-first AI decision engine. It progressively understands, filters, narrows, scores, verifies, and selects—using the cheapest reliable mechanism at each stage—so expensive generative AI only runs when necessary.
That distinction is important.
text → classifier → label
UNSTRUCTURED INPUT
│
▼
┌───────────────────┐
│ Normalize / Parse │
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ Deterministic │
│ Rules / Filters │
└─────────┬─────────┘
│
candidate set
│
▼
┌───────────────────┐
│ Fast NLP / │
│ Semantic Analysis │
└─────────┬─────────┘
│
narrowed set
│
▼
┌───────────────────┐
│ Decision Model │
│ Choice/Score/Bool │
└─────────┬─────────┘
│
confidence
│
┌───────┴───────┐
│ │
certain uncertain
│ │
▼ ▼
accept verifier
│
┌──────┴──────┐
│ │
agree disagree
│ │
▼ ▼
accept abstain
/ escalate
And that entire thing can be represented as a Decision Graph.
This is much closer to what we should build.
The research reinforces this architecture. Jev itself is explicitly positioned as a typed-decision system rather than a chat model: unstructured state goes in, typed probabilistic decisions come out; its public primitives are essentially choice, score, and boolean/noul, with calibrated probabilities and parallel evaluation. (TypeSafe AI)
The open ecosystem has independently converged on several useful pieces: Laya uses non-autoregressive typed decisions; Von targets calibrated local decisions; open-jev demonstrates one-pass candidate scoring with a normal language model; and jevlike/related projects demonstrate candidate-conditioned scoring and learned decision heads. (GitHub)
So OpenCodifier should not be "our Jev clone." It should be the broader deterministic-first decision architecture that can implement Jev-like behavior as one mode.
Status: Greenfield implementation plan Language: Rust Primary targets: Windows, Linux, macOS, WASM License: Apache-2.0 recommended Primary inference target: ONNX Runtime / ONNX models Secondary/future runtime: Burn Primary integration: Amortyx Protocol compatibility: OpenCodifier native + Jev/System One + OpenAI + Anthropic schemas Default behavior: Local, offline, private, no telemetry
Build OpenCodifier as an open-source, local-first decision engine that transforms unstructured state into narrow, machine-actionable decisions.
OpenCodifier must:
-
Prefer deterministic logic before ML inference.
-
Narrow candidate sets before expensive semantic inference.
-
Use lightweight NLP/semantic models for unresolved decisions.
-
Support typed probabilistic decisions:
- Choice
- Boolean
- Score
-
Support dynamically supplied candidates.
-
Support multiple decision stages.
-
Support confidence, calibration, abstention, and verification.
-
Cache reusable work.
-
Batch independent decisions.
-
Produce machine-readable outputs without generated prose.
-
Support Jev-compatible decision schemas.
-
Accept OpenAI and Anthropic-style structured schemas.
-
Run completely locally and offline.
-
Be embeddable as a Rust library.
-
Expose CLI, HTTP, MCP, and WASM interfaces.
-
Remain useful without a neural model for deterministic workloads.
-
Allow the underlying model to be replaced without changing the decision API.
The project must optimize for:
correctness → determinism → latency → resource efficiency → privacy → usability → extensibility.
OpenCodifier is NOT:
- a chatbot
- a generative LLM
- an LLM wrapper
- an agent framework
- a vector database
- merely a classifier
- merely a decision tree
- a replacement for GPT/Claude/etc.
OpenCodifier IS:
A deterministic-first decision runtime that progressively reduces uncertainty and candidate space until it can make, verify, or safely defer a machine-readable decision.
Its fundamental operation is:
STATE
+
DECISION QUESTION
+
CANDIDATE SPACE
↓
DECISION
Examples:
Which model should process this request?
Does this request require external information?
How difficult is this task?
Which tool should be used?
Should this tool call be allowed?
Does this cached result apply?
Which skill should be activated?
Does this context belong in the active context window?
Should this request escalate to a larger model?
Which of these 50 documents are relevant?
Does this result require verification?
OpenCodifier must use the cheapest reliable mechanism first.
Preferred execution order:
1. Exact deterministic rule
2. Cached decision
3. Metadata/filtering
4. Lexical matching
5. Lightweight semantic similarity
6. Small classifier
7. Candidate-conditioned decision model
8. Secondary verifier
9. Larger local model
10. External generative model
Never invoke a more expensive layer if a previous layer can establish the decision with sufficient confidence.
This is the central optimization principle of OpenCodifier.
Use the following terminology consistently.
A machine-actionable result.
One possible answer.
Example:
qwen
glm
kimi
claude
The semantic decision being made.
The input/context against which the question is evaluated.
A directed acyclic graph of deterministic operations and decision nodes.
A node that produces one or more typed decisions.
A deterministic threshold or condition controlling whether execution proceeds.
A secondary model or deterministic mechanism used when the primary decision is uncertain or high-risk.
Explicitly refusing to make a sufficiently reliable decision.
Abstention is a successful outcome, not an error.
Calibrated estimate associated with the decision.
Do not treat raw neural softmax probability as calibrated confidence.
V1 must support three primary decision types.
Select one candidate from a dynamic candidate list.
{
"type": "choice",
"question": "Which model should handle this request?",
"candidates": {
"qwen": "General coding and reasoning",
"glm": "Complex reasoning",
"kimi": "Very large context"
}
}Response:
{
"type": "choice",
"choice": "qwen",
"probabilities": {
"qwen": 0.78,
"glm": 0.17,
"kimi": 0.05
},
"confidence": 0.78
}Equivalent to Jev's noul concept.
{
"type": "boolean",
"question": "Does this request require external information?"
}Response:
{
"type": "boolean",
"value": true,
"probability": 0.93,
"confidence": 0.93
}Ordered decision.
{
"type": "score",
"question": "How difficult is this request?",
"levels": [
"trivial",
"easy",
"moderate",
"difficult",
"expert"
]
}Return:
{
"type": "score",
"value": 3.8,
"level": "difficult",
"probabilities": {
"trivial": 0.01,
"easy": 0.04,
"moderate": 0.17,
"difficult": 0.61,
"expert": 0.17
}
}The engine should retain the complete distribution rather than only the selected answer.
Everything must normalize into a canonical Rust representation.
pub enum DecisionQuestion {
Choice(ChoiceQuestion),
Boolean(BooleanQuestion),
Score(ScoreQuestion),
}Common structure:
pub struct DecisionRequest {
pub state: State,
pub questions: Vec<DecisionQuestion>,
pub policy: DecisionPolicy,
pub metadata: RequestMetadata,
}Response:
pub struct DecisionResponse {
pub answers: Vec<DecisionAnswer>,
pub confidence: ConfidenceReport,
pub trace: Option<DecisionTrace>,
pub metrics: DecisionMetrics,
}Do not make OpenAI, Anthropic, or Jev formats the internal representation.
They are adapters.
External Schema
│
┌─────────────┼─────────────┐
▼ ▼ ▼
OpenAI Anthropic Jev
│ │ │
└─────────────┼─────────────┘
▼
Canonical OC IR
│
▼
OpenCodifier Engine
OpenAI structured outputs and Anthropic tool schemas must be translated into the internal IR rather than executed directly. This keeps the core independent of vendor-specific semantics. OpenAI documents JSON Schema structured outputs, while Anthropic exposes JSON-schema-based input_schema for tools. Validate against the current vendor specifications during implementation rather than assuming all JSON Schema features are equivalent.
Support the decision-oriented subset first.
type
enum
const
boolean
integer
number
oneOf
anyOf where safely reducible
required
description
title
minimum
maximum
enum
↓
Choice
boolean
↓
Boolean
ordered enum
↓
Score
numeric bounded range
↓
Score / bucketed Score
discriminator
↓
Choice
A free-form string:
{
"explanation": {
"type": "string"
}
}must not become an OpenCodifier task.
Return:
unsupported_generation_field
The generative model remains responsible for prose.
This is the heart of OpenCodifier.
A graph consists of nodes.
V1 node types:
input
normalize
rule
cache
filter
lexical
embedding
classify
score
boolean
choice
verify
fuse
threshold
branch
retrieve
rerank
transform
output
Example:
INPUT
│
▼
NORMALIZE
│
▼
CACHE?
├── HIT ───────────────► OUTPUT
│
└── MISS
│
▼
FILTER MODELS
│
▼
CLASSIFY TASK
│
├── coding
│
├── research
│
└── general
│
▼
SCORE COMPLEXITY
│
▼
CHOOSE MODEL
│
▼
CONFIDENCE GATE
│
├── ACCEPT ────────► OUTPUT
│
└── VERIFY
│
├── AGREE ─► OUTPUT
│
└── DISAGREE → ABSTAIN/ESCALATE
Graphs must be declarative and serializable.
Do NOT create an embedded scripting language in V1.
Implement this before sophisticated ML.
The engine must support:
- topological execution
- dependency tracking
- parallel independent nodes
- short-circuiting
- conditional branches
- result propagation
- timeout
- cancellation
- cache lookup
- trace generation
Example:
nodes:
normalize:
type: normalize
cache:
type: cache
depends_on: [normalize]
context:
type: score
depends_on: [normalize]
tools:
type: boolean
depends_on: [normalize]
route:
type: choice
depends_on:
- context
- toolscontext and tools can execute concurrently.
Implement a rule engine.
Example:
rules:
- when:
context_tokens:
gt: 100000
set:
requires_long_context: true
- when:
modality:
contains: image
set:
requires_vision: true
- when:
privacy:
equals: local_only
exclude_models:
- cloudRules must execute before ML wherever possible.
This is one of OpenCodifier's major differentiators.
Example:
Registered models:
50
Deterministic filtering:
50 → 22
Capability filtering:
22 → 11
Embedding similarity:
11 → 6
Decision model:
6 → 2
Verifier:
2 → 1
Never feed all candidates into the semantic model if deterministic constraints can eliminate them first.
Do not build only a conventional fixed-label classifier.
The system must support:
STATE
+
QUESTION
+
N dynamically supplied candidates
The candidate set must be runtime-defined.
This is one of the most important lessons from the current Jev-like ecosystem. open-jev demonstrates scoring pre-written options in one forward pass, while the jevlike-derived design uses a learned option/context scoring head. (GitHub)
The architecture should therefore support:
context representation
+
candidate representation
↓
candidate score
↓
softmax
rather than:
context
↓
fixed classifier
↓
class 1 / class 2 / class 3
Build the first OpenCodifier model as a small candidate-conditioned encoder.
Target characteristics:
non-autoregressive
small
CPU friendly
quantizable
ONNX exportable
WASM-capable
dynamic candidate count
batch-friendly
The model should not generate text.
It should return scores.
Potential architecture:
Tokenizer
│
▼
Context Encoder
│
├─────────────────┐
│ │
▼ ▼
Context tokens Candidate encoder
│ │
└────────┬────────┘
▼
Cross-attention
│
▼
scalar logit
Repeat candidate scoring in a batch.
Training is not the initial blocker.
First establish the inference contract.
Then construct:
teacher data
↓
labeled decision examples
↓
student model
↓
calibration
↓
evaluation
Potential teacher sources:
- synthetic deterministic rules
- human labels
- existing LLMs
- successful Amortyx routing outcomes
- public classification datasets
- Jev-like benchmark tasks
- specialized task datasets
Use teacher LLMs only during dataset construction.
The deployed OpenCodifier must not require those APIs.
Use a simple JSONL representation.
{
"state": "...",
"question": {
"type": "choice",
"text": "Which model should process this?"
},
"candidates": [
{
"id": "qwen",
"description": "..."
},
{
"id": "glm",
"description": "..."
}
],
"label": "qwen"
}For confidence-aware training, optionally retain:
{
"label": "qwen",
"quality": 0.92,
"teacher_confidence": 0.88
}Do not blindly train on teacher confidence as ground truth.
Calibration is mandatory.
Measure:
ECE
Brier score
NLL
reliability
selective accuracy
coverage
Implement temperature scaling first.
Later support:
isotonic regression
conformal prediction
task-specific calibration
The current Jev/open ecosystem strongly reinforces this requirement. Jev explicitly emphasizes calibrated decisions; Von reports Brier/CE-based calibration; poorjev demonstrates temperature scaling plus conformal abstention. (TypeSafe AI)
Do not expose only:
confidence: 0.91
Internally track:
top_probability
margin
entropy
calibration
OOD_score
candidate_count
model_quality
verifier_agreement
Example:
{
"top_probability": 0.82,
"margin": 0.57,
"entropy": 0.31,
"calibrated_confidence": 0.79,
"ood_score": 0.06,
"verifier_agreement": true
}Implement a cascade.
Primary classifier
│
▼
Uncertainty gate
│
├── confident → accept
│
└── uncertain
│
▼
verifier
│
┌─────┴─────┐
▼ ▼
agree disagree
│ │
▼ ▼
accept abstain
Do NOT execute both models on every request.
Trigger verification when:
entropy > threshold
OR margin < threshold
OR confidence < threshold
OR OOD > threshold
OR high-risk action
OR irreversible action
OR candidate ambiguity
The verifier should preferably use a meaningfully different architecture or training source to reduce correlated errors.
Every graph can classify actions:
LOW
MEDIUM
HIGH
CRITICAL
Examples:
Choose formatting style → LOW
Choose AI model → LOW/MEDIUM
Select tool → MEDIUM
Modify source code → MEDIUM/HIGH
Delete data → HIGH
Production deployment → CRITICAL
Higher-risk decisions require stronger confidence or verification.
This must be deterministic policy, not model preference.
Abstention is a core feature.
Possible outputs:
ACCEPT
VERIFY
ABSTAIN
ESCALATE
NO_VALID_CANDIDATE
Never force the classifier to select an answer merely because a candidate exists.
If ten questions use the same state:
Q1 task type
Q2 complexity
Q3 tool requirement
Q4 context size
Q5 privacy
Q6 model
...
the engine must batch them.
Jev explicitly emphasizes parallel evaluation; Laya likewise describes typed questions evaluated in a single forward pass. (TypeSafe AI)
OpenCodifier should make this automatic.
Implement multiple caches.
Key:
hash(
normalized_state
+
question
+
candidate_set
+
model_version
+
calibration_version
+
policy_version
)
Cache parsed schemas.
Cache repeated candidate tokenization.
Cache candidate embeddings/representations.
Optional.
Use similarity only when policy permits approximate reuse.
Never silently reuse a semantic result where correctness requirements demand exact evaluation.
Embeddings should be included as an optional subsystem, not the core identity.
Useful for:
- semantic cache
- candidate retrieval
- skill selection
- memory selection
- duplicate detection
- document selection
- model capability matching
- graph routing
Rust fastembed-rs is a useful reference because it already provides local embedding and reranking support and includes quantized embedding models. (GitHub)
Architecture:
OpenCodifier Core
│
└── optional semantic feature
├── embeddings
├── similarity
├── retrieval
└── reranking
V1 must work without:
- vector DB
- database server
- Python
- Docker
- cloud
- API key
For local semantic retrieval, investigate:
- USearch
- embedded HNSW
- simple memory-resident vectors
For lexical retrieval, Tantivy is an excellent Rust reference and already provides BM25, low startup overhead, SIMD-related optimizations, and mmap-based storage. (GitHub)
Expose a reranker interface:
trait Reranker {
fn rerank(
&self,
query: &str,
candidates: &[Candidate]
) -> Result<Vec<ScoredCandidate>>;
}Do not make reranking mandatory in V1.
Pipeline:
50 candidates
↓
metadata
↓
BM25 / embedding
↓
10
↓
reranker
↓
3
↓
decision model
fastembed-rs already provides a useful reference implementation and supports multiple reranking models. (GitHub)
Implement a runtime abstraction.
trait InferenceBackend {
fn load(&self, model: &ModelArtifact) -> Result<ModelHandle>;
fn infer(
&self,
model: &ModelHandle,
input: &InferenceInput
) -> Result<InferenceOutput>;
}V1:
ONNX Runtime
Future/parallel:
Burn
Do not hard-code OpenCodifier to ONNX APIs.
ONNX provides a portable model interchange layer.
Burn is particularly interesting because its Rust ecosystem supports CPU/GPU/WebAssembly backends and ONNX import. Its current ONNX tooling can generate native Burn Rust code and target WebAssembly, CUDA and WebGPU, although operator coverage and portability must still be tested against the exact OpenCodifier model. (GitHub)
Recommended architecture:
OpenCodifier Model
│
├── ONNX artifact
│
├── native ORT
│
└── optional Burn-native artifact
Browser deployment is a first-class target.
ONNX Runtime Web supports WASM CPU execution and browser GPU execution through WebGPU/WebNN paths; it also supports SIMD and threaded WASM builds. (ONNX Runtime)
However, do not assume the Rust ort crate itself gives us a clean browser build.
Therefore define:
Native:
Rust → ORT
Browser:
Rust/WASM core
+
ONNX Runtime Web bridge
or
Burn WASM backend
The browser build must not require a server.
Use Hugging Face Tokenizers as the initial tokenizer foundation.
It is itself implemented in Rust and is designed for high-performance tokenization and production use. (GitHub)
Tokenizer must be encapsulated behind:
trait Tokenizer {
fn encode(&self, text: &str) -> Result<Encoding>;
}This prevents future model changes from contaminating the engine.
MCP is a first-class interface.
Use the official Rust rmcp SDK. The current SDK implements the stable MCP 2026-07-28 specification and includes server/client functionality. (GitHub)
Command:
opencodifier mcp serveInitial tools:
codify_decide
codify_batch
codify_graph
codify_validate
codify_explain
codify_classify
codify_score
codify_verify
Do not expose chain-of-thought.
codify_explain returns an execution trace:
{
"node": "model_selection",
"candidate_count_before": 18,
"candidate_count_after": 5,
"confidence": 0.91,
"verified": false,
"cache_hit": false
}OpenCodifier should eventually consume MCP tool schemas.
Input:
MCP server
↓
tool schemas
↓
candidate registry
↓
capability index
Then OpenCodifier can decide:
Which tool?
Is a tool required?
Which tool is safest?
Does this tool require confirmation?
Can another tool satisfy the request?
This should be implemented after the core MCP server works.
Ship a skills directory:
skills/
routing/
tool-selection/
tool-gating/
context-pruning/
model-selection/
escalation/
verification/
memory-selection/
A skill describes how an AI should use OpenCodifier.
It should not contain large amounts of duplicated decision logic.
Example:
When deciding which model to use:
1. Send request to OpenCodifier.
2. Provide candidate model capabilities.
3. Apply deterministic constraints first.
4. Inspect confidence.
5. Verify low-confidence decisions.
6. Escalate if OpenCodifier abstains.
Ship ready-to-use graphs.
recipes/
model-routing
task-classification
tool-selection
tool-gating
context-pruning
cache-eligibility
skill-selection
memory-selection
escalation
verification
document-relevance
code-review-risk
Usage:
opencodifier recipe install model-routingThis is essential for adoption.
Commands:
opencodifier init
opencodifier decide
opencodifier batch
opencodifier graph
opencodifier graph validate
opencodifier graph run
opencodifier model list
opencodifier model install
opencodifier model remove
opencodifier model inspect
opencodifier recipe list
opencodifier recipe install
opencodifier benchmark
opencodifier serve
opencodifier mcp serve
opencodifier doctor
Example:
opencodifier decide \
--schema model-routing.json \
--state request.jsonOutput:
{
"decision": "qwen",
"confidence": 0.91
}Human-readable mode:
Decision: qwen
Confidence: 91%
Verified: no
Latency: 3.8 ms
Cache: miss
Candidates: 8 → 3
Primary:
POST /v1/decide
POST /v1/batch
POST /v1/graph/run
POST /v1/validate
Metadata:
GET /v1/health
GET /v1/models
GET /v1/capabilities
Compatibility:
POST /v1/systemone
The compatibility endpoint must emulate the supported Jev/System One schema, while the native /v1/decide API remains the canonical OpenCodifier API.
Amendment (2026-10-15): shipped reality — there is no /v1/systemone
route. Jev/System One compatibility lives in opencodifier-schema as a
wire adapter (Jev in, canonical IR out), and the HTTP surface is
POST /v1/decide (+ /v1/chat/completions, /v1/messages,
/v1/batch). The adapter-normalizes-into-the-IR rule supersedes a
dedicated compat endpoint; see docs/DECISIONS.md D12.
Default:
opencodifier serveMust bind locally by default:
127.0.0.1
Do not expose network access unless explicitly requested.
Example:
opencodifier serve --host 0.0.0.0must require an explicit flag.
Default:
No telemetry
No analytics
No cloud calls
No account
No API key
No remote logging
No automatic uploads
All model downloads must be explicit.
Model integrity:
SHA-256
signature
manifest
license
Use a Rust workspace:
opencodifier/
├── Cargo.toml
├── Cargo.lock
├── README.md
├── LICENSE
├── SECURITY.md
├── CONTRIBUTING.md
├── CHANGELOG.md
│
├── crates/
│ ├── opencodifier-core/
│ ├── opencodifier-schema/
│ ├── opencodifier-engine/
│ ├── opencodifier-runtime/
│ ├── opencodifier-model/
│ ├── opencodifier-http/
│ ├── opencodifier-mcp/
│ ├── opencodifier-cli/
│ └── opencodifier-wasm/
│
├── adapters/
│ ├── openai/
│ ├── anthropic/
│ └── jev/
│
├── recipes/
├── skills/
├── models/
├── benchmarks/
├── fixtures/
├── examples/
├── tests/
└── docs/
Keep dependencies directional.
core must not depend on:
HTTP
MCP
CLI
Tokio
specific ML runtime
- Initialize Rust workspace.
- Select Apache-2.0.
- Add README.
- Add security policy.
- Add contribution policy.
- Add CODEOWNERS if appropriate.
- Add issue templates.
- Add CI.
- Configure rustfmt.
- Configure Clippy.
- Configure cargo-deny.
- Configure dependency auditing.
- Add MSRV policy.
- Add semantic versioning policy.
Every PR:
cargo fmt --check
cargo clippy --all-targets --all-features
cargo test --all
cargo test --doc
cargo deny check
Implement:
DecisionRequest
DecisionQuestion
DecisionAnswer
Candidate
ScoreLevel
DecisionPolicy
Confidence
DecisionTrace
Implement serialization.
Tests:
Choice round trip
Boolean round trip
Score round trip
empty candidates
duplicate candidates
invalid score ordering
missing question
malformed JSON
Deliverable:
cargo testwith no ML dependency.
Implement adapters:
native OpenCodifier
OpenAI
Anthropic
Jev
Create fixture suites.
Example:
fixtures/openai/
fixtures/anthropic/
fixtures/jev/
Every fixture must normalize into identical canonical IR where semantics are equivalent.
Deliverable:
schema → IR → schema
round-trip/compatibility tests.
Implement:
rules
filters
candidate pruning
branching
thresholds
DAG scheduling
parallel independent nodes
short circuit
Build a fake deterministic classifier:
MockClassifier
This allows graph testing before the real model exists.
Deliverable:
complete functioning Decision Graph engine
without ML.
Implement:
exact cache
schema cache
candidate cache
tokenization cache
Requirements:
- deterministic keys
- model versioning
- policy versioning
- TTL
- max size
- persistence optional
- in-memory default
Test:
same input → same key
model update → cache invalidated
policy update → cache invalidated
candidate order normalization
Implement:
capability filters
metadata filters
rule filters
lexical matching
Measure:
candidate reduction ratio
latency
false elimination rate
Never optimize candidate count at the expense of silently removing the correct candidate.
Add a "safe mode":
never eliminate based solely on weak semantic evidence
Implement tokenizer abstraction.
Implement first ONNX backend.
Load a known small encoder/classifier.
Do NOT train OpenCodifier yet.
The first objective is proving:
Rust
→ tokenizer
→ ONNX
→ inference
→ score
→ decision
Deliverable:
real local NLP decision
Build/train the first candidate-conditioned model.
Initial architecture:
small encoder
+
candidate representation
+
cross-attention/scoring head
Output:
one scalar per candidate
For N candidates:
N logits
Then:
softmax
Do not generate text.
Implement all three through the same model infrastructure.
candidate logits → softmax
true/false logits → softmax
or:
binary logit → sigmoid
ordered levels → softmax
Calculate:
expected score
Also retain the full probability distribution.
Create calibration dataset.
Run:
raw model
→ temperature scaling
→ calibrated model
Measure:
accuracy
ECE
Brier
NLL
coverage
selective accuracy
Reject deployment if calibration regression exceeds defined thresholds.
Implement:
uncertainty detector
verifier
fusion
abstention
Test:
high confidence
low confidence
near tie
OOD
high risk
verifier disagreement
Produce explicit states:
accepted
verified
abstained
escalated
Once the graph engine works, add optimization passes.
Example:
Before:
rule A
↓
classifier
↓
rule B
↓
classifier
Optimizer:
rule A + rule B
↓
candidate reduction
↓
single classifier
Other optimizations:
common subexpression elimination
constant folding
dead-node elimination
parallelization
cache insertion
batching
candidate pruning
early exit
This is where OpenCodifier starts behaving like a decision compiler, rather than merely a classifier.
That is a useful conceptual direction for the name "Codifier":
It converts semantic decision logic into an executable, optimized decision graph.
Implement feature-gated embedding subsystem.
Interface:
trait Embedder {
fn embed(&self, texts: &[String]) -> Result<Vec<Vector>>;
}Implement:
local ONNX embedding
Use cases:
candidate retrieval
semantic cache
skill retrieval
memory selection
document relevance
model capability matching
Do not allow embeddings to override deterministic constraints.
Implement optional:
lexical retrieval
dense retrieval
hybrid retrieval
Use Tantivy as the lexical reference/implementation where practical; it is a Rust full-text engine with BM25 and low startup overhead. (GitHub)
Use a lightweight vector implementation rather than requiring an external database.
Implement:
Reranker trait
Then:
retrieve top-N
↓
rerank
↓
top-K
↓
decision
Keep this optional.
Add rmcp.
Implement:
codify_decide
codify_batch
codify_graph
codify_validate
codify_verify
codify_explain
Add integration tests against MCP clients.
Then implement:
MCP schema discovery
MCP tool indexing
tool candidate generation
Ship initial decision graphs:
model-routing
tool-selection
tool-gating
context-pruning
cache-selection
skill-selection
memory-selection
escalation
verification
Each recipe must have:
graph
schema
example input
example output
benchmark
documentation
Build:
opencodifier-wasm
Support:
WASM CPU
WASM SIMD
optional threads
WebGPU path
Use ONNX Runtime Web and/or Burn according to the model/runtime implementation. ONNX Runtime Web supports WASM CPU and browser GPU paths, while Burn provides a Rust-native route to WASM/WebGPU. (ONNX Runtime)
Browser demo:
drag/drop text
↓
OpenCodifier WASM
↓
local decision
Show a network inspector demonstrating that inference occurs locally.
Build binaries for:
Windows x86_64
Linux x86_64
Linux ARM64
macOS x86_64
macOS ARM64
Provide:
GitHub Releases
Cargo
Homebrew
winget
install scripts
Model downloads remain separate.
Create a permanent benchmark suite.
Measure:
accuracy
top-k accuracy
precision
recall
F1
ECE
Brier
NLL
cold-start latency
warm latency
p50
p95
p99
throughput
RAM
model size
CPU utilization
candidate reduction
cache hit rate
verification rate
abstention rate
LLM escalation rate
tokens avoided
The last metrics are especially important for Amortyx.
Amortyx should consume OpenCodifier as an external decision subsystem.
Architecture:
Amortyx
│
▼
OpenCodifier
│
├── classify task
├── estimate complexity
├── estimate context requirement
├── determine tool requirement
├── determine freshness requirement
└── rank eligible models
│
▼
Amortyx policy engine
│
├── price
├── latency
├── availability
├── quota
├── user policy
└── privacy
│
▼
actual provider/model
OpenCodifier makes semantic decisions.
Amortyx makes economic/operational decisions.
Neither should absorb the other's responsibilities.
Before OpenCodifier controls routing:
request
├── existing Amortyx route
└── OpenCodifier shadow route
Record:
OC prediction
actual selected model
actual latency
actual cost
actual success
quality
Then calculate:
Would OC have chosen correctly?
Would OC have saved cost?
Would OC have reduced latency?
Would verification have been necessary?
Only after sufficient evidence should OpenCodifier become an active routing component.
Build an optional dataset pipeline:
production/shadow logs
↓
privacy filter
↓
deduplication
↓
label extraction
↓
training dataset
↓
OpenCodifier model
↓
benchmark
↓
candidate deployment
Do not automatically train from every production result.
Use explicit dataset/version controls.
Implement a registry concept.
Example:
decision:
id: amortyx.model_selection
version: 1
question:
type: choice
text: "Which model should process the request?"
candidates:
dynamic: true
policy:
min_confidence: 0.80
verify_below: 0.65
abstain_below: 0.50This lets decision definitions become reusable artifacts.
Graphs must be versioned.
Cache keys must include:
graph_version
model_version
calibration_version
policy_version
Never allow a new graph/model/calibration artifact to accidentally reuse incompatible cached decisions.
OpenCodifier must never expose chain-of-thought.
Instead expose deterministic execution facts.
Example:
{
"decision": "qwen",
"confidence": 0.91,
"trace": [
{
"node": "candidate_filter",
"before": 18,
"after": 7
},
{
"node": "semantic_classifier",
"model": "oc-lite-1",
"latency_ms": 4.7
},
{
"node": "confidence_gate",
"threshold": 0.80,
"passed": true
}
]
}Treat all input as hostile.
Particularly:
candidate descriptions
tool descriptions
MCP schemas
JSON schemas
state text
graph files
model manifests
model files
Do not let input text modify:
policy
thresholds
graph structure
filesystem paths
network configuration
tool permissions
Separate data from executable configuration.
Every request must have:
max input bytes
max tokens
max candidates
max questions
max graph nodes
max execution time
max memory
max semantic retrieval results
Defaults should be conservative.
Browser builds must have:
no filesystem assumptions
no arbitrary networking
no telemetry
no dynamic native code
Model downloads must be explicit and integrity checked.
Use browser storage such as IndexedDB only through the WASM/web adapter.
OpenCodifier V1 is complete when all of the following work:
- Rust library
- Decision IR
- Choice
- Boolean
- Score
- Dynamic candidates
- Confidence
- Abstention
- Rules
- Candidate filtering
- DAG
- Branching
- Parallel execution
- Cache
- Trace
- Native OC schema
- Jev/System One compatibility
- OpenAI structured schema
- Anthropic tool schema
- ONNX inference
- local tokenizer
- OpenCodifier model
- calibration
- verifier
- CLI
- HTTP
- MCP
- Rust API
- WASM
- Windows
- Linux
- macOS
- browser
- checksums
- reproducible release artifacts
- recipes
- skills
- documentation
- examples
- benchmark suite
- shadow integration
- routing integration
- decision telemetry
- cost/latency comparison
These are engineering targets, not claims.
The implementation agent must benchmark rather than assume them.
Target:
<100 µs
for trivial in-memory rules where practical.
Target:
sub-millisecond
where practical.
Target:
single-digit to low-double-digit milliseconds
on ordinary desktop CPU for short inputs.
Target:
amortized <5 ms/question
where hardware/model size permits.
Default runtime should remain small.
The base binary must not require a large ML model.
Model size is separate from executable size.
After V1 proves the architecture:
better model
multilingual
semantic cache
reranker
hybrid retrieval
graph optimizer
more recipes
more MCP integrations
Potential future work:
adaptive decision graphs
online calibration
federated/private training
automatic distillation
model specialization
learned candidate pruning
distributed decision cache
GPU batching
mobile
edge workers
Do not implement these during V1 unless required to validate the architecture.
The implementing AI must follow these rules.
Do not start by training a model.
Build the contract and deterministic engine first.
Do not create a monolithic Rust crate.
Maintain clear boundaries.
Do not make ONNX Runtime a hard dependency of the core.
Do not require Python for runtime.
Python may exist in a separate training/research repository if necessary.
Do not make a vector database mandatory.
Do not force every request through a neural model.
Do not force every decision to produce an answer.
Abstention is valid.
Do not expose raw probability as calibrated confidence.
Do not execute two classifiers on every request.
Use confidence-gated verification.
Do not use generated prose where a typed decision will suffice.
Do not silently discard candidates because of weak semantic similarity.
Do not invent support for unsupported JSON Schema constructs.
Do not add telemetry.
Do not add cloud dependencies to the local runtime.
Do not optimize before benchmarks exist.
Input:
{
"request": "Refactor this Rust parser to remove the lifetime bug and add tests.",
"context_tokens": 42000,
"tools_available": [
"filesystem",
"shell",
"git"
],
"models": [
"local-qwen",
"local-glm",
"cloud-qwen",
"cloud-kimi"
]
}Deterministic filtering:
vision required? no
context > 32K? yes
local-only? no
tools required? yes
4 models
Capability filtering:
local-qwen eligible
local-glm eligible
cloud-qwen eligible
cloud-kimi eligible
Semantic classification:
coding 0.99
reasoning 0.88
tool-use 0.94
complexity 0.79
Candidate scoring:
local-qwen 0.44
local-glm 0.39
cloud-qwen 0.12
cloud-kimi 0.05
Amortyx policy then applies:
cost
latency
availability
user policy
current provider health
Final decision:
local-glm
OpenCodifier did not make the economic decision.
It supplied the semantic decision information.
Amortyx remains responsible for routing policy.
Input:
"Find the PR that introduced this regression."
Candidates:
git_log
github_search
filesystem_search
ripgrep
Deterministic:
requires repository history → eliminate filesystem_search
Semantic:
git_log 0.52
github_search 0.39
ripgrep 0.09
Margin is insufficient.
Verifier executes.
Result:
git_log 0.67
github_search 0.28
ripgrep 0.05
Accept.
No LLM generation was required.
Question:
Does this document belong in the active context?
Deterministic:
same repository? yes
same task? yes
modified recently? yes
Semantic:
relevance = 0.91
Decision:
KEEP
Another document:
repository mismatch
old timestamp
semantic relevance 0.21
Decision:
DEMOTE
This becomes directly useful for the broader AI memory/context architecture.
Incoming task
↓
OpenCodifier
↓
task family = GIS
↓
candidate skills:
GIS
CAD
Rust
web
database
↓
semantic ranking
↓
GIS = 0.94
CAD = 0.73
Rust = 0.52
↓
activate GIS
↓
optional CAD
Again, no generated response required.
simple request
↓
OC Micro
↓
confidence = 0.96
↓
local model
Versus:
ambiguous request
↓
OC Micro
↓
confidence = 0.54
↓
OC Small verifier
↓
confidence = 0.61
↓
abstain
↓
larger LLM
This is exactly the kind of token-saving architecture OpenCodifier should enable.
The final AI stack should look like:
USER / AGENT
│
▼
┌───────────────────┐
│ OpenCodifier │
│ │
│ deterministic │
│ decision graphs │
│ NLP │
│ classifiers │
│ semantic search │
│ verification │
│ calibration │
│ caching │
└─────────┬─────────┘
│
machine-actionable intent
│
▼
┌───────────────────┐
│ Amortyx │
│ │
│ routing │
│ cost │
│ latency │
│ provider health │
│ model selection │
│ prompt batching │
│ caching │
└─────────┬─────────┘
│
┌───────────┼────────────┐
▼ ▼ ▼
Local Cheap Frontier
Models APIs APIs
OpenCodifier becomes the semantic decision plane.
Amortyx becomes the inference optimization/routing plane.
Generative models remain the execution/generation plane.
That separation is the key architectural decision.
OpenCodifier should be built around one fundamental idea:
Don't ask a large generative model to make a decision that can be reduced to deterministic logic, classification, scoring, retrieval, or a narrowly constrained probabilistic decision.
Instead:
Understand
↓
Normalize
↓
Filter
↓
Narrow
↓
Classify
↓
Score
↓
Verify
↓
Decide
↓
Escalate only if necessary
The result is not another AI assistant.
It is a decision substrate for AI systems.
That is why I would keep it separate from Amortyx.
Amortyx can consume it.
Coding agents can consume it.
MCP servers can consume it.
Your future memory/context system can consume it.
A web application can run it locally through WASM.
An enterprise can embed it without sending data anywhere.
And an AI agent can use it as a deterministic/semantic control layer before invoking an expensive generative model.
That is a much larger and more defensible project than simply implementing "an open-source Jev."
I would not market OpenCodifier as a "decision tree classifier." Keep that as an intuitive explanation, but the technical/product term should be decision runtime, decision engine, or decision fabric.
A literal decision tree is only one possible implementation. OpenCodifier's real architecture is a deterministic-first decision DAG with progressively more expensive semantic operators.
That distinction gives us room for:
- deterministic rules
- NLP classifiers
- dynamic candidate scoring
- embeddings
- retrieval
- reranking
- calibration
- dual-model verification
- caching
- graph optimization
- MCP
- WASM
- model routing
- context pruning
- skill selection
without having to redesign the product later.
And the research supports the specific technical direction: Jev validates typed probabilistic decisions and parallel evaluation; open-jev validates one-pass candidate scoring; Laya and Von demonstrate small/non-autoregressive local decision models; and the broader ecosystem already includes calibration, abstention, distillation, and tiny models. (TypeSafe AI)
I would therefore create OpenCodifier as the independent greenfield project now, and make Amortyx its first deep integration rather than putting the technology inside Amortyx.