A dynamic agent orchestrator in TypeScript. It analyzes every request locally — spending zero tokens — and only then decides which agent, or combination of agents, should handle it.
Say hi and it answers in 5 ms for $0, never touching a model. Ask it to research, plan and estimate costs, and it fans out to three agents in parallel and synthesizes the result.
$ pnpm dev --trace "hi"
[reflex] direct · reflex.smalltalk
Hey! What can I help you with?
4ms analyze intent=greeting complexity=0.01
5ms route direct -> reflex.smalltalk
5ms done 1 agent, $0.00000Most agent systems send everything to the biggest model. Orchestati picks the path first, using a deterministic analyzer that runs in microseconds:
input ──► Analyzer ──► Router ──► Executor ──► response
(local) (local) (agents)
0 tokens 0 tokens
The routing decision costs nothing, so the system can afford to make it carefully — and the money is only spent where it actually buys something.
Real output from pnpm table, the repository's calibration bench. The lexicon is bilingual (English/Spanish); these are the English prompts, and the Spanish equivalents route identically.
| Request | Intent | Cpx | Tier | Strategy | Agents |
|---|---|---|---|---|---|
hi |
greeting | 0.01 | reflex | direct | reflex.smalltalk |
who are you and what can you do |
identity | 0.08 | reflex | direct | reflex.identity |
what is a closure in javascript |
factual_qa | 0.21 | light | direct | llm.quick |
summarize this paragraph in two lines |
summarize | 0.26 | standard | direct | llm.writer |
how much is 15% of 2340 |
math | 0.32 | standard | direct | llm.analyst |
delete every row in the users table in production |
tool_action | 0.36 | standard | direct | llm.tools |
write a typescript function that validates an email |
code_generate | 0.38 | standard | direct | llm.coder |
refactor this payments module, it became a mess |
refactor | 0.44 | standard | direct | llm.coder |
my app throws TypeError: cannot read property map of undefined |
code_debug | 0.51 | deep | chain | llm.debugger → llm.critic |
design the full architecture of a multi-tenant billing system |
planning | 0.54 | deep | chain | llm.planner → llm.researcher → llm.critic |
research and compare vector DBs, then a migration plan and cost estimates |
research | 0.60 | swarm | parallel | researcher ∥ planner ∥ analyst → synthesizer |
Trying it out:
pnpm install
pnpm chat # interactive chat with live routing, tokens and costPutting it in your own project:
npm install orchestatiimport { Orchestrator } from 'orchestati';
const o = new Orchestrator({ maxCostUsd: 0.25 });
const res = await o.run('refactor this payments module', { sessionId: user.id });Already using the AI SDK? It is a drop-in model:
import { generateText } from 'ai';
import { Orchestrator, orchestatiModel } from 'orchestati';
const model = orchestatiModel(new Orchestrator());
const { text, providerMetadata } = await generateText({ model, prompt: userMessage });
providerMetadata.orchestati.tier; // 'reflex' — that call cost $0 and 0 tokensSee INTEGRATION.md for the full guide — framework routes, domain agents, tools against your own systems, the confirmation gate wired to your UI, sessions in your database, cost control, and a production checklist.
With no credentials the system runs on MockModel: the routing is real, the answers are not. That is enough to develop and test the entire orchestrator without spending anything.
To hit real models, copy .env.example to .env and set one key. Entry points load it automatically, so you never have to remember your shell's export syntax.
| Backend | Variable | Notes |
|---|---|---|
| OpenAI | OPENAI_API_KEY |
defaults to the gpt-4.1 family |
| Gemini | GEMINI_API_KEY |
has a free tier; used through its OpenAI-compatible endpoint |
| Groq | GROQ_API_KEY |
free tier, runs open-weight models (Llama, Kimi) |
| Moonshot / Kimi | MOONSHOT_API_KEY |
|
| DeepSeek | DEEPSEEK_API_KEY |
|
| Cerebras | CEREBRAS_API_KEY |
open-weight models, very fast |
| OpenRouter | OPENROUTER_API_KEY |
|
| Vercel AI Gateway | AI_GATEWAY_API_KEY |
one key for every provider |
| Ollama / LM Studio | — | local; must be requested with ORCHESTATI_PROVIDER=ollama |
pnpm models asks each configured provider what it actually offers, so you do not have to trust a hardcoded list of model ids.
This is the setting that decides whether routing actually saves money.
ORCHESTATI_MODEL_LIGHT=groq:llama-3.1-8b-instant
ORCHESTATI_MODEL_STANDARD=groq:llama-3.3-70b-versatile
ORCHESTATI_MODEL_DEEP=openai:gpt-4.1Within one provider, adjacent tiers are typically ~5× apart in price. Since chain and parallel run two or three agents, the middle tier ends up costing about the same as a single call to the expensive one — and the routing stops paying for itself. Across providers the gap is 10×–40×, which is where the cheap tier earns its place. Measured numbers are in the evaluation section.
Everything except the gateway goes through a single OpenAI-compatible client, so adding a new destination means adding a preset, not a provider.
Local backends are deliberately not auto-detected: a server listening on a port is not the same as a server you meant to use.
pnpm chat shows what a normal chat hides — where each request was routed, which agent handled it, which tools ran, how many tokens it cost, and how much the session has spent so far.
backend OpenAI
light gpt-4.1-nano
standard gpt-4.1-mini
deep gpt-4.1
reflex — no model, immediate response
/examples list the paths · /1 .. /11 run one
/cost running total · /trace detail · /exit
Type /examples for one request per routing path, then /cost for the breakdown. A real session against gpt-4.1:
Session usage
3 messages · 759 in + 86 out = 845 tokens · $0.00036
by tier
reflex 1 msg 0 tok $0 0%
light 1 msg 183 tok $0.00003 8%
standard 1 msg 662 tok $0.00033 92%
by agent
llm.analyst 1× 662 tok $0.00033
llm.quick 1× 183 tok $0.00003
reflex.smalltalk 1× 0 tok $0
tools calculator×1
2 of 3 requests did not need the expensive tier
1 was answered without calling any model
That breakdown is the project's thesis measured in money: the greeting was free, the simple question cost three hundred-thousandths of a dollar, and 92% of the spend went to the single request that actually warranted it.
Deterministic, synchronous, no network. From every request it extracts:
- intent — 20 types, bilingual lexicon, with the evidence that triggered it;
- artifacts — code blocks, stack traces, URLs, file paths, JSON;
- structure — words, questions, list items, chained requests;
- risk (0–1) — irreversible verbs like delete, deploy to prod, charge;
- complexity (0–1), with a breakdown you can audit.
The complexity model treats a task's intrinsic difficulty as a floor, not as one term among many: "design the architecture of X" is a heavy request even when it is said in twelve words. Everything else amplifies from there.
import { analyze } from 'orchestati';
analyze('hi').complexity; // 0.01 → reflex
analyze('design the architecture…').complexity; // 0.54 → deepA greeting glued to a real request does not hijack the routing: conversational intents scale their score by the fraction of the message they occupy, so "hi, refactor this for me" routes to refactor, not greeting.
A regex lexicon has a structural weakness: somebody has to maintain it, and when it does not match there is no safety net. Measured on a held-out set of 62 phrases that appear nowhere in the codebase, the lexicon alone is right 35.5% of the time — the rest falls through to unknown.
The net is a local classifier: character n-grams (3–5) hashed with TF-IDF weights, compared by cosine against 223 labeled prototype phrases. No dependencies, no downloads, no tokens. Character n-grams do the heavy lifting: refactorizame, refactorizar and refactor share nearly all their trigrams, so they land together without anyone writing the rule — and they absorb typos, which is exactly where a lexicon breaks.
$ pnpm eval $ pnpm eval b
set A · 62 cases set B · 39 cases (control)
lexicon only 35.5% lexicon only 23.1%
+ semantic 82.3% + semantic 79.5%Two thresholds, because they are two different decisions. Measuring the score's calibration: at similarity ≥ 0.30 the classifier is right 100% of the time, at ≥ 0.22 it is right 81%, and with no floor at all, 63%. So:
- Filling in an
unknown(floor 0.15) — being right 63% of the time beatsunknown, which carries no routing information at all. - Overruling the lexicon (floor 0.30) — that requires the band where the classifier does not get it wrong.
The lexicon still wins when it is confident: it is exact, auditable and free. The semantic classifier is consulted only when the lexicon hesitated, so the fast path pays nothing — 27 µs versus 513 µs for the doubtful case.
Every prediction carries its nearest prototype as evidence (≈ "let's draw up the quarterly roadmap" (0.41)), so you can always audit why it said what it said.
Two evaluation sets, and the second is the one that matters. Measuring repeatedly against the same held-out set wears it out: every adjustment you make while looking at its errors turns it, bit by bit, into a training set. Set B was written before the prototypes were expanded and without looking at set A's failures. When coverage was expanded, A gained 8 points and B, never inspected, gained 18 — so the improvement generalizes rather than overfitting. A test also asserts that no evaluation phrase appears verbatim among the prototypes.
This is the half of the thesis that went unmeasured the longest, and the answer is not unconditionally — it depends on your traffic mix, and on mixing providers.
pnpm eval:quality runs each request twice: once through the orchestrator, once by sending it straight to the expensive model — the strawman this project claims to beat — and has a blind judge compare the answers with alternating positions. It costs real money, so it is opt-in and never runs in CI.
Measured on 14 self-contained requests against OpenAI, with the baseline pinned to gpt-4.1:
Quality
wins or ties 79% (2 wins, 9 ties)
loses 21% (3)
Cost per tier (routed vs. sending that same request to the expensive model)
reflex 2 cases $0.00000 vs $0.00039 100% cheaper
light 3 cases $0.00020 vs $0.00352 94% cheaper
standard 7 cases $0.01086 vs $0.00967 12% more expensive
deep 2 cases $0.01581 vs $0.00726 118% more expensive
The global number was hiding the interesting one. Savings on the cheap tiers are enormous — 94% to 100% — but chain and parallel run two or three agents, so the expensive tiers cost more than a single call. With this configuration the break-even point sits at 59% of traffic landing on the cheap tiers. A support inbox clears that easily; a set of hard engineering questions does not.
That is also what makes mixing providers the decisive setting rather than a nicety. Same 14 requests, same fixed baseline, the only change being cheaper models on the middle tiers:
| Default config | Cheap middle tiers | |
|---|---|---|
| Cost vs. always-expensive | 29% more expensive | 50% saved |
| Quality (wins or ties) | 79% | 71% |
| Break-even traffic mix | 59% cheap | 25% cheap |
standard tier |
12% more expensive | 74% cheaper |
deep tier |
118% more expensive | 9% more expensive |
So the trade is legible: eight points of quality buy a swing from losing money to saving half of it, and the break-even drops to a level most real traffic clears comfortably. Widening the gap further with a genuinely cheap provider on the light tier — an 8B Llama on Groq, a Gemini Flash — moves it further still.
The honest summary is that Orchestati is not unconditionally cheaper, and its default configuration is not the one that saves money. Routing pays when the cheap tiers are genuinely cheap and enough of your traffic lands on them. pnpm eval:quality is there so you can measure that for your own traffic instead of taking this table's word for it.
Two findings came out of building this, both of them defects rather than tradeoffs:
llm.criticcapped its output at 700 tokens, and it is the last link of every chain — so it was the one producing the final answer. Architecture answers came back truncated. Removing the cap took quality from 57% to 71%.- The first version of the experiment moved its own control. The baseline resolved through the
deeptier, so making that tier cheaper made the baseline cheaper too. Pinning the baseline to a fixed model took the reading from 64% to 79%. An experiment whose control moves measures nothing.
Knowing that the default configuration loses money is only useful if you can fix it for your requests. pnpm tune does that, and it is nearly free.
The trick is that routing is deterministic and the tokens a request consumes barely depend on which model answers it — they depend on the request, the agent's prompt and how many turns it takes. The price does depend on the model. So you profile once against the real API, and then sweep configurations with pure arithmetic.
pnpm tune --profile --from=sessions # spends money once, over your real requests
pnpm tune # sweep configurations, free, as often as you likeRequests can come from your saved sessions (--from=sessions), a file with one per line (--from=requests.txt), or the bundled curated set. Real traffic is the point: tuning against a hand-picked set optimizes for the sample, not for what your users actually ask.
The sweep reports three things — where the money goes (tokens and calls per tier), what each priced model would cost on each tier, and a few complete configurations with the env vars to try them. Models whose price came from the conservative fallback rather than the table are flagged, so you know which numbers are solid.
Cost is tokens × price. The sweep knows the price exactly — it comes from a table — and used to assume the token count stays put when the model changes. The entire error lived in that one term.
Two recommendations validated end to end: a predicted 67% saving came out at 31%, and a predicted 72% came out at 13%. The ranking of the options held both times; the magnitude never did.
The second gap has a sharp cause: reasoning models break the assumption badly. On the same prompt, openai/gpt-oss-120b emitted 3,072 output tokens against gpt-4.1's 965. At a third of the price per token it still came out more expensive — and the sweep was recommending it.
The fix is to measure that difference instead of assuming it away:
pnpm tune --calibrateThis sends three short prompts to every usable model and records how much each one writes. The sweep then corrects the profile's token counts by that ratio. It is cheap for a reason worth stating: input tokens barely change between models — the prompt is ours — so only output verbosity needs measuring.
Models without a measurement are not silently assumed to write the same amount; the sweep flags them and tells you to calibrate. Re-calibrate when you change the models in play, and after adopting a configuration, re-profile with it (ORCHESTATI_MODEL_DEEP=… pnpm tune --profile) for the ground truth.
The first calibration run produced a result worth the whole feature:
openai:gpt-4.1 655 tok 1.0×
openai:gpt-4.1-nano 700 tok 1.1×
openai:gpt-4.1-mini 718 tok 1.1×
openai:gpt-4o 845 tok 1.3×
openai:gpt-5-mini 2162 tok 3.3×
openai:gpt-5 3546 tok 5.4×
openai:gpt-5-nano 5328 tok 8.1×
gpt-5-nano — the cheapest model in the table by price per token — writes 8.1× more than gpt-4.1. Picking it to save money would have cost considerably more. The sweep now says so out loud before you try it.
This is the general shape of the trap: price per token is not price per answer, and the gap is largest exactly where a reasoning model is marketed as the budget option.
It also only measures money. A cheaper model can answer worse and that does not show up here, which is what pnpm eval:quality is for. Running that loop on the bundled set:
| Configuration | Quality (wins or ties) | Cost vs. always-expensive |
|---|---|---|
| Default, one provider | 79% | 29% more expensive |
| Cheapest everywhere, picked by hand | 71% | 50% saved |
What tune recommended |
86% | 31% saved |
Open 120B model on deep (Groq) |
86% | 13% saved |
The tool's recommendation beat the hand-picked one on both axes. It changes a single tier — the one carrying 84% of the spend — which is also why a quality drop would have had an obvious cause.
The last row carries the most interesting result and the most useful warning. An open-weight model held the hardest tier: openai/gpt-oss-120b scored the same 86% as a frontier model on deep. Capability was not the problem. What stopped it was quota — see below.
A configuration that cannot handle your traffic is not cheaper, it is unusable, and the sweep used to ignore this entirely.
Running that Groq configuration failed with a limit its own console does not list: 1,000 output tokens per minute. A request whose expected output exceeds it is rejected outright — not queued, not slowed. Retrying cannot help, because the same request will always be too large.
That rules the free tier out of standard and deep in this system, whose agents routinely produce more than that. pnpm tune now knows: it reads limits from the presets, checks them against the largest request in the profile (the limit applies per request, so one is enough to fail), and stops proposing configurations it knows will be rejected — instead of recommending one with a warning nobody reads in time.
The same failure exposed a real bug in the retry layer: it kept retrying a rejection the provider had already marked x-should-retry: false, with a message that literally said reduce max_tokens. Retrying a request that is too large burns time and quota to fail identically.
Intent accuracy measures a component. The routing decision is the product, and it used to be judged by eyeballing a table. pnpm eval:routing measures it against 33 labeled cases — requests that are not the ones the thresholds were calibrated against:
exact tier 93.9% (31/33)
tier within ±1 100.0% (33/33)
expected agent 93.3% (28/30)
over-routed 0 wastes money
under-routed 2 risks answer quality
The two error directions do not cost the same, so they are reported separately: routing more expensively than needed wastes money, routing more cheaply risks the answer. Tests enforce that over-routing stays at zero — the bias has to sit on the side the system actually promises.
That split is what made the first measurement useful. At 84.8%, every single failure was an under-route, and they clustered on one pattern: English debugging symptoms with no stack trace (times out, memory leak, race condition). The lexicon's debug vocabulary leaned on the word "error". Adding symptom vocabulary took it to 93.9% — and the untouched control set for intent classification rose from 79.5% to 84.6%, which is what tells you the fix generalized instead of just fitting the routing set.
This is what changed the design most. A classifier that is right 74% of the time is wrong 26% of the time, and the system has to know it. Confidence now travels all the way down:
- A guessed intent's complexity regresses toward the mean. If the classifier says farewell for a refactor request, believing its 0.00 complexity sends the request to the reflex agent.
- The router stops ranking by intent when it is unsure. The
intentweight is scaled by confidence and the remainder is handed to capability coverage — which is also inferred from hard evidence (code blocks, stack traces, paths), not just phrasing. When in doubt, an agent with the right capabilities beats a specialist for an intent you may have guessed wrong. - A guessed intent cannot trigger the reflex path. It is the only path with no recovery — it answers with a fixed string and it is done — so it requires confidence ≥ 0.6. A real greeting has it:
hiscores 1.00.
Without this, "separate the business logic from the view" was classified as farewell and the system answered "Bye! Ping me anytime." Now it goes to an agent that can actually respond.
Scores every agent in the pool against the signals and picks. The score has five terms, all visible in decision.ranking:
| Term | Weight | What it measures |
|---|---|---|
capability |
0.30 | coverage of the capabilities the request requires |
intent |
0.26 | whether the agent declares that intent — scaled by confidence |
tierFit |
0.24 | distance to the target power tier (penalizes undershooting and overshooting) |
prior |
0.12 | how that agent has historically performed on that intent |
cost |
0.08 | relative cost penalty |
Agents can also veto themselves through accepts(). That is how the reflex agent excludes itself the moment a real request shows up, instead of depending on an if inside the router.
Then it picks an execution shape:
direct— one agent.chain— planner → worker → critic, where each link is included only if it contributes (planning a stack trace buys nothing; that is a diagnosis, not a plan).parallel— fan-out plus a synthesizer.swarmis not reached by a scalar threshold but by an explicit rule: the request has to be heavy AND multi-part. A single request, however hard, gains nothing from fanning out.
In a fan-out, every agent must contribute at least one capability the request actually requires that is not yet covered — without that condition the parallel branch fills up with irrelevant agents.
An agent declares what it can do, what it costs, and how hard a request it is willing to take on (comfortMax). When a request exceeds that, it returns escalate instead of delivering a poor answer, and the orchestrator re-routes it upward.
| Agent | Tier | Role | Purpose |
|---|---|---|---|
reflex.smalltalk |
reflex | responder | greetings, thanks, goodbyes — no LLM |
reflex.identity |
reflex | responder | "who are you?" — describes the real pool |
llm.quick |
light | responder | direct questions, short translations |
llm.writer |
standard | worker | prose, summaries, explanations |
llm.coder |
standard | worker | code — reads and writes files |
llm.analyst |
standard | worker | arithmetic and costs — uses calculator |
llm.tools |
standard | worker | actions with side effects — runs commands |
llm.debugger |
deep | worker | stack traces and root cause — reads the real code |
llm.researcher |
deep | worker | comparisons and trade-offs |
llm.planner |
deep | planner | breaks work into actionable steps |
llm.critic |
standard | critic | reviews the previous work |
llm.synthesizer |
standard | synthesizer | merges parallel outputs |
Agents actually execute. Every tool declares a risk level, and that level defines what it takes to run it:
| Tool | Risk | What it does |
|---|---|---|
read_file · list_dir · search_code |
safe |
read the project — run without asking |
calculator |
safe |
exact arithmetic, without eval |
write_file · http_fetch |
confirm |
write to disk or reach the network |
run_command |
destructive |
execute a project binary |
Orchestati runs the tool loop, not the provider's SDK. That is the design decision holding everything else up: between the model asking for a tool and that tool running, a confirmation gate has to happen. If the loop lives inside the SDK, that gate does not exist.
model asks ─► exists? ─► valid args? ─► authorized? ─► execute
│ │ │
└────────────┴──────────────┘
the model is told what happened and continues
A denial is not an error: the model is handed "this needs the user's permission" and keeps working without that tool.
Confirmation policies (src/tools/confirm.ts):
autoSafe()— the default: reads without asking, writes nothing without permission.askUser(fn)—safepasses through, everything else goes to whoever decides.denyAll()— rejects everything, to see what an agent would do (--dry-run).allowAll()— no prompting, only for environments where authorization already happened elsewhere (--yes). Never the default.
$ pnpm dev "run the project tests with pnpm test"
⊘ run_command (not authorized)
Did not run "execute: pnpm test": the active policy only allows reads.
$ pnpm dev --yes "run the project tests with pnpm test"
✓ run_command
RUN v2.1.9 — 131 tests passedContainment: file tools cannot leave the project root (.., absolute paths and null bytes are rejected); run_command uses an allowlist — not a denylist — and rejects shell metacharacters that would let commands be chained; http_fetch blocks local network destinations, including the cloud metadata endpoint; and calculator parses the expression instead of evaluating it, so a "sum" cannot execute code.
Runs the strategy with a budget (maxCostUsd, maxMs), an escalation loop, fault tolerance — if one agent in a parallel branch blows up, the run continues — and a complete trace of what was decided and why.
After every run the router receives feedback and updates its memory — an EWMA per intent↔agent pair, persisted to .orchestati/memory.json.
What it learns from matters more than the mechanism. An earlier version fed the router the agent's own self-reported confidence, which was computed from the request's complexity and the agent's comfortMax — both known before the agent ran. The router was learning from its own prior decision: a closed loop with no external signal, which cannot converge on anything. Feedback now comes from facts that could only be known afterwards, weighted by how much they are worth:
| Signal | Weight | What it tells you |
|---|---|---|
| The run finished without incident | 0.25 | very little — a mediocre answer and an excellent one look identical from outside |
| Something concrete happened (escalation, error, a tool failed) | 0.6 | the run went badly |
| The user rephrased the same request | 0.7 | the previous answer did not solve it |
The user rated it (recordFeedback) |
1.0 | the only signal that speaks to quality |
The rephrasing signal reuses the n-gram vectorizer: if a new request in a session is very similar to the previous one, the agent that answered it is penalized. In pnpm chat, /good and /bad rate the last answer.
Streaming here is not just text token by token. A UI needs to know which agent is working, which tool ran, and when the system decided to escalate — the trace is part of the product, not a log. So stream() emits typed events:
for await (const ev of orchestrator.stream('refactor this')) {
if (ev.type === 'route') showPipeline(ev.decision);
if (ev.type === 'tool') showTool(ev.record);
if (ev.type === 'text') write(ev.delta); // ev.agentId says whose
if (ev.type === 'escalate') notify(ev.from, ev.to);
if (ev.type === 'done') finish(ev.result);
}In parallel three agents write at once, which is why every text event carries its agentId and the consumer decides which one to render.
Sessions — InMemorySessionStore or FileSessionStore (append-only JSONL, one file per session, so two processes cannot clobber each other), plus SQL and Redis adapters for deployments with more than one instance. The adapters take a client you already have rather than importing a driver, so the package stays dependency-free for everyone who does not use them.
await o.run('write me a CSV parser', { sessionId: 'javier' });
await o.run('now port it to python', { sessionId: 'javier' }); // has contextHTTP server — node:http, no framework:
pnpm serve # http://127.0.0.1:3000| Endpoint | Purpose |
|---|---|
POST /chat |
run and return the full result |
POST /chat/stream |
the same, as SSE, event by event |
POST /inspect |
analysis and routing decision, without executing |
GET /info |
models and prices per tier, agents, tools, active policy |
GET /agents |
the pool with tiers, capabilities and costs |
GET · DELETE /session/:id |
a session's history |
GET / |
demo UI: pipeline, tools and trace, live |
Behind an HTTP boundary there is nobody to ask whether an rm is authorized. That is why the server's default policy is autoSafe — reads yes, writes no — and raising it is an explicit decision by whoever starts it.
No need to touch the router — register it and it joins the competition.
import { llmAgent, createDefaultRegistry, Orchestrator } from 'orchestati';
const sql = llmAgent({
id: 'llm.sql',
name: 'SQL',
description: 'Writes and optimizes SQL queries.',
tier: 'standard',
capabilities: ['code', 'analysis'],
intents: ['code_generate', 'data_analysis'],
cost: 0.4,
comfortMax: 0.7,
tools: ['read_file', 'search_code'],
system: 'You are a SQL expert…',
accepts: (s) => (/\b(select|join|query|sql)\b/.test(s.normalized) ? 0.5 : 0),
});
const registry = createDefaultRegistry().register(sql);
const o = new Orchestrator({ registry });For an agent that does not use an LLM (an API, a database, a local computation), implement the Agent interface directly — src/agents/reflex.ts is the worked example.
import { z } from 'zod';
import type { Tool } from 'orchestati';
export const jiraTicket: Tool<{ key: string }> = {
name: 'jira_ticket',
description: 'Fetches a Jira ticket by key.',
risk: 'safe', // safe | confirm | destructive
schema: z.object({ key: z.string() }),
summarize: (a) => `fetch ${a.key}`, // shown in the confirmation prompt
execute: async (a, ctx) => ({ ok: true, content: await fetchTicket(a.key) }),
};import { Orchestrator, askUser, denyAll } from 'orchestati';
new Orchestrator({ confirm: denyAll() }); // no tools at all
new Orchestrator({ root: '/path/to/project' }); // a different sandbox
new Orchestrator({ confirm: askUser(async (req) => { // your own gate
return await myUI.confirm(req.summary, req.risk);
}) });| Variable | Purpose |
|---|---|
ORCHESTATI_PROVIDER |
force a backend: openai, gemini, groq, openrouter, gateway, ollama, lmstudio, mock |
ORCHESTATI_MODEL_LIGHT … _SWARM |
override the model for each tier |
ORCHESTATI_BASE_URL |
your own OpenAI-compatible endpoint |
PORT |
HTTP server port (default 3000) |
Per-tier prices come from a table in src/llm/openai-compatible.ts. It feeds the budget cutoff, so an inflated number is not harmless: it cuts runs that would actually have fit. A model with no known rate is assumed expensive — overestimating cuts early and gets noticed; underestimating overspends and shows up on the invoice.
| Command | Purpose |
|---|---|
pnpm chat |
interactive chat: routing, tokens and cost, live |
pnpm dev "<request>" |
one-shot run |
pnpm dev --explain "<request>" |
analysis and decision, without executing |
pnpm dev --trace "<request>" |
run and print the trace |
pnpm dev --dry-run "<request>" |
deny every tool: see what it would do |
pnpm dev --yes "<request>" |
authorize tools without prompting |
pnpm serve |
HTTP server, SSE and demo UI |
pnpm table |
routing calibration bench |
pnpm eval · pnpm eval b |
classifier accuracy on each held-out set |
pnpm eval:routing |
routing quality: tier accuracy and cost-error direction |
pnpm eval:quality |
does the cheap tier answer well enough? (needs a key, spends money) |
pnpm tune |
sweep model configurations for cost against a one-time profile |
pnpm tune --calibrate |
measure how verbose each model is, so the sweep stops guessing |
pnpm cycle |
sweep, then validate the recommended configuration in one go |
pnpm models |
ask each configured provider which models it actually offers |
pnpm info |
what this system is right now: models and prices per tier, agents, tools |
pnpm smoke |
smoke test against the real API, one request per tier |
pnpm test |
the test suite |
pnpm verify:package |
installs the built package in a temp project and checks it imports, types and runs |
src/
analysis/ analyzer, lexicon, arbiter
semantic/ n-gram classifier, prototypes, vectorizer
router/ registry, multi-factor ranking, EWMA memory
agents/ the pool, LLM agent factory, reflex agents
tools/ types, registry, confirmation policies, sandbox
builtin/ fs, shell, calculator, http
runtime/ orchestrator, tool loop, sessions, trace
llm/ AI SDK base, gateway, OpenAI-compatible presets
core/ types, events, .env loading
ui/ demo page served at /
dev/ chat, smoke, routing table, evaluation
131 tests cover the analyzer, the router's ranking, the escalation loop, the budget cutoff, fault tolerance, the tool loop, sandbox containment (path escapes, binary allowlist, SSRF, eval), the semantic classifier, sessions, streaming and the server's endpoints.
They include accuracy floors on both held-out sets, so a routing regression breaks the build instead of going unnoticed.
pnpm test
pnpm typecheckWhat is not there yet: the ~20% of the evaluation sets the classifier still gets wrong, and tools executable from the server behind a real interactive gate (today the server deliberately stays read-only).
| MANUAL.md | complete reference: commands, API, configuration, measurement |
| INTEGRATION.md | putting Orchestati inside an existing application |
| CONTRIBUTING.md | working on Orchestati itself |
| README.es.md | esta documentación en español |
See CONTRIBUTING.md. The short version: you do not need an API key to develop — the mock backend exercises every layer — and if you touch the analyzer or the router, run pnpm table and mention which rows moved.
MIT © Javier D'Accorso