This guide explains how to configure, run, inspect, and troubleshoot Dispatch's deterministic agent, provider, and model router.
The Dispatch router chooses the best policy-compliant execution route for an agent step. A route is the combination of:
- an agent definition and compatible runtime handler;
- a provider;
- a model from an approved, versioned AI SDK catalog.
The router is deterministic rather than LLM-driven. It first rejects candidates that violate hard constraints, then ranks the eligible candidates using a stable quality-first policy. Dispatch persists the selected route before execution, records the reasons for the choice, keeps ordinary retries separate from provider/model fallback, and reuses the persisted decision on resume.
Routing is opt-in. Existing workflows continue to use their configured agent.model unchanged unless routing.enabled is true.
Responsibility is deliberately split across the Kujo SDKs:
- Dispatch owns routing policy, selection, persistence, retries, fallback, evaluation, and audit artifacts.
- AI SDK owns the versioned provider/model catalog and its canonical hash.
- Agents SDK defines the shared agent-side vocabulary:
handler_id, versionedexecution_contract,capabilities,model_candidates, and routing metadata.
The canonical example requires no provider credentials:
cd /path/to/dispatch
DISPATCH_OFFLINE_FIXTURE=true kujo run dispatch.kujo demo "Routing review" \
--workflow-file examples/workflows/routed-review.json \
--yes \
--non-interactive \
--output-root tests/tmp/routing-howtoThe command prints a run ID. Inspect the completed run with:
kujo run dispatch.kujo inspect <run-id> \
--output-root tests/tmp/routing-howto \
--jsonThe example gives Dispatch two fixture models. Both satisfy the step's hard constraints, so the quality-first policy selects dispatch-frontier deterministically.
If you installed the ecosystem AI profile, the same example is ready immediately:
cd "$HOME/.kujo/sources/dispatch"
dispatch validate --workflow-file examples/workflows/routed-review.json --json
dispatch explain-route --workflow-file examples/workflows/routed-review.json --json
DISPATCH_OFFLINE_FIXTURE=true dispatch demo "Routing review" \
--workflow-file examples/workflows/routed-review.json \
--yes \
--non-interactive \
--output-root tests/tmp/routing-howtoThe installer configures the dispatch shim with the matching Dispatch and AI SDK source paths. No additional dependency install or provider credential is required for this fixture-backed walkthrough.
For each routed agent step, Dispatch performs this sequence:
- Merge workflow, agent, and step routing constraints.
- Load the requested agent and, when enabled, compatible substitute agents.
- Expand each agent's
model_candidatesagainst the AI SDK model catalog. - Reject candidates that violate any hard constraint.
- Rank the remaining candidates with the quality-first policy.
- Create and persist the route decision before the model call.
- Execute the selected handler using an effective cloned agent whose model is replaced by the selected provider/model.
- Evaluate the output when deterministic evaluation is configured.
- Retry the same route or select an explicit fallback route according to policy.
Candidate declaration order does not decide the winner. Dispatch uses stable candidate ordering and a deterministic route-key tie-break.
Eligible candidates are ranked in this order:
quality_tier:frontier,premium,standard,economy,fixture, thenunknown;- higher
relative_quality; - higher
reliability_score; - lower combined input/output cost;
- lower typical latency;
- higher
token_efficiencywhen supplied; - lexicographically stable
agent::provider::modelroute key.
Unknown cost or latency values are ranked after known values. Dispatch never invents missing operational metadata.
A routed workflow needs four pieces:
- routable agent metadata;
- model candidates;
- a valid model catalog;
- workflow and/or step routing policy.
{
"id": "writer",
"name": "Report Writer",
"role": "writer",
"purpose": "Write a structured report.",
"instructions": "Write a concise, evidence-backed report.",
"handler_id": "writer",
"execution_contract": {
"id": "report-writer",
"version": "1"
},
"capabilities": {
"write_report": true
},
"model": {
"provider": "openai",
"model": "legacy-default-model"
},
"model_candidates": [
{"provider": "openai", "model": "approved-standard-model"},
{"provider": "openrouter", "model": "approved-frontier-model"}
],
"tools": [],
"output_schema": {"type": "dict"},
"routing": {"risk": "low"},
"uses_model": true
}Important rules:
modelremains the legacy/default route and preserves backward compatibility.model_candidatesidentify possible routes; they do not own cost, quality, latency, or reliability facts.- The AI SDK catalog is authoritative for model metadata. A candidate cannot override catalog facts.
handler_ididentifies the executable handler.execution_contractmust contain non-emptyidandversionfields.- A substitute agent is compatible only when it uses the same handler or has an exact execution-contract ID/version match.
uses_modelcontrols whether model-call lifecycle events are emitted for the handler.
Generate catalog entries in the AI SDK repository rather than maintaining a second provider table in Dispatch:
from src.model_catalog import create_model_metadata, create_model_catalog
catalog := create_model_catalog("production-approved", "2026-08-24", [
create_model_metadata("openai", "approved-standard-model", {
"quality_tier": "standard",
"relative_quality": 0.80,
"context_window": 16000,
"supports_tools": true,
"supports_structured_output": true,
"input_cost_per_million": null,
"output_cost_per_million": null,
"typical_latency_ms": null,
"reliability_score": null,
"source": "platform-policy"
}),
create_model_metadata("openrouter", "approved-frontier-model", {
"quality_tier": "frontier",
"relative_quality": 0.95,
"context_window": 32000,
"supports_tools": true,
"supports_structured_output": true,
"source": "platform-policy"
})
], {"owner": "ai-platform"})
print(to_json(catalog))
Copy the generated JSON object into workflow.routing.model_catalog, or inject the same generated object when constructing a workflow in Kujo code. Do not type a catalog hash manually: AI SDK calculates the canonical catalog_hash from the catalog identity, sorted model entries, and metadata.
An enabled Dispatch router requires:
schema_name: "ai-sdk-model-catalog";schema_version: "1.0.0";- a non-empty catalog
idandversion; - a valid canonical
catalog_hash; - a
modelsarray containing uniqueprovider::modelentries.
Unknown context, price, latency, or reliability values should be null. If a hard constraint depends on unknown metadata, the default unknown_metadata: "reject_constrained" policy rejects that candidate.
{
"routing": {
"enabled": true,
"policy_version": "1.0.0",
"objective": "quality_first",
"allow_agent_substitution": false,
"constraints": {
"quality_floor": "standard",
"unknown_metadata": "reject_constrained"
},
"fallback": {
"max_fallbacks": 1,
"max_evaluation_retries": 1,
"fallback_on_codes": [
"provider_unavailable",
"structured_output_invalid"
]
},
"model_catalog": {
"schema_name": "ai-sdk-model-catalog",
"schema_version": "1.0.0",
"id": "production-approved",
"version": "2026-08-24",
"catalog_hash": "<generated-by-ai-sdk>",
"models": []
}
}
}The abbreviated catalog above shows placement only; it is not runnable. Use the complete object generated by AI SDK.
Routing policy version 1.0.0 supports quality_first, cost_first, latency_first, and balanced. Hard constraints are identical for every objective; the objective changes only deterministic ranking among eligible candidates. Unsupported versions or objectives fail explicitly.
{
"id": "write",
"name": "Write Report",
"type": "agent",
"agent_id": "writer",
"input_from": ["input", "research"],
"output_key": "report",
"routing": {
"allow_agent_substitution": true,
"constraints": {
"required_capabilities": ["write_report"],
"required_tools": [],
"allowed_providers": ["openai", "openrouter"],
"quality_floor": "standard",
"required_context_tokens": 12000,
"requires_structured_output": true,
"minimum_reliability": 0.95,
"max_latency_ms": 3000,
"maximum_risk": "low"
}
}
}Step constraints may only narrow agent and workflow policy. Allowlists intersect, requirements and denylists accumulate, minimums increase, and maximums decrease. A step cannot weaken a broader constraint. Dispatch also uses a step's declared output_schema as a required agent output contract.
dispatch validate --workflow-file workflow.json --json
dispatch explain-route --workflow-file workflow.json --step-id write --jsonAdd --live to validate to check the AI SDK bridge and provider credential environment variables without making a model call. Validation rejects unknown fields, unsupported providers, missing handlers, incomplete execution contracts, invalid fallback/evaluation policy, and missing model candidates.
Set routing.model_catalog_file to a JSON file path relative to the workflow file:
{"routing": {"enabled": true, "model_catalog_file": "catalogs/production.json"}}Dispatch loads and validates it before execution, then embeds the resolved catalog in persisted run state so resume does not depend on the original file. Generate it with AI SDK:
cd /path/to/ai-sdk
kujo run scripts/generate_model_catalog.kujo -- \
examples/dispatch-model-catalog.config.json --output /path/to/dispatch/catalogs/production.jsonUse handler_id: "model" for an agent that does not need custom Kujo handler code. Supply instructions, model_candidates, optional registered tools, model_options, and an optional output_schema. Dispatch sends tool schemas to the model, executes only the declared tools the model actually requests, reapplies authorization on every call, deduplicates repeated tool-call IDs, and stops at the configured bounds (max_tool_rounds 0–16 and max_tool_calls 0–64).
To persist incremental AI SDK stream events while retaining the same normalized final result:
{
"handler_id": "model",
"model_options": {
"stream": true,
"max_tool_rounds": 4,
"max_tool_calls": 16
}
}The run directory receives stream-<step-id>.jsonl as deltas arrive.
Agents SDK users can call convert_agent_to_dispatch(agent, options) or validate_dispatch_agent_conversion(agent, options) from src.agents.integrations.adapters to create this definition without manually translating fields.
When deterministic evaluation returns request_human_review, Dispatch creates a native approval request, persists the candidate output and route decision, and pauses the run. Resume with the normal resume or resume-decision flow. Approval accepts the stored output without another model call; rejection fails the step; requested changes rerun the persisted route.
All hard constraints are applied before ranking.
| Constraint | Meaning |
|---|---|
required_capabilities |
Every named capability must be enabled on the agent. |
required_tools |
The agent must expose every named tool, and the model must support tools. |
allowed_agents / denied_agents |
Agent ID allowlist or denylist. |
allowed_providers / denied_providers |
Provider ID allowlist or denylist. |
allowed_models / denied_models |
Model ID allowlist or denylist. |
quality_floor |
Minimum accepted quality tier. |
requires_tools |
Requires model tool-call support. |
requires_structured_output |
Requires model structured-output support. |
output_schema_required |
Requires the agent to declare an output schema. |
required_output_schema |
Requires compatible output type and required fields. Normally derived from the step schema. |
required_context_tokens |
Minimum catalog context window. |
max_input_cost_per_million |
Maximum catalog input price. |
max_output_cost_per_million |
Maximum catalog output price. |
max_latency_ms |
Maximum typical catalog latency. |
minimum_reliability |
Minimum reliability score from 0 to 1. |
maximum_risk |
Maximum agent/model risk: low, medium, high, or critical. |
unknown_metadata |
reject_constrained rejects unknown values needed by an active constraint. |
require_catalog_membership is enforced by default. A workflow may set allow_uncataloged_models: true, but production workflows should normally keep catalog membership mandatory.
Agent substitution is disabled by default. Enable it at workflow or step level:
{
"routing": {
"enabled": true,
"allow_agent_substitution": true
}
}Dispatch considers substitute agents in stable agent-ID order. A substitute must:
- satisfy all capability, tool, schema, policy, and risk constraints;
- use the same
handler_idas the requested agent, or declare the exact same versionedexecution_contract.
This prevents a semantically attractive agent definition from being routed to an incompatible runtime implementation.
Plugin-provided agents can participate in routing. Apply the plugin when starting the run with --plugin; Dispatch persists the active plugin reference and rehydrates built-in plugins on resume.
Evaluation runs after a successful handler/model call:
{
"evaluation": {
"required_fields": ["summary", "citations"],
"minimum_confidence": 0.80,
"max_unsupported_claims": 0,
"on_failure": "upgrade_model"
}
}Supported outcomes are:
| Outcome | Behavior |
|---|---|
accepted |
Output passed evaluation and the step completes. |
retry_same_route |
Re-execute the same agent/provider/model up to max_evaluation_retries. |
upgrade_model |
Select another eligible route that is a strict quality upgrade. |
reroute_agent |
Exclude the current agent and select a compatible substitute. |
request_human_review |
Create a native Dispatch approval request and pause with the candidate output and route persisted. |
failed |
Fail with quality_evaluation_failed. |
accepted is produced by the evaluator; configure one of the other values through on_failure.
Dispatch treats retry and fallback as different operations.
Transient, retryable failures remain on the same agent/provider/model. Examples include:
timeout;rate_limitorhttp_429;- retryable network or bridge execution errors;
- retryable
http_5xxresponses.
The step's normal retry_policy controls these attempts.
Fallback deliberately selects a different route. Built-in fallback categories include:
provider_unavailable;context_incompatible;structured_output_invalid;quality_evaluation_failed;unsupported_feature;- any code explicitly listed in
fallback.fallback_on_codes.
Fallback is bounded by max_fallbacks. Each transition records:
fallback_from;fallback_reason;fallback_to;- the previous and next decision IDs.
Dispatch never silently changes a provider or model.
Route decisions are persisted before execution. When a run resumes, Dispatch reuses the stored decision instead of resolving a new route.
Resume fails with persisted_route_unavailable when the stored route can no longer be executed—for example, when:
- the selected agent is missing;
- the agent's handler changed;
- the execution-contract ID or version changed;
- the selected provider/model is absent from the active catalog;
- the catalog hash changed;
- a required plugin or plugin handler is unavailable.
This behavior is intentional. Update or migrate the persisted workflow/run explicitly instead of expecting resume to silently select a new route.
Live SDK calls are the default and fail closed when credentials or configuration are missing. Do not set fixture mode for a live run:
unset DISPATCH_OFFLINE_FIXTURE
export AI_SDK_PATH=/path/to/ai-sdk
# Configure only the providers present in the approved catalog.
export OPENAI_API_KEY=...
export OPENROUTER_API_KEY=...
export DEEPSEEK_API_KEY=...
kujo run dispatch.kujo demo "Live routed review" \
--workflow-file path/to/live-routed-workflow.json \
--yes \
--non-interactiveThe bridge currently recognizes the built-in provider IDs openai, openrouter, and deepseek, plus custom for an OpenAI-compatible endpoint. A custom agent's legacy model configuration should include base_url and api_key_env; the selected model candidate supplies the model ID. CUSTOM_API_KEY is the default permitted credential variable for custom endpoints. To use a different, purpose-specific variable, explicitly allow its name:
export DISPATCH_ALLOWED_CUSTOM_API_KEY_ENVS=WATCHDOG_PROXY_TOKEN
export DISPATCH_ALLOWED_CUSTOM_PROVIDER_ORIGINS=https://watchdog.example/v1Use a comma-separated list for multiple approved names. Dispatch rejects arbitrary process environment variables so an untrusted workflow cannot select an unrelated secret and forward it to a custom endpoint.
Workflow files are limited to 1 MiB and external model catalogs to 5 MiB by default. Controlled deployments may override these bounds with DISPATCH_WORKFLOW_MAX_BYTES and DISPATCH_MODEL_CATALOG_MAX_BYTES.
Never put API-key values in a workflow, catalog, route decision, or model candidate. Use provider environment-variable names and the environment itself for secrets.
Before production rollout, validate every allowed provider/model route with the real credentials and deployment network policy. Offline fixture validation proves routing behavior, not external provider availability.
Run the live preflight before starting a live workflow:
dispatch validate --workflow-file path/to/live-routed-workflow.json --live --jsonFor an intentionally local OpenAI-compatible endpoint over HTTP, set KUJO_AI_SDK_ALLOW_INSECURE_LOCALHOST=1, DISPATCH_ALLOW_INSECURE_LOCAL_CUSTOM_PROVIDER=true, KUJO_ALLOW_PRIVATE_NETWORK_DESTINATIONS=1, and include the exact localhost base URL in DISPATCH_ALLOWED_CUSTOM_PROVIDER_ORIGINS. These opt-ins apply only to local development; production endpoints must use HTTPS.
Add workflow-level budgets. Dispatch persists cumulative use and fails with workflow_budget_exceeded before starting more work or immediately after a model response crosses a resource limit.
{
"budgets": {
"max_steps": 20,
"max_total_tokens": 100000,
"max_cost_usd": 10,
"max_wall_time_ms": 900000
}
}max_cost_usd requires explicit non-negative input and output token pricing on every eligible live catalog route. Dispatch rejects missing pricing instead of treating it as free. Hard wall-time or step deadlines allow the reviewed bounded tools shipped with Dispatch; unreviewed custom/plugin handlers are rejected before execution. Put custom side-effecting tools behind a separately supervised service when you require hard preemption.
Cost enforcement uses only operator-supplied catalog rates and normalized AI SDK usage. Unknown cost metadata is never invented, and a cost-bounded live route with missing pricing is rejected before model execution.
Concurrency is explicit and side-effect safe. Set a workflow bound and opt individual tool steps in with a stable idempotency key:
{
"max_parallel_steps": 4,
"steps": [
{
"id": "fetch-a",
"name": "Fetch A",
"type": "tool",
"tool_name": "fetch_source",
"parallel_safe": true,
"idempotency_key": "fetch-a-v1",
"output_key": "source_a"
}
]
}Only dependency-ready tool steps can opt in. Dispatch starts at most max_parallel_steps, preserves deterministic result merge order, and keeps agent/approval/handoff/report execution serialized.
Filesystem state is the zero-dependency default. SQLite is available for stronger restart and shared-root state authority:
export DISPATCH_STATE_BACKEND=sqlite
dispatch doctor --output-root outputs --jsonSQLite uses WAL, full synchronous durability, and revision compare-and-swap. Human-readable JSON artifacts are still written as mirrors. Every run also has an owner-bound lock; tune waits only when deployment needs differ:
export DISPATCH_RUN_LOCK_TIMEOUT_MS=5000HTTP lifecycle events are recorded before delivery. Network delivery requires an exact HTTPS origin allowlist and a signing key:
export DISPATCH_WEBHOOK_SIGNING_KEY='from-your-secret-manager'
export DISPATCH_WEBHOOK_SIGNING_KEY_ID='events-2026-08'
export DISPATCH_ALLOWED_WEBHOOK_ORIGINS='https://collector.example'
dispatch run "Task" --workflow-file workflow.json --webhook-url https://collector.example/events --yes --non-interactive
dispatch webhooks status --output-root outputs --json
dispatch webhooks replay --output-root outputs --url https://collector.example/events --destination lifecycle --jsonThe outbox stores a destination hash rather than the URL. Replay therefore requires the operator to supply the same URL. The active outbox rotates to one bounded backup at DISPATCH_WEBHOOK_OUTBOX_MAX_BYTES (10 MiB by default).
Receivers verify signature.value as HMAC-SHA256 over the canonical JSON object containing schema_version, event_id, issued_at, destination, url_hash, and payload, using the key selected by signature.key_id. Reject stale issued_at values and duplicate event_id values.
doctor always checks the selected state backend. --live additionally validates the workflow's environment and performs a small real request against every unique catalog route:
dispatch doctor --live --workflow-file workflow.json --output-root outputs --jsonThis command uses real provider credentials and may incur minimal provider usage. Do not run --live in fixture-only CI.
Each run stores routing data in:
state.json:route_decisions,route_attempts, and per-step route fields;trace.json: structured route and model-call lifecycle events;trace.md: human-readable routing timeline and event counts;otel-traces.json: OTLP JSON spans correlated by run/step/trace IDs;stream-<step-id>.jsonl: incremental model events when streaming is enabled;report.json: routing decisions and attempts when a report is generated;inspect --json: aroutingenvelope containing decisions and attempts.
Useful trace event types include:
route_requested;route_candidates_evaluated;route_rejected;route_selected;route_fallback;model_call_started;model_call_completed;model_call_failed.
Inspect a decision directly:
jq '.route_decisions' outputs/<run-id>/state.json
jq '.route_attempts' outputs/<run-id>/state.json
jq '[.events[] | select(.type | startswith("route_") or startswith("model_call_"))]' \
outputs/<run-id>/trace.jsonA persisted decision includes the selected agent, provider, model, quality tier, catalog identity/hash, policy version, constraints, candidate count, reason codes, estimated cost/latency when known, selection time, and deterministic decision ID.
Dispatch applies its normal sensitive-field redaction before writing artifacts. Token-count metrics are retained; API keys, authorization data, secrets, and passwords are redacted.
Check that:
- the catalog was produced by the AI SDK
1.0.0catalog contract; - the catalog ID and version are non-empty;
- provider/model pairs are unique;
- bounded numeric metadata is valid;
- the canonical hash still matches the catalog content;
- a stale
validation.ok: falseresult was not embedded.
Changing any hashed catalog content requires generating and persisting a new hash.
Inspect error.details.candidate_evaluations. Every candidate includes rejection reason_codes, such as:
capability_mismatch;provider_denied;quality_below_floor;context_capacity_insufficientorcontext_capacity_unknown;structured_output_unsupported;reliability_below_floor;risk_exceededorrisk_unknown;output_schema_incompatible.
Do not weaken several constraints at once. Correct stale metadata or relax the single policy that is intentionally too strict, then rerun with a newly versioned catalog when catalog content changes.
Confirm, in order:
- both candidates passed hard constraints;
- the selected model's catalog
quality_tier; relative_qualityandreliability_score;- known versus unknown cost and latency;
- the stable route-key tie-break.
The route_candidates_evaluated and route_selected trace payloads are the source of truth.
Compare the stored decision with the persisted workflow definition and current plugin/catalog configuration. Do not delete the stored route merely to force a new decision; that would remove the audit guarantee. Start a new run or perform an explicit, reviewed migration.
Unset DISPATCH_OFFLINE_FIXTURE in the environment used by Dispatch and verify AI_SDK_PATH, KUJO_BIN, the provider key, and DISPATCH_SDK_BRIDGE_SCRIPT when overridden.
Routing is disabled unless explicitly enabled. In a legacy workflow:
- Dispatch uses the step's requested agent;
- the agent's existing
modelis preserved; - no model catalog is required;
- the route decision is marked with
legacy_static; - existing workflow, retry, approval, handoff, plugin, fixture, and CLI behavior remains intact.
This makes it safe to adopt routing one workflow at a time.
Before merging a routed workflow:
- Generate the catalog and hash through AI SDK.
- Keep unknown operational data as
null. - Give every substitutable agent a versioned execution contract.
- Confirm every model candidate exists in the catalog.
- Set a deliberate quality floor and provider policy.
- Bound fallback and evaluation retries.
- Run the workflow in fixture mode.
- Inspect
state.json,trace.json,trace.md, andreport.json. - Exercise at least one no-route case and one fallback case in tests.
- Verify resume reuses the persisted route.
- Validate all approved routes live before production rollout.
examples/workflows/routed-review.json: complete offline routed workflow.src/core/routing.kujo: filtering, ranking, decision, and evaluation contracts.src/core/runner.kujo: persistence, execution, retry, fallback, and resume behavior.src/bridge/sdk_adapter.kujoandsrc/bridge/bridge_chat.kujo: AI SDK bridge boundary.tests/routing_tests.kujo: executable routing behavior examples.../ai-sdk/src/model_catalog.kujo: model catalog construction and validation.../agents-sdk/src/agents/core_types.kujo: routable agent metadata and compatibility.