You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Astra has one durable Agent Backbone on Server and multiple bounded capacity providers. Server correctly owns ContextPipeline, the Agent Loop, model routing, policy, retry, trace, usage aggregation, and durable run state. A Runner should execute work where the relevant credentials, network, and authority live.
The implemented external model path does not yet satisfy that contract. AdmittedModelExecution::from_endpoint places provider authorization and endpoint material in Server memory, ServerAgenticLoopHost converts it into a normal Server LLM route, and the Server HTTP client opens the provider connection. The invocation can nevertheless be recorded as execution_placement = edge.
The current Edge WebSocket protocol also supports only tool request/result/cancel/ack. It has no inference preparation, exact provider-attempt admission, normalized stream, usage, terminal, or reconciliation lifecycle.
This creates four product and correctness problems:
A user- or organization-owned provider secret crosses into Astra Server memory.
Device-local and private-network model endpoints may be unreachable from Server.
Persisted placement does not describe the real transport boundary.
The current path cannot recover a streamed inference across Runner disconnects without risking duplicate provider work or inventing a terminal result.
The target behavior is already owned by docs/design/model-access-and-inference.md: non-TaaS personal credentials remain in a Runner-local secret store, Server remains authoritative for canonical run and invocation state, and the selected Runner acts only as an inference executor.
Design intent and first principles
This feature extends Astra's existing product thesis to inference:
One semantic owner. Web, CLI, Server, and Runner share one Agent Loop, ContextPipeline, inference lifecycle, retry policy, trace, and usage model.
Execution follows authority. Provider credentials, private endpoint configuration, proxy credentials, and private CA material are resolved only inside the Runner that owns them.
Durable authority precedes irreversible work. The exact logical invocation and provider attempt are durable before provider I/O; the Runner writes its local dispatch fence before opening the provider connection.
Evidence does not become authority. Runner advertisements and outcomes are authenticated, versioned evidence. Server remains authoritative for Offering eligibility, policy, admission, fallback, and canonical settlement.
Uncertainty remains explicit. Astra does not claim exactly-once provider execution without downstream idempotency or reconciliation, and it does not blindly retry after ambiguous dispatch.
Failures are typed, scoped, and bounded. Losing one Runner blocks only inference branches that require that Offering. Deadlines, queues, frames, replay, journals, and concurrency are bounded.
The product states the real guarantee. Runner BYOK protects provider credentials and the provider network boundary; it does not imply that prompt or model output bypasses Server.
Describe the feature you would like
Add first-class This device and Enterprise Runner model execution. The Agent Backbone remains on Server. An authenticated personal or organization-owned Runner executes one Server-admitted physical provider attempt using a local credential or private endpoint and returns normalized events and one canonical terminal result to the existing Agent Loop.
Web / CLI / API
|
v
Server: one durable Agent Backbone
ContextPipeline | Agent Loop | Policy | Route | Ledger | Retry | Trace
|
+-- ServerAttemptExecutor --> managed/self-hosted model
|
+-- RunnerAttemptExecutor --> BYOK/private/local model
secret and endpoint stay in Runner
Terminology and product dimensions
Runner is the execution boundary. Edge is a deployment topology, not a second runtime or the canonical execution-placement name. CLI-local, personal Edge, and enterprise deployments can all supply Runner capacity.
The following facts are independent and must not be collapsed into one enum:
Model Access source and owner;
credential owner;
billing owner;
data boundary;
execution placement (server or runner);
Runner class (personal or enterprise);
exact Runner and model-binding identity;
current availability.
For example, an organization-owned Workspace Offering may execute through an Enterprise Runner. Enterprise Runner does not automatically require a separate Model Access kind.
Product, privacy, and trust contract
A Runner-executed Offering must guarantee:
API keys, bearer tokens, provider credentials, private endpoint values, proxy credentials, and private CA material never enter Server requests, state, logs, trace, SSE, snapshots, or debug output.
Astra Server does not open the provider transport; the admitted attempt is delivered to the selected authenticated Runner.
Durable Server routes contain only opaque references, ownership, policy, and revision facts.
The UI states execution placement, Runner, credential owner, billing owner, availability, and recovery actions.
Server never silently falls back across Runner, device, workspace, credential, billing, or data boundaries.
Runner-reported BYOK usage is labeled by source and must not be presented as an authoritative provider billing ledger.
Within Astra's trust model, execution_placement = runner means Server did not materialize provider connection material or execute provider transport, and the attempt was bound to the named Runner. It does not cryptographically prove the behavior of a compromised Runner. Deployments that require stronger proof may add workload/host attestation or independently observed egress identity without changing the base inference lifecycle.
This mode does not claim that Server cannot see the prompt, tool schemas, or model result. Server still owns ContextPipeline and the Agent Loop. A deployment where prompt and agent state never reach Astra Cloud requires the same Backbone to be self-hosted inside the enterprise boundary; it is not a local=true variation or a client-side Agent Loop.
Product journey
Provide a local CLI/Runner setup flow similar to:
$ astra model add
Where is this model reachable?
> This device
Enterprise Runner
Astra Server (administrator-managed)
Provider: OpenAI-compatible
Endpoint: https://api.example.com/v1
Model: glm-5
API key: ********
✓ Endpoint reachable observed now
✓ Authentication accepted observed now
✓ Streaming probe passed observed now
✓ Tool calling declared / probe passed evidence shown
✓ Saved to system keychain
✓ Runner online
GLM-5 · This device · Your billing · Online
The API key never leaves this device.
The prompt and model output are processed by your Astra Server.
Capability diagnostics distinguish declared capability from observed probe evidence and show freshness. A probe that creates provider work is explicit and bounded because it may incur cost.
Known Offerings remain visible when unavailable so the user can understand and repair an existing selection. Visibility and selectability are separate:
GLM-5
Xupeng's Mac · Your billing · Offline
Not selectable now
Reconnect device Choose another model Cancel
This device is displayed only when the current client can establish that it is colocated with the selected personal Runner. Other clients display the stable Runner name instead.
Canonical design
1. Implement the canonical secret-free route
Replace the combined AdmittedModelExecution material with the existing target ResolvedInferenceRoute contract rather than introducing a smaller parallel route type:
Server credential references remain in the separate ResolvedCredentialBinding. A Runner binding contains only an opaque local credential generation/reference; Server cannot dereference it and never receives the local endpoint.
The existing provider-authorized gateway path must not continue reporting Runner/Edge placement while Server performs the HTTP request. Until Runner inference transport exists, classify that path truthfully as Server execution or reject it as unsupported; do not retain two interpretations.
2. Add one exact-attempt executor boundary
Introduce one typed inference facet consumed by a shared inference coordinator:
ServerAttemptExecutor wraps the current provider HTTP implementation.
RunnerAttemptExecutor uses the durable Runner transport.
One call performs at most one physical provider request.
Provider adapters do not implement hidden retries.
The Server inference coordinator owns logical invocation admission, retry, fallback, deadline, usage settlement, and Agent Loop continuation.
A fallback that changes an Offering, provider, credential, Runner, billing owner, or data boundary creates a new immutable route/attempt and an explicit fallback_from fact.
Server and Runner adapters consume the same versioned canonical request and emit the same normalized event/terminal contract. Shared adapters use one conformance suite rather than diverging copies.
3. Add Runner-local model bindings and secret storage
A local model binding owns:
binding id and revision
provider family and protocol
upstream model name
local endpoint reference
local credential reference or workload identity
local proxy/TLS profile references
adapter revision
declared and observed capabilities with provenance/freshness
enabled state
Runner advertisement contains only non-secret metadata, opaque binding identity, revisions, health evidence, supported inference purposes, and a finite lease.
Provide a RunnerSecretStore abstraction rather than a device-only vault:
an explicitly disclosed encrypted-file fallback whose unlock key is not stored beside the ciphertext.
Secret values use redacting, non-serializable, zeroizing wrappers and are materialized only for one local attempt. Neither secrets nor endpoint values are written to advertisements or journals. Redirect, proxy, DNS, TLS, and private-CA behavior is explicit; authorization is never forwarded across an unapproved origin change.
Rotation increments the local binding/credential revision and affects the next attempt, not an active one.
4. Add a dedicated versioned inference protocol
Do not overload EdgeServerMessage::ToolRequest. Inference has a distinct typed facet and lifecycle.
Server -> Runner
inference_prepare
inference_dispatch
inference_cancel
inference_progress_ack
inference_terminal_ack
inference_reconcile_request
Runner -> Server
inference_prepared
inference_rejected
inference_dispatch_started
inference_event_batch
inference_terminal
inference_reconcile_state
The authenticated connection negotiates the inference protocol and adapter versions before a Runner Offering becomes selectable. Every message is bound to:
invocation, preparation, and provider-attempt identity;
authenticated principal, Runner, and workspace/organization scope;
Runner lease epoch and delivery generation;
model-binding, capability, and adapter revisions;
canonical request hash and prepared provider-wire identity;
finite deadline;
monotonic event-batch sequence;
typed dispatch certainty and outcome.
Frames have strict schema and size limits. Unknown critical fields/versions fail closed for the affected attempt; malformed sibling activity must not poison unrelated runs.
5. Use a two-phase prepare/dispatch boundary
Server authenticates the principal, resolves and revalidates the effective Offering, policy, purpose, data boundary, Runner lease, and revisions.
Server reserves budget/concurrency and persists the immutable route, logical invocation, and an attempt intent that carries no provider-I/O authority.
Runner validates the exact local binding and capacity, materializes local execution material, prepares the provider-native request, and writes a local prepared intent without provider I/O.
Runner returns a non-secret preparation and provider-wire identity.
Server durably binds that identity to the exact provider attempt and commits dispatch authorization.
Server sends the exact dispatch grant.
Runner compare-and-sets the local record to dispatch_fenced and fsyncs before provider I/O.
Runner emits dispatch_started, normalized event batches, and a terminal aggregate.
Server persists the provider terminal, canonical response/result reference, usage status, and logical settlement before issuing terminal ACK.
Prepared state may cache material only in bounded process memory. The durable local journal stores identity, revisions, hashes, custody state, bounded response/replay data, and terminal evidence; it never stores provider secrets. Any prompt/output persistence in the Runner journal is encrypted, bounded, TTL-governed, and disclosed because it is user data even when it is not a provider credential.
6. Represent delivery truth with orthogonal facts
Do not encode all semantics in one linear status enum. Persist at least:
streaming is progress, not execution authority. DeliveryUnknown prevents a new provider attempt unless policy has downstream idempotency/reconciliation evidence. Later Runner evidence appends a reconciliation fact; it does not erase the original period of uncertainty.
Run/invocation cancellation and provider-attempt outcome are separate facts. If cancellation races a provider completion, the run may remain cancelled while the provider attempt records completion and billable usage. Exactly one terminal wins only within the same entity and fencing generation.
7. Define stream and ACK durability precisely
Runner batches normalized deltas; never emit one WebSocket frame or database row per token.
progress_ack is a flow-control/replay watermark, not proof that the final response is durable.
Runner retains a bounded cumulative full/partial canonical response and terminal evidence until terminal_ack.
terminal_ack is sent only after Server durably stores the exact attempt terminal, usage status, response/result hash or artifact reference, and required canonical transition.
Live deltas may use a bounded low-latency cross-pod relay. Agent Loop correctness and recovery depend on the durable terminal aggregate or durable batched chunks, not on an in-memory relay.
Sequence gaps are NACKed/replayed. Duplicate batches and terminals are idempotent by exact identity and payload hash; the same identity with a different hash is quarantined as a conflict.
Slow Server/client consumption applies backpressure. Journal or spool exhaustion is a typed condition and must not silently drop terminal evidence.
8. Support horizontal Server scaling
Do not bind correctness to the in-process Runner connection pool.
Reuse the existing durable Edge dispatch claim/lease/generation/wakeup pattern for low-volume Runner inference control messages, but do not reuse the tool payload/result state machine for inference streams.
Durable command, route, attempt, terminal, and reconciliation facts remain in MatrixOne.
A socket-owner pod claims and delivers commands to the authenticated Runner.
A run-owner pod consumes normalized progress and the canonical durable terminal.
Terminal ingestion is idempotent and durable before Runner ACK, regardless of which pod owns the socket.
In-memory notification and live relays are disposable accelerators; database facts and the Runner journal recover missed notifications, reconnects, and pod failure.
Large canonical requests/results use bounded owner-scoped payload/artifact references rather than unbounded JSON queue rows.
9. Use one typed error and recovery contract
Every failure crossing the executor boundary includes:
Preserve success/failure with usage_unavailable; numeric zero is not measured zero billing.
Cancel before dispatch
Settle cancelled/not-dispatched and guarantee no provider I/O.
Cancel after dispatch races provider completion
Preserve separate user-control and provider-attempt facts; retain real usage and one terminal per entity/generation.
Terminal persisted but terminal ACK lost
Runner replays the exact terminal until Server idempotently accepts and ACKs it.
Binding/endpoint/credential rotates during an attempt
Active attempt remains pinned; the next attempt must resolve the new revision.
Socket owner and run owner are different pods
Durable command/terminal plus bounded relay; no sticky-session requirement.
Socket-owner, run-owner, Server, or Runner restart
Recover from canonical attempt facts and Runner journal without double dispatch.
Slow Runner, Server, or client; relay/channel saturation
Bounded queue/backpressure and typed degradation; never unbounded memory or silent critical-event loss.
Browser disconnect
Does not cancel the run; reconnect from durable run/transcript/provider projection.
Fallback would cross Runner, credential, billing, workspace, or data boundary
Deny by default; require explicit policy/user action and create a new observable route.
Malformed/oversized/malicious Runner event
Isolate the attempt, rate-limit/quarantine as appropriate, and keep unrelated runs usable.
Model Access and long-running behavior
A known personal or enterprise Runner Offering remains visible while offline but is not selectable for new inference.
An existing selection remains visible with typed reason, observed freshness, Runner identity, and recovery actions; it does not silently disappear or change model.
A run records the exact route and Runner binding used by every attempt. An active attempt never moves to another device.
Enterprise Runner pool failover, if supported, occurs only before provider dispatch and creates a new exact route/attempt under explicit organization policy.
Background, subagent, required compaction, reflection, memory, and verification purposes each require explicit eligibility to consume personal or organization-owned Runner capacity and billing.
Optional purposes degrade without blocking unrelated primary Work. A required inference pauses only the dependent branch and preserves completed transcript/task state.
Prompt-cache identity includes actual provider/model/protocol, canonical serialization version, Runner binding/capability/adapter revision, trust/data boundary, and credential generation where required by cache isolation.
Delivery plan
Implement as reviewed, independently truthful slices rather than one parallel runtime:
Canonical contract and threat model — update the owning design document, Runner terminology, guarantees/non-goals, state dimensions, error envelope, and fault matrix.
Truth cleanup — split route/material ownership and reject or reclassify the current fake Runner/Edge placement.
Exact-attempt executor seam — wrap the existing Server HTTP path without changing behavior; move retry ownership above the executor.
Runner binding and secret store — local setup, capability evidence, lease/revision handling, personal platform stores, and enterprise secret-provider interface.
Runner inference v1 — version negotiation and prepare/dispatch/event/terminal/ACK using the OpenAI-compatible adapter first.
Horizontal scale and load — durable control routing, cross-pod terminal convergence, bounded relays/backpressure, request/result artifacts, and multi-pod tests.
Product surfaces — Model Access projection, known-versus-selectable Offerings, CLI wizard, Web pairing/status, billing/credential/execution labels, and repair actions.
Additional adapters — Anthropic, Bedrock, Ollama, LM Studio, and private gateways through the same exact-attempt and conformance contract.
Acceptance criteria
Architecture and truth
Server, CLI, Web, and all Runner topologies share one Agent Loop and one canonical inference lifecycle.
Agent Loop code does not branch into separate Server/Runner context, policy, retry, trace, usage, or recovery semantics.
One canonical secret-free route and one inference ledger own resolution, admission, attempts, and settlement; no shadow route/state machine is introduced.
The executor interface performs exactly one physical provider attempt and has no hidden adapter retry.
Existing Server-originated gateway requests no longer report Runner/Edge placement.
Server and Runner adapters pass the same canonical request/event/terminal conformance suite.
A test provider/network harness verifies that a Runner attempt does not originate provider traffic from Server.
Server never receives or dereferences a Runner-local provider secret, endpoint, proxy credential, or private CA value.
Forged or cross-owner Offering, Runner, lease, workspace, binding, revision, attempt, stream, or terminal identity is rejected.
Redirect and proxy handling cannot leak authorization to an unapproved origin.
Endpoint/credential changes are locally validated and affect only later attempts.
No silent fallback crosses Runner, credential, billing, execution, workspace, or data boundaries.
Product text accurately states that Server still processes Agent context and model output.
Durability and unhappy paths
Fault-injection tests cover every boundary before/after attempt intent, prepare journal, provider-attempt commit, dispatch fence, provider connect, first response byte, terminal journal, Server terminal commit, and terminal ACK.
Disconnect before dispatch proves no provider I/O and permits safe same-attempt replay.
Disconnect after dispatch preserves uncertainty until Runner/provider reconciliation; it never triggers a blind new attempt.
Duplicate and conflicting protocol messages have deterministic idempotent/conflict outcomes.
Stream gaps replay from a watermark without corrupting the canonical response.
Cancel/provider-complete races preserve both control and provider/billing truth.
Server and Runner crash/restart do not double-dispatch an attempt.
401, 403, 429, retryable 5xx, connect failure, upload/header/body timeout, partial stream, missing usage, context overflow, vault failure, journal exhaustion, and terminal-ACK loss have typed outcomes.
Losing Runner capacity pauses only dependent inference branches; completed Work remains visible and recoverable.
Performance and scale
Persistent provider connections and bounded event batching avoid one frame or database write per token.
Slow Runners, providers, Server pods, and clients cannot create unbounded memory, relay, or local-journal growth.
Multi-pod tests cover different run-owner and Runner-socket-owner pods plus failure of each owner.
Terminal correctness does not depend on an in-process connection pool or live relay.
Load tests cover 100, 500, and 1,000 concurrent sessions without a global resolver/executor lock and report prepare, dispatch, TTFB, stream, settlement, replay, and backpressure metrics.
Product journey
A user can add, validate, save, select, diagnose, rotate, disable, and remove a personal model without exposing its secret.
An administrator can bind and govern an Enterprise Runner without routing its private endpoint or secret through Server.
Model Access independently displays source, Runner, execution placement, credential owner, billing owner, data boundary where relevant, availability, freshness, and actions.
Offline known Offerings remain visible but non-selectable; existing selections do not silently disappear or change.
Browser-only use never implies local inference and never stores a provider key in browser storage.
This device is shown only for a colocated client; remote clients show the stable Runner name.
BYOK usage is labeled as provider/Runner reported and distinguishes exact, partial, and unavailable usage.
Is there an existing issue for the same feature?
Is your feature request related to a problem?
Astra has one durable Agent Backbone on Server and multiple bounded capacity providers. Server correctly owns ContextPipeline, the Agent Loop, model routing, policy, retry, trace, usage aggregation, and durable run state. A Runner should execute work where the relevant credentials, network, and authority live.
The implemented external model path does not yet satisfy that contract.
AdmittedModelExecution::from_endpointplaces provider authorization and endpoint material in Server memory,ServerAgenticLoopHostconverts it into a normal Server LLM route, and the Server HTTP client opens the provider connection. The invocation can nevertheless be recorded asexecution_placement = edge.The current Edge WebSocket protocol also supports only tool request/result/cancel/ack. It has no inference preparation, exact provider-attempt admission, normalized stream, usage, terminal, or reconciliation lifecycle.
This creates four product and correctness problems:
The target behavior is already owned by
docs/design/model-access-and-inference.md: non-TaaS personal credentials remain in a Runner-local secret store, Server remains authoritative for canonical run and invocation state, and the selected Runner acts only as an inference executor.Design intent and first principles
This feature extends Astra's existing product thesis to inference:
The design follows these invariants:
Describe the feature you would like
Add first-class This device and Enterprise Runner model execution. The Agent Backbone remains on Server. An authenticated personal or organization-owned Runner executes one Server-admitted physical provider attempt using a local credential or private endpoint and returns normalized events and one canonical terminal result to the existing Agent Loop.
Terminology and product dimensions
Runneris the execution boundary.Edgeis a deployment topology, not a second runtime or the canonical execution-placement name. CLI-local, personal Edge, and enterprise deployments can all supply Runner capacity.The following facts are independent and must not be collapsed into one enum:
serverorrunner);personalorenterprise);For example, an organization-owned Workspace Offering may execute through an Enterprise Runner.
Enterprise Runnerdoes not automatically require a separate Model Access kind.Product, privacy, and trust contract
A Runner-executed Offering must guarantee:
Within Astra's trust model,
execution_placement = runnermeans Server did not materialize provider connection material or execute provider transport, and the attempt was bound to the named Runner. It does not cryptographically prove the behavior of a compromised Runner. Deployments that require stronger proof may add workload/host attestation or independently observed egress identity without changing the base inference lifecycle.This mode does not claim that Server cannot see the prompt, tool schemas, or model result. Server still owns ContextPipeline and the Agent Loop. A deployment where prompt and agent state never reach Astra Cloud requires the same Backbone to be self-hosted inside the enterprise boundary; it is not a
local=truevariation or a client-side Agent Loop.Product journey
Provide a local CLI/Runner setup flow similar to:
Capability diagnostics distinguish declared capability from observed probe evidence and show freshness. A probe that creates provider work is explicit and bounded because it may incur cost.
Known Offerings remain visible when unavailable so the user can understand and repair an existing selection. Visibility and selectability are separate:
This deviceis displayed only when the current client can establish that it is colocated with the selected personal Runner. Other clients display the stable Runner name instead.Canonical design
1. Implement the canonical secret-free route
Replace the combined
AdmittedModelExecutionmaterial with the existing targetResolvedInferenceRoutecontract rather than introducing a smaller parallel route type:Server credential references remain in the separate
ResolvedCredentialBinding. A Runner binding contains only an opaque local credential generation/reference; Server cannot dereference it and never receives the local endpoint.The existing provider-authorized gateway path must not continue reporting Runner/Edge placement while Server performs the HTTP request. Until Runner inference transport exists, classify that path truthfully as Server execution or reject it as unsupported; do not retain two interpretations.
2. Add one exact-attempt executor boundary
Introduce one typed inference facet consumed by a shared inference coordinator:
ServerAttemptExecutorwraps the current provider HTTP implementation.RunnerAttemptExecutoruses the durable Runner transport.fallback_fromfact.3. Add Runner-local model bindings and secret storage
A local model binding owns:
Runner advertisement contains only non-secret metadata, opaque binding identity, revisions, health evidence, supported inference purposes, and a finite lease.
Provide a
RunnerSecretStoreabstraction rather than a device-only vault:Secret values use redacting, non-serializable, zeroizing wrappers and are materialized only for one local attempt. Neither secrets nor endpoint values are written to advertisements or journals. Redirect, proxy, DNS, TLS, and private-CA behavior is explicit; authorization is never forwarded across an unapproved origin change.
Rotation increments the local binding/credential revision and affects the next attempt, not an active one.
4. Add a dedicated versioned inference protocol
Do not overload
EdgeServerMessage::ToolRequest. Inference has a distinct typed facet and lifecycle.The authenticated connection negotiates the inference protocol and adapter versions before a Runner Offering becomes selectable. Every message is bound to:
Frames have strict schema and size limits. Unknown critical fields/versions fail closed for the affected attempt; malformed sibling activity must not poison unrelated runs.
5. Use a two-phase prepare/dispatch boundary
dispatch_fencedand fsyncs before provider I/O.dispatch_started, normalized event batches, and a terminal aggregate.Prepared state may cache material only in bounded process memory. The durable local journal stores identity, revisions, hashes, custody state, bounded response/replay data, and terminal evidence; it never stores provider secrets. Any prompt/output persistence in the Runner journal is encrypted, bounded, TTL-governed, and disclosed because it is user data even when it is not a provider credential.
6. Represent delivery truth with orthogonal facts
Do not encode all semantics in one linear status enum. Persist at least:
streamingis progress, not execution authority.DeliveryUnknownprevents a new provider attempt unless policy has downstream idempotency/reconciliation evidence. Later Runner evidence appends a reconciliation fact; it does not erase the original period of uncertainty.Run/invocation cancellation and provider-attempt outcome are separate facts. If cancellation races a provider completion, the run may remain cancelled while the provider attempt records completion and billable usage. Exactly one terminal wins only within the same entity and fencing generation.
7. Define stream and ACK durability precisely
progress_ackis a flow-control/replay watermark, not proof that the final response is durable.terminal_ack.terminal_ackis sent only after Server durably stores the exact attempt terminal, usage status, response/result hash or artifact reference, and required canonical transition.8. Support horizontal Server scaling
Do not bind correctness to the in-process Runner connection pool.
9. Use one typed error and recovery contract
Every failure crossing the executor boundary includes:
Retry safety distinguishes:
These are not interchangeable operations.
Required unhappy-path behavior
wait_for_runner,choose_model, andcancel.ActionRequired; no secret-shaped error crosses to Server.NotDispatchedwith bounded occupancy and retry-after evidence.MayHaveDispatched/reconciliation pending; never blindly create a new attempt.DeliveryUnknown; no retry without provider idempotency/reconciliation.usage_unavailable; numeric zero is not measured zero billing.Model Access and long-running behavior
Delivery plan
Implement as reviewed, independently truthful slices rather than one parallel runtime:
DeliveryUnknown, and reconciliation.Acceptance criteria
Architecture and truth
Security, privacy, and policy
Debug, logs, trace, SSE, MatrixOne rows, artifacts, Runner journal, errors, crash diagnostics, and setup probes.Durability and unhappy paths
Performance and scale
Product journey
This deviceis shown only for a colocated client; remote clients show the stable Runner name.Non-goals
Additional information
Canonical design owners and current implementation boundaries:
README.md— Astra product thesis and Runner terminologydocs/design/ARCHITECTURE.mddocs/design/agent-backbone-capacity-provider.mddocs/design/model-access-and-inference.mddocs/design/capability-provider-runtime.mddocs/design/edge-cloud-execution.mddocs/design/runtime-lifecycle.mddocs/architecture/edge-cloud-sync-architecture.mdcrates/services/src/models.rscrates/services/src/inference_execution.rscrates/services/src/multi_agent/edge_dispatch.rscrates/runtime/src/server/model_execution_admission.rscrates/runtime/src/server/server_loop_host.rscrates/runtime/src/turn/llm/client.rscrates/runtime/src/turn/llm/durable.rscrates/astra-server-types/src/edge_ws_protocol.rscrates/astra-server-types/src/edge_connection_pool.rscrates/astra-edge/src/main.rscrates/astra-edge/src/invocation_journal.rs