Grid Ingress Gateway

[shaneutt]

In addition to egress capabilities to cloud services and 3rd party APIs, we need ingress capabilities so that outside clients can use the grid for inference, and have their requests intelligently routed much like internal workloads would, but with additional emphasis on geographical routing cues and regional affinities.

Re-use Praxis as the ingress Gateway for grid networks, where there can be multiple joined from different geographical locations. Allow for both k8s and non-k8s deployments.

[nerdalert]

Attaching WIP details here on this current spike:

Praxis Grid External Client Ingress and Global Load Balancing Plan

Scope: Praxis Grid, Praxis AI, Praxis core, and praxis-forge

Primary goal: Provide one stable OpenAI-compatible endpoint for external clients while preserving Grid's distributed control-plane and data-plane ownership boundaries.

Overview

Praxis Grid needs two complementary access patterns:

1. Workload ingress: A workload inside a Kubernetes cluster calls its cluster-local Praxis gateway. That local gateway acts as a logical inference provider over the Grid.
2. External client ingress: A customer calls one public DNS name. Global traffic management sends the connection to a healthy Praxis edge-ingress instance. That Praxis instance uses a Grid-rendered routing snapshot to select a provider site for the requested model.

The second pattern adds a logically global endpoint without introducing one central gateway process:

one public service name
does not mean
one public gateway instance

The recommended design is an active-active fleet of Praxis AI edge-ingress gateways behind managed DNS, Anycast, or a cloud global load balancer. Each edge has a location-specific Grid routing snapshot. Grid determines provider eligibility and candidate order; Praxis AI makes the request-time model match and forwards through Pingora.

Praxis should be described as the GLB data-plane target and L7 AI router, not as the complete global traffic-management system. DNS health steering, Anycast advertisement, DDoS protection, and internet-scale traffic absorption should remain external platform infrastructure.

For the first POC:

• Forge creates the shared container network and KIND clusters.
• Provider gateways are exposed on that network through MetalLB.
• A standalone Praxis AI edge container joins the same network and binds a host port.
• A Grid-specific overlay synchronizer copies the edge gateway's ConfigMap snapshot to a bind-mounted file.
• grid_route hot-reloads that file in-process.
• New external requests reroute after provider failure without restarting the Praxis edge container.
• Location affinity is demonstrated by giving the edge and provider sites regions and preferring a same-region provider when health and capacity are otherwise equal.
• Optional session affinity is keyed from authenticated tenant identity, an explicit client session identifier, and the normalized requested capability. It is bounded to Grid's currently eligible provider tier and never overrides health, authorization, residency, or failover.

Terminology

Term	Meaning in this plan
Global traffic manager	Managed DNS, Anycast, CDN, or cloud GLB that selects a healthy edge-ingress instance. It does not select an inference model provider.
Edge-ingress gateway	Public-facing Praxis AI instance receiving external OpenAI-compatible requests.
Site/consumer gateway	Cluster-local Praxis gateway used by Kubernetes workloads.
Provider gateway	Praxis gateway at a provider site that terminates Grid mTLS and forwards to local inference infrastructure.
Routing overlay	Grid-rendered, bounded snapshot of eligible and ordered model/tool candidates.
Location affinity	A preference for a nearby provider when policy, freshness, health, and capacity allow it. It is not a hard residency guarantee.
Session affinity	A bounded preference that sends requests carrying the same authenticated tenant, session ID, and capability to the same eligible provider when possible. It is not a guarantee and is abandoned on provider ineligibility.
GLB mode	A deployment/configuration profile for Praxis AI. It should not initially be a separate proxy implementation or a large runtime fork.

Answering the centralization question

The customer requirement is valid: one DNS record must provide access to models distributed across a fleet of clusters that are not directly exposed to users.

Note: The diagram shows roles from the external client’s point of view, not exclusive node types. A Praxis instance or cluster can act as an edge/consumer, a provider-side egress proxy, or both. In simple deployments the selected edge may also be the final provider hop; in larger deployments it may forward to a separate provider site.

The solution is logically centralized but physically distributed:

flowchart TB
    C["External client"] --> D["api.example.com<br/>DNS / Anycast / managed GLB"]
    D --> EA["Praxis edge A"]
    D --> EB["Praxis edge B"]
    EA --> PA["Provider site A"]
    EA --> PB["Provider site B"]
    EB --> PA
    EB --> PB
    G["Grid control planes"] -. "per-edge overlays" .-> EA
    G -. "per-edge overlays" .-> EB

Loading

• The name and API contract are global.
• The edge execution tier is replicated and active-active.
• The Grid control plane remains distributed.
• The provider clusters remain private.
• No single Grid operator or Praxis edge is required on the live request path for all traffic.

The workload path continues to coexist:

in-cluster workload
  -> cluster-local Praxis gateway
  -> local or remote provider gateway

external customer
  -> public DNS/global traffic manager
  -> nearest healthy Praxis edge-ingress gateway
  -> local or remote provider gateway

Recommended architecture

Two-stage routing

Global ingress needs two different routing decisions because they use different information.

Stage 1: client to edge

The global traffic manager chooses a healthy edge based on network proximity, latency, geography, policy, and edge health.

This decision happens before the HTTP request body is available. DNS-based systems commonly use resolver location or EDNS Client Subnet, and therefore their result is an approximation of client location. AWS Route 53 documents that latency steering chooses among regional load balancers from measured client-to-region latency, while Cloudflare exposes geo, proximity, dynamic, and least-outstanding-request policies over healthy pools.

Stage 2: edge to provider

Praxis AI parses the OpenAI-compatible request, extracts the model, and invokes grid_route. The loaded Grid snapshot provides eligible provider candidates ordered for that edge's routing perspective. Praxis selects a candidate, sets ctx.cluster, and the existing load_balancer/Pingora path forwards to the provider gateway.

DNS should not select the provider site because it does not know the requested model, tenant policy, provider capacity, or Grid freshness.

External request flow

sequenceDiagram
    participant Client
    participant GTM as Global traffic manager
    participant Edge as Praxis edge
    participant Grid as Loaded Grid snapshot
    participant Site as Provider gateway
    participant Model as llm-d / model backend

    Client->>GTM: TLS connection to api.example.com
    GTM->>Edge: Select healthy nearby edge
    Client->>Edge: POST /v1/chat/completions
    Edge->>Edge: Authenticate, authorize, limit, parse model
    Edge->>Grid: In-memory candidate lookup
    Grid-->>Edge: Selected provider cluster
    Edge->>Site: Grid mTLS request
    Site->>Site: Verify edge peer identity
    Site->>Model: Provider-local endpoint selection
    Model-->>Client: OpenAI-compatible response / SSE stream

Loading

The request path does not call Kubernetes, Grid operators, SWIM, CRDT, DNS control APIs, or the filesystem.

Control-plane flow

provider CRDs + metrics + SWIM/CRDT state
  -> Grid reconciliation
  -> edge-specific routing overlay ConfigMap
  -> projected ConfigMap volume in Kubernetes deployments
     OR Grid overlay-sync adapter for the standalone-container POC
  -> atomic file replacement
  -> grid_route validates and atomically swaps the in-memory snapshot
  -> subsequent requests use the new snapshot

Praxis AI itself should not become a SWIM member. The Grid operator or a future Grid edge agent represents the site in the Grid control plane. Praxis remains the data plane and consumes a local validated snapshot.

In the preferred production shape, every edge location has a colocated Grid operator or edge agent that joins the Grid and renders that edge's snapshot locally. In the first POC, an edge-control KIND cluster can provide that control-plane companion while the Praxis process remains a standalone container. The documentation must not shorten this to “Praxis joins SWIM.”

Current implementation baseline

The following was verified against the repositories on 2026-07-23:

Area	Current state	Consequence for this plan
Grid deployment package	Initial /deploy package landed in Grid commit a6e9d76.	Forge and the POC should consume this package rather than duplicating Grid installation manifests.
Forge	Initial CLI crate landed in Grid commit df54396. It currently implements doctor, plan, and config commands plus strict schema/validation and a structured CommandRunner.	Cluster, state, service, stack, certificate, and network lifecycle phases still need implementation before the edge service is fully declarative.
grid_route	Praxis AI PR #339 is open. The public PR description still describes static candidates.	The edge failover proof must be gated on the overlay-file hot-reload work landing.
Provider credential injection	Praxis AI PR #386 is open as a draft and depends on #339.	Outbound API-provider fallback remains separate from external caller authentication.
Gateway peer identity	Praxis core PR #801 merged.	Verified downstream mTLS peer identity is available to filters.
Gateway peer trust	Praxis core PR #803 merged.	Provider gateways can fail closed on untrusted edge/site certificates.
Consumer config	Grid can render a complete consumer Praxis config from a GatewayRef, including explicit endpoint transport.	A stable edge configuration can reuse this contract, but dynamic overlay data should be split from static endpoint/TLS topology.
Overlay loading	Current Grid docs still state that a ConfigMap change alone does not prove a gateway loaded the new routes.	Documentation must change only after #339's in-process overlay reload is merged and proven.
Provider scoring	Grid sorts candidates using backend kind, metrics, cost, and GridNetwork.spec.region. Provider region is not currently derived from GridSite.spec.region for each gateway perspective.	True per-edge location affinity requires a scoring correction; changing only local_site is not enough.

Relevant repository material:

• Grid architecture overview
• Grid routing architecture
• Grid consumer config
• Grid authentication and trust
• Forge issue #2
• Forge source

POC topology

Required components

1. Three KIND clusters on one Forge-managed Docker/Podman network:
   • provider-east
   • provider-west
   • edge-control or existing consumer/control cluster
2. Grid operator and CRDs installed from /deploy.
3. One Praxis provider gateway per provider cluster, reachable through MetalLB addresses on the Forge network.
4. Two inference simulators serving the same model, one in each region.
5. One standalone Praxis AI edge-ingress container:
   • connected to the Forge network;
   • bound to 127.0.0.1:8443 on the host;
   • configured as routing perspective edge-us-east;
   • using Grid mTLS when calling provider gateways.
6. One Grid overlay-sync process:
   • read-only access to the edge overlay ConfigMap;
   • writes to .forge/runtime/edge-us-east/grid-config.json;
   • shares that directory with Praxis read-only;
   • performs atomic file replacement.
7. Optional second edge container representing edge-eu-west to demonstrate that identical model/provider state produces a different location preference.

The edge-control component must have an explicit site identity. If currentGridSite lifecycle semantics cannot represent a consumer-only ingress site cleanly, add that as a control-plane contract instead of leaving the edge in an ambiguous perpetual Pending state.

POC DNS

Use:

curl --resolve api.grid.test:8443:127.0.0.1 \
  https://api.grid.test:8443/v1/chat/completions \
  -H 'Content-Type: application/json' \
  --data '{"model":"shared-model","messages":[{"role":"user","content":"hello"}]}'

This emulates the stable public name without changing host DNS. It is not a claim that Forge implements production DNS or Anycast.

POC network path

host client
  -> 127.0.0.1:8443
  -> standalone Praxis edge container
  -> Forge container network
  -> provider-site MetalLB address
  -> provider Praxis gateway
  -> llm-d-inference-sim

Provider gateways and inference services must not bind public host ports.

Location-aware affinity

Recommended initial behavior

Location is a property of the edge deployment, not a client-controlled routing
header:

edge-us-east has routing perspective:
  site=edge-us-east
  region=us-east
  zone=optional
  sovereigntyZone=optional

The global traffic manager first steers the client to edge-us-east. Grid then orders provider candidates for that edge:

1. Policy and trust eligibility.
2. Available/fresh providers.
3. Hard residency constraints, when implemented.
4. Provider health and capacity.
5. Same site, zone, or region preference.
6. Cost and fallback category.
7. Stable deterministic tie-break.

Location is an affinity, not a hard pin. A healthy cross-region provider must win over an unavailable same-region provider.

Required Grid scoring correction

Today, backendKind: local describes the provider relative to the operator/site that declared it. It does not necessarily mean local to an external edge. A provider that is local to provider-east is remote from edge-us-east.

Add a consumer-perspective structure:

struct RoutingPerspective<'a> {
    site: &'a str,
    region: Option<&'a str>,
    zone: Option<&'a str>,
    sovereignty_zone: Option<&'a str>,
}

enum SiteDistance {
    SameSite,
    SameZone,
    SameRegion,
    CrossRegion,
    Unknown,
}

For each GatewayRef:

• Resolve the consumer GridSite from localSiteName.
• Resolve the candidate's provider GridSite.
• Derive site distance from the two GridSite records.
• Use provider category and site distance as separate scoring inputs.
• Apply metrics/cost/freshness.
• Render an overlay ordered specifically for that gateway.

Fallback behavior must be explicit:

• Missing edge GridSite: fail location scoring or use a documented neutral perspective; do not silently label providers local.
• Missing provider region: use Unknown, not same-region.
• Sovereignty mismatch: exclude before scoring once residency policy exists.

Make Grid ordering authoritative

Grid's scoring is richer than the current grid_route fresh/local/tie-break logic. Overlay mode should not recompute or override that formula.

Recommended compatibility contract:

• Static candidates mode: preserve current grid_route freshness/local-site behavior.
• Overlay-file mode: Grid-rendered order, or an explicit bounded rank, is authoritative within matching candidates. Praxis may retain a hard fresh-over-stale safety rule but should not apply a second location formula.

Prefer an integer rank or ordered list over exposing floating-point internal scores as a public API. Add optional overlay metadata:

{
  "apiVersion": "grid.praxis-proxy.io/v1alpha1",
  "revision": "sha256:...",
  "generatedAt": "2026-07-23T18:00:00Z",
  "local_site": "edge-us-east",
  "candidates": []
}

Older overlays should remain readable during migration.

GLB and edge best practices

1. Active-active edge deployment

• Run at least two edge instances in separate failure domains for production.
• Do not make one "primary" edge the normal path and leave the other cold.
• Each edge receives its own local routing snapshot.
• An edge must continue serving its loaded snapshot during temporary control-plane disconnection.
• The POC may start with one edge but should make a two-edge topology an explicit follow-up acceptance test.

2. Keep global and model routing separate

• Managed DNS/Anycast steers only to healthy edge ingress.
• Grid/Praxis performs model, capability, policy, and provider routing.
• Do not publish provider gateways directly in global DNS.
• Do not require the global traffic manager to parse OpenAI request bodies.

AWS documents latency-based DNS steering to regional load balancers, while Cloudflare documents geo, proximity, dynamic, and least-outstanding-request steering among healthy pools:

• AWS Route 53 latency routing
• Cloudflare global traffic steering

3. Layered health and readiness

Use separate signals:

Signal	Question answered
Liveness	Is the Praxis process alive?
Edge readiness	Has it loaded a valid, sufficiently fresh overlay and required TLS/auth material?
Route readiness	Is at least one eligible route available for the public service?
Provider health	Is a particular model/provider currently usable?

The external global health check should remove a broken edge, not remove a
healthy edge merely because one provider cluster is unavailable.

Praxis's generic /ready behavior must be reviewed carefully: marking the whole
edge unready because any configured provider cluster has zero healthy endpoints
is too strict for a multi-provider Grid. A route-aware readiness policy is
preferable.

Route 53 health checks illustrate the standard separation between resource
health and DNS failover:

• Route 53 health checks and DNS failover

4. Bound stale control-plane state

Last-known-good routing is necessary, but serving an old snapshot forever is not a complete production policy.

Add:

• overlay revision and generation timestamp;
• reload-success/reload-failure counters;
• current overlay age;
• not_ready_after_seconds;
• optional hard expiry for environments that must fail closed.

Recommended behavior:

1. Bad/missing replacement keeps last-known-good.
2. Edge reports degraded when update age crosses the warning threshold.
3. Edge becomes globally unready after the configured readiness threshold.
4. Existing in-flight streams are not terminated solely because the overlay aged.
5. A stricter hard-expiry policy may reject new requests where stale routing
   violates security or residency requirements.

5. Retry safety

OpenAI-compatible inference calls are normally POST and can consume expensivecapacity even when the client never receives the response.

• Do not automatically replay a request after request bytes might have reached a provider.
• Safe pre-connect failover is acceptable before the upstream request is sent.
• Add request idempotency keys only with a defined cross-edge/provider dedupe contract.
• Bound attempts, total timeout, and retry budget.
• Never retry indefinitely across providers.

RFC 9110 states that proxies must not automatically retry non-idempotent requests unless their semantics are known to be idempotent:

• RFC 9110, idempotent methods

6. Streaming and connection draining

• Treat SSE inference streams as long-lived requests.
• Set read/idle timeouts intentionally for inference, not web-page defaults.
• Stop sending new traffic to an edge before termination.
• Give existing streams time to complete.
• Do not attempt to migrate an active stream between edges or providers.
• Document the maximum drain time and what happens to streams exceeding it.

Connection draining conventionally stops new traffic while allowing in-progress
requests to complete:

• Google Cloud connection draining

7. Overload protection

At the edge, configure:

• maximum connections;
• maximum request and response body sizes;
• header and model-name bounds;
• per-client and global request rate limits;
• concurrency/admission limits appropriate for long-running inference;
• memory ceiling;
• upstream connection/read/idle timeouts;
• circuit breaking;
• 429/503 responses with Retry-After where appropriate.

Praxis already provides connection limits, body limits, rate limiting, active and passive health checks, circuit breaking, memory pressure handling, and Pingora load balancing. Local per-process buckets are appropriate for the POC; production tenant quota needs shared or externally coordinated state.

8. Authentication and authorization

External caller authentication and provider credential injection are different:

customer bearer/JWT/API key
  -> authenticates and authorizes the customer at the edge

Grid site certificate
  -> authenticates the edge to a provider gateway

provider credential
  -> authenticates the final-hop gateway to a SaaS/cloud provider

Requirements:

• Terminate public TLS at the edge or a trusted managed front door.
• Authenticate customers before grid_route.
• Authorize tenant-to-model access before routing.
• Strip untrusted internal and location headers.
• Do not forward the customer's Authorization header to provider gateways.
• Replace it only at the authorized final hop when provider credentials are required.
• Keep public server certificates separate from Grid site mTLS certificates.
• Provider gateways trust the edge's Grid identity, not the end customer's certificate.

Current Grid provider access policy is site-oriented, not tenant-oriented. An edge site's provider eligibility does not authorize every customer to every model. Production external service must add request-time tenant/model policy.

For the localhost POC, bind the public port to 127.0.0.1 and clearly label missing customer auth as non-production. Praxis's CPEX policy filter is a possible JWT/OIDC path, but it is currently an optional/experimental build feature and should not be silently treated as a finished multi-tenant product contract.

9. Source location trust

• Prefer location inferred by the trusted global traffic manager.
• Give each edge a configured location identity.
• Do not trust an arbitrary client X-Region, X-Country, or forwarding header.
• If a CDN/LB supplies location headers, accept them only from trusted upstream
  addresses and strip client copies first.
• Do not log full client IP addresses unless operational and privacy policy requires them.
• Keep residency policy distinct from latency affinity.

10. Observability

Record bounded metadata:

• request ID and trace ID;
• edge site/region;
• requested model;
• selected provider site/cluster;
• overlay revision and age;
• route decision reason;
• auth/rate-limit outcome;
• connection, first-token, total, and stream duration latency;
• response status and token counts where available.

Expose:

• edge request/error/active-stream counts;
• auth and policy denials;
• rate-limit and overload rejections;
• route selections and failovers by model/site;
• provider connect failures;
• overlay reload success/failure/hash/age;
• edge readiness state;
• mTLS trust failures.

Do not log prompts, tool arguments, bearer tokens, API keys, certificate private keys, or provider credential values. Do not expose internal provider topology in public response headers.

11. DDoS and internet edge protection

Praxis should not initially replace a managed DDoS/WAF/Anycast service. Production deployment should put the Praxis edge fleet behind the organization's approved internet edge, while keeping Praxis responsible for AI-aware L7 authorization, routing, and provider forwarding.

12. Deterministic failure behavior

• Unknown model: bounded 404 or documented OpenAI-compatible error.
• No eligible provider: 503, not random fallback.
• Invalid overlay update: last-known-good remains active.
• Expired overlay: readiness degrades or requests fail according to policy.
• Provider loss: only new requests move; in-flight requests are not replayed.
• All candidate providers unavailable: fail closed with bounded error and
  Retry-After.

13. Anti-flapping and controlled failback

• Use consecutive health thresholds rather than one failed probe.
• Add debounce/hold-down windows to provider and edge state changes.
• Keep last-known-good configuration through transient control-plane errors.
• When a failed region returns, ramp traffic back with weights or a canary instead of immediately moving all new requests.
• Expose a manual drain/maintenance state distinct from unhealthy.
• Ensure DNS/GLB health thresholds and Grid provider thresholds do not create competing oscillations.

DNS responses are cached, so DNS failover should not be described as instantaneous even with a low TTL. Long-lived connections also remain on their selected edge until they close or are drained.

14. Session and state affinity

• Do not enable sticky sessions for stateless Chat Completions by default.
• An SSE stream remains on the selected edge/provider for the life of that connection.
• If Responses, Threads, agents, or tools rely on edge-local state, move that state to a shared tenant-scoped store or define explicit bounded affinity.
• Never depend on source-IP affinity as a durable customer identity.
• Affinity must not override provider unavailability, residency, or authorization policy.

Affinity contract

There are two distinct decisions:

1. Edge affinity: The managed global traffic manager selects a healthy, nearby Praxis edge. The initial design does not require a client to remain on one edge; every edge must be able to serve the request from shared authentication/policy state and its current Grid snapshot. HTTP keep-alive, HTTP/2, and an active SSE stream naturally remain on their established connection. Do not add source-IP stickiness.
2. Provider affinity: After Grid eligibility and ranking have been applied, Praxis may consistently prefer one provider for related requests. This is useful for provider-side prompt/KV cache locality or an explicitly stateful
   conversation, but it must remain subordinate to Grid's routing decision.

The proposed provider-affinity input is:

authenticated tenant ID
  + explicit session/conversation ID
  + normalized capability kind and name

example:
tenant-42 + conversation-7 + inference_model/shared-model

The affinity key and per-candidate rendezvous score should be computed with
keyed hashes:

affinity_key = HMAC-SHA-256(
  affinity_secret,
  version || tenant_id || session_id || capability_kind || capability_name
)

candidate_score = HMAC-SHA-256(
  affinity_key,
  stable_candidate_id
)

selected_provider = candidate with the highest candidate_score

Use length-prefixed or another unambiguous encoding rather than raw string concatenation. The HMAC prevents clients from predicting provider placement or brute-forcing session identifiers to target a particular provider. The affinity secret is runtime secret material distributed to the Praxis edge fleet; it must not appear in a Grid overlay, status output, request log, or metric label. Secret rotation needs a documented overlap strategy if preserving placement through rotation is required. Use the same secret only across edges that intentionally share an affinity domain; a different per-edge secret makes affinity local to that edge.

Identity and input rules:

• tenant_id comes only from the validated authentication/authorization context. Never accept it from an untrusted X-Tenant-ID request header.
• Define an explicit API header such as X-Praxis-Session-ID for the first implementation. Treat it as an opaque identifier, not as authorization.
• Validate the session ID before hashing: 1–128 bytes, no control characters, and exactly one value. An absent header disables affinity for that request; a present but malformed, oversized, or repeated header returns a bounded 400 error instead of silently changing routing behavior.
• Normalize the requested model/tool only after the request-format filter has parsed it. Include both capability kind and name so affinity cannot cross model or tool boundaries.
• If authenticated tenant context or the session header is absent, use ordinary Grid routing with no session affinity. A supplied invalid session header is still a 400. The localhost POC may use a fixed documented demo tenant, but that is not production behavior.
• Never log the raw tenant/session tuple. Log only a bounded route-decision reason and, if operationally necessary, a short non-reversible diagnostic digest.

Selection rules:

1. Grid excludes unauthorized, residency-incompatible, stale/unavailable, and otherwise ineligible providers.
2. Grid establishes the best eligible selection tier for the edge's routing perspective. Praxis must not use affinity to reach into a worse tier.
3. Praxis uses rendezvous hashing only among candidates in that best tier. Initial selection is uniform. Weighted selection must wait for an explicit, bounded Grid-owned weight in the overlay rather than inferring a weight from
   candidate order.
4. If the selected provider becomes ineligible, the same key deterministically maps to a surviving candidate. There is no attempt to preserve affinity to the failed provider.
5. A returning provider becomes eligible only after Grid's recovery and hold-down criteria. Stateless hashing may then remap some sessions. Strict “remain on the failover provider until the conversation ends” behavior
   requires shared or otherwise coordinated session state and is not part of the initial POC.

The overlay contract must make the affinity boundary explicit. Add an optionalselectionTier (lower is better) or equivalent Grid-owned grouping. Candidate order/rank remains authoritative across tiers; affinity is allowed only within the lowest eligible tier. For backward compatibility, an overlay without selection tiers keeps existing ordered selection and does not silently enable cross-candidate affinity.

A possible future Praxis configuration shape is:

- filter: grid_route
  local_site: edge-us-east
  overlay_file: /etc/grid/grid-config.json
  affinity:
    enabled: true
    session_header: X-Praxis-Session-ID
    secret_file: /etc/praxis/affinity/key
    algorithm: rendezvous
    max_session_id_bytes: 128

This is a proposed contract, not a claim that these fields already exist.
Configuration must reject affinity without overlay mode, without a secret, or
with an unsupported algorithm. The request path may read an already-loaded
secret but must not read the secret file, overlay file, Kubernetes, or Grid.

15. Capacity and autoscaling

• Scale on active requests/streams, admission queue, time to first token, token
  throughput, and memory pressure—not CPU alone.
• Keep spare edge connection capacity for regional failover.
• Apply load shedding before process exhaustion.
• Bound per-tenant concurrency separately from global process capacity.
• Test the “one edge region disappears” load transfer, not only steady-state
  horizontal scaling.

Security and trust model

External TLS

• Public certificate for api.example.com.
• TLS 1.3 preferred.
• Public key and certificate rotation independent of Grid CA rotation.
• Admin/metrics listener stays on loopback or a private management network.

Edge-to-provider mTLS

• Edge receives a distinct Grid site certificate.
• Provider gateway verifies the certificate chain.
• peer_identity_trust verifies the expected edge certificate identity.
• Production should prefer certificate digest or SPIFFE-style identity over
  organization-only matching.
• Direct provider network paths are unavailable to public clients.

Grid control-plane access

The overlay synchronizer requires only:

• get, list, and watch on ConfigMaps in a dedicated namespace;
• no Secret read;
• no provider CRD mutation;
• no cluster-admin kubeconfig.

The resulting overlay contains credential references only, never provider token
values.

Overlay distribution for the standalone-container POC

Kubernetes Praxis deployments should use a projected ConfigMap directory and
the grid_route parent-directory watcher. The standalone Docker/Podman edge
cannot directly mount a Kubernetes ConfigMap, so the POC needs an adapter.

Proposed grid-overlay-sync

Implement a small Grid-owned binary or service, not Forge-specific logic:

grid-overlay-sync
  --context kind-grid-edge-control
  --namespace grid-system
  --configmap grid-overlay-production-edge-us-east
  --key grid-config.json
  --output /runtime/grid-config.json

Behavior:

1. Perform an initial GET and fail startup if no valid non-empty snapshot exists.
2. Watch the named ConfigMap and retain resourceVersion.
3. Re-list and resume with exponential backoff and jitter after watch failure.
4. Enforce the same size and semantic bounds as the consumer.
5. Hash content and no-op when unchanged.
6. Write a sibling temporary file, flush, and atomically rename.
7. Never truncate or delete the last-known-good output after startup.
8. Publish health, revision, and update-age metrics.
9. Handle cancellation and join worker tasks on shutdown.

This is control-plane plumbing. It must never run synchronously from an HTTP request.

Production edge deployments should prefer:

• a local Kubernetes control-plane/agent and projected ConfigMap volume; or
• a defined Grid edge-agent distribution mechanism.

The POC sync adapter is not a reason to put Kubernetes access into Praxis AI.

Praxis AI edge profile

Do not initially add a separate proxy binary or hide behavior behind a large--mode glb code path. Add a reviewed grid-edge-ingress configuration profile using existing composable filters.

Illustrative chain:

listeners:
  - name: public-openai
    address: "0.0.0.0:8080"
    max_connections: 1000
    filter_chains:
      - observability
      - edge-security
      - grid-inference

filter_chains:
  - name: observability
    filters:
      - filter: request_id
      - filter: access_log

  - name: edge-security
    filters:
      - filter: timeout
        timeout_ms: 120000
      - filter: rate_limit
        mode: per_ip
        rate: 10
        burst: 20

  - name: grid-inference
    filters:
      - filter: json_body_field
        field: model
        header: X-Model

      - filter: grid_route
        local_site: edge-us-east
        model_header: X-Model
        overlay_file: /etc/grid/grid-config.json
        reload:
          enabled: true
          debounce_ms: 500
          keep_last_good: true

      - filter: load_balancer
        clusters:
          # Fixed, pre-authorized provider gateway topology.
          # Overlay reload changes candidate use, not endpoint/TLS definitions.

admin:
  address: "127.0.0.1:9901"

shutdown_timeout_secs: 120

This shape is illustrative until #339's final schema is merged. The final
example must be generated/tested against the actual Praxis AI schema.

Filter ordering for production:

request ID / tracing
-> trusted forwarded-header handling
-> external caller authentication
-> tenant/model authorization
-> rate/concurrency/body limits
-> model extraction
-> grid_route snapshot selection
-> optional final-hop provider credential injection
-> circuit breaker / load_balancer

The load-balancer cluster set remains static in the first version. Overlay hot-reload may change candidate membership/order/freshness only for clusters whose endpoints and TLS identities are already configured.

Forge integration

Current prerequisite

Forge F1 exists, but service and environment lifecycle do not yet. Implement generic Forge phases before adding Grid-specific edge configuration.

Generic Forge changes

Extend ServiceSpec in forge/src/config/mod.rs:

services:
  - name: grid-edge-us-east
    image: ghcr.io/praxis-proxy/ai:sha-...
    network: environment
    dependsOn:
      - grid-overlay-sync-us-east
    ports:
      - bindAddress: 127.0.0.1
        host: 8443
        container: 8080
        protocol: tcp
    volumes:
      - source: .forge/runtime/edge-us-east/praxis.yaml
        target: /etc/praxis/praxis.yaml
        readOnly: true
      - source: .forge/runtime/edge-us-east/grid-config.json
        target: /etc/grid/grid-config.json
        readOnly: true
      - source: .forge/runtime/edge-us-east/tls
        target: /etc/praxis/tls
        readOnly: true
    args:
      - --config
      - /etc/praxis/praxis.yaml
    healthCheck:
      type: tcp
      port: 8080
      interval: 2s
      timeout: 1s
      retries: 30

Required generic additions:

• environment network creation and ownership labels;
• KIND clusters attached to that network;
• service volumes;
• explicit port bind address;
• dependencies and health checks;
• restart policy;
• service start/stop/status/logs;
• atomic state store and ownership checks;
• up, down, status, and bounded parallel execution;
• Secret/certificate paths marked as sensitive and never printed.

Grid-specific values belong outside forge/, for example:

environments/grid-edge/
  forge.yaml
  README.md
  manifests/
  configs/
  policies/

Forge code must not know what a Grid overlay, GridSite, model, or Praxis filter is.

Recommended CLI proof

praxis-forge config validate --config environments/grid-edge/forge.yaml
praxis-forge doctor --config environments/grid-edge/forge.yaml
praxis-forge plan --config environments/grid-edge/forge.yaml
praxis-forge up --config environments/grid-edge/forge.yaml --parallel 3
praxis-forge status --config environments/grid-edge/forge.yaml
praxis-forge service logs grid-edge-us-east --follow
praxis-forge down --config environments/grid-edge/forge.yaml

Implementation phases

Phase 0 — Architecture contract and terminology

Files:

• docs/architecture/external-ingress.md — new
• docs/architecture/overview.md
• docs/architecture/routing.md
• docs/architecture/auth.md
• docs/architecture/consumer-config.md

Deliver:

• Document workload ingress and external ingress as complementary.
• Define global traffic manager versus Praxis edge-ingress gateway.
• State that the edge data plane is active-active.
• State that Praxis does not directly join SWIM.
• State that one DNS endpoint is a logical service, not a single process.
• Document two-stage location routing.
• Document tenant authorization and global traffic management as current gaps.

Acceptance:

• No claim that the current POC implements DNS, Anycast, DDoS, or production tenant isolation.

Phase 1 — Land grid_route overlay hot reload

Primary repository: praxis-proxy/ai, PR #339.

Inspect/change:

• filters/src/grid/mod.rs
• filters/src/grid/descriptor.rs
• filters/src/grid/overlay.rs
• filters/src/grid/route.rs
• server/src/server.rs
• server/src/watcher.rs
• server/src/reload.rs
• docs/filters/grid_route.md
• examples/configs/grid-route-overlay.yaml
• schema/integration tests

Requirements:

• static and overlay modes mutually exclusive;
• initial overlay required and valid;
• parent directory watched without canonicalizing the projected-file path;
• debounce and content hash;
• max size;
• semantic validation;
• atomic ArcSwap snapshot replacement;
• invalid/missing/empty replacement retains last-known-good;
• no request-path file or control-plane I/O;
• bounded watcher lifecycle and cleanup;
• reload success/failure/hash/age metrics;
• overlay order/rank contract documented.

Acceptance:

• Update the overlay from provider A to B and prove a new request selects B without process/listener restart.

Phase 2 — Per-gateway routing perspective

Primary repository: praxis-proxy/grid.

Inspect/change:

• operator/src/crd/grid_network.rs
• operator/src/crd/grid_site.rs
• operator/src/resources/routing_overlay.rs
• operator/src/controller/grid_network.rs
• scoring/src/scoring.rs
• generated CRDs under deploy/crds/
• routing/scoring docs and tests

Requirements:

• use GatewayRef.localSiteName to resolve the edge/consumer GridSite;
• derive consumer region/zone/sovereignty from that site;
• derive provider location from the candidate's GridSite;
• separate site distance from provider backend category;
• render candidate order per gateway perspective;
• preserve deterministic ties;
• fail safely on missing location;
• retain existing ordering behavior where no gateway perspective is configured;
• add optional overlay revision/generated timestamp/rank and selectionTier without breaking older readers;
• make the stable candidate identity used for affinity explicit and invariant under candidate reordering.

Tests:

• edge-east prefers provider-east over provider-west with equal metrics;
• edge-west prefers provider-west;
• unavailable same-region provider is excluded;
• stale same-region provider loses to fresh cross-region provider;
• a busy same-region versus idle cross-region decision follows documented scoring weights;
• spoofed request location cannot affect ordering;
• provider access policy evaluates the edge site's labels;
• selection tiers preserve policy, health, residency, and location ordering;
• a candidate's stable identity does not change when its list position changes;
• older overlays without selectionTier preserve their current ordered behavior.

Phase 2B — Bounded provider session affinity

Primary repository: praxis-proxy/ai, with the overlay schema/grouping supplied by praxis-proxy/grid.

Inspect/change:

• filters/src/grid/mod.rs
• filters/src/grid/descriptor.rs
• filters/src/grid/overlay.rs
• filters/src/grid/route.rs
• the existing authenticated request-context plumbing used by authorization
  filters
• docs/filters/grid_route.md
• examples/configs/grid-route-overlay.yaml
• schema and integration tests

Requirements:

• add a strict AffinityConfig under grid_route;
• load the HMAC secret during filter construction, never on the request path;
• reject affinity outside overlay mode, without a readable non-empty secret, or with an unsupported algorithm;
• derive tenant identity from authenticated context, never an untrusted tenant header;
• accept one bounded X-Praxis-Session-ID value;
• return 400 for a present malformed, oversized, or repeated session header;
• normalize capability kind/name before building the key;
• use HMAC-SHA-256 rendezvous hashing against a stable candidate ID;
• select only within Grid's lowest eligible selectionTier;
• preserve existing ordered selection and keep affinity disabled for older overlays without tiers;
• immediately remap when the selected provider is absent or ineligible in a newly loaded snapshot;
• expose bounded decision metrics without tenant/session labels;
• zeroize secret buffers where practical and never serialize or log the secret;
• keep request processing free of filesystem, Kubernetes, Grid, SWIM, or CRDT calls.

Expected configuration structures:

#[derive(Debug, Deserialize)]
#[serde(deny_unknown_fields)]
struct AffinityConfig {
    enabled: bool,
    session_header: String,
    secret_file: PathBuf,
    algorithm: AffinityAlgorithm,
    max_session_id_bytes: usize,
}

#[derive(Debug, Deserialize)]
#[serde(rename_all = "snake_case")]
enum AffinityAlgorithm {
    Rendezvous,
}

struct AffinityRuntime {
    session_header: HeaderName,
    secret: SecretBytes,
    algorithm: AffinityAlgorithm,
    max_session_id_bytes: usize,
}

Construction should validate the header name and bounds, then replace secret_file with a non-serializable in-memory secret held by the filter. For the initial implementation, read the secret once at construction; rotation
requires a documented filter/full-config reconstruction. Secret-file hot reload is a separate lifecycle feature and must not be accidentally coupled to overlay reload. The request-time helper should accept already parsed authenticated tenant identity, session bytes, normalized capability, and an Arc<RouteSnapshot>; it should not accept a path or a control-plane client.

Tests:

• equal affinity inputs select the same provider within the best tier;
• a fixed corpus of session IDs deterministically reaches every equivalent provider without relying on a probabilistic assertion;
• affinity never selects a provider from a worse selection tier;
• provider failure deterministically remaps the session to a surviving candidate;
• a client-supplied tenant header cannot alter the affinity key;
• missing session ID uses ordinary non-affinity selection;
• malformed, oversized, or repeated session header returns 400;
• source IP is not part of the affinity key;
• raw tenant/session identifiers and the affinity secret do not appear in logs, metrics, overlays, or status.

Phase 3 — Grid overlay synchronization adapter

Primary repository: praxis-proxy/grid.

Suggested files:

• operator/src/bin/grid_overlay_sync.rs
• operator/src/overlay_sync.rs
• focused tests using a mock/watch stream and temporary directory

Requirements:

• initial GET;
• get/list/watch only;
• resourceVersion resume;
• reconnect with exponential backoff and jitter;
• bounded snapshot;
• hash no-op;
• atomic write;
• last-known-good;
• graceful cancellation;
• no Secret access;
• health and update-age diagnostics.

Alternative:

• If the edge POC is moved into Kubernetes, skip this binary and mount the ConfigMap directory directly. Do not build both mechanisms without a clear need.

Phase 4 — Forge service/network lifecycle

Primary repository: praxis-proxy/grid, generic code only in forge/.

Inspect/change:

• forge/src/cli.rs
• forge/src/config/mod.rs
• forge/src/config/validate.rs
• forge/src/command/mod.rs
• new forge/src/state/
• new forge/src/runtime/
• new forge/src/cluster/
• new forge/src/service/
• integration tests

Requirements:

• implement atomic locked state;
• create one owned environment network;
• attach KIND and standalone services;
• implement service volume/port/dependency/health configuration;
• ensure 127.0.0.1 bind is representable;
• add up/down/status;
• never shell-concatenate configuration values;
• remove only Forge-owned resources;
• preserve state on partial failure;
• make reruns idempotent.

Acceptance:

• up/status/down works twice;
• interrupted up resumes;
• edge container can reach both provider MetalLB endpoints;
• provider endpoints are not exposed on the host.

Phase 5 — Edge deployment profile

Repositories:

• Praxis AI owns the tested edge filter-chain example.
• Grid owns the Grid-specific Forge environment and POC wiring.
• Praxis core remains generic.

Add:

environments/grid-edge/forge.yaml
environments/grid-edge/README.md
environments/grid-edge/configs/praxis-edge.yaml
environments/grid-edge/manifests/
environments/grid-edge/policies/

Requirements:

• pinned Praxis AI image;
• host listener;
• loopback-only POC bind;
• separate admin listener;
• explicit body/connection/time limits;
• edge site certificate;
• provider mTLS;
• peer_identity_trust at providers;
• Grid overlay hot reload;
• selected route evidence in internal logs/metrics;
• no provider port exposed to the host.

Phase 6 — Edge ingress E2E

Keep Grid assertions out of generic Forge code. Add a Grid-specific verifier, initially under xtask or a dedicated grid-e2e binary.

Test:

1. Start the Forge environment.
2. Record edge container ID, start timestamp, PID, and restart count.
3. Send shared-model request through api.grid.test.
4. Prove provider-east handled it.
5. Mark provider-east unavailable or stop it.
6. Wait for Grid overlay revision to change.
7. Wait for sync and Praxis reload success.
8. Send a new request.
9. Prove provider-west handled it.
10. Assert the edge container identity/PID/start time did not change.
11. Assert no rollout or restart command ran.
12. Write an invalid overlay update.
13. Prove last-known-good provider-west remains active.
14. Restore provider-east and prove deterministic recovery.
15. Send repeated requests with the same authenticated demo tenant, session header, and model; prove they select the same healthy provider.
16. Send a fixed corpus of session IDs; prove every equivalent provider is selected while no request crosses Grid selection tiers.
17. Fail the selected provider; prove the existing session remaps without an edge restart.

Security assertions:

• unknown model fails cleanly;
• direct provider endpoints are unreachable from the host;
• missing/untrusted edge certificate is rejected by provider gateway;
• public/client authorization is not forwarded as a provider credential;
• token bytes are absent from overlay, config, logs, and status;
• client-supplied location header does not change routing.

Phase 7 — Production GLB deployment

After the POC:

• deploy two or more edge instances per supported geography;
• integrate with the platform's managed DNS/Anycast/GLB;
• define public certificate and DNS ownership;
• implement route-aware readiness;
• implement overlay-age readiness policy;
• implement external JWT/API-key identity and tenant/model authorization;
• use shared tenant quota/accounting state;
• integrate WAF/DDoS controls;
• define autoscaling signals;
• define drain and long-stream termination policy;
• run regional edge and provider failure exercises;
• add SLOs and route-decision dashboards.

Acceptance criteria

Architecture

• One stable public DNS name can target multiple active Praxis edges.
• External and workload ingress are both documented.
• Grid remains the provider discovery/scoring/overlay control plane.
• Praxis AI remains the request-time data plane.
• Provider clusters are not directly public.
• Praxis does not call Kubernetes or SWIM from the request path.

POC

• Standalone Praxis edge container runs on the Forge network.
• Host client reaches only the edge port.
• Same OpenAI-compatible endpoint reaches models in multiple clusters.
• Location-affinity test prefers same-region provider under equal conditions.
• When enabled, session affinity uses authenticated tenant + validated session ID + normalized capability and remains inside Grid's best eligible selection tier.
• Provider failure overrides session affinity and remaps new requests to a surviving eligible provider.
• Source IP and untrusted tenant/location headers cannot control affinity.
• Provider loss regenerates and propagates the overlay.
• Praxis applies it in-process.
• New traffic reaches a surviving provider.
• Edge process/container restart count remains zero.
• Invalid update retains last-known-good.

Production readiness before customer exposure

• Active-active edge fleet.
• Managed global traffic steering and health checks.
• Public TLS ownership and rotation.
• Customer authentication and tenant/model authorization.
• Shared quota/accounting or explicitly bounded local-only behavior.
• Route-aware readiness and overlay-age policy.
• Connection draining for long-lived streams.
• A documented affinity contract, secret-rotation procedure, and recovery
  behavior if session affinity is enabled.
• No unsafe automatic POST replay.
• DDoS/WAF integration.
• Regional failover and recovery tests.

Non-goals

• Building authoritative DNS or an Anycast BGP controller in Grid.
• Making Praxis AI speak SWIM/CRDT.
• Adding Kubernetes calls to the request path.
• Recomputing the complete Grid scoring formula in Praxis.
• Exposing provider clusters directly to customers.
• Treating a single host-port container as production GLB.
• Solving all tenant billing, quota, and residency policy in the POC.
• Retrying arbitrary inference POSTs across providers.
• Migrating an in-flight SSE stream between edges.

Open decisions

1. Global traffic manager owner: Which platform product owns public DNS, Anycast/CDN, DDoS, and edge health steering?
2. Edge control-plane attachment: Does each production edge have a local Kubernetes/Grid operator, or does Grid need a standalone edge agent?
3. Overlay order contract: Is ordered candidate position sufficient, or should the wire format add explicit integer rank?
4. Overlay staleness: When should an edge become unready, and when—if ever—should it stop serving last-known-good?
5. External identity: Which JWT/OIDC/API-key mechanism is the supported product contract?
6. Tenant policy: Where is tenant-to-model/region authorization enforced and audited?
7. Global quota: Which shared store or external service coordinates limits across edge instances?
8. Streaming: What are the maximum request, idle, and drain durations?
9. Model discovery: Is /v1/models generated from the same authorized snapshot, and how is it filtered per tenant?
10. Endpoint topology lifecycle: How are newly joined provider gateway endpoints added without making routing churn require full Praxis reload?
11. Consumer-only edge lifecycle: Does a public ingress location use an existing GridSite lifecycle, or does Grid need an explicit consumer/edge site role and readiness state?

Research references

• Praxis Grid architecture
• Praxis Grid routing
• Praxis Grid consumer config
• Praxis Grid authentication
• Praxis AI grid_route PR #339
• Praxis AI credential injection PR #386
• Praxis peer identity PR #801
• Praxis peer trust PR #803
• Pingora
• AWS Route 53 latency routing
• AWS Route 53 health checks
• Cloudflare global traffic steering
• RFC 9110 HTTP semantics
• Google Cloud connection draining
• OWASP API resource-consumption guidance