Grid Kind E2E Image Dependencies

[nerdalert]

Summary

Grid's Kind demos/E2Es currently validate the intended data-plane behavior, but some tests depend on temporary development images built from local Praxis AI and Praxis branches. We need to replace those temporary images with project-owned published images once the dependent Praxis AI and Praxis changes land.

This issue tracks what is blocked, what is already merged upstream, and what environment overrides are currently required to run the Grid demos with the temporary development images. Since these are all temporary I am avoiding adding them to the repo unless it becomes a blocker for anyone else looking into the project and getting the dependancies merged take longer than expected.

Status

Grid's xtask harness can run the full local Kind stack, but the gateway image must contain several AI/Praxis features that are not all available in a single published project image yet:

• grid_route
• grid_credential_inject
• openai_responses_format
• llm-d-compatible ext_proc
• mock EPP support for the llm-d-compatible provider-gateway path
• Praxis core support needed by the above filters

Until those are merged and published together, Grid's local demos use temporary GHCR images under ghcr.io/nerdalert/*.

These images are for development validation only. They should not be referenced from production manifests or production docs.

Upstream dependency status

Praxis AI

PR	Status	Blocks
praxis-proxy/ai#334 — feat(llm-d): move ext_proc compatibility into AI	Open	Canonical AI-owned llm-d-compatible ext_proc provider-gateway path. Needed for verify-llmd-compatible-routing.
praxis-proxy/ai#339 — feat(filters): add grid_route	Open; blocked by Praxis publish/version alignment	Canonical grid_route filter. Needed for project-owned Grid overlay/model-routing E2Es.
praxis-proxy/ai#386 — Grid credential inject	Open; depends on #339	Canonical grid_credential_inject filter. Needed for native credential-injection E2E without temporary images.

Praxis core

The local dev image currently depends on Praxis core changes that are merged but not yet available through the published dependency path used by Praxis AI:

PR	Status	Grid relevance
praxis-proxy/praxis#803 — feat(filter): add peer identity trust enforcement	Merged	Required for gateway-side peer identity trust enforcement.
praxis-proxy/praxis#776 — feat(ext-proc): add response-header lifecycle processing	Merged	Required by llm-d/ext_proc compatibility behavior.

Additional AI follow-up needed: mock EPP image

Grid's llm-d-compatible E2E needs a mock EPP image so the test can exercise the provider-gateway path without requiring a full llm-d deployment.

Current temporary dev image:

• ghcr.io/nerdalert/praxis-ai-mock-epp:dev-20260718-d560a3a

Current local reference source:

• /home/ubuntu/praxxis/ai-grid/prs/temp-ai-grid-e2e-combined/ai
• branch: temp/grid-e2e-combined
• commit: c194439

The current local AI fork contains the llm-d/ext_proc integration and test support patterns. A canonical AI PR should add the mock EPP test image/build path beside the llm-d integration, so Grid does not own or carry an AI-specific mock EPP.

Recommendation:

• open a small Praxis AI PR for the mock EPP test-support image after or alongside the llm-d integration path
• keep only one Grid E2E that depends on mock EPP: verify-llmd-compatible-routing
• use non-llm-d provider gateway paths for the rest of the Grid routing E2Es

Temporary development image overrides

To run the current Grid Kind demos against the temporary development images:

  export GRID_XTASK_GATEWAY_IMAGE=ghcr.io/nerdalert/praxis-ai:dev-20260720-a3e2a3b
  export GRID_XTASK_MOCK_EPP_IMAGE=ghcr.io/nerdalert/praxis-ai-mock-epp:dev-20260720-a3e2a3b
  export GRID_XTASK_MOCK_PROVIDER_IMAGE=ghcr.io/nerdalert/grid-mock-providers:dev-20260720-a3e2a3b
  export GRID_XTASK_OPERATOR_IMAGE=ghcr.io/nerdalert/grid-operator:dev-20260720-a3e2a3b
  export GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent

Meaning:

• GRID_XTASK_GATEWAY_IMAGE: Praxis AI gateway with llm-d ext_proc, grid_route, and grid_credential_inject.
• GRID_XTASK_MOCK_EPP_IMAGE: AI mock EPP for llm-d endpoint-picking tests.
• GRID_XTASK_MOCK_PROVIDER_IMAGE: Grid mock backend providers.
• GRID_XTASK_OPERATOR_IMAGE: Grid operator image.
• GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent: required so Kind pulls registry images instead of expecting local-only images.

Notes:

• GRID_XTASK_IMAGE_PULL_POLICY=IfNotPresent is required for registry images.
• Without it, xtask preserves the local-development default: imagePullPolicy: Never.
• These overrides are intentionally temporary and should move to praxis-proxy/* images after the upstream PRs are merged and published.

Demo/E2E commands using these images

Validates operator routing overlay generation in Kind. The temporary image overrides above need to be in place for the majority of the following demos.

cargo xtask env validate-operator-routing -c tests/env/operator-routing.toml

Responses API routing

Validates /v1/responses routing through openai_responses_format and
grid_route.

cargo xtask env verify-responses-routing -c tests/env/operator-routing-multisite.toml

llm-d-compatible provider-gateway routing

Validates the dedicated llm-d-compatible path through provider gateways: mock EPP, ext_proc, endpoint_selector, and provider routing.

This is the only Grid test that should require mock EPP.

cargo xtask env verify-llmd-compatible-routing -c tests/env/operator-routing-multisite.toml

Full Grid routing smoke

Validates full-grid routing across local, remote, cloud mock, and API mock backends.

cargo xtask env verify-full-grid-routing -c tests/env/operator-routing-two-provider.toml

API-provider fallback with static header injection

Validates API-provider fallback using the older static header-injection harness path.

cargo xtask env verify-api-fallback -c tests/env/operator-routing.toml

Native API credential injection

Validates grid_route plus grid_credential_inject using a mounted Secret file. Token bytes must stay out of the overlay and consumer ConfigMap.

cargo xtask env verify-api-fallback-native -c tests/env/operator-routing.toml

SWIM membership from env-var startup seeds

Validates SWIM membership from environment-variable startup seeds.

cargo xtask env verify-swim-membership -c tests/env/operator-routing.toml

SWIM membership from GridNetwork seeds

Validates SWIM membership from GridNetwork.spec.seeds.

cargo xtask env verify-swim-crd-seeds -c tests/env/operator-routing.toml

CRDT provider-state propagation

Validates CRDT provider-state propagation over SWIM.

cargo xtask env verify-swim-state -c tests/env/operator-routing.toml

Encrypted SWIM transport

Validates encrypted SWIM transport behavior and failure cases.

Desired end state

1. Praxis core publishes/releases a dependency version containing the merged functionality needed by Praxis AI.
2. Praxis AI merges and publishes images containing:
   • grid_route
   • grid_credential_inject
   • openai_responses_format
   • llm-d-compatible ext_proc
   • canonical mock EPP test image, or a documented replacement
3. Grid xtask image overrides point at project-owned images, not temporary nerdalert images.
4. Only the dedicated llm-d-compatible E2E depends on mock EPP.
5. Other Grid routing E2Es use non-llm-d provider paths to keep the validation matrix understandable and less brittle.

Acceptance criteria

• Grid demos can run from project-owned published images.
• No production Grid docs or manifests reference temporary fork images.
• verify-llmd-compatible-routing remains the single mock-EPP-dependent Grid smoke test.
• verify-responses-routing, verify-api-fallback-native, and general routing tests do not require mock EPP.
• The required image override environment variables are documented for local development until the project-owned images are available.