Skip to content

feat(inference): add The Grid as a remote inference provider - #6500

Open
CastilloLuis wants to merge 3 commits into
ogx-ai:mainfrom
the-gridai:thegridai/thegrid-provider
Open

CastilloLuis wants to merge 3 commits into
ogx-ai:mainfrom
the-gridai:thegridai/thegrid-provider

Conversation

@CastilloLuis

Copy link
Copy Markdown

Adds The Grid as a remote inference provider. Discussion issue: #6499.

Disclosure: I work on The Grid.

What it is

The Grid is a spot market for inference. Callers pick a market instrument — a task type and a quality tier (text-standard, code-prime, agent-max), or a lab-scoped market (claude-opus-latest) — and the request is filled by whichever supplier is competitive at the time. Because of that, the model field of a response names the model that actually served the request, not the instrument requested; asking for text-standard can return openai/gpt-oss-120b. That is called out in the adapter docstring so it doesn't read as a bug later.

/v1/chat/completions is OpenAI-compatible and /v1/models is served, so OpenAIMixin does the work and instruments are discovered rather than hardcoded. The adapter is shaped after remote::deepseek.

Capability surface — measured, not assumed

I probed the live API before writing the adapter, so it advertises only what The Grid actually serves:

Surface Live result What the adapter does
/v1/chat/completions works, incl. streaming + tool calls inherited from OpenAIMixin
/v1/models works inherited — instruments auto-discovered
response_format: json_schema works (returned valid structured output) left enabled — deliberately not copied from deepseek, which raises here
/v1/completions 404 NotImplementedError
/v1/embeddings 404 NotImplementedError
n > 1 accepted, but only one choice returned added to the n > 1 skip tuple

The json_schema line is the one worth a second look: deepseek is the template and it raises on json_schema, so copying it wholesale would have silently disabled a capability that works.

Live integration run

uv run --group test pytest --inference-mode live \
  --stack-config inference=remote::thegrid \
  --text-model thegrid/text-standard \
  tests/integration/inference

18 passed, 2 failed, 73 skipped in 51.89s

Both failures are test_inference_store_disabled.py::test_{non_,}streaming_chat_completion_without_store. That module boots its own in-process stack pinned to OpenAI with OPENAI_API_KEY="fake-key-for-replay"; under --inference-mode live the placeholder reaches the real OpenAI API and returns 401 Incorrect API key provided: fake-key*******play. The same two tests pass under --inference-mode replay:

uv run --group test pytest --inference-mode replay \
  --stack-config inference=remote::thegrid --text-model thegrid/text-standard \
  tests/integration/inference/test_inference_store_disabled.py

6 passed in 9.03s

So it's a live-mode harness artifact that would hit any provider, not something this adapter causes. Flagging rather than trimming it out of the numbers.

Other checks

  • pytest tests/unit/providers/inference533 passed (7 of them new, in test_thegrid_config.py)
  • uv run pre-commit run --all-files → clean
  • uv run pre-commit run mypy-full --hook-stage manual --all-files → passed
  • docs/docs/providers/inference/remote_thegrid.mdx is generated by scripts/provider_codegen.py, not hand-written

Files

File Change
src/ogx/providers/remote/inference/thegrid/{__init__,config,thegrid}.py new provider
src/ogx/providers/registry/inference.py RemoteProviderSpec entry
scripts/generate_target_models_docs.py "thegrid" in INTENTIONALLY_UNMAPPED_REGISTRY_PROVIDERS
tests/integration/inference/test_openai_completion.py two skip entries
tests/unit/providers/inference/test_thegrid_config.py new, 7 tests
docs/docs/providers/inference/remote_thegrid.mdx generated
docs/sidebars.ts nav entry

Two notes on scope. The Grid serves /v1/responses natively, but since OGX implements Responses server-side over chat completions this PR does not wire that up. And while adding the sidebar entry I noticed remote_deepseek and remote_mistral are missing from docs/sidebars.ts — I've left those alone rather than widen the diff, but happy to send a one-line follow-up.

Ask

Could a maintainer add a THEGRID_API_KEY CI secret (or trigger the recording workflow)? Until then the integration tests skip rather than fail.

The Grid is a spot market for inference. Its model ids are market
instruments -- a task type and a quality tier, such as text-standard,
code-prime or agent-max, plus lab-scoped instruments like
claude-opus-latest -- and a request is filled by whichever supplier is
competitive at the time, so the model named in a response differs from
the instrument that was requested.

The chat completions API is OpenAI-compatible and /v1/models is served,
so OpenAIMixin covers the adapter and instruments are discovered rather
than hardcoded. Only the surfaces The Grid actually serves are exposed:
/v1/completions and /v1/embeddings both return 404 upstream and raise
NotImplementedError here.

Signed-off-by: CastilloLuis <luisedcastillog@gmail.com>

@mattf mattf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CastilloLuis i don't have an api key for thegrid. will you run the integration suite in record-if-missing mode and include the recordings in the PR?

…uite

@mattf asked for the integration suite run in record-if-missing mode with
the recordings included. The provider was not reachable from any stack
config, so this adds the wiring first, then the recordings.

- ci-tests registers a thegrid provider behind ${env.THEGRID_API_KEY:+thegrid}
  and pre-registers thegrid/text-standard, mirroring how watsonx is added.
  Ids name market instruments rather than fixed models and the catalog is
  discovered from /v1/models, so pinning one keeps replay deterministic.
- A thegrid setup and a thegrid suite over test_openai_completion.py, the
  OpenAI-compatible chat file. /v1/completions and /v1/embeddings both 404
  upstream, so there is nothing else to point at.
- 20 recordings under tests/integration/inference/recordings, captured live.
- THEGRID_API_KEY: replay-mode-dummy-key in the workflow env and the docker
  passthrough, so replay needs no real key.

Recorded: 20 passed, 13 skipped against the live API.
Replayed with the dummy key, which is what CI does: 20 passed, 13 skipped
in 1.4s, no network.

Signed-off-by: CastilloLuis <luisedcastillog@gmail.com>
@CastilloLuis

Copy link
Copy Markdown
Author

@mattf done, recordings are in.

The provider was not reachable from any stack config, so I had to wire it up before anything could be recorded. Three commits' worth of change in one:

  • ci-tests now registers a thegrid provider behind ${env.THEGRID_API_KEY:+thegrid} and pre-registers thegrid/text-standard, mirroring how watsonx is added in ci_tests.py. Ids on The Grid name market instruments rather than fixed models and the catalog is discovered from /v1/models, so pinning one keeps replay deterministic.
  • A thegrid setup, and a thegrid suite over test_openai_completion.py. That is the whole applicable surface: /v1/completions and /v1/embeddings both 404 upstream, which is why the PR already adds The Grid to the skip lists in that file.
  • THEGRID_API_KEY: replay-mode-dummy-key in the workflow env and the docker passthrough, so nobody needs a real key to replay.
  • 20 recordings under tests/integration/inference/recordings/.

Verified both directions:

# recorded against the live API with a real key
--setup thegrid --inference-mode record-if-missing --file tests/integration/inference/test_openai_completion.py
20 passed, 13 skipped in 55.08s

# replayed the way CI runs it, dummy key, no network
THEGRID_API_KEY=replay-mode-dummy-key --setup thegrid --suite thegrid
20 passed, 13 skipped in 1.38s

Also added {"suite": "thegrid", "setup": "thegrid"} to ci_matrix.json and regenerated TARGET_MODELS.md, so the suite actually runs. Drop that entry if you would rather not spend a CI job on it; the recordings stand on their own either way.

The recordings hold url, method, payload and response only, no headers, so no credential is in them. I checked before committing.

@mergify

mergify Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be merged. @CastilloLuis please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants