feat(inference): add The Grid as a remote inference provider - #6500
CastilloLuis wants to merge 3 commits into
Conversation
The Grid is a spot market for inference. Its model ids are market instruments -- a task type and a quality tier, such as text-standard, code-prime or agent-max, plus lab-scoped instruments like claude-opus-latest -- and a request is filled by whichever supplier is competitive at the time, so the model named in a response differs from the instrument that was requested. The chat completions API is OpenAI-compatible and /v1/models is served, so OpenAIMixin covers the adapter and instruments are discovered rather than hardcoded. Only the surfaces The Grid actually serves are exposed: /v1/completions and /v1/embeddings both return 404 upstream and raise NotImplementedError here. Signed-off-by: CastilloLuis <luisedcastillog@gmail.com>
mattf
left a comment
There was a problem hiding this comment.
@CastilloLuis i don't have an api key for thegrid. will you run the integration suite in record-if-missing mode and include the recordings in the PR?
…uite @mattf asked for the integration suite run in record-if-missing mode with the recordings included. The provider was not reachable from any stack config, so this adds the wiring first, then the recordings. - ci-tests registers a thegrid provider behind ${env.THEGRID_API_KEY:+thegrid} and pre-registers thegrid/text-standard, mirroring how watsonx is added. Ids name market instruments rather than fixed models and the catalog is discovered from /v1/models, so pinning one keeps replay deterministic. - A thegrid setup and a thegrid suite over test_openai_completion.py, the OpenAI-compatible chat file. /v1/completions and /v1/embeddings both 404 upstream, so there is nothing else to point at. - 20 recordings under tests/integration/inference/recordings, captured live. - THEGRID_API_KEY: replay-mode-dummy-key in the workflow env and the docker passthrough, so replay needs no real key. Recorded: 20 passed, 13 skipped against the live API. Replayed with the dummy key, which is what CI does: 20 passed, 13 skipped in 1.4s, no network. Signed-off-by: CastilloLuis <luisedcastillog@gmail.com>
|
@mattf done, recordings are in. The provider was not reachable from any stack config, so I had to wire it up before anything could be recorded. Three commits' worth of change in one:
Verified both directions: Also added The recordings hold url, method, payload and response only, no headers, so no credential is in them. I checked before committing. |
|
This pull request has merge conflicts that must be resolved before it can be merged. @CastilloLuis please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork |
Adds The Grid as a remote inference provider. Discussion issue: #6499.
Disclosure: I work on The Grid.
What it is
The Grid is a spot market for inference. Callers pick a market instrument — a task type and a quality tier (
text-standard,code-prime,agent-max), or a lab-scoped market (claude-opus-latest) — and the request is filled by whichever supplier is competitive at the time. Because of that, themodelfield of a response names the model that actually served the request, not the instrument requested; asking fortext-standardcan returnopenai/gpt-oss-120b. That is called out in the adapter docstring so it doesn't read as a bug later./v1/chat/completionsis OpenAI-compatible and/v1/modelsis served, soOpenAIMixindoes the work and instruments are discovered rather than hardcoded. The adapter is shaped afterremote::deepseek.Capability surface — measured, not assumed
I probed the live API before writing the adapter, so it advertises only what The Grid actually serves:
/v1/chat/completionsOpenAIMixin/v1/modelsresponse_format: json_schemadeepseek, which raises here/v1/completionsNotImplementedError/v1/embeddingsNotImplementedErrorn > 1n > 1skip tupleThe
json_schemaline is the one worth a second look:deepseekis the template and it raises onjson_schema, so copying it wholesale would have silently disabled a capability that works.Live integration run
Both failures are
test_inference_store_disabled.py::test_{non_,}streaming_chat_completion_without_store. That module boots its own in-process stack pinned to OpenAI withOPENAI_API_KEY="fake-key-for-replay"; under--inference-mode livethe placeholder reaches the real OpenAI API and returns401 Incorrect API key provided: fake-key*******play. The same two tests pass under--inference-mode replay:So it's a live-mode harness artifact that would hit any provider, not something this adapter causes. Flagging rather than trimming it out of the numbers.
Other checks
pytest tests/unit/providers/inference→ 533 passed (7 of them new, intest_thegrid_config.py)uv run pre-commit run --all-files→ cleanuv run pre-commit run mypy-full --hook-stage manual --all-files→ passeddocs/docs/providers/inference/remote_thegrid.mdxis generated byscripts/provider_codegen.py, not hand-writtenFiles
src/ogx/providers/remote/inference/thegrid/{__init__,config,thegrid}.pysrc/ogx/providers/registry/inference.pyRemoteProviderSpecentryscripts/generate_target_models_docs.py"thegrid"inINTENTIONALLY_UNMAPPED_REGISTRY_PROVIDERStests/integration/inference/test_openai_completion.pytests/unit/providers/inference/test_thegrid_config.pydocs/docs/providers/inference/remote_thegrid.mdxdocs/sidebars.tsTwo notes on scope. The Grid serves
/v1/responsesnatively, but since OGX implements Responses server-side over chat completions this PR does not wire that up. And while adding the sidebar entry I noticedremote_deepseekandremote_mistralare missing fromdocs/sidebars.ts— I've left those alone rather than widen the diff, but happy to send a one-line follow-up.Ask
Could a maintainer add a
THEGRID_API_KEYCI secret (or trigger the recording workflow)? Until then the integration tests skip rather than fail.