Skip to content

Support Cloudflare AI Gateway as a saved provider - #4

Closed
rknightion wants to merge 8 commits into
dzhng:mainfrom
rknightion:feat/cloudflare-ai-gateway
Closed

rknightion wants to merge 8 commits into
dzhng:mainfrom
rknightion:feat/cloudflare-ai-gateway

Conversation

@rknightion

@rknightion rknightion commented Sep 27, 2026 •

Copy link
Copy Markdown

Adds Cloudflare AI Gateway as a saved provider, alongside the presets and the custom endpoint from #28.

Cloudflare serves Jev (typesafe/jev) with Unified Billing, but only on the gateway's Workers AI route, <gateway>/workers-ai/run/typesafe/jev. That route takes the model from the path and the token as cf-aig-authorization, and wraps the answer as {state: "Completed", result}. The TypeSafe paths (/typesafe/v1/systemone and friends) return 2008 Invalid provider, so it can't be set up as a custom endpoint.

Setup

jg auth --provider cloudflare --base-url https://gateway.ai.cloudflare.com/v1/ACCOUNT_ID/GATEWAY --stdin

Or pick "Cloudflare AI Gateway" in jg auth. The gateway URL can also be the gateway's custom domain. The token needs the account's AI Gateway Run permission.

How it works

  • The saved record is {provider: "cloudflare", baseURL, apiKey}. It shares baseURL, --base-url and validateBaseURL with the custom provider and has no model; --model is rejected.
  • endpointFor strips a trailing slash and rejects a URL that already has a /workers-ai or /compat route on it.
  • The evaluator rewrites the one request in its existing fetch hook: route, header, no model in the body, and redirect: "error" so the custom header can't follow a cross-origin redirect. Only a Completed envelope yields an answer. Anything else fails validation and is never scored.
  • Errors pass through unchanged, so 401/403/429/5xx keep their current handling. Cache identity needs nothing extra, provider plus gateway URL already separate it.
  • credentialProviders in providers.ts owns the provider order (presets, Cloudflare, custom last) for the auth menu, argument checks and error text.

The spec record, research and choices ledger are in specs/done/cloudflare-provider/.

Testing

  • bun run test (Docker): 125 bun, 18 provider protocol, 39 parser, 4 release and 30 harness tests pass
  • bun run test:installed: 42/42, including the cloudflare auth, doctor, search, cache and replacement journey and invalid Cloudflare records
  • Each transport rule (header, redirect refusal, Completed-only) fails its own evaluator test when removed
  • Live against a gateway custom domain with a Run-only token: auth, doctor, real searches, and a route-suffixed URL rejected
  • oxfmt --check . only flags upstream files this PR doesn't touch

For you to decide

  • validateBaseURL's empty-value message is now neutral, since two providers use it
  • Cloudflare uses the 15s default timeout and no token pacing. One live run hit 42 Workers AI 429s without retry-after and ended incomplete

rknightion and others added 3 commits September 27, 2026 12:03
Cloudflare serves Jev as a Workers AI model at
<gateway>/workers-ai/run/typesafe/jev with Unified Billing. Auth saves the
account's gateway URL (gateway.ai.cloudflare.com/v1/ACCOUNT/GATEWAY or a
custom domain) alongside the token. The evaluator adapts that one request:
path from the saved URL, token as cf-aig-authorization, model taken from the
path, {state, result} envelope unwrapped, redirects refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dpmZNjz5QApmRBSwvz2Ne
… the spec

Adds cloudflare to the reference transport and provider replay, the installed
saved-provider journey, conflicting-environment and invalid-record cases, and
the interactive auth prompt, all through a canonical fixture gateway routed by
the test-only preload. Only a {state: "Completed", result} envelope yields an
answer. specs/done/cloudflare-provider records the route evidence, choices and
verification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016dpmZNjz5QApmRBSwvz2Ne
@dzhng

dzhng commented Sep 27, 2026

Copy link
Copy Markdown
Owner

thanks! did you use the repo's spec and implement skills by any chance? looks like claude followed the internal conventions pretty well

and did you test it?

rknightion and others added 3 commits September 29, 2026 12:21
Resolves to upstream's tree; the Cloudflare provider is re-ported onto the
custom-endpoint seams in the next commit.
Cloudflare becomes a credential provider beside custom: its record is
{provider, baseURL, apiKey}, auth takes the gateway URL through the shared
--base-url flag and validateBaseURL, and endpointFor resolves it. The
evaluator still adapts the one request to the Workers AI route
(cf-aig-authorization, no model field, Completed envelope, no redirects).
A gateway URL that already carries a /workers-ai or /compat route is
rejected instead of doubled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UsM3MRh2GKBUmaPwA5E322
…skills

refactor-clean: drop the Cloudflare-only cache namespace (provider and
gateway URL already isolate it), simplify the envelope unwrap, and give
the saved-provider order one owner, credentialProviders, used by the
menu, argument validation and errors. code-review: no unused-binding
suppression or type cast. write-docs: link the provider record from the
CLI README. audit-choices: rewrite the choices ledger per the skill.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UsM3MRh2GKBUmaPwA5E322
@rknightion

Copy link
Copy Markdown
Author

No, to be honest. I had Claude build it, but it matched the conventions by reading the existing specs in specs/done (mostly provider-support) and copying their shape. It never ran write-spec or implement-spec.

Since then I've merged main in (merge commit, no force push) and re-ported it onto the custom endpoint work from #28. Cloudflare still can't just be a custom endpoint - CF only serves Jev on the Workers AI route (/workers-ai/run/typesafe/jev) with the token in cf-aig-authorization, no model in the body and the answer wrapped as {state: "Completed", result}. Posting to /systemone gets 2008 Invalid provider back. It does reuse --base-url / baseURL now instead of its own --gateway-url.

I did run the repo's skills over it this time: review (refactor-clean, code-review, write-docs) then audit-choices on the whole change. That dropped a Cloudflare-specific cache namespace it didn't need (provider + baseURL already keep it separate), simplified the envelope unwrap, gave the provider order one owner (credentialProviders) and rewrote choices.md in the skill's format.

Two things in choices.md are your call:

  • validateBaseURL's empty-value message is now "Provide the endpoint base URL." instead of "...for the custom endpoint." since Cloudflare calls it too. Happy to revert that.
  • Cloudflare gets the default 15s timeout and no token pacing. One live run hit 42 Workers AI 429s with no retry-after and finished incomplete, so it might want a budget at some point.

Testing, yes. bun run test in Docker is green (125 bun, 18 provider protocol, 39 parser, 4 release, 30 harness) and test:installed is 42/42 including a cloudflare saved-provider journey. I also broke each transport behaviour on purpose to check its test fails. Live against a real gateway with an AI Gateway Run-only token: auth, doctor, real searches, and a URL with the route pasted on the end gets rejected. Details are in specs/done/cloudflare-provider/assets/verification.md.

@rknightion rknightion closed this Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants