Skip to content

feat(providers): Cloudflare Workers AI provider + the two runtime fixes it needs - #80

Closed
mikeumus wants to merge 4 commits into
sapientinc:mainfrom
mikeumus:feat/cf-deepseek
Closed

mikeumus wants to merge 4 commits into
sapientinc:mainfrom
mikeumus:feat/cf-deepseek

Conversation

@mikeumus

@mikeumus mikeumus commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a cloudflare model provider so Praxist can run peers against Cloudflare
Workers AI, plus the two runtime-side fixes that provider turned out to require.

Workers AI exposes one OpenAI-compatible surface at
https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1
and publishes no Anthropic Messages route, so the plugin rides the existing
OpenAI-compatible client and only swaps the base URL and the bearer credential
(CLOUDFLARE_API_KEY). The account id is part of the URL rather than the
credential, so it is interpolated once in praxist/core/cloudflare.py and
shared by the CLI registry, the Codex relay, and the research-loop stage.
CLOUDFLARE_BASE_URL overrides the whole URL when an operator needs it.

Model ids are full @cf/vendor/model paths (@cf/deepseek-ai/deepseek-v4-flash-0731,
@cf/zai-org/glm-5.3-flash). That is why the plugin declares a distinct
cloudflare_workers_ai api_format instead of reusing openai_compatible: the
generic format strips everything before the first / during model-name
normalization, which would rewrite @cf/deepseek-ai/deepseek-v4-flash-0731 to
deepseek-ai/deepseek-v4-flash-0731 — a name Workers AI rejects. The format is
handled exactly like openrouter in _normalize_model_for_api_format; it
changes nothing else about the wire protocol.

The provider is selectable through the normal configure-llm path
(PROVIDER_KEY_MAP / PROVIDER_REF_FOR_SHORT_NAME / CredentialResolver), with
@cf/deepseek-ai/deepseek-v4-flash-0731 as cheap_peer and
@cf/deepseek-ai/deepseek-v4-pro-0813 as strong_reasoner.

The two runtime fixes

Both were required to make the provider usable, and both surfaced only at
resolve/start time — praxist doctor passed while neither was in place, so
they are included here rather than split out:

  1. codex_sdk allowlist (plugins/agent_runtimes/codex_sdk/plugin.yaml) —
    compatible_model_providers did not list model_provider:cloudflare, so
    runtime resolution rejected the pairing even though Workers AI reaches the
    Codex runtime through the ordinary relay. claude_sdk is deliberately not
    extended: Workers AI has no Anthropic route.
  2. Credential forwarding (workflow_stages/research_loop/backend/agent.py,
    provider_env.py, stage.py) — _scoped_legacy_provider_env had no
    cloudflare branch, so the key and account id were scrubbed out of the peer
    subprocess environment and the relay started unauthenticated. The new branch
    forwards CLOUDFLARE_API_KEY (also mirrored to OPENAI_API_KEY, which is
    what the relay reads), CLOUDFLARE_ACCOUNT_ID, and CLOUDFLARE_BASE_URL,
    and leaves the Anthropic surface unset. freeze_provider_env and
    _provider_env gained matching entries so the variables are explicitly
    nulled for every other provider rather than leaking across a switch.

Scope

  • Commits: 8dd890f (provider), b7dfc9e (codex_sdk allowlist),
    e904864 (credential forwarding), e49c2a2 (docs).
  • Affected modules: new praxist/core/cloudflare.py; new
    praxist/plugins/model_providers/cloudflare/; touched praxist/cli/_env.py,
    praxist/cli/start.py, praxist/core/credentials.py,
    praxist/core/modeling.py, praxist/plugins/agent_runtimes/codex_sdk/
    (_relay.py, adapter.py, plugin.yaml), and
    praxist/plugins/workflow_stages/research_loop/ (backend/agent.py,
    provider_env.py, stage.py).
  • Public contracts: additive. One new api_format value
    (cloudflare_workers_ai), one new provider ref (model_provider:cloudflare),
    three new environment variables. _relay.py gains an exported
    provider_upstream() helper because Cloudflare's upstream cannot live in the
    static _PROVIDER_UPSTREAMS table; existing table entries are unchanged.
  • Compatibility: no existing provider, runtime, or env var changes behavior.
    A missing CLOUDFLARE_ACCOUNT_ID raises a named ValueError explaining which
    variable to set, surfaced as a RuntimeError from start_relay.
  • Task-agnostic rationale: the provider is a transport, and both fixes are
    gaps in shared provider plumbing that any OpenAI-compatible, account-scoped
    provider would hit.

Verification

Run on macOS 27.0 (arm64), Praxist 0.5.0, codex_sdk runtime via the relay:

  • praxist doctor --target claude — clean.
  • Real generations completed through the relay (HTTP 200) against
    @cf/deepseek-ai/deepseek-v4-flash-0731 and @cf/zai-org/glm-5.3-flash.
  • Test suite: 37 failures on this branch vs 41 on the main baseline
    (aa198d0). Every remaining delta was reviewed; the one difference not
    explained by the baseline is a timing flake, confirmed by re-running it.
  • ruff clean.

Not verified, and deliberately called out:

  • Only single-generation runs were exercised. Multi-generation behavior with
    this provider is unproven.
  • Under a long multi-peer run, streaming hit one reconnect. It recovered and the
    run continued, but the cause was not root-caused, so I would not claim the
    streaming path is production-hardened on this route yet.
  • No new unit tests are included in these commits. The behavior was validated
    end-to-end against the live endpoint rather than in the test suite; happy to
    add focused tests for workers_ai_base_url resolution order,
    _normalize_model_for_api_format on @cf/... ids, and the
    _scoped_legacy_provider_env cloudflare branch if you want them in this PR.

Checklist

  • The change is focused and does not include unrelated generated files.
  • Tests cover the affected behavior. — see the note above; validated
    end-to-end against the live endpoint, unit tests not yet written.
  • Documentation, templates, examples, and skills were updated where needed.
    (docs/guides/model-providers.md, docs/guides/agent-runtimes.md,
    docs/reference/plugins.md.)
  • No credentials, private task data, or research-run artifacts are included.
  • New dependencies and copied assets include their source and license terms.
    (No new dependencies; the provider reuses the existing OpenAI-compatible
    client.)
  • I have read .github/CONTRIBUTING.md and the contribution terms in LICENSE.md.

mikeumus and others added 4 commits September 1, 2026 00:16
Adds a first-class `cloudflare` model provider backed by Cloudflare
Workers AI's OpenAI-compatible endpoint, so research runs can use the
DeepSeek models already served on our account without a new external
provider key.

Workers AI publishes only the OpenAI chat-completions surface at
/client/v4/accounts/{account_id}/ai/v1 (probing /v1/messages and
/anthropic/v1/messages returns "No route for that URI"), so the provider
rides the existing OpenAI-compatible path and the Codex relay rather
than the Anthropic transport used by deepseek_alias/openrouter.

Two details drove the shape of the change:

- The account id lives in the URL, not the credential, so the base URL
  is interpolated at call time from CLOUDFLARE_ACCOUNT_ID (overridable
  with an explicit account id or a whole-URL CLOUDFLARE_BASE_URL)
  instead of being a static table entry.
- Workers AI model ids are full `@cf/vendor/model` paths. The generic
  openai_compatible normalizer strips everything before the first "/",
  which would corrupt them, so the provider declares its own
  `cloudflare_workers_ai` api_format that preserves the whole name.

Auth is a bearer token from CLOUDFLARE_API_KEY. Verified end-to-end
through the relay against both @cf/deepseek-ai/deepseek-v4-flash-0731
and @cf/deepseek-ai/deepseek-v4-pro-0813.
`praxist resolve` refused the new provider with "agent_runtime:codex_sdk is
not compatible with model_provider:cloudflare". The runtime plugin declares
an explicit provider allowlist that the previous commit did not update, so
the provider registered cleanly and passed `praxist doctor` but could not
actually start a run.

Adds cloudflare to codex_sdk's compatible_model_providers. It is left out
of claude_sdk's allowlist on purpose: Workers AI publishes no Anthropic
Messages route, so that pairing could never work.

Note that `praxist doctor` does not exercise this conformance check; only
`praxist resolve`/`start` do.
A launched run died immediately with "CLOUDFLARE_API_KEY is required for
Codex SDK provider cloudflare", both peers red after 6 seconds.

`_scoped_legacy_provider_env` builds the runtime credential environment from
a per-provider allowlist. cloudflare had no branch, so it fell through to the
legacy default that forwards only the Anthropic/OpenRouter variables, and the
bearer token never reached the Codex client — even though the key was present
in the launcher's own environment and in the persisted config.

Adds the cloudflare branch (CLOUDFLARE_API_KEY, OPENAI_API_KEY for the relay,
plus CLOUDFLARE_ACCOUNT_ID/CLOUDFLARE_BASE_URL so the account-scoped upstream
can be interpolated in the run process), mirroring the deepseek_alias branch.

Also adds CLOUDFLARE_API_KEY to the Codex adapter's _MODEL_CREDENTIAL_ENV_KEYS
so it is stripped from task shell/MCP child environments unconditionally. The
`_looks_secret` "KEY" heuristic already covered it, but that path has a
task_keys escape hatch and every sibling provider key is listed explicitly.

Neither gap is visible to `praxist doctor` or `praxist resolve`; only a real
run reaches this code.
Adds `model_provider:cloudflare` to the built-in provider list, the
codex_sdk relay routing table, and the plugin API reference. Records the
two things that are not obvious from the manifest: the endpoint base is
account-scoped (interpolated from CLOUDFLARE_ACCOUNT_ID, overridable via
CLOUDFLARE_BASE_URL), and the distinct `cloudflare_workers_ai` API format
exists only so `@cf/vendor/model` ids survive model-name normalization.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A5c7aqDR1adMAtrFtzbw7o

@rajpratham1 rajpratham1 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. The change correctly merges duplicate --disable-features / --enable-features switches instead of allowing later flags to override earlier ones. The added regression tests also cover the main cases, including default flags and extra_args. No issues found.

@electrixoul

Copy link
Copy Markdown
Collaborator

Thank you @mikeumus for contributing the Cloudflare Workers AI provider path and for documenting your live endpoint checks.

We have preserved your commits and integrated them into the shared validation/model-provider-expansion branch alongside the other model-provider contributions. During combination review, we retained the account-scoped Workers AI endpoint and Codex relay design, aligned the credential/runtime documentation, and added offline regression coverage for account/base-URL resolution, secret scoping, relay routing, model-ID preservation, plugin resolution, and CLI setup. The branch passed the complete local suite (3,091 tests, 6 expected skips), blocking local CI guardrails, unit/integration coverage, docs generation, and compile checks before push.

Our maintainer workflow is documented here: https://github.com/sapientinc/PRAXIST/blob/main/docs/guides/contributing.md#what-happens-after-submission. The validation branch is now ready for the separately scheduled real-task experiment stage. If that stage confirms there is no cross-task regression, we will promote the validated work to main while retaining your authorship.

We are closing this source PR because its implementation has been transferred to the validation branch, not because the contribution was rejected. This discussion and attribution remain available.

@electrixoul electrixoul closed this Sep 3, 2026
@mikeumus

mikeumus commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

You're welcome @electrixoul and @rajpratham1 ❤️

@mikeumus
mikeumus deleted the feat/cf-deepseek branch September 3, 2026 23:24
@Glen-SP

Glen-SP commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Update: this shipped. Your Cloudflare Workers AI provider and the two runtime
fixes it needed are now on main in 34e6de7, merged via #200, with co-author
credit.

Thank you for your contribution, @mikeumus

@mikeumus

Copy link
Copy Markdown
Contributor Author

You're welcome @Glen-SP 😀

Thank you for opening, sourcing and maintaining such a powerful tool. We use it now with a lot of the AI research that we do. 🙇🏻‍♂️💕

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants