Skip to content

✨ [v5] Keep every coding CLI's model list current, like Claude's: copilot, codex, agy and omp #19028

Closed
3 of 3 issues completed

Description

@Danathar

Summary

#18788 added a weekly job that keeps the Claude model allowlist in step with the Claude Code version pinned in the image. Every other coding CLI still relies on hand-edited lists. This issue extends the same treatment to every backend, so all providers work the same way:

  • CI keeps the upper limit: the models the CLI version pinned in src/Dockerfile can actually launch.
  • Each hive shows what its own login can use, filtered by that limit. CI never pushes a model list to running hives.

Do it in parts. Each part below is self-contained and can be its own PR. Write it for v5; it will be forward-ported to v6.

All links are pinned to v5 at c30d0a91dc2e66aef1eb5cf09b22c2124290e5c7. Re-check them against a fresh pull before editing.

Background: how the Claude version works today

  • discoverClaudeModels (cli_models.go:1979) calls Anthropic's /v1/models with the hive's own credential, then keeps only ids in claudePinnedCLIModels (cli_models.go:461).
  • pinnedCLIModelAllowlist (cli_models.go:1210) has allowlists for claude, copilot and gemini only.
  • .github/workflows/claude-model-freshness.yml plus .github/scripts/claude-model-freshness.sh and check-static-model-freshness.py --apply claude find new candidates, install the pinned CLI, ask it offline whether it recognises each id (with control checks), and open one PR on a bot branch. That PR always carries needs-human and is never auto-merged. Read ✨ Weekly PR adds new Claude models to the lists and price table once the pinned Claude Code accepts them #18788's description before starting; it explains the gate and the control checks.
  • .github/workflows/static-model-freshness.yml only files a drift report issue for the other lists. It does not change them.

Current state per backend

Backend Pin in src/Dockerfile What the hive asks at runtime Allowlist CI keeps it fresh
claude CLAUDE_CODE_VERSION=2.1.327 Anthropic /v1/models with the hive's own login claudePinnedCLIModels yes (#18788)
copilot COPILOT_VERSION=1.0.88 Copilot SDK catalogue with the hive's login copilotPinnedCLIModels no (report only)
gemini none (no Gemini CLI installed) Google models API with GEMINI_API_KEY geminiPinnedCLIModels no (report only)
codex CODEX_VERSION=0.170.0 codex app-server model/list from the local binary none no; list comment says 0.166.0/0.167.0
agy AGY_VERSION=1.5.0 agy models from the local binary none no; static floor from agy 1.1.18
omp OMP_VERSION=18.22.0 omp models --json from the local binary none no
goose GOOSE_VERSION=1.60.1 goose ACP probe, per configured provider none no

Part 1: copilot (same pattern as Claude)

Where: copilotPinnedCLIModels (cli_models.go:342), copilotStaticModels (:304), the matching JS list in static-src/js/backends-tokens.js, and copilotCLIAcceptedModels in src/crux/copilot_models.go (the comment at :331 says to keep them in sync).

Do:

  1. Find an offline gate first. Install @github/copilot@<pin> the same way the Dockerfile does (npm --ignore-scripts) and find a signal that separates "id the CLI knows" from "id it doesn't" without a login. Note 🐛 copilot: picking claude-sonnet-5-5 or claude-opus-5-5 runs the agent on claude-sonnet-5 (Copilot CLI 1.0.88) #9927: the CLI silently swaps an unknown dashed id for another model (Model "..." from --model flag is not available. Using "claude-sonnet-5" instead.). That message may itself be the signal. Record the exact command and output in the PR.
  2. If there is a reliable offline signal, add --apply copilot next to --apply claude, using the same three control checks (version equals the pin, a listed id is recognised, a made-up id is not). Copilot ids are dotted (claude-sonnet-5.5), not dashed. Keep that spelling.
  3. If there is no offline signal, stop after step 1 and write the finding on this issue. Don't build a gate that guesses.

Done when: a dry run against the real pin prints the edits it would make, a re-run changes nothing, and every edited list stays in sync.

Part 2: codex (refresh the static list from the pinned binary)

Codex has no per-account catalogue: the model list is built into the CLI binary. So the fix is not an allowlist. It is keeping codexStaticModels (cli_models.go:429) equal to what the pinned binary reports. That list is what hosted hives without the binary always serve.

Do:

  1. Install @openai/codex@<pin> in CI and run the same app-server model/list exchange that discoverCodexModels (cli_models_codex.go:100) speaks. Check first whether model/list answers without a ChatGPT login. If it doesn't, stop and write the finding here.
  2. Rewrite codexStaticModels and CODEX_CLI_MODELS (JS) in the order orderCodexServedModels uses: visible ids first, then hidden ones. Update the "matches codex X" comment to the pin.
  3. Open or update one PR on a bot branch with needs-human, same as ✨ Weekly PR adds new Claude models to the lists and price table once the pinned Claude Code accepts them #18788.

Done when: HIVE_MODEL_CHECK_STRICT=1 src/scripts/check-cli-model-lists.sh no longer reports "codex static list comment says … but src/Dockerfile pins …". That finding is live today (0.166.0 vs 0.170.0).

Part 3: agy and omp (refresh the static floor)

Both are probed from the local binary at runtime (agy models; omp models --json), so the running CLI already sets the limit and no allowlist is needed. Their static floors are stale, though: agyStaticModels (cli_models_agy.go:28) is a snapshot from agy 1.1.18, and the pin is 1.5.0.

Do: for each one, check whether the subcommand runs without a login on the pinned binary. If it does, refresh the static list (Go and JS) from it in the same weekly PR. If it doesn't, list it as "needs a manual capture" in the PR body. Don't guess.

Part 4: gemini and goose (report only)

  • gemini: no Gemini CLI is installed in src/Dockerfile or src/Dockerfile.contributor, so there is no pinned binary to check against. Leave geminiPinnedCLIModels hand-kept, and say so in src/docs/cli-pins.md.
  • goose: its models depend on whichever provider an operator configures. There is no single list to keep fresh. Document that it is out of scope.

How to ship

Tests

  • Hermetic Python tests with a fake CLI per backend, like test-claude-model-freshness.py: candidate detection, gate pass/fail, control-check failures add nothing, dry run writes nothing, and a re-apply is a no-op.
  • The Go tests that check list shape and Go/JS sync (TestStaticModelListsShape) still pass.
  • The PR's self-test job dry-runs the whole pipeline against the real pins.
  • Locally, only gofmt -l and node --check (see AGENTS.md). CI decides the rest.

Out of scope

Refs #18754 #18788

Activity

  1. hivecommons-hive commented on Nov 18, 2026

    @hivecommons-hive
    Contributor

    🔓 Claim by agent scanner released (covering PR merged or closed).

    Claim history
    • 2026-11-18 16:34 UTC — claimed by scanner (agent on hive hosted-available-oke-11-placeholder-r05x)
    • 2026-11-18 17:29 UTC — released by scanner (agent; covering PR merged or closed)
  2. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Nov 18, 2026
  3. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor

    @Danathar this issue was closed by @hivecommons-hive[bot] as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions