Skip to content

✨ [v5] Keep the Claude model lists and prices current: weekly PR adding new models the pinned Claude Code accepts #18754

Closed

Description

@Danathar

Target: v5.

Target branch: v5. Open the PR with --base v5. The code paths are the same on v6 (line numbers can differ), and the regular v5 → v6 top-up carries it there, so do not open a second PR for v6. Citations are path:line on v5 @ 7ca94dd, plus the function name, so the pointers still work after the lines drift.

In the PR body write Refs #<this issue>. Do not use a closing keyword: a PR based on v5 cannot close an issue that way, so a maintainer closes this one after the merge.

Executive summary

Every time Anthropic releases a model, someone has to hand-edit five places before the hive's dashboard offers it and prices it correctly. Nobody is told to, so the lists fall behind. Fable 5.2, Fable 5.1 and Haiku 5.5 have all been missed.

The hive already has weekly jobs that notice part of this:

  • one reports newer models as an issue;
  • one updates prices, but only for models already in the table.

This issue turns that into a weekly pull request that adds new Claude models to the lists, their prices and their Claude Code version requirement. A reviewer then only has to check and merge it.

The closed allowlist stays exactly as it is: a model is added only once the pinned Claude Code version accepts it.

Problem

Adding a Claude model takes edits in five places, all by hand:

  1. claudePinnedCLIModels, the closed allowlist (🐛 bug: agent model dropdown offers models the installed CLI can't run (all backends, not just claude) #8418);
  2. claudeStaticModels;
  3. CLAUDE_CLI_MODELS, plus a rebuild of static/index.html;
  4. src/scripts/cli-model-requirements.txt (✨ [v5] cli-pin-bump: check Hive's model lists and prices against the new CLI pin (and bump patch-versioned CLIs like Claude Code) #9805);
  5. a modelPrices row.

What exists today covers only parts of it:

  • static-model-freshness.yml (🌱 Weekly static model list freshness check and list-shape test #18474, Mondays 07:17 UTC) runs .github/scripts/check-static-model-freshness.py. It reports "newer models of the same family on OpenRouter that the list lacks" in one tracking issue, and changes nothing. It has issues: write only.
  • price-table-refresh.yml (Mondays 06:41 UTC) runs src/internal/tools/pricetable, which reprices existing rows from OpenRouter and opens a PR. It never adds a row for a model the table lacks.
  • cli-pin-bump.yml bumps the Claude Code pin and runs check-cli-model-lists.sh. That check only compares against cli-model-requirements.txt, which has one line (Sonnet 5.5), and it runs with HIVE_MODEL_CHECK_STRICT: '0', so drift is a warning.

So a new model is detected at best as an issue line, and nothing adds it.

Evidence

Root cause

  • .github/scripts/check-static-model-freshness.py@7ca94dd: main (:112) builds newer (:147-167) and only writes report text.
  • .github/workflows/static-model-freshness.yml:34-36@7ca94dd: contents: read, issues: write.
  • src/internal/tools/pricetable/reconcile.go@7ca94dd: reconciles rows that exist, recording change and unverified. There is no path that adds a row.
  • .github/workflows/cli-pin-bump.yml:105@7ca94dd: HIVE_MODEL_CHECK_STRICT: '0'.
  • The lists: src/pkg/dashboard/cli_models.go:450 (claudePinnedCLIModels), :272 (claudeStaticModels), :392 (claudeAlwaysIncludeModels), and src/pkg/dashboard/static-src/js/backends-tokens.js:53 (CLAUDE_CLI_MODELS), all @7ca94dd.

Wanted behavior

A weekly job that opens, or updates, one PR adding the Claude models the lists lack:

  1. Find candidates. Reuse the freshness script's comparison. Candidates are Claude ids newer than the lists' entries in the same line (Fable, Opus, Sonnet, Haiku), from OpenRouter's public catalog. Use Anthropic's /v1/models only if a key is configured for CI, and never require one. Map each to the Claude CLI's dashed spelling (anthropic/claude-fable-5.2 → claude-fable-5-2).
  2. Gate on the pinned Claude Code version. Add a candidate only when the Claude Code version in src/Dockerfile accepts it, and record that version in cli-model-requirements.txt. Work out how to decide "accepts" without credentials, and document it in the PR:
    • First choice: in the pin-smoke image cli-pin-bump.yml already builds, check whether an unauthenticated claude --model <id> -p x answers with the CLI's "version X or newer is required" refusal, as distinct from an authentication error. If it reliably does, that is the gate.
    • Fallback: if the CLI does not tell these apart offline, open the PR as a draft, listing each candidate under "needs a check against the Claude Code changelog". Never add an id as accepted on a guess.
  3. Make all five edits for each accepted candidate:
    • the three lists, with a display label ("Fable 5.2");
    • a rebuilt static/index.html (make dashboard-gen);
    • the requirements line;
    • a modelPrices row from the OpenRouter quote, through the pricetable tool's existing plausibility checks. A quote that fails them is left out and flagged, not guessed.
  4. One PR, kept current. Reuse the price-table PR pattern (price-table-refresh.yml already holds contents: write and pull-requests: write). Use a fixed branch and update the open PR in place on later runs. With no candidates, do nothing.
  5. Make the requirements check strict once the list is current: HIVE_MODEL_CHECK_STRICT: '1' in cli-pin-bump.yml. A pin bump then fails while a model it newly accepts is still missing from the allowlist.

Acceptance criteria

  • With a fixture catalog containing a newer Fable and Haiku than the lists hold, a dry run prints the five edits per model and opens no PR.
  • A candidate the pinned CLI is not shown to accept is never added as accepted; it appears under "needs a check".
  • A real run opens one PR on a fixed branch. A second run with the same candidates updates that PR and does not open another.
  • The generated PR passes make dashboard-gen-check, TestStaticModelListsShape and check-cli-model-lists.sh with HIVE_MODEL_CHECK_STRICT=1.
  • Price rows come only through the pricetable plausibility checks.
  • Mythos (limited-access) ids are never proposed. Retired ids are never added.
  • With no candidates, no PR and no branch.

Test plan (CI decides)

  • Python unit tests for the freshness script's new candidate and edit logic, against fixture catalogs: same-line newer detection, the dotted→dashed mapping, Mythos excluded, idempotent edits (running twice changes nothing more). Put them next to the script, in the style of .github/scripts/test-ci-infra.py.
  • src/internal/tools/pricetable/main_test.go: adding a missing row from a quote; refusing an implausible one.
  • src/scripts/test-check-cli-model-lists.sh: strict mode fails on a requirements line whose model is missing from the allowlist, and passes when present.
  • A workflow_dispatch dry run on a fork, linked from the PR.

Suggested implementation

  1. Split the freshness script's comparison into a function that returns structured candidates. The report and the new --apply claude mode share it.
  2. --apply edits the Go and JS lists as text, keeping order and comments, and runs make dashboard-gen.
  3. Extend pricetable with an "add missing ids" mode that takes the accepted ids.
  4. Add the PR step, either to static-model-freshness.yml with the price workflow's permissions, or as a job in price-table-refresh.yml so lists and prices land in one PR. Say which in the PR and why.
  5. Flip HIVE_MODEL_CHECK_STRICT only after the sibling issue's refresh has merged, so strict mode starts green.

Rejected options

Out of scope / do not change

  • Copilot, Codex, Gemini and other backends' lists. Same pattern, but do Claude first. The freshness report keeps covering the rest.
  • The Claude Code pin bump itself (cli-pin-bump.yml), other than strict mode.

Note for the implementer

This touches .github/workflows/, so the PR needs a lane that may push workflow files.

Dependencies

Activity

  1. added
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    on Nov 17, 2026
  2. added and removed
    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
    stage/010-actionableTriaged; eligible for the hive to pick up
    on Nov 17, 2026
  3. hivecommons-hive commented on Nov 17, 2026

    @hivecommons-hive
    Contributor

    Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

    PR #18788 covers this issue: #18788

    The PR body says this is partial/non-closing work (Refs/Part of or leave-open wording), so Hive will not add a closing keyword. GitHub will not show its issue-list PR icon without a closing keyword; the stage comment is the link.

  4. added
    hive/partially-addressedA merged PR addressed only part of this issue; remainder still actionable
    stage/010-actionableTriaged; eligible for the hive to pick up
    and removed on Nov 18, 2026
  5. clubanderson commented on Nov 18, 2026

    @clubanderson
    Member

    Closing by hand: delivered by #18788 (merged to v5). v5 merges with Refs do not auto-close, so a maintainer closes it after the merge.

  6. github-actions commented on Nov 18, 2026

    @github-actions
    Contributor

    @Danathar this issue was closed by @clubanderson as completed.

    If this was closed by a merged PR, the fix only takes effect once it ships in a tagged release and your deployment updates to it — check your running version against the fix before concluding the fix failed.

    If you are still seeing the problem, comment /reopen and it will be reopened for you — you do not need write access. Please add what you are still observing (running version, logs, steps) so it is not closed the same way again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

hive/partially-addressedA merged PR addressed only part of this issue; remainder still actionable

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions