Repository navigation
✨ Weekly PR adds new Claude models to the lists and price table once the pinned Claude Code accepts them - #18788
Conversation
Adds .github/workflows/claude-model-freshness.yml: every Monday it finds Claude ids on OpenRouter newer than the dashboard's Claude lists (same comparison as the static-model-freshness report), probes each with the Claude Code version pinned in src/Dockerfile (unauthenticated, empty HOME: the CLI prints an unrecognized_model diagnostic for ids it does not know), and for recognised ids edits claudePinnedCLIModels, claudeStaticModels, CLAUDE_CLI_MODELS, static/index.html, cli-model-requirements.txt and modelPrices (via the new `pricetable --add`, which applies the existing plausibility rules). One needs-human PR on bot/claude-model-freshness, updated in place; no accepted model means no branch and no PR. Refs hivecommons#18754 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
src/scripts/test-check-cli-model-lists.sh changed, so the pinned tree in integration-governance.cjs moves to git rev-parse HEAD:src/scripts. Refs hivecommons#18754 Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head
No merged by Hive trusted-author auto-merge: author |
|
@Danathar this PR is back in your court ( Failing checks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (0 waiting): fixes are handed out before new work as soon as you have capacity. |
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether a new scheduled workflow with contents: write/pull-requests: write and the TOPUP_PUSH_TOKEN PAT, which runs downloaded vendor code, may land.
This adds .github/workflows/claude-model-freshness.yml, a weekly workflow with write permissions and a push PAT, and it edits .github/scripts/integration-governance.cjs. Those are CI-permission and credential surfaces, so a person has to approve them. I did not run the pipeline. The one verified concern is on the line comment below: credentials stay on disk while the pinned npm package's install.cjs and claude binary run.
What I read at 2cadacb0 that looks sound: the PR-triggered selftest job has read-only permissions and persist-credentials: false (workflow line 87). The write job only runs on schedule and workflow_dispatch, and the PR is labelled needs-human and never auto-merged. The probe runs the CLI with an empty HOME and a minimal env.
— hive: agent=reviewer-2 backend=claude model=claude-sonnet-5-5 claude=2.1.326
| with: | ||
| ref: ${{ env.BASE_BRANCH }} | ||
| fetch-depth: 0 | ||
| token: ${{ secrets.TOPUP_PUSH_TOKEN != '' && secrets.TOPUP_PUSH_TOKEN || github.token }} |
There was a problem hiding this comment.
This checkout takes the write token (TOPUP_PUSH_TOKEN or github.token) and does not set persist-credentials: false, so the token is left in .git/config for the rest of the job. The next step runs claude-model-freshness.sh, which does npm install and then node .../install.cjs (script line 43) and executes the downloaded claude binary. --ignore-scripts stops lifecycle scripts, but install.cjs and the binary are still vendor code running as the same user with a readable checkout. A compromised release of the pinned package could read the token and push to the repo. The probe's empty-env HOME does not hide .git/config from a process that can reach the checkout. Suggestion: check out with persist-credentials: false and pass the token only to the final push and gh steps, or run the install and probe in a job without write permissions and hand over accepted.txt as an artifact.
|
Needs a decision, not a merge:
Head: verdict requires_human @5c0caf0, 31/58 checks green, 27 pending, DCO yes, soak 0s of 10m0s. How to release it:
Clear the cause and the hive re-evaluates on its next pass; the stage moves to Updated 2026-11-17 23:39 UTC. |
|
/lgtm Posted by @clubanderson from the Hive dashboard; the merge gate decides from here. |
|
@hivecommons-hive[bot]: changing LGTM is restricted to collaborators DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/approve Posted by @clubanderson from the Hive dashboard; the merge gate decides from here. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: hivecommons-hive[bot] The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Re-pin TRUSTED_SCRIPTS_TREE to the merged src/scripts tree. Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether a new scheduled workflow holding contents: write + pull-requests: write and the TOPUP_PUSH_TOKEN secret, which installs and executes the pinned Claude Code CLI, should land.
Routing, not a code defect: this PR adds .github/workflows/claude-model-freshness.yml (a CODEOWNERS-covered path, .github/workflows/**) and re-pins TRUSTED_SCRIPTS_TREE in .github/scripts/integration-governance.cjs:12. Both are trust surfaces I should not wave through.
What I verified in the diff:
- The
refreshjob (schedule/dispatch only) holdscontents: writeandpull-requests: write, checks out withTOPUP_PUSH_TOKEN(so the credential is persisted in.git/config; nopersist-credentials: false), then runsclaude-model-freshness.sh, whichnpm installs the pinned CLI (--ignore-scripts, then the vendor'sinstall.cjs) and executes it. The probe itself scrubs the env (empty HOME, no keys), but the npm install/install.cjsstep runs in the same job as the persisted token. The version is pinned fromsrc/Dockerfile, so the exposure is a supply-chain one, not a new input. Open question for the owner: is that acceptable, or should the install/probe run in a job without the push token and hand edits to a second job? - The
selftestjob onpull_requestis read-only (contents: read), no secrets, and checks out withpersist-credentials: false. That looks right. TRUSTED_SCRIPTS_TREEchanges fromc269cb8…to22b8f6c…. Editing anything in the enrolled.githubclosure necessarily moves this; I did not recompute the tree hash, so someone should confirm it matches the head tree rather than trusting the literal.- Inputs (
inputs.base, step outputs) reach shell only viaenv:, not inline${{ }}interpolation. Good. - The PR is labelled
needs-humanby the workflow and says it is never auto-merged; that matches the body.
Not reviewed in depth: the Go pricetable --add changes and the Python candidate/insert logic (I read the Python at a skim level only; nothing wrong stood out). Treat my other perspectives as "no verified findings", not as a full audit.
— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326
There was a problem hiding this comment.
Decision needed: @clubanderson should decide whether a new scheduled workflow with contents: write + pull-requests: write (and a changed TRUSTED_SCRIPTS_TREE pin) may land.
This PR touches CI trust surfaces, so I am routing it rather than judging it alone.
Verified at 5c0caf0:
.github/workflows/claude-model-freshness.ymlis new. Itsrefreshjob (cron47 7 * * 1plusworkflow_dispatch) holdscontents: writeandpull-requests: write, and pushesbot/claude-model-freshnessusingTOPUP_PUSH_TOKENwhen set, elsegithub.token. Workflow-level permissions arecontents: read, and thepull_requestselftestjob inherits only that, which is the right shape..github/scripts/integration-governance.cjs:12changesTRUSTED_SCRIPTS_TREEfromc269cb8…to22b8f6c…. The comment above it says the value must not be chosen by a caller. That is expected when anything under the enrolled.githubclosure changes, but a human should confirm the new tree hash matches the closure this PR produces.- The job runs a pinned Claude Code binary with an empty
HOMEand no key, and runspricetable --add, which writespricing.go.
I did not find a defect in what I read of the workflow and script. The open question is policy: is a weekly bot-authored PR path that can push to a branch and edit the price table acceptable, and is TOPUP_PUSH_TOKEN the right credential for it? The PR says it is never auto-merged and carries needs-human.
I did not review the Go pricetable changes line by line. I have not approved or closed anything.
— hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.326
Every Monday a new workflow opens or updates one PR that adds new Claude models to the dashboard's lists and price table. A model is added only when the Claude Code version pinned in
src/Dockerfilerecognises it. Operators get new models in the dropdown and priced correctly without anyone hand-editing five files. Any model the pinned CLI doesn't recognise is listed for a manual check and is not added. The PR always carriesneeds-humanand is never auto-merged.Refs #18754
What this changes
claude-model-freshness.yml(Mondays 07:47 UTC, plusworkflow_dispatchwithbaseanddry_run).claudePinnedCLIModels,claudeStaticModels,CLAUDE_CLI_MODELSwith a label (for example "Fable 5.2"), a rebuiltstatic/index.html, acli-model-requirements.txtline at the pin, and amodelPricesrow.pricetablegets an--addmode. It inserts rows the table lacks, and only through the existing plausibility rules. It never reprices a row and never movespriceTableDate.--apply claudemode both use it. The report's output is unchanged (checked against the live catalogue).Why
Nothing added new Claude models. The freshness report only filed an issue line, and the price refresh only repriced rows the table already had. Fable 5.1, Fable 5.2 and Haiku 5.5 were all missed.
How it works
Candidates. These are Claude ids on OpenRouter's public catalogue that are newer than the lists' newest model of the same line (Fable, Opus, Sonnet, Haiku), mapped to the CLI's dashed spelling.
anthropic/claude-fable-5.2becomesclaude-fable-5-2. The following are never candidates:expiration_date:batch~router aliasesGate. The issue's first choice, the "version X or newer is required" refusal, doesn't work offline. That refusal is an API 400 that comes back after login (see the error text in 🐛 bug: pinned agent CLIs are stale — Claude Code 2.1.226 rejects claude-opus-5-5, Codex 0.153.4 lacks GPT-6 Sol/Luna #8417). An unauthenticated CLI only says "Not logged in". The CLI does have an offline signal, though: for an id its built-in model table doesn't know,
claude --model <id> -p xprints[claude-code:unrecognized_model]before "Not logged in", and for a known id it prints nothing extra. I checked this by hand with Claude Code 2.1.311. It recognisedclaude-fable-5-1andclaude-haiku-5-5and did not recogniseclaude-fable-5-2or a made-up id. The job installs the pinned version the same way the image does (npm --ignore-scripts, theninstall.cjs) and probes each candidate with an empty HOME and no key. Before trusting any result it runs three checks:--versionmust equal the pin;If any check fails, nothing is added and every candidate goes under "needs a check".
Edits. List entries go next to the same line's entries, newest first, in the same position in Go and JS, so the lists stay exact copies. Price rows are compared with the newest existing row of the same line. A quote more than 5× away from it, or one that fails the ratio rules, or one with no cache-read rate, is held and reported instead of added.
PR. There is one PR on
bot/claude-model-freshness, force-pushed and edited in place by later runs. If no model is accepted, there is no branch and no PR. Tokens and DCO sign-off followprice-table-refresh.yml.This is its own workflow rather than a job in
static-model-freshness.yml(which hasissues: writeonly) orprice-table-refresh.yml. That keeps the write permissions on the one job that opens this PR, and it means a model addition and a repricing get reviewed separately.Example: this PR's own self-test dry run (live catalogue, pinned Claude Code 2.1.326) passed all three probe checks. It would add Fable 5.1, Fable 5.2 and Haiku 5.5 to the lists. On prices it would add a Fable 5.2 row (8.00 / 40.00 / 0.20 / 10.00 per MTok), skip Fable 5.1 because it is already priced, and hold Haiku 5.5. The Haiku 5.5 hold is because OpenRouter quotes it at 0.10 / 0.50, a tenth of Haiku 4.5's price, which trips the 5× rule. A human checks that one against Anthropic's price page.
Not in this PR:
HIVE_MODEL_CHECK_STRICT: '1'incli-pin-bump.yml. The issue says to flip it only after #18753 merges. Even then, strict mode would currently fail on two findings that already exist:claudePinnedCLIModelshas no "version pinned in src/Dockerfile (X" comment;It needs a small follow-up once those are fixed.
Files
.github/workflows/claude-model-freshness.yml: schedule and dispatch job that opens the PR; a read-only PR self-test plus a dry run.github/scripts/claude-model-freshness.sh: fetch the catalogue, install the pinned CLI, apply,pricetable --add,make dashboard-gen.github/scripts/check-static-model-freshness.py:compare_list,index_catalog,claude_candidates,probe_claude,gate_candidates,insert_listed,apply_claude,--apply claude.github/scripts/test-claude-model-freshness.pyandtestdata/claude-model-freshness/: hermetic tests with a fake CLIsrc/internal/tools/pricetable/add.go:planAdditions,proposeRow,siblingEntry,applyAdditions,renderAddReport,executeAddsrc/internal/tools/pricetable/reconcile.go:implausiblesplit intomoveRulesandshapeRules(same behaviour)src/internal/tools/pricetable/table.go: rows record their end offsetsrc/internal/tools/pricetable/main.go:--addflagsrc/internal/tools/pricetable/add_test.go,testdata/openrouter-add.jsonsrc/scripts/test-check-cli-model-lists.sh: strict-mode cases for an appended requirements linesrc/docs/cli-pins.md,changelog.d/added-18754-claude-model-freshness.mdTests
CI: this PR's own checks; only gofmt -l / node --check run locally.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WcWm7qFmFJp26r9LPdmi3t