Skip to content

Setup wizard: validate API keys against current models - #227

Open
kamal0808 wants to merge 1 commit into
elder-plinius:mainfrom
kamal0808:fix/retired-setup-validation-models
Open

kamal0808 wants to merge 1 commit into
elder-plinius:mainfrom
kamal0808:fix/retired-setup-validation-models

Conversation

@kamal0808

Copy link
Copy Markdown

Summary

The setup wizard tests a newly entered API key by sending a tiny prompt, but it
does that against models that have been retired, so a valid key fails the check
and never gets saved.

  • Anthropic keys are tested with claude-sonnet-4-20250514, retired 2026-06-15.
  • OpenAI keys are tested with gpt-4-turbo-preview, retired 2026-03-26.

Both calls come back as errors, so setupAnthropicKey/setupOpenAIKey print
"API key validation failed" and drop the key. The OpenAI provider's
defaultModel in src/config/index.ts is the same retired gpt-4-turbo-preview,
so even a saved OpenAI key would point at a dead model.

Sources:

What changed

  • Validate Anthropic keys with claude-sonnet-5-5 (active; recommended replacement).
  • Validate OpenAI keys with gpt-5.6-sol, and use it as the OpenAI provider
    defaultModel. I picked gpt-5.6-sol over gpt-5/gpt-4.1 because those have
    near-term shutdown dates, while gpt-5.6-sol currently has none — so this
    shouldn't need touching again at the next retirement.
  • Stop sending temperature to the Anthropic Messages API. This part is needed
    for the fix to actually work: temperature/top_p/top_k were removed in API
    version 2026-10-21 and are already rejected (400) as non-default values on
    Claude 4.7+ models (which claude-sonnet-5-5 is) on earlier API versions. The
    adapter sent temperature: 0, so swapping in a current model alone would still
    fail validation. Anthropic's own guidance is to omit these and use prompting.
    (This also unblocks every other Anthropic call in the app, which hit the same
    wall with the claude-opus-4-8 default.)

Added a static invariant test in the repo's existing style
(src/__tests__/setup-model-validation-static.test.ts, modeled on
local-openai-wire-static.test.ts) that pins the current model ids and the
dropped sampling params.

Linked issue

None — this is an unreported break (no open or recent issue references these
model ids or the validation failure).

Prior work and attribution

  • Prior PR or issue used: none
  • Reused/adapted areas: none
  • Fresh verification performed for reused material: n/a

Contribution receipt

  • Scope class: local_lab (static config/code change; no live target)
  • Target authority: n/a
  • Network use: none
  • Claims or evidence changed: none
  • Redaction/provenance notes: none

Verification

I didn't run the suite locally for this draft, so the checklist below is honest
about that. The change is small and pinned by a static invariant test; worth a
npm run test:pr before merge.

  • npm run typecheck
  • npm test
  • npm run doctor

Risk and rollback

  • Residual risk: dropping temperature means Anthropic calls use the API default
    sampling; the app can no longer pin temperature for Anthropic, but it couldn't
    anyway on current models (the API rejects it). OpenAI is unchanged and still
    accepts temperature.
  • Rollback: revert this commit.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant