Skip to content

fix(config): treat placeholder provider keys as unset when picking the LLM provider - #1588

Merged
rohitg00 merged 8 commits into
rohitg00:mainfrom
dmazhukov:fix/1586-placeholder-provider-keys
Nov 8, 2026
Merged

rohitg00 merged 8 commits into
rohitg00:mainfrom
dmazhukov:fix/1586-placeholder-provider-keys

Conversation

@dmazhukov

@dmazhukov dmazhukov commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Provider selection, the opt-in notice and the key each LLM provider sends now skip placeholder values: your-key-here, changeme, xxxx-xxxx and anything ending in ..., the form .env.example samples use. Fixes #1586.

The daemon counted any non-empty key, so AGENTMEMORY_LLM_PROVIDER=anthropic with ANTHROPIC_API_KEY=sk-ant-... ran anthropic instead of zero-LLM with the missing-key notice, a sample key alone drew a notice offering its provider, and GEMINI_API_KEY=your-key-here next to a real GOOGLE_API_KEY sent Bearer your-key-here. Doctor and the daemon now share the rule. With a plain-http OPENAI_BASE_URL, OPENAI_API_KEY keeps any non-empty value, as the README promises for local servers.

Not covered: embedding provider detection still checks raw presence.
Not covered: env-placeholder-keys reads OPENAI_BASE_URL from .env only.


Devin Review

Summary by CodeRabbit

  • Bug Fixes
    • Provider selection and diagnostics now treat blank values and common placeholder API keys as missing credentials, while continuing to recognize usable keys.
    • OpenAI configurations using plain-HTTP endpoints accept any nonblank API key. Placeholder keys remain invalid for HTTPS endpoints and other providers.
    • Valid Google credentials are used when the Gemini key is a placeholder.
    • Provider setup and key checks now apply consistent validation, including when keys are read from environment variables or configuration files.

…e LLM provider

Doctor reports keys such as sk-ant-... or your-key-here as missing, but
the daemon accepted any non-empty value. It picked a provider for a
placeholder and sent that string with every request, and a placeholder
GEMINI_API_KEY shadowed a real GOOGLE_API_KEY.

Move the placeholder rule from doctor into config, widen it to any value
ending in "..." as .env.example ships them, and apply it both to
provider detection and to the key each provider sends. OPENAI_API_KEY
keeps any non-empty value while OPENAI_BASE_URL is a plain-http address,
because local servers ignore it and the README tells users to put any
string there.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
@vercel

vercel Bot commented Oct 19, 2026

Copy link
Copy Markdown

@dmazhukov is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 20dc1484-eb48-49ca-9557-279329e524c4
Commits

Reviewing files that changed from the base of the PR and between c137db6 and 73a575a.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

Provider-key validation now rejects placeholder-like values during provider detection and credential retrieval. Doctor diagnostics use the same validation. OpenAI keys with plain-HTTP base URLs follow a separate nonblank-key rule.

Changes: Provider Key Validation

Layer / File(s) Summary
Shared provider-key validation and selection
src/config.ts, test/provider-placeholder-keys.test.ts
Shared validation checks placeholder-like values and is used for provider detection, opt-in hints, and Gemini compatibility. Tests cover placeholder classification, provider selection, and .env.example samples.
Doctor diagnostics
src/cli/doctor-diagnostics.ts, test/cli-doctor-fixes.test.ts
Doctor diagnostics use the shared classifier. Provider detection receives the merged environment without removing placeholder keys. Tests cover placeholder reporting and OpenAI keys with plain-HTTP base URLs.
Validated credential retrieval
src/config.ts, src/providers/index.ts, test/provider-placeholder-keys.test.ts
getProviderKey returns a key only when validation accepts it. Provider construction uses validated keys, with Google-key fallback when the Gemini key is invalid. Tests cover OpenAI URL handling, provider construction, and Gemini key selection.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

Suggested reviewers: rohitg00

Merge Risk: Minimal · previous review c137d

The reported test compatibility and OpenAI configuration concerns do not affect supported workflows. No actionable merge-blocking risk remains after normal checks.

Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change to LLM provider selection.
Linked Issues check Passed [#1586] Provider selection and credential lookup use shared placeholder validation. Gemini falls back to a real GOOGLE_API_KEY when GEMINI_API_KEY is a placeholder. Tests cover selection, credenti…
Out of Scope Changes check Passed [#1586] The config, provider, and doctor changes share the provider-key rule. The tests cover the related selection and credential behavior. No unrelated changes are identified.
Docstring Coverage Passed Docstring coverage is inconclusive: files from earlier reviews still need to be rechecked within the check's analysis limits. Not blocking.
Full details: Linked Issues check

Explanation

[#1586] Provider selection and credential lookup use shared placeholder validation. Gemini falls back to a real GOOGLE_API_KEY when GEMINI_API_KEY is a placeholder. Tests cover selection, credentials, diagnostics, and the plain-HTTP OpenAI exception. Embedding detection remains outside the issue scope.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 720e15de-d93d-4f5a-be09-a5498d99d7a9
Commits

Reviewing files that changed from the base of the PR and between 958c6e9 and c137db6.

Files selected for processing: 4

  • src/cli/doctor-diagnostics.ts
  • src/config.ts
  • test/cli-doctor-fixes.test.ts
  • test/provider-placeholder-keys.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

coderabbitai[bot]

This comment was marked as resolved.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
…ion skips placeholders

detectLlmProviderKind already treats placeholder keys as unset, so deleting them before the check changed nothing. The test title claimed the daemon disagrees, which is no longer true.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
The plain-http exception for OPENAI_API_KEY depends on OPENAI_BASE_URL, so without it a local-server key like xxx was dropped and doctor reported the agent-sdk fallback while the daemon picked openai.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Devin Review

Comment thread src/config.ts Resolved
coderabbitai[bot]

This comment was marked as resolved.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
…al Gemini keys and blank keys

The plain-http rule counts only a non-empty OPENAI_API_KEY, the Gemini provider prefers GEMINI_API_KEY without a warning when both keys are real, and a blank or space-padded key is not real; each had a surviving mutant.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
…d a real key starting with x

An empty or blank OPENAI_API_KEY next to a plain-http base URL is not a key, and a real key such as xai-... is not taken for the xxxx placeholder; both had a surviving mutant.

Signed-off-by: Dmitrii Zhukov <dmitry0983@gmail.com>
@dmazhukov

Copy link
Copy Markdown
Contributor Author

The rule covers the forms .env.example samples use. Other spellings such as your_key_here, <your-key-here> and YOUR_API_KEY still count as keys (src/config.ts:91-99).

@rohitg00
rohitg00 merged commit 77f4535 into rohitg00:main Nov 8, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Daemon picks an LLM provider for a placeholder key that doctor reports as missing

2 participants