[Abhishekpandey123-git]

Summary

• Add llama_cpp as an explicit LLM provider.
• Register llama.cpp in the current provider specification system.
• Use the existing OpenAI-compatible LLM transport with the default llama.cpp server endpoint.
• Keep the provider API-key-free, matching a local llama.cpp deployment.

Implementation

The current LLM architecture already supports OpenAI-compatible endpoints, so no separate llama.cpp service implementation is required.

The provider is registered with:

llama_cpp
http://127.0.0.1:8080/v1
needs_key=False

Validation

• git diff --check passes.
• PYTHONPATH=. pytest -q tests/test_llm_service.py tests/test_llm_registry.py
  • 226 passed
• PYTHONPATH=. pytest -q tests/test_llm_service.py tests/test_llm_registry.py tests/test_api.py
  • 272 passed

The only reported warnings are existing Pydantic deprecation warnings.

Context

This PR is a fresh replacement for the previous llama.cpp provider PR, which was closed after its target branch was deleted. It is based on the current development branch as requested.

The benchmark work will be handled separately so the explicit provider implementation can be reviewed independently.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[Abhishekpandey123-git]

@marcvergees This PR addresses the explicit llama.cpp provider portion of #702.

The Ollama-vs-llama.cpp benchmark work will be handled separately so the provider implementation and benchmarking can be reviewed independently.

[vharkins1]

Thanks @Abhishekpandey123-git. A few notes on nice to haves you could add:

• llama_cpp in contracts/schemas/enums.yaml
• an entry in LLM_PROVIDER_ENV so it can actually be configured more easily

[marcvergees]

Hi @Abhishekpandey123-git, Imma review this asap, but me and @vharkins1 are working rn on the desktop deployment, which is the most urgent thing rn because we might be onboarding again next week. I haven't forgot your stuff, don't worry!
M