feat(providers): add DeepInfra provider and curated models (#619) - #624
Open
cagdasyurekli wants to merge 2 commits into
Open
feat(providers): add DeepInfra provider and curated models (#619)#624cagdasyurekli wants to merge 2 commits into
cagdasyurekli wants to merge 2 commits into
Conversation
…#619) Add DeepInfra as a first-class model provider and curate its latest model catalog: - Register deepinfra in coworker/providers/registry.py via _compat() pointing to https://api.deepinfra.com/v1/openai with DEEPINFRA_API_KEY. - Add 8 curated DeepInfra models to MATRIX in coworker/providers/matrix.py (DeepSeek V4 Flash/Pro including 0731 pinned snapshots, GLM 5.3 Flash/Pro, Kimi K3 and Kimi K2.7 Code) with verified context windows (128k, 256k, 1M). - Enables desktop context fill meter and compaction trigger without UI changes. - Add unit tests for routing, builder, key verification, context windows, and key isolation. Closes andrewyng#619
deepseek-ai/DeepSeek-V4-Pro-0731 does not exist on DeepInfra; the valid snapshot is deepseek-ai/DeepSeek-V4-Pro-0813. Verified against the live DeepInfra model catalog. Also updates the pinned-snapshot comment which referenced 0731 for both Flash and Pro.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses and closes #619 by registering DeepInfra as a first-class model provider and curating its latest generation of frontier open-weight models (DeepSeek V4, GLM 5.3, Kimi K3) with verified context window metadata.
Motivation & Background
DeepInfra hosts high-performance inference endpoints for leading open-weight frontier models. While users could previously configure a generic custom endpoint, doing so caused:
TurnEngine._compaction_config) to fall back to default thresholds, prematurely compacting long-context models like GLM 5.3 and Kimi K3 (which support 1,000,000 tokens).Changes
Provider Registration (
coworker/providers/registry.py):deepinfrainDESCRIPTORSvia_compat()pointing tohttps://api.deepinfra.com/v1/openai.DEEPINFRA_API_KEY.deepseek-ai/DeepSeek-V4-Flash.Curated Model Matrix (
coworker/providers/matrix.py):deepinfra:deepseek-ai/DeepSeek-V4-Flash: 128,000 tokens (_AGENTIC)deepinfra:deepseek-ai/DeepSeek-V4-Flash-0731: 128,000 tokens (_AGENTIC)deepinfra:deepseek-ai/DeepSeek-V4-Pro: 128,000 tokens (_AGENTIC)deepinfra:deepseek-ai/DeepSeek-V4-Pro-0731: 128,000 tokens (_AGENTIC)deepinfra:zai-org/GLM-5.3-Flash: 1,000,000 tokens (_AGENTIC_VISION)deepinfra:zai-org/GLM-5.3: 1,000,000 tokens (_AGENTIC)deepinfra:moonshotai/Kimi-K3: 1,000,000 tokens (_AGENTIC_VISION)deepinfra:moonshotai/Kimi-K2.7-Code: 256,000 tokens (_AGENTIC)Automated Tests:
tests/test_token_usage.py: verified all 8 DeepInfra models export exact context windows inmodel_context_windows().tests/test_provider_router.py: verified client builder default base URL and router prefix stripping for DeepInfra models.tests/test_providers.py: verified capabilities (including vision), lockstep reseller descriptor checks, and key isolation (ensuringOPENAI_API_KEYis not leaked whenDEEPINFRA_API_KEYis missing).tests/test_provider_verify.py: verified test probe targetshttps://api.deepinfra.com/v1/openai/modelswith Bearer auth.Verification
pytest tests/test_token_usage.py tests/test_provider_router.py tests/test_providers.py tests/test_provider_verify.py tests/test_compaction.py tests/test_engine.py -q-> 144 passed.model_context_windowsfrom/v1/settings.Closes #619