Repository navigation
Allow cold model startup and configure embedding timeouts - #1
Merged
Merged
Conversation
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (10)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local MCP searches could fall back to keyword results while a cold model exceeded the fixed two-second wait, or immediately when another request occupied the worker. Allow 30 seconds for startup and 10 seconds for warm queries, configurable through the embedding JSON. Waiting and inference share one bounded per-call budget; outstanding work remains single-flight and reusable.
Remote embedding queries now default to 10 seconds, background requests to 30 seconds, and client reads to 30 seconds. Remote budgets are independent. Existing
TimeoutErrorfallback metadata is preserved, with safe local reason codes distinguishing startup, query, contention, and stalled-worker limits.Adds regression coverage for cold/warm budgets, pending-result reuse and failure isolation, bounded admission, and configuration validation. Native qualification now checks both normal cold startup and forced timeout recovery with OS networking denied. Usage documentation explains the settings and their limits. No schema or embedding-identity changes.
Validation: targeted tests, Ruff, actionlint, and diff checks passed. Full CI passed on Linux and macOS with Python 3.11 and 3.14, including offline and recovery checks. Native qualification passed cold startup and forced timeout recovery with OS networking denied. An independent subagent reviewed the implementation; both findings were fixed and regression-tested.