Fix tool-calling resilience (retry + circuit breaker)#181
Open
imnishantmehra wants to merge 1 commit intolemony-ai:mainfrom
Open
Fix tool-calling resilience (retry + circuit breaker)#181imnishantmehra wants to merge 1 commit intolemony-ai:mainfrom
imnishantmehra wants to merge 1 commit intolemony-ai:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Tool-calling flows were bypassing
BaseProvider.complete()by callingcomplete_with_tools()directly in multiple internal paths, which meant tool calls didn’t benefit from the standard retry + circuit breaker + retry-metrics wrapper. This PR routes tool execution through the same resilience layer as normal completions, while remaining compatible with existing provider stubs/mocks used in tests.Type of Change
What Changed
BaseProvider:complete_with_tools(...)._complete_with_tools_impl(...)to dispatch tocomplete_with_tools(...)(normalizes messages/system prompt) so tool calls run through_execute_with_retry(...).tool_choicethrough as-is (supports object/dict-style tool forcing).cascadeflow/agent.py,cascadeflow/core/cascade.py, andcascadeflow/streaming/tools.pyto go through the retry/CB path when the provider is a realBaseProvider.BaseProviderprovider stubs by falling back tocomplete_with_tools(...)where appropriate (test/mocks).VLLMProvidernormalizes host-only base URLs to include/v1(prevents misconfig and fixes failing tests).tests/test_provider_tool_retry.py.clear=Truewhere needed.Testing
pytest tests/test_provider_tool_retry.py -vpytest tests/ -v -m "not integration and not requires_api and not requires_ollama and not requires_vllm"