fix(lmi): bound Redis I/O in GlobalRateLimiter [CLD-320] - #464
Merged
Conversation
GlobalRateLimiter.try_acquire gates every LLM/embedding call on a Redis op via coredis. The client was created with stream_timeout=None, so a silently dropped idle connection made the await block forever; acquire_timeout only bounds the asyncio.sleep polling loop, not the Redis I/O. Pass stream_timeout and connect_timeout to RedisStorage so a stalled op raises a bounded coredis error instead of hanging. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ypicard
marked this pull request as ready for review
July 1, 2026 23:21
ypicard
enabled auto-merge (squash)
July 1, 2026 23:22
jabra
approved these changes
Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Agent trajectories were freezing forever, at random. Every LLM and embedding call goes through
GlobalRateLimiter.try_acquire, which checks a Redis-backed rate limit. The coredis client was built with no socket timeout, so when an idle Redis connection got silently dropped (for example, an AWS NAT Gateway reaping the flow without sending a reset), the next Redis call blocked with nothing to break it. The existingacquire_timeoutonly bounds the polling sleep loop, not the Redis I/O, so the run hung indefinitely.What changed
Pass
stream_timeoutandconnect_timeouttoRedisStorage. These flow throughlimitstocoredis.Redis.from_url, so a stalled op now raises a bounded error instead of hanging. The error propagates to the caller rather than freezing the trajectory.This is a minimal alternative to #460, which builds fail-open behavior, in-memory degradation, and auto-recovery on top of the same root fix. This PR does only the part that stops the freeze.