feat: add client-side rate limiting - #262
GenZ-CODER-X wants to merge 3 commits into
Conversation
Design assumptions / implementation notesWhile implementing #51, I made the following design decisions based on the issue requirements:
I also added tests covering token consumption, refill, waiting, capacity limits, configuration validation, write request limiting, query-page limiting, and paginator exhaustion. These are the main implementation assumptions I made for #51. Please let me know if any of these differ from the intended behavior. |
This reverts commit fd4a061.
CI status updateThe current
I previously added From my side, the rate-limiting implementation and tests are passing, and the Please let me know whether the existing |
Opt-in TokenBucket limiters (put_rps / query_rps in config) throttle S3 Vectors put/query calls to stay under account/service limits. Rebased onto current development by the maintainer (its CI typecheck failure was a stale base, not a code issue); mypy + full suite green. Co-authored-by: GenZ-CODER-X <GenZ-CODER-X@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Landed in |
Description
Adds client-side token-bucket rate limiting to S3 Vectors requests to help avoid exceeding AWS request-rate limits.
The limiter is configurable independently for write and query operations and is disabled by default for backward compatibility.
Related issue
Fixes #51
Changes
TokenBucketrate limiter with configurable RPS and capacity.put_rpsandquery_rpsconfiguration with validation.PutVectorsbatches and paginatedQueryVectorsrequests.Testing
Validation performed:
uv run --no-sync pytest -q --ignore=tests/test_langchain.py --ignore=tests/test_llamaindex.pygit diff --checkpassed.Checklist