Skip to content

fix: apply request limits and feedback permissions - #78

Merged
moe-berri merged 1 commit into
mainfrom
litellm_lens_request_limits
Oct 27, 2026
Merged

moe-berri merged 1 commit into
mainfrom
litellm_lens_request_limits

Conversation

@moe-berri

@moe-berri moe-berri commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Lens now applies consistent request and query limits and uses the same read-only role checks for feedback as other write surfaces. Accepted trace content is preserved without truncation

Changes

  • Limit API request bodies to 2 MiB and ingest/internal spend bodies to 32 MiB
  • Bound total gzip expansion to 32 MiB and OTLP nesting to 64, with finite decode defaults
  • Limit buffered database responses to 64 MiB and ad-hoc SQL results to 50,000 rows
  • Reject feedback writes and deletes for read-only delegated roles
  • Restrict bundled ClickHouse ingress to this release's Lens workload
  • Update deployment and query guidance and add regression coverage

Compatibility

Oversized requests now return 413. Clients may need smaller upload batches or API requests, and large SQL queries may need narrower selections or pagination. The nesting ceiling remains enforced when an operator configures a higher depth

The bundled-storage NetworkPolicy requires a CNI that enforces policies. Other policies selecting the same pods can broaden access; external storage remains operator-managed

Validation

252 Rust tests passed locally with networking disabled, covering OTLP decoding, feedback scope checks, request-body limits, gzip boundaries, and the real ingest router through in-memory requests. Rust formatting and diff checks pass

The ClickHouse query-access, HTTP transport, and database-backed feedback regression targets compile. Their runtime checks and the Helm suite have not been run locally. This remains a draft pending CI and those integration checks


Note

Medium Risk
Breaking change for clients sending bodies above 2–32 MiB or large ad-hoc SQL results; feedback write denial for some roles may surprise integrators; bundled Helm installs need a CNI that enforces NetworkPolicy.

Overview
Introduces enforced size and decode budgets across ingest, HTTP APIs, and ClickHouse reads, and tightens feedback write authorization and bundled ClickHouse network isolation.

Request and ingest limits: General Lens API routes now use a shared 2 MiB DefaultBodyLimit (replacing disabled limits). OTLP trace/log uploads and internal spend/feedback batch inserts cap bodies at 32 MiB, with gzip expansion bounded to the same size and HTTP 413 on overflow. OTLP decoding gets finite defaults (e.g. 16,384 spans, 64 MiB decoded span bytes) and an absolute nesting ceiling of 64; JSON/protobuf preflight always runs.

Query and read limits: Buffered ClickHouse responses default to 64 MiB; ad-hoc trace SQL uses 50,000 rows plus that byte cap. Query help text reflects the new reader limits.

Authorization: Feedback PUT/DELETE now rejects InternalUserViewer and Customer delegated roles (aligned with other read-only surfaces), with tests ensuring 403 does not mutate stored feedback.

Kubernetes: The Helm chart adds an ingress NetworkPolicy so bundled ClickHouse 8123 is reachable only from this release’s Lens workload pods (with unit tests for selector overrides).

Deploy/Helm READMEs document the limits, 413 behavior, and NetworkPolicy expectations. Regression tests cover gzip boundaries, oversized ingest, body limits before auth/storage, and query reader caps.

Reviewed by Cursor Bugbot for commit 99d74bc. Bugbot is set up for automated code reviews on this repo. Configure here.

@moe-berri
moe-berri marked this pull request as ready for review October 27, 2026 03:39
@moe-berri
moe-berri merged commit b2ac5f8 into main Oct 27, 2026
12 of 13 checks passed
@moe-berri
moe-berri deleted the litellm_lens_request_limits branch October 27, 2026 03:39

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 99d74bc. Configure here.

{{- include "lens.selectorLabels" . | nindent 14 }}
ports:
- protocol: TCP
port: 8123

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NetworkPolicy can block storage probes

Medium Severity

The new ClickHouse ingress NetworkPolicy allows TCP 8123 only from Lens pods. Bundled startup, readiness, and liveness probes also use that port from the node, not from a matching pod, so a CNI that enforces policies without a host exception can keep ClickHouse unready after upgrade.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 99d74bc. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant