Repository navigation
Page large admin results without dropping data - #2
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Large user, team, model and request-log reads currently embed entire related collections or histories. This makes compact results the default without discarding fields: small responses remain complete, while large responses provide complete small values, record previews and exact paths to unread data. Agents can read the original result or selected sections without repeating the gateway operation.
All 65 administrative actions retain their names, arguments and server-side validation. Standard MCP discovery returns complete schemas by default, so clients with native tool search can defer loading them directly. The server does not enable a client setting. An optional lightweight discovery mode supports clients that eagerly load every schema.
Changes
read_admin_resultfor exact field, collection and Unicode string reads from immutable, caller-bound snapshots. Indexes distinguish unloaded data from null, false, zero and empty values. Keep records whole at page boundaries when they fit.describe_admin_toolfor complete schemas, including advanced parameters and referenced definitions. Inline top-level body references so clients can display their fields. Discovery mode retains individual tool names and safety annotations; there is no execution dispatcher.data[i].model_info.keyas a catalog identifier only in the two reviewed model-info reads. Ordinary model identifiers no longer disable paging. The field's value is still checked for recognizable credentials, and other fields/endpoints retain conservative secret detection.mainand refresh the connector on each MCP startup, including when its package version is unchanged. Document the one-time migration for existing pinned installs, restart behavior, fixed-version alternatives and Docker redeployment.response.view=fullremains available for complete inline results.LITELLM_ADMIN_RESPONSE_VIEWdefaults tocompact;LITELLM_ADMIN_SCHEMA_MODEdefaults tofull. Agents remain free to request full results.Validation
87 tests pass. The two new model-info paging regressions failed before the fix and now verify complete reconstruction, full overrides and no repeated gateway fetches. Additional cases check that secrets in the identifier value, nested metadata, other paths and other endpoints still prevent caching.
Wheel and source distribution build successfully. Live verification uses a separately installed wheel, not an editable source import.
Verify automatic updates with a moving Git branch and unchanged package version: the same launcher/cache loaded the first commit, the next commit, and then the unchanged latest commit. The exact README launcher also initialized against the live gateway and completed a read-only key-list call; installed Git provenance matched upstream
main.Post-fix live
list_models: initial text decreased from 194,169 bytes to 840 bytes (approximately 52,244 to 267o200k_basecomparison tokens). Every field and catalog identifier reconstructed exactly; full responses match the previous PR build. These are response measurements, not billing or end-to-end task-cost claims.Post-fix live
get_model: the unfiltered 8.36 MB catalog returned a compact index, and all 1,290 records reconstructed exactly across 629 detail reads. Snapshot contents, full overrides, previous-build data and gateway catalog identifiers matched in the recheck.Independent live audit of the preceding head
b11adc1verified all 65 argument schemas against the installed base; exact user/team/key/model/budget output parity; lossless Unicode/edge-value paging; temporary team/key/budget lifecycles and cleanup; rotation, HTTP authentication, read-only POST reads, caller isolation, real credential revocation, cache capacity and real five-minute expiry.Four fresh Codex/OpenCode client tasks in that audit returned correct results, including the full 67-tool catalog and optional discovery mode. Claude Code connected to the MCP, but an upstream Anthropic insufficient-credit error blocked inference. Native tool-search deferral remains client-controlled and was not established by those runs. Not every administrative write endpoint was exercised live.
Full reads and reading every page can still be large. The benchmark verifies exact reconstruction and reports first-page and all-field retrieval costs; a smaller first response does not establish equivalent reasoning quality or lower total task cost.
GitHub Actions passed for
8b41440, including tests, distribution builds, Docker build and container smoke test.