Gate unbounded buffered Cosmos queries - #5301
Open
Tomas Varon (tvaron3) wants to merge 6 commits into
Open
Tomas Varon (tvaron3) wants to merge 6 commits into
Tomas Varon (tvaron3) wants to merge 6 commits into
Conversation
Require a finite global TOP/LIMIT for unordered DISTINCT and non-streaming ORDER BY, with a layered allow_unbounded_queries opt-out and shared 400/20126 admission status. Preserve bounded execution and continuation restrictions. Cover admission, option precedence, routing, encoding, and buffered execution; update SDK and driver APIs, changelogs, and query specifications. Native pagination remains out of scope. Fixes Azure#5122 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream main at ec9b993 while preserving buffered-query admission and opt-out behavior. Resolve SDK and driver API conflicts by regenerating artifacts with the updated upstream generator. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Link the SDK and driver feature and breaking-change entries to the upstream pull request. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover default rejection and explicit opt-out for unbounded DISTINCT in the production query-plan comparison tests. Scope retry-count overrides to the injected read via the test client helper, keeping metadata setup retries intact. Inject a one-shot metadata 429 while preserving exact read attempt assertions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Public documentation currently overstates continuation restrictions and does not clearly identify the query shapes governed by the new option.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds admission controls for potentially unbounded Cosmos query pipelines while preserving bounded execution and routing exemptions.
Changes:
- Adds layered
allow_unbounded_queriesSDK and driver APIs. - Gates unordered
DISTINCTand non-streamingORDER BYwith 400/20126. - Implements and tests opted-in unbounded buffered ordering.
File summaries
| File | Description |
|---|---|
sdk/cosmos/docs/specs/0013-query-engine.md |
Documents buffered-query admission. |
sdk/cosmos/docs/specs/0006-error-codes-and-retries.md |
Documents admission errors. |
sdk/cosmos/docs/specs/0001-configuration-options.md |
Documents option layering. |
sdk/cosmos/azure_data_cosmos/tests/split_tests/cosmos_query_distinct_split.rs |
Opts split tests into unbounded DISTINCT. |
sdk/cosmos/azure_data_cosmos/tests/in_memory_emulator_tests/query_comparison.rs |
Tests precedence and HPK routing. |
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_vector_query.rs |
Tests unbounded vector admission. |
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_query.rs |
Tests DISTINCT admission and bounds. |
sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs |
Updates HPK DISTINCT coverage. |
sdk/cosmos/azure_data_cosmos/tests/binary_roundtrip_fuzzer.rs |
Enables unbounded DISTINCT fuzz cases. |
sdk/cosmos/azure_data_cosmos/src/options/feed.rs |
Adds the query convenience setter. |
sdk/cosmos/azure_data_cosmos/CHANGELOG.md |
Records SDK API and behavior changes. |
sdk/cosmos/azure_data_cosmos/api/API.metadata.yml |
Updates SDK API metadata. |
sdk/cosmos/azure_data_cosmos/api/API.md |
Updates SDK API surface. |
sdk/cosmos/azure_data_cosmos_driver/tests/in_memory_emulator_tests/distinct.rs |
Tests driver admission behavior. |
sdk/cosmos/azure_data_cosmos_driver/tests/gateway_query_plan_comparison.rs |
Tests Gateway-plan opt-out behavior. |
sdk/cosmos/azure_data_cosmos_driver/tests/framework/test_client.rs |
Adds per-read test options. |
sdk/cosmos/azure_data_cosmos_driver/tests/emulator_tests/driver_fault_injection.rs |
Scopes retry options to the tested read. |
sdk/cosmos/azure_data_cosmos_driver/src/options/operation_options.rs |
Adds the layered driver option. |
sdk/cosmos/azure_data_cosmos_driver/src/error/cosmos_status.rs |
Generalizes status 20126 with aliases. |
sdk/cosmos/azure_data_cosmos_driver/src/driver/dataflow/planner.rs |
Enforces admission and supports unbounded plans. |
sdk/cosmos/azure_data_cosmos_driver/src/driver/dataflow/non_streaming_ordered_merge.rs |
Adds unbounded candidate buffering. |
sdk/cosmos/azure_data_cosmos_driver/src/driver/cosmos_driver.rs |
Applies admission during planning. |
sdk/cosmos/azure_data_cosmos_driver/CHANGELOG.md |
Records driver API and behavior changes. |
sdk/cosmos/azure_data_cosmos_driver/api/API.metadata.yml |
Updates driver API metadata. |
sdk/cosmos/azure_data_cosmos_driver/api/API.md |
Updates driver API surface. |
Review details
- Files reviewed: 25/25 changed files
- Comments generated: 3
- Review effort level: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
Share continuation failures across buffered query shapes and compact the client error range to 20124-20126. Preserve existing constant names as aliases while updating symbolic diagnostics and regression coverage. Clarify partition-key exemptions in query specs, document numeric changes in both changelogs, and regenerate SDK and driver API artifacts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Member
Author
|
/azp run rust - cosmos - weekly |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Ashley Stanton-Nurse (analogrelay)
approved these changes
Sep 15, 2026
Fabian Meiswinkel (FabianMeiswinkel)
requested changes
Sep 15, 2026
Fabian Meiswinkel (FabianMeiswinkel)
left a comment
Fabian Meiswinkel (FabianMeiswinkel)
left a comment
Member
There was a problem hiding this comment.
LGTM except for one comment regarding public API - please discuss and if others feel like current API is ok you can proceed . Don't block on me rereviewing - just reset my vote if this gets triaged as won't fix.
Merge upstream main and address query-planning review feedback. Require an explicit finite global window capped at 1000 by default, including OFFSET, with a per-query configurable maximum and no unbounded opt-out. Move planning settings to PlanOptions and SDK QueryOptions, remove layered planning defaults, and separate live-only query coverage from emulator tests. Refresh tests, specs, changelogs, and generated API artifacts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Require a finite global TOP/LIMIT for unordered DISTINCT and non-streaming ORDER BY, with a layered allow_unbounded_queries opt-out and shared 400/20126 admission status. Preserve bounded execution and continuation restrictions.
Cover admission, option precedence, routing, encoding, and buffered execution; update SDK and driver APIs, changelogs, and query specifications. Native pagination remains out of scope.
Fixes #5122