Skip to content

Add unified Cosmos read-many operations - #5385

Draft
Tomas Varon (tvaron3) wants to merge 17 commits into
Azure:mainfrom
tvaron3:tvaron3-cosmos-read-many
Draft

Tomas Varon (tvaron3) wants to merge 17 commits into
Azure:mainfrom
tvaron3:tvaron3-cosmos-read-many

Conversation

@tvaron3

@tvaron3 Tomas Varon (tvaron3) commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

Add one read-many execution path for exact item identities or complete logical partition keys, with optional parameterized filters that only narrow the selection.

  • Group selections by physical partition and execute bounded query batches, using point reads for unfiltered singleton item groups.
  • Expose SDK pages and explicit collect_all() aggregation with request charge, diagnostics, and a shared collection deadline.
  • Add a versioned native read-many open request using the retained cursor lifecycle from Add native Cosmos feed cursors #5334, preserving existing ABI layouts and complete item-buffer ownership.
  • Deduplicate selections, omit missing items, and explicitly reject durable checkpoint/resume.
  • Update documentation, changelogs, generated APIs/header, and Rust/emulator/linked C coverage.

Tomas Varon (tvaron3) and others added 10 commits September 21, 2026 18:07
Retain driver plans across native feed pages independently of durable checkpoints. Expose complete result buffers while preserving legacy ABI layouts and rejecting incomplete one-shot results.

Add cursor lifecycle, cancellation, queue-capacity and payload ownership coverage in Rust and linked C tests. Regenerate API artifacts and document the design and wrapper-managed prefetch. Preserve existing read-feed routing.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve upstream query planning changes and cursor APIs. Adapt buffered-query tests and checkpoint status handling, regenerate merged API artifacts, and renumber the cursor design to 0029 alongside the upstream 0028 spec.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude the test fixture from the generated public header through cbindgen configuration. Exercise binary DISTINCT payloads through linked C cursor tests alongside text scenarios.

Give cursor lifecycle errors appropriate messages and cover migration guidance with a regression test. Link the driver status-constant changelog entry to the upstream PR.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream API artifact renames and emulator/E2E updates while preserving native cursor behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Import ETAG within the emulator test module so change-feed tests compile without preview_dtx. Gate HPK patch test imports with preview_patch to keep default-feature lint clean.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream main while preserving retained feed cursors. Adopt the upstream removal of handle cancellation and carry diagnostics through cursor completion ownership, retaining timeout and delivery-loss behavior.

Regenerate native headers and API artifacts and update lifecycle coverage and documentation for the merged contract.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve retained cursor support while integrating native CI coverage, preview feature updates and crate-root API artifacts. Resolve the lockfile from upstream versions plus the cursor time dependency, regenerate merged APIs and retain upstream C test cleanup.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adapt native cursor status constants and checkpoint handling to upstream error::status_codes while preserving numeric values. Regenerate merged driver and native API artifacts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Request a fresh pipeline run without changing source files.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream main and reuse the merged native feed cursor implementation.

Add item and complete-partition selections with scoped parameterized filters, bounded query batches, SDK paging and explicit aggregation, and an additive native cursor entry point.

Preserve diagnostics and collection deadlines, add Rust and linked C regressions, and refresh public API artifacts and documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the Cosmos The azure_cosmos crate label Sep 30, 2026
Adopt upstream main at 420f173, including the beta.2 package versions and dependency updates.

Keep read-many entries under the unreleased Features Added sections and link them to upstream PR Azure#5385 without changing released history.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tvaron3

Copy link
Copy Markdown
Member Author

/azp run rust - cosmos - weekly

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

Add an unreleased native changelog entry for the versioned read-many request and cursor APIs, including scoped filters and checkpoint limitations. Link the entry to PR Azure#5385.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tvaron3

Copy link
Copy Markdown
Member Author

/azp run rust - cosmos - weekly

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Use queries for singleton selections on RID-addressed containers to avoid mixed name/RID document paths. Derive collected request charge from aggregate diagnostics so charged retry attempts are included. Add regressions for both cases.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tvaron3

Copy link
Copy Markdown
Member Author

/azp run rust - cosmos - weekly

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Keep ReadManyResponse debug output redacted without requiring customer item types to implement Debug under workspace-wide features; refresh the API artifact and add regression coverage.

Include connection failures in bounded external query-test setup retries, preserving the existing budget and immediate failures for authentication and other service errors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream main at 26ecc26 while preserving read-many's unreleased native changelog entry and the published 0.2.0 history.

Retain shared cursor construction with upstream option-snapshot admission validation, and refresh native API and header artifacts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pin native request rejection, byte-limited batching, populated split results, and collection deadlines across container recreation.

Preserve undefined partition-key and escaped filter-key semantics. Retain attempt diagnostics and charges on failed topology retries, repair failures, and split-retry exhaustion without double counting accepted replacements.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate upstream main at 019eaad, including rustup bootstrap, normalized User-Agent formatting, and terminal service error handling.

Preserve both read-many and upstream Features Added entries when resolving the driver changelog conflict.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@tvaron3

Copy link
Copy Markdown
Member Author

/azp run rust - cosmos - weekly

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Cosmos The azure_cosmos crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant