From 09915d1c8bcbf4935b760cb350e135306a43cc0b Mon Sep 17 00:00:00 2001 From: Toby Martin Date: Fri, 2 Oct 2026 22:52:08 +0000 Subject: [PATCH 1/2] Support PostgreSQL text group pages and SQL-visible aggregates --- .github/workflows/ci.yml | 12 +- .github/workflows/release.yml | 3 +- CHANGELOG.md | 21 +- Cargo.lock | 4 +- Cargo.toml | 2 +- MIGRATION.md | 30 +- crates/graphql-orm-macros/Cargo.toml | 2 +- crates/graphql-orm-macros/README.md | 8 +- .../fixtures/cross-crate-relations/Cargo.lock | 4 +- crates/graphql-orm-macros/src/entity.rs | 3 + crates/graphql-orm/Cargo.toml | 4 +- crates/graphql-orm/README.md | 6 +- crates/graphql-orm/src/db.rs | 20 + crates/graphql-orm/src/graphql/orm/query.rs | 170 +++- .../tests/complete_group_pages_unsupported.rs | 2 +- .../fixtures/backend-coexistence/Cargo.lock | 4 +- .../fixtures/federation-end-to-end/Cargo.lock | 4 +- .../migration-api-compatibility/Cargo.lock | 4 +- .../repository-aggregate-consumer/Cargo.lock | 4 +- .../examples/complete_events.rs | 69 +- .../tests/portable_groups.rs | 22 + .../graphql-orm/tests/portable_group_pages.rs | 859 ++++++++++++++++++ .../reference/graphql-orm/typed-aggregates.md | 97 +- docs/reference/workspace-packages.md | 4 +- scripts/run-owned-database-lanes.sh | 6 +- 25 files changed, 1251 insertions(+), 113 deletions(-) create mode 100644 crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/tests/portable_groups.rs create mode 100644 crates/graphql-orm/tests/portable_group_pages.rs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2e6165c2..d1dbb4f5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -75,20 +75,22 @@ jobs: with: components: clippy - uses: Swatinem/rust-cache@v2 - - name: Execute complete SQLite groups and unchanged aggregate behavior - run: cargo test -p graphql-orm --locked --no-default-features --features sqlite --lib --test complete_group_pages --test grouped_aggregates + - name: Execute complete SQLite groups and authorized aggregates + run: cargo test -p graphql-orm --locked --no-default-features --features sqlite --lib --test complete_group_pages --test portable_group_pages --test grouped_aggregates - name: Execute the dependency-minimal private repository example run: cargo run --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features sqlite --example complete_events - name: Verify unsupported profiles before pool I/O run: | - cargo test -p graphql-orm --locked --no-default-features --features postgres --lib --test complete_group_pages_unsupported + cargo test -p graphql-orm --locked --no-default-features --features postgres --lib cargo test -p graphql-orm --locked --no-default-features --features mssql --lib --test complete_group_pages_unsupported - name: Execute existing aggregates on a test-owned PostgreSQL container - run: cargo test -p graphql-orm --locked --no-default-features --features postgres --test grouped_aggregates_postgres -- --ignored --test-threads=1 + run: cargo test -p graphql-orm --locked --no-default-features --features postgres --test grouped_aggregates_postgres --test portable_group_pages -- --ignored --test-threads=1 + - name: Execute the private PostgreSQL consumer without query SQL + run: cargo test --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features postgres --test portable_groups -- --ignored --test-threads=1 - name: Check explicit backend Clippy and Rustdoc lanes run: | for backend in sqlite postgres mssql; do - cargo clippy -p graphql-orm -p graphql-orm-macros --locked --no-default-features --features "$backend" --lib --test complete_group_pages --test complete_group_pages_unsupported -- -D warnings + cargo clippy -p graphql-orm -p graphql-orm-macros --locked --no-default-features --features "$backend" --lib --test complete_group_pages --test portable_group_pages --test complete_group_pages_unsupported -- -D warnings RUSTDOCFLAGS="-D warnings" cargo doc -p graphql-orm -p graphql-orm-macros --locked --no-default-features --features "$backend" --no-deps done cargo clippy --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features sqlite --all-targets -- -D warnings diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 61e7f222..85b97b5c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -156,9 +156,10 @@ jobs: cargo check -p graphql-orm --no-default-features --features postgres --locked cargo test -p graphql-orm --no-default-features --features postgres --test federation_entity_keys_postgres --locked cargo test -p graphql-orm --no-default-features --features postgres --test projection_visibility --test cross_crate_relations_fixture --test repository_counts --locked - cargo test -p graphql-orm --no-default-features --features postgres --test grouped_aggregates_postgres --locked -- --ignored --test-threads=1 + cargo test -p graphql-orm --no-default-features --features postgres --test grouped_aggregates_postgres --test portable_group_pages --locked -- --ignored --test-threads=1 cargo run --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features sqlite --example policy_projections cargo run --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features sqlite --example complete_events + cargo test --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features postgres --test portable_groups -- --ignored --test-threads=1 cargo check -p graphql-orm --no-default-features --features mssql --locked cargo test -p graphql-orm --no-default-features --features mssql --test federation_entity_keys_mssql --locked cargo clippy -p graphql-orm --all-targets --locked -- -D warnings diff --git a/CHANGELOG.md b/CHANGELOG.md index eca8ba56..b389bae3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,7 +3,7 @@ title: "Changelog" kind: reference status: active owner: workspace-maintainers -last_reviewed: 2026-10-01 +last_reviewed: 2026-10-02 review_by: 2027-02-01 supersedes: [] --- @@ -14,6 +14,25 @@ This file is the authoritative user-facing release chronology. The former [release-notes ledger](docs/archive/2026/graphql-orm-release-notes.md) is retained for historical context. +## 0.38.0 - Unreleased + +Companion macros crate: `graphql-orm-macros` **0.38.0**. + +- Add PostgreSQL `fetch_group_page` execution for one TEXT/VARCHAR grouping + field, preserving exact original values with explicit bytewise equality and + ASCII-only case-insensitive ordering plus a binary tie-breaker. Preserve + SQLite native group equality, ordering, cursor wire format and page bounds. +- Let ordinary aggregates, including ungrouped COUNT/SUM and GraphQL aggregate + roots, consume current explicit unrestricted or complete SQL read visibility. + Intersect policy and caller predicates before grouping; retain entity/field + authority and reject residual authorization before query execution. +- Add a provided aggregate entity-identity hook, generated by the macro without + requiring changes to existing handwritten aggregate-field implementations. +- Execute over 1,000-group SQLite/PostgreSQL parity, nullable/metric/cursor/policy + regressions and a framework-neutral standalone event-enumeration example. + MSSQL pages remain explicitly unsupported. Separate reads do not constitute a + cross-request snapshot under concurrent writes; see the aggregate reference. + ## 0.35.2 - 2026-10-01 - Keep standalone fixture lockfiles aligned with this package set and execute diff --git a/Cargo.lock b/Cargo.lock index 0eb2769e..a48dbb8a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3146,7 +3146,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "agql-auth", "async-graphql", @@ -3245,7 +3245,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case 0.7.1", "proc-macro2", diff --git a/Cargo.toml b/Cargo.toml index a5d5df9d..5c7aa5ff 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -37,7 +37,7 @@ cynic-parser = { version = "=0.11.2", features = ["pretty"] } futures = "0.3" getrandom = "0.3" graphql-composition = "=0.12.2" -graphql-orm = { path = "crates/graphql-orm", version = "0.35.2", default-features = false } +graphql-orm = { path = "crates/graphql-orm", version = "0.38.0", default-features = false } graphql-orm-ai-tool-profiles = { path = "crates/graphql-orm-ai-tool-profiles", version = "0.15.0" } graphql-orm-backup = { path = "crates/graphql-orm-backup", version = "0.7.2", default-features = false } graphql-orm-operation-catalog = { path = "crates/graphql-orm-operation-catalog", version = "0.4.0" } diff --git a/MIGRATION.md b/MIGRATION.md index 2f2420a7..c10eced7 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -3,7 +3,7 @@ title: "Migration Guide" kind: reference status: active owner: workspace-maintainers -last_reviewed: 2026-10-01 +last_reviewed: 2026-10-02 review_by: 2027-02-01 supersedes: [] --- @@ -13,6 +13,34 @@ supersedes: [] `graphql-orm` is distributed from GitHub only. Use a reviewed full 40-character commit in `rev`; neither the runtime nor macros crate is published to crates.io. +## Portable group pages and SQL-visible aggregates (0.38.0) + +Adopt aligned ORM/macros 0.38.0. No physical schema, data, GraphQL SDL, existing +SQLite cursor wire format or call-site change is required. `fetch_group_page` +remains supported. PostgreSQL now supports single TEXT/VARCHAR group pages: +`Binary` uses bytewise `C` comparison; the retained `SqliteNoCase` variant uses +explicit ASCII A–Z folding then bytewise original-value tie-breaking. PostgreSQL +page grouping uses exact original text equality, independently of source collation; +ordinary aggregates and SQLite page native equality remain unchanged. + +An installed row policy no longer automatically blocks ordinary `fetch`: current +`Unrestricted` or `Complete` visibility is required, with policy and caller filters +intersected before aggregation. Residual/callback-only policies still fail closed. +Generated aggregate-field enums supply an entity TypeId through a provided hook; +handwritten enums remain compatible, but must implement `entity_type_id` correctly +to use complete SQL visibility. Unrestricted visibility needs no token. + +Keep trusted authorization/public-revision context in page requests, protect +host cursor envelopes as appropriate, and recheck current policy on every call. +Page cursors do not pin a database snapshot. Concurrent insertion/deletion/renaming +can change a multi-page enumeration; refresh from None when refreshing the dropdown. +MSSQL pages and other scalar/multiple grouping keys reject before I/O. Existing +MSSQL ordinary aggregate/provider/schema gates are unchanged. + +See [provider, authorization, wire and consistency contract](docs/reference/graphql-orm/typed-aggregates.md) +and its executable standalone example and owned PostgreSQL test. This capability +is independent of runtime contracts A–D and host-managed repository timestamps. + ## Policy-aware transaction counts (0.35.2) Adopt aligned runtime/macros 0.35.2. No schema, stored-data, public API or diff --git a/crates/graphql-orm-macros/Cargo.toml b/crates/graphql-orm-macros/Cargo.toml index f7cf6ac6..a2dbb08c 100644 --- a/crates/graphql-orm-macros/Cargo.toml +++ b/crates/graphql-orm-macros/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" edition = "2024" authors = ["Toby Martin"] description = "Procedural macros for async-graphql and ORM-backed entities, relations, and CRUD operations." diff --git a/crates/graphql-orm-macros/README.md b/crates/graphql-orm-macros/README.md index 418a352d..afa22716 100644 --- a/crates/graphql-orm-macros/README.md +++ b/crates/graphql-orm-macros/README.md @@ -3,7 +3,7 @@ title: "graphql-orm-macros" kind: reference status: active owner: graphql-orm-macros-maintainers -last_reviewed: 2026-10-01 +last_reviewed: 2026-10-02 review_by: 2027-02-01 supersedes: [] --- @@ -16,13 +16,13 @@ macro/runtime versions aligned: ```toml [dependencies] -graphql-orm = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.35.2", default-features = false, features = ["sqlite"] } +graphql-orm = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.38.0", default-features = false, features = ["sqlite"] } ``` Direct use is supported for tooling that needs the macro package: ```toml -graphql-orm-macros = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.35.2", default-features = false, features = ["sqlite"] } +graphql-orm-macros = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.38.0", default-features = false, features = ["sqlite"] } ``` The direct dependency still requires a compatible `graphql-orm` runtime in the @@ -174,3 +174,5 @@ Version 0.35.1 generates projection identity for complete SQL row visibility and aligns with the core policy-aware projection fix. Existing generated aggregate enums/builders and ordinary GraphQL aggregate SDL remain unchanged; repository-only consumers still need no direct async-graphql dependency. + +Complete SQLite/PostgreSQL text-group pages and SQL-visible ordinary aggregates: see [typed aggregates](../../docs/reference/graphql-orm/typed-aggregates.md). diff --git a/crates/graphql-orm-macros/fixtures/cross-crate-relations/Cargo.lock b/crates/graphql-orm-macros/fixtures/cross-crate-relations/Cargo.lock index 2c8a3bb8..e4efbdc9 100644 --- a/crates/graphql-orm-macros/fixtures/cross-crate-relations/Cargo.lock +++ b/crates/graphql-orm-macros/fixtures/cross-crate-relations/Cargo.lock @@ -1028,7 +1028,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "async-graphql", "chacha20poly1305", @@ -1054,7 +1054,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case", "proc-macro2", diff --git a/crates/graphql-orm-macros/src/entity.rs b/crates/graphql-orm-macros/src/entity.rs index 0cb5cf7a..29ca299c 100644 --- a/crates/graphql-orm-macros/src/entity.rs +++ b/crates/graphql-orm-macros/src/entity.rs @@ -4573,6 +4573,9 @@ fn generate_entity_impl( impl ::graphql_orm::graphql::orm::TypedAggregateField<#struct_name> for #aggregate_field_name { + fn entity_type_id() -> Option<::std::any::TypeId> { + Some(::std::any::TypeId::of::<#struct_name>()) + } fn aggregate_field(self) -> &'static ::graphql_orm::graphql::orm::AggregateFieldRef { match self { #(#aggregate_field_match_arms),* diff --git a/crates/graphql-orm/Cargo.toml b/crates/graphql-orm/Cargo.toml index d394800e..6d00629e 100644 --- a/crates/graphql-orm/Cargo.toml +++ b/crates/graphql-orm/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" edition = "2024" description = "Runtime support crate for graphql-orm-macros" license = "MIT" @@ -68,7 +68,7 @@ futures = "0.3" geo = { version = "0.33", optional = true, default-features = false } geo-types = { version = "0.7", optional = true } geojson = { version = "1", optional = true, default-features = true } -graphql-orm-macros = { path = "../graphql-orm-macros", version = "0.35.2", default-features = false } +graphql-orm-macros = { path = "../graphql-orm-macros", version = "0.38.0", default-features = false } graphql-orm-operation-catalog = { workspace = true } rust_decimal = { workspace = true } serde = { version = "1", features = ["derive"] } diff --git a/crates/graphql-orm/README.md b/crates/graphql-orm/README.md index 46a2b450..7f37bac0 100644 --- a/crates/graphql-orm/README.md +++ b/crates/graphql-orm/README.md @@ -3,7 +3,7 @@ title: "graphql-orm" kind: reference status: active owner: graphql-orm-maintainers -last_reviewed: 2026-10-01 +last_reviewed: 2026-10-02 review_by: 2027-02-01 supersedes: [] --- @@ -29,7 +29,7 @@ backend: ```toml [dependencies] -graphql-orm = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.35.2", default-features = false, features = ["sqlite"] } +graphql-orm = { git = "https://github.com/Dastari/graphql-orm.git", rev = "", version = "0.38.0", default-features = false, features = ["sqlite"] } ``` This unpublished package has no docs.rs release. Use this Git README and the @@ -192,3 +192,5 @@ Complete bounded SQLite text-group pages reuse generated aggregate enums and the existing aggregate builder. See [typed aggregates](../../docs/reference/graphql-orm/typed-aggregates.md) and the SQL-free private consumer example for native distinct identities, current SQL visibility and typed continuation. Existing aggregate SDL/cursors are unchanged. + +Complete SQLite/PostgreSQL text-group pages and SQL-visible ordinary aggregates: see [typed aggregates](../../docs/reference/graphql-orm/typed-aggregates.md). diff --git a/crates/graphql-orm/src/db.rs b/crates/graphql-orm/src/db.rs index 98d20338..040b64ee 100644 --- a/crates/graphql-orm/src/db.rs +++ b/crates/graphql-orm/src/db.rs @@ -1019,6 +1019,26 @@ impl Database { Ok(visibility) } + pub(crate) async fn aggregate_read_visibility( + &self, + ctx: Option<&async_graphql::Context<'_>>, + ) -> async_graphql::Result + where + T: crate::graphql::orm::Entity, + A: crate::graphql::orm::TypedAggregateField, + { + let surface = if ctx.is_some() { + crate::graphql::orm::EntityAccessSurface::GraphqlQuery + } else { + crate::graphql::orm::EntityAccessSurface::Repository + }; + let visibility = self + .read_visibility_for(T::metadata(), ctx, surface) + .await?; + visibility.validate_identity(A::entity_type_id(), B::DIALECT)?; + Ok(visibility) + } + async fn read_visibility_for( &self, metadata: &'static crate::graphql::orm::EntityMetadata, diff --git a/crates/graphql-orm/src/graphql/orm/query.rs b/crates/graphql-orm/src/graphql/orm/query.rs index e375b40c..f4c72cd2 100644 --- a/crates/graphql-orm/src/graphql/orm/query.rs +++ b/crates/graphql-orm/src/graphql/orm/query.rs @@ -697,6 +697,13 @@ impl AggregateFieldRef { pub trait TypedAggregateField: Copy + Eq + Send + Sync + 'static { /// Returns immutable generated field evidence. fn aggregate_field(self) -> &'static AggregateFieldRef; + /// Generated entity identity for validating complete SQL visibility. + /// Handwritten implementations remain compatible; Complete visibility requires + /// this token, while explicitly Unrestricted reads need no token. + #[doc(hidden)] + fn entity_type_id() -> Option { + None + } } /// One requested aggregate expression. @@ -1944,17 +1951,28 @@ fn render_grouped_aggregate_query_page( query: &GroupedAggregateSqlQuery, page: Option<(AggregateGroupOrder, Option<&AggregateGroupCursor>)>, ) -> RenderedQuery { - let quoted_groups = query + let mut quoted_groups = query .groups .iter() .map(|field| dialect.quote_identifier_path(field.column_name)) .collect::>(); - // Preserve native GROUP BY equality. A deterministic original representative + // PostgreSQL page identities are exact text, independent of the source's + // locale/nondeterministic collation. Ordinary aggregates retain native equality. + if page.is_some() && dialect == DatabaseBackend::Postgres { + quoted_groups = quoted_groups + .iter() + .map(|column| format!("CAST({column} AS TEXT) COLLATE \"C\"")) + .collect(); + } + // Preserve native SQLite GROUP BY equality. A deterministic original representative // makes continuation stable even if native equality folds text variants. let ordered_groups = if page.is_some() { quoted_groups .iter() - .map(|column| format!("MIN({column} COLLATE BINARY)")) + .map(|column| match dialect { + DatabaseBackend::Postgres => column.clone(), + _ => format!("MIN({column} COLLATE BINARY)"), + }) .collect::>() } else { quoted_groups.clone() @@ -1987,18 +2005,25 @@ fn render_grouped_aggregate_query_page( let column = &ordered_groups[0]; let boundary = match &cursor.key { None => FilterExpression::trusted_fragment(format!("{column} IS NOT NULL"), vec![]), - Some(key) => match order { - AggregateGroupOrder::Binary => FilterExpression::trusted_fragment( - format!("{column} COLLATE BINARY > ?"), - vec![SqlValue::String(key.clone())], - ), - AggregateGroupOrder::SqliteNoCase => FilterExpression::trusted_fragment( - format!( - "({column} COLLATE NOCASE > ? OR ({column} COLLATE NOCASE = ? AND {column} COLLATE BINARY > ?))" - ), - vec![SqlValue::String(key.clone()); 3], - ), - }, + Some(key) => { + let binary = group_binary_comparison(dialect, column); + let (clause, values) = match order { + AggregateGroupOrder::Binary => { + (format!("{binary} > ?"), vec![SqlValue::String(key.clone())]) + } + AggregateGroupOrder::SqliteNoCase => { + let folded = group_ascii_comparison(dialect, column); + let bound = group_ascii_comparison(dialect, "?"); + ( + format!( + "({folded} > {bound} OR ({folded} = {bound} AND {binary} > ?))" + ), + vec![SqlValue::String(key.clone()); 3], + ) + } + }; + FilterExpression::trusted_fragment(clause, values) + } }; sql.push_str(" HAVING "); sql.push_str(&render_filter_expression( @@ -2017,9 +2042,9 @@ fn render_grouped_aggregate_query_page( vec![format!("CASE WHEN {column} IS NULL THEN 0 ELSE 1 END ASC")]; if let Some((mode, _)) = page { if mode == AggregateGroupOrder::SqliteNoCase { - order.push(format!("{column} COLLATE NOCASE ASC")); + order.push(format!("{} ASC", group_ascii_comparison(dialect, column))); } - order.push(format!("{column} COLLATE BINARY ASC")); + order.push(format!("{} ASC", group_binary_comparison(dialect, column))); } else { order.push(format!("{column} ASC")); } @@ -2033,13 +2058,31 @@ fn render_grouped_aggregate_query_page( RenderedQuery { sql, values } } -/// Ascending SQLite text-group comparison. Group equality remains native. +fn group_binary_comparison(dialect: DatabaseBackend, value: &str) -> String { + match dialect { + DatabaseBackend::Postgres => format!("({value}) COLLATE \"C\""), + _ => format!("{value} COLLATE BINARY"), + } +} + +fn group_ascii_comparison(dialect: DatabaseBackend, value: &str) -> String { + match dialect { + DatabaseBackend::Postgres => format!( + "translate(({value})::text, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') COLLATE \"C\"" + ), + _ => format!("{value} COLLATE NOCASE"), + } +} + +/// Ascending text-group comparison on supported page providers. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] pub enum AggregateGroupOrder { - /// SQLite BINARY byte comparison, with SQL NULL first. + /// Byte comparison (SQLite BINARY / PostgreSQL C), with SQL NULL first. #[default] Binary, - /// SQLite NOCASE comparison (ASCII folding), then BINARY for deterministic ties. + /// ASCII-only folding followed by byte comparison for deterministic ties. + /// The historical name is retained: PostgreSQL uses explicit ASCII translation + /// and C comparison, equivalent to SQLite NOCASE on UTF-8 text. SqliteNoCase, } @@ -2277,7 +2320,6 @@ where async fn authorize( &self, graphql_context: Option<&async_graphql::Context<'_>>, - sql_visibility: bool, ) -> crate::Result<()> { let metadata = T::metadata(); let surface = if graphql_context.is_some() { @@ -2295,11 +2337,6 @@ where ) .await .map_err(crate::graphql::errors::sqlx_error_from_graphql)?; - if self.db.row_policy().is_some() && !sql_visibility { - return Err(invalid_aggregate( - "aggregate queries require database row security or a typed SQL-renderable filter", - )); - } let mut fields = self .groups .iter() @@ -2358,10 +2395,10 @@ where Ok(()) } - /// Enumerate all distinct string groups through bounded SQLite pages. + /// Enumerate all distinct string groups through bounded SQLite/PostgreSQL pages. /// - /// Initial capability: exactly one TEXT-affinity group, ascending BINARY/NOCASE, - /// optional existing metrics and fully SQL-authorized rows. Other backends, + /// Exactly one text group, ascending byte/ASCII-folded ordering, optional + /// existing metrics and fully SQL-authorized rows. Unsupported backends, /// scalar groups and residual policies fail before the aggregate SELECT. /// The existing group_limit bounds each page (1..=1000 plus one lookahead). pub async fn fetch_group_page( @@ -2372,7 +2409,11 @@ where where T: 'static, { - if B::DIALECT != DatabaseBackend::Sqlite || self.groups.len() != 1 { + if !matches!( + B::DIALECT, + DatabaseBackend::Sqlite | DatabaseBackend::Postgres + ) || self.groups.len() != 1 + { return Err(invalid_aggregate( "unsupported aggregate group pagination capability", )); @@ -2383,7 +2424,11 @@ where .find(|c| c.name == field.column_name) .is_some_and(|c| { let ty = c.sql_type.to_ascii_uppercase(); - ty.contains("TEXT") || ty.contains("CHAR") || ty.contains("CLOB") + if B::DIALECT == DatabaseBackend::Postgres { + ty == "TEXT" || ty.starts_with("VARCHAR") || ty.starts_with("CHARACTER VARYING") + } else { + ty.contains("TEXT") || ty.contains("CHAR") || ty.contains("CLOB") + } }); if field.kind != AggregateFieldKind::Text || !text_storage { return Err(invalid_aggregate( @@ -2412,7 +2457,7 @@ where if let Some(filter) = &self.filter { filter.validate()?; } - self.authorize(None, true).await?; + self.authorize(None).await?; let visibility = self .db .read_visibility::(None, super::EntityAccessSurface::Repository) @@ -2538,11 +2583,29 @@ where if let Some(filter) = &self.filter { filter.validate()?; } - self.authorize(context, false).await?; - let filter = self + self.authorize(context).await?; + let visibility = self + .db + .aggregate_read_visibility::(context) + .await + .map_err(crate::graphql::errors::sqlx_error_from_graphql)?; + if visibility.requires_residual_checks() { + return Err(invalid_aggregate( + "aggregate queries require complete SQL row authorization", + )); + } + let filters: Vec<_> = self .filter .as_ref() - .and_then(DatabaseFilter::to_filter_expression); + .and_then(DatabaseFilter::to_filter_expression) + .into_iter() + .chain(visibility.predicate_expression().cloned()) + .collect(); + let filter = if filters.is_empty() { + None + } else { + Some(FilterExpression::And(filters)) + }; let query = GroupedAggregateSqlQuery { table: T::TABLE_NAME, groups: self.groups.clone(), @@ -2558,6 +2621,7 @@ where self.auth.as_ref(), ) .await?; + self.db.observe_read(&rendered.sql, rows.len()); rows.iter() .map(|row| decode_aggregate_row::(row, &query.groups, &query.metrics)) .collect() @@ -4612,6 +4676,42 @@ mod grouped_aggregate_tests { AggregateFieldRef::generated(name, name, name, kind, false, true, OPERATORS) } + #[test] + fn postgres_page_comparisons_bind_ascii_folded_boundaries_and_exact_groups() { + let query = GroupedAggregateSqlQuery { + table: "history", + groups: vec![field("event", AggregateFieldKind::Text)], + metrics: vec![], + filter: Some(FilterExpression::trusted_fragment( + "tenant = ?", + vec![SqlValue::String("alpha".into())], + )), + group_limit: 101, + }; + let cursor = AggregateGroupCursor { + format_version: 1, + fingerprint: "a".repeat(64), + key: Some("Event".into()), + }; + let rendered = render_grouped_aggregate_query_page( + DatabaseBackend::Postgres, + &query, + Some((AggregateGroupOrder::SqliteNoCase, Some(&cursor))), + ); + assert!( + rendered + .sql + .contains("GROUP BY CAST(\"event\" AS TEXT) COLLATE \"C\" HAVING") + ); + assert!(!rendered.sql.contains("NOCASE")); + assert!(!rendered.sql.contains("lower(")); + assert!(rendered.sql.contains("translate(($2)::text")); + assert!(rendered.sql.contains("translate(($3)::text")); + assert!(rendered.sql.contains(" > $4")); + assert_eq!(rendered.values.len(), 4); + assert!(rendered.sql.ends_with("LIMIT 101")); + } + #[test] fn write_decision_selects_use_backend_specific_locks() { let query = SelectQuery { diff --git a/crates/graphql-orm/tests/complete_group_pages_unsupported.rs b/crates/graphql-orm/tests/complete_group_pages_unsupported.rs index afb22d2c..2415e3aa 100644 --- a/crates/graphql-orm/tests/complete_group_pages_unsupported.rs +++ b/crates/graphql-orm/tests/complete_group_pages_unsupported.rs @@ -1,4 +1,4 @@ -#![cfg(all(any(feature = "postgres", feature = "mssql"), not(feature = "sqlite")))] +#![cfg(all(feature = "mssql", not(any(feature = "sqlite", feature = "postgres"))))] use graphql_orm::prelude::*; #[derive(RepositoryEntity, Clone, serde::Serialize, serde::Deserialize)] diff --git a/crates/graphql-orm/tests/fixtures/backend-coexistence/Cargo.lock b/crates/graphql-orm/tests/fixtures/backend-coexistence/Cargo.lock index 4ab287c2..ae344fa2 100644 --- a/crates/graphql-orm/tests/fixtures/backend-coexistence/Cargo.lock +++ b/crates/graphql-orm/tests/fixtures/backend-coexistence/Cargo.lock @@ -1350,7 +1350,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "agql-auth", "async-graphql", @@ -1420,7 +1420,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case", "proc-macro2", diff --git a/crates/graphql-orm/tests/fixtures/federation-end-to-end/Cargo.lock b/crates/graphql-orm/tests/fixtures/federation-end-to-end/Cargo.lock index 40cebd67..72790b98 100644 --- a/crates/graphql-orm/tests/fixtures/federation-end-to-end/Cargo.lock +++ b/crates/graphql-orm/tests/fixtures/federation-end-to-end/Cargo.lock @@ -2508,7 +2508,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "async-graphql", "chacha20poly1305", @@ -2531,7 +2531,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case 0.7.1", "proc-macro2", diff --git a/crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.lock b/crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.lock index 13852ec0..e77ae08d 100644 --- a/crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.lock +++ b/crates/graphql-orm/tests/fixtures/migration-api-compatibility/Cargo.lock @@ -1004,7 +1004,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "async-graphql", "chacha20poly1305", @@ -1030,7 +1030,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case", "proc-macro2", diff --git a/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.lock b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.lock index b74527bc..7d76a2c5 100644 --- a/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.lock +++ b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.lock @@ -1004,7 +1004,7 @@ dependencies = [ [[package]] name = "graphql-orm" -version = "0.35.2" +version = "0.38.0" dependencies = [ "async-graphql", "chacha20poly1305", @@ -1030,7 +1030,7 @@ dependencies = [ [[package]] name = "graphql-orm-macros" -version = "0.35.2" +version = "0.38.0" dependencies = [ "convert_case", "proc-macro2", diff --git a/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/examples/complete_events.rs b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/examples/complete_events.rs index de21b67d..db026f9e 100644 --- a/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/examples/complete_events.rs +++ b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/examples/complete_events.rs @@ -1,13 +1,24 @@ -//! Run with --no-default-features --features sqlite --example complete_events. +//! SQLite executable; PostgreSQL execution is covered by the owned portable_groups test. //! Private models and complete enumeration, without application query SQL or GraphQL roots. -#[cfg(feature = "sqlite")] +#[cfg(any(feature = "sqlite", feature = "postgres"))] use graphql_orm::prelude::*; -#[cfg(feature = "sqlite")] +#[cfg(any(feature = "sqlite", feature = "postgres"))] #[derive(RepositoryEntity, Clone, serde::Serialize, serde::Deserialize)] -#[repository_entity( - backend = "sqlite", - table = "private_event_history", - plural = "PrivateEventHistory" +#[cfg_attr( + feature = "sqlite", + repository_entity( + backend = "sqlite", + table = "private_event_history", + plural = "PrivateEventHistory" + ) +)] +#[cfg_attr( + all(feature = "postgres", not(feature = "sqlite")), + repository_entity( + backend = "postgres", + table = "private_event_history", + plural = "PrivateEventHistory" + ) )] struct PrivateEvent { #[primary_key] @@ -19,10 +30,8 @@ struct PrivateEvent { #[graphql_orm(private, sensitive)] custody: Option, } -#[cfg(feature = "sqlite")] -#[tokio::main] -async fn main() -> graphql_orm::Result<()> { - let db = Database::::connect_sqlite("sqlite::memory:").await?; +#[cfg(any(feature = "sqlite", feature = "postgres"))] +pub async fn run(db: &Database) -> graphql_orm::Result<()> { let plan = db .schema() .plan_migration_to_entities( @@ -34,20 +43,20 @@ async fn main() -> graphql_orm::Result<()> { db.schema() .apply_migration(&plan, Default::default()) .await?; - for index in 0..225 { + for index in 0..1225 { PrivateEvent::insert( - &db, + db, CreatePrivateEventInput { id: index.to_string(), tenant: "tenant-a".into(), - event: format!("Event-{index:03}"), + event: format!("Event-{index:04}"), custody: None, }, ) .await?; } PrivateEvent::insert( - &db, + db, CreatePrivateEventInput { id: "other".into(), tenant: "tenant-b".into(), @@ -64,7 +73,7 @@ async fn main() -> graphql_orm::Result<()> { let mut cursor = None; let mut events = Vec::new(); loop { - let page = PrivateEvent::aggregate(&db) + let page = PrivateEvent::aggregate(db) .filter(PrivateEventWhereInput { tenant: Some(StringFilter { eq: Some("tenant-a".into()), @@ -73,7 +82,7 @@ async fn main() -> graphql_orm::Result<()> { ..Default::default() }) .group_by(PrivateEventAggregateField::Event)? - .group_limit(37)? + .group_limit(100)? .fetch_group_page(options.clone(), cursor.as_ref()) .await?; for row in page.rows { @@ -90,16 +99,34 @@ async fn main() -> graphql_orm::Result<()> { break; } } - assert_eq!(events.len(), 225); - assert_eq!(events[0], "Event-000"); - assert_eq!(events.last().unwrap(), "Event-224"); + assert_eq!(events.len(), 1225); + assert_eq!(events[0], "Event-0000"); + assert_eq!(events.last().unwrap(), "Event-1224"); + let rows = PrivateEvent::aggregate(db) + .filter(PrivateEventWhereInput { + tenant: Some(StringFilter { + eq: Some("tenant-a".into()), + ..Default::default() + }), + ..Default::default() + }) + .count_rows()? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(1225)); println!( "Enumerated {} original event types in bounded authorized pages", events.len() ); Ok(()) } +#[cfg(feature = "sqlite")] +#[tokio::main] +async fn main() -> graphql_orm::Result<()> { + let db = Database::::connect_sqlite("sqlite::memory:").await?; + run(&db).await +} #[cfg(not(feature = "sqlite"))] fn main() { - println!("Complete text-group pages currently require the SQLite feature"); + println!("Use the owned portable_groups PostgreSQL test; MSSQL pages are unsupported"); } diff --git a/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/tests/portable_groups.rs b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/tests/portable_groups.rs new file mode 100644 index 00000000..4816b15a --- /dev/null +++ b/crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/tests/portable_groups.rs @@ -0,0 +1,22 @@ +#![cfg(all(feature = "postgres", not(any(feature = "sqlite", feature = "mssql"))))] +#[allow(dead_code)] +#[path = "../examples/complete_events.rs"] +mod example; +#[path = "../../../support/owned_postgres.rs"] +mod owned_postgres; +#[tokio::test] +#[ignore = "owns verified disposable PostgreSQL 17 container"] +async fn postgres_private_consumer_enumerates_every_event_without_query_sql() +-> Result<(), Box> { + let mut postgres = owned_postgres::OwnedPostgres::start("consumer-portable-groups")?; + let db = + graphql_orm::db::Database::::connect_postgres( + &postgres.url, + ) + .await?; + example::run(&db).await?; + db.pool().close().await; + drop(db); + postgres.cleanup()?; + Ok(()) +} diff --git a/crates/graphql-orm/tests/portable_group_pages.rs b/crates/graphql-orm/tests/portable_group_pages.rs new file mode 100644 index 00000000..1a1b1d2b --- /dev/null +++ b/crates/graphql-orm/tests/portable_group_pages.rs @@ -0,0 +1,859 @@ +#![cfg(all(any(feature = "sqlite", feature = "postgres"), not(feature = "mssql")))] + +use graphql_orm::prelude::*; +use std::sync::{Arc, Mutex}; +#[cfg(feature = "postgres")] +#[path = "support/owned_postgres.rs"] +mod owned_postgres; +#[cfg(feature = "sqlite")] +type Backend = SqliteBackend; +#[cfg(all(feature = "postgres", not(feature = "sqlite")))] +type Backend = PostgresBackend; + +#[derive(RepositoryEntity, Clone, serde::Serialize, serde::Deserialize)] +#[cfg_attr( + feature = "sqlite", + repository_entity( + backend = "sqlite", + table = "portable_history", + plural = "PortableHistory" + ) +)] +#[cfg_attr( + all(feature = "postgres", not(feature = "sqlite")), + repository_entity( + backend = "postgres", + table = "portable_history", + plural = "PortableHistory" + ) +)] +struct History { + #[primary_key] + #[graphql_orm(auto_generated = false)] + id: String, + #[filterable(type = "string")] + tenant: String, + #[filterable(type = "string")] + event: Option, + units: i64, +} + +struct Owned { + db: Database, + #[cfg(feature = "postgres")] + postgres: owned_postgres::OwnedPostgres, +} +impl Owned { + async fn start() -> Result> { + #[cfg(feature = "postgres")] + let postgres = owned_postgres::OwnedPostgres::start("portable-groups")?; + #[cfg(feature = "sqlite")] + let db = Database::new( + sqlx::sqlite::SqlitePoolOptions::new() + .max_connections(1) + .connect("sqlite::memory:") + .await?, + ); + #[cfg(all(feature = "postgres", not(feature = "sqlite")))] + let db = Database::new( + sqlx::postgres::PgPoolOptions::new() + .max_connections(1) + .connect(&postgres.url) + .await?, + ); + let db = db.with_pagination_config(PaginationConfig::legacy()); + let plan = db + .schema() + .plan_migration_to_entities("fixture", "owned portable groups", &[History::metadata()]) + .await?; + db.schema() + .apply_migration(&plan, Default::default()) + .await?; + Ok(Self { + db, + #[cfg(feature = "postgres")] + postgres, + }) + } + async fn finish(self) -> Result<(), Box> { + self.db.pool().close().await; + #[cfg(feature = "postgres")] + { + let mut postgres = self.postgres; + postgres.cleanup()?; + } + Ok(()) + } +} +fn options(exclude_blank: bool) -> AggregateGroupPageOptions { + AggregateGroupPageOptions { + order: AggregateGroupOrder::SqliteNoCase, + exclude_blank, + context: "verified-alpha:policy-r1".into(), + } +} +fn tenant(owner: &str) -> HistoryWhereInput { + HistoryWhereInput { + tenant: Some(StringFilter { + eq: Some(owner.into()), + ..Default::default() + }), + ..Default::default() + } +} +async fn insert( + db: &Database, + id: &str, + owner: &str, + event: Option<&str>, + units: i64, +) -> graphql_orm::Result<()> { + History::insert( + db, + CreateHistoryInput { + id: id.into(), + tenant: owner.into(), + event: event.map(str::to_owned), + units, + }, + ) + .await?; + Ok(()) +} +#[derive(Clone, Default)] +struct Reads(Arc>>); +impl ReadQueryObserver for Reads { + fn on_read(&self, _: &str, rows: usize) { + self.0.lock().unwrap().push(rows); + } +} +#[derive(Clone)] +struct Visibility { + owner: Arc>, + mode: u8, +} +impl RowPolicy for Visibility { + fn read_visibility<'a>( + &'a self, + _: Option<&'a async_graphql::Context<'_>>, + _: &'a Database, + _: &'static str, + _: Option<&'static str>, + _: EntityAccessSurface, + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async move { + Ok(match self.mode { + 0 => ReadVisibility::Unrestricted, + 1 => ReadVisibility::Complete(ReadPredicate::from_filter::(&tenant( + &self.owner.lock().unwrap(), + ))?), + 2 => ReadVisibility::Prefilter(ReadPredicate::from_filter::(&tenant( + &self.owner.lock().unwrap(), + ))?), + _ => ReadVisibility::CallbackOnly, + }) + }) + } + fn can_read_row<'a>( + &'a self, + _: Option<&'a async_graphql::Context<'_>>, + _: &'a Database, + _: &'static str, + _: Option<&'static str>, + _: EntityAccessSurface, + _: &'a (dyn std::any::Any + Send + Sync), + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async { panic!("residual evaluation must never run after aggregation") }) + } + fn can_write_row<'a>( + &'a self, + _: Option<&'a async_graphql::Context<'_>>, + _: &'a Database, + _: &'static str, + _: Option<&'static str>, + _: EntityAccessSurface, + _: &'a (dyn std::any::Any + Send + Sync), + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async { Ok(true) }) + } +} + +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn complete_original_groups_and_metrics_match_ascii_byte_order_across_pages() +-> Result<(), Box> { + let mut owned = Owned::start().await?; + let mut keys: Vec<_> = (0..1305).map(|i| format!("Event-{i:04}")).collect(); + keys.extend( + [ + "event-0000", + "EVENT-0000", + " Case ", + "Case", + "case", + "é", + "É", + "İ", + "i", + "ß", + "SS", + "中", + "e\u{301}", + "\t", + "\n", + "😀", + ] + .into_iter() + .map(str::to_owned), + ); + for (i, key) in keys.iter().enumerate() { + insert(&owned.db, &format!("a-{i}"), "alpha", Some(key), 1).await?; + insert(&owned.db, &format!("duplicate-{i}"), "alpha", Some(key), 2).await?; + insert(&owned.db, &format!("b-{i}"), "beta", Some(key), 1000).await?; + } + for (i, value) in [None, Some(""), Some(" "), Some(" ")] + .into_iter() + .enumerate() + { + insert(&owned.db, &format!("blank-{i}"), "alpha", value, 4).await?; + } + keys.sort_by(|a, b| { + a.to_ascii_lowercase() + .as_bytes() + .cmp(b.to_ascii_lowercase().as_bytes()) + .then_with(|| a.as_bytes().cmp(b.as_bytes())) + }); + let observer = Reads::default(); + owned.db = owned.db.with_read_query_observer(observer.clone()); + owned.db.set_row_policy(Visibility { + owner: Arc::new(Mutex::new("alpha".into())), + mode: 1, + }); + for (order, limit) in [ + (AggregateGroupOrder::SqliteNoCase, 100), + (AggregateGroupOrder::Binary, 100), + (AggregateGroupOrder::SqliteNoCase, 1000), + ] { + let mut expected = keys.clone(); + if order == AggregateGroupOrder::Binary { + expected.sort(); + } + let mut cursor = None; + let mut actual = Vec::new(); + let mut pages = 0; + loop { + let page = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .count_rows()? + .count(HistoryAggregateField::Event)? + .sum(HistoryAggregateField::Units)? + .min(HistoryAggregateField::Units)? + .max(HistoryAggregateField::Units)? + .group_limit(limit)? + .fetch_group_page( + AggregateGroupPageOptions { + order, + ..options(true) + }, + cursor.as_ref(), + ) + .await?; + assert!(page.rows.len() <= limit as usize); + for row in page.rows { + let AggregateValue::Text(key) = &row.groups[0].value else { + panic!("excluded null group") + }; + actual.push(key.clone()); + assert_eq!( + row.metrics + .iter() + .map(|m| m.value.clone()) + .collect::>(), + vec![ + AggregateValue::Count(2), + AggregateValue::Count(2), + AggregateValue::Integral(3), + AggregateValue::Integral(1), + AggregateValue::Integral(2) + ] + ); + } + pages += 1; + cursor = page + .end_cursor + .map(|c| AggregateGroupCursor::from_json(&c.to_json()?)) + .transpose()?; + if !page.has_next_page { + break; + } + assert!(cursor.is_some()); + } + assert_eq!(actual, expected); + assert_eq!(pages, keys.len().div_ceil(limit as usize)); + } + assert!(observer.0.lock().unwrap().iter().all(|rows| *rows <= 1001)); + owned.finish().await +} + +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn nullable_groups_and_policy_aware_ordinary_metrics_are_exact() +-> Result<(), Box> { + let mut owned = Owned::start().await?; + for (id, owner, value, units) in [ + ("a", "alpha", None, 2), + ("b", "alpha", None, 3), + ("c", "alpha", Some(""), 4), + ("d", "alpha", Some("X"), 5), + ("e", "beta", Some("Hidden"), 100), + ] { + insert(&owned.db, id, owner, value, units).await?; + } + let owner = Arc::new(Mutex::new("alpha".into())); + owned.db.set_row_policy(Visibility { + owner: owner.clone(), + mode: 1, + }); + let mut cursor = None; + let mut values = vec![]; + loop { + let page = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .count_rows()? + .sum(HistoryAggregateField::Units)? + .group_limit(1)? + .fetch_group_page(options(false), cursor.as_ref()) + .await?; + values.push(( + page.rows[0].groups[0].value.clone(), + page.rows[0].metrics[0].value.clone(), + )); + cursor = page.end_cursor; + if !page.has_next_page { + break; + } + } + assert_eq!( + values, + vec![ + (AggregateValue::Null, AggregateValue::Count(2)), + (AggregateValue::Text("".into()), AggregateValue::Count(1)), + (AggregateValue::Text("X".into()), AggregateValue::Count(1)) + ] + ); + let rows = History::aggregate(&owned.db) + .count_rows()? + .sum(HistoryAggregateField::Units)? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(4)); + assert_eq!(rows[0].metrics[1].value, AggregateValue::Integral(14)); + // Caller and policy filters are intersected, never substituted. + let rows = History::aggregate(&owned.db) + .filter(tenant("beta")) + .count_rows()? + .sum(HistoryAggregateField::Units)? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(0)); + assert_eq!(rows[0].metrics[1].value, AggregateValue::Null); + for mode in [2, 3] { + owned.db.set_row_policy(Visibility { + owner: owner.clone(), + mode, + }); + assert!( + History::aggregate(&owned.db) + .count_rows()? + .fetch() + .await + .is_err() + ); + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(false), None) + .await + .is_err() + ); + } + owned.db.set_row_policy(Visibility { owner, mode: 0 }); + let rows = History::aggregate(&owned.db) + .count_rows()? + .sum(HistoryAggregateField::Units)? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(5)); + assert_eq!(rows[0].metrics[1].value, AggregateValue::Integral(114)); + owned.finish().await +} + +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn cursor_bindings_and_current_policy_reject_before_query() +-> Result<(), Box> { + let mut owned = Owned::start().await?; + for (id, owner, value) in [ + ("a", "alpha", "A"), + ("b", "alpha", "a"), + ("c", "beta", "Hidden"), + ] { + insert(&owned.db, id, owner, Some(value), 1).await?; + } + let owner = Arc::new(Mutex::new("alpha".into())); + owned.db.set_row_policy(Visibility { + owner: owner.clone(), + mode: 1, + }); + let observer = Reads::default(); + owned.db = owned.db.with_read_query_observer(observer.clone()); + let cursor = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .group_limit(1)? + .fetch_group_page(options(true), None) + .await? + .end_cursor + .unwrap(); + let mut changed = options(true); + changed.context = "alpha:r2".into(); + for opts in [ + changed, + options(false), + AggregateGroupPageOptions { + order: AggregateGroupOrder::Binary, + ..options(true) + }, + ] { + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(opts, Some(&cursor)) + .await + .is_err() + ); + } + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .count_rows()? + .fetch_group_page(options(true), Some(&cursor)) + .await + .is_err() + ); + assert!( + History::aggregate(&owned.db) + .filter(tenant("beta")) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), Some(&cursor)) + .await + .is_err() + ); + *owner.lock().unwrap() = "beta".into(); + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), Some(&cursor)) + .await + .is_err() + ); + for wire in ["{}".into(), "null".into(), "x".repeat(256 * 1024 + 1)] { + assert!(AggregateGroupCursor::from_json(&wire).is_err()); + } + let mut wire: serde_json::Value = serde_json::from_str(&cursor.to_json()?)?; + for key in [ + serde_json::json!(42), + serde_json::json!(true), + serde_json::json!("x".repeat(65537)), + ] { + wire["key"] = key; + assert!(AggregateGroupCursor::from_json(&wire.to_string()).is_err()); + } + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Units)? + .fetch_group_page(options(true), None) + .await + .is_err() + ); + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .group_by(HistoryAggregateField::Tenant)? + .fetch_group_page(options(true), None) + .await + .is_err() + ); + assert_eq!(observer.0.lock().unwrap().len(), 1); + let page = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), None) + .await?; + assert_eq!( + page.rows[0].groups[0].value, + AggregateValue::Text("Hidden".into()) + ); + owned.finish().await +} + +struct DenyField(&'static str); +impl FieldPolicy for DenyField { + fn can_read_field<'a>( + &'a self, + _: &'a async_graphql::Context<'_>, + _: &'a Database, + _: &'static str, + field: &'static str, + _: Option<&'static str>, + _: Option<&'a (dyn std::any::Any + Send + Sync)>, + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async move { Ok(field != self.0) }) + } + fn can_write_field<'a>( + &'a self, + _: &'a async_graphql::Context<'_>, + _: &'a Database, + _: &'static str, + _: &'static str, + _: Option<&'static str>, + _: Option<&'a (dyn std::any::Any + Send + Sync)>, + _: Option<&'a (dyn std::any::Any + Send + Sync)>, + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async { Ok(false) }) + } + fn can_read_repository_field<'a>( + &'a self, + _: Option>, + _: &'a Database, + _: &'static str, + field: &'static str, + _: Option<&'static str>, + _: Option<&'a (dyn std::any::Any + Send + Sync)>, + ) -> graphql_orm::futures::future::BoxFuture<'a, async_graphql::Result> { + Box::pin(async move { Ok(field != self.0) }) + } +} +struct Root; +#[async_graphql::Object] +impl Root { + async fn visible_count(&self, ctx: &async_graphql::Context<'_>) -> async_graphql::Result { + let db = ctx.data::>()?; + let rows = History::aggregate(db) + .count_rows()? + .sum(HistoryAggregateField::Units)? + .fetch_with_graphql_context(ctx) + .await?; + match rows[0].metrics[0].value { + AggregateValue::Count(count) => Ok(count), + _ => Err("unexpected count".into()), + } + } +} +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn group_metric_filter_and_entity_authority_remain_fail_closed() +-> Result<(), Box> { + let mut owned = Owned::start().await?; + insert(&owned.db, "a", "alpha", Some("A"), 1).await?; + insert(&owned.db, "b", "beta", Some("B"), 2).await?; + let observer = Reads::default(); + owned.db = owned.db.with_read_query_observer(observer.clone()); + let visibility = Visibility { + owner: Arc::new(Mutex::new("alpha".into())), + mode: 1, + }; + owned.db.set_row_policy(visibility); + for field in ["event", "units", "tenant"] { + owned.db.set_field_policy(DenyField(field)); + assert!( + History::aggregate(&owned.db) + .filter(tenant("alpha")) + .group_by(HistoryAggregateField::Event)? + .sum(HistoryAggregateField::Units)? + .fetch_group_page(options(true), None) + .await + .is_err() + ); + assert!( + History::aggregate(&owned.db) + .filter(tenant("alpha")) + .group_by(HistoryAggregateField::Event)? + .sum(HistoryAggregateField::Units)? + .fetch() + .await + .is_err() + ); + } + assert!(observer.0.lock().unwrap().is_empty()); + owned.db.set_field_policy(DenyField("unrelated")); + let schema = async_graphql::Schema::build( + Root, + async_graphql::EmptyMutation, + async_graphql::EmptySubscription, + ) + .data(owned.db.clone()) + .finish(); + let response = schema.execute("{ visibleCount }").await; + assert!(response.errors.is_empty(), "{:?}", response.errors); + assert_eq!(response.data, async_graphql::value!({"visibleCount": 1})); + drop(schema); + owned.db.set_field_policy(DenyField("units")); + let schema = async_graphql::Schema::build( + Root, + async_graphql::EmptyMutation, + async_graphql::EmptySubscription, + ) + .data(owned.db.clone()) + .finish(); + assert!(!schema.execute("{ visibleCount }").await.errors.is_empty()); + drop(schema); + assert_eq!(observer.0.lock().unwrap().len(), 1); + owned + .db + .set_entity_policy(ScopeEntityPolicy::authenticated()); + assert!( + History::aggregate(&owned.db) + .count_rows()? + .fetch() + .await + .is_err() + ); + assert!( + History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), None) + .await + .is_err() + ); + assert_eq!(observer.0.lock().unwrap().len(), 1); + owned.finish().await +} + +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn concurrent_new_groups_are_not_a_cross_request_snapshot() +-> Result<(), Box> { + let owned = Owned::start().await?; + insert(&owned.db, "b", "alpha", Some("B"), 1).await?; + insert(&owned.db, "d", "alpha", Some("D"), 1).await?; + let first = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .group_limit(1)? + .fetch_group_page(options(true), None) + .await?; + insert(&owned.db, "a", "alpha", Some("A"), 1).await?; + insert(&owned.db, "c", "alpha", Some("C"), 1).await?; + let next = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), first.end_cursor.as_ref()) + .await?; + assert_eq!( + next.rows + .iter() + .map(|r| r.groups[0].value.clone()) + .collect::>(), + vec![ + AggregateValue::Text("C".into()), + AggregateValue::Text("D".into()) + ] + ); + // A inserted behind the boundary is skipped; C ahead is seen. Re-enumerate + // from None for a refreshed dropdown instead of treating cursors as snapshots. + owned.finish().await +} + +#[cfg(all(feature = "postgres", not(feature = "sqlite")))] +#[tokio::test] +#[ignore = "owns disposable loopback PostgreSQL 17 with ICU"] +async fn postgres_nondeterministic_source_collation_does_not_merge_original_groups() +-> Result<(), Box> { + let owned = Owned::start().await?; + sqlx::query("CREATE COLLATION fixture_case_insensitive (provider = icu, locale = 'und-u-ks-level2', deterministic = false)").execute(owned.db.pool()).await?; + sqlx::query("ALTER TABLE portable_history ALTER COLUMN event TYPE TEXT COLLATE fixture_case_insensitive").execute(owned.db.pool()).await?; + for (id, key) in [("a", "A"), ("b", "a"), ("c", "É"), ("d", "é")] { + insert(&owned.db, id, "alpha", Some(key), 1).await?; + } + let mut cursor = None; + let mut keys = vec![]; + loop { + let page = History::aggregate(&owned.db) + .group_by(HistoryAggregateField::Event)? + .count_rows()? + .group_limit(1)? + .fetch_group_page(options(true), cursor.as_ref()) + .await?; + assert_eq!(page.rows[0].metrics[0].value, AggregateValue::Count(1)); + keys.push(page.rows[0].groups[0].value.clone()); + cursor = page.end_cursor; + if !page.has_next_page { + break; + } + } + assert_eq!( + keys, + ["A", "a", "É", "é"].map(|s| AggregateValue::Text(s.into())) + ); + owned.finish().await +} + +#[derive(Clone, Copy, PartialEq, Eq)] +struct HandwrittenField; +impl TypedAggregateField for HandwrittenField { + fn aggregate_field(self) -> &'static AggregateFieldRef { + HistoryAggregateField::Event.aggregate_field() + } +} +#[derive(Clone, Copy, PartialEq, Eq)] +struct WrongIdentityField; +impl TypedAggregateField for WrongIdentityField { + fn aggregate_field(self) -> &'static AggregateFieldRef { + HistoryAggregateField::Event.aggregate_field() + } + fn entity_type_id() -> Option { + Some(std::any::TypeId::of::<()>()) + } +} +#[tokio::test] +#[cfg_attr( + feature = "postgres", + ignore = "owns disposable loopback PostgreSQL 17" +)] +async fn handwritten_aggregate_identity_remains_compatible_and_complete_visibility_is_bound() +-> Result<(), Box> { + let mut owned = Owned::start().await?; + insert(&owned.db, "a", "alpha", Some("A"), 1).await?; + let owner = Arc::new(Mutex::new("alpha".into())); + owned.db.set_row_policy(Visibility { + owner: owner.clone(), + mode: 1, + }); + assert!( + GroupedAggregateQuery::::new( + &owned.db + ) + .count_rows()? + .fetch() + .await + .is_err() + ); + assert!( + GroupedAggregateQuery::::new( + &owned.db + ) + .count_rows()? + .fetch() + .await + .is_err() + ); + owned.db.set_row_policy(Visibility { owner, mode: 0 }); + let rows = GroupedAggregateQuery::::new( + &owned.db, + ) + .count_rows()? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(1)); + owned.finish().await +} + +#[cfg(all(feature = "postgres", not(feature = "sqlite")))] +#[tokio::test] +#[ignore = "owns disposable loopback PostgreSQL 17 with nonowner RLS role"] +async fn postgres_db_auth_context_filters_before_pages_and_ordinary_aggregation() +-> Result<(), Box> { + let owned = Owned::start().await?; + for (id, owner, key, units) in [ + ("a", "alpha", "A", 1), + ("b", "alpha", "B", 2), + ("c", "beta", "Other", 100), + ] { + insert(&owned.db, id, owner, Some(key), units).await?; + } + for statement in [ + "CREATE ROLE fixture_reader NOLOGIN NOSUPERUSER NOBYPASSRLS", + "GRANT SELECT ON portable_history TO fixture_reader", + "ALTER TABLE portable_history ENABLE ROW LEVEL SECURITY", + "CREATE POLICY tenant_read ON portable_history FOR SELECT TO fixture_reader USING (tenant = current_setting('app.tenant_id', true))", + ] { + sqlx::query(statement).execute(owned.db.pool()).await?; + } + let pool = sqlx::postgres::PgPoolOptions::new() + .max_connections(1) + .after_connect(|connection, _| { + Box::pin(async move { + sqlx::query("SET ROLE fixture_reader") + .execute(connection) + .await?; + Ok(()) + }) + }) + .connect(&owned.postgres.url) + .await?; + let db = Database::::new(pool); + let user: String = sqlx::query_scalar("SELECT current_user") + .fetch_one(db.pool()) + .await?; + assert_eq!(user, "fixture_reader"); + let alpha = DbAuthContext::from_parts("reader-a", vec![], vec![], Some("alpha".into())); + let beta = DbAuthContext::from_parts("reader-b", vec![], vec![], Some("beta".into())); + let first = History::aggregate(&db) + .with_auth(Some(&alpha)) + .group_by(HistoryAggregateField::Event)? + .group_limit(1)? + .fetch_group_page(options(true), None) + .await?; + assert_eq!( + first.rows[0].groups[0].value, + AggregateValue::Text("A".into()) + ); + assert!(first.has_next_page); + assert!( + History::aggregate(&db) + .with_auth(Some(&beta)) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), first.end_cursor.as_ref()) + .await + .is_err() + ); + let next = History::aggregate(&db) + .with_auth(Some(&alpha)) + .group_by(HistoryAggregateField::Event)? + .fetch_group_page(options(true), first.end_cursor.as_ref()) + .await?; + assert_eq!( + next.rows[0].groups[0].value, + AggregateValue::Text("B".into()) + ); + assert!(!next.has_next_page); + for (auth, count, sum) in [ + (Some(&alpha), 2, AggregateValue::Integral(3)), + (Some(&beta), 1, AggregateValue::Integral(100)), + (None, 0, AggregateValue::Null), + ] { + let rows = History::aggregate(&db) + .with_auth(auth) + .count_rows()? + .sum(HistoryAggregateField::Units)? + .fetch() + .await?; + assert_eq!(rows[0].metrics[0].value, AggregateValue::Count(count)); + assert_eq!(rows[0].metrics[1].value, sum); + } + db.pool().close().await; + drop(db); + owned.finish().await +} diff --git a/docs/reference/graphql-orm/typed-aggregates.md b/docs/reference/graphql-orm/typed-aggregates.md index 9e1b56bb..f1ff75d0 100644 --- a/docs/reference/graphql-orm/typed-aggregates.md +++ b/docs/reference/graphql-orm/typed-aggregates.md @@ -3,7 +3,7 @@ title: Typed grouped aggregates kind: reference status: active owner: graphql-orm-maintainers -last_reviewed: 2026-10-01 +last_reviewed: 2026-10-02 review_by: 2027-02-01 supersedes: [] --- @@ -72,10 +72,21 @@ Generated filters must be completely SQL-renderable. PostgreSQL authenticated execution keeps the supplied `DbAuthContext`, so transaction-local RLS is applied before aggregation. -An application-side `RowPolicy` cannot safely inspect rows after aggregation; -ordinary `fetch` therefore retains its rejection of that configuration. Move the restriction into a -typed SQL filter or database RLS instead of aggregating unauthorized rows and -filtering the result. +From ORM/macros 0.38.0, both ordinary `fetch` (including generated GraphQL +aggregate resolvers) and `fetch_group_page` accept an installed `RowPolicy` only +when its current decision is `ReadVisibility::Unrestricted` or `Complete`. +Complete SQL predicates are intersected with caller filters before aggregation, +including ungrouped COUNT/SUM and empty-source aggregates. `CallbackOnly` and +`Prefilter` are rejected before the aggregate SELECT: no post-aggregate callback +can recover correct counts or groups. A caller filter never replaces policy. +The host must accurately declare completeness and reevaluate current identity, +ownership and permissions on every request. + +Generated aggregate enums carry their entity TypeId to validate predicates. +Handwritten `TypedAggregateField` implementations remain source compatible; +complete visibility requires implementing the optional hidden `entity_type_id` +hook with the associated entity's TypeId. Unrestricted reads need no token. +No group, metric or filter field policy is bypassed by complete visibility. ## Opt-in GraphQL aggregate root @@ -122,14 +133,13 @@ them into SQL. Logical backup rows retain the exact Decimal value together with its definition, allowing SQLite and PostgreSQL exports to restore without rounding or treating a native numeric as text. -## Complete SQLite text-group pages +## Complete SQLite and PostgreSQL text-group pages -ORM/macros 0.35.0 adds `fetch_group_page` to the existing generated aggregate -builder. Its initial profile supports exactly one SQLite TEXT-affinity grouping -field, optional existing metrics, and ascending `Binary` or `SqliteNoCase` -comparison. Other scalar families, multi-key groups and other backends reject -before the aggregate query; ordinary `fetch` and GraphQL aggregate SDL/cursors -remain unchanged. Repository-only entities remain private and need no direct +ORM/macros 0.35.0 introduced `fetch_group_page`; 0.38.0 adds PostgreSQL +execution. The API remains supported for complete event-name enumeration, with +exactly one text grouping field, optional existing metrics, and ascending `Binary` +or `SqliteNoCase` comparison. Other scalar families, multi-key groups and MSSQL +reject before pool I/O. Existing GraphQL SDL and cursor wire shapes are unchanged. Repository-only entities remain private and need no direct async-graphql dependency. ```rust,ignore @@ -139,7 +149,7 @@ let page = PrivateEvent::aggregate(&database) ..Default::default() }) .group_by(PrivateEventAggregateField::Event)? - .group_limit(37)? + .group_limit(100)? .fetch_group_page(AggregateGroupPageOptions { order: AggregateGroupOrder::SqliteNoCase, exclude_blank: true, @@ -156,7 +166,7 @@ Use existing `PaginationConfig::legacy()` or an explicit 1,000-row maximum only when that larger bound is intended. A complete set larger than a single-page cap does not require increasing that cap or loading the ledger into Rust. -Group equality remains the original column's native SQL equality. Ordering does +On SQLite, group equality remains the original column's native SQL equality. Ordering does not case-fold or trim group identities. For deterministic representative values, paging selects the BINARY minimum original text in each native group. On ordinary BINARY text storage that is the exact original distinct string. NOCASE storage @@ -167,19 +177,52 @@ tie-breaker; SQL NULL sorts first. `exclude_blank` applies native SQL are excluded, padded nonblank strings are retained unchanged, and tabs/newlines are not removed by SQLite's default TRIM. +PostgreSQL pages group the original TEXT/VARCHAR value under explicit `C` +comparison, even if the source column has a locale-dependent or nondeterministic +collation. Case-distinct original strings therefore remain separate groups; +there is no locale case conversion or identity trimming. `Binary` uses byte +ordering. The historical `SqliteNoCase` spelling also works on PostgreSQL: it +translates only ASCII A–Z to a–z, orders that expression under `C`, then uses +the original value under `C` as tie-breaker. Unicode is retained without folding +or normalization. On UTF-8 databases this matches SQLite's ASCII NOCASE plus +BINARY ordering. PostgreSQL NULL sorts first; default TRIM removes ordinary +spaces only, matching blank exclusion on SQLite. PostgreSQL fixed-width CHAR, +citext/custom storage, numeric and other nontext declarations are not supported +page profiles; TEXT and VARCHAR are. Other database encodings are not a Unicode +parity claim. [PostgreSQL collation documentation](https://www.postgresql.org/docs/17/collation.html) +and [ASCII translation](https://www.postgresql.org/docs/17/functions-string.html) +describe the primitives; no application SQL fragment is required. + +| Provider | Group pages | Ordinary aggregates with complete SQL visibility | +| --- | --- | --- | +| SQLite | BINARY/native equality; Binary/SqliteNoCase ordering | Supported | +| PostgreSQL | Exact text identity; C/ASCII-folded C ordering | Supported; DbAuthContext/RLS retained | +| MSSQL | Explicitly rejected before pool I/O | Existing COUNT/MIN/MAX/SUM renderers supported | + +MSSQL page support remains separate: native text equality, trailing-space rules, +collation, conversion width and provider version require their own faithful +boundary contract. SQLite/PostgreSQL semantics are not silently substituted. +No new provider is implied by portable ordinary aggregates. + Continuation uses HAVING over the same grouping representative and comparisons as ordering. It never filters source rows with a group cursor, which could truncate metrics or split a group. Metrics include every authorized source row in each returned group before the group limit. The database computes groups on each request; this API does not promise a cross-request snapshot under concurrent -ledger writes. +ledger writes. Separate page calls use separate reads/transactions (PostgreSQL's +ordinary statement snapshots), not one pinned repeatable-read snapshot. Insertions +before a consumed boundary may be missed, insertions after it may appear, deletion +or renaming may remove/move groups, and metrics may change between requests. +Stable source rows yield complete enumeration without duplicate/skipped groups; +refresh from a null cursor for a refreshed dropdown. A cursor is not a data revision +or durable replay token. Concurrent authorization changes are always rechecked. Every request rechecks entity and group/metric/filter field authority. The page path permits current `ReadVisibility::Complete` SQL predicates or explicit `Unrestricted`; callback-only or prefilter/residual policies fail closed because post-aggregate row callbacks cannot produce correct groups/counts. Ordinary -aggregate `fetch` keeps its previous row-policy contract. SQL visibility is -applied before grouping and pagination, and contributes to cursor identity. +aggregate `fetch` uses the same complete-visibility contract from 0.38.0. SQL +visibility is applied before grouping and pagination, and contributes to cursor identity. `AggregateGroupCursor::to_json`/`from_json` provide a strict framework-neutral boundary for host-owned opaque envelopes. The wire object has exactly @@ -200,8 +243,9 @@ authority or provide encryption. Protect the complete envelope when the host requires confidentiality; existing static/runtime cursor formats are unchanged. The [standalone private consumer example](../../../crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/examples/complete_events.rs) -creates its own disposable SQLite database through ORM migrations/typed inserts -and collects 225 original event types in 37-group pages without application query +runs with a disposable SQLite database or the owned PostgreSQL test through ORM +migrations/typed inserts +and collects 1,225 original event types in 100-group pages without application query SQL or public GraphQL roots: ```sh @@ -212,7 +256,16 @@ cargo run --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate cover native identities, SQL NULL/blank/padding, literal characters, more groups than the secure maximum, 1,000 groups plus lookahead, tenant/current policy isolation, denied fields/entities, strict cursors and complete metrics after -continuation. PostgreSQL/MSSQL only compile and reject this new capability before -pool I/O; existing PostgreSQL aggregate behavior is separately executed against -test-owned infrastructure. Computed SQL Server grouping/whole-set totals and +continuation. [Portable execution parity](../../../crates/graphql-orm/tests/portable_group_pages.rs) +covers over 1,000 distinct groups in 100-group and 1,000-group pages, ASCII/case/Unicode +ordering, nullable/blank values, metrics, current tenant policy, ordinary aggregates, +GraphQL field authority, concurrent-write behavior and invalid cursor bindings. +PostgreSQL tests explicitly own disposable loopback containers; MSSQL pages reject +before pool I/O. Run the external PostgreSQL example with: + +```sh +cargo test --manifest-path crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml --locked --no-default-features --features postgres --test portable_groups -- --ignored --test-threads=1 +``` + + Computed SQL Server grouping/whole-set totals and joined/computed record queries remain separate capabilities. diff --git a/docs/reference/workspace-packages.md b/docs/reference/workspace-packages.md index 8fffbffa..d00181d3 100644 --- a/docs/reference/workspace-packages.md +++ b/docs/reference/workspace-packages.md @@ -18,11 +18,11 @@ changes. | Package | Version | Path | Default features | Direct internal dependencies | | --- | --- | --- | --- | --- | -| `graphql-orm` | `0.35.2` | `crates/graphql-orm` | `sqlite` | `graphql-orm-macros`, `graphql-orm-operation-catalog`, `graphql-orm-router-protocol` (dev-only) | +| `graphql-orm` | `0.38.0` | `crates/graphql-orm` | `sqlite` | `graphql-orm-macros`, `graphql-orm-operation-catalog`, `graphql-orm-router-protocol` (dev-only) | | `graphql-orm-ai` | `0.106.1` | `crates/graphql-orm-ai` | `sqlite` | `graphql-orm`, `graphql-orm-ai-tool-profiles`, `graphql-orm-storage` | | `graphql-orm-ai-tool-profiles` | `0.15.0` | `crates/graphql-orm-ai-tool-profiles` | none | `graphql-orm-operation-catalog`, `graphql-orm-router-protocol` (dev-only) | | `graphql-orm-backup` | `0.7.2` | `crates/graphql-orm-backup` | `local` | `graphql-orm` (optional), `graphql-orm-storage` | -| `graphql-orm-macros` | `0.35.2` | `crates/graphql-orm-macros` | `sqlite` | none | +| `graphql-orm-macros` | `0.38.0` | `crates/graphql-orm-macros` | `sqlite` | none | | `graphql-orm-operation-catalog` | `0.4.0` | `crates/graphql-orm-operation-catalog` | none | `graphql-orm-router-protocol` (optional) | | `graphql-orm-router` | `0.5.4` | `crates/graphql-orm-router` | none | `graphql-orm-router-protocol` | | `graphql-orm-router-protocol` | `0.2.1` | `crates/graphql-orm-router-protocol` | none | none | diff --git a/scripts/run-owned-database-lanes.sh b/scripts/run-owned-database-lanes.sh index bd7a5661..801024b0 100755 --- a/scripts/run-owned-database-lanes.sh +++ b/scripts/run-owned-database-lanes.sh @@ -27,7 +27,7 @@ cd "${repository_root}" run_sqlite() { cargo test -p graphql-orm --no-default-features --features sqlite \ - --test grouped_aggregates --locked + --test grouped_aggregates --test complete_group_pages --test portable_group_pages --locked cargo test --manifest-path \ crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml \ --no-default-features --features sqlite --locked @@ -35,10 +35,10 @@ run_sqlite() { run_postgres() { cargo test -p graphql-orm --no-default-features --features postgres \ - --test grouped_aggregates_postgres --locked -- --ignored --test-threads=1 + --test grouped_aggregates_postgres --test portable_group_pages --locked -- --ignored --test-threads=1 cargo test --manifest-path \ crates/graphql-orm/tests/fixtures/repository-aggregate-consumer/Cargo.toml \ - --no-default-features --features postgres --locked --test owned_backend \ + --no-default-features --features postgres --locked --test owned_backend --test portable_groups \ -- --ignored --test-threads=1 } From c7d564ffab8105e67c6f1932cc99a572c347313a Mon Sep 17 00:00:00 2001 From: Toby Martin Date: Fri, 2 Oct 2026 23:36:29 +0000 Subject: [PATCH 2/2] Pin PostgreSQL group comparisons to built-in byte semantics --- crates/graphql-orm/src/graphql/orm/query.rs | 24 +++++++++++-------- .../graphql-orm/tests/portable_group_pages.rs | 16 +++++++++++-- .../reference/graphql-orm/typed-aggregates.md | 10 +++++--- 3 files changed, 35 insertions(+), 15 deletions(-) diff --git a/crates/graphql-orm/src/graphql/orm/query.rs b/crates/graphql-orm/src/graphql/orm/query.rs index f4c72cd2..5b6cc18e 100644 --- a/crates/graphql-orm/src/graphql/orm/query.rs +++ b/crates/graphql-orm/src/graphql/orm/query.rs @@ -1961,7 +1961,7 @@ fn render_grouped_aggregate_query_page( if page.is_some() && dialect == DatabaseBackend::Postgres { quoted_groups = quoted_groups .iter() - .map(|column| format!("CAST({column} AS TEXT) COLLATE \"C\"")) + .map(|column| format!("CAST({column} AS TEXT) COLLATE pg_catalog.\"C\"")) .collect(); } // Preserve native SQLite GROUP BY equality. A deterministic original representative @@ -2060,7 +2060,7 @@ fn render_grouped_aggregate_query_page( fn group_binary_comparison(dialect: DatabaseBackend, value: &str) -> String { match dialect { - DatabaseBackend::Postgres => format!("({value}) COLLATE \"C\""), + DatabaseBackend::Postgres => format!("({value}) COLLATE pg_catalog.\"C\""), _ => format!("{value} COLLATE BINARY"), } } @@ -2068,7 +2068,7 @@ fn group_binary_comparison(dialect: DatabaseBackend, value: &str) -> String { fn group_ascii_comparison(dialect: DatabaseBackend, value: &str) -> String { match dialect { DatabaseBackend::Postgres => format!( - "translate(({value})::text, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') COLLATE \"C\"" + "pg_catalog.translate(({value})::pg_catalog.text, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') COLLATE pg_catalog.\"C\"" ), _ => format!("{value} COLLATE NOCASE"), } @@ -2479,10 +2479,14 @@ where } if options.exclude_blank { filters.push(FilterExpression::trusted_fragment( - format!( - "TRIM({}) <> ?", - B::DIALECT.quote_identifier(field.column_name) - ), + { + let trim = format!("TRIM({})", B::DIALECT.quote_identifier(field.column_name)); + if B::DIALECT == DatabaseBackend::Postgres { + format!("{} <> ?", group_binary_comparison(B::DIALECT, &trim)) + } else { + format!("{trim} <> ?") + } + }, vec![SqlValue::String(String::new())], )); } @@ -4701,12 +4705,12 @@ mod grouped_aggregate_tests { assert!( rendered .sql - .contains("GROUP BY CAST(\"event\" AS TEXT) COLLATE \"C\" HAVING") + .contains("GROUP BY CAST(\"event\" AS TEXT) COLLATE pg_catalog.\"C\" HAVING") ); assert!(!rendered.sql.contains("NOCASE")); assert!(!rendered.sql.contains("lower(")); - assert!(rendered.sql.contains("translate(($2)::text")); - assert!(rendered.sql.contains("translate(($3)::text")); + assert!(rendered.sql.contains("translate(($2)::pg_catalog.text")); + assert!(rendered.sql.contains("translate(($3)::pg_catalog.text")); assert!(rendered.sql.contains(" > $4")); assert_eq!(rendered.values.len(), 4); assert!(rendered.sql.ends_with("LIMIT 101")); diff --git a/crates/graphql-orm/tests/portable_group_pages.rs b/crates/graphql-orm/tests/portable_group_pages.rs index 1a1b1d2b..0a35f0ad 100644 --- a/crates/graphql-orm/tests/portable_group_pages.rs +++ b/crates/graphql-orm/tests/portable_group_pages.rs @@ -684,7 +684,19 @@ async fn postgres_nondeterministic_source_collation_does_not_merge_original_grou let owned = Owned::start().await?; sqlx::query("CREATE COLLATION fixture_case_insensitive (provider = icu, locale = 'und-u-ks-level2', deterministic = false)").execute(owned.db.pool()).await?; sqlx::query("ALTER TABLE portable_history ALTER COLUMN event TYPE TEXT COLLATE fixture_case_insensitive").execute(owned.db.pool()).await?; - for (id, key) in [("a", "A"), ("b", "a"), ("c", "É"), ("d", "é")] { + // A legal host search_path may put a user collation named C ahead of + // pg_catalog. Fixed ordering must still select the built-in byte collation. + sqlx::query("CREATE COLLATION public.\"C\" (provider = icu, locale = 'und-u-ks-level2', deterministic = false)").execute(owned.db.pool()).await?; + sqlx::query("SET search_path TO public, pg_catalog") + .execute(owned.db.pool()) + .await?; + for (id, key) in [ + ("a", "A"), + ("b", "a"), + ("c", "É"), + ("d", "é"), + ("z", "\u{200b}"), + ] { insert(&owned.db, id, "alpha", Some(key), 1).await?; } let mut cursor = None; @@ -705,7 +717,7 @@ async fn postgres_nondeterministic_source_collation_does_not_merge_original_grou } assert_eq!( keys, - ["A", "a", "É", "é"].map(|s| AggregateValue::Text(s.into())) + ["A", "a", "É", "é", "\u{200b}"].map(|s| AggregateValue::Text(s.into())) ); owned.finish().await } diff --git a/docs/reference/graphql-orm/typed-aggregates.md b/docs/reference/graphql-orm/typed-aggregates.md index f1ff75d0..2e3b37c1 100644 --- a/docs/reference/graphql-orm/typed-aggregates.md +++ b/docs/reference/graphql-orm/typed-aggregates.md @@ -178,17 +178,21 @@ are excluded, padded nonblank strings are retained unchanged, and tabs/newlines are not removed by SQLite's default TRIM. PostgreSQL pages group the original TEXT/VARCHAR value under explicit `C` -comparison, even if the source column has a locale-dependent or nondeterministic +comparison (`pg_catalog."C"`), even if the source column has a locale-dependent or nondeterministic collation. Case-distinct original strings therefore remain separate groups; there is no locale case conversion or identity trimming. `Binary` uses byte ordering. The historical `SqliteNoCase` spelling also works on PostgreSQL: it -translates only ASCII A–Z to a–z, orders that expression under `C`, then uses +uses `pg_catalog.translate` for only ASCII A–Z to a–z, orders that expression under +`pg_catalog."C"`, then uses the original value under `C` as tie-breaker. Unicode is retained without folding or normalization. On UTF-8 databases this matches SQLite's ASCII NOCASE plus BINARY ordering. PostgreSQL NULL sorts first; default TRIM removes ordinary spaces only, matching blank exclusion on SQLite. PostgreSQL fixed-width CHAR, citext/custom storage, numeric and other nontext declarations are not supported -page profiles; TEXT and VARCHAR are. Other database encodings are not a Unicode +page profiles; TEXT and VARCHAR are. Built-in collation/function names are qualified, so a host search_path cannot +substitute a user-defined C collation or translate function. Blank exclusion also +uses byte comparison: non-empty Unicode values are retained even when a source +ICU collation regards them as equal to empty. Other database encodings are not a Unicode parity claim. [PostgreSQL collation documentation](https://www.postgresql.org/docs/17/collation.html) and [ASCII translation](https://www.postgresql.org/docs/17/functions-string.html) describe the primitives; no application SQL fragment is required.