Reject DISTINCT window aggregates

[fulghum]

Match PostgreSQL by rejecting DISTINCT window aggregates with SQLSTATE 0A000 after resolving the exact function overload, instead of silently ignoring DISTINCT. Preserve PostgreSQL scalar-function diagnostics, including user-defined and schema-qualified name collisions.

Depends on: go-mysql-server #3732

[github-actions]

	Main	PR
Total	42090	42090
Successful	19600	19601
Failures	22490	22489
Partial Successes1	5463	5463

	Main	PR
Successful	46.5669%	46.5693%
Failures	53.4331%	53.4307%

${\color{lightgreen}Progressions (1)}$

subselect

QUERY: select count(*) from tenk1 t
where (exists(select 1 from tenk1 k where k.unique1 = t.unique2) or ten < 0);

Footnotes

1. These are tests that we're marking as Successful, however they do not match the expected output in some way. This is due to small differences, such as different wording on the error messages, or the column names being incorrect while the data itself is correct. ↩

[coffeegoddd]

@fulghum DOLT

read_tests	from_latency	to_latency	percent_change
covering_index_scan_postgres	2.43	2.43	0.0
groupby_scan_postgres	75.82	81.48	7.47
index_join_postgres	2.22	2.22	0.0
index_join_scan_postgres	1.61	1.61	0.0
index_scan_postgres	467.3	475.79	1.82
oltp_point_select	0.36	0.36	0.0
oltp_read_only	6.32	6.32	0.0
select_random_points	0.7	0.7	0.0
select_random_ranges	1.01	1.01	0.0
table_scan_postgres	475.79	475.79	0.0
types_table_scan_postgres	1170.65	1191.92	1.82

write_tests	from_latency	to_latency	percent_change
oltp_delete_insert_postgres	6.67	6.67	0.0
oltp_insert	3.3	3.3	0.0
oltp_read_write	13.22	13.22	0.0
oltp_update_index	3.55	3.55	0.0
oltp_update_non_index	3.25	3.25	0.0
oltp_write_only	6.91	6.91	0.0
types_delete_insert_postgres	7.17	7.17	0.0

[itoqa]

Commit: 1bd08c5: 12 test cases ran, 12 passed ✅.

Summary

Coverage spans normal aggregate and ordered window calculations, duplicate and null handling, partitioning and frame behavior, function-name resolution, and invalid-query safeguards. It also exercises edge cases involving nested queries, mixed valid and invalid expressions, concurrent requests, and unavailable prepared-statement support, with healthy behavior across the exercised application paths.

Safe to merge — the run found no PR-attributable regressions, new failures, or previously flagged failures, and all exercised behaviors matched expectations. The prepared-statement scenario is an environment capability limitation rather than a product failure and is not a merge blocker.

Tests run by Ito

Result	Severity	Type	Description
✅	—	Aggregate	The DISTINCT aggregate window query was rejected before returning rows with SQLSTATE 0A000 and the expected PostgreSQL error message.
✅	—	Aggregate	The aggregate returned one copy of the repeated value and kept the NULL value, as expected.
✅	—	General	Built-in and user-defined aggregate queries with DISTINCT and OVER were rejected with the expected PostgreSQL error, without returning rows or an internal error.
✅	—	General	The typed NULL query was rejected with SQLSTATE 0A000 and the expected message before any rows were returned.
✅	—	General	Partitioned window queries returned the expected values, including running and previous-row frames.
✅	—	General	Running invalid and ordinary aggregate queries at the same time kept their results separate. Invalid window queries were rejected, while ordinary distinct aggregates returned [1].
✅	—	Rev	The prepared-statement check could not run because this local database build does not support PREPARE or EXECUTE. A direct query still returned the expected DISTINCT-window error, and source review found the validation path in place.
✅	—	Rev	The derived query was rejected before it returned rows, with the expected PostgreSQL error for a DISTINCT window aggregate.
✅	—	Rev	A query with one valid window expression and one invalid DISTINCT window expression was rejected as a whole. It returned SQLSTATE 0A000 with the expected message and produced no result rows.
✅	—	Rev	A user-defined scalar function takes precedence over the built-in aggregate name, and the query returns the expected PostgreSQL error instead of using aggregate behavior.
✅	—	Scalar	The query was rejected with SQLSTATE 42809 and clearly identified abs as a non-aggregate function, as expected.
✅	—	Window	The ordered window query returned ["a"], ["a", "b"], and ["a", "b", "c"] in order.

Tip

Reply with @itoqa to send us feedback on this test run.

[itoqa]

History reset (rebase or force-push detected). Starting test narrative over.

Commit: 0af4c78: 10 test cases ran, 9 passed ✅, 1 additional finding ⚠️.

Summary

Coverage focuses on database query behavior across normal scalar, aggregate, and window operations, along with edge cases involving duplicate values, nulls, overloaded functions, expression rewrites, rejected queries, session recovery, and concurrent requests. The exercised behaviors are broadly healthy, including correct results, precise error classification, isolation between requests, and continued operation after failures.

Safe to merge — the only failure is a medium-severity, pre-existing diagnostic issue unrelated to this PR, with no observed regressions or PR-attributable failures. It is a flag for later because it affects error classification for a specific unsupported query shape but is not a merge blocker for this change.

Tests run by Ito

Result	Severity	Type	Description
✅	—	General	Queries with type-changing expressions returned the correct diagnostic for the final function type. The aggregate returned SQLSTATE 0A000, while the scalar call returned SQLSTATE 42809 and named abs.
✅	—	General	The database rejected the unsupported DISTINCT window query with the expected error. The ordinary aggregate, scalar, window, and rewritten queries that followed all returned the expected results in the same session.
✅	—	General	All 12 concurrent query pairs kept their own error code and message. Aggregate window queries returned the feature-not-supported error, while scalar queries named their own function.
✅	—	Rev	Both integer and numeric versions of the custom aggregate were rejected before producing rows. Each returned SQLSTATE 0A000 with the expected message, so changing the argument type does not bypass the rule.
✅	—	Rev	Counting and summing distinct values worked correctly, including duplicate and NULL values. The query returned count 2 and sum 3 without an error.
✅	—	Rev	A rejected query did not affect the next statement. The following running total completed successfully and returned 2, 5, and 10.
✅	—	Rewrite	Regular scalar, aggregate, and window queries all completed and returned the expected values. The database kept normal behavior while expressions were rewritten during analysis.
✅	—	Scalar	The query was rejected during analysis with SQLSTATE 42809 and the message "DISTINCT specified, but abs is not an aggregate function" before any rows were evaluated.
✅	—	Window	The database rejected the query during analysis with SQLSTATE 0A000 and the expected message. No result rows were returned.
⚠️		Rev	The database rejected the query with SQLSTATE XX000 and the message "function filters are not yet supported". It should reject the DISTINCT window use with SQLSTATE 0A000 and the message "DISTINCT is not implemented for window functions".

Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 FILTER query returns the wrong error

• Severity: Medium
• Description: The database rejected the query with SQLSTATE XX000 and the message "function filters are not yet supported". It should reject the DISTINCT window use with SQLSTATE 0A000 and the message "DISTINCT is not implemented for window functions".
• Impact: Applications that send this query receive the wrong database error code and message, so they cannot reliably handle the unsupported query as expected. The query produces no results, but clients may show the wrong failure or take the wrong recovery path.
• Steps to Reproduce:
  1. Connect to the local PostgreSQL-compatible endpoint.
  2. Create a temporary table with duplicate, NULL, and non-duplicate integer values.
  3. Run SELECT sum(DISTINCT value) FILTER (WHERE value IS NOT NULL) OVER (ORDER BY id) FROM the temporary table.
  4. Inspect the statement error code and message without reading result rows.
• Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
• Code Analysis: The observed result follows directly from the production code. In server/ast/func_expr.go, nodeFuncExpr returns errors.Errorf("function filters are not yet supported") at lines 33-35 whenever node.Filter is non-nil, before it resolves the function, builds the window definition, or performs semantic validation. The PostgreSQL semantic checker in postgres/parser/sem/tree/type_check.go:948-956 first recognizes a window function application and then returns pgcode.FeatureNotSupported (SQLSTATE 0A000) with "DISTINCT is not implemented for window functions" when expr.Type is DistinctFuncType; its FILTER handling is later at lines 965-978. Therefore the FILTER guard makes the required DISTINCT-window diagnostic unreachable for this query. The PR diff changes server/functions/framework/compiled_function.go and related framework code to preserve the DISTINCT-window marker after overload resolution, plus tests and dependency versions, but it does not change server/ast/func_expr.go. The smallest fix is to let FILTER-bearing function expressions reach the existing semantic validation, or to add a narrowly scoped equivalent check in the AST path that preserves the 0A000 DISTINCT-window error before rejecting unsupported FILTER syntax; the implementation should not return the generic XX000 error first.

Evidence Package

• text/REV-1/REV-1-filter-distinct-window.txt — 2 KB

Tip

Reply with @itoqa to send us feedback on this test run.

[zachmu]

LGTM!

[itoqa]

History reset (rebase or force-push detected). Starting test narrative over.

Commit: f85838f: 9 test cases ran, 8 passed ✅, 1 additional finding ⚠️.

Summary

Coverage focused on database query behavior across normal window calculations, malformed and unsupported queries, session recovery after errors, function-name collisions, partitions, ordering, frames, duplicates, and null values. The exercised paths represent happy-path business logic plus edge-case and adversarial validation of error handling and result correctness.

Safe to merge — the sole observed defect is unrelated to this PR and is a medium-severity database behavior issue, while the PR has no regressions, new failures, or previously flagged failures. The unrelated finding should be tracked separately rather than treated as a merge blocker.

Tests run by Ito

Result	Severity	Type	Description
✅	—	General	Malformed window queries returned clear errors, and later valid queries still returned the right errors and rolling totals. The database kept each statement's behavior separate.
✅	—	General	An invalid window query returned a clear error, but the same database session stayed usable. The next window query returned 10, 30, and 5, and a later DISTINCT query returned its expected rejection.
✅	—	General	A schema-qualified function that shares a name with a built-in aggregate kept its own identity after repeated lookups and expression changes. It returned the expected 42809 error, while the built-in aggregate returned 0A000 and the session continued to work.
✅	—	General	Repeated DISTINCT window queries returned the expected PostgreSQL error each time, even when casts, aliases, partitions, and frames changed. A normal rolling-sum query still returned 10, 30, and 5 afterward.
✅	—	General	The ordinary query kept each group, sort order, and bounded frame aligned, including duplicate and NULL values. The matching DISTINCT queries were rejected before returning rows with the expected PostgreSQL error.
✅	—	Distinct	The query is rejected before returning rows, with SQLSTATE 0A000 and the message "DISTINCT is not implemented for window functions".
✅	—	Function	Using DISTINCT with the scalar absolute-value function correctly returned SQLSTATE 42809 and the message that it is not an aggregate function. The session remained usable afterward.
✅	—	Rebuild	The window query returned the expected rolling sums for both groups: 10, 30, and 5. The rows stayed in the requested group and value order.
⚠️		General	The count query returned rows 1, 2, and 0. It should have failed before producing rows with SQLSTATE 0A000 and the message DISTINCT is not implemented for window functions. The same rejection worked for sum(DISTINCT ...) OVER and json_agg(DISTINCT ...) OVER, so this is specific to the count path rather than a client display problem.

Additional Findings Details

These findings are unrelated to the current changes but were observed during testing.

🟡 Count window query returns incorrect rows

• Severity: Medium
• Description: The count query returned rows 1, 2, and 0. It should have failed before producing rows with SQLSTATE 0A000 and the message DISTINCT is not implemented for window functions. The same rejection worked for sum(DISTINCT ...) OVER and json_agg(DISTINCT ...) OVER, so this is specific to the count path rather than a client display problem.
• Impact: Users running a count query with DISTINCT and a window may receive results instead of a clear unsupported-feature error. They could trust those results even though the query does not follow the expected database behavior.
• Steps to Reproduce:
  1. Connect to the local Doltgres PostgreSQL endpoint.
  2. Run SELECT count(DISTINCT v) OVER () FROM (VALUES (1),(1),(NULL)) AS t(v);.
  3. Check the response and whether the statement returns any rows.
  4. Compare it with SELECT sum(DISTINCT v) OVER () FROM (VALUES (1),(1),(NULL)) AS t(v);, which should return SQLSTATE 0A000 with the message DISTINCT is not implemented for window functions.
• Stub / mock content: No stubs, mocks, or bypasses were applied for this test in the recorded run.
• Code Analysis: The PostgreSQL semantic checker in postgres/parser/sem/tree/type_check.go:948-956 performs the generic DISTINCT-over-window check after overload resolution and returns pgcode.FeatureNotSupported with the expected message. The Doltgres change in server/functions/framework/compiled_function.go:76-97 also validates the selected overload and returns the same 0A000 error for aggregate and window overloads. However, the source-analysis evidence identifies a separate special case in .build-agent/go-mysql-server/sql/planbuilder/aggregates.go:348-350: when the parsed function is count and DISTINCT is set, the planner constructs aggregation.NewCountDistinct(args...). That count-specific construction bypasses the normal compiled-function validation path, allowing a window expression to execute and emit rows. The PR diff contains compiled_aggregate_function.go, compiled_function.go, compiled_window_function.go, and testing/go/functions_test.go, but does not contain the dependency planner branch that bypasses validation. The smallest practical fix is to reject DISTINCT count window expressions before or at that CountDistinct construction, using SQLSTATE 0A000 and the existing PostgreSQL-compatible message, or to route that branch through the already-existing validator.

Evidence Package

• text/BF-INTEGRATION-1/BF-INTEGRATION-1-count-window-evidence.txt — 3 KB

Tip

Reply with @itoqa to send us feedback on this test run.