Add sqllogictest coverage for unused UNNEST pruning edge cases#22074
Open
kosiew wants to merge 2 commits intoapache:mainfrom
Open
Add sqllogictest coverage for unused UNNEST pruning edge cases#22074kosiew wants to merge 2 commits intoapache:mainfrom
kosiew wants to merge 2 commits intoapache:mainfrom
Conversation
…d cardinality issues - Added tests to check for unused UNNEST under GROUP BY; confirmed plan includes Unnest / UnnestExec. - Introduced a counterexample for plain SELECT id showcasing cardinality changes. - Added a test for empty/null array cases where rows are dropped but count remains 2.
…dling and cardinality contract
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
This PR adds test-only coverage documenting the current optimization gap around unused
UNNESToutputs.The new tests capture a case where the unnested column becomes duplicate-insensitive under a
GROUP BY, while also documenting counterexamples where removingUNNESTwould incorrectly change row cardinality or null/empty-array semantics. These tests are intended to guide future optimizer work without changing current behavior.What changes are included in this PR?
sqllogictest/test_files/unnest.slt.UNNESToutput underGROUP BY.EXPLAINassertions documenting that the current logical and physical plans still containUnnest/UnnestExec.UNNESTwould change result cardinality.NULLarray semantics to document current select-listUNNESTbehavior.Are these changes tested?
Yes.
This PR adds SQL logic tests in
datafusion/sqllogictest/test_files/unnest.slt, including:UNNESToutput belowGROUP BYEXPLAINplan assertions forUnnestandUnnestExecAre there any user-facing changes?
No. This PR only adds tests and documentation of current behavior; it does not change optimizer behavior or query semantics.
LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed and tested.