Commit 8ab8342
authored
Implement Substrait Support for
## Which issue does this PR close?
* Closes #16294.
## Rationale for this change
This change adds complete Substrait round‑trip support for `GROUPING SET
CUBE`, allowing logical plans containing cubes to successfully convert
to and from Substrait. Previously, cube handling in the Substrait
producer returned a hard `NotImplemented` error, preventing several
SQLLogicTest cases from running under round‑trip mode.
Supporting `CUBE` brings consistency with existing `ROLLUP` and
`GROUPING SETS` handling, ensures correct logical plan serialization,
and enables successful execution of tests that rely on cube semantics.
### Before
```
❯ cargo test --test sqllogictests -- --substrait-round-trip grouping.slt:52
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.60s
Running bin/sqllogictests.rs (target/debug/deps/sqllogictests-917e139464eeea33)
Completed 1 test files in 0 seconds External error: 1 errors in file /Users/kosiew/GitHub/datafusion/datafusion/sqllogictest/test_files/grouping.slt
1. query failed: DataFusion error: This feature is not implemented: GroupingSet CUBE is not yet supported
```
### After
```
❯ cargo test --test sqllogictests -- --substrait-round-trip grouping.slt:52
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.65s
Running bin/sqllogictests.rs (target/debug/deps/sqllogictests-917e139464eeea33)
Completed 1 test files in 0 seconds
```
## What changes are included in this PR?
* Introduces a shared internal helper `powerset_indices` for efficient
subset generation.
* Refactors `powerset` to use DataFusion error types and removes the
string‑based error.
* Adds a new `powerset_cloned` function for owned‑value subsets needed
in the Substrait adapter.
* Implements full Substrait producer support for `GroupingSet::Cube`
using `powerset_cloned`.
* Updates aggregate Substrait translation to correctly assemble grouping
sets derived from cube expansions.
* Adds a new Substrait round‑trip test case for `GROUP BY CUBE`.
## Are these changes tested?
Yes. A new Substrait round‑trip test (`aggregate_grouping_cube`)
validates the logical plan after translation. Existing
grouping/aggregate tests continue to pass, covering other grouping‑set
variants.
## Are there any user-facing changes?
There are no user‑facing API changes. The behavior of `GROUP BY CUBE` is
now consistent under Substrait round‑trip mode, which may allow
previously failing queries to succeed.
## LLM-generated code disclosure
This PR includes LLM‑generated code and comments. All LLM‑generated
content has been manually reviewed and tested.GROUPING SET CUBE (#18798)1 parent 0871741 commit 8ab8342
File tree
3 files changed
+66
-30
lines changed- datafusion
- expr/src
- substrait
- src/logical_plan/producer/rel
- tests/cases
3 files changed
+66
-30
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
| 37 | + | |
| 38 | + | |
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
69 | 86 | | |
70 | 87 | | |
71 | 88 | | |
| |||
83 | 100 | | |
84 | 101 | | |
85 | 102 | | |
86 | | - | |
| 103 | + | |
87 | 104 | | |
88 | | - | |
| 105 | + | |
89 | 106 | | |
90 | 107 | | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
106 | 111 | | |
107 | 112 | | |
108 | 113 | | |
| |||
207 | 212 | | |
208 | 213 | | |
209 | 214 | | |
210 | | - | |
211 | | - | |
| 215 | + | |
212 | 216 | | |
213 | 217 | | |
214 | 218 | | |
| |||
Lines changed: 21 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
91 | 92 | | |
92 | 93 | | |
93 | 94 | | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
98 | 111 | | |
99 | 112 | | |
100 | 113 | | |
| |||
104 | 117 | | |
105 | 118 | | |
106 | 119 | | |
107 | | - | |
| 120 | + | |
108 | 121 | | |
109 | 122 | | |
110 | 123 | | |
111 | 124 | | |
112 | 125 | | |
113 | | - | |
114 | | - | |
| 126 | + | |
115 | 127 | | |
116 | 128 | | |
117 | 129 | | |
| |||
121 | 133 | | |
122 | 134 | | |
123 | 135 | | |
124 | | - | |
| 136 | + | |
125 | 137 | | |
126 | 138 | | |
127 | 139 | | |
| |||
Lines changed: 20 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
322 | 322 | | |
323 | 323 | | |
324 | 324 | | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
325 | 345 | | |
326 | 346 | | |
327 | 347 | | |
| |||
0 commit comments