Extremely Limited Support for GROUPBY Function

This is a partial response to issue #4282. The actual logic to implement GROUPBY is probably very complicated. And, even worse, Excel has thrown a whole new way of (internally) specifying one of the arguments into the mix. That argument is a function name, expressed not as a mapped integer (as SUBTOTAL does), nor even as a string, but as the unquoted function name prefixed by `_xleta.`. And, unlike its `_xlfn.` and `_xlws.` predecessors, it is difficult to figure out when the new prefix needs to be added, and when it needs to be ignored. I am not even going to attempt that task with this ticket.

So, what does this change do? Like earlier attempts to introduce limited functionality (such as with form controls), it is there so that using GROUPBY can be passed through - you can load a spreadsheet that contains it, and save it to a new spreadsheet, and the function and its results are preserved. Some cautionary notes. Dynamic arrays must be enabled (the function makes no sense without doing that). Changing any of the inputs used in the function may result in internal inconsistencies between PhpSpreadsheet and Excel; this is especially so if the dimensions of the returned array change as a result of changes to the input data. The programmer can avoid some of these problems by changing the formulatAttributes of the cell where the function is used; this may be difficult to do in practice. Oh, yes, using the GROUPBY cell as an argument in another formula will probably lead to problems. Finally, I confess that part of this solution looks awfully kludgey to me.

With its limitations and those cautions, is it worth proceeding with this change? My gut feel is that it is more useful to proceed than not. However, I will give others the opportunity to weigh in. I will wait at least a couple of weeks into the new year before proceeding with this.

Author: oleibman

docs/references/function-list-by-category.md

@@ -245,6 +245,7 @@ COLUMNS                  | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\RowCo
 FILTER                   | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\Filter::filter
 FORMULATEXT              | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\Formula::text
 GETPIVOTDATA             | **Not yet Implemented**
+GROUPBY                  | **Not yet Implemented**
 HLOOKUP                  | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\HLookup::lookup
 HYPERLINK                | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\Hyperlink::set
 INDEX                    | \PhpOffice\PhpSpreadsheet\Calculation\LookupRef\Matrix::index

docs/references/function-list-by-name.md

@@ -239,6 +239,7 @@ GCD                      | CATEGORY_MATH_AND_TRIG         | \PhpOffice\PhpSpread
 GEOMEAN                  | CATEGORY_STATISTICAL           | \PhpOffice\PhpSpreadsheet\Calculation\Statistical\Averages\Mean::geometric
 GESTEP                   | CATEGORY_ENGINEERING           | \PhpOffice\PhpSpreadsheet\Calculation\Engineering\Compare::GESTEP
 GETPIVOTDATA             | CATEGORY_LOOKUP_AND_REFERENCE  | **Not yet Implemented**
+GROUPBY                  | CATEGORY_LOOKUP_AND_REFERENCE  | **Not yet Implemented**
 GROWTH                   | CATEGORY_STATISTICAL           | \PhpOffice\PhpSpreadsheet\Calculation\Statistical\Trends::GROWTH
 
 ## H

src/PhpSpreadsheet/Calculation/Calculation.php

@@ -1256,6 +1256,11 @@ public static function getExcelConstants(string $key): bool|null
             'functionCall' => [Functions::class, 'DUMMY'],
             'argumentCount' => '2+',
         ],
+        'GROUPBY' => [
+            'category' => Category::CATEGORY_LOOKUP_AND_REFERENCE,
+            'functionCall' => [Functions::class, 'DUMMY'],
+            'argumentCount' => '3-7',
+        ],
         'GROWTH' => [
             'category' => Category::CATEGORY_STATISTICAL,
             'functionCall' => [Statistical\Trends::class, 'GROWTH'],
@@ -4601,7 +4606,7 @@ private static function dataTestReference(array &$operandData): mixed
     private static int $matchIndex10 = 10;
 
     /**
-     * @return array<int, mixed>|false
+     * @return array<int, mixed>|false|string
      */
     private function processTokenStack(mixed $tokens, ?string $cellID = null, ?Cell $cell = null)
     {
@@ -5182,6 +5187,9 @@ private function processTokenStack(mixed $tokens, ?string $cellID = null, ?Cell
                 } elseif (preg_match('/^' . self::CALCULATION_REGEXP_DEFINEDNAME . '$/miu', $token, $matches)) {
                     // if the token is a named range or formula, evaluate it and push the result onto the stack
                     $definedName = $matches[6];
+                    if (str_starts_with($definedName, '_xleta')) {
+                        return Functions::NOT_YET_IMPLEMENTED;
+                    }
                     if ($cell === null || $pCellWorksheet === null) {
                         return $this->raiseFormulaError("undefined name '$token'");
                     }
@@ -5214,6 +5222,7 @@ private function processTokenStack(mixed $tokens, ?string $cellID = null, ?Cell
                     }
 
                     $result = $this->evaluateDefinedName($cell, $namedRange, $pCellWorksheet, $stack, $specifiedWorksheet !== '');
+
                     if (isset($storeKey)) {
                         $branchStore[$storeKey] = $result;
                     }

src/PhpSpreadsheet/Worksheet/Worksheet.php

@@ -49,6 +49,8 @@ class Worksheet
     public const MERGE_CELL_CONTENT_HIDE = 'hide';
     public const MERGE_CELL_CONTENT_MERGE = 'merge';
 
+    public const FUNCTION_LIKE_GROUPBY = '/\\b(groupby|_xleta)\\b/i'; // weird new syntax
+
     protected const SHEET_NAME_REQUIRES_NO_QUOTES = '/^[_\p{L}][_\p{L}\p{N}]*$/mui';
 
     /**
@@ -3701,7 +3703,9 @@ public function calculateArrays(bool $preCalculateFormulas = true): void
             $keys = $this->cellCollection->getCoordinates();
             foreach ($keys as $key) {
                 if ($this->getCell($key)->getDataType() === DataType::TYPE_FORMULA) {
-                    $this->getCell($key)->getCalculatedValue();
+                    if (preg_match(self::FUNCTION_LIKE_GROUPBY, $this->getCell($key)->getValue()) !== 1) {
+                        $this->getCell($key)->getCalculatedValue();
+                    }
                 }
             }
         }

src/PhpSpreadsheet/Writer/Xlsx/FunctionPrefix.php

@@ -142,6 +142,7 @@ class FunctionPrefix
         . '|drop'
         . '|expand'
         . '|filter'
+        . '|groupby'
         . '|hstack'
         . '|isomitted'
         . '|lambda'

src/PhpSpreadsheet/Writer/Xlsx/Worksheet.php

@@ -1578,7 +1578,11 @@ private function writeCell(XMLWriter $objWriter, PhpspreadsheetWorksheet $worksh
         $mappedType = $pCell->getDataType();
         if ($mappedType === DataType::TYPE_FORMULA) {
             if ($this->useDynamicArrays) {
-                $tempCalc = $pCell->getCalculatedValue();
+                if (preg_match(PhpspreadsheetWorksheet::FUNCTION_LIKE_GROUPBY, $cellValue) === 1) {
+                    $tempCalc = [];
+                } else {
+                    $tempCalc = $pCell->getCalculatedValue();
+                }
                 if (is_array($tempCalc)) {
                     $objWriter->writeAttribute('cm', '1');
                 }

tests/PhpSpreadsheetTests/Reader/Xlsx/GroupByLimitedTest.php

@@ -0,0 +1,34 @@
+<?php
+
+declare(strict_types=1);
+
+namespace PhpOffice\PhpSpreadsheetTests\Reader\Xlsx;
+
+use PhpOffice\PhpSpreadsheet\Reader\Xlsx;
+use PhpOffice\PhpSpreadsheetTests\Functional\AbstractFunctional;
+
+class GroupByLimitedTest extends AbstractFunctional
+{
+    private static string $testbook = 'tests/data/Reader/XLSX/excel-groupby-one.xlsx';
+
+    public function testRowBreaks(): void
+    {
+        $reader = new Xlsx();
+        $spreadsheet = $reader->load(self::$testbook);
+        $reloadedSpreadsheet = $this->writeAndReload($spreadsheet, 'Xlsx');
+        $spreadsheet->disconnectWorksheets();
+        $reloadedSheet = $reloadedSpreadsheet->getActiveSheet();
+        self::assertSame(['t' => 'array', 'ref' => 'E3:F7'], $reloadedSheet->getCell('E3')->getFormulaAttributes());
+        $group = $reloadedSheet->rangeToArray('E3:F8');
+        $expected = [
+            ['Design', '$505,000 '],
+            ['Development', '$346,000 '],
+            ['Marketing', '$491,000 '],
+            ['Research', '$573,000 '],
+            ['Total', '$1,915,000 '],
+            [null, null],
+        ];
+        self::assertSame($expected, $reloadedSheet->rangeToArray('E3:F8'));
+        $reloadedSpreadsheet->disconnectWorksheets();
+    }
+}