Skip to content

Commit 404a9ca

Browse files
fqazifqazi
committed
sql/schemachanger: avoid early drop of secondary indexes during PK swap
Previously, the declarative schema changer was dropping secondary indexes too early during a PK swap. This would cause them to be unavailable earlier then expected potentially impacting query performacne for a brief period. This patch adjust the order of operations to ensure that when the new primary key is published, the secondary indexes for the old primary key are removed first. Fixes: #153728 Note: This patch also cleans up some of the rules which are not constrained correctly for IsPotentialSecondaryIndexSwap. Release note: None
1 parent 574b651 commit 404a9ca

File tree

232 files changed

+1435
-1282
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

232 files changed

+1435
-1282
lines changed

‎pkg/ccl/schemachangerccl/testdata/end_to_end/add_column_multiple_regional_by_row/add_column_multiple_regional_by_row.explain‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -346,14 +346,14 @@ Schema change plan for ALTER TABLE ‹multiregion_db›.‹public›.‹table_re
346346
│ │ └── ValidateIndex {"IndexID":8,"TableID":108}
347347
│ ├── Stage 16 of 23 in PostCommitPhase
348348
│ │ ├── 14 elements transitioning toward PUBLIC
349-
│ │ │ ├── ABSENT → BACKFILL_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
349+
│ │ │ ├── ABSENT → BACKFILL_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
350350
│ │ │ ├── ABSENT → PUBLIC IndexPartitioning:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+)}
351351
│ │ │ ├── ABSENT → PUBLIC IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 4 (crdb_region), IndexID: 2 (table_regional_by_row_j_key+)}
352352
│ │ │ ├── ABSENT → PUBLIC IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 5 (j+), IndexID: 2 (table_regional_by_row_j_key+)}
353353
│ │ │ ├── ABSENT → PUBLIC IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 1 (k), IndexID: 2 (table_regional_by_row_j_key+)}
354354
│ │ │ ├── ABSENT → PUBLIC IndexData:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+)}
355355
│ │ │ ├── ABSENT → PUBLIC IndexName:{DescID: 108 (table_regional_by_row), Name: "table_regional_by_row_j_key", IndexID: 2 (table_regional_by_row_j_key+)}
356-
│ │ │ ├── ABSENT → BACKFILL_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
356+
│ │ │ ├── ABSENT → BACKFILL_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
357357
│ │ │ ├── ABSENT → PUBLIC IndexPartitioning:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+)}
358358
│ │ │ ├── ABSENT → PUBLIC IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 4 (crdb_region), IndexID: 4 (table_regional_by_row_l_key+)}
359359
│ │ │ ├── ABSENT → PUBLIC IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 6 (l+), IndexID: 4 (table_regional_by_row_l_key+)}
@@ -413,40 +413,40 @@ Schema change plan for ALTER TABLE ‹multiregion_db›.‹public›.‹table_re
413413
│ │ └── UpdateSchemaChangerJob {"RunningStatus":"Pending: Backfil..."}
414414
│ ├── Stage 18 of 23 in PostCommitPhase
415415
│ │ ├── 2 elements transitioning toward PUBLIC
416-
│ │ │ ├── BACKFILL_ONLY → BACKFILLED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
417-
│ │ │ └── BACKFILL_ONLY → BACKFILLED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
416+
│ │ │ ├── BACKFILL_ONLY → BACKFILLED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
417+
│ │ │ └── BACKFILL_ONLY → BACKFILLED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
418418
│ │ └── 2 Backfill operations
419419
│ │ ├── BackfillIndex {"IndexID":2,"SourceIndexID":8,"TableID":108}
420420
│ │ └── BackfillIndex {"IndexID":4,"SourceIndexID":8,"TableID":108}
421421
│ ├── Stage 19 of 23 in PostCommitPhase
422422
│ │ ├── 2 elements transitioning toward PUBLIC
423-
│ │ │ ├── BACKFILLED → DELETE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
424-
│ │ │ └── BACKFILLED → DELETE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
423+
│ │ │ ├── BACKFILLED → DELETE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
424+
│ │ │ └── BACKFILLED → DELETE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
425425
│ │ └── 4 Mutation operations
426426
│ │ ├── MakeBackfillingIndexDeleteOnly {"IndexID":2,"TableID":108}
427427
│ │ ├── MakeBackfillingIndexDeleteOnly {"IndexID":4,"TableID":108}
428428
│ │ ├── SetJobStateOnDescriptor {"DescriptorID":108}
429429
│ │ └── UpdateSchemaChangerJob {"RunningStatus":"Pending: Updatin..."}
430430
│ ├── Stage 20 of 23 in PostCommitPhase
431431
│ │ ├── 2 elements transitioning toward PUBLIC
432-
│ │ │ ├── DELETE_ONLY → MERGE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
433-
│ │ │ └── DELETE_ONLY → MERGE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
432+
│ │ │ ├── DELETE_ONLY → MERGE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
433+
│ │ │ └── DELETE_ONLY → MERGE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
434434
│ │ └── 4 Mutation operations
435435
│ │ ├── MakeBackfilledIndexMerging {"IndexID":2,"TableID":108}
436436
│ │ ├── MakeBackfilledIndexMerging {"IndexID":4,"TableID":108}
437437
│ │ ├── SetJobStateOnDescriptor {"DescriptorID":108}
438438
│ │ └── UpdateSchemaChangerJob {"RunningStatus":"Pending: Merging..."}
439439
│ ├── Stage 21 of 23 in PostCommitPhase
440440
│ │ ├── 2 elements transitioning toward PUBLIC
441-
│ │ │ ├── MERGE_ONLY → MERGED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
442-
│ │ │ └── MERGE_ONLY → MERGED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
441+
│ │ │ ├── MERGE_ONLY → MERGED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
442+
│ │ │ └── MERGE_ONLY → MERGED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
443443
│ │ └── 2 Backfill operations
444444
│ │ ├── MergeIndex {"BackfilledIndexID":2,"TableID":108,"TemporaryIndexID":3}
445445
│ │ └── MergeIndex {"BackfilledIndexID":4,"TableID":108,"TemporaryIndexID":5}
446446
│ ├── Stage 22 of 23 in PostCommitPhase
447447
│ │ ├── 2 elements transitioning toward PUBLIC
448-
│ │ │ ├── MERGED → WRITE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
449-
│ │ │ └── MERGED → WRITE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
448+
│ │ │ ├── MERGED → WRITE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
449+
│ │ │ └── MERGED → WRITE_ONLY SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
450450
│ │ ├── 2 elements transitioning toward TRANSIENT_ABSENT
451451
│ │ │ ├── WRITE_ONLY → TRANSIENT_DELETE_ONLY TemporaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 3, ConstraintID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+)}
452452
│ │ │ └── WRITE_ONLY → TRANSIENT_DELETE_ONLY TemporaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 5, ConstraintID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+)}
@@ -459,16 +459,16 @@ Schema change plan for ALTER TABLE ‹multiregion_db›.‹public›.‹table_re
459459
│ │ └── UpdateSchemaChangerJob {"RunningStatus":"Pending: Validat..."}
460460
│ └── Stage 23 of 23 in PostCommitPhase
461461
│ ├── 2 elements transitioning toward PUBLIC
462-
│ │ ├── WRITE_ONLY → VALIDATED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
463-
│ │ └── WRITE_ONLY → VALIDATED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
462+
│ │ ├── WRITE_ONLY → VALIDATED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
463+
│ │ └── WRITE_ONLY → VALIDATED SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
464464
│ └── 2 Validation operations
465465
│ ├── ValidateIndex {"IndexID":2,"TableID":108}
466466
│ └── ValidateIndex {"IndexID":4,"TableID":108}
467467
└── PostCommitNonRevertiblePhase
468468
├── Stage 1 of 5 in PostCommitNonRevertiblePhase
469469
│ ├── 2 elements transitioning toward PUBLIC
470-
│ │ ├── VALIDATED → PUBLIC SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
471-
│ │ └── VALIDATED → PUBLIC SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0}
470+
│ │ ├── VALIDATED → PUBLIC SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key+), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
471+
│ │ └── VALIDATED → PUBLIC SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key+), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey+), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
472472
│ ├── 20 elements transitioning toward TRANSIENT_ABSENT
473473
│ │ ├── TRANSIENT_DELETE_ONLY → TRANSIENT_ABSENT TemporaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 7, ConstraintID: 7, SourceIndexID: 1 (table_regional_by_row_pkey-)}
474474
│ │ ├── PUBLIC → TRANSIENT_ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 4 (crdb_region), IndexID: 7}

‎pkg/ccl/schemachangerccl/testdata/end_to_end/add_column_multiple_regional_by_row/add_column_multiple_regional_by_row__rollback_17_of_23.explain‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Schema change plan for rolling back ALTER TABLE multiregion_db.public.table_regi
4444
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 5 (j-), IndexID: 7}
4545
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 5 (j-), IndexID: 8 (table_regional_by_row_pkey-)}
4646
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 5 (j-), IndexID: 9}
47-
│ │ ├── BACKFILL_ONLY → ABSENT SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key-), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey-), RecreateSourceIndexID: 0}
47+
│ │ ├── BACKFILL_ONLY → ABSENT SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key-), ConstraintID: 2, TemporaryIndexID: 3, SourceIndexID: 8 (table_regional_by_row_pkey-), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
4848
│ │ ├── PUBLIC → ABSENT IndexPartitioning:{DescID: 108 (table_regional_by_row), IndexID: 2 (table_regional_by_row_j_key-)}
4949
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 4 (crdb_region), IndexID: 2 (table_regional_by_row_j_key-)}
5050
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 5 (j-), IndexID: 2 (table_regional_by_row_j_key-)}
@@ -61,7 +61,7 @@ Schema change plan for rolling back ALTER TABLE multiregion_db.public.table_regi
6161
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 6 (l-), IndexID: 7}
6262
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 6 (l-), IndexID: 8 (table_regional_by_row_pkey-)}
6363
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 6 (l-), IndexID: 9}
64-
│ │ ├── BACKFILL_ONLY → ABSENT SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key-), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey-), RecreateSourceIndexID: 0}
64+
│ │ ├── BACKFILL_ONLY → ABSENT SecondaryIndex:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key-), ConstraintID: 4, TemporaryIndexID: 5, SourceIndexID: 8 (table_regional_by_row_pkey-), RecreateSourceIndexID: 0, RecreateTargetIndexID: 0}
6565
│ │ ├── PUBLIC → ABSENT IndexPartitioning:{DescID: 108 (table_regional_by_row), IndexID: 4 (table_regional_by_row_l_key-)}
6666
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 4 (crdb_region), IndexID: 4 (table_regional_by_row_l_key-)}
6767
│ │ ├── PUBLIC → ABSENT IndexColumn:{DescID: 108 (table_regional_by_row), ColumnID: 6 (l-), IndexID: 4 (table_regional_by_row_l_key-)}

0 commit comments

Comments
 (0)