Commit 3cf81a8
Merge #151074
151074: backfill: merge vectors from a temp forward index into a vector index r=mw5h a=mw5h
Previously mutation of values in a vector index during backfill was prohibited, effectively taking the table offline while the index is built. This patch does a few things:
* Re-enables mutations on vector indexes undergoing backfill.
* Changes the temporary index created for vector indexes to capture changes during backfill from a VECTOR index to a FORWARD index.
* Changes the schema of the temporary index so that it is keyed by the index prefix and primary key columns, with the vector as a payload column.
* Refactors the mvcc merger to take a functor argument to merge individual entries, with the existing kv merger as the default.
* Adds a second merger that re-encodes vector index entries read from the temporary index.
Because forward indexes don't record values for tombstone entries, we don't delete vectors from the vector index that are deleted during backfill. Since reading from a vector index requires an index join on the PK, this doesn't create a correctness issue, but it could cause accuracy issues if a large number of vector index entries are deleted during backfill.
Fixes: #144443
Release note (sql change): Tables with vector indexes will no longer be taken offline while the vector index builds.
Co-authored-by: Matt White <matt.white@cockroachlabs.com>File tree
18 files changed
+851
-119
lines changed- pkg/sql
- backfill
- logictest/testdata/logic_test
- opt/optbuilder
- schemachanger
- scbuild/internal/scbuildstmt
- scdeps/sctestdeps
- sctest
- vecindex
- vecencoding
- vecstore
18 files changed
+851
-119
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| 43 | + | |
43 | 44 | | |
44 | 45 | | |
45 | 46 | | |
| |||
0 commit comments