Improve locking iterator performance #1
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue: locking iterator is several times slower than the default
iterator in queries based on secondary keys.
For example:
select COUNT(*) from sbtest1 where k % 10 = 1 for update;
This query ends up iterating over the entire table, and in the end locks
the entire table, but does so by lots of small locks and seeks every
time.
This diff shows that by simply locking the entire table instead we can
improve the performance of the iterator by 50% - it's still slower than
the default implementation, but now it's much closer to it.
This modification of course results in overlocking in queries such as
select COUNT(*) from sbtest1 where k % 10 = 1 for update limit 3000;
where instead of locking only a few thousands records, with this change
we lock the entire table instead.
This change is in this form a proof of concept, for a real patch I
think we could go in two directions:
variable
linear or exponential batches