-
Notifications
You must be signed in to change notification settings - Fork 719
Commit sst in dedicated threads #1275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: fb-mysql-8.0.28
Are you sure you want to change the base?
Conversation
| } | ||
|
|
||
| void Rdb_sst_info::commit_sst_file(Rdb_sst_file_ordered *sst_file) { | ||
| m_commiting_threads_mutex.lock(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
consider std::lock_guardstd::mutex? we use RDB_MUTEX_LOCK_CHECK macros elsewhere in this file
| close_curr_sst_file(); | ||
| } | ||
|
|
||
| for (auto& thr : m_commiting_threads) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to grab a log here too?
could you please provide some rough calculations about the time savings? |
In our private branch, when |
Rdb_index_merge and Rdb_sst_file_ordered both use cf's comparator, when Rdb_index_merge write to Rdb_sst_file_ordered, the keys should be in cf's increasing order, m_use_stack should be false in this case. |
We removed |
Execute
Rdb_sst_info::commit_sst_filein dedicated threads, this improves performance:Rdb_sst_file_ordered::commitmay use stack to reverse input data, this is time consumingm_sst_file_writer->Finishmay be consuming, at least it need to callfsyncExecute
Rdb_sst_info::commit_sst_filein dedicated threads increase parallelization with mimimal code changes.