Issues with load testing of the quorum queue in RabbitMQ 4.0.9

[caojiaqiang]

Is your feature request related to a problem? Please describe.

When load testing the quorum queue with broker version 4.0.9 and a message size of 1KB, further load testing did not increase TPS, while there was significant backlog in ra_log_wal. The monitoring graph is shown below. Therefore, I want to confirm whether ra_log_wal is written in a single thread at the broker level.

Describe the solution you'd like

Does multi-threaded writing of ra_log_wal support this?

Describe alternatives you've considered

No response

Additional context

No response

[mkuratczyk]

A single Erlang process always runs on 1 CPU core at at time. Currently, the write-ahead log process (ra_log_wal) is shared by all quorum queues on a given node and with enough traffic across many queues, it will become the bottleneck sooner or later. This process writes data to disk, so a faster disk, especially in terms of fsync() time, may help.

[michaelklishin]

I'd add that the options to consider if faster disks are not an option are:

1. Using one or more streams with a RabbitMQ Stream Protocol client (streams do use Raft for the coordinator part but the disk I/O part is handled by sendfile(2), that is, the kernel
2. Using classic queues where replication is not necessary (they do not use the Raft WAL)