-
Notifications
You must be signed in to change notification settings - Fork 28
Fix for mariadb batch operations #214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
I tried running replication using this branch, I'm getting a ton of debug events, and it appears to process the same events over and over again. $ cat binlog/binlog_replicator.log | grep '18926311, 1285'[binlogrepl 2025-11-11 11:12:55,227 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:55,773 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:55,953 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:56,340 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:56,524 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:56,701 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:57,062 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:57,243 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:57,598 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:57,967 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:58,134 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:58,314 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:58,497 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:58,881 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:12:59,635 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:00,522 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:00,754 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:01,464 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:01,622 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:01,794 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:01,967 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:02,172 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:02,354 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:02,680 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:03,036 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:03,672 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:03,856 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:04,046 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:04,650 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:04,992 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:05,170 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:05,356 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:05,541 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:05,920 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:06,465 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:06,648 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)
[binlogrepl 2025-11-11 11:13:06,824 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000028', 18926311, 1285)Afterwards, I tried to run replication again, I'm getting this error, unless I clean the binlog directory [binlogrepl 2025-11-11 11:17:40,027 INFO] state start position: ('mysql-bin.000028', 4, 0)
Traceback (most recent call last):
File "mysql_ch_replicator/.venv/bin/mysql_ch_replicator", line 8, in <module>
sys.exit(main())
~~~~^^
File "mysql_ch_replicator/mysql_ch_replicator/main.py", line 200, in main
run_binlog_replicator(args, config)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "mysql_ch_replicator/mysql_ch_replicator/main.py", line 64, in run_binlog_replicator
binlog_replicator = BinlogReplicator(
settings=config,
)
File "mysql_ch_replicator/mysql_ch_replicator/binlog_replicator.py", line 363, in __init__
log_file, log_pos = self.state.prev_last_seen_transaction
^^^^^^^^^^^^^^^^^
ValueError: too many values to unpack (expected 2) |
|
You should remove existing state and restart replication from scratch. The change is not backward compatible. |
|
I pulled the latest changes, the replication restart works fine now, but the same events are still processed in an infinite loop. |
|
Is there any errors in logs? Could you share some more logs? It's hard to understand what's going on with only this grep.. |
|
Steps to reproduce:
FLUSH LOGS;
PURGE BINARY LOGS BEFORE NOW();
rm -rf binlog
-- you may need to adjust the condition, in my case I tested updating 2k rows
UPDATE sites SET active = 0
WHERE active = 1;And then you should see the same events processed over and over again. Logs[binlogrepl 2025-11-11 12:26:15,801 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.TableMapEvent'>, ('mysql-bin.000029', 427, 1)
[binlogrepl 2025-11-11 12:26:15,801 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 2)
[binlogrepl 2025-11-11 12:26:15,802 DEBUG] event matched ('mysql-bin.000029', 427, 2), test_database, sites
[binlogrepl 2025-11-11 12:26:15,815 DEBUG] store event ('mysql-bin.000029', 427, 2), event type: 1, database: test_database table: sites records: 85
[binlogrepl 2025-11-11 12:26:15,816 DEBUG] store event ('mysql-bin.000029', 427, 2)
[binlogrepl 2025-11-11 12:26:15,818 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 3)
[binlogrepl 2025-11-11 12:26:15,818 DEBUG] event matched ('mysql-bin.000029', 427, 3), test_database, sites
[binlogrepl 2025-11-11 12:26:15,822 DEBUG] store event ('mysql-bin.000029', 427, 3), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,822 DEBUG] store event ('mysql-bin.000029', 427, 3)
[binlogrepl 2025-11-11 12:26:15,822 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 4)
[binlogrepl 2025-11-11 12:26:15,822 DEBUG] event matched ('mysql-bin.000029', 427, 4), test_database, sites
[binlogrepl 2025-11-11 12:26:15,830 DEBUG] store event ('mysql-bin.000029', 427, 4), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,830 DEBUG] store event ('mysql-bin.000029', 427, 4)
[binlogrepl 2025-11-11 12:26:15,830 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 5)
[binlogrepl 2025-11-11 12:26:15,830 DEBUG] event matched ('mysql-bin.000029', 427, 5), test_database, sites
[binlogrepl 2025-11-11 12:26:15,836 DEBUG] store event ('mysql-bin.000029', 427, 5), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,852 DEBUG] store event ('mysql-bin.000029', 427, 5)
[binlogrepl 2025-11-11 12:26:15,854 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 6)
[binlogrepl 2025-11-11 12:26:15,857 DEBUG] event matched ('mysql-bin.000029', 427, 6), test_database, sites
[binlogrepl 2025-11-11 12:26:15,867 DEBUG] store event ('mysql-bin.000029', 427, 6), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,868 DEBUG] store event ('mysql-bin.000029', 427, 6)
[binlogrepl 2025-11-11 12:26:15,868 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 7)
[binlogrepl 2025-11-11 12:26:15,868 DEBUG] event matched ('mysql-bin.000029', 427, 7), test_database, sites
[binlogrepl 2025-11-11 12:26:15,871 DEBUG] store event ('mysql-bin.000029', 427, 7), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,871 DEBUG] store event ('mysql-bin.000029', 427, 7)
[binlogrepl 2025-11-11 12:26:15,871 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 8)
[binlogrepl 2025-11-11 12:26:15,872 DEBUG] event matched ('mysql-bin.000029', 427, 8), test_database, sites
[binlogrepl 2025-11-11 12:26:15,875 DEBUG] store event ('mysql-bin.000029', 427, 8), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,875 DEBUG] store event ('mysql-bin.000029', 427, 8)
[binlogrepl 2025-11-11 12:26:15,875 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 9)
[binlogrepl 2025-11-11 12:26:15,875 DEBUG] event matched ('mysql-bin.000029', 427, 9), test_database, sites
[binlogrepl 2025-11-11 12:26:15,879 DEBUG] store event ('mysql-bin.000029', 427, 9), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,880 DEBUG] store event ('mysql-bin.000029', 427, 9)
[binlogrepl 2025-11-11 12:26:15,880 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 10)
[binlogrepl 2025-11-11 12:26:15,881 DEBUG] event matched ('mysql-bin.000029', 427, 10), test_database, sites
[binlogrepl 2025-11-11 12:26:15,884 DEBUG] store event ('mysql-bin.000029', 427, 10), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,884 DEBUG] store event ('mysql-bin.000029', 427, 10)
[binlogrepl 2025-11-11 12:26:15,885 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 11)
[binlogrepl 2025-11-11 12:26:15,885 DEBUG] event matched ('mysql-bin.000029', 427, 11), test_database, sites
[binlogrepl 2025-11-11 12:26:15,888 DEBUG] store event ('mysql-bin.000029', 427, 11), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,888 DEBUG] store event ('mysql-bin.000029', 427, 11)
[binlogrepl 2025-11-11 12:26:15,888 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 12)
[binlogrepl 2025-11-11 12:26:15,888 DEBUG] event matched ('mysql-bin.000029', 427, 12), test_database, sites
[binlogrepl 2025-11-11 12:26:15,890 DEBUG] store event ('mysql-bin.000029', 427, 12), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,890 DEBUG] store event ('mysql-bin.000029', 427, 12)
[binlogrepl 2025-11-11 12:26:15,891 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 13)
[binlogrepl 2025-11-11 12:26:15,891 DEBUG] event matched ('mysql-bin.000029', 427, 13), test_database, sites
[binlogrepl 2025-11-11 12:26:15,893 DEBUG] store event ('mysql-bin.000029', 427, 13), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:15,893 DEBUG] store event ('mysql-bin.000029', 427, 13)
[binlogrepl 2025-11-11 12:26:15,893 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 14)
[binlogrepl 2025-11-11 12:26:15,893 DEBUG] event matched ('mysql-bin.000029', 427, 14), test_database, sites
[binlogrepl 2025-11-11 12:26:15,906 DEBUG] store event ('mysql-bin.000029', 427, 14), event type: 1, database: test_database table: sites records: 82
[binlogrepl 2025-11-11 12:26:15,906 DEBUG] store event ('mysql-bin.000029', 427, 14)
[binlogrepl 2025-11-11 12:26:15,906 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 15)
[binlogrepl 2025-11-11 12:26:15,906 DEBUG] event matched ('mysql-bin.000029', 427, 15), test_database, sites
[binlogrepl 2025-11-11 12:26:15,910 DEBUG] store event ('mysql-bin.000029', 427, 15), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,910 DEBUG] store event ('mysql-bin.000029', 427, 15)
[binlogrepl 2025-11-11 12:26:15,910 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 16)
[binlogrepl 2025-11-11 12:26:15,910 DEBUG] event matched ('mysql-bin.000029', 427, 16), test_database, sites
[binlogrepl 2025-11-11 12:26:15,913 DEBUG] store event ('mysql-bin.000029', 427, 16), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,913 DEBUG] store event ('mysql-bin.000029', 427, 16)
[binlogrepl 2025-11-11 12:26:15,913 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 17)
[binlogrepl 2025-11-11 12:26:15,913 DEBUG] event matched ('mysql-bin.000029', 427, 17), test_database, sites
[binlogrepl 2025-11-11 12:26:15,916 DEBUG] store event ('mysql-bin.000029', 427, 17), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,917 DEBUG] store event ('mysql-bin.000029', 427, 17)
[binlogrepl 2025-11-11 12:26:15,917 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 18)
[binlogrepl 2025-11-11 12:26:15,917 DEBUG] event matched ('mysql-bin.000029', 427, 18), test_database, sites
[binlogrepl 2025-11-11 12:26:15,919 DEBUG] store event ('mysql-bin.000029', 427, 18), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,919 DEBUG] store event ('mysql-bin.000029', 427, 18)
[binlogrepl 2025-11-11 12:26:15,920 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 19)
[binlogrepl 2025-11-11 12:26:15,920 DEBUG] event matched ('mysql-bin.000029', 427, 19), test_database, sites
[binlogrepl 2025-11-11 12:26:15,922 DEBUG] store event ('mysql-bin.000029', 427, 19), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,922 DEBUG] store event ('mysql-bin.000029', 427, 19)
[binlogrepl 2025-11-11 12:26:15,922 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 20)
[binlogrepl 2025-11-11 12:26:15,922 DEBUG] event matched ('mysql-bin.000029', 427, 20), test_database, sites
[binlogrepl 2025-11-11 12:26:15,925 DEBUG] store event ('mysql-bin.000029', 427, 20), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,925 DEBUG] store event ('mysql-bin.000029', 427, 20)
[binlogrepl 2025-11-11 12:26:15,925 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 21)
[binlogrepl 2025-11-11 12:26:15,925 DEBUG] event matched ('mysql-bin.000029', 427, 21), test_database, sites
[binlogrepl 2025-11-11 12:26:15,927 DEBUG] store event ('mysql-bin.000029', 427, 21), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,927 DEBUG] store event ('mysql-bin.000029', 427, 21)
[binlogrepl 2025-11-11 12:26:15,928 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 22)
[binlogrepl 2025-11-11 12:26:15,928 DEBUG] event matched ('mysql-bin.000029', 427, 22), test_database, sites
[binlogrepl 2025-11-11 12:26:15,930 DEBUG] store event ('mysql-bin.000029', 427, 22), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,930 DEBUG] store event ('mysql-bin.000029', 427, 22)
[binlogrepl 2025-11-11 12:26:15,930 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 23)
[binlogrepl 2025-11-11 12:26:15,930 DEBUG] event matched ('mysql-bin.000029', 427, 23), test_database, sites
[binlogrepl 2025-11-11 12:26:15,933 DEBUG] store event ('mysql-bin.000029', 427, 23), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,933 DEBUG] store event ('mysql-bin.000029', 427, 23)
[binlogrepl 2025-11-11 12:26:15,933 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 24)
[binlogrepl 2025-11-11 12:26:15,933 DEBUG] event matched ('mysql-bin.000029', 427, 24), test_database, sites
[binlogrepl 2025-11-11 12:26:15,935 DEBUG] store event ('mysql-bin.000029', 427, 24), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,935 DEBUG] store event ('mysql-bin.000029', 427, 24)
[binlogrepl 2025-11-11 12:26:15,935 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 25)
[binlogrepl 2025-11-11 12:26:15,935 DEBUG] event matched ('mysql-bin.000029', 427, 25), test_database, sites
[binlogrepl 2025-11-11 12:26:15,938 DEBUG] store event ('mysql-bin.000029', 427, 25), event type: 1, database: test_database table: sites records: 80
[binlogrepl 2025-11-11 12:26:15,938 DEBUG] store event ('mysql-bin.000029', 427, 25)
[binlogrepl 2025-11-11 12:26:15,938 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 26)
[binlogrepl 2025-11-11 12:26:15,938 DEBUG] event matched ('mysql-bin.000029', 427, 26), test_database, sites
[binlogrepl 2025-11-11 12:26:15,939 DEBUG] store event ('mysql-bin.000029', 427, 26), event type: 1, database: test_database table: sites records: 40
[binlogrepl 2025-11-11 12:26:15,939 DEBUG] store event ('mysql-bin.000029', 427, 26)
[binlogrepl 2025-11-11 12:26:15,939 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.XidEvent'>, ('mysql-bin.000029', 201017, 0)
[binlogrepl 2025-11-11 12:26:16,257 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.RotateEvent'>, ('mysql-bin.000029', 4, 0)
[binlogrepl 2025-11-11 12:26:16,258 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.FormatDescriptionEvent'>, ('mysql-bin.000029', 256, 0)
[binlogrepl 2025-11-11 12:26:16,258 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.MariadbGtidListEvent'>, ('mysql-bin.000029', 299, 0)
[binlogrepl 2025-11-11 12:26:16,259 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.MariadbBinLogCheckPointEvent'>, ('mysql-bin.000029', 342, 0)
[binlogrepl 2025-11-11 12:26:16,259 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.MariadbBinLogCheckPointEvent'>, ('mysql-bin.000029', 385, 0)
[binlogrepl 2025-11-11 12:26:16,259 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.event.MariadbGtidEvent'>, ('mysql-bin.000029', 427, 0)
[binlogrepl 2025-11-11 12:26:16,260 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.TableMapEvent'>, ('mysql-bin.000029', 427, 1)
[binlogrepl 2025-11-11 12:26:16,260 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 2)
[binlogrepl 2025-11-11 12:26:16,260 DEBUG] event matched ('mysql-bin.000029', 427, 2), test_database, sites
[binlogrepl 2025-11-11 12:26:16,268 DEBUG] store event ('mysql-bin.000029', 427, 2), event type: 1, database: test_database table: sites records: 85
[binlogrepl 2025-11-11 12:26:16,268 DEBUG] store event ('mysql-bin.000029', 427, 2)
[binlogrepl 2025-11-11 12:26:16,268 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 3)
[binlogrepl 2025-11-11 12:26:16,269 DEBUG] event matched ('mysql-bin.000029', 427, 3), test_database, sites
[binlogrepl 2025-11-11 12:26:16,274 DEBUG] store event ('mysql-bin.000029', 427, 3), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:16,274 DEBUG] store event ('mysql-bin.000029', 427, 3)
[binlogrepl 2025-11-11 12:26:16,274 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 4)
[binlogrepl 2025-11-11 12:26:16,275 DEBUG] event matched ('mysql-bin.000029', 427, 4), test_database, sites
[binlogrepl 2025-11-11 12:26:16,280 DEBUG] store event ('mysql-bin.000029', 427, 4), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:16,280 DEBUG] store event ('mysql-bin.000029', 427, 4)
[binlogrepl 2025-11-11 12:26:16,280 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 5)
[binlogrepl 2025-11-11 12:26:16,280 DEBUG] event matched ('mysql-bin.000029', 427, 5), test_database, sites
[binlogrepl 2025-11-11 12:26:16,284 DEBUG] store event ('mysql-bin.000029', 427, 5), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:16,284 DEBUG] store event ('mysql-bin.000029', 427, 5)
[binlogrepl 2025-11-11 12:26:16,285 DEBUG] received event <class 'mysql_ch_replicator.pymysqlreplication.row_event.UpdateRowsEvent'>, ('mysql-bin.000029', 427, 6)
[binlogrepl 2025-11-11 12:26:16,285 DEBUG] event matched ('mysql-bin.000029', 427, 6), test_database, sites
[binlogrepl 2025-11-11 12:26:16,289 DEBUG] store event ('mysql-bin.000029', 427, 6), event type: 1, database: test_database table: sites records: 83
[binlogrepl 2025-11-11 12:26:16,289 DEBUG] store event ('mysql-bin.000029', 427, 6)I can see only one error that appears a few times in the logs, unfortunately, there are no more details: |
|
It's not reproducing for me but I see from logs where is the issue. I tried to fix it, could you please check if it works? (still need some time to fix all test cases) |
|
Still processes over and over again, even though LogsThis is how it looks in the binlog |
|
Yeah, this is the issue i am still trying to figure out how to fix. This mariadb binlog thing is pretty challenging (
We receive a rotate event from mariadb, and then it start sending same events. Probably it's some old binlog records, not yet quite sure where did this came from, why it sends this RotateEvent and start from scratch. |
Fix for #209