Efficient ways for whitelisting more tables in Debezium Mysql Connector

问题

Are there any best practices that are followed for whitelisting a new table to the debezium mysql connector ?

We are using debezium mysql connector for our CDC flows and a use case has arised to whitelist more tables to the connector configuration. Here are the version details of the debezium being used and the

{
    "class": "io.debezium.connector.mysql.MySqlConnector",
    "version": "0.8.0.Final",
    "snapshot.mode": "schema_only"
}

There is a debezium ticket https://issues.redhat.com/browse/DBZ-906 which indicates to run the connector in

"snapshot.mode": "schema_only_recover"

mode and to recreate the history topic before restarting the connector. I had a couple of questions regarding this approach

Is this option available for the above mentioned version as well ?

How can I get the updates to the new table made before the table was whitelisted ?

回答1:

first of all, pleas eupgrade Debezium as version 0.8.0 is very old.

Generally there are multiple approaches to solving your issue but if you can afford a pause in streaming than I'd recommend to 1) Stop the current connector 2) Start a new connector with whitelisted new table to execute the snapshot on it 3) When snapshot is completed then start the original connector with updated whitelist table list

See for example https://groups.google.com/forum/?fromgroups=#!msg/debezium/Iw4BgLZ8Mq4/SZBLGwEaAQAJ;context-place=forum/debezium for exact details

If you cannot afford the downtime then please check the experimental approach introduced in DBZ-175

来源：https://stackoverflow.com/questions/62060026/efficient-ways-for-whitelisting-more-tables-in-debezium-mysql-connector