Can the JDBC Kafka Connector pull data from multiple databases?

a 夏天 提交于 2021-02-10 07:17:08

问题


I would love to setup a cluster of JDBC Kafka Connectors and configure them to pull from multiple databases running on the same host.

I've been looking through the Kafka Connect documentation, and it appears that after you configure the JDBC connector that it can only pull data from a single database.

Can anyone confirm this?


回答1:


Depending on the mode you start your workers (standalone or distributed) :

In the standalone mode, you can start multiple jdbc connectors by using :

bin/connect-standalone worker.properties connector1.properties [connector2.properties connector3.properties ...]

Where each connector.properties match one database.

More details here : Running workers

In the distributed mode, first start workers with :

 bin/connect-distributed worker.properties

Then push each configuration with POST http requests like :

$ curl -X POST -H "Content-Type: application/json" --data '{"name": "jdbc-source", "config": {"connector.class":"JdbcSourceConnector", "tasks.max":"1", "connection.url":"jdbc:sqlite:test.db", "topic.prefix":"connect-jdbc-test-", "mode":"bulk" }}' http://worker_host:8083/connectors

Or, to use a file containing the JSON-formatted configuration

$ curl -X POST -H "Content-Type: application/json" --data @config.json http://worker_host:8083/connectors


来源:https://stackoverflow.com/questions/42907982/can-the-jdbc-kafka-connector-pull-data-from-multiple-databases

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!