Kafka Connect can't find connector

Question

I\'m trying to use the Kafka Connect Elasticsearch connector, and am unsuccessful. It is crashing with the following error:

[2018-11-21 14:48:29,096] ERROR S         


        

Answer 1

I ran jdbc connector yesterday manually on kafka in docker without confluent platform etc just to learn how those things works underneath. I did not have to build jar on my side or anyhing like this. Hopefully it will be relevant for you - what I did is ( I will skip docker parts howto mount dir with connector etc ):

• download connector from https://www.confluent.io/connector/kafka-connect-jdbc/, unpack zip

• put contents of zip to directory in path configured in properties file ( shown below in 3rd point ) -

  plugin.path=/plugins
  

  so tree looks something like this:

  /plugins/
  └── jdbcconnector
      └──assets
      └──doc
      └──etc
      └──lib
  

  Note the lib dir where are the dependencies are, one of them is kafka-connect-jdbc-5.0.0.jar

• Now you can try to run connector

  ./connect-standalone.sh connect-standalone.properties jdbc-connector-config.properties
  

  connect-standalone.properties are common properties needed for kafka-connect, in my case:

  bootstrap.servers=localhost:9092
  key.converter=org.apache.kafka.connect.json.JsonConverter
  value.converter=org.apache.kafka.connect.json.JsonConverter
  key.converter.schemas.enable=true
  value.converter.schemas.enable=true
  offset.storage.file.filename=/tmp/connect.offsets
  offset.flush.interval.ms=10000
  plugin.path=/plugins
  rest.port=8086
  rest.host.name=127.0.0.1
  

  jdbc-connector-config.properties is more involving, as it's just configuration for this particular connector, you need to dig into connector docs - for jdbc source it is https://docs.confluent.io/current/connect/kafka-connect-jdbc/source-connector/source_config_options.html