Kafka Connect can't find connector

Question

I\'m trying to use the Kafka Connect Elasticsearch connector, and am unsuccessful. It is crashing with the following error:

[2018-11-21 14:48:29,096] ERROR S         


        

Answer 1

The compiled JAR needs to be available to Kafka Connect. You have a few options here:

1. Use Confluent Platform, which includes the Elasticsearch (and others) pre-built: https://www.confluent.io/download/. There's zip, rpm/deb, Docker images etc available.

2. Build the JAR yourself. This typically involves:

   cd kafka-connect-elasticsearch-5.0.1
   mvn clean package
   

   Then take the resulting kafka-connect-elasticsearch-5.0.1.jar JAR and put it in a path as configured in Kafka Connect with plugin.path.

You can find more info on using Kafka Connect here:

• https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-1/
• https://www.confluent.io/blog/blogthe-simplest-useful-kafka-connect-data-pipeline-in-the-world-or-thereabouts-part-2/
• https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-3/

Disclaimer: I work for Confluent, and wrote the above blog posts.