Kafka Connect can't find connector

Question

I\'m trying to use the Kafka Connect Elasticsearch connector, and am unsuccessful. It is crashing with the following error:

[2018-11-21 14:48:29,096] ERROR S         


        

Answer 1

The plugin path must load JAR files, containing compiled code, not raw Java classes of the source code (src/main/java).

It also needs to be the parent directory of other directories which are containing those plug-ins.

plugin.path=/opt/kafka-connect/plugins/

Where

$ ls - lR /opt/kafka-connect/plugins/
kafka-connect-elasticsearch-x.y.z/
    file1.jar
    file2.jar 
    etc

Ref - Manually installing Community Connectors

The Kafka Connect startup scripts in the Confluent Platform automatically (used to?) read all folders that match share/java/kafka-connect-*, too, so that's one way to go. At least, it will continue doing so, if you include the path to the share/java folder of the Confluent package installation in the plugin path as well

If you are not very familiar with Maven, or even if you are, then you actually cannot just clone the Elasticsearch connector repo and build the master branch; it has prerequisites of first Kafka, then the common Confluent repo first. Otherwise, you must checkout a Git tag like 5.0.1-post that matches a Confluent release.

An even simpler option would be to grab the package using Confluent Hub CLI

And if none of that works, just downloading the Confluent Platform and using the Kafka Connect scripts would be the most easiest. This does not imply you need to use the Kafka or Zookeeper configurations from that