apache-zookeeper

For particular reasons I need to use both - ConsumerGroup (a.k.a. high-level consumer) and SimpleConsumer (a.k.a. low-level consumer) to read from Kafka. For ConsumerGroup I use ZooKeeper-based config and am completely satisfied with it, but SimpleConsumer requires seed brokers to be instantiated. I don't want to keep list of both - ZooKeeper and broker hosts. Thus, I'm looking for a way to automatically discover brokers for a particular topic from ZooKeeper . Because of some indirect information I belief that these data is stored in ZooKeeper under one of the following paths: /brokers/topics/

I'm trying to install Kafka message queue on Windows for testing purposes (not for production). I found this article on how to install Apache Kafka 0.8 on Windows: http://janschulte.wordpress.com/2013/10/13/apache-kafka-0-8-on-windows/ It's a good article, but it's outdated, unfortunately. Does someone know a way to achieve that? Ok, it's finally not complicated :) The only steps are: Download Kafka and uncompress it somewhere nice (let's say C:/Kafka ) Install Cygwin Edit \bin\kafka-run-class.sh and at the end of the file, change exec $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA

Calling org.apache.zookeeper.server.quorum.QuorumPeerMain.main() isn't working. To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use following code to start ZooKeeper in embedded mode. Properties startupProperties = ... QuorumPeerConfig quorumConfiguration = new QuorumPeerConfig(); try { quorumConfiguration.parseProperties(startupProperties); } catch(Exception e) { throw new RuntimeException(e); } zooKeeperServer = new ZooKeeperServerMain(); final ServerConfig configuration = new ServerConfig(); configuration.readFrom(quorumConfiguration); new Thread() { public void

I am trying to understand ZooKeeper, how it works and what it does. Is there any application which is comparable to ZooKeeper? If you know, then how would you describe ZooKeeper to a layman? I have tried apache wiki, zookeeper sourceforge...but I am still not able to relate to it. I just read thru http://zookeeper.sourceforge.net/index.sf.shtml , so aren't there more services like this? Is it as simple as just replicating a server service? Luca Geretti In a nutshell, ZooKeeper helps you build distributed applications. How it works You may describe ZooKeeper as a replicated synchronization

I'm using kafka_2.9.2-0.8.1.1 with zookeeper 3.4.6. Is there a utility that can automatically remove a consumer group from zookeeper? Or can I just remove everything under /consumers/[group_id] in zookeeper? If the latter, is there anything else I'm missing & can this be done with a live system? Update: As of kafka version 2.3.0, there is a new utility: > bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --delete --group my-group Related doc: http://kafka.apache.org/documentation/#basic_ops_consumer_lag See below for more discussion Heejin Currently, as I know, the only way to