apache-zookeeper | 易学教程

supervisord stopping child processes

阅读更多关于 supervisord stopping child processes

问题 One of the problems, I face with supervisord is that when I have a command which in turn spawns another process, supervisord is not able to kill it. For example I have a java process which when runs normally is like $ zkServer.sh start-foreground $ ps -eaf | grep zk user 30404 28280 0 09:21 pts/2 00:00:00 bash zkServer.sh start-foreground user 30413 30404 76 09:21 pts/2 00:00:10 java -Dzookeeper.something..something The supervisord config file looks like: [program:zookeeper] command=zkServer

Why do Kafka consumers connect to zookeeper, and producers get metadata from brokers?

阅读更多关于 Why do Kafka consumers connect to zookeeper, and producers get metadata from brokers?

Why is it that consumers connect to zookeeper to retrieve the partition locations? And kafka producers have to connect to one of the brokers to retrieve metadata. My point is, what exactly is the use of zookeeper when every broker already has all the necessary metadata to tell producers the location to send their messages? Couldn't the brokers send this same information to the consumers? I can understand why brokers have the metadata, to not have to make a connection to zookeeper each time a new message is sent to them. Is there a function that zookeeper has that I'm missing? I'm finding it

Is Zookeeper a must for Kafka?

阅读更多关于 Is Zookeeper a must for Kafka?

In Kafka, I would like to use only a single broker, single topic and a single partition having one producer and multiple consumers (each consumer getting its own copy of data from the broker). Given this, I do not want the overhead of using Zookeeper; Can I not just use the broker only? Why is a Zookeeper must? John Petrone Yes, Zookeeper is required for running Kafka. From the Kafka Getting Started documentation: Step 2: Start the server Kafka uses zookeeper so you need to first start a zookeeper server if you don't already have one. You can use the convenience script packaged with kafka to

Why do we need ZooKeeper in the Hadoop stack?

阅读更多关于 Why do we need ZooKeeper in the Hadoop stack?

问题 I am new to Hadoop/ZooKeeper. I cannot understand the purpose of using ZooKeeper with Hadoop, is ZooKeeper writing data in Hadoop? If not, then why we do we use ZooKeeper with Hadoop? 回答1: Hadoop 1.x does not use Zookeeper. HBase does use zookeeper even in Hadoop 1.x installations. Hadoop adopted Zookeeper as well starting with version 2.0. The purpose of Zookeeper is cluster management. This fits with the general philosophy of *nix of using smaller specialized components - so components of

Is there a way to delete all the data from a topic or delete the topic before every run?

阅读更多关于 Is there a way to delete all the data from a topic or delete the topic before every run?

问题 Is there a way to delete all the data from a topic or delete the topic before every run? Can I modify the KafkaConfig.scala file to change the logRetentionHours property? Is there a way the messages gets deleted as soon as the consumer reads it? I am using producers to fetch the data from somewhere and sending the data to a particular topic where a consumer consumes, can I delete all the data from that topic on every run? I want only new data every time in the topic. Is there a way to

SolrCloud 5 and Zookeeper config upload

阅读更多关于 SolrCloud 5 and Zookeeper config upload

问题 I'm in the middle of configuring SolrCloud with Zookeeper but I struggle to load the config on ZK. Here my steps: Configure an ensemble of 3 ZK, I see 1 leader and 2 follower Configure a small cluster of 2 of SolrCloud that is started as followed /bin/solr start -c -z <ip1>:2181,<ip2>:2181,<ip3>:2181 -noprompt Then I tried to load the config on ZK using zkCli.sh: ./bin/zkCli.sh -zkhost <ip1>:2181,<ip2>:2181,<ip3>:2181 -cmd upconfig -confname config1 -confdir /folder/with/schema.xml (it come

Connecting to a remote HBase instance

阅读更多关于 Connecting to a remote HBase instance

问题 I have an HBase-0.98.3-hadoop2 instance running on a ubuntu VM, I could not connect to it (the main thread stuck when creating HTable instance) with the following code: Configuration config = HBaseConfiguration.create(); config.set("hbase.zookeeper.quorum", "192.168.56.101"); HTableInterface usersTable = new HTable(config, "users"); Here is what I got in the console SLF4J: The requested version 1.5.8 by your slf4j binding is not compatible with [1.6, 1.7] SLF4J: See http://www.slf4j.org/codes

Apache Kafka error on windows - Couldnot find or load main class QuorumPeerMain

阅读更多关于 Apache Kafka error on windows - Couldnot find or load main class QuorumPeerMain

I just downloaded Kafka 2.8.0 from Apache website, and I am trying to setup using the instructions given on the website. But when I try to start zookeper server, I am getting below error: Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain My environment is Windows 7 64 bit. I tried to follow below e-mail chain: Apache Email Chain . But still it's having same issue. Can anyone guide me in this? As I am very new to this and couldn't find many information on Google/Apache Kafka email chain. Run these commands from your Kafka root folder: cd bin/windows Then

Kafka SASL zookeeper authentication

阅读更多关于 Kafka SASL zookeeper authentication

I am facing the following error while enabling SASL on Zookeeper and broker authentication. [2017-04-18 15:54:10,476] DEBUG Size of client SASL token: 0 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-18 15:54:10,476] ERROR cnxn.saslServer is null: cnxn object did not initialize its saslServer properly. (org.apache.zookeeper.server. ZooKeeperServer) [2017-04-18 15:54:10,478] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient) [2017-04-18 15:54:10,478] DEBUG Received event: WatchedEvent state:AuthFailed type:None path:null (org

Kafka SASL zookeeper authentication

阅读更多关于 Kafka SASL zookeeper authentication

问题 I am facing the following error while enabling SASL on Zookeeper and broker authentication. [2017-04-18 15:54:10,476] DEBUG Size of client SASL token: 0 (org.apache.zookeeper.server.ZooKeeperServer) [2017-04-18 15:54:10,476] ERROR cnxn.saslServer is null: cnxn object did not initialize its saslServer properly. (org.apache.zookeeper.server. ZooKeeperServer) [2017-04-18 15:54:10,478] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)