apache-zookeeper

Solr issue: ClusterState says we are the leader, but locally we don't think so

*爱你&永不变心* 提交于 2019-12-07 03:02:42
问题 So today we run into a disturbing solr issue. After a restart of the whole cluster one of the shard stop being able to index/store documents. We had no hint about the issue until we started indexing (querying the server looks fine). The error is: 2014-05-19 18:36:20,707 ERROR o.a.s.u.p.DistributedUpdateProcessor [qtp406017988-19] ClusterState says we are the leader, but locally we don't think so 2014-05-19 18:36:20,709 ERROR o.a.s.c.SolrException [qtp406017988-19] org.apache.solr.common

Zookeeper CLI failing - IOException Packet <len12343123123> is out of range

独自空忆成欢 提交于 2019-12-07 02:56:26
问题 Running zookeeper 3.3.3. I have a znode that I am just trying to list, via the CLI, as in: ls /myznode/subznode This crashes with an IOException in org.apache.ClientCnxn$SendThread.readLength at line 710. Anyone seen this?? Someone suggested that maybe bad data is in the znode. Not sure if, or how... but I cannot delete it either, as it has something in it. 回答1: I was able to work around this by increasing the max size of my listing call. I added the "-Djute.maxbuffer=50111000" to my zkCli.sh

Kafka partitions out of sync on certain nodes

為{幸葍}努か 提交于 2019-12-07 00:42:16
问题 I'm running a Kafka cluster on 3 EC2 instances. Each instance runs kafka (0.11.0.1) and zookeeper (3.4). My topics are configured so that each has 20 partitions and ReplicationFactor of 3. Today I noticed that some partitions refuse to sync to all three nodes. Here's an example: bin/kafka-topics.sh --zookeeper "10.0.0.1:2181,10.0.0.2:2181,10.0.0.3:2181" --describe --topic prod-decline Topic:prod-decline PartitionCount:20 ReplicationFactor:3 Configs: Topic: prod-decline Partition: 0 Leader: 2

How to access a protected znode from ZooKeeper using zkCli?

安稳与你 提交于 2019-12-06 16:06:26
I have created a znode using: zookeeper-0:/opt/zookeeper/bin # ./zkCli.sh create /mynode content digest:user:pass:cdrwa How to access the znode using the zkCli.sh utility now? zookeeper-0:/opt/zookeeper/bin # ./zkCli.sh get /mynode Connecting to localhost:2181 WATCHER:: WatchedEvent state:SyncConnected type:None path:null Authentication is not valid : /mynode zookeeper-0:/opt/zookeeper/bin # The getAcl is showing the following: zookeeper-0:/opt/zookeeper/bin # ./zkCli.sh getAcl /mynode Connecting to localhost:2181 WATCHER:: WatchedEvent state:SyncConnected type:None path:null 'digest,'user

Solr 5.3 & Zookeeper Security Authentication & Authorization

安稳与你 提交于 2019-12-06 14:22:55
问题 There are a few topics and articles on Solr authentication & authorization, but I cannot get it to work (the way I like). I followed these tutorials / information sources: https://cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins and https://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules/ Then I created this security.json and I confirmed it is active in Zookeeper: { "authentication":{ "class":"solr.BasicAuthPlugin", "credentials":{

Connecting to Zookeeper in a Apache Kafka Multi Node cluster

瘦欲@ 提交于 2019-12-06 13:33:35
I followed the following instructions to set up a multi node kafka cluster. Now, how to connect to the zookeeper ? Is it okay to connect to just one zookeeper from the Producer/consumer side in JAVA or is there a way to connect all the zookeeper nodes ? Setting a multi node Apache ZooKeeper cluster On every node of the cluster add the following lines to the file kafka/config/zookeeper.properties server.1=zNode01:2888:3888 server.2=zNode02:2888:3888 server.3=zNode03:2888:3888 #add here more servers if you want initLimit=5 syncLimit=2 On every node of the cluster create a file called myid in the

is kafka reliable when used as a message bus in micro services

跟風遠走 提交于 2019-12-06 13:32:57
I am using kafka as a message bus for Micro Service architecture, hence multiple services listen on a topic for a message. Therefore, the services are highly dependent on the topic to be live. But, there are many instances where I get leader not available , broker not available and leader= - 1 on the topics. Now, I am not sure if I can rely on the kafka topics, as services get interrupted when there are issues on the topics with cause issues in the platform. Can someone throw some light on the reliability and dependability on the topics and can we recover if we can across the above issues. I

Error producing to embedded kafka queue after upgrade from 0.7 to 0.8.1.1

北城余情 提交于 2019-12-06 12:06:25
I haven't been able to find anything that directly handled the problem I'm facing, so I'm posting here. I have JUnit/JBehave tests that spin up an embedded ZooKeeper server, embedded Kafka server, and kafka producers and consumers. After upgrading kafka from 0.7 to 0.8.1.1, I'm encountering the following types of errors: ERROR [kafka-request-handler-5] state.change.logger - Error on broker 1 while processing LeaderAndIsr request correlationId 7 received from controller 1 epoch 1 for partition [topicName,8] java.lang.NullPointerException: null at kafka.log.Log.<init>(Log.scala:60) ~[kafka_2.10

When and why does Curator throw ConnectionLossException?

情到浓时终转凉″ 提交于 2019-12-06 11:44:12
问题 I use Curator 1.2.4 and I keep getting ConnectionLossException when I want to monitor one znode for its children's changes. I then implemented a watcher like this public class CuratorChildWatcherImpl implements CuratorWatcher { private CuratorFramework client; public CuratorChildWatcherImpl(CuratorFramework client) { this.client = client; } @Override public void process(WatchedEvent event) throws Exception { List<String> children=client.getChildren().usingWatcher(this).forPath(event.getPath()

Starting Zookeeper and Kafka servers from Java application

泄露秘密 提交于 2019-12-06 09:16:56
I am using a Java application which start a consumer reading Kafka topics. Every time I need to start the consumer application, I have to start the Zookeeper and Kafka servers with commands in cmd. Is it possible to start/stop them with small Java programs? Thank you. Please see this topic, it's about starting Zookeeper in Java: Best way to start zookeeper server from java program directing to this: Is it possible to start a zookeeper server instance in process, say for unit tests? If you need this for something like testing the I think that the better way is using the Debezium library which