apache-zookeeper

How to get All Topics in apache kafka?

☆樱花仙子☆ 提交于 2019-12-11 03:32:37
问题 @RequestMapping(value = "/getTopics",method = RequestMethod.GET) @ResponseBody public Response getAllTopics() { ZkClient zkClient = new ZkClient(ZookeeperProps.zookeeperURL, ZookeeperProps.connectionTimeoutMs, ZookeeperProps.sessionTimeoutMs, ZKStringSerializer$.MODULE$); Seq<String> topics = ZkUtils.getAllTopics(zkClient); scala.collection.Iterator<String> topicIterator = topics.iterator(); String allTopics = ""; while(topicIterator.hasNext()) { allTopics+=topicIterator.next(); allTopics+="

How to get Stats of a Zookeeper node using Curator framework

最后都变了- 提交于 2019-12-11 02:19:35
问题 I am using curator framework in java to interact with ZNodes. How do we get node stats like last_modified time and creation time etc. I could do same in python using kazoo framework. from kazoo.client import KazooClient zk_client = KazooClient(hosts='127.0.0.1:2181') zk_client.start() data, stat = zk_client.get("/my/favorite") Ref. Link: kazoo I tried searching similar support via curator and couldn't get result. Kindly help here. Thanks. 回答1: Curator (note I'm the main author of Curator)

how to turn off zookeeper log info?

三世轮回 提交于 2019-12-11 01:39:25
问题 I am using zookeeper c client library. When I run my program, it will output ZOO_INFO to console. Part of log messages looks like: 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.6 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@716: Client environment:host.name=myhost 2015-03-26 20:08:22,115:15978(0x7f83a1fbc740):ZOO_INFO@log_env@723: Client environment:os.name=Linux 2015-03-26 20:08:22,115:15978

HBase: /hbase/meta-region-server node does not exist

旧巷老猫 提交于 2019-12-11 00:38:23
问题 I have installed cloudera and hdfs, mapreduce, zookeper, hbase on it. 4 nodes with these services (3 zookeeper). All are installed by cloudera wizard and have no configuration issues in cloudera. On connect from java I have got an error: 9:32:23.020 [main-SendThread()] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server /172.20.7.6:2181 09:32:23.020 [main] INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x301abf87 connecting

Not able to send json tweets events to Kafka topic/producer using kafka command line

大城市里の小女人 提交于 2019-12-10 23:21:11
问题 I have created a python script raw_tweets_stream.py to stream twitter data using twitter api. The json data from twitter is pipped to kafka producer using the script below. `python raw_tweets_stream.py | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list localhost:2181 --topic raw_json_tweets` raw_json_tweets is the kafka topic created for these tweets. The python script raw_tweets_stream.py runs just fine but it throws error while sending it to the kafka producer. I am

How does Zookeeper synchronize the clock in the cluster

有些话、适合烂在心里 提交于 2019-12-10 18:46:09
问题 From the Zookeeper docs: ctime The time in milliseconds from epoch when this znode was created. mtime The time in milliseconds from epoch when this znode was last modified. I presume that Zookeeper somehow syncs the (internal) real clock of all the nodes in the ZK cluster in order the time stamps to be consistent regardless of the node where the client connects. Is that so? If yes, how does it work? All I found is: Real time ZooKeeper doesn't use real time, or clock time, at all except to put

Kafka Zookeeper security

前提是你 提交于 2019-12-10 18:06:02
问题 I am using Kafka Version 0.10.2.0. Is there a way to secure communication between Zookeper Client i.e ZkClient and zookeper server with SSL. I found some way to do through SASL but i want it through SSL. 回答1: Zookeeper 3.5 includes SSL support but it is still in alpha so Kafka doesn't yet support it. The highest supported version is 3.4 which only includes sasl. Ref: https://issues.apache.org/jira/browse/ZOOKEEPER-1000 回答2: This task can still be achieved by a simple workaround mentioned in

How to reset Kafka offsets to match tail position?

风流意气都作罢 提交于 2019-12-10 17:54:10
问题 We're using Storm with Kafka and ZooKeeper. We had a situation where we had to delete some topics and recreate them with different names. Our Kafka spouts stayed the same, aside from now reading from the new topic names. However now the spouts are using the offsets from the old topic partitions when trying to read from the new topics. So the tail position of my-topic-name partition 0 will be 500 but the offset will be something like 10000. Is there a way to reset the offset position so it

Solr: Adding cores using a configSet in a SolrCloud setup does not work

混江龙づ霸主 提交于 2019-12-10 12:12:50
问题 I have a SolrCloud (v5.2) setup with 2 different cores [ users, docs ] working properly. Now I need to be able to dynamically add new cores that are configuration-wise identical to the docs core that already exists. I don't want to duplicate the configuration (I tried an it works OK) as I need to actually share the synonyms.txt and some other files that constantly change. I think that I need "configsets" but I cannot get them to work. I uploaded the "template" conf using this command: sh

Copy/Migrate old zookeeper znode/data to new zookeeper

雨燕双飞 提交于 2019-12-10 09:26:37
问题 We have an old zookeeper 3-cluster quorum, with some permanent routing info stored in znodes/data. Now we are setting up another brand new 3-cluster quorum on a different data center, and we want to migrate the routing info to it. What's the best reliable way to do this? Will simply copy the transaction log do? Or better with snapshot? Or if there are some tools to extract data from old znodes and replay creation onto the new ones? Similar question found, but not answered best way to copy