apache-zookeeper

zookeeper zknodes traversal to update acl's

空扰寡人 提交于 2019-12-24 11:08:47
问题 Currently, I have zknodes with ACL property world:anyone:crdwa. With new requirement, I need to update ACL to use auth scheme. For that, I need to update all current zknodes with auth scheme ACL too. I am planning to write a python script and do DFS traversal doing so. I want to reach out to learn, if there is an open source library or better way to achieve the same? 回答1: Scripting the traversal in Python (or any language of choice) is a viable solution. Current releases of Apache ZooKeeper

Kafka on Cloudera - test=TOPIC_AUTHORIZATION_FAILED

北城余情 提交于 2019-12-24 10:35:17
问题 We just upgraded from CDH 5.3.6 to 5.10.0, and started getting errors when trying to write to Kafka topics. We have the default settings on everything, no SSL or Kerberos authentication enabled. When use the console producer to write to one of my topics, I get this error: /usr/bin/kafka-console-producer --broker-list=myhost1.dev.com:9092,myhost2.dev.com:9092 --topic test 17/03/06 21:00:57 INFO utils.AppInfoParser: Kafka version : 0.10.0-kafka-2.1.0 17/03/06 21:00:57 INFO utils.AppInfoParser:

how to change log levels of 3rd party library in java

浪子不回头ぞ 提交于 2019-12-24 09:05:31
问题 The console logs is cluttered with logs from 3rd part libraries. For example my project uses kafka and zooker keeper client libraries because of this there are too many logs from them 2018-05-08 10:30:38.250 INFO 2968 --- [0:0:0:0:1:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 2018-05-08 10:30:38.309 INFO 2968 --- [ main] o.a.k.clients.producer.ProducerConfig :

Hbase setup configuration: HMaster is not running

风格不统一 提交于 2019-12-24 09:04:04
问题 I am trying to setup HBase in a fully distributed mode: consisting of 1 master and 2 region servers. I have set HBASE_MANAGES_ZK = true in hbase-env.sh. The hadoop cluster is running on the cluster with following configurations: Master: node-master Regionserver1: node1 Regionserver2: node2 When I am starting HBase, I can see that RegionServers are getting started and HQuorumPeer on master also, but HMaster is not showing. Please find the logs as below: Master hbase-site.xml <configuration>

Zookeeper - upgrade from standalone to quorum

白昼怎懂夜的黑 提交于 2019-12-24 08:39:05
问题 Currently I have a standalone ZK instances used in a test system. But this test system has become production system and i would like to upgrade from 1 ZK instance to 3 without compromising availability of the SolrCloud system that ZK is overseeing. From what i've read upgrading from 3 to 5 and so on is pretty easy using rolling restarts, but haven't found any info on going from standalone (1 instance) to 3. Does anyone have any insight on this (anyone who might have tried it)? Thanks! 回答1: I

Exception while doing hbase scan

心不动则不痛 提交于 2019-12-24 07:59:22
问题 I was trying out hbase spark distributed scan example. My simple code looks like this: public class DistributedHBaseScanToRddDemo { public static void main(String[] args) { JavaSparkContext jsc = getJavaSparkContext("hbasetable1"); Configuration hbaseConf = getHbaseConf(0, "", ""); JavaHBaseContext javaHbaseContext = new JavaHBaseContext(jsc, hbaseConf); Scan scan = new Scan(); scan.setCaching(100); JavaRDD<Tuple2<ImmutableBytesWritable, Result>> javaRdd = javaHbaseContext.hbaseRDD(TableName

WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

自闭症网瘾萝莉.ら 提交于 2019-12-24 07:30:03
问题 I have two nodes that each one has docker with Mesos,marathon,and zookeeper have been installed on it. This is my docker compose file on master node: version: '3.7' services: zookeeper: image: ubuntu_mesos_home_marzieh command: /home/zookeeper-3.4.8/bin/zkServer.sh restart environment: ZOOKEEPER_SERVER_ID: 1 ZOOKEEPER_CLIENT_PORT: 2190 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_INIT_LIMIT: 10 ZOOKEEPER_SYNC_LIMIT: 5 ZOOKEEPER_SERVERS: 150.20.11.133:2888:3888;150.20.11.136:2888:3888 network_mode:

Solr and Zookeeper Configuration

筅森魡賤 提交于 2019-12-24 06:35:55
问题 In Production environment, should Solr be setup on every server possible including the one having Zookeeper? Talking about External Zookeeper Total Servers : 5 Case 1: Solr on all 5 servers. Zookeeper on 3 servers. Case 2: Solr on 2 servers. Zookeeper on 3 servers. Case 3: Solr on 5 servers. Zookeeper on 5 servers. What is the best practice? What are the advantages of using one case over another? I have read that it's better to have Zookeeper in a separate server. 回答1: At any point of time

How change Kafka committed consumer offset with required offset

烂漫一生 提交于 2019-12-24 06:33:31
问题 I have Kafka Stream application. My application is processing the events successfully. How to change Kafka committed consumer offset with required offset to reprocess/skip the events. I tried How to change start offset for topic?. But I got 'Node does not exist:' error. Please help me. 回答1: The question/answer you are referring to is based on an older Kafka version. Since Kafka 0.9, offsets are not committed to ZooKeeper but store in a special Kafka topic called the offset topic (topic name

Are Dockerfiles available for Google's sample images on Google Container Registry?

纵饮孤独 提交于 2019-12-24 06:30:15
问题 I'm using the official stable ZooKeeper Helm chart for Kubernetes which pulls a ZooKeeper Docker image from Google's sample images on Google Container Registry. That ZooKeeper image is available here, however, I can't seem to find any reference to the Dockerfile for how it is built or if its Dockerfile is generated from some other representation (e.g., via Bazel). I'd like to know info like what else is installed on the image, what OS it's based on, etc. In general are Dockerfiles for the