apache-zookeeper

Weird DNS server causes Hadoop and HBase to malfunction

旧巷老猫 提交于 2019-11-30 14:25:18
问题 I have a network with some weird (as I understand) DNS server which causes Hadoop or HBase to malfunction. It resolves my hostname to some address my machine doesn't know about (i.e. there is no such interface). Hadoop does work if I have following entries in /etc/hosts: 127.0.0.1 localhost 127.0.1.1 myhostname If entry "127.0.1.1 myhostname" is not present uploading file to HDFS fails and complains that it can replicate the file only to 0 datanodes instead of 1. But in this case HBase does

how to integrate cassandra with zookeeper to support transactions

半城伤御伤魂 提交于 2019-11-30 11:44:56
I have a Cassandra cluster and Zookeeper server installed. Now I want to support transactions in cassandra using zookeeper. How do i do that. Zookeeper creates znodes to perform read and write operations and data to and fro goes through znodes in Zookeeper. I want to know that how to support rollback and commit feature in cassandra using Zookeeper. Is there any way by which we can specify cassandra configurations in zookeeper or zookeeper configurations in cassandra. I know cassandra and zookeeper individually how data is read and written but I dont know how to integrate both of them using

Offsets stored in Zookeeper or Kafka?

大兔子大兔子 提交于 2019-11-30 11:28:40
问题 I'm a bit confused about where offsets are stored when using Kafka and Zookeeper. It seems like offsets in some cases are stored in Zookeeper, in other cases they are stored in Kafka. What determines whether the offset is stored in Kafka or in Zookeeper? And what the pros and cons? NB: Of course I could also store the offset on my own in some different data store but that is not part of the picture for this post. Some more details about my setup: I run these versions: KAFKA_VERSION="0.10.1.0"

Why does zookeeper not use my log4j.properties file log directory

泄露秘密 提交于 2019-11-30 11:16:57
问题 In my zookeeper/conf/log4j.properties file I set the zookeeper.log.dir to $HOME/zklogs When I use zkServer.sh it does not use that directory. Instead it uses the ${ZOO_LOG_DIR} which when I echo it, comes out to "." I don't understand how fix this issue, I don't see the ${ZOO_LOG_DIR} set anywhere. I am not sure how it gets set to "." at all. I also don't know how to launch zookeeper without zkServer.sh. I am noobish at linux too and a little lost on this issue... Does anybody know how I can

What exactly is the zookeeper quorum setting in hbase-site.xml?

放肆的年华 提交于 2019-11-30 11:03:10
What exactly is the zookeeper quorum setting in hbase-site.xml? As described in hbase-default.xml, here's the setting: Comma separated list of servers in the ZooKeeper Quorum. For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". By default this is set to localhost for local and pseudo-distributed modes of operation. For a fully-distributed setup, this should be set to a full list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh this is the list of servers which we will start/stop ZooKeeper on. What this actually does has been answered by Edward J.

Weird DNS server causes Hadoop and HBase to malfunction

不想你离开。 提交于 2019-11-30 10:38:10
I have a network with some weird (as I understand) DNS server which causes Hadoop or HBase to malfunction. It resolves my hostname to some address my machine doesn't know about (i.e. there is no such interface). Hadoop does work if I have following entries in /etc/hosts: 127.0.0.1 localhost 127.0.1.1 myhostname If entry "127.0.1.1 myhostname" is not present uploading file to HDFS fails and complains that it can replicate the file only to 0 datanodes instead of 1. But in this case HBase does not work: creating a table from HBase shell causes NotAllMetaRegionsOnlineException (caused actually by

Why is kafka not creating a topic? bootstrap-server is not a recognized option

萝らか妹 提交于 2019-11-30 08:45:16
I am new to Kafka and trying to create a new topic on my local machine. I am following this link . Here are the steps which i followed: Start zookeeper bin/zookeeper-server-start.sh config/zookeeper.properties Start kafka-server bin/kafka-server-start.sh config/server.properties Create a topic bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test but when creating the topic, i am getting the following error: Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option at joptsimple

How to configure dynamic properties while using spring boot?

被刻印的时光 ゝ 提交于 2019-11-30 08:24:53
I'm planning to use Spring Boot for my assignment. Its a typical server application with connection to database. I know I can use Spring Configuration to externalize my properties e.g. db connection details. But I also have other dynamic properties which needs be updated at runtime. e.g. flippers/feature flags. Certain features of my application needs to be controlled dynamically e.g. imagine a property like app.cool-feature.enable=true and then after a while the same feature would be turned off by app.cool-feature.enable=false Any suggestions what is the best practice around ingesting such

How to recover Zookeeper from java.io.EOFException after a server crash?

淺唱寂寞╮ 提交于 2019-11-30 08:20:46
问题 How to recover from the following error that started happening after a server crash? Zookeeper won’t start and the following message is showing repeatedly on the log. 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] -

Rebalancing issue while reading messages in Kafka

怎甘沉沦 提交于 2019-11-30 06:53:05
I am trying to read messages on Kafka topic, but I am unable to read it. The process gets killed after sometime, without reading any messages. Here is the rebalancing error which I get: [2014-03-21 10:10:53,215] ERROR Error processing message, stopping consumer: (kafka.consumer.ConsoleConsumer$) kafka.common.ConsumerRebalanceFailedException: topic-1395414642817-47bb4df2 can't rebalance after 4 retries at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:428) at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer