Cloudera

Kafka on Cloudera - test=TOPIC_AUTHORIZATION_FAILED

由 北城余情 提交于 2019-12-24 10:35:17
问题 We just upgraded from CDH 5.3.6 to 5.10.0, and started getting errors when trying to write to Kafka topics. We have the default settings on everything, no SSL or Kerberos authentication enabled. When use the console producer to write to one of my topics, I get this error: /usr/bin/kafka-console-producer --broker-list=myhost1.dev.com:9092,myhost2.dev.com:9092 --topic test 17/03/06 21:00:57 INFO utils.AppInfoParser: Kafka version : 0.10.0-kafka-2.1.0 17/03/06 21:00:57 INFO utils.AppInfoParser:

hadoop入门介绍（一）

由 ぃ、小莉子 提交于 2019-12-24 09:07:04
hadoop是什么？（一） hadoop发展史 （二） Hadoop三大发行版本（三） Hadoop三大发行版本：Apache、Cloudera、Hortonworks。 Apache版本最原始（最基础）的版本，对于入门学习最好。 Cloudera在大型互联网企业中用的较多。 Hortonworks文档较好。 Apache Hadoop 官网地址：http://hadoop.apache.org/releases.html 下载地址： https://archive.apache.org/dist/hadoop/common/ Cloudera Hadoop 官网地址：https://www.cloudera.com/downloads/cdh/5-10-0.html 下载地址：http://archive-primary.cloudera.com/cdh5/cdh/5/ （1）2008年成立的Cloudera是最早将Hadoop商用的公司，为合作伙伴提供Hadoop的商用解决方案，主要是包括支持、咨询服务、培训。 （ 2 ） 2009 年 Hadoop 的创始人 Doug Cutting 也加盟 Cloudera 公司 。Cloudera产品主要为CDH，Cloudera Manager，Cloudera Support （3）CDH是Cloudera的Hadoop发行版

cdh3 client interoperable with apache hadoop server 0.20.xx?

由 落花浮王杯 提交于 2019-12-24 08:35:42
问题 we have a hadoop-core-0.20.2-cdh3u1.jar based java client. Would it be safe to work against a server (cluster) running apache distributed 0.20.xx? Or did cloudera break some apache APIs? Thanks 回答1: probably a bad idea. Tried working with a client using hadoop-core version 0.20.203.0 against a server with cdh3u1 and got: org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 61, server = 63) Didn't try the other way

CM搭建

由 試著忘記壹切 提交于 2019-12-24 02:31:14
搭建环境必看 https://blog.csdn.net/qq_35315256/article/details/88989194 环境准备好开始搭建 1、所有节点创建mkdir /opt/cloudera-manager 创建用户 useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 1、上传相关包 2、 所有节点创建 mkdir /opt/cloudera-manager useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 3、解压cm包 tar -zxvf cloudera-manager-el6-cm5.4.3_x86_64.tar.gz -C /opt/cloudera-manager 4、配置 1、vim /opt/cloudera-manager/cm-5.4.3/etc/cloudera-scm-agent/config.ini server_host=node1 # cm server的hosts # 默认 server_port=7182 2、配置CM Server数据库

Cannot Compile WordCount.java

由 不羁的心 提交于 2019-12-24 01:27:12
问题 mark@maestro1:/usr/lib/hadoop/wordcount_classes$ javac -classpath /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/client/hadoop-mapreduce-client-core-2.0.0-cdh4.0.1.jar -d /usr/lib/hadoop/wordcount_classes /usr/lib/hadoop/wordcount_classes/WordCount.java /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.0.1.jar(org/apache/hadoop/fs/Path.class): warning: Cannot find annotation method 'value()' in type 'LimitedPrivate': class file for org.apache.hadoop.classification.InterfaceAudience

Why does start-all.sh from root cause “failed to launch org.apache.spark.deploy.master.Master: JAVA_HOME is not set”?

由 跟風遠走 提交于 2019-12-24 01:09:27
问题 I am trying to execute a Spark application built through Scala IDE through my standalone Spark service running on cloudera quickstart VM 5.3.0. My cloudera account JAVA_HOME is /usr/java/default However, I am facing the below error message while executing the start-all.sh command from cloudera user as below: [cloudera@localhost sbin]$ pwd /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark/sbin [cloudera@localhost sbin]$ ./start-all.sh chown: changing ownership of `/opt/cloudera

Cloudera Manager. Failed to detect Cloudera Manager Server

由 拥有回忆 提交于 2019-12-23 18:24:22
问题 I have two PC's with CentOS 6.5 client86-101.aihs.net 80.94.86.101 client86-103.aihs.net 80.94.86.103 cloudera-manager-server installed on client86-101.aihs.net. I have the problem on detecting Cloudera Manager Server(3rd step on cluster installation.) Issue trace: BEGIN host -t PTR 80.94.86.101 101.86.94.80.in-addr.arpa domain name pointer client86-101.aihs.net. END (0) using client86-101.aihs.net as scm server hostname BEGIN which python END (0) BEGIN python -c 'import socket; import sys; s

VM cloudera - user cloudera and permissions?

由 ∥☆過路亽．° 提交于 2019-12-23 17:11:25
问题 I downloaded and installed the VM Cloudera 4.4 to play with Hadoop. I have already a cluster on a platform for my job, so I know a little how works hadoop. So I think my problem comes from my misunderstanding of linux and his users and group. With Hive : I try to create a hive table with the shell, and it works. I have a table in /user/hive/warehouse/test witch belongs to user cloudera of group cloudera. I have some data files (.txt) in hdfs : /user/cloudera ( user:cloudera and group: hive)

Clouder Manager安装时出现please remove the following file: /etc/cloudera-scm-server/db.properties问题解决（图文详解）

由 左心房为你撑大大i 提交于 2019-12-23 13:04:46
说在前面的话 　　我的机器是总共4台，分别为 ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。（注意啦，以下是针对Ubuntu系统的） 　　在 ubuntucmbigdata1 上执行了 sudo apt-get install cloudera-manager-daemons cloudera-manager-server sudo apt-get install cloudera-manager-daemons cloudera-manager-agent 　　即，ubuntucmbigdata1是既做server，又做agent。 　　在 ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4 上执行了 sudo apt-get install cloudera-manager-daemons cloudera-manager-agent 　　即，ubuntucmbigdata2、3、和4是只做agent。 　　在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上 新建文件 sudo vi /etc/apt/sources.list.d/cloudera

Spark Streaming - java.io.IOException: Lease timeout of 0 seconds expired

由 。_饼干妹妹 提交于 2019-12-23 12:38:52
问题 I have spark streaming application using checkpoint writing on HDFS. Has anyone know the solution? Previously we were using the kinit to specify principal and keytab and got the suggestion to specify these via spark-submit command instead kinit but still this error and cause spark streaming application down. spark-submit --principal sparkuser@HADOOP.ABC.COM --keytab /home/sparkuser/keytab/sparkuser.keytab --name MyStreamingApp --master yarn-cluster --conf "spark.driver.extraJavaOptions=-XX:
订阅 Cloudera