安装elasticsearch 、redis、zookeeper 、kafka

前言

[TOC]

以前的笔记，重新整理了一下，都在这里了。
操作系统是Centos 7

安装 elasticsearch

rpm -ivh jdk-8u171-linux-x64.rpm  cd /mnt/data/source wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.2.tar.gz tar zxmf elasticsearch-1.5.2.tar.gz  mv elasticsearch-1.5.2 /mnt/data/apps/

插件 analysis-ik 及 head

analysis-ik

mkdir -p /mnt/data/apps/elasticsearch-1.5.2/plugins/analysis-ik  wget https://github.com/medcl/elasticsearch-analysis-ik/archive/v1.2.6.zip -O analysis-ik.zip  unzip analysis-ik.zip  cd elasticsearch-analysis-ik-1.2.6  mvn clean mvn compile mvn package cp target/elasticsearch-analysis-ik-1.2.6.jar /mnt/data/apps/elasticsearch-1.5.2/plugins/analysis-ik  cat >>/mnt/data/apps/elasticsearch-1.5.2/config/elasticsearch.yml <<EOF index:   analysis:     analyzer:       ik:           alias: [ik_analyzer]           type: org.elasticsearch.index.analysis.IkAnalyzerProvider       ik_max_word:           type: ik           use_smart: false       ik_smart:           type: ik           use_smart: true EOF  注意：如果你的版本不对应，可能会出现如下错误： {"error":"IndexCreationException[[index] failed to create index]; nested: ElasticsearchIllegalArgumentException[failed to find analyzer type [ik] or tokenizer for [ik_max_word]]; nested: NoClassSettingsException[Failed to load class setting [type] with value [ik]]; nested: ClassNotFoundException[org.elasticsearch.index.analysis.ik.IkAnalyzerProvider]; ","status":400}

elasticsearch-head

./plugin -install mobz/elasticsearch-head/1.x

http://localhost:9200/_plugin/head/

配置服务启动

cat >/usr/lib/systemd/system/elasticsearch.service <<EOF [Unit] Description=The ElasticSearch server After=network.target remote-fs.target nss-lookup.target  [Service] Type=forking ExecStart=/mnt/data/apps/elasticsearch-1.5.2/bin/elasticsearch -d  ExecReload=/bin/kill -s HUP \$MAINPID KillSignal=SIGQUIT TimeoutStopSec=5 KillMode=process PrivateTmp=true  [Install] WantedBy=multi-user.target EOF  systemctl enable elasticsearch systemctl start elasticsearch

redis 安装配置

yum install redis  mkdir -p /mnt/data/apps/redis/data chown -R redis:redis /mnt/data/apps/redis  redis 配置文件(/etc/redis.conf)修改如下几个参数  bind <hostip>   ##绑定服务的IP ，可以被局域网中的其他机器访问  dir /mnt/data/apps/redis/data  ## 数据目录，数据库的写入会在这个目录 ## 1G maxmemory 1073741824   ##   （https://www.cnblogs.com/changbosha/p/5849982.html） maxmemory-policy volatile-lru   ## 内存  systemctl enable redis systemctl start redis 

启动文件

[Unit] Description=Redis persistent key-value database After=network.target  [Service] ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd ExecStop=/usr/libexec/redis-shutdown Type=notify User=redis Group=redis RuntimeDirectory=redis RuntimeDirectoryMode=0755  [Install] WantedBy=multi-user.target 

zookeeper 1台运行3个实例

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.12/zookeeper-3.4.12.tar.gz mkdir -p /mnt/data/apps/zookeeper-data/zk1/data /mnt/data/apps/zookeeper-data/zk2/data /mnt/data/apps/zookeeper-data/zk3/data mkdir -p /mnt/data/apps/zookeeper-data/zk1/logs /mnt/data/apps/zookeeper-data/zk2/logs /mnt/data/apps/zookeeper-data/zk3/logs cd /mnt/data/apps/zookeeper-3.4.12/conf  cat > zoo1.conf <<EOF tickTime=2000 initLimit=10 syncLimit=5 dataDir=/mnt/data/apps/zookeeper-data/zk1/data dataLogDir=/mnt/data/apps/zookeeper-data/zk1/logs clientPort=2181  server.1=localhost:2287:3387 server.2=localhost:2288:3388 server.3=localhost:2289:3389 EOF  cat > zoo2.conf <<EOF tickTime=2000 initLimit=10 syncLimit=5 dataDir=/mnt/data/apps/zookeeper-data/zk2/data dataLogDir=/mnt/data/apps/zookeeper-data/zk2/logs clientPort=2182  server.1=localhost:2287:3387 server.2=localhost:2288:3388 server.3=localhost:2289:3389 EOF  cat > zoo3.conf <<EOF tickTime=2000 initLimit=10 syncLimit=5 dataDir=/mnt/data/apps/zookeeper-data/zk3/data dataLogDir=/mnt/data/apps/zookeeper-data/zk3/logs clientPort=2181  server.1=localhost:2287:3387 server.2=localhost:2288:3388 server.3=localhost:2289:3389 EOF  echo "1" >/mnt/data/apps/zookeeper-data/zk1/data/myid echo "2" >/mnt/data/apps/zookeeper-data/zk2/data/myid echo "3" >/mnt/data/apps/zookeeper-data/zk3/data/myid  ./bin/zkServer.sh start conf/zoo1.conf  ./bin/zkServer.sh start conf/zoo2.conf  ./bin/zkServer.sh start conf/zoo3.conf 

kafka

配置参数见https://blog.csdn.net/lizhitao/article/details/25667831)

wget https://archive.apache.org/dist/kafka/0.9.0.1/kafka_2.10-0.9.0.1.tgz  tar zxmf kafka_2.10-0.9.0.1.tgz -C /mnt/data/apps/  ## 配置中主意把自己的IP填上  listeners=PLAINTEXT://IP:9092  kafka-server-start.sh  -daemon  config/server.properties  & 

配置服务

cat >/usr/lib/systemd/system/kafka.service <<EOF [Unit] Description=The Kafka server After=network.target remote-fs.target nss-lookup.target  [Service] Type=forking ExecStart=/mnt/data/apps/kafka_2.10-0.9.0.1/bin/kafka-server-start.sh -daemon /mnt/data/apps/kafka_2.10-0.9.0.1/config/server.properties ExecStop=/mnt/data/apps/kafka_2.10-0.9.0.1/bin/kafka-server-stop.sh  /mnt/data/apps/kafka_2.10-0.9.0.1/config/server.properties &  [Install] WantedBy=multi-user.target  EOF  systemctl enable kafka systemctl start kafka

问题集锦

• 现象：redis运行一段时间之后就会出现假死的现象

1. 查看日志是否有关overcommit_memory 的报错，若有则修改如下：
   echo 1 > /proc/sys/vm/overcommit_memory
   sysctl -p
2. 检查redis.conf，确认如下参数是否正确
   maxmemory <bytes> ## 默认是无限制，建议为物理内存3/4
   maxmemory-policy allkeys-lru ## 数据清除策略
   maxmemory-samples 3 ##上面策略的取样值以便检查