搭建大数据平台系列(2)-zookeeper环境搭建
1.安装步骤 Zookeeper集群一般配置奇数个,在本次测试机是部署到slave1,slave2,slave3这3台机器上。1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 1 .下载zookeeper-3.4 .5 -cdh5.5 .0 .tar.gz到slave1机器的~/bigdataspacce文件夹下2 .解压安装包的命令:[hadoop@slave1 bigdataspace]$ tar -zxvf zookeeper-3.4 .5 -cdh5.5 .0 .tar.gz 3 .解压完成后删除压缩包:[hadoop@slave1 bigdataspace]$ rm zookeeper-3.4 .5 -cdh5.5 .0 .tar.gz 4 .配置zoo.cfg文件 $ cd /home/hadoop/bigdataspace/zookeeper-3.4 .5 -cdh5.5 .0 /conf $ cp zoo_sample.cfg zoo.cfg $ vi zoo.cfg (修改文件中的dataDir配置) dataDir=/data/zookeeper-3.4 .5 -cdh5.5 .0 /var/data dataLogDir=/data/zookeeper-3.4 .5 -cdh5.5 .0 /var/dataLog/ (并在clientPort下面新增如下配置) server.1 =slave1:2888 :3888 server.2 =slave2:2888 :3888 server.3 =slave3:2888 :3888 5 .建立dataDir对应路径的文件夹,并进入该data文件夹下新建一个文件myid: $ mkdir -p /data/zookeeper-3.4 .5 -cdh5.5 .0 /var/dataLog $ mkdir -p /data/zookeeper-3.4 .5 -cdh5.5 .0 /var/data $ cd /data/zookeeper-3.4 .5 -cdh5.5 .0 /var/data $ vi myid (myid文件内容为zoo.cfg中配的server号码,如server.1 则myid文件中只保存1 ,每台机器都配自己对应的号码) $ cat myid 1 $ 6 .以上对zookeeper的配置基本完成,下面使用scp把zookeeper发到各个节点:$ scp -r zookeeper-3.4 .5 -cdh5.5 .0 / hadoop@slave2:~/bigdataspace/ $ scp -r zookeeper-3.4 .5 -cdh5.5 .0 / hadoop@slave3:~/bigdataspace/ 7 .通过scp把myid传到各个节点,并修改其zoo.cfg配置文件对应的server号码(如server.2 =slave1:52888 :53888 则myid文件存入2 ) $ scp -r /data/zookeeper-3.4 .5 -cdh5.5 .0 / hadoop@slave2:/data/ $ scp -r /data/zookeeper-3.4 .5 -cdh5.5 .0 / hadoop@slave3:/data/ (然后到到各个节点上修改/data/zookeeper-3.4 .5 -cdh5.5 .0 /var/data/myid文件),如: [hadoop@slave2 ~]$ vi /data/zookeeper-3.4 .5 -cdh5.5 .0 /var/data/myid 2 [hadoop@slave2 ~]$ 8. zookeeper.out以及log4j日志文件的设置[hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4 .5 -cdh5.5 .0 /conf [hadoop@slave1 conf]$ vi log4j.properties # Define some default values that can be overridden by system properties zookeeper.root.logger=INFO, ROLLINGFILE …… log4j.appender.ROLLINGFILE=org.apache.log4j.DailyRollingFileAppender 查看zkServer.sh脚本,发现运行时会先加载zookeeper-3.4 .5 -cdh5.5 .0 /libexec/zkEnv.sh,不存在会加载zookeeper-3.4 .5 -cdh5.5 .0 /bin/zkEnv.sh [hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4 .5 -cdh5.5 .0 /libexec [hadoop@slave1 libexec]$ vi zkEnv.sh if [ "x${ZOO_LOG_DIR}" = "x" ] then ZOO_LOG_DIR="/data/zookeeper-3.4.5-cdh5.5.0/logs" fi if [ "x${ZOO_LOG4J_PROP}" = "x" ] then ZOO_LOG4J_PROP="INFO,ROLLINGFILE" 也可以把zookeeper-3.4 .5 -cdh5.5 .0 /bin/zkEnv.sh文件的配置修改成上面一样.
1 2 3 9 .启动zookeeper服务: [hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4 .5 -cdh5.5.0 /bin [hadoop@slave1 bin]$ ./ zkServer.sh start
2.验证 1 2 3 4 5 6 7 [hadoop@slave1 bin]$ jps #使用jps命令 25906 Jps 20536 QuorumPeerMain #zookeeper的进程19994 JobHistoryServer19068 NameNode19422 ResourceManager19263 SecondaryNameNode
如上,含有QuorumPeerMain表明安装成功1 2 3 4 5 6 7 8 9 10 11 12 13 [hadoop@master bin]$ ./ zkServer.sh stop #停止zookeeper(每台机器都要执行此命令) [hadoop@master bin]$ ./zkServer.sh status #查看角色状态命令 JMX enabled by default Using config: /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/bin/.. /conf/zoo.cfg Mode: follower (Mode: follower/leader,leader这个角色只有一台机器,是通过zookeeper的选举算法产生) 如果出现如下错误, [hadoop@master bin]$ ./zkServer.sh status JMX enabled by default Using config: /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/bin/.. /conf/zoo.cfg Error contacting service. It is probably not running.极大可能是因为防火墙端口被限制了,可以打开这些被用到的端口 (注意:只启用一台zookeeper也是会 出现这个错误,需要启动2台以上的节点)
1 2 [hadoop@master bin ]$ bin/zkCli.sh -server master :2181