CM5.11.0安装笔记

• 1.配置网络

[root@server1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0 

DEVICE=eth0
HWADDR=00:0C:29:2b:f8:e0
TYPE=Ethernet
UUID=c2618c28-e1b5-46da-9124-2af371aa5df1
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.80.165
NETMASK=255.255.255.0
GATEWAY=192.168.80.2

• 2.安装jdk
  略
• 3.配置hosts

[root@server1 ~]# vi /etc/hosts

192.168.80.165 server1
192.168.80.166 server2
192.168.80.167 server3

• 4.关闭防火墙和selinux

chkconfig iptables off

[root@server1 ~]# vi /etc/selinux/config 
SELINUX=disabled

• 5.ssh免密码

[root@server1 ~]# ssh-keygen -t rsa
每台都要执行

每台机器运行命令
ssh-copy-id server1
ssh-copy-id server2
ssh-copy-id server3

• 6.安装ntp

设置开机启动
yum install –y ntp
chkconfig ntpd on
设置时间同步

• 7.配置安装mysql

[root@server1 mysource]# yum install -y mysql mysql-server mysql-devel

[root@server1 mysource]# chkconfig mysqld on

[root@server1 mysource] # service mysqld start

[root@server1 mysource] # mysql

mysql> use mysql;

mysql> grant all privileges on *.* to 'root'@'%' identified by '111111' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> delete from user where host !='%';

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

• 8 安装依赖包 (3台机器都要装)

yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

• 9.安装cm server和angent

上传cm 安装包 和 cdh的文件到server1的 /usr/java/mysource目录

其他台机器也要创建 mkdir /usr/java/mysource 目录

并且把cm安装包上传到该目录

• 10.建立目录(默认的目录) 三台都要创建 并解压

mkdir /opt/cloudera-manager
tar xzvf cloudera-manager*.tar.gz -C /opt/cloudera-manager

• 11.创建用户 cloudera-scm

[root@server1 mysource]# useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

• 12.配置cm agent

修改文件 /opt/cloudera-manager/cm-5.11.0/etc/cloudera-scm-agent/config.ini
修改: server_host=server1 (指定server是server1 3台都改)

• 13.拷贝mysql的驱动包到 /usr/share/java (名称改为mysql-connector-java.jar,没有文件夹就创建)
• 14.Mysql 创建用户

mysql> grant all on *.* to 'cmmanager'@'%' identified by 'cmmanager' with grant option;
mysql> flush privileges;

• 15.执行脚本(只在server1)

切换目录: [root@server1 java]# cd /opt/cloudera-manager/cm-5.11.0/share/cmf/schema/
执行:
[root@server1 schema]# ./scm_prepare_database.sh mysql cmmanager -h server1 -ucmmanager -pcmmanager --scm-host server1 scm scm scm

• 16.创建parcel目录:(注意如果不赋予权限安装cm不会出现本地源的parcel)

server: 
[root@server1 schema]# mkdir -p /opt/cloudera/parcel-repo
[root@server1 schema]# chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/*

agent:
[root@server1 schema]# mkdir -p /opt/cloudera/parcels
[root@server1 schema]# chown cloudera-scm:cloudera-scm /opt/cloudera/parcels/*

• 17.copy parcel文件

[root@server1 parcel-repo]# cp /usr/java/mysource/CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel* ./
[root@server1 parcel-repo]# cp /usr/java/mysource/manifest.json ./

• 18.启动cm

三台机器都切换到: cd /opt/cloudera-manager/cm-5.11.0/etc/init.d/
启动server: [root@server1 init.d]# ./cloudera-scm-server start
启动agent(3台都执行): ./cloudera-scm-agent start

[root@server1 init.d]# ./cloudera-scm-server start
Starting cloudera-scm-server:                              [  OK  ]
[root@server1 init.d]# ./cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]
[root@server1 init.d]#

• 19.观察log:

/opt/cloudera-manager/cm-5.11.0/log
执行成功:log为:
2017-05-24 21:13:21,335 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

• 20.访问cm

http://192.168.80.165:7180/cmf/login

• 22.登录

账号密码默认admin

选择所有主机,下一步:

没有本地的5.11的源,查看原来是

所有节点执行: chown cloudera-scm:cloudera-scm /opt/cloudera/*
然后重启server和agent,然后在访问cm的web界面重复上面的操作

不知道为啥… 还是没有cm5.11的源…..

注意:

后面终于发现,原来是CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha1文件的后缀应为sha,修改了CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha1为CDH-5.11.0-1.cdh5.11.0.p0.34-el6.parcel.sha 然后又重启服务,发现cm5.11 终于出来了!!!然后点击下一步

执行:
[root@server1 vm]# echo 10 > /proc/sys/vm/swappiness
[root@server1 vm]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@server1 vm]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

vi /etc/rc.local 并保存

#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

自定义安装(只安装hdfs yarn zk):

角色分配:

默认配置继续:

执行安装界面:

安装失败:

内存不够,命令超时,重试….

还是失败

将server的内存,重启系统,重试…

还是不行,界面如下:

再重启整个集群,终于成功了:

不过用的自己笔记本的虚拟机,实在是太卡了,不过还是搞定了,睡觉..