＜Zhuuu_ZZ＞Cloudera CDH6安装教学

一 前置条件

• CentOS7/RHEL7以上
• 本教程使用5台虚拟机搭配，真实集群环境类似
• 配置

主机名	Server	Agent	MySQL
cdh1	√	√
cdh2		√
cdh3		√
cdh4		√
cdh5		√	√

二 安装前准备

注：除特别声明，本教程所有步骤都在/root目录下执行

1、所有机器安装JDK(必须安装在/usr/java目录下)

链接: jdk-8u221-linux-x64.tar.提取码：hok4

mkdir /usr/java
tar -zxvf jdk-8u221-linux-x64.tar.gz -C /usr/java
cat << EOF >> /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_221
export PATH=\$PATH:\$JAVA_HOME/bin
EOF

source /etc/profile

jps

2、所有机器关闭防火墙

systemctl disable firewalld

vi /etc/selinux/config
//把这个变为：
SELINUX=disabled

//完成后所有机器都需要重启
reboot

3、所有机器配置主机名与添加互信

• 配置主机名

hostnamectl set-hostname cdh1

hostnamectl set-hostname cdh2

hostnamectl set-hostname cdh3

hostnamectl set-hostname cdh4

hostnamectl set-hostname cdh5

• 配置ip主机名映射vi /etc/hosts

192.168.198.211 cdh1
192.168.198.212 cdh2
192.168.198.213 cdh3
192.168.198.214 cdh4
192.168.198.215 cdh5

• 互信

--第一步：
ssh-keygen
--第二步：
cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys
--第三步：
ssh-copy-id -i /root/.ssh/id_rsa.pub -p22 root@主机名

• 所有机器ping下集群机器防止互信失败

ping cdh1

ping cdh2

ping cdh3

ping cdh4

ping cdh5

4、各个节点上传对应安装包

链接: cdh6.2.提取码：v67n
链接: cm6.2.提取码：98pr
链接: mysql-community-…-5.7.提取码：xunm

##cdh1:
CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1
manifest.json
cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
mysql-connector-java-5.1.38.jar
PHOENIX-1.0.jar
PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel
PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha

##cdh2-cdh5
cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
mysql-connector-java-5.1.38.jar

##cdh5
mysql-community-client-5.7.32-1.el7.x86_64.rpm
mysql-community-common-5.7.32-1.el7.x86_64.rpm
mysql-community-devel-5.7.32-1.el7.x86_64.rpm
mysql-community-libs-5.7.32-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.32-1.el7.x86_64.rpm
mysql-community-server-5.7.32-1.el7.x86_64.rpm

5、所有机器安装一些依赖以及删除部分依赖

yum ‐y install gcc python‐devel cyrus‐sasl* chrony iotop yum‐utils yum‐pl ugin‐downloadonly psmisc ntp ntpdate httpd createrepo wget git rsync mod_ss l cyrus‐sasl‐plain cyrus‐sasl‐devel cyrus‐sasl‐gssapi libxslt krb5‐devel cy rus‐sasl‐gssapi cyrus‐sasl‐devel libxml2‐devel libxslt‐devel openldap‐devel python‐devel python‐simplejson sqlite‐devel net-tools

yum ‐y remove mariadb‐libs

• 若以上依赖下载出现Loaded plugins: fastestmirror

1. 
vi /etc/yum/pluginconf.d/fastestmirror.conf
enabled = 1修改为enabled = 0

2.
vi /etc/yum.conf
修改为plugins=0 不使用插件

3.
yum clean all
yum clean dbcache
yum makecache

6、所有机器修改一些系统参数

• 直接copy在/root下执行即可

sysctl vm.swappiness=10
echo 'vm.swappiness=10'>> /etc/sysctl.conf
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local

7、所有节点配置时间同步

#确认是否安装过ntp:
rpm -qa | grep ntp
#若有的话使用:yum -y remove 相关文件名命令卸载

#下载ntp
yum -y install ntp

#ntp服务器配置，这里为cdh1即192.168.198.211
vi /etc/ntp.conf
①注释掉所有的 restrict 开头的配置
②修改#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap
为 restrict 192.168.1.102 mask 255.255.255.0 nomodify notrap
③将所有 server 配置进行注释
④添加下面两行内容
server 127.127.1.0
fudge 127.127.1.0 stratum 10

#ntp客户端配置即同步服务器时间的其余四台机器都配置
vi /etc/ntp.conf
①注释所有 restrict 和 server 配置
②添加 server 192.168.198.211

#所有机器启动服务和开机自启动
service ntpd start
chkconfig ntpd on

#等待一段时间查看是否同步,所有机器输入
ntpstat

8、cdh5节点安装mysql

• 安装本地压缩包

yum -y localinstall mysql-community-common-5.7.32-1.el7.x86_64.rpm
yum -y localinstall mysql-community-libs-5.7.32-1.el7.x86_64.rpm
yum -y localinstall mysql-community-libs-compat-5.7.32-1.el7.x86_64.rpm
yum -y localinstall mysql-community-devel-5.7.32-1.el7.x86_64.rpm
yum -y localinstall mysql-community-client-5.7.32-1.el7.x86_64.rpm
yum -y localinstall mysql-community-server-5.7.32-1.el7.x86_64.rpm

• • 启动

systemctl start mysqld

• 开机自启动

systemctl enable mysqld

• 查看临时密码

cat /var/log/mysqld.log | grep password

• 登录

mysql ‐uroot ‐p'临时密码'

## 首次登录设置密码
set password=password('zhuuu_zz@163.com');
quit;

## 登陆设置允许外网访问当前的MySQL
mysql -uroot -p'zhuuu_zz@163.com'
use mysql;
update user set host='%' where host='localhost';
delete from user where host != '%';
flush privileges;
quit;

mysql -uroot -p'zhuuu_zz@163.com'

##创建ClouderaManager需要用到的几个数据库
create database cmf default character set = 'utf8';
create database amon default character set = 'utf8';
create database hive default character set = 'latin1';
create database hue default character set = 'utf8';
create database oozie default character set = 'utf8';
quit;

9、所有机器整理mysql的驱动包

mv mysql-connector-java-5.1.38.jar mysql‐connector‐java.jar
mkdir /usr/share/java
mv mysql‐connector‐java.jar /usr/share/java/

三 安装CM组件

• 全部节点都安装daemons和agent

yum ‐y localinstall cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm
//上面daemons可能会安装失败，提示下方agent安装需要daemons组件
//那就用rpm -ivh 安装
//rpm -ivh cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm

yum -y localinstall cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm

• 上述安装若遇Transaction check error：Error Summary

##如下方错误
Transaction check error: 
file /usr/share/mysql/ukrainian/errmsg.sys from install of mariadb-libs-1:5.5.68-1.el7.x86_64 conflicts with file from package MySQL-server-5.6.46-1.el7.x86_64
......
Error Summary
-------------
##包冲突，找到from package后面内容删除即可
yum remove MySQL-server-5.6.46-1.el7.x86_64 再重新安装即可

• cdh上安装server

rpm -ivh cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm
//yum ‐y localinstall cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm

四 cdh1安装CDH组件

mv CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel* /opt/cloudera/parcel-repo/
mv PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel* /opt/cloudera/parcel-repo/
mv manifest.json /opt/cloudera/parcel-repo/
mv PHOENIX-1.0.jar /opt/cloudera/csd/
cd /opt/cloudera/parcel-repo/

##校验sha1
sha1sum CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel
cat CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1
##若发现出现的hash运算值是一样的，那就意味着下载的parcel包是完整的,否则需要重新下载
mv CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1 CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha

##校验phoenix，不过不用修改后缀
sha1sum PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel
cat PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha

五 修改server和agent的配置文件

# cdh1上修改server
vi /etc/cloudera-scm-server/db.properties
# 8行为com.cloudera.cmf.db.type=mysql
# 12行去掉注释把localhost改成cdh5
# 15行去掉注释为com.cloudera.cmf.db.name=cmf
# 18行去掉注释，把cmf改成root
# 21行去掉注释，密码那里填写zhuuu_zz@163.com
# 28行改为:com.cloudera.cmf.db.setupType=EXTERNAL
 
 # 全部节点修改agent
 vi /etc/cloudera-scm-agent/config.ini
 # 15行把localhost修改为cdh1

六 启动CM，开始CDH集群

# 在cdh1上启动server
systemctl start cloudera-scm-server

#对cdh1另外一个窗口，查看相关日志。有异常就解决异常(一般ERROR不要慌，比如什么Cloudera 测试，连接超时啥的)
tail -f 200 /var/log/cloudera‐scm‐server/cloudera‐scm‐server.log

# 这个异常可以忽略 ERROR ParcelUpdateService:com.cloudera.parcel.component s.ParcelDownloaderImpl: Unable to retrieve remote parcel repository manifes t

# 在全部节点上启动agent
systemctl start cloudera-scm-agent

# 当在 cdh001上 netstat ‐tunlp | grep 7180 有内容时，说明我们可以访问web页面
192.168.198.211:7180
#用户名和密码都为admin

七 更多

注：安装过程如有问题，可下方留言或私信我邮箱zhuuu_zz@163.com