CDH5.16.1新增节点

1.环境

CentOS 7.6
CDH 5.16.1

2.服务器环境准备

1.设置 hostname 和 hosts

vim /etc/hostname

vim /etc/hosts

2.关闭SELINUX,将SELINUX=enforcing 改为SELINUX=disabled

vim /etc/selinux/config

3.配置ssh免密

#1.在新增节点生成公钥
ssh-keygen -t rsa

#2.将本机的公钥复制到集群中其他服务器上
cd ~/.ssh
ssh-copy-id cdh01

4.设置vm.swappiness=10

cat /proc/sys/vm/swappiness #查看当前值，设置后需要重启才生效

vi /etc/sysctl.conf
vm.swappiness=10 #在文件最后增加一行

5.安装Java,Scala

6.配置Java的快捷方式

mkdir /usr/java
ln -s /opt/module/jdk1.8.0_144/ /usr/java/default

7.拷贝mysql jar文件到目录 /usr/share/java/

mkdir /usr/share/java/
cp /opt/software/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /usr/share/java/

#注意jar包名称要修改为mysql-connector-java.jar
mv /usr/share/java/mysql-connector-java-5.1.27-bin.jar  /usr/share/java/mysql-connector-java.jar

3.Cloudera Manager安装

1.在新增节点上下载第三方依赖

yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

2.新增节点中下载CDH相关文件

3.新增节点上解压 cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz

mkdir /opt/module/cloudera-manager
tar -zxvf /opt/software/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/module/cloudera-manager/

4.新增节点创建用户cloudera-scm

useradd --system --home=/opt/module/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

# --system 创建一个系统账户
# --home 指定用户登入时的主目录，替换系统默认值/home/<用户名>
# --no-create-home 不要创建用户的主目录
# --shell 用户的登录 shell 名
# --comment 用户的描述信息

5.创建Parcel-repo 目录，将CDH相关文件拷贝到/opt/cloudera/parcel-repo中

mkdir -p /opt/cloudera/parcel-repo

6.配置CM Agent

vim /opt/module/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini

7.启动 CM Agent

/opt/module/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent start