CDH 6.3.1 史上最全安装手册

因项目需要CDH，经过十来次的重复安装，反复踩坑、填坑、验证，终于了今日的成功。

基础设施

设置主机

Node	cloudera-scm-server	cloudera-scm-agent	操作系统	cpu	内存
cdh-guance01	✅	✅	Centos7.4	4核	16G
cdh-guance02	❎	✅	Centos7.4	4核	16G
cdh-guance03	❎	✅	Centos7.4	4核	16G

cloudera-scm-server 一般是一个节点,cloudera-scm-agent可以是多个节点

• 主机名

hostnamectl set-hostname cdh-guance01

hostnamectl set-hostname cdh-guance02

hostnamectl set-hostname cdh-guance03

• hosts 配置（所有节点）

echo "10.0.16.217 cdh-guance01">> /etc/hosts
echo "10.0.16.218 cdh-guance02">> /etc/hosts
echo "10.0.16.219 cdh-guance03">> /etc/hosts

免密互通（所有节点）

• 生成密钥

ssh-keygen -t rsa

• 复制密钥

ssh-copy-id cdh-guance01
ssh-copy-id cdh-guance02
ssh-copy-id cdh-guance03

资源（所有节点）

echo '* soft nproc 65535' >> /etc/security/limits.conf
echo '* hard nproc 65535' >> /etc/security/limits.conf
echo '* soft nofile 65535' >> /etc/security/limits.conf
echo '* hard nofile 65535' >> /etc/security/limits.conf

CDH 资源下载（主节点）

[root@cdh-guance01 opt]# ll -h
总用量 3.9G
drwxr-xr-x 2 root root 4.0K 8月  12 09:57 bin
-rw-r--r-- 1 root root 3.9G 8月  12 09:49 cdh6.3.1-all.zip

安装 MySQL（主节点）

• 安装

yum install libaio -y

yum -y install autoconf

wget https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm

rpm -ivh mysql57-community-release-el7-11.noarch.rpm

rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022

yum -y install mysql-community-server

systemctl start mysqld

• 查看初始密码

[root@cdh-guance01 opt]# grep "password" /var/log/mysqld.log
2024-08-12T09:23:08.653194Z 1 [Note] A temporary password is generated for root@localhost: kWTrBTxyq8/U

• 设置数据库

[root@cdh-guance01 opt]# mysql_secure_installation

Securing the MySQL server deployment.

Enter password for user root: 

修改密码后，后面全部输入 Y。

• 修改密码策略（按需，可以不用）

> mysql -uroot -p

# 查看当前安全策略
mysql> show variables like 'validate_password%';
+--------------------------------------+--------+
| Variable_name                        | Value  |
+--------------------------------------+--------+
| validate_password_check_user_name    | OFF    |
| validate_password_dictionary_file    |        |
| validate_password_length             | 8      |
| validate_password_mixed_case_count   | 1      |
| validate_password_number_count       | 1      |
| validate_password_policy             | MEDIUM |
| validate_password_special_char_count | 1      |
+--------------------------------------+--------+
7 rows in set (0.00 sec)

# 最低安全策略(0->LOW ,1->MEDIUM, 2->STRONG)
mysql>set global validate_password_policy=0; 
# 最小密码字符长度是4
mysql>set global validate_password_length=4; 
# 设置本机root密码
mysql>alter user 'root'@'localhost' identified by 'root';

# 刷新权限
mysql>flush privileges;
#退出
mysql>exit

mysql 的密码可以设置复杂点，因为我设置的过于简单，结果数据库被莫名其妙的清空了。

• 关闭 ssl

初始化 scm 的时候一直报数据库异常，ssl有点问题，我被这个问题困了一天，关闭 ssl 提前脱困。

mysql> SHOW VARIABLES LIKE 'have_ssl';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| have_ssl      | YES   |
+---------------+-------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES LIKE 'ssl%';
+---------------+-----------------+
| Variable_name | Value           |
+---------------+-----------------+
| ssl_ca        | ca.pem          |
| ssl_capath    |                 |
| ssl_cert      | server-cert.pem |
| ssl_cipher    |                 |
| ssl_crl       |                 |
| ssl_crlpath   |                 |
| ssl_key       | server-key.pem  |
+---------------+-----------------+
7 rows in set (0.00 sec)

如果 have_ssl 的值是 YES 就需要关闭。

找到MySQL的配置文件my.cnf或my.ini，通常位于/etc/mysql/目录下。在[mysqld]区块中添加以下配置来禁用SSL：

[mysqld]
ssl=0

调整后，重启 mysql 服务，在进行查看。

systemctl restart mysqld

查看 ssl。

mysql> SHOW VARIABLES LIKE 'have_ssl';
+---------------+----------+
| Variable_name | Value    |
+---------------+----------+
| have_ssl      | DISABLED |
+---------------+----------+
1 row in set (0.00 sec)

• 创建scm 帐号和数据库

后面安装集群会用到，在 mysql 命令行依次执行以下命令

set global validate_password_policy=0;
set global validate_password_length=4;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON *.* TO 'scm'@'%' IDENTIFIED BY 'sQnj2CzMHPSm7BJc';

安装 rsync（所有节点）

用于同步文件，非必选，建议安装，提升安装效率。

yum install -y rsync

JDK（所有节点）

• 安装

yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y

[root@cdh-guance01 opt]# java -version
openjdk version "1.8.0_412"
OpenJDK Runtime Environment (build 1.8.0_412-b08)
OpenJDK 64-Bit Server VM (build 25.412-b08, mixed mode

• 配置环境变量

新增环境变量文件 /etc/profile.d/my_env.sh，内容如下：

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-1.el7_9.x86_64
export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin

• 生效

source /etc/profile.d/my_env.sh

CM 安装

下载解压

这里没有提供下载地址，百度网盘下载太慢了，4G 的压缩包，有需要的话，给我留言，通过 QQ 发送。

[root@cdh-guance01 opt]# ll
总用量 4018204
drwxr-xr-x 2 root root       4096 8月  13 14:09 bin
-rw-r--r-- 1 root root 4114603334 8月  13 14:09 cdh6.3.1-all.zip
-rw-r--r-- 1 root root      25680 4月  27 2017 mysql57-community-release-el7-11.noarch.rpm
[root@cdh-guance01 opt]# unzip cdh6.3.1-all.zip 
Archive:  cdh6.3.1-all.zip
   creating: cdh6.3.1/
  inflating: cdh6.3.1/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel  
  inflating: cdh6.3.1/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1  
   creating: cdh6.3.1/cm6.3.1/
  inflating: cdh6.3.1/cm6.3.1/cm6.3.1-redhat7.tar.gz  
  inflating: cdh6.3.1/cm6.3.1/jre-8u181-linux-x64.tar.gz  
  inflating: cdh6.3.1/cm6.3.1/mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz  
  inflating: cdh6.3.1/cm6.3.1/mysql-connector-java-5.1.47.jar  
  inflating: cdh6.3.1/manifest.json  

分发 mysql-connector-java.jar

CM 是java程序，链接数据库可以使用多种数据库类型，这里主要使用 mysql，需要添加相应的依赖。将 mysql-connector-java-5.1.27-bin.jar上传到/usr/share/java路径下，并重命名mysql-connector-java.jar,分发出去

• 编写分发程序 xsync

#在该文件中编写如下代码
#!/bin/bash
#1 获取输入参数个数，如果没有参数，直接退出
pcount=$#
if((pcount==0)); then
echo no args;
exit;
fi
 
#2 获取文件名称
p1=$1
fname=`basename $p1`
echo fname=$fname
 
#3 获取上级目录到绝对路径
pdir=`cd -P $(dirname $p1); pwd`
echo pdir=$pdir
 
#4 获取当前用户名称
user=`whoami`
 
#5 循环
for((host=1; host<4; host++)); do
        echo ------------------- cdh-guance0$host --------------
        rsync -av $pdir/$fname $user@cdh-guance0$host:$pdir
done

• 分发

[root@cdh-guance01 opt]# cd cdh6.3.1/cm6.3.1/
[root@cdh-guance01 cm6.3.1]# mv mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
[root@cdh-guance01 bin]# ./xsync /usr/share/java/
fname=java
pdir=/usr/share
------------------- cdh-guance01 --------------
sending incremental file list

sent 94 bytes  received 17 bytes  74.00 bytes/sec
total size is 1,007,502  speedup is 9,076.59
------------------- cdh-guance02 --------------
sending incremental file list
java/
java/mysql-connector-java.jar

sent 1,007,878 bytes  received 39 bytes  2,015,834.00 bytes/sec
total size is 1,007,502  speedup is 1.00
------------------- cdh-guance03 --------------
sending incremental file list
java/
java/mysql-connector-java.jar

sent 1,007,878 bytes  received 39 bytes  2,015,834.00 bytes/sec
total size is 1,007,502  speedup is 1.00

准备工作

[root@cdh-guance01 opt]# mkdir /opt/cloudera-manager
[root@cdh-guance01 opt]# cd cdh6.3.1/cm6.3.1/
[root@cdh-guance01 cm6.3.1]# ll
总用量 1993012
-rw-r--r-- 1 root root 1411444147 6月  16 2021 cm6.3.1-redhat7.tar.gz
-rw-r--r-- 1 root root   81191893 6月  16 2021 jre-8u181-linux-x64.tar.gz
-rw-r--r-- 1 root root  548193637 6月  16 2021 mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
[root@cdh-guance01 cm6.3.1]# tar -zxvf cm6.3.1-redhat7.tar.gz 
...
[root@cdh-guance01 cm6.3.1]# cd cm6.3.1/RPMS/x86_64/
[root@cdh-guance01 x86_64]# mv cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@cdh-guance01 x86_64]# mv cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@cdh-guance01 x86_64]# mv cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm /opt/cloudera-manager/
[root@cdh-guance01 x86_64]# cd /opt/cloudera-manager/
[root@cdh-guance01 cloudera-manager]# ll
总用量 1185876
-rw-r--r-- 1 2001 2001   10483568 9月  25 2019 cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001 1203832464 9月  25 2019 cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
-rw-r--r-- 1 2001 2001      11488 9月  25 2019 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
[root@cdh-guance01 cloudera-manager]# cd /opt/bin/
# 分发文件到其他机器
[root@cdh-guance01 bin]# ./xsync /opt/cloudera-manager/

安装 cloudera-manager-daemons （所有节点）

[root@cdh-guance01 x86_64]# cd /opt/cloudera-manager/
[root@cdh-guance01 cloudera-manager]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm

安装 cloudera-manager-agent（所有节点）

yum install bind-utils psmisc cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs /lib/lsb/init-functions httpd mod_ssl openssl-devel python-psycopg2 MySQL-python libxslt

rpm -ivh /opt/cloudera-manager/cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm

调整 server host，修改/etc/cloudera-scm-agent/config.ini的server_host，设置为cloudera-manager-server 主机名，如server_host=cdh-guance01。

等 cloudera-manager-server 安装启动成功后，再启动 agent，否则会启动失败。

systemctl start cloudera-scm-agent

安装 cloudera-manager-server（主节点）

• 安装 cloudera-manager-server

# 安装cloudera-manager-server
[root@cdh-guance01 cloudera-manager]# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm 

• 配置离线仓库

[root@cdh-guance01 cdh6.3.1]# pwd
/opt/cdh6.3.1
[root@cdh-guance01 cdh6.3.1]# ll
总用量 2035088
-rw-r--r-- 1 root root 2083878000 6月  16 2021 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
-rw-r--r-- 1 root root         40 6月  16 2021 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
drwxr-xr-x 3 root root       4096 8月  12 19:26 cm6.3.1
-rw-r--r-- 1 root root      33887 6月  16 2021 manifest.json
[root@cdh-guance01 cdh6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel* /opt/cloudera/parcel-repo
[root@cdh-guance01 cdh6.3.1]# mv manifest.json /opt/cloudera/parcel-repo
[root@cdh-guance01 cdh6.3.1]# cd /opt/cloudera/parcel-repo
[root@cdh-guance01 parcel-repo]# ll
总用量 2035084
-rw-r--r-- 1 root root 2083878000 6月  16 2021 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
-rw-r--r-- 1 root root         40 6月  16 2021 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
-rw-r--r-- 1 root root      33887 6月  16 2021 manifest.json
[root@cdh-guance01 parcel-repo]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha

• 调整 DB 配置

/etc/cloudera-scm-server/db.properties

com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=cdh-guance01:3306
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=root
com.cloudera.cmf.db.setupType=External

• 启动 cloudera-scm-server

# 初始化数据库
[root@cdh-guance01 parcel-repo]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root
Enter SCM password: 
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-1.el7_9.x86_64
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.412.b08-1.el7_9.x86_64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
log4j:ERROR Could not find value for key log4j.appender.A
log4j:ERROR Could not instantiate appender named "A".
Mon Aug 12 20:08:42 CST 2024 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[2024-08-12 20:08:42,790] INFO     0[main] - com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java) - Successfully connected to database.
All done, your SCM database is configured correctly!
# 启动服务
[root@cdh-guance01 parcel-repo]# systemctl start cloudera-scm-server 

登陆配置 CDH 组件

http://cdh-guance01:7180，帐号信息 admin/admin

• 组件安装，按实际需求选择对应的组件
  
  
  

• 数据库配置

为了简便操作，我直接授权 scm 所有的 databases 权限，弱密码有风险，建议设置复杂点。

常见问题

NFS Gateway 启动异常

解决方案：启动 rpcbind 服务

[root@cdh-guance02 ~]# service rpcbind start
Redirecting to /bin/systemctl start rpcbind.service
[root@cdh-guance02 ~]# systemctl status rpcbind
● rpcbind.service - RPC bind service
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.service; enabled; vendor preset: enabled)
   Active: active (running) since 二 2024-08-13 09:19:23 CST; 5s ago
  Process: 30123 ExecStart=/sbin/rpcbind -w $RPCBIND_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 30124 (rpcbind)
   CGroup: /system.slice/rpcbind.service
           └─30124 /sbin/rpcbind -w

8月 13 09:19:23 cdh-guance02 systemd[1]: Starting RPC bind service...
8月 13 09:19:23 cdh-guance02 systemd[1]: Started RPC bind service.

MainThread agent ERROR Error, CM server guid updated

[14/Aug/2024 14:01:30 +0000] 29251 MainThread agent        ERROR    Error, CM server guid updated, expected 2dab60bc-d941-438b-8454-c7cb271f4ad6, received cd48128c-8c13-4b96-ab8f-4b8cc20008d4
[14/Aug/2024 14:01:30 +0000] 29251 MainThread agent        ERROR    Error, CM server guid updated, expected 2dab60bc-d941-438b-8454-c7cb271f4ad6, received cd48128c-8c13-4b96-ab8f-4b8cc20008d4
[14/Aug/2024 14:01:34 +0000] 29251 MainThread agent        ERROR    Error, CM server guid updated, expected 2dab60bc-d941-438b-8454-c7cb271f4ad6, received cd48128c-8c13-4b96-ab8f-4b8cc20008d4
[14/Aug/2024 14:01:34 +0000] 29251 MainThread agent        ERROR    Error, CM server guid updated, expected 2dab60bc-d941-438b-8454-c7cb271f4ad6, received cd48128c-8c13-4b96-ab8f-4b8cc20008d4
[14/Aug/2024 14:01:39 +0000] 29251 MainThread agent        ERROR    Error, CM server guid updated, expected 2dab60bc-d941-438b-8454-c7cb271f4ad6, received cd48128c-8c13-4b96-ab8f-4b8cc20008d4

解决方案：

删除 cm_guid

sudo rm /var/lib/cloudera-scm-agent/cm_guid

重启 cloudera-scm-agent

systemctl restart cloudera-scm-agent