CM和CDH安装

最新推荐文章于 2024-04-26 16:31:32 发布

奋进的小马

最新推荐文章于 2024-04-26 16:31:32 发布

阅读量2.8k

点赞数

分类专栏： CDH 文章标签： linux hadoop centos

本文链接：https://blog.csdn.net/weixin_42507440/article/details/121683101

版权

CDH 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

CDH安装

文章目录

CDH安装
1、虚拟机配置
2、安装包和yum准备
- 2.1 yum配置
- 2.2 安装准备
3、前置操作
4、CM安装
- 4.1 性能优化
5、集群安装
- 5.1 安装
- 5.2 问题：

1、虚拟机配置

主机	内存(G)	cpu	磁盘
10.11.90.11	32	16	500g
10.11.90.12	32	16	500g
10.11.90.13	32	16	500g
10.11.90.14	32	16	500g

2、安装包和yum准备

2.1 yum配置

#ansible，ansible机器10.11.90.10
1. ansible机器准备好CentOS7-Base-163.repo和host文件
```
[all:vars]
ansible_ssh_user=root
ansible_ssh_pass=root
[master]
10.11.90.11
[node]
10.11.90.12
10.11.90.13
10.11.90.14
[all]
10.11.90.11
10.11.90.12
10.11.90.13
10.11.90.14
```
2.dns配置
ansible -i /root/host/cdh all -m shell -a "echo 'nameserver 8.8.8.8' >> /etc/resolv.conf"
3. ansible机器执行分发yum源
ansible -i /root/host/cdh all -m copy -a "src=/root/conf/CentOS7-Base-163.repo dest=/etc/yum.repos.d"
4. yum编译
ansible -i /root/host/cdh all -m shell -a "yum clean all"
ansible -i /root/host/cdh all -m shell -a "yum makecache"

#yum源网址
http://mirrors.163.com/

[root@cdh-master ~]# cd /etc/yum.repos.d/
[root@cdh-master yum.repos.d]# mkdir bak
[root@cdh-master yum.repos.d]# mv ./CentOS7-* ./bak/
[root@cce-master1 yum.repos.d]# pwd
/etc/yum.repos.d
[root@cce-master1 yum.repos.d]# cat CentOS7-Base-163.repo 
# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the 
# remarked out baseurl= line instead.
#
#
[base]
name=CentOS-$releasever - Base - 163.com
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
baseurl=http://mirrors.163.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirrors.163.com/centos/RPM-GPG-KEY-CentOS-7

#released updates
[updates]
name=CentOS-$releasever - Updates - 163.com
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
baseurl=http://mirrors.163.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirrors.163.com/centos/RPM-GPG-KEY-CentOS-7

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - 163.com
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
baseurl=http://mirrors.163.com/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=http://mirrors.163.com/centos/RPM-GPG-KEY-CentOS-7

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.163.com/centos/RPM-GPG-KEY-CentOS-7
[root@cce-master1 yum.repos.d]# 

yum clean all
yum makecache

2.2 安装准备

#CDH官网：https://www.cloudera.com

#我这下载的是6.3.2
CM:
[root@cdh001 CM]# ll
total 1380436
-rwxrwxrwx. 1 root root      14040 Oct  9 02:43 allkeys.asc
-rwxrwxrwx. 1 root root   10483568 Oct  9 02:43 cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
-rwxrwxrwx. 1 root root 1203832464 Oct  9 03:12 cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
-rwxrwxrwx. 1 root root      11488 Oct  9 02:43 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
-rwxrwxrwx. 1 root root      10996 Oct  9 02:43 cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
-rwxrwxrwx. 1 root root   14209868 Oct  9 02:43 enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
-rwxrwxrwx. 1 root root  184988341 Oct  9 02:45 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm

CDH:
[root@cdh001 CDH]# ll
total 2033432
-rwxrwxrwx. 1 root root 2082186246 Oct  9 03:18 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
-rwxrwxrwx. 1 root root         40 Oct  9 02:43 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
-rwxrwxrwx. 1 root root         64 Oct  9 02:43 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
-rwxrwxrwx. 1 root root      33887 Oct  9 02:43 manifest.json

mysql:
[root@cdh001-master mysql]# ll
total 185520
-rwxrwxrwx. 1 root root    277604 Oct  9 02:43 01_mysql-community-common-5.7.16-1.el7.x86_64.rpm
-rwxrwxrwx. 1 root root   2237116 Oct  9 02:43 02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm
-rwxrwxrwx. 1 root root   2112700 Oct  9 02:43 03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm
-rwxrwxrwx. 1 root root  25034716 Oct  9 02:44 04_mysql-community-client-5.7.16-1.el7.x86_64.rpm
-rwxrwxrwx. 1 root root 159295840 Oct  9 02:45 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm
-rwxrwxrwx. 1 root root   1006956 Oct  9 02:44 mysql-connector-java-5.1.48.jar

3、前置操作

3.1 host配置

3.1.1 修改hostname

#10.11.90.11
hostnamectl set-hostname cdh-master

#10.11.90.12
hostnamectl set-hostname cdh-node1

#10.11.90.13
hostnamectl set-hostname cdh-node2

#10.11.90.14
hostnamectl set-hostname cdh-node3

3.1.2 修改hosts文件(所有节点都一致)

#absilbe
1. 准备一个脚本
#!/bin/bash
cat >> /etc/hosts <<EOF
10.11.90.11 cdh-master
10.11.90.12 cdh-node1
10.11.90.13 cdh-node2
10.11.90.14 cdh-node3
EOF
2. 脚本分发
nsible -i /root/host/cdh all -m copy -a "src=/roo/shell/uphosts.sh dest=/root/"
3. 脚本修改可执行权限
ansible -i /root/host/cdh all -m shell -a "chmod +x /root/uphosts.sh"
4. 修改hosts配置
ansible -i /root/host/cdh all -m shell -a "/root/uphosts.sh"

#普通配置
[root@cdh001 etc]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

10.11.90.11 cdh-master
10.11.90.12 cdh-node1
10.11.90.13 cdh-node2
10.11.90.14 cdh-node3

#单行末尾追加
echo '10.11.90.11 cdh-master' >> /etc/hosts

#EOF末尾追加
cat >> /etc/hosts <<EOF
10.11.90.11 cdh-master
10.11.90.12 cdh-node1
10.11.90.13 cdh-node2
10.11.90.14 cdh-node3
EOF

3.2 防火墙配置

所有节点都需要设置，可以使用ansible

3.2.1 关闭防火墙

systemctl stop firewalld
systemctl disable firewalld.service
iptables -F

#ansible
ansible -i /root/host/cdh all -m shell -a "systemctl stop firewalld"
ansible -i /root/host/cdh all -m shell -a "systemctl status firewalld"
ansible -i /root/host/cdh all -m shell -a "systemctl disable firewalld.service"
ansible -i /root/host/cdh all -m shell -a "iptables -F"

3.2.2 关闭selinux

#ansible
ansible -i /root/host/cdh all -m shell -a "sed  -i 's#=enforcing#=disable#g' /etc/selinux/config"

#普通修改
vim /etc/selinux/config

SELINUX=disabled

[root@cdh001 etc]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

3.3 免密登陆配置

master节点操作
[root@cdh001 ~]# ssh root@10.11.90.12
[root@cdh001 ~]# cd .ssh/
[root@cdh001 .ssh]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:VkIrD9Jkzs8NtG9sNGJjV4nGl1Uv87rV2PtFwEFBPgk root@cdh001-master
The key's randomart image is:
+---[RSA 2048]----+
|      o o. .E**o.|
|     * o o+.++ o.|
|    . * X.=.  O .|
|     . O % .   * |
|        S *     o|
|       . o     =.|
|              o =|
|               oo|
|              ..o|
+----[SHA256]-----+
[root@cdh001 .ssh]# ssh-copy-id 10.11.90.12
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@10.11.90.12's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '10.11.90.12'"
and check to make sure that only the key(s) you wanted were added.

[root@cdh001 .ssh]# ssh 10.11.90.12
Last login: Sun Oct 10 23:37:19 2021 from cdh001-master
[root@cdh002-node2 ~]# exit
logout
Connection to 10.11.90.12 closed.

#把公钥分发到 每个节点

3.4 NTP配置

3.4.1 NTP client安装

NTP server端在别的机器，这里就不明说了，所有机器都装

#安装ntp服
yum install -y ntp
ansible -i /root/host/cdh all -m shell -a "yum install -y ntp"

#备份NTP文件
cp /etc/ntp.conf /etc/ntp.conf.bak

#修改配置文件 
[root@chushi ~]# vi /etc/ntp.conf
server 10.11.90.100      ## IP为NTP server端的IP

#启动前先ntpdate⼿动同步下时间，避免客户端与server端时间差过⼤，导致ntp不能同步。 
ntpdate -u 10.11.90.100 (#serverIP) 

[root@demon-harbor etc]# ntpdate -u 10.11.90.100
17 Sep 13:53:46 ntpdate[15258]: step time server 10.11.90.237 offset -24.875959 sec

ansible -i /root/host/cdh all -m shell -a "ntpdate -u 10.11.90.100"

#启动NTP服务器
systemctl enable ntpd && systemctl start ntpd
ansible -i /root/host/cdh all -m shell -a "systemctl enable ntpd && systemctl start ntpd"

#查看是否时间同步(#如显示unsynchronised，同步时间大概需要几分钟，几分钟后再次查看即可)
ntpstat 
ansible -i /root/host/cdh all -m shell -a "ntpstat"

[root@demon-harbor etc]# ntpstat 
synchronised to NTP server (10.11.90.237) at stratum 7
   time correct to within 16 ms
   polling server every 64 s

3.4 JDK安装

3.4.1 检查并卸载自带jdk

#rpm包安装
rpm -qa | grep jdk 
rpm -qa|grep openjdk |xargs -I {} rpm -e --nodeps {}  #所有节点执行

#jar包安装
java -version

3.4.2 安装jdk

所有节点都安装jdk

#ansible
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "mkdir -p /usr/local/java"
[root@ansible ~]# ansible -i /root/host/cdh all -m copy -a "src=/root/package/jdk-8u301-linux-x64.tar.gz dest=/usr/local/java"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "tar -xvf /usr/local/java/jdk-8u301-linux-x64.tar.gz -C /usr/local/java"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "chmod +x /root/up_bash_profile.sh"
#实际操作这两个没有生效
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "/root/up_bash_profile.sh"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "source /root/.bash_profile"

[root@ansible scripts]# cat up_bash_profile.sh 
#!/bin/bash
cat >> /root/.bash_profile <<EOF
export JAVA_HOME=/usr/local/java/jdk1.8.0_301
export PATH=$JAVA_HOME/bin:$PATH
export JRE_HOME=$JAVA_HOME/jre
EOF


mkdir -p /usr/local/java
cp /root/zhs/jdk-8u301-linux-x64.tar.gz /usr/local/java
cd /usr/local/java
tar -xvf jdk-8u301-linux-x64.tar.gz

vim ~/.bash_profile
加入内容：
export JAVA_HOME=/usr/local/java/jdk1.8.0_301
export PATH=$JAVA_HOME/bin:$PATH
export JRE_HOME=$JAVA_HOME/jre

source ~/.bash_profile

验证是否安装成功：
java -version

#这个必须做，否则启动cloudera-scm-server报错java——home找不到
mkdir -p /usr/java
cd jdk1.8.0_301/
ln -s /usr/local/java/jdk1.8.0_301 /usr/java/default

3.5 安装http和repo

#安装repo
#ansible
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "yum install -y createrepo"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "yum install -y httpd"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "systemctl start httpd && systemctl enable httpd"

yum install -y createrepo

#安装http
yum install -y httpd
systemctl start httpd && systemctl enable httpd

3.6 安装mysql

3.6.1 安装mysql8的驱动

所有节点装mysql驱动

# 下载
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar -zxf mysql-connector-java-5.1.46.tar.gz

# 创建目录
mkdir /usr/share/java/
# 放到指定目录
（一定要改名为mysql-connector-java.jar）
cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

#ansible
ansible -i /root/host/cdh all -m shell -a "mkdir /usr/share/java/"
ansible -i /root/host/cdh all -m copy -a "src=/root/package/mysql-connector-java-5.1.48.jar dest=/usr/share/java"
ansible -i /root/host/cdh all -m shell -a "mv /usr/share/java/mysql-connector-java-5.1.48.jar /usr/share/java/mysql-connector-java.jar"

3.6.2 mysql安装master

#contos7默认安装了mariadb数据库,这里把他移除：yum remove mariadb-libs.x86_64
yum remove mariadb-libs.x86_64

#mysql rpm安装
rpm -ivh 01_mysql-community-common-5.7.16-1.el7.x86_64.rpm
rpm -ivh 02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm
rpm -ivh 04_mysql-community-client-5.7.16-1.el7.x86_64.rpm
rpm -ivh 03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm
rpm -ivh 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm
-----------------------------------------------------------------------
[root@cdh-master mysql]# rpm -ivh 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm
warning: 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
error: Failed dependencies:
	/usr/bin/perl is needed by mysql-community-server-5.7.16-1.el7.x86_64
	net-tools is needed by mysql-community-server-5.7.16-1.el7.x86_64
	perl(Getopt::Long) is needed by mysql-community-server-5.7.16-1.el7.x86_64
	perl(strict) is needed by mysql-community-server-5.7.16-1.el7.x86_64
[root@cdh001-master mysql]# yum install -y net-tools
[root@cdh-master mysql]# yum install -y perl
---------------------------------------------------------------------------------------
yum install -y net-tools 
rpm -ivh 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm

3.6.3 mysql配置

#启动mysql
systemctl start mysqld && systemctl enable mysqld

#查看mysql初始密码
[root@cdh001-master mysql]# grep 'temporary password' /var/log/mysqld.log
2021-10-11T07:41:18.527744Z 1 [Note] A temporary password is generated for root@localhost: 7/ixefDd2w64

root/7/ixefDd2w64
root/?Mbxj_GUe8c%

#修改Mysql密码
mysql -uroot -p7/ixefDd2w64   #输入上面获取到的密码
alter user root@localhost identified by '2wsx#EDC';
flush privileges;
-----------------------------------------------------------------------------------------------------------
[root@cdh001-master mysql]# mysql -uroot -p7/ixefDd2w64
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.16

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> alter user root@localhost identified by '2wsx#EDC';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> 
----------------------------------------------------------------------------------------------------------

3.6.4 创建所需要的库和表

mysql -u root --password='2wsx#EDC' -e 'create database metastore default character set utf8';
mysql -u root --password='2wsx#EDC' -e "create user 'hive'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database hive default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on hive.* to 'hive'@'%'";
mysql -u root --password='2wsx#EDC' -e "grant all privileges on metastore.* to 'hive'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'amon'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database amon default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on amon.* to 'amon'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'rman'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database rman default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on rman.* to 'rman'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'sentry'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database sentry default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on sentry.* to 'sentry'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'nav'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database nav default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on nav.* to 'nav'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'navms'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database navms default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on navms.* to 'navms'@'%'"
mysql -u root --password='2wsx#EDC' -e "create user 'cm'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database cm default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on cm.* to 'cm'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'oos'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database oos default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on oos.* to 'oos'@'%'";
mysql -u root --password='2wsx#EDC' -e "create user 'hue'@'%' identified by '2wsx#EDC'";
mysql -u root --password='2wsx#EDC' -e 'create database hue default character set utf8';
mysql -u root --password='2wsx#EDC' -e "grant all privileges on hue.* to 'hue'@'%'";

[root@cdh001-master rc.d]# mysql -u root --password='2wsx#EDC'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 62
Server version: 5.7.16 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| amon               |
| cm                 |
| hive               |
| hue                |
| metastore          |
| mysql              |
| nav                |
| navms              |
| oos                |
| performance_schema |
| rman               |
| sentry             |
| sys                |
+--------------------+
14 rows in set (0.01 sec)

[root@cdh001-master ~]#  mysql -uroot -p2wsx#EDC
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 63
Server version: 5.7.16 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------------+
| Tables_in_mysql           |
+---------------------------+
| columns_priv              |
| db                        |
| engine_cost               |
| event                     |
| func                      |
| general_log               |
| gtid_executed             |
| help_category             |
| help_keyword              |
| help_relation             |
| help_topic                |
| innodb_index_stats        |
| innodb_table_stats        |
| ndb_binlog_index          |
| plugin                    |
| proc                      |
| procs_priv                |
| proxies_priv              |
| server_cost               |
| servers                   |
| slave_master_info         |
| slave_relay_log_info      |
| slave_worker_info         |
| slow_log                  |
| tables_priv               |
| time_zone                 |
| time_zone_leap_second     |
| time_zone_name            |
| time_zone_transition      |
| time_zone_transition_type |
| user                      |
+---------------------------+
31 rows in set (0.00 sec)
mysql> select user,host from user;
+-----------+-----------+
| user      | host      |
+-----------+-----------+
| amon      | %         |
| cm        | %         |
| hive      | %         |
| hue       | %         |
| nav       | %         |
| navms     | %         |
| oos       | %         |
| rman      | %         |
| sentry    | %         |
| mysql.sys | localhost |
| root      | localhost |
+-----------+-----------+
11 rows in set (0.00 sec)

4、CM安装

4.1 性能优化

4.1.1 删除虚拟网卡

virsh net-destroy default
virsh net-undefine default

4.1.2 静止透明大页

所有节点执行

大页面（Huge Page）是内存分页管理的一种实现方式。计算机内存是通过表映射（页表）的方式进行内存寻址，目前系统内存以4KB为一个页，作为内存寻址的最小单元。随着内存不断增大，页表的大小将会不断增大。一台256G内存的机器，如果使用4KB小页，仅页表大小就要4G左右。页表是必须装在内存的，而且是在CPU内存，太大就会发生大量miss，内存寻址性能就会下降。

Huge Page就是为了解决这个问题，它使用2MB的大页代替传统小页来管理内存，这样页表大小就可以控制的很小，进而全部装在CPU内存，防止出现miss。它有两种实现方式，一是静态大页面（Static Huge Pages，SHP），二是透明大页面（Transparent Huge Pages，THP）。从它们的名字就可以看出，SHP是静态的，而THP是动态的。由于THP是在运行期做分配和管理，因此会有一定程度的延迟，对于内存密集型的应用十分不利，必须关闭它

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

vim /etc/rc.d/rc.local
#最后添加
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@cdh001-master rc.d]# cat /etc/rc.d/rc.local
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@cdh001-master rc.d]#

4.1.3 减少集群使用交换内存

所有节点执行

sysctl -w vm.swappiness=0
echo 'vm.swappiness=0' >> /etc/sysctl.conf
sysctl -p

#ansible
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "sysctl -w vm.swappiness=0"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "echo 'vm.swappiness=0' >> /etc/sysctl.conf"
[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "sysctl -p"

4.1.4 CM安装

4.1.4.1 daemons安装

所有节点都执行

cloudera-manager-daemons是cm server和agent的基础，要先进性安装
yum  -y localinstall cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm

ansible -i /root/host/cdh all -m shell -a "yum  -y localinstall /root/cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm"

4.1.4.2 server安装

master节点执行

yum -y localinstall cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm

修改配置
vim /etc/cloudera-scm-server/db.properties

# Copyright (c) 2012 Cloudera, Inc. All rights reserved.
#
# This file describes the database connection.
#

# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com.cloudera.cmf.db.type=mysql

# The database host
# If a non standard port is needed, use 'hostname:port'
com.cloudera.cmf.db.host=cdh-master

# The database name
com.cloudera.cmf.db.name=cm

# The database user
com.cloudera.cmf.db.user=cm

# The database user's password
com.cloudera.cmf.db.password=2wsx#ED

# The db setup type
# After fresh install it is set to INIT
# and will be changed post config.
# If scm-server uses Embedded DB then it is set to EMBEDDED
# If scm-server uses External DB then it is set to EXTERNAL
com.cloudera.cmf.db.setupType=EXTERNAL
=========================================================================================
db和user查看mysql数据库
[root@cdh001-master ~]#  mysql -uroot -p2wsx#EDC
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 63
Server version: 5.7.16 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+---------------------------+
| Tables_in_mysql           |
+---------------------------+
| columns_priv              |
| db                        |
| engine_cost               |
| event                     |
| func                      |
| general_log               |
| gtid_executed             |
| help_category             |
| help_keyword              |
| help_relation             |
| help_topic                |
| innodb_index_stats        |
| innodb_table_stats        |
| ndb_binlog_index          |
| plugin                    |
| proc                      |
| procs_priv                |
| proxies_priv              |
| server_cost               |
| servers                   |
| slave_master_info         |
| slave_relay_log_info      |
| slave_worker_info         |
| slow_log                  |
| tables_priv               |
| time_zone                 |
| time_zone_leap_second     |
| time_zone_name            |
| time_zone_transition      |
| time_zone_transition_type |
| user                      |
+---------------------------+
31 rows in set (0.00 sec)
mysql> select user,host from user;
+-----------+-----------+
| user      | host      |
+-----------+-----------+
| amon      | %         |
| cm        | %         |
| hive      | %         |
| hue       | %         |
| nav       | %         |
| navms     | %         |
| oos       | %         |
| rman      | %         |
| sentry    | %         |
| mysql.sys | localhost |
| root      | localhost |
+-----------+-----------+
11 rows in set (0.00 sec)
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| amon               |
| cm                 |
| hive               |
| hue                |
| metastore          |
| mysql              |
| nav                |
| navms              |
| oos                |
| performance_schema |
| rman               |
| sentry             |
| sys                |
+--------------------+
14 rows in set (0.00 sec)

4.1.4.3 agent安装

所有节点都执行

yum  -y localinstall cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm

[root@cdh001-master ~]# vim /etc/cloudera-scm-agent/config.ini
```
[General]
# Hostname of the CM server.
server_host=cdh-master

# Port that the CM server is listening on.
server_port=7182
```

4.1.4.4 CDH包准备

master节点

#CDH包放在/opt/cloudera/parcel-repo

[root@cdh001-master parcel-repo]# ll
total 2033432
-rwxr-xr-x. 1 root root 2082186246 Oct 11 05:08 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
-rwxr-xr-x. 1 root root         40 Oct 11 05:08 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
-rwxr-xr-x. 1 root root         64 Oct 11 05:08 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
-rwxr-xr-x. 1 root root      33887 Oct 11 05:08 manifest.json
[root@cdh001-master parcel-repo]# pwd
/opt/cloudera/parcel-repo

#CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1重命名为CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
[root@cdh001-master parcel-repo]# mv  ./CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 ./CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

#CDH包http配置
mkdir -p /var/www/html/parcel-repo
cd  /var/www/html/parcel-repo
cp /opt/cloudera/parcel-repo/*  /var/www/html/parcel-repo/
createrepo .
chmod 777 -R /var/www/html/parcel-repo/*

node节点

vim cloudera-manager.repo
[cloudera-manager]
name=Cloudera Manager 6.3.2
baseurl=http://10.11.90.11/parcel-repo/
gpgcheck=0
enabled=1

yum clean all
yum makecache

4.1.4.5 初始化CM

 [root@cdh001-master schema]# ll
total 60
drwxr-xr-x. 4 root root  8192 Oct 11 04:47 mysql
drwxr-xr-x. 4 root root  8192 Oct 11 04:47 oracle
drwxr-xr-x. 4 root root 12288 Oct 11 04:47 postgresql
-rw-r--r--. 1 root root  1437 Sep 25  2019 scm_database_functions.sh
-rwxr-xr-x. 1 root root 12450 Sep 25  2019 scm_prepare_database.sh
[root@cdh001-master schema]# pwd
/opt/cloudera/cm/schema
[root@cdh001-master schema]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql cm cm 2wsx#EDC

cd /opt/cloudera/cm/schema
./scm_prepare_database.sh -h 10.11.90.11 -P 3306 mysql cm cm 2wsx#EDC
 #参数说明： -h MysqlHost -P MysqlPort dbType dbName dbUser dbPasswd
 
 [root@cdh001-master schema]# ./scm_prepare_database.sh -h 10.11.90.11 -P 3306 mysql cm cm 2wsx#EDC
JAVA_HOME=/usr/local/java/jdk1.8.0_301
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/local/java/jdk1.8.0_301/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Mon Oct 11 23:19:23 EDT 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

报错:

Establishing SSL connection without server’s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn’t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false’. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification

看报错信息是mysql5版本以上默认是开启SSL，可以查看SSL状态，并在my.cnf修改配置关闭SSL

[root@cdh001-master schema]# ./scm_prepare_database.sh -h 10.11.90.11 -P 3306 mysql cm root 2wsx#EDC
JAVA_HOME=/usr/local/java/jdk1.8.0_301
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/local/java/jdk1.8.0_301/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Mon Oct 11 22:38:17 EDT 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[                          main] DbCommandExecutor              ERROR Error when connecting to database.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 439 milliseconds ago.  The last packet sent successfully to the server was 428 milliseconds ago.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.8.0_301]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)[:1.8.0_301]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_301]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_301]
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:201)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4869)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1656)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1217)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2189)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2220)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2015)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:768)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)[mysql-connector-java.jar:5.1.48]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)[:1.8.0_301]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)[:1.8.0_301]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_301]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_301]
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:385)[mysql-connector-java.jar:5.1.48]
	at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:323)[mysql-connector-java.jar:5.1.48]
	at java.sql.DriverManager.getConnection(DriverManager.java:664)[:1.8.0_301]
	at java.sql.DriverManager.getConnection(DriverManager.java:247)[:1.8.0_301]
	at com.cloudera.enterprise.dbutil.DbCommandExecutor.testDbConnection(DbCommandExecutor.java:263)[db-common-6.3.1.96818eaab0a222aa84a7854b8d22c0c7.jar:]
	at com.cloudera.enterprise.dbutil.DbCommandExecutor.main(DbCommandExecutor.java:139)[db-common-6.3.1.96818eaab0a222aa84a7854b8d22c0c7.jar:]
[                          main] DbCommandExecutor              ERROR Exiting with exit code 4
--> Error 4, giving up (use --force if you wish to ignore the error)

解决办法：

[root@cdh001-master ~]# mysql -u cm -p2wsx#EDC
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 73
Server version: 5.7.16 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW VARIABLES LIKE '%ssl%';
+---------------+-----------------+
| Variable_name | Value           |
+---------------+-----------------+
| have_openssl  | YES             |
| have_ssl      | YES             |
| ssl_ca        | ca.pem          |
| ssl_capath    |                 |
| ssl_cert      | server-cert.pem |
| ssl_cipher    |                 |
| ssl_crl       |                 |
| ssl_crlpath   |                 |
| ssl_key       | server-key.pem  |
+---------------+-----------------+
9 rows in set (0.01 sec)

mysql> 

[root@cdh001-master ~]# vim /etc/my.cnf
#===关闭ssl====
# disable_ssl
skip_ssl
[root@cdh001-master ~]# systemctl restart mysqld
[root@cdh001-master ~]# mysql -u cm -p2wsx#EDC
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.16 MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW VARIABLES LIKE '%ssl%';
+---------------+----------+
| Variable_name | Value    |
+---------------+----------+
| have_openssl  | DISABLED |
| have_ssl      | DISABLED |
| ssl_ca        |          |
| ssl_capath    |          |
| ssl_cert      |          |
| ssl_cipher    |          |
| ssl_crl       |          |
| ssl_crlpath   |          |
| ssl_key       |          |
+---------------+----------+
9 rows in set (0.01 sec)

4.1.4.6 启动CM

4.1.4.6.1 启动agent

所有节点执行

[root@ansible ~]# ansible -i /root/host/cdh all -m shell -a "systemctl restart cloudera-scm-agent"
systemctl restart cloudera-scm-agent

systemctl enable cloudera-scm-agent

4.1.4.6.2 启动server

master节点执行

systemctl restart cloudera-scm-server
systemctl enable cloudera-scm-server

# 另开一个窗口，查看相关日志。有异常就解决异常
tail -200f /var/log/cloudera-scm-server/cloudera-scm-server.log

等待数据库的表都创建完成，显示web可以访问，此时可以访问chd
url:
http://10.11.90.11:7180/
默认账号:
admin/admin

报错1：

SqlExceptionHelper - Table ‘scm.CM_VERSION’ doesn’t exist

#linux系统，mysql修改为大小写不敏感，修改/etc/my.conf文件中 [mysqld] 的内容
vim /etc/my.conf
[mysqld]
lower_case_table_names=1

systemctl restart mysqld

#删除数据库，重新创建，重新初始化cm数据库

cd /opt/cloudera/cm/schema
./scm_prepare_database.sh -h 10.11.90.11 -P 3306 mysql cm cm 2wsx#EDC
 #参数说明： -h MysqlHost -P MysqlPort dbType dbName dbUser dbPasswd
 
 #重新启动agent和server

报错2：

[root@cdh-master java]# systemctl status cloudera-scm-server.service
● cloudera-scm-server.service - Cloudera CM Server Service
   Loaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Wed 2021-11-17 21:48:36 EST; 28s ago
  Process: 27782 ExecStart=/opt/cloudera/cm/bin/cm-server (code=exited, status=1/FAILURE)
  Process: 27779 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)
 Main PID: 27782 (code=exited, status=1/FAILURE)

Nov 17 21:48:36 cdh-master systemd[1]: cloudera-scm-server.service failed.
Nov 17 21:48:36 cdh-master systemd[1]: cloudera-scm-server.service holdoff time over, scheduling restart.
Nov 17 21:48:36 cdh-master systemd[1]: Stopped Cloudera CM Server Service.
Nov 17 21:48:36 cdh-master systemd[1]: start request repeated too quickly for cloudera-scm-server.service
Nov 17 21:48:36 cdh-master systemd[1]: Failed to start Cloudera CM Server Service.
Nov 17 21:48:36 cdh-master systemd[1]: Unit cloudera-scm-server.service entered failed state.
Nov 17 21:48:36 cdh-master systemd[1]: cloudera-scm-server.service failed.
Nov 17 21:48:38 cdh-master systemd[1]: start request repeated too quickly for cloudera-scm-server.service
Nov 17 21:48:38 cdh-master systemd[1]: Failed to start Cloudera CM Server Service.
Nov 17 21:48:38 cdh-master systemd[1]: cloudera-scm-server.service failed.
====================================================================================

[root@cdh-master java]# journalctl -xe
Nov 17 21:56:37 cdh-master systemd[1]: Started Cloudera CM Server Service.
-- Subject: Unit cloudera-scm-server.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit cloudera-scm-server.service has finished starting up.
-- 
-- The start-up result is done.
Nov 17 21:56:37 cdh-master cm-server[28249]: +======================================================================+
Nov 17 21:56:37 cdh-master cm-server[28249]: |      Error: Unable to find a compatible version of Java on this host,|
Nov 17 21:56:37 cdh-master cm-server[28249]: |             either because JAVA_HOME has not been set or because a   |
Nov 17 21:56:37 cdh-master cm-server[28249]: |             compatible version of Java is not installed.             |
Nov 17 21:56:37 cdh-master cm-server[28249]: +----------------------------------------------------------------------+
Nov 17 21:56:37 cdh-master cm-server[28249]: | Please install either:                                               |
Nov 17 21:56:37 cdh-master cm-server[28249]: | - a supported version of the Oracle JDK from the Oracle Java web     |
Nov 17 21:56:37 cdh-master cm-server[28249]: |   site:                                                              |
Nov 17 21:56:37 cdh-master cm-server[28249]: |   > http://www.oracle.com/technetwork/java/javase/index.html <       |
Nov 17 21:56:37 cdh-master cm-server[28249]: | OR                                                                   |
Nov 17 21:56:37 cdh-master cm-server[28249]: | - a supported version of the OpenJDK from your OS vendor.  Help for  |
Nov 17 21:56:37 cdh-master cm-server[28249]: |   some OSes are available at:                                        |
Nov 17 21:56:37 cdh-master cm-server[28249]: |   > http://openjdk.java.net/install/ <                               |
Nov 17 21:56:37 cdh-master cm-server[28249]: |                                                                      |
Nov 17 21:56:37 cdh-master cm-server[28249]: | Cloudera Manager requires Oracle JDK or OpenJDK 1.8 or later.        |
Nov 17 21:56:37 cdh-master cm-server[28249]: | NOTE: Cloudera Manager will find the Oracle JDK when starting,       |
Nov 17 21:56:37 cdh-master cm-server[28249]: |       regardless of whether you installed the JDK using a binary     |
Nov 17 21:56:37 cdh-master cm-server[28249]: |       installer or the RPM-based installer.                          |
Nov 17 21:56:37 cdh-master cm-server[28249]: +======================================================================+
Nov 17 21:56:37 cdh-master systemd[1]: cloudera-scm-server.service: main process exited, code=exited, status=1/FAILURE
Nov 17 21:56:37 cdh-master systemd[1]: Unit cloudera-scm-server.service entered failed state.
Nov 17 21:56:37 cdh-master systemd[1]: cloudera-scm-server.service failed.
Nov 17 21:56:37 cdh-master systemd[1]: cloudera-scm-server.service holdoff time over, scheduling restart.
Nov 17 21:56:37 cdh-master systemd[1]: Stopped Cloudera CM Server Service.
-- Subject: Unit cloudera-scm-server.service has finished shutting down

在这里插入图片描述

解决方法

#然后通过命令journalctl -xe发现了一些端倪，提示JAVA_HOME找不到，但是我明明已经安装过了jdk，使用java -version也可以正常列出版本信息，怎么还会找不到呢？之后在一个脚本文件中找到了些答案，原来程序默认会去使用/usr/java下的jdk，所以解决办法执行以下两条命令即可：
mkdir -p /usr/java
cd jdk1.8.0_301/
ln -s /usr/local/java/jdk1.8.0_301 /usr/java/default

5、集群安装

5.1 安装

依据步骤进行安装，集群安装完可以依据知己的情况安装需要的组件

5.2 问题：

在install parcels时候各个节点一直报红
在这里插入图片描述

解决办法：

主机运行状况不良的错误-这是由于server和agent的id标识不一致导致的

删除agent的cm_guid并重启（有问题的都进行这个操作)
rm -rf /var/lib/cloudera-scm-agent/cm_guid
systemctl restart cloudera-scm-agent

在这里插入图片描述

奋进的小马

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
CM和CDH安装

CDH安装文章目录CDH安装1、虚拟机配置2、安装包和yum准备2.1 yum配置2.2 安装准备3、前置操作3.1 host配置3.1.1 修改hostname3.1.2 修改hosts文件(所有节点都一致)3.2 防火墙配置3.2.1 关闭防火墙3.2.2 关闭selinux3.3 免密登陆配置3.4 NTP配置3.4.1 NTP client安装3.4 JDK安装3.4.1 检查并卸载自带jdk3.4.2 安装jdk3.5 安装http和repo3.6 安装mysql3.6.1 安装mysql8的驱
复制链接

扫一扫