安装环境
cdh-6.3.2
服务器环境 centos7.9
name | ip |
master.hadoop | 192.168.10.187 |
node1.hadoop | 192.168.10.188 |
node2.hadoop | 192.168.10.189 |
node3.hadoop | 192.168.10.190 |
第一步
注意把dns解析做好
或者更改etc文件把所有的服务器的/etc/hosts文件增加手工解析
第二步
关闭selinux 和防火墙,每台机器上都要做
systemctl stop firewalld
systemctl disable firewalld
sed -i '/SELINUX/s/enforcing/disabled/g' /etc/selinux/config
cat /etc/selinux/config
第三步
做免密
在master机器上生成密钥,通过ssh-copy-id拷贝到四台机器
[root@master ~]#ssh_keygen
# 一直回车
[root@master ~]# ssh-copy-id node{1-3}.hadoop
第四步 安装时间同步
yum -y install ntpdate
ntpdate cn.pool.ntp.org
第五步 安装jdk
yum -y install java java-devel
第六步,使用swap
sysctl -w vm.swappiness=0
echo 'vm.swappiness=0' >> /etc/sysctl.conf
sysctl -p
第7步 ,在master机器上安装mysql5.7
[root@master ~]# wget https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm
--2022-12-31 20:40:10-- https://dev.mysql.com/get/mysql80-community-release-el7-3.noarch.rpm
Resolving dev.mysql.com (dev.mysql.com)... 23.63.37.189, 2600:140b:2:99d::2e31, 2600:140b:2:99c::2e31
Connecting to dev.mysql.com (dev.mysql.com)|23.63.37.189|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm [following]
--2022-12-31 20:40:11-- https://repo.mysql.com//mysql80-community-release-el7-3.noarch.rpm
Resolving repo.mysql.com (repo.mysql.com)... 23.39.0.221
Connecting to repo.mysql.com (repo.mysql.com)|23.39.0.221|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26024 (25K) [application/x-redhat-package-manager]
Saving to: ‘mysql80-community-release-el7-3.noarch.rpm.1’
100%[===============================================================================================>] 26,024 162KB/s in 0.2s
[root@master ~]# rpm -ivh mysql80-community-release-el7-3.noarch.rpm
warning: mysql80-community-release-el7-3.noarch.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:mysql80-community-release-el7-3 ################################# [100%]
[root@master ~]#
让5.7 mysql生效
vi /etc/yum.repos.d/mysql-community.repo
# Enable to use MySQL 5.7
[mysql57-community]
name=MySQL 5.7 Community Server
baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/7/$basearch/
enabled=1 # 改成1
gpgcheck=0 # 改成0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
[mysql80-community]
name=MySQL 8.0 Community Server
baseurl=http://repo.mysql.com/yum/mysql-8.0-community/el/7/$basearch/
enabled=0 改成0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
然后安装mysql
[root@master ~]#yum -y install mysql-community-server
....
[root@master ~]#systemctl start mysqld
[root@master ~]#systemctl enable mysqld
[root@master ~]# cat /var/log/mysqld.log |grep password
2022-12-31T12:48:20.149714Z 1 [Note] A temporary password is generated for root@localhost: 2p#GJBWtg1g4
正常安装完毕,初始化cdh所需要的账户名和密码.
mysql5.7/8.0 需要修改初始密码才能执行创建用户
mysql> alter user user() identified by 'Root@123';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql>
执行一下脚本,把下面脚本保存成jiaoben.sql
create database scm DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database rman DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database metastore DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database sentry DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database nav DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database navms DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
grant all on scm.* to 'scm'@'%' identified by 'Root@123';
grant all on hive.* to 'hive'@'%' identified by 'Root@123';
grant all on amon.* to 'amon'@'%' identified by 'Root@123';
grant all on rman.* to 'rman'@'%' identified by 'Root@123';
grant all on hue.* to 'hue'@'%' identified by 'Root@123';
grant all on metastore.* to 'hive'@'%' identified by 'Root@123';
grant all on sentry.* to 'sentry'@'%' identified by 'Root@123';
grant all on nav.* to 'nav'@'%' identified by 'Root@123';
grant all on navms .* to navms @'%' identified by 'Root@123';
grant all on oozie.* to 'oozie'@'%' identified by 'Root@123';
flush privileges;
修改密码并执行jiaoben.sql
[root@master ~]# mysql -uroot -p"2p#GJBWtg1g4"
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.7.40
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> alter user user() identified by 'Root@123';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
[root@master ~]# mysql -uroot -p"Root@123" < jiaoben.sql
第八步 建立仓库
建立yum源方便安装cdh。
建立仓库的主要代码是创建nginx一个虚拟网站,然后创建仓库。
mkdir -p /www/wwwroot/yum.in.io/cm6
将下载的cm包文件移到此目录下: /www/wwwroot/yum.in.io/cm6
包含这些文件:
cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
oracle-j2sdk1.8-1.8.0update181-1.x86_64.rpm
cd /www/wwwroot/yum.in.io/cm6
创建仓库: createrepo .
如无此命令: yum install createrepo 后再执行 createrepo .
成功后会出现:repodata文件夹
在master 上创建仓库文件
tee <<EOF > /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name=cloudera-manager
baseurl=http://yum.in.io/cm6/
enabled=1
gpgcheck=0
EOF
在master上执行
yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
把以下文件 移动到 /opt/cloudera/parcel-repo
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
初始化mysql脚本
opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm Root@123
发现没有mysql驱动,下载mysql驱动,再执行代码完成。
[root@master parcel-repo]# cp -i ~/mysql-connector-java-5.1.49-bin.jar /opt/cloudera/cm/lib/
[root@master parcel-repo]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm Root@123 JAVA_HOME=/usr/lib/jvm/java-openjdk
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/lib/jvm/java-openjdk/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Sat Dec 31 21:22:32 CST 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!
[root@master parcel-repo]#
至此进入模块安装就完成了