自测–CDH6.3.1 搭建 (2核16G阿里云环境)
1.准备工作
1.1部署方式
本次测试采用离线部署的方式 :
主要分为三块 :
- Cloudera-Manager离线部署
- MySQL离线部署
- Parcel文件离线源部署
1.2集群规划
| 节点 | MySQL部署组件 | Parcel⽂文件离线源 | CM服务进程 | 大数据组件 |
|---|---|---|---|---|
| hadoop001 | MySQL | Parcel | Activity Monitor | NN RM DN NM |
| hadoop002 | Alert Publisher Event Server | DN NM | ||
| hadoop003 | Host Monitor Service Monitor | DN NM |
1.3下载源
-
CM
-
Parcel
-
JDK
-
MySQL
-
MySQL jdbc jar
-
下载完成后要重命名去掉版本号:
mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar
2.集群节点初始化
2.1购买阿里云服务器
-
按量量付费虚拟机
-
CentOS7.2操作系统，2核16G最低配置(8G 勉强也可以)
-
-
-
-
-
-
-
-
2.2当前机器配置hosts文件
- Window: C:\windows\System32\drivers\etc\hosts
- ps : hosts文件里面只放公网ip
# 私人阿里云环境
# 公网
161.xxx.52.206 hadoop001
161.xxx.0.68 hadoop002
47.xx.214.28 hadoop003
# 内网
172.21.102.204 hadoop001
172.21.102.202 hadoop002
172.21.102.203 hadoop003
2.3设置所有节点的hosts文件
echo "172.21.102.204 hadoop001">> /etc/hosts
echo "172.21.102.202 hadoop002">> /etc/hosts
echo "172.21.102.203 hadoop003">> /etc/hosts
-
-
查看主机之前能否ping通
2.4关闭所有节点的防⽕火墙及清空规则
systemctl stop firewalld
systemctl disable firewalld
iptables -F
2.5关闭所有节点的selinux
阿里云默认是disabled的
vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能⽣生效
2.6设置所有节点的时区⼀一致及时钟同步 (云主机不用配置)
[root@hadoop001 /]# date
Sun Jun 28 16:04:00 CST 2020
[root@hadoop001 /]# timedatectl
Local time: Sun 2020-06-28 16:04:04 CST
Universal time: Sun 2020-06-28 08:04:04 UTC
RTC time: Sun 2020-06-28 16:04:02
Time zone: Asia/Shanghai (CST, +0800)
NTP enabled: yes
NTP synchronized: no
RTC in local TZ: yes
DST active: n/a
Warning: The system is configured to read the RTC time in the local time zone.
This mode can not be fully supported. It will create various problems
with time zone changes and daylight saving time adjustments. The RTC
time is never updated, it relies on external facilities to maintain it.
If at all possible, use RTC in UTC by calling
'timedatectl set-local-rtc 0'.
#查看哪些时区
[root@hadoop001 /]# timedatectl list-timezones
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
Africa/Bamako
...
#所有节点设置亚洲上海海时区
[root@hadoop001 /]# timedatectl set-timezone Asia/Shanghai
[root@hadoop002 /]# timedatectl set-timezone Asia/Shanghai
[root@hadoop003 /]# timedatectl set-timezone Asia/Shanghai
#所有节点安装ntp
[root@hadoop001 ~]# yum install -y ntp
[root@hadoop002 ~]# yum install -y ntp
[root@hadoop003 ~]# yum install -y ntp
#选取hadoop001为ntp的主节点
[root@hadoop001 ~]# vi /etc/ntp.conf
-
增加
-
#当外部时间不不可⽤用时，可使⽤用本地硬件时间 server 127.127.1.0 iburst local clock #允许哪些网段的机器来同步时间 restrict 172.21.102.0 mask 255.255.255.0 nomodify notrap
# 开启ntpd
[root@hadoop001 /]# systemctl start ntpd
# 查看状态
[root@hadoop001 /]# systemctl status ntpd
● ntpd.service - Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2020-06-28 16:08:45 CST; 6min ago
Main PID: 15948 (ntpd)
CGroup: /system.slice/ntpd.service
└─15948 /usr/sbin/ntpd -u ntp:ntp -g
Jun 28 16:08:45 hadoop001 systemd[1]: Starting Network Time Service...
Jun 28 16:08:45 hadoop001 ntpd[15948]: proto: precision = 0.185 usec
Jun 28 16:08:45 hadoop001 ntpd[15948]: 0.0.0.0 c01d 0d kern kernel time sync enabled
Jun 28 16:08:45 hadoop001 systemd[1]: Started Network Time Service.
# 验证
[root@hadoop001 /]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
LOCAL(0) .LOCL. 10 l 401 64 100 0.000 0.000 0.000
-120.25.115.20 10.137.53.7 2 u 3 64 177 47.747 -226.06 9.561
10.143.33.49 .INIT. 16 u - 1024 0 0.000 0.000 0.000
-100.100.3.1 10.137.55.181 2 u 1 64 177 73.610 -224.33 0.021
-100.100.3.2 10.137.55.181 2 u 31 64 377 77.180 -226.06 0.199
+100.100.3.3 10.137.55.181 2 u 30 64 377 73.864 -228.00 0.015
-203.107.6.88 10.137.38.86 2 u 25 64 377 82.009 -230.21 7.446
10.143.33.50 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.33.51 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.44 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.45 .INIT. 16 u - 1024 0 0.000 0.000 0.000
10.143.0.46 .INIT. 16 u - 1024 0 0.000 0.000 0.000
+100.100.5.1 10.137.55.181 2 u 42 64 177 73.779 -227.82 0.191
-100.100.5.2 10.137.55.181 2 u 21 64 377 74.274 -227.93 0.109
-100.100.5.3 10.137.55.181 2 u 17 64 377 73.638 -224.35 0.030
*100.100.61.88 .BD. 1 u 36 64 177 12.647 -226.23 0.035
[root@hadoop001 /]#
#其他从节点停⽌止禁⽤用ntpd服务 hadoop002
[root@hadoop002 ~]# systemctl stop ntpd
[root@hadoop002 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop002 ~]# /usr/sbin/ntpdate hadoop001
28 Jun 16:17:15 ntpdate[9951]: adjust time server 172.21.102.204 offset -0.133138 sec
[root@hadoop002 ~]#
#每天凌晨同步hadoop001节点时间
[root@hadoop002 ~]# crontab -e
00 00 * * * /usr/sbin/ntpdate hadoop001
#其他从节点停⽌止禁⽤用ntpd服务 hadoop003
[root@hadoop003 ~]# systemctl stop ntpd
[root@hadoop003 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop003 ~]# /usr/sbin/ntpdate hadoop001
28 Jun 16:19:18 ntpdate[10000]: adjust time server 172.21.102.204 offset -0.047496 sec
[root@hadoop003 ~]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
[root@hadoop003 ~]#
00 00 * * * /usr/sbin/ntpdate hadoop001
2.7每个节点部署JDK
-
先把所有软件上传到 hadoop001当中
-
[root@hadoop001 CDH6.3.1]# ll total 4131080 -rw-r--r-- 1 root root 2083878000 Jun 28 16:26 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel -rw-r--r-- 1 root root 40 Jun 28 16:26 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 -rw-r--r-- 1 root root 1411444147 Jun 28 16:28 cm6.3.1-redhat7.tar.gz -rw-r--r-- 1 root root 185646832 Jun 28 16:28 jdk-8u181-linux-x64.tar.gz -rw-r--r-- 1 root root 33887 Jun 28 16:28 manifest.json -rw-r--r-- 1 root root 548193637 Jun 28 16:29 mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -rw-r--r-- 1 root root 1007502 Jun 28 16:29 mysql-connector-java-5.1.47.jar
-
-
其他节点创建目录
-
[root@hadoop002 ~]# mkdir CDH6.3.1 [root@hadoop003 ~]# mkdir CDH6.3.1
-
-
传输JDK
-
[root@hadoop001 CDH6.3.1]# scp jdk-8u181-linux-x64.tar.gz hadoop002:/root/CDH6.3.1/ [root@hadoop001 CDH6.3.1]# scp jdk-8u181-linux-x64.tar.gz hadoop003:/root/CDH6.3.1/ # 输入yes和root账号的密码
-
-
所有节点创建目录
-
mkdir /usr/java
-
-
所有节点解压JDK包
-
tar -xzvf jdk-8u181-linux-x64.tar.gz -C /usr/java
-
-
所有节点修正用户和用户组
-
cd /usr/java chown -R root:root /usr/java/jdk1.8.0_181
-
-
所有节点配置环境变量
-
echo "export JAVA_HOME=/usr/java/jdk1.8.0_181" >> /etc/profile echo "export PATH=/usr/java/jdk1.8.0_181/bin:${PATH}" >> /etc/profile source /etc/profile which java
-
3.离线部署MySQL5.7
我们选择hadoop001为我们的mysql节点
-
解压
-
sudo tar -xzvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -C /usr/local
-
-
重新命名
-
cd /usr/local mv mysql-5.7.11-linux-glibc2.5-x86_64 mysql
-
-
修正用户组
-
chown -R root:root /usr/local/mysql/
-
-
创建文件夹
-
mkdir mysql/arch/ mysql/data/ mysql/tmp/
-
-
创建配置文件
-
vi /etc/my.cnf把默认的内容删掉 替换如下
-
[client] port = 3306 socket = /usr/local/mysql/data/mysql.sock default-character-set=utf8mb4 [mysqld] port = 3306 socket = /usr/local/mysql/data/mysql.sock skip-slave-start skip-external-locking key_buffer_size = 256M sort_buffer_size = 2M read_buffer_size = 2M read_rnd_buffer_size = 4M query_cache_size= 32M max_allowed_packet = 16M myisam_sort_buffer_size=128M tmp_table_size=32M table_open_cache = 512 thread_cache_size = 8 wait_timeout = 86400 interactive_timeout = 86400 max_connections = 600 # Try number of CPU's*2 for thread_concurrency #thread_concurrency = 32 #isolation level and default engine default-storage-engine = INNODB transaction-isolation = READ-COMMITTED server-id = 1739 basedir = /usr/local/mysql datadir = /usr/local/mysql/data pid-file = /usr/local/mysql/data/hostname.pid #open performance schema log-warnings sysdate-is-now binlog_format = ROW log_bin_trust_function_creators=1 log-error = /usr/local/mysql/data/hostname.err log-bin = /usr/local/mysql/arch/mysql-bin expire_logs_days = 7 innodb_write_io_threads=16 relay-log = /usr/local/mysql/relay_log/relay-log relay-log-index = /usr/local/mysql/relay_log/relay-log.index relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info log_slave_updates=1 gtid_mode=OFF enforce_gtid_consistency=OFF # slave slave-parallel-type=LOGICAL_CLOCK slave-parallel-workers=4 master_info_repository=TABLE relay_log_info_repository=TABLE relay_log_recovery=ON #other logs #general_log =1 #general_log_file = /usr/local/mysql/data/general_log.err #slow_query_log=1 #slow_query_log_file=/usr/local/mysql/data/slow_log.err #for replication slave sync_binlog = 500 #for innodb options innodb_data_home_dir = /usr/local/mysql/data/ innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend innodb_log_group_home_dir = /usr/local/mysql/arch innodb_log_files_in_group = 4 innodb_log_file_size = 1G innodb_log_buffer_size = 200M #根据生产需要，调整pool size innodb_buffer_pool_size = 2G #innodb_additional_mem_pool_size = 50M #deprecated in 5.6 tmpdir = /usr/local/mysql/tmp innodb_lock_wait_timeout = 1000 #innodb_thread_concurrency = 0 innodb_flush_log_at_trx_commit = 2 innodb_locks_unsafe_for_binlog=1 #innodb io features: add for mysql5.5.8 performance_schema innodb_read_io_threads=4 innodb-write-io-threads=4 innodb-io-capacity=200 #purge threads change default(0) to 1 for purge innodb_purge_threads=1 innodb_use_native_aio=on #case-sensitive file names and separate tablespace innodb_file_per_table = 1 lower_case_table_names=1 [mysqldump] quick max_allowed_packet = 128M [mysql] no-auto-rehash default-character-set=utf8mb4 [mysqlhotcopy] interactive-timeout [myisamchk] key_buffer_size = 256M sort_buffer_size = 256M read_buffer = 2M write_buffer = 2M
-
-
创建用户组及用户
-
groupadd -g 101 dba useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
-
-
copy配置文件
-
cp /etc/skel/.* /usr/local/mysql
-
-
切换用户
-
su - mysqladmin ll -a #查看文件是否copy exit
-
-
编辑配置文件
-
vi ./bashrc -
# .bashrc # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # Uncomment the following line if you don't like systemctl's auto-paging feature: # export SYSTEMD_PAGER= export MYSQL_BASE=/usr/local/mysql export PATH=${MYSQL_BASE}/bin:$PATH
-
-
赋予权限 记得切换为root
-
[root@hadoop001 local]# chown mysqladmin:dba /etc/my.cnf [root@hadoop001 local]# chmod 640 /etc/my.cnf [root@hadoop001 local]# chown -R mysqladmin:dba /usr/local/mysql [root@hadoop001 local]# chmod -R 755 /usr/local/mysql
-
-
配置服务和开机自启动
-
[root@hadoop001 local]# cd /usr/local/mysql #将服务文件拷贝到init.d下，并重命名为mysql [root@hadoop001 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysql #赋予可执行权限 [root@hadoop001 mysql]# chmod +x /etc/rc.d/init.d/mysql #删除服务 [root@hadoop001 mysql]# chkconfig --del mysql #添加服务 [root@hadoop001 mysql]# chkconfig --add mysql
-
-
安装libaio及安装mysql的初始db
-
[hejiahao@hadoop001 mysql]$ sudo yum -y install libaio # 初始化启动 [root@hadoop001 mysql]# bin/mysqld \ > --defaults-file=/etc/my.cnf \ > --user=mysqladmin \ > --basedir=/usr/local/mysql/ \ > --datadir=/usr/local/mysql/data/ \ > --initialize [root@hadoop001 mysql]# [root@hadoop001 mysql]# [root@hadoop001 mysql]# cd data/ # 查看临时密码 [root@hadoop001 data]# cat hostname.err |grep password 2020-06-28T10:00:55.318092Z 1 [Note] A temporary password is generated for root@ localhost: Wwh*do<PE1sH [root@hadoop001 data]#
-
-
启动并登陆
-
su - mysqladmin service mysql start -
mysql -uroot -p Wwh*do<PE1sH
-
-
重置密码
-
alter user root@localhost identified by 'hadoop'; GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' ; flush privileges; exit;mysql> show databases; ERROR 1820 (HY000): You must reset your password using ALTER USER statement before executing this statement. mysql> alter user root@localhost identified by 'hadoop'; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' ; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit; Bye [mysqladmin@hadoop001 ~]$ # 再次登陆 mysql -uroot -p hadoop
-
-
创建CDH的元数据库和⽤用户、amon服务的数据库及⽤用户
-
create database cmf DEFAULT CHARACTER SET utf8; grant all on cmf.* TO 'cmf'@'%' IDENTIFIED BY 'cmf'; create database amon DEFAULT CHARACTER SET utf8; grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon'; flush privileges;
-
-
hadoop001节点部署mysql jdbc jar
-
cd ~ mkdir -p /usr/share/java/ # 不要版本号 cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
-
4.CDH部署
4.1离线部署cm server及agent
-
解压rpm包并安装server
-
[root@hadoop001 CDH6.3.1]# tar -xzvf cm6.3.1-redhat7.tar.gz -
cd cm6.3.1 cd RPMS/ cd x86_64/ [root@hadoop001 x86_64]# ll total 1380424 -rw-r--r-- 1 2001 2001 10483568 Sep 25 2019 cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm -rw-r--r-- 1 2001 2001 1203832464 Sep 25 2019 cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm -rw-r--r-- 1 2001 2001 11488 Sep 25 2019 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm # 官方提供的数据库 (不用) : 因为你不知道你安装在哪里 也不知道密码 -rw-r--r-- 1 2001 2001 10996 Sep 25 2019 cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm -rw-r--r-- 1 2001 2001 14209868 Sep 25 2019 enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm -rw-rw-r-- 1 2001 2001 184988341 Sep 25 2019 oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm [root@hadoop001 x86_64]# server需要安装: cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm agent需要安装: cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm -
# 裸装 不用yum源 rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force [root@hadoop001 x86_64]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force warning: cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY Preparing... ################################# [100%] Updating / installing... 1:cloudera-manager-daemons-6.3.1-14################################# [100%] [root@hadoop001 x86_64]# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force warning: cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY Preparing... ################################# [100%] Updating / installing... 1:cloudera-manager-server-6.3.1-146####################### ########## [100%] Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-server.service to /usr/lib/systemd/system/cloudera-scm-server.service. [root@hadoop001 x86_64]# [root@hadoop001 cloudera-scm-server]# cd /etc/cloudera-scm-server/ [root@hadoop001 cloudera-scm-server]# ll total 20 -rw-r--r-- 1 root root 8197 Sep 25 2019 cmSubCaCert.profile -rw------- 1 cloudera-scm cloudera-scm 757 Sep 25 2019 db.properties -rw-r--r-- 1 root root 3027 Sep 25 2019 log4j.properties [root@hadoop001 cloudera-scm-server]# -
vi db.properties # Copyright (c) 2012 Cloudera, Inc. All rights reserved. # # This file describes the database connection. # # The database type # Currently 'mysql', 'postgresql' and 'oracle' are valid databases. com.cloudera.cmf.db.type=mysql # The database host # If a non standard port is needed, use 'hostname:port' com.cloudera.cmf.db.host=hadoop001:3306 # The database name com.cloudera.cmf.db.name=cmf # The database user com.cloudera.cmf.db.user=cmf # The database user's password com.cloudera.cmf.db.password=cmf # The db setup type # After fresh install it is set to INIT # and will be changed post config. # If scm-server uses Embedded DB then it is set to EMBEDDED # If scm-server uses External DB then it is set to EXTERNAL com.cloudera.cmf.db.setupType=EXTERNAL
-
-
启动server
-
service cloudera-scm-server start # 需要等待一分钟 -
# 去 /var/log/cloudera-scm-server/ 下查看日志 tail -F /var/log/cloudera-scm-server/cloudera-scm-server.log -
-
去阿里云开放7180端口
-
-
- 出现登陆页面后不要动
- 出现登陆页面后不要动
-
hadoop001继续安装rpm包 先前安装了daemons和server
-
cd /root/CDH6.3.1/cm6.3.1/RPMS/x86_64 [root@hadoop001 x86_64]# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force warning: cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY Preparing... ################################# [100%] Updating / installing... 1:cloudera-manager-agent-6.3.1-1466################################# [100%] Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-agent.service to /usr/lib/systemd/system/cloudera-scm-agent.service. Created symlink from /etc/systemd/system/multi-user.target.wants/supervisord.service to /usr/lib/systemd/system/supervisord.service. [root@hadoop001 x86_64]#
-
-
编辑文件
-
vi /etc/cloudera-scm-agent/config.ini 修改 server_host=hadoop001
-
-
其他的机器安装rpm包
-
# 先传输软件包 scp cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm hadoop002:/root/CDH6.3.1/ scp cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm hadoop003:/root/CDH6.3.1/ scp cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm hadoop002:/root/CDH6.3.1/ scp cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm hadoop003:/root/CDH6.3.1/ # hadoop002 和 hadoop003 安装 rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
-
-
其他的机器编辑文件 hadoop002 和 hadoop003
-
vi /etc/cloudera-scm-agent/config.ini 修改 server_host=hadoop001
-
-
所有机器启动 agent
-
service cloudera-scm-agent start
-
5.浏览器登陆
默认账号密码
admin
admin
5.1进入欢迎页面
5.2同意许可证
5.3选择免费版本
5.4集群安装
5.5设置集群名称
5.6选择主机
5.7选择存储库
-
配置
-
cd CDH6.3.1/ yum install -y httpd mkdir /var/www/html/cdh6_parcel [root@hadoop001 CDH6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel/ # 这里记得要不sha1的1去掉 [root@hadoop001 CDH6.3.1]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha [root@hadoop001 CDH6.3.1]# mv manifest.json /var/www/html/cdh6_parcel/ [root@hadoop001 CDH6.3.1]# [root@hadoop001 CDH6.3.1]# cd /var/www/html/cdh6_parcel/ [root@hadoop001 cdh6_parcel]# ll total 2035084 -rw-r--r-- 1 root root 2083878000 Jun 28 16:26 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel -rw-r--r-- 1 root root 40 Jun 28 16:26 CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha -rw-r--r-- 1 root root 33887 Jun 28 16:28 manifest.json [root@hadoop001 cdh6_parcel]#
-
-
启动httpd 服务
-
[root@hadoop001 CDH6.3.1]# service httpd start Redirecting to /bin/systemctl start httpd.service [root@hadoop001 CDH6.3.1]#
-
-
记得阿里云开放80端口 浏览器
-
设置路径
-
-
这时候页面自动找到了我们的parcel包了
5.8安装 Parcels
5.9集群检测
-
-
显示检查器结果
-
所有机器运行
-
echo never > /sys/kernel/mm/transparent_hugepage/defrag echo never > /sys/kernel/mm/transparent_hugepage/enabled -
-
-
回到页面选择忽略警告,再点击继续
5.10群集设置
-
-
选择自定义服务
- 先选择 ZooKeeper , YARN , HDFS
- 先选择 ZooKeeper , YARN , HDFS
5.11自定义角色分配
5.12数据库设置
5.13审核更改
5.14首次运行
5.15汇总
6.解决报错
因为官方配置里的内存大小设置跟我们的不一样
生产上要设置成推荐大小, 不设置可以先点击Suppress
大部分可以抑制掉
- 解决
7.停止关闭集群
7.1先停止集群
7.2再停止Cloudera Management Service
7.3停止所有机器的Agent
service cloudera-scm-agent stop
7.4 停止Server
service cloudera-scm-server stop
7.5 停止MySQL
[root@hadoop001 CDH6.3.1]# su - mysqladmin
Last login: Sun Jun 28 18:03:00 CST 2020 on pts/0
[mysqladmin@hadoop001 ~]$ service mysql stop
Shutting down MySQL..[ OK ]
[mysqladmin@hadoop001 ~]$
7.6阿里云服务器选择停机不收费
8.卸载集群
-> 停止Cloudera Manager Server
sudo systemctl stop cloudera-scm-server
-> 停止Cloudera Manager Agent
sudo systemctl stop cloudera-scm-agent
-> 卸载Cloudera Manager Server
sudo yum remove cloudera-manager-server
-> 卸载Cloudera Manager Agent（所有Agent节点）
sudo systemctl stop supervisord
sudo yum remove 'cloudera-manager-*'
-> 清理
yum clean all
-> 清库
drop database cmf;
drop user cmf;
2223
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
