文章目录
- 1.为什么使用离线部署
- 2.准备工作
- 3.集群节点初始化
- 3. CDH部署
- 3.1 离线部署cm server及agent
- 3.2 hadoop001节点部署离线parcel源
- 3.2.1 部署离线parcel源
- 3.2.2 目录修改用户及用户组
- 3.2.3 所有节点创建软件安装目录、用户及用户组权限
- 3.2.4 hadoop001节点启动Server
- 3.2.5 所有节点启动Agent
- 3.2.6 进入Web界面进行后续操作
- 3.2.7 欢迎使用 Cloudera Manager 最终用户许可条款和条件
- 3.2.8 欢迎使⽤用Cloudera Manager--您想要部署哪个版本?选择Cloudera Express免费 版本
- 3.2.9 为CDH集群安装指导主机。选择[当前管理理的主机],全部勾选
- 3.2.10 选择存储库
- 3.2.11 集群安装
- 3.2.12 检查主机正确
- 3.2.13 自定义服务,选择部署Zookeeper、HDFS、Yarn服务
- 3.2.14 自定义角色分配
- 3.2.15 数据库设置
- 3.2.16 审改设置,默认即可
- 3.2.17 首次运行
- 3.2.18 恭喜您!安装成功!
- 3.2.19 查看主页
1.为什么使用离线部署
CDH 部署方式有三种:
1.使用bin 在线部署,这个需要集群机器能够访问外网
2.rpm 离线部署,需要下载相应依赖的rpm包,不是真正的离线部署(需要访问外网或者私服)
3.tar 真正离线部署,适合大多数场景,集群无需外网权限即可完成部署
在生产上我们如果需要集群访问外网往往要向网络部门申请权限,而且带来未知隐患,所以离线部署节能很好的解决这些问题。
2.准备工作
2.1 离线部署主要分为三个模块:
- MySQL离线部署
- CM离线部署
- Parcel⽂件离线源部署
2.2 集群规划
这里使用阿里云按量付费三台机器Centos 7
作为演示。
各节点规划如下:
节点名 | MySQL组件 | Parcel⽂件离线源 | CM服务进程 | 大数据组件 |
---|---|---|---|---|
hadoop001 | MySQL | Parcel | Alert Publisher Event Server | NN RM DN NM ZK |
hadoop001 | – | – | Alert Publisher Event Server | DN NM ZK |
hadoop001 | – | – | Host Monitor Service Monitor | DN NM ZK |
2.3 离线部署包下载
名称 | 官方下载链接 |
---|---|
CM | cloudera-manager-centos7-cm5.16.1x8664.tar.gz |
Parcel | CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 manifest.json |
Jdk | JDK8 |
MySQL | MySQL5.7 |
MySQL jdbc jar | mysql-connector-java-5.1.47.jar 下载完成后要重命名去掉版本号 mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar |
百度云打包下载:
链接: https://pan.baidu.com/s/16NTp7kqJ4K7Z8QPYt2EUGg 提取码: j7n4
3.集群节点初始化
3.1 准备三台机器
笔者是从阿里云购买的三台按年付费机器,读者可以用其他主机
3.2 修改当前笔记本或台式机配置hosts文件
- Linux/Mac: /etc/hosts
- Window: C:\windows\system32\drivers\etc\hosts
为三台主机分别设置别名
47.108.80.** hadoop001
47.108.81.** hadoop002
47.108.82.** hadoop003
3.3 设置所有节点的hosts文件
三台机器均修改hosts,配置别名以便互通,注意这里设置的是内网ip!
# 添加别名
172.24.103.2 hadoop001
172.24.103.1 hadoop002
172.24.103.0 hadoop003
3.4 关闭所有节点的防火墙及清空规则
云主机一般默认关闭了内网防火墙,需要手动去安全组进行配置
systemctl stop firewalld
systemctl disable firewalld
iptables -F
3.5 关闭所有节点的selinux
阿里云服务器已经将selinux关闭了,使用阿里云主机可以跳过本步
vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能⽣生效
3.6 设置所有节点的时区一致及时钟同步
阿里云已经将节点时区和时间做了同步,使用阿里云主机可以跳过本步
1.时区
[root@hadoop001 ~]# date
Sat Oct 5 16:11:35 CST 2019
[root@hadoop001 ~]# timedatectl
Local time: Sat 2019-10-05 16:11:45 CST
Universal time: Sat 2019-10-05 08:11:45 UTC
RTC time: Sat 2019-10-05 16:11:44
Time zone: Asia/Shanghai (CST, +0800)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: yes
DST active: n/a
#查看哪些时区
[root@hadoop001 ~]# timedatectl list-timezones
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
........
#所有节点设置亚洲上海海时区
[root@hadoop001 ~]# timedatectl set-timezone Asia/Shanghai
[root@hadoop002 ~]# timedatectl set-timezone Asia/Shanghai
[root@hadoop003 ~]# timedatectl set-timezone Asia/Shanghai
2.时间
#所有节点安装ntp
[root@hadoop001 ~]# yum install -y ntp
#选取hadoop001为ntp的主节点
[root@hadoop001 ~]# vi /etc/ntp.conf
#time
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org
#当外部时间不可用时,可使用本地硬件时间
server 127.127.1.0 iburst local clock
#允许哪些网段的机器来同步时间 修改成自己的内网网段
restrict 172.17.144.0 mask 255.255.255.0 nomodify notrap
#开启ntpd及查看状态
[root@hadoop001 ~]# systemctl start ntpd
[root@hadoop001 ~]# systemctl status ntpd
● ntpd.service - Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: d
isabled)
Active: active (running) since Sat 2019-05-11 10:15:00 CST; 11min ago
Main PID: 18518 (ntpd)
CGroup: /system.slice/ntpd.service
!"18518 /usr/sbin/ntpd -u ntp:ntp -g
May 11 10:15:00 hadoop001 systemd[1]: Starting Network Time Service...
May 11 10:15:00 hadoop001 ntpd[18518]: proto: precision = 0.088 usec
May 11 10:15:00 hadoop001 ntpd[18518]: 0.0.0.0 c01d 0d kern kernel time sync enabl
ed
May 11 10:15:00 hadoop001 systemd[1]: Started Network Time Service.
#验证
[root@hadoop001 ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
LOCAL(0) .LOCL. 10 l 726 64 0 0.000 0.000 0.000
#其他从节点停⽌禁⽤ntpd服务
[root@hadoop002 ~]# systemctl stop ntpd
[root@hadoop002 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop002 ~]# /usr/sbin/ntpdate hadoop001
11 May 10:29:22 ntpdate[9370]: adjust time server 172.19.7.96 offset 0.000867 sec
#每天凌晨同步hadoop001节点时间
[root@hadoop002 ~]# crontab -e
00 00 * * * /usr/sbin/ntpdate hadoop001
[root@hadoop003 ~]# systemctl stop ntpd
[root@hadoop004 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop005 ~]# /usr/sbin/ntpdate hadoop001
11 May 10:29:22 ntpdate[9370]: adjust time server 172.19.7.96 offset 0.000867 sec
#每天凌晨同步hadoop001节点时间
[root@hadoop003 ~]# crontab -e
00 00 * * * /usr/sbin/ntpdate hadoop001
3.7 JDK部署
三台机器都必须配置
[root@hadoop001 software]# mkdir /usr/java
[root@hadoop001 software]# tar -zxvf jdk-8u45-linux-x64.gz -C /usr/java # -C 指定解压目录
#切记必须修正所属⽤户及⽤户组
[root@hadoop001 software]# chown -R root:root /usr/java/jdk1.8.0_45
#添加JAVA环境变量
[root@hadoop001 software]# vim /etc/profile
# 底部添加
export JAVA_HOME=/usr/java/jdk1.8.0_45
export PATH=${JAVA_HOME}/bin:${PATH}
# 使配置生效并测试
[root@hadoop001 software]# source /etc/profile
[root@hadoop001 software]# which java
/usr/java/jdk1.8.0_45/bin/java
3.8 hadoop001节点离线部署MySQL5.7
3.8.1 解压及创建目录
[root@hadoop001 software]# tar -zxvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz -C /usr/local/
[root@hadoop001 software]# cd /usr/local/
[root@hadoop001 local]# mv mysql-5.7.11-linux-glibc2.5-x86_64/ mysql
[root@hadoop001 local]# mkdir mysql/arch mysql/data mysql/tmp
3.8.2 创建my.cnf
[root@hadoop001 local]# rm -rf /etc/my.cnf
[root@hadoop001 local]# vim /etc/my.cnf
添加以下内容
[client]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
default-character-set=utf8mb4
[mysqld]
port = 3306
socket = /usr/local/mysql/data/mysql.sock
skip-slave-start
skip-external-locking
key_buffer_size = 256M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 32M
max_allowed_packet = 16M
myisam_sort_buffer_size=128M
tmp_table_size=32M
table_open_cache = 512
thread_cache_size = 8
wait_timeout = 86400
interactive_timeout = 86400
max_connections = 600
# Try number of CPU's*2 for thread_concurrency
#thread_concurrency = 32
#isolation level and default engine
default-storage-engine = INNODB
transaction-isolation = READ-COMMITTED
server-id = 1739
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
pid-file = /usr/local/mysql/data/hostname.pid
#open performance schema
log-warnings
sysdate-is-now
binlog_format = ROW
log_bin_trust_function_creators=1
log-error = /usr/local/mysql/data/hostname.err
log-bin = /usr/local/mysql/arch/mysql-bin
expire_logs_days = 7
innodb_write_io_threads=16
relay-log = /usr/local/mysql/relay_log/relay-log
relay-log-index = /usr/local/mysql/relay_log/relay-log.index
relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info
log_slave_updates=1
gtid_mode=OFF
enforce_gtid_consistency=OFF
# slave
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=4
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON
#other logs
#general_log =1
#general_log_file = /usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err
#for replication slave
sync_binlog = 500
#for innodb options
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend
innodb_log_group_home_dir = /usr/local/mysql/arch
innodb_log_files_in_group = 4
innodb_log_file_size = 1G
innodb_log_buffer_size = 200M
#根据生产需要,调整pool size
innodb_buffer_pool_size = 2G
#innodb_additional_mem_pool_size = 50M #deprecated in 5.6
tmpdir = /usr/local/mysql/tmp
innodb_lock_wait_timeout = 1000
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2
innodb_locks_unsafe_for_binlog=1
#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on
#case-sensitive file names and separate tablespace
innodb_file_per_table = 1
lower_case_table_names=1
[mysqldump]
quick
max_allowed_packet = 128M
[mysql]
no-auto-rehash
default-character-set=utf8mb4
[mysqlhotcopy]
interactive-timeout
[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
3.8.3 创建用户组及用户
[root@hadoop001 local]# groupadd -g 101 dba
[root@hadoop001 local]# useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
[root@hadoop001 local]# id mysqladmin
uid=514(mysqladmin) gid=101(dba) groups=101(dba),0(root)
# 一般不需要设置mysqladmin的密码,直接从root或者LDAP用户sudo切换
3.8.4 复制环境变量配置文件
[root@hadoop001 local]# cp /etc/skel/.* /usr/local/mysql
3.8.5 配置环境变量
[root@hadoop001 local]# vi mysql/.bash_profile
# 修改底部配置
# User specific environment and startup programs
export MYSQL_BASE=/usr/local/mysql
export PATH=${MYSQL_BASE}/bin:$PATH
unset USERNAME
#stty erase ^H
set umask to 022
umask 022
PS1=`uname -n`":"'$USER'":"'$PWD'":>"; export PS1
3.8.6 赋权限和用户组 切换用户mysqladmin 安装
[root@hadoop001 local]# chown mysqladmin:dba /etc/my.cnf
[root@hadoop001 local]# chmod 640 /etc/my.cnf
[root@hadoop001 local]# chown -R mysqladmin:dba /usr/local/mysql
[root@hadoop001 local]# chmod -R 755 /usr/local/mysql
3.8.7 配置服务及开机自启动
[root@hadoop001 local]# cd /usr/local/mysql
#将服务文件拷贝到init.d下,并重命名为mysql
[root@hadoop001 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysql
#赋予可执行权限
[root@hadoop001 mysql]# chmod +x /etc/rc.d/init.d/mysql
#删除服务
[root@hadoop001 mysql]# chkconfig --del mysql
#添加服务
[root@hadoop001 mysql]# chkconfig --add mysql
[root@hadoop001 mysql]# chkconfig --level 345 mysql on
3.8.8 安装libaio及安装mysql的初始db
[root@hadoop001 mysql]# yum -y install libaio
[root@hadoop001 mysql]# sudo su - mysqladmin
hadoop001:mysqladmin:/usr/local/mysql:>bin/mysqld \
--defaults-file=/etc/my.cnf \
--user=mysqladmin \
--basedir=/usr/local/mysql/ \
--datadir=/usr/local/mysql/data/ \
--initialize
在初始化时如果加上 –initial-insecure,则会创建空密码的 root@localhost 账号,否则会创建带密码的 root@localhost 账号,密码直接写在 log-error 日志文件中
(在5.6版本中是放在 ~/.mysql_secret 文件里,更加隐蔽,不熟悉的话可能会无所适从)
3.8.9 查看临时密码
hadoop001:mysqladmin:/usr/local/mysql:>cd /usr/local/mysql/data
hadoop001:mysqladmin:/usr/local/mysql/data:>cat hostname.err |grep password
2019-10-05T10:56:08.109910Z 1 [Note] A temporary password is generated for root@localhost: xxxxxx,
3.8.9 启动
hadoop001:mysqladmin:/usr/local/mysql/data:>/usr/local/mysql/bin/mysqld_safe --defaults-file=/etc/my.cnf &
3.8.10 登录及修改用户密码
hadoop001:mysqladmin:/usr/local/mysql/data:>mysql -uroot -p
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> alter user root@localhost identified by 'hadoop001&123!';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop001&123!' ;
mysql> flush privileges;
3.8.11 重启
hadoop001:mysqladmin:/usr/local/mysql/data:>service mysql restart
Shutting down MySQL..2019-10-05T11:05:47.622134Z mysqld_safe mysqld from pid file /usr/local/mysql/data/hostname.pid ended
[ OK ]
Starting MySQL..[ OK ]
[1]+ Done /usr/local/mysql/bin/mysqld_safe --defaults-file=/etc/my.cnf
3.9 创建CDH的元数据库和用户、amon服务的数据库及用户
mysql> create database cmf DEFAULT CHARACTER SET utf8;
mysql> create database amon DEFAULT CHARACTER SET utf8;
mysql> grant all on cmf.* TO 'cmf'@'%' IDENTIFIED BY 'hadoop001&123!';
mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'hadoop001&123!';
mysql> flush privileges;
3.10 hadoop001节点部署mysql jdbc jar
[root@hadoop001 mysql]# mkdir -p /usr/share/java/
[root@hadoop001 mysql]# cp mysql-connector-java.jar /usr/share/java/
3. CDH部署
3.1 离线部署cm server及agent
3.1.1.所有节点创建目录及解压
[root@hadoop001 software]# mkdir /opt/cloudera-manager
[root@hadoop001 software]# tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/
3.1.2.所有节点修改agent的配置,指向server的节点
[root@hadoop001 software]# sed -i "s/server_host=localhost/server_host=hadoop001/g" /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini
3.1.3.主节点修改server的配置:
[root@hadoop001 software]#vi /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-server/db.properties
#修改配置
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=hadoop001
com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.password=hadoop001&123!
com.cloudera.cmf.db.setupType=EXTERNAL
3.1.4.所有节点创建用户
[root@hadoop001 software]# useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
3.1.5.目录修改用户及用户组
[root@hadoop001 software]# chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
3.2 hadoop001节点部署离线parcel源
3.2.1 部署离线parcel源
[root@hadoop001 software]# mkdir -p /opt/cloudera/parcel-repo
[root@hadoop001 software]# cp CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel /opt/cloudera/parcel-repo/
#切记cp时,重命名去掉1,不不然在部署过程CM认为如上⽂文件下载未完整,会持续下载
[root@hadoop001 software]# cp CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
[root@hadoop001 software]# cp manifest.json /opt/cloudera/parcel-repo/
3.2.2 目录修改用户及用户组
[root@hadoop001 software]# chown -R cloudera-scm:cloudera-scm /opt/cloudera/
3.2.3 所有节点创建软件安装目录、用户及用户组权限
[root@hadoop001 software]# mkdir -p /opt/cloudera/parcels
[root@hadoop001 software]# chown -R cloudera-scm:cloudera-scm /opt/cloudera/
3.2.4 hadoop001节点启动Server
1.启动server /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-server start
2.设置hadoop001节点防火墙放开7180端口 (阿里云主机到安全组里设置)
3.等待1min,打开 http://hadoop001:7180 账号密码:admin/admin
4.假如打不不开,去看server的log,根据错误仔细排查错误
log路径在/opt/cloudera-manager/cm-5.16.1/log/cloudera-scm-server
3.2.5 所有节点启动Agent
/opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent start
3.2.6 进入Web界面进行后续操作
http://hadoop001:7180/
账号密码:admin/admin
3.2.7 欢迎使用 Cloudera Manager 最终用户许可条款和条件
3.2.8 欢迎使⽤用Cloudera Manager–您想要部署哪个版本?选择Cloudera Express免费 版本
3.2.9 为CDH集群安装指导主机。选择[当前管理理的主机],全部勾选
3.2.10 选择存储库
3.2.11 集群安装
正在安装选定Parcel
假如本地parcel离线源配置正确,则"下载"阶段瞬间完成,其余阶段视节点数与内部⽹络情况决定。
遇到的问题:
集群安装报错 ERROR Failure due to stall on seeded torrent.
解决:重启agent服务
[root@hadoop001 cloudera-scm-agent]# /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent restart
3.2.12 检查主机正确
解决透明大页问题
# 临时生效
[root@hadoop003 cloudera-scm-agent]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@hadoop003 cloudera-scm-agent]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
# 添加到/etc/rc.local使其永久生效
[root@hadoop003 cloudera-scm-agent]# vim /etc/rc.local
#添加以下内容
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@hadoop003 cloudera-scm-agent]# chmod +x /etc/rc.local
重新运行检查