CDH5.16.1集群企业真正离线部署

最新推荐文章于 2020-05-25 21:48:53 发布

钻石大门

最新推荐文章于 2020-05-25 21:48:53 发布

阅读量448

点赞数

分类专栏： CDH

本文链接：https://blog.csdn.net/weixin_41158277/article/details/103391208

版权

CDH 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

一.准备工作

1.离线部署主要分为三块

a.MySQL离线部署
b.CM离线部署
c.Parcel文件离线源部署

2.规划

在这里插入图片描述

3.下载源

CM
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz
Parcel
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
http://archive.cloudera.com/cdh5/parcels/5.16.1/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.16.1/manifest.json
JDK
https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html
下载jdk-8u202-linux-x64.tar.gz
MySQL https://dev.mysql.com/downloads/mysql/5.7.html#downloads
下载mysql-5.7.26-el7-x86_64.tar.gz
MySQL jdbc jar
http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar
下载完成后要重命名去掉版本号， mv mysql-connector-java-5.1.47.jar
mysql-connector-java.jar

准备好百度云,下载安装包:
链接:https://pan.baidu.com/s/10s-NaFLfztKuWImZTiBMjA 密码:viqp

二.集群节点初始化

1.阿里云华南区购买3台，按量付费虚拟机

CentOS7.2操作系统，2核8G最低配置
密码：Diamond8888

2.当前笔记本或台式机配置hosts文件

MAC: /etc/hosts
Window: C:\windows\system32\drivers\etc\hosts

公网地址:
47.112.166.64 hadoop001
47.112.157.186 hadoop002
47.112.150.172 hadoop003

3.设置所有节点的hosts文件

私有地址、内网地址:
echo "172.18.98.143 hadoop001">> /etc/hosts
echo "172.18.98.144 hadoop002">> /etc/hosts
echo "172.18.98.142 hadoop003">> /etc/hosts

4.关闭所有节点的防火墙及清空规则

systemctl stop firewalld
systemctl disable firewalld
iptables -F

5.关闭所有节点的selinux

vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled
设置后需要重启才能生效

6.设置所有节点的时区一致及时钟同步

6.1.时区
[root@hadoop001 ~]# date
Sun Dec 22 00:34:05 CST 2019
[root@hadoop001 ~]# timedatectl
      Local time: Sun 2019-12-22 00:34:14 CST
  Universal time: Sat 2019-12-21 16:34:14 UTC
        RTC time: Sun 2019-12-22 00:34:14
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: yes
      DST active: n/a
#查看命令帮助，学习至关重要，无需百度
[root@hadoop001 ~]# timedatectl --help
timedatectl [OPTIONS...] COMMAND ...

Query or change system time and date settings.

  -h --help                Show this help message
     --version             Show package version
     --no-pager            Do not pipe output into a pager
     --no-ask-password     Do not prompt for password
  -H --host=[USER@]HOST    Operate on remote host
  -M --machine=CONTAINER   Operate on local container
     --adjust-system-clock Adjust system clock when changing local RTC mode

Commands:
  status                   Show current time settings
  set-time TIME            Set system time
  set-timezone ZONE        Set system time zone
  list-timezones           Show known time zones
  set-local-rtc BOOL       Control whether RTC is in local time
  set-ntp BOOL             Control whether NTP is enabled

#查看哪些时区
[root@hadoop001 ~]# timedatectl list-timezones
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
Africa/Bamako
Africa/Bangui
Africa/Banjul
......

#所有节点设置亚洲上海海时区
[root@hadoop001 ~]# timedatectl set-timezone Asia/Shanghai
[root@hadoop002 ~]# timedatectl set-timezone Asia/Shanghai
[root@hadoop003 ~]# timedatectl set-timezone Asia/Shanghai

6.2.时间
#所有节点安装ntp
[root@hadoop001 ~]# yum install -y ntp

#选取hadoop001为ntp的主节点
[root@hadoop001 ~]# vi /etc/ntp.conf

#time
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org
#当外部时间不不可用时，可使用本地硬件时间
server 127.127.1.0 iburst local clock
#允许哪些网段的机器器来同步时间
restrict 172.18.98.0 mask 255.255.255.0 nomodify notrap

#开启ntpd及查看状态
[root@hadoop001 ~]# systemctl start ntpd
[root@hadoop001 ~]# systemctl status ntpd
● ntpd.service - Network Time Service
   Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2019-12-22 00:39:46 CST; 7min ago
 Main PID: 19022 (ntpd)
   CGroup: /system.slice/ntpd.service
           └─19022 /usr/sbin/ntpd -u ntp:ntp -g

Dec 22 00:39:46 hadoop001 systemd[1]: Starting Network Time Service...
Dec 22 00:39:46 hadoop001 ntpd[19022]: proto: precision = 0.088 usec
Dec 22 00:39:46 hadoop001 ntpd[19022]: 0.0.0.0 c01d 0d kern kernel time sync enabled
Dec 22 00:39:46 hadoop001 systemd[1]: Started Network Time Service.

#其他从节点停止禁用ntpd服务
[root@hadoop002 ~]# systemctl stop ntpd
[root@hadoop002 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop002 ~]# /usr/sbin/ntpdate hadoop001
22 Dec 00:49:59 ntpdate[19067]: adjust time server 172.18.98.143 offset 0.000812 sec
#每天凌晨同步hadoop001节点时间
[root@hadoop002 ~]# crontab -e 
00 00 * * * /usr/sbin/ntpdate hadoop001

[root@hadoop003 ~]# systemctl stop ntpd
[root@hadoop003 ~]# systemctl disable ntpd
Removed symlink /etc/systemd/system/multi-user.target.wants/ntpd.service.
[root@hadoop003 ~]# /usr/sbin/ntpdate hadoop001
22 Dec 00:53:57 ntpdate[19075]: adjust time server 172.18.98.143 offset 0.000027 sec
#每天凌晨同步hadoop001节点时间
[root@hadoop003 ~]# crontab -e
00 00 * * * /usr/sbin/ntpdate hadoop001

7.部署集群的JDK

mkdir /usr/java
tar -xzvf jdk-8u45-linux-x64.tar.gz -C /usr/java/
#切记必须修正所属用户及用户组
chown -R root:root /usr/java/jdk1.8.0_45
echo "export JAVA_HOME=/usr/java/jdk1.8.0_45" >> /etc/profile
echo "export PATH=${JAVA_HOME}/bin:${PATH}" >> /etc/profile
source /etc/profile
which java

8.hadoop001节点离线部署MySQL5.7

8.1.解压及创建目录
[root@hadoop001 local]# tar xzvf mysql-5.7.11-linux-glibc2.5-x86_64.tar.gz
[root@hadoop001 local]# mv mysql-5.7.11-linux-glibc2.5-x86_64 mysql

[root@hadoop001 local]# mkdir mysql/arch mysql/data mysql/tmp

8.2.创建my.cnf(见文件)
[root@hadoop39 local]# vi /etc/my.cnf
[client]
port            = 3306
socket          = /usr/local/mysql/data/mysql.sock
default-character-set=utf8mb4

[mysqld]
port            = 3306
socket          = /usr/local/mysql/data/mysql.sock

skip-slave-start

skip-external-locking
key_buffer_size = 256M
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 4M
query_cache_size= 32M
max_allowed_packet = 16M
myisam_sort_buffer_size=128M
tmp_table_size=32M

table_open_cache = 512
thread_cache_size = 8
wait_timeout = 86400
interactive_timeout = 86400
max_connections = 600

# Try number of CPU's*2 for thread_concurrency
#thread_concurrency = 32 

#isolation level and default engine 
default-storage-engine = INNODB
transaction-isolation = READ-COMMITTED

server-id  = 1739
basedir     = /usr/local/mysql
datadir     = /usr/local/mysql/data
pid-file     = /usr/local/mysql/data/hostname.pid

#open performance schema
log-warnings
sysdate-is-now

binlog_format = ROW
log_bin_trust_function_creators=1
log-error  = /usr/local/mysql/data/hostname.err
log-bin = /usr/local/mysql/arch/mysql-bin
expire_logs_days = 7

innodb_write_io_threads=16

relay-log  = /usr/local/mysql/relay_log/relay-log
relay-log-index = /usr/local/mysql/relay_log/relay-log.index
relay_log_info_file= /usr/local/mysql/relay_log/relay-log.info

log_slave_updates=1
gtid_mode=OFF
enforce_gtid_consistency=OFF

# slave
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=4
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON

#other logs
#general_log =1
#general_log_file  = /usr/local/mysql/data/general_log.err
#slow_query_log=1
#slow_query_log_file=/usr/local/mysql/data/slow_log.err

#for replication slave
sync_binlog = 500


#for innodb options 
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:1G;ibdata2:1G:autoextend

innodb_log_group_home_dir = /usr/local/mysql/arch
innodb_log_files_in_group = 4
innodb_log_file_size = 1G
innodb_log_buffer_size = 200M

#根据生产需要，调整pool size 
innodb_buffer_pool_size = 2G
#innodb_additional_mem_pool_size = 50M #deprecated in 5.6
tmpdir = /usr/local/mysql/tmp

innodb_lock_wait_timeout = 1000
#innodb_thread_concurrency = 0
innodb_flush_log_at_trx_commit = 2

innodb_locks_unsafe_for_binlog=1

#innodb io features: add for mysql5.5.8
performance_schema
innodb_read_io_threads=4
innodb-write-io-threads=4
innodb-io-capacity=200
#purge threads change default(0) to 1 for purge
innodb_purge_threads=1
innodb_use_native_aio=on

#case-sensitive file names and separate tablespace
innodb_file_per_table = 1
lower_case_table_names=1

[mysqldump]
quick
max_allowed_packet = 128M

[mysql]
no-auto-rehash
default-character-set=utf8mb4

[mysqlhotcopy]
interactive-timeout

[myisamchk]
key_buffer_size = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M

8.33.创建用户组及用户
[root@hadoop001 local]# groupadd -g 101 dba
[root@hadoop001 local]# useradd -u 514 -g dba -G root -d /usr/local/mysql mysqladmin
[root@hadoop001 local]# id mysqladmin
uid=514(mysqladmin) gid=101(dba) groups=101(dba),0(root)

## 一般不需要设置mysqladmin的密码，直接从root或者LDAP用户sudo切换
#[root@hadoop001 local]# passwd mysqladmin
Changing password for user mysqladmin.
New UNIX password: 123456
BAD PASSWORD: it is too simplistic/systematic
Retype new UNIX password: 
passwd: all authentication tokens updated successfully.

## if user mysqladmin is existing,please execute the following command of usermod.
#[root@hadoop001 local]# usermod -u 514 -g dba -G root -d /usr/local/mysql mysqladmin

8.4.copy 环境变量配置文件至mysqladmin用户的home目录中,为了以下步骤配置个人环境变量
[root@hadoop001 local]# cp /etc/skel/.* /usr/local/mysql  
###important

8.5.配置环境变量
[root@hadoop001 local]# vi mysql/.bash_profile
# .bash_profile
# Get the aliases and functions

if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs
export MYSQL_BASE=/usr/local/mysql
export PATH=${MYSQL_BASE}/bin:$PATH


unset USERNAME

#stty erase ^H
set umask to 022
umask 022
PS1=`uname -n`":"'$USER'":"'$PWD'":>"; export PS1

8.6.赋权限和用户组，切换用户mysqladmin，安装
[root@hadoop001 local]# chown  mysqladmin:dba /etc/my.cnf 
[root@hadoop001 local]# chmod  640 /etc/my.cnf  


[root@hadoop001 local]# chown -R mysqladmin:dba /usr/local/mysql
[root@hadoop001 local]# chmod -R 755 /usr/local/mysql

8.7.配置服务及开机自启动
[root@hadoop001 local]# cd /usr/local/mysql
#将服务文件拷贝到init.d下，并重命名为mysql
[root@hadoop001 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysql 
#赋予可执行权限
[root@hadoop001 mysql]# chmod +x /etc/rc.d/init.d/mysql
#删除服务
[root@hadoop001 mysql]# chkconfig --del mysql
#添加服务
[root@hadoop001 mysql]# chkconfig --add mysql
[root@hadoop001 mysql]# chkconfig --level 345 mysql on

8.8.安装libaio及安装mysql的初始db
[root@hadoop001 mysql]# yum -y install libaio
[root@hadoop001 mysql]# su - mysqladmin

hadoop001:mysqladmin:/usr/local/mysql:> bin/mysqld \
--defaults-file=/etc/my.cnf \
--user=mysqladmin \
--basedir=/usr/local/mysql/ \
--datadir=/usr/local/mysql/data/ \
--initialize

在初始化时如果加上 –initial-insecure，则会创建空密码的 root@localhost 账号，否则会创建带密码的 root@localhost 账号，密码直接写在 log-error 日志文件中
（在5.6版本中是放在 ~/.mysql_secret 文件里，更加隐蔽，不熟悉的话可能会无所适从）

8.9.查看临时密码
hadoop001:mysqladmin:/usr/local/mysql/data:>cat hostname.err |grep password
2019-12-22T10:03:36.640568Z 1 [Note] A temporary password is generated for root@localhost: DjeuJCqdh9?7
hadoop001:mysqladmin:/usr/local/mysql/data:>

8.10.启动
/usr/local/mysql/bin/mysqld_safe --defaults-file=/etc/my.cnf &

8.11.登录及修改用户密码
hadoop001:mysqladmin:/usr/local/mysql:>mysql -uroot -p'DjeuJCqdh9?7'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.11-log

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> alter user root@localhost identified by 'ruozedata123';

Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'ruozedata123' ;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> exit;
Bye

8.12.重启
hadoop001:mysqladmin:/usr/local/mysql:> service mysql restart

hadoop001:mysqladmin:/usr/local/mysql:>service mysql restart
Shutting down MySQL..[  OK  ]
rm: cannot remove '/var/lock/subsys/mysql': Permission denied
Starting MySQL.[  OK  ]
hadoop001:mysqladmin:/usr/local/mysql:>mysql -uroot -pruozedata123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.11-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

9.创建CDH的元数据库和用户、amon服务的数据库及用户

create database cmf DEFAULT CHARACTER SET utf8;
create database amon DEFAULT CHARACTER SET utf8;
grant all on cmf.* TO 'cmf'@'%' IDENTIFIED BY 'Ruozedata123456!';
grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'Ruozedata123456!';
flush privileges;

10.hadoop001节点部署mysql jdbc jar

[root@hadoop001 cdh5.16.1]# mkdir -p /usr/share/java/
[root@hadoop001 cdh5.16.1]# cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar

三.CDH部署

1.离线部署cm server及agent

1.0.上传cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz到hadoop001，并且scp到hadoop002和hadoop003
[root@hadoop001 cdh5.16.1]# rz
[root@hadoop001 cdh5.16.1]# scp cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz hadoop002:/root/cdh5.16.1
[root@hadoop001 cdh5.16.1]# scp cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz hadoop003:/root/cdh5.16.1

1.1.所有节点创建目录及解压

[root@hadoop001 cdh5.16.1]# mkdir /opt/cloudera-manager
[root@hadoop001 cdh5.16.1]# tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/
[root@hadoop002 cdh5.16.1]# mkdir /opt/cloudera-manager
[root@hadoop002 cdh5.16.1]# tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/
[root@hadoop003 cdh5.16.1]# mkdir /opt/cloudera-manager
[root@hadoop003 cdh5.16.1]# tar -zxvf cloudera-manager-centos7-cm5.16.1_x86_64.tar.gz -C /opt/cloudera-manager/

1.2.所有节点修改agent的配置，指向server的节点hadoop001
[root@hadoop001 cdh5.16.1]# sed -i "s/server_host=localhost/server_host=hadoop001/g" /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini
[root@hadoop002 cdh5.16.1]# sed -i "s/server_host=localhost/server_host=hadoop001/g" /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini
[root@hadoop003 cdh5.16.1]# sed -i "s/server_host=localhost/server_host=hadoop001/g" /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-agent/config.ini

1.3.主节点修改server的配置:
[root@hadoop001 cdh5.16.1]# vi /opt/cloudera-manager/cm-5.16.1/etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=hadoop001
com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.password=Ruozedata123456!
com.cloudera.cmf.db.setupType=EXTERNAL

1.4.所有节点创建用户
[root@hadoop001 cdh5.16.1]# useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
[root@hadoop002 cdh5.16.1]# useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
[root@hadoop003 cdh5.16.1]# useradd --system --home=/opt/cloudera-manager/cm-5.16.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

1.5.目录修改用户及用户组
[root@hadoop001 cdh5.16.1]# chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
[root@hadoop002 cdh5.16.1]# chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager
[root@hadoop003 cdh5.16.1]# chown -R cloudera-scm:cloudera-scm /opt/cloudera-manager

2.hadoop001节点部署离线parcel源

2.1.部署离线parcel源
$ mkdir -p /opt/cloudera/parcel-repo
$ ll
total 3081664
-rw-r--r-- 1 root root 2127506677 May 9 18:04 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.par
cel
-rw-r--r-- 1 root root 41 May 9 18:03 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.par
cel.sha1
-rw-r--r-- 1 root root 841524318 May 9 18:03 cloudera-manager-centos7-cm5.16.1_x
86_64.tar.gz
-rw-r--r-- 1 root root 185515842 Aug 10 2017 jdk-8u144-linux-x64.tar.gz
-rw-r--r-- 1 root root 66538 May 9 18:03 manifest.json
-rw-r--r-- 1 root root 989495 May 25 2017 mysql-connector-java.jar
$ cp CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel /opt/cloudera/parcel-repo/
#切记cp时，重命名去掉1，不不然在部署过程CM认为如上文件下载未完整，会持续下载
$ cp CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
$ cp manifest.json /opt/cloudera/parcel-repo/
#校验下载的文件完整无缺损
[root@hadoop001 parcel-repo]# ll
total 2077720
-rw-r--r-- 1 root root 2127506677 Dec 23 00:47 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
-rw-r--r-- 1 root root         41 Dec 23 00:47 CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha
-rw-r--r-- 1 root root      66538 Dec 23 00:48 manifest.json
[root@hadoop001 parcel-repo]# cat CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel.sha 
703728dfa7690861ecd3a9bcd412b04ac8de7148
[root@hadoop001 parcel-repo]# /usr/bin/sha1sum CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel
703728dfa7690861ecd3a9bcd412b04ac8de7148  CDH-5.16.1-1.cdh5.16.1.p0.3-el7.parcel

2.2.目录修改用户及用户组
$ chown -R cloudera-scm:cloudera-scm /opt/cloudera/

3.所有节点创建软件安装目录、用户及用户组权限

[root@hadoop001 parcel-repo]# mkdir -p /opt/cloudera/parcels
[root@hadoop001 parcel-repo]# chown -R cloudera-scm:cloudera-scm /opt/cloudera/

4.hadoop001节点启动Server

4.1.启动server
/opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-server start
4.2.阿里云web界面，设置该hadoop001节点防火墙放开7180端口
4.3.等待1min，打开 http://hadoop001:7180 账号密码:admin/admin
4.4.假如打不不开，去看server的log，根据错误仔细排查错误

5.所有节点启动Agent

[root@hadoop001 cdh5.16.1]# /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]
[root@hadoop002 cdh5.16.1]# /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]
[root@hadoop003 cdh5.16.1]# /opt/cloudera-manager/cm-5.16.1/etc/init.d/cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]

6.接下来，全部Web界面操作

http://hadoop001:7180/
账号密码:admin/admin

7.欢迎使用Cloudera Manager–最终用户许可条款与条件。勾选

在这里插入图片描述

8.欢迎使用Cloudera Manager–您想要部署哪个版本？选择Cloudera Express免费版本

在这里插入图片描述

9.感谢您选择Cloudera Manager和CDH

在这里插入图片描述

10.为CDH集群安装指导主机。选择[当前管理理的主机]，全部勾选

在这里插入图片描述

11.选择存储库

在这里插入图片描述

12.集群安装–正在安装选定Parcel假如

本地parcel离线源配置正确，则"下载"阶段瞬间完成，其余阶段视节点数与内部网络情况决定。
在这里插入图片描述

13.检查主机正确性

在这里插入图片描述

13.1.建议将/proc/sys/vm/swappiness设置为最大值10。
swappiness值控制操作系统尝试交换内存的积极；
swappiness=0：表示最⼤大限度使用物理理内存，之后才是swap空间；
swappiness=100：表示积极使⽤用swap分区，并且把内存上的数据及时搬迁到swap空间；
如果是混合服务器器，不不建议完全禁用swap，可以尝试降低swappiness。
临时调整：
sysctl vm.swappiness=10
永久调整：
cat << EOF >> /etc/sysctl.conf
# Adjust swappiness value
vm.swappiness=10
EOF
13.2.已启用透明大页面压缩，可能会导致重大性能问题，建议禁用此设置。
临时调整：
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
永久调整：
cat << EOF >> /etc/rc.d/rc.local
# Disable transparent_hugepage
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
EOF
# centos7.x系统，需要为"/etc/rc.d/rc.local"文件赋予执行行权限
chmod +x /etc/rc.d/rc.local

14.自定义服务，选择部署Zookeeper、HDFS、Yarn服务

在这里插入图片描述

15.自定义⻆角色分配

在这里插入图片描述

16.数据库设置

password: Ruozedata123456!
在这里插入图片描述

17.审改设置，默认即可

在这里插入图片描述

18.首次运行

在这里插入图片描述

19.恭喜您!

在这里插入图片描述

20.主页

在这里插入图片描述

钻石大门

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
CDH5.16.1集群企业真正离线部署

一.准备工作1.离线部署主要分为三块a.MySQL离线部署b.CM离线部署c.Parcel文件离线源部署2.规划3.下载源CMhttp://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.16.1_x86_64.tar.gzParcelhttp://archive.cloudera.com/cdh5/pa...
复制链接

扫一扫

专栏目录