笨鸟的平凡之路-安装CDH

最新推荐文章于 2022-08-31 10:30:53 发布

笨鸟的平凡之路

最新推荐文章于 2022-08-31 10:30:53 发布

阅读量557

点赞数

分类专栏： CDH平台问题文章标签：大数据

本文链接：https://blog.csdn.net/weixin_45109718/article/details/91854236

版权

CDH平台问题专栏收录该内容

5 篇文章 0 订阅

订阅专栏

准备工作

相关节点及压缩包的准备,从官网下载cloudera-manager和CDH 5.11.2 parcel文件.

CM 下载地址:
http://archive.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.11.2_x86_64.tar.gz

CDH parcel 下载地址:
http://archive.cloudera.com/cdh5/parcels/5.11.2/CDH-5.16.1-1.cdh5.16.1.p0.3-el6.parcel
http://archive.cloudera.com/cdh5/parcels/5.11.2/CDH-5.16.1-1.cdh5.16.1.p0.3-el6.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.11.2/manifest.json
特别注意要将下载的.sha1文件后缀更改为.sha, 并将文件上传到CM管理节点的/opt目录下.

1.所有节点JDK版本替换

CDH的运行依赖JDK的版本,但不支持系统默认的OpenJDK,需要先卸载操作的OpenJDK,然后重新1.7以上的JDK.

JDK 下载地址:
https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

所有节点删除旧的JDK

[root@cm ~]# rpm -qa | grep jdk
java-1.6.0-openjdk-1.6.0.41-1.13.13.1.el6_8.x86_64
java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el6_8.x86_64
[root@cm ~]# rpm -e java-1.6.0-openjdk-1.6.0.41-1.13.13.1.el6_8.x86_64
[root@cm ~]# rpm -e java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el6_8.x86_64
[root@cm ~]# rpm -qa | grep jdk
[root@cm ~]#

安装JDK 1.8:
将rpm结尾的包上传到CM节点,然后安装JDK

[root@cm ~]# rpm -ivh jdk-8u212-linux-x64.rpm
warning: jdk-8u212-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ########################################### [100%]
   1:jdk1.8                 ########################################### [100%]
Unpacking JAR files...
	tools.jar...
	plugin.jar...
	javaws.jar...
	deploy.jar...
	rt.jar...
	jsse.jar...
	charsets.jar...
	localedata.jar...
[root@cm ~]#

配置环境变量，在/etc/profile文件中添加如下内容：

#java
export JAVA_HOME=/usr/java/default
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

保存退出后使更改生效:

source /etc/profile

检查是否安装成功:

[root@cm ~]# java -version
java version "1.8.0_212"
Java(TM) SE Runtime Environment (build 1.8.0_212-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.212-b10, mixed mode)

注意:我这里只是安装了一台,是所有的节点都需要重新安装JDK

2.所有节点修改hosts文件

[root@cm ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.20.190 cm
192.168.20.191 master01
192.168.20.192 master02
192.168.20.193 worker01
192.168.20.194 worker02
[root@cm ~]#

3.关闭所有节点的SELinux

修改/etc/selinux/config文件如下:

[root@cm ~]# cat /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 
[root@cm ~]# setenforce 0

4.关闭所有节点的防火墙

[root@cm ~]# service iptables stop
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Unloading modules:                               [  OK  ]
[root@cm ~]# chkconfig iptables off
[root@cm ~]# chkconfig iptables --list
iptables       	0:关闭	1:关闭	2:关闭	3:关闭	4:关闭	5:关闭	6:关闭
[root@cm ~]#

5.所有节点调整Linux 系统参数

修改 swappiness=0,最大限度使用物理内存,然后才是 swap 交换分区,命令如下:

[root@cm ~]# echo 0 >/proc/sys/vm/swappiness
[root@cm ~]# echo "vm.swappiness=0">>/etc/sysctl.conf
[root@cm ~]# echo "echo 0>/proc/sys/vm/swappiness">>/etc/rc.d/rc.local

禁用 hugepage 透明大页,因为它可能会导致 CPU 使用率过高.将这个参数设 nerver,并添加到/etc/re.local 中,命令如下:

[root@cm ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@cm ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@cm ~]# echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.d/rc.local
[root@cm ~]# echo "echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag" >> /etc/rc.d/rc.local

修改 Linux 最大文件句柄数（默认 Linux 最大文件句柄数为 1024 ），命令如下：

[root@cm ~]# echo "* soft nofile 128000" >> /etc/security/limits.conf
[root@cm ~]# echo "* hard nofile 128000" >> /etc/security/limits.conf
[root@cm ~]# echo "* soft nproc 128000" >> /etc/security/limits.conf
[root@cm ~]# echo "* hard nproc 128000" >> /etc/security/limits.conf
[root@cm ~]# sed -i 's/1024/unlimited/' /etc/security/limits.d/90-nproc.conf
[root@cm ~]# ulimit -SHn 128000
[root@cm ~]# ulimit -SHu 128000

6.配置SSH免密登陆

为避免频繁输入密码,使用RSA非对称加密进行免密登陆配置,操作如下:
在所有节点都生成密钥(pub公钥):

[root@cm ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
27:39:a5:53:71:df:63:86:b9:4d:27:d6:b1:ed:7a:2b root@cm
The key's randomart image is:
+--[ RSA 2048]----+
|          . .  . |
|           o .oo+|
|          o  o+*=|
|         =   .*oo|
|        S .  . ..|
|         =     . |
|              . .|
|             E ..|
|              .. |
+-----------------+
[root@cm ~]# cd ~/.ssh/
[root@cm .ssh]# ls
id_rsa  id_rsa.pub
[root@cm .ssh]# cat ~/.ssh/id_rsa.pub >> authorized_keys

其他节点使用ssh-copy-id命令全部写入CM节点：

[root@master01 ~]# ssh-copy-id -i /root/.ssh/id_rsa.pub 192.168.20.190

cm再把合并之后的authorized_keys 分发到所有节点(必须是~/.ssh目录下)：

[root@cm .ssh]# scp authorized_keys master01:`pwd`

对所有节点的authorized_keys 文件修改权限,注意: authorized_keys 一定要其他人没有w(写的)权限，不然ssh秘钥是不工作的，700 744 755 都可以只要其他人没有写的权限,最好是600

[root@cm .ssh]# chmod 600 ./authorized_keys

在所有节点进行验证ssh操作:

[root@cm .ssh]# ssh cm date
[root@cm .ssh]# ssh master01 date
[root@cm .ssh]# ssh master02 date
[root@cm .ssh]# ssh worker01 date
[root@cm .ssh]# ssh worker02 date

之后可以查看known_hosts文件,会发现记录了所有节点的访问公钥.

7.配置时间同步

CDH集群中所有节点需要设置统一的时间，所以我们需要配置时间同步，可以选择NTP
在CM管理节点上配置NTP
在/etc/ntp.conf文件中先注释掉以下内容:

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst

然后再添加如下内容:

#中国国家受时中心地址
server 210.72.145.44 prefer 
#外部的时间服务器不可用时，以本地时间作为时间服务
server 127.127.1.0  
fudge 127.127.1.0 stratum 10

在cm上启动ntpd

[root@cm ~]# ntpdate cn.pool.ntp.org
[root@cm ~]# service ntpd start
Starting ntpd:                                             [  OK  ]

[root@cm ~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 210.72.145.44   .INIT.          16 u    -   64    0    0.000    0.000   0.000
*LOCAL(0)        .LOCL.          10 l   47   64    3    0.000    0.000   0.000

[root@cm ~]# ntpstat
synchronised to local net at stratum 11 
   time correct to within 1949 ms
   polling server every 64 s

在其他节点同步CM的时间:

[root@master01 .ssh]# ntpdate cm

8.CM管理节点安装HTTP服务

我这里已经装过了,没有装过的同学烦请自行百度

[root@cm ~]# rpm -qa|grep httpd
httpd-tools-2.2.15-59.el6.x86_64
httpd-2.2.15-59.el6.x86_64

启动httpd服务

service httpd start 
chkconfig httpd on

9.安装MySQL数据库

CM的元数据需要存储在数据库中,这里选择MySQL5.6版本的数据库来存储.
下载Mysql,官方网址：https://www.mysql.com/
下载:https://cdn.mysql.com//Downloads/MySQL-5.6/MySQL-5.6.44-1.el6.x86_64.rpm-bundle.tar

在cm节点上创建文件夹,并解压tar包

[root@cm ~]# mkdir /opt/mysql
[root@cm ~]# tar -xvf MySQL-5.6.44-1.el6.x86_64.rpm-bundle.tar -C /opt/mysql/
MySQL-test-5.6.44-1.el6.x86_64.rpm
MySQL-shared-5.6.44-1.el6.x86_64.rpm
MySQL-devel-5.6.44-1.el6.x86_64.rpm
MySQL-shared-compat-5.6.44-1.el6.x86_64.rpm
MySQL-client-5.6.44-1.el6.x86_64.rpm
MySQL-embedded-5.6.44-1.el6.x86_64.rpm
MySQL-server-5.6.44-1.el6.x86_64.rpm

添加执行权限

[root@cm mysql]# chmod u+x /opt/mysql/*.rpm

安装MySQL-shared包:

[root@cm mysql]# rpm -ivh ./MySQL-shared-compat-5.6.44-1.el6.x86_64.rpm 
warning: ./MySQL-shared-compat-5.6.44-1.el6.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                ########################################### [100%]
   1:MySQL-shared-compat    ########################################### [100%]

卸载系统自带的mysql-libs包,注意这里必选先安装mysql-shared包,否则这里依赖关系会有问题

[root@cm mysql]# rpm -qa|grep mysql
mysql-libs-5.1.73-8.el6_8.x86_64
[root@cm mysql]# rpm -e mysql-libs-5.1.73-8.el6_8.x86_64

安装MySQL server 和Client:

[root@cm mysql]# rpm -ivh MySQL-server-5.6.44-1.el6.x86_64.rpm
[root@cm mysql]# rpm -ivh MySQL-client-5.6.44-1.el6.x86_64.rpm
[root@cm mysql]# rpm -ivh MySQL-devel-5.6.44-1.el6.x86_64.rpm
[root@cm mysql]# chkconfig mysql --list

初始化MySQL

[root@cm mysql]# /usr/bin/mysql_install_db
[root@cm mysql]# service mysql start
Starting MySQL.Logging to '/var/lib/mysql/cm.err'.
.                                                          [  OK  ]

配置MySQL配置文件

[root@cm mysql]# cp /usr/share/mysql/my-default.cnf /etc/my.cnf
[root@cm mysql]# vi /etc/my.cnf

my.cnf文件内容修改如下:

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0


key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

首先查看mysql初始密码并重置密码:

[root@cm mysql]# more ~/.mysql_secret
# The random password set for the root user at Sun Dec 10 19:17:06 2028 (local t
ime): jbhvJqdULQVfI3J8

[root@cm mysql]# mysql -uroot -pjbhvJqdULQVfI3J8
mysql> set password=password('111111');

添加允许远程登录

mysql> Grant all privileges on *.* to 'root'@'%' identified by '111111' with grant option;

重新加载权限

mysql> flush privileges;

创建数据库及用户并添加权限
其他的数据库实例使用uft8编码，hive和oozie使用默认的编码方式
如下，可直接复制

create database if not exists amon default charset utf8 collate utf8_general_ci;
create database if not exists rman default charset utf8 collate utf8_general_ci;
create database if not exists nav default charset utf8 collate utf8_general_ci;
create database if not exists navms default charset utf8 collate utf8_general_ci;
create database if not exists hue default charset utf8 collate utf8_general_ci;
create database if not exists sentry default charset utf8 collate utf8_general_ci;
create database if not exists hive;
create database if not exists oozie;
grant all on hive.* to 'hive'@'%' identified by 'hive' with grant option;
grant all on oozie.* to 'oozie'@'%' identified by 'oozie' with grant option;
grant all on hue.* to 'hue'@'%' identified by 'hue' with grant option;
grant all on amon.* to 'amon'@'%' identified by 'amon' with grant option;
grant all on rman.* to 'rman'@'%' identified by 'rman' with grant option;
grant all on nav.* to 'nav'@'%' identified by 'nav' with grant option;
grant all on navms.* to 'navms'@'%' identified by 'navms' with grant option;
grant all on sentry.* to 'sentry'@'%' identified by 'sentry' with grant option;
flush privileges;

查看创建的数据库

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| amon               |
| hive               |
| hue                |
| mysql              |
| nav                |
| navms              |
| oozie              |
| performance_schema |
| rman               |
| sentry             |
| test               |
+--------------------+
12 rows in set (0.00 sec)

10.在所有节点安装MySQL JDBC驱动

从官网下载：
https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz

[root@cm ~]# tar -xvf mysql-connector-java-5.1.47.tar.gz
[root@cm ~]# cp mysql-connector-java-5.1.47/mysql-connector-java-* /usr/share/java
[root@cm ~]# cd /usr/share/java
[root@cm java]# ll
总用量 1968
-rw-r--r--. 1 root root 1007505 12月 10 20:09 mysql-connector-java-5.1.47-bin.jar
-rw-r--r--. 1 root root 1007502 12月 10 20:09 mysql-connector-java-5.1.47.jar
[root@cm java]# mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar

注意这里必须修改驱动文件的名称，不能带版本号，否则会在安装CDH时会报java.lang.ClassNotFoundException: com.mysql.jdbc.Driver的错误.
分发驱动包

[root@cm java]# scp /usr/share/java/* root@master01:/usr/share/java/

11.CM安装准备工作

[root@cm java]# cd /opt/
[root@cm opt]# ls
CDH-5.11.2-1.cdh5.11.2.p0.4-el6.parcel       manifest.json
CDH-5.11.2-1.cdh5.11.2.p0.4-el6.parcel.sha   mysql
cloudera-manager-el6-cm5.11.2_x86_64.tar.gz  rh
[root@cm opt]# tar -zxvf /opt/cloudera-manager-el6-cm5.11.2_x86_64.tar.gz -C /opt

准备Parcels：

[root@cm opt]# cp /opt/CDH-5.11.2-1.cdh5.11.2.p0.4-el6.parcel /opt/cloudera/parcel-repo/
[root@cm opt]# cp /opt/CDH-5.11.2-1.cdh5.11.2.p0.4-el6.parcel.sha /opt/cloudera/parcel-repo/
[root@cm opt]# cp /opt/manifest.json /opt/cloudera/parcel-repo/

修改config.ini 配置文件，将config.ini配置文件中server_host中改成CM管理节点的名称:cm:

[root@cm ~]# cd /opt/cm-5.11.2/etc/cloudera-scm-agent
[root@cm cloudera-scm-agent]# head config.ini 
[General]
# Hostname of the CM server.
server_host=cm

复制MySQL驱动到/opt/cm-5.16.1/share/cmf/lib/中

[root@cm cloudera-scm-agent]# cp /usr/share/java/mysql-connector-java.jar /opt/cm-5.11.2/share/cmf/lib/

同步Agent到所有节点：

[root@cm cloudera-scm-agent]# nohup scp -r /opt/cm-5.11.2/ root@master01:/opt/ &
[root@cm cloudera-scm-agent]# nohup scp -r /opt/cm-5.11.2/ root@master02:/opt/ &
[root@cm cloudera-scm-agent]# nohup scp -r /opt/cm-5.11.2/ root@worker01:/opt/ &
[root@cm cloudera-scm-agent]# nohup scp -r /opt/cm-5.11.2/ root@worker02:/opt/ &

在所有节点创建cloudera-scm用户：

[root@cm cloudera-scm-agent]# useradd --system --home=/opt/cm-5.11.2/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

在CM管理节点初始化配置数据库，命令参数可以查看帮助

[root@cm cloudera-scm-agent]# /opt/cm-5.11.2/share/cmf/schema/scm_prepare_database.sh mysql -hlocalhost -uroot -p111111 --scm-host localhost scm scm scm

CM管理节点启动CM服务和Agent：

[root@cm cloudera-scm-agent]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:                              [确定]
[root@cm cloudera-scm-agent]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-agent start
正在启动 cloudera-scm-agent：                              [确定]

其他节点启动Agent:

[root@master01 cm-5.11.2]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-agent start
正在启动 cloudera-scm-agent：                              [确定]
[root@master02 ~]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-agent start
正在启动 cloudera-scm-agent：                              [确定]
[root@worker01 ~]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-agent start
正在启动 cloudera-scm-agent：                              [确定]
[root@worker02 ~]# /opt/cm-5.11.2/etc/init.d/cloudera-scm-agent start
正在启动 cloudera-scm-agent：                              [确定]

12.Cloudera Manager 及CDH 安装

CM Server 和 Agent 启动以后就可以通过CM的Web界面进行CDH的安装，默认的用户名和密码都是admin。直接通过7180端口访问CM 管理节点即可：
http://192.168.20.190:7180/cmf/login

同意条款后继续:
在这里插入图片描述
选择继续:

选择继续:

安装指定主机:

选择继续:

集群安装:

集群设置(建议选择Hbase内核):

可以根据需求自定义各节点对应的角色:

测试连接Mysql数据库(这里和之前Mysql建库对应上,测试连接后选择继续):
在这里插入图片描述
集群设置,审核更改(一些配置项):

首次运行:

其中对于kafka的安装可能会出现问题,最好参考以下简书:
https://www.jianshu.com/p/c275edb69ec2
当出现权限问题时,需要到hdfs用户更改权限:
hdfs dfs -chmod -R 755 /
hdfs dfs -chmod -R 775 /

[hdfs@master01 ~]$ hdfs dfs -chmod 775 /user
[hdfs@master01 ~]$ hdfs dfs -chmod 775 /
Found 2 items
drwxrwxrwt   - hdfs supergroup          0 2028-12-11 13:19 /tmp
drwxrwxr-x   - hdfs supergroup          0 2028-12-11 13:20 /user

对于角色的安排可以参考我的博客.