CDH6.1安装五之一(基础搭建)
总概
本文将介绍Centos7.6(通过cat /etc/centos-release命令查看centos版本号) 离线安装CDH和Cloudera Manager过程,软件版本如下表1-1所示:
表1-1 软件版本选取
NO. | 软件名称 | 版本 |
---|---|---|
1 | 操作系统 | CentOS Linux release 7.6.1810 |
2 | JDK | |
3 | Clouder Manager | |
4 | CDH |
服务器规划
本次安装共8台服务器,服务器配置及用途如下表1-2所示:
表1-2 服务器基本规划配置
NO. | hostname | ip | 配置 | 用途 |
---|---|---|---|---|
1 | xxxxx | xxxxx | xxxx | xxxx |
1 | xxxxx | xxxxx | xxxx | xxxx |
1 | xxxxx | xxxxx | xxxx | xxxx |
CheckList
本部分是讲述,安装过程的checklist,也就是安装步骤总览,如表1-3所示:
类别 | 项目 | 当前进度 |
---|---|---|
基础 | 配置tinyproxy代理 | |
基础 | 配置sucurecrt脚本登陆跳板机之后登陆内网机 | |
资源 | 校核操作系统 | Centos7.5 64 位 |
资源 | 资源下载 | Jdk+cm+cdh |
Linux | Hostname-Hosts 互ping |
基础环境
tinyproxy代理设置
hadoop集群采用专用交换机纯内网模式,为方便使用yum等工具故在跳板机上安装代理。
tinyproxy依赖yum第三方EPEL存储库,首先我们要安装EPEL存储库
yum list | grep epel-release;
yum install -y epel-release
安装tinyproxy
yum -y install tinyproxy
配置 TinyProxy
vi /etc/tinyproxy/tinyproxy.conf
修改 Port 端口,默认为 8888
Port 12801
注释掉 Allow,表示允许所有人访问代理
#Allow 127.0.0.1
隐藏掉Via请求头部,去掉下面的注释
DisableViaHeader Yes
启动 TinyProxy
systemctl start tinyproxy.service
开放防火墙端口
iptables -A INPUT -p tcp --dport 12801 -j ACCEPT
永久保存
service iptables save
重启iptables
systemctl restart iptables;
检查是否启用端口
iptables -L -n
配置sucurecrt跳板机转内网机
8台内网机,每次通过跳板机登陆再ssh到内网机会很麻烦,可以通过录制脚本功能,可以很容易的录制一个脚本。然后修改,使用此脚本就可以轻易的登录内网机。
步骤
1、打开script–》Start Recording Script,然后就进行登录操作。当完成登录之后,停止录制。
2、保存并修改录制文件。
3、右击连接,properties,connection,Logon Actions,勾选Logon script,选中刚录制的脚本。
4、重新登录,就会发现已经能够自动操作刚才的登录过程。
PS:针对录制的脚本可以进行进一步修改,如果属性相关脚本,可以个性化定制一些打开的操作。
以下为录制好的脚本
#$language = "VBScript"
#$interface = "1.0"
crt.Screen.Synchronous = True
' This automatically generated script may need to be
' edited in order to work correctly.
Sub Main
crt.Screen.Send "ssh root@192.168.1.101" & chr(13)
crt.Screen.WaitForString "root@192.168.1.101's password: "
crt.Screen.Send "密码" & chr(13)
crt.Screen.WaitForString chr(27) & "]0;root@localhost:" & chr(126) & chr(7) & chr(27) & "[?1034h[root@localhost " & chr(126) & "]# "
crt.Screen.Send "hostname" & chr(13)
End Sub
跳板机安装图形界面
CENTOS是最小化安装的默认都是不带X WINDOWS的,首先安装X WINDOWS
yum upgrade
yum -y groupinstall "X Window System"
然后安装gnome
yum -y groups install "GNOME Desktop"
启动图形界面,重启
startx
reboot
重启后,使用vnc连接
跳板机yum安装chrome
创建yum源文件
cd /etc/yum.repos.d/
touch google-chrome.repo
配置yum源信息
[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
安装google chrome
yum -y install google-chrome-stable --nogpgcheck
系统基础配置
配置Hostname、yum、ntp、透明大页面等
Configure Network Names
配置hostname,集群所有节点通过hostname互相访问
sudo hostnamectl set-hostname sms-hdpc-cm01.transtech.com.cn
配置hosts文件
192.168.1.100 sms-hdpc-cm01.transtech.com.cn sms-hdpc-cm01
192.168.1.101 sms-hdpc-node01.transtech.com.cn sms-hdpc-node01
192.168.1.102 sms-hdpc-node02.transtech.com.cn sms-hdpc-node02
192.168.1.103 sms-hdpc-node03.transtech.com.cn sms-hdpc-node03
192.168.1.104 sms-hdpc-node04.transtech.com.cn sms-hdpc-node04
192.168.1.105 sms-hdpc-node05.transtech.com.cn sms-hdpc-node05
192.168.1.106 sms-hdpc-node06.transtech.com.cn sms-hdpc-node06
192.168.1.107 sms-hdpc-node07.transtech.com.cn sms-hdpc-node07
脚本如下
echo '192.168.1.100 sms-hdpc-cm01.transtech.com.cn sms-hdpc-cm01' >>/etc/hosts
echo '192.168.1.101 sms-hdpc-node01.transtech.com.cn sms-hdpc-node01' >>/etc/hosts
echo '192.168.1.102 sms-hdpc-node02.transtech.com.cn sms-hdpc-node02' >>/etc/hosts
echo '192.168.1.103 sms-hdpc-node03.transtech.com.cn sms-hdpc-node03' >>/etc/hosts
echo '192.168.1.104 sms-hdpc-node04.transtech.com.cn sms-hdpc-node04' >>/etc/hosts
echo '192.168.1.105 sms-hdpc-node05.transtech.com.cn sms-hdpc-node05' >>/etc/hosts
echo '192.168.1.106 sms-hdpc-node06.transtech.com.cn sms-hdpc-node06' >>/etc/hosts
echo '192.168.1.107 sms-hdpc-node07.transtech.com.cn sms-hdpc-node07' >>/etc/hosts
配置yum代理
通过跳板机的tinyproxy访问yum源
echo "proxy=http://192.168.1.204:12801" >>/etc/yum.conf
然后yum clean all、yum makecache
关闭防火墙
centos默认使用了firewall防火墙
sudo systemctl disable firewalld
sudo systemctl stop firewalld
关闭SELinux
本次安装的cdh6支持SELinux宽容模式,以防万一我们还是根据以往的已经设置为关闭状态,具体参考官方6.1文档
getenforce
sed -i '7d' /etc/selinux/config
echo 'SELINUX=disabled' >>/etc/selinux/config
配置ntp时间同步
采用ntpd,所以安装ntpd之前要卸载chronyd,否则会影响ntpd正常工作
首先在跳板机中安装ntp服务
yum install ntp -y
可以在/etc/ntp.conf中指定远端时间服务器地址和允许访问的网段等。
启动,并查看运行状态
systemctl start ntpd
ntpq -p
可以手动同步一次,
ntpdate 0.centos.pool.ntp.org
把ntp需要的udp 123端口加入到防火墙策略中
ntp时间服务器至此安装结束,剩下的配置在所有集群节点中完成,
所有节点卸载chrony,并安装ntp
yum remove chrony
yum install ntp
编辑/etc/ntp.conf,修改时间服务器为我们刚刚配置的跳板机
server 192.168.1.204
开启ntpd服务,并开机启动,
sudo systemctl start ntpd```
sudo systemctl enable ntpd
过一点时间观察ntpstat命令,查看是否同步成功。
最后,将系统时钟同步到硬件时钟
hwclock --systohc
关闭tuned服务
Ensure that the tuned service is started:
systemctl start tuned
Turn the tuned service off:
tuned-adm off
Ensure that there are no active profiles:
tuned-adm list
The output should contain the following line:
No current active profile
Shutdown and disable the tuned service:
systemctl stop tuned
systemctl disable tuned
关闭透明大页面
执行一下命令,查看thp状态
cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/enabled
正常是打开状态,如下,
[always] never
增加如下脚本到/etc/rc.d/rc.local
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
命令如下
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.d/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.d/rc.local
给/etc/rc.d/rc.local赋权确保启动生效
chmod +x /etc/rc.d/rc.local
增加如下脚本到/etc/default/grub
transparent_hugepage=never
命令如下
echo 'transparent_hugepage=never' >>/etc/default/grub
执行命令
grub2-mkconfig -o /boot/grub2/grub.cfg
调整vm.swappiness
vm.swappiness大小代表交换到虚拟内存的积极性,越大越容易交换到虚拟内存中,cloudera建议设置为0
首先查看vm.swappiness的值,
cat /proc/sys/vm/swappiness
修改vm.swappiness为0
echo vm.swappiness = 0 >> /etc/sysctl.conf
解除文件句柄数限制
解除 Linux 系统的最大进程数和最大文件打开数限制:
在/etc/security/limits.conf中添加如下的行
- soft noproc 11000
- hard noproc 11000
- soft nofile 65535
- hard nofile 65535
命令如下:
echo '* soft noproc 11000'>>/etc/security/limits.conf
echo '* hard noproc 11000'>>/etc/security/limits.conf
echo '* soft nofile 65535'>>/etc/security/limits.conf
echo '* hard nofile 65535'>>/etc/security/limits.conf
无密钥登陆
关闭ipv6
cdh仅支持ipv4,不支持ipv6
命令如下
echo 'net.ipv6.conf.all.disable_ipv6 = 1'>>/etc/sysctl.conf
echo 'net.ipv6.conf.default.disable_ipv6 = 1'>>/etc/sysctl.conf
磁盘挂载
建议通过uuid挂载,使用blkid命令查询磁盘uuid,文件系统选择ext4,比xfs随机读写性能更好,优化参数选择NOATIME,选择NOATIME之后无需再加NODIRATIME,修改完fstab之后记得用mount -a进行检测
在内核源代码 linux-2.6.33/fs/inode.c 文件里有一个 touch_atime 函数,可以看出如果 inode 的标记位是 NOATIME 的话就直接返回了,根本就走不到 NODIRATIME 那里去,所以只设置 noatime 就可以了,不必再设置
安装cm和cdh
网速好的话可以使用cloudera提供的远程仓库,由于cloudera网络不稳定,时快时慢,这里建议使用下载cm和cdh并配置成本地仓库
首先我们要部署一个web服务器
Install Apache HTTP Server:
sudo yum install httpd
Start Apache HTTP Server:
sudo systemctl start httpd
开机启动,
sudo systemctl enable httpd
在httpd配置文件/etc/httpd/conf/httpd.conf中的的IfModule mime_module模块中添加如下脚本
AddType application/x-gzip .gz .tgz .parcel
官方文档里说,如果不添加会有hash校验错误
Warning: Skipping this step could result in an error message Hash verification failed when trying to download the parcel from a local repository, especially in Cloudera Manager 6 and higher.
cm节点创建cm本地永久yum仓库
下载cm6.1.1的yum仓库到httpd中
sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cm6/6.1.1/redhat7/ -P /var/www/html/cloudera-repos
sudo wget https://archive.cloudera.com/cm6/6.1.1/allkeys.asc -P /var/www/html/cloudera-repos/cm6/6.1.1/
赋权
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm6
创建yum配置文件/etc/yum.repos.d/cloudera-repo.repo,并填入以下信息
[cloudera-repo]
name=cloudera-repo
baseurl=http://192.168.1.100/cloudera-repos/cm6/6.1.1/redhat7/yum
enabled=1
gpgcheck=0
cm节点创建cdh本地永久parcel仓库
下载cdh6.1.1到httpd中
sudo mkdir -p /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/cdh6/6.1.1/parcels/ -P /var/www/html/cloudera-repos
sudo wget --recursive --no-parent --no-host-directories https://archive.cloudera.com/gplextras6/6.1.1/parcels/ -P /var/www/html/cloudera-repos
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cdh6
sudo chmod -R ugo+rX /var/www/html/cloudera-repos/gplextras6
至此cdh本地仓库配置完毕,下一步安装cdh的时候填入http://<web_server>/cloudera-parcels/cdh6/6.1.1/即可。
cm节点安装oracle jdk
cdh6目前已经支持openjdk,我们还是使用oracle jdk,配置完前一步的cm本地源之后,我们可以通过yum直接安装oracle jdk,如果有系统自带的openjdk记得卸载,安装命令如下
sudo yum install oracle-j2sdk1.8
安装cm
使用前一步的cm yum源安装cm,命令如下:
sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
开启Auto-TLS
cloudera推荐开始Auto-TLS证书,命令如下:
sudo JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera /opt/cloudera/cm-agent/bin/certmanager setup --configure-services
安装并配置mysql
配置mysql的yum仓库并安装
wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm
sudo yum update
sudo yum install mysql-server
sudo systemctl start mysqld
sudo systemctl enable mysqld
配置mysql配置文件/etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M
#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log
#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES
开启mysql
sudo systemctl start mysqld
运行/usr/bin/mysql_secure_installation配置root密码等
sudo /usr/bin/mysql_secure_installation
[...]
Enter current password for root (enter for none):
OK, successfully used password, moving on...
[...]
Set root password? [Y/n] Y
New password:
Re-enter new password:
Remove anonymous users? [Y/n] Y
[...]
Disallow root login remotely? [Y/n] N
[...]
Remove test database and access to it [Y/n] Y
[...]
Reload privilege tables now? [Y/n] Y
All done!
下载mysql jdbc驱动
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
解压
tar zxvf mysql-connector-java-5.1.46.tar.gz
复制到/usr/share/java/
sudo mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46
sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar
初始化cm相关数据库
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'transtech9ol.';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'transtech9ol.';
配置scm数据库
执行以下脚本
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
如果报 Access denied错误,试试删除空用户
mysql脚本如下
DELETE FROM `mysql`.`user` WHERE `user`='';
FLUSH PRIVILEGES;
##cm控制台安装cdh和其他软件