cdh6.3.x安装文档

一.安装文档链接

官方 https://docs.cloudera.com/documentation/enterprise/6/6.0/topics/installation.html

网友贡献1 https://www.cnblogs.com/swordfall/p/10816797.html

网友贡献2 https://blog.csdn.net/TXBSW/article/details/84648269

二.配置hosts

vim /etc/hosts

将集群的每台机器的域名与ip 的映射追加至/etc/hosts

三.关闭防火墙

临时关闭防火墙

# systemctl stop firewalld

永久防火墙开机自启动

# systemctl disable firewalld

查看防火墙状态

# systemctl status firewalld

四.selinux关闭

# vim /etc/sysconfig/selinux

SELINUX=disabled

重启机器，命令：

# reboot

重启后检查：

# sestatus –v

五.mysql

mysql版本建议 5.7.29 MySQL Community Server (GPL)

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

GRANT ALL ON scm.* TO 'cdh6'@'%' IDENTIFIED BY '12345678';

GRANT ALL ON amon.* TO 'cdh6'@'%' IDENTIFIED BY '12345678';

GRANT ALL ON hue.* TO 'cdh6'@'%' IDENTIFIED BY '12345678';

GRANT ALL ON metastore.* TO 'cdh6'@'%' IDENTIFIED BY '12345678';

GRANT ALL ON rman.* TO 'cdh6'@'%' IDENTIFIED BY '12345678’;

GRANT ALL ON sentry.* TO 'cdh6'@'%' IDENTIFIED BY 'aB!12345678';

FLUSH PRIVILEGES;

六.安装jdk

安装之前先卸载CentOS7自带的JDK

选择 jdk-8u181-linux-x64.tar.gz 下载 https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html

查看系统已经装的jdk：

# rpm -qa|grep jdk

卸载jdk：

# rpm -e --nodeps java-1.6.0-openjdk-javadoc-1.6.0.0-1.66.1.13.0.el6.x86_64

1.下载“jdk-8u131-linux-x64.tar.gz”，放到/usr/java/目录下

2.解压

# tar -xzvf jdk-8u131-linux-x64.tar.gz

3.在任何目录下均可编辑

# vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_131

export JRE_HOME=$JAVA_HOME/jre

export PATH=$PATH:$JAVA_HOME/bin

export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib

七.配置NTP服务（所有节点）

a.修改时区（改为中国标准时区）ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

b.安装ntp yum -y install ntp

c.ntp主机配置 vim /etc/ntp.conf

sed 's/tinker/#tinker/g' -i /etc/ntp.conf

重新启动 ntp 服务：systemctl restart ntpd.service

设置开机自启：systemctl enable ntpd.service

ntpdc -c loopinfo #查看与时间同步服务器的时间偏差

ntpq -p #查看当前同步的时间服务器

ntpstat #查看状态

5分钟后执行 ntptime 查看是否存在error，没有的话则代表启动成功

八.配置各节点之间ssh免密登录

@运维

九.优化虚拟内存需求率

检查虚拟内存需求率

# cat /proc/sys/vm/swappiness

显示如下：

30

临时降低虚拟内存需求率

# sysctl vm.swappiness=10

永久降低虚拟内存需求率

# echo 'vm.swappiness = 10' > /etc/sysctl.d/swappiness.conf

并运行如下命令使生效

# sysctl -p

十.检查透明大页面问题

# cat /sys/kernel/mm/transparent_hugepage/defrag

如果显示为：

[always] madvise never

临时关闭透明大页面问题

# echo never > /sys/kernel/mm/transparent_hugepage/defrag

确认配置生效

# cat /sys/kernel/mm/transparent_hugepage/defrag

应该显示为：

always madvise [never]

配置开机自动生效

# echo never > /sys/kernel/mm/transparent_hugepage/enabled

十一.拷贝mysql-connector-java到各个节点指定目录下

mkdir -p /usr/share/java

cp mysql-connector-java-5.1.44-bin.jar /usr/share/java/mysql-connector-java.jar

十二.配置 Cloudera Manager 仓库

a.wget ' https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/cloudera-manager.repo' -P /etc/yum.repos.d/

b.rpm --import https://archive.cloudera.com/cm6/6.3.1/redhat7/yum/RPM-GPG-KEY-cloudera

c.yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server ( 主节点，其余执行 yum install cloudera-manager-daemons cloudera-manager-agent )

十三.设置 Cloudera Manager 数据库

cd /opt/cloudera/cm/schema/

sh scm_prepare_database.sh mysql -h127.0.0.1 -uroot -p'Root!!2018' --scm-host 10.9.43.67 db_cdh6_scm root 'Root!!2018’

最后出现 All done, your SCM database is configured correctly ! 代表初始化数据库成功

十四.修改scm-agent配置

sudo vi /etc/cloudera-scm-agent/config.ini

server_host=cdh1 （这里我是以cdh1作为scm-server节点）

十五.启动 cloudera-scm-server

systemctl start cloudera-scm-server

十六.错误汇总

a.kudu 启动失败问题

Check failed: _s.ok() Bad status : Service unavailable: Cannot initialize clock: Error reading clock. Clock considered unsynchronized

sed -i 's/tinker/#tinker/g' /etc/ntp.conf

详见 https://blog.csdn.net/liaynling/article/details/87164805