从零开始虚拟机安装单节点CDH5.15

环境和离线文件

软件	VMwave Workstation15
	Xshell6
文件	CentOS-7-x86_64-Minimal-1804.iso
	mysql-connector-java-5.1.47.tar.gz
	CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel
	CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1
	manifest.json
	cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz
配置分配	cpu i5-8300H 8核
	内存 13.4G
	硬盘 40G固态
占用情况	启动CDH后占9.4G内存
	关闭Cloudera Management Service可以节省2G内存
	硬盘总占用12G

安装CentOS7

图形界面选择语言中文,打开网络设置,配置静态ip 192.168.5.140 主机名master.hadoop,配置root密码123456
使用xshell登陆,
配置终端提示符颜色

vi .bashrc
export PS1='[\[\e[1;34;1m\]\u@\h \W\[\e[0m\]]$ '
source .bashrc

更换yum源

yum install -y wget
cd /etc/yum.repos.d
sudo mv CentOS-Base.repo CentOS-Base.repo.bak
wget http://mirrors.163.com/.help/CentOS7-Base-163.repo
mv CentOS7-Base-163.repo CentOS-Base.repo
yum clean all
yum makecache
#更新软件,安装常用软件
yum update -y
yum install -y openssh-server vim gcc gcc-c++ glibc-headers bzip2-devel lzo-devel curl openssh-clients zlib-devel autoconf automake cmake libtool openssl-devel fuse-devel snappy-devel telnet unzip zip net-tools.x86_64 firewalld systemd lrzsz psmisc

配置host

vi /etc/hosts
尾部添加192.168.5.140 master.hadoop

配置SSH免密登陆

主要分为两个步骤：首先在所有的节点生成公钥
ssh-keygen -t rsa
然后将所有的节点执行拷贝公钥
ssh-copy-id root@master.hadoop

来自 https://blog.csdn.net/fffsssfff6/article/details/81215416

关闭selinux

vim /etc/selinux/config
把文件里的SELINUX=disabled
关闭防火墙和查看防火墙状态：
systemctl stop firewalld  
systemctl disable firewalld
systemctl status firewalld

来自 https://blog.csdn.net/fffsssfff6/article/details/81215416

安装jdk1.8

使用rz上传jdk-8u191-linux-x64.tar.gz  mysql-connector-java-5.1.47.tar.gz
mkdir -p /usr/java
tar -xzf jdk-8u191-linux-x64.tar.gz -C /usr/java/
 
vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_191
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
 
source /etc/profile

安装mysql

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
rpm -ivh mysql-community-release-el7-5.noarch.rpm
yum update
yum install mysql-server
systemctl enable mysqld
systemctl start mysqld
#设置mysql root密码
/usr/bin/mysql_secure_installation

添加jdbc驱动

mkdir -p /usr/share/java/
tar xzf mysql-connector-java-5.1.47.tar.gz
cd mysql-connector-java-5.1.47
mv mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar

来自 https://www.cloudera.com/documentation/enterprise/5-15-x/topics/cm_ig_mysql.html#cmig_topic_5_5

为CDH创建数据库和用户

mysql -u root -p123456 
#输入密码登陆mysql ，然后创建多个数据库，并完成授权。
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';

来自 https://blog.csdn.net/fffsssfff6/article/details/81215416

导入repo(可不做)

wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo -P /etc/yum.repos.d/
rpm --import https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera

来自https://www.cloudera.com/documentation/enterprise/5-15-x/topics/configure_cm_repo.html
https://www.cloudera.com/documentation/enterprise/5-15-x/topics/configure_cm_repo.html

解压cm

tar zxf cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/

初始化CM5的数据库

/opt/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm

配置cloudera-manger-agent指向主节点服务器

vim /opt/cm-5.15.1/etc/cloudera-scm-agent/config.ini
server_host=master.hadoop

来自 https://blog.csdn.net/shawnhu007/article/details/52579204

创建cloudera-scm用户

useradd --system --home=/opt/cm-5.15.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
#来自 <https://blog.csdn.net/gtsina/article/details/78048925> 
 
#将cdh安装包放到/opt/cloudera/parcel-repo/下,其中CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1要把末尾1去掉
cd /opt/cloudera/parcel-repo/
mv ~/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel ./
mv ~/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha ./
mv ~/manifest.json ./

/opt/cm-5.15.1/etc/init.d/cloudera-scm-server start启动服务端。
/opt/cm-5.15.1/etc/init.d/cloudera-scm-agent start启动Agent服务。
通过http://192.168.5.140:7180访问

• 安装问题

• 问题:hue 数据库连接不上

1，Unexpected error. Unable to verify database connection
查看日志报错是：ImportError: libxslt.so.1: cannot open shared object file: No such file or directory
原因是centos缺少库文件，执行如下命令即可
解决：

yum install krb5-devel cyrus-sasl-gssapi cyrus-sasl-deve libxml2-devel libxslt-devel mysql mysql-devel openldap-devel python-devel python-simplejson sqlite-devel

---

作者：陈苏漾
来源：CSDN
原文：https://blog.csdn.net/chshgod1/article/details/80475596

• 问题:启动HUE Load Balancer角色失败。

原因：服务器环境没有预先安装httpd，mod_ssl服务
解决方法：

yum -y install httpd
yum -y install mod_ssl

来自 https://segmentfault.com/q/1010000013552053

• 配置HUE 遇到Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found

解决方法：

yum install cyrus-sasl-plain  cyrus-sasl-devel  cyrus-sasl-gssapi

来自 https://blog.csdn.net/youngqj/article/details/50954275?locationNum=2

结果

参考:https://blog.csdn.net/gtsina/article/details/78048925