centos7安装cdh

CDH 安装

---

三台服务器配置如下：

IP	hostname	配置
192.168.13.121	centos-13-121	4C16G
192.168.13.122	centos-13-122	4C16G
192.168.13.123	centos-13-123	4C16G

软件如下：

MySQL-5.6.43-1.el7.x86_64.rpm-bundle.tar
cm5.15.0-centos7.tar.gz
jdk-8u162-linux-x64.rpm
CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel

1.1 ntp 时间同步

timedatectl set-timezone Asia/Shanghai
yum install  ntpdate
yum install  ntp 
systemctl restart ntpd

1.2 java配置

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_162
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

1.3 安装mysql

略

2 安装cloudera manager

2.1 安装一些依赖

每个节点都要安装
yum install net-tools lsof sysstat lrzsz  rsync cyrus-sasl-plain  cyrus-sasl-devel  cyrus-sasl-gssapi   psmisc dstat openssh-server openssl -y
yum install python-psycopg2  mod_ssl openssl-devel fuse-libs portmap  fuse bind-utils libxslt perl lsb MySQL-python  -y

2.2 安装服务端

服务端
rpm -ivh cloudera-manager-server-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm  cloudera-manager-daemons-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm cloudera-manager-agent-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm 

客户端（每个节点都要安装）
rpm -ivh cloudera-manager-daemons-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm cloudera-manager-agent-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm 

2.3 配置安装源

在server端执行
cd  /opt/cloudera/parcel-repo
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel.sha1
# 将后缀改成 sha
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/manifest.json

2.4 初始化数据库

在mysql里面创建scm数据库，数据库的用户 和密码都是scm，scm用户对scm数据库拥有所有的权限
cd /usr/share/cmf/schema/
./scm_prepare_database.sh mysql -h 192.168.13.121 --scm-host centos-13-121  scm scm scm

检查配置
cat /etc/cloudera-scm-server/db.properties

com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=192.168.13.121
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.setupType=EXTERNAL
com.cloudera.cmf.db.password=scm

启动server进程
/etc/init.d/cloudera-scm-server start 

初始化数据会有一段时间

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

成功后出现如下日志

2019-03-29 10:24:49,075 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180
2019-03-29 10:24:49,075 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
2019-03-29 10:24:49,745 INFO SearchRepositoryManager-0:com.cloudera.server.web.cm

2.5 客户端配置

vi   /etc/cloudera-scm-agent/config.ini
server_host=centos-13-121
需要同步配置到各节点

2.6 开始安装

浏览器访问http://192.168.13.121:7180， 用户名/密码: admin/admin

安装hive报错

解决方式

cp mysql-connector-java-5.1.46.jar /opt/cloudera/parcels/CDH-5.13.2-1.cdh5.13.2.p0.3/lib/hive/lib/

安装oozie报错

解决方式

cp mysql-connector-java-5.1.46.jar /var/lib/oozie/
chown -R oozie.oozie /var/lib/oozie/

安装完成，如下图