CDH 安装
三台服务器配置如下:
IP | hostname | 配置 |
---|---|---|
192.168.13.121 | centos-13-121 | 4C16G |
192.168.13.122 | centos-13-122 | 4C16G |
192.168.13.123 | centos-13-123 | 4C16G |
软件如下:
MySQL-5.6.43-1.el7.x86_64.rpm-bundle.tar
cm5.15.0-centos7.tar.gz
jdk-8u162-linux-x64.rpm
CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel
1.1 ntp 时间同步
timedatectl set-timezone Asia/Shanghai
yum install ntpdate
yum install ntp
systemctl restart ntpd
1.2 java配置
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_162
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
1.3 安装mysql
略
2 安装cloudera manager
2.1 安装一些依赖
每个节点都要安装
yum install net-tools lsof sysstat lrzsz rsync cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi psmisc dstat openssh-server openssl -y
yum install python-psycopg2 mod_ssl openssl-devel fuse-libs portmap fuse bind-utils libxslt perl lsb MySQL-python -y
2.2 安装服务端
服务端
rpm -ivh cloudera-manager-server-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm cloudera-manager-daemons-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm cloudera-manager-agent-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm
客户端(每个节点都要安装)
rpm -ivh cloudera-manager-daemons-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm cloudera-manager-agent-5.15.0-1.cm5150.p0.62.el7.x86_64.rpm
2.3 配置安装源
在server端执行
cd /opt/cloudera/parcel-repo
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel.sha1
# 将后缀改成 sha
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/CDH-5.13.2-1.cdh5.13.2.p0.3-el7.parcel
wget http://archive.cloudera.com/cdh5/parcels/5.13.2/manifest.json
2.4 初始化数据库
在mysql里面创建scm数据库,数据库的用户 和密码都是scm,scm用户对scm数据库拥有所有的权限
cd /usr/share/cmf/schema/
./scm_prepare_database.sh mysql -h 192.168.13.121 --scm-host centos-13-121 scm scm scm
检查配置
cat /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=192.168.13.121
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.setupType=EXTERNAL
com.cloudera.cmf.db.password=scm
启动server进程
/etc/init.d/cloudera-scm-server start
初始化数据会有一段时间
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
成功后出现如下日志
2019-03-29 10:24:49,075 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180
2019-03-29 10:24:49,075 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
2019-03-29 10:24:49,745 INFO SearchRepositoryManager-0:com.cloudera.server.web.cm
2.5 客户端配置
vi /etc/cloudera-scm-agent/config.ini
server_host=centos-13-121
需要同步配置到各节点
2.6 开始安装
浏览器访问http://192.168.13.121:7180, 用户名/密码: admin/admin
安装hive报错
解决方式
cp mysql-connector-java-5.1.46.jar /opt/cloudera/parcels/CDH-5.13.2-1.cdh5.13.2.p0.3/lib/hive/lib/
安装oozie报错
解决方式
cp mysql-connector-java-5.1.46.jar /var/lib/oozie/
chown -R oozie.oozie /var/lib/oozie/
安装完成,如下图