cdh-6离线安装

一.环境基础操作

1.配置host
集群节点
xxxx master
xxxx masterb
xxxx slave1
xxxx slave2
xxxx slave3
xxxx slave4
xxxx slave5

2.jdk
配置环境变量 /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_201-amd64
export PATH=$PATH:$JAVA_HOME/bin

3.
基础依赖软件
yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb mod_ssl unzip lrzsz

4.
关闭防火墙
systemctl stop firewalld && systemctl disable firewalld

5.
时间同步
yum install ntp ntpdate -y
systemctl start ntpd && systemctl enable ntpd

6.
免密登录配置

ssh-keygen -t rsa

ssh-copy-id masterb
ssh-copy-id slave1
ssh-copy-id slave2
ssh-copy-id slave3
ssh-copy-id slave4
ssh-copy-id slave5

master,masterb 需要与各个 slave 之间互相免密
...

CDH官网

2021-01-31 下载所有版本均需持有License

https://cn.cloudera.com/downloads/paywall-expansion.html
https://docs.cloudera.com/cdp-private-cloud-base/7.1.3/managing-clusters/topics/cm-accessing-license-page.html

二.下载资源 

1.官网
Cloudera Manager CDH

2.百度云资源
1).
下载cm-6.3.2

cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
allkeys.asc

2).
下载cdh-6.3.2

CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256
manifest.json

三.安装

#说明
cm6及cdh6安装所有操作仅在master节点即可,无需scp文件到其它节点:安装agent会自动copy文件到不同节点

1.
创建目录,供httpd服务访问
mkdir -p /var/www/html/cm-6.3.2
mkdir -p /var/www/html/cdh-6.3.2

2.
上传cm文件到
/var/www/html/cm-6.3.2

上传cdh文件到
/var/www/html/cdh-6.3.2

mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha

3.
创建cm rpm元数据
cd /var/www/html/cm-6.3.2
yum install createrepo
createrepo .

4.
编辑本地yum资源

1).
vim /etc/yum.repos.d/os.repo

[osrepo]
name=os_repo
baseurl=http://master/cm-6.3.2
enabled=true
gpgcheck=false

2).
vim /etc/yum.repos.d/cm.repo

[cmrepo]
name=cm_repo
baseurl=http://master/cm-6.3.2
enable=true
gpgcheck=false

3).
vim /etc/httpd/conf/httpd.conf

AddType application/x-gzip .gz .tgz
修改为
AddType application/x-gzip .gz .tgz .parcel

yum repolist

5.安装httpd服务

yum install httpd
systemctl start httpd
ps -ef | grep httpd 
systemctl restart httpd

浏览器访问
http://master/cdh6.2/
http://master/cm6.2/

6.创建mysql jdbc连接环境
mkdir -p /usr/share/java
上传mysql-connector-java-5.6.x.jar
mv mysql-connector-java-5.6.x.jar mysql-connector-java.jar
mkdir -p /usr/java
ln -s /opt/jdk1.8.0_141  /usr/java/default

7.
安装cm
yum -y install cloudera-manager-server

8.
初始化mysql的cm元数据

1).
#root登录mysql
mysql -uroot -hmaster -proot

#创建数据库
CREATE DATABASE IF NOT EXISTS cm;

#创建用户及权限
GRANT ALL ON cm.* TO 'cm'@'%' IDENTIFIED BY 'cm';
GRANT ALL ON cm.* TO 'cm'@'localhost' IDENTIFIED BY 'cm';
GRANT ALL ON cm.* TO 'cm'@'master' IDENTIFIED BY 'cm';

flush privileges;

exit

2).
#执行初始化 cm 用户 cm 密码 cm 数据库
#/opt/cloudera/cm/schema/scm_prepare_database.sh 为yum默认安装cm路径
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h master -uroot -proot cm cm cm

9.启动Cloudera Manager Server

1).
systemctl start cloudera-scm-server

ps -ef | grep cloudera-scm-server

systemctl status cloudera-scm-server

yum install net-tools

netstat -lnpt | grep 7180

2).
http://master:7180   admin/admin

10.添加集群

1).
如有:Setup Auto-TLS 跳过

2).
jdk 跳过

3).
Cloudera Manager Agent选:自定义存储库
http://master/cm-6.3.2   #上面自定义yum指定

4).
使用Parcel:
远程Parcel存储库URL:
http://master/cdh-6.3.2  #上面自定义yum指定 且 其它的全部删掉

11.组件安装

hive/hue/oozie元数据库配置

mysql -uroot -hmaster -proot

CREATE DATABASE IF NOT EXISTS hive;
CREATE DATABASE IF NOT EXISTS hue;
CREATE DATABASE IF NOT EXISTS oozie;

GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
GRANT ALL ON hive.* TO 'hive'@'master' IDENTIFIED BY 'hive';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
GRANT ALL ON hue.* TO 'hue'@'master' IDENTIFIED BY 'hue';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';
GRANT ALL ON oozie.* TO 'oozie'@'master' IDENTIFIED BY 'oozie';

CREATE DATABASE IF NOT EXISTS hive;
CREATE DATABASE IF NOT EXISTS hue;
CREATE DATABASE IF NOT EXISTS oozie;

flush privileges;

参考链接:

https://blog.csdn.net/zimiao552147572/article/details/89817025

https://blog.csdn.net/zimiao552147572/article/details/87194241