CDH 6.1安装过程

目录

一、JDK

二、配置镜像文件

三、上传数据库上传 mysql-5.6.39-linux-glibc2.12-x86_64.tar 到 /usr/local/

四、安装Mysql

五、安装所有节点：yum clean allyum makecache

五、运行

---

前文：

          CDH的好处太多不一一列举了。

一、JDK

cd /opt/software
rm -rf  jdk1.8.0_181
rm -rf /opt/software/jdk-8u181-linux-x64.tar.gz   

所有节点：
yum -y install rsync
yum -y install rpcbind

mkdir /usr/java/

主节点：
上传 oracle-j2sdk1.8-1.8.0+update141-1.x86_64 到/usr/java/

cd /usr/java/
rpm -ivh oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm
mv jdk1.8.0_141-cloudera/ jdk1.8/

scp -r /usr/java/jdk1.8 root@bt-02:/usr/java/
scp -r /usr/java/jdk1.8 root@bt-03:/usr/java/

所有节点：

vim /etc/profile

JAVA_HOME=/usr/java/jdk1.8
JRE_HOME=/usr/java/jdk1.8/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
export JAVA_HOME JRE_HOME CLASS_PATH PATH

source /etc/profile

java -version

二、配置镜像文件

主节点：
mkdir /var/www/html/cm6.1.0
mkdir /var/www/html/cdh6.1.0

cp /opt/CDH610/* /var/www/html/cdh6.1.0/
cp /opt/CM610/* /var/www/html/cm6.1.0/

createrepo /var/www/html/cm6.1.0/
createrepo /var/www/html/cdh6.1.0/

cd /etc/yum.repos.d/ 
    
vim cm.repo
[cmrepo]
name = cm_repo
baseurl =http://bt-01/cm6.1.0/
enable = true
gpgcheck = false

vim cdh.repo
[cdhrepo]
name = cdh_repo
baseurl =http://bt-01/cdh6.1.0/
enable = true
gpgcheck = false

rsync /etc/yum.repos.d/* root@bt-02:/etc/yum.repos.d/
rsync /etc/yum.repos.d/* root@bt-03:/etc/yum.repos.d/

ll /etc/yum.repos.d/

主：资源包只要放在主节点

切换镜像：
所有节点：
cd /etc/yum.repos.d/
rename .repo .repo.bak *  
mv cm.repo.bak  cm.repo
mv cdh.repo.bak  cdh.repo

三、上传数据库
上传 mysql-5.6.39-linux-glibc2.12-x86_64.tar 到 /usr/local/

***********************************************************
        保        存
***********************************************************

四、安装Mysql

所有节点：
rpm -qa | grep -i mysql
rpm -e --nodeps MySQL-python-1.2.5-1.el7.x86_64  (不该执行这一步)

rpm -qa | grep -i mariadb
rpm -e --nodeps mariadb-libs-5.5.60-1.el7_5.x86_64

find / -name mysql
rm -rf /etc/selinux/targeted/active/modules/100/mysql
rm -rf /usr/lib64/mysql

主节点：
useradd mysql
passwd mysql

cd /usr/local
tar -xzvf mysql-5.6.39-linux-glibc2.12-x86_64.tar.gz
mv mysql-5.6.39-linux-glibc2.12-x86_64 mysql
chown -R mysql:mysql /usr/local/mysql

/usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data
cp my-default.cnf  /etc/my.cnf 
cp mysql.server /etc/init.d/mysql

vi /etc/my.cnf
basedir=/usr/local/mysql
datadir=/usr/local/mysql/data

vi /etc/init.d/mysql
basedir=/usr/local/mysql
datadir=/usr/local/mysql/data

chkconfig --add mysql
chkconfig mysql on

service mysql start

/usr/local/mysql/bin/mysqladmin -u root password 123456

vi /etc/profile

JAVA_HOME=/usr/java/jdk1.8
JRE_HOME=/usr/java/jdk1.8/jre
MYSQL_HOME=/usr/local/mysql
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:$MYSQL_HOME/bin

export JAVA_HOME JRE_HOME CLASS_PATH PATH MYSQL_HOME

source /etc/profile

mysql -u root -p123456
grant all privileges on *.* to root@"%" identified by "123456";
flush privileges;

mkdir /usr/share/java
把 mysql-connector-java.jar 放入/usr/share/java中

这一步权限非常重要，最后一次还要操作一次

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456';
flush privileges;

主节点：
上传 MySQL-shared-5.6.47-1.el7.x86_64.rpm 到 /opt/software
rpm -ivh MySQL-shared-5.6.47-1.el7.x86_64.rpm

##################################

五、安装
所有节点：
yum clean all
yum makecache

重装：(误删)
cd /etc/yum.repos.d/
mv CentOS-Base.repo.bak CentOS-Base.repo
yum -y install MySQL-python
mv CentOS-Base.repo CentOS-Base.repo.bak

注意：作为依赖被安装:
  mariadb-libs.x86_64 1:5.5.64-1.el7 

主节点：
yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

其他节点：
yum install cloudera-manager-daemons cloudera-manager-agent

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
cp -r /var/www/html/cdh6.1.0/* /opt/cloudera/parcel-repo/

五、运行

主节点启动：
systemctl start cloudera-scm-server

#所有节点启动：
#systemctl start cloudera-scm-agent

netstat -ntlp |grep 7180
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

数据库用户权限访问主机确认

进入http://bt-01:7180

bt-01,bt-02,bt-03
http://bt-01/cdh6.1.0/
http://bt-01/cm6.1.0/

六、关闭集群
systemctl stop cloudera-scm-server
systemctl stop cloudera-scm-agent

 

相关参考：https://blog.csdn.net/qq_39680564/article/details/100923644#comments

 

六、补充

6.1 界面

6.2 发行版本

组件	版本	发行版	CDH 版本
Supervisord	3.0	不可用	不适用
Cloudera Manager Agent	6.1.0	769885.el7	不适用
Cloudera Manager Management Daemon	6.1.0	769885.el7	不适用
Flume NG	1.8.0+cdh6.1.0	770702	CDH 6.1.0
Hadoop	3.0.0+cdh6.1.0	770702	CDH 6.1.0
HDFS	3.0.0+cdh6.1.0	770702	CDH 6.1.0
HttpFS	3.0.0+cdh6.1.0	770702	CDH 6.1.0
hadoop-kms	3.0.0+cdh6.1.0	770702	CDH 6.1.0
MapReduce 2	3.0.0+cdh6.1.0	770702	CDH 6.1.0
YARN	3.0.0+cdh6.1.0	770702	CDH 6.1.0
HBase	2.1.0+cdh6.1.0	770702	CDH 6.1.0
Lily HBase Indexer	1.5+cdh6.1.0	770702	CDH 6.1.0
Hive	2.1.1+cdh6.1.0	770702	CDH 6.1.0
HCatalog	2.1.1+cdh6.1.0	770702	CDH 6.1.0
Hue	3.9.0+cdh6.1.0	770702	CDH 6.1.0
Impala	3.1.0+cdh6.1.0	770702	CDH 6.1.0
Java 8	java version "1.8.0_162" Java(TM) SE Runtime Environment (build 1.8.0_162-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)	不可用	不适用
Kafka	2.0.0+cdh6.1.0	770702	CDH 6.1.0
Kite（仅限 CDH 5 ）	1.0.0+cdh6.1.0	770702	CDH 6.1.0
kudu	1.8.0+cdh6.1.0	770702	CDH 6.1.0
Oozie	5.0.0-beta1+cdh6.1.0	770702	CDH 6.1.0
Parquet	1.9.0+cdh6.1.0	770702	CDH 6.1.0
Pig	0.17.0+cdh6.1.0	770702	CDH 6.1.0
sentry	2.1.0+cdh6.1.0	770702	CDH 6.1.0
Solr	7.4.0+cdh6.1.0	770702	CDH 6.1.0
spark	2.4.0+cdh6.1.0	770702	CDH 6.1.0
Sqoop	1.4.7+cdh6.1.0	770702	CDH 6.1.0
ZooKeeper	3.4.5+cdh6.1.0	770702	CDH 6.1.0