CDH6.3.1-安装步骤

最新推荐文章于 2024-06-12 18:02:46 发布

果丶果

最新推荐文章于 2024-06-12 18:02:46 发布

阅读量2.5k

点赞数 5

分类专栏：大数据

本文链接：https://blog.csdn.net/m0_37986160/article/details/108034803

版权

大数据专栏收录该内容

4 篇文章 0 订阅

订阅专栏

CDH6.3.1-安装步骤

注意-所有主机密码要一致

准备安装包

MySQL5.7

mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar

MySQL驱动包

mysql-connector-java.jar

cloudera-manager包

cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
allkeys.asc

parcel数据包

CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha256
manifest.json

初始化安装环境

安装一些依赖库(每台主机都要运行)

yum install -y vim wget ntp net-tools httpd createrepo

每台机器中分别设置主机名(每台主机都要运行)

hostnamectl set-hostname {每台机器的主机名}

修改每台主机的hosts文件(每台主机都要运行)

IP1 主机名1
IP2 主机名2
IP3 主机名3

关闭防火墙(每台主机都要运行)

systemctl stop firewalld \
&& systemctl disable firewalld \
&& systemctl status firewalld

关闭Selinux(每台主机都要运行)

setenforce 0 #临时关闭
vim /etc/selinux/config #永久关闭
# SELINUX=enforcing 改成 SELINUX=permissive
getenforce #查看状态为Permissive

启动NTP服务(每台主机都要运行)

vim /etc/ntp.conf
# 将时钟服务器更改为如下4个
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org	
server 3.cn.pool.ntp.org

------------------------------------

执行命令：

systemctl start ntpd \
&& systemctl enable ntpd \
&& ntpdate -u 0.cn.pool.ntp.org \
&& hwclock --systohc \
&& date

设置vm.swappiness(每台主机都要运行)

sysctl vm.swappiness=10
echo 'vm.swappiness=10'>> /etc/sysctl.conf

设置透明大页面压缩(每台主机都要运行)

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

安装Cloudera Manager

启动httpd(安装主机运行)

systemctl start httpd

搭建本地yum源(安装主机运行)

将Cloudera Manager文件拷贝到安装主机中的httpd访问目录中,即/var/www/html中

需要拷贝的文件

cloudera-repos/
├── cm6
│   └── 6.3.1
│       ├── allkeys.asc
│       └── redhat7
│           ├── index.html
│           └── yum
│               ├── cloudera-manager.repo
│               ├── index.html
│               ├── repodata
│               │   ├── 3662f97de72fd44c017bb0e25cee3bc9398108c8efb745def12130a69df2ecb2-filelists.sqlite.bz2
│               │   ├── 43f3725f730ee7522712039982aa4befadae4db968c8d780c8eb15ae9872cd4d-primary.xml.gz
│               │   ├── 49e4d60647407a36819f1d8ed901258a13361749b742e3be9065025ad31feb8e-filelists.xml.gz
│               │   ├── 8afda99b921fd1538dd06355952719652654fc06b6cd14515437bda28376c03d-other.sqlite.bz2
│               │   ├── b9300879675bdbc300436c1131a910a535b8b5a5dc6f38e956d51769b6771a96-primary.sqlite.bz2
│               │   ├── e28836e19e07f71480c4dad0f7a87a804dc93970ec5277ad95614e8ffcff0d58-other.xml.gz
│               │   ├── index.html
│               │   ├── repomd.xml
│               │   ├── repomd.xml.asc
│               │   └── repomd.xml.key
│               ├── RPM-GPG-KEY-cloudera
│               └── RPMS
│                   ├── index.html
│                   └── x86_64
│                       ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
│                       ├── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
│                       ├── index.html
│                       └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
└── robots.txt

授予权限

sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm6

配置yum源(每台主机都运行)

(
cat <<EOF
[cloudera-manager]
name=Cloudera Manager 6.3.1
baseurl=http://{安装主机的IP地址}/cloudera-repos/cm6/6.3.1/redhat7/yum
gpgcheck=0
enabled=1
EOF
) >> /etc/yum.repos.d/cloudera-manager.repo

安装JDK(每台主机都要运行)

yum -y install oracle-j2sdk1.8

安装Cloudera Manager(安装主机运行)

yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

配置本地Parcel存储库(安装主机运行)

将本地存储库放到httpd可访问目录下

cdh6/
└── 6.3.1
    └── parcels
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha256
        └── manifest.json

配置权限

sudo chmod -R ugo+rX /var/www/html/cdh6

安装MySQL

卸载自带的mariadb(安装主机运行)

yum -y remove mariadb-libs

安装(安装主机运行)

yum install -y libaio \
&& rpm -ivh mysql-community-common-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-libs-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-client-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-server-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm

配置my.cnf(安装主机运行)

echo character-set-server=utf8 >> /etc/my.cnf

启动MySQL(安装主机运行)

systemctl enable mysqld && systemctl start mysqld && grep password /var/log/mysqld.log | sed 's/.*\(............\)$/\1/'

创建数据库并授权(安装主机运行)

(
cat <<EOF
set password for root@localhost = password('123456Aa.');
grant all privileges on *.* to 'root'@'%' identified by '123456Aa.';
flush privileges;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE monitor DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON monitor.* TO 'monitor'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456Aa.';
SHOW DATABASES;
EOF
) >> /root/cdhinit.sql

运行脚本

source /root/cdhinit.sql

配置JDBC驱动包(安装主机运行)

mkdir -p /usr/share/java/
cp mysql-connector-java.jar /usr/share/java/

初始化scm库(安装主机运行)

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456Aa.

启动CM(安装主机运行)

systemctl start cloudera-scm-server 
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

访问界面(安装主机运行)

http://安装主机IP:7180

登陆-默认账号和密码都是admin

添加集群

配置需要增加的机器

配置存储库地址

配置数据包地址

配置集群密码

之后正常下一步安装即可

所需资源：

链接：https://pan.baidu.com/s/1cNj28m8XkQcRED47iex29Q

提取码：qn71

果丶果

关注

5
点赞
踩
15

收藏

觉得还不错? 一键收藏
打赏
9
评论
CDH6.3.1-安装步骤

CDH6.3.1-安装步骤注意-所有主机密码要一致准备安装包MySQL5.7mysql-5.7.27-1.el7.x86_64.rpm-bundle.tarMySQL驱动包mysql-connector-java.jarcloudera-manager包cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpmcloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpmclouder
复制链接

扫一扫