CDH6.3.2安装部署(Ubuntu1804)（附安装包）

一位编程的学习者

已于 2023-11-28 10:49:53 修改

阅读量1.4k

点赞数 13

文章标签：大数据 java hadoop hive kafka ubuntu

于 2023-11-28 10:00:33 首次发布

本文链接：https://blog.csdn.net/u013522746/article/details/134660483

版权

本文详细描述了在多台Ubuntu18.04机器上安装ClouderaManager，包括设置主机名、root用户、阿里云源、JDK、MySQL，修改数据库和用户权限，以及安装NTP服务和启动ClouderaManager的过程。

摘要由CSDN通过智能技术生成

资源下载

太大了就放百度盘了
链接：https://pan.baidu.com/s/1WyxHKzLjshfvI42Es28uaA?pwd=iozr
提取码：iozr

安装前的准备工作

准备3台以上机器

操作系统为ubuntu 18.04。为了方便使用，在hosts文件中配置机器的ip：

sudo vim /etc/hosts

设置root用户密码

sudo passwd root

允许使用root用户登录ssh：

sudo vim /etc/ssh/sshd_config

添加：PermitRootLogin yes

切换阿里源

sudo vim /etc/apt/sources.list

替换为：

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

本地源安装

每个节点都需要配置Cloudera Manager的离线源：

sudo vim /etc/apt/sources.list.d/cloudera-manager.list

添加本地源：

deb [arch=amd64] http://172.16.20.2:12345/cm/ubuntu1804/apt/ bionic-cm6 contrib

下载archive.key文件

wget http://172.16.20.2:12345/cm/ubuntu1804/apt/archive.key

添加archive.key文件，并更新源

sudo apt-key add archive.key
sudo apt-get update -y

安装JDK

JDK需要在所有机器上安装：

sudo apt-get install oracle-j2sdk1.8 -y

安装mysql

mysql只需要在第一台机器上安装即可

sudo apt-get install -yq mysql-server mysql-client libmysqlclient-dev libmysql-java

查看原始用户的信息：

sudo cat /etc/mysql/debian.cnf

    # Automatically generated for Debian scripts. DO NOT TOUCH!
    [client]
    host     = localhost
    user     = debian-sys-maint
    password = Uvqpha8PHlwtsfXz
    socket   = /var/run/mysqld/mysqld.sock
    [mysql_upgrade]
    host     = localhost
    user     = debian-sys-maint
    password = Uvqpha8PHlwtsfXz
    socket   = /var/run/mysqld/mysqld.sock

使用debian-sys-maint用户以及密码登录mysql。

修改root用户的密码：

update mysql.user set authentication_string=password('123456') where user='root';
update mysql.user set plugin="mysql_native_password";
flush privileges;

允许root用户远程连接：

sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

将bind-address=127.0.0.1改成bind-address=0.0.0.0

设置root用户的连接权限：

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;
FLUSH PRIVILEGES;

安装ntp服务

CDH要求集群中的每台机器上配置网络时间协议（NTP）进行时间同步：

安装ntp

sudo apt-get install ntp

修改配置文件

sudo vim /etc/ntp.conf

添加以下的配置

server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org

重启ntp服务

sudo service ntp restart

设置ntp服务开机自启

sudo systemctl enable ntp

设置时区
执行tzselect，选择北京时间
复制配置文件到/etc目录下:sudo cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

安装Cloudera Manager

安装Cloudera Manager包

在master节点中执行：

sudo apt-get install cloudera-manager-daemons cloudera-manager-server cloudera-manager-agent -y

在slave节点中执行：

sudo apt-get install cloudera-manager-daemons cloudera-manager-agent -y

设置数据库

创建CDH依赖的数据库以及用户

#创建数据库
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
 
#创建用户并授权
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm123456';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon123456';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman123456';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue123456';
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive123456';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry123456';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav123456';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms123456';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie123456';

设置Cloudera Manager数据库

cd /opt/cloudera/cm/schema/
sudo ./scm_prepare_database.sh mysql scm scm scm123456
sudo ./scm_prepare_database.sh mysql amon amon amon123456
sudo ./scm_prepare_database.sh mysql hue hue hue123456
sudo ./scm_prepare_database.sh mysql nav nav nav123456
sudo ./scm_prepare_database.sh mysql navms navms navms123456
sudo ./scm_prepare_database.sh mysql oozie oozie oozie123456
sudo ./scm_prepare_database.sh mysql rman rman rman123456
sudo ./scm_prepare_database.sh mysql sentry sentry sentry123456

下载CDH的parcels包

cd /opt/cloudera/parcel-repo
wget http://172.16.20.2:12345/CDH/CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel
wget http://172.16.20.2:12345/CDH/CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha1
wget http://172.16.20.2:12345/CDH/CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha256
wget http://172.16.20.2:12345/CDH/manifest.json
mv CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha

注意要将CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha1文件重命名为CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha。