ubuntu 18.04服务器版 在线搭建cdh6.2.1

一、准备三台虚拟机（全部root用户下操作）
1、规划
master server，agent mysql
slave1 agent
slave2 agent
2、修改主机名
Vim /etc/hostname
Reboot
3、配置静态ip(root用户下)
cd /etc/netplan
Vim 00-installer-config.yaml
配置如下：

   # This is the network config written by 'subiquity'
network:
  ethernets:
    enp0s3:
      addresses:
      - 192.168.20.47/24
      gateway4: 192.168.20.1
      nameservers:
        addresses: [8.8.8.8,8.8.4.4]
  version: 2

Netplan apply （使修改生效）
配置DNS：vi /etc/systemd/resolved.conf
[Resolve]
DNS=8.8.8.8,8.8.4.4
4、关闭防火墙（root用户下）
防火墙状态 ufw status
防火墙激活 ufw enable
防火墙禁用 ufw disable
5、禁用SELinux
Getenforce 命令查看状态
未安装则 apt-get install selinux-utils 安装
安装后查看状态为disable
6、配置host
Vim /etc/hosts
192.168.20.40 master
192.168.20.41 slave1
192.168.20.42 slave2
7、设置时钟
ntp方式统一时钟：
master作为服务端，使用命令 apt-get install ntp安装ntp，配置/etc/ntp.conf

 #注释掉linux自带的系统时钟
 #pool 0.ubuntu.pool.ntp.org iburst
#pool 1.ubuntu.pool.ntp.org iburst
#pool 2.ubuntu.pool.ntp.org iburst
#pool 3.ubuntu.pool.ntp.org iburst
#Use Ubuntu's ntp server as a fallback.
#pool ntp.ubuntu.com
 #然后设置server
server 192.168.20.45
restrict 192.168.20.45 mask 255.255.255.0 nomodify notrap 

slave1、slave2作为客户端，使用命令apt-get install ntpdate 安装ntp，配置/etc/ntp.conf

  #注释掉linux自带的系统时钟
 #pool 0.ubuntu.pool.ntp.org iburst
#pool 1.ubuntu.pool.ntp.org iburst
#pool 2.ubuntu.pool.ntp.org iburst
#pool 3.ubuntu.pool.ntp.org iburst
#Use Ubuntu's ntp server as a fallback.
#pool ntp.ubuntu.com
#添加server配置
server 192.168.20.45
restrict source notrap nomodify noquery

三个节点分别重启ntp服务，service ntp restart
设置三个节点的ntp服务开机启动 systemctl enable ntp
使用ntpq -p 查看状态

 统一时钟：
  1）sudo tzselect
  2）sudo ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
  3）sudo hwclock -r

8、ssh免密
1）在三台机器root用户下，生成秘钥，ssh-keygen -t rsa
2）将三台机器的公钥拷贝到master的authorized_keys文件（cd .ssh目录下），然后将authorized_keys文件分发到slave1、slave2节点
3）设置允许远程登录
vim /etc/ssh/sshd_config 添加配置 PermitRootLogin yes
9、安装jdk
1）将jdk包上传到cdhmaster节点的app目录下
2）创建java目录 sudo mkdir /usr/java
3）cd app
sudo mv jdk-8u231-linux-x64.tar.gz /usr/java
Cd /usr/java
sudo tar -zxvf jdk-8u231-linux-x64.tar.gz
sudo ln -s jdk1.8.0_231/ jdk
将jar包分发到其他两台机器
sudo scp jdk-8u231-linux-x64.tar.gz sunqing@cdhslave1:/home/sunqing/app/
4）配置环境变量
需要配置下~/.bashrc文件下的环境变量，否则重启后失效。

  export JAVA_HOME=/usr/java/jdk
  export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
  export PATH=.:$JAVA_HOME/bin:$PATH

10、1）在master上安装mysql，在线安装
apt-get install mysql-server
2)设置初始密码
查看配置文件 sudo cat /etc/mysql/deblian.cnf
查看用户密码，然后登陆 mysql -u debian-sys-maint -p
登陆后

use mysql;
update mysql.user set authentication_string=password('root') where user='root' and Host='localhost';
 update user set plugin="mysql_native_password";
 flush privileges;

3)设置root用户权限

4）修改配置文件

将bind-address=127.0.0.1 修改为bind-address=0.0.0.0
重新启动 service mysql restart
5）安装mysql驱动包
sudo apt-get install libmysql-java
5)创建cm所需要的数据库

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';
----Activity Monitor amon
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon';
----Reports Manager rman
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman';
----Hue hue
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
----Hive Metastore Server hive
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive';
----Sentry Server sentry
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry';
----Cloudera Navigator Audit Server nav
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav';
----Cloudera Navigator Metadata Server navms
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms';
----Oozie oozie
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';

二、安装cm
1、配置源
在三台机器中执行如下操作：
下载cloudera-manager.list文件放到/etc/apt/sources.list.d/目录下
下载地址为：https://archive.cloudera.com/cm6/6.2.1/ubuntu1804/apt/cloudera-manager.list
导入GPGkey：
wget https://archive.cloudera.com/cm6/6.2.1/ubuntu1604/apt/archive.key
sudo apt-key add archive.key
2、安装cm
Master节点： sudo apt-get install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
Slave节点：sudo apt-get install cloudera-manager-daemons cloudera-manager-agent
3、配置mysql数据库
停止mysql服务：sudo service mysqld stop
配置mysql的配置文件，在 cd /etc/mysql/mysql.conf.d/mysqld.cnf文件
配置文件如下：（参考官网）

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

启动mysql服务：sudo service mysqld start
初始化数据库：

sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql amon amon 
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql rman  rman
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql hue  hue
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql hive  hive 
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql sentry sentry 
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql nav nav
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql navms navms
sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql oozie  oozie

（密码见创建cm所需要的数据库）

4、启动cm
Master节点：sudo systemctl start cloudera-scm-server
sudo systemctl start cloudera-scm-agent
Slave节点：sudo systemctl start cloudera-scm-agent
查看日志
sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

当出现如下所示则启动成功：

5、web登陆
http://master:7180/ 登陆用户名密码为admin/admin，windows用户下要设置host以识别master主机。