VirtualBox搭建CDH集群，配置VirtualBox网络，设置免密，搭建mysql

1.配置JDK

     设置JDK环境变量，并生效

2.配置虚拟机网络

cd 到 /etc/sysconfig/network-scripts
vi ifcfg-enp0s3

HWADDR=08:00:27:FC:F8:C4
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp0s3
UUID=ad31ffd1-6fb9-487b-9fab-436688355592
DEVICE=enp0s3
ONBOOT=yes

 

 cd 到 /etc/sysconfig/network-scripts
vi ifcfg-enp0s8

vi ifcfg-enp0s8
HWADDR=08:00:27:0B:D4:C8
TYPE=Ethernet
BROWSER_ONLY=no
BOOTPROTO=static
IPADDR=192.168.56.101
GATEWAY=192.168.56.1
NETMASK=255.255.255.0
DNS1=192.168.56.1
NAME=enp0s8
UUID=ad31ffd1-6fb9-487b-9fab-436688355593
DEVICE=enp0s8
ONBOOT=yes

service network restart 重启网卡服务

#创建递归目录
mkdir -p /opt/module/cm

#解压缩
tar -zxvf cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz -C /opt/module/cm

#创建用户
useradd \
--system \
--home=/opt/module/cm/cm-5.12.1/run/cloudera-scm-server \
--no-create-home \
--shell=/bin/false \
--comment "Cloudera SCM User" cloudera-scm

检查用户是否创建

id cloudera-scm

安装mysql

1.linux 新版中  自带了数据库 mariadb

[root@localhost ~]# rpm -qa|grep -i mariadb
mariadb-libs-5.5.68-1.el7.x86_64

2. 卸载mariadb

[root@localhost ~]# rpm -e  --nodeps mariadb-libs-5.5.68-1.el7.x86_64

3.查查看看有没有mysql

[root@localhost ~]# rpm -qa|grep -i mysql

[root@localhost ~]# find / -name mysql

不用刪

除了标箭头的。把其他栏目全都删掉

比如 rm -rf /usr/share/mysql 

4.利用wget 下载 mysql的rpm

wget http://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm

5.在线安装

yum -y install mysql57-community-release-el7-11.noarch.rpm

6.查看一下安装效果

yum repolist enabled | grep mysql.*

7.直接安装服务

yum install mysql-community-server

8.启动mysql服务

systemctl start  mysqld.service

9.查看状态

systemctl status mysqld.service

10.查看安装的默认密码

grep "password" /var/log/mysqld.log

#添加弱密码策略

vi /etc/my.cnf

#添加validate_password_policy配置
validate_password_policy=0
#关闭密码策略
validate_password = off

重启mysql     systemctl restart  mysqld.service

11.登录

mysql -uroot -p   单独输入登录的原始密码

12.修改密码

ALTER USER 'root'@'localhost' IDENTIFIED BY 'root';

13.从window电脑通过mysql的客户端连接 linux上的mysql

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'root' WITH GRANT OPTION;

必须刷新

FLUSH PRIVILEGES;

14.开机自启

systemctl enable mysqld
systemctl daemon-reload
 

/opt/module/cm/cm-5.12.1/etc/cloudera-scm-agent

在此目录下导入mysql jar包 （如果没有此目录 创建目录   并导入mysql数据库jar包 改名如下）

/usr/share/java    

mv mysql-connector-java.5.5.45.jar mysql-connector-java.jar

三台都需要安装驱动

yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

配置SSH免密登录（RSA MD5）

ssh-keygen -t rsa      #一直回车
cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys

或
ssh-keygen -t rsa
ssh-copy-id hdp5

初始化数据库

/opt/module/cm/cm-5.12.1/share/cmf/schema/scm_prepare_database.sh mysql cm -hcdh1 -uroot -proot --scm-host cdh1 scm scm scm

创建parcel-repo

mkdir -p /opt/cloudera/parcel-repo

导入三个文件

然后执行

chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo

把文件sha1 改成sha

mv CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1 CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha

 该目录用于保存安装完成的文件

mkdir -p /opt/cloudera/parcels

chown -R cloudera-scm:cloudera-scm /opt/cloudera/parcels

把两个文件夹开始分发

scp -r cloudera/ module/ cdh2:$PWD

scp -r cloudera/ module/ cdh3:$PWD

三台都创建此用户

useradd \

--system \

--home=/opt/module/cm/cm-5.12.1/run/cloudera-scm-server \

--no-create-home \

--shell=/bin/false \

--comment "Cloudera SCM User" cloudera-scm

三台都要

chown -R cloudera-scm:cloudera-scm cloudera/

第一台启动server  

/opt/module/cm/cm-5.12.1/etc/init.d/cloudera-scm-server start

第一，二，三台启动agent

/opt/module/cm/cm-5.12.1/etc/init.d/cloudera-scm-agent start

关闭server和agent

/opt/module/cm/cm-5.12.1/etc/init.d/cloudera-scm-server stop

/opt/module/cm/cm-5.12.1/etc/init.d/cloudera-scm-agent stop

登录网址   主机名:7180