CDH集群安装

CDH集群安装

1、准备软件包
软件包下载地址：链接: https://pan.baidu.com/s/1J-P9LKhLx4W55CSLHg7kIw?pwd=kxmm 提取码: kxmm
2、服务器规划

主机名	IP地址	角色
hadoop102	192.168.10.102	主
hadoop103	192.168.10.103	从
hadoop104	192.168.10.104	从

3、修改主机名(分别在三台服务器操作)

#hadoop1节点执行
hostname hadoop102
echo hadoop102 > /etc/hostname
#hadoop2节点执行
hostname hadoop103
echo hadoop103 > /etc/hostname
#hadoop3节点执行
hostname hadoop104
echo hadoop104 > /etc/hostname    

4、配置主机名映射(分别在三台服务器操作)：vi /etc/hosts

192.168.10.102 hadoop102
192.168.10.103 hadoop103
192.168.10.104 hadoop104           

5、配置节点间免密（在hadoop102节点执行）

ssh-keygen -t rsa -f ~/.ssh/id_rsa -N ''
ssh-copy-id hadoop102
ssh-copy-id hadoop103
ssh-copy-id hadoop104 

6、安装系统软件包

yum install -y wget ntpd net-tools httpd createrepo     

7、关闭防火墙

systemctl disable --now firewalld  
iptables -F  

8、安装java和mysql驱动

mkdir -p /usr/java/
mkdir -p /usr/share/java/
ln -s /opt/jdk /usr/java/default
cp /opt/mysql-connector*.jar /usr/share/java/mysql-connector-java.jar

9、关闭selinux

setenforce 0
sed -i "s/enforcing/disabled/g" /etc/selinux/config   

10、配置ntp

systemctl disable --now chronyd
systemctl enable --now ntpd            

11、配置httpd服务

mv /opt/cdh /var/www/html/
mv /opt/clouderamanager /var/www/html/
cd /var/www/html/cdh
createrepo .
cd /var/www/html/clouderamanager
createrepo .
systemctl restart httpd      
#说明：createrepo后面有一个点

12、安装数据库

yum -y install mariadb-server
systemctl enable --now mariadb        

13、创建scm用户及对应的数据库

CREATE USER 'scm'@'%' IDENTIFIED BY '123456';
grant all privileges on *.* to root@'%'identified by '123456';
grant all privileges on *.* to 'scm'@'%' with grant option;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456';
GRANT ALL ON scm.* TO 'scm'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456';
GRANT ALL ON amon.* TO 'amon'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456';
GRANT ALL ON rman.* TO 'rman'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456';
GRANT ALL ON hue.* TO 'hue'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY '123456';
GRANT ALL ON hive.* TO 'hive'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456';
GRANT ALL ON sentry.* TO 'sentry'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456';
GRANT ALL ON nav.* TO 'nav'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456';
GRANT ALL ON navms.* TO 'navms'@'localhost' IDENTIFIED BY '123456';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456';
GRANT ALL ON oozie.* TO 'oozie'@'localhost' IDENTIFIED BY '123456';
flush privileges;   

15、配置yum源:/etc/yum.repos.d/cloudera-manager.repo

[cloudera-manager]
name=Cloudera-Manager
#baseurl=http://192.168.10.102/cdh
baseurl=http://192.168.10.102/clouderamanager
enabled=1
gpgcheck=0            

16、安装cloudera-manager-server，只在server节点，根据已经配置好的本地yum，拉取安装包

yum install -y cloudera-manager-daemons cloudera-manager-server  

17、移动安装包parcel

cp /opt/CDH/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel* /opt/cloudera/parcel-repo
sha1sum /opt/cloudera/parcel-repo/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel | awk '{ print $1 }' > /opt/cloudera/parcel-repo/CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha       

18、注入CM-server库 与 CM-agent端的server

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456

19、启动cloudera-scm-server

systemctl enable cloudera-scm-server
systemctl start cloudera-scm-server
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log   

20、登录网页

http://hadoop102:7180
#用户名/密码: admin/admin   

21、开始安装