安装 CDH

最新推荐文章于 2024-06-12 15:24:38 发布

weixin_34008933

最新推荐文章于 2024-06-12 15:24:38 发布

阅读量280

点赞数

原文链接：https://my.oschina.net/AnnaWu/blog/1862381

版权

为什么80%的码农都做不了架构师？>>>

Centos 7 安装CDH

准备四台服务器

cdh -1 139.162.4.28
cdh-2 139.162.16.65
cdh-3 139.162.35.124

Step1: Set hostname

说明：四台需要设置hostname

[root@localhost ~]# hostnamectl set-hostname cdh-1

Step2: Set hosts

说明：四台需要配置hosts

 vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
139.162.4.28 cdh-1
139.162.16.65 cdh-2
139.162.35.124 cdh-3
172.104.47.188 cdh-4

Step3:Off selinux

说明：四台需要关闭selinux

[root@cdh-1 ~]# vim /etc/sysconfig/selinux 


*****************
*****************
*****************
SELINUX=enforcing  修改为：SELINUX=disabled

Step4:Download

[root@cdh-1] wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15.1/RPMS/x86_64/jdk-6u31-linux-amd64.rpm

[root@cdh-1] wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15.1/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

[root@cdh-1]wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15.1/RPMS/x86_64/cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

[root@cdh-1] wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15.1/RPMS/x86_64/cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

[root@cdh-1]wget https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15.1/RPMS/x86_64/cloudera-manager-server-db-2-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

Step5:all server install

[root@cdh-1 src]# yum install -y  oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

[root@cdh-1 src]# yum install -y  jdk-6u31-linux-amd64.rpm

Step5: Install mysql in cdh-1

[root@cdh-2 src]# wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

[root@cdh-2 src]# rpm -ivh mysql-community-release-el7-5.noarch.rpm

[root@cdh-2 src]# yum install mysql-server -y

[root@cdh-2 src]# systemctl start mysqld

Step6:install clound manager server

[root@cdh-1 src]# yum install -y cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

[root@cdh-1 src]# yum install -y cloudera-manager-server-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

Step6:set mysql passwd & create user databases in cdh-1

mysql> create database cm default character set utf8 default collate utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql>  grant all on cm.* to 'cm'@'%' identified by 'password';
Query OK, 0 rows affected (0.00 sec)

mysql> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.01 sec)


mysql> create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql> grant all on hive.* to 'root'@'%' identified by 'root' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on oozie.* to 'root'@'%' identified by 'root' with grant option;
Query OK, 0 rows affected (0.00 sec)

mysql> create database reports DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

mysql> grant all on reports.* to 'root'@'%' identified by 'root' with grant option;
Query OK, 0 rows affected (0.00 sec)

Step7:Install MySQL JDBC In cdh-1

Description:Download adress:http://www.mysql.com/downloads/connector/j/5.1.html

[root@cdh-1 src]# tar zxvf mysql-connector-java-5.1.47.tar.gz

[root@cdh-1 src]# mkdir -p /usr/share/java/

[root@cdh-1 src]# cd mysql-connector-java-5.1.47

[root@cdh-1 mysql-connector-java-5.1.47]# cp mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar

Step8: Install Cloudera Manager Server

[root@cdh-1 src]# yum install -y cloudera-manager-daemons-5.5.0-1.cm550.p0.61.el7.x86_64.rpm

[root@cdh-1 src]# yum install -y cloudera-manager-server-5.5.0-1.cm550.p0.61.el7.x86_64.rpm

[root@cdh-1 src]# yum instsall -y  cloudera-manager-agent-5.5.0-1.cm550.p0.61.el7.x86_64.rpm

Step9: import Cloudera Manager Server Metadata

[root@cdh-2 ~]# /usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm root
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.7.0_67-cloudera/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!

Step10:Start Cloudera Manager Server Metadata

[root@cdh-1 ~]# systemctl start cloudera-scm-server

[root@cdh-1 ~]# !net
netstat -nvlpt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:7180            0.0.0.0:*               LISTEN      3002/java           
tcp        0      0 0.0.0.0:7182            0.0.0.0:*               LISTEN      3002/java           
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      870/sshd            
tcp6       0      0 :::3306                 :::*                    LISTEN      29497/mysqld        
tcp6       0      0 :::22                   :::*                    LISTEN      870/sshd

Step11:

Access Adress :http://139.162.16.65:7180

User:admin Passwd:admin

Step11.1 : 1.选择：是的，我接受最终用户许可条款和条件 2.选择：继续

Step11.2:选择继续

Step11.3: 选择继续

Step11.4：输入 Cloudera Manager Server master IP

Step11.5: 选择继续

Step11.6: 选择继续

Step11.7:选择继续

Step11.8:1.所有主机接受相同私钥：选中
2.私钥文件：选择cdh-1 的私钥
3.输入密码短语:密钥密码
4:确认密码短语：密钥密码

Step11.9：选择继续

Step11.10:根据提示修改配置

[root@cdh-1 ~]# less /proc/sys/vm/swappiness
30

#修改为10
[root@cdh-1 ~]# echo "10" > /proc/sys/vm/swappiness

[root@cdh-1 ~]# echo never > /sys/kernel/mm/transparent_hugepage/defrag

[root@cdh-1 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

[root@cdh-1 ~]# vim /etc/rc.local

*************************
*************************
*************************
#加入
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Step11.12:选择要安装的组件

说明：根据自己需要选择了：HBase HDFS Hive Kafka Oozie Spark Sqoop YARN ZooKeeper

Step12: Install cloudera-manager-agent

[root@cdh-2 src]# yum install -y jdk-6u31-linux-amd64.rpm

[root@cdh-2 src]# yum install -y cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

[root@cdh-2 src]# yum install -y oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

[root@cdh-2 src]# yum install -y  cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm
   

[root@cdh-3 src]# yum install -y jdk-6u31-linux-amd64.rpm

[root@cdh-3 src]# yum install -y cloudera-manager-agent-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

[root@cdh-3 src]# yum install -y oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

[root@cdh-3 src]# yum install -y  cloudera-manager-daemons-5.15.1-1.cm5151.p0.3.el7.x86_64.rpm

Step12.1:set cloudera-manager-agent

[root@cdh-3 ~]# vim /etc/cloudera-scm-agent/config.ini

[General]
# Hostname of the CM server.
server_host=localhost  更改为cloudera-manager 的IP：server_host=139.162.16.65

Step13:Start cloudera-scm-agent

[root@cdh-2 src]# systemctl start cloudera-scm-agent

[root@cdh-3 src]# systemctl start cloudera-scm-agent

安装文档://www.cloudera.com/documentation/cdh/5-0-x/CDH5-Installation-Guide/cdh5ig_oracle_jdk_installation.html
yum 源：https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/cloudera-cdh5.repo

初始化数据库

https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.5.0/RPMS/x86_64/

https://my.oschina.net/epoch/blog/1633971

转载于:https://my.oschina.net/AnnaWu/blog/1862381