Cloudera Manager平台部署说明书

最新推荐文章于 2024-08-16 15:24:10 发布

王者的路注定孤独

最新推荐文章于 2024-08-16 15:24:10 发布

阅读量10w+

点赞数 3

文章标签： Cloudera Manager 大数据

本文链接：https://blog.csdn.net/volitationlong/article/details/82460975

版权

一、基础环境

1. 修改主机名

# vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=master1

# vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=master2

# vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=node1

# vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=node2

# vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=node3

2. 配置网络映射

# vim /etc/hosts

192.168.233.101     master1
192.168.233.102     master2
192.168.233.103     node1
192.168.233.104     node2
192.168.233.105     node3

3. 关闭防火墙

   # service iptables stop
   # chkconfig iptables off
   # chkconfig --list | grep iptables

4. 禁用selinux

# vim /etc/selinux/config

SELINUX=disabled

5. 禁用IPv6

   # echo " " >> /etc/modprobe.d/dist.conf
   # echo "alias net-pf-10 off" >> /etc/modprobe.d/dist.conf
   # echo "alias ipv6 off" >> /etc/modprobe.d/dist.conf

6. 配置SSH无秘钥登录

6.1 生产公钥和私钥

# ssh-keygen

6.2 拷贝公钥给目标服务器

   # ssh-copy-id -i master1
   # ssh-copy-id -i master2
   # ssh-copy-id -i node1
   # ssh-copy-id -i node2
   # ssh-copy-id -i node3

7. 集群时间同步

# ntpdate -u time.nuri.net

8. 设置用户的可打开最大文件数及进程数

# vim /etc/security/limits.conf

*     soft     nofile     32728
*     hard     nofile     1024567
*     soft     nproc      65535
*     hard     nproc      unlimited
*     soft     memlock    unlimited
*     hard     memlock    unlimited

二、基础软件

1. JDK安装

1.1 下载JDK安装包

本平台采用 JDK-8u172 的版本：jdk-8u172-linux-x64.tar.gz
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

1.2 解压JDK安装程序

# mkdir -p /opt/java/
# tar zxf /opt/tools/jdk-8u172-linux-x64.tar.gz -C /opt/java/

1.3 拷贝JDK至其它节点

   # scp -r java/ master2:/opt/
   # scp -r java/ node1:/opt/
   # scp -r java/ node2:/opt/
   # scp -r java/ node3:/opt/

1.4 配置JDK环境变量

vim /etc/profile

# JAVA_HOME
export JAVA_HOME=/opt/java/jdk1.8.0_172
export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
export PATH=$PATH:$JAVA_HOME/bin

1.5 验证JDK环境

# source /etc/profile
# java -version

2. MySQL安装

CentOS-6.9 二进制安装Mysql-5.7.12

三、Cloudera Manager安装

1. 安装介质

1.1 下载Cloudera Manager

本平台采用 Cloudera Manager-5.12.0的版本：cloudera-manager-el6-cm5.12.0_x86_64.tar.gz
http://archive.cloudera.com/cm5/cm/5/

1.2 下载MySQL JDBC驱动

本平台采用 MySQL-5.1.46的JDBC驱动：mysql-connector-java-5.1.46.jar
http://dev.mysql.com/downloads/connector/j/

2. 部署准备

2.1 MySQL创建元数据库

# mysql -uroot -pmysqlpasswd

create database activity default charset utf8 collate utf8_general_ci;
grant all on activity.* to 'cm'@'%' identified by 'cm!@#edc';
flush privileges;

2.2 创建cloudera-scm用户

参数	参数说明
--system	表示系统用户
--home=/opt/cm-5.12.0/run/cloudera-scm-server --no-create-home	指定用户的主目录
--shell=/bin/false	不作为登陆用户
--comment "Cloudera SCM User"	不作为登陆用户
cloudera-scm	用户名

useradd --system \
--home=/opt/cm-5.12.0/run/cloudera-scm-server --no-create-home \
--shell=/bin/false \
--comment "Cloudera SCM User" cloudera-scm

3. 部署Cloudera Manager

3.1 解压Cloudera Manager安装程序

# tar zxf /opt/tools/cloudera-manager-el6-cm5.12.0_x86_64.tar.gz -C /opt/

3.2 初始化Cloudera Manager数据库

拷贝MySQL驱动

# cp /opt/tools/mysql-connector-java-5.1.46.jar /opt/cm-5.12.0/share/cmf/lib/

初始化Cloudera Manager数据库，创建的数据库名为cm，对应的用户名为cm，密码为cm!@#edc

# /opt/cm-5.12.0/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -p'mysqlpasswd' cm 'cm!@#edc'

3.3 修改配置

# vim /opt/cm-5.12.0/etc/cloudera-scm-agent/config.ini

server_host=master1

3.4 拷贝CM至其它节点

   # scp -r /opt/cm-5.12.0 master2:/opt/
   # scp -r /opt/cm-5.12.0 node1:/opt/
   # scp -r /opt/cm-5.12.0 node2:/opt/
   # scp -r /opt/cm-5.12.0 node3:/opt/