centos6.8部署cloudera-manager

 

环境

centos 6.8

cm6.2

cdh6.2

jdk1.8

机器 node1   node2   node3

1、下载安装包

下载地址https://archive.cloudera.com/cm6/6.2.0/redhat6/yum/RPMS/x86_64/

cm6.2的包

cdh6.2的包https://archive.cloudera.com/cdh6/6.2.0/parcels/

2、安装前准备

以下每台都执行

2.1.0、jdk 需要安装到/usr/java/下

我用的就是yum本地源安装的，或者rpm安装，自动安装到/usr/java下

这一步可以等到后面配置好yum本地源之后执行下面命令

yum install oracle-j2sdk1.8

2.1.1、ssh免密互信登陆

#每台机器都执行
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa 直接生成
 
# 我的是3台机器，每台机器都执行
ssh-copy-id -i node1
ssh-copy-id -i node2
ssh-copy-id -i node3

2.1.2、hosts主机名映射

# 每台都执行
vim /etc/hosts

192.168.88.151 node1
192.168.88.152 node2
192.168.88.153 node3

2.1.3、时间同步,这个需要安装，因为cm监控会检查

https://blog.csdn.net/qq_35315256/article/details/88684154（我的另外一篇博客）

2.1.4、普通用户的sudu免密（root不需要）

chmod +w /etc/sudoers
vim /etc/sudoers
wangyf ALL=(ALL) NOPASSWD: ALL
chmod -w /etc/sudoers

2.1.5、防火墙关闭

service iptables stop 
chkconfig iptables off

2.1.6、禁用透明页

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

vim /etc/rc.local #这是centos 6的写法
加入上面两句话

2.1.7、关闭SELinux

vim /etc/selinux/config
disabled 就可以
setenforce 0 # 临时生效

2.1.8、swap分区优化

echo "vm.swappiness = 10" >> /etc/sysctl.conf
sysctl -p

2.1.9、如果用的mysql库，驱动放到这，每台机器都放

/usr/share/java/ 
mv mysql-connector-java-5.1.47.jar mysql-connector-java.jar

2.1.10、查看这些依赖是否缺少

yum install -y chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb

2.1.11、安装python2.7，我是yum在线安装的

 

2.2、配置yum内部存储库

2.2.1、安装http服务，我是选择的node3这台机器

yum  install  httpd
service  httpd  start
chkconfig  httpd  on 

2.2.2、安装createrepo服务，(node3这台机器)

yum -y install createrepo

2.2.3、如下图操作，createrepo生成xml文件

2.2.4、测试一下

2.2.5、配置yum源，在node1上

cd  /etc/yum.repos.d/

vim  cloudera-repo.repo

[cloudera-repo]
name=cloudera-repo,version=6.2
baseurl=http://node3/cm6.2/
gpgcheck=1

执行命令

yum clean all
yum makecache
yum repolist 

3、安装cloudera-manager6.2（在node1）

yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

如果报密钥没有被安装，就rpm --import 一下密钥，密匙在官网对应版本

sudo rpm --import https://archive.cloudera.com/cm6/6.2.1/redhat6/yum/RPM-GPG-KEY-cloudera

官网文档：https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/configure_cm_repo.html

4、配置mysql数据库

    CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123';
    CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon@123';
    CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123';
    CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123';
    CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123';
    CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123';
    CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123';
    CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123';
    CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123';
    CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive@123';
    flush privileges;

执行脚本 ,生成cm元数据

/opt/cloudera/cm/schema/scm_prepare_database.sh -h 127.0.0.1 -P 3306 mysql scm scm scm@123

5、配置parcel包

文章最一开始的cdh6.2的截图包全部放到/opt/cloudera/parcel-repo/下

把.sha文件的字符串替换为.json文件最后面的那个hash值

"hash": "5dc82b17dbdbb8ffd9bcd05e98358c111f2c0f12"

6、启动

service cloudera-scm-server start

查看日志

tailf /var/log/cloudera-scm-server/cloudera-scm-server.log

访问：192.168.88.151:7180

7、配置向导

 

 

我的最后一个警告没解决，也没啥影响，其他参考上面准备环节都可以解决。

跟着一步一步的来，安装工作完成。