CDH6.2搭建之安装

最新推荐文章于 2024-04-21 18:26:03 发布

u014010342

最新推荐文章于 2024-04-21 18:26:03 发布

阅读量3k

点赞数 1

分类专栏：大数据 CDH 文章标签： CDH

本文链接：https://blog.csdn.net/u014010342/article/details/89473486

版权

CDH 同时被 2 个专栏收录

1 篇文章 0 订阅

订阅专栏

大数据

0 篇文章 0 订阅

订阅专栏

环境准备：
宿主机 Centos Liunx release 7.6.1810 128G内存 2T硬盘
VirtualBox 6.0.x86_64

虚拟机： 3台；用户cdh
   配置
       内存：32G
       存储：200G
       系统：Centos Liunx release 7.6.1810
       网络：
一、准备工作

1、网络设置，要求每台机器可以访问互联网，并且可以互通；

这里我的VirtualBox设置是：

网卡一设置为：网络地址转换（NAT）

网卡二设置为：桥接网卡，指定到联网的那个网卡（找不到，自行百度）;

2、主机名设置（所有机器）

vim /etc/hostname

修改为 cdh.a.com

196.168.89.61 cdh.a.com 主节点
196.168.89.62 cdh.b.com
196.168.89.63 cdh.c.com

3、防火墙设置

sudo systemctl stop firewalld.service

 sudo systemctl disable firewalld.service

sudo vim /etc/selinux/config

修改SELINUX=disabled

sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

4、ssh免密登陆

所有节点 [cdh.a.com,cdh.b.com,cdh.c.com]

sudo ssh-keygen -t rsa

主节点 cdh.a.com

 [cdh@com ~]$ ssh-keygen -t rsa
            Generating public/private rsa key pair.
            Enter file in which to save the key (/home/cdh/.ssh/id_rsa): 
            Created directory '/home/cdh/.ssh'.
            Enter passphrase (empty for no passphrase): 
            Enter same passphrase again: 
            Your identification has been saved in /home/cdh/.ssh/id_rsa.
            Your public key has been saved in /home/cdh/.ssh/id_rsa.pub.
            The key fingerprint is:
            SHA256:kggyD9C2N4dDzRjlxwPkzM9hFEO/81AwWOlUNFo7jiE cdh@com.a.com
            The key's randomart image is:
            +---[RSA 2048]----+
            |..  .B+.=+++=    |
            |. o o=o+.oo= o   |
            |+..o .= Eo+ +    |
            | =..=..* +.* .   |
            |  ...+o S = .    |
            |       .   +     |
            |            .    |
            |                 |
            |                 |
            +----[SHA256]-----+
        [cdh@com ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub cdh@cdh.a.com
            /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/cdh/.ssh/id_rsa.pub"
            The authenticity of host 'cdh.a.com (196.168.89.61)' can't be established.
            ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
            ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
            Are you sure you want to continue connecting (yes/no)? yes
            /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
            /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
            cdh@cdh.a.com's password: 

            Number of key(s) added: 1

            Now try logging into the machine, with:   "ssh 'cdh@cdh.a.com'"
            and check to make sure that only the key(s) you wanted were added.

        [cdh@com ~]$ scp ~/.ssh/authorized_keys cdh@cdh.b.com:~/.ssh/
            The authenticity of host 'cdh.b.com (196.168.89.62)' can't be established.
            ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
            ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
            Are you sure you want to continue connecting (yes/no)? yes
            Warning: Permanently added 'cdh.b.com,196.168.89.62' (ECDSA) to the list of known hosts.
            cdh@cdh.b.com's password: 
            authorized_keys                                                          100%  395   339.7KB/s   00:00    
        [cdh@com ~]$ scp ~/.ssh/authorized_keys cdh@cdh.c.com:~/.ssh/
            The authenticity of host 'cdh.c.com (196.168.89.63)' can't be established.
            ECDSA key fingerprint is SHA256:fLmm8WivB6o5YzGaMuWPB2sZQvk1W+T3XXc6uCMKWi8.
            ECDSA key fingerprint is MD5:33:8d:96:e2:1e:0e:76:fe:4e:04:fc:bb:24:47:0a:2a.
            Are you sure you want to continue connecting (yes/no)? yes
            Warning: Permanently added 'cdh.c.com,196.168.89.63' (ECDSA) to the list of known hosts.
            cdh@cdh.c.com's password: 
            scp: /home/cdh/.ssh/: Is a directory

5、阿里 yum repo (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])

sudo mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak
[cdh@com ~]$ sudo curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
		  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
										 Dload  Upload   Total   Spent    Left  Speed
		100  2523  100  2523    0     0  45165      0 --:--:-- --:--:-- --:--:-- 45872

6、lib install (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])

 sudo yum install net-tools vim -y

7、安装NTP (所有节点 [cdh.a.com,cdh.b.com,cdh.c.com])

sudo yum install  -y ntp

sudo vim /etc/ntp.conf

主节点（cdh.a.com）修改

server time.windows.com prefer
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst

time.windows.com 是时间服务器，在内网环境可让用户提供统一的时间服务器地址

从节点（ [cdh.b.com,cdh.c.com]）修改

server cdh.a.com
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst

8、JDK安装这里我选择使用oracle-jkd 所以卸载Centos自带的openjdk

官网JDK要求

JDK 安装，环境变量配置完成以后记得备份一下虚拟机，至此还没有和CDH产生一毛钱关系

二、CM安装 所有节点 [cdh.a.com,cdh.b.com,cdh.c.com]

官方安装说明

yum repo 地址

sudo wget https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/

sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

数据库安装配置：CM初始化需要执行数据库脚本，需要一个数据库数据库说明

安装好以后需要创建一系列的数据库（我这里使用的是一个已经存在的postgresql，因此没有安装数据库）需要准备一些库

Cloudera Manager Server包含一个可以为自己创建和配置数据库的脚本。该脚本可以：

创建Cloudera Manager Server数据库配置文件。
（MariaDB，MySQL和PostgreSQL）为Cloudera Manager Server创建和配置数据库以供使用。
（MariaDB，MySQL和PostgreSQL）为Cloudera Manager Server创建和配置用户帐户。

sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql -h 196.168.65.136 --scm-host cdh.a.com scm scm

[cdh@com ~]$ sudo /opt/cloudera/cm/schema/scm_prepare_database.sh postgresql -h 196.168.65.136 --scm-host cdh.a.com scm scm
[sudo] cdh 的密码：
Enter SCM password: 
JAVA_HOME=/usr/java/jdk1.8.0_181-amd64
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing:  /usr/java/jdk1.8.0_181-amd64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
	[                          main] DbCommandExecutor              INFO  Successfully connected to database.
	All done, your SCM database is configured correctly!

表示我们的数据库已经可以连上了

启动 Cloudera Manager Server

sudo systemctl start cloudera-scm-server

查看启动日志

sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

管网交代当看到"INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server."时表示成功启动了

2019-04-23 15:16:14,568 INFO SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:2019-04-23T07:16:14.568Z
2019-04-23 15:16:15,596 INFO WebServerImpl:org.eclipse.jetty.server.Server: jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_181-b13
2019-04-23 15:16:15,772 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@78fdd57a{HTTP/1.1,[http/1.1]}{0.0.0.0:7180}
2019-04-23 15:16:15,791 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @175529ms
2019-04-23 15:16:15,791 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

千呼万唤始出来，OK!CDH至此安装成功

记得虚拟机在备份一下，鬼知道配置的时候会不会调到坑里出不来

http://<server_host>:7180访问试试

u014010342

关注

1
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
CDH6.2搭建之安装

环境准备：宿主机 Centos Liunx release 7.6.1810 128G内存 2T硬盘VirtualBox 6.0.x86_64虚拟机： 3台；用户cdh 配置内存：32G 存储：200G 系统：Centos Liunx release 7.6.1810 网络：一、准备工作1、网络设置，要求每...
复制链接

扫一扫