CentOS7.6 在线安装 CDH6.3.0

本教程介绍了如何在CentOS7.6环境下进行CDH6.3.0的在线安装,包括服务器准备、网络配置、SSH免密登录、时间同步、CDH仓库配置、Oracle JDK与MySQL数据库的安装、namenode与datanode的部署。通过此过程,读者可以掌握大数据平台CDH的搭建步骤。
摘要由CSDN通过智能技术生成

Created with Raphaël 2.2.0 开始 配置网络&hosts&ssh 安装JDK 安装MySQL & 配置MySQL 安装cloudera-scm-server & 配置SCM数据库 安装cloudera-scm-agent & 配置 host 结束

准备工作

准备服务器

准备4台centos8 一台master 3台slave在这里插入图片描述

配置网络

启用网卡

centos7最小化安装 无ifconfig 启用网卡.

  • 修改hostname
sudo hostnamectl set-hostname vic01
sudo hostnamectl

在这里插入图片描述

  • 修改hosts
sudo vim /etc/hosts

将以下内容添加到hosts里

10.2.0.10 vic0 vic0
10.2.0.11 vic1 vic1
10.2.0.12 vic2 vic2
10.2.0.13 vic3 vic3

在这里插入图片描述

配置集群SSH免密登录

scp免密码拷贝和ssh免密码登录
.
在主节点上执行ssh-keygen -t rsa一路回车,生成无密码的密钥对。

将公钥添加到认证文件中:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

并设置authorized_keys的访问权限

chmod 600 ~/.ssh/authorized_keys

scp文件到所有datenode节点:

scp ~/.ssh/authorized_keys root@vic1:~/.ssh
scp ~/.ssh/authorized_keys root@vic2:~/.ssh
scp ~/.ssh/authorized_keys root@vic3:~/.ssh

PS :shell 分发

同时配置多台机器可设置 Xshell 脚本分发
在这里插入图片描述

时间同步配置

yum -y install ntp
systemctl start ntpd 
systemctl enable ntpd 

在vic0节点

vi /etc/ntp.conf
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org
server 3.cn.pool.ntp.org
restrict 0.cn.pool.ntp.org nomodify notrap noquery
restrict 1.cn.pool.ntp.org nomodify notrap noquery
restrict 2.cn.pool.ntp.org nomodify notrap noquery
restrict 3.cn.pool.ntp.org nomodify notrap noquery
server 127.127.1.0
fudge 127.127.1.0 stratum 10
systemctl start ntpd.service
systemctl enable ntpd.service
ntpstat

在datanode节点

vim /etc/ntp.conf
server vic0 prefer
systemctl start ntpd.service
systemctl enable ntpd.service
ntpdate -u vic0
ntpstat

配置CDH Repository

导入repo

sudo curl -o /etc/yum.repos.d/cloudera-manager.repo https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/cloudera-manager.repo
sudo rpm --import https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPM-GPG-KEY-cloudera 

在这里插入图片描述

安装依赖

sudo yum install -y bind-utils psmisc libxslt cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs httpd mod_ssl openssl-devel python-psycopg2 MySQL-python /lib/lsb/init-functions libpq.so.5
wget http://www.percona.com/redir/downloads/Percona-XtraDB-Cluster/5.5.37-25.10/RPM/rhel6/x86_64/Percona-XtraDB-Cluster-shared-55-5.5.37-25.10.756.el6.x86_64.rpm
sudo rpm -ivh Percona-XtraDB-Cluster-shared-55-5.5.37-25.10.756.el6.x86_64.rpm

安装Oracle JDK

centos7.6 安装配置Oracle JDK

分发JDK

scp -r /usr/java/jdk1.8.0_271 root@vic1:/usr/java/
scp -r /usr/java/jdk1.8.0_271 root@vic2:/usr/java/
scp -r /usr/java/jdk1.8.0_271 root@vic3:/usr/java/

安装数据库

centos7.6 安装配置 mysql5.7

配置mysql

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit  = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

安装mysql 驱动

下载解压驱动

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
tar zxvf mysql-connector-java-5.1.46.tar.gz
sudo mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46
sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

一定定要改名mysql-connector-java.jar 放到/usr/share/java路径

进入数据库创建数据库

mysql -u root -p
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

设置CDH的Database

创建数据库账号

创建cdh6给cdh使用

create user chd6@% indentified by '123456'

给账号赋权

GRANT ALL ON scm.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON amon.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON rman.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON hue.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON metastore.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON sentry.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON nav.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON navms.* TO 'cdh6'@'%' IDENTIFIED BY '123456';
GRANT ALL ON oozie.* TO 'cdh6'@'%' IDENTIFIED BY '123456';

在这里插入图片描述

namenode安装

sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
systemctl start cloudera-scm-server 

查看日志

sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

官方推荐Enable Auto-TLS(配置复杂不建议启用)

sudo JAVA_HOME=/usr/java/jdk1.8.0_271 /opt/cloudera/cm-agent/bin/certmanager --location /opt/cloudera/CMCA setup --configure-services

设置CDH的Database

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm cdh6

在这里插入图片描述
重启

systemctl restart cloudera-scm-server 

地址:http://vic0:7180
用户:admin
密码:admin

datanode安装

sudo yum install cloudera-manager-daemons cloudera-manager-agent
vi /etc/cloudera-scm-agent/config.ini
server_host=vic0

启动agent

systemctl start cloudera-scm-agent
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值