一、前期准备
1、关闭防火墙
2、配置NTP时间同步服务
3、设置SELinux模式
前期准备就不展开网上一搜一大堆,如果你是使用阿里云的服务器那么防火墙和SElnux啥的都不用管。如果你是单节点ntp也不用配
二、准备安装包
1、MYSQL安装包 https://dev.mysql.com/downloads/mysql/5.7.html#downloads
2、JDBC 驱动包 wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz(如果wget不下来,就直接浏览器输入上述url)
3、cm安装包 (1281944) https://archive.cloudera.com/cm6/6.3.0/redhat7/yum/RPMS/x86_64/
4、parcel 安装包 (可选,如果不配置没关系可以省略,如果真的需要加我微信我给你网盘 微信--hanyeah5288)
5. 准备allkey wget https://archive.cloudera.com/cm6/6.3.0/allkeys.asc
最终目录树:
└── hadoop_CHD
├── cloudera-repos
│ ├── allkeys.asc
│ ├── cloudera-manager-agent-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-daemons-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-server-6.3.0-1281944.el7.x86_64.rpm
│ ├── cloudera-manager-server-db-2-6.3.0-1281944.el7.x86_64.rpm
│ ├── enterprise-debuginfo-6.3.0-1281944.el7.x86_64.rpm
│ └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
├── mysql
│ └── mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar
├── mysql-jdbc
│ └── mysql-connector-java-5.1.47.tar.gz
└── parcel
├── CDH-6.3.0-1.cdh6.3.0.p0.1279813-el7.parcel
└── manifest.json
二、配置JDK
1、java version "1.8.0_231" --这是我用的版本大家自己安装就好了
三、搭建本地yum源
1、安装httpd和createrepo
yum -y install httpd createrepo
2、启动httpd服务
systemctl start httpd && systemctl enable httpd
3、生成RPM元数据
这些软件包我放在了/mnt02/hadoop_CHD/cloudera-repos/下,你们根据自己的需求自己创建。
cd /mnt02/hadoop_CHD/cloudera-repos/ && createrepo .
4、移动目录
mv /mnt02/hadoop_CHD/cloudera-repos /var/www/html/
5、访问一下
http://服务器外网ip/cloudera-repos/
四、安装配置mysql库
1、卸载自带的mariadb-libs
yum -y remove mariadb-libs
2、解压
tar -xvf mysql-5.7.31-1.el7.x86_64.rpm-bundle.tar
3、安装(MySQL 依赖 libaio,所以先要安装 libaio)
yum install -y libaio
&& rpm -ivh mysql-community-common-5.7.31-1.el7.x86_64.rpm
&& rpm -ivh mysql-community-libs-5.7.31-1.el7.x86_64.rpm
&& rpm -ivh mysql-community-client-5.7.31-1.el7.x86_64.rpm
&& rpm -ivh mysql-community-server-5.7.31-1.el7.x86_64.rpm
&& rpm -ivh mysql-community-libs-compat-5.7.31-1.el7.x86_64.rpm
4、vim /etc/my.cnf
写入官网推荐配置:
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M
#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log
#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
character-set-server=utf8
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
sql_mode=STRICT_ALL_TABLES
5、启动服务并查看
启动:systemctl start mysqld
查看:systemctl start mysqld
6、获取初始密码—grep password /var/log/mysqld.log
阿里云服务器获取不到初始密码解决方法:http://www.mamicode.com/info-detail-2505104.html
7、更改密码-----这里有个坑,密码必须大于八位并且有大小写字母特殊字符和数字,要不然后面认证很麻烦!!!
mysql -uroot -p
set password for root@localhost = password(‘123456Aa.’);
8、设置远程登录权限–进入mysql输入下列命令
grant all privileges on . to ‘root’@’%’ identified by ‘你的密码’;
flush privileges;
五、JDBC驱动
1、解压之后放入文件夹内并更名
tar -zxvf mysql-connector-java-5.1.47.tar.gz
mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.47
cp mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar
六、创建CDH需要的mysql元数据库
1、建库–把下面的语句粘贴复制,搞定!!!
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
2、授权
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '你的密码';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '你的密码';
3、检查一下—检查一下授权
SHOW GRANTS FOR 'scm'@'%';
SHOW GRANTS FOR 'amon'@'%';
SHOW GRANTS FOR 'rman'@'%';
SHOW GRANTS FOR 'hue'@'%';
SHOW GRANTS FOR 'hive'@'%';
SHOW GRANTS FOR 'sentry'@'%';
SHOW GRANTS FOR 'nav'@'%';
SHOW GRANTS FOR 'navms'@'%';
SHOW GRANTS FOR 'oozie'@'%';
五、安装Cloudera Manager
1、编辑repo文件
命令=vim /etc/yum.repos.d/cloudera-manager.repo
输入内容:
[cloudera-manager]
name=Cloudera Manager 6.3.0
baseurl=http://你的服务器IP/cloudera-repos/
gpgcheck=0
enabled=1
2、生成缓存
yum clean all && yum makecache
3、安装
yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server
4、检查
rpm -qa|grep cloudera-manager
5、启动
systemctl start cloudera-scm-server
6、查看日志
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
当日志出现以下提示时,说明启动成功
INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
7、注意
cm的webUI默认是7180端口,使用阿里的服务器时一定要开放安全组端口,并且白名单不要设置为0.0.0.0,因为一旦白名单设置为0.0.0.0,就意味着任何ip都能访问你的服务器,中毒的概率非常大!!!!!!!!
8、安装CDH
跟着步骤走就好了。没什么难的,有什么问题可以留言