centos7搭建CDH

2019独角兽企业重金招聘Python工程师标准>>>

CDH CENTOS7环境搭建
CentOS-7-x86_64-DVD-1810.iso
最大磁盘80G 内存8G
网络适配器是仅主机模式
时区选择上海  soft selection选择server with gui/右边marialDB database server2 development tool 
enable kdump关掉
security policy关掉
network &HostName
HostName CDH11
ETHERNET ON 
CONFIGURE-》geneeral auto matically connnect to this network 
192.168.119.128
ipv4 manual ->address-netmast
选择installtion destion begin installation
准备工作:
关闭防火墙
systemctl stop firewalld.service
[root@CDH11 ~]# systemctl stop firewalld.service
[root@CDH11 ~]# systemctl disable  firewalld.service
[root@CDH11 ~]# vi /etc/hosts
[root@CDH11 ~]# ssh-keygen -t rsa
[root@CDH11 ~]# ssh-copy-id -i .ssh/id_rsa.pub root@CDH11

[root@CDH11 ~]# ssh CDH11

设置主机名
安装JDK
卸载自带的jdk
[root@CDH11 ~]# rpm -qa|grep java
java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64
python-javapackages-3.4.1-11.el7.noarch
javapackages-tools-3.4.1-11.el7.noarch
java-1.8.0-openjdk-headless-1.8.0.181-7.b13.el7.x86_64
tzdata-java-2018e-3.el7.noarch
[root@CDH11 ~]# rpm -e --nodeps java-1.8.0-openjdk-1.8.0.181-7.b13.el7.x86_64

[root@CDH11 ~]# rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch
[root@CDH11 ~]# rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch
[root@CDH11 ~]# rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.181-7.b13.el7.x86_64
[root@CDH11 ~]# rpm -e --nodeps tzdata-java-2018e-3.el7.noarch
[paohaiijiao@CDH11 opt]$ rpm -ivh jdk-8u201-linux-x64.rpm
echo JAVA_HOME="/usr/java/latest" >>/etc/enviroment

[root@CDH11 opt]# echo JAVA_HOME="/usr/java/latest" >>/etc/enviroment
[root@CDH11 opt]# java -version
配置mysql(marial db) 数据库 
[root@CDH11 opt]# systemctl start mariadb
[root@CDH11 opt]# systemctl enable mariadb
[root@CDH11 opt]# mysql_secure_installation设置密码
创建数据库
mysql -u -p105104gou
create database hive;
create database amon;必须
create database oozie;
create database hue;

 create database sqoop;
安装数据库的驱动程序需要使用yum
设置-》光盘-》已连接
[root@CDH11 opt]# mkdir /root/cdroom/
[root@CDH11 opt]# mount /dev/cdrom /root/cdroom/
mount: /dev/sr0 is write-protected, mounting read-only
删除所有的repo
[root@CDH11 opt]# cd /etc/yum.repos.d/
[root@CDH11 yum.repos.d]# ls
CentOS-Base.repo  CentOS-Debuginfo.repo  CentOS-Media.repo    CentOS-Vault.repo
CentOS-CR.repo    CentOS-fasttrack.repo  CentOS-Sources.repo
[root@CDH11 yum.repos.d]# rm -rf *
[root@CDH11 yum.repos.d]# vi my.repo
[centos-yum]
baseurl=file:///root/cdroom
enabled=1
gpgcheck=0
开启NTP
[root@CDH11 yum.repos.d]# yum install ntp
systemctl is-enabled ntpd
systemctl enable ntpd
systemctl start ntpd
root@CDH11 yum.repos.d]# yum install mysql-connector-java
3开始安装CDH
安装配置clouder manager
1.将cloudera-manager-centos-cm5.12.1_x86_64上传到/opt目录并解压
[root@CDH11 opt]# tar -zxvf cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz
2.将mysql的驱动放到/opt/cm-5.12.1/share/cmf/lib
cp /usr/share/java/mysql-connector-java.jar /opt/cm-5.12.1/share/cmf/lib
3.初始化cloudera manager 的数据库元信息
/opt/cm-5.12.1/share/cmf/schema/scm_prepare_database.sh  mysql cm -hlocalhost -uroot -p105104gou --scm-host localhost scm scm scm 
g that we can write to /opt/cm-5.12.1/etc/cloudera-scm-server
Creating SCM configuration file in /opt/cm-5.12.1/etc/cloudera-scm-server
groups: cloudera-scm: no such user
Executing:  /usr/java/jdk1.8.0_201-amd64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.12.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.12.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[                          main] DbCommandExecutor              INFO  Successfully connected to database.
All done, your SCM database is configured correctly!--执行成功了
4.创建cloudera-scm用户
useradd --system --home=/opt/cm-5.12.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm 
5.配置介质
默认路径/opt/cloudera/parcel-repo
[root@CDH11 parcel-repo]# ls
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel       manifest.json
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1
[root@CDH11 parcel-repo]# mv CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1 CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha
[root@CDH11 parcel-repo]# ls
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel      manifest.json
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha
修改/opt/cm-5.12.1/etc/cloudera-scm-agent/config.ini 中的server_host为主节点的主机名
[root@CDH11 parcel-repo]# vi /opt/cm-5.12.1/etc/cloudera-scm-agent/config.ini
server_host=CDH11
启动服务
/opt/cm-5.12.1/etc/init.d/cloudera-scm-server start
/opt/cm-5.12.1/etc/init.d/cloudera-scm-agent start
[root@CDH11 opt]# vi /opt/cm-5.12.1/etc/cloudera-scm-agent/config.ini
[root@CDH11 opt]# /opt/cm-5.12.1/etc/init.d/cloudera-scm-server start
Starting cloudera-scm-server:                              [  OK  ]
[root@CDH11 opt]# /opt/cm-5.12.1/etc/init.d/cloudera-scm-agent start
Starting cloudera-scm-agent:                               [  OK  ]
http://192.168.174.132:7180/cmf/login[启动时间长，刷新admin admin]17分钟
接受协议-》继续当前管理的主机-》继续
在选择存储库-》更多选项删除远程的东西
继续当前管理的主机-》继续
echo 10 >/proc/sys/vm/swappiness
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
重新运行
完成
自定义
  Cloudera Management Service              
report manager
create database report;

https://blog.csdn.net/xjping0794/article/details/77746630
[root@CDH11 opt]# cd /opt/cloudera/parcel-repo
输入 sudo，然后输入密码，回到root用户
cd ~ 进入 /root目录

[paohaiijiao@CDH11 ~]$ su -
Password: 
su: Authentication failure
[paohaiijiao@CDH11 ~]$ sudo passwd root
Changing password for user root.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.
[paohaiijiao@CDH11 ~]$ su -

转载于:https://my.oschina.net/goudingcheng/blog/3014087