CDH-离线安装

环境清单：

主机1: 内网: 125.10.1.17
主机2: 内网: 125.10.1.18
主机3: 内网: 125.10.1.19

 系统配置清单：

IP	HOSTNAME	CPU	MEM	DISK	RELEASE
125.10.1.17	cdh01	4核	3G	90G	CentOS release 6.4 (Final)
125.10.1.18	cdh02	4核	2G	90G	CentOS release 6.4 (Final)
125.10.1.19	cdh03	4核	2G	90G	CentOS release 6.4 (Final)

★配置本地yum源

--请百度利用ISO文件配置yum本地源

★安装yum依赖

yum -y  install  lrzsz
yum -y  install  createrepo
yum -y  install  httpd
yum -y  install  nscd
yum -y  install  unzip
yum -y  install  openssh-clients
yum -y  install  bind-utils 
yum -y  install  tree

★启动对应服务

service httpd start
service nscd  start
chkconfig httpd on
chkconfig nscd  on

★上传parcel包到 【/opt/cloudera/parcel-repo/】下并解压

mkdir -p /opt/cloudera/parcel-repo
cd  /opt/cloudera/parcel-repo

sz   manifest.json
sz   CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel.sha1
sz   CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel
或
wget http://archive.cloudera.com/cdh5/parcels/5.9.0/manifest.json
wget http://archive.cloudera.com/cdh5/parcels/5.9.0/CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel.sha1
wget http://archive.cloudera.com/cdh5/parcels/5.9.0/CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel

cp CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel.sha1 CDH-5.9.0-1.cdh5.9.0.p0.23-el6.parcel.sha

★上传包【cm5.9.0-centos6.tar.gz】到 /opt/cloudera/下并解压

--上传这个包到master上
cd /opt/cloudera/

sz cm5.9.0-centos6.tar.gz
或
http://archive.cloudera.com/cm5/repo-as-tarball/5.9.0/cm5.9.0-centos6.tar.gz
http://archive.cloudera.com/cm5/repo-as-tarball/5.9.0/cm5.9.0-centos7.tar.gz

tar zxvf cm5.9.0-centos6.tar.gz

★制作本地CDH repos源

mkdir -p  /var/www/html/cloudera-repos
cd    /var/www/html/cloudera-repos
ln -s /opt/cloudera/cm cm
ls -l
-- lrwxrwxrwx 1 root root 22 Sep 30 00:38 5.9.0 -> /opt/cloudera/cm/5.9.0

cd /var/www/html/cloudera-repos/cm/5.9.0/RPMS/x86_64
createrepo .
chmod -R ugo+rX /var/www/html

-- IE验证
http://XXX.XXX.XXX.XXX/cloudera-repos

★制作cdh yum本地源

cp -R /etc/yum.repos.d /etc/yum.repos.d.bak
cd /etc/yum.repos.d/

cat >cloudera-manager.repo <<\EOF
[cloudera-manager]
name = Cloudera Manager, Version 5.9.0
baseurl = http://125.10.1.17/cloudera-repos/cm/5.9.0/RPMS/x86_64/
gpgcheck = 0
enabled=1
EOF

yum clean all
yum makecache
yum list|grep cloudera

★编辑/etc/hosts

125.10.1.17 cdh01
125.10.1.18 cdh02
125.10.1.19 cdh03

 ★配置/etc/sysctl.conf

echo 'vm.swappiness=0' >>/etc/sysctl.conf

sysctl -p 

★命令批量执行

--预留接口，当需要发送所有机器，在此步骤执行
--采用的是greenplum的分发工具操作
gpssh -f all_hosts "echo never > /sys/kernel/mm/transparent_hugepage/defrag"   
gpssh -f all_hosts "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
gpssh -f all_hosts "cat /sys/kernel/mm/transparent_hugepage/defrag"
gpssh -f all_hosts "cat /sys/kernel/mm/transparent_hugepage/enabled"

或
各位可以使用ssh到其他机器执行命令
ssh 其他机器  "指令"

★命令分发到其他机器

--采用的是greenplum的分发工具操作
cd /opt/cloudera
gpscp -f all_hosts /etc/hosts =:/etc
gpscp -f all_hosts /etc/sysctl.conf =:/etc
gpscp -f all_hosts /etc/yum.repos.d/cloudera-manager.repo =:/etc/yum.repos.d/

或
各位可以使用scp发送到其他机器
scp 文件   其他某个机器:目录

 

★在非master上安装外部的postgres数据库

yum -y  install postgres*
service postgresql initdb
service postgresql start

su - postgres
vi postgresql.conf
listen_addresses = '*'    
port = 5432 

vi pg_hba.conf
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD
local   all         all                               md5
# IPv4 local connections:
host    all         all         127.0.0.1/32          md5
# IPv6 local connections:
host    all         all         ::1/128               md5
host    all         all         0/0                   md5

psql 
db=# create user root        SUPERUSER password '123456';
db=# alter  user postgres    password '123456';
db=# create user hive        password '123456';
db=# create user amon        password '123456';
db=# create user hue         password '123456';
db=# create user oozie       password '123456';
db=# create database root    ENCODING ='utf8';
db=# create database hive    ENCODING ='utf8';
db=# create database amon    ENCODING ='utf8';
db=# create database hue     ENCODING ='utf8';
db=# create database oozie   ENCODING ='utf8';

service postgresql restart

★安装CDH

cd /opt/cloudera/

sz cloudera-manager-installer.bin
或
wget http://archive.cloudera.com/cm4/installer/latest/cloudera-manager-installer.bin

chmod u+x cloudera-manager-installer.bin
./cloudera-manager-installer.bin --skip_repo_package=1

service cloudera-scm-server restart

注意: 【安装日志路径】
/var/log/cloudera-manager-installer

★IE验证

http://XXX.XXX.XXX.XXX:7180

注意: 【运行日志】
/var/log/cloudera-scm-server/cloudera-scm-server.log

 

★IE上Use Parcels配置

Choose Method

Parcel Directory                                 /opt/cloudera/parcels Local Parcel Repository Path          /opt/cloudeta/parcel-repo Remote Rarcel Repository URLs     为空

Select the specific release of the Cloudera Manager Agent you want to install on your hosts.

Custom Repository

http://125.10.1.17/cloudera-repos/cm/5.9.0

 Enter a custom URL for the location of the GPG signing key (applies to all custom repositories and without Internet access).

Custom GPG Key URL:

http://125.10.1.17/cloudera-repos/cm/RPM-GPG-KEY-cloudera

 

★IE上Enable Single User Mode配置

Single User Mode 不选 

★IE上配置外部数据库-postgres

 

★IE后面一路自行安装即可

 

参考:

       https://docs.cloud.inspur.com/software/cloudera.html

       https://segmentfault.com/a/1190000013814973