cdh

准备工作


下载安装包
CDH5： http://archive.cloudera.com/cdh5/parcels/latest/  选择相关版本，内容全下载下来。http://archive.cloudera.com/cdh5/parcels/5.7.1/
Cloudera Manager: 
http://archive.cloudera.com/cm5/cm/5/   下载相关版本，并解压缩
http://archive.cloudera.com/cm5/redhat/5/x86_64/cm/5.7.1/RPMS/x86_64/  所有依赖包。


1.安装Apache 服务
yum install httpd createrepo
cd /var/www/html/
mkdir clouderamanager
cd /var/www/html/clouderamanager/
createrepo .
/etc/init.d/httpd start
httpd：httpd: Could not reliably determine the server's fully qualified domain name, using 192.168.1.11 for ServerName

http://192.168.1.11/clouderamanager/

2.配置本地yum源
所有节点添加:
vim /etc/yum.repos.d/cloudera-manager.repo
name = Cloudera Manager, Version x.x.x
baseurl = http://192.168.1.11/clouderamanager/
gpgcheck = 0

3.主机名配置
所有节点:
vim /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=CDH1

vim /etc/hosts
192.168.1.11 CDH1
192.168.1.12 CDH2
192.168.1.13 CDH3
192.168.1.14 CDH4


4.关闭防火墙
所有节点:
/etc/init.d/iptables stop
chkconfig iptables off


5.安装NTP服务
所有节点:
/etc/init.d/gmond start
chkconfig gmond on


6.关闭HUGEPAGE
所有节点:
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag

vim /etc/rc.local
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag


7.操作系统参数优化
所有节点:
echo "* soft stack 8192" >> /etc/security/limits.conf
echo "* hard stack 8192" >> /etc/security/limits.conf
echo "* soft nofile 65535" >> /etc/security/limits.conf
echo "* hard nofile 65535" >> /etc/security/limits.conf
echo "* soft nproc 65535" >> /etc/security/limits.conf
echo "* hard nproc 65535" >> /etc/security/limits.conf


8.交换分区设置
所有节点:
vm.swappiness = 0
sysctl -p

9.SSH免密码登陆配置
所有节点：
yum install openssh-clients wget bind-utils -y
ssh-keygen -t rsa –
cd /root/.ssh/
cp id_rsa.pub authorized_keys
CDH2,CDH3,CDH4上操作：
scp authorized_keys root@CDH1:/root/.ssh/CDH2_keys
scp authorized_keys root@CDH1:/root/.ssh/CDH3_keys
scp authorized_keys root@CDH1:/root/.ssh/CDH4_keys
生成集群主机公钥文件:在CDH1上操作
cat CDH2_keys>> authorized_keys
cat CDH3_keys>> authorized_keys
cat CDH4_keys>> authorized_keys
分发至集群所有节点:在CDH1上操作
scp authorized_keys root@CDH2:/root/.ssh/authorized_keys
scp authorized_keys root@CDH3:/root/.ssh/authorized_keys
scp authorized_keys root@CDH4:/root/.ssh/authorized_keys
测试验证:在CDH1上操作
ssh CDH2 date



开始安装
1.CDH1上操作：
cd /var/www/html/clouderamanager/
chmod +x cloudera-manager-installer.bin
./cloudera-manager-installer.bin
如长时间无响应，确认CDH1节点cloudera manager repository配置正确:
vi /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
name = Cloudera Manager, Version 5.7.1
baseurl = http://192.168.1.11/clouderamanager/
gpgcheck = 0 


2.CDH集群服务安装
http://192.168.1.11:7180
默认用户名: admin
默认密码: admin
A.选择Cloudera Express版本
B.添加主机
C.添加自定义Repository(http://192.168.1.11/clouderamanager/)并删除所有在线Repository
D.选择Single User Mode
E.输入root密码,需保证所有节点密码一致:
如发生异常:
每个节点删除lock文件:
rm -rf /tmp/.scm_prepare_node.lock 
rm -rf /var/run/yum.pid