CDH安装1(CM安装)

设置HOST======================================================================================================================
所有节点
centos 6.x
echo NETWORKING=yes >> /etc/sysconfig/network
echo HOSTNAME=CDH1 >> /etc/sysconfig/network


centos 7.x
hostnamectl set-hostname CDH1
该命令会修改/etc/hostname文件


echo 10.10.10.11 CDH1 >> /etc/hosts
echo 10.10.10.12 CDH2 >> /etc/hosts
echo 10.10.10.13 CDH3 >> /etc/hosts
echo 10.10.10.14 CDH4 >> /etc/hosts




安装HTTP服务======================================================================================================================
其中一个节点
yum install httpd createrepo
centos 6.x
/etc/init.d/httpd start


centos 7.x
/bin/systemctl start  httpd.service


验证
http://10.10.10.11/clouderamanager/




下载相关文件======================================================================================================================
CDH1
EL是enterprise linux的意思
1.下载CDH包
http://archive.cloudera.com/cdh5/parcels/ 找到相关版本


wget http://archive.cloudera.com/cdh5/parcels/5.10.1/CDH-5.10.1-1.cdh5.10.1.p0.10-el7.parcel
wget http://archive.cloudera.com/cdh5/parcels/5.10.1/CDH-5.10.1-1.cdh5.10.1.p0.10-el7.parcel.sha1
wget http://archive.cloudera.com/cdh5/parcels/5.10.1/manifest.json
2.下载Cloudera manager 5 安装文件
http://archive.cloudera.com/cm5/  找到相关版本
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/cloudera-manager-agent-5.10.1-1.cm5101.p0.6.el7.x86_64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/cloudera-manager-daemons-5.10.1-1.cm5101.p0.6.el7.x86_64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/cloudera-manager-server-5.10.1-1.cm5101.p0.6.el7.x86_64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/cloudera-manager-server-db-2-5.10.1-1.cm5101.p0.6.el7.x86_64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/enterprise-debuginfo-5.10.1-1.cm5101.p0.6.el7.x86_64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/jdk-6u31-linux-amd64.rpm
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.10.1/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

3.下载Cloudera manager 5 安装引导文件
http://archive.cloudera.com/cm5/  找到相关版本
wget http://archive.cloudera.com/cm5/installer/5.10.1/cloudera-manager-installer.bin


cd /var/www/html/
mkdir clouderamanager
下载到/var/www/html/clouderamanager
createrepo .
创建repo


下载完了才能“createrepo .”  “createrepo .” 和下载的文件有关系，修改文件也要重新执行该操作


各种优化======================================================================================================================
以下都是所有节点。
1.关闭HUGEPAGE 
centos 6.x
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag


vim /etc/rc.local
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag


centos 7.x
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local
echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.local

2.关闭防火墙
centos 6.x
/etc/init.d/iptables stop
chkconfig iptables off

centos 7.x
/bin/systemctl status firewalld.service
/bin/systemctl stop firewalld.service
3.安装NTP服务 
yum install -y ntp

centos 6.x
/etc/init.d/ntpd start
chkconfig ntpd on 345【跟随系统启动而启动】
chkconfig --list ntpd【查看服务是否添加到自启动】


centos 7.x
/bin/systemctl start  ntpd.service
systemctl enable ntpd.service【跟随系统启动而启动】
systemctl list-unit-files【查看服务是否添加到自启动】
4.操作系统参数优化
echo "* soft stack 8192" >> /etc/security/limits.conf
echo "* hard stack 8192" >> /etc/security/limits.conf
echo "* soft nofile 65535" >> /etc/security/limits.conf
echo "* hard nofile 65535" >> /etc/security/limits.conf
echo "* soft nproc 65535" >> /etc/security/limits.conf
echo "* hard nproc 65535" >> /etc/security/limits.conf
5.交换分区设置
echo vm.swappiness = 0  >> /etc/sysctl.conf
sysctl -p
6.安装SSH CLIENT
yum install openssh-clients wget bind-utils -y

设置免密登录======================================================================================================================
1.生成本机authorized_keys[所有节点]
ssh-keygen -t rsa    三个回车
cd /root/.ssh/
cp id_rsa.pub authorized_keys
2.copy authorized_keys 到node A[除了node A] 
scp authorized_keys root@CDH1:/root/.ssh/cdh2_keys
scp authorized_keys root@CDH1:/root/.ssh/cdh3_keys
scp authorized_keys root@CDH1:/root/.ssh/cdh4_keys
3.生成集群主机公钥文件[node A] 
cat cdh2_keys >> authorized_keys
cat cdh3_keys >> authorized_keys
cat cdh4_keys >> authorized_keys
4.将公钥文件分发至集群所有节点[node A] 
scp authorized_keys root@CDH2:/root/.ssh/authorized_keys
scp authorized_keys root@CDH3:/root/.ssh/authorized_keys
scp authorized_keys root@CDH4:/root/.ssh/authorized_keys
5.测试[node A] 
ssh CDH2 date
ssh CDH3 date
ssh CDH4 date
安装clouderamanager======================================================================================================================
[node A] 


echo [cloudera-manager] > /etc/yum.repos.d/cloudera-manager.repo
echo name = Cloudera Manager, Version 5.10.1 >> /etc/yum.repos.d/cloudera-manager.repo
echo baseurl = http://10.10.10.11/clouderamanager/ >> /etc/yum.repos.d/cloudera-manager.repo
echo gpgcheck = 0 >> /etc/yum.repos.d/cloudera-manager.repo


备份并删除/etc/yum.repos.d下所有除了CentOS-Base.repo的文件，如果全删除会报 postgresql-server >= 8.4安装异常。
每次安装失败后cloudera-manager.repo文件会被删除，需要重新生成。


cd /var/www/html/clouderamanager/
chmod +x cloudera-manager-installer.bin
./cloudera-manager-installer.bin