Install Client Node for Cloudera Manager Hadoop Cluster

分区考虑，不要使用LVM
root -- >50G

opt -- >50G

/tmp -- >100G (run job失败的话请查看此目录空间)

swap -- 2倍系统内存

RAM -- >8GB

1. disable selinux and iptables
service iptables stop
chkconfig iptables off; chkconfig ip6tables off

setenforce 0
sed -i 's,SELINUX=enforcing,SELINUX=disabled,g' /etc/selinux/config

2. disable ipv6
echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf

sysctl -p

3. vi /etc/hosts to add all hosts FQDN, like below:
192.168.1.19    cm5.local cm5 archive.cloudera.com

192.168.1.20    master1.local master1  # HDFS NameNode
192.168.1.21    master2.local master2  # YARN ResourceManager
192.168.1.22    slave1.local slave1
192.168.1.23    slave2.local slave2

192.168.1.24    slave3.local slave3

192.168.1.25    client1.local client1

vi /etc/sysconfig/network to set FQDN

yum -y install ntp openssh-clients lzo

service ntpd start; chkconfig ntpd on

cat << EOF > /etc/yum.repos.d/iso.repo
[iso]
name=iso
baseurl=http://mirrors.aliyun.com/centos/6.5/os/x86_64
enable=1
gpgcheck=0
EOF

4. cat << EOF > /etc/yum.repos.d/cm520.repo

[cm520]
name=cm520
baseurl=http://192.168.1.19/cm520
enable=1
gpgcheck=0
EOF

yum -y install oracle-j2sdk1.7 cloudera-manager-agent cloudera-manager-daemons

ln -s  /usr/java/jdk1.7.0_67-cloudera /usr/java/default
echo 'export JAVA_HOME=/usr/java/default' >> /etc/profile
echo 'export PATH=$JAVA_HOME/bin:$PATH' >> /etc/profile
source /etc/profile

vi /etc/cloudera-scm-agent/config.ini
server_host=cm5.local
server_port=7182

service cloudera-scm-agent start

5. add client node host to cluster and add gateway roles to it

You can configure gateway roles for HDFS, YARN, Hive, HBase, MapReduce, Solr, Sqoop 1 Client, Accumulo, Spark.

ok

转载于:https://blog.51cto.com/hj192837/1568341