CentOS7.2系统Hadoop集群部署

最新推荐文章于 2023-11-01 23:46:51 发布

置顶思考v

最新推荐文章于 2023-11-01 23:46:51 发布

阅读量675

点赞数

分类专栏：云计算

本文链接：https://blog.csdn.net/xiegh2014/article/details/53364916

版权

云计算专栏收录该内容

13 篇文章 0 订阅

订阅专栏

1.基本环境：
操作系统：
[root@hadoop-master ~]# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core) 

设置主机名：
10.0.8.10主机上执行
echo "Hadoop-master" > /etc/hostname

10.0.8.11主机上执行
echo "Hadoop-node1" > /etc/hostname

10.0.8.12主机上执行
echo "Hadoop-node2" > /etc/hostname

在三台服务上执行
cat >>/etc/hosts<< EOF
10.0.8.10         Hadoop-master
10.0.8.11         Hadoop-node1
10.0.8.12         Hadoop-node2
EOF

ping  Hadoop-master
ping  Hadoop-node1
ping  Hadoop-node2

关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
grep SELINUX=disabled /etc/selinux/config
setenforce 0

软件包
hadoop-2.7.3.tar.gz
jdk-8u101-linux-x64.tar.gz

安装JDK
yum install glibc.i686 -y
tar -xvf jdk-8u101-linux-x64.tar.gz -C /usr/local/src/
cat >>/etc/profile<< EOF
export JAVA_HOME=/usr/local/src/jdk1.8.0_101
export JRE_HOME=/usr/local/src/jdk1.8.0_101/jre
export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
EOF
source /etc/profile
java -version

创建用户
useradd hadoop
echo hadoop |passwd --stdin hadoop
echo hadoop  "ALL=(ALL)       NOPASSWD: ALL" >>/etc/sudoers
grep -w hadoop  /etc/sudoers

配置key无密钥连接

ssh-keygen -t rsa 
cd /home/hadoop/.ssh/ 
cp id_rsa.pub  authorized_keys 
chmod go-wx  authorized_keys

scp ~/.ssh/authorized_keys hadoop-node1:/home/hadoop/.ssh/
scp ~/.ssh/authorized_keys hadoop-node2:/home/hadoop/.ssh/

验证登录：
[root@hadoop-master ~]# ssh hadoop-node1
[root@hadoop-master ~]# ssh hadoop-node2

为方便node1，node2中hadoop用户可以同时登录master，这里我们集群中的所有用户共用一个密钥
scp ~/.ssh/id_rsa hadoop-node1:/home/hadoop/.ssh/
scp ~/.ssh/id_rsa hadoop-node2:/home/hadoop/.ssh/