假设有三台虚拟机其ip分别为(在别的网络环境以当时ip地址为准):
master:192.168.1.140
slave1:192.168.1.141
slave2:192.168.1.142
配置主机名(分别在各自主机中运行上述命令,改为各自的主机名)
$ vi /etc/hostnname
修改每台机器的hosts文件,将主机名及ip地址映射填写进去
$ vi /etc/hosts
## 在文件底部添加:
192.168.1.140 master
192.168.1.141 slave1
192.168.1.142 slave2
配置后可用ping命令检测是否能够连通
$ ping master
$ ping slave1
$ ping slave2
配置免密登录
$ ssh-keygen -t rsa
$ cd /root/.ssh
$ cat id_rsa.pub >> ./authorized_keys
运行过ssh-keygen后会产生俩个文件
- id_rsa私钥
- id_rsa.pub公钥
将master的公钥传送给slave节点
配置jdk
$ vi /etc/profile
### 在底部追加:
export JAVA_HOME=/usr/local/share/jdk
export PATH=$JAVA_HOME/bin:$PATH
export HADOOP_HOME=/home/hadoop
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
解压Hadoop压缩包
暂定hadoop目录为/home
$ cd /home/hadoop/etc/hadoop
$ vi hadoop-env.sh
### 补齐JAVA_HOME HADOOP_HOME
### 创建一个临时文件路径
mkdir /home/tmp
配置core-site.xml
<property>
<name>fs.default.name</name>
<value>hdfs://master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/tmp</value>
</property>
配置hdfs-site.xml
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/hdfs/data</value>
</property>
配置mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>master:19888</value>
</property>
配置yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
将设置完成的hadoop发送至slave1和slave2对应的路径上
添加USER定义
$ vi start-dfs.sh
#!/usr/bin/env bash
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
$ vi stop-dfs.sh
#!/usr/bin/env bash
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
$ vi start-yarn.sh stop-yarn.sh
#!/usr/bin/env bash
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
$ vi stop-yarn.sh
#!/usr/bin/env bash
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
查看服务
$ jps