Hadoop HDFS SingleCluster安装

环境准备

机器列表

主机ip	主机名	用途
10.80.3.55	node1	namenode
10.80.3.163	node2	datanode/secondrynamenode
10.80.3.132	node3	datanode
10.80.3.164	node4	datanode

确保每台机器已经安装好jdk环境

下载解压

# 下载安装包
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz
# 解压到`/home`目录下
tar -xvzf hadoop-2.7.2.tar.gz -C /home

配置ssh免密登录

# node1的公钥加入到所有机器的`authorized_keys`中
ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys

修改hadoop的JAVA环境变量etc/hadoop/hadoop-env.sh

# set to the root of your Java installation
export JAVA_HOME=/usr/local/jdk1.8

修改core-site.xml

<configuration>
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://node1:9000</value>
</property>
<property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/hadoop-2.7.2</value>
</property>
</configuration>

修改hdfs-site.xml

<configuration>
<property>
 <name>dfs.namenode.secondary.http-address</name>
<value>node2:50090</value>
</property>
<property>
 <name>dfs.namenode.secondary.https-address</name>
<value>node2:50091</value>
</property>
</configuration>

修改masters文件和slaves文件

# vi masters #配置secondrynamenode节点机器
node2
# vi slaves #配置datanode节点机器
node2
node3
node4

格式化namenode

bin/hdfs namenode -format

start-hdfs.sh启动

sbin/start-dfs.sh