安装Hadoop

在Centos7下

192.168.6.201  master
192.168.6.202  node1
192.168.6.203  node2

192.168.6.201主机：
    echo "master" > /etc/hostname
192.168.6.202主机：
    echo "node1" > /etc/hostname
192.168.6.203主机：
    echo "node2" > /etc/hostnamemaster上修改hosts文件

echo "192.168.6.201 master" >>/etc/hosts
echo "192.168.6.202 node1" >>/etc/hosts
echo "192.168.6.203 node2" >>/etc/hosts

同步到node1，node2上的主机
scp /etc/hosts node1:/etc/
scp /etc/hosts node2:/etc/

下面安装java:

rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

yum install -y java-1.8.0-openjdk

master，node1，node2上创建hadoop用户:
useradd hadoop;passwd  hadoop;

将hadoop用户加入sudo权限管理:
visudo在root    ALL=(ALL)      ALL 后加入
hadoop    ALL=(ALL)      ALL
master，node1，node2上进入hadoop用户：
su - hadoop

在master，node1，node2上无密钥连接:
master上的hadoop用户通过rsa算法生成非对称密钥对：
ssh-keygen -t rsa
cd /home/hadoop/.ssh/
cp id_rsa.pub  authorized_keys
chmod go-wx  authorized_keys

master上的公钥authorized_keys拷贝到node1，node2上的hadoop用户
scp authorized_keys node1:/home/hadoop/.ssh/
scp authorized_keys node2:/home/hadoop/.ssh/

可通过如下命令测试
ssh node1
ssh node2
 为方便node1，node2中hadoop用户可以同时登录master，这里我们集群中的所有用户共用一个密钥
scp ~/.ssh/id_rsa node1:/home/hadoop/.ssh/
scp ~/.ssh/id_rsa node2:/home/hadoop/.ssh/

vi /home/hadoop/.bash_profile
最后加上:
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
su - hadoop  #重新读取环境变量

master，node1，node2上创建相关目录
  sudo mkdir -p /usr/local/hadoop
  sudo chown -R hadoop:hadoop  /usr/local/hadoop
  sudo mkdir -p /data/hadoop/  #创建 hadoop 数据目录结构
  sudo chown -R hadoop:hadoop  /data/hadoop/
  mkdir -p /data/hadoop/tmp/  #创建 tmp
  mkdir -p /data/hadoop/hdfs/  #创建hdfs
  mkdir -p /data/hadoop/hdfs/data  #创建datanode目录
  mkdir -p /data/hadoop/hdfs/name  #创建namenode目录
  mkdir -p /data/hadoop/hdfs/namesecondary

下载hadoop:
在这里http://hadoop.apache.org/releases.html，找binary，我用的是:
wget http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz

然后安装:
tar -zxvf hadoop-2.7.3.tar.gz 
mv hadoop-2.7.3 /usr/local/hadoop
chown -R hadoop:hadoop /usr/local/hadoop/

在master上配置:
cd /usr/local/hadoop/etc/hadoop:

vi core-site.xml:
<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.6.201:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/data/hadoop/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
 </configuration>

vi hdfs-site.xml:
<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/data/hadoop/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/data/hadoop/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>192.168.6.201:9001</value>
    </property>
    <property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
    </property>
 </configuration>

cp mapred-site.xml.template mapred-site.xml
vi mapred-site.xml
 <configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>192.168.6.201:10020</value>
    </property>
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>192.168.6.201:19888</value>
    </property>
 </configuration>

vi yarn-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>master:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>master:8088</value>
</property>
<property>  
    <name>yarn.nodemanager.aux-services</name>  
    <value>mapreduce_shuffle</value>  
</property>
</configuration>

vi hadoop-env.sh:改下java路径
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/

vi yarn-env.sh:改下java路径
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/

vi slaves:
node1
node2

开工:
cd /usr/local/hadoop
初始化:
bin/hdfs namenode -format
全部启动:
sbin/start-all.sh
停止:
sbin/stop-all.sh
看到相关信息:
jps