Ubuntu 14.04.1 LTS 安装Hadoop 2.7.0(单节点) 一

      自己想玩hadoop，但总是由于种种异常导致不想玩了。今天呢天气太热了，没地方玩只能在家搞hadoop了，不过还好最终能启动成功了。以此来记录下过程方便以后自己参考，也给想玩hadoop的程序员提供一个参考

1.下载解压就不说了，我下载的是2.7.0版本的。

2.hadoop解压目录/opt/app/hadoop/hadoop-2.7.0。

在/opt/app/hadoop/目录下创建一下目录

tmp、hdfs、hdfs/data、hdfs/name

3.配置/etc/hadoop目录下的core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.56.101:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/hadoop/tmp</value>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131702</value>
    </property>
</configuration>

4.配置/etc/hadoop目录下的hdfs-site.xml

<configuration>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:/opt/app/hadoop/hdfs/name</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:/opt/app/hadoop/hdfs/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>192.168.56.101:9001</value>
    </property>
    <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
</configuration>

5.配置/etc/hadoop目录下mapred-site.xml

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
         <property>
                <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
                <value>org.apache.hadoop.mapred.ShuffleHandler</value>
        </property>
        <property>
                <name>yarn.resourcemanager.address</name>
                <value>192.168.56.101:8032</value>
        </property>
        <property>
                <name>yarn.resourcemanager.scheduler.address</name>
                <value>192.168.56.101:8030</value>
        </property>
        <property>
                <name>yarn.resourcemanager.resource-tracker.address</name>
                <value>192.168.56.101:8031</value>
        </property>
        <property>
                <name>yarn.resourcemanager.admin.address</name>
                <value>192.168.56.101:8033</value>
        </property>
        <property>
                <name>yarn.resourcemanager.webapp.address</name>
                <value>192.168.56.101:8088</value>
        </property>
        <property>
                <name>yarn.nodemanager.resource.memory-mb</name>
                <value>1024</value>
        </property>
        <property>
                 <name>mapreduce.jobhistory.address</name>
                 <value>192.168.56.101:10020</value>
        </property>
        <property>
                 <name>mapreduce.jobhistory.webapp.address</name>
                 <value>192.168.56.101:19888</value>
        </property>
</configuration>

6.配置/etc/hadoop目录下etc/hadoop/yarn-site.xml

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

7.配置/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME

#Java setting
JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79

8.配置环境变量

在~/.profile文件中添加环境变量

vim ~/.profile

#setting java
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_79
export JRE_HOME=${JAVA_HOME}/jre


#setting hadoop
export HADOOP_HOME=/opt/app/hadoop/hadoop-2.7.0
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"


export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

不设置的话启动hadoop会报错。

错误如下：

hadoop@ubuntu:~$ /opt/app/hadoop/hadoop-2.7.0/sbin/start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
15/01/23 20:23:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [Java HotSpot(TM) Client VM warning: You have loaded library /hadoop/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
hd-m1]
sed: -e expression #1, char 6: unknown option to `s'
-c: Unknown cipher type 'cd'
hd-m1: starting namenode, logging to /hadoop/hadoop-2.6.0/logs/hadoop-root-namenode-hd-m1.out
HotSpot(TM): ssh: Could not resolve hostname HotSpot(TM): Temporary failure in name resolution
Java: ssh: Could not resolve hostname Java: Temporary failure in name resolution
Client: ssh: Could not resolve hostname Client: Temporary failure in name resolution
You: ssh: Could not resolve hostname You: Temporary failure in name resolution
warning:: ssh: Could not resolve hostname warning:: Temporary failure in name resolution
VM: ssh: Could not resolve hostname VM: Temporary failure in name resolution......等等

9.格式化NameNode

/opt/app/hadoop/hadoop-2.7.0/bin/hadoop namenode -format

10.启动Hadoop

/opt/app/hadoop/hadoop-2.7.0/sbin/start-all.sh

11.启动Hadoop任务历史服务器

 ./sbin/mr-jobhistory-daemon.sh   start historyserver

http://192.168.56.101:19888

12.启动成功，访问管理界面

All Applications:   http://192.168.56.101:8088

Node:   http://192.168.56.101:8042

Status:   http://192.168.56.101:9001

好吧，现在只是简单的启动会成功了，说实话没技术含量，但是要一步一步来哇，该去吃饭了~