Hadoop2.7.3的安装详细过程(伪分布式)

1.安装java
2.设置ssh免密码登入
3.安装Hadoop

1.安装java:

rpm -ivh jdk-8u101-linux-i586.rpm
然后设置环境变量
vi /etc/profile
在最末端添加:
#set java env
    JAVA_HOME=/usr/java/jdk1.8.0_101
    PATH=$JAVA_HOME/bin:$PATH
    CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
    export JAVA_HOME
    export PATH
    export CLASSPATH

最后,再在终端输入java -version测试一下,到此,java便安装结束了。

2.设置ssh免密码登入
ssh-keygen -t rsa
这将会产生一个公钥和私钥
ssh-copy-id -i slaver1
连续按几次enter直至结束
再来一次
ssh-copy-id -i slaver2

3.安装hadoop
tar -xzvf hadoop-2.7.3.tar.gz
cd进入解压后的文件夹
首先,先修改hadoop下面etc/hadoop里面的5个文件:
cd /usr/hadoop/hadoop-2.7.3/etc/hadoop
1.core-site.xml


<configuration>
     <property>
             <name>fs.default.name</name>
             <value>hdfs://master:9000</value>
     </property>
     <property>
             <name>hadoop.tmp.dir</name>
             <value>file:/usr/hadoop/hadoop-2.7.3/tmp</value>
     </property>
     <property>
             <name>dfs.replication</name>
             <value>1</value>
     </property>
</configuration>
2.yarn-site.xml
<configuration>


<!-- Site specific YARN configuration properties -->
        <property>
                        <name>yarn.resourcemanager.hostname</name>
                        <value>master</value>
        </property>
                <!-- reducer获取数据的方式 -->
        <property>
                        <name>yarn.nodemanager.aux-services</name>
                        <value>mapreduce_shuffle</value>
        </property>
        <property>
                        <value>org.apache.mapred.ShuffleHandler</value>
        </property>
        <property>
                        <name>yarn.resourcemanager.address</name>
                        <value>master:8032</value>
        </property>
        <property>
                        <name>yarn.resourcemanager.scheduler.address</name>
                        <value>master:8030</value>
        </property>
        <property>
                        <name>yarn.resourcemanager.resource-tracker.address</name>
                        <value>master:8031</value>
        </property>
        <property>
                        <name>yarn.resourcemanager.admin.address</name>
                        <value>master:8033</value>
        </property>
        <property>
                        <name>yarn.resourcemanager.webapp.address</name>
                        <value>master:8088</value>
        </property>


</configuration>
3.mapred-site.xml
<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>master:10020</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>master:19888</value>
  </property>
  <property>
     <name>mapred.job.tracker</name>
     <value>master:9001</value>
  </property>


</configuration>
4.hdfs-site.xml
<configuration>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>/usr/hadoop/hadoop-2.7.3/hdf/data</value>
                <final>true</final>
        </property>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>/usr/hadoop/hadoop-2.7.3/hdf/name</value>
                <final>true</final>
        </property>


</configuration>
5.hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_101
export HADOOP_HOME=/usr/hadoop/hadoop-2.7.3
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export  HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

6.yarn-env.sh

export JAVA_HOME=/usr/java/jdk1.8.0_101
export HADOOP_HOME=/usr/hadoop/hadoop-2.7.3
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native
export  HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

hadoop.env.sh和yarn-env.sh这里的后三行网上很多人都没有写,听说64位的机器才需要加上,但是我的是32位的,没有加上去仍然不行。。

然后把Hadoop文件夹发到其他主机上
scp -r /usr/hadoop slaver1:/usr/hadoop
scp -r /usr/hadoop slaver1:/usr/hadoop

在最后加上环境变量,所有的主机都要设置。
vi /etc/profile

#set hadoop env
export HADOOP_HOME=/usr/hadoop/hadoop-2.7.3
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
export HADOOP_LOG_DIR=/usr/hadoop/hadoop-2.7.3/logs
export YARN_LOG_DIR=$HADOOP_LOG_DIR

最后,启动hadoop集群
./sbin/start-all.sh

在终端输入
[root@master ~]# jps
28597 NameNode
28934 ResourceManager
29800 Jps
28792 SecondaryNameNode
[root@master hadoop-2.7.3]# ./bin/hdfs dfsadmin -report
16/10/29 10:02:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
Configured Capacity: 97575411712 (90.87 GB)
Present Capacity: 79984197632 (74.49 GB)
DFS Remaining: 79984140288 (74.49 GB)
DFS Used: 57344 (56 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0


Live datanodes (2):

Name: 192.168.183.84:50010 (slaver1)
Hostname: localhost.localdomain
Decommission Status : Normal
Configured Capacity: 49795620864 (46.38 GB)
DFS Used: 28672 (28 KB)
Non DFS Used: 8847638528 (8.24 GB)
DFS Remaining: 40947953664 (38.14 GB)
DFS Used%: 0.00%
DFS Remaining%: 82.23%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 29 10:02:28 EDT 2016

Name: 192.168.183.51:50010 (slaver2)
Hostname: localhost.localdomain
Decommission Status : Normal
Configured Capacity: 47779790848 (44.50 GB)
DFS Used: 28672 (28 KB)
Non DFS Used: 8743575552 (8.14 GB)
DFS Remaining: 39036186624 (36.36 GB)
DFS Used%: 0.00%
DFS Remaining%: 81.70%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 29 10:02:29 EDT 2016

进入http://192.168.183.70:50070/
可以看到
这里写图片描述

到此,Hadoop安装成功。

  • 0
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值