hadoop本地模式、伪分布式模式

一、系统环境

  Linux: centos 6.5 x64

  JDK: jdk-8u45-linux-x64.tar.gz

  hadoop: hadoop-2.7.0.tar

二、安装配置

1.安装jdk

[root@bogon /usr/local/src]#tar zxvf jdk-8u45-linux-x64.tar.gz

[root@bogon /usr/local/src]#mkdir /usr/java

[root@bogon /usr/local/src]#mv jdk-8u45-linux-x64 /usr/java/jdk

2.安装hadoop

[root@bogon /usr/local/src]#tar zxvf hadoop-2.7.0.tar.gz

3.配置

[root@bogon /usr/local/src]#mv hadoop-2.7.0 /usr/local/

[root@bogon /usr/local/src]#cd /usr/local/

[root@bogon /usr/local]#ln -sv hadoop-2.7.0 hadoop

编写两个shell，进行路径配置 bin还是sbin看你的.sh在哪里，本机配置时sbin下启动sh，bin下hadoop都添加

java.sh

JAVA_HOME=/usr/java/jdk

PATH=$JAVA_HOME/bin:$PATH

export JAVA_HOME PATH

hadoop.sh

HADOOP_HOME=/usr/local/hadoop

PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

export HADOOP_HOME PATH

移动到/etc/profile.d/

重新打开终端，检查安装

[root@bogon /usr/local]#useradd hduser

[root@bogon /usr/local]#passwd hduser

[root@bogon /usr/local]#chown -R hduser.hduser /usr/local/hadoop/              （斜杠不能少）

[root@bogon /usr/local]#su hduser

[hduser@bogon /usr/local/hadoop/etc/hadoop]#cd /usr/local/hadoop/etc/hadoop/

[hduser@bogon /usr/local/hadoop/etc/hadoop]#vim core-site.xml

[hduser@bogon /usr/local/hadoop/etc/hadoop]#vim mapred-site.xml.template

[hduser@bogon /usr/local/hadoop/etc/hadoop]#vim hdfs-site.xml

[hduser@localhost hadoop]$ vim yarn-site.xml

[hduser1@localhost ~]$ ssh-keygen -t rsa -P ''

[hduser1@localhost sbin]$ cd ~
[hduser1@localhost ~]$ ssh-copy-id -i .ssh/id_rsa.pub hduser@localhost

[hduser@bogon /usr/local/hadoop/sbin]#cd /usr/local/hadoop/sbin/

[root@bogon /usr/local/hadoop/sbin]#./start-all.sh

[hduser@bogon sbin]$ jps

6339 SecondaryNameNode

6486 ResourceManager

6214 DataNode

6119 NameNode

6616 Jps

6585 NodeManager

 

4.出错检查log

[hduser@bogon sbin]$hadoop namenode -format         //格式化namenode

[hduser@bogon sbin]$start-all.sh

[hduser@bogon sbin]$jps

 

 查看各个操作有没有出现ERROR，注意用户、755权限

不能创建

mkdir -p /hadoop/temp/

chown -R hduser.hduser /hadoop/temp/

 

namenode没起来

stop-all.sh

rm /hadoop/temp/dfs/name/current/VERSION

hadoop namenode -format

start-all.sh

 

datanode没起来

移除VERSION

格式化datanode

格式化namenode 

 

 native-hadoop library未加载

http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.6.0.tar
下载解压到lib/native下

转载于:https://my.oschina.net/charlock/blog/651662