hadoop 2.7 java_hadoop2.7.2修改配置文件，配置linux java环境变量

1.修改core-site.xml，加上

fs.defaultFS

hdfs://backup02:9000

hadoop.tmp.dir

file:/home/zhongml/hadoop-2.7.2/tmp

io.file.buffer.size

131702

2.修改hdfs-site.xml，加上

dfs.namenode.name.dir

file:/home/zhongml/hadoop-2.7.2/hdfs/name

dfs.datanode.data.dir

file:/home/zhongml/hadoop-2.7.2/hdfs/data

dfs.replication

2

dfs.namenode.secondary.http-address

backup02:9001

dfs.webhdfs.enabled

true

3.先复制一个mapred-site.xml

cp mapred-site.xml.template mapred-site.xml

修改mapred-site.xml，加上

mapreduce.framework.name

yarn

mapreduce.jobhistory.address

backup02:10020

mapreduce.jobhistory.webapp.address

backup02:19888

4.修改yarn-site.xml，加上

yarn.nodemanager.aux-services

mapreduce_shuffle

yarn.nodemanager.auxservices.mapreduce.shuffle.class

org.apache.hadoop.mapred.ShuffleHandler

yarn.resourcemanager.address

backup02:8032

yarn.resourcemanager.scheduler.address

backup02:8030

yarn.resourcemanager.resource-tracker.address

backup02:8031

yarn.resourcemanager.admin.address

backup02:8033

yarn.resourcemanager.webapp.address

backup02:8088

yarn.nodemanager.resource.memory-mb

768

5.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下hadoop-env.sh、yarn-env.sh的JAVA_HOME

6.配置/home/zhongml/hadoop-2.7.2/etc/hadoop目录下slaves

7.验证启动是否成功

bin/hadoop fs -ls

http://localhost:50030(MapReduce的页面)

http://localhost:50070(HDFS的页面)

8.如果是64位的linux，需要覆盖native

9.配置java环境变量/etc/profile

先卸载linux自带的jdk，

执行以下命令查看需要卸载的选项:

rpm -qa | grep gcj

执行卸载操作:

yum -y remove java-1.4.2-gcj-compat-1.4.2.0-40jpp.115

将以下内容添加到/etc/profile末尾

JAVA_HOME=/jdk/jdk1.8.0_101/

PATH=$JAVA_HOME/bin:$PATH

CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export JAVA_HOME

export PATH

export CLASSPATH

注意:装好hadoop环境必须关闭防火墙:

关闭防火墙服务

[root@cluster3 hadoop]# service iptables stop

关闭开机自动启动

[root@cluster3 hadoop]# chkconfig iptables off

安装以下软件:

yum install svn

yum install autoconfautomake libtool cmake

yum install ncurses-devel

yum install openssl-devel

yum install gcc*