本文参照 hadoop2.2在window7上搭建单节点环境进行实施
想了解hadoop,所以就下了最新的文件,搭建相关的环境,以下为自己在win7上搭建hadoop的相关步骤。要在64位windows7中进行,否则需要自己编译hadoop-commin中的动态库和可执行文件。
1、下载hadoop-2.4.0.tar.gz,用winRAR解压。(路径不能有空格)解压到d:\hadoop目录下
2、安装java,此处就不做讲解了。jdk版本在1.7+
3、配置hadoop的环境变量HADOOP_HOME="HADOOP的安装目录" 如d:\hadoop\hadoop.2.4.0
4、在环境变量的PATH中加入HADOOP的安装目录/bin 如%HADOOP_HOME%/bin
5、配置JAVA_HOME变量为当前java的安装路径。
6、下载hadoop-commin-2.2.zip(由于没找到2.4对应的编译版本。采用2.2版本的也可以用。),因为在windows下运行,压缩包里面缺少 winutils.exe, hadoop.dll等文件,下载完成后,将要报下面的bin目录下的所有文件全部拷贝到hadoop目录下的bin文件夹下。
7、修改配置文件,以下四个文件全部在hadoop目录下的etc/hadoop目录下
修改core-site.xml,如下:
01 | <? xml version = "1.0" encoding = "UTF-8" ?> |
02 | <? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 | <!-- |
04 | Licensed under the Apache License, Version 2.0 (the "License"); |
05 | you may not use this file except in compliance with the License. |
06 | You may obtain a copy of the License at |
07 | |
09 | |
10 | Unless required by applicable law or agreed to in writing, software |
11 | distributed under the License is distributed on an "AS IS" BASIS, |
12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 | See the License for the specific language governing permissions and |
14 | limitations under the License. See accompanying LICENSE file. |
15 | --> |
16 | |
17 | <!-- Put site-specific property overrides in this file. --> |
18 | |
19 | < configuration > |
20 | < property > |
21 | < name >fs.defaultFS</ name > |
22 | < value >hdfs://localhost:9000</ value > |
23 | </ property > |
24 | </ configuration > |
修改hdfs-site.xml如下:
01 | <? xml version = "1.0" encoding = "UTF-8" ?> |
02 | <? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 | <!-- |
04 | Licensed under the Apache License, Version 2.0 (the "License"); |
05 | you may not use this file except in compliance with the License. |
06 | You may obtain a copy of the License at |
07 | |
09 | |
10 | Unless required by applicable law or agreed to in writing, software |
11 | distributed under the License is distributed on an "AS IS" BASIS, |
12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 | See the License for the specific language governing permissions and |
14 | limitations under the License. See accompanying LICENSE file. |
15 | --> |
16 | |
17 | <!-- Put site-specific property overrides in this file. --> |
18 | |
19 | < configuration > |
20 | < property > |
21 | < name >dfs.replication</ name > |
22 | < value >1</ value > |
23 | </ property > |
24 | < property > |
25 | < name >dfs.namenode.name.dir</ name > |
26 | < value >file:/hadoop/data/dfs/namenode</ value > |
27 | </ property > |
28 | < property > |
29 | < name >dfs.datanode.data.dir</ name > |
30 | < value >file:/hadoop/data/dfs/datanode</ value > |
31 | </ property > |
32 | </ configuration > |
修改yarn-site.xml如下:
01 | <? xml version = "1.0" ?> |
02 | <!-- |
03 | Licensed under the Apache License, Version 2.0 (the "License"); |
04 | you may not use this file except in compliance with the License. |
05 | You may obtain a copy of the License at |
06 | |
08 | |
09 | Unless required by applicable law or agreed to in writing, software |
10 | distributed under the License is distributed on an "AS IS" BASIS, |
11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
12 | See the License for the specific language governing permissions and |
13 | limitations under the License. See accompanying LICENSE file. |
14 | --> |
15 | < configuration > |
16 | < property > |
17 | < name >yarn.nodemanager.aux-services</ name > |
18 | < value >mapreduce_shuffle</ value > |
19 | </ property > |
20 | < property > |
21 | < name >yarn.nodemanager.aux-services.mapreduce.shuffle.class</ name > |
22 | < value >org.apache.hadoop.mapred.ShuffleHandler</ value > |
23 | </ property > |
24 | </ configuration > |
修改mapred-site.xml如下:
01 | <? xml version = "1.0" ?> |
02 | <? xml-stylesheet type = "text/xsl" href = "configuration.xsl" ?> |
03 | <!-- |
04 | Licensed under the Apache License, Version 2.0 (the "License"); |
05 | you may not use this file except in compliance with the License. |
06 | You may obtain a copy of the License at |
07 | |
09 | |
10 | Unless required by applicable law or agreed to in writing, software |
11 | distributed under the License is distributed on an "AS IS" BASIS, |
12 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
13 | See the License for the specific language governing permissions and |
14 | limitations under the License. See accompanying LICENSE file. |
15 | --> |
16 | |
17 | <!-- Put site-specific property overrides in this file. --> |
18 | |
19 | < configuration > |
20 | < property > |
21 | < name >mapreduce.framework.name</ name > |
22 | < value >yarn</ value > |
23 | </ property > |
24 | </ configuration > |
然后打开cmd,运行hadoop namenode -format命令,运行结果基本如下:
d:\hadoop\hadoop-2.4.0\bin>hdfs namenode -format
注意观查控制台信息如果没有报错会在/hadoop/data/dfs/namenode下和/hadoop/data/dfs/datanode下生成一些文件。
然后在cmd下切换目录到hadoop2.4.0目录下的sbin目录下,运行start-all 会打开四个cmd窗口,可以打开浏览器输入 http://localhost:8042以及http://localhost:50070查看是否配置成功!