Hadoop学习笔记-windows环境搭建

下载解压

Hadoop下载地址http://hadoop.apache.org/

Windows下运行需要hadoop.dll和winutils.exe,2.7.3版本对应的可以从以下地址下载，下载完放到bin下面，下载地址: https://github.com/rucyang/hadoop.dll-and-winutils.exe-for-hadoop2.7.3-on-windows_X64/tree/master/bin

配置环境变量

设置环境变量JAVA_HOME和HADOOP_HOME，默认的JAVA_HOME默认C:\Program Files\Java\jdk1.8.0_60会出错，可能目录中有空格吧，我换了个地址。

配置core-site和hdfs-site.xml

D:\hadoop-2.7.3\etc\hadoop\core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

D:\hadoop-2.7.3\etc\hadoop\hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

执行hadoop version测试

启动HDFS

格式化namenode

sbin/start-dfs.cmd启动HDFS

启动完可以打开网页查看：http://127.0.0.1:50070/

HDFS Shell命令参考http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_shell.html

Utilities/Browse the file system可以查看HDFS目录结构

测试YARN

http://127.0.0.1:8088/

测试MapReduce计算PI

执行hadoop jar hadoop-mapreduce-examples-2.7.3.jar pi 5 10

转载于:https://my.oschina.net/Endless2010/blog/1422378