windows安装hadoop-单节点
步骤说明
- 下载jdk,hadoop。
- 安装jdk,解压hadoop。
- 配置环境变量,配置hadoop。
- 初始化hdfs
- 开始使用
详细
- 下载JDK,版本1.8.0_65;
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
- 下载hadoop,版本2.6.3;
http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.6.3/hadoop-2.6.3.tar.gz - 下载hadoop-windows组件,适用于2.6;
http://pan.baidu.com/s/1o6RX2IY - 安装JDK;
默认安装目录为C:\Program Files\Java\jdk1.8.0_65 - 解压hadoop;
解压目录为D:\hadoop\hadoop-2.6.3 - 配置环境变量;
变量名 HADOOP_HOME 变量值 D:\hadoop\hadoop-2.6.3
变量名 JAVA_HOME 变量值 C:\ progra~1 \Java\jdk1.8.0_65 修改Hadoop配置,都在hadoop-2.6.3\etc\hadoop\ 目录下
修改 hadoop-env.cmd文件,在文件末尾追加
set HADOOP_PREFIX=D:\hadoop\hadoop-2.6.3 set HADOOP_CONF_DIR=%HADOOP_PREFIX%\etc\hadoop set YARN_CONF_DIR=%HADOOP_CONF_DIR% set PATH=%PATH%;%HADOOP_PREFIX%\bin
修改core-site.xml文件
<configuration> <property> <name>fs.default.name</name> <value>hdfs://127.0.0.1:19000</value> </property> </configuration>
修改hdfs-site.xml文件
<configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration>
复制mapred-site.xml.template文件,重命名为mapred-site.xml,用户名改为当前登录用户名
<configuration> <property> <name>mapreduce.job.user.name</name> <value>用户名</value> </property> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>yarn.apps.stagingDir</name> <value>/user/用户名/staging</value> </property> <property> <name>mapreduce.jobtracker.address</name> <value>local</value> </property> </configuration>
- 修改yarn-site.xml文件
<configuration> <property> <name>yarn.server.resourcemanager.address</name> <value>127.0.0.1:8020</value> </property> <property> <name>yarn.server.resourcemanager.application.expiry.interval</name> <value>60000</value> </property> <property> <name>yarn.server.nodemanager.address</name> <value>127.0.0.1:45454</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> <property> <name>yarn.server.nodemanager.remote-app-log-dir</name> <value>/app-logs</value> </property> <property> <name>yarn.nodemanager.log-dirs</name> <value>/dep/logs/userlogs</value> </property> <property> <name>yarn.server.mapreduce-appmanager.attempt-listener.bindAddress</name> <value>127.0.0.1</value> </property> <property> <name>yarn.server.mapreduce-appmanager.client-service.bindAddress</name> <value>127.0.0.1</value> </property> <property> <name>yarn.log-aggregation-enable</name> <value>true</value> </property> <property> <name>yarn.log-aggregation.retain-seconds</name> <value>-1</value> </property> <property> <name>yarn.application.classpath</name> <value>%HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_COMMON_HOME%/share/hadoop/common/lib/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/*,%HADOOP_HDFS_HOME%/share/hadoop/hdfs/lib/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/*,%HADOOP_MAPRED_HOME%/share/hadoop/mapreduce/lib/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/lib/*</value> </property> </configuration>
- 增加windows下所需文件,将第8步中下载的hadoop-windows组件解压,文件全部复制到hadoop-2.6.3\bin目录下,重名文件覆盖即可。
- 执行下hadoop-2.6.3\etc\hadoop\hadoop-env.cmd文件
- 初始化HDFS
- 打开cmd控制台,输入d: 回车进入d盘操作。
- 输入hdfs namenode -format 回车
- D盘下会多出一个tmp文件夹就是了。
- 启动 hadoop,执行hadoop-2.6.3\sbin\start-all.cmd 文件
- 浏览器输入http://127.0.0.1:50070 即可查看hdfs的使用情况等。