hadoop安装步骤

最新推荐文章于 2023-09-07 16:22:30 发布

hhp_hhp_hhp

最新推荐文章于 2023-09-07 16:22:30 发布

阅读量333

点赞数

分类专栏： hadoop 文章标签： hadoop bash eclipse character ssh windows

hadoop 专栏收录该内容

9 篇文章 0 订阅

订阅专栏

1将压缩包复制到crywin/home/hhp下

2解压 tar -xzf hadoop-0.20.2.tar.gz
3配置文件：包括 hadoop-env.sh ， core-site.xml, hdfs-site.xml, mapred-site.xml ， masters ， slaves一共 6 个文件。
hadoop-env.sh
export JAVA_HOME=/cygdrive/d/jdk1.6.0_10
export HADOOP_CLASSPATH=/home/hhp/hadoop-0.20.2

参考命令：
路径转换：
How do I convert between Windows and UNIX paths?

    Use the 'cygpath' utility. Type 'cygpath --help' for information. For example (on my installation):

   bash$ cygpath --windows ~/.bashrc
        D:\starksb\.bashrc
        bash$ cygpath --unix C:/cygwin/bin/cygwin.bat
        /usr/bin/cygwin.bat
        bash$ cygpath --unix C:\\cygwin\\bin\\cygwin.bat
        /usr/bin/cygwin.bat
Note that bash interprets the backslash '\' as an escape character, so you must type it twice in the bash shell if you want it to be recognized as such.
文件名包含空格：
空格被作为单词分隔符；
   bash-2.03$ cd '/cygdrive/c/Program Files'
   or
   bash-2.03$ cd /cygdrive/c/Program\ Files

core-site.xml：

<property>

<name>fs.default.name</name>

<value>hdfs://202.112.1.50:9000</value>
</property>

hdfs-site.xml
<property>

<name>dfs.replication</name>

<value>1</value>
</property>

mapred-site.xml :
<property>

<name>mapred.job.tracker</name>

<value> 202.112.1.50 :9001</value>

</property>
4免密码ssh设置
ssh localhost yes 输入密码

4格式化namenode
bin/hadoop namenode -format

5启动hadoop守护进程
bin/start-all.sh
NameNode - http://localhost:50070/
JobTracker - http://localhost:50030/

6配置eclipse：
将插件hadoop-eclipse-plugin-0.20.3-SNAPSHOT 放入eclipse
启动eclipse
配置9001 9000

停止hadoop：
bin/stop-all.sh