Windows--hadoop-CSDN博客

本文链接：https://blog.csdn.net/windowsLearner/article/details/8598068

本文介绍了在Windows上使用Cygwin安装和运行Hadoop的步骤，包括如何通过Cygwin运行Hadoop，配置Eclipse进行Hadoop和Hive的编译与调试，并解决Cygwin依赖问题和遇到的常见错误。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Cygwin 的使用

Windows--Cygwin--运行hadoop http://wenku.baidu.com/view/6af47921af45b307e8719799.html http://wenku.baidu.com/view/45409fc289eb172ded63b775.html

Windows--Cygwin--Eclipse--编译hadoop http://wenku.baidu.com/view/c1ad44323968011ca3009199.html http://wenku.baidu.com/view/f57268a4b0717fd5360cdc97.html 将Cygwin加入环境变量

Windows--Eclipse--调试hadoop http://blog.csdn.net/yanical/article/details/4474830

Windows--Eclipse--调试Hive http://wenku.baidu.com/view/f4d9b407e87101f69e319599.html

------------

Cgwin--Resolving Dependencies

crypt (1.2-1)
Encryption/Decryption utility and library
Required by: openssh, libkrb5_26, libaprutil1, libsasl2, perl, libroken18, libopenldap2_3_0, libpq5

csih (0.9.6-1)
Provides support for installing cygwin services
Required by: openssh

cygrunsrv (1.40-2)
NT/W2K service initiator
Required by: openssh

libapr1 (1.4.6-1)
Apache Portable Runtime core library (runtime)
Required by: subversion, libaprutil1, libserf1_0, libserf0_1

libaprutil1 (1.4.1-1)
Apache Portable Runtime utility library (runtime)
Required by: subversion, libserf1_0, libserf0_1

libasn1_8 (1.5.2-4)
Kerberos 5 implementation (ASN.1 library)
Required by: libgssapi3, libkafs0, libkrb5_26, libhx509_5

libcom_err2 (1.42.6-1)
Common error description library (runtime)
Required by: libgssapi3, libkrb5_26, libasn1_8, libheimntlm0, libhx509_5, libwind0

libdb4.5 (4.5.20.2-3)
Oracle Berkeley DB (db4.5 - C and C++ libs)
Required by: subversion, libaprutil1, libsasl2, perl

libedit0 (20120311-1)
The NetBSD Editline library (runtime)
Required by: openssh

libexpat1 (2.1.0-1)
Expat XML parser library (runtime)
Required by: subversion, libaprutil1, libneon27, perl_vendor

libgdbm4 (1.8.3-20)
GNU dbm database routines (runtime)
Required by: libaprutil1, perl

libgssapi3 (1.5.2-4)
Kerberos 5 implementation (GSSAPI library)
Required by: openssh

libheimbase1 (1.5.2-4)
Kerberos 5 implementation (base library)
Required by: libgssapi3, libkrb5_26

libheimntlm0 (1.5.2-4)
Kerberos 5 implementation (NTLM library)
Required by: libgssapi3

libhx509_5 (1.5.2-4)
Kerberos 5 implementation (HX509 library)
Required by: libkrb5_26

libkafs0 (1.5.2-4)
Kerberos 5 implementation (AFS library)
Required by: openssh

libkrb5_26 (1.5.2-4)
Kerberos 5 implementation (Krb5 library)
Required by: openssh, libgssapi3, libkafs0, libheimntlm0

libneon27 (0.29.6-2)
HTTP and WebDAV library - (runtime)
Required by: subversion

libopenldap2_3_0 (2.3.43-3)
Lightweight Directory Access Protocol suite - (runtime)
Required by: libaprutil1

csih有此目录却有错误

参考

http://www.cnblogs.com/yutenghit/archive/2012/07/23/2605483.html

重新安装问题解决，只是这次重新安装时去掉了一些不需要安装的包的选项(不记得如何勾选的) Strange

设置无密码访问

ssh-copy-id -i id_rsa.pub Administrator@localhost 要求输入密码 w上的密码已经取消了这里还要求输入填写以前的密码可以设置了
http://blog.csdn.net/cybercode/article/details/7080743

遇到一个cygwin错误

http://hi.baidu.com/windhl/item/bd455715be49b80ed1d66d58

http://ljhzzyx.blog.163.com/blog/static/383803122012811101934797/ http://blog.sina.com.cn/s/blog_625defdd0100ov0n.html 不管用http://hi.baidu.com/windhl/item/bd455715be49b80ed1d66d58 从linux机器直接拷贝下来不管用

http://blog.sina.com.cn/s/blog_7226f3660100uipz.html

http://blog.csdn.net/kkdelta/article/details/7699088

问题解决http://blog.sina.com.cn/s/blog_764bad0f0100t8nf.html

结果出现形如“./bin/hadoop:Line 2:$'r':Command Not Found ”的错误，需要运行如下命令修复shell文件：

dos2unix bin/hadoop
dos2unix bin/*.sh
dos2unix conf/*.sh

如此，方可无错误的运行hadoop命令。

启动每个node log错误 java.lang.IllegalArgumentException: Does not contain a valid host:port authority:file:///
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:162)

http://bbs.csdn.net/topics/380004117

http://www.cnblogs.com/ggjucheng/archive/2012/01/08/2316692.html

http://blog.csdn.net/21aspnet/article/details/1535741

配置有误 http://hadoop.apache.org/docs/r1.0.4/single_node_setup.html 问题解决同样遇到了一个0700错误

将4个函数注释掉启动Hadoop ，Cygwin中jps 只能看到启动了jobtracker 和namenode 但是log和关闭时显示均启动另一个cygwin的，将cygwin的path加入环境变量，path却看不到它但是可以运行cygwin里的exe

由于 Cannot run program "autoreconf" (in directory

使用Cygwin 安装autoconf http://www.gnu.org/software/autoconf/#downloading http://man.chinaunix.net/linux/lfs/htmlbook/chapter06/autoconf.html

安装时停在checking for GNU m4 that supports accurate traces...

换一种方法只用到Java的包又没有别的方法可以安装windows eclipse autoconf 把build.xml文件从后往前凡用到autoreconf的都去掉在cmd直接可以编译成功了在Eclipse还需clean 一下，不过重新打开之后clean是灰的 build project不起作用但是eclipse上的小红叉没有了