我终于用Eclipse成功连接上Hadoop了!!
啊啊啊啊啊啊!!来自少女的狂欢啊!!!!太快乐了!!!!
我遇到的情况如下:
然后就是还有下面这一大长串:(看着就头疼!!)
org.apache.hadoop.net.ConnectTimeoutException: Call From LAPTOP-JNDAQOOH/192.168.137.1 to 192.168.100.11:9000 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=192.168.100.11/192.168.100.11:9000]; For more details see: http://wiki.apache.org/hadoop/SocketTimeout
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:751)
at org.apache.hadoop.ipc.Client.call(Client.java:1480)
at org.apache.hadoop.ipc.Client.call(Client.java:1413)
at org.apache.hadoop.ipc.ProtobufRpcEngine
I
n
v
o
k
e
r
.
i
n
v
o
k
e
(
P
r
o
t
o
b
u
f
R
p
c
E
n
g
i
n
e
.
j
a
v
a
:
229
)
a
t
c
o
m
.
s
u
n
.
p
r
o
x
y
.
Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.
Invoker.invoke(ProtobufRpcEngine.java:229)atcom.sun.proxy.Proxy13.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:563)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy14.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3014)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2984)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1047)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1043)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1043)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1036)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1880)
at com.hadoop.hdfs.hfs.test(hfs.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner
2.
e
v
a
l
u
a
t
e
(
P
a
r
e
n
t
R
u
n
n
e
r
.
j
a
v
a
:
268
)
a
t
o
r
g
.
j
u
n
i
t
.
r
u
n
n
e
r
s
.
P
a
r
e
n
t
R
u
n
n
e
r
.
r
u
n
(
P
a
r
e
n
t
R
u
n
n
e
r
.
j
a
v
a
:
363
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
4.
r
u
n
n
e
r
.
J
U
n
i
t
4
T
e
s
t
R
e
f
e
r
e
n
c
e
.
r
u
n
(
J
U
n
i
t
4
T
e
s
t
R
e
f
e
r
e
n
c
e
.
j
a
v
a
:
86
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
.
r
u
n
n
e
r
.
T
e
s
t
E
x
e
c
u
t
i
o
n
.
r
u
n
(
T
e
s
t
E
x
e
c
u
t
i
o
n
.
j
a
v
a
:
38
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
.
r
u
n
n
e
r
.
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
r
u
n
T
e
s
t
s
(
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
j
a
v
a
:
459
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
.
r
u
n
n
e
r
.
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
r
u
n
T
e
s
t
s
(
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
j
a
v
a
:
675
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
.
r
u
n
n
e
r
.
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
r
u
n
(
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
j
a
v
a
:
382
)
a
t
o
r
g
.
e
c
l
i
p
s
e
.
j
d
t
.
i
n
t
e
r
n
a
l
.
j
u
n
i
t
.
r
u
n
n
e
r
.
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
m
a
i
n
(
R
e
m
o
t
e
T
e
s
t
R
u
n
n
e
r
.
j
a
v
a
:
192
)
C
a
u
s
e
d
b
y
:
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
n
e
t
.
C
o
n
n
e
c
t
T
i
m
e
o
u
t
E
x
c
e
p
t
i
o
n
:
20000
m
i
l
l
i
s
t
i
m
e
o
u
t
w
h
i
l
e
w
a
i
t
i
n
g
f
o
r
c
h
a
n
n
e
l
t
o
b
e
r
e
a
d
y
f
o
r
c
o
n
n
e
c
t
.
c
h
:
j
a
v
a
.
n
i
o
.
c
h
a
n
n
e
l
s
.
S
o
c
k
e
t
C
h
a
n
n
e
l
[
c
o
n
n
e
c
t
i
o
n
−
p
e
n
d
i
n
g
r
e
m
o
t
e
=
192.168.100.11
/
192.168.100.11
:
9000
]
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
n
e
t
.
N
e
t
U
t
i
l
s
.
c
o
n
n
e
c
t
(
N
e
t
U
t
i
l
s
.
j
a
v
a
:
534
)
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
n
e
t
.
N
e
t
U
t
i
l
s
.
c
o
n
n
e
c
t
(
N
e
t
U
t
i
l
s
.
j
a
v
a
:
495
)
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
i
p
c
.
C
l
i
e
n
t
2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=192.168.100.11/192.168.100.11:9000] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:534) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.ipc.Client
2.evaluate(ParentRunner.java:268)atorg.junit.runners.ParentRunner.run(ParentRunner.java:363)atorg.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)atorg.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)atorg.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)atorg.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)atorg.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)atorg.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)Causedby:org.apache.hadoop.net.ConnectTimeoutException:20000millistimeoutwhilewaitingforchanneltobereadyforconnect.ch:java.nio.channels.SocketChannel[connection−pendingremote=192.168.100.11/192.168.100.11:9000]atorg.apache.hadoop.net.NetUtils.connect(NetUtils.java:534)atorg.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)atorg.apache.hadoop.ipc.ClientConnection.setupConnection(Client.java:615)
at org.apache.hadoop.ipc.Client
C
o
n
n
e
c
t
i
o
n
.
s
e
t
u
p
I
O
s
t
r
e
a
m
s
(
C
l
i
e
n
t
.
j
a
v
a
:
713
)
a
t
o
r
g
.
a
p
a
c
h
e
.
h
a
d
o
o
p
.
i
p
c
.
C
l
i
e
n
t
Connection.setupIOstreams(Client.java:713) at org.apache.hadoop.ipc.Client
Connection.setupIOstreams(Client.java:713)atorg.apache.hadoop.ipc.ClientConnection.access$2900(Client.java:376)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
at org.apache.hadoop.ipc.Client.call(Client.java:1452)
… 43 more
被这个bug扰了我两天两夜啊!!
我尝试了很多方法都没有用,然后让我一直以为是我的hadoop-eclipse-plugin-2.7.7.jar出了问题,不过它确实是有一点问题:
看我的博客:编译hadoop-eclipse-2.7.7插件
里面最后Success的代码前是有提示警告,这就涉及到你的jdk版本了,一般1.8的比较好,也得跟你的hadoop集群一样。
接着就是关键问题所在了!!
- Hadoop集群防火墙是否关闭
我一直坚信我的防火墙是关闭的了。
特意看了我的集群搭建博客:Hadoop集群搭建(超详细)(伪分布式配置) 是关了的
你们千万千万不要跟我一样啊!!!赶紧的,不管它关没关,咋们再关它一次!!
有几个集群就给我关它几次
systemctl stop firewalld.service
systemctl disable firewalld.service
2.eclipse的版本问题,最好是使用Eclipse的Mars版本。
- 在cmd里检查下master node上的9000端口是否开启了,可以通过telnet的方式测试
telnet 192.168.100.11 9000
第一次基本上是无法连接的,因为一般win7或win10系统中telnet是禁用状态,需要手动启动它。
控制面板-- 系统与安全--左边列里的程序--程序与功能下有个管理员的标志--启用或关闭Windows功能-- √ Telnet Client
这样就打开了Telnet功能了,然后重新在cmd里执行就成功了
telnet 192.168.100.11 9000
成功连接后一般不会有信息,所以不要以为是有错误哦!
回到Eclipse后再尝试连接hadoop就成功了(记得hadoop先连上集群哦)。
集群查看就可以看到新建的文件夹 /dt/tmp1