java.net.ConnectException: Connection timed out: no further information

最新推荐文章于 2024-07-17 08:45:47 发布
最新推荐文章于 2024-07-17 08:45:47 发布
阅读量6.6k 收藏 3
点赞数 2
分类专栏： bugbug
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_33286695/article/details/90753077
版权

首先，本地读取云集群hdfs文件报连接超时，这个问题困的我。。。。。。。
百度，谷歌很多都是防火墙，端口，路径的错误，这些都不是这个问题的根源，防火墙开着也可以读取hdfs中的文件，所以有必要记下这个问题，错误如下：

19/06/03 15:25:09 WARN BlockReaderFactory: I/O error constructing remote block reader.
	java.net.ConnectException: Connection timed out: no further information
		at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
		at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
		at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
		at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
		at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3101)
		at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:755)
		at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:670)
		at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:337)
		at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:576)
		at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:800)
		at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:847)
		at java.io.DataInputStream.read(DataInputStream.java:100)
		at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
		at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
		at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
		at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:134)
		at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.liftedTree1$1(HadoopRDD.scala:257)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:256)
		at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:214)
		at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:94)
		at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
		at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
		at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
		at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
		at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
		at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
		at org.apache.spark.scheduler.Task.run(Task.scala:109)
		at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)
	19/06/03 15:25:09 WARN BlockReaderFactory: I/O error constructing remote block reader.
	java.net.ConnectException: Connection timed out: no further information
		at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
		at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
		at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
		at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
		at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3101)
		at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:755)
		at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:670)
		at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:337)
		at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:576)
		at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:800)
		at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:847)
		at java.io.DataInputStream.read(DataInputStream.java:100)
		at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
		at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
		at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
		at org.apache.hadoop.mapred.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:206)
		at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:244)
		at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:277)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:214)
		at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
		at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
		at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
		at scala.collection.Iterator$class.foreach(Iterator.scala:893)
		at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
		at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$28.apply(RDD.scala:927)
		at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$28.apply(RDD.scala:927)
		at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
		at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
		at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
		at org.apache.spark.scheduler.Task.run(Task.scala:109)
		at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)
	19/06/03 15:25:09 WARN DFSClient: Failed to connect to /IP:50010 for block, add to deadNodes and continue. java.net.ConnectException: Connection timed out: no further information
	java.net.ConnectException: Connection timed out: no further information
		at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
		at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
		at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
		at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
		at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3101)
		at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:755)
		at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:670)
		at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:337)
		at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:576)
		at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:800)
		at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:847)
		at java.io.DataInputStream.read(DataInputStream.java:100)
		at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
		at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
		at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
		at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:134)
		at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.liftedTree1$1(HadoopRDD.scala:257)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:256)
		at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:214)
		at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:94)
		at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
		at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
		at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
		at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
		at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
		at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
		at org.apache.spark.scheduler.Task.run(Task.scala:109)
		at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)
	19/06/03 15:25:09 WARN DFSClient: Failed to connect to /IP:50010 for block, add to deadNodes and continue. java.net.ConnectException: Connection timed out: no further information
	java.net.ConnectException: Connection timed out: no further information
		at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
		at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
		at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
		at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
		at org.apache.hadoop.hdfs.DFSClient.newConnectedPeer(DFSClient.java:3101)
		at org.apache.hadoop.hdfs.BlockReaderFactory.nextTcpPeer(BlockReaderFactory.java:755)
		at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:670)
		at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:337)
		at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:576)
		at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:800)
		at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:847)
		at java.io.DataInputStream.read(DataInputStream.java:100)
		at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:180)
		at org.apache.hadoop.util.LineReader.readDefaultLine(LineReader.java:216)
		at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174)
		at org.apache.hadoop.mapred.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:206)
		at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:244)
		at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:47)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:277)
		at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:214)
		at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
		at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
		at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
		at scala.collection.Iterator$class.foreach(Iterator.scala:893)
		at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
		at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$28.apply(RDD.scala:927)
		at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$28.apply(RDD.scala:927)
		at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
		at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2074)
		at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
		at org.apache.spark.scheduler.Task.run(Task.scala:109)
		at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)

解决方案：
问题在于但我们使用本地读取hdfs文件时，云集群datanode使用的是内网进行通信，所以我们把hdfs-site.xml和core-site.xml文件导入到Idea中时，需要修改hdfs-site.xml中的以下配置：

<property>
        <name>dfs.datanode.use.datanode.hostname</name>
        <value>true</value>
</property>

通过查阅资料，建议在hdfs-site.xml中增加dfs.datanode.
use.datanode.hostname属性，表示datanode之间的通信也通过域名方式

binbincoder
关注
  • 2
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 3
    评论
专栏目录
异常:java.net.ConnectException: Connection timed out: no further information
wuzj1314的博客
04-08 1万+
异常:java.net.ConnectException: Connection timed out: no further information Zookeeper注册中心连接超时 原因：可能是注册中心所在的计算机【Linux ContOS7】的防火墙未关闭。 解决方法：关闭防火墙 centOS7防火墙的开启、关闭、禁用命令 （1）设置开机启用防火墙：systemctl enable firewalld.service （2）设置开机禁用防火墙：systemctl disable fire.
出现Exception in thread “main“ java.net.ConnectException: Connection timed out: no further等解决方法
码农研究僧的博客
01-18 1万+
出现如下问题： 在连接rabbitmq的时候出现了连接超时的提示 Exception in thread "main" java.net.ConnectException: Connection timed out: no further information at java.base/sun.nio.ch.Net.pollConnect(Native Method) at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) at java.b
Hdfs连接报错java.net.ConnectException: Connection timed out: no further information
赵广陆
10-11 4417
目录1 连接超时2 排查具体问题2.1.检查网络能不能ping通,防火墙问题2.2 检查zookeeper有没有开启2.3 最后检查Hue中Hdfs节点挂掉3 解决方法 1 连接超时 2 排查具体问题 2.1.检查网络能不能ping通,防火墙问题 由于有Hue可以访问,网络没有问题 2.2 检查zookeeper有没有开启 2.3 最后检查Hue中Hdfs节点挂掉 最后可以看出上述问题都没有所以大环境是没有错,那就是代码的问题 3 解决方法 问题在于使用本地读取hdfs文件时，服务器datanode使
Final Shell。遇到java.net.ConnectException: Connection timed out: connect如何处理
最新发布
qq_62752002的博客
07-17 855
在使用 FinalShell 时，遇到错误通常表示网络连接问题。
Caused by java.net.ConnectException Connection timed out connect
m0_67393342的博客
03-20 2万+
jdbc远程连接数据库超时 标题主要可能有连个原因：mysql服务没开，防火墙没有放行3306端口 七月 14, 2020 5:04:18 下午 com.alibaba.druid.pool.DruidDataSource error 严重: create connection error, url: jdbc:mysql://192.168.184.130:3306/project_crowd, errorCode 0, state 08S01 com.mysql.jdbc.exceptions.jdbc
解决：java.net.ConnectException: Connection timed out: no further information
T
11-05 3550
我今天在搭建cloud的时候 用zookeeper做注册中心的时候包的错 报错信息： java.net.ConnectException: Connection timed out: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_131] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImp
Caused by: java.net.ConnectException: Connection timed out: no further information
12-03
Caused by: java.net.ConnectException: Connection timed out: no further information通常是由于网络连接问题引起的。可能的原因包括： 1. 目标主机不可达或已关闭。 2. 防火墙阻止了连接请求。 3. 网络延迟或不...
【已解决】java.net.ConnectException: Connection timed out: no further information
热门推荐
wzder
12-22 5万+
前言： 今天连接zookeepter的时候出现问题(上午连接的时候还没问题的，下午就出现了问题，很是无奈) 报错信息如下： java.net.ConnectException: Connection timed out: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at s...
异常：java.net.ConnectException: Connection timed out: no further information
weixin_43656426的博客
12-13 2302
今天安装完zookeeper连接的时候报了这么一个错误，目标主机无法访问（也就是无法连接zookeeper），检查了一下自己的连接路径和主机，后来在同事的提醒下终于找到了问题的关键-----------防火墙 由于没有关闭linux中的防火墙，所以导致安装的zookeeper无法连接，一直报这个错误。 解决 进入Linux命令中，输入命令： 第一种： sudo service firewalld ...
java api连接hbase一直卡住 java.net.ConnectException: Connection timed out: no further information
qq_40694036的博客
07-03 1710
一般是没在host中指明路径 C:\Windows\System32\drivers\etc\hosts 中最后加一行 192.168.25.164 master 连接时有可能还报错 java.net.ConnectException: Connection timed out: no further information 原因是没关防火墙 systemctl stop iptables //关...
评论 3
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值