java web:Broken Pipe问题排查始末

声明

首次,查找到博客等材料,只有 BrokenPipe错误终极解释,比较靠谱,但是实际的操作中,发现表现并不一样,具体的可见该博客的评论,或者自己实现下。

接下来就是尝试,不过事先声明,不要在windows os上测试,该问题无法完全复现,均会报“Connection reset by peer: socket write error”。

结论

好了,不废话,上结论,原因是:socket的client单方面关闭(不论是client所在进程崩溃还是timeout)后,且已经向server发送了RST,server端才向response写入data,这样,server端(目前tomcat的实现是会在error时再次write一次)第一次write时报“Connection reset by peer”,第二次重试又write时,才会报“Broken pipe”。

测试demo

过程说明

  1. server端暴露3113端口,并等待接收数据
  2. client端连接3113,并发送数据,立即close
    1. socket设置为接收到close命令后,立即向server发送RST
  3. server休眠10s(只要保证client能关掉即可)
  4. server再次发送数据,并在捕获到异常后,再次尝试发送(模拟tomcat在接收到异常时的再次flush行为)

环境说明

jdk版本:1.8

1、server端代码

package com.socket;

import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class Server {
    //server程序
    public static void main(String[] args) {
        try {
            ServerSocket ss = new ServerSocket(3113);
            Socket s = ss.accept();
            InputStream is = s.getInputStream();
            byte[] buf =new byte[1024];
            int len = is.read(buf);
            System.out.println("recv:"+new String(buf,0,len));

            Thread.sleep(10000);
            // 模拟tomcat的两次flush
            try {
                // rst后第一次写
                s.getOutputStream().write("hello1".getBytes());
            } catch (Throwable e) {
                e.printStackTrace();
                // 接收到connection reset by peer后第二次写
                s.getOutputStream().write("hello2".getBytes());
            }

            System.out.println("send over");
            System.in.read();
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

编译命令:javac Server.java

运行命令:java Server

2、client端代码

package com.socket;

import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Socket;

public class Client {
    //client程序
    public static void main(String[] args) {
        try {
            Socket s = new Socket();
            s.setSoLinger(true,0);//设置调用close就发送RST(很关键)
            s.connect(new InetSocketAddress("0.0.0.0",3113));

            OutputStream os = s.getOutputStream();
            os.write("hello".getBytes());
            Thread.sleep(1000);
            s.close();

            System.in.read();//防止程序退出
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

编译命令:javac Client.java

运行命令:java Client  

3、结果(server端)

recv:hello
java.net.SocketException: Connection reset by peer (Write failed)
        at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
        at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:138)
        at Server.main(Server.java:18)
java.net.SocketException: Broken pipe (Write failed)
        at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
        at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:138)
        at Server.main(Server.java:21)

附:tomcat中该问题的表现

可根据两者不同的栈信息进行排查tomcat的具体行为

java.io.IOException: Connection reset by peer

	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)

	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)

	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)

	at sun.nio.ch.IOUtil.write(IOUtil.java:65)

	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470)

	at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:134)

	at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)

	at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:157)

	at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1306)

	at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:731)

	at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:684)

	at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:674)

	at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.flush(Http11OutputBuffer.java:646)

	at org.apache.coyote.http11.filters.ChunkedOutputFilter.flush(ChunkedOutputFilter.java:169)

	at org.apache.coyote.http11.Http11OutputBuffer.flush(Http11OutputBuffer.java:252)

	at org.apache.coyote.http11.Http11Processor.flush(Http11Processor.java:1561)

	at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:399)

	at org.apache.coyote.Response.action(Response.java:206)

	at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:318)

	at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:285)

	at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:118)

	at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1100)

	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:915)

	at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.writeInternal(AbstractJackson2HttpMessageConverter.java:285)

	at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write(AbstractGenericHttpMessageConverter.java:106)

	at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:231)

	at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.handleReturnValue(RequestResponseBodyMethodProcessor.java:174)

	at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:81)

	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:113)

	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854)

	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765)

	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)

	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)

	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)

	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)

	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)

	at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)

	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)

	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)

	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at com.megvii.framework.log.filter.LogTraceFilter.doFilter(LogTraceFilter.java:28)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111)

	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at io.micrometer.spring.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:106)

	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)

	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103)

	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)

	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)

	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)

	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)

	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)

	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)

	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)

	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)

	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)

	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)

	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)

	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)

	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)

	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

	at java.lang.Thread.run(Thread.java:748)

java.io.IOException: Broken pipe

	at sun.nio.ch.FileDispatcherImpl.write0(Native Method)

	at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)

	at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)

	at sun.nio.ch.IOUtil.write(IOUtil.java:65)

	at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470)

	at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:134)

	at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)

	at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:157)

	at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1306)

	at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:731)

	at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:684)

	at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:674)

	at org.apache.coyote.http11.Http11OutputBuffer$SocketOutputBuffer.flush(Http11OutputBuffer.java:646)

	at org.apache.coyote.http11.filters.ChunkedOutputFilter.flush(ChunkedOutputFilter.java:169)

	at org.apache.coyote.http11.Http11OutputBuffer.flush(Http11OutputBuffer.java:252)

	at org.apache.coyote.http11.Http11Processor.flush(Http11Processor.java:1561)

	at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:399)

	at org.apache.coyote.Response.action(Response.java:206)

	at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:318)

	at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:285)

	at org.apache.catalina.connector.Response.flushBuffer(Response.java:500)

	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:89)

	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)

	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)

	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)

	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)

	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)

	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)

	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)

	at java.lang.Thread.run(Thread.java:748)

  • 0
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值