基于Netty实现https代理客户端

最新推荐文章于 2024-09-14 17:21:40 发布

想去滨海看日出

最新推荐文章于 2024-09-14 17:21:40 发布

阅读量1.6k

点赞数

文章标签： https 服务器网络

本文链接：https://blog.csdn.net/qq_43520928/article/details/129658268

版权

文章介绍了如何基于Netty实现一个HTTPS代理客户端，重点在于理解HTTP隧道代理协议和TLS加密，以及解决在Java中携带自定义请求头的问题。作者提到了在实现过程中遇到的挑战，如两次TLS握手和HttpClientCodec的配置，并提供了相应的解决方案。

摘要由CSDN通过智能技术生成

基于Netty实现https代理客户端

业务场景和需求

说一说我的场景和需求,我的场景是需要一个能在http的connect请求头上携带自定义的请求头,而标准的jdk实现和apache client都没办法做到,那么我就想自己实现这个功能.后来发现curl的高版本能实现这个功能,但是毕竟用Java调用shell不太方便,所以还是研究了一番

一,https代理原理

https代理就是在http代理协议的基础上加一层tls,由于tls是加密的,所以需要代理之间建立一个隧道来传输密文.所以https代理协议就是利用http的隧道代理协议+tls加密协议.组成了https代理协议.

http隧道代理协议建立的过程:首先客户端会发送一个请求类型为connect请求的,http请求,该请求和普通的GET/POST http请求不同的是,他会在url上放目标网站的域名和端口,以及在host上放目标网站的域名和端口,代理服务端收到请求就会与目标网站建立tcp连接,当连接建立完成就会返回客户端200,连接建立成功报文,客户端再向目标网站发起tls握手请求完后发起正常http请求,代理服务只做tcp层数据流的中转.

connect请求报文如下:

 CONNECT www.baidu.com:443 HTTP/1.1
 User-Agent: curl
 Host: www.baidu.com
 Content-Length: 0
 Connection: Keep-Alive

connect 响应报文

 HTTP/1.1 200 Connection Established

交互流程图

二,netty实现代理客户端源码

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioChannelOption;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.*;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import io.netty.handler.proxy.HttpProxyHandler;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import io.netty.util.AttributeKey;
import io.netty.util.CharsetUtil;
import io.netty.util.NetUtil;

import java.net.InetSocketAddress;


public class NettyClient {

    static final String proxy_auth =  "";
    static final AttributeKey<HttpEntityPromise> key = AttributeKey.valueOf("response");

    public static void main(String[] args) throws Exception {

        Bootstrap bootstrap = new Bootstrap();
        bootstrap.channel(NioSocketChannel.class);

        bootstrap.option(NioChannelOption.CONNECT_TIMEOUT_MILLIS, 30 * 1000);

        NioEventLoopGroup group = new NioEventLoopGroup();
        try {

            bootstrap.group(group);

            LoggingHandler loggingHandler = new LoggingHandler(LogLevel.INFO);

            SslContextBuilder sslContextBuilder = SslContextBuilder.forClient();
            //下面这行，直接信任自签证书
            sslContextBuilder.trustManager(InsecureTrustManagerFactory.INSTANCE);
            SslContext sslContext = sslContextBuilder.build();

            bootstrap.handler(new ChannelInitializer<NioSocketChannel>() {
                @Override
                protected void initChannel(NioSocketChannel ch) throws Exception {
                    ChannelPipeline pipeline = ch.pipeline();
                    pipeline.addLast("ssl", sslContext.newHandler(ch.alloc()));
                    pipeline.addLast("http-codec",new HttpClientCodec());
                    pipeline.addLast("log", loggingHandler);
                    pipeline.addLast("biz", new HandleBiz());

                }
            });
            ChannelFuture channelFuture = bootstrap.connect("xxx.x.x.x", 8088);
            channelFuture.sync();
            HttpEntityPromise httpEntityPromise = new HttpEntityPromise();
            channelFuture.channel().attr(key).set(httpEntityPromise);


            channelFuture.channel().writeAndFlush(getConnect());

            HttpEntityPromise httpEntityPromise1 = httpEntityPromise.get();
            System.out.println("=============================");
            System.out.println(httpEntityPromise1.toString());
            System.out.println("******************************");
			
			// 因为目标网站是https所以我们在connect请求成功之后,要再加一层tls,使其能和目标网站tls握手
            channelFuture.channel().pipeline().addAfter("ssl", "target-ssl", sslContext.newHandler(channelFuture.channel().alloc()));
            
            HttpEntityPromise httpEntityPromise2 = new HttpEntityPromise();
            channelFuture.channel().attr(key).set(httpEntityPromise);
            channelFuture.channel().writeAndFlush(getGet()).addListener(f -> {
                System.out.println(f.isSuccess());
            });
            HttpEntityPromise httpEntityPromise3 = httpEntityPromise2.get();
            System.out.println("proxy=============================");
            System.out.println(httpEntityPromise3.toString());

        } finally{
            group.shutdownGracefully();
        }

    }

    private static FullHttpRequest getConnect() {
        FullHttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.CONNECT, "www.baidu.com:443", Unpooled.EMPTY_BUFFER, false);
        request.headers().set("host", "www.baidu.com:443");
        request.headers().set("proxy-authorization", proxy_auth); // 可去掉
        request.headers().set("my-header", "hello"); // 可去掉
        request.headers().set("User-Agent", "curl"); // 可去掉

        return request;
    }

    private static DefaultHttpRequest getGet() {
        DefaultHttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET,"/");
        request.headers().set("host", "www.baidu.com");

        return request;
    }


	/**
	* 业务处理handle
	*/
    static class HandleBiz extends SimpleChannelInboundHandler<HttpObject> {

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, HttpObject o) throws Exception {
            HttpEntityPromise httpEntityPromise = ctx.channel().attr(key).get();

            if (o instanceof HttpResponse) {
                HttpHeaders headers = ((HttpResponse) o).headers();
                System.out.println("响应头:"+ headers);
                httpEntityPromise.addHeader(headers);
                httpEntityPromise.setCode(((HttpResponse) o).status().code());
            }else if (o instanceof LastHttpContent){
                String s = ((HttpContent) o).content().toString(CharsetUtil.UTF_8);
                httpEntityPromise.addBody(s);
                httpEntityPromise.finish();
                System.out.println("最后的包:"+s);
            }else if (o instanceof HttpContent){
                String s = ((HttpContent) o).content().toString(CharsetUtil.UTF_8);
                httpEntityPromise.addBody(s);
                System.out.println("body内容:"+s);
            }else {
                System.out.println("未知类:"+o.getClass());
                System.out.println("toString:"+o);
            }
        }

        @Override
        public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
            super.channelReadComplete(ctx);
        }

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            ctx.channel().close();
            cause.printStackTrace();
        }
    }
}

三,其中遇到的一些坑

3.1 https目标网站无法访问
原因是我没有在请求建立连接之后再添加一个tls编解码器,导致传输给目标网站的是http报文,不是https报文,虽然我传输给proxy是https报文,但是经过代理后自动进行了tls解码,所以需要两层tls,第一层是客户端和代理设备的tls握手,.第二层是客户端和目标网站的tls握手.对应源代码中的channelFuture.channel().pipeline().addAfter(“ssl”, “target-ssl”, sslContext.newHandler(channelFuture.channel().alloc()));这一行
3.2 请求https目标网站能收到connect请求的响应但是后面的get请求响应就无法解析
这是因为netty的HttpClientCodec编解码器默认在处理完connect请求后就不处理了,这是为代理服务端做的准备,我们作为客户端是需要继续处理的,所以有个构造方式会有一个参数:parseHttpAfterConnectRequest,设置为true就可以了
3.3 请求https目标网站报tls握手失败

控制台报错如下

Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133)
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:358)
	at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293)

我遇到的报错是tls handshake fail,打开tls握手debug看到是server name not know什么的,原来有的网站开启了tls服务认证,需要域名一致.这个在netty中创建tls处理器的时候有两个参数,preeHost和preePort就是目标网站的主机和端口
sslContext.newHandler(channelFuture.channel().alloc(), host, port)