smiley-http-proxy-servlet做代理application/x-www-form-urlencoded请求类型，报错failed to respond

铁憨憨b

已于 2024-03-12 16:31:57 修改

阅读量1.1k

点赞数 19

文章标签： http servlet 网络协议

于 2024-03-12 16:30:04 首次发布

本文链接：https://blog.csdn.net/qq_38730338/article/details/136655615

版权

背景

关于springboot项目用来做类型nginx的反向代理，利用smiley-http-proxy-servlet既可以做出，

但是如果请求application/x-www-form-urlencoded 会报错failed to respond，对此我前后梳理原因并做出解决

一、smiley-http-proxy-servlet的使用

引入依赖

        <!--http 反向代理-->
        <dependency>
            <groupId>org.mitre.dsmiley.httpproxy</groupId>
            <artifactId>smiley-http-proxy-servlet</artifactId>
            <version>1.12.1</version>
        </dependency>

编写动态生成prpxyServlet,通过yml配置，动态生成

yml配置

proxy:
  url_mapping:
    {
      #请求前置代理地址
      "[/proxy/front/*]": "http://10.203.40.3:8080/yyds",
      ##请求饿了么地址的
      "[/proxy/elm/]": "https://ppe-api-be.ele.me/"

    }

获取配置信息

package cn.hsa.gyjg.config.httpproxy;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

import java.util.Map;

/**
 * <h3>net_agent</h3>
 * <p>代理地址映射配置</p>
 *
 * @author : AbstractBin
 * @date : 2022/3/10 0:11
 **/
@Configuration
@ConfigurationProperties(prefix = "proxy")
public class ProxyMappingConfig {
    //将yml中的proxy.url_mapping自动转为map
    private Map<String, String> urlMapping;

    public void setUrlMapping(Map<String, String> urlMapping) {
        this.urlMapping = urlMapping;
    }
    public Map<String, String> getUrlMapping() {
        return urlMapping;
    }
}

编写动态生成prpxyServlet,

package cn.hsa.gyjg.config.httpproxy;
import org.mitre.dsmiley.httpproxy.ProxyServlet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.servlet.ServletContextInitializer;
import org.springframework.stereotype.Component;

import javax.servlet.Servlet;
import javax.servlet.ServletContext;
import javax.servlet.ServletRegistration;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * <h3>net_agent</h3>
 * <p></p>
 *
 * @author : AbstractBin
 * @date : 2022/3/10 9:35
 **/
@Component
public class ServletContextInitListener implements ServletContextInitializer {
    Logger log = LoggerFactory.getLogger(ServletContextInitializer.class);
    @Autowired
    private ProxyMappingConfig proxyMappingConfig;

    //项目启动时，会读取yml配置里所有的路径映射，每个映射注册一个ProxyServlet
    @Override
    public void onStartup(ServletContext servletContext) {
        Map<String, String> urlMapping = proxyMappingConfig.getUrlMapping();
        if (null == urlMapping || urlMapping.size() < 1) {
            return;
        }
        //计数
        AtomicInteger count = new AtomicInteger(0);
        urlMapping.forEach((serverUrl, targetUrl) -> {

            log.info("【请求代理注册，serverUrl={}，targetUrl】", serverUrl, targetUrl);
            ProxyServlet servlet = new ProxyServlet();
            //每个servlet名字不能重复
            String servletName = "servlet" + count.get();
            ServletRegistration.Dynamic dynamic = servletContext.addServlet(servletName, servlet);
            Map<String, String> paramMap = new HashMap<>();
            paramMap.put(ProxyServlet.P_TARGET_URI, targetUrl);
            paramMap.put(ProxyServlet.P_LOG, "true");
            dynamic.setInitParameters(paramMap);
            dynamic.addMapping(serverUrl);
            count.incrementAndGet();

        });
    }
}

启动项目，访问对应yml配置接口即可代理到对应地址

二、smiley-http-proxy-servlet做代理时遇到的问题，报错failed to respond

源码分析

smiley-http-proxy-servlet的ProxyServlet就是一个servlet；
当前端请求代理地址时，首先经过各种FIlter，然后再调用该代理ProxyServlet的 service() 方法；
下面是service()方法重点部分

Object proxyRequest;
        if (servletRequest.getHeader("Content-Length") == null && servletRequest.getHeader("Transfer-Encoding") == null) {
            proxyRequest = new BasicHttpRequest(method, proxyRequestUri);
        } else {
            proxyRequest = this.newProxyRequestWithEntity(method, proxyRequestUri, servletRequest);
        }

当然最终走的是newProxyRequestWithEntity方法，下面看看该方法源码

protected HttpRequest newProxyRequestWithEntity(String method, String proxyRequestUri, HttpServletRequest servletRequest) throws IOException {
        HttpEntityEnclosingRequest eProxyRequest = new BasicHttpEntityEnclosingRequest(method, proxyRequestUri);
        eProxyRequest.setEntity(new InputStreamEntity(servletRequest.getInputStream(), this.getContentLength(servletRequest)));
        return eProxyRequest;
    }

这里看到了最终是通过getInputStream将请求的Stream流传给新的代理请求Request
再调试的时候getInputStream一直拿不到数据，所以请求代理的时候报错failed to respond

问题原因
由于application/x-www-form-urlencoded该类型，在项目中通过servlet时候项目已经将其流获取转成ParameterMap存储，所以获取不到流数据

request.getParameter()、request.getInputStream()和request.getReader()的区别
如果Post请求体是application/x- www-form-urlencoded，或是form表单提交。可以通过request.getParameter()方法来获取请求参数值。但当请求体不是该类型时，需要通过request.getInputStream()或request.getReader()方法来获取请求参数值。
当然请求体类型是application/x- www-form-urlencoded时也可以直接调用request.getInputStream()或request.getReader()方法来解析并获取请求参数，前提是还没调用request.getParameter()方法；
此时当request.getInputStream()或request.getReader()获取到请求数据后，无法再调request.getParameter()获取请求数据；
即对该类型的请求，三个方法互斥，只能调其中一个。

如何解决？

通过源码我们知道smiley-http-proxy-servlet代理底层是以httpcilent 作为转发请求工具，参考httpcilent 请求application/x- www-form-urlencoded需要如何入参可以知道

import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.util.ArrayList;
import java.util.List;

public class HttpClientExample {
    public static void main(String[] args) {
        // 1. 创建HttpClient对象
        HttpClient httpClient = new DefaultHttpClient();

        // 2. 创建HttpPost对象，并设置请求的URL
        HttpPost httpPost = new HttpPost("

        try {
            // 3. 创建参数列表，并进行参数编码
            List<NameValuePair> params = new ArrayList<>();
            params.add(new BasicNameValuePair("param1", "value1"));
            params.add(new BasicNameValuePair("param2", "value2"));
            UrlEncodedFormEntity entity = new UrlEncodedFormEntity(params, "UTF-8");

            // 4. 将参数设置到HttpPost对象的Entity中
            httpPost.setEntity(entity);

            // 5. 发送请求，并获取响应结果
            HttpResponse response = httpClient.execute(httpPost);
            String responseBody = EntityUtils.toString(response.getEntity());
            System.out.println(responseBody);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

所以自定义ProxyServlet，在对应参数转换里，加入改代码，判断请求类型，转换参数

if (MediaType.APPLICATION_FORM_URLENCODED_VALUE.equals(servletRequest.getContentType())
        || servletRequest.getContentType().contains(MediaType.APPLICATION_FORM_URLENCODED_VALUE)) {
    // 添加参数
    Map<String, String[]> mapdata = servletRequest.getParameterMap();
    List<NameValuePair> nameValuePairs = new ArrayList<>();
    if (mapdata.size() != 0) {
        // 将mapdata中的key存在set集合中，通过迭代器取出所有的key，再获取每一个键对应的值
        Set keySet = mapdata.keySet();
        Iterator it = keySet.iterator();
        while (it.hasNext()) {
            String k = it.next().toString();
            String[] v = mapdata.get(k);
            nameValuePairs.add(new BasicNameValuePair(k, v[0]));
        }
    }
    eProxyRequest.setEntity(new UrlEncodedFormEntity(nameValuePairs, servletRequest.getCharacterEncoding()));
} else {
    eProxyRequest.setEntity(new InputStreamEntity(servletRequest.getInputStream(), this.getContentLength(servletRequest)));
}

package cn.hsa.gyjg.config.httpproxy;

import lombok.extern.slf4j.Slf4j;
import org.apache.http.HttpEntityEnclosingRequest;
import org.apache.http.HttpRequest;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.entity.InputStreamEntity;
import org.apache.http.message.BasicHttpEntityEnclosingRequest;
import org.apache.http.message.BasicHttpRequest;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;
import org.mitre.dsmiley.httpproxy.ProxyServlet;
import org.springframework.http.MediaType;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.*;

@Slf4j
public class MyProxyServlet extends ProxyServlet {
    @Override
    public void service(HttpServletRequest servletRequest, HttpServletResponse servletResponse) throws ServletException, IOException {
        if (servletRequest.getAttribute(ATTR_TARGET_URI) == null) {
            servletRequest.setAttribute(ATTR_TARGET_URI, this.targetUri);
        }

        if (servletRequest.getAttribute(ATTR_TARGET_HOST) == null) {
            servletRequest.setAttribute(ATTR_TARGET_HOST, this.targetHost);
        }

        String method = servletRequest.getMethod();
        String proxyRequestUri = this.rewriteUrlFromRequest(servletRequest);
        Object proxyRequest;
        if (servletRequest.getHeader("Content-Length") == null && servletRequest.getHeader("Transfer-Encoding") == null) {
            proxyRequest = new BasicHttpRequest(method, proxyRequestUri);
        } else {
            proxyRequest = this.newProxyRequestWithEntity(method, proxyRequestUri, servletRequest);
        }

        this.copyRequestHeaders(servletRequest, (HttpRequest) proxyRequest);
        this.setXForwardedForHeader(servletRequest, (HttpRequest) proxyRequest);
        HttpResponse proxyResponse = null;

        try {
            if (this.doLog) {
                log.info("proxyRequestUri:"+ proxyRequestUri);
                log.info("Content-Type:"+ servletRequest.getContentType());
//                log.info("params:"+getRequestBodyByInputStream(((HttpEntityEnclosingRequest)proxyRequest).getEntity().getContent()));
            }
            proxyResponse = this.doExecute(servletRequest, servletResponse, (HttpRequest) proxyRequest);
            int statusCode = proxyResponse.getStatusLine().getStatusCode();
            servletResponse.setStatus(statusCode, proxyResponse.getStatusLine().getReasonPhrase());
            this.copyResponseHeaders(proxyResponse, servletRequest, servletResponse);
            if (statusCode == 304) {
                servletResponse.setIntHeader("Content-Length", 0);
            } else {
                this.copyResponseEntity(proxyResponse, servletResponse, (HttpRequest) proxyRequest, servletRequest);
            }
            if (this.doLog) {
                log.info("respStatusCode:"+ statusCode);
            }
        } catch (Exception var11) {
            this.handleRequestException((HttpRequest) proxyRequest, proxyResponse, var11);
        } finally {
            if (proxyResponse != null) {
                EntityUtils.consumeQuietly(proxyResponse.getEntity());
            }

        }

    }

    /**
     * 获取http中的请求报文
     *
     * @return
     */
    private String getRequestBodyByInputStream(InputStream inputStream) {
        try {
            // 获取ServletInputStream
            BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
            StringBuilder requestBody = new StringBuilder();
            char[] buffer = new char[1024];
            int read;
            while ((read = reader.read(buffer)) != -1) {
                requestBody.append(buffer, 0, read);
            }
            // 现在requestBody包含了请求体的字符串内容
            return requestBody.toString();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return "";
    }

    @Override
    protected HttpRequest newProxyRequestWithEntity(String method, String proxyRequestUri, HttpServletRequest servletRequest) throws IOException {
        HttpEntityEnclosingRequest eProxyRequest = new BasicHttpEntityEnclosingRequest(method, proxyRequestUri);
        if (MediaType.APPLICATION_FORM_URLENCODED_VALUE.equals(servletRequest.getContentType())
                || servletRequest.getContentType().contains(MediaType.APPLICATION_FORM_URLENCODED_VALUE)) {
            // 添加参数
            Map<String, String[]> mapdata = servletRequest.getParameterMap();
            List<NameValuePair> nameValuePairs = new ArrayList<>();
            if (mapdata.size() != 0) {
                // 将mapdata中的key存在set集合中，通过迭代器取出所有的key，再获取每一个键对应的值
                Set keySet = mapdata.keySet();
                Iterator it = keySet.iterator();
                while (it.hasNext()) {
                    String k = it.next().toString();
                    String[] v = mapdata.get(k);
                    nameValuePairs.add(new BasicNameValuePair(k, v[0]));
                }
            }
            eProxyRequest.setEntity(new UrlEncodedFormEntity(nameValuePairs, servletRequest.getCharacterEncoding()));
        } else {
            eProxyRequest.setEntity(new InputStreamEntity(servletRequest.getInputStream(), this.getContentLength(servletRequest)));
        }
        return eProxyRequest;
    }

    private long getContentLength(HttpServletRequest request) {
        String contentLengthHeader = request.getHeader("Content-Length");
        return contentLengthHeader != null ? Long.parseLong(contentLengthHeader) : -1L;
    }


    private void setXForwardedForHeader(HttpServletRequest servletRequest, HttpRequest proxyRequest) {
        if (this.doForwardIP) {
            String forHeaderName = "X-Forwarded-For";
            String forHeader = servletRequest.getRemoteAddr();
            String existingForHeader = servletRequest.getHeader(forHeaderName);
            if (existingForHeader != null) {
                forHeader = existingForHeader + ", " + forHeader;
            }

            proxyRequest.setHeader(forHeaderName, forHeader);
            String protoHeaderName = "X-Forwarded-Proto";
            String protoHeader = servletRequest.getScheme();
            proxyRequest.setHeader(protoHeaderName, protoHeader);
        }

    }
}

最后在动态生成代理时候转成自己的代理类

package cn.hsa.gyjg.config.httpproxy;
import org.mitre.dsmiley.httpproxy.ProxyServlet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.servlet.ServletContextInitializer;
import org.springframework.stereotype.Component;

import javax.servlet.Servlet;
import javax.servlet.ServletContext;
import javax.servlet.ServletRegistration;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

/**
 * <h3>net_agent</h3>
 * <p></p>
 *
 * @author : AbstractBin
 * @date : 2022/3/10 9:35
 **/
@Component
public class ServletContextInitListener implements ServletContextInitializer {
    Logger log = LoggerFactory.getLogger(ServletContextInitializer.class);
    @Autowired
    private ProxyMappingConfig proxyMappingConfig;

    //项目启动时，会读取yml配置里所有的路径映射，每个映射注册一个ProxyServlet
    @Override
    public void onStartup(ServletContext servletContext) {
        Map<String, String> urlMapping = proxyMappingConfig.getUrlMapping();
        if (null == urlMapping || urlMapping.size() < 1) {
            return;
        }
        //计数
        AtomicInteger count = new AtomicInteger(0);
        urlMapping.forEach((serverUrl, targetUrl) -> {

            log.info("【请求代理注册，serverUrl={}，targetUrl】", serverUrl, targetUrl);
            ProxyServlet servlet = new MyProxyServlet();
            //每个servlet名字不能重复
            String servletName = "servlet" + count.get();
            ServletRegistration.Dynamic dynamic = servletContext.addServlet(servletName, servlet);
            Map<String, String> paramMap = new HashMap<>();
            paramMap.put(ProxyServlet.P_TARGET_URI, targetUrl);
            paramMap.put(ProxyServlet.P_LOG, "true");
            dynamic.setInitParameters(paramMap);
            dynamic.addMapping(serverUrl);
            count.incrementAndGet();

        });
    }
}

铁憨憨b

关注

19
点赞
踩
20

收藏

觉得还不错? 一键收藏
0
评论
smiley-http-proxy-servlet做代理application/x-www-form-urlencoded请求类型，报错failed to respond

smiley-http-proxy-servlet做代理application/x-www-form-urlencoded请求类型，报错failed to respond
复制链接

扫一扫