一,HTTP概述
HTTP协议是无状态的协议,即每一次请求都是互相独立的。因此它的最初实现是,每一个http请求都会打开一个tcp socket连接,当交互完毕后会关闭这个连接。
HTTP协议是全双工的协议,所以建立连接与断开连接是要经过三次握手与四次挥手的。显然在这种设计中,每次发送Http请求都会消耗很多的额外资源,即连接的建立与销毁。
于是,HTTP协议的也进行了发展,通过持久连接的方法来进行socket连接复用。
HTTP/1.0+的Keep-Alive
使用HTTP/1.0的客户端在首部中加上"Connection:Keep-Alive",请求服务端将一条连接保持在打开状态。服务端如果愿意将这条连接保持在打开状态,就会在响应中包含同样的首部。如果响应中没有包含"Connection:Keep-Alive"首部,则客户端会认为服务端不支持keep-alive,会在发送完响应报文之后关闭掉当前连接。
通过keep-alive补充协议,客户端与服务器之间完成了持久连接,然而仍然存在着一些问题:
- 在HTTP/1.0中keep-alive不是标准协议,客户端必须发送Connection:Keep-Alive来激活keep-alive连接。
- 代理服务器可能无法支持keep-alive,因为一些代理是"盲中继",无法理解首部的含义,只是将首部逐跳转发。所以可能造成客户端与服务端都保持了连接,但是代理不接受该连接上的数据。
HTTP/1.1的持久连接
HTTP/1.1采取持久连接的方式替代了Keep-Alive。
HTTP/1.1的连接默认情况下都是持久连接。如果要显式关闭,需要在报文中加上Connection:Close首部。即在HTTP/1.1中,所有的连接都进行了复用。
然而如同Keep-Alive一样,空闲的持久连接也可以随时被客户端与服务端关闭。不发送Connection:Close不意味着服务器承诺连接永远保持打开。
二,HttpURLConnection简介
在JDK的java.net包中已经提供了访问HTTP协议的基本功能的类:HttpURLConnection。
HttpURLConnection是Java的标准类,它继承自URLConnection,可用于向指定网站发送GET请求、POST请求。它在URLConnection的基础上提供了如下便捷的方法:
int getResponseCode(); // 获取服务器的响应代码。
String getResponseMessage(); // 获取服务器的响应消息。
String getResponseMethod(); // 获取发送请求的方法。
void setRequestMethod(String method); // 设置发送请求的方法。
- HttpURLConnection对象不能直接构造,需要通过URL类中的openConnection()方法来获得。
- HttpURLConnection的connect()函数,实际上只是建立了一个与服务器的TCP连接,并没有实际发送HTTP请求。HTTP请求实际上直到我们获取服务器响应数据(如调用getInputStream()、getResponseCode()等方法)时才正式发送出去。
- 对HttpURLConnection对象的配置都需要在connect()方法执行之前完成。
- HttpURLConnection是基于HTTP协议的,其底层通过socket通信实现。如果不设置超时(timeout),在网络异常的情况下,可能会导致程序僵死而不继续往下执行。
- HTTP正文的内容是通过OutputStream流写入的, 向流中写入的数据不会立即发送到网络,而是存在于内存缓冲区中,待流关闭时,根据写入的内容生成HTTP正文。
- 调用getInputStream()方法时,返回一个输入流,用于从中读取服务器对于HTTP请求的返回信息。
- 我们可以使用HttpURLConnection.connect()方法手动的发送一个HTTP请求,但是如果要获取HTTP响应的时候,请求就会自动的发起,比如我们使用HttpURLConnection.getInputStream()方法的时候,所以完全没有必要调用connect()方法。
HttpURLConnection工具类封装:
package com.example.demohttp.httpurlconnection;
import org.apache.commons.lang.StringUtils;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.util.HashMap;
import java.util.Map;
/**
* @Date: 2021/12/1 4:39 下午
*/
public class HttpUrlConnectionUtils {
private static int DEFAULT_READTIME = 30000;
private static int DEFAULT_CONNECTTIME = 30000;
private static String READTIME_KEY = "readTime";
private static String CONNECTTIME_KEY = "connectTime";
/**
* Send a get request
* @param url
* @return response
* @throws IOException
*/
static public String get(String url) throws IOException {
return get(url, null);
}
/**
* Send a get request
* @param url Url as string
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String get(String url,
Map<String, String> headers) throws IOException {
return fetch("GET", url, null, headers);
}
/**
* Send a post request
* @param url Url as string
* @param body Request body as string
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String post(String url, String body,
Map<String, String> headers) throws IOException {
return fetch("POST", url, body, headers);
}
/**
* Send a post request
* @param url Url as string
* @param body Request body as string
* @return response Response as string
* @throws IOException
*/
static public String post(String url, String body) throws IOException {
return post(url, body, null);
}
/**
* Post a form with parameters
* @param url Url as string
* @param params map with parameters/values
* @return response Response as string
* @throws IOException
*/
static public String postForm(String url, Map<String, String> params)
throws IOException {
return postForm(url, params, null);
}
/**
* Post a form with parameters
* @param url Url as string
* @param params Map with parameters/values
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String postForm(String url, Map<String, String> params,
Map<String, String> headers) throws IOException {
// set content type
if (headers == null) {
headers = new HashMap<String, String>();
}
if (StringUtils.isBlank(headers.get("Content-Type"))){
headers.put("Content-Type", "application/x-www-form-urlencoded");
}
// parse parameters
String body = "";
if (params != null) {
boolean first = true;
for (String param : params.keySet()) {
if (first) {
first = false;
} else {
body += "&";
}
String value = params.get(param);
body += URLEncoder.encode(param, "UTF-8") + "=";
body += URLEncoder.encode(value, "UTF-8");
}
}
return post(url, body, headers);
}
/**
* Send a put request
* @param url Url as string
* @param body Request body as string
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String put(String url, String body,
Map<String, String> headers) throws IOException {
return fetch("PUT", url, body, headers);
}
/**
* Send a put request
* @param url Url as string
* @return response Response as string
* @throws IOException
*/
static public String put(String url, String body) throws IOException {
return put(url, body, null);
}
/**
* Send a delete request
* @param url Url as string
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String delete(String url,
Map<String, String> headers) throws IOException {
return fetch("DELETE", url, null, headers);
}
/**
* Send a delete request
* @param url Url as string
* @return response Response as string
* @throws IOException
*/
static public String delete(String url) throws IOException {
return delete(url, null);
}
/**
* Append query parameters to given url
* @param url Url as string
* @param params Map with query parameters
* @return url Url with query parameters appended
* @throws IOException
*/
static public String appendQueryParams(String url,
Map<String, String> params) throws IOException {
String fullUrl = url;
if (params != null) {
boolean first = (fullUrl.indexOf('?') == -1);
for (String param : params.keySet()) {
if (first) {
fullUrl += '?';
first = false;
} else {
fullUrl += '&';
}
String value = params.get(param);
fullUrl += URLEncoder.encode(param, "UTF-8") + '=';
fullUrl += URLEncoder.encode(value, "UTF-8");
}
}
return fullUrl;
}
/**
* Retrieve the query parameters from given url
* @param url Url containing query parameters
* @return params Map with query parameters
* @throws IOException
*/
static public Map<String, String> getQueryParams(String url)
throws IOException {
Map<String, String> params = new HashMap<String, String>();
int start = url.indexOf('?');
while (start != -1) {
// read parameter name
int equals = url.indexOf('=', start);
String param = "";
if (equals != -1) {
param = url.substring(start + 1, equals);
} else {
param = url.substring(start + 1);
}
// read parameter value
String value = "";
if (equals != -1) {
start = url.indexOf('&', equals);
if (start != -1) {
value = url.substring(equals + 1, start);
} else {
value = url.substring(equals + 1);
}
}
params.put(URLDecoder.decode(param, "UTF-8"),
URLDecoder.decode(value, "UTF-8"));
}
return params;
}
/**
* Returns the url without query parameters
* @param url Url containing query parameters
* @return url Url without query parameters
* @throws IOException
*/
static public String removeQueryParams(String url)
throws IOException {
int q = url.indexOf('?');
if (q != -1) {
return url.substring(0, q);
} else {
return url;
}
}
/**
* Send a request
* @param method HTTP method, for example "GET" or "POST"
* @param url Url as string
* @param body Request body as string
* @param headers Optional map with headers
* @return response Response as string
* @throws IOException
*/
static public String fetch(String method, String url, String body,
Map<String, String> headers) throws IOException {
// connection
URL u = new URL(url);
HttpURLConnection conn = (HttpURLConnection) u.openConnection();
conn.setConnectTimeout(DEFAULT_CONNECTTIME);
conn.setReadTimeout(DEFAULT_READTIME);
// method
if (method != null) {
conn.setRequestMethod(method);
}
// headers
if (headers != null) {
for (String key : headers.keySet()) {
if (READTIME_KEY.equals(key)) {
conn.setReadTimeout(Integer.parseInt(headers.get(key)));
continue;
}
if (CONNECTTIME_KEY.equals(key)) {
conn.setConnectTimeout(Integer.parseInt(headers.get(key)));
continue;
}
conn.addRequestProperty(key, headers.get(key));
}
}
// body
if (body != null) {
conn.setDoOutput(true);
OutputStream os = conn.getOutputStream();
os.write(body.getBytes());
os.flush();
os.close();
}
// response
InputStream is = null;
if (conn.getResponseCode() >= 400) {
is = conn.getErrorStream();
} else {
is = conn.getInputStream();
}
String response = streamToString(is);
is.close();
// handle redirects
if (conn.getResponseCode() == 301) {
String location = conn.getHeaderField("Location");
return fetch(method, location, body, headers);
}
return response;
}
/**
* Read an input stream into a string
* @param in
* @return
* @throws IOException
*/
static public String streamToString(InputStream in) throws IOException {
StringBuffer out = new StringBuffer();
byte[] b = new byte[4096];
for (int n; (n = in.read(b)) != -1; ) {
out.append(new String(b, 0, n));
}
return out.toString();
}
}
三,HttpClient简介
HttpClient是Apache开源组织提供的它是一个简单的HTTP客户端,不是浏览器,用于发送HTTP请求,接收HTTP响应。但不会缓存服务器的响应。它只是关注于如何发送请求、接收响应,以及管理HTTP连接。
HttpClient相比较来说简化了HttpURLConnection对象对Session、Cookie的处理。
可以说HttpClient就是一个增强版的HttpURLConnection,HttpClient可以做HttpURLConnection对象所有能做的事。
HttpClien中使用了连接池来管理持有连接,同一条TCP链路上,连接是可以复用的。HttpClient通过连接池的方式进行连接持久化。
HttpClient链接池的实现:
- 当有连接第一次使用的时候建立连接。
- 结束时对应连接不关闭,归还到池中。
- 下次同个目的的连接可从池中获取一个可用连接。
- 定期清理过期连接。
HttpClient连接池工具类封装
导入
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.13</version>
</dependency>
HttpClient工具类:
public class HttpClientUtil {
private static Logger logger = LoggerFactory.getLogger(HttpClientUtil.class);
private volatile static CloseableHttpClient httpClient;
private HttpClientUtil(){}
private static CloseableHttpClient getHttpClient() {
if (httpClient == null) {
synchronized (HttpClientUtil.class) {
if (httpClient == null) {
httpClient = HttpPoolConManager.getClient();
}
}
}
return httpClient;
}
/**
* 自定义配置方式初使化httpclient
* 此方法在一个应用系统里面,只需要执行一次。
* 应用程序自行控制。
* @param config
*/
public static void config(HttpClientConfig config) {
if (httpClient != null) {
logger.warn("http client already init ! ");
}
httpClient = HttpPoolConManager.getClient(config);
}
/**
* 发送get请求
* @param url 请求地址
* @return 返回请求结果
* @throws Exception
*/
public static String get(String url) throws Exception {
return get(url, Collections.EMPTY_MAP);
}
/**
* 发送get请求
* @param url 请求地址
* @param params 请求参数
* @return
* @throws Exception
*/
public static String get(String url, Map<String, String> params) throws Exception {
return get(url, Collections.EMPTY_MAP, params);
}
/**
* 发送get请求
* @param url 请求地址
* @param headers 请求头
* @param params 请求参数
* @return
* @throws IOException
*/
public static String get(String url,Map<String, String> headers,Map<String, String> params ) throws IOException{
// *) 构建GET请求头
String apiUrl = appendQueryParams(url, params);
HttpGet httpGet = new HttpGet(apiUrl);
return response(httpGet,headers);
}
/**
* 发送post请求
* @param apiUrl 请求地址
* @return 请求结果
*/
public static String post(String apiUrl){
return post(apiUrl,Collections.EMPTY_MAP);
}
/**
* 发送post请求
* @param apiUrl 请求地址
* @param params 请求参数
* @return 请求结果
*/
public static String post(String apiUrl, Map<String, String> params){
return post(apiUrl,Collections.EMPTY_MAP,params);
}
/**
* 发送post请求
* @param apiUrl 请求地址
* @param headers header
* @param params 请求参数
* @return 返回参数
* @throws IOException
*/
public static String post(String apiUrl, Map<String, String> headers, Map<String, String> params) {
HttpPost httpPost = new HttpPost(apiUrl);
// *) 配置请求参数
if ( params != null && params.size() > 0 ) {
HttpEntity entityReq = getUrlEncodedFormEntity(params);
httpPost.setEntity(entityReq);
}
return response(httpPost,headers);
}
/**
* 发送post请求
* @param apiUrl 请求地址
* @param params 请求参数
* @return 请求结果
*/
public static String postJson(String apiUrl, String params) throws IOException {
return postJson(apiUrl,Collections.EMPTY_MAP,params);
}
/**
* 发送post json 请求
* @param apiUrl 请求地址
* @param headers 请求头
* @param body 请求json内容
* @return 返回请求结果
* @throws IOException
*/
public static String postJson(String apiUrl, Map<String, String> headers, String body) throws IOException {
HttpPost httpPost = new HttpPost(apiUrl);
httpPost.addHeader("Content-Type", "application/json");
// *) 配置请求参数
if ( body != null && !"".equals(body) ) {
HttpEntity entityReq = new StringEntity(body,"UTF-8");
httpPost.setEntity(entityReq);
}
return response(httpPost,headers);
}
/**
* 将参数转为httpEntity实体。
* @param params 请求参数
* @return
*/
private static HttpEntity getUrlEncodedFormEntity(Map<String, String> params) {
List<NameValuePair> pairList = new ArrayList<>(params.size());
for (Map.Entry<String, String> entry : params.entrySet()) {
NameValuePair pair = new BasicNameValuePair(entry.getKey(), entry
.getValue());
pairList.add(pair);
}
return new UrlEncodedFormEntity(pairList, Charset.forName("UTF-8"));
}
/**
* @param request 请求方式
* @param headers 请求头
* @return 返回请求结果
*/
private static String response(HttpRequestBase request, Map<String, String> headers){
// *) 设置header信息
if ( headers != null && headers.size() > 0 ) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
request.addHeader(entry.getKey(), entry.getValue());
}
}
CloseableHttpResponse response = null;
int statusCode = -1;
try {
response = getHttpClient().execute(request);
if (response == null || response.getStatusLine() == null) {
return null;
}
statusCode = response.getStatusLine().getStatusCode();
if ( statusCode == HttpStatus.SC_OK ) {
HttpEntity entityRes = response.getEntity();
if (entityRes != null) {
return EntityUtils.toString(entityRes, "UTF-8");
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if ( response != null ) {
try {
response.close();
} catch (IOException e) {
//ignore
}
}
}
BaseResponse responseObject = new BaseResponse(BaseResponse.RespCode.ERROR);
return JSONObject.toJSONString(responseObject);
}
/**
* Append query parameters to given url
* @param url Url as string
* @param params Map with query parameters
* @return url Url with query parameters appended
* @throws IOException
*/
static public String appendQueryParams(String url,
Map<String, String> params) throws IOException {
String fullUrl = url;
if (params != null) {
boolean first = (fullUrl.indexOf('?') == -1);
for (String param : params.keySet()) {
if (first) {
fullUrl += '?';
first = false;
} else {
fullUrl += '&';
}
String value = params.get(param);
fullUrl += URLEncoder.encode(param, "UTF-8") + '=';
fullUrl += URLEncoder.encode(value, "UTF-8");
}
}
return fullUrl;
}
}
连接池管理类:
public class HttpPoolConManager {
/**默认创建连接超时时间*/
private final static int DEFAULT_CONNECTIONTIMEOUT = 3000;
/**默认获取连接超时时间*/
private final static int DEFAULT_CONNECTIONREQUESTTIMEOUT = 3000;
/**默认数据传输超时时间*/
private final static int DEFAULT_READIMEOUT = 3000;
/**默认最大连接数*/
private final static int DEFAULT_MAX = 400;
/**每个路由最大并发数*/
private final static int DEFAULT_MAX_PERROUTE= 400;
/**
* 配置PoolingHttpClientConnectionManager
* 例如默认每路由最高50并发,具体依据业务来定
* @return
*/
private static PoolingHttpClientConnectionManager poolingHttpClientConnectionManager(HttpClientConfig config) {
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(config.getMaxConnect()>0?config.getMaxConnect():DEFAULT_MAX);
connectionManager.setDefaultMaxPerRoute(config.getMaxPerRoute()>0?config.getMaxPerRoute():DEFAULT_MAX_PERROUTE);
if(config.getValidateAfterInactivity()>0){
connectionManager.setValidateAfterInactivity(config.getValidateAfterInactivity());
}
connectionManager.closeIdleConnections(
config.getConnectionRequestTimeout()>0?config.getConnectionRequestTimeout() * 2 :
DEFAULT_CONNECTIONREQUESTTIMEOUT * 2, TimeUnit.MILLISECONDS);
return connectionManager;
}
/**
* 配置RequestConfig
* @return
*/
private static RequestConfig requestConfig(HttpClientConfig config) {
return RequestConfig.custom()
.setConnectionRequestTimeout(config.getConnectionRequestTimeout()>0?config.getConnectionRequestTimeout():DEFAULT_CONNECTIONREQUESTTIMEOUT)
.setConnectTimeout(config.getConnectionTimeout()>0?config.getConnectionTimeout():DEFAULT_CONNECTIONTIMEOUT)
.setSocketTimeout(config.getSocketTimeout()>0?config.getSocketTimeout():DEFAULT_READIMEOUT)
.setRedirectsEnabled(false)
.build();
}
public static CloseableHttpClient getClient(HttpClientConfig config) {
return HttpClients.custom()
.setConnectionManager(poolingHttpClientConnectionManager(config))
.setKeepAliveStrategy(new MyConnKeepAliveStrategy())
.setDefaultRequestConfig(requestConfig(config))
.build();
}
public static CloseableHttpClient getClient() {
HttpClientConfig config = HttpClientConfig.newBuilder().build();
return HttpClients.custom()
.setConnectionManager(poolingHttpClientConnectionManager(config))
.setKeepAliveStrategy(new MyConnKeepAliveStrategy())
.setDefaultRequestConfig(requestConfig(config))
.build();
}
}
HttpClient配置类
public class HttpClientConfig {
/**
* 连接池最大的连接数
*/
private int maxConnect;
/**
* 连接每个域名的最大连接数
*/
private int maxPerRoute;
/**
* 获取从连接池获取连接的最长时间,单位是毫秒;
*/
private int connectionRequestTimeout;
/**
* 获取创建连接的最长时间,单位是毫秒;
*/
private int connectionTimeout;
/**
* 获取数据传输的最长时间,单位是毫秒
*/
private int socketTimeout;
/**
* 用空闲连接过期时间,重用空闲连接时会先检查是否空闲时间超过这个时间,如果超过,释放socket重新建
*/
private int validateAfterInactivity;
/**
* 是否保持长连接
*/
private boolean keepAlive;
public int getValidateAfterInactivity() {
return validateAfterInactivity;
}
public void setValidateAfterInactivity(int validateAfterInactivity) {
this.validateAfterInactivity = validateAfterInactivity;
}
private HttpClientConfig(Builder builder) {
setMaxConnect(builder.maxConnect);
setMaxPerRoute(builder.maxPerRoute);
setConnectionRequestTimeout(builder.connectionRequestTimeout);
setConnectionTimeout(builder.connectionTimeout);
setSocketTimeout(builder.socketTimeout);
setValidateAfterInactivity(builder.validateAfterInactivity);
setKeepAlive(builder.keepAlive);
}
public static Builder newBuilder() {
return new Builder();
}
public int getMaxConnect() {
return maxConnect;
}
public void setMaxConnect(int maxConnect) {
this.maxConnect = maxConnect;
}
public int getMaxPerRoute() {
return maxPerRoute;
}
public void setMaxPerRoute(int maxPerRoute) {
this.maxPerRoute = maxPerRoute;
}
public int getConnectionRequestTimeout() {
return connectionRequestTimeout;
}
public void setConnectionRequestTimeout(int connectionRequestTimeout) {
this.connectionRequestTimeout = connectionRequestTimeout;
}
public int getConnectionTimeout() {
return connectionTimeout;
}
public void setConnectionTimeout(int connectionTimeout) {
this.connectionTimeout = connectionTimeout;
}
public int getSocketTimeout() {
return socketTimeout;
}
public void setSocketTimeout(int socketTimeout) {
this.socketTimeout = socketTimeout;
}
public boolean isKeepAlive() {
return keepAlive;
}
public void setKeepAlive(boolean keepAlive) {
this.keepAlive = keepAlive;
}
public static final class Builder {
private int maxConnect;
private int maxPerRoute;
private int connectionRequestTimeout;
private int connectionTimeout;
private int socketTimeout;
private int validateAfterInactivity;
private boolean keepAlive;
private Builder() {
}
/**
* 连接池最大的连接数
*/
public Builder maxConnect(int maxConnect) {
this.maxConnect = maxConnect;
return this;
}
/**
* 连接每个域名的最大连接数
*/
public Builder maxPerRoute(int maxPerRoute) {
this.maxPerRoute = maxPerRoute;
return this;
}
/**
* 获取从连接池获取连接的最长时间,单位是毫秒;
*/
public Builder connectionRequestTimeout(int connectionRequestTimeout) {
this.connectionRequestTimeout = connectionRequestTimeout;
return this;
}
/**
* 获取创建连接的最长时间,单位是毫秒;
*/
public Builder connectionTimeout(int connectionTimeout) {
this.connectionTimeout = connectionTimeout;
return this;
}
/**
* 获取数据传输的最长时间,单位是毫秒
*/
public Builder socketTimeout(int socketTimeout) {
this.socketTimeout = socketTimeout;
return this;
}
/**
* 用空闲连接过期时间,重用空闲连接时会先检查是否空闲时间超过这个时间,如果超过,释放socket重新建
*/
public Builder validateAfterInactivity(int validateAfterInactivity) {
this.validateAfterInactivity = validateAfterInactivity;
return this;
}
/**
* 是否保持长连接
*/
public Builder keepAlive(boolean keepAlive) {
this.keepAlive = keepAlive;
return this;
}
public HttpClientConfig build() {
return new HttpClientConfig(this);
}
}
}
Keep-Alive策略类
public class MyConnKeepAliveStrategy implements ConnectionKeepAliveStrategy {
@Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
HeaderIterator headerIterator = response.headerIterator(HTTP.CONN_KEEP_ALIVE);
HeaderElementIterator iterator = new BasicHeaderElementIterator(headerIterator);
while (iterator.hasNext()) {
HeaderElement element = iterator.nextElement();
String name = element.getName();
String value = element.getValue();
if (name != null && name.equalsIgnoreCase("timeout")) {
return Long.parseLong(value) * 1000;
}
}
System.out.println("没有保持");
// 如果没有约定,设置默认时长为60
return 30 * 1000;
}
}
响应封装类:
public class BaseResponse {
private int code;
private String message;
private Object object;
public BaseResponse(RespCode respCode) {
this.code = respCode.getCode();
this.message = respCode.getMessage();
}
public BaseResponse(RespCode respCode, Object data) {
this(respCode);
this.object = data;
}
public BaseResponse(int code, String message, Object object) {
super();
this.code = code;
this.message = message;
this.object = object;
}
public enum RespCode {
/**/
SUCCESS(0, "请求成功"), ERROR(-1, "网络异常,请稍后重试...");
private int code;
private String message;
private RespCode(int code, String message) {
this.code = code;
this.message = message;
}
public int getCode() {
return code;
}
public String getMessage() {
return message;
}
}
public int getCode() {
return code;
}
public void setCode(int code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
public Object getObject() {
return object;
}
public void setObject(Object object) {
this.object = object;
}
}
四,OKHttp简介
OKHttp与HttpClient类似,也是一个Http客户端,提供了对 HTTP/2 和 SPDY 的支持,并提供了连接池,GZIP 压缩和 HTTP 响应缓存功能。
官网地址:http://square.github.io/okhttp/
工具类封装:
导入依赖:
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.9.2</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>21.0</version>
</dependency>
OKHttpUtil:
package com.example.demohttp.okhttp;
import com.google.common.net.HttpHeaders;
import okhttp3.*;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.util.*;
import java.util.concurrent.TimeUnit;
/**
* @Date: 2021/12/1 4:39 下午
*/
public final class OKHttpClientUtil {
private static final String DEFAULT_USER_AGENT = "httpclient";
private static final Logger LOGGER = LoggerFactory.getLogger(OKHttpClientUtil.class);
private static final int fastTimeout = 1000;
private static final int midTimeout = 3000;
private static final int maxRequests = 200;
private static final int maxRequestPerHost = 200;
// for local http calls
private static final OkHttpClient fastHttpClient;
// for normal outside http/https calls
private static final OkHttpClient midHttpClient;
// for overseas http/https calls
private static Map<String, String> defaultHeader = null;
static {
OkHttpClient.Builder fastBuilder = new OkHttpClient.Builder();
OkHttpClient.Builder midBuilder = new OkHttpClient.Builder();
OkHttpClient.Builder slowBuilder = new OkHttpClient.Builder();
OkHttpClient.Builder slowestBuilder = new OkHttpClient.Builder();
fastBuilder.connectTimeout(fastTimeout, TimeUnit.MILLISECONDS)
.readTimeout(fastTimeout, TimeUnit.MILLISECONDS)
.writeTimeout(fastTimeout, TimeUnit.MILLISECONDS);
fastHttpClient = fastBuilder.build();
fastHttpClient.dispatcher().setMaxRequests(maxRequests);
fastHttpClient.dispatcher().setMaxRequestsPerHost(maxRequestPerHost);
midBuilder.connectTimeout(midTimeout, TimeUnit.MILLISECONDS)
.readTimeout(midTimeout, TimeUnit.MILLISECONDS)
.writeTimeout(midTimeout, TimeUnit.MILLISECONDS);
midHttpClient = midBuilder.build();
midHttpClient.dispatcher().setMaxRequests(maxRequests);
midHttpClient.dispatcher().setMaxRequestsPerHost(maxRequestPerHost);
Map<String, String> tmpDefaultHeaderMap = new HashMap<>();
tmpDefaultHeaderMap.put(HttpHeaders.USER_AGENT, DEFAULT_USER_AGENT);
defaultHeader = Collections.unmodifiableMap(tmpDefaultHeaderMap);
}
public static int getStatus(String url) {
return getStatus(url, null, 0);
}
public static int getStatus(String url, int timeoutMills) {
return getStatus(url, null, timeoutMills);
}
public static int getStatus(String url, Map<String, String> headers, int timeoutMills) {
SimpleHttpResponse response = get(getURL(url), headers, timeoutMills);
return response.getStatusCode();
}
public static Optional<String> getContent(String url) {
return getContent(url, null, 0);
}
public static Optional<String> getContent(String url, int timeoutMills) {
return getContent(url, null, timeoutMills);
}
public static Optional<String> getContent(String url, Map<String, String> headers, int timeoutMills) {
return Optional.ofNullable(get(getURL(url), headers, timeoutMills).getBody());
}
public static SimpleHttpResponse get(String url) {
return get(getURL(url), null, 0);
}
public static SimpleHttpResponse get(String url, int timeoutMills) {
return get(getURL(url), null, timeoutMills);
}
public static int postStatus(String url, Map<String, String> parameterMap) {
return post(getURL(url), null, parameterMap, 0).getStatusCode();
}
public static int postStatus(String url, Map<String, String> parameterMap, int timeoutMills) {
return post(getURL(url), null, parameterMap, timeoutMills).getStatusCode();
}
public static int postStatus(String url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
return post(getURL(url), headers, parameterMap, timeoutMills).getStatusCode();
}
public static Optional<String> postResult(String url, Map<String, String> parameterMap) {
return postResult(url, null, parameterMap, 0);
}
public static Optional<String> postResult(String url, Map<String, String> parameterMap, int timeoutMills) {
return postResult(url, null, parameterMap, timeoutMills);
}
public static Optional<String> postResult(String url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
return Optional.ofNullable(post(getURL(url), headers, parameterMap, timeoutMills).getBody());
}
/**
* 暴露原始的resp,便于获取文件stream等,需要自行关闭response,并捕捉IOException
*
* @param reqUrl
* @param headers
* @param timeoutMills
* @return
* @throws IOException
*/
public static Response getRawResp(String reqUrl, Map<String, String> headers, int timeoutMills) throws IOException {
URL url = getURL(reqUrl);
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
Headers h = Headers.of(finalHeaders);
Request req = new Request.Builder().url(url).headers(h).build();
long start = Instant.now().toEpochMilli();
boolean succ = false;
long size = 0;
try {
Response resp = client.newCall(req).execute();
succ = resp.isSuccessful();
return resp;
} finally {
long cost = Instant.now().toEpochMilli() - start;
LOGGER.info("http call success[{}] url[{}] method[{}] costMills[{}] size[{}] ", succ, req.url().url().toString(), req.method(), cost, size);
}
}
/**
* get url with specified headers
*
* @param url the specified url
* @param headers the request http headers
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
public static SimpleHttpResponse get(String url, Map<String, String> headers, int timeoutMills) {
return get(getURL(url), headers, timeoutMills);
}
/**
* post parameterMap as "application/x-www-form-urlencoded"
*
* @param url the specified url
* @param headers http request headers
* @param parameterMap http post parameters
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
public static SimpleHttpResponse post(String url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
return post(getURL(url), headers, parameterMap, timeoutMills);
}
public static SimpleHttpResponse put(String url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
return put(getURL(url), headers, parameterMap, timeoutMills);
}
/**
* post body as raw data
*
* @param url the specified url
* @param headers http request headers
* @param body http post body raw data ,the default body type is json or specified in headers Content-type field
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
public static SimpleHttpResponse postRawBody(String url, Map<String, String> headers, String body, int timeoutMills) {
return postRawBody(getURL(url), headers, body, timeoutMills);
}
public static SimpleHttpResponse putRawBody(String url, Map<String, String> headers, String body, int timeoutMills) {
return putRawBody(getURL(url), headers, body, timeoutMills);
}
public static SimpleHttpResponse deleteRawBody(String url, Map<String, String> headers, String body, int timeoutMills) {
return deleteRawBody(getURL(url), headers, body, timeoutMills);
}
/**
* the real get api,
*
* @param url the specified url
* @param headers the http request headers
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
private static SimpleHttpResponse get(URL url, Map<String, String> headers, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
Headers h = Headers.of(finalHeaders);
Request req = new Request.Builder().url(url).headers(h).build();
return doCall(client, req);
}
/**
* post body as raw data
*
* @param url the specified url
* @param headers http request headers
* @param body http post body raw data ,the default body type is json or specified in headers Content-type field
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
private static SimpleHttpResponse postRawBody(URL url, Map<String, String> headers, String body, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
// rare condition: post nothing to a url
if (body == null) {
body = StringUtils.EMPTY;
}
String type = finalHeaders.containsKey(HttpHeaders.CONTENT_TYPE) ? finalHeaders.get(HttpHeaders.CONTENT_TYPE) :
com.google.common.net.MediaType.JSON_UTF_8.toString();
RequestBody requestBody = RequestBody.create(MediaType.parse(type), body.getBytes(StandardCharsets.UTF_8));
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).post(requestBody);
Request req = builder.build();
return doCall(client, req);
}
private static SimpleHttpResponse putRawBody(URL url, Map<String, String> headers, String body, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
// rare condition: post nothing to a url
if (body == null) {
body = StringUtils.EMPTY;
}
String type = finalHeaders.containsKey(HttpHeaders.CONTENT_TYPE) ? finalHeaders.get(HttpHeaders.CONTENT_TYPE) :
com.google.common.net.MediaType.JSON_UTF_8.toString();
RequestBody requestBody = RequestBody.create(MediaType.parse(type), body.getBytes(StandardCharsets.UTF_8));
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).put(requestBody);
Request req = builder.build();
return doCall(client, req);
}
private static SimpleHttpResponse deleteRawBody(URL url, Map<String, String> headers, String body, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
// rare condition: post nothing to a url
if (body == null) {
body = StringUtils.EMPTY;
}
String type = finalHeaders.containsKey(HttpHeaders.CONTENT_TYPE) ? finalHeaders.get(HttpHeaders.CONTENT_TYPE) :
com.google.common.net.MediaType.JSON_UTF_8.toString();
RequestBody requestBody = RequestBody.create(MediaType.parse(type), body.getBytes(StandardCharsets.UTF_8));
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).delete(requestBody);
Request req = builder.build();
return doCall(client, req);
}
/**
* post parameterMap as "application/x-www-form-urlencoded"
*
* @param url the specified url
* @param headers http request headers
* @param parameterMap http post parameters
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
private static SimpleHttpResponse post(URL url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
if (parameterMap == null) {
parameterMap = new HashMap<>();
}
Map<String, String> finalHeaders = new HashMap<>();
finalHeaders.putAll(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
FormBody.Builder fbBuilder = new FormBody.Builder();
for (Map.Entry<String, String> entry : parameterMap.entrySet()) {
fbBuilder.add(entry.getKey(), entry.getValue());
}
RequestBody body = fbBuilder.build();
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).post(body);
Request req = builder.build();
return doCall(client, req);
}
private static SimpleHttpResponse put(URL url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
if (parameterMap == null) {
parameterMap = new HashMap<>();
}
Map<String, String> finalHeaders = new HashMap<>(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
FormBody.Builder fbBuilder = new FormBody.Builder();
for (Map.Entry<String, String> entry : parameterMap.entrySet()) {
fbBuilder.add(entry.getKey(), entry.getValue());
}
RequestBody body = fbBuilder.build();
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).put(body);
Request req = builder.build();
return doCall(client, req);
}
/**
* transform the checked exception to runtime exception
*
* @param url the specified String url
* @return URL Object
*/
private static URL getURL(String url) {
URL u;
try {
u = new URL(url);
} catch (MalformedURLException e) {
LOGGER.warn("try getContent with Malformed URL {}", url);
throw new RuntimeException("url error: " + url, e);
}
return u;
}
private static SimpleHttpResponse doCall(OkHttpClient client, Request req) {
long start = Instant.now().toEpochMilli();
boolean succ = false;
long size = 0;
SimpleHttpResponse response;
try {
Response res = client.newCall(req).execute();
succ = res.isSuccessful();
size = res.body() == null ? 0 : res.body().contentLength();
response = SimpleHttpResponse.newHttpResponse(res);
return response;
} catch (IOException e) {
LOGGER.warn("OkHttpClient {} request {}", client, req, e);
response = SimpleHttpResponse.Error(e);
return response;
} finally {
long cost = Instant.now().toEpochMilli() - start;
LOGGER.info("http call success[{}] url[{}] method[{}] costMills[{}] size[{}] ", succ, req.url().url().toString(), req.method(), cost, size);
}
}
/**
* auto find best client for url
*
* @param url the specified url
* @param timeoutMills the timeout milliseconds for client
* @return the best http client
*/
private static OkHttpClient findBestClient(String url, int timeoutMills) {
OkHttpClient client = midHttpClient;
// first find by timeout
if (timeoutMills > 0) {
if (timeoutMills <= fastTimeout) {
client = fastHttpClient;
} else if (timeoutMills <= midTimeout) {
client = midHttpClient;
}
}
return client;
}
/**
* asyn post body as raw data
*
* @param url the specified url
* @param headers http request headers
* @param body http post body raw data ,the default body type is json or specified in headers Content-type field
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
public static void asynPostRawBody(String url, Map<String, String> headers, String body, int timeoutMills) {
asynPostRawBody(getURL(url), headers, body, timeoutMills);
}
/**
* asyn post body as raw data
*
* @param url the specified url
* @param headers http request headers
* @param body http post body raw data ,the default body type is json or specified in headers Content-type field
* @param timeoutMills 0 for using default timeout by client
* @return SimpleHttpResponse
*/
private static void asynPostRawBody(URL url, Map<String, String> headers, String body, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
Map<String, String> finalHeaders = new HashMap<>();
finalHeaders.putAll(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
// rare condition: post nothing to a url
if (body == null) {
body = StringUtils.EMPTY;
}
String type = finalHeaders.containsKey(HttpHeaders.CONTENT_TYPE) ? finalHeaders.get(HttpHeaders.CONTENT_TYPE) :
com.google.common.net.MediaType.JSON_UTF_8.toString();
RequestBody requestBody = RequestBody.create(MediaType.parse(type), body);
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).post(requestBody);
Request req = builder.build();
//异步调用
asyncDoCall(client, req);
}
public static void asynPost(String url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
asynPost(getURL(url), headers, parameterMap, timeoutMills);
}
private static void asynPost(URL url, Map<String, String> headers, Map<String, String> parameterMap, int timeoutMills) {
OkHttpClient client = findBestClient(url.toString(), timeoutMills);
if (parameterMap == null) {
parameterMap = new HashMap<>();
}
Map<String, String> finalHeaders = new HashMap<>();
finalHeaders.putAll(defaultHeader);
if (headers != null) {
finalHeaders.putAll(headers);
}
FormBody.Builder fbBuilder = new FormBody.Builder();
for (Map.Entry<String, String> entry : parameterMap.entrySet()) {
fbBuilder.add(entry.getKey(), entry.getValue());
}
RequestBody body = fbBuilder.build();
Request.Builder builder = new Request.Builder().headers(Headers.of(finalHeaders)).url(url).post(body);
Request req = builder.build();
//异步调用
asyncDoCall(client, req);
}
/**
* 方法实现说明 异步调用
*
* @param client
* @param req
* @return com.ksyun.http.SimpleHttpResponse
* @throws
* @author Ivan
* @date 2019/1/10 16:49
*/
private static void asyncDoCall(OkHttpClient client, Request req) {
long start = Instant.now().toEpochMilli();
//异步执行
client.newCall(req).enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
LOGGER.warn("OkHttpClient {} fail request {} Exception is {}", client, call.request(), e);
}
@Override
public void onResponse(Call call, Response response) {
if (response.isSuccessful()) {
long cost = Instant.now().toEpochMilli() - start;
LOGGER.info("http call success[true] url[{}] method[{}] costMills[{}] size[{}] ", call.request().url().url().toString(), call.request().method(), cost, response.body().contentLength());
} else {
long cost = Instant.now().toEpochMilli() - start;
LOGGER.warn("http call success[false] url[{}] method[{}] costMills[{}] msg [{}] ", call.request().url().url().toString(), call.request().method(), cost, response.message());
}
response.close();
}
});
}
}
SimpleHttpResponse
package com.example.demohttp.okhttp;
import java.net.ProtocolException;
import java.util.List;
import java.util.Map;
/**
* @Date: 2021/12/1 4:39 下午
*/
public class SimpleHttpResponse {
private int statusCode;
private String body;
private Exception exception;
private Map<String, List<String>> headers;
private SimpleHttpResponse(int statusCode, String body, Map<String, List<String>> headers) {
this.statusCode = statusCode;
this.body = body;
this.exception = null;
this.headers = headers;
}
private SimpleHttpResponse(int statusCode, String body, Map<String, List<String>> headers, Exception e) {
this.statusCode = statusCode;
this.body = body;
this.exception = e;
this.headers = headers;
}
static SimpleHttpResponse Error(Exception e) {
return new SimpleHttpResponse(500, null, null, e);
}
static SimpleHttpResponse newHttpResponse(okhttp3.Response theResponse) {
try {
if (theResponse.isSuccessful()) {
return new SimpleHttpResponse(theResponse.code(), theResponse.body().string(), theResponse.headers().toMultimap());
} else {
return new SimpleHttpResponse(theResponse.code(), theResponse.body().string(), theResponse.headers().toMultimap(),
new ProtocolException("Http StatusCode error: " + theResponse.code() + " (" + theResponse.message() + ")"));
}
} catch (Exception e) {
return Error(e);
} finally {
theResponse.close();
}
}
public Map<String, List<String>> getHeaders() {
return headers;
}
public int getStatusCode() {
return statusCode;
}
public String getBody() {
return body;
}
public Exception getException() {
return exception;
}
public boolean isSuccess() {
return exception == null && getStatusCode() < 300;
}
@Override
public String toString() {
return "SimpleHttpResponse{" +
"statusCode=" + statusCode +
", body='" + body + '\'' +
", exception=" + exception +
'}';
}
}
五,HttpURLConnection和HttpClient池化性能对比
个数 | HttpClient池化 | HttpUrlConnation |
---|---|---|
1000 | 520ms | 178ms |
10000 | 1378ms | 869ms |
50000 | 3732ms | 3476ms |
100000 | 6752ms | 7370ms |
200000 | 12454ms | 14362ms |