springboot中如何向外部发起一次http请求

当想要向外部发起一次简单的http请求时, 会出现很多选项:

1. hutool的HttpUtil
2. springBoot的restTemplate
3. okHttp
4. apache的httpclient
   …等等
   具体http到底是如何发起的, 这些http工具之间又是什么关系, 对于初接触的人还是一顿浆糊, 这里试图简单明了的说清这些事, 让小白也能明白springboot框架下发起的http到底是什么
   本次设计以下关键包和关键类

• org.apache.httpcomponents:httpclient

• org.springframework:spring-web

  • RestTemplate
  • HttpRequest
  • ClientHttpRequestFactory
  • HttpComponentsClientHttpRequestFactory

• cn.hutool:hutool-all

• • HttpUtil
  • HttpRequest
  • HttpConnection

• jdk.rt

  • HttpURLConnection
    先说结论, 再看源码

hutool中用的jdk.rt中的net包下的HttpURLConnection
apache的httpcomponents用的是自己包内的HttpConnection
springboot提供了RestTemplate , 不会操作底层的HttpURLConnection, RestTemplate默认使用的是标准的 java.net.HttpURLConnection, 其工厂是SimpleClientHttpRequestFactory , 也可以更换为Apache HttpClient的HttpComponentsClientHttpRequestFactory或者OkHttp的OkHttp3ClientHttpRequestFactory

源码分析:

Hutool包的HttpUtil

先构造了一个HttpRequest, 然后执行execute方法
HttpUtil.class

	public static String post(String urlString, Map<String, Object> paramMap, int timeout) {
		return HttpRequest.post(urlString).form(paramMap).timeout(timeout).execute().body();
	}

HttpRequest.class

	public HttpResponse execute(boolean isAsync) {
		return doExecute(isAsync, config.requestInterceptors, config.responseInterceptors);
	}
private HttpResponse doExecute(boolean isAsync, HttpInterceptor.Chain<HttpRequest> requestInterceptors,
								   HttpInterceptor.Chain<HttpResponse> responseInterceptors) {
		if (null != requestInterceptors) {
			for (HttpInterceptor<HttpRequest> interceptor : requestInterceptors) {
				interceptor.process(this);
			}
		}

		// 初始化URL
		urlWithParamIfGet();
		// 初始化 connection
		initConnection();
		// 发送请求
		send();

		// 手动实现重定向
		HttpResponse httpResponse = sendRedirectIfPossible(isAsync);

		// 获取响应
		if (null == httpResponse) {
			httpResponse = new HttpResponse(this.httpConnection, this.config, this.charset, isAsync, isIgnoreResponseBody());
		}

		// 拦截响应
		if (null != responseInterceptors) {
			for (HttpInterceptor<HttpResponse> interceptor : responseInterceptors) {
				interceptor.process(httpResponse);
			}
		}

		return httpResponse;
	}

	/**
	 * 初始化网络连接
	 */
	private void initConnection() {
		if (null != this.httpConnection) {
			// 执行下次请求时自动关闭上次请求（常用于转发）
			this.httpConnection.disconnectQuietly();
		}

		this.httpConnection = HttpConnection
				// issue#I50NHQ
				// 在生成正式URL前，设置自定义编码
				.create(this.url.setCharset(this.charset).toURL(this.urlHandler), config.proxy)//
				.setConnectTimeout(config.connectionTimeout)//
				.setReadTimeout(config.readTimeout)//
				.setMethod(this.method)//
				.setHttpsInfo(config.hostnameVerifier, config.ssf)//
				// 关闭JDK自动转发，采用手动转发方式
				.setInstanceFollowRedirects(false)
				// 流方式上传数据
				.setChunkedStreamingMode(config.blockSize)
				// 覆盖默认Header
				.header(this.headers, true);

		if (null != this.cookie) {
			// 当用户自定义Cookie时，全局Cookie自动失效
			this.httpConnection.setCookie(this.cookie);
		} else {
			// 读取全局Cookie信息并附带到请求中
			GlobalCookieManager.add(this.httpConnection);
		}

		// 是否禁用缓存
		if (config.isDisableCache) {
			this.httpConnection.disableCache();
		}
	}

可以看到主要的操作是围绕httpConnection对象进行操作的

/**
* 连接对象
*/
private HttpConnection httpConnection;

进入HttpConnection类

private final URL url;
private final Proxy proxy;
private HttpURLConnection conn;

这里的HttpURLConnection 就是来自于java的net包下

RestTemplate

	public <T> T postForObject(String url, @Nullable Object request, Class<T> responseType,
			Object... uriVariables) throws RestClientException {

		RequestCallback requestCallback = httpEntityCallback(request, responseType);
		HttpMessageConverterExtractor<T> responseExtractor =
				new HttpMessageConverterExtractor<>(responseType, getMessageConverters(), logger);
		return execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables);
	}
	
	public <T> T execute(String url, HttpMethod method, @Nullable RequestCallback requestCallback,
			@Nullable ResponseExtractor<T> responseExtractor, Object... uriVariables) throws RestClientException {

		URI expanded = getUriTemplateHandler().expand(url, uriVariables);
		return doExecute(expanded, method, requestCallback, responseExtractor);
	}
protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
			@Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {

		Assert.notNull(url, "URI is required");
		Assert.notNull(method, "HttpMethod is required");
		ClientHttpResponse response = null;
		try {
			ClientHttpRequest request = createRequest(url, method);
			if (requestCallback != null) {
				requestCallback.doWithRequest(request);
			}
			response = request.execute();
			handleResponse(url, method, response);
			return (responseExtractor != null ? responseExtractor.extractData(response) : null);
		}
		catch (IOException ex) {
			String resource = url.toString();
			String query = url.getRawQuery();
			resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
			throw new ResourceAccessException("I/O error on " + method.name() +
					" request for \"" + resource + "\": " + ex.getMessage(), ex);
		}
		finally {
			if (response != null) {
				response.close();
			}
		}
	}

	protected ClientHttpRequest createRequest(URI url, HttpMethod method) throws IOException {
		ClientHttpRequest request = getRequestFactory().createRequest(url, method);
		if (logger.isDebugEnabled()) {
			logger.debug("HTTP " + method.name() + " " + url);
		}
		return request;
	}

可以看到RestTemplate是通过注入一个ClientHttpRequestFactory
ClientHttpRequestFactory是来自spring-web包, 由spring定义的
看一下ClientHttpRequestFactory的哪些实现

进入HttpComponentsClientHttpRequestFactory.class

public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
		HttpClient client = getHttpClient();

		HttpUriRequest httpRequest = createHttpUriRequest(httpMethod, uri);
		postProcessHttpRequest(httpRequest);
		HttpContext context = createHttpContext(httpMethod, uri);
		if (context == null) {
			context = HttpClientContext.create();
		}

		if (context.getAttribute(HttpClientContext.REQUEST_CONFIG) == null) {

			RequestConfig config = null;
			if (httpRequest instanceof Configurable) {
				config = ((Configurable) httpRequest).getConfig();
			}
			if (config == null) {
				config = createRequestConfig(client);
			}
			if (config != null) {
				context.setAttribute(HttpClientContext.REQUEST_CONFIG, config);
			}
		}

		if (this.bufferRequestBody) {
			return new HttpComponentsClientHttpRequest(client, httpRequest, context);
		}
		else {
			return new HttpComponentsStreamingClientHttpRequest(client, httpRequest, context);
		}
	}

可以看到, HttpComponentsClientHttpRequestFactory的构造和创建的ClientHttpRequest的使用与内部对象HttpClient息息相关.
这里的HttpClient其实就来自于org.apache.httpclient

至此, 可以看到spring中一条清晰的线:
HttpRequest -> ClientHttpRequest -> ClientHttpRequestFactory
在各种自定义的factory中持有不同来源的httpClient, 并且统一转换为ClientHttpRequest, 在执行execute()方法时调用内部的httpClient

com.apache.httpcomponents:httpclient

HttpClient -> CloseableHttpClient -> AbstractHttpClient -> CloseableHttpResponseProxy

首先由HttpComponentsClientHttpRequestFactory开始引入了HttpClient, 其实现类CloseableHttpClient

    public CloseableHttpResponse execute(HttpHost target, HttpRequest request, HttpContext context) throws IOException, ClientProtocolException {
        return this.doExecute(target, request, context);
    }

其实现类AbstractHttpClient

protected final CloseableHttpResponse doExecute(HttpHost target, HttpRequest request, HttpContext context) throws IOException, ClientProtocolException {
        Args.notNull(request, "HTTP request");
        HttpContext execContext = null;
        RequestDirector director = null;
        HttpRoutePlanner routePlanner = null;
        ConnectionBackoffStrategy connectionBackoffStrategy = null;
        BackoffManager backoffManager = null;
        synchronized(this) {
            HttpContext defaultContext = this.createHttpContext();
            if (context == null) {
                execContext = defaultContext;
            } else {
                execContext = new DefaultedHttpContext(context, defaultContext);
            }

            HttpParams params = this.determineParams(request);
            RequestConfig config = HttpClientParamConfig.getRequestConfig(params);
            ((HttpContext)execContext).setAttribute("http.request-config", config);
            director = this.createClientRequestDirector(this.getRequestExecutor(), this.getConnectionManager(), this.getConnectionReuseStrategy(), this.getConnectionKeepAliveStrategy(), this.getRoutePlanner(), this.getProtocolProcessor(), this.getHttpRequestRetryHandler(), this.getRedirectStrategy(), this.getTargetAuthenticationStrategy(), this.getProxyAuthenticationStrategy(), this.getUserTokenHandler(), params);
            routePlanner = this.getRoutePlanner();
            connectionBackoffStrategy = this.getConnectionBackoffStrategy();
            backoffManager = this.getBackoffManager();
        }

        try {
            if (connectionBackoffStrategy != null && backoffManager != null) {
                HttpHost targetForRoute = target != null ? target : (HttpHost)this.determineParams(request).getParameter("http.default-host");
                HttpRoute route = routePlanner.determineRoute(targetForRoute, request, (HttpContext)execContext);

                CloseableHttpResponse out;
                try {
                    out = CloseableHttpResponseProxy.newProxy(director.execute(target, request, (HttpContext)execContext));
                } catch (RuntimeException var15) {
                    if (connectionBackoffStrategy.shouldBackoff(var15)) {
                        backoffManager.backOff(route);
                    }

                    throw var15;
                } catch (Exception var16) {
                    if (connectionBackoffStrategy.shouldBackoff(var16)) {
                        backoffManager.backOff(route);
                    }

                    if (var16 instanceof HttpException) {
                        throw (HttpException)var16;
                    }

                    if (var16 instanceof IOException) {
                        throw (IOException)var16;
                    }

                    throw new UndeclaredThrowableException(var16);
                }

                if (connectionBackoffStrategy.shouldBackoff(out)) {
                    backoffManager.backOff(route);
                } else {
                    backoffManager.probe(route);
                }

                return out;
            } else {
                return CloseableHttpResponseProxy.newProxy(director.execute(target, request, (HttpContext)execContext));
            }
        } catch (HttpException var17) {
            throw new ClientProtocolException(var17);
        }
    }

这里很长一段, 其主要作用是构建RequestDirector和HttpRoute, 进入RequestDirector的execute方法

...省略, 只关注ClientConnectionManager connManager
if (this.managedConn == null) {
                    ClientConnectionRequest connRequest = this.connManager.requestConnection(route, userToken);
                    if (orig instanceof AbortableHttpRequest) {
                        ((AbortableHttpRequest)orig).setConnectionRequest(connRequest);
                    }

                    duration = HttpClientParams.getConnectionManagerTimeout(this.params);

                    try {
                        this.managedConn = connRequest.getConnection(duration, TimeUnit.MILLISECONDS);
                    } catch (InterruptedException var19) {
                        Thread.currentThread().interrupt();
                        throw new InterruptedIOException();
                    }

                    if (HttpConnectionParams.isStaleCheckingEnabled(this.params) && this.managedConn.isOpen()) {
                        this.log.debug("Stale connection check");
                        if (this.managedConn.isStale()) {
                            this.log.debug("Stale connection detected");
                            this.managedConn.close();
                        }
                    }
                }

这里的connManager就是httpComponents包对http实现的关键类, 其内部做了线程的创建, 复用和销毁
以PoolingClientConnectionManager举例

    public ClientConnectionRequest requestConnection(HttpRoute route, Object state) {
        Args.notNull(route, "HTTP route");
        if (this.log.isDebugEnabled()) {
            this.log.debug("Connection request: " + this.format(route, state) + this.formatStats(route));
        }

        final Future<HttpPoolEntry> future = this.pool.lease(route, state);
        return new ClientConnectionRequest() {
            public void abortRequest() {
                future.cancel(true);
            }

            public ManagedClientConnection getConnection(long timeout, TimeUnit timeUnit) throws InterruptedException, ConnectionPoolTimeoutException {
                return PoolingClientConnectionManager.this.leaseConnection(future, timeout, timeUnit);
            }
        };
    }

其核心是内部持有的

private final HttpConnPool pool

进入HttpConnPool

public Future<E> lease(final T route, final Object state, final FutureCallback<E> callback) {
        Args.notNull(route, "Route");
        Asserts.check(!this.isShutDown, "Connection pool shut down");
        return new Future<E>() {
            private final AtomicBoolean cancelled = new AtomicBoolean(false);
            private final AtomicBoolean done = new AtomicBoolean(false);
            private final AtomicReference<E> entryRef = new AtomicReference((Object)null);

            public boolean cancel(boolean mayInterruptIfRunning) {
                if (this.cancelled.compareAndSet(false, true)) {
                    this.done.set(true);
                    AbstractConnPool.this.lock.lock();

                    try {
                        AbstractConnPool.this.condition.signalAll();
                    } finally {
                        AbstractConnPool.this.lock.unlock();
                    }

                    if (callback != null) {
                        callback.cancelled();
                    }

                    return true;
                } else {
                    return false;
                }
            }

            public boolean isCancelled() {
                return this.cancelled.get();
            }

            public boolean isDone() {
                return this.done.get();
            }

            public E get() throws InterruptedException, ExecutionException {
                try {
                    return this.get(0L, TimeUnit.MILLISECONDS);
                } catch (TimeoutException var2) {
                    throw new ExecutionException(var2);
                }
            }

            public E get(long timeout, TimeUnit timeUnit) throws InterruptedException, ExecutionException, TimeoutException {
                E entry = (PoolEntry)this.entryRef.get();
                if (entry != null) {
                    return entry;
                } else {
                    synchronized(this) {
                        try {
                            while(true) {
                                E leasedEntry = AbstractConnPool.this.getPoolEntryBlocking(route, state, timeout, timeUnit, this);
                                if (AbstractConnPool.this.validateAfterInactivity <= 0 || leasedEntry.getUpdated() + (long)AbstractConnPool.this.validateAfterInactivity > System.currentTimeMillis() || AbstractConnPool.this.validate(leasedEntry)) {
                                    this.entryRef.set(leasedEntry);
                                    this.done.set(true);
                                    AbstractConnPool.this.onLease(leasedEntry);
                                    if (callback != null) {
                                        callback.completed(leasedEntry);
                                    }

                                    PoolEntry var10000 = leasedEntry;
                                    return var10000;
                                }

                                leasedEntry.close();
                                AbstractConnPool.this.release(leasedEntry, false);
                            }
                        } catch (IOException var8) {
                            this.done.set(true);
                            if (callback != null) {
                                callback.failed(var8);
                            }

                            throw new ExecutionException(var8);
                        }
                    }
                }
            }
        };
    }

这里就是线程的创建和释放了, 和大部分的线程池思想一样, 就不详细解读了
简单看一下HttpPoolEntry extends PoolEntry<HttpRoute, OperatedClientConnection>

private final C conn;
 private long expiry;

在HttpPoolEntry 中, 终于看到了最终的http连接, 其中final C conn就是HttpClientConnection的实现类了, 也是实际的http连接