3.2.2 HttpClient请求URL

1.创建HttpClient实例
HttpClient的重要功能是执行HTTP请求方法，获取响应资源。在执行具体的请求方法之前，需要实例化HttpClient。 实例化HttpClient的方式主要有以下5种。

• HttpClient httpClient = Httpclients.custom().build();
• HttpClient httpClient = Httpclientbuilder.create().build();
• HttpClient httpClient = Httpclients.createSystem();
• HttpClient httpClient = Httpclients.createMinimal();
• CloseableHttpClient httpClient = Httpclients.createDefault();

2.创建请求方法的实例
在HttpClient中，支持HTTP/1.1的HTTP方法，即GET、POST、HEAD、PUT、DELETE、OPTIONS和TRACE。其中，每种方法都对应一个类，即HttpGet、HttpPost、HttpHead、HttpPut、HttpDelete、HttpOption和HttpTrace。在网络爬虫中，常用的类是HttpGet与HttpPost。从HttpClient源码中，可以发现这些类的实例化方式各有三种，三种实例化使用方式如下面代码所示。

//第一种方式
String personalUrl = "https://searchcustomerexperience.techtarget.com/info/news";
URI uri = new URIBuilder(personalUrl).build();
HttpGet getMethod = new HttpGet();
getMethod.setURI(uri);
System.out.println(getMethod);

//第二种方式
HttpGet httpGetUri = new HttpGet(uri);
System.out.println(httpGetUri);

//第三种方式
HttpGet httpGetStr = new HttpGet(personalUrl);
System.out.println(httpGetStr);

3.执行请求
基于实例化的HttpClient，可以调用execute(HttpUriRequest request)方法来执行请求，返回HttpResponse。HttpClient也提供了三种操作方式，代码示例如下。

//第一种方式
HttpResponse httpResponse = new BasicHttpResponse(HttpVersion.HTTP_1_1,
				HttpStatus.SC_OK, "OK");                      
httpResponse = client.execute(getMethod);                
		
//第二种方式
HttpResponse httpResponse = null;
try {
	httpResponse = httpClient.execute(httpGet,localContext);
} catch (IOException e) {
	e.printStackTrace();
}

//第三种方式
CloseableHttpClient httpClient = HttpClients.createDefault(); 
HttpGet httpGet = new HttpGet("https://searchcustomerexperience.techtarget.com/info/news");
CloseableHttpResponse httpResponse = null;
try {
	httpResponse = httpClient.execute(httpGet);
} catch (IOException e) {
	e.printStackTrace();
}

4.获取响应信息
基于上述方法3获取的HttpResponse，可以继续执行一些方法获取响应状态码、响应头和响应实体等信息，如程序3-14所示，在执行请求时，使用了HttpContext，即HTTP上下文环境。

//程序3-14
public class HttpclientInit {
    public static void main(String[] args) throws Exception {
        //初始化HttpContext
        HttpContext localContext = new BasicHttpContext();
        String url = "https://searchcustomerexperience.techtarget.com/info/news";
        //初始化httpClient
        HttpClient httpClient = HttpClients.custom().build();
        HttpGet httpGet = new HttpGet(url);
        //执行请求获取HttpResponse
        HttpResponse httpResponse = null;
        try {
            httpResponse = httpClient.execute(httpGet,localContext);
        } catch (IOException e) {
            e.printStackTrace();
        }
        //获取具体响应信息
        System.out.println("response:" + httpResponse );
        //响应状态
        String status = httpResponse .getStatusLine().toString();
        System.out.println("status:" + status);
        //获取响应状态码
        int statusCode = httpResponse .getStatusLine().getStatusCode();
        System.out.println("statusCode:" + statusCode);
        //协议的版本号
        ProtocolVersion protocolVersion = httpResponse .getProtocolVersion();
        System.out.println("protocolVersion:" + protocolVersion);
        //是否是ok
        String phrase = httpResponse .getStatusLine().getReasonPhrase();
        System.out.println("phrase:" + phrase);
        //头信息
        Header [] headers = httpResponse.getAllHeaders();
        System.out.println("输出头信息为：");
        for (int i = 0; i < headers.length; i++) {
            System.out.println(headers[i]);
        }
        System.out.println("头信息输出结束");
        if(statusCode == HttpStatus.SC_OK){//状态码200表示响应成功
            //获取实体内容
            HttpEntity entity = httpResponse.getEntity();
            //注意设置编码
            String entityString = EntityUtils.toString (entity,"gbk");
            //输出实体内容
            System.out.println(entityString);
            EntityUtils.consume(httpResponse.getEntity());
        }else {
            //关闭HttpEntity的流实体
            EntityUtils.consume(httpResponse.getEntity());
        }
    }
}