httpclient读取数据不完整

最新推荐文章于 2024-07-13 02:19:17 发布

weixin_34072857

最新推荐文章于 2024-07-13 02:19:17 发布

阅读量2.1k

点赞数

文章标签： java 爬虫 json

原文链接：https://my.oschina.net/wy08/blog/132258

版权

为什么80%的码农都做不了架构师？>>>

读取不完整，可能时服务端开启了gzip的原因，至少本人遇到的情况是这样

// 实例化请求对象
            HttpPost request = new HttpPost(Url);
            
            // 设置要发送的数据 以 name=value 键值对
            HttpEntity requestEntity = new UrlEncodedFormEntity(params, "UTF-8");
            request.setEntity(requestEntity);
            //获取HttpClient
            DefaultHttpClient httpClient = new DefaultHttpClient();
            
            httpClient.addRequestInterceptor(new HttpRequestInterceptor() {  
                public void process(final HttpRequest request,  
                        final HttpContext context) throws HttpException,  
                        IOException {  
                    if (!request.containsHeader("Accept-Encoding")) {  
                        request.addHeader("Accept-Encoding", "gzip");  
                    }  
                }  
              
            });  

            //设置请求超时时间
            /tpClient.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 3000);
            // 发送请求 其实底层就是 OutputStream,并获得响应
            HttpResponse response = httpClient.execute(request);

            if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
                // 获得服务器响应返回的数据
                //         
                HttpEntity entity = response.getEntity();
                Header ceheader = entity.getContentEncoding();
                if (ceheader != null) {
                    for (HeaderElement element : ceheader.getElements()) {
                        if ("gzip".equalsIgnoreCase(element.getName())) {
                            entity = new GzipDecompressingEntity(response.getEntity());
                            break;
                        }
                    }
                }

                String html = EntityUtils.toString(entity, "UTF-8");
                System.out.println(html);

转载于:https://my.oschina.net/wy08/blog/132258