小白Java自编ip代理池

最新推荐文章于 2023-11-27 10:13:55 发布
最新推荐文章于 2023-11-27 10:13:55 发布
阅读量450 收藏 2
点赞数
文章标签： json java 爬虫
原文链接：https://my.oschina.net/u/4131417/blog/3048247
版权

各们大佬们，小白第一次在github上发布源码，不足之处还往多多指教。

     这是给朋友爬虫使用的ip代理池，保证ip的有效性和可靠性，使用java实现。

      话不多说开始上干货

     本次采用的是apache开源对象池PoolableObjectFactory管理所有ip

    首先引项目所需依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.3.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.shimmer</groupId>
    <artifactId>iphttps</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>iphttps</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.7</version>
        </dependency>
        <dependency>
            <groupId>commons-codec</groupId>
            <artifactId>commons-codec</artifactId>
        </dependency>
        <dependency>
            <groupId>commons-httpclient</groupId>
            <artifactId>commons-httpclient</artifactId>
            <version>3.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi</artifactId>
            <version>3.9</version>
        </dependency>
        <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi-ooxml</artifactId>
            <version>3.9</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>
        <dependency>
            <groupId>commons-pool</groupId>
            <artifactId>commons-pool</artifactId>
            <version>1.6</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.2</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-test</artifactId>
            <version>2.1.3.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>5.1.5.RELEASE</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

核心配置文件application.yml

# PROXY代理配置
ip-proxy:
  # 最大活动数
  maxActive: 200
  # 最大空闲数
  maxIdle: 20
  # 最大等待时间
  maxWait: 100
  #返回是否校验
  testOnReturn: false
  #借出是否校验
  testOnBorrow: true
  #是否后进先出
  lifo: false
  #提前多少分钟销毁对象
  testTime: 3
  #设定间隔每过多少毫秒进行一次后台对象清理的行动。如果这个值不是正数，则实际上不会进行后台对象清理。
  timeBetweenEvictionRunsMillis: -1000
  #设定在进行后台对象清理时，是否还对没有过期的池内对象进行有效性检查。不能通过有效性检查的对象也将被回收。
  testWhileIdle: true
  #设定在进行后台对象清理时，每次检查几个对象。如果这个值不是正数，则每次检查的对象数是检查时池内对象的总数乘以这个值的负倒数再向上取整的结果――也就是说，
  #如果这个值是-2（-3、-4、-5……）的话，那么每次大约检查当时池内对象总数的1/2（1/3、1/4、1/5……）左右。
  numTestsPerEvictionRun: 3
  # 获取ip 方式 0 为免费 1 为付费 2 为免费和付费同时进行
  iPType: 2
  # IP代理请求路径 某某第三方ip代理
  httpPath: "http://"
  # IP代理请求参数
  httpProxy:
  #所有的参数和对应的值 如下：
    num: "1"
    type: "2"
    pro: ""
    city: ""
    yys: "0"
    port: "1"
    pack: "1111"
    amount: "1"
    ts: "1"
    ys: "1"
    cs: "1"
    lb: "1"
    sb: "0"
    pb: "4"
    mr: "1"
    regions: ""

获取配置类 

package com.sunshine.http_proxy.utils;

import java.util.HashMap;
import java.util.Map;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;

/**
 * @author xaingzi
 * 配置类
 * 从配置文件中读取数据映射到map
 * 注意：必须实现set方法
 */
@Configuration
@ConfigurationProperties(prefix = "ip-proxy")
public class HttpProxyConfig {
   private int maxActive;
   private int maxIdle;
   private int maxWait;
   private int testTime;
   private int numTestsPerEvictionRun;
   private int iPType;
   private boolean testOnReturn;
   private boolean testOnBorrow;
   private boolean lifo;
   private String httpPath;
   private boolean testWhileIdle;
   private long timeBetweenEvictionRunsMillis;
   private Map<String, String> httpProxy = new HashMap<>();
   
   public int getMaxActive() {
      return maxActive;
   }
   public void setMaxActive(int maxActive) {
      this.maxActive = maxActive;
   }
   public int getMaxIdle() {
      return maxIdle;
   }
   public void setMaxIdle(int maxIdle) {
      this.maxIdle = maxIdle;
   }
   public int getMaxWait() {
      return maxWait;
   }
   public void setMaxWait(int maxWait) {
      this.maxWait = maxWait;
   }
   public boolean isTestOnReturn() {
      return testOnReturn;
   }
   public void setTestOnReturn(boolean testOnReturn) {
      this.testOnReturn = testOnReturn;
   }
   public boolean isTestOnBorrow() {
      return testOnBorrow;
   }
   public void setTestOnBorrow(boolean testOnBorrow) {
      this.testOnBorrow = testOnBorrow;
   }
   public boolean isLifo() {
      return lifo;
   }
   public void setLifo(boolean lifo) {
      this.lifo = lifo;
   }
   public String getHttpPath() {
      return httpPath;
   }
   public void setHttpPath(String httpPath) {
      this.httpPath = httpPath;
   }
   public Map<String, String> getHttpProxy() {
      return httpProxy;
   }
   public void setHttpProxy(Map<String, String> httpProxy) {
      this.httpProxy = httpProxy;
   }
   public int getTestTime() {
      return testTime;
   }
   public void setTestTime(int testTime) {
      this.testTime = testTime;
   }

   public boolean isTestWhileIdle() {
      return testWhileIdle;
   }

   public void setTestWhileIdle(boolean testWhileIdle) {
      this.testWhileIdle = testWhileIdle;
   }

   public long getTimeBetweenEvictionRunsMillis() {
      return timeBetweenEvictionRunsMillis;
   }

   public void setTimeBetweenEvictionRunsMillis(long timeBetweenEvictionRunsMillis) {
      this.timeBetweenEvictionRunsMillis = timeBetweenEvictionRunsMillis;
   }

    public int getNumTestsPerEvictionRun() {
        return numTestsPerEvictionRun;
    }

    public void setNumTestsPerEvictionRun(int numTestsPerEvictionRun) {
        this.numTestsPerEvictionRun = numTestsPerEvictionRun;
    }

    public int getiPType() {
        return iPType;
    }

    public void setiPType(int iPType) {
        this.iPType = iPType;
    }
}

 

ip对象存储基类

package com.sunshine.http_proxy.pool;
/**
 * 代理对象
 *
 * @author xaingzi
 */

import java.io.Serializable;

public class ProxyHost implements Serializable {
    /**
     * 序列化
     */
    private static final long serialVersionUID = 1L;
    private String ip;
    private Integer port;
    private String creaTime;
    private String expireTime;
    private String city;
    private String isp;

    public String getIp() {
        return ip;
    }

    public void setIp(String ip) {
        this.ip = ip;
    }

    public Integer getPort() {
        return port;
    }

    public void setPort(Integer port) {
        this.port = port;
    }

    public String getCreaTime() {
        return creaTime;
    }

    public void setCreaTime(String creaTime) {
        this.creaTime = creaTime;
    }

    public String getExpireTime() {
        return expireTime;
    }

    public void setExpireTime(String expireTime) {
        this.expireTime = expireTime;
    }

    public String getCity() {
        return city;
    }

    public void setCity(String city) {
        this.city = city;
    }

    public String getIsp() {
        return isp;
    }

    public void setIsp(String isp) {
        this.isp = isp;
    }

    @Override
    public String toString() {
        return "ip：" + ip + " 端口：" + port + "  到期时间：" + expireTime + " 创建时间：" + creaTime + " 城市：" + city + " isp：" + isp;
    }
}

http请求获取第三方代理ip和扫描免费代理ip 

package com.sunshine.http_proxy;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.net.MalformedURLException;
import java.net.Proxy;
import java.net.URL;
import java.net.URLConnection;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

import com.sunshine.http_proxy.utils.DateUtil;
import com.sunshine.http_proxy.utils.HttpClientUtil;
import com.sunshine.http_proxy.utils.HttpProxyConfig;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
/**
 * 
 * @author xaingzi
 *
 */ 
@SuppressWarnings({"unchecked", "rawtypes"})
public class HttpProxyClientLocat implements Runnable {
    public static Logger log = LoggerFactory.getLogger(HttpProxyClientLocat.class);
    private static final String BAIDU_PATH = "http://youxingyang.com";
    private static Map<String, String> hashMap = new Hashtable<String, String>();
    private static JSONObject json = new JSONObject();
    private static int[] ports = new int[]{80, 443, 8883, 4271, 8886, 8080, 8888, 8887, 8889, 8081};
    private static String expireTime, city, isp;
    private static int i = 0;

    /**
     * @param path
     * @param map
     * @return
     */
    public static JSONObject getHttpClientProxyIp(String path, Map<String, String> map) {
        List list = new ArrayList<HashMap<String, String>>();
        JSONObject json = new JSONObject();
        do {
            String result = HttpClientUtil.httpGet(path, map);
            log.info("获取ip信息：" + result);
            List<String> parseArray = JSON.parseArray(JSON.parseObject(result).getString("data"), String.class);
            for (String str : parseArray) {
                String ip = JSON.parseObject(str).getString("ip");
                String port = JSON.parseObject(str).getString("http_port_secured");
                // String id =
                // JSON.parseObject(result).getJSONArray("data").getJSONObject(0).getString("id");
                Map<String, Integer> map1 = new HashMap<String, Integer>();
                map1.put(ip, Integer.valueOf(port));
                checkProxyIp(map1, BAIDU_PATH);
            }
        } while ((i < 3) && (hashMap.size() < 1));
        list.add(hashMap);
        json.put("status", 0);
        json.put("data", list);
        return json;
    }

    /**
     * @param httpProxyConfig
     * @return
     */
    public static JSONObject getHttpClientProxyIp(HttpProxyConfig httpProxyConfig) {
        hashMap.clear();
        JSONObject json = new JSONObject();
        if (httpProxyConfig.getiPType() == 0) {
            try {
                json = getHttpChilentProxyApi();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        } else if (httpProxyConfig.getiPType() == 1) {
            json = getHttpClient(httpProxyConfig);
        } else if (httpProxyConfig.getiPType() == 2) {
            if (Math.random() > 0.3) {
                json = getHttpClient(httpProxyConfig);
            } else {
                try {
                    json = getHttpChilentProxyApi();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
        return json;
    }

    private static JSONObject getHttpClient(HttpProxyConfig httpProxyConfig) {
        List list = new ArrayList<HashMap<String, String>>();
        do {
            log.info("发送Get请求：" + httpProxyConfig.getHttpPath());
            String result = HttpClientUtil.httpGet(httpProxyConfig.getHttpPath(), httpProxyConfig.getHttpProxy());
            log.info("获取ip信息：" + result);
            List<String> parseArray = JSON.parseArray(JSON.parseObject(result).getString("data"), String.class);
            for (String str : parseArray) {
                String ip = JSON.parseObject(str).getString("ip");
                String port = JSON.parseObject(str).getString("port");
                expireTime = JSON.parseObject(str).getString("expire_time");
                isp = city = JSON.parseObject(str).getString("isp");
                city = JSON.parseObject(str).getString("city");
                // String id =
                // JSON.parseObject(result).getJSONArray("data").getJSONObject(0).getString("id");
                Map<String, Integer> map1 = new HashMap<String, Integer>();
                map1.put(ip, Integer.valueOf(port));
                checkProxyIp(map1, BAIDU_PATH);
            }
        } while ((i < 3) && (hashMap.size() < 1));
        list.add(hashMap);
        json.put("status", 0);
        json.put("data", list);
        return json;
    }

    /**
     * 单个代理IP有效检测
     * 
     * @param ip
     * @param port
     */
    public void createIPAddress(String ip, int port) {
        URL url = null;
        try {
            url = new URL("http://www.baidu.com");
        } catch (MalformedURLException e) {
            log.info("url invalidate");
        }
        InetSocketAddress addr = null;
        addr = new InetSocketAddress(ip, port);
        Proxy proxy = new Proxy(Proxy.Type.HTTP, addr); // http proxy
        InputStream in = null;
        try {
            URLConnection conn = url.openConnection(proxy);
            conn.setConnectTimeout(1000);
            in = conn.getInputStream();
        } catch (Exception e) {
            log.info(Thread.currentThread().getName() + "  ip " + ip + " : " + port + "   is not aviable");// 异常IP
        }
        String s = convertStreamToString(in);
        //log.info(s);
        // log.info(s);http://ip.taobao.com/service/getIpInfo.php?ip=210.32.158.14
            synchronized (this){
                if (s.indexOf("html") > 0) {// 有效IP

                        log.info(Thread.currentThread().getName() + "   " + ip + ":" + port + " is ok");
                        hashMap.put("ip", ip);
                        hashMap.put("port", port + "");
                        String s1 = HttpClientUtil.getResult("http://ip.taobao.com/service/getIpInfo.php", "ip=" + ip, "utf-8");
                        //String[] split = JSON.parseObject(s1).getJSONArray("data").getJSONObject(0).getString("location")
                        //.split(" ");
                        JSONObject data = JSON.parseObject(JSON.parseObject(s1).getString("data"));
                        hashMap.put("ip", ip);
                        hashMap.put("port", port + "");
                        hashMap.put("creaTime", (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")).format(new Date()));
                        hashMap.put("expireTime", DateUtil.format(DateUtil.addMinute(new Date(), 30), DateUtil.TIME_PATTERN_DEFAULT));
                        if (StringUtils.equals(data.getString("region"), data.getString("city"))) {
                            hashMap.put("city", data.getString("region") + "市" + data.getString("city") + "市");
                        } else {
                            hashMap.put("city", data.getString("region") + "省" + data.getString("city") + "市");
                        }
                        hashMap.put("isp", data.getString("isp"));

                }
            }
    }

    public static JSONObject getHttpChilentProxyApi() throws InterruptedException {
        Vector vector = new Vector();
        log.info("获取免费ip");
        Vector<Thread> vec = new Vector<Thread>();
        HttpProxyClientLocat httptest2 = new HttpProxyClientLocat();
        int threadNumber = 320;
        for (int i = 0; i < threadNumber; i++) {
            Thread thread = new Thread(httptest2);
            thread.start();
            vec.add(thread);
        }
        for (Thread thread : vec) {
            thread.join();
        }
        vector.add(hashMap);
        json.put("status", 0);
        json.put("data", vector);
        log.info(json.toString());
        return json;
    }

    @Override
    public void run() {
        while (hashMap.size() < 1) {
            String randomIp = getRandomIp();
            log.info(randomIp);
            for (int port : ports) {
                createIPAddress(randomIp, port);
            }
        }
    }

    public static String convertStreamToString(InputStream is) {
        if (is == null)
            return "";
        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();
        String line = null;
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "/n");
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return sb.toString();

    }

    /**
     * 批量代理IP有效检测
     *
     * @param proxyIpMap
     * @param reqUrl
     */
    public static void checkProxyIp(Map<String, Integer> proxyIpMap, String reqUrl) {
        log.info("检测ip是否有效");
        for (String proxyHost : proxyIpMap.keySet()) {
            Integer proxyPort = proxyIpMap.get(proxyHost);
            int statusCode = 0;
            try {
                HttpClient httpClient = new HttpClient();
                httpClient.getHostConfiguration().setProxy(proxyHost, proxyPort);

                // 连接超时时间（默认10秒 10000ms） 单位毫秒（ms）
                int connectionTimeout = 10000;
                // 读取数据超时时间（默认30秒 30000ms） 单位毫秒（ms）
                int soTimeout = 30000;
                httpClient.getHttpConnectionManager().getParams().setConnectionTimeout(connectionTimeout);
                httpClient.getHttpConnectionManager().getParams().setSoTimeout(soTimeout);

                String httpGet = HttpClientUtil.httpGet(reqUrl, new HashMap<>());
                log.info(httpGet);
                statusCode = httpGet.indexOf("html");
            } catch (Exception e) {
                i++;
                log.error("ip " + proxyHost + " is not aviable");
            }
            Lock l = new ReentrantLock();
            l.lock();
            try {
                if (statusCode > 0) {
                    log.info(Thread.currentThread().getName() + "   " + proxyHost + ":" + proxyPort + " is ok");
                    System.out.format("%s:%s-->%s/n", proxyHost, proxyPort, statusCode);
                    hashMap.put("creaTime", (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")).format(new Date()));
                    hashMap.put("expireTime", expireTime);
                    hashMap.put("city", city);
                    hashMap.put("ip", proxyHost);
                    hashMap.put("port", proxyPort + "");
                }
            } finally {
                l.unlock();
            }
        }
    }

    public static JSONObject connectHttpChilentStatus(String http) {
        List list = new ArrayList<HashMap<String, String>>();
        log.info("开始校验" + http);
        JSONObject json = new JSONObject();
        if (null != http && "" != http) {
            String[] ips = http.split(",");
            for (String ip : ips) {
                if (null != ip && "" != ip) {
                    Map<String, Object> map = new HashMap<>();
                    String[] str = ip.split(":");
                    int statusCode = 0;
                    map.put("ip", str[0]);
                    map.put("port", str[1]);
                    try {
                        HttpClient httpClient = new HttpClient();
                        httpClient.getHostConfiguration().setProxy(str[0], Integer.valueOf(str[1]));
                        // 连接超时时间（默认10秒 10000ms） 单位毫秒（ms）
                        int connectionTimeout = 10000;
                        // 读取数据超时时间（默认30秒 30000ms） 单位毫秒（ms）
                        int soTimeout = 30000;
                        httpClient.getHttpConnectionManager().getParams().setConnectionTimeout(connectionTimeout);
                        httpClient.getHttpConnectionManager().getParams().setSoTimeout(soTimeout);

                        String httpGet = HttpClientUtil.httpGet(BAIDU_PATH, new HashMap<>());
                        log.info(httpGet);
                        statusCode = httpGet.indexOf("html");
                    } catch (Exception e) {
                        map.put("status", 500);
                        map.put("msg", "连接超时");
                    }
                    if (statusCode > 0) {
                        map.put("status", 0);
                        map.put("msg", "连接成功");
                    }
                    list.add(map);
                }
            }
        }
        json.put("status", 0);
        json.put("data", list);
        return json;
    }

    public static String getRandomIp() {
        // ip范围
        int[][] range = {{607649792, 608174079}, // 36.56.0.0-36.63.255.255
                {1038614528, 1039007743}, // 61.232.0.0-61.237.255.255
                {1783627776, 1784676351}, // 106.80.0.0-106.95.255.255
                {2035023872, 2035154943}, // 121.76.0.0-121.77.255.255
                {2078801920, 2079064063}, // 123.232.0.0-123.235.255.255
                {-1950089216, -1948778497}, // 139.196.0.0-139.215.255.255
                {-1425539072, -1425014785}, // 171.8.0.0-171.15.255.255
                {-1236271104, -1235419137}, // 182.80.0.0-182.92.255.255
                {-770113536, -768606209}, // 210.25.0.0-210.47.255.255
                {-569376768, -564133889}, // 222.16.0.0-222.95.255.255
        };

        Random rdint = new Random();
        int index = rdint.nextInt(10);
        String ip = num2ip(range[index][0] + new Random().nextInt(range[index][1] - range[index][0]));
        return ip;
    }

    public static String num2ip(int ip) {
        int[] b = new int[4];
        String x = "";

        b[0] = (int) ((ip >> 24) & 0xff);
        b[1] = (int) ((ip >> 16) & 0xff);
        b[2] = (int) ((ip >> 8) & 0xff);
        b[3] = (int) (ip & 0xff);
        x = Integer.toString(b[0]) + "." + Integer.toString(b[1]) + "." + Integer.toString(b[2]) + "."
                + Integer.toString(b[3]);

        return x;
    }
}

因为用到了时间转换和 httpClient请求  所以我也把这两个工具类贴出来 方便使用

httpClientUtil

package com.sunshine.http_proxy.utils;

import java.io.*;

import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;

import java.util.Iterator;

import java.util.List;

import java.util.Map;


import org.apache.http.HttpEntity;

import org.apache.http.NameValuePair;

import org.apache.http.client.config.CookieSpecs;

import org.apache.http.client.config.RequestConfig;

import org.apache.http.client.entity.UrlEncodedFormEntity;

import org.apache.http.client.methods.CloseableHttpResponse;

import org.apache.http.client.methods.HttpGet;

import org.apache.http.client.methods.HttpPost;

import org.apache.http.entity.StringEntity;

import org.apache.http.impl.client.CloseableHttpClient;

import org.apache.http.impl.client.HttpClients;

import org.apache.http.message.BasicNameValuePair;

import org.apache.http.util.EntityUtils;

public class HttpClientUtil {
    public static String httpGet(String url, Map<String, String> params) {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        Iterator<String> keysIte = params.keySet().iterator();
        int index = 0;
        while (keysIte.hasNext()) {
            String key = keysIte.next();
            if (index == 0) {
                url = url + "?" + key + "=" + params.get(key);
            } else {
                url = url + "&" + key + "=" + params.get(key);
            }
            index++;
        }
        HttpGet httpget = new HttpGet(url);
        CloseableHttpResponse response = null;
        StringBuilder result = new StringBuilder();
        // 设置请求和传输超时时间5s,设置cookie策略
        RequestConfig requestconfig = RequestConfig.custom()
                .setSocketTimeout(5000).setConnectTimeout(5000)
                .setCookieSpec(CookieSpecs.BEST_MATCH).build();
        httpget.setConfig(requestconfig);
        try {
            response = httpClient.execute(httpget);
            HttpEntity entity = response != null ? response.getEntity() : null;
            if (entity != null) {
                InputStream in = entity.getContent();
                BufferedReader br = new BufferedReader(new InputStreamReader(
                        in, "UTF-8"));
                String line;
                while ((line = br.readLine()) != null) {
                    result.append(line);
                    result.append("\n");
                }
                in.close();
                br.close();
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (response != null) response.close();
                httpClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return result.toString();
    }

    public static String httpPost(String url, String body) {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        HttpPost httpPost = new HttpPost(url);
        CloseableHttpResponse response = null;
        String result = null;
        try {
            StringEntity stringEntity = new StringEntity(body, "UTF-8");
            httpPost.setEntity(stringEntity);
            response = httpClient.execute(httpPost);
            HttpEntity entity = response != null ? response.getEntity() : null;
            if (entity != null) {
                result = EntityUtils.toString(entity, "UTF-8");
            }
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (response != null) response.close();
                httpClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return result;
    }

    public static String httpPost(String url, Map<String, String> params) {
        CloseableHttpClient httpClient = HttpClients.createDefault();
        HttpPost httpPost = new HttpPost(url);
        CloseableHttpResponse response = null;
        String result = null;
        try {
            List<NameValuePair> nvps = new ArrayList<NameValuePair>();
            for (String key : params.keySet()) {
                nvps.add(new BasicNameValuePair(key, params.get(key)));
            }
            httpPost.setEntity(new UrlEncodedFormEntity(nvps, "UTF-8"));
            response = httpClient.execute(httpPost);
            HttpEntity entity = response != null ? response.getEntity() : null;
            if (entity != null) {
                result = EntityUtils.toString(entity, "UTF-8");
            }
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (response != null) response.close();
                httpClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return result;
    }

    public static String getBodyString(BufferedReader br) {
        String inputLine;
        String str = "";
        try {
            while ((inputLine = br.readLine()) != null) {
                str += inputLine;
            }
            br.close();
        } catch (IOException e) {
            System.out.println("IOException: " + e);
        }
        return str;
    }

    /**
     * 设置url以及键值对参数
     *
     * @param url
     * @param params
     * @return
     */
    public static String getUrlWithParams(String url, Map<String, String> params) {
        Iterator<String> keysIte = params.keySet().iterator();
        int index = 0;
        while (keysIte.hasNext()) {
            String key = keysIte.next();
            if (index == 0) {
                url = url + "?" + key + "=" + params.get(key);
            } else {
                url = url + "&" + key + "=" + params.get(key);
            }
            index++;
        }
        return url;
    }

    /**
     * 按照一定顺序设置url以及键值对参数
     *
     * @param url
     * @param params
     * @param orderList
     * @return
     */
    public static String getUrlWithParams(String url, Map<String, String> params, List<String> orderList) {
        int index = 0;
        for (String key : orderList) {
            if (index == 0) {
                url = url + "?" + key + "=" + params.get(key);
            } else {
                url = url + "&" + key + "=" + params.get(key);
            }
            index++;
        }
        return url;
    }

    /**
     * @param urlStr   请求的地址
     * @param content  请求的参数 格式为：name=xxx&pwd=xxx
     * @param encoding 服务器端请求编码。如GBK,UTF-8等
     * @return
     */
    public static String getResult(String urlStr, String content, String encoding) {
        URL url = null;
        HttpURLConnection connection = null;
        try {
            url = new URL(urlStr);
            connection = (HttpURLConnection) url.openConnection();// 新建连接实例
            connection.setConnectTimeout(2000);// 设置连接超时时间，单位毫秒
            connection.setReadTimeout(2000);// 设置读取数据超时时间，单位毫秒
            connection.setDoOutput(true);// 是否打开输出流 true|false
            connection.setDoInput(true);// 是否打开输入流true|false
            connection.setRequestMethod("GET");// 提交方法POST|GET
            connection.setUseCaches(false);// 是否缓存true|false
            connection.connect();// 打开连接端口
            DataOutputStream out = new DataOutputStream(connection
                    .getOutputStream());// 打开输出流往对端服务器写数据
            out.writeBytes(content);// 写数据,也就是提交你的表单 name=xxx&pwd=xxx
            out.flush();// 刷新
            out.close();// 关闭输出流
            BufferedReader reader = new BufferedReader(new InputStreamReader(
                    connection.getInputStream(), encoding));// 往对端写完数据对端服务器返回数据
            // ,以BufferedReader流来读取
            StringBuffer buffer = new StringBuffer();
            String line = "";
            while ((line = reader.readLine()) != null) {
                buffer.append(line);
            }
            reader.close();
            return buffer.toString();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (connection != null) {
                connection.disconnect();// 关闭连接
            }
        }
        return null;
    }
}

DateUtil

/**
 * 时间操作工具
 *
 * @author xaingzi
 *
 */

package com.sunshine.http_proxy.utils;

import org.apache.commons.lang3.StringUtils;

import javax.xml.datatype.DatatypeConfigurationException;
import javax.xml.datatype.DatatypeFactory;
import javax.xml.datatype.XMLGregorianCalendar;
import java.sql.Timestamp;
import java.text.DateFormatSymbols;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;

/**
 * 日期转换工具
 */
public class DateUtil {
   public static final String DATE_DIVISION = "-";
   /**
    * yyyy-MM-dd HH:mm:ss
    */
   public static final String TIME_PATTERN_DEFAULT = "yyyy-MM-dd HH:mm:ss";

   public static final String TIME_PATTERN_DEFAULT_NO_SS = "yyyy-MM-dd HH:mm";
   /**
    * yyyy-MM-dd
    */
   public static final String DATE_PATTERN_DEFAULT = "yyyy-MM-dd";
   /**
    * yyyyMMdd
    */
   public static final String DATA_PATTERN_YYYYMMDD = "yyyyMMdd";
   /**
    * HH:mm:ss
    */
   public static final String TIME_PATTERN_HHMMSS = "HH:mm:ss";

   /**
    * yyyy-MM-dd HH-mm-ss
    */
   public static final String TIME_PATTERN = "yyyy-MM-dd HH-mm-ss";

   /**
    * yyyyMMddHHmmss
    */
   public static final String TIME_PATTERN_DEFAULT_STR = "yyyyMMddHHmmss";

   public static final String TIME_PATTERN_SSSS_STR = "yyyyMMddHHmmssSSSS";

   public static final int SECOND = 1000;
   public static final int MINUTE = 60 * SECOND;
   public static final int HOUR = 60 * MINUTE;
   public static final long DAY = 24l * HOUR;

   /**
    * Return the current date
    * 
    * @return － DATE<br>
    */
   public static Date now() {
      Calendar cal = Calendar.getInstance();
      Date currDate = cal.getTime();

      return currDate;
   }

   public static Timestamp nowTimestamp() {
      Calendar cal = Calendar.getInstance();
      return new Timestamp(cal.getTimeInMillis());
   }
   /**
    * 给一个时间加上N分种或减去N分种后得到一个新的日期
    *
    * @param startDate
    *            需要增加的日期时间
    * @param addnos
    *            添加的分钟数，可以是正数也可以是负数
    * @return 操作后的日期
    */
   public static Date addMinute(Date startDate, int addnos) {
      if (startDate == null) {
         return null;
      }
      Calendar cc = Calendar.getInstance();
      cc.setTime(startDate);
      cc.add(Calendar.MINUTE, addnos);
      return cc.getTime();
   }
   /**
    * Return the current date string
    * 
    * @return － 产生的日期字符串<br>
    */
   public static String nowString() {
      Calendar cal = Calendar.getInstance();
      Date currDate = cal.getTime();

      return formatDate(currDate);
   }

   /**
    * Return the current date in the specified format
    * 
    * @param strFormat
    * @return
    */
   public static String nowString(String pattern) {
      Calendar cal = Calendar.getInstance();
      Date currDate = cal.getTime();

      return format(currDate, pattern);
   }

   public static Timestamp parseTimestampDefault(String dateValue) {
      return new Timestamp(parse(dateValue, TIME_PATTERN_DEFAULT).getTime());
   }

   public static Timestamp parseTimestamp(String dateValue) {
      return new Timestamp(parse(dateValue, TIME_PATTERN_HHMMSS).getTime());
   }

   public static Timestamp parseTimestampYYMMDD(String dateValue) {
      return new Timestamp(parse(dateValue, DATE_PATTERN_DEFAULT).getTime());
   }
   
   public static Timestamp parseTimestamp(String dateValue,String partten) {
      return new Timestamp(parse(dateValue, partten).getTime());
   }

   public static Timestamp parseTimeNoStamp(String dateValue) {
      if(null==dateValue||dateValue.trim().length()==0) {
         dateValue=getNowDateString();  
      }
      return new Timestamp(parse(dateValue, DATE_PATTERN_DEFAULT).getTime());

   }

   public static String getNowDateString() {
      return format(new Date(),DATE_PATTERN_DEFAULT);
   }

   /**
    * Parse a string and return a date value
    * 
    * @param dateValue
    * @return
    * @throws Exception
    */
   public static Date parseDate(String dateValue) {
      return parse(dateValue, DATE_PATTERN_DEFAULT);
   }

   /**
    * Parse a strign and return a datetime value
    * 
    * @param dateValue
    * @return
    */
   public static Date parseTime(String dateValue) {
      return parse(dateValue, TIME_PATTERN_DEFAULT);
   }

   /**
    * Parse a string and return the date value in the specified format
    * 
    * @param strFormat
    * @param dateValue
    * @return
    * @throws ParseException
    * @throws Exception
    */
   public static Date parse(String dateValue, String pattern) {
      if (StringUtils.isEmpty(dateValue))
         return null;

      SimpleDateFormat dateFormat = new SimpleDateFormat(pattern);

      try {
         return dateFormat.parse(dateValue);
      } catch (ParseException pe) {
         return null;
      }
   }

   /**
    * 将Timestamp类型的日期转换为系统参数定义的格式的字符串。
    * 
    * @param aTs_Datetime
    *            需要转换的日期。
    * @return 转换后符合给定格式的日期字符串
    */
   public static String formatDate(Date d) {
      return format(d, DATE_PATTERN_DEFAULT);
   }

   public static String formatDate(Timestamp d) {
      return format(d, DATE_PATTERN_DEFAULT);
   }

   
   
   public static String formatDateddmmss(Date d) {
      return format(d, TIME_PATTERN_DEFAULT);
   }
   
   /**
    * 将Timestamp类型的日期转换为系统参数定义的格式的字符串(只有数字)。
    * 
    * @param aTs_Datetime
    *            需要转换的日期。
    * @return 转换后符合给定格式的日期字符串
    */
   public static String formatTimeTargetStr(Date t) {
      String timestr = null;
      timestr = format(t, TIME_PATTERN_DEFAULT_STR);
      return timestr;
   }

   /**
    * 将Timestamp类型的日期转换为系统参数定义的格式的字符串(只有数字)。
    * 
    * @param aTs_Datetime
    *            需要转换的日期。
    * @return 转换后符合给定格式的日期字符串
    */
   public String formatTimeStr(Date t) {
      String timestr = null;
      synchronized (this) {
         timestr = format(t, TIME_PATTERN_DEFAULT_STR);
      }
      return timestr;
   }

   /**
    * 豪秒 将Timestamp 类型的日期转换为系统参数定义的格式的字符串(只有数字)。
    * 
    * @param aTs_Datetime
    *            需要转换的日期。
    * @return 转换后符合给定格式的日期字符串
    */
   public String formatTimeSSSSStr(Date t) {
      String timestr = null;
      synchronized (this) {
         timestr = format(t, TIME_PATTERN_SSSS_STR);
      }
      return timestr;
   }

   /**
    * 将Timestamp类型的日期转换为系统参数定义的格式的字符串。
    * 
    * @param aTs_Datetime
    *            需要转换的日期。
    * @return 转换后符合给定格式的日期字符串
    */
   public static String formatTime(Date t) {
      return format(t, TIME_PATTERN_DEFAULT);
   }

   /**
    * 将Date类型的日期转换为系统参数定义的格式的字符串。
    * 
    * @param aTs_Datetime
    * @param as_Pattern
    * @return
    */
   public static String format(Date d, String pattern) {
      if (d == null)
         return null;

      SimpleDateFormat dateFromat = new SimpleDateFormat(pattern);
      return dateFromat.format(d);
   }
   
   /**
    * 将Timestamp类型的日期转换为系统参数定义的格式的字符串。
    * 
    * @param t
    * @param pattern
    * @return
    */
   public static String format(Timestamp t, String pattern) {
      if (t == null)
         return null;

        Date date = new Date(t.getTime());
        
        return DateUtil.format(date, pattern);
   }
   
   /**
    * 将Timestamp类型的日期转换Date类型。
    * 
    * @param t
    * @return
    */
   public static Date timestamp2Date(Timestamp t) {
      if (t == null)
         return null;

        return new Date(t.getTime());
   }

   /**
    * 取得指定日期N天后的日期
    * 
    * @param date
    * @param days
    * @return
    */
   public static Date addDays(Date date, int days) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);

      cal.add(Calendar.DAY_OF_MONTH, days);

      return cal.getTime();
   }

   /**
    * 计算两个日期之间相差的天数
    * 
    * @param date1
    *            start
    * @param date2
    *            end
    * @return
    */
   public static int daysBetween(Date date1, Date date2) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      long time1 = cal.getTimeInMillis();
      cal.setTime(date2);
      long time2 = cal.getTimeInMillis();
      long between_days = (time2 - time1) / (1000 * 3600 * 24);

      return Integer.parseInt(String.valueOf(between_days));
   }

   /**
    * 计算两个日期之间相差的月数 ,请用 后面参数-前面参数
    * 
    * @param date1
    * @param date2
    * @return
    */
   public static int MonthsBetween(Date date1, Date date2) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      int year1 = cal.get(Calendar.YEAR);
      int month1 = cal.get(Calendar.MONTH);
      cal.setTime(date2);
      int year2 = cal.get(Calendar.YEAR);
      int month2 = cal.get(Calendar.MONTH);

      int m = (year2 - year1) * 12 + (month2 - month1);

      return m;
   }

   /**
    * 计算开始时间到月末与结束时间到月初之和
    * 
    * @param StarDate
    * @param EndDate
    * @return day
    */
   public static int getDayByStarDateEndDate(Date StarDate, Date EndDate) {
      int dayStar = daysBetween(StarDate, getFirstDateOfNextMonth(StarDate));

      int dayEnd = daysBetween(getFirstDateOfCurrentMonth(EndDate), EndDate);

      int day = dayStar + dayEnd;
      if (day > 30) {
         day = 30;
      }
      return day;
   }

   /**
    * 计算两个相同天相差的天数
    * 
    * @param date1
    * @param date2
    * @return
    */
   public static int DaysBetween(Date date1, Date date2) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      int day1 = cal.get(Calendar.DATE);
      cal.setTime(date2);
      int day2 = cal.get(Calendar.DATE);

      int d = day2 - day1;
      if (d > 30) {
         d = 30;
      }
      if (d < -30) {
         d = -30;
      }
      return d;
   }

   /**
    * 判断两个日期是否为同月
    * 
    * @param date1
    * @param date2
    * @return
    */
   public static boolean booleanBetweenDate(Date date1, Date date2) {
      boolean falg = false;
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      int month1 = cal.get(Calendar.MONTH);
      cal.setTime(date2);
      int month2 = cal.get(Calendar.MONTH);
      int m = month2 - month1;
      if (m == 0) {
         falg = true;
      }
      return falg;
   }

   /**
    * 计算两个相同月相差的天数
    * 
    * @param date1
    * @param date2
    * @return
    */
   public static int MonthCountBetween(Date date1, Date date2) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      int year1 = cal.get(Calendar.YEAR);
      int month1 = cal.get(Calendar.MONTH);
      cal.setTime(date2);
      int year2 = cal.get(Calendar.YEAR);
      int month2 = cal.get(Calendar.MONTH);
      int m = month2 - month1;
      int y = year2 - year1;

      return y * 12 + m;
   }

   public static Date getDateBeforTwelveMonth() {
      Calendar cal = Calendar.getInstance();
      cal.set(Calendar.DAY_OF_MONTH, 1);
      cal.set(Calendar.HOUR_OF_DAY, 0);
      cal.set(Calendar.MINUTE, 0);
      cal.set(Calendar.SECOND, 0);
      cal.add(Calendar.MONTH, -12);

      return cal.getTime();

   }

   /**
    * 传入时间字符串,加一天后返回Date
    * 
    * @param date
    *            时间 格式 YYYY-MM-DD
    * @return
    */
   public static Date addDate(String date) {
      if (date == null)
         return null;

      Date tmpDate = parse(date, DATE_PATTERN_DEFAULT);

      Calendar cal = Calendar.getInstance();
      cal.setTime(tmpDate);
      cal.add(Calendar.DAY_OF_MONTH, 1);

      return cal.getTime();
   }


   /**
    * 得到某个日期的星期
    * 
    * @param dt
    * @return Day of the week (0 - sunday)
    */
   public static int getWeekOfDate(Date dt) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(dt);

      int w = cal.get(Calendar.DAY_OF_WEEK) - 1;
      if (w < 0)
         w = 0;

      return w;
   }

   /**
    * 得某(月，季度，上半年，下半年，全年)第1天日期
    */
   public static Date getCurrentMonFirstDate(int month) {
      Calendar cal = Calendar.getInstance();
      if (month == 13) {
         cal.setTime(now());
         cal.set(Calendar.DAY_OF_YEAR, 1);
      } else if (month == 14 || month == 17) {
         cal.set(Calendar.MONTH, 5);
         cal.set(Calendar.DATE, 1);
      } else if (month == 15 || month == 19) {
         cal.set(Calendar.MONTH, 11);
         cal.set(Calendar.DATE, 1);
      } else if (month == 16) {
         cal.set(Calendar.MONTH, 2);
         cal.set(Calendar.DATE, 1);
      } else if (month == 18) {
         cal.set(Calendar.MONTH, 8);
         cal.set(Calendar.DATE, 1);
      } else {
         cal.set(Calendar.MONTH, (month - 1));
         cal.set(Calendar.DATE, 1);
      }
      return cal.getTime();
   }

   /**
    * 得某(月，季度，上半年，下半年，全年)最后1天日期
    */
   public static Date getCurrentMouLastDate(int month) {
      Calendar cal = Calendar.getInstance();
      if (month == 13) {
         cal.setTime(now());
         cal.set(Calendar.DAY_OF_YEAR, 1);
         cal.roll(Calendar.DAY_OF_YEAR, -1);
      } else if (month == 14 || month == 17) {
         cal.set(Calendar.MONTH, 5);
         cal.set(Calendar.DATE, 1);
         cal.roll(Calendar.DATE, -1);
      } else if (month == 15 || month == 19) {
         cal.set(Calendar.MONTH, 11);
         cal.set(Calendar.DATE, 1);
         cal.roll(Calendar.DATE, -1);
      } else if (month == 16) {
         cal.set(Calendar.MONTH, 2);
         cal.set(Calendar.DATE, 1);
         cal.roll(Calendar.DATE, -1);
      } else if (month == 18) {
         cal.set(Calendar.MONTH, 8);
         cal.set(Calendar.DATE, 1);
         cal.roll(Calendar.DATE, -1);
      } else {
         cal.set(Calendar.MONTH, (month - 1));
         cal.set(Calendar.DATE, 1);
         cal.roll(Calendar.DATE, -1);
      }
      return cal.getTime();
   }

   /**
    * 得到当月前几个月的列表 yyyy-mm-01
    * 
    * @return
    */
   public static List<String> getBeforeMonth(int number) {
      List<String> list = new ArrayList<String>();
      Calendar cal = Calendar.getInstance();
      int month = cal.get(Calendar.MONTH);
      int year = cal.get(Calendar.YEAR);
      for (int i = 0; i < number; i++) {
         String yearMonth = new String();

         yearMonth = year + "-" + (month + 1);
         list.add(yearMonth);
         if (month == 0) {
            year--;
            month = 11;
            cal.set(Calendar.YEAR, year);
            cal.set(Calendar.MONTH, month);
         } else {
            month--;
            cal.set(Calendar.MONTH, month);
         }
      }
      return list;
   }

   /**
    * 查出某月的最后一天
    * 
    * @param date
    *            Date
    * @return lastDayofMonthStr
    */
   public static String getLastDayofMonth(Date date) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      int month = cal.get(Calendar.MONTH) + 1;
      cal.set(Calendar.MONTH, month);
      cal.set(Calendar.DAY_OF_MONTH, 1);
      long lastTime = cal.getTimeInMillis() - (1000 * 3600 * 24);
      cal.setTimeInMillis(lastTime);
      return format(cal.getTime(), DATE_PATTERN_DEFAULT);
   }

   /**
    * 查出上月的最后一天
    * 
    * @param date
    *            Date
    * @return lastDayofMonthStr
    */
   public static String getLastDayofMonthBeforeDate(Date date) {
      if (date == null) {
         date = now();
      }
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month);
      cal.set(Calendar.DAY_OF_MONTH, 1);
      long lastTime = cal.getTimeInMillis() - (1000 * 3600 * 24);
      cal.setTimeInMillis(lastTime);
      return format(cal.getTime(), DATE_PATTERN_DEFAULT);
   }

   /**
    * 返回当前日期
    * 
    * @return Date
    * */
   public static Date getCurrentDate() {
      return parseDate(formatDate(now()));
   }

   /**
    * 返回当前日期
    * 
    * @return Date
    * */
   public static String getCurrentDateStr() {
      return format(getCurrentDate(), DATE_PATTERN_DEFAULT);
   }
   /**
    * 返回当前日期
    * 
    * @return Date
    * */
   public static String getCurrentDateStr2() {
      return format(getCurrentDate(), TIME_PATTERN_DEFAULT);
   }
   /**
    * 生成字符日期类型，无分隔符
    * 
    * @param date
    * @param days
    * @return
    */
   public static String parseDate(Date date) {
      String name = "";
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      Integer year = cal.get(Calendar.YEAR);
      Integer month = cal.get(Calendar.MONTH) + 1;
      Integer day = cal.get(Calendar.DATE);
      String syear = year.toString();
      String smonth;
      String sday;
      if (month.intValue() < 10)
         smonth = "0" + month.toString();
      else
         smonth = month.toString();

      if (day.intValue() < 10)
         sday = "0" + day.toString();
      else
         sday = day.toString();
      name = syear + smonth + sday;

      return name;
   }

   /**
    * 根据指定日期月份加n且取下个月,得到个新日期
    * 
    * @param date
    * @return date
    */
   public static Date getDateForMonthAddNew(java.util.Date date, int month) {
      java.util.Calendar c = java.util.Calendar.getInstance();
      c.setTime(date);
      c.set(c.get(java.util.Calendar.YEAR), c.get(java.util.Calendar.MONTH)
            + month, c.get(java.util.Calendar.DATE));
      Date nextDate = c.getTime();
      return nextDate;
   }

   /**
    * 得到某天当月第一天日期
    * 
    * @param dateOfMonth
    * @return
    */
   public static Date getFirstDateOfCurrentMonth(Date dateOfMonth) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(dateOfMonth);
      cal.set(Calendar.DAY_OF_MONTH, 1);

      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 得到某天当月某一天号
    * 
    * @param dateOfMonth
    * @param theDay
    *            到某天
    * @return
    */
   public static Date getTheDateOfCurrentMonth(Date dateOfMonth, int theDay) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(dateOfMonth);
      cal.set(Calendar.DAY_OF_MONTH, theDay);

      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 得到某天下个月第一天日期
    * 
    * @param dateOfMonth
    * @return
    */
   public static Date getFirstDateOfNextMonth(Date dateOfMonth) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(dateOfMonth);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month + 1);
      cal.set(Calendar.DAY_OF_MONTH, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 得到某天下下个月第一天日期
    * 
    * @param dateOfMonth
    * @return
    */
   public static Date getFirstDateOfNextTwoMonth(Date dateOfMonth) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(dateOfMonth);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month + 2);
      cal.set(Calendar.DAY_OF_MONTH, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 取得某日m个月后得日子
    * 
    * @param beforeDate
    * @param m
    * @return afterDate
    */
   public static Date getDateAfterCountMonth(Date beforeDate, int m) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(beforeDate);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month + m);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 取得某日m个月后得日子(改进版)
    * 
    * @param beforeDate
    * @param m
    * @return afterDate
    */
   public static Date getDateAfterCountMonthByCal(Date beforeDate, int m) {

      Calendar cal = Calendar.getInstance();
      cal.setTime(beforeDate);
      int month = cal.get(Calendar.MONTH);
      int day = cal.get(Calendar.DAY_OF_MONTH);
      // 判断是否为月末
      boolean isLastDay = isLastDayOfMonth(beforeDate);
      if (isLastDay) {
         // 是 返回m+1个月初
         cal.set(Calendar.MONTH, month + m + 1);
         cal.set(Calendar.DAY_OF_MONTH, 1);
         return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
      } else {
         // 否
         cal.set(Calendar.MONTH, month + m);
         Date newDate = parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));

         Calendar newCal = Calendar.getInstance();
         newCal.setTime(newDate);
         int newDay = newCal.get(Calendar.DAY_OF_MONTH);
         // 判断 m 个后,相同号是否存在，如1月30日之后的1个月没有30日
         if (newDay == day) {
            // 如果存在，直接返回
            return newDate;
         } else {
            // 如果不存在，说明已经被推到下个月了（3月2日），直接该月初,如3月1日
            return getFirstDateOfCurrentMonth(newDate);
         }
      }

   }

   /**
    * 取得某日m个月前得日子
    * 
    * @param beforeDate
    * @param m
    * @return afterDate
    */
   public static Date getDateBeforeCountMonth(Date beforeDate, int m) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(beforeDate);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month - m);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 取得某年第1天时期
    * 
    * @param
    * @param
    * @return startDate
    */
   public static String getDateStratYearMonthDay() {
      Calendar cal = Calendar.getInstance();
      cal.setTime(now());
      cal.set(Calendar.DAY_OF_YEAR, 1);
      return format(cal.getTime(), DATE_PATTERN_DEFAULT);
   }

   /**
    * 取得某年下一年第1天时期
    * 
    * @param
    * @param
    * @return endDate
    */
   public static String getDateNextYearMonthDay() {
      Calendar cal = Calendar.getInstance();
      cal.setTime(now());
      int year = cal.get(Calendar.YEAR);
      cal.set(Calendar.YEAR, year + 1);
      cal.set(Calendar.DAY_OF_YEAR, 1);
      return format(cal.getTime(), DATE_PATTERN_DEFAULT);
   }

   /**
    * 根据计算日期，取得下个季度的第一天
    * 
    * @param date
    * @return
    */
   public static Date getFirstDateOfNextSeason(Date date) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      int month = cal.get(Calendar.MONTH);
      int year = cal.get(Calendar.YEAR);
      int returnMonth = 0;
      int returnYear = year;
      switch (month) {
      case 0:
         returnMonth = 3;
         break;
      case 1:
         returnMonth = 3;
         break;
      case 2:
         returnMonth = 3;
         break;
      case 3:
         returnMonth = 6;
         break;
      case 4:
         returnMonth = 6;
         break;
      case 5:
         returnMonth = 6;
         break;
      case 6:
         returnMonth = 9;
         break;
      case 7:
         returnMonth = 9;
         break;
      case 8:
         returnMonth = 9;
         break;
      case 9:
         returnMonth = 0;
         returnYear = year + 1;
         break;
      case 10:
         returnMonth = 0;
         returnYear = year + 1;
         break;
      case 11:
         returnMonth = 0;
         returnYear = year + 1;
         break;
      }
      cal.set(Calendar.MONTH, returnMonth);
      cal.set(Calendar.YEAR, returnYear);
      cal.set(Calendar.DATE, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 得到某天的上一年的同天日期
    * 
    * @param dateOfMonth
    * @return
    */
   public static Date getDateOfPreviousYear(Date date) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      int year = cal.get(Calendar.YEAR);
      cal.set(Calendar.YEAR, year - 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 得到某天的年初日期
    * 
    * @param dateOfMonth
    * @return
    */
   public static Date getFirstYearDate(Date date) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      cal.set(Calendar.MONTH, 0);
      cal.set(Calendar.DATE, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 取得上个月的同天日期
    */
   public static Date getDateOfPreviousMonth(Date date) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month - 1);
      cal.set(Calendar.DATE, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   /**
    * 判断给定日期是否为月末的一天
    * 
    * @param date
    * @return true:是|false:不是
    */
   public static boolean isLastDayOfMonth(Date date) {
      Calendar calendar = Calendar.getInstance();
      calendar.setTime(date);
      calendar.set(Calendar.DATE, (calendar.get(Calendar.DATE) + 1));
      if (calendar.get(Calendar.DAY_OF_MONTH) == 1) {
         return true;
      }
      return false;
   }

   /**
    * 判断是否月初
    * 
    * @param date
    * @return
    */
   public static boolean isFirstDayOfMonth(Date date) {
      Calendar calendar = Calendar.getInstance();
      calendar.setTime(date);
      if (calendar.get(Calendar.DAY_OF_MONTH) == 1) {
         return true;
      }
      return false;
   }

   /**
    * yyyy-MM-dd HH-mm-ss
    * 
    * @return
    */

   /**
    * 取得某日m个月后月初的日子
    * 
    * @param beforeDate
    * @param m
    * @return afterDate
    */
   public static Date getBeginDateAfterCountMonth(Date beforeDate, int m) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(beforeDate);
      int month = cal.get(Calendar.MONTH);
      cal.set(Calendar.MONTH, month + m);
      cal.set(Calendar.DATE, 1);
      return parseDate(format(cal.getTime(), DATE_PATTERN_DEFAULT));
   }

   public static String getDateMinute() {
      return nowString(TIME_PATTERN);
   }

   /**
    * 取得当前日期的下季度第一天
    * 
    * @param args
    */
   public static Date getNextQuarterFirst(Date nowDate) {
      Calendar cal = Calendar.getInstance();
      cal.setTime(nowDate);
      int month = cal.get(Calendar.MONTH);
      int times = 3 - month % 3;
      return getDateAfterCountMonth(nowDate, times);

   }

   /**
    * @Description TODO(获取当天剩余分钟数)
    * @Author lvhaosir 2019/2/14 19:19
    * @Param
    * @return
    */
   public static long getDayRemaining(String claimGoodTime) {
      // 传入的时间为 YYYY-MM-dd 需改变为 yyyyMMdd
      claimGoodTime = claimGoodTime.replaceAll("-","");
      // 使用预定的 日期+ 235959  减去 当前时间
      String newYmd = claimGoodTime + "235959";
      String ymdhms = DateUtil.formatTimeTargetStr(new Date());
      System.out.println(ymdhms);
      System.out.println(" 净菜自提预定时间 >>> "+newYmd);
      System.out.println(" 净菜自提当前时间 >>> "+ymdhms);
      // 预定时间
      Date newYmdData = DateUtil.parse(newYmd, "yyyyMMddHHmmss");
      Date ymdhmsData = DateUtil.parse(ymdhms, "yyyyMMddHHmmss");
      long diff = newYmdData.getTime() - ymdhmsData.getTime();
      long days = diff / (1000 * 60 * 60 * 24);
      long hours = (diff-days*(1000 * 60 * 60 * 24))/(1000* 60 * 60);
      long minutes = (diff-days*(1000 * 60 * 60 * 24)-hours*(1000* 60 * 60))/(1000* 60);
      System.out.println(""+days+"天"+hours+"小时"+minutes+"分");
      long time = hours * 60 + minutes;
      System.out.println(time);
      return time;
   }

   /**
    * @Description TODO(获取传入时间与当前时间相差的小时数)
    * @Author lvhaosir 2019/4/2 17:43
    * @param startTime
    */
   public static long getDayRemainHours(Timestamp startTime) {

      // System.out.println(" 传入的时间为 >>> "+startTime);
      Timestamp nowTimestamp = DateUtil.nowTimestamp();
      // System.out.println(" 当前时间为 >>> "+nowTimestamp);
      // 预定时间

      long diff = startTime.getTime() - nowTimestamp.getTime();
      if (diff > 0) {
         long days = diff / (1000 * 60 * 60 * 24);
         long hours = (diff-days*(1000 * 60 * 60 * 24))/(1000* 60 * 60);
         return hours;
      }
      // long minutes = (diff-days*(1000 * 60 * 60 * 24)-hours*(1000* 60 * 60))/(1000* 60);
      // System.out.println(""+days+"天"+hours+"小时"+minutes+"分");
      // long time = hours * 60 + minutes;
      // System.out.println(time);
      return -1;
   }

   /*public static void main(String[] args) {

      String offWorkDate = "2019-03-15-00:00";
      System.out.println(offWorkDate.substring(0,10));




   }*/

   public static List<String> getResultTableName(Date date1, Date date2) {
      List<String> list = new ArrayList<String>();
      int daysBetween = daysBetween(date1, date2);
      if (daysBetween < 0) {
         return list;
      }
      Calendar cal = Calendar.getInstance();
      cal.setTime(date1);
      int month = cal.get(Calendar.MONTH) + 1;
      int year = cal.get(Calendar.YEAR);
      String result = "";
      //判断两个日期是否为同月,如果是直接返回当前月。
      boolean b = booleanBetweenDate(date1, date2);
      if (b) {
         if (month < 10) {
            result = "" + year + 0 + month;
         } else {
            result = "" + year + month;
         }
         list.add(result);
         return list;
      }

      int MonthsBetween = MonthsBetween(date1, date2);
      Calendar cal2 = Calendar.getInstance();
      cal2.setTime(date2);
      for (int i = 0; i <= MonthsBetween; i++) {
         if (month <= 12) {
            if ((month + i) < 10) {
               result = "" + year + 0 + (month + i);
            } else {
               result = "" + year + (month + i);
            }
         } else {
            if ((month + i - 12) < 10) {
               result = "" + (year + 1) + (month + i - 12);
            } else {
               result = "" + (year + 1) + (month + i - 12);
            }
         }
         list.add(result);
      }
      return list;
   }
   
   /**
    * 得到当天的后二天的日期
    * 
    * @param day 
    * @return
    */
   public static String getCurrentDateAfterDay(int day) {
      Calendar cal = Calendar.getInstance();
      long lastTime = cal.getTimeInMillis() + (1000 * 3600 * 24 * day);
      cal.setTimeInMillis(lastTime);
      return format(cal.getTime(), DATE_PATTERN_DEFAULT);
   }
   
   /**
    * 获取当前系统时间的星期数。
    * <p>
    * 结果显示格式根据系统语言环境确定，如zh_CN时显示为星期一
    * </p>
    * 
    * @return
    */
   public static String getLocaleDayOfWeek() {
      Locale usersLocale = Locale.getDefault();
      DateFormatSymbols dfs = new DateFormatSymbols(usersLocale);
      String weekdays[] = dfs.getWeekdays();
      return weekdays[Calendar.getInstance().get(Calendar.DAY_OF_WEEK)];
   }
   
   /**
    * 获取友好时间
    * @param datetime 格式：2012-8-21 17:53:20
    * @return
    */
   public static String getFriendTime(Date datetime) { // 传入的时间格式必须类似于2012-8-21 17:53:20这样的格式

      if(datetime == null){
         return "";
      }
      String interval = null;

      Date d1 = datetime;

      // 用现在距离1970年的时间间隔new
      // Date().getTime()减去以前的时间距离1970年的时间间隔d1.getTime()得出的就是以前的时间与现在时间的时间间隔
      long time = new Date().getTime() - d1.getTime();// 得出的时间间隔是毫秒

      if (time / 1000 < 10 && time / 1000 >= 0) {
         // 如果时间间隔小于10秒则显示“刚刚”time/10得出的时间间隔的单位是秒
         interval = "刚刚";
         return interval;
      } else if (time / 1000 < 60 && time / 1000 > 0) {
         // 如果时间间隔小于60秒则显示多少秒前
         int se = (int) ((time % 60000) / 1000);
         interval = se + "秒前";
         return interval;
      } else if (time / 60000 < 60 && time / 60000 > 0) {
         // 如果时间间隔小于60分钟则显示多少分钟前
         int m = (int) ((time % 3600000) / 60000);// 得出的时间间隔的单位是分钟
         interval = m + "分钟前";
         return interval;
      } else if (time / 3600000 < 24 && time / 3600000 >= 0) {
         // 如果时间间隔小于24小时则显示多少小时前
         int h = (int) (time / 3600000);// 得出的时间间隔的单位是小时
         interval = h + "小时前";
         return interval;
      } else if (time / 3600000 >= 24 && time / 3600000 < 48) {
         // 如果时间间隔大于24个小时，小于48个小时
         interval = "昨天";
         return interval;
      } else if (time / 3600000 >= 48 && time / 3600000 < 72) {
         // 如果时间间隔大于48个小时，小于72个小时
         interval = "前天";
         return interval;
      } else {
         // 大于24小时，则显示正常的时间，但是不显示秒
         SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
         interval = sdf.format(datetime);
         return interval;
      }
      
   }
   
   public static String strToDateFormat(String date,String strPattern,String toPattern) throws ParseException {
       SimpleDateFormat formatter = new SimpleDateFormat(strPattern);
       Date newDate= formatter.parse(date);
       formatter = new SimpleDateFormat(toPattern);
       return formatter.format(newDate);
   }
   
   /**
    * 计算两个日期之间相差的分钟数 (向上取整的整数),请用 后面参数-前面参数
    * 
    * @param befDate
    * @param aftDate
    * @return min
    */
   public static int minsBetween(Date befDate, Date aftDate) {
      long befTime = befDate.getTime();
      long aftTime = aftDate.getTime();
      int mins = (int)Math.ceil((aftTime-befTime)/(60*1000.0));
      return mins;
   }
   //取得一个月的第一天
   public static String getFirstDay(Date d) {
      Calendar c = Calendar.getInstance();
      c.setTime(d);
      c.set(Calendar.DAY_OF_MONTH, 1);
      SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
      return sdf.format(c.getTime());
   }

   /**
    * 将Date类转换为XMLGregorianCalendar
    * @param date
    * @return
    */
   public static XMLGregorianCalendar dateToXmlDate(Date date){
      Calendar cal = Calendar.getInstance();
      cal.setTime(date);
      DatatypeFactory dtf = null;
      try {
         dtf = DatatypeFactory.newInstance();
      } catch (DatatypeConfigurationException e) {
      }
      XMLGregorianCalendar dateType = dtf.newXMLGregorianCalendar();
      dateType.setYear(cal.get(Calendar.YEAR));
      //由于Calendar.MONTH取值范围为0~11,需要加1
      dateType.setMonth(cal.get(Calendar.MONTH)+1);
      dateType.setDay(cal.get(Calendar.DAY_OF_MONTH));
      dateType.setHour(cal.get(Calendar.HOUR_OF_DAY));
      dateType.setMinute(cal.get(Calendar.MINUTE));
      dateType.setSecond(cal.get(Calendar.SECOND));
      return dateType;
   }

   /**
    * 将XMLGregorianCalendar转换为Date
    * @param cal
    * @return
    */
   public static Timestamp xmlDate2Date(XMLGregorianCalendar cal){
      return new Timestamp(cal.toGregorianCalendar().getTime().getTime());
   }
   /**
    * 获取包含任意时间的星期一和星期天的日期
    * @param d
    * @return
    */
   public static Map<String, Object> getWeekday(Date d) {
      SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd"); // 设置时间格式
      Calendar cal = Calendar.getInstance();
      cal.setTime(d);
      // 判断要计算的日期是否是周日，如果是则减一天计算周六的，否则会出问题，计算到下一周去了
      int dayWeek = cal.get(Calendar.DAY_OF_WEEK);// 获得当前日期是一个星期的第几天
      if (1 == dayWeek) {
         cal.add(Calendar.DAY_OF_MONTH, -1);
      }
      cal.setFirstDayOfWeek(Calendar.MONDAY);// 设置一个星期的第一天，按中国的习惯一个星期的第一天是星期一
      int day = cal.get(Calendar.DAY_OF_WEEK);// 获得当前日期是一个星期的第几天
      cal.add(Calendar.DATE, cal.getFirstDayOfWeek() - day);// 根据日历的规则，给当前日期减去星期几与一个星期第一天的差值
      String monday = sdf.format(cal.getTime());
      cal.add(Calendar.DATE, 6);
      String sunday = sdf.format(cal.getTime());
      Map<String, Object> weekday=new HashMap<>();
      weekday.put("monday", monday);
      weekday.put("sunday", sunday);
      return weekday;
   }
   
   /**
    * 某一月的天数
    */
   public static int whichMonth(String date) {
      int days = 0;
      if (date.equals(
            (Calendar.getInstance().get(Calendar.YEAR) + "-" + (Calendar.getInstance().get(Calendar.MONTH) + 1))
                  .toString())) {
         days = Calendar.getInstance().get(Calendar.DAY_OF_MONTH);
      } else {
         SimpleDateFormat sdf = new SimpleDateFormat("", Locale.ENGLISH);
         sdf.applyPattern("yyyy-MM");
         Calendar calendar = new GregorianCalendar();
         try {
            calendar.setTime(sdf.parse(date));
         } catch (ParseException e) {
            e.printStackTrace();
         }
         days = calendar.getActualMaximum(Calendar.DAY_OF_MONTH);
      }
      return days;
   }

   /**
    * 查询所有时间、今天0、最近三天3、最近一周7、最近一月30
    */
   public static String queryDate(Integer days) {
       SimpleDateFormat sdf=new SimpleDateFormat("yyyy-MM-dd");
       Date date=new Date();
       Calendar calendar = Calendar.getInstance();
       calendar.setTime(date);
       calendar.add(Calendar.DAY_OF_MONTH, -days);
       date = calendar.getTime();
       System.out.println(sdf.format(date));
      return sdf.format(date);
   }
   
   /**
    * 根据年月查询对应月的天数
    * @param year
    * @param month
    * @return
    */
   public static int getDaysByYearMonth(int year, int month) {  
        Calendar a = Calendar.getInstance();  
        a.set(Calendar.YEAR, year);  
        a.set(Calendar.MONTH, month - 1);  
        a.set(Calendar.DATE, 1);  
        a.roll(Calendar.DATE, -1);  
        int maxDate = a.get(Calendar.DATE);  
        return maxDate;  
 
    }
   
    /** 
     * 获取当月的 天数 
     * */  
 
   public static int getCurrentMonthDay() {  
        Calendar a = Calendar.getInstance();  
        a.set(Calendar.DATE, 1); 
        a.roll(Calendar.DATE, -1);  
        int maxDate = a.get(Calendar.DATE);  
        return maxDate;  
 
    }


   public static boolean isValidDate(String str) {
      boolean convertSuccess = true;
      // 指定日期格式为四位年/两位月份/两位日期，注意yyyy/MM/dd区分大小写；
      SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd");
      try {
         // 设置lenient为false. 否则SimpleDateFormat会比较宽松地验证日期，比如2007/02/29会被接受，并转换成2007/03/01
         format.setLenient(false);
         format.parse(str);
      } catch (Exception e) {
         // e.printStackTrace();
         // 如果throw java.text.ParseException或者NullPointerException，就说明格式不对
         convertSuccess = false;
      }
      return convertSuccess;
   }
   /**
    * @Description TODO(获取下个月的时间，具体到 秒)
    * @Author lvhaosir 2019/3/29 17:04
    * @param startTime
    */
   public static Timestamp getNextMonth(Timestamp startTime) {
      String end1 = DateUtil.format(startTime, DateUtil.TIME_PATTERN_HHMMSS);
      System.out.println(startTime);
      Date dateAfterCountMonthByCal = DateUtil.getDateAfterCountMonthByCal(startTime, 1);
      String start1 = DateUtil.format(dateAfterCountMonthByCal, DateUtil.DATE_PATTERN_DEFAULT);
      StringBuilder sb = new StringBuilder(start1);
      sb.append(" ");
      sb.append(end1);
      Timestamp endTime = DateUtil.parseTimestampDefault(sb.toString());
      System.out.println(endTime);
      return endTime;
   }

   /**
    * 获取指定年月的最后一天
    * @param year
    * @param month
    * @return
    */
   public static String getLastDayOfMonth(int year, int month) {
      Calendar cal = Calendar.getInstance();
      //设置年份
      cal.set(Calendar.YEAR, year);
      //设置月份
      cal.set(Calendar.MONTH, month-1);
      //获取某月最大天数
      int lastDay = cal.getActualMaximum(Calendar.DATE);
      //设置日历中月份的最大天数
      cal.set(Calendar.DAY_OF_MONTH, lastDay);
      //格式化日期
      SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
      return sdf.format(cal.getTime()) + " 23:59:59";
   }

   public static Timestamp getLastDayOfMonth(Timestamp timestamp) {
      int year = timestamp.getYear()+1900;
      int month = timestamp.getMonth() + 1;
      String lastDayOfMonth = getLastDayOfMonth(year, month);
      System.out.println(lastDayOfMonth);
      return DateUtil.parseTimestampDefault(lastDayOfMonth);
   }
}

 

对象池操作工具类

package com.sunshine.http_proxy.pool;

import com.sunshine.http_proxy.utils.HttpProxyConfig;
import org.apache.commons.pool.PoolableObjectFactory;
import org.apache.commons.pool.impl.GenericObjectPool;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * 对象池操作操作工具
 * @author xaingzi
 */
@SuppressWarnings({ "unchecked", "rawtypes" })
public class ObjectPool {
   public static Logger log = LoggerFactory.getLogger(ObjectPool.class);
   private static ObjectPool objectPool;
   private GenericObjectPool pool;

   private ObjectPool() {}

   private ObjectPool(Class cls, HttpProxyConfig httpProxyConfig) {
      this.pool = new GenericObjectPool(new ObjectPoolFactory(cls, httpProxyConfig));
      pool.setMaxActive(httpProxyConfig.getMaxActive());// 最大活动对象
      pool.setMaxIdle(httpProxyConfig.getMaxIdle());// 最大空闲对象
      pool.setMaxWait(httpProxyConfig.getMaxWait());// 最大等待时间
      pool.setTestOnReturn(httpProxyConfig.isTestOnReturn());
      pool.setTestOnBorrow(httpProxyConfig.isTestOnBorrow());
      pool.setLifo(httpProxyConfig.isLifo());
      pool.setTimeBetweenEvictionRunsMillis(httpProxyConfig.getTimeBetweenEvictionRunsMillis());
      pool.setNumTestsPerEvictionRun(httpProxyConfig.getNumTestsPerEvictionRun());
      pool.setTestWhileIdle(true);
   }

   public static ObjectPool getObjecPool(Class cls, HttpProxyConfig httpProxyConfig) {
      if (objectPool == null) {
         synchronized (ObjectPool.class) {
            if (objectPool == null) {
               objectPool = new ObjectPool(ProxyHost.class, httpProxyConfig);
            }
         }
      }
      return objectPool;
   }

   /**
    * 创建对象
    * 
    * @return
    */
   public void makeObject() {
      try {
         log.info("创建对象");
         pool.addObject();
      } catch (Exception e) {
         // TODO Auto-generated catch block
         e.printStackTrace();
      }
   }

   /**
    * 池中取出对象
    * 
    * @param <T>
    * @return
    */
   public <T> T borrowObject() {
      T obj = null;
      try {
         obj = (T) pool.borrowObject();
         log.info("获得对象");
      } catch (Exception e) {
         System.out.println(e);
      }
      return obj;
   }

   /**
    * 对象放回池中
    * 
    * @param obj
    */
   public void returnObject(Object obj) {
      try {
         pool.returnObject(obj);
         log.info("返还对象");
      } catch (Exception e) {
         System.out.println(e);
      }
   }

   /**
    * 检测对象
    * 
    * @param obj
    * @return
    */
   public boolean validateObject(Object obj) {
      try {
         log.info("检测对象");
         return ((PoolableObjectFactory) pool).validateObject(obj);
      } catch (Exception e) {
         System.out.println(e);
      }
      return false;
   }

   /**
    * 销毁对象
    * 
    * @param obj
    */
   public void destroyObject(Object obj) {
      try {
         pool.invalidateObject(obj);
         log.info("销毁对象");
      } catch (Exception e) {
         System.out.println(e);
      }
   }

   // 是否后进先出
   public void setLifo(boolean bool) {
      pool.setLifo(bool);
   }

   // 对象将在返回到后进行验证。
   public void setTestOnReturn(boolean bool) {
      pool.setTestOnReturn(bool);
   }

   // 是否在借用对象之前应该进行验证。
   public void setTestOnBorrow(boolean bool) {
      pool.setTestOnBorrow(bool);
   }

   // 返回最大活动对象
   public int getMaxActive() {
      return pool.getMaxActive();
   }

   // 返回最多可空闲对象
   public int getMaxIdle() {
      return pool.getMaxIdle();
   }

   // 返回当前在此对象池中休眠的对象的数目。
   public int getNumIdle() {
      return pool.getNumIdle();
   }

   // 返回已经从此对象池中借出的对象的总数
   public int getNumActive() {
      return pool.getNumActive();
   }

   // 将当前对象池与参数中给定的PoolableObjectFactory相关联。如果在当前状态下，无法完成这一操作，会有一个IllegalStateException异常抛出。
   public void setFactory(PoolableObjectFactory factory) {
      pool.setFactory(factory);
   }

   // 清空对象池，并销毁对象池中的所有对象
   public void clear() {
      pool.clear();
   }

   // 关闭对象池
   public void close() {
      try {
         pool.close();
      } catch (Exception e) {
         e.printStackTrace();
      }
   }

}

 

 

对象池管理工厂

package com.sunshine.http_proxy.pool;

import java.lang.reflect.Method;
import java.util.Date;

import com.sunshine.http_proxy.HttpProxyClientLocat;
import com.sunshine.http_proxy.utils.HttpProxyConfig;
import org.apache.commons.pool.PoolableObjectFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONObject;
import com.sunshine.http_proxy.utils.DateUtil;
/**
 * 对象池管理工厂
* @author xaingzi
*/
@SuppressWarnings({"unchecked", "rawtypes" })
public class ObjectPoolFactory implements PoolableObjectFactory {
   public static Logger log = LoggerFactory.getLogger(ObjectPoolFactory.class);
   private Class cls;
   private HttpProxyConfig httpProxyConfig;
   private static final String INIT_METHOD = "clearObject";// 有状态对象恢复初始化的方法

   public ObjectPoolFactory(Class cls, HttpProxyConfig httpProxyConfig) {
      this.cls = cls;
      this.httpProxyConfig = httpProxyConfig;
   }

   public void activateObject(Object arg0) throws Exception {
      log.info("有状态对象恢复初始化");
      try {
         cls.getDeclaredMethod(INIT_METHOD).invoke(arg0);// 有状态对象恢复初始化
      } catch (Exception e) {
      }
   }

   public void destroyObject(Object arg0) throws Exception {
      log.info("销毁要过期IP对象："+arg0.toString());
   }

   public Object makeObject() throws Exception {
      synchronized (this){
         JSONObject json = HttpProxyClientLocat.getHttpClientProxyIpZhiMa8(httpProxyConfig);
         Object object = JSON.parseObject(json.getJSONArray("data").getJSONObject(0).toString(), cls);
         log.info("创建新IP对象："+object.toString());
         return object;
      }
   }

   public void passivateObject(Object arg0) throws Exception {
   }

   public boolean validateObject(Object arg0) {
      //log.info("验证对象："+arg0.toString());
      try {
         String fieldName = "expireTime";
         String firstLetter = fieldName.substring(0, 1).toUpperCase();
         String getter = "get" + firstLetter + fieldName.substring(1);
         Method method = arg0.getClass().getMethod(getter, new Class[] {});
         String value = (String) method.invoke(arg0, new Object[] {});
         Date date = DateUtil.parse(value, DateUtil.TIME_PATTERN_DEFAULT);
         log.info("距离截止有效期还剩：" + DateUtil.minsBetween(new Date(), date)+" 分钟");
         if (DateUtil.minsBetween(new Date(), date) <= httpProxyConfig.getTestTime()) {
            log.info(arg0.toString() + " 即将被销毁");
            return false;
         }
      } catch (Exception e) {
         log.error(e.toString());
      }
      return true;
   }
}

 

好了，今天的分享就到这，有什么疑问可以留言讨论。。。

更详细的代码https://github.com/Shimmer8618/sunshine

 

 

 

 

小白来自农村，如果你有爱心就给我一点阳光。。。

在这里祝所有努力的年轻人，都能活成自己想要的模样。。。

我在努力中~  加油！！！

一个小白为之奋斗的起点

转载于:https://my.oschina.net/u/4131417/blog/3048247

chuangliaopin9247
关注
  • 0
    点赞
  • 踩
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Java 实现ip代理池请求-爬虫防封、文章阅读刷量
洛阳泰山的博客
12-15 4021
实现过程主要分两步： 第一步，需要到ip代理平台，注册开通获取代理ip的api接口 第二步，请求api接口，获得代理ip列表，实现ip代理请求指定网址。 pom需要依赖 <!-- hutool--> <dependency> <groupId>cn.hutool</groupId> <artifactId>hutool-all</artifactId> <version>5.3.6&lt..
java多线程代理IP池
11-30
多线程代理IP池，一直看到有关这方面的技术，最近实现了一个。简单的来说，启动后，会一直定时的获取代理ip，并自动检测代理ip的活跃度。运用多线程的技术，在极短的时间内获取大量的ip进行筛选。架构也比较清楚，方便大家学习参考 需要主要的是，代理ip获取成功后是放在redis里面的，有关redis的参数配置在baseDB里面配置。当然对于没有开redis，我也进行处理放在了static里面。难者不会会者不难-=-。
爬虫必学：Java创建代理ip池详细教程
weixin_44617651的博客
11-27 1061
闲来无事，在网上瞎看看，正好看见一篇有关python爬虫代理Ip池建立的方法，详细查看验证之后觉得非常有趣。正好利用我空余时间，写了一篇java语言创建代理IP池的通用模板，对于爬虫新手来说非常实用，我将从几个方面详细阐述我的步骤，希望能帮助更多的新手学习并入门爬虫。
java 手写IP代理池 (开放源代码)
qzqanzc的博客
01-03 3271
最近的项目在做爬虫，ip代理一直是一个很让人头疼的问题，看了python 开源版的爬虫代理池，修改起来也比较吃力。本着自己动手丰衣足食的理念还是用java 写个IP代理池的项目吧，解决IP代理的问题。源码地址https://github.com/zhiquanqiao/IPProxyPool。 需要手动维护被爬网站的信息和代理网站信息，维护好以后就可以通过接口获取对应的被爬网站的IP代理信息。 ...
JAVA 配置动态转发代理IP做数据爬虫采集
无忧代理IP
03-04 993
这篇文章介绍的是使用动态转发代理IP（也叫隧道代理IP），参考http://www.xiaozhudaili.com/buy/tunnel.html 首先效果是很不错的，只需要设置好一次代理IP，然后每次都会自动换一个IP，不需要代码做什么了，看下我的效果： 标题 下面附上代码：JDK1.8 package com.xiaozhudaili.test; import java.io...
一个专科小白Java的学习路线
05-04
冲冲冲！！！ 加油加油！！！ 一定要在一年内把整个思维导图填满。 相信一年之后的我将不再是个小白(ovo)
小白自己学习Java，记录Java基础.rar
06-27
1. java开发环境： - java编译运行过程： - 编译期：.java源文件，经过编译，生成.class字节码文件 - 运行期：JVM加载.class并运行.class(0和1) > 特点：跨平台、一次编译到处运行 - 名词解释： - JVM：...
JAVA教程，适合小白看
最新发布
06-19
JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看JAVA教程，适合小白看...
适用于小白查看ip地址.bat
04-30
适用于小白查看ip地址.bat
帮助新手小白JAVA环境搭建
10-25
教会新手学习怎么搭建JAVA环境
java 设置代理ip
热门推荐
独恋幽兰的专栏
05-17 1万+
java设置代理IP的两种方式。第一种简便快捷，第二种可以避免代理失效时使用本地连接访问网络。
Java——Jsoup方式的IP代理服务器的使用
qq2981118661的博客
07-06 1233
代理服务器的基本了解和使用
Java 设置代理IP 访问网络
泉诚软件设计工作室
11-09 6655
一. 工具准备 为了能够验证Java程序是否真的通过代理访问网络，还是准备一个代理工具：fiddler，很好的一个能监视http、https的debug工具，附件中有。 二. 开始 先写一段简单的网络访问程序，如下： import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream;
简单的java爬虫及ip代理
文殊的博客
06-23 885
package com.platform.modules.napoleon.common.util; import java.io.BufferedInputStream; import java.io.IOException; import java.net.*; import java.util.ArrayList; import static java.lang.Thread.sleep; public class TextProxy { public static void mai...
Java 实现 OkHttp 设置proxy代理
qq_42697946的博客
07-15 4256
java 实用工具 OkHttp 设置代理Ip
java爬虫如何使用动态代理ip
wq10_12的博客
07-04 1934
通过寻找可靠的代理服务提供商，配置代理池，实现代理IP的动态切换，并实施错误处理和IP重试机制，您可以在Java爬虫中使用动态IP代理。为了确保代理IP的稳定性和可用性，建议定期监控代理IP的状态。您可以编写监控脚本，定期检查代理IP的连通性、响应时间和可用性，并将不可用的代理IP从代理池中移除或标记为不可用状态。在爬虫代码中，使用代理池的相关功能实现代理IP的动态切换。根据您选择的HTTP请求库，将代理IP配置到请求的设置中，并确保在每次请求时使用不同的代理IP。
Java实现一个ip池,让你的蜘蛛畅行无阻
Linch的博客
09-15 2092
玩爬虫的时候,如果访问的太频繁的话,很容易被封ip,一物降一物,有反爬,当然就有反反爬╰(￣▽￣)╭ 为了防止ip被封,就可以使用ip代理,让代理服务器帮你完成这个请求,再将请求结果返回给你,是不是很像平时我们用的梯子( ‵▽′)ψ ; 使用代理之后,你的每个请求都是由很多个代理服务器帮你完成.国内用的比较多的就是西刺代理,还有其他代理也不错; 下面用java实现一个ip代理池: 这...
java 代理ip工具类_Java基础之java处理ip的工具类
weixin_39560066的博客
02-13 224
java处理ip的工具类，包括把long类型的Ip转为一般Ip类型、把xx.xx.xx.xx类型的转为long类型、根据掩码位获取掩码、根据 ip/掩码位 计算IP段的起始IP、根据 ip/掩码位 计算IP段的终止IP等方法，可以直接使用！package com.hh.test;import java.util.HashMap;import java.util.Map;import org.apa...
java使用ip代理池模拟不同地区客户端访问
菜鸟一个
07-08 1402
前言     有时候需要快速提高自己一篇文章的活跃度，对于一些没有身份认证的只靠ip和http header统计访问量的系统，我们可以使用脚本+ip代理池来进行模拟访问 代码 proxy文件为楼主存储ip代理池的文件，ip代理池可以从网上找ip获取或者直接找一份长期的ip存在本地，楼主用的是后者（需要的ip量不大，可以重复），格式如下： 58.218.200...
小白java如何学习
06-13
以下是小白学习Java的一些建议： 1. **基础知识了解**：开始之前，了解计算机科学的基础概念，如数据类型、变量、控制结构（如循环和条件语句）等。这些都是Java语言的基础。 2. **选择教材或课程**：推荐使用官方...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值