关闭

OkHttp与Cookie及Cookie的持久化

1327人阅读 评论(1) 收藏 举报
分类:

参考:OkkHttp3之Cookie管理 、 模拟登录知乎并抓取用户信息

一、OkHttp3下的Cookie的使用

①、OkHttpClient取消了setCookieHandler(CookieHandler cookieHandler);

改而使用:

setCookieJar(CookieJar cookieJar);

CookieJar是一个接口,需要自己实现CookieJar的定义。

CookieJar cookieJar = new CookieJar() {
	//存储数据		
	@Override
	public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
		// TODO Auto-generated method stub
			
<span style="white-space:pre">	</span>}
<span style="white-space:pre">	</span>//读取数据	
	@Override
	public List<Cookie> loadForRequest(HttpUrl arg0) {
		// TODO Auto-generated method stub
		return null;
	}
};
并且使用OkHttp的Cookie类来存储Cookie,而不使用自带的HttpCookie类。


②、OkHttp推荐使用Builder来创建OkHttpClient对象,来配置OkHttpClient

OkHttpClient.Builder builder = new OkHttpClient.Builder();
//设置超时时常
builder.connectTimeout(timeout, unit);
//设置Cookie管理器
builder.cookieJar(cookieJar);
//等等。。查看文档


二、使用OkHttpClient获取知乎关注的人的数据(昵称和简介)  非持久化



①、初始化CookieJar

//初始化Cookie管理器
		CookieJar cookieJar = new CookieJar() {
			//Cookie缓存区
			private final Map<String, List<Cookie>> cookiesMap = new HashMap<String, List<Cookie>>();
			@Override
			public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
				// TODO Auto-generated method stub
				//移除相同的url的Cookie
				String host = arg0.host();
				List<Cookie> cookiesList = cookiesMap.get(host);
				if (cookiesList != null){
					cookiesMap.remove(host);
				}
				//再重新天添加
				cookiesMap.put(host, <span style="font-family: Arial, Helvetica, sans-serif;">arg1</span><span style="font-family: Arial, Helvetica, sans-serif;">);</span>
			}
			
			@Override
			public List<Cookie> loadForRequest(HttpUrl arg0) {
				// TODO Auto-generated method stub
				List<Cookie> cookiesList = cookiesMap.get(arg0.host());
				//注:这里不能返回null,否则会报NULLException的错误。
				//原因:当Request 连接到网络的时候,OkHttp会调用loadForRequest()
				return cookiesList != null ? cookiesList : new ArrayList<Cookie>();
			}
		};

②、登陆知乎

1、查看知乎的Post请求

首先通过F12打开浏览器的开发者工具(本人用的Chrome)


点击到红色的位置,并勾选蓝色位置的选项。

2、登陆知乎:https://www.zhihu.com/

登陆完成后,会在开发者工具发现该文件(本人是手机登陆所以是phone_num,如果是邮箱登陆是emila文件)


然后我们查看我们要上传的数据和上传的地址


这里有个_xsfr:需要在登陆的html中的form表单中查找其真正的value


本人的值是:bf284aba4cc706ebfc5ebcba1c4f97fc。  据测试,该值在同一个浏览器中提交是不会变的。

③、使用OkHttp登陆,并获取Cookie

//创建OkHttpClient
		OkHttpClient client = new OkHttpClient.Builder()
						.connectTimeout(5000, TimeUnit.MILLISECONDS)
						.cookieJar(cookieJar)
						.build();
		//创建登陆的表单
		FormBody loginBody = new FormBody.Builder()
		  			.add("_xsrf", "bf284aba4cc706ebfc5ebcba1c4f97fc")
		  			.add("password", "cay1314159")
		  			.add("captcha_type", "cn")
		  			.add("remember_me", "true")
		  			.add("phone_num", "15520762775")
		  			.build();//账号密码自己填
		//创建Request请求
		Request loginRequest = new Request.Builder()
						.url("https://www.zhihu.com/login/phone_num")
						.post(loginBody)
						.build();
		//上传
		Call loginCall = client.newCall(loginRequest);
		
		try {
			//非异步执行
			Response loginResponse = loginCall.execute();
			//测试是否登陆成功
			System.out.println(loginResponse.body().string());
			//获取返回数据的头部
			Headers headers = loginResponse.headers();
			HttpUrl loginUrl = loginRequest.url();
			//获取头部的Cookie,注意:可以通过Cooke.parseAll()来获取
			List<Cookie> cookies = Cookie.parseAll(loginUrl, headers);
			//防止header没有Cookie的情况
			if (cookies != null){
				//存储到Cookie管理器中
				client.cookieJar().saveFromResponse(loginUrl, cookies);//这样就将Cookie存储到缓存中了
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

④、获取关注的人

本人获取数据的地址:https://www.zhihu.com/people/chen-yan-xiang-83/followees

		//获取需要提交的CookieStr
		StringBuilder cookieStr = new StringBuilder();
		//从缓存中获取Cookie
		List<Cookie> cookies = client.cookieJar().loadForRequest(loginRequest.url());
		//将Cookie数据弄成一行
		for(Cookie cookie : cookies){
			cookieStr.append(cookie.name()).append("=").append(cookie.value()+";");
		}
		System.out.println(cookieStr.toString());
		//设置提交的请求
		Request attentionRequest = new Request.Builder()
							.url("https://www.zhihu.com/people/chen-yan-xiang-83/followees")
							.header("Cookie", cookieStr.toString())
							.build();
		Call attentionCall = client.newCall(attentionRequest);
		try {
			//连接网络
			Response attentionResponse = attentionCall.execute();
			if (attentionResponse.isSuccessful()){
				//获取返回的数据
				String data = attentionResponse.body().string();
				//测试
				System.out.println(data);
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

⑤、通过Jsoup获取html文件的数据

Document document = Jsoup.parse(data);
Elements attentions = document.select("div.zm-profile-card");
for(Element attention : attentions){
	System.out.println("name:"+attention.select("h2").text()+"  简介:"+attention.select("span").text());
}

⑥、完整代码

public static void main(String[]args){
		//初始化Cookie管理器
		CookieJar cookieJar = new CookieJar() {
			//Cookie缓存区
			private final Map<String, List<Cookie>> cookiesMap = new HashMap<String, List<Cookie>>();
			@Override
			public void saveFromResponse(HttpUrl arg0, List<Cookie> arg1) {
				// TODO Auto-generated method stub
				//移除相同的url的Cookie
				String host = arg0.host();
				List<Cookie> cookiesList = cookiesMap.get(host);
				if (cookiesList != null){
					cookiesMap.remove(host);
				}
				//再重新天添加
				cookiesMap.put(host, arg1);
			}
			
			@Override
			public List<Cookie> loadForRequest(HttpUrl arg0) {
				// TODO Auto-generated method stub
				List<Cookie> cookiesList = cookiesMap.get(arg0.host());
				//注:这里不能返回null,否则会报NULLException的错误。
				//原因:当Request 连接到网络的时候,OkHttp会调用loadForRequest()
				return cookiesList != null ? cookiesList : new ArrayList<Cookie>();
			}
		};
		//创建OkHttpClient
		OkHttpClient client = new OkHttpClient.Builder()
						.connectTimeout(5000, TimeUnit.MILLISECONDS)
						.cookieJar(cookieJar)
						.build();
		//创建登陆的表单
		FormBody loginBody = new FormBody.Builder()
		  			.add("_xsrf", "bf284aba4cc706ebfc5ebcba1c4f97fc")
		  			.add("password", "cay1314159")
		  			.add("captcha_type", "cn")
		  			.add("remember_me", "true")
		  			.add("phone_num", "15520762775")
		  			.build();//账号密码自己填
		//创建Request请求
		Request loginRequest = new Request.Builder()
						.url("https://www.zhihu.com/login/phone_num")
						.post(loginBody)
						.build();
		//上传
		Call loginCall = client.newCall(loginRequest);
		
		try {
			//非异步执行
			Response loginResponse = loginCall.execute();
			//测试是否登陆成功
			System.out.println(loginResponse.body().string());
			//获取返回数据的头部
			Headers headers = loginResponse.headers();
			HttpUrl loginUrl = loginRequest.url();
			//获取头部的Cookie,注意:可以通过Cooke.parseAll()来获取
			List<Cookie> cookies = Cookie.parseAll(loginUrl, headers);
			//防止header没有Cookie的情况
			if (cookies != null){
				//存储到Cookie管理器中
				client.cookieJar().saveFromResponse(loginUrl, cookies);//这样就将Cookie存储到缓存中了
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		
		
		//获取需要提交的CookieStr
		StringBuilder cookieStr = new StringBuilder();
		//从缓存中获取Cookie
		List<Cookie> cookies = client.cookieJar().loadForRequest(loginRequest.url());
		//将Cookie数据弄成一行
		for(Cookie cookie : cookies){
			cookieStr.append(cookie.name()).append("=").append(cookie.value()+";");
		}
		System.out.println(cookieStr.toString());
		//设置提交的请求
		Request attentionRequest = new Request.Builder()
							.url("https://www.zhihu.com/people/chen-yan-xiang-83/followees")
							.header("Cookie", cookieStr.toString())
							.build();
		Call attentionCall = client.newCall(attentionRequest);
		try {
			//连接网络
			Response attentionResponse = attentionCall.execute();
			if (attentionResponse.isSuccessful()){
				//获取返回的数据
				String data = attentionResponse.body().string();
				//测试
				System.out.println(data);
				//解析数据
				Document document = Jsoup.parse(data);
				Elements attentions = document.select("div.zm-profile-card");
				for(Element attention : attentions){
					System.out.println("name:"+attention.select("h2").text()+"  简介:"+attention.select("span").text());
				}
			}
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

⑦、残留的问题

有些时候登陆的时候会要求填写验证码 ,由于验证码需要手动数据,还要使用到GUI交互,所以就没写。所以改代码只能在不需要提交验证码的情况下使用。之后会用Android的方式补上

⑧、封装CookieJar

根据上面的使用情况,我们可以封装三个部分:
1、当获取到Response类的时候,我们将获取到的Cookie放入CookieJar,是可封装的
2、从CookieJar中获取Cookie,并连成一个CookieString是可以封装了。
3、设定Request请求头的Cookie是可以封装的。

将其封装在HttpEngine类中
1、第一类封装
public void receiveHeaders(Headers headers) throws IOException {
    if (client.cookieJar() == CookieJar.NO_COOKIES) return;

    List<Cookie> cookies = Cookie.parseAll(userRequest.url(), headers);
    if (cookies.isEmpty()) return;

    client.cookieJar().saveFromResponse(userRequest.url(), cookies);
  }
2、第二类封装
private String cookieHeader(List<Cookie> cookies) {
    StringBuilder cookieHeader = new StringBuilder();
    for (int i = 0, size = cookies.size(); i < size; i++) {
      if (i > 0) {
        cookieHeader.append("; ");
      }
      Cookie cookie = cookies.get(i);
      cookieHeader.append(cookie.name()).append('=').append(cookie.value());
    }
    return cookieHeader.toString();
  }
3、第三类封装
private Request networkRequest(Request request) throws IOException {
    Request.Builder result = request.newBuilder();
    
    List<Cookie> cookies = client.cookieJar().loadForRequest(request.url());
    if (!cookies.isEmpty()) {
      result.header("Cookie", cookieHeader(cookies));
    }

    return result.build();
}

三、Cookie的持久化

参考android-async-http的PersistentCookieStore类与SerializableHttpCookie类

*PersistentCookieStore类:该类用的是SharePreference存储,也可以换成外置的文件存储
public class PersistentCookieStore {
    private static final String LOG_TAG = "PersistentCookieStore";
    private static final String COOKIE_PREFS = "Cookies_Prefs";

    private final Map<String, ConcurrentHashMap<String, Cookie>> cookies;
    private final SharedPreferences cookiePrefs;


    public PersistentCookieStore(Context context) {
        cookiePrefs = context.getSharedPreferences(COOKIE_PREFS, 0);
        cookies = new HashMap<>();

        //将持久化的cookies缓存到内存中 即map cookies
        Map<String, ?> prefsMap = cookiePrefs.getAll();
        for (Map.Entry<String, ?> entry : prefsMap.entrySet()) {
            String[] cookieNames = TextUtils.split((String) entry.getValue(), ",");
            for (String name : cookieNames) {
                String encodedCookie = cookiePrefs.getString(name, null);
                if (encodedCookie != null) {
                    Cookie decodedCookie = decodeCookie(encodedCookie);
                    if (decodedCookie != null) {
                        if (!cookies.containsKey(entry.getKey())) {
                            cookies.put(entry.getKey(), new ConcurrentHashMap<String, Cookie>());
                        }
                        cookies.get(entry.getKey()).put(name, decodedCookie);
                    }
                }
            }
        }
    }

    protected String getCookieToken(Cookie cookie) {
        return cookie.name() + "@" + cookie.domain();
    }

    public void add(HttpUrl url, Cookie cookie) {
        String name = getCookieToken(cookie);

        //将cookies缓存到内存中 如果缓存过期 就重置此cookie
        if (!cookie.persistent()) {
            if (!cookies.containsKey(url.host())) {
                cookies.put(url.host(), new ConcurrentHashMap<String, Cookie>());
            }
            cookies.get(url.host()).put(name, cookie);
        } else {
            if (cookies.containsKey(url.host())) {
                cookies.get(url.host()).remove(name);
            }
        }

        //讲cookies持久化到本地
        SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
        prefsWriter.putString(url.host(), TextUtils.join(",", cookies.get(url.host()).keySet()));
        prefsWriter.putString(name, encodeCookie(new SerializableOkHttpCookies(cookie)));
        prefsWriter.apply();
    }

    public List<Cookie> get(HttpUrl url) {
        ArrayList<Cookie> ret = new ArrayList<>();
        if (cookies.containsKey(url.host()))
            ret.addAll(cookies.get(url.host()).values());
        return ret;
    }

    public boolean removeAll() {
        SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
        prefsWriter.clear();
        prefsWriter.apply();
        cookies.clear();
        return true;
    }

    public boolean remove(HttpUrl url, Cookie cookie) {
        String name = getCookieToken(cookie);

        if (cookies.containsKey(url.host()) && cookies.get(url.host()).containsKey(name)) {
            cookies.get(url.host()).remove(name);

            SharedPreferences.Editor prefsWriter = cookiePrefs.edit();
            if (cookiePrefs.contains(name)) {
                prefsWriter.remove(name);
            }
            prefsWriter.putString(url.host(), TextUtils.join(",", cookies.get(url.host()).keySet()));
            prefsWriter.apply();

            return true;
        } else {
            return false;
        }
    }

    public List<Cookie> getCookies() {
        ArrayList<Cookie> ret = new ArrayList<>();
        for (String key : cookies.keySet())
            ret.addAll(cookies.get(key).values());

        return ret;
    }

    /**
     * cookies 序列化成 string
     *
     * @param cookie 要序列化的cookie
     * @return 序列化之后的string
     */
    protected String encodeCookie(SerializableOkHttpCookies cookie) {
        if (cookie == null)
            return null;
        ByteArrayOutputStream os = new ByteArrayOutputStream();
        try {
            ObjectOutputStream outputStream = new ObjectOutputStream(os);
            outputStream.writeObject(cookie);
        } catch (IOException e) {
            Log.d(LOG_TAG, "IOException in encodeCookie", e);
            return null;
        }

        return byteArrayToHexString(os.toByteArray());
    }

    /**
     * 将字符串反序列化成cookies
     *
     * @param cookieString cookies string
     * @return cookie object
     */
    protected Cookie decodeCookie(String cookieString) {
        byte[] bytes = hexStringToByteArray(cookieString);
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
        Cookie cookie = null;
        try {
            ObjectInputStream objectInputStream = new ObjectInputStream(byteArrayInputStream);
            cookie = ((SerializableOkHttpCookies) objectInputStream.readObject()).getCookies();
        } catch (IOException e) {
            Log.d(LOG_TAG, "IOException in decodeCookie", e);
        } catch (ClassNotFoundException e) {
            Log.d(LOG_TAG, "ClassNotFoundException in decodeCookie", e);
        }

        return cookie;
    }

    /**
     * 二进制数组转十六进制字符串
     *
     * @param bytes byte array to be converted
     * @return string containing hex values
     */
    protected String byteArrayToHexString(byte[] bytes) {
        StringBuilder sb = new StringBuilder(bytes.length * 2);
        for (byte element : bytes) {
            int v = element & 0xff;
            if (v < 16) {
                sb.append('0');
            }
            sb.append(Integer.toHexString(v));
        }
        return sb.toString().toUpperCase(Locale.US);
    }

    /**
     * 十六进制字符串转二进制数组
     *
     * @param hexString string of hex-encoded values
     * @return decoded byte array
     */
    protected byte[] hexStringToByteArray(String hexString) {
        int len = hexString.length();
        byte[] data = new byte[len / 2];
        for (int i = 0; i < len; i += 2) {
            data[i / 2] = (byte) ((Character.digit(hexString.charAt(i), 16) << 4) + Character.digit(hexString.charAt(i + 1), 16));
        }
        return data;
    }
}

*SerializableHttpCookie类
public class SerializableOkHttpCookies implements Serializable {

    private transient final Cookie cookies;
    private transient Cookie clientCookies;

    public SerializableOkHttpCookies(Cookie cookies) {
        this.cookies = cookies;
    }

    public Cookie getCookies() {
        Cookie bestCookies = cookies;
        if (clientCookies != null) {
            bestCookies = clientCookies;
        }
        return bestCookies;
    }

    private void writeObject(ObjectOutputStream out) throws IOException {
        out.writeObject(cookies.name());
        out.writeObject(cookies.value());
        out.writeLong(cookies.expiresAt());
        out.writeObject(cookies.domain());
        out.writeObject(cookies.path());
        out.writeBoolean(cookies.secure());
        out.writeBoolean(cookies.httpOnly());
        out.writeBoolean(cookies.hostOnly());
        out.writeBoolean(cookies.persistent());
    }

    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        String name = (String) in.readObject();
        String value = (String) in.readObject();
        long expiresAt = in.readLong();
        String domain = (String) in.readObject();
        String path = (String) in.readObject();
        boolean secure = in.readBoolean();
        boolean httpOnly = in.readBoolean();
        boolean hostOnly = in.readBoolean();
        boolean persistent = in.readBoolean();
        Cookie.Builder builder = new Cookie.Builder();
        builder = builder.name(name);
        builder = builder.value(value);
        builder = builder.expiresAt(expiresAt);
        builder = hostOnly ? builder.hostOnlyDomain(domain) : builder.domain(domain);
        builder = builder.path(path);
        builder = secure ? builder.secure() : builder;
        builder = httpOnly ? builder.httpOnly() : builder;
        clientCookies =builder.build();
    }
}







0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:17421次
    • 积分:715
    • 等级:
    • 排名:千里之外
    • 原创:56篇
    • 转载:1篇
    • 译文:0篇
    • 评论:6条
    最新评论