jsoup学习整理（模拟登陆，乱码问题）

学习至jsoup中文网http://www.open-open.com/jsoup/

获取cookies（针对没有验证码或者有漏洞绕过验证的模拟登陆）【例如正方教务】

Response res=null;

        res=Jsoup.connect(urlCookies)//要请求的链接

       .followRedirects(false)//不处理自动重定向

.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0")//伪装浏览器
.data("TextBox1",username)//设置请求时需要的数据
.data("TextBox2",password)
.data("Button1","")
.data("RadioButtonList1","学生")
.data("__VIEWSTATE","dDwxMTE4MjQwNDc1Ozs+MzFt0h81g6NGHTq1L9P2NfWUGLA=")
.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")//设置请求头
.header("Accept-Language","zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3")
.header("Accept-Encoding", "gzip, deflate")
.header("Connection","keep-alive")
.header("Upgrade-Insecure-Requests", "1")

.execute();

        cookies=res.cookies();

-------------------------------------------------------------

得到cookies后这样就可以去得到页面的document对象了

elemts=Jsoup.connect(urlSource+username)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:56.0) Gecko/20100101 Firefox/56.0")
//.header("Referer", "http://172.16.1.8/default2.aspx")
.cookies(cookies)

.get();

----------------------------------------------------------------

这里有个东西说下有些网站有设置Referer，这个时候你就要填写发出请求的页面

且jsoup的Jsoup.connect是无法设置编码 的所以可能会乱码

一般情况下得到document取得elemt后new String(gt.text().getBytes("ISO8859-1"),"GBK")

转码就ok了但是有一次遇见一些偏僻的中文就不行（可能我的版本低了没设置编码功能）

最后找到了处理方法：

URL url = new URL(ppx.toString());//ppx是要访问的链接
HttpURLConnection connection = (HttpURLConnection)url.openConnection();//这里只是创建一个链接实例并没有真实网络链接
connection.setRequestMethod("GET");
//connection.addRequestProperty("ddlXQ", "1");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Upgrade-Insecure-Requests", "1");
connection.addRequestProperty("Host", "61.142.33.204");
connection.addRequestProperty("Referer", "http://61.142.33.204/xs_main.aspx?xh="+username);
connection.addRequestProperty("Accept-Language", "zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3");
connection.addRequestProperty("Accept-Encoding", "gzip, deflate");
connection.addRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*;q=0.8");
connection.addRequestProperty("Cookie", "ASP.NET_SessionId="+cookies.get("ASP.NET_SessionId"));

elemts = Jsoup.parse(connection.getInputStream(), "GBK", ppx.toString());//这个方法才去访问链接并可以设置编码

这样就算是生僻字也没问题了