jsoup实现微博登陆及发微博(带验证码)

城堡

前言

很早之前用别的语言写过一次微博登陆,现在改成java版,主要是为了验证jsoup的强大,一款完全可胜任http请求和html解析的工具。抓包,抽取js的方法这里就不在赘述。

工具

  • jsoup-1.11.2(使用最新)
  • 打码账号及官方提供的java类(市面上很多,这里用若快)

开始

  1. 首先是分析登录过程,常登陆的没有验证码,新的会有验证码,保险起见这里都采用有验证码的方式登录。
  2. Base64编码用户名后,get请求获取servertime,nonce,pubkey,pcid,及返回的cookie

    //Base64编码用户名
    su = new BASE64Encoder().encode(usename.getBytes());
    String url = "http://login.sina.com.cn/sso/prelogin.php?entry=weibo&callback=sinaSSOController.preloginCallBack&su=" + su + "&rsakt=mod&checkpin=1&client=ssologin.js(v1.4.18)&_=" +
    getTimestamp();
    Connection.Response execute = Jsoup.connect(url).ignoreContentType(true).execute();
    //获取返回数据
    String body = execute.body();
    //获取返回cookie
    Map<String, String> photocookie = execute.cookies();
    JSONObject jsonObject = JSONObject.parseObject(StringUtils.substringBetween(body, "(", ")"));
    servertime = jsonObject.getString("servertime");
    nonce = jsonObject.getString("nonce");
    pubkey = jsonObject.getString("pubkey");
    pcid = jsonObject.getString("pcid");
  3. 带着返回的cookie及pcid,get请求获取验证码

    url = "http://login.sina.com.cn/cgi/pin.php?r=54474015&s=0&p=" + pcid;
    byte[] bytes = Jsoup.connect(url).ignoreContentType(true).cookies(photocookie).execute().bodyAsBytes();
  4. 调用打码平台的类实现打码,返回结果

    public static String createByPost(String username, String password, String typeid, String timeout, String softid, String softkey,byte[] byteArr) {
    String result = "";
    String param = String
    .format(
    "username=%s&password=%s&typeid=%s&timeout=%s&softid=%s&softkey=%s",
    username, password, typeid, timeout, softid, softkey);
    try {
    result = RuoKuai.httpPostImage("http://api.ruokuai.com/create.xml", param, byteArr);
    // jsoup去解析xml (略坑,加了好多参数进去)
    result = Jsoup.parse(result).select("body > root > result").text();
    } catch(Exception e) {
    result = "未知问题";
    }
    return result;
    }
  5. 调用js,对密码动态加密,返回参数sp

    private boolean encodePwd() {
    ScriptEngineManager sem = new ScriptEngineManager();
    ScriptEngine se = sem.getEngineByName("javascript");
    try {
    // FileReader fr = new FileReader("");
    se.eval(LOGIN_JS);
    Invocable invocableEngine = (Invocable) se;
    sp = (String) invocableEngine.invokeFunction("getPW", password, servertime, nonce, pubkey);
    return true;
    } catch (ScriptException e) {
    } catch (NoSuchMethodException e) {
    }
    return false;
    }
  6. 带着参数pcid,code,sp,su,servertime,nonce,sp及cookie,post请求返回cookies及下一个url,此时可从返回的body中判断登录成功与否

        //retcode=101 账号密码错误
        //retcode=80 请输入正确的密码
        //retcode=4049 输入验证码
        //retcode=2070 验证码错误
        url = "http://login.sina.com.cn/sso/login.php?client=ssologin.js(v1.4.18)";
        String post = "entry=weibo&gateway=1&from=&savestate=7&useticket=1&pagerefer=http%3A%2F%2Fpassport.weibo" +
                ".com%2Fvisitor%2Fvisitor%3Fentry%3Dminiblog%26a%3Denter%26url%3Dhttp%253A%252F%252Fweibo.com%252F%26domain%3D.weibo" +
                ".com%26ua%3Dphp-sso_sdk_client-0.6.14%26_rand%3D1441434306.495&pcid=" + pcid + "&door=" + code + "&vsnf=1&su=" + su + "&service=miniblog&servertime=" + servertime + "&nonce=" + nonce
                + "&pwencode=rsa2&rsakv=1330428213&sp=" + sp + "&sr=1366*768&encoding=UTF-8&url=http%3A%2F%2Fweibo.com%2Fajaxlogin.php%3Fframelogin%3D1%26callback%3Dparent.sinaSSOController.feedBackUrlCallBack&returntype=META";
    
        Connection.Response execute1 = Jsoup.connect(url).method(Connection.Method.POST).requestBody(post).cookies(photocookie).ignoreContentType(true).execute().charset("GBK");
        String body1 = execute1.body();
        Map<String, String> cookies = execute1.cookies();
        System.out.println(body1);
        System.out.println(body1.indexOf("正在登录") != -1 ? "登录成功" : "登录失败");
    
        url = StringUtils.substringBetween(body1, "location.replace('", "'");
    
  7. 带着上次返回的cookie,get请求(禁止重定向)获取最终的cookie

    Connection.Response execute2 = Jsoup.connect(url).cookies(cookies).followRedirects(false).ignoreContentType(true).execute();
    Map<String, String> cookies1 = execute2.cookies();
    //从返回协议头中获取location,重定向地址,一般是固定
    Map<String, String> headers = execute2.headers();

结束

获取到cookie,基本之后的操作都可以,那就简单的发条微博,发微博其实只带post数据和cookie就能提交,协议头完全可以不带,偏偏jsoup默认带个UA,结果要多带个refrere头才可以。
1. 通过步骤7,获取的重定向地址,get请求获取refrere参数uniqueid

                        url = "http://weibo.com/ajaxlogin.php?framelogin=1&callback=parent.sinaSSOController.feedBackUrlCallBack&sudaref=login.sina.com.cn";
                        String body2 = Jsoup.connect(url).cookies(cookies1).ignoreContentType(true).execute().body();
                        String uid = StringUtils.substringBetween(body2, "uniqueid\":\"", "\",\"userid");
                        String referer = "https://weibo.com/u/" + uid + "/home";
  1. 带着cookie,及协议头referer,post请求,发一条微博吧
        url = "https://weibo.com/aj/mblog/add?ajwvr=6&__rnd=" + getTimestamp();
        post = "location=v6_content_home&appkey=&style_type=1&pic_id=&text=" + text + "&pdetail=&rank=0&rankid=&module=stissue&pub_source=main_&pub_type=dialog&_t=0";

        Connection.Response execute3 = Jsoup.connect(url).method(Connection.Method.POST).requestBody(post).cookies(cookies1).referrer(referer).ignoreContentType(true).execute().charset("GBK");
        System.out.println(execute3.body());

Success

  • 1
    点赞
  • 6
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Jsoup+httpclient 模拟登陆和抓取页面 package com.app.html; import java.io.BufferedReader; import java.io.BufferedWriter; import java.io.File; import java.io.FileOutputStream; import java.io.FileReader; import java.io.IOException; import java.io.OutputStreamWriter; import java.io.Writer; import java.text.SimpleDateFormat; import java.util.Date; import org.apache.commons.httpclient.Cookie; import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.NameValuePair; import org.apache.commons.httpclient.cookie.CookiePolicy; import org.apache.commons.httpclient.cookie.CookieSpec; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.params.HttpMethodParams; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import com.app.comom.FileUtil; public class HttpClientHtml { private static final String SITE = "login.goodjobs.cn"; private static final int PORT = 80; private static final String loginAction = "/index.php/action/UserLogin"; private static final String forwardURL = "http://user.goodjobs.cn/dispatcher.php/module/Personal/?skip_fill=1"; private static final String toUrl = "d:\\test\\"; private static final String css = "http://user.goodjobs.cn/personal.css"; private static final String Img = "http://user.goodjobs.cn/images"; private static final String _JS = "http://user.goodjobs.cn/scripts/fValidate/fValidate.one.js"; /** * 模拟等录 * @param LOGON_SITE * @param LOGON_PORT * @param login_Action * @param params * @throws Exception */ private static HttpClient loginHtml(String LOGON_SITE, int LOGON_PORT,String login_Action,String ...params) throws Exception { HttpClient client = new HttpClient(); client.getHostConfiguration().setHost(LOGON_SITE, LOGON_PORT); // 模拟登录页面 PostMethod post = new PostMethod(login_Action); NameValuePair userName = new NameValuePair("memberName",params[0] ); NameValuePair password = new NameValuePair("password",params[1] ); post.setRequestBody(new NameValuePair[] { userName, password }); client.executeMethod(post); post.releaseConnection(); // 查看cookie信息 CookieSpec cookiespec = CookiePolicy.getDefaultSpec(); Cookie[] cookies = cookiespec.match(LOGON_SITE, LOGON_PORT, "/", false, client.getState().getCookies()); if (cookies != null) if (cookies.length == 0) { System.out.println("Cookies is not Exists "); } else { for (int i = 0; i < cookies.length; i++) { System.out.println(cookies[i].toString()); } } return client; } /** * 模拟等录 后获取所需要的页面 * @param client * @param newUrl * @throws Exception */ private static String createHtml(HttpClient client, String newUrl) throws Exception { SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd"); String filePath = toUrl + format.format(new Date() )+ "_" + 1 + ".html"; PostMethod post = new PostMethod(newUrl); client.executeMethod(post); //设置编码 post.getParams().setParameter(HttpMethodParams.HTTP_CONTENT_CHARSET, "GBK"); String content= post.getResponseBodyAsString(); FileUtil.write(content, filePath); System.out.println("\n写入文件成功!"); post.releaseConnection(); return filePath; } /** * 解析html代码 * @param filePath * @param random * @return */ private static String JsoupFile(String filePath, int random) { SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd"); File infile = new File(filePath); String url = toUrl + format.format(new Date()) + "_new_" + random+ ".html"; try { File outFile = new File(url); Document doc = Jsoup.parse(infile, "GBK"); String html="<!DOCTYPE HTML PUBLIC '-//W3C//DTD HTML 4.01 Transitional//EN'>"; StringBuffer sb = new StringBuffer(); sb.append(html).append("\n"); sb.append("<html>").append("\n"); sb.append("<head>").append("\n"); sb.append("<title>欢迎使用新安人才网个人专区</title>").append("\n"); Elements meta = doc.getElementsByTag("meta"); sb.append(meta.toString()).append("\n"); ////////////////////////////body////////////////////////// Elements body = doc.getElementsByTag("body"); ////////////////////////////link////////////////////////// Elements links = doc.select("link");//对link标签有href的路径都作处理 for (Element link : links) { String hrefAttr = link.attr("href"); if (hrefAttr.contains("/personal.css")) { hrefAttr = hrefAttr.replace("/personal.css",css); Element hrefVal=link.attr("href", hrefAttr);//修改href的属性值 sb.append(hrefVal.toString()).append("\n"); } } ////////////////////////////script////////////////////////// Elements scripts = doc.select("script");//对script标签 for (Element js : scripts) { String jsrc = js.attr("src"); if (jsrc.contains("/fValidate.one.js")) { String oldJS="/scripts/fValidate/fValidate.one.js";//之前的css jsrc = jsrc.replace(oldJS,_JS); Element val=js.attr("src", jsrc);//修改href的属性值 sb.append(val.toString()).append("\n").append("</head>"); } } ////////////////////////////script////////////////////////// Elements tags = body.select("*");//对所有标签有src的路径都作处理 for (Element tag : tags) { String src = tag.attr("src"); if (src.contains("/images")) { src = src.replace("/images",Img); tag.attr("src", src);//修改src的属性值 } } sb.append(body.toString()); sb.append("</html>"); BufferedReader in = new BufferedReader(new FileReader(infile)); Writer out = new BufferedWriter(new OutputStreamWriter( new FileOutputStream(outFile), "gbk")); String content = sb.toString(); out.write(content); in.close(); System.out.println("页面已经爬完"); out.close(); } catch (IOException e) { e.printStackTrace(); } return url; } public static void main(String[] args) throws Exception { String [] params={"admin","admin123"}; HttpClient client = loginHtml(SITE, PORT, loginAction,params); // 访问所需的页面 String path=createHtml(client, forwardURL); System.out.println( JsoupFile(path,1)); } }

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值