最近因为同事的毕业设计聊天时谈及了抓包这个词,还记得当时在114的时候也用了httpwatch抓过一次学校的课表,当然demo是来自于csdn中网友的分享,开始自己的课表还是可以抓的,最后因为学校教务系统升级了,也就抓不了了。如今又来学习一下抓包,这个小技巧还是很不错的,可以抓取网上的数据作为资源。
抓包一般是返回的json数据,由此也联想到了抓html文本,比如从360文库里解析最新的hosts文本。
因为在下喜欢的是android开发,所以都是基于安卓来说的。使用抓包工具可以抓电脑和手机的数据,为了方便,就直接抓取手机app的数据。
首先要做的就是电脑和手机相连,通过wifi连到同一个局域网,下面是步骤:
- 电脑:Win+R==>>cmd回车==>>ipconfig回车==>>找到无线局域网ip;
- 手机:连上电脑的wifi,或和电脑连接到同一个wifi,在手机wifi设置里更改“无”代理为“手动”代理,ip填入步骤一的ip,端口手机为8888;
- Fiddler:Tools==>>Fiddler Options…==>>Connections==>>Allow remote computers to connect复选框打勾。
这样子,准备工作就可以了,可以试着抓取数据了,如下:
- app界面:确实有“猜你喜欢”这一栏
- 请求链接:http://v2.api.dmzj.com/recommend/batchUpdate?category_id=50
- Json数据:
这样子,你就可以使用请求到的数据,做自己想做的了。
下面从基础开始试一试,这里分为好几类开始:
- 解析app工程res/raw/文件夹下的.json本地数据,简单例子;
- 解析上述抓包到的json数据,url方式;
- 推广,想到了解析html,使用jsoup,以从360文库解析最新hosts为例子。
先看第一个简单例子:
rawjson.json
****.java
JSONObject obj;
try
{
InputStream is =
this
.getResources().openRawResource(R.raw.rawjson);
byte[] buffer =
new
byte[is.available()];
is.read(buffer);
String json =
new
String(buffer,
"utf-8"
);
obj =
new
JSONObject(json);
x = obj.getString(
"姓名"
);
Log.d(
"======姓名========"
, x);
y = obj.getString(
"性别"
);
Log.d(
"======性别========"
, x);
z = obj.getString(
"年龄"
);
Log.d(
"======年龄========"
, x);
JSONObject obj1 = obj.getJSONObject(
"学习成绩"
);
x1 = obj1.getString(
"数学"
);
Log.d(
"======数学========"
, x);
y1 = obj1.getString(
"语文"
);
Log.d(
"======语文========"
, x);
z1 = obj1.getString(
"英语"
);
Log.d(
"======英语========"
, x);
JSONArray array = obj1.getJSONArray(
"综合"
);
obj = array.getJSONObject(0);
x2 = obj.getString(
"文科综合"
);
Log.d(
"======文科综合========"
, x);
obj = array.getJSONObject(1);
y2 = obj.getString(
"理科综合"
);
Log.d(
"======理科综合========"
, x);
tv.setText(
"hello"
+ x + y + z + x1 + y1 + z1 + x2 + y2);
}
catch
(Exception e) {
e.printStackTrace();
}
再看第二个和抓上述动漫之间数据:
这里需要用到工具jar包:xUtils
****.java
import com.lidroid.xutils.HttpUtils;
import com.lidroid.xutils.exception.HttpException;
import com.lidroid.xutils.http.ResponseInfo;
import com.lidroid.xutils.http.callback.RequestCallBack;
import com.lidroid.xutils.http.client.HttpRequest;
HttpUtils http =
new
HttpUtils();
http.send(HttpRequest.HttpMethod.GET,
"http://v2.api.dmzj.com/recommend/batchUpdate?category_id=50"
,
new
RequestCallBack <
String > () {
@
Override
public void onFailure(HttpException arg0, String arg1) {
// TODO Auto-generated method stub
}
@
Override
public void onSuccess(ResponseInfo < String > responseInfo) {
// TODO Auto-generated method stub
json = responseInfo.result;
JSONObject obj;
try
{
obj =
new
JSONObject(json);
code = obj.getString(
"code"
);
msg = obj.getString(
"msg"
);
JSONObject data = obj.getJSONObject(
"data"
);
data_category_id = data.getString(
"category_id"
);
data_title = data.getString(
"title"
);
data_sort = data.getString(
"sort"
);
data_data = data.getJSONArray(
"data"
);
obj = data_data.getJSONObject(0);
data_data_id = obj.getString(
"id"
);
data_data_title = obj.getString(
"title"
);
data_data_authors = obj.getString(
"authors"
);
data_data_status = obj.getString(
"status"
);
data_data_cover = obj.getString(
"cover"
);
data_data_num = obj.getString(
"num"
);
obj = data_data.getJSONObject(1);
data_data_id1 = obj.getString(
"id"
);
data_data_title1 = obj.getString(
"title"
);
data_data_authors1 = obj.getString(
"authors"
);
data_data_status1 = obj.getString(
"status"
);
data_data_cover1 = obj.getString(
"cover"
);
data_data_num1 = obj.getString(
"num"
);
obj = data_data.getJSONObject(2);
data_data_id2 = obj.getString(
"id"
);
data_data_title2 = obj.getString(
"title"
);
data_data_authors2 = obj.getString(
"authors"
);
data_data_status2 = obj.getString(
"status"
);
data_data_cover2 = obj.getString(
"cover"
);
data_data_num2 = obj.getString(
"num"
);
tv.setText(
"id:"
+ data_data_id +
",title:"
+ data_data_title +
",authors:"
+ data_data_authors +
",cover:"
+
data_data_cover +
",num:"
+ data_data_num +
";"
);
tv1.setText(
"id:"
+ data_data_id1 +
",title:"
+ data_data_title1 +
",authors:"
+ data_data_authors1 +
",cover:"
+ data_data_cover1 +
",num:"
+ data_data_num1 +
";"
);
tv2.setText(
"id:"
+ data_data_id2 +
",title:"
+ data_data_title2 +
",authors:"
+ data_data_authors2 +
",cover:"
+ data_data_cover2 +
",num:"
+ data_data_num2 +
";"
);
tv3.setText(
"code:"
+ code +
",msg:"
+ msg +
",category_id:"
+ data_category_id +
",title:"
+ data_title +
",sort:"
+ data_sort);
}
catch
(Exception e) {
e.printStackTrace();
}
}
});
可以看出,主要是要对json的数据格式有所理解,无外乎就是对象object和数组arrary两种组合;当然以上解析的只是得到了数据资源,要使用这些资源的话,那就使用就是了~
下面看第三个实用的例子,获取hosts:
这里需要用到工具jar包:jsoup
参考博客园
****.java
Document doc;
try
{
doc = Jsoup.parse(
new
URL(
"http://www.360kb.com/kb/2_122.html"
), 5000);
String title1 = doc.head().getElementsByTag(
"title"
).text();
Document content = Jsoup.parse(doc.toString());
Elements divs = content.select(
"#storybox"
);
Document divcontions = Jsoup.parse(divs.toString());
Elements element1 = divcontions.getElementsByTag(
"pre"
);
String hostsStr = element1.text();
String[] tmp =
null
;
tmp = hostsStr.split(
"====="
);
hostsStr = tmp[1];
tv3.setText(
"jsoup html"
);
tv4.setText(
"HOSTS:"
+ hostsStr);
}
catch
(Exception e) {
e.printStackTrace();
}
public String getHtmlString(String urlString) {
URL url;
try
{
url =
new
URL(urlString);
URLConnection ucon = url.openConnection();
InputStream instr = ucon.getInputStream();
BufferedInputStream bis =
new
BufferedInputStream(instr);
ByteArrayBuffer baf =
new
ByteArrayBuffer(500);
int current = 0;
while
((current = bis.read()) != -1) {
baf.append((byte) current);
}
return
EncodingUtils.getString(baf.toByteArray(),
"utf-8"
);
}
catch
(Exception e) {
return
""
;
}
}
这样,就可以解析到最新的hosts,简单吧。
深圳南山科技园
2016-04-15