DevStore,一个源码的下载站。下载源码一个一个下载太麻烦了,所以就想着写个批量下载的。
首先看看单个下载的过程。
例如按关键词转发短信至指定手机号,当下载时需要登录,这里注册一个账号,进行下载。
这里会有两个Post
一个是检查花费的活力值,一个是扣除活力值
Post->Get
两个地址分别为:
http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax?sourceCodeId=715
响应:<pre role="list"><code class="wrappedText focusRow" role="listitem">{"price":0,"status":2}</code>
http://www.devstore.cn/code/costSourceCodeUserPointsAjax?sourceCodeId=715
响应:{
"status": 1,
"url": "http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip?87d45372b2babb3a34570fd107cac9747900e64e21876aa613ac0910e7d5d689b9a4cdf80e67f3bac16cad6fa4ca5b88d818e8a07f8f44e19604f2362a677492501b7eed6bbbc114e1687347f00ea624a069c08baf721201f502b82d3ccc3197"
}
真实的下载地址是
http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip
下载的代码
/**
*
* @param url 下载地址
* @param localPath 保存的路径
*/
public static void downLoad(String url,String localPath){
URL fileUrl = null;
HttpURLConnection httpUrl = null;
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
File file = new File(localPath);
try {
fileUrl = new URL(url);
httpUrl = (HttpURLConnection) fileUrl.openConnection();
httpUrl.connect();
bis = new BufferedInputStream(httpUrl.getInputStream());
bos = new BufferedOutputStream(new FileOutputStream(file));
int len = 2048;
byte[] b = new byte[len];
while ((len = bis.read(b))!= -1) {
bos.write(b,0,len);
}
bos.flush();
bis.close();
httpUrl.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try {
bis.close();
bos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
下载的代码好了,接下来就是登陆和获取下载地址的代码
直接把代码贴上吧
public class DownLoad {
/**
* @param 用户名
* @param 密码
* @return cookie
*/
public static Map<String, String> login(String userName,String passWord){
Map<String, String> cookies = null;
Map<String, String> map = new HashMap<String, String>();
map.put("pwd", passWord);
map.put("userName", userName);
Response response = null;
try {
response = Jsoup.connect("http://www.devstore.cn/user/login").
timeout(5000).data(map).ignoreContentType(true).execute();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
cookies = response.cookies();
return cookies;
}
/**
* 获取下载的链接
* @param url 下载的原地址
* @param cookies
* @return
*/
public static String getdownUrl(String url,Map<String, String> cookies){
String result = null;
Response response = null;
JSONObject json = null;
String id = (getInfoByUrl(url)).split("\\.")[0];
try {
response = Jsoup.connect("http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax").
timeout(5000).data("sourceCodeId",id).cookies(cookies).ignoreContentType(true).execute();
json = new JSONObject(response.body());
result = json.get("url").toString().split("\\?")[0];
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return result;
}
/**
* 截取
* @param url
* @return
*/
public static String getInfoByUrl(String url){
String result = null;
int length = url.split("\\/").length;
result = url.split("\\/")[length - 1];
return result;
}
/**
*
* @param url 下载地址
* @param localPath 保存的路径
*/
public static void downLoad(String url,String localPath){
String fileName = null;
URL fileUrl = null;
HttpURLConnection httpUrl = null;
BufferedInputStream bis = null;
BufferedOutputStream bos = null;
fileName = getInfoByUrl(url);
File file = new File(localPath + File.separator + fileName);
try {
fileUrl = new URL(url);
httpUrl = (HttpURLConnection) fileUrl.openConnection();
httpUrl.connect();
bis = new BufferedInputStream(httpUrl.getInputStream());
bos = new BufferedOutputStream(new FileOutputStream(file));
int len = 2048;
byte[] b = new byte[len];
while ((len = bis.read(b))!= -1) {
bos.write(b,0,len);
}
bos.flush();
bis.close();
httpUrl.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
try {
bis.close();
bos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
测试代码
Map<String, String> cookies = DownLoad.login("用户名", "密码");
String url = DownLoad.getdownUrl("http://www.devstore.cn/code/info/715.html", cookies);
DownLoad.downLoad(url, "G:\\src");
批量下载的话可以先获取页面中所有的下载链接
Document doc = Jsoup.connect("http://www.devstore.cn/code/list/pn1-or0.html").get();
Element element = doc.select("ul.source_list_ul").first();
//System.out.println(element);
Elements links = element.select("h4").select("a");
for (Element link : links) {
System.out.println(link.attr("href"));
}
代码如上。
返回结果类似
/code/info/715.html
/code/info/712.html
/code/info/713.html
/code/info/714.html
/code/info/708.html
....
....
链接有了,然后循环执行下载代码即可