登录下载网络资源之Java代码

DevStore，一个源码的下载站。下载源码一个一个下载太麻烦了，所以就想着写个批量下载的。

首先看看单个下载的过程。

例如按关键词转发短信至指定手机号，当下载时需要登录，这里注册一个账号，进行下载。

这里会有两个Post

一个是检查花费的活力值，一个是扣除活力值

Post->Get

两个地址分别为：

http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax?sourceCodeId=715
响应：<pre role="list"><code class="wrappedText focusRow" role="listitem">{"price":0,"status":2}</code>

http://www.devstore.cn/code/costSourceCodeUserPointsAjax?sourceCodeId=715

响应：{
    "status": 1,
    "url": "http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip?87d45372b2babb3a34570fd107cac9747900e64e21876aa613ac0910e7d5d689b9a4cdf80e67f3bac16cad6fa4ca5b88d818e8a07f8f44e19604f2362a677492501b7eed6bbbc114e1687347f00ea624a069c08baf721201f502b82d3ccc3197"
}

真实的下载地址是

http://ds.devstore.cn/dev_store/user/souce_code/file/20150227043055827e0vbr74d/按关键词转发短信至指定手机号.zip

下载的代码

/**
	 * 
	 * @param url  下载地址
	 * @param localPath 保存的路径
	 */
	public static void downLoad(String url,String localPath){
		
		URL fileUrl = null;
		
		HttpURLConnection httpUrl = null;
		
		BufferedInputStream bis = null;
		
		BufferedOutputStream bos = null;
		
		File file = new File(localPath);
		
		
		try {
			fileUrl = new URL(url);
			httpUrl = (HttpURLConnection) fileUrl.openConnection();
			httpUrl.connect();
			bis = new BufferedInputStream(httpUrl.getInputStream());
			bos = new BufferedOutputStream(new FileOutputStream(file));
			int len = 2048;
			byte[] b = new byte[len];
			while ((len = bis.read(b))!= -1) {
				bos.write(b,0,len);
				
			}
			bos.flush();
			bis.close();
			httpUrl.disconnect();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}finally{
			try {
				bis.close();
				bos.close();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			
			
		}
	}

下载的代码好了，接下来就是登陆和获取下载地址的代码

直接把代码贴上吧

public class DownLoad {

	/**
	 * @param 用户名
	 * @param 密码
	 * @return cookie
	 */
	public static Map<String, String> login(String userName,String passWord){
		Map<String, String> cookies = null;
		
		Map<String, String> map = new HashMap<String, String>();
		
		map.put("pwd", passWord);
		map.put("userName", userName);
		Response response = null;
		try {
			response = Jsoup.connect("http://www.devstore.cn/user/login").
						timeout(5000).data(map).ignoreContentType(true).execute();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		cookies = response.cookies();
		return cookies;
	}
	
	/**
	 * 获取下载的链接
	 * @param url  下载的原地址
	 * @param cookies 
	 * @return 
	 */
	public static String getdownUrl(String url,Map<String, String> cookies){
		String result = null;
		Response response = null;
		JSONObject json = null;
		String id = (getInfoByUrl(url)).split("\\.")[0];
		try {
			response = Jsoup.connect("http://www.devstore.cn/code/checkDownSourceCodeUserPointsAjax").
						timeout(5000).data("sourceCodeId",id).cookies(cookies).ignoreContentType(true).execute();
			json = new JSONObject(response.body());
			result = json.get("url").toString().split("\\?")[0];
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (JSONException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		return result;
	}
	
	
	
	/**
	 * 截取
	 * @param url
	 * @return
	 */
	public static String getInfoByUrl(String url){
		String result = null;
		int length = url.split("\\/").length;
		result = url.split("\\/")[length - 1];		
		return result;
	}
	
	
	
	
	/**
	 * 
	 * @param url  下载地址
	 * @param localPath 保存的路径
	 */
	public static void downLoad(String url,String localPath){
		String fileName = null;
		
		URL fileUrl = null;
		
		HttpURLConnection httpUrl = null;
		
		BufferedInputStream bis = null;
		
		BufferedOutputStream bos = null;
		
		fileName = getInfoByUrl(url);
		
		File file = new File(localPath + File.separator + fileName);
		
		
		try {
			fileUrl = new URL(url);
			httpUrl = (HttpURLConnection) fileUrl.openConnection();
			httpUrl.connect();
			bis = new BufferedInputStream(httpUrl.getInputStream());
			bos = new BufferedOutputStream(new FileOutputStream(file));
			int len = 2048;
			byte[] b = new byte[len];
			while ((len = bis.read(b))!= -1) {
				bos.write(b,0,len);
				
			}
			bos.flush();
			bis.close();
			httpUrl.disconnect();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}finally{
			try {
				bis.close();
				bos.close();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
			
			
		}
	}
}

测试代码

		Map<String, String> cookies = DownLoad.login("用户名", "密码");
		
		String url = DownLoad.getdownUrl("http://www.devstore.cn/code/info/715.html", cookies);
		
		DownLoad.downLoad(url, "G:\\src");

批量下载的话可以先获取页面中所有的下载链接

		Document doc = Jsoup.connect("http://www.devstore.cn/code/list/pn1-or0.html").get();

		Element element = doc.select("ul.source_list_ul").first();
		//System.out.println(element);
		Elements links = element.select("h4").select("a");
		for (Element link : links) {
			System.out.println(link.attr("href"));
		}

代码如上。

返回结果类似

/code/info/715.html
/code/info/712.html
/code/info/713.html
/code/info/714.html
/code/info/708.html
....
....

链接有了，然后循环执行下载代码即可