利用java/android 模拟网页表单提交。可用于爬虫和wif登录器制作

最新推荐文章于 2021-12-04 19:04:37 发布

Winney_x

最新推荐文章于 2021-12-04 19:04:37 发布

阅读量3.8k

点赞数

分类专栏： android java 文章标签：表单 java 浏览器

本文链接：https://blog.csdn.net/lingchixin/article/details/40753007

版权

java 同时被 2 个专栏收录

48 篇文章 0 订阅

订阅专栏

android

18 篇文章 0 订阅

订阅专栏

学校的wifi登录页又改了。。之前做的wifi登录器没用了。按理说是没换什么接口啊什么的。可是就是不能用了。

我之前做的登录器倒也是省事。直接把填好信息的网页存本地一个。到时候直接用webview加载一下就ok了。简单粗暴。不好的地方就是不能及时的获取返回值。虽然我用webview和网页里面的js代码做了交互，不过可能是因为执行顺序的问题导致不能及时的取得类似于密码错误和用户名不存在这样的消息。所以这次打算时隔半年。用学到的东西再重新写一遍

前一个暑假学会了爬虫，爬虫说简单点实际上就是一般只能抓你看到的网页的内容。优点就是它可以非常快，非常准。所以当时要分析网页里边的各种结构。当时抓取的是人人网的东西，但是它用的是异步加载，一个网页实际上是先加载大体框架再逐步的去加载子网页的。这就对当时的抓取信息造成了一定的不方便，因为之前看过浏览器确实是有开发者这个选项的。就随便去试了一下。然后比较了各浏览器的开发者功能之后发现firefox无论从执行速度还是从开发的功能上都非常厉害。我在研究网页的时候safari经常假死，firefox就非常的快。

以百度为例。想要模拟表单提交，主要是看参数这一栏，然后用namevaluepair去写入到请求中去，再去执行。我们都知道请求分为两种方法。一个是get一个是post。所以只要弄清楚这几个。然后对比一下网页。只要基本上一致就没有问题。之前在创业的时候认识了个做爬虫方面的大神，他告诉我说这种测试要用java去执行一下试试才能知道具体卡在哪，才方便去找问题。比如我的问题是卡在httpclient.executemethod这行中了。然后他告诉我就是参数写的不对。后来才发现的应该是https请求。

这次还是弄了好久。主要遇到的问题就是jar包上的选择。二是wifi登录用的是https的表单，之前都是http方式进行的测试。找了半天才发现还多了一个s呢。talk is cheap .show code。另外commons.httpclient那几个包建议用最新的。百度第一个就是。现在的版本是4.5.3.下好了之后里面不光是只有这一个包。其它的辅助包也要。

import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

public class Test {
	public static void main(String[] args) {
		Map<String, String> params = new HashMap<String, String>();
		params.put("username", "2012011294");
		params.put("password", "*******");
		params.put("buttonClicked", "4");
		sendSSLPostRequest("https://1.1.1.1/login.html", params);
		// logout();
	}

	//
	// public static void Login() {
	// HttpClient httpClient = new HttpClient();
	// String url = "http://1.1.1.1/login.html";
	// PostMethod postMethod = new PostMethod(url);
	// postMethod
	// .addRequestHeader(new Header(
	// "User-Agent",
	// "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:33.0) Gecko/20100101 Firefox/33.0"));
	// postMethod.addRequestHeader(new Header("Referer",
	// "http://1.1.1.1/login.html"));
	// postMethod
	// .addRequestHeader(new Header("Accept",
	// "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"));
	// postMethod.addRequestHeader(new Header("Accept-Encoding",
	// "gzip, deflate"));
	// postMethod.addRequestHeader(new Header("Connection", "keep-alive"));
	// postMethod.addRequestHeader(new Header("Host", "1.1.1.1"));
	//
	// postMethod.addParameter("Content-Type",
	// "application/x-www-form-urlencoded");
	//
	// // 填入各个表单域的值
	// NameValuePair[] data = { new NameValuePair("username", "2012011294"),
	// new NameValuePair("password", "******"),
	// new NameValuePair("buttonClicked", "4") };
	// // 将表单的值放入postMethod中
	// postMethod.setRequestBody(data);
	// // 执行postMethod
	// int statusCode = 0;
	// try {
	// statusCode = httpClient.executeMethod(postMethod);
	// } catch (HttpException e) {
	// e.printStackTrace();
	// } catch (IOException e) {
	// e.printStackTrace();
	// }
	// // HttpClient对于要求接受后继服务的请求，象POST和PUT等不能自动处理转发
	// // 301或者302
	// if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY
	// || statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
	// // 从头中取出转向的地址
	// Header locationHeader = postMethod.getResponseHeader("location");
	// String location = null;
	// if (locationHeader != null) {
	// location = locationHeader.getValue();
	// System.out.println("diandianLogin:" + location);
	// } else {
	// System.err.println("Location field value is null.");
	// }
	// return;
	// } else {
	// System.out.println(postMethod.getStatusLine());
	// String str = "";
	// try {
	// str = postMethod.getResponseBodyAsString();
	// } catch (IOException e) {
	// e.printStackTrace();
	// }
	// System.out.println(str);
	// }
	// postMethod.releaseConnection();
	// return;
	// }
	//
	public static void logout() {
		org.apache.commons.httpclient.HttpClient httpClient = new org.apache.commons.httpclient.HttpClient();
		String url = "http://1.1.1.1/logout.html";
		PostMethod postMethod = new PostMethod(url);
		// 填入各个表单域的值
		org.apache.commons.httpclient.NameValuePair[] data = { new org.apache.commons.httpclient.NameValuePair(
				"userStatus", "1"), };
		// 将表单的值放入postMethod中
		postMethod.setRequestBody(data);
		// 执行postMethod
		int statusCode = 0;
		try {
			statusCode = httpClient.executeMethod(postMethod);
		} catch (HttpException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}
		// HttpClient对于要求接受后继服务的请求，象POST和PUT等不能自动处理转发
		// 301或者302
		if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY
				|| statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
			// 从头中取出转向的地址
			Header locationHeader = postMethod.getResponseHeader("location");
			String location = null;
			if (locationHeader != null) {
				location = locationHeader.getValue();
				System.out.println("diandianLogin:" + location);
			} else {
				System.err.println("Location field value is null.");
			}
			return;
		} else {
			System.out.println(postMethod.getStatusLine());
			String str = "";
			try {
				str = postMethod.getResponseBodyAsString();
			} catch (IOException e) {
				e.printStackTrace();
			}
			System.out.println(str);
		}
		postMethod.releaseConnection();
		return;
	}

		@SuppressWarnings("finally")
	public static String sendSSLPostRequest(String reqURL,
			Map<String, String> params) {
		long responseLength = 0; // 响应长度
		String responseContent = null; // 响应内容
		HttpClient httpClient = new DefaultHttpClient(); // 创建默认的httpClient实例
		X509TrustManager xtm = new X509TrustManager() { // 创建TrustManager
			public java.security.cert.X509Certificate[] getAcceptedIssuers() {
				return null;
			}

			public void checkClientTrusted(
					java.security.cert.X509Certificate[] arg0, String arg1)
					throws java.security.cert.CertificateException {
				// TODO Auto-generated method stub

			}

			public void checkServerTrusted(
					java.security.cert.X509Certificate[] arg0, String arg1)
					throws java.security.cert.CertificateException {
				// TODO Auto-generated method stub

			}
		};
		try {
			// TLS1.0与SSL3.0基本上没有太大的差别，可粗略理解为TLS是SSL的继承者，但它们使用的是相同的SSLContext
			SSLContext ctx = SSLContext.getInstance("TLS");

			// 使用TrustManager来初始化该上下文，TrustManager只是被SSL的Socket所使用
			ctx.init(null, new TrustManager[] { xtm }, null);

			// 创建SSLSocketFactory
			SSLSocketFactory socketFactory = new SSLSocketFactory(ctx);

			// 通过SchemeRegistry将SSLSocketFactory注册到我们的HttpClient上
			httpClient.getConnectionManager().getSchemeRegistry()
					.register(new Scheme("https", 443, socketFactory));

			HttpPost httpPost = new HttpPost(reqURL); // 创建HttpPost
			List<NameValuePair> formParams = new ArrayList<NameValuePair>(); // 构建POST请求的表单参数
			for (Map.Entry<String, String> entry : params.entrySet()) {
				formParams.add(new BasicNameValuePair(entry.getKey(), entry
						.getValue()));
			}
			httpPost.setEntity(new UrlEncodedFormEntity(formParams, "UTF-8"));

			HttpResponse response = httpClient.execute(httpPost); // 执行POST请求
			HttpEntity entity = response.getEntity(); // 获取响应实体

			if (null != entity) {
				responseLength = entity.getContentLength();
				responseContent = EntityUtils.toString(entity, "UTF-8");
				EntityUtils.consume(entity); // Consume response content
			}
			System.out.println("请求地址: " + httpPost.getURI());
			System.out.println("响应状态: " + response.getStatusLine());
			System.out.println("响应长度: " + responseLength);
			System.out.println("响应内容: " + responseContent);
		} catch (KeyManagementException e) {
			e.printStackTrace();
		} catch (NoSuchAlgorithmException e) {
			e.printStackTrace();
		} catch (UnsupportedEncodingException e) {
			e.printStackTrace();
		} catch (ClientProtocolException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		} finally {
			httpClient.getConnectionManager().shutdown(); // 关闭连接,释放资源
			return responseContent;
		}
	}
}

注释掉的login方法是http请求的。那个sendssl的是用了https的。那个logout也是http请求就可以的虽然网址是https。再次修改的时候也都改成https请求吧，这样请求的方法写一个就好，引用包也不会这么乱了。这里主要是给大家一个参考。二是刚写完，有时间再改。

一天之后。

今天试图将java程序移植到android上。以为可以。但是中间遇到了很多问题。而且就算能够把错误都消除了，当new sslsocketfactory的时候还是会卡在这。首先android里边有一个和commen.httpclient一样的包。包名和路径都一样。所以如果出现问题的话需要改一下order and expert里边的包。画上勾。然后将你导入的把移到top。这样可以最先去引用你所导入的包，就酱。然后下面附上android的httpspost

package com.daxiong.wifilogin;

import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLEncoder;
import java.security.SecureRandom;
import java.security.cert.X509Certificate;
import java.util.Map;

import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.KeyManager;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.X509TrustManager;

import org.apache.http.conn.ssl.AllowAllHostnameVerifier;

public class HttpClient {
	private static final AllowAllHostnameVerifier HOSTNAME_VERIFIER = new AllowAllHostnameVerifier();
	private static X509TrustManager xtm = new X509TrustManager() {
		public void checkClientTrusted(X509Certificate[] chain, String authType) {
		}

		public void checkServerTrusted(X509Certificate[] chain, String authType) {
		}

		public X509Certificate[] getAcceptedIssuers() {
			return null;
		}
	};
	private static X509TrustManager[] xtmArray = new X509TrustManager[] { xtm };
	private static HttpsURLConnection conn = null;

	public static InputStream sendPOSTRequestForInputStream(String path,
			Map<String, String> params, String encoding) throws Exception {
		// 1> 组拼实体数据
		// method=save&name=liming&timelength=100
		StringBuilder entityBuilder = new StringBuilder("");
		if (params != null && !params.isEmpty()) {
			for (Map.Entry<String, String> entry : params.entrySet()) {
				entityBuilder.append(entry.getKey()).append('=');
				entityBuilder.append(URLEncoder.encode(entry.getValue(),
						encoding));
				entityBuilder.append('&');
			}
			entityBuilder.deleteCharAt(entityBuilder.length() - 1);
		}
		byte[] entity = entityBuilder.toString().getBytes();
		URL url = new URL(path);
		conn = (HttpsURLConnection) url.openConnection();
		if (conn instanceof HttpsURLConnection) {
			// Trust all certificates
			SSLContext context = SSLContext.getInstance("TLS");
			context.init(new KeyManager[0], xtmArray, new SecureRandom());
			SSLSocketFactory socketFactory = context.getSocketFactory();
			((HttpsURLConnection) conn).setSSLSocketFactory(socketFactory);
			((HttpsURLConnection) conn).setHostnameVerifier(HOSTNAME_VERIFIER);
		}
		conn.setConnectTimeout(5 * 1000);
		conn.setRequestMethod("POST");
		conn.setDoOutput(true);// 允许输出数据
		conn.setRequestProperty("Content-Type",
				"application/x-www-form-urlencoded");
		conn.setRequestProperty("Content-Length", String.valueOf(entity.length));
		OutputStream outStream = conn.getOutputStream();
		outStream.write(entity);
		outStream.flush();
		outStream.close();
		if (conn.getResponseCode() == 200) {
			return conn.getInputStream();
		}
		return conn.getInputStream();
	}

	public static void closeConnection() {
		if (conn != null)
			conn.disconnect();
	}
}

另外这个返回值是一个输入流需要用下面的这个方法转化一下。

	public String inputStream2String(InputStream in) throws IOException {
		StringBuffer out = new StringBuffer();
		byte[] b = new byte[4096];
		for (int n; (n = in.read(b)) != -1;) {
			out.append(new String(b, 0, n));
		}
		return out.toString();