HtmlUnit
简单介绍
- HtmlUnit相比于HttpClient功能更加强大,就像一个浏览器,是Junit的扩展测试框架之一,该框架模拟浏览器的行为,开发者可以使用其提供的API对
页面的元素
进行操作。支持HTTP, HTTPS,COOKIE,表单的POST和GET方法
,能够对html文档进行包装,页面的各种元素都可以被当作对象进行调用,对JavaScript的支持也比较好。 - 官网地址:HtmlUnit官网地址
下载后的如下图所示:
当缺少什么包时,可以尝试去该地址mvnrepository下载响应的jar包。
我有下载了一个:jquery-3.5.1.jar
简单demo
- 模拟bugfree登录。并不会真正的打开浏览器
代码如下:
@Test
public void testBugfree() throws FailingHttpStatusCodeException, MalformedURLException, IOException {
String url = "http://146.56.246.116:8081";
WebClient client = new WebClient();
HtmlPage page1 = client.getPage(url);
HtmlElement username= page1.getHtmlElementById("LoginForm_username");
HtmlElement password = page1.getHtmlElementById("LoginForm_password");
username.type("admin");
password.type("123456");
HtmlPage page2 = page1.getHtmlElementById("SubmitLoginBTN").click();
System.out.println(page2.asXml());
}
打印出来的部分内容如下:
- getDemo
注意第一个测试方法中的url2,不用使用encode方法,和httpclient中的不同。而带中文的参数,只能用第二个测试方法,机型编码设置。
package demo;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import org.eclipse.jetty.util.UrlEncoded;
import org.testng.annotations.Test;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.HttpMethod;
import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebRequest;
import com.gargoylesoftware.htmlunit.WebResponse;
import com.gargoylesoftware.htmlunit.util.NameValuePair;
public class GetDemo {
@Test
public void testGetNopara() throws FailingHttpStatusCodeException, IOException {
String url = "http://146.56.246.116:8899/common/skuList";
String url2 = "http://146.56.246.116:8080/Supermarket/analysis/lookupprice?goodsCode={\"pId\":\"123457\"}";
//String url3 = "https://movie.douban.com/j/search_subjects?type=tv&tag=热门&page_limit=50&page_start=0";
// 1.创建client对象
WebClient client = new WebClient();
// 2.创建request
WebRequest request = new WebRequest(new URL(url2), HttpMethod.GET);
// 3.执行请求
Page page = client.getPage(request);
// 4.获得响应
WebResponse response = page.getWebResponse();
// 5.获得响应正文
String result = response.getContentAsString(Charset.defaultCharset().forName("utf-8"));
System.out.println(result);
// 6.关闭Client对象
client.close();
}
@Test
public void testGetBypara() throws FailingHttpStatusCodeException, IOException {
String url3 = "https://movie.douban.com/j/search_subjects";
// 1.创建client对象
WebClient client = new WebClient();
// 2.创建WebRequest
List<NameValuePair> para = new ArrayList<NameValuePair>();
NameValuePair p1 = new NameValuePair("type", "tv");
NameValuePair p2 = new NameValuePair("tag", "热门");
NameValuePair p3 = new NameValuePair("page_limit", "50");
NameValuePair p4 = new NameValuePair("page_start", "0");
para.add(p1);
para.add(p2);
para.add(p3);
para.add(p4);
WebRequest request = new WebRequest(new URL(url3), HttpMethod.GET);
request.setCharset(Charset.forName("utf-8"));
request.setRequestParameters(para);
// 3.执行请求
Page page = client.getPage(request);
// 4.获得响应
WebResponse response = page.getWebResponse();
// 5.获得响应正文
String result = response.getContentAsString(Charset.defaultCharset().forName("utf-8"));
System.out.println(result);
// 6.关闭
client.close();
}
}
结果如下:
- postDemo
该demo展示了两种(form、json)请求体的设置,重点在请求体的设置区别。
package demo;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.Charset;
import org.testng.annotations.Test;
import com.alibaba.fastjson2.JSONObject;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.HttpMethod;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebRequest;
import com.gargoylesoftware.htmlunit.WebResponse;
public class PostDemo {
@Test
public void testByForm() throws FailingHttpStatusCodeException, IOException {
String url = "http://httpbin.org/post";
// 1.创建Client对象
WebClient client = new WebClient();
// 2.创建request
WebRequest request = new WebRequest(new URL(url), HttpMethod.POST);
client.addRequestHeader("Content-Type", "application/x-www-form-urlencoded");
request.setRequestBody("username=vip&password=secret");
WebResponse response = client.getPage(request).getWebResponse();
String result = response.getContentAsString(Charset.forName("utf-8"));
System.out.println(result);
client.close();
}
@Test
public void testByJSON() throws FailingHttpStatusCodeException, IOException {
String url = "http://146.56.246.116:8899/common/fgadmin/login";
WebClient client = new WebClient();
WebRequest request = new WebRequest(new URL(url), HttpMethod.POST);
// 设置请求体
JSONObject user = new JSONObject();
user.put("phoneArea", "86");
user.put("phoneNumber", "2000");
user.put("password", "123456");
client.addRequestHeader("Content-Type", "application/json");
request.setRequestBody(user.toString());
WebResponse response = client.getPage(request).getWebResponse();
String result = response.getContentAsString();
System.out.println(result);
client.close();
}
}
- cookie的获取及使用。
两个client之间的关联。
核心代码:
//获取
Set<Cookie> cookies = client1.getCookieManager().getCookies();
//使用
Iterator<Cookie> i =cookies .iterator();
while (i.hasNext())
{
client2.getCookieManager().addCookie(i.next());
}
package demo;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.charset.Charset;
import java.util.Iterator;
import java.util.Set;
import org.omg.CORBA.PUBLIC_MEMBER;
import org.testng.annotations.Test;
import com.alibaba.fastjson2.JSONObject;
import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;
import com.gargoylesoftware.htmlunit.HttpMethod;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebRequest;
import com.gargoylesoftware.htmlunit.WebResponse;
import com.gargoylesoftware.htmlunit.util.Cookie;
public class CookieDemo {
// 登录的方法,返回cookie
public static Set<Cookie> getCookies(JSONObject user) throws FailingHttpStatusCodeException, IOException {
String url = "http://146.56.246.116:8899/common/fgadmin/login";
WebClient client = new WebClient();
WebRequest request = new WebRequest(new URL(url), HttpMethod.POST);
client.addRequestHeader("Content-Type", "application/json");
request.setRequestBody(user.toString());
WebResponse response = client.getPage(request).getWebResponse();
String result = response.getContentAsString();
System.out.println(result);
client.close();
return client.getCookieManager().getCookies();
}
@Test
public void testGetAddress() throws FailingHttpStatusCodeException, IOException {
String url = "http://146.56.246.116:8899/fgadmin/address/list";
WebClient client = new WebClient();
JSONObject user = new JSONObject();
user.put("phoneArea", "86");
user.put("phoneNumber", "2000");
user.put("password", "123456");
// 调用上述登录方法,并得到cookie
Set<Cookie> cookies_old = getCookies(user);
Iterator<Cookie> cookie = cookies_old.iterator();
// 将登录产生cookie加入到当前的client对象中。
while (cookie.hasNext()) {
client.getCookieManager().addCookie(cookie.next());
}
WebRequest request = new WebRequest(new URL(url), HttpMethod.GET);
WebResponse response = client.getPage(request).getWebResponse();
String result = response.getContentAsString(Charset.forName("utf-8"));
System.out.println(result);
}
}
传入当前已有的cookie,get请求,返回json格式的数据
//传入当前已有的cookie,get请求,返回json格式的数据。
public static JSONObject doGet(String url, Set<Cookie> cookies) throws FailingHttpStatusCodeException, IOException {
//1.创建client对象
WebClient client = new WebClient();
Iterator<Cookie> cookie = cookies.iterator();
while (cookie.hasNext()) {
client.getCookieManager().addCookie(cookie.next());
}
// 2.创建request
WebRequest request = new WebRequest(new URL(url), HttpMethod.GET);
// 3.执行请求
Page page = client.getPage(request);
// 4.获得响应
WebResponse response = page.getWebResponse();
// 5.获得响应正文
String result = response.getContentAsString(Charset.forName("utf-8"));
System.out.println(result);
client.close();
return JSONObject.parseObject(result);
}
更多例子可以去官网快速开始页面查看,部分内容如下:
OkHttp
简单了解
-
OkHttp是square公司推出的一款Android和Java网络请求库,是Android目前最流行的网络库之一。
官网地址
github地址 -
RequestBody类:上传数据的核心类,请求体又文件、JSON字符串等多种形式,通常使用的
FormBody
和MultipartBody
简单demo
- getDemo
package http;
import java.io.IOException;
import org.testng.annotations.Test;
import okhttp3.Call;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.Response;
import okhttp3.ResponseBody;
public class GetDemo {
@Test
public void testGetNoPara() throws IOException {
String url = "http://146.56.246.116:8899/common/skuList";
//1.创建客户端
OkHttpClient client = new OkHttpClient();
//2.创建请求Request
Request request = new Request.Builder().url(url).get().build();
//3.获得Call对象,发送请求,获得响应
Call call=client.newCall(request);
Response response = call.execute();
String result = response.code() + " " + response.message();
ResponseBody body = response.body();
System.out.println(result);
System.out.println(response.headers());
System.out.println(body.string());
body.close();
}
}
- postDemo
package http;
import java.io.IOException;
import org.testng.annotations.Test;
import com.alibaba.fastjson2.JSONObject;
import okhttp3.Call;
import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;
import okhttp3.ResponseBody;
public class PostDemo {
@Test
public void testPostJSON() throws IOException {
String url = "http://146.56.246.116:8899/common/fgadmin/login";
JSONObject user = new JSONObject();
user.put("phoneArea", "86");
user.put("phoneNumber", "2000");
user.put("password", "123456");
OkHttpClient client = new OkHttpClient();
RequestBody body = RequestBody.create(user.toString(), MediaType.parse("application/json"));
Request request = new Request.Builder().url(url).post(body).build();
Call call = client.newCall(request);
Response response = call.execute();
ResponseBody responseBody = response.body();
System.out.println(responseBody.string());
responseBody.close();
}
}
Form类型的代码部分如下:
FormBody.Builder builder = new FormBody.Builder(); builder.add("userName","zhangsan");
builder.add("password","123");
FormBody body = builder.build();
Request request = new Request.Builder().post(body) .url(BASE_URL).build();