htmlunit webrequest ajax,HtmlUnit ajax执行问题解决

1.问题

严重: runtimeError: message=[An invalid or illegal selector was specified (selector: 'meta:eq(0)' error: Invalid selector: meta:eq(0)).] sourceName=[] line=[11] lineSource=[null] lineOffset=[0]

@Test

public void submittingForm() throws Exception {

final WebClient webClient = new WebClient();

// Get the first page

final HtmlPage page1 = webClient.getPage("");

// Get the form that we are dealing with and within that form,

// find the submit button and the field that we want to change.

final HtmlForm form = page1.getFormByName("f");

final HtmlSubmitInput button = form.getInputByValue("百度一下");

final HtmlTextInput textField = form.getInputByName("wd");

// Change the value of the text field

textField.setValueAttribute("htmlUnit");

// Now submit the form by clicking the button and get back the second

// page.

final HtmlPage page2 = button.click();

System.out.println(page2.asXml());

webClient.closeAllWindows();

}

2.查找原因

(1)从错误结果看，是jquery 的选择器非法或无效。

(2)选择器怎么会无法定位到元素的呢？是不是html元素没有加载到，而执行的js需要选择定位它，然后导      上述错误？

(3)在百度 上真实搜索htmlUnit 与上述程序打印的html文档做比较。发现确实有不同，前者html页body中确实存在"

(4)怎么会没有的呢？html元素除了静态请求得到外，还有一种是ajax动态请求得到，会不会ajax动态请求在我执行page2.asXml时还没执行？

Usually, you should wait() or sleep() a little, as HtmlUnit can finish before the AJAX response is retrieved from the server, please read this FAQ”，原来ajax请求真会在htmlUnit finish 后执行。点击this FAQ找到解决方法

3.解决方法

(1)第一种方法貌似是让ajax的异步执行重新同步，测试下来,仍然报错。

(2)  第二种方法是让使用HtmlUnit的main线程等待指定的毫秒数，给ajax执行时间，一旦时间用完，返回还未执行的ajax数量，并继续执行main线程，测试下来可行。

(3)  第三种方法是挂起main线程直到条件满足(比如页面中某个元素存储)，或者超出一定的次数。测试下来可行。

第三种方法个人觉得比较好，所以我采用了第三种方式，代码如下：

@Test

public void submittingForm() throws Exception {

final WebClient webClient = new WebClient();

// Get the first page

final HtmlPage page1 = webClient.getPage("");

// Get the form that we are dealing with and within that form,

// find the submit button and the field that we want to change.

final HtmlForm form = page1.getFormByName("f");

final HtmlSubmitInput button = form.getInputByValue("百度一下");

final HtmlTextInput textField = form.getInputByName("wd");

// Change the value of the text field

textField.setValueAttribute("htmlUnit");

// Now submit the form by clicking the button and get back the second

// page.

final HtmlPage page2 = button.click();

for (int i = 0; i < 20; i++) {

if (!page2.getByXPath("//div[@class='c-gap-top c-recommend']").isEmpty()) {

break;

}

synchronized (page2) {

page2.wait(500);

}

}

FileWriter writer = new FileWriter("baidu.html");

writer.write(page2.asXml());

writer.close();

// System.out.println(page2.asXml());

webClient.closeAllWindows();

}