Ajax testing with Selenium using waitForCondition
An often-asked question on the selenium-users mailing list is how to test Ajax-specific functionality with Selenium. The problem with Ajax testing is that the HTML page under test is modified asynchronously, so a plain Selenium assert or verify command might very well fail because the element being tested has not been created yet by the Ajax call. A quick-and-dirty solution is to put a pause command before the assert, but this is error-prone, since the pause might be not sufficient on a slow machine, while being unnecessarily slow on a faster one.
A better solution is to use Dan Fabulich's waitForCondition extension. But first, a word about Selenium extensions.
If you've never installed a Selenium extension, it's actually pretty easy. You should have a file called user-extensions.js.sample in the same directory where you installed the other core Selenium files (such as TestRunner.html and selenium-api.js). You need to rename that file as user-extensions.js, so that it will be automatically picked up by Selenium the next time you run a test. To install a specific extension such as waitForCondition, you need to download and unpack the extension's zip file, then add the contents of the user-extensions.js.waitForCondition file to user-extensions.js. That's all there is to it.
Now back to testing Ajax functionality. For the MailOnnaStick application, Titus and I used Ian Bicking's Commentary application as an example of Ajax-specific functionality that we wanted to test with Selenium. See this post of mine for details on how Commentary works and how we wrote our initial tests. The approach we took initially was the one I mentioned in the beginning, namely putting pause commands before the Ajax-specific asserts. Interestingly enough, this was the only Selenium test that was breaking consistently in our buildbot setup, precisely because of speed differences between the machines that were running buildbot. So I rewrote the tests using waitForCondition.
What does waitForCondition buy you? It allows you to include arbitrary Javascript code in your commands and assert that a condition (written in Javascript) is true. The test will not advance until the condition becomes true (hence the wait prefix). Or, to put it in the words of Dan Fabulich:
waitForCondition: Waits for any arbitrary condition, by running a JavaScript snippet of your choosing. When the snippet evaluates to "true", we stop waiting.
Here's a quick example of a Selenium test table row that uses waitForCondition (note that the last value in the 3rd cell is a timeout value, in milliseconds):
| waitForCondition | var value = selenium.getText("//textarea[@name='comment']"); value == "" | 10000 |
What I'm doing here is asserting that a certain HTML element is present in the page under test. For the Commentary functionality, the element I chose is the text area of the form that pops up when you double-click on the page. This element did not exist before the double-click event, so by asserting that its value is empty, I make sure that it exists, which means that the asynchronous Ajax call has completed. If the element is not there after the timeout has expired (10 seconds in my case), the assertion is marked as failed.
To get to the element, I used the special variable selenium, which is available for use in Javascript commands that you want to embed in your Selenium tables. The methods that you can call on this variable are the same methods that start with Selenium.prototype in the file selenium-api.js. In this case, I called getText, which is defined as follows in selenium-api.js:
Selenium.prototype.getText = function(locator) {
var element = this.page().findElement(locator);
return getText(element).trim();
};
This function gets a locator as its only argument. In the example above, I used the XPath-style locator "//textarea[@name='comment']" -- which means "give me the HTML element identified by the tag textarea, and whose attribute name has the value 'comment'". The value of this HTML element is empty, so this is exactly what I'm asserting in the test table: value == "".
You might wonder how I figured out which element to use in the assertion. Easy: I inspected the HTML source of the page under test before and after I double-clicked on the page, and I identified an element which was present only after the double-click event.
The other scenario I had to test was that the Commentary post-it note is not present anymore after deleting the commentary. Again, I looked at the HTML page under test before and after clicking on the Delete link, and I identified an element which was present before, and not present after the deletion. Here is the waitForCondition assertion I came up with:
| waitForCondition | var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1; | 10000 |
Here I used selenium.page() to get to the HTML page under test, then bodyText() to get to the text of the body tag. I then searched for the text that I was NOT expecting to find anymore, and I asserted that the Javascript indexOf() method returned -1 (i.e. the text was indeed not found.)
Here is the test table for the Commentary functionality in its entirety:
| TestCommentary | ||
| open | /message/20050409174524.GA4854@highenergymagic.org | |
| dblclick | //blockquote | |
| waitForCondition | var value = selenium.getText("//textarea[@name='comment']"); value == "" | 10000 |
| store | javascript{Math.round(1000*Math.random())} | var |
| type | username | user${var} |
| type | user${var}@mos.org | |
| type | comment | hello there from user${var} |
| click | //form//button[1] | |
| waitForCondition | var value = selenium.getText("//div[@class='commentary-comment commentary-inline']"); value.match(/hello there from user${var}/); | 10000 |
| verifyText | //div[@class="commentary-comment commentary-inline"] | regexp:hello there from user${var} |
| clickAndWait | //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"] | |
| type | q | user${var} |
| clickAndWait | //input[@type='submit' and @value='search'] | |
| verifyValue | q | user${var} |
| assertTextPresent | Query: user${var} | |
| assertTextPresent | in Re: [socal-piggies] meeting Tues Apr 12th: confirmed | |
| open | /message/20050409174524.GA4854@highenergymagic.org | |
| assertTextPresent | hello there from user${var} | |
| assertTextPresent | delete | |
| click | link=delete | |
| waitForCondition | var allText = selenium.page().bodyText(); var unexpectedText = "hello there from user${var}" allText.indexOf(unexpectedText) == -1; | 10000 |
| assertTextNotPresent | hello there from user${var} | |
| assertTextNotPresent | delete | |
| clickAndWait | //div/div[position()="1" and @style="font-size: 80%;"]/a[position()="2" and @href="/search"] | |
| type | q | user${var} |
| clickAndWait | //input[@type='submit' and @value='search'] | |
| verifyValue | q | user${var} |
| assertTextPresent | Query: user${var} | |
| assertTextPresent | no matches | |
For more details on how to identify HTML elements that you want to test using Selenium test tables, see this post on useful Selenium tools, and this post on using the Selenium IDE.
Update: Since images are worth thousands and thousands of words, here are 2 screencasts (no sound) of running the Commentary test with Selenium: one in Windows AVI format, and the other one in Quicktime MOV format (you might be better off saving the files to your local disk before viewing them.)
扫一扫
579
到【灌水乐园】发言
13 Comments:
As usual, great work, Grig! :-)
The screencasts truly are worth a thousand words. :-) The syntax of the test (especially the waitForCondition command) is starting to look scary, though. Very powerful, but scary looking-- like PHP or Perl. :-) I'm kinda craving a mini-port of Ruby for the JavaScript "platform" so we can get some of its DSL and "anonymous code block" goodness.
By Jason Huggins, at 6:13 PM
Jason,
Thanks for the comments. Yeah, it is kind of scary to see all that Javascript code in there, but on the other hand it does offer you tremendous flexibility (so you can shoot yourself in the foot at your ease :-)
Ruby-in-Javascript sounds pretty enticing to me :-)
By Grig Gheorghiu, at 6:18 PM
On the first example you need to place a ';' at the end of the first example or it doesn't work.
var value = selenium.getText("//textarea[@name='comment']"); value == "";
By Anonymous, at 12:28 PM
Fascination blog..u made my life simpler..
I have a requirement to test the AJAX pages in the web app.
Scenario :
1)Enter a value in a edit box . (Hello)
2) Click on a button which does a partial page refresh or a partial post back.
3)The value entered in the edit box will show up in the page on step (2).
4)Need to check or read this value.(Check for Hello)
sel.waitForCondition("String str=sel.getBodyText();str.indexOf(Hello)!=-1;","30000");
error on running:
com.thoughtworks.selenium.SeleniumException: missing ; before statement
Whts to be done..??
By Hari, at 7:55 AM
In my next project I have to test an ajax application. I am trying to find out the challenges posed by ajax applications w.r.t testing. Will the normal automation tools work or are there any specific tools to test ajax apps? I am also trying to findout why these specific tools are required? Why the already existing tools cant do the job?
Any useful links in this regard will be helpful.
By Nagendra, at 1:35 AM
So waitForCondition works by polling for a condition, but could some method of hooking in to the AJAX success callback or the general AJAX response handler be feasible?
-Trey
By Anonymous, at 10:12 PM
At my previous company we were using Selenium to test an AJAX-y app.
In the end we put in some code that extended the existing xxxAndWait commands to check that any outstanding requests had been serviced (a la COMs ->AddReference model).
It worked pretty well, and didn't need us to re-implement too much code. Although it was pretty ugly code :)
By Paul Ingles, at 1:39 PM
Thanks, this is good, I was able to use the WaitForCondition command right away after reading your post..
Dr. Java
By Dr. Java, at 12:52 PM
Hi,
In my version of selenium (0.8.1) the function call getText(..) will fail with an error.
I browsed a little through the selenium-browserbot.js and noticed that the function PageBot.findElement() will throw a SeleniumError if the element is not found. Probably this is a change in the newer versions of Selenium.
My quick and dirty fix was to surround the getText() and findElement() calls with try..catch blocks.
Thank you for this usefull post,
Stefan Hornea
By Anonymous, at 9:58 AM
Check out WebAii automation at www.artoftest.com. Seems like a new Ajax framework for anyone doing .NET testing. Looks like the beta is out and the final release will be sometime mid year...
By Anonymous, at 8:51 PM
Wow, people love to make thinks more and more complicated :)
An simpler solution would be to pass your XMLHTTPRequest
from async to sync mode for in-browser testing session. Of course the ajax framework needs to support this some.
By Vitaliy Shevchuk, at 1:51 AM
Very useful, Grig!. I am having trouble with mouseover in selenium.
The page that I am testing has a AJAX call only when you go to the bottom of the page. I get that with mouseover I can do that in selenium, but havent been able to successfully do it.
Any pointers?
By Anonymous, at 11:46 AM
The AJAX creates problems for all testing tools, because the calls to XMLHTTPRequest do not generate Browser events - there are no “complete” events for the AJAX and there is no way to get real browser ready status. Without the "complete" events it is hard to create reliable automation solution for AJAX.
To my knowledge only SWExplorerAutomation (SWEA) from Webius(http://webiussoft.com) monitors the Browser network activity to get true browser ready status.
By alex_f_il, at 9:21 AM
Post a Comment