Migrating Unit Tests fromSelenium to Watir Webdriver
Posted by JordonPhillips
Until recently,HiretheWorld did all it’s front-end testing using Selenium.Our tests were quite old and they were pretty slow and unstable, they wouldfail fairly often for no reason, and would pass if the test was run again. Iwas tasked with improving our front-end testing process, the options were totry to optimize our selenium tests or find a new testing framework. Since bothwould require going through every test one by one, I decided to do someresearch of what different frameworks were out there.
Truthis, there aren’t that many – the big two are Selenium and Watir, with a bunchof other, much smaller, less well supported frameworks like windmill.
Watir
Weeventually decided to use watir because it has a very clean and straightforward api, and Watir tests are written in ruby, like our old selenium basedtests were, so it made the transition easier. We used watir webdriver, whichuses selenium webdriver to drive the browser. The main difference between usingwebdriver through watir rather than selenium is that watir’s api is easier touse and has more features, making it much easier to write clean, efficienttests. Watir also has a ton of documentation and examples, thischeat sheet aswell as the watir-webdriversite weregreat resources.
Diagramshowing how our front-end testing setup fits together - test-unit plus Watir,driven by a test script - talking to web-driver, which drives the browser,which then outputs to the screen, or into a virtual framebuffer
Setting UpWatir
Settingup watir is easy, you just need ruby, and a few gems. Some additional downloadscan make the whole process better, such as using xfvb to do headless testing(no browser window) and using chromedriver to use google chrome as your testbrowser, which will speed things up. Also, a unit testing framework is neededto make batch testing easier; we use the test-unit framework.
- Install Ruby
- Ubuntu:
sudo apt-get install ruby
- Windows: http://rubyinstaller.org/
- Ubuntu:
- Install Ruby Gems
- Ubuntu:
sudo apt-get install rubygems
- Windows: comes with installer
- Ubuntu:
- Install xvfb (if you want headless tests, linux only)
- Ubuntu:
sudo apt-get install xfvb
- Ubuntu:
- Install the following gems: watir-webdriver, test-unit, headless (if you want headless tests)
- Ubuntu:
sudo gem install watir-webdriver etc...
- Windows:
gem install watir-webdriver etc...
- Ubuntu:
- Download chromedriver if you want to use google chrome for testing (recommended)
- http://code.google.com/p/chromedriver/downloads/list
- Place in your PATH
- Ubuntu:
usr/bin
- Windows: The ruby install folder is a good place, (usually c:/ruby###)/bin
- Ubuntu:
- More detailed instructions here: http://watir.com/installation/
Setting UpTests with Test-Unit
First you need to setup your test unitclass in a new ruby file
require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver" class TestExample <Test::Unit::TestCase |
Next we create the setup and teardownfunctions – these are the functions that test-unit will run before and aftereach test:
# setup is run before every test def setup $browser = 'chrome'if$browser.nil? $site = 'http://test.localhost'if$site.nil? if$headless require'headless' $headless = Headless.new $headless.start end if$browser == 'chrome' $b = Watir::Browser.new:chrome elsif$browser == 'firefox' $b = Watir::Browser.new:ff elsif$browser == 'ie' $b = Watir::Browser.new:ie end $b.goto$site end # teardown is run after every test def teardown $b.close if$headless $headless.destroy end end |
The setup code will pick a default browserand site if none is specified, then create the browser and go to the specifiedsite. The teardown will close the browser; it can be useful to take ascreenshot here so you can see where your test failed:
# take screenshot of end of test, useful for failures/errors time = Time.new $b.driver.save_screenshot(File.dirname(__FILE__)+'/screenshots/'+@method_name+'_'+ time.strftime('%Y%m%d_%H%M%S')+'.png'); |
We also add a bit of code at the start ofthe file, after the requires, to handle arguments so that it’s easy to pickyour browser and turn headless on/off using command line arguments:
require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver" # check arguments for browser or headless specification ARGV.each{|arg| if arg.downcase.include? 'chrome' $browser = 'chrome' elsif arg.downcase.include? 'firefox' $browser = 'firefox' elsif arg.downcase.include? 'ff' $browser = 'firefox' elsif arg.downcase.include? 'ie' $browser = 'ie' elsif arg.downcase.include? 'headless' $headless = true end} class TestExample <Test::Unit::TestCase |
Writing Tests
Foreach test case you need a ruby function starting with the word test
.Tests generally start with some manipulations of your site, followed by assertsto make sure everything behaved as expected. An example for our site is to fillin some of our forms, press the save button, then go back and see if everythingis still there.
Intest-unit you have three main assert functions, assert_true
, assert_false
and assert_equal
.Your test will pass if the expression after the assert function give theexpected outcome.
assert_equal 'Magic/More Magic', $b.text_field(:name, 'organization_name').value assert_equal 'As mentioned above, we make magic and more magic.', $b.text_field(:name, 'question_38').value assert_equal 'People who like magic and more magic, as opposed to less magic.', $b.text_field(:name,'question_39').value assert_equal 'Im putting stuff into question 41', $b.text_field(:name,'question_41').value |
Running Tests
You can run your tests just like runningany ruby script, and you can pass parameters to which we can use to changethings like the browser like above.
- Run test:
ruby testexample.rb
- Run test in headless firefox:
ruby testexample.rb headless firefox
Tips forwriting good tests
The key to writing efficient tests is notto use explicit waits, our old tests were full of sleep functions to wait forpages to load or for ajax to load, this made them both slow and unstable.Instead, you should wait for an event, watir makes this very easy. I have a fewother tips/tricks that I learned through the process of creating our tests aswell as researching online.
Put everything you can infunctions.
Putting everything you can in functionsmakes it easy to change repeated code, and when writing tests you will repeatthings often. This will make it much easier to change your tests later whiledebugging or when you change your site.
def browse_to_new_project $b.goto$site+"/designtourney/projects/new" end def click_logo_design $b.link(:class, 'logo-design').click end def form_fill_first_page $b.text_field(:name, 'organization_name').set('Magic/More Magic') $b.text_field(:name, 'question_38').set('As mentioned above, we make magic and more magic.') $b.text_field(:name, 'question_39').set('People who like magic and more magic, as opposed to less magic.') $b.link(:id=>'show-more').click $b.text_field(:name, 'question_41').set('Im putting stuff into question 41') $b.text_field(:name, 'question_45').set('Im putting stuff into question 45') end |
Waiting for Ajax
Watirwill automatically wait for a page to load, but that won’t help you if yoursite uses a lot of ajax. A really handy function in watir for dealing with ajaxloads is the wait_while_present
function.It’s good practice to have an ajax loader animation displayed while your siteloads an ajax call, using thewait_while_present
functionyou can take advantage of this by waiting while the loader is present:
def wait_for_ajax $b.div(:id, 'ajax-loader').wait_while_present end |
Waiting for javascriptanimation
We usethe fallr jqueryplugin on HiretheWorld. It’s a slick looking animated modal box for prompts andalerts. These animations can cause trouble for automated tests. Using the waitfunctions in watir and a little javascript you can make this a little easier. Istill had to use a short sleep here as waiting for a javascript function can bea little finicky, but it still helps make the test more stable.
$b.div(:id, 'fallr').wait_until_present $b.wait_until{$b.execute_script('return $(\'#fallr-wrapper\').is(\':animated\')') == false} sleep0.5 $b.link(:id, 'fallr-button-yes').click $b.div(:id, 'fallr-overlay').wait_while_present |
Dealing with timeouts
Nomatter how good you make your tests you will still occasionally get timeouts,you can make watir retry your timeouts using the ruby timeout
class.I based this off of code I found here
def load_link(waittime) begin Timeout::timeout(waittime) do yield end rescueTimeout::Error=> e puts"Page load timed out: #{e}" retry end end def browse_to_new_project load_link(30){$b.goto$site+"/designtourney/projects/new"} end def click_logo_design load_link(30){$b.link(:class, 'logo-design').click} end |
Logging errors to a file
Anotheruseful trick is to log your errors to a file. You can do this by overriding thetest-unit run
method.Do this at the top of your test script. I based this code off of this stackoverflow post
moduleTest module Unit class TestSuite alias:old_run:run def run(result, &progress_block) old_run(result, &progress_block) File.open('errors.log', 'w'){|f| result.faults.each{|err| case err whenTest::Unit::Error, Test::Unit::Failure f << err.test_name f <<"\n" #not in log file whenTest::Unit::Pending, Test::Unit::Notification, Test::Unit::Omission end } } end end end end |
Retrying errors
Even with well written tests, you can stilloccasionally get random errors for no apparent reason. To help combat this, Icreated a script that will retry all the errors in the error log file fromabove. You can simply run this script after your tests if any errors happen, tosee if they were repeatable errors.
# create string of all args args = "" ARGV.each{|arg| args+=" "+arg } f = File.open("errors.log")or die "Unable to open file..." # start with an empty array errors=[] f.each_line{|line| errors.push line } if errors.length>0 puts'Attempting to resolve errors' try = 1 while try <= 3 puts"Try number: "+try.to_s errors.each_with_index{|name, i| test = /(.+?)\((.+?)\)/.match(name) ifsystem"ruby \""+test[2]+".rb"+args+"\"" errors[i] = false end } errors.delete(false) if errors.length == 0 puts'All errors resolved successfully!' break end try+=1 end File.open('errors.log', 'w'){|f| errors.each{|error| f << error f <<"\n" } } if errors.length != 0 puts'Errors unresolved' end else puts'There are no errors in errors.log' end |
Conclusion
Rewriting our tests turned out to be agreat success for us. Not only are our tests a lot more stable now, but we alsosaw a 40% speed increase in running our entire testing suite! Below is thefinal code for this example:
require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver" # check arguments for browser or headless specification ARGV.each{|arg| if arg.downcase.include? 'chrome' $browser = 'chrome' elsif arg.downcase.include? 'firefox' $browser = 'firefox' elsif arg.downcase.include? 'ff' $browser = 'firefox' elsif arg.downcase.include? 'ie' $browser = 'ie' elsif arg.downcase.include? 'headless' $headless = true end} moduleTest module Unit class TestSuite alias:old_run:run def run(result, &progress_block) old_run(result, &progress_block) File.open('errors.log', 'w'){|f| result.faults.each{|err| case err whenTest::Unit::Error, Test::Unit::Failure f << err.test_name f <<"\n" #not in log file whenTest::Unit::Pending, Test::Unit::Notification, Test::Unit::Omission end } } end end end end class TestExample <Test::Unit::TestCase # setup is run before every test def setup $browser = 'chrome'if$browser.nil? $site = 'http://test.localhost'if$site.nil? if$headless require'headless' $headless = Headless.new $headless.start end if$browser == 'chrome' $b = Watir::Browser.new:chrome elsif$browser == 'firefox' $b = Watir::Browser.new:ff elsif$browser == 'ie' $b = Watir::Browser.new:ie end $timeout_length = 30 load_link($timeout_length){$b.goto$site} end # teardown is run after every test def teardown # take screenshot of end of test, useful for failures/errors time = Time.new $b.driver.save_screenshot(File.dirname(__FILE__)+'/screenshots/'+@method_name+'_'+ time.strftime('%Y%m%d_%H%M%S')+'.png'); $b.close if$headless $headless.destroy end end def browse_to_new_project load_link($timeout_length){$b.goto$site+"/designtourney/projects/new"} end def click_logo_design load_link($timeout_length){$b.link(:class, 'logo-design').click} end def form_fill_first_page $b.text_field(:name, 'organization_name').set('Magic/More Magic') $b.text_field(:name, 'question_38').set('As mentioned above, we make magic and more magic.') $b.text_field(:name, 'question_39').set('People who like magic and more magic, as opposed to less magic.') $b.link(:id=>'show-more').click $b.text_field(:name, 'question_41').set('Im putting stuff into question 41') $b.text_field(:name, 'question_45').set('Im putting stuff into question 45') end def first_page_asserts type = 'regular' assert_equal 'Magic/More Magic', $b.text_field(:name, 'organization_name').value assert_equal 'As mentioned above, we make magic and more magic.', $b.text_field(:name, 'question_38').value assert_equal 'People who like magic and more magic, as opposed to less magic.', $b.text_field(:name,'question_39').value assert_equal 'Im putting stuff into question 41', $b.text_field(:name,'question_41').value end def wait_for_ajax $b.div(:id, 'ajax-loader').wait_while_present end def load_link(waittime) begin Timeout::timeout(waittime) do yield end rescueTimeout::Error=> e puts"Page load timed out: #{e}" retry end end def test_save_for_later browse_to_new_project click_logo_design form_fill_first_page $b.link(:class, 'save').click wait_for_ajax assert_true $b.div(:id, 'fallr').visible? browse_to_new_project $b.div(:id, 'fallr').wait_until_present $b.wait_until{$b.execute_script('return $(\'#fallr-wrapper\').is(\':animated\')') == false} sleep0.5 $b.link(:id, 'fallr-button-yes').click $b.div(:id, 'fallr-overlay').wait_while_present # These assertions make sure the stuff for the first page is still all there first_page_asserts end end |
If you’ve got any questions about this, justleave a comment and we’ll try to help you out!
From: http://www.hiretheworld.com/blog/tech-blog/migrating-unit-tests-from-selenium-to-watir-webdriver