Migrating Unit Tests from Selenium to Watir Webdriver

Migrating Unit Tests fromSelenium to Watir Webdriver

Posted by JordonPhillips

Until recently,HiretheWorld did all it’s front-end testing using Selenium.Our tests were quite old and they were pretty slow and unstable, they wouldfail fairly often for no reason, and would pass if the test was run again. Iwas tasked with improving our front-end testing process, the options were totry to optimize our selenium tests or find a new testing framework. Since bothwould require going through every test one by one, I decided to do someresearch of what different frameworks were out there.

Truthis, there aren’t that many – the big two are Selenium and Watir, with a bunchof other, much smaller, less well supported frameworks like windmill.

Watir

Weeventually decided to use watir because it has a very clean and straightforward api, and Watir tests are written in ruby, like our old selenium basedtests were, so it made the transition easier. We used watir webdriver, whichuses selenium webdriver to drive the browser. The main difference between usingwebdriver through watir rather than selenium is that watir’s api is easier touse and has more features, making it much easier to write clean, efficienttests. Watir also has a ton of documentation and examples, thischeat sheet aswell as the watir-webdriversite weregreat resources.

Diagramshowing how our front-end testing setup fits together - test-unit plus Watir,driven by a test script - talking to web-driver, which drives the browser,which then outputs to the screen, or into a virtual framebuffer

Setting UpWatir

Settingup watir is easy, you just need ruby, and a few gems. Some additional downloadscan make the whole process better, such as using xfvb to do headless testing(no browser window) and using chromedriver to use google chrome as your testbrowser, which will speed things up. Also, a unit testing framework is neededto make batch testing easier; we use the test-unit framework.

• Install Ruby
  • Ubuntu: sudo apt-get install ruby
  • Windows: http://rubyinstaller.org/
• Install Ruby Gems
  • Ubuntu: sudo apt-get install rubygems
  • Windows: comes with installer
• Install xvfb (if you want headless tests, linux only)
  • Ubuntu: sudo apt-get install xfvb
• Install the following gems: watir-webdriver, test-unit, headless (if you want headless tests)
  • Ubuntu: sudo gem install watir-webdriver etc...
  • Windows: gem install watir-webdriver etc...
• Download chromedriver if you want to use google chrome for testing (recommended)
  • http://code.google.com/p/chromedriver/downloads/list
  • Place in your PATH
    • Ubuntu: usr/bin
    • Windows: The ruby install folder is a good place, (usually c:/ruby###)/bin
• More detailed instructions here: http://watir.com/installation/

Setting UpTests with Test-Unit

First you need to setup your test unitclass in a new ruby file

require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver"   class TestExample <Test::Unit::TestCase

Next we create the setup and teardownfunctions – these are the functions that test-unit will run before and aftereach test:

# setup is run before every test def setup   $browser = 'chrome'if$browser.nil?   $site = 'http://test.localhost'if$site.nil?     if$headless     require'headless'     $headless = Headless.new     $headless.start   end   if$browser == 'chrome'     $b = Watir::Browser.new:chrome   elsif$browser == 'firefox'     $b = Watir::Browser.new:ff   elsif$browser == 'ie'     $b = Watir::Browser.new:ie   end     $b.goto$site end   # teardown is run after every test def teardown   $b.close   if$headless       $headless.destroy   end end

The setup code will pick a default browserand site if none is specified, then create the browser and go to the specifiedsite. The teardown will close the browser; it can be useful to take ascreenshot here so you can see where your test failed:

# take screenshot of end of test, useful for failures/errors time = Time.new $b.driver.save_screenshot(File.dirname(__FILE__)+'/screenshots/'+@method_name+'_'+ time.strftime('%Y%m%d_%H%M%S')+'.png');

We also add a bit of code at the start ofthe file, after the requires, to handle arguments so that it’s easy to pickyour browser and turn headless on/off using command line arguments:

require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver"   # check arguments for browser or headless specification ARGV.each{|arg|     if arg.downcase.include? 'chrome'         $browser = 'chrome'     elsif arg.downcase.include? 'firefox'         $browser = 'firefox'     elsif arg.downcase.include? 'ff'         $browser = 'firefox'     elsif arg.downcase.include? 'ie'         $browser = 'ie'     elsif arg.downcase.include? 'headless'         $headless = true     end}   class TestExample <Test::Unit::TestCase

Writing Tests

Foreach test case you need a ruby function starting with the word test.Tests generally start with some manipulations of your site, followed by assertsto make sure everything behaved as expected. An example for our site is to fillin some of our forms, press the save button, then go back and see if everythingis still there.

Intest-unit you have three main assert functions, assert_true, assert_false and assert_equal.Your test will pass if the expression after the assert function give theexpected outcome.

assert_equal 'Magic/More Magic', $b.text_field(:name, 'organization_name').value assert_equal 'As mentioned above, we make magic and more magic.', $b.text_field(:name, 'question_38').value assert_equal 'People who like magic and more magic, as opposed to less magic.', $b.text_field(:name,'question_39').value assert_equal 'Im putting stuff into question 41', $b.text_field(:name,'question_41').value

Running Tests

You can run your tests just like runningany ruby script, and you can pass parameters to which we can use to changethings like the browser like above.

• Run test:
  • ruby testexample.rb
• Run test in headless firefox:
  • ruby testexample.rb headless firefox

Tips forwriting good tests

The key to writing efficient tests is notto use explicit waits, our old tests were full of sleep functions to wait forpages to load or for ajax to load, this made them both slow and unstable.Instead, you should wait for an event, watir makes this very easy. I have a fewother tips/tricks that I learned through the process of creating our tests aswell as researching online.

Put everything you can infunctions.

Putting everything you can in functionsmakes it easy to change repeated code, and when writing tests you will repeatthings often. This will make it much easier to change your tests later whiledebugging or when you change your site.

def browse_to_new_project   $b.goto$site+"/designtourney/projects/new" end   def click_logo_design   $b.link(:class, 'logo-design').click end   def form_fill_first_page   $b.text_field(:name, 'organization_name').set('Magic/More Magic')   $b.text_field(:name, 'question_38').set('As mentioned above, we make magic and more magic.')   $b.text_field(:name, 'question_39').set('People who like magic and more magic, as opposed to less magic.')   $b.link(:id=>'show-more').click   $b.text_field(:name, 'question_41').set('Im putting stuff into question 41')   $b.text_field(:name, 'question_45').set('Im putting stuff into question 45') end

Waiting for Ajax

Watirwill automatically wait for a page to load, but that won’t help you if yoursite uses a lot of ajax. A really handy function in watir for dealing with ajaxloads is the wait_while_present function.It’s good practice to have an ajax loader animation displayed while your siteloads an ajax call, using thewait_while_present functionyou can take advantage of this by waiting while the loader is present:

def wait_for_ajax   $b.div(:id, 'ajax-loader').wait_while_present end

Waiting for javascriptanimation

We usethe fallr jqueryplugin on HiretheWorld. It’s a slick looking animated modal box for prompts andalerts. These animations can cause trouble for automated tests. Using the waitfunctions in watir and a little javascript you can make this a little easier. Istill had to use a short sleep here as waiting for a javascript function can bea little finicky, but it still helps make the test more stable.

$b.div(:id, 'fallr').wait_until_present $b.wait_until{$b.execute_script('return $(\'#fallr-wrapper\').is(\':animated\')') == false} sleep0.5 $b.link(:id, 'fallr-button-yes').click $b.div(:id, 'fallr-overlay').wait_while_present

Dealing with timeouts

Nomatter how good you make your tests you will still occasionally get timeouts,you can make watir retry your timeouts using the ruby timeout class.I based this off of code I found here

def load_link(waittime)   begin     Timeout::timeout(waittime)  do     yield   end   rescueTimeout::Error=> e     puts"Page load timed out: #{e}"     retry   end end   def browse_to_new_project load_link(30){$b.goto$site+"/designtourney/projects/new"} end   def click_logo_design load_link(30){$b.link(:class, 'logo-design').click} end

Logging errors to a file

Anotheruseful trick is to log your errors to a file. You can do this by overriding thetest-unit run method.Do this at the top of your test script. I based this code off of this stackoverflow post

moduleTest   module Unit     class TestSuite       alias:old_run:run       def run(result, &progress_block)         old_run(result, &progress_block)         File.open('errors.log', 'w'){|f|           result.faults.each{|err|             case err               whenTest::Unit::Error, Test::Unit::Failure                 f << err.test_name                 f <<"\n"               #not in log file               whenTest::Unit::Pending, Test::Unit::Notification, Test::Unit::Omission               end           }         }       end     end   end end

Retrying errors

Even with well written tests, you can stilloccasionally get random errors for no apparent reason. To help combat this, Icreated a script that will retry all the errors in the error log file fromabove. You can simply run this script after your tests if any errors happen, tosee if they were repeatable errors.

# create string of all args args = "" ARGV.each{|arg| args+=" "+arg } f = File.open("errors.log")or die "Unable to open file..."   # start with an empty array errors=[] f.each_line{|line|   errors.push line }   if errors.length>0   puts'Attempting to resolve errors'   try = 1   while try <= 3     puts"Try number: "+try.to_s     errors.each_with_index{|name, i|       test = /(.+?)\((.+?)\)/.match(name)       ifsystem"ruby \""+test[2]+".rb"+args+"\""         errors[i] = false       end     }     errors.delete(false)     if errors.length == 0       puts'All errors resolved successfully!'       break     end     try+=1   end   File.open('errors.log', 'w'){|f|     errors.each{|error|       f << error       f <<"\n"     }   }   if errors.length != 0     puts'Errors unresolved'   end else   puts'There are no errors in errors.log' end

Conclusion

Rewriting our tests turned out to be agreat success for us. Not only are our tests a lot more stable now, but we alsosaw a 40% speed increase in running our entire testing suite! Below is thefinal code for this example:

require"rubygems" gem "test-unit" require"test/unit" require"watir-webdriver"   # check arguments for browser or headless specification ARGV.each{|arg|     if arg.downcase.include? 'chrome'         $browser = 'chrome'     elsif arg.downcase.include? 'firefox'         $browser = 'firefox'     elsif arg.downcase.include? 'ff'         $browser = 'firefox'     elsif arg.downcase.include? 'ie'         $browser = 'ie'     elsif arg.downcase.include? 'headless'         $headless = true     end}   moduleTest   module Unit     class TestSuite       alias:old_run:run       def run(result, &progress_block)         old_run(result, &progress_block)         File.open('errors.log', 'w'){|f|           result.faults.each{|err|             case err               whenTest::Unit::Error, Test::Unit::Failure                 f << err.test_name                 f <<"\n"               #not in log file               whenTest::Unit::Pending, Test::Unit::Notification, Test::Unit::Omission               end           }         }       end     end   end end   class TestExample <Test::Unit::TestCase   # setup is run before every test   def setup     $browser = 'chrome'if$browser.nil?     $site = 'http://test.localhost'if$site.nil?       if$headless       require'headless'       $headless = Headless.new       $headless.start     end     if$browser == 'chrome'       $b = Watir::Browser.new:chrome     elsif$browser == 'firefox'       $b = Watir::Browser.new:ff     elsif$browser == 'ie'       $b = Watir::Browser.new:ie     end       $timeout_length = 30       load_link($timeout_length){$b.goto$site}   end     # teardown is run after every test   def teardown     # take screenshot of end of test, useful for failures/errors     time = Time.new     $b.driver.save_screenshot(File.dirname(__FILE__)+'/screenshots/'+@method_name+'_'+ time.strftime('%Y%m%d_%H%M%S')+'.png');     $b.close     if$headless         $headless.destroy     end   end     def browse_to_new_project     load_link($timeout_length){$b.goto$site+"/designtourney/projects/new"}   end     def click_logo_design     load_link($timeout_length){$b.link(:class, 'logo-design').click}   end     def form_fill_first_page     $b.text_field(:name, 'organization_name').set('Magic/More Magic')     $b.text_field(:name, 'question_38').set('As mentioned above, we make magic and more magic.')     $b.text_field(:name, 'question_39').set('People who like magic and more magic, as opposed to less magic.')     $b.link(:id=>'show-more').click     $b.text_field(:name, 'question_41').set('Im putting stuff into question 41')     $b.text_field(:name, 'question_45').set('Im putting stuff into question 45')   end     def first_page_asserts type = 'regular'     assert_equal 'Magic/More Magic', $b.text_field(:name, 'organization_name').value     assert_equal 'As mentioned above, we make magic and more magic.', $b.text_field(:name, 'question_38').value     assert_equal 'People who like magic and more magic, as opposed to less magic.', $b.text_field(:name,'question_39').value     assert_equal 'Im putting stuff into question 41', $b.text_field(:name,'question_41').value   end     def wait_for_ajax     $b.div(:id, 'ajax-loader').wait_while_present   end     def load_link(waittime)     begin       Timeout::timeout(waittime)  do       yield     end     rescueTimeout::Error=> e       puts"Page load timed out: #{e}"       retry     end   end     def test_save_for_later     browse_to_new_project       click_logo_design       form_fill_first_page     $b.link(:class, 'save').click     wait_for_ajax       assert_true $b.div(:id, 'fallr').visible?       browse_to_new_project       $b.div(:id, 'fallr').wait_until_present     $b.wait_until{$b.execute_script('return $(\'#fallr-wrapper\').is(\':animated\')') == false}     sleep0.5     $b.link(:id, 'fallr-button-yes').click     $b.div(:id, 'fallr-overlay').wait_while_present       # These assertions make sure the stuff for the first page is still all there     first_page_asserts   end end

If you’ve got any questions about this, justleave a comment and we’ll try to help you out!

 

 

From: http://www.hiretheworld.com/blog/tech-blog/migrating-unit-tests-from-selenium-to-watir-webdriver