Headless Execution of Selenium Tests in Jenkins

As you probably know, Continuous Testing is becoming a more and more important part of the software development life cycle. Since any change to the product needs to be verified prior to going to production, the idea behind Continuous Testing is to make the whole workflow from a single commit to publishing the new release, 100% automated, and as a result - unattended.

When it comes to functional testing of web applications, the same rule should be applied. In addition to backend unit testing and performance testing, it makes sense to run a set of automated functional tests on the client side, to ensure that everything is working fine, all elements are visible and operating correctly, rendering speed is acceptable and so on. As of now, the most popular and powerful tool for browser automation is Selenium - a free and open source browser automation framework.

We already covered the point of having Selenium automated tests as a part of the Continuous Delivery process in the How to Automate Testing Using Selenium WebDriver, Jenkins and Allure post. Today’s article is about running Selenium tests in “headless” mode, i.e. when no browser window is being displayed.

The importance of having “headless” tests is:

• The majority of Linux server deployments don’t have GUI at all

• The same applies to Windows Core Servers

• When it’s not possible to open the browser in the foreground, for example when the build agent is being used for desktop applications testing which require focus, or you would like to continue using your machine normally while Selenium tests are in progress

Today you will learn how you can launch Selenium tests when Jenkins is in “headless” mode.

Quick Solution

The fastest and the easiest way would be switching to “headless” Selenium WebDriver implementations such as:

• HtmlUnitDriver - wrapper for HtmlUnit - a Java-based browser implementation

• PhantomJSDriver - wrapper for PhantomJS, so called “headless WebKit”, non-GUI browser engine. By the way, WebKit is under the hood of such browsers as Chrome (and derivatives), Safari (including mobile versions), Dolphin browser (Android mobiles), etc.

However, in some situations switching to a headless browser might not be an option, like if the application is under test restrictions (for example application supports Internet Explorer browser only), there is a requirement to test plugin content (Flash, Silverlight, Java), or there is a requirement to support certain technologies like ActiveX or authentication types like single sign-on)

So the remaining part of the article will cover running Selenium tests using real browsers in headless mode on Jenkins nodes.

Test Project

All of the following examples assume the following minimal test scenario:

• Use Selenium Firefox Driver open as an instance of the Firefox browser

• Open BlazeMeter main page

• Print the global title

• Close the browser

The test class is simple:

package com.blazemeter.junit.selenium;


import org.junit.Test;

import org.openqa.selenium.By;

import org.openqa.selenium.WebElement;

import org.openqa.selenium.firefox.FirefoxDriver;


public class JMeterSeleniumDemoTest {

   @Test
   public void openBlazeMeterTest() {

       FirefoxDriver driver = new FirefoxDriver();

       driver.get("http://blazemeter.com");

       WebElement title = driver.findElement(By.xpath("//title"));

       System.out.println("*************************************");
       System.out.println("*" + title.getAttribute("text") + "*");
       System.out.println("*************************************");
       driver.quit();
   }

}

And this is what it looks like in action:

Now let’s see how this test can be executed in Jenkins on different operating systems.

Windows

In general, running Selenium tests on Windows is not possible. The Windows operating system family always has GUI, and even Windows Core Server deployments are capable of running GUI applications without any extra configuration (you just need to get the GUI application somehow using the command-line, for example using Invoke-WebRequest PowerShell cmdlet).

However, it is possible to run your test on behalf of a different user. This way, you can continue working while the test is being executed in the background, without having to worry that your normal keyboard and mouse activity will break the test, and vice versa.

We recommend you install Jenkins master (or node) as a Windows service. This will kill two birds with one stone:

• You will get failover and resilience if the machine is restarted after installing updates, if there is a power outage or if the Jenkins process dies for some reason.

• The process will be run under a different user account, for instance Local System, so the browser window won’t appear in the current session.

Here is the demo of the Test Project execution on a Jenkins node running as a Windows service

As you can see, no Firefox browser window is present, but the test execution is successful. The BlazeMeter page title, which is “JMeter and Performance Testing for DevOps | BlazeMeter” is being printed to the build log.

N.B. in some cases (for example if you need to use Internet Explorer browser or other applications requiring interacting with the Desktop) you may also need to set java.awt.headless Java system property to “false” for Jenkins master and/or slave processes.