Exploring Spring Boot Testing 1 - Unit Test using Junit

Exploring Spring Boot Testing

• Unit Test using Junit
• SpringBoot Mocks with Mockito
• SpringBoot Testing Spring MVC Web App
• SpringBoot Testing Spring REST APIs

Unit Testing vs Integration Testing

What is Unit Testing?

• Testing an individual unit of code for correctness.
• Provide a fixed inputs
• Expect known output

Example 1 TestCases: Calculator add(int x, int y): int

• input: add(5, 2), output: 7
• input: add(-3, -8), output: -11
• input: add(-3, 1), output: -2
• input: add(0, 0), output: 0

Example 2 TestCases: StringUtils captilize(String data): String

• input: captilize(“Java666Learn”), output: “JAVA666LEARN”
• input: captilize(“JAVA666LEARN”), output: “JAVA666LEARN”
• input: captilize(“Java666Learn”), output: “JAVA666LEARN”
• input: captilize(null), output: ???

Benifit of Unit Testing

• Automated tests
• Better code design
• Fewer bugs and higher reliability
• Increased confidence for code refactoring, did I break anything?
• Basic requirements for DevOps and build pipeline, CICD(continuous integration and continous deployment)

Unit Testing Frameworks

• Junit: Supports creating test cases, Automation of the test cases with pass/fail, Utilities for test setup, teardown and assertions.
• Mockito: Create mocks and stubs, Minimize dependencies on external components

What is Integration Testing?

• Test multiple components together as part of a test plan
• Determine if software units work together as expected
• Identify any negative side effects due to integration
• Can test using mocks/stubs
• Can also test using live integration(database, filesystem)

Junit

How to write Unit Testing?(Demo)

1. Add Maven dependencies for Junit

// pom.xml
<dependencies>
	// ......
   <dependency>
       <groupId>org.junit.jupiter</groupId>
       <artifactId>junit-jupiter</artifactId>
       <version>5.8.2</version>
       <scope>test</scope>
   </dependency>
</dependencies>

2. Create Applicaion Code

// src/main/java/com/example/junitlearndemo/DemoUtils.java
public class DemoUtils {
    public int add(int x, int y) {
        return x + y;
    }
    public Object checkNull(Object object) {
    	return object;
    }
}

3. Create Unit Testing Code
Unit tests have the following structure:

• SetUp: create instance of the class to test
• Execute: call the method you want to test
• Assert: check the result and verify that it is the expected result, click and get more assertions

// src/test/java/com/example/junitlearndemo/DemoUtilsTest.java

public class DemoUtilsTest {
    @Test
    void testDemoUtilsAdd(){
        // set up
        DemoUtils utils = new DemoUtils();
        int expected = 6;
        int unexpected = 8;

        // execute
        int actual = utils.add(2, 4);

        // assert
        Assertions.assertEquals(expected, actual, "2+4 must equal 6");
        Assertions.assertNotEquals(unexpected, actual, "2+4 must eaual 8");
    }


    @Test
    void testCheckNull(){
        // set up
        DemoUtils utils = new DemoUtils();
        String s1 = "hello";
        String s2 = null;

        Assertions.assertNotNull(s1, "Object should not be null");
        Assertions.assertNull(s2, "Object should be null");
    }
}

Junit Lifecycle Methods

When developing tests, we may need to perform common operations.

• Before each test: create objects, setup test data
• After each test: release resource, clean up test data

When developing test, we may need to perform one-time operations.

• One-time setup before all tests: get database connections, connect to remote servers
• One-time clean up after all tests: release database connections, disconnect from remote servers
  

Junit Custom Display Names

We’d like to give custom diaplay names, because of 3 reasons:

• Provide a more descriptive name for the test
• Includes spaces, special characters
• Useful for sharing test reports with project management and non-techies
  

class xxxTest{
	@Test
	@DisplayName("test add function")
    void testDemoUtilsAdd() {
    	// ....
    }
}

But I wish Junit could generate a display name for me, How could we do? Display Name Generators

// demo code
@DisplayNameGeneration(DisplayNameGenerator.Simple.class)
class xxxTest{
	// ...
}

Junit Assertions

• Test Same and NotSame
  

• Test for True or False
  

• Test for Arrays, Iterables and Lines

• Test for Throws and Timeout

Application code:

// com/example/junitlearndemo/DemoUtils.java
public class DemoUtils {
	// ...
    public String throwException(int a) throws Exception {
        if (a < 0) {
            throw new Exception("Value should be greater or equal to 0");
        }
        return "Value is greater or equal to 0";
    }

    public void checkTimeout() throws InterruptedException {
        System.out.println("it's going to sleep!");
        Thread.sleep(2000);
        System.out.println("sleep over!");
    }
}

Unit Testing code:

// com/example/junitlearndemo/DemoUtilsTest.java


public class DemoUtilsTest {
	// ......
    @Test
    @DisplayName("Throw and not throws")
    void testThrowException(){
        DemoUtils utils = new DemoUtils();
        Assertions.assertThrows(Exception.class, ()-> {utils.throwException(-1);}, "should throw exception");
        Assertions.assertDoesNotThrow(() -> {utils.throwException(9);}, "should not throw exception");
    }

    @Test
    @DisplayName("test timeout")
    void testCheckTimeout(){
        DemoUtils utils = new DemoUtils();
        Assertions.assertTimeout(Duration.ofSeconds(3), utils::checkTimeout, "Method should execute in 3 seconds");
    }
}

Ordering Junit Tests

In general, order should not be a factor in unit tests, there should no dependency between tests, all tests should pass regardless of the order in which they are run.However, there are some use cases when we want to control the order.

• We want tests to appear in alphabetical order for reporting purposes.
• Sharing reports with project management and QA team etc.
• Group tests based on functionality or requirements.

How to configure the order/sort algorithm for the test methods?
Annotation @TestMethodOrder

How many kinds of order algorithms?

• MethodOrderer.DisplayName: sorts test methods alphaumerically based on display name.
• MethodOrderer.MethodName: sorts test methods alphaumerically based on method name.
• MethodOrderer.Random: pseudo-random order based on method name.
• MethodOrderer.OrderAnnotation: sort test methods numerically based on @Order annotation.

Example for MethodOrderer.OrderAnnotation
the display order is: testCheckNull(-2) > testDemoUtilsAdd(-1) > testThrowException(1) > testCheckTimeout(5)

// com/example/junitlearndemo/DemoUtilsTest.java
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class DemoUtilsTest {
    @Test
    @Order(-1)
    void testDemoUtilsAdd() {
		// ...
    }

    @Test
    @Order(-2)
    void testCheckNull() {
      // ...
    }

    @Test
    @DisplayName("Throw and not throws")
    @Order(1)
    void testThrowException(){
    	// ...
    }

    @Test
    @DisplayName("test timeout")
    @Order(5)
    void testCheckTimeout(){
       // ...
    }
}

Code Coverage and Test Report

Code Coverage measures how many methods/lines are called by tests. On most teams, 70%~80% is acceptable.

Code Coverage and Test Report with IntelliJ

Run Unit Test with Code Coverage

Generate Coverage Report

Generate Test Report with IntelliJ

Code Coverage and Test Report with Maven

Useful when running as part of DevOps build process.

Install Maven in MacOS
Download Maven or using brew install mvn.

(base) fan@fandeMacBook-Pro junit-learn-demo % mvn --version
Apache Maven 3.8.4

Run unit tests

(base) fan@fandeMacBook-Pro junit-learn-demo % mvn clean test
# ...
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.019 s - in com.example.junitlearndemo.DemoUtilsTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.104 s

Generate unit test reports
configure the pom.xml

// ....
 <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-report-plugin</artifactId>
      <version>3.0.0</version>
      <executions>
          <execution>
              <phase>test</phase>
              <goals>
                  <goal>report</goal>
              </goals>
          </execution>
      </executions>
  </plugin>
</plugins>
// ...

rerun the mvn code

# run tests and execute Surfire report plugin to generate html report.
mvn clean test
# site: add website resources(images, css, etc).
# -DgenerateReports=false don't overwrite existing html reports.
mvn site -DgenerateReports=false 

view the html report in target/site/surefire-report.html.

Handling test failure
By default, Maven Surefire plugin will not generate reports if tests fail. So we need to add configuration in pom.xml.

// ...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0</version>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
    </configuration>
</plugin>
// ...

Rerun the code and view the report, we can build success and show failure tests in report.

mvn clean test
mvn site -DgenerateReports=false 

Show @DisplayName in reports
By default, Maven Surefire plugin will not show @DisplayName in reports. So we need to add configuration in pom.xml.

// ...
<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>3.0.0</version>
   <configuration>
       <testFailureIgnore>true</testFailureIgnore>
       <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
           <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
       </statelessTestsetReporter>
   </configuration>
</plugin>
// ...

Generate code coverage reports

JaCoCo(Java Code Coverage) is a free code coverage library for Java. So we need to add configuration in pom.xml, run the code mvn clean test, view the html in target/site/jacoco/index.html.

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.7</version>
    <executions>
        <execution>
            <id>jacoco-prepare</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>

        <execution>
            <id>jacoco-report</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Conditional Test

We may not need to run all of the tests, there are some use cases:

• Don’t run a test because the method to test is broken, and we are waiting on dev team to fix it.
• A test should only run for a specific version of Java(18) or range of version(13-18).
• A test should only run on a given OS: MacOS, Linux, etc.
• A test should only run if specific environment variables or system properties are set.
  Annotations can be applied at the class level or method level.

Annotations: @Disabled, @EnabledOnOs

• @Disabled: disable a test method.
• @EnabledOnOs: enable test when running on a given OS.

// com/example/junitlearndemo/ConditionalTest.java
public class ConditionalTest {
    @Test
    @Disabled("don't run until jira #111 is resolved")
    public void basicTest() {
        System.out.println("basic test");
    }

    @Test
    @EnabledOnOs(OS.WINDOWS)
    public void testForWindowsOnly() {
        System.out.println("testForWindowsOnly");
    }

    @Test
    @EnabledOnOs(OS.MAC)
    public void testForMACOnly() {
        System.out.println("testForMACOnly");
    }

    @Test
    @EnabledOnOs({OS.MAC, OS.WINDOWS})
    public void testForWindowsMAC() {
        System.out.println("testForWindowsMAC");
    }
}

Annotations: @EnabledOnJre,@EnabledForJreRange

• @EnabledOnJre: enable test for a given java version.
• @EnabledForJreRange: enable test for a given java version range.

// com/example/junitlearndemo/ConditionalTest.java
public class ConditionalTest {
	// ...
    @Test
    @EnabledOnJre(JRE.JAVA_17)
    public void testForJava17Only() {
        System.out.println("testForJava17Only");
    }
    
    @Test
    @EnabledForJreRange(min = JRE.JAVA_8, max = JRE.JAVA_18)
    public void testForJava8To18() {
        System.out.println("testForJava8To18");
    }

    @Test
    @EnabledForJreRange(min = JRE.JAVA_18)
    public void testForMinJava18() {
        System.out.println("testForMinJava18");
    }
}

Annotations: @EnabledIfSystemProperty, @EnabledIfEnvironmentVariable

• @EnabledIfSystemProperty: enable test based on System Property.
• @EnabledIfEnvironmentVariable: enable test based on Environment Variable.

How to set System Property and Environment Variable in IntelliJ?

public class ConditionalTest {
	// ...
    @Test
    @EnabledIfSystemProperty(named = "SYS_PROP", matches = "CI_CD_DEPLOY")
    public void testOnlyForSystemProperty(){
        System.out.println("testOnlyForSystemProperty");
    }
    
    @Test
    @EnabledIfEnvironmentVariable(named = "ENV", matches = "DEV")
    public void testOnlyForEnvironmentVariable(){
        System.out.println("testOnlyForEnvironmentVariable");
    }
}