ADS-Lecture 07 Testing

Lecture 07 Testing

How Does a Programmer Know That Their Code Works?

In the real world, programmers believe their code works because of tests they write themselves.

• Knowing that it works for sure is usually impossible.
• This will be our new way.

Sorting: The McGuffin for Our Testing Adventure

To try out this new way™, we need a task to complete.

• Let’s try to write a method that sorts arrays of Strings.

In this lecture we’ll write sort, as well as our own test for sort.

• Even crazier idea: We’ll start by writing testSort first!

Ad-Hoc Testing vs. JUnit

Ad-Hoc Testing is Tedious

public class TestSort {
    /** Tests the sort method of the Sort class. */
    public static void testSort() {
        String[] input = {"beware", "of", "falling", "rocks"};
        String[] expected = {"beware", "falling", "of", "rocks"};
        Sort.sort(input);
 
        for (int i = 0; i < input.length; i += 1) {
    	      if (!input[i].equals(expected[i])) {
    	          System.out.println("Mismatch at position " + i + ",   									   
    	                                 expected: '" + expected[i] + 
                           				   "', but got '" + input[i] + "'");
    	          return;
  	      	  }
        }
        //JUnit saves us the trouble of writing code like this (and more!).
        
    }
 
    public static void main(String[] args) {
		testSort();
    }
}

JUnit: A Library for Making Testing Easier (example below)

public class TestSort {
  /** Tests the sort method of the Sort class. */  
  public static testSort() {
    String[] input = {"cows", "dwell", "above", "clouds"};
    String[] expected = {"above", "cows", "clouds", "dwell"};
    Sort.sort(input);
 
    org.junit.Assert.assertArrayEquals(expected, input);
  }
 
  public static void main(String[] args) {
    testSort();
  }
}

Selection Sort

Back to Sorting: Selection Sort

Selection sorting a list of N items:

• Find the smallest item.

• Move it to the front.

• Selection sort the remaining N-1 items (without touching front item!).

As an aside: Can prove correctness of this sort using invariants.

public class Sort {
	/** Sorts strings destructively. */
	public static void sort(String[] x) {
        sort(x, 0);
	}

	/** Sorts x starting at position start. */
	private static void sort(String[] x, int start) {
	    if (start == x.length) {
	        return;
        }
	    int smallestIndex = findSmallest(x, start);
	    swap(x, start, smallestIndex);
	    sort(x, start + 1);
    }

	/** Swap item a with b. */
	public static void swap(String[] x, int a, int b) {
	    String temp = x[a];
	    x[a] = x[b];
	    x[b] = temp;
    }

	/** Return the index of the smallest String in x, starting at start. */
	public static int findSmallest(String[] x, int start) {
        int smallestIndex = start;
        for (int i = start; i < x.length; i += 1) {
            int cmp = x[i].compareTo(x[smallestIndex]);
            // from the internet, if x[i] < x[smallestIndex], cmp will be -1.
            if (cmp < 0) {
                smallestIndex = i;
            }
        }
        return smallestIndex;
    }
}

import org.junit.Test;
import static org.junit.Assert.*;

/** Tests the the Sort class. */
public class TestSort {
    /** Test the Sort.sort method. */
    @Test
    public void testSort() {
        String[] input = {"i", "have", "an", "egg"};
        String[] expected = {"an", "egg", "have", "i"};
        Sort.sort(input);

        assertArrayEquals(expected, input);
    }

    /** Test the Sort.findSmallest method. */
    @Test
    public void testFindSmallest() {
        String[] input = {"i", "have", "an", "egg"};
        int expected = 2;

        int actual = Sort.findSmallest(input, 0);
        assertEquals(expected, actual);

        String[] input2 = {"there", "are", "many", "pigs"};
        int expected2 = 2;

        int actual2 = Sort.findSmallest(input2, 2);
        assertEquals(expected2, actual2);
    }

    /** Test the Sort.swap method. */
    @Test
    public void testSwap() {
        String[] input = {"i", "have", "an", "egg"};
        int a = 0;
        int b = 2;
        String[] expected = {"an", "have", "i", "egg"};

        Sort.swap(input, a, b);
        assertArrayEquals(expected, input);
    }
}

The Evolution of Our Design

• Created testSort: testSort()

• Created a sort skeleton: sort(String[] inputs)

• Created testFindSmallest: testFindSmallest()

• Created findSmallest: String findSmallest(String[] input)

  • Used Google(or whatever else) to find out how to compare strings.

• Created testSwap: testSwap()

• Created swap: swap(String[] input, int a, int b)

  • Used debugger to fix.

• Changed findSmallest: int findSmallest(String[] input)

Now we have all the helper methods we need, as well as tests that make us pretty sure that they work! All that’s left is to write the sort method itself.

Very Tricky Problem

Without changing the signature of public static void sort(String[] a), how can we use recursion? What might the recursive call look like?

public static void sort(String[] x) {
    int smallest = findSmallest(x);
    swap(inputs, 0, smallest);
    // recursive call??
}

Bad But Tempting Solution

public static void sort(String[] x) {
    int smallest = findSmallest(x);
    swap(inputs, 0, smallest);
    //sort(x[1:]); ← Would be nice, but not possible!
    //java is not python
}

Some languages support sub-indexing into arrays. Java does not.

• Bottom line: No way to get address of the middle of an array.

Good Solution

Define a private helper method.

public static void sort(String[] x) {
    sort(x, 0); 
}

/** Destructively sorts x starting at index k */
public static void sort(String[] x, int k) {
    ...
    sort(x, k + 1);
}

Major Design Flaw in findSmallest

We didn’t properly account for how findSmallest would be used.

• Example: Want to find smallest item from among the last 4:

• We need another parameter so that it’s actually useful for sorting.

Improvement of our Design

• Added helper method: sort(String[] inputs, int k)

• Used debugger to realize fundamental design flaw in findSmallest

• Modified findSmallest: int findSmallest(String[] input, int k)

And We’re Done!

Often, development is an incremental process that involves lots of task switching and on the fly design modification.

Tests provide stability and scaffolding.

• Provide confidence in basic units and mitigate possibility of breaking them.

• Help you focus on one task at a time.

In larger projects, tests also allow you to safely refactor! Sometimes code gets ugly, necessitating redesign and rewrites .

One remaining problem: Sure was annoying to have to constantly edit which tests were running. Let’s take care of that.

Simpler JUnit Tests (using two new syntax tricks)

Simple JUnit

New Syntax #1: org.junit.Assert.assertEquals(expected, actual);

• Tests that expected equals actual.

• If not, program terminates with verbose message.

JUnit does much more:

• Other methods like assertEquals include assertFalse, assertNotNull, etc., see http://junit.org/junit4/javadoc/4.12/org/junit/Assert.html
• Other more complex behavior to support more sophisticated testing.

Better JUnit

The messages output by JUnit are kind of ugly, and invoking each test manually is annoying.

New Syntax #2 (just trust me):

• Annotate each test with @org.junit.Test.

• Change all test methods to non-static.

  • Yes this is weird, as it implies you’d be instantiating TestSort.java. In fact, JUnit runners do this. I don’t know why.

• Use a JUnit runner to run all tests and tabulate results.

  • IntelliJ provides a default runner/renderer. OK to delete main.
  • If you want to use the command line instead, see google/stack overflow/CSDN etc. Not preferred.
  • Rendered output is easier to read, no need to manually invoke tests!

There is a lot of black magic happening here! Just accept it all for now.

Even Better JUnit

It is annoying to type out the name of the library repeatedly, e.g. org.junit.Test and org.junit.Assert.assertEquals.

New Syntax #3: To avoid this we’ll start every test file with:

import org.junit.Test;
import static org.junit.Assert.*;

This will magically eliminate the need to type org.junit or org.junit.Assert (more later on what these imports really mean).

Testing Philosophy

Autograder Driven Development (ADD)

The worst way to approach programming:

• Read and (mostly) understand the spec.

• Write entire program.

• Compile. Fix all compilation errors.

• Send to autograder. Get many errors.

• Until correct, repeat randomly:

  • Run autograder.
  • Add print statements to zero in on the bug.
  • Make changes to code to try to fix bug.

This workflow is slow and unsafe!

Note: Print statements are not inherently evil. While they are a weak tool, they are very easy to use.

Test-Driven Development (TDD)

Steps to developing according to TDD:

• Identify a new feature.

• Write a unit test for that feature.

• Run the test. It should fail. (RED)

• Write code that passes test. (GREEN)

  • Implementation is certifiably good!

• Optional: Refactor code to make it faster, cleaner, etc.

Not required. You might hate this!

• But testing is a good idea.
  • Unit Test
  • Integration Test

More On JUnit (Extra)

What is an Annotation?

Annotations (like org.junit.Test) don’t do anything on their own.

@Test
public void testSort() {
    ...
}

Runner uses reflections library to iterate through all methods with “Test” annotation.

Sample Runner Pseudocode

List<Method> L = getMethodsWithAnnotation(TestSort.class, org.junit.Test);
int numTests = L.size();
int numPassed = 0;
for (Method m : L) {
    result r = m.execute();
    if (r.passed == true) { numPassed += 1; }
    if (r.passed == false) { System.out.println(r.message); }
}
System.out.println(numPassed + “/” + numTests + “ passed!”);