Use JUnit rules to debug failed API tests

Last Updated on by

Post summary: What are JUnit rules and how to use them to improve debugging of failed API tests.

What are JUnit rules

Rules are easy way to separate tests from non-tests code. In many cases some kind of setup is required before starting the tests. Rules provide a way to define this setup code externally and just access it from your tests.

How to use JUnit rules

Instantiate a public variable with rule class you want to use. Depending on specific rule you can invoke methods on this public variable. Example bellow is the simplest that gives you the name of current test method being executed.

@Rule
public TestName name = new TestName();

@Test
public void testPrintMethodName() {
	assertEquals("testPrintMethodName", name.getMethodName());
}

Types of JUnit rules

Bellow are rules that JUnit provides:

  • TemporaryFolder – allows creation of files and folders that gets deleted when test method finishes.
  • ExternalResource – sets up external resources (file, socket, database connection) and then releases them. Same can be accomplished in @Before and @After methods.
  • ErrorCollector – execution of test continues after the first error and successive errors are collected and reported after test finishes.
  • Verifier – additional asserting on test conditions.
  • TestWatcher – has access to tests output – when test starts, finishes and the test result.
  • TestName – gives current executing test method name.
  • Timeout – sets timeout for all the test methods in the class. If some test takes longer it is terminated and failed.
  • ExpectedException – very handy way to test whether method throws correct exception. It is possible to use @Test(expected = NullPointerException.class), but it does not allow you to check what is the exception message.

ClassRule

@Rule annotation creates new instance of the rule class before each and every test method is run. In some cases rule object is needed into test class initialisation method (annotated with @BeforeClass). In order to have it into initialisation method a @ClassRule annotation has to be used instead. Then the rule object is instantiated only once before @BeforeClass method has run, so rule is available in it. @ClassRule is good to be used in situations where there are expensive resources to be created – better to create them on test class initialisation rather before each test method. More details about execution sequence can be found in JUnit methods execution sequence post.

Debug API tests

API tests generally are sequence of requests. One request depends from previous as it takes some data out of it. If some of the request in the chain fails you will need whole chain to be able to debug and trace why exactly whole scenario failed.

Store API calls in a Queue

It is good idea to have one class that is sending API requests and returning responses. Bellow is a simple example of such class. In real life makeRequest methods will accept some parameters or request object and result will be some response object, not String.

public class RequestUtils {

	private static final Queue<String> MESSAGES_QUEUE = new LinkedList<>();

	public static String makeSomeRequest(String request) {
		getMessages().add(request);
		String response = "makeSomeRequestResponse";
		getMessages().add(response);
		return response;
	}

	public static String makeAnotherRequest(String request) {
		getMessages().add(request);
		String response = "makeAnotherRequestResponse";
		getMessages().add(response);
		return response;
	}

	public static void printMessages() {
		for (String message : getMessages()) {
			System.out.println(message);
		}
		clearMessages();
	}

	public static void clearMessages() {
		getMessages().clear();
	}

	private static Queue<String> getMessages() {
		return MESSAGES_QUEUE;
	}
}

Class collects all requests and responses in a queue of Strings. It has printMessages() and clearMessages() methods. It also has a getMessages() method which just returns the queue. This method is not bringing real value to code, but rather used to easy switch to different types of queues.

Extend TestWatcher to have access to test results

As stated above TestWatcher provides access to tests output without ability to modify it. Extending TestWatcher gives you access to those methods:

public class MessagesQueueRule extends TestWatcher {

	protected void succeeded(Description description) {
		RequestUtils.clearMessages();
	}

	protected void failed(Throwable e, Description description) {
		RequestUtils.printMessages();
	}

	protected void skipped(AssumptionViolatedException e, 
			Description description) {
		RequestUtils.printMessages();
	}
}

On success messages queue get cleared, on skipped or failed it gets printed to enable you to debug.

Tests are sequence of requests and responses

Test bellow is just an example to show how an API test generally looks like. It is possible to do some method chaining, but this is out of scope of current post.

@Test
public void test1() {
	String result1 = RequestUtils.makeSomeRequest("test1request1");
	String result2 = RequestUtils.makeAnotherRequest(result1);
	String actualResult = RequestUtils.makeAnotherRequest(result2);
	assertEquals("makeAnotherRequestResponse", actualResult);
}

If assertEquals() fails all requests/response in the test method will get printed into the logs.

Multi threading

You might have noticed that current solution is not thread safe as Queue is one and the same and can be accessed from many threads which will lead to ConcurrentModification exception. In Avoid multithreading problems in Java using ThreadLocal post there is solution using ThreadLocal.

Conclusion

Rules provide flexibility make what ever is needed in your tests. They are easy way to extract code which is not test logic to external classes. All the details about rules are available at JUnit Rules page. In current post I showed you easy way to store and output requests in case of API testing. Code shown above is available in GitHub java-samples/junit repository.

If you find this post useful, please share it to reach more people. Sharing is caring!
Share on FacebookShare on LinkedInTweet about this on TwitterShare on Google+Email this to someone
Category: API Automation, Java, Unit testing | Tags: