by Andres Almiray
Part 2 in a series that explores libraries for improving Java developers' productivity and enhancing the development experience
Table of Contents
Choosing the Base Testing Framework
Testing the Controller
Integration Testing
Conclusion
See Also
About the Author
In the first part of this series, we learned how to build a simple JavaFX application that’s able to query a REST-based API and display the results. To summarize, we used Retrofit to define a Java-based wrapper on top of GitHub’s REST API. The data coming from the server was transformed from JSON into Java beans using Jackson. The source code of the Java beans was kept to a minimum with Lombok, a bytecode generation tool. JDeferred was used to invoke the network call outside of the UI thread, leveraging the concept of promises and callbacks. Any errors during processing were sent as events through a lightweight event bus provider, MBassador, ensuring low coupling between components as a result. Finally, all components were glued together using Google Guice, the reference implementation of JSR 330, the dependency injection JSR for Java SE.
Now it’s the time to figure out how these components can be tested. Remember that the full source code is also available under the GPL3 license at javatrove/github-api-01.
Choosing the Base Testing Framework
When it comes to testing Java code, there are a few options that pop to mind. There’s JUnit, a testing framework that almost every Java developer has seen before, used, or heard of at some point. JUnit is the project that revolutionized how we wrote tests in Java back in the day because, well, back when JUnit appeared there were no testing frameworks of note. JUnit became wildly popular among developers who wrote tests—believe it or not, at the time there were people that didn’t see the benefits of writing any kind of automated tests, to the point where Alberto Savoia said in 2004 that if you were just getting into unit testing, you were too late.
The popularity of JUnit meant many extensions popped up all over the place in order to accommodate new requirements. For example, if you need to seed a database with test data, execute operations on that database, and then verify that it has the right contents DbUnit, is your option. Have you ever had the need to verify the contents of an XML file, or perhaps even compare it with an existing sample? XMLUnit comes to the rescue. Or perhaps you would like to test out a shiny web service without having to rely on a real browser; HttpUnit did that before it was cool.
As good as JUnit was, the project stagnated in version 3 for many years. The Java language added new features during that time; features such as generics, varargs, annotations. JUnit 3 lacked support for these features. Then a new contender appeared on scene: TestNG. TestNG was able to take advantage of the new features available in Java, offering more options for developers and enabling them to write effective and productive tests more easily. The creators of TestNG also made sure their project could consume JUnit extensions, in order to avoid a split of the testing ecosystem and to embrace existing codebases. TestNG delivered more bang for your buck, and by all means, it should be the first choice for many.
Alas, we know that’s not how the testing landscape looks today. Development of JUnit was rekindled after some years, perhaps inspired by the amazing work done by TestNG. JUnit 4 was reworked from the ground up, taking advantage of language features added in Java 5 and also taking inspiration from TestNG. Today, JUnit continues to be the dominant testing framework. There’s even an upcoming version, JUnit 5, that harnesses the power of lambda expressions and other new language features available since Java 8.
And we have just covered the traditional Java-based testing frameworks. We could expand our search to behavior-driven development (BDD) frameworks such as JBehave and Cucumber-JVM. We can’t speak about BDD without mentioning its creator, Dan North, who just recently released a new testing extension inspired by the testing support found in the Go language. Have a look at JGoTesting. Personally I’d choose another option named Spock. You can think of Spock as a testing DSL. You write Spock specifications (or tests, if you will) using Groovy code. Spock is also JUnit-compatible, meaning many extensions can be reused. And if we’re opening the door for other Java Virtual Machine (JVM) languages, then you might also be interested in what ScalaTest and Spek bring to the table.
All this being said, we’ll stick with what most people might find familiar and continue with JUnit for now.
Testing the Controller
Let’s begin by testing the controller code. If you remember from the previous part, the controller’s responsibility is to react to user input, issue a network call on a thread different from the UI thread, and then update the UI when results are available. In the case of an error or an unexpected problem, the controller will trigger an event that will be handled by another component in an asynchronous way. The code looks like Listing 1:
public class AppController {
@Inject private AppModel model;
@Inject private Github github;
@Inject private ApplicationEventBus eventBus;
public void loadRepositories() {
model.setState(RUNNING);
github.repositories(model.getOrganization())
.done(model.getRepositories()::addAll)
.fail(throwable -> eventBus.publishAsync(new ThrowableEvent(throwable)))
.always((state, resolved, rejected) -> model.setState(READY));
}
}
Listing 1. AppController.java
The controller requires three collaborators in order to fulfill its responsibilities. Somehow these collaborators need to be initialized when you are testing the controller. You could say that, given the size of the application and its current complexity, it wouldn’t be much of a problem to manually create the required instances and inject them. That’s the easy way out, and we very well know that it’s hard to resist the siren’s call of the easy way. It usually gets us in trouble down the road, though. The simpler way, on the other hand, would give us better results. However, the simpler way is obtained through a complex set of steps of carefully choosing alternatives until we can distill the code into its shortest, more readable version.
What do we do now? We know we’ll use JUnit as the base test framework, so chalk that one down. We know that if we follow proper unit testing practices, we must test the controller in isolation, making sure its collaborators are faked out, or better yet, mocked. Of the several choices we have in the Java space for mocking, one project rises to the top: Mockito. Mockito uses a neat feature found in Java named proxies. It can dynamically create type-safe proxies of a particular type in order to intercept method calls. These method calls can be primed with canned results or fake data, ready to be triggered when the class under test (in our current case, the controller) invokes the methods. The following snippet shows how a mocked GitHub instance can be configured with a canned result, and then verified after the controller invokes it:
// given:
Collection<Repository> repositories = TestHelper.createSampleRepositories();
Promise<Collection<Repository>, Throwable, Void> promise = new DeferredObject<Collection<Repository>, Throwable, Void>().resolve(repositories);
when(github.repositories(ORGANIZATION)).thenReturn(promise);
// when:
model.setOrganization(ORGANIZATION);
controller.loadRepositories();
// then:
assertThat(model.getRepositories(), hasSize(3));
assertThat(model.getRepositories(), equalTo(repositories));
verify(github, only()).repositories(ORGANIZATION);
First, the mock is configured for an specific method call and argument. The mock returns a predefined value when that combination is triggered. The mock is verified once checks performed on the class under test have been cleared. You can specify expected cardinality and order of invocations during verification; thus, a test may fail if there are too many or too few invocations, and also if the invocations were triggered in the wrong order.
This is great, but we’re missing one last thing. The whole application is glued together using Google Guice; as we saw, the controller class requires its collaborators to be injected by Guice. We need to find a way to inject the mock collaborators during testing. Fortunately there’s a handy JUnit extension that combines Guice and Mockito together: it’s called Jukito. There are three things that Jukito will do for you:
- It enables dependency injection on the test case itself. This means you can annotate fields and methods belonging to the test case with
@Inject.
- It automatically mock types when an explicit binding is not provided. That’s right: mocks materialize immediately and are available via the dependency injection container.
- It overrides bindings using a custom module.
Jukito is enabled on a test case by configuring a particular runner. Runners are one of the many extension points offered by JUnit’s API. Listing 2 shows how the full test case looks when we want to test out the happy path.
@RunWith(JukitoRunner.class)
public class AppControllerTest {
private static final String ORGANIZATION = "foo";
@Inject private AppController controller;
@Inject private AppModel model;
@Test
public void happyPath(Github github) {
// given:
Collection<Repository> repositories = TestHelper.createSampleRepositories();
Promise<Collection<Repository>, Throwable, Void> promise = new DeferredObject<Collection<Repository>, Throwable, Void>().resolve(repositories);
when(github.repositories(ORGANIZATION)).thenReturn(promise);
// when:
model.setOrganization(ORGANIZATION);
controller.loadRepositories();
// then:
assertThat(model.getRepositories(), hasSize(3));
assertThat(model.getRepositories(), equalTo(repositories));
verify(github, only()).repositories(ORGANIZATION);
}
}
Listing 2. AppControllerTest.java: happy path
Let’s deconstruct this test step by step. First, we mark it as a Jukito-enabled test case by annotating it with @RunWith(JukitoRunner.class); this tells Jukito to properly initialize the test case and perform the required injection. Notice that we have defined two injection points: one for the class under test itself (the controller)and another for one of its collaborators (the model). We can safely use a real SampleModel instance here, because this particular type defines JavaFX properties and no additional behavior.
Next, have a look at the definition of the happyPath method. It takes a single argument of type Github. We did not define an injection point for this type as we did for the previous fields. This instructs Jukito to automatically create a mock for that type. The mock is given to the test method as an argument, so that we can configure its expectations, but it’s also injected into the controller instance because the controller has an injection point that matches the given type—neat! This means our class under test is fully configured now, and this includes a mock for the ApplicationEventBus type that the controller defines as an injection point, although we don’t make use of that mock in this test. Notice the use of Hamcrest matchers in the assertions.
We should get in a green bar if we run the test as it stands right now. But don’t trust it yet! Did the test pass because it tested the right behavior or because there was no actual test at all and we just fooled ourselves? In situations like this, I always go with the following adage: "never trust a test you haven’t seen fail at least once." To verify that we have the correct test code it would be as simple as changing one of the assertions, for example, the one that checks the size of the returned collection to exactly 3. Change that number to something else and rerun the test. Did it break? Hooray! We’ve got it. Fix the number back to 3 and rerun the test. Did you get a green bar again? Awesome! We’re in business.
We’re ready to move on to the next step which would be testing the failure path. This can be achieved by throwing an exception at the right time. We can rely on JDeferred’s DeferredObject to configure the return value that we need. We’re also going to rely on a real implementation of the ApplicationEventBus, thus forcing us to take multiple threads into account. If you remember how the controller’s code looks, it triggers an event when an error is sent back from the promise. This event is triggered asynchronously from the current point of execution; the controller waits for nobody and the event is handled in a different thread. Our test must be able to wait for a certain condition to be met or fail in the process. Testing this scenario is cumbersome to say the least; it’s likely that many of us reach for the easy way and litter the test code with calls to Thread.sleep(). Deep down, we know there has to be a better way, and that way my friend is Awaitility.
Awaitility has been engineered to enable you to avoid using Thread.sleep() in your tests. Its API is Java 8-friendly, thus enabling you to use lambda expressions to define conditions or use Hamcrest matchers, too. The basic idea is that you set up a condition to be evaluated at some point in time. When the time comes, the condition is either successful (and so the test continues) or it fails (and the test fails). You may specify timeouts, minimum and/or maximum time elapsed, and more. Listing 3 shows how the test for the failure scenario looks when we add Awaitility into the mix.
@RunWith(JukitoRunner.class)
public class AppControllerTest {
private static final String ORGANIZATION = "foo";
@Inject private AppController controller;
@Inject private AppModel model;
@Inject private ApplicationEventBus eventBus;
@Test
public void failurePath(Github github) {
// given:
ApplicationEventHandlerStub eventHandler = new ApplicationEventHandlerStub();
eventBus.subscribe(eventHandler);
Throwable exception = new RuntimeException("boom");
Promise<Collection<Repository>, Throwable, Void> promise = new DeferredObject<Collection<Repository>, Throwable, Void>().reject(exception);
when(github.repositories(ORGANIZATION)).thenReturn(promise);
// when:
model.setOrganization(ORGANIZATION);
controller.loadRepositories();
await().timeout(2, SECONDS).until(eventHandler::getEvent, notNullValue());
// then:
assertThat(model.getRepositories(), hasSize(0));
assertThat(eventHandler.getEvent().getThrowable(), equalTo(exception));
verify(github, only()).repositories(ORGANIZATION);
}
public static class ApplicationEventHandlerStub {
@Getter
private ThrowableEvent event;
@Handler
public void handleThrowable(ThrowableEvent event) {
this.event = event;
}
}
}
Listing 3. AppControllerTest.java: failure path
We define an injection point for ApplicationEventBus because we want to use the real deal. The test method receives a mocked instance of Github, just as we did in the happy-path scenario. Next, we create a witness for the triggered event and register it with the event bus. The witness type is a plain box around the event we’re interested in; thus, we can use Lombok to generate the required boilerplate code. We’ll use the witness later in the test as part of the condition we must write. Following up is setting up the expected exception to be thrown.
Moving your attention to the stimuli block, did you notice the comments resembling labels? This style comes in handy when moving to Spock-based specifications. This is where we put the condition to the test, pardon the pun. The important line is this one:
await().timeout(2, SECONDS).until(eventHandler::getEvent, notNullValue());
This code tells the test case to wait for a maximum of 2 seconds to fail if the condition has not succeeded. It’s worth mentioning that the condition may be evaluated before the timeout runs out. The first argument to the until() call is of type Callable. I’m so glad we can use lambda expressions— even better, method references—with Awaitility’s API, because that leads to very concise and readable code. The second argument is a Matcher; we grab one from the big collection of predefined matchers coming from Hamcrest. All that’s left to do now is to write the proper assertions and verify mocked invocations. These are pretty straightforward given the following:
- There should not be a list of results returned from the controller invocation. We didn’t set any predefined values, but just in case we check, we get no results at all.
- The thrown exception should have been caught by the event handler (this is checked by the condition’s not-null matcher), but it should also be of the right type and have the expected values.
- The GitHub mock is verified exactly as we did before.
We now have a test case that’s able to check both the happy and failure paths of execution. Perhaps you can come up with additional tests that might be missing. Looking back to what we just accomplished, some will claim that what we wrote is not a pure unit test, because we rely on some real collaborators and a dependency injection container. Technically they’d be correct; this test sits somewhere in between unit and integration test. And that’s … a good thing. It’s OK to lean into one type more than the other if it gets the job done. But just to be fair we’ll cover an integration test in the next section.
Integration Testing
Let’s try to test the implementation of the Github API. You might remember its implementation looks like Listing 4:
public class GithubImpl implements Github {
@Inject private GithubAPI api;
@Inject private DeferredManager deferredManager;
@Override
public Promise<Collection<Repository>, Throwable, Void> repositories(final String name) {
return deferredManager.when(() -> {
Response<List<Repository>> response = api.repositories(name).execute();
if (response.isSuccess()) { return response.body(); }
throw new IllegalStateException(response.message());
});
}
}
Listing 4. GithubImpl.java
An instance of this type depends on the Retrofit-powered proxy created around the GithubAPI type shown in Listing 5:
public interface GithubAPI {
String GITHUB_API_URL_KEY = "GITHUB_API_URL";
@GET("/orgs/{organization}/repos")
Call<List<Repository>> repositories(@Path("organization") String organization);
}
Listing 5. GithubAPI.java
This means that if we want to test GithubImpl, we either mock out the proxy that Retrofit would create or find a way to point the real proxy to a REST resource under our control. Mocking out the Retrofit proxy sounds like overkill, because we must provide a somewhat working Call<List<Repository>> that can respond to an execute() call that returns a Response<List<Repository>> with the expected results. It’s turtles way down this route. Setting up a REST resource starts to sound much better. The easy way would be to fire up an HTTP server with an static JSON payload; however, can we be sure the server will be running whenever we run the tests? This is when an integrated solution starts to sound like the way to go. There are a couple of options out there that enable you to set up a fake HTTP server, complete with canned responses and the ability to verify requests and responses. We’re going to take WireMock for a spin in our integration test.
Similar to Mockito, WireMock let’s you define expectations before the stimuli is triggered and verify those expectations after the stimuli has been executed. However, expectations in WireMock take the form of requests, where you can define the HTTP verb, URL pattern, header values, and more. Once a match is done, you can also return an expected response.
We’ll use Jukito again to configure our test case, and pair it up with a JUnit @Rule than sets up WireMock. Listing 6 shows both the happy and failure paths, similar to what was done with the controller:
@RunWith(JukitoRunner.class)
@UseModules({GithubImplTest.AppTestModule.class})
public class GithubImplTest {
private static final String ORGANIZATION = "foo";
@Rule
public WireMockRule wireMockRule = new WireMockRule(8080);
@Inject private ObjectMapper objectMapper;
@Inject private Github github;
@Test
public void happyPath() throws Exception {
// given:
Collection<Repository> repositories = createSampleRepositories();
stubFor(get(urlEqualTo("/orgs/" + ORGANIZATION + "/repos"))
.willReturn(aResponse()
.withStatus(200)
.withHeader("Content-Type", "text/json")
.withBody(repositoriesAsJSON(repositories, objectMapper))));
// when:
Promise<Collection<Repository>, Throwable, Void> promise = github.repositories(ORGANIZATION);
await().timeout(2, SECONDS).until(promise::state, equalTo(Promise.State.RESOLVED));
// then:
promise.done(result -> assertThat(result, equalTo(repositories)));
verify(getRequestedFor(urlEqualTo("/orgs/" + ORGANIZATION + "/repos")));
}
@Test
public void failurePath() {
// given:
stubFor(get(urlEqualTo("/orgs/" + ORGANIZATION + "/repos"))
.willReturn(aResponse()
.withStatus(500)
.withStatusMessage("Internal Error")));
// when:
Promise<Collection<Repository>, Throwable, Void> promise = github.repositories(ORGANIZATION);
await().timeout(2, SECONDS).until(promise::state, equalTo(Promise.State.REJECTED));
// then:
promise.fail(throwable -> assertThat(throwable.getMessage(), equalTo("Internal Error")));
verify(getRequestedFor(urlEqualTo("/orgs/" + ORGANIZATION + "/repos")));
}
public static class AppTestModule extends AppModule {
@Override
protected void bindGithubApiUrl() {
bindConstant()
.annotatedWith(named(GithubAPI.GITHUB_API_URL_KEY))
.to("http://localhost:8080");
}
}
}
Listing 6. GithubImplTest.java
All right—given that the test case is Jukito-based, we start again by annotating it with @RunWith(JukitoRunner.class). Next, we specify that we want to tweak how bindings are defined for this test case. We want to use all production bindings except the one that defines the URL that functions as the entry point for the REST calls. We’ll define this module as an inner class of the test case itself, a fairly common idiom found in this type of tests. Thus, we annotate the test case with @UseModules({GithubImplTest.AppTestModule.class}). The implementation of GithubImplTest.AppTestModule extends the module class that delivers production-ready bindings and overrides a single binding, pointing the value to the URL WireMock will set up during the test.
Next, we see the use of WireMock as JUnit @Rule. This rule sets up the fake server before every test method, and it also tears the server down after each test method has been executed. Inside of each of the test methods, we set up the right expectations for the particular case. In the happy path, we expect a URL of the form "/orgs/${organization}/repos"; the expectation replies with a JSON payload when this URL matches an incoming GET call. For the failure path, the expectation matches against the same URL but returns a 500 error code and an error status message. Next, we invoke the real Retrofit proxy, trusting that WireMock will react at the appropriate time. We must rely on Awaitility again because we’re dealing with operations that run in multiple threads—remember that we’re making use of DeferredManager in order to create a Promise object; the body of the promise is executed on a background thread.
Once the stimuli block has been executed, it's time for verifying the state of the promise and the state of the expected WireMock invocations. Like the other libraries we’ve seen so far (Mockito, Awaitility), WireMock provides a set of static methods that can be used to compose a readable set of expectations and verifications. It almost reads like plain text while still being a type-safe Java-based DSL. Feel free to tweak the values in order to make the tests fail at least once—remember not to trust the first green bar you get!
That’s all for now, because we’ve covered a lot of ground. Speaking of coverage, the source code for this article contains code coverage configuration via JaCoCo. You can obtain the report by running the following Gradle tasks:
gradle test jacocoTestReport
We’ll cover builds tools in a later installment of this series.
Conclusion
In summary, testing our code is always a good thing to do, but sometimes we can get in trouble with a particular testing scenario. Mockito facilitates setting up fake collaborators, expectations, and mocks. Jukito ups the ante by adding support for automatic mocks and Guice support—an excellent choice when the production code relies on Guice as a dependency injection container. Testing concurrency and multithreading is always a pain, but Awaitility can reduce a big chunk of that pain. Finally, WireMock can be used to set up a fake HTTP server that can handle expectations in a similar way as Mockito does.
To be covered in the next installment: handling paginated results and more tests.
See Also
About the Author
Andres is a Java/Groovy developer and a Java Champion with more than 16 years of experience in software design and development. He has been involved in web and desktop application development since the early days of Java. Andres is a true believer in open source and has participated on popular projects such as Groovy, Griffon, and DbUnit, as well as starting his own projects (Json-lib, EZMorph, GraphicsBuilder, JideBuilder). He is a founding member of the Griffon framework and Hackergarten community event. Andres maintains a blog and you can reach him at @aalmiray.
Join the Conversation
Join the Java community conversation on Facebook, Twitter, and the Oracle Java Blog!