Vert.x JUnit 5 integration

This module offers integration and support for writing Vert.x tests with JUnit 5.

Use it in your build

groupId: io.vertx
artifactId: vertx-junit5
version: (current Vert.x release or SNAPSHOT)

Why testing asynchronous code is different

Testing asynchronous operations requires more tools than what a test harness like JUnit provides. Let us consider a typical Vert.x creation of a HTTP server, and put it into a JUnit test:

class ATest {
  Vertx vertx = Vertx.vertx();

  @Test
  void start_server() {
    vertx.createHttpServer()
      .requestHandler(req -> req.response().end("Ok"))
      .listen(16969, ar -> {
        // (we can check here if the server started or not)
      });
  }
}

There are issues here since listen does not block as it tries to start a HTTP server asynchronously. We cannot simply assume that the server has properly started upon a listen invocation return. Also:

the callback passed to listen will be executed from a Vert.x event loop thread, which is different from the thread that runs the JUnit test, and
right after calling listen, the test exits and is being considered to be passed, while the HTTP server may not even have finished starting, and
since the listen callback executes on a different thread than the one executing the test, then any exception such as one thrown by a failed assertion cannot be capture by the JUnit runner.

A test context for asynchronous executions

The first contribution of this module is a VertxTestContext object that:

allows waiting for operations in other threads to notify of completion, and
supports assertion failures to be received to mark a test as failed.

Here is a very basic usage:

class BTest {
  Vertx vertx = Vertx.vertx();

  @Test
  void start_http_server() throws Throwable {
    VertxTestContext testContext = new VertxTestContext();

    vertx.createHttpServer()
      .requestHandler(req -> req.response().end())
      .listen(16969)
      .onComplete(testContext.succeedingThenComplete()); (1)

    assertThat(testContext.awaitCompletion(5, TimeUnit.SECONDS)).isTrue(); (2)
    if (testContext.failed()) {  (3)
      throw testContext.causeOfFailure();
    }
  }
}

succeedingThenComplete returns an asynchronous result handler that is expected to succeed and then make the test context pass.
awaitCompletion has the semantics of a java.util.concurrent.CountDownLatch, and returns false if the waiting delay expired before the test passed.
If the context captures a (potentially asynchronous) error, then after completion we must throw the failure exception to make the test fail.

Use any assertion library

This module does not make any assumption on the assertion library that you should be using. You can use plain JUnit assertions, AssertJ, etc.

To make assertions in asynchronous code and make sure that VertxTestContext is notified of potential failures, you need to wrap them with a call to verify, succeeding, or failing:

HttpClient client = vertx.createHttpClient();

client.request(HttpMethod.GET, 8080, "localhost", "/")
  .compose(req -> req.send().compose(HttpClientResponse::body))
  .onComplete(testContext.succeeding(buffer -> testContext.verify(() -> {
    assertThat(buffer.toString()).isEqualTo("Plop");
    testContext.completeNow();
  })));

The useful methods in VertxTestContext are the following:

completeNow and failNow to notify of a success or failure
succeedingThenComplete to provide Handler<AsyncResult<T>> handlers that expect a success and then completes the test context
failingThenComplete to provide Handler<AsyncResult<T>> handlers that expect a failure and then completes the test context
succeeding to provide Handler<AsyncResult<T>> handlers that expect a success and pass the result to another callback, any exception thrown from the callback is considered as a test failure
failing to provide Handler<AsyncResult<T>> handlers that expect a failure and pass the exception to another callback, any exception thrown from the callback is considered as a test failure
verify to perform assertions, any exception thrown from the code block is considered as a test failure.

Warning

Unlike succeedingThenComplete and failingThenComplete, calling succeeding and failing methods can only make a test fail (e.g., succeeding gets a failed asynchronous result). To make a test pass you still need to call completeNow, or use checkpoints as explained below.

Checkpoint when there are multiple success conditions

Many tests can be marked as passed by simply calling completeNow at some point of the execution. That being said there are also many cases where the success of a test depends on different asynchronous parts to be validated.

You can use checkpoints to flag some execution points to be passed. A Checkpoint can require a single flagging, or multiple flags. When all checkpoints have been flagged, then the corresponding VertxTestContext makes the test pass.

Here is an example with checkpoints on the HTTP server being started, 10 HTTP requests having being responded, and 10 HTTP client requests having been made:

Checkpoint serverStarted = testContext.checkpoint();
Checkpoint requestsServed = testContext.checkpoint(10);
Checkpoint responsesReceived = testContext.checkpoint(10);

vertx.createHttpServer()
  .requestHandler(req -> {
    req.response().end("Ok");
    requestsServed.flag();
  })
  .listen(8888)
  .onComplete(testContext.succeeding(httpServer -> {
    serverStarted.flag();

    HttpClient client = vertx.createHttpClient();
    for (int i = 0; i < 10; i++) {
      client.request(HttpMethod.GET, 8888, "localhost", "/")
        .compose(req -> req.send().compose(HttpClientResponse::body))
        .onComplete(testContext.succeeding(buffer -> testContext.verify(() -> {
          assertThat(buffer.toString()).isEqualTo("Ok");
          responsesReceived.flag();
        })));
    }
  }));

Tip	Checkpoints should be created only from the test case main thread, not from Vert.x asynchronous event callbacks.

Integration with JUnit 5

JUnit 5 provides a different model compared to the previous versions.

Test methods

The Vert.x integration is primarily done using the VertxExtension class, and using test parameter injection of Vertx and VertxTestContext instances:

@ExtendWith(VertxExtension.class)
class SomeTest {

  @Test
  void some_test(Vertx vertx, VertxTestContext testContext) {
    // (...)
  }
}

Note	The `Vertx` instance is not clustered and has the default configuration. If you need something else then just don’t use injection on that parameter and prepare a `Vertx` object by yourself.

The test is automatically wrapped around the VertxTestContext instance lifecycle, so you don’t need to insert awaitCompletion calls yourself:

@ExtendWith(VertxExtension.class)
class SomeTest {

  @Test
  void http_server_check_response(Vertx vertx, VertxTestContext testContext) {
    vertx.deployVerticle(new HttpServerVerticle(), testContext.succeeding(id -> {
      HttpClient client = vertx.createHttpClient();
      client.request(HttpMethod.GET, 8080, "localhost", "/")
        .compose(req -> req.send().compose(HttpClientResponse::body))
        .onComplete(testContext.succeeding(buffer -> testContext.verify(() -> {
          assertThat(buffer.toString()).isEqualTo("Plop");
          testContext.completeNow();
        })));
    }));
  }
}

You can use it with standard JUnit annotations such as @RepeatedTest or lifecycle callbacks annotations:

@ExtendWith(VertxExtension.class)
class SomeTest {

  // Deploy the verticle and execute the test methods when the verticle
  // is successfully deployed
  @BeforeEach
  void deploy_verticle(Vertx vertx, VertxTestContext testContext) {
    vertx.deployVerticle(new HttpServerVerticle(), testContext.succeedingThenComplete());
  }

  // Repeat this test 3 times
  @RepeatedTest(3)
  void http_server_check_response(Vertx vertx, VertxTestContext testContext) {
    HttpClient client = vertx.createHttpClient();
    client.request(HttpMethod.GET, 8080, "localhost", "/")
      .compose(req -> req.send().compose(HttpClientResponse::body))
      .onComplete(testContext.succeeding(buffer -> testContext.verify(() -> {
        assertThat(buffer.toString()).isEqualTo("Plop");
        testContext.completeNow();
      })));
  }
}

It is also possible to customize the default VertxTestContext timeout using the @Timeout annotation either on test classes or methods:

@ExtendWith(VertxExtension.class)
class SomeTest {

  @Test
  @Timeout(value = 10, timeUnit = TimeUnit.SECONDS)
  void some_test(Vertx vertx, VertxTestContext context) {
    // (...)
  }
}

Lifecycle methods

JUnit 5 provides several user-defined lifecycle methods annotated with @BeforeAll, @BeforeEach, @AfterEach and @AfterAll.

These methods can request the injection of Vertx instances. By doing so, they are likely to perform asynchronous operations with the Vertx instance, so they can request the injection of a VertxTestContext instance to ensure that the JUnit runner waits for them to complete, and report possible errors.

Here is an example:

@ExtendWith(VertxExtension.class)
class LifecycleExampleTest {

  @BeforeEach
  @DisplayName("Deploy a verticle")
  void prepare(Vertx vertx, VertxTestContext testContext) {
    vertx.deployVerticle(new SomeVerticle(), testContext.succeedingThenComplete());
  }

  @Test
  @DisplayName("A first test")
  void foo(Vertx vertx, VertxTestContext testContext) {
    // (...)
    testContext.completeNow();
  }

  @Test
  @DisplayName("A second test")
  void bar(Vertx vertx, VertxTestContext testContext) {
    // (...)
    testContext.completeNow();
  }

  @AfterEach
  @DisplayName("Check that the verticle is still there")
  void lastChecks(Vertx vertx) {
    assertThat(vertx.deploymentIDs())
      .isNotEmpty()
      .hasSize(1);
  }
}

Scope of `VertxTestContext` objects

Since these objects help waiting for asynchronous operations to complete, a new instance is created for any @Test, @BeforeAll, @BeforeEach, @AfterEach and @AfterAll method.

Scope of `Vertx` objects

The scope of a Vertx object depends on which lifecycle method in the JUnit relative execution order first required a new instance to be created. Generally-speaking, we respect the JUnit extension scoping rules, but here are the specifications.

If a parent test context already had a Vertx instance, it is being reused in children extension test contexts.
Injecting in a @BeforeAll method creates a new instance that is being shared for injection in all subsequent test and lifecycle methods.
Injecting in a @BeforeEach with no parent context or previous @BeforeAll injection creates a new instance shared with the corresponding test and AfterEach method(s).
When no instance exists before running a test method, an instance is created for that test (and only for that test).

Configuring `Vertx` instances

By default, the Vertx objects get created with Vertx.vertx(), using the default settings for Vertx. However, you have the ability to configure VertxOptions to suit your needs. A typical use case would be "extending blocking timeout warning for debugging". To configure the Vertx object you must:

create a json file with the VertxOptions in json format
create an environment variable VERTX_PARAMETER_FILENAME, or a system property vertx.parameter.filename, pointing to that file

Tip	The environment variable value takes precedence over the system property value, if both are present.

Example file content for extended timeouts:

{
 "blockedThreadCheckInterval" : 5,
 "blockedThreadCheckIntervalUnit" : "MINUTES",
 "maxEventLoopExecuteTime" : 360,
 "maxEventLoopExecuteTimeUnit" : "SECONDS"
}

With these conditions met, the Vertx object will be created with the configured options

Closing and removal of `Vertx` objects

Injected Vertx objects are being automatically closed and removed from their corresponding scopes.

For instance if a Vertx object is created for the scope of a test method, it is being closed after the test completes. Similarly, when it is being created by a @BeforeEach method, it is being closed after possible @AfterEach methods have completed.

Support for additional parameter types

The Vert.x JUnit 5 extension is extensible: you can add more types through the VertxExtensionParameterProvider service provider interface.

If you use RxJava, instead of io.vertx.core.Vertx you can inject:

io.vertx.rxjava3.core.Vertx, or
io.vertx.reactivex.core.Vertx, or
io.vertx.rxjava.core.Vertx.

To do so, add the corresponding library to your project:

io.vertx:vertx-junit5-rx-java3, or
io.vertx:vertx-junit5-rx-java2, or
io.vertx:vertx-junit5-rx-java.

On Reactiverse you can find a growing collection of extensions for vertx-junit5 that integrates with Vert.x stack in the reactiverse-junit5-extensions project: https://github.com/reactiverse/reactiverse-junit5-extensions.

Parameter ordering

It may be the case that a parameter type has to be placed before another parameter. For instance the Web Client support in the vertx-junit5-extensions project requires that the Vertx argument is before the WebClient argument. This is because the Vertx instance needs to exist to create the WebClient.

It is expected that parameter providers throw meaningful exceptions to let users know of possible ordering constraints.

In any case it is a good idea to have the Vertx parameter first, and the next parameters in the order of what you’d need to create them manually.

Parameterized tests with `@MethodSource`

You can use parameterized tests with @MethodSource with vertx-junit5. Therefore you need to declare the method source parameters before the vertx test parameters in the method definition.

@ExtendWith(VertxExtension.class)
static class SomeTest {

  static Stream<Arguments> testData() {
    return Stream.of(
      Arguments.of("complex object1", 4),
      Arguments.of("complex object2", 0)
    );
  }

  @ParameterizedTest
  @MethodSource("testData")
  void test2(String obj, int count, Vertx vertx, VertxTestContext testContext) {
    // your test code
    testContext.completeNow();
  }
}

The same holds for the other ArgumentSources. See the section Formal Parameter List in the API doc of ParameterizedTest

Running tests on a Vert.x context

By default the thread invoking the test methods is the JUnit thread. The RunTestOnContext extension can be used to alter this behavior by running test methods on a Vert.x event-loop thread.

Caution

Keep in mind that you must not block the event loop when using this extension.

For this purpose, the extension needs a Vertx instance. By default, it creates one automatically but you can provide options for configuration or a supplier method.

The Vertx instance can be retrieved when the test is running.

@ExtendWith(VertxExtension.class)
class RunTestOnContextExampleTest {

  @RegisterExtension
  RunTestOnContext rtoc = new RunTestOnContext();

  Vertx vertx;

  @BeforeEach
  void prepare(VertxTestContext testContext) {
    vertx = rtoc.vertx();
    // Prepare something on a Vert.x event-loop thread
    // The thread changes with each test instance
    testContext.completeNow();
  }

  @Test
  void foo(VertxTestContext testContext) {
    // Test something on the same Vert.x event-loop thread
    // that called prepare
    testContext.completeNow();
  }

  @AfterEach
  void cleanUp(VertxTestContext testContext) {
    // Clean things up on the same Vert.x event-loop thread
    // that called prepare and foo
    testContext.completeNow();
  }
}

When used as a @RegisterExtension instance field, a new Vertx object and Context are created for each tested method. @BeforeEach and @AfterEach methods are executed on this context.

When used as a @RegisterExtension static field, a single Vertx object and Context are created for all the tested methods. @BeforeAll and @AfterAll methods are executed on this context too.