Real-time bidding with Websockets and Vert.x

The ex­pec­ta­tions of users for in­ter­ac­tiv­ity with web ap­pli­ca­tions have changed over the past few years. Users dur­ing bid­ding in auc­tion no longer want to press the re­fresh but­ton to check if the price has changed or the auc­tion is over. This made bid­ding dif­fi­cult and less fun. In­stead, they ex­pect to see the up­dates in ap­pli­ca­tion in real-​time.

In this ar­ti­cle I want to show how to cre­ate a sim­ple ap­pli­ca­tion that pro­vides real-​time bid­ding. We will use Web­Sock­ets, SockJS and Vert.x.

We will cre­ate a front-​end for fast bid­ding that com­mu­ni­cates with a micro-​service writ­ten in Java and based on Vert.x.

What are Websockets?

Web­Socket is asyn­chro­nous, bidi­rec­tional, full-​duplex pro­to­col that pro­vides a com­mu­ni­ca­tion chan­nel over a sin­gle TCP con­nec­tion. With the Web­Socket API it pro­vides bidi­rec­tional com­mu­ni­ca­tion be­tween the web­site and a re­mote server.

Web­Sock­ets solve many prob­lems which pre­vented the HTTP pro­to­col from being suit­able for use in mod­ern, real-​time ap­pli­ca­tions. Workarounds like polling are no longer needed, which sim­pli­fies ap­pli­ca­tion ar­chi­tec­ture. Web­Sock­ets do not need to open mul­ti­ple HTTP con­nec­tions, they pro­vide a re­duc­tion of un­nec­es­sary net­work traf­fic and re­duce la­tency.

Websocket API vs SockJS

Un­for­tu­nately, Web­Sock­ets are not sup­ported by all web browsers. How­ever, there are li­braries that pro­vide a fall­back when Web­Sock­ets are not avail­able. One such li­brary is SockJS. SockJS starts from try­ing to use the Web­Socket pro­to­col. How­ever, if this is not pos­si­ble, it uses a va­ri­ety of browser-​specific trans­port pro­to­cols. SockJS is a li­brary de­signed to work in all mod­ern browsers and in en­vi­ron­ments that do not sup­port Web­Socket pro­to­col, for in­stance be­hind re­stric­tive cor­po­rate proxy. SockJS pro­vides an API sim­i­lar to the stan­dard Web­Socket API.

Frontend to fast bidding

Auc­tion web page con­tains the bid­ding form and some sim­ple JavaScript which loads cur­rent price from the ser­vice, opens an event bus con­nec­tion to the SockJS server and of­fers bid­ding. HTML source code of sam­ple web page on which we bid might look like this:

<h3>Auction 1</h3>
<div id="error_message"></div>
<form>
    Current price:
    <span id="current_price"></span>
    <div>
        <label for="my_bid_value">Your offer:</label>
        <input id="my_bid_value" type="text">
        <input type="button" onclick="bid();" value="Bid">
    </div>
    <div>
        Feed:
        <textarea id="feed" rows="4" cols="50" readonly></textarea>
    </div>
</form>

We use the vertx-eventbus.js li­brary to cre­ate a con­nec­tion to the event bus. vertx-eventbus.js li­brary is a part of the Vert.x dis­tri­b­u­tion. vertx-eventbus.js in­ter­nally uses SockJS li­brary to send the data to the SockJS server. In the code snip­pet below we cre­ate an in­stance of the event bus. The pa­ra­me­ter to the con­struc­tor is the URI where to con­nect to the event bus. Then we reg­is­ter the han­dler lis­ten­ing on ad­dress auction.<auction_id>. Each client has a pos­si­bil­ity of reg­is­ter­ing at mul­ti­ple ad­dresses e.g. when bid­ding in the auc­tion 1234, they reg­is­ter on the ad­dress auction.1234 etc. When data ar­rives in the han­dler, we change the cur­rent price and the bid­ding feed on the auc­tion’s web page.

function registerHandlerForUpdateCurrentPriceAndFeed() {
    var eventBus = new EventBus('http://localhost:8080/eventbus');
    eventBus.onopen = function () {
        eventBus.registerHandler('auction.' + auction_id, function (error, message) {
            document.getElementById('current_price').innerHTML = JSON.parse(message.body).price;
            document.getElementById('feed').value += 'New offer: ' + JSON.parse(message.body).price + '\n';
        });
    }
};

Any user at­tempt to bid gen­er­ates a PATCH Ajax re­quest to the ser­vice with in­for­ma­tion about the new offer made at auc­tion (see bid() func­tion). On the server side we pub­lish this in­for­ma­tion on the event bus to all clients reg­is­tered to an ad­dress. If you re­ceive an HTTP re­sponse sta­tus code other than 200 (OK), an error mes­sage is dis­played on the web page.

function bid() {
    var newPrice = document.getElementById('my_bid_value').value;

    var xmlhttp = (window.XMLHttpRequest) ? new XMLHttpRequest() : new ActiveXObject("Microsoft.XMLHTTP");
    xmlhttp.onreadystatechange = function () {
        if (xmlhttp.readyState == 4) {
            if (xmlhttp.status != 200) {
                document.getElementById('error_message').innerHTML = 'Sorry, something went wrong.';
            }
        }
    };
    xmlhttp.open("PATCH", "http://localhost:8080/api/auctions/" + auction_id);
    xmlhttp.setRequestHeader("Content-Type", "application/json");
    xmlhttp.send(JSON.stringify({price: newPrice}));
};

Auction Service

SockJS client re­quires the server-​side part. Now we are going to cre­ate a light-​weight REST­ful auc­tion ser­vice. We will send and re­trieve data in JSON for­mat. Let’s start by cre­at­ing a ver­ti­cle. First we need to in­herit from AbstractVerticle and over­ride the start method. Each ver­ti­cle in­stance has a mem­ber vari­able called vertx. This pro­vides ac­cess to the Vert.x core API. For ex­am­ple, to cre­ate an HTTP server you call the createHttpServer method on vertx in­stance. To tell the server to lis­ten on port 8080 for in­com­ing re­quests you use the listen method.

We need a router with routes. A router takes an HTTP re­quest and finds the first match­ing route. The route can have a han­dler as­so­ci­ated with it, which re­ceives the re­quest (e.g. route that matches path /eventbus/* is as­so­ci­ated with eventBusHandler).

We can do some­thing with the re­quest, and then, end it or pass it to the next match­ing han­dler.

If you have a lot of han­dlers it makes sense to split them up into mul­ti­ple routers.

You can do this by mount­ing a router at a mount point in an­other router (see auctionApiRouter that cor­re­sponds to /api mount point in code snip­pet below).

Here’s an ex­am­ple ver­ti­cle:

public class AuctionServiceVerticle extends AbstractVerticle {

    @Override
    public void start() {
        Router router = Router.router(vertx);

        router.route("/eventbus/*").handler(eventBusHandler());
        router.mountSubRouter("/api", auctionApiRouter());
        router.route().failureHandler(errorHandler());
        router.route().handler(staticHandler());

        vertx.createHttpServer().requestHandler(router::accept).listen(8080);
    }

    //…
}

Now we’ll look at things in more de­tail. We’ll dis­cuss Vert.x fea­tures used in ver­ti­cle: error han­dler, SockJS han­dler, body han­dler, shared data, sta­tic han­dler and rout­ing based on method, path etc.

Error handler

As well as set­ting han­dlers to han­dle re­quests you can also set a han­dler for fail­ures in rout­ing. Fail­ure in rout­ing oc­curs if a han­dler throws an ex­cep­tion, or if a han­dler calls fail method. To ren­der error pages we use error han­dler pro­vides by Vert.x:

private ErrorHandler errorHandler() {
    return ErrorHandler.create();
}

SockJS handler

Vert.x pro­vides SockJS han­dler with the event bus bridge which ex­tends the server-​side Vert.x event bus into client side JavaScript.

Con­fig­ur­ing the bridge to tell it which mes­sages should pass through is easy. You can spec­ify which matches you want to allow for in­bound and out­bound traf­fic using the BridgeOptions. If a mes­sage is out­bound, be­fore send­ing it from the server to the client side JavaScript, Vert.x will look through any out­bound per­mit­ted matches. In code snip­pet below we allow any mes­sages from ad­dresses start­ing with “auc­tion.” and end­ing with dig­its (e.g. auction.1, auction.100 etc).

If you want to be no­ti­fied when an event oc­curs on the bridge you can pro­vide a han­dler when call­ing the bridge. For ex­am­ple, SOCKET_CREATED event will occur when a new SockJS socket is cre­ated. The event is an in­stance of Future. When you are fin­ished han­dling the event you can com­plete the fu­ture with “true” to en­able fur­ther pro­cess­ing.

To start the bridge sim­ply call bridge method on the SockJS han­dler:

private SockJSHandler eventBusHandler() {
    BridgeOptions options = new BridgeOptions()
            .addOutboundPermitted(new PermittedOptions().setAddressRegex("auction\\.[0-9]+"));
    return SockJSHandler.create(vertx).bridge(options, event -> {
         if (event.type() == BridgeEventType.SOCKET_CREATED) {
            logger.info("A socket was created");
        }
        event.complete(true);
    });
}

Body handler

The Body­Han­dler al­lows you to re­trieve the re­quest body, limit the body size and to han­dle the file up­load. Body han­dler should be on a match­ing route for any re­quests that re­quire this func­tion­al­ity. We need Body­Han­dler dur­ing the bid­ding process (PATCH method re­quest /auctions/<auction_id> con­tains re­quest body with in­for­ma­tion about a new offer made at auc­tion). Cre­at­ing a new body han­dler is sim­ple:

BodyHandler.create();

If re­quest body is in JSON for­mat, you can get it with getBodyAsJson method.

Shared data

Shared data con­tains func­tion­al­ity that al­lows you to safely share the data be­tween dif­fer­ent ap­pli­ca­tions in the same Vert.x in­stance or across a clus­ter of Vert.x in­stances. Shared data in­cludes local shared maps, dis­trib­uted, cluster-​wide maps, asyn­chro­nous cluster-​wide locks and asyn­chro­nous cluster-​wide coun­ters.

To sim­plify the ap­pli­ca­tion we use the local shared map to save in­for­ma­tion about auc­tions. The local shared map al­lows you to share data be­tween dif­fer­ent ver­ti­cles in the same Vert.x in­stance. Here’s an ex­am­ple of using a shared local map in an auc­tion ser­vice:

public class AuctionRepository {

    //…

    public Optional<Auction> getById(String auctionId) {
        LocalMap<String, String> auctionSharedData = this.sharedData.getLocalMap(auctionId);

        return Optional.of(auctionSharedData)
            .filter(m -> !m.isEmpty())
            .map(this::convertToAuction);
    }

    public void save(Auction auction) {
        LocalMap<String, String> auctionSharedData = this.sharedData.getLocalMap(auction.getId());

        auctionSharedData.put("id", auction.getId());
        auctionSharedData.put("price", auction.getPrice());
    }

    //…
}

If you want to store auc­tion data in a data­base, Vert.x pro­vides a few dif­fer­ent asyn­chro­nous clients for ac­cess­ing var­i­ous data stor­ages (Mon­goDB, Redis or JDBC client).

Auction API

Vert.x lets you route HTTP re­quests to dif­fer­ent han­dlers based on pat­tern match­ing on the re­quest path. It also en­ables you to ex­tract val­ues from the path and use them as pa­ra­me­ters in the re­quest. Cor­re­spond­ing meth­ods exist for each HTTP method. The first match­ing one will re­ceive the re­quest. This func­tion­al­ity is par­tic­u­larly use­ful when de­vel­op­ing REST-​style web ap­pli­ca­tions.

To ex­tract pa­ra­me­ters from the path, you can use the colon char­ac­ter to de­note the name of a pa­ra­me­ter. Reg­u­lar ex­pres­sions can also be used to ex­tract more com­plex matches. Any pa­ra­me­ters ex­tracted by pat­tern match­ing are added to the map of re­quest pa­ra­me­ters.

Consumes de­scribes which MIME types the han­dler can con­sume. By using produces you de­fine which MIME types the route pro­duces. In the code below the routes will match any re­quest with content-type header and accept header that matches application/json.

Let’s look at an ex­am­ple of a sub­router mounted on the main router which was cre­ated in start method in ver­ti­cle:

private Router auctionApiRouter() {
    AuctionRepository repository = new AuctionRepository(vertx.sharedData());
    AuctionValidator validator = new AuctionValidator(repository);
    AuctionHandler handler = new AuctionHandler(repository, validator);

    Router router = Router.router(vertx);
    router.route().handler(BodyHandler.create());

    router.route().consumes("application/json");
    router.route().produces("application/json");

    router.get("/auctions/:id").handler(handler::handleGetAuction);
    router.patch("/auctions/:id").handler(handler::handleChangeAuctionPrice);

    return router;
}

The GET re­quest re­turns auc­tion data, while the PATCH method re­quest al­lows you to bid up in the auc­tion. Let’s focus on the more in­ter­est­ing method, namely handleChangeAuctionPrice. In the sim­plest terms, the method might look like this:

public void handleChangeAuctionPrice(RoutingContext context) {
    String auctionId = context.request().getParam("id");
    Auction auction = new Auction(
        auctionId,
        new BigDecimal(context.getBodyAsJson().getString("price"))
    );

    this.repository.save(auction);
    context.vertx().eventBus().publish("auction." + auctionId, context.getBodyAsString());

    context.response()
        .setStatusCode(200)
        .end();
}

PATCH re­quest to /auctions/1 would re­sult in vari­able auctionId get­ting the value 1. We save a new offer in the auc­tion and then pub­lish this in­for­ma­tion on the event bus to all clients reg­is­tered on the ad­dress on the client side JavaScript. After you have fin­ished with the HTTP re­sponse you must call the end func­tion on it.

Static handler

Vert.x pro­vides the han­dler for serv­ing sta­tic web re­sources. The de­fault di­rec­tory from which sta­tic files are served is webroot, but this can be con­fig­ured. By de­fault the sta­tic han­dler will set cache head­ers to en­able browsers to cache files. Set­ting cache head­ers can be dis­abled with setCachingEnabled method. To serve the auc­tion HTML page, JS files (and other sta­tic files) from auc­tion ser­vice, you can cre­ate a sta­tic han­dler like this:

private StaticHandler staticHandler() {
    return StaticHandler.create()
        .setCachingEnabled(false);
}

Let’s run!

Full ap­pli­ca­tion code is avail­able on github.

Clone the repos­i­tory and run ./gradlew run.

Open one or more browsers and point them to http://localhost:8080. Now you can bid in auc­tion:

Real time bidding in application

Summary

This ar­ti­cle presents the out­line of a sim­ple ap­pli­ca­tion that al­lows real-​time bid­ding. We cre­ated a light­weight, high-​performance and scal­able micro-​service writ­ten in Java and based on Vert.x. We dis­cussed what Vert.x of­fers, among oth­ers, a dis­trib­uted event bus and an el­e­gant API that al­lows you to cre­ate ap­pli­ca­tions in no time.

Next post

Using Hamcrest Matchers with Vert.x Unit

Vert.x Unit is a very elegant library to test asynchronous applications developed with vert.x. However because of this asynchronous aspect, reporting test failures is not natural for JUnit users.

Read more
Previous post

ECMAScript 6/7 on the JVM with TypeScript and Vert.x

I figured it would be a good idea to give you a couple of examples how teaming Vert.x and TypeScript helps you use ECMAScript 6 and 7 features on the JVM today.

Read more
Related posts

Things to keep in mind concerning CSRF attacks

Eclipse Vert.x like most frameworks provides an anti-CSRF defense. However, no framework can prevent all attack vectors that exist in the web. Therefore, developers need to be aware of some dangers and common attack vectors concerning CSRF defenses.

Read more

Some Rest with Vert.x

This post is part of the Introduction to Vert.x series. Let’s go a bit further this time and develop a CRUD-ish application

Read more

Building services and APIs with AMQP 1.0

Microservices and APIs are everywhere. Everyone talks about them, presentation slides are full of them ... some people are actually even building them.

Read more