Dec 30, 2015

WireSpock - Testing REST service client components with Spock and WireMock

In a previous post I have written about using the Spock framework for the exploratory testing of open source software. In this post I want to showcase a neat technology integration between Spock and the WireMock framework for testing your REST service client components. This is especially useful when testing micro service based architectures, since you want to test the individual service integrations without firing up all the collaborators.

Introducing WireMock

As stated on it's webpage, WireMock is "a web service test double for all occasions". It supports stubbing and mocking of HTTP calls, as well as request verification, record and playback of stubs, fault injection and much more. It actually fires up a small embedded HTTP server, so your code and test interacts with it on the protocol level.

The most convenient way to use WireMock in your test cases is via a JUnit 4.x rule that handles the lifecycle of starting and stopping the mock server before and after each test. There also is a class rule available in case it is sufficient to use the same WireMock instance for the lifetime of the whole test case. The official documentation for the rule can be found here.

The good thing is that you can use the WireMock rule in your Spock specification just like in an ordinary JUnit based test. No magic here. Let's have a look at the following example.
class BookServiceClientSpec extends Specification {

    @Rule
    WireMockRule wireMockRule = new WireMockRule(18080)

    @Shared
    def client = new BookServiceClient("http://localhost:18080")

    def "Find all books using a WireMock stub server"() {
        given: "a stubbed GET request for all books"
        // TODO

        when: "we invoke the REST client to find all books"
        def books = client.findAll()

        then: "we expect two books to be found"
        books.size() == 2

        and: "the mock to be invoked exactly once"
        // TODO
    }
}
First, the JUnit WireMock rule is created and initialized to listen on port 18080. Next the REST client component under test is created and configured to access the local wire mock server. The test method itself does not do much yet. For it to work we need to stub the response for the findAll() query and we want to check that the mock has been invoked once. Before we continue, let's have a look at the test dependencies required to compile and run the example.
dependencies {
    testCompile 'junit:junit:4.12'
    testCompile 'org.spockframework:spock-core:1.0-groovy-2.4'

    testCompile 'com.github.tomakehurst:wiremock:1.57'
    testCompile 'com.github.tomjankes:wiremock-groovy:0.2.0'
}

Making WireMock Groovy

The last dependency is a small Groovy binding library for WireMock that plays together nicely with Spock. It allows for a more concise stubbing and verification syntax instead of using WireMock's default static imports API. Have a look at the following example to get the idea.
def wireMock = new WireMockGroovy(18080)

def "Find all books using a WireMock stub server"() {
    given: "a stubbed GET request for all books"
    wireMock.stub {
        request {
            method "GET"
            url "/book"
        }
        response {
            status 200
            body """[
                      {"title": "Book 1", "isbn": "4711"},
                      {"title": "Book 2", "isbn": "4712"}
                    ]
                 """
            headers { "Content-Type" "application/json" }
        }
    }

    when: "we invoke the REST client to find all books"
    def books = client.findAll()

    then: "we expect two books to be found"
    books.size() == 2

    and: "the mock to be invoked exactly once"
    1 == wireMock.count {
        method "GET"
        url "/book"
    }
}
First, we create the WireMock Groovy binding to create stubbed requests and responses. The stub closure takes the definitions of the REST request and response using the WireMock JSON API. As you can see we can even specify the response body as inline JSON multiline GString. Finally, we check that the invocation count for the expected request is correct.

Clearly, specifying the responses inline is not very maintenance friendly especially for large response structures. So a better alternative is to externalize the response body in a separate file. The file needs to be located in a directory named __files within src/test/resources.
The bodyFileName value is relative to the __files directory and contain any content. You could even return binary files like JPEGs using this mechanism.
response {
    status 200
    bodyFileName "books.json"
    headers { "Content-Type" "application/json" }
}
A further way of specifying the response body is by using plain Java or Groovy objects that get serialized to JSON automatically.
response {
    status 200
    jsonBody new Book(title: "WireSpock Showcase", isbn: "4713")
    headers { "Content-Type" "application/json" }
}
The stubbing capabilities of WireMock are quite powerful. You can perform different matchings on the URL, request headers, query parameters or the request body to determine the correct response. Have a look at the WireMock stubbing documentation for a complete description of all features.

So there is only one thing left to say: Test long and prosper with Spock!

References

Dec 17, 2015

Open Source Project Chronix: An efficient and fast time series database based on Apache Solr.


www.chronix.io
We are pleased to announce the open source project Chronix. Chronix is a fast and efficient time series storage. It is based on Apache Solr, a distributed NoSQL database with impressive search capabilities. Chronix uses the features of Solr and enriches it with specialized concepts for storing time series data. Thereby Chronix allows you to store about 15 GB of raw time series data (csv files) in about 238 MB. An average query for a bunch of time series data needs 21 ms using a single Solr server and one core. On Benchmarks Chronix outperforms related time series databases like OpenTSDB, InfluxDB or Graphite in both storage demand and query times.

Why is Chronix Open Source?
We use Chronix in several applications like the Software EKG or as central time series storage in a research project called "Design for Diagnosability". We believe that the community can benefit from using Chronix in other projects and hope that their experiences flow back into Chronix. Thus download and use Chronix, fork it, improve it, and raise a pull request. :-)

How can I start?
The homepage of Chronix contains a 5 minute quick start guide. The guide uses an example JavaFX application for time series exploration. You can easily perform range queries, do some analyses and examine the results in real time.
Chronix Quick Start Guide - Time Series Exploration

The listing below shows an example integration using Chronix-API, the Chronix-Kassiopeia time series package, and the Chronix-Solr storage. All libraries are available on bintray. A build script for use in all Gradle versions is:
repositories {
mavenCentral()
    maven {
        url "http://dl.bintray.com/chronix/maven"
    }
}
dependencies {
   compile 'de.qaware.chronix:chronix-api:0.1'
   compile 'de.qaware.chronix:chronix-server-client:0.1'
   compile 'de.qaware.chronix:chronix-kassiopeia-simple:0.1'
   compile 'de.qaware.chronix:chronix-kassiopeia-simple-converter:0.1'
}
Full Source build.gradle

The following snipped first constructs a Chronix client with a connection to Apache Solr, and then streams the maximum from all time series whose metric matches *Load*.
//Connection to Solr
SolrClient solr = new HttpSolrClient("http://host:8983/solr/chronix/");
//Define a group by function for the time series records
Function<MetricTimeSeries, String> groupBy = 
     ts -> ts.getMetric() + "-" + ts.attribute("host");

//Define a reduce function for the grouped time series records
BinaryOperator<MetricTimeSeries> reduce = (ts1, ts2) -> {
      MetricTimeSeries.Builder reduced = new MetricTimeSeries
         .Builder(ts1.getMetric())
         .data(concat(ts1.getTimestamps(), ts2.getTimestamps()),
               concat(ts1.getValues(), ts2.getValues()))
         .attributes(ts1.attributes());
     return reduced.build();
}

//Instantiate a Chronix Client
ChronixClient<MetricTimeSeries,SolrClient,SolrQuery> chronix = 
  new ChronixClient(new KassiopeiaSimpleConverter(),
            new ChronixSolrStorage<>(200,groupBy,reduce));

//We want the maximum of all time series that metric matches *Load*.
SolrQuery query = new SolrQuery("metric:*Load*");
query.addFilterQuery("ag=max");

//The result is a Java Stream. We simply collect the result into a list.
List<MetricTimeSeries> maxTS = chronix.stream(solr, query)
.collect(Collectors.toList());

//Just print it out.
LOGGER.info("Result for query {} is: {}", query, prettyPrint(maxTS));

But I want to use my own fancy time series implementation! No worries!
In the example above we use the default Chronix time series class but you can use Chronix to store your own time series. You only have to implement two simple methods of the TimeSeriesConverter interface shown below:
//Binary time series (data as blob) into your custom time series
T from(BinaryTimeSeries binaryTimeSeries, long queryStart, long queryEnd);

//Your custom time series into a binary time series
BinaryTimeSeries to(T document);

Afterwards you can use Chronix to store and stream you custom time series. For more details check the Chronix GitHub repository and website.

Can I contribute to Chronix?
You are highly welcome to contribute your improvements to the Chronix project. All you have to do is to fork the public GitHub repository, improve the code and issue a pull request.

What do I have do now?
Fork Chronix on GitHub and follow us on Twitter. :-)