The Test Pyramid in Practice 4/5
In the last article we described component tests: tests which are half unit and half integration test, that allow us to validate both integration within our application (via dependency injection) and also with peripheral components. All of which while remaining sufficiently isolated, to limit friction during execution. Because this isolation works perfectly, our API client tests suffer from a major flaw: if and when the supplier changes the service signature we learn about it much too late. This is what contract tests attempt to avoid, and which we will address in this article.
This article originally appeared on our French Language Blog on 28/06/2018.
body .gist .highlight {<br /> background: #202020;<br />}<br />body .gist tr:nth-child(2n+1) {<br /> background: #202020;<br />}<br />body .gist tr:nth-child(2n) {<br /> background: #202020;<br />}<br />body .gist .gist-meta {<br /> display:none;<br />}<br />body .gist .blob-num,<br />body .gist .blob-code-inner,<br />body .gist .pl-s2,<br />body .gist .pl-stj {<br /> color: #f8f8f2;<br />}<br />body .gist .pl-c1 {<br /> color: #ae81ff;<br />}<br />body .gist .pl-enti {<br /> color: #a6e22e;<br /> font-weight: 700;<br />}<br />body .gist .pl-st {<br /> color: #66d9ef;<br />}<br />body .gist .pl-mdr {<br /> color: #66d9ef;<br /> font-weight: 400;<br />}<br />body .gist .pl-ms1 {<br /> background: #fd971f;<br />}<br />body .gist .pl-c,<br />body .gist .pl-c span,<br />body .gist .pl-pdc {<br /> color: #75715e;<br /> font-style: italic;<br />}<br />body .gist .pl-cce,<br />body .gist .pl-cn,<br />body .gist .pl-coc,<br />body .gist .pl-enc,<br />body .gist .pl-ens,<br />body .gist .pl-kos,<br />body .gist .pl-kou,<br />body .gist .pl-mh .pl-pdh,<br />body .gist .pl-mp,<br />body .gist .pl-mp1 .pl-sf,<br />body .gist .pl-mq,<br />body .gist .pl-pde,<br />body .gist .pl-pse,<br />body .gist .pl-pse .pl-s2,<br />body .gist .pl-mp .pl-s3,<br />body .gist .pl-smi,<br />body .gist .pl-stp,<br />body .gist .pl-sv,<br />body .gist .pl-v,<br />body .gist .pl-vi,<br />body .gist .pl-vpf,<br />body .gist .pl-mri,<br />body .gist .pl-va,<br />body .gist .pl-vpu {<br /> color: #66d9ef;<br />}<br />body .gist .pl-cos,<br />body .gist .pl-ml,<br />body .gist .pl-pds,<br />body .gist .pl-s,<br />body .gist .pl-s1,<br />body .gist .pl-sol {<br /> color: #e6db74;<br />}<br />body .gist .pl-e,<br />body .gist .pl-ef,<br />body .gist .pl-en,<br />body .gist .pl-enf,<br />body .gist .pl-enm,<br />body .gist .pl-entc,<br />body .gist .pl-entm,<br />body .gist .pl-eoac,<br />body .gist .pl-eoac .pl-pde,<br />body .gist .pl-eoi,<br />body .gist .pl-mai .pl-sf,<br />body .gist .pl-mm,<br />body .gist .pl-pdv,<br />body .gist .pl-som,<br />body .gist .pl-sr,<br />body .gist .pl-vo {<br /> color: #a6e22e;<br />}<br />body .gist .pl-ent,<br />body .gist .pl-eoa,<br />body .gist .pl-eoai,<br />body .gist .pl-eoai .pl-pde,<br />body .gist .pl-k,<br />body .gist .pl-ko,<br />body .gist .pl-kolp,<br />body .gist .pl-mc,<br />body .gist .pl-mr,<br />body .gist .pl-ms,<br />body .gist .pl-s3,<br />body .gist .pl-smc,<br />body .gist .pl-smp,<br />body .gist .pl-sok,<br />body .gist .pl-sra,<br />body .gist .pl-src,<br />body .gist .pl-sre {<br /> color: #f92672;<br />}<br />body .gist .pl-mb,<br />body .gist .pl-pdb {<br /> color: #e6db74;<br /> font-weight: 700;<br />}<br />body .gist .pl-mi,<br />body .gist .pl-pdi {<br /> color: #f92672;<br /> font-style: italic;<br />}<br />body .gist .pl-pdc1,<br />body .gist .pl-scp {<br /> color: #ae81ff;<br />}<br />body .gist .pl-sc,<br />body .gist .pl-sf,<br />body .gist .pl-mo,<br />body .gist .pl-entl {<br /> color: #fd971f;<br />}<br />body .gist .pl-mi1,<br />body .gist .pl-mdht {<br /> color: #a6e22e;<br /> background: rgba(0, 64, 0, .5);<br />}<br />body .gist .pl-md,<br />body .gist .pl-mdhf {<br /> color: #f92672;<br /> background: rgba(64, 0, 0, .5);<br />}<br />body .gist .pl-mdh,<br />body .gist .pl-mdi {<br /> color: #a6e22e;<br /> font-weight: 400;<br />}<br />body .gist .pl-ib,<br />body .gist .pl-id,<br />body .gist .pl-ii,<br />body .gist .pl-iu {<br /> background: #a6e22e;<br /> color: #272822;<br />}<br />body .gist .gist-file,<br />body .gist .gist-data {<br /> border: 0px;<br /> border-bottom: 0px;<br />}<br />
Contract tests
What to test?
In a microservice architecture, it’s likely that the services will communicate amongst themselves, and we’ll run into disaster if we don’t adopt certain practices, in particular autonomy:
- A shared nothing architecture in the technical implementation
- Team autonomy in the organisation
- Deployment autonomy as part of the process.
It’s on this last point that contract tests help us. They’re an excellent complement to component tests when testing microservices. Indeed, contract tests make it possible to verify that the contract between producer and consumer is always the one defined between the two parties:
- On the producer side, we verify that we’re providing the service that consumers expect to receive (the implementation of the contract). If this isn’t the case it’s because our interface has changed, intentional or not. If it is unintentional, the test will fail and we avoid deploying a service that might cause our consumers to crash. If it’s intentional, then it’s necessary to communicate the change to our consumers and increase the service’s version.
- On the consumer side, we check that the contract hasn’t changed. If it has changed, we check with the producer to identify the change and adapt our code and the associated Component tests.
We frequently hear about Consumer-Driven Contracts tests, a pattern that aims to construct contracts based on consumers' needs rather than imposing one defined by the producer. The intention is to limit friction as the service evolves (i.e. upstream and downstream compatibility management, versions, etc. generally imposed by the producer), and to better understand how the service is used in reality (i.e. which endpoints and which values are useful).
In our example, this is how contract tests will look:
Implementation
To implement our contract tests, we stay within the Spring ecosystem by using Spring Cloud Contract, there are other frameworks out there, Pact being one of the more popular.
Let's zoom in on these tests:
Producer side
Maven Configuration
On the producer side we’ll need two things:
- A library to check that our controller is implementing the contract:
- A plugin which will generate tests from the contract and generate stubs to be used on the consumer side:
The baseClassForTests is a parent class that will be used by all our generated tests, we’ll come back to it later.
Contract definition
Our first step is to create a contract describing the request and response (in yaml format or with a DSL such as groovy). The following example is simplistic, and it’s possible to do much more advanced things (including with regular expressions and other features, we encourage you to read the docs). We’ll store this contract on the producer side in the directory src/test/resources/contracts, with the response body also stored in a file:
At this point, if you run Maven the mechanism will come into place, at least in part. The test class will be generated (in target/generated-test-sources/contracts):
But the test will fail.
Contract tests’ base class
To make the test pass, we need the basic class we referred to above to initialize a context:
There’s a lot going on at this level:
- The primary objective is to provide what the Controller needs in order to answer the test, so we provide a MockMvc environment with RestAssuredMockMvc.
- We could settle on the last line, but in this case the Controller would use a real implementation of the Service which in turn uses a real client which would then call the real transport API. To avoid this and provide more isolation, we’ll do what we did earlier, and provide a MockBean that simulates the Service’s response by deserializing a json file, which is the same file used in the contract.
- It’s generally be necessary to create as many base classes as you have Controllers to simulate their behaviour. The maven configuration is somewhat different in this case.
At this point the producer-side tests should pass, and if you’ve run mvn install or deploy, the jar containing the stubs should be published in the Maven repo, be it local or remote:
Installing connection-lookup/target/connection-lookup-0.0.1-SNAPSHOT-stubs.jar to ~/.m2/repository/ch/octo/blog/connection-lookup/0.0.1-SNAPSHOT/connection-lookup-0.0.1-SNAPSHOT-stubs.jar
Now let’s look at the consumer side: our journey-booking module.
Consumer side
Maven Configuration
Here, we need to depend on the stub generated on the producer side:
Moreover, we have to replace the wiremock dependency with the Spring Cloud dependency or we'll have an exception. If you remember our client test, we used wiremock to simulate a server. Spring Cloud does the same but is based on the self-generated stub, which allows it to be aligned with the producer contract.
The test
The code is as follows (gitlab link) :
The test is similar to our component tests (ConnectionLookupClientCompTest), except that we no longer use wiremock directly. The @AutoConfigureStubRunner annotation configures the stub by fetching the Maven dependency specified by the ids attribute (in groupId:artifactId:version:classifier format) and exposing it on port 8090.
By using “+” as the version number, we’ll always take the most recent version which makes it possible to verify that our test always passes despite the evolving producer. On the day the test fails, we’ll know the contract has changed, logically before the producer has put the new version into production.
Contract test or component test?
As we’ve just seen, the contract test is nothing other than a component test, but it also has the advantage of validating that the producer and consumer are aligned. Our recommendation is to always use contract testing, at least in a controlled environment (typically microservices within your company), but this doesn’t make sense when we’re using an open API (in our case the transport API), in this case we prefer component tests.
In both cases, I consider these tests sufficiently important, and fast enough, to run to be integrated into a continuous integration pipeline, being run again and again with the objective of getting rapid feedback.
Contract tests are often associated with the Consumer-Driven Contracts pattern, and are an excellent way to verify that service consumers and providers (whether via REST or via messaging) are aligned on a common, shared contract. They also have the advantage of running fairly quickly (because of the insulation provided by wiremock) and can therefore be integrated into the continuous integration pipeline. In the next article, we’ll discuss much less straightforward tests to run: integration and end-to-end tests.