Efficient integration testing in Go with Go-VCR
Integration Testing, an overview
note
This post is a translation of the (in our opinion) much nicer presentation-format version of this content, which can be found at /thinking/decks/go-vcr
An integration involves connecting system boundaries through inter-system communication. This can be between internal systems, external systems, or a combination of both. The key point is that any code you write to connect external systems is considered part of your internal system.
OK, but is testing integrations worth it?
Yes, you should test integrations if you expect specific responses over communication protocols or file formats. Essentially, if your application sends data and expects a response, that interaction should be tested.
Testing integrations allows you to trust that the code you’ve written, which you may assume (or test) to be functionally correct, is actually correct when tested against the reality of an external interface (read: not just testing against what, ’the external [API] documentation says’).
Common Misconceptions About Integration Testing
It’s easy to fall into the trap of thinking that some code is so simple it doesn’t need testing, or that internal services are safe as long as they follow the documentation. These assumptions can lead to issues in production when things don’t work as expected, despite passing unit tests.
But, you might be thinking, don’t our unit tests have us covered?
Unit tests are great for verifying that your code behaves as expected, but they often miss issues that arise in production. For example, mocked data might pass unit tests, but the actual integration could still fail. This is where integration testing shines, ensuring that your code interacts with other systems as expected in a real-world scenario.
Integration Testing Challenges
While integration testing has its benefits, it definitely comes with its own special suite of challenges. Primarily:
- Slowness: Testing the entirety of your integration naturally takes time.
- Flakiness: The further the integrated system is from your control, the more likely you’ll encounter issues like intermittent failures.
To mitigate these issues, you might consider running integration tests selectively, such as during continuous integration (CI) processes.
Enter Go-VCR
Go-VCR helps tackle the challenges of integration testing by recording HTTP interactions and replaying them in future test runs. This makes tests faster and more reliable while maintaining the integrity of the integration.
What is a “VCR”?
A VCR connects a data input (e.g., a Cable-TV coaxial cable) to a display device (e.g., a TV). It can optionally record the input stream onto a medium called a “cassette” during playback and can later replay the recordings stored on the cassette.
How Does Go-VCR Work?
Go-VCR provides an HTTP transport that wraps live or recorded HTTP requests and responses. The recordings are stored in a cassette, which allows for transparent replayability. This makes your tests believe they’re interacting with live systems when they’re actually using the recorded data.
Go-VCR Recording Modes
Go-VCR offers various recording modes to suit your testing needs:
- Record-only: No playback, only recording.
- Replay-only: No recording, only playback.
- Replay-with-new-episodes: Record missing interactions but keep the old ones.
- Record-once: Record once and replay, error if an interaction is missing.
- No replay/record: Pass-through only, no recording or playback.
Putting It All Together
Using Go-VCR effectively in your Go-based application’s test suite is pretty straight-forward:
- Configure your test suite to use Go-VCR’s HTTP transport.
- Optionally, add a caching mechanism for unmarshaled HTTP response results.
- Run the tests, which should now be faster thanks to the recorded cassettes.
The first run will be as slow as usual since the cache is empty, but subsequent runs will benefit from the cached data, making the tests significantly faster.
Thus, in one fell-swoop, we can eliminate the slowness and flakiness associated with integration testing, as we can configure when/where our test suite fetches new/live data.
tip
We can configure when our tests do the slow, but necessary, updating of the assumption that the external interface responds as we expect it to!
A great place & time for this would be a periodic (e.g. nightly) Continuous Integration (CI) job!
Conclusion
Go-VCR is a powerful tool that can make your integration tests more efficient and reliable. By recording HTTP interactions and replaying them in future test runs, you can ensure that your code behaves as expected without the drawbacks of slow and flaky tests. If you’re looking to streamline your testing process, Go-VCR might just be the solution you need.
For more details, check out the slides and real-world usage of Go-VCR:
Feel free to reach out if you’re interested in discussing testing strategies at mo [at] dcek.com.