Integration Testing: Key Tool or Waste of Time?

Integration tests are that part of programming where we often get stuck or leave aside during the normal application development process, but it's a crucial element in the entire software lifecycle. This post comes as a result of a message I received through the /ama section for anonymous questions. 

Remember, if you have any questions you can leave them anonymously at the following link https://www.netmentor.es/ama.

1 - What are integration tests? 

I don't want to go too deep into what integration tests are or how to implement them because I have a post about it. 

What I do want to mention is that in different companies, anything that integrates external services is called an integration test, whether that's the database, the file system, or another internal company service. 

And whether you're testing locally with dependencies in docker, pointing to dev from local, or running the tests within an environment, you're testing the dependencies.  So in many cases, everything that's not a unit test will be considered an integration test, and that's how we'll treat it in this post.

2 - Are integration tests useful or do they cause more problems than benefits?

Here's the question, which I found interesting enough to make a video about:

I know a lot of you will know the answer before I say anything, since most of you who read this blog or watch my YouTube channel have many years of experience. But I think this could still be very useful for many people. 

And the short answer is yes, they are very useful. I usually answer /ama questions with a tweet and that's it, but for this one I want to add an explanation about why they're useful. 

The reason is obvious: they help to check that your application's entire structure works as expected. In every environment.  

This is where the important part comes in, the AMA question points out that testing a MySQL database running in docker is fine. But what if we have something in Azure (cosmosDb), etc.? 

We can mention two parts.

The first is that you can use Azurite locally to emulate Azure on your machine. For AWS there's localstack. Still, many companies give access to the dev environment from local to avoid all the individual project configurations, etc. which over time is actually much easier. 

The second, and most important: the main thing about integration tests is that they work in production environments. That they work on your local machine is fine, of course, since that's where you create the test. But where it really matters is that they work in PRODUCTION.

This is because a well-structured system needs to have only the correct and necessary permissions, never more, never less. The original message references that these systems “depend on nobody having made changes”; that's why you have tests. In fact, it's the best reason; this way if something works with the current implementation, you deploy your new code, and it doesn't work, you know someone has made changes and you should roll back to the previous version. 

The more environments you have, the better. If you have an error in one, it should happen in all of them. (Except for app communication/configuration issues).

A common error example: you need to access a table where you don't have permissions. On your local machine everything works fine because you have access to everything, and in the unit tests you're mocking the database. So, once you deploy, you realize there are tests that fail, and looking at the logs you see you're getting an access error for a certain table.

2.1 - Ways to run integration tests.

In my opinion, there are two ways to run integration tests; both are similar but not exactly the same. One is "safer" than the other, but requires more configuration.

Have integration tests (or end to end) in a separate project. 

This means the tests are completely independent from the code, in code as well as in deployment. 

So, when you make a code change, the test DOESN'T KNOW anything has changed, because it's a totally different project. 

The process might look like this: you make a change, approve the Pull Request, do the merge and in CI/CD a package is created so you can deploy (or it's deployed automatically) in the development environment.

Once it's deployed in this environment, the tests must be run, whether automatically or manually or by a specific person for QA. We have to wait for it to finish and for everything to be ok before propagating the change further through the pipeline, which in this case is a mix of manual and automatic. 

Repeating this process in each environment you have.

When everything goes well, at the end we give the green light and that's it. And if something fails, we have to do a rollback and deploy the previous version. 

Key points of this approach:

• Tests are in a separate project, so if something changes in the app, we must update them independently. If a functionality change introduces a breaking change, the test will break.
• In many companies, this is a manual step, not automatic, and the result must be verified to make sure everything worked.
• They're not run in the application's main pipeline, so you could have an app that's not working deployed to production.
• Since the tests aren't directly linked to the referenced service, they're often forgotten and not updated. 

The second way is to have the integration tests in the main project itself. If you come from C#, I'm not talking about testing an API with testserver since that functionality just spins up an in-memory copy of your app, but rather, whether as an additional project in our repository and a CI/CD step, we have tests that make real http calls to the service. 

Ideally, we could combine deployment with a Blue/Green release where we have both the new and old instances running in parallel, configuring the load balancer so our calls always go to the new version.  At the same time, the pipeline has a step to run the test, and once it's executed and everything works, we replace the existing instances with the new ones, and if something fails, we roll back the change in the load balancer so the old ones keep serving instead of the new ones. 

Key points of this approach

• Tests are in the codebase itself.
• Tests are fully automated within the pipeline.
• Much more CI/CD configuration required (greater resource consumption).
• Errors don't make it to the production environment. If something doesn't work, we'll spot it in the tests, and since the load balancer still points to the old instance, clients won't be affected. 

In the end, which option you use depends on two main factors: resources, both budgetary and personnel, because it's not cheap and needs maintenance.  And how important it is for a service to not function 100% for a few minutes while you run tests and deploy the previous version. 

Most commonly, companies use the first option, but I have also worked at one where we used the second option.

If you like the way this post is written, I have a book to learn about distributed systems using the same formula, which you can buy at the following link:  https://www.netmentor.es/libros/construyendo-sistemas-distribuidos  

2.2 - Using integration tests exclusively

I want to end by giving my opinion about projects or companies that use only integration tests. Because they exist. If the company is small, or a startup, I can understand it, since an integration test will check that everything that needs to work does work. 

Still, it's much easier to write unit tests, because as the name suggests, you test a single piece of functionality, you can simulate different dependency responses, etc. 

In summary, both types of tests have their uses and both are necessary for the proper functioning of our application.