Health checks in ASP.NET Core

If we work with microservices, there will come a point where we need to constantly know if our application is running and hasn’t encountered any failure that might bring it down.

We could perform this check manually, but we would end up spending the whole day refreshing the page.

What we’re going to do instead is use health checks to automate this process.

1 - What is a health check?

A health check is what we use to verify whether an application or a part of our infrastructure is running as it should.

This validation is not limited to simply checking if it works or not. We can also include checks for response time, memory usage, or even dependency checks.

1.1 - Types of health checks

You can include as many as you want, and adapt them to your needs. The most common types are:

• Basic check, known as the “basic probe”: an endpoint that will simply return if the application is responding or not.
• Basic check with dependencies: Similar to the previous one, but it will check that all dependencies are functioning as expected. (This needs to be built manually). Dependencies include other microservices as well as infrastructure components such as the database, service bus, etc.
• System checks: In my personal experience, this kind of check is especially useful when deploying cloud applications, whether serverless or in containers, to ensure we aren’t allocating more resources than necessary. Commonly checked elements include CPU usage, memory, disk usage, and so on.

1.2 - Health check response levels

Now that we have the types, let’s go over the response levels.

Generally, there are three response levels when answering:

• Healthy: everything is working normally
• Degraded: The application works but is very slow, for example, an API call that usually takes 120ms now takes 5000ms.
• Unhealthy: The application is down or not functioning as expected.

2 - How and when to use health checks

In my opinion, we should have health checks for every element in our system. For both the applications and the infrastructure.

This means that for every application we create from now on, we should include an endpoint that exclusively checks that everything is functioning.

The same principle applies to infrastructure elements. Every service we use should be checkable, and keep in mind that different services may require different methods of validation, though most of them will use commands or HTTP requests.

And we should always include them—a system failure should never go unnoticed because it can have very negative consequences for the customer.

3 - Implementing Health checks in ASP.NET Core

To implement the code for this application, we’re going to continue with the code from the distributed systems course Distribt, but the code is the same for any other system.

The first thing we’ll do is add a basic health check to ALL our services.

So, navigate to where we have our setup abstraction and include in services .AddhealthChecks().

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);builder.Services.AddHealthChecks();

Subsequently, when we have built our WebApplication, we need to call MapHealthChecks passing the path where the health check will be located.

WebApplication app = builder.Build();app.MapHealthChecks("/health");...

Once configured, if we run the application and call the /health endpoint, we’ll get the following result:

Additionally, its status code will be 200, which indicates it worked correctly.

• NOTE: If you’re using the Distribt library, this information is inside Distribt.Shared.Setup.API in the DefaultDistribtWebApplication class, which means every application will have the health check by default.

3.1 - Checking dependencies in a health check

But what if we want to check more things, not just that the app is running, but also validate dependencies?

Most things we want to check can be done using their own nuget library. If you search nuget for AspNetCore.HealthChecks. you’ll see 70 results—these are libraries already built for such health checks.

In our specific case, we use two types of databases as we saw in the post on CQRS, so we’ll import AspNetCore.HealthChecks.MongoDb and AspNetCore.HealthChecks.MySql.

You can do this either in the affected application, or, as in my case, in the abstraction layer.

If you’re not using Distribt, just add the extension method for the system you’re going to configure right after .AddHealthChecks();:

builder.Services.AddHealthChecks.AddMongoDb(configuration.GetSection("Database:MongoDb"));

And you’ll need to repeat this process for every dependency.

On the other hand, if you’re using the Distribt library, this code will go abstracted in every service you use, and will be added automatically when configuring that service. For MongoDb, for example, we create a method to calculate the information:

public static IServiceCollection AddMongoHealthCheck(this IServiceCollection serviceCollection){    ServiceProvider serviceProvider = serviceCollection.BuildServiceProvider();    string mongoConnectionString = serviceProvider.GetRequiredService<IMongoDbConnectionProvider>().GetMongoConnectionString();    serviceCollection.AddHealthChecks().AddMongoDb(mongoConnectionString);    return serviceCollection;}

And then, in the method called by our applications

public static IServiceCollection AddDistribtMongoDbConnectionProvider(this IServiceCollection serviceCollection,    IConfiguration configuration){    return serviceCollection        .AddMongoDbConnectionProvider()        .AddMongoDbDatabaseConfiguration(configuration)        .AddMongoHealthCheck();}

This way, our applications don’t need to configure the health check themselves—adding the parent functionality is enough.

We must repeat this for every software we’re using.

Now if we go back to the /health endpoint, we’ll see the same “healthy” result, but if we stop the database in Docker, it will return “Unhealthy”

Which will also have a status code of 503, meaning there was an error.

3.2 - Creating a custom health check in .NET

Besides the built-in health checks provided by libraries, we can create our own.

As I mentioned before, it’s common to have health checks for response time or memory usage, but those are more often created at the infrastructure level, since many services already provide them.

For example, if everything is on the cloud, all providers offer ways to get this information, in addition to sending alerts, etc.

We’re going to create a health check that verifies another microservice we depend on is functioning properly.

In the Distribt Project, we have a direct relationship (via RabbitMq) between the Orders microservice and the Products microservice; thus, to always keep information accurate in an order (for instance, the name), the products microservice has to be running properly.

If it isn’t, the Orders microservice will keep working, since we have eventual consistency and duplicate the information that’s essential for us.

To create a custom health check, we need to create a class that inherits from IHealthCheck and only implement the following method:

public class ProductsHealthCheck : IHealthCheck{    public Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context,        CancellationToken cancellationToken = new CancellationToken())    {        throw new NotImplementedException();    }}

In our case, we’ll make an HTTP call to the products microservice, so our code will be:

public class ProductsHealthCheck : IHealthCheck{    private readonly IHttpClientFactory _httpClientFactory;    private readonly IServiceDiscovery _discovery;    public ProductsHealthCheck(IHttpClientFactory httpClientFactory, IServiceDiscovery discovery)    {        _httpClientFactory = httpClientFactory;        _discovery = discovery;    }    public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context,        CancellationToken cancellationToken = new CancellationToken())    {        //TODO: abstract out all the HTTP calls to other distribt microservices #26        HttpClient client = _httpClientFactory.CreateClient();        string productsReadApi =            await _discovery.GetFullAddress(DiscoveryServices.Microservices.ProductsApi.ApiRead, cancellationToken);        client.BaseAddress = new Uri(productsReadApi);        HttpResponseMessage responseMessage = await client.GetAsync($"health", cancellationToken);        if (responseMessage.IsSuccessStatusCode)        {            return HealthCheckResult.Healthy("Product service is healthy");        }                return HealthCheckResult.Degraded("Product service is down");    }}

We mark it as Degraded because orders can still be created, but product names won’t be updated if the service is down.

Note: If an exception is thrown during execution of the health check, its state will be set as Unhealthy.

Now we just need to add this healthcheck to our list of health checks with .AddCheck<T>

webappBuilder.Services.AddHealthChecks().AddCheck<ProductsHealthCheck>(nameof(ProductsHealthCheck));

Now, the result of our health check is the combination of all health checks we have configured.

But this only gives us a global result, which is not ideal. What we need to do is import the nuget package AspNetCore.HealthChecks.UI.Client and use the following configuration:

webApp.UseHealthChecks("/health", new HealthCheckOptions(){    Predicate = _ => true,    ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse});

This way, we can see individual results for what is down and what isn’t.

As we can see, we now have a more detailed explanation of what’s failing.

3.3 - Including an Interface for health checks

So far, all we have is an endpoint returning the response.

What if we want a visual interface?

For this, there is a nuget package called AspNetCore.HealthChecks.UI which gives us a small interface.

The change is very simple; in the services, we just indicate

services.AddHealthChecksUI();

And since we need to store this information somewhere to display it, we’ll use the package AspNetCore.HealthChecks.UI.InMemory.Storage, which allows us to store it in memory:

builder.Services.AddHealthChecksUI().AddInMemoryStorage();

And in the app configuration section, we indicate which URL will host the interface and which endpoint to check for getting the results.

webApp.UseHealthChecksUI(config =>{    config.UIPath = "/health-ui";            });

Now, in each of our microservices, we need to indicate which endpoint to call to get this configuration in the appsettings.json file.

{  ...  "HealthChecksUI": {    "HealthChecks": [      {        "Name": "Orders health check",        "Uri": "/health"      }    ]  }  ...}

And if we run the application, we can see the result.

• Note: In the Distribt project, this information is configured by default to work with Distribt.Shared.Setup without needing to modify appsettings.

This is only for the configured microservice; we could configure a microservice to add observability for all microservices, since the configuration file contains an array, but this global view of what’s up and down will be covered in another video about observability.

Conclusion

In this post we’ve reviewed what a health check is.

What are the states of a health check

When to set up health checks

How to build and configure health checks with .NET