Service Registration with Consul

When working with microservices, we eventually find ourselves managing dozens, hundreds, or in some companies, even thousands of services. Managing these services becomes exponentially more complicated as you continue to add more and more.

One of the main problems is knowing where these services are located, and therefore the URL we need to access in order to communicate with them.

1 - Service registries in microservices

When you have a very large system with many microservices that communicate with each other, managing the entire configuration can be quite complex and costly. Furthermore, with a classic approach, you would store all the information about the services to be consumed in each service's configuration file, and you would have to do this for each service.

When I say configuration, I mainly mean the URLs of a microservice or some architectural service, such as the database host or message system host, for asynchronous communication.

And not only that, but for each environment you have, you must store that information, which becomes very tedious.

2 - The service registry

To manage all this information, we use a service registry (service registry or service discovery in English) that allows us to register all this information. At the end of the day, such a service acts as a database that contains the information for all services, such as URL, port, protocol, etc.

Therefore, when we want to call Microservice A from Microservice B, we must first query the registry service to find out where it is located.

Having a service that acts as a service registry allows us to delegate all the complexity of managing URLs/hosts to that service.

And in our own service we only need a reference, similar to a key, which points to the record in the service discovery system.

As we saw in the RabbitMQ post or the Vault post, the creation of the architecture will be independent from the code itself, but you can make it so a service registers itself automatically when it is deployed, if you want.

A comparison would be something similar to a DNS service but working only inside our own architecture.

3 - What is Hashicorp Consul?

Just as we saw in the previous post about credential management, I'm going to use another Hashicorp software here, in this case Consul.

The main reason is that Consul is not just a service discovery tool, but it also works as an app mesh if you need it. This means it includes elements such as observability, security between components, health checks, etc., all of this being 100% compatible with services like AWS, Azure, Google Cloud and so on.

But for this post, we will only use the registering and discovery part of the service.

3.1 - Adding Consul with Docker

As in previous posts, the first thing we'll do is add Consul to our infrastructure. To do that, we just include it in our docker file, where we already have Vault and RabbitMQ:

consul:
container_name: consul
image: consul
ports:
    - 8500:8500 # this is the UI/API calls
    - 8400:8400
    - 8600:8600
    - 8600:8600/udp

With this configuration, you can access the UI if needed at localhost:8500. Now all that's left is to register the services inside Consul. You can do this via the UI, the API, or the command line.

In our case, as with Vault, we will create the elements through the command line, using a bash file, which we'll execute in our architecture's initialization script:

#!/bin/bash
docker exec -it consul consul services register -name=RabbitMQ -address=localhost
docker exec -it consul consul services register -name=SecretManager -address=http://localhost -port=8200
docker exec -it consul consul services register -name=EmailsApi -address=http://localhost -port=50120
docker exec -it consul consul services register -name=ProductsApi -address=http://localhost -port=50220
docker exec -it consul consul services register -name=OrdersApi -address=http://localhost -port=50320
docker exec -it consul consul services register -name=SubscriptionsApi -address=http://localhost -port=50420

Consul has excellent documentation, but for this example, we only need the register command with these properties:

• name for the service name
• address for the location (in production, this will be different)
• port for the port; if not specified, port 0 will be used automatically

If we go to the Consul interface, we can see how all the services have been registered correctly.

Of course, we can fetch all this information via the API, the command line, or as we'll do, with the SDK.

3.2 - Consul implementation in .NET

For our implementation, we'll follow the logic of abstracting the configuration so that if we ever want to switch from Consul to another service, we only need to update the implementation in one place.

And in our particular case, since we register the services through our infrastructure, we only need to read those records.

To do this, we create a project called Distribt.Shared.Discovery where we'll create our abstraction.

First, we need to add the reference or include the Consul NuGet package. Personally, I'll also add in-memory caching, since with the current architecture the addresses will not change. But this 100% depends on your architecture.

If they are likely to change month by month, you can implement a wrapper to retry the call if it fails.

For example: use the cache first, and if it fails, query the records again and update the cache.

Once we have the reference, we create the interface to fetch the information:

public interface IServiceDiscovery
{
    Task<string> GetFullAddress(string serviceKey, CancellationToken cancellationToken = default);
}

And now we create the Consul implementation in C# to read the service information, where we only need to inject the IConsulClient interface to use the entire API. In our case, we only need to read a service from the catalog:

public class ConsulServiceDiscovery : IServiceDiscovery
{
    private readonly IConsulClient _client;
    private readonly MemoryCache _cache;

    public ConsulServiceDiscovery(IConsulClient client)
    {
        _client = client;
        _cache = new MemoryCache(new MemoryCacheOptions());
    }


    public async Task<string> GetFullAddress(string serviceKey, CancellationToken cancellationToken = default)
    {
        if (_cache.TryGetValue(serviceKey, out string serviceAddress))
        {
            return serviceAddress;
        }

        return await GetAddressFromService(serviceKey, cancellationToken);
    }


    private async Task<string> GetAddressFromService(string serviceKey, CancellationToken cancellationToken = default)
    {
        
        var services = await _client.Catalog.Service(serviceKey, cancellationToken);
        if (services.Response != null && services.Response.Any())
        {
            var service = services.Response.First();
            StringBuilder serviceAddress = new StringBuilder();
            serviceAddress.Append(service.ServiceAddress);
            if (service.ServicePort != 0)
            {
                serviceAddress.Append($":{service.ServicePort}");
            }

            string serviceAddressString = serviceAddress.ToString();
            
            AddToCache(serviceKey, serviceAddressString);
            return serviceAddressString;
        }

        throw new ArgumentException($"seems like the service your are trying to access ({serviceKey}) does not exist ");
    }

    private void AddToCache(string serviceKey, string serviceAddress)
    {
        _cache.Set(serviceKey, serviceAddress);
    }
}

To use the library, we must add it to the dependency container as well as the client configuration, where we specify the URL Consul is running at:

public static IServiceCollection AddDiscovery(this IServiceCollection services, IConfiguration configuration)
{
    return services.AddSingleton<IConsulClient, ConsulClient>(provider => new ConsulClient(consulConfig =>
        {
            var address = configuration["Discovery:Address"];
            consulConfig.Address = new Uri(address);
        }))
        .AddSingleton<IServiceDiscovery, ConsulServiceDiscovery>();
}

We must also update the projects that will use this functionality.

The first step is to add the correct configuration to the appsettings.json file

{
  ...
  "Discovery": {
    "Address": "http://localhost:8500"
  },
  ...
}

Note: in our case, service discovery will be part of the application's core and it will be configured in DefaultDistribtWebApplication, so this information should be added to all services.

After this, we must update the libraries/services to fetch the service discovery URL instead of from the configuration file.

For the case of Vault, for example, we just create a method to get that URL and add it to the configuration.

public static void AddSecretManager(this IServiceCollection serviceCollection, IConfiguration configuration)
{
    //TODO: create an awaiter project instead of .result everywhere in the config
    string discoveredUrl = GetVaultUrl(serviceCollection.BuildServiceProvider()).Result;
    serviceCollection.AddVaultService(configuration, discoveredUrl); 
}

private static async Task<string> GetVaultUrl(IServiceProvider serviceProvider)
{
    var serviceDiscovery = serviceProvider.GetService<IServiceDiscovery>();
    return await serviceDiscovery?.GetFullAddress(DiscoveryServices.Secrets)!;
}

Note: In our example the URLs are static. If they can change in your service, you will have to manage that rotation manually in your code.

Also, when you use .Result you block the thread; if you do this at app startup, it doesn't really matter, but don't run this action outside initialization.

However, when configuring, you can't run async code, so that's just how it is.

If we run the code, we see that it works without problems:

It is very common as a practice to have all the keys for the services you will use in a common file, so you don't have to remember the keys and instead use the reference itself:

public class DiscoveryServices
{
    public const string RabbitMQ = "RabbitMQ";
    public const string Secrets = "SecretManager";
    
    public class Microservices
    {
        public const string Emails = "EmailsApi";
        public const string Orders = "OrdersAPi";
        public const string Products = "ProductsAPi";
        public const string Subscriptions = "SubscriptionsAPi";
    }
}

Finally, we need to update all the appsettings files and the libraries affected by this.

Don't forget to remove references to all hostnames/urls from the appsettings files, including rabbitMQ or Vault among other services.

Conclusion

• In this post, we've seen what a service registry is in a distributed architecture
• What Consul is and how to configure it with Docker
• How to implement Consul with C#