What is Rate Limiting? Implementation in .NET

This is a post I've been wanting to make for a while, and I decided to wait for the release of .NET 7 because the new version of .NET brings native rate limiting functionality, which simplifies things significantly.

1 - What is Rate Limiting?

When we create an API, sometimes—especially when the API is externally accessible—we want to limit its usage.

This is done to prevent overuse of the API. A clear example is in applications with paid plans: the paid version allows X number of calls, while the free version allows far fewer:

A real-world example is the YouTube API, which only allows "queries" for 10,000 units (videos, comments, etc.) per day.

What is NOT rate limiting: a filter that limits where a client can access or permissions. For that, we use either API Keys or JWT.

2- Why use rate limiting?

One reason is to directly limit clients, for instance, accept only X number of calls, and then have a higher-tier or paid version that allows more calls.

Another example, though less common nowadays, is that the app or database can only handle X concurrent requests; if you exceed that number, the app degrades or even crashes. To make sure this doesn’t happen, we limit the API usage.

And, of course, it protects you from DDoS attacks and similar threats.

These reasons, or a mix of them, are the most common. A typical situation: we have clients who can make all the calls they want, but there's an asterisk: 10 per minute or something similar.

3 - Implementing rate limiting in .NET

As I mentioned, in the new version of .NET (version 7), rate limiting comes out of the box. So if you create or update a project to net7 or above, you can use the middleware that allows you to configure rate limiting.

For this post, I implemented rate limiting in the app from the Distributed Systems course Distribt, with code available at GitHub.

This is the complete diagram, but for the example, just keep in mind the green dot and a bit about the API key from the previous post.

The library is pretty complete, and we'll look at its different options. In any case, we must add the extension method to configure the rate limiter middleware UseRateLimiter().

webappBuilder.Services.AddRateLimiter(options => {...})app.UseRateLimiter();

By the way, place the UseRateLimiter middleware before your first endpoint, since as you’ll remember, middlewares execute in order.

Before moving to the different types of rate limiting, I want to explain a bit how the library works. Here’s a very basic example:

webappBuilder.Services.AddRateLimiter(options =>    options.AddConcurrencyLimiter(policyName: "concurrencyPolicy", limiterOptions =>    {        limiterOptions.PermitLimit = 10;    }));

This is the simplest example: any API with this configuration will only accept 10 simultaneous calls. But what happens to the rest?

3.1 - Queues in Rate Limiting

One option is to enable a queue. You specify its size and order (FIFO or LIFO).

webappBuilder.Services.AddRateLimiter(options =>    options.AddConcurrencyLimiter(policyName: "concurrencyPolicy", limiterOptions =>    {        limiterOptions.PermitLimit = 10;        limiterOptions.QueueLimit = 100;        limiterOptions.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;    }));

3.2 - Rejecting calls with Rate Limiting

But once the total call count and queue are full, what happens to new requests?

Well, if everything’s "full", we’ll reject them, and you can specify which status code to return with RejectionStatusCode.

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = 429; // Too many request    options.AddConcurrencyLimiter(policyName: "concurrencyPolicy", limiterOptions =>    {        limiterOptions.PermitLimit = 10;        limiterOptions.QueueLimit = 100;        limiterOptions.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;    });});

In our case, we return 429, which is commonly used to indicate too many requests.

3.3 - Response message in Rate Limiting

Returning only the error code is good but incomplete; ideally, include an error message with the OnRejected delegate:

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = 429; // Too many request    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };    options.AddConcurrencyLimiter(policyName: "concurrencyPolicy", limiterOptions =>    {        limiterOptions.PermitLimit = 10;        limiterOptions.QueueLimit = 100;        limiterOptions.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;    });});

Ideally we should standardize the response format to match the normalization of our other APIs.

3.4 - Identify who makes the call in rate limiting

To identify who's making the call and thus apply different rate limiting filters, we use PartitionKey, and we'll see how to configure it next.

In summary, it's like this:

partitionKey: “here the Identifier”

4 - Types of Rate limiting

Within rate limiting, we have several types of configurations we can use, and not only that, there are different ways (algorithms) to build such configurations. So far, we've seen options.AddConcurrencyLimiter(...), that's one way, but there's also options.GlobalLimiter, which we'll look at in a couple of examples.

Note: GlobalLimiter limits everything, while .Add.. must be specified per endpoint/controller, etc.

4.1 - Concurrency limit

This is the clearest and simplest example: just allow X simultaneous calls. If you allow 10 simultaneous calls, number 11 will be denied

And this would be its code:

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>        RateLimitPartition.GetConcurrencyLimiter(            partitionKey: "here the identifier",            factory: _ => new ConcurrencyLimiterOptions()            {                PermitLimit = 10,                QueueLimit = 0,                QueueProcessingOrder =  QueueProcessingOrder.OldestFirst            }));});

4.2 - Token bucket limit

The name comes from its real-world representation.

We have a limited number of tokens (calls), say 100. Each request spends one token; once they're all spent, you can't use the service anymore.

Here's the code:

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };    options.AddTokenBucketLimiter("policy-name", limiterOptions =>    {        limiterOptions.TokensPerPeriod = 2;        limiterOptions.TokenLimit = 100;        limiterOptions.ReplenishmentPeriod = TimeSpan.FromMinutes(5);    });});

In this example, we have a limit of 100 tokens and 2 tokens become available every 5 minutes.

4.3 - Fixed window limit

Similar to the previous case, the difference is there's no token count limit:

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };    options.AddFixedWindowLimiter("policy-name", limiterOptions =>    {        limiterOptions.PermitLimit = 2;        limiterOptions.QueueLimit = 10;        limiterOptions.Window = TimeSpan.FromMinutes(5);    });});

4.4 - Sliding window limit

Similar to the previous one, but each window has a specified number of segments. In this example:

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };        options.AddSlidingWindowLimiter("policy-name", limiterOptions =>    {        limiterOptions.PermitLimit = 5;        limiterOptions.QueueLimit = 10;        limiterOptions.SegmentsPerWindow = 5;        limiterOptions.Window = TimeSpan.FromMinutes(5);            });});

We have 5 calls per window, the window lasts 5 minutes, but at the same time, there are 5 segments, so we can only execute one call per minute.

5 - Custom Rate Limiting Policies

You can create a totally custom policy for your system. To do this, you just need to create a class that inherits from IRateLimiterPolicy<T> where T is the partition key you’re going to use—usually a string.

When you create the class, implement the members you need. You’ll get something like the following:

public class DistribtRateLimiterPolicy : IRateLimiterPolicy<string>{    public RateLimitPartition<string> GetPartition(HttpContext httpContext)    {        return RateLimitPartition.GetFixedWindowLimiter(            partitionKey: httpContext.Request.Headers["apiKey"].ToString(),            partition => new FixedWindowRateLimiterOptions            {                PermitLimit = 10,                Window = TimeSpan.FromMinutes(60),            });    }    public Func<OnRejectedContext, CancellationToken, ValueTask>? OnRejected { get; } =        (context, _) =>        {            context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;            context.HttpContext.Response.WriteAsync("too many calls, please try again later ");            return new ValueTask();        };}

Since this is Distribt and we know we have an API Key, we just read that key for the partition key, which represents a client. But for public services, you might have a policy where if an API key exists, it gets 10 calls per hour, otherwise only one call per hour.

• Note: this is a regular class, so it supports dependency injection, etc.

Now, we need to tweak the configuration a bit. We no longer use services.AddRateLimiter; instead, when using useRateLimiter, specify the policy.

app.UseRateLimiter();app.MapGet("/rate-limiting-test", () =>{    return "Hello World!";}).RequireRateLimiting(new DistribtRateLimiterPolicy());

6 - Rate limiting per endpoint

It's true that most rate limiting configurations are usually set for the whole API, but you can also limit per endpoint.

To do this, instead of app.UseRateLimiting() you configure a policy and then use it with RequireRateLimiting.

webappBuilder.Services.AddRateLimiter(options =>{    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    options.OnRejected = async (context, token) =>    {        await context.HttpContext.Response.WriteAsync("too many calls, please try again later ");    };    options.AddConcurrencyLimiter("policy-name", limiterOptions =>    {        limiterOptions.PermitLimit = 2;        limiterOptions.QueueLimit = 0;    });});....app.MapGet("/rate-limiting-test", () =>{    Thread.Sleep(2 * 60 * 1000);    return "Hello World!";}).RequireRateLimiting("policy-name");

Similarly, if you want to disable rate limiting, use the DisableRateLimiting filter or extension method.

app.MapGet("/", () => "Hello World!").DisableRateLimiting()

As I mentioned, this works for both minimal APIs and regular MVC controllers.

In MVC, we use the filter attributes [EnableRateLimiting("policy-name")] and [DisableRateLimiting] either at the controller or endpoint level.

Conclusion

In this post we've seen how to create Rate Limiting in .NET

We discussed the different types of Rate Limiting