Implementing HTTP Resilience by Microsoft

I was working on a project that needed to call multiple webservices, so I wanted to add resilience. Luckily, Microsoft has created a HTTP Resilience package as a wrapper for Polly, adding sensible defaults. In this blog I'll explore how easy it is to add the new package to our services, including deferring to configuration.

Idea

Our solution has 4 components:

We'll use a named HTTP client, as this makes it easier to bind it to our configuration.
We're using the Microsoft.Extensions.Http.Resilience package to implement resilience, configuration and defaults.
We'll create an option class that we'll use to change defaults and bind to the configuration.
We'll create some extension methods to hook it all up and make our setup reusable.

Background

At Wehkamp, we use many micro services; in 99% of the cases the resilience of a dependency is exactly the same for its endpoints. In other words: we can define the resilience per dependent service.

The Microsoft.Extensions.Http.Resilience package has the following defaults:

Order	Strategy	Description	Defaults
1	Rate limiter	The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency.	Queue: `0` Permit: `1_000`
2	Total timeout	The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including retry attempts, doesn't exceed the configured limit.	Total timeout: 30s
3	Retry	The retry pipeline retries the request in case the dependency is slow or returns a transient error.	Max retries: `3` Backoff: `Exponential` Use jitter: `true` Delay:2s
4	Circuit breaker	The circuit breaker blocks the execution if too many direct failures or timeouts are detected.	Failure ratio: 10% Min throughput: `100` Sampling duration: 30s Break duration: 5s
5	Attempt timeout	The attempt timeout pipeline limits each request attempt duration and throws if it's exceeded.	Attempt timeout: 10s

These are pretty sensible defaults. The only thing I want to change is setting the Retry.Delay to 100ms.

Example

Let's use the excellent httpstat.us service to get some error responses. The random endpoint will randomly return on of the specified status codes.

namespace Ktt.Resilience.Clients.HttpClients;

public class HttpStatusApiService(HttpClient client)
{
    public async Task<string> Get()
    {
        var path = "/random/200,500-508";
        return await client.GetStringAsync(path);
    }
}

In most cases this request will return an error in the 500 range, causing a retry. To show resilience in action, we'll create an app that will poll our service 10 times:

using ICustomHttpStatusApiClient = Ktt.Resilience.Clients.HttpClients.HttpStatusApiService;
using KiotaHttpStatusClient = Ktt.Resilience.Clients.Kiota.HttpClients.HttpStatus.HttpStatusClient;

public class DemoRetry(
    ICustomHttpStatusApiClient customHttpStatusApiClient,
    KiotaHttpStatusClient kiotaHttpStatusClient
)
{
    public async Task RunAsync()
    {
        await Execute("CustomHttpStatusApiClient", async () => await customHttpStatusApiClient.Get());
        await Execute("KiotaHttpStatusClient", async () => await kiotaHttpStatusClient.Random.TwoZeroZeroFiveZeroZeroFiveZeroTwoFiveZeroThree.GetAsync());

        Console.WriteLine("");
    }

    private async Task Execute(string name, Func<Task<string?>> execute)
    {
        Console.WriteLine($"Retry Demo for {name}...");

        for (var i = 0; i < 4; i++)
        {
            Console.WriteLine("");
            Console.WriteLine($"Calling {name} iteration {i}");

            try
            {
                var str = await execute();
                Console.WriteLine($"Result: {str}");
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error: {ex.Message}");
            }
        }
    }
}

If we don't add the resilience handlers, our program will fail pretty quickly.

Config

Any HttpClient needs to have a URL. I hate to ship such a URL in my source code, so let's define an options class that stores it. When we inherit from HttpStandardResilienceOptions, we get a nice way of overriding the retry delay to 500ms (without having to add it to all configs).

using Microsoft.Extensions.Http.Resilience;
using System.ComponentModel.DataAnnotations;

namespace Ktt.Resilience.Clients.Config;

public class HttpClientOptions : HttpStandardResilienceOptions
{
    [Required]
    [Url]
    [RegularExpression(@"^(http|https)://.*$", ErrorMessage = "The URL must start with http:// or https://")]
    public string BaseUrl { get; set; } = string.Empty;

    public HttpClientOptions()
    {
        Retry.Delay = TimeSpan.FromMilliseconds(100);
    }

    public void CopyTo(HttpStandardResilienceOptions other)
    {
        // I hate that this is needed...

        other.AttemptTimeout = AttemptTimeout;
        other.CircuitBreaker = CircuitBreaker;
        other.RateLimiter = RateLimiter;
        other.Retry = Retry;
        other.TotalRequestTimeout = TotalRequestTimeout;
    }
}

Notice the CopyTo method. It'll be used to copy the settings from our config to the HttpStandardResilienceOptions. I wanted to used the same class / config, but I can't get it injected into the standard pipeline. Copying the settings seems the next best thing.

Now, we want to override some of the details of the retry specifically for the HttpStatusApi. We want to set the BaseUrl and let's set the number of retries to 7. We don't want to wait too long for every retry, let's set the backoff type to Linear.

{
  "HttpClients": {
    "HttpStatusApi": {
      "BaseUrl": "https://httpstat.us",
      "Retry": {
        "MaxRetryAttempts": 7,
        "BackoffType": "Linear"
      }
    }
  }
}

Dependency injection 1: configuration

Let's create an extension method to bind the config to our HttpClientOptions to a named section. To make things easier, let's name the option also after our section.

public static IServiceCollection AddNamedOptionsForHttpClient(this IServiceCollection services, string sectionName)
{
    return AddNamedOptionsForHttpClient<HttpClientOptions>(services, sectionName);
}

public static IServiceCollection AddNamedOptionsForHttpClient<TOptions>(this IServiceCollection services, string sectionName)
    where TOptions : HttpClientOptions, new()
{
    services
        .AddOptionsWithValidateOnStart<TOptions>(sectionName)
        .BindConfiguration(sectionName)
        .Validate(options =>
        {
            var results = new List<ValidationResult>();
            var context = new ValidationContext(options);
            if (!Validator.TryValidateObject(options, context, results, validateAllProperties: true))
            {
                throw new OptionsValidationException(
                    sectionName,
                    typeof(TOptions),
                    results.Select(r => $"[{sectionName}] {r.ErrorMessage}")
                );
            }
            return true;
        });

We'll need to add a named option to configure our resilience pipeline. When you dive into the code of the package, you'll discover that the AddStandardResilienceHandler will request a named option with the name builder-name-standard. I don't want to register the section twice and I want to reuse the defaults of the HttpClientOptions class. Solution: use a IOptionsMonitor to get the HttpClientOptions and copy the settings to HttpStandardResilienceOptions.

Dependency injection 2: http client with resilience pipeline

Finally, we're ready to setup our named client. We use generics to bind it to the target class and instantiate the options. We reuse the name of the section as name for the client.

{
    public static IHttpStandardResiliencePipelineBuilder AddHttpClientWithResilienceHandler<TClass>(
        this IServiceCollection services,
        string sectionName
    )
        where TClass : class
    {
        return services.AddHttpClientWithResilienceHandler<TClass, HttpClientOptions>(sectionName);
    }

    public static IHttpStandardResiliencePipelineBuilder AddHttpClientWithResilienceHandler<TClass, TConfig>(
        this IServiceCollection services,
        string sectionName)
        where TClass : class
        where TConfig : HttpClientOptions, new()
    {
        var httpClientBuilder = services
            // bind the configuration section
            .AddNamedOptionsForHttpClient<TConfig>(sectionName)
            // add the HttpClient itself, configure the base URL
            .AddHttpClient<TClass>(sectionName, (serviceProvider, client) =>
            {
                var monitor = serviceProvider.GetRequiredService<IOptionsMonitor<TConfig>>();
                var config = monitor.Get(sectionName);
                if (!string.IsNullOrWhiteSpace(config?.BaseUrl))
                {
                    client.BaseAddress = new Uri(config.BaseUrl);
                }
            });

        // add resilience handler
        return httpClientBuilder.AddStandardResilienceHandler();

Now, we can register it all in our program:

// the order matters!
services.AddTransient<HttpStatusApiService>();
services
    .AddHttpClientWithResilienceHandler<HttpStatusApiService>("HttpClients:HttpStatusApi")
    .Configure(config =>
    {
        config.Retry.OnRetry = async args =>
        {
            Console.WriteLine($"Retry {args.AttemptNumber}: Retrying after {args.RetryDelay} due to {args.Outcome.Result?.StatusCode}");
            await Task.CompletedTask;
        };
    });

When we run our App, we see the following: