Adding observability to micro services is vital if you want to discover bottle necks. In this blog, I'll show how we implemented Jaeger in .NET Core to observe incoming and outgoing requests. We'll also use a Jaeger decorator to observe spans in classes.
Jaeger describes itself as:
As on-the-ground microservice practitioners are quickly realizing, the majority of operational problems that arise when moving to a distributed architecture are ultimately grounded in two areas: networking and observability. It is simply an orders of magnitude larger problem to network and debug a set of intertwined distributed services versus a single monolithic application.
Why Jaeger
Run Jaeger locally for testing
To do some local development, you'll want to run Jaeger locally. The documentation describes how to run the stack locally with a single Docker container:
docker run -d --name jaeger \ -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \ -p 5775:5775/udp \ -p 6831:6831/udp \ -p 6832:6832/udp \ -p 5778:5778 \ -p 16686:16686 \ -p 14268:14268 \ -p 14250:14250 \ -p 9411:9411 \ jaegertracing/all-in-one:1.24
docker run -d --name jaeger ` -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 ` -p 5775:5775/udp ` -p 6831:6831/udp ` -p 6832:6832/udp ` -p 5778:5778 ` -p 16686:16686 ` -p 14268:14268 ` -p 14250:14250 ` -p 9411:9411 ` jaegertracing/all-in-one:1.24
The UI runs on http://localhost:16686. Super easy.
.NET Core Setup
Before we can start, we'll need to set some things up: NuGet Packages, environment variables and the Jaeger dependency injection (DI) configuration.
NuGet packages
Never go anywhere without a package. We'll be using the following packages:
- Install-Package Jaeger.Core -Version 1.0.2Install-Package Jaeger.Senders.Thrift -Version 1.0.2Install-Package OpenTracing.Contrib.NetCore -Version 0.7.1Install-Package Scrutor -Version 3.3.0
- dotnet add package Jaeger.Core --version 1.0.2dotnet add package Jaeger.Senders.Thrift --version 1.0.2dotnet add package OpenTracing.Contrib.NetCore --version 0.7.1dotnet add package Scrutor --version 3.3.0
- <PackageReference Include="Jaeger.Core" Version="1.0.2" /><PackageReference Include="Jaeger.Senders.Thrift" Version="1.0.2" /><PackageReference Include="OpenTracing.Contrib.NetCore" Version="0.7.1" /><PackageReference Include="Scrutor" Version="3.3.0" />
As always we're using Scrutor to write a decorator.
Environment variables
By default most Jaeger clients use environment variables (although I noticed that some implementation might use slightly different variable names). Locally we'll be using:
JAEGER_AGENT_HOST=localhost
JAEGER_AGENT_PORT=6831
JAEGER_SAMPLER_TYPE=const
JAEGER_SAMPLER_PARAM=1
This means we're tracing all requests. You don't want to do this in production, cause it might generate too much data.
Jaeger configuration
I like to have my configuration in a separate class to keep my Startup
class cleaner. This class will initialize the Jaeger tracer:
using Jaeger;
using Jaeger.Samplers;
using Jaeger.Senders;
using Jaeger.Senders.Thrift;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using OpenTracing;
using OpenTracing.Contrib.NetCore.Configuration;
using OpenTracing.Util;
public static class JaegerConfiguration
{
public static void AddJaeger(this IServiceCollection services)
{
// Use "OpenTracing.Contrib.NetCore" to automatically generate spans for ASP.NET Core, Entity Framework Core, ...
// See https://github.com/opentracing-contrib/csharp-netcore for details.
services.AddOpenTracing();
// Adds the Jaeger Tracer.
services.AddSingleton<ITracer>(serviceProvider =>
{
var serviceName = Program.Name;
var loggerFactory = serviceProvider.GetRequiredService<ILoggerFactory>();
// This is necessary to pick the correct sender, otherwise a NoopSender is used!
Jaeger.Configuration.SenderConfiguration.DefaultSenderResolver = new SenderResolver(loggerFactory)
.RegisterSenderFactory<ThriftSenderFactory>();
// This will log to a default localhost installation of Jaeger.
var tracer = new Tracer.Builder(serviceName)
.WithLoggerFactory(loggerFactory)
.Build();
// Allows code that can't use DI to also access the tracer.
if (!GlobalTracer.IsRegistered())
{
GlobalTracer.Register(tracer);
}
return tracer;
});
services.Configure<AspNetCoreDiagnosticOptions>(options =>
{
options.Hosting.IgnorePatterns.Add(context => context.Request.Path.Value.StartsWith("/status"));
options.Hosting.IgnorePatterns.Add(context => context.Request.Path.Value.StartsWith("/metrics"));
options.Hosting.IgnorePatterns.Add(context => context.Request.Path.Value.StartsWith("/swagger"));
});
}
}
Note: we're skipping the health check, metrics and documentation end points.
Startup: hook it up in DI
Add the following to your Startup
class DI:
public void ConfigureServices(IServiceCollection services)
{
services.AddJaeger();
}
Congratulations, Jaeger now works with you .NET Core application. By default it includes spans for your controllers and your outgoing HTTP requests. In the next section I'll be showing how to create a decorator, to log more information.
Jaeger decorator
First, make sure you've copied the Decorator
base class from here. The base class takes care of getting the class name from the decorated object. Next, you can copy this JaegerDecorator
:
using OpenTracing;
using System;
using System.Runtime.CompilerServices;
using System.Threading.Tasks;
/// <summary>
/// Implements Jaeger tracing as a decorator.
/// </summary>
/// <typeparam name="TDecorated">The decorated type.</typeparam>
public abstract class JaegerDecorator<TDecorated> : Decorator<TDecorated>
{
private ITracer _tracer;
protected JaegerDecorator(TDecorated decorated, ITracer tracer) : base(decorated)
{
_tracer = tracer ?? throw new ArgumentNullException(nameof(tracer));
}
protected virtual IScope StartScope(string methodName)
{
return _tracer
.BuildSpan($"{DecoratedClassName}.{methodName}")
.WithTag("method", methodName)
.WithTag("class", DecoratedFullClassName)
.StartActive(true);
}
protected async Task Decorate(
Func<Task> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
await action();
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected async Task Decorate(
Func<IScope, Task> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
await action(scope);
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected async Task<TReturn> Decorate<TReturn>(
Func<Task<TReturn>> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
return await action();
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected async Task<TReturn> Decorate<TReturn>(
Func<IScope, Task<TReturn>> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
return await action(scope);
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected TReturn Decorate<TReturn>(
Func<TReturn> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
return action();
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected TReturn Decorate<TReturn>(
Func<IScope, TReturn> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
return action(scope);
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected void Decorate(
Action action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
action();
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected void Decorate(
Action<IScope> action,
[CallerMemberName] string methodName = "")
{
using var scope = StartScope(methodName);
try
{
action(scope);
}
catch (Exception ex)
{
HandleException(ex, scope);
throw;
}
}
protected virtual void HandleException(Exception ex, IScope scope)
{
scope.Span.SetTag("error", true);
scope.Span.Log($"Exception: {ex.Message}");
}
}
Note: the code uses the simple using syntax - a C# 8.0 feature.
The basis of the class is the ITracer
object. It allows us to generate new spans. When an exception is caught, we'll set the tag error=true
, which will show up in the Jaeger UI like this:

Jaeger Decorator Implementation Example
The JaegerDecorator
exposes an IScope
object that can be used to log additional information in the request. It is optional, so you don't have to use it:
public interface IMyInterface
{
bool InOutMethod(int x);
void VoidMethod();
}
public class MyInterfaceLatencyDecorator : JaegerDecorator<IMyInterface>, IMyInterface
{
protected MyInterfaceLatencyDecorator(IMyInterface decorated, ITracer tracer) : base(decorated, tracer)
{
}
public bool InOutMethod(int x) =>
Decorate((scope) =>
{
scope.Span.SetTag("x", x);
return Decorated.InOutMethod(x);
});
public void VoidMethod() => Decorate(Decorated.VoidMethod);
}