Set up end-to-end tracing

Granit.Observability configures Serilog (structured logs) and OpenTelemetry (traces and metrics) with OTLP export to the LGTM stack (Loki/Grafana/Tempo/Mimir). Combined with Granit.Wolverine’s trace context propagation, you get a single trace spanning HTTP requests, asynchronous message handlers, and database queries.

Prerequisites

• A .NET 10 project with Granit module system configured
• An OpenTelemetry Collector (or Grafana Alloy) reachable via gRPC
• Grafana with Tempo (traces) and Loki (logs) data sources configured

Step 1 — Install packages

dotnet add package Granit.Observability

If you use Wolverine messaging, the Granit.Wolverine package provides trace context propagation automatically — no additional packages are needed.

Step 2 — Configure the OTLP endpoint

Add the Observability section to appsettings.json:

{
  "Observability": {
    "ServiceName": "my-backend",
    "ServiceVersion": "1.0.0",
    "OtlpEndpoint": "http://otel-collector:4317",
    "ServiceNamespace": "my-company",
    "Environment": "production",
    "EnableTracing": true,
    "EnableMetrics": true
  }
}

Property	Type	Default	Description
ServiceName	string	unknown-service	Identifies the service in Tempo and Loki
ServiceVersion	string	0.0.0	Displayed in trace attributes
OtlpEndpoint	string	http://localhost:4317	gRPC endpoint of the OpenTelemetry Collector
ServiceNamespace	string	my-company	Groups related services in dashboards
Environment	string	development	Deployment environment tag
EnableTracing	bool	true	Enable/disable distributed tracing
EnableMetrics	bool	true	Enable/disable metrics export

Step 3 — Register observability

Module system (recommended)

With the Granit module system, GranitObservabilityModule is loaded automatically via AddGranit<T>(). No additional code is needed.

Direct registration

builder.AddGranitObservability();

This single call configures:

• Serilog with console sink (development) and OpenTelemetry sink (OTLP to Loki)
• OpenTelemetry tracing with ASP.NET Core, HttpClient, and EF Core instrumentation
• OpenTelemetry metrics with ASP.NET Core and HttpClient instrumentation
• Resource attributes: service.name, service.version, service.namespace, deployment.environment

Step 4 — Configure log levels

Serilog reads additional configuration from appsettings.json:

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft.AspNetCore": "Warning",
        "Microsoft.EntityFrameworkCore": "Warning"
      }
    }
  }
}

Automatic instrumentations

Traces

Instrumentation	Data collected
ASP.NET Core	Incoming HTTP requests (except /healthz)
HttpClient	Outgoing HTTP requests
Entity Framework Core	SQL queries

Exceptions are recorded automatically (RecordException = true).

Metrics

Instrumentation	Metrics collected
ASP.NET Core	Request duration, response codes
HttpClient	Outgoing call duration

Correlating across Wolverine (async boundaries)

The main challenge in distributed tracing is maintaining trace correlation across asynchronous boundaries. When an HTTP request publishes a Wolverine message via the transactional outbox, the worker that processes that message seconds (or hours) later must appear under the same trace in Tempo.

How it works

Granit.Wolverine handles this automatically:

1. Sender side — OutgoingContextMiddleware captures the current Activity.Current.Id (W3C traceparent) and injects it into the Wolverine envelope headers along with TenantId and UserId

2. Receiver side — TraceContextBehavior reads the traceparent header, parses it with ActivityContext.TryParse(), and starts a bridge activity named wolverine.message.handle with the original trace as parent

The result: every span created during handler execution (EF Core queries, HTTP calls, custom spans) appears as a child of the original HTTP request trace.

Execution pipeline

[Incoming message]
  -> TenantContextBehavior.Before()    -- restores ICurrentTenant
  -> UserContextBehavior.Before()      -- restores ICurrentUserService
  -> TraceContextBehavior.Before()     -- restores trace-id (Activity bridge)
  -> [Handler execution]
  -> TraceContextBehavior.After()      -- disposes the bridge activity
  -> UserContextBehavior.After()
  -> TenantContextBehavior.After()

Complete example

HTTP endpoint — publishes an event:

internal static class OrderEndpoints
{
    internal static async Task<Accepted> PlaceOrderAsync(
        PlaceOrderRequest request,
        IMessageBus bus,
        ILogger<PlaceOrderRequest> logger,
        CancellationToken cancellationToken)
    {
        logger.LogInformation(
            "Placing order for {ProductId}, quantity {Quantity}",
            request.ProductId, request.Quantity);

        await bus.PublishAsync(new OrderPlacedEvent(
            request.ProductId,
            request.Quantity));

        return TypedResults.Accepted(value: (string?)null);
    }
}

public sealed record PlaceOrderRequest(Guid ProductId, int Quantity);
public sealed record OrderPlacedEvent(Guid ProductId, int Quantity);

Wolverine handler — processes the event with the same trace-id:

public static class OrderPlacedEventHandler
{
    public static async Task HandleAsync(
        OrderPlacedEvent evt,
        AppDbContext db,
        ILogger logger,
        CancellationToken cancellationToken)
    {
        // This log shares the same TraceId as the HTTP request
        logger.LogInformation(
            "Processing order for {ProductId}, quantity {Quantity}",
            evt.ProductId, evt.Quantity);

        var order = new Order
        {
            ProductId = evt.ProductId,
            Quantity = evt.Quantity,
            Status = OrderStatus.Processing
        };

        db.Orders.Add(order);
        await db.SaveChangesAsync(cancellationToken);
    }
}

Adding custom spans to your modules

Each Granit module that performs significant I/O declares a dedicated ActivitySource. Follow this pattern for your own modules:

1. Create an ActivitySource

using System.Diagnostics;

internal static class InventoryActivitySource
{
    internal const string Name = "MyApp.Inventory";
    internal static readonly ActivitySource Source = new(Name);
    internal const string CheckStock = "inventory.check-stock";
}

2. Register it

using Granit.Observability;

public static IServiceCollection AddInventory(this IServiceCollection services)
{
    GranitActivitySourceRegistry.Register(InventoryActivitySource.Name);
    // ... other registrations
    return services;
}

3. Instrument I/O operations

public async Task<int> CheckStockAsync(
    Guid productId, CancellationToken cancellationToken)
{
    using var activity = InventoryActivitySource.Source.StartActivity(
        InventoryActivitySource.CheckStock);
    activity?.SetTag("inventory.product_id", productId.ToString());

    var stock = await _dbContext.Stock
        .Where(s => s.ProductId == productId)
        .Select(s => s.Quantity)
        .FirstOrDefaultAsync(cancellationToken);

    activity?.SetTag("inventory.stock_level", stock);
    return stock;
}

Good to know

If no OpenTelemetry listener is registered (e.g., Granit.Observability is not installed), StartActivity() returns null and all instrumentation becomes a no-op. This is the soft-dependency pattern — modules never reference Granit.Observability directly.

Registered ActivitySources in Granit

Source	Module	Spans
Granit.Wolverine	Granit.Wolverine	wolverine.message.handle
Granit.Webhooks	Granit.Webhooks	webhooks.deliver, webhooks.fanout
Granit.Notifications	Granit.Notifications	notifications.deliver, notifications.fanout
Granit.BackgroundJobs	Granit.BackgroundJobs	backgroundjobs.trigger
Granit.BlobStorage.S3	Granit.BlobStorage.S3	blobstorage.upload-ticket, blobstorage.download-url, blobstorage.delete
Granit.Identity.Keycloak	Granit.Identity.Keycloak	identity.keycloak.*

Viewing traces in Grafana

Grafana Tempo

Search by trace ID to see the complete waterfall:

POST /api/orders (12ms)
  |-- PublishAsync: OrderPlacedEvent (2ms)
  +-- [Wolverine] OrderPlacedEventHandler (45ms)
        |-- EF Core: INSERT INTO orders (8ms)
        +-- Commit transaction (3ms)

Grafana Loki

Filter logs by TraceId to see all log entries from a single operation:

{service_name="my-backend"} | json | TraceId = "abc-123"

Every Serilog log entry includes TraceId and SpanId. In Grafana, clicking a log entry opens the corresponding trace in Tempo.

OTLP pipeline

The typical production pipeline:

Application -> OTLP gRPC -> OpenTelemetry Collector -> Tempo (traces)
                                                     -> Mimir (metrics)
                                                     -> Loki (logs)

Edge cases

Situation	Behavior
No traceparent header in Wolverine envelope	No-op — handler runs normally
Malformed traceparent header	Warning log + no-op (no exception)
Granit.Wolverine source not listened	StartActivity() returns null — silent no-op
No active Activity at publish time	traceparent header is absent

Pro tip

AddGranitObservability() must be called after all module registrations. This is the default in the Granit pattern — observability is always the last configuration step. The method iterates GranitActivitySourceRegistry and calls .AddSource() for each registered ActivitySource.

Next steps

• Configure blob storage — blob storage operations emit their own ActivitySource spans
• Implement a workflow to trace workflow transitions across async handlers
• Observability reference for the complete configuration options