Observability is not logging. Logging tells you what happened. Observability lets you ask arbitrary questions about your system's behavior and get answers without deploying new code. In modern .NET, OpenTelemetry provides the foundation for all three pillars: traces, metrics, and logs.
Setting Up OpenTelemetry in .NET
The OpenTelemetry SDK for .NET integrates directly with the framework's built-in abstractions. ASP.NET Core, HttpClient, and EF Core all emit telemetry data that OpenTelemetry can capture automatically.
builder.Services.AddOpenTelemetry()
.ConfigureResource(resource => resource
.AddService(serviceName: "OrderService", serviceVersion: "1.0.0"))
.WithTracing(tracing => tracing
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddEntityFrameworkCoreInstrumentation()
.AddSource("OrderService.Domain")
.AddOtlpExporter())
.WithMetrics(metrics => metrics
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddRuntimeInstrumentation()
.AddMeter("OrderService.Metrics")
.AddOtlpExporter());
Custom Traces for Business Operations
Auto-instrumentation captures HTTP requests and database queries, but the most valuable traces come from your business logic. Use ActivitySource to create custom spans that represent meaningful domain operations.
public class OrderProcessingService
{
private static readonly ActivitySource ActivitySource = new("OrderService.Domain");
public async Task<Order> ProcessOrderAsync(CreateOrderCommand command, CancellationToken ct)
{
using var activity = ActivitySource.StartActivity("ProcessOrder");
activity?.SetTag("order.customer_id", command.CustomerId.ToString());
activity?.SetTag("order.item_count", command.Items.Count);
// Validate inventory
using (ActivitySource.StartActivity("ValidateInventory"))
{
await ValidateInventoryAsync(command.Items, ct);
}
// Calculate pricing
using (var pricingActivity = ActivitySource.StartActivity("CalculatePricing"))
{
var total = await CalculatePricingAsync(command.Items, ct);
pricingActivity?.SetTag("order.total", total);
}
// Persist order
using (ActivitySource.StartActivity("PersistOrder"))
{
return await SaveOrderAsync(command, ct);
}
}
}
Custom Metrics for Business KPIs
Metrics should go beyond request latency and error rates. Track business-relevant metrics that help you understand whether your system is doing its job.
public class OrderMetrics
{
private readonly Counter<long> _ordersCreated;
private readonly Histogram<double> _orderValue;
private readonly UpDownCounter<long> _pendingOrders;
public OrderMetrics(IMeterFactory meterFactory)
{
var meter = meterFactory.Create("OrderService.Metrics");
_ordersCreated = meter.CreateCounter<long>("orders.created", "orders");
_orderValue = meter.CreateHistogram<double>("orders.value", "EUR");
_pendingOrders = meter.CreateUpDownCounter<long>("orders.pending", "orders");
}
public void OrderCreated(decimal value, string region)
{
_ordersCreated.Add(1, new KeyValuePair<string, object?>("region", region));
_orderValue.Record((double)value, new KeyValuePair<string, object?>("region", region));
_pendingOrders.Add(1);
}
}
Structured Logging Completes the Picture
Logs should be structured, not free-text strings. Use semantic logging with named properties so your log aggregation tool can index and query them.
logger.LogInformation(
"Order {OrderId} created for customer {CustomerId} with {ItemCount} items totaling {Total:C}",
order.PublicId, command.CustomerId, command.Items.Count, total);
The three pillars — traces, metrics, and logs — are interconnected through correlation IDs. A trace ID links a distributed operation across services. Log entries within that trace carry the same trace ID. Metrics can be correlated with traces through exemplars. This interconnection is what makes true observability possible: you see a spike in the error rate metric, drill into exemplar traces, and read the associated logs to understand the root cause.
Comments (2)
Great post! I learned a lot from this.
I agree, great article!
Interesting perspective, I hadn't thought of it that way.
Leave a Comment