PII-Safe Telemetry

Observability systems (tracing, metrics, logging) can inadvertently capture personal data — user IDs, email addresses, tenant identifiers — in span tags, log properties, and metric dimensions. Excalibur.Dispatch provides a sanitization layer that hashes, suppresses, or passes through telemetry values based on tag classification.

Package: Excalibur.Dispatch.Observability

Architecture

ITelemetrySanitizer (interface, 2 methods)
├── HashingTelemetrySanitizer      (default, SHA-256 hashing)
└── ComplianceTelemetrySanitizer   (regex pattern detection + hashing)

• HashingTelemetrySanitizer — Classifies tags into three categories: hashed (SHA-256), suppressed (tag omitted entirely), or passthrough (unchanged). Uses a bounded cache (1024 entries) for hash performance.
• ComplianceTelemetrySanitizer — Layers regex-based PII pattern detection (emails, phone numbers, SSNs) on top of the baseline hashing sanitizer.

Setup

Basic (Registered Automatically)

UseObservability() registers HashingTelemetrySanitizer as the default ITelemetrySanitizer:

builder.Services.AddDispatch(dispatch =>
{
    dispatch.UseObservability();
});

No additional configuration is needed — the default configuration hashes common PII tags and suppresses highly sensitive tags.

Security Auditing (Safe by Default)

AddSecurityAuditing() installs a masking sanitizer as the default ITelemetrySanitizer (registered with TryAdd), so a security-audit sink never emits raw PII out of the box — the previous no-op passthrough is now opt-in only. Tag values are replaced with a stable sha256:-prefixed fingerprint and secret-shaped free-form payloads are redacted, so the same identifier correlates across events without exposing the raw value.

The default fingerprint is a stable pseudonym suitable for correlation. For cryptographic protection of low-entropy identifiers (for example a short user ID or an IP address, where an attacker could brute-force the hash domain), register the keyed sanitizer instead:

// Raw passthrough is opt-in only — register it explicitly if you truly need raw values:
// builder.Services.AddSingleton<ITelemetrySanitizer>(NullTelemetrySanitizer.Instance);

UseObservability() still overrides the default via TryAdd precedence when you want the full HashingTelemetrySanitizer classification described above.

Custom Tag Classification

Configure which tags are hashed vs suppressed:

builder.Services.AddDispatch(dispatch =>
{
    dispatch.UseObservability();
});

builder.Services.Configure<TelemetrySanitizerOptions>(opt =>
{
    // Tags whose values are hashed (SHA-256) before emission
    opt.SensitiveTagNames =
    [
        "user.id",
        "user.name",
        "auth.user_id",
        "auth.subject_id",
        "auth.identity_name",
        "auth.tenant_id",
        "audit.user_id",
        "tenant.id",
        "tenant.name",
        "dispatch.messaging.tenant_id",
    ];

    // Tags suppressed entirely (not emitted)
    opt.SuppressedTagNames =
    [
        "auth.email",
        "auth.token",
    ];
});

Development Override

Bypass all sanitization in development environments for debugging:

builder.Services.Configure<TelemetrySanitizerOptions>(opt =>
{
    opt.IncludeRawPii = true; // Development only!
});

A startup warning is emitted if IncludeRawPii is true in non-Development environments.

Keyed fingerprints (pepper)

By default, sensitive tag values are fingerprinted with an unkeyed SHA-256 digest. That is a stable pseudonym — the same input always yields the same sha256: fingerprint, so identifiers correlate across events — but for low-entropy values (a short numeric user ID, a source IP) an attacker who observes the fingerprints can brute-force or use a rainbow table to recover the original value, because the hash domain is small and unkeyed.

Supply a secret pepper to upgrade fingerprinting to keyed HMAC-SHA-256. Without the key, the fingerprints cannot be reversed by brute force:

// Observability sanitizer (HashingTelemetrySanitizer)
builder.Services.Configure<TelemetrySanitizerOptions>(opt =>
{
    // High-entropy secret sourced from a secret manager / KMS — never a hard-coded literal.
    opt.Pepper = pepperBytes;
});

The security-audit masking sanitizer exposes the same knob:

// Security-audit sanitizer (MaskingTelemetrySanitizer), installed by AddSecurityAuditing()
builder.Services.Configure<MaskingTelemetrySanitizerOptions>(opt =>
{
    opt.Pepper = pepperBytes;
});

• Pepper is byte[]?. When null (the default), fingerprinting falls back to the unkeyed SHA-256 digest.
• Source the key from a secret manager / KMS. It is a secret: rotating it changes every fingerprint (breaking cross-time correlation), so treat rotation like a key-management event.
• Fingerprinting never throws on the telemetry/audit path regardless of this setting (fail-open) — a derivation problem degrades to a correlation-only fingerprint, it never breaks the operation being observed.

Compliance-Level Sanitization

For regulated environments, add regex-based PII detection that catches patterns even in unclassified tags:

builder.Services.AddDispatch(dispatch =>
{
    dispatch.UseObservability();
});
builder.Services.AddComplianceTelemetrySanitizer(opt =>
{
    // Add custom patterns beyond the built-in email/phone/SSN detection
    opt.CustomPatterns.Add(new PiiPattern("medical-record", @"MRN-\d{8}"));
});

ITelemetrySanitizer Interface

namespace Excalibur.Dispatch.Telemetry;

public interface ITelemetrySanitizer
{
    /// <summary>
    /// Sanitizes a telemetry tag value. Returns null to suppress the tag entirely.
    /// </summary>
    string? SanitizeTag(string tagName, string? rawValue);

    /// <summary>
    /// Sanitizes a payload string (e.g., message body or log message).
    /// </summary>
    string SanitizePayload(string payload);
}

How Values Are Sanitized

Classification	Behavior	Example
Sensitive (in SensitiveTagNames)	Hashed to sha256:<hex>	user.id: "john" → user.id: "sha256:a8cfcd..."
Suppressed (in SuppressedTagNames)	Tag omitted entirely	auth.email → not emitted
Passthrough (not in either list)	Returned unchanged	http.method: "GET" → http.method: "GET"
IncludeRawPii = true	All values passed through	No sanitization applied

Middleware Integration

The following middleware inject ITelemetrySanitizer and sanitize telemetry data before emission:

Middleware	What It Sanitizes
TracingMiddleware	Span tags and error descriptions
LoggingMiddleware	Log properties
AuditLoggingMiddleware	Audit trail entries
MetricsLoggingMiddleware	Metric dimension values
AuthenticationMiddleware	Identity-related span tags
AuthorizationMiddleware	Authorization context tags
TenantIdentityMiddleware	Tenant identification tags
JwtAuthenticationMiddleware	JWT claim values in spans
RetryMiddleware	Error descriptions in retry spans
CircuitBreakerMiddleware	Error descriptions in circuit breaker spans

SetSanitizedErrorStatus Extension

When recording exceptions on OpenTelemetry spans, use the sanitized extension to prevent PII in error messages from leaking:

using Excalibur.Dispatch.Extensions;

// Instead of:
activity.SetStatus(ActivityStatusCode.Error, exception.Message); // PII risk!

// Use:
activity.SetSanitizedErrorStatus(exception, sanitizer);

This method:

1. Returns only the exception type name for well-known system exceptions (no PII risk)
2. Sanitizes the message using ITelemetrySanitizer.SanitizePayload() for all other exceptions
3. Records a sanitized exception event on the span
4. Sets the span status to Error

SensitiveDataPostConfigureOptions

The SensitiveDataPostConfigureOptions class automatically flows the IncludeRawPii toggle into all IncludeSensitiveData flags across the framework:

Options Class	Property
TracingOptions	IncludeSensitiveData
AuditLoggingOptions	IncludeSensitiveData
ObservabilityOptions	IncludeSensitiveData

When TelemetrySanitizerOptions.IncludeRawPii = true, all three are set to true automatically. This ensures a single toggle controls PII inclusion across the entire pipeline.

TelemetrySanitizerOptions Reference

Property	Type	Default	Description
IncludeRawPii	bool	false	Bypass all sanitization (development only)
Pepper	byte[]?	null	Secret key for keyed HMAC-SHA-256 fingerprints (see Keyed fingerprints); null falls back to unkeyed SHA-256
SensitiveTagNames	IList<string>	10 common PII tags	Tags hashed before emission
SuppressedTagNames	IList<string>	auth.email, auth.token	Tags suppressed entirely

See Also

• Telemetry Configuration — Configure OpenTelemetry metrics and tracing
• Metrics Reference — Complete metrics catalog
• Production Observability — Monitoring guide