Skip to main content

Outbox Pattern

New to reliable messaging?

Start with the Idempotent Consumer Guide to understand why messages get duplicated and how the Outbox and Inbox patterns work together.

The outbox pattern ensures reliable message publishing by storing messages in the same database transaction as your domain changes.

Delivery guarantee: at-least-once

Every staged message is delivered to its transport at least once. Under a dispatcher crash or a retry a message may be delivered more than once — the outbox is not exactly-once, and no configuration makes it so.

What you get is that a message cannot be lost: it commits with your state change, so an unreachable broker cannot cause a silent drop. What you owe in return is that your consumers must be idempotent — handling the same message twice must have the same effect as handling it once. The Inbox pattern provides the receive-side deduplication for this.

Before You Start​

  • .NET 10.0
  • Install the required packages: 
    dotnet add package Excalibur.Dispatch.Patterns
    dotnet add package Excalibur.EventSourcing.SqlServer  # or your provider
  • Familiarity with Dispatch pipeline and database transactions
  • A SQL Server or PostgreSQL database for outbox storage

The Problem​

Without the outbox pattern, you risk inconsistency:

The Solution​

Store messages in an outbox table within the same transaction:

Quick Start​

Configuration​

The outbox is configured through a single callback. The provider call and the processing settings both belong inside it:

services.AddDispatch(dispatch =>
{
    dispatch.AddHandlersFromAssembly(typeof(Program).Assembly);
});

// Configure the outbox and its store together. The provider call belongs INSIDE the
// AddOutbox callback -- that call is what registers the IOutboxStore the outbox drains
// and the one your own types inject.
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql
        .ConnectionString(connectionString)
        .SchemaName("outbox"));
}));

Processing settings​

Processing is configured with WithProcessing, and background draining is turned on with EnableBackgroundProcessing. Both hang off the same AddOutbox callback as the provider call:

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox => outbox
    .UseSqlServer(sql => sql
        .ConnectionString(connectionString)
        .SchemaName("outbox"))
    .WithProcessing(processing => processing
        .BatchSize(1000)
        .PollingInterval(TimeSpan.FromMilliseconds(100))
        .MaxRetryCount(3)
        .RetryDelay(TimeSpan.FromMinutes(1))
        .ProcessorId("worker-1")
        .EnableParallelProcessing(maxDegreeOfParallelism: 8))
    .EnableBackgroundProcessing()));

EnableParallelProcessing takes the degree of parallelism directly and defaults to 4. To drain sequentially, leave it off rather than passing 1.

Suggested profiles​

These are starting points, not a mechanism -- each is just the set of WithProcessing values below, which you type out for the profile you want. There is no preset argument to pass.

SettingHigh throughputBalancedHigh reliability
BatchSize100010010
PollingInterval100ms1s5s
MaxRetryCount3510
RetryDelay1 min5 min15 min
EnableParallelProcessing84omit (sequential)
ProfileUse case
High throughputReal-time event processing, high-volume systems
BalancedMost production workloads
High reliabilityFinancial transactions, critical systems

Balanced, spelled out:

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox => outbox
    .UseSqlServer(sql => sql.ConnectionString(connectionString))
    .WithProcessing(processing => processing
        .BatchSize(100)
        .PollingInterval(TimeSpan.FromSeconds(1))
        .MaxRetryCount(5)
        .RetryDelay(TimeSpan.FromMinutes(5))
        .EnableParallelProcessing(maxDegreeOfParallelism: 4))
    .EnableBackgroundProcessing()));

High reliability -- small batches, long backoff, sequential:

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox => outbox
    .UseSqlServer(sql => sql.ConnectionString(connectionString))
    .WithProcessing(processing => processing
        .BatchSize(10)
        .PollingInterval(TimeSpan.FromSeconds(5))
        .MaxRetryCount(10)
        .RetryDelay(TimeSpan.FromMinutes(15)))
    .EnableBackgroundProcessing()));

Usage in Handlers​

Inject IOutboxWriter into your handler and call WriteAsync to stage outbound messages. The consistency guarantee (eventually-consistent vs. transactional) is determined by configuration -- your handler code stays the same regardless of mode:

using Excalibur.Dispatch.Outbox;

public class CreateOrderHandler : IDispatchHandler<CreateOrderAction>
{
    private readonly IDbConnection _db;
    private readonly IOutboxWriter _outboxWriter;

    public CreateOrderHandler(IDbConnection db, IOutboxWriter outboxWriter)
    {
        _db = db;
        _outboxWriter = outboxWriter;
    }

    public async Task<IMessageResult> HandleAsync(
        CreateOrderAction action,
        IMessageContext context,
        CancellationToken ct)
    {
        using var transaction = _db.BeginTransaction();
        context.SetItem("Transaction", transaction);

        // Save domain changes
        var orderId = Guid.NewGuid();
        await _db.ExecuteAsync(
            "INSERT INTO Orders (Id, CustomerId) VALUES (@Id, @CustomerId)",
            new { Id = orderId, action.CustomerId },
            transaction);

        // Write to outbox -- behavior depends on configured ConsistencyMode
        await _outboxWriter.WriteAsync(
            new OrderCreatedEvent(orderId, action.CustomerId),
            destination: "orders",
            ct);

        transaction.Commit();
        return MessageResult.Success();
    }
}

For scheduled delivery, use the WriteScheduledAsync extension method:

await _outboxWriter.WriteScheduledAsync(
    new ReminderEvent(orderId),
    destination: "reminders",
    scheduledAt: DateTimeOffset.UtcNow.AddHours(24),
    ct);

Consistency Modes​

Configure the outbox consistency mode via OutboxStagingOptions:

services.AddDispatch(dispatch =>
{
    dispatch.UseOutbox(outbox =>
    {
        // Default: startup requires an IOutboxStore, because UseOutbox() states that this host
        // stages. It adds no requirement on the transaction middleware.
        outbox.ConsistencyMode = OutboxConsistencyMode.EventuallyConsistent;

        // OR: additionally refuse to start unless the transaction middleware is registered too.
        outbox.ConsistencyMode = OutboxConsistencyMode.Transactional;
    });
});
ConsistencyMode is a startup requirement, not a write-path selector

Calling UseOutbox() at all makes startup fail unless an IOutboxStore is registered — the call is how you state that this host stages, so a missing store is a misconfiguration rather than a host without an outbox. Setting ConsistencyMode to Transactional adds the transaction middleware to that requirement. Neither setting chooses how writes are staged, and a configuration that starts successfully is not thereby transactional.

For event-sourced aggregates the write path is selected by the staging strategy on the event-sourcing builder, which prefers the transactional path only when a transactional outbox writer and an event store that supports transactional staging are both present, and falls back to the eventually-consistent path otherwise. That check lives in the event-sourcing packages, so ConsistencyMode cannot perform it — it can only refuse to start without the parts the dispatch side can see.

If you need transactional staging, configure the staging strategy where your aggregates are registered and verify the path your store actually takes; ConsistencyMode alone will not give it to you.

ModeBehaviorRiskRequires
EventuallyConsistent (default)Messages buffered during handler execution, flushed to outbox after handler + transaction completeMessages lost if process crashes between commit and flushNothing extra
TransactionalMessages written to IOutboxStore within the ambient transactionNone (atomic with business data)IOutboxStore registration + TransactionMiddleware

Outbox Stores​

SQL Server​

services.AddSqlServerOutboxStore(options =>
{
    options.ConnectionString = connectionString;
    options.SchemaName = "outbox";
    options.OutboxTableName = "OutboxMessages";
});

PostgreSQL​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UsePostgres(postgres =>
    {
        postgres.ConnectionString(connectionString)
                .SchemaName("outbox")
                .TableName("outbox_messages");
    });
}));

Redis​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseRedis(redis =>
    {
        redis.ConnectionString("localhost:6379")
             .KeyPrefix("outbox:");
    });
}));

// Or with an existing ConnectionMultiplexer from DI
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseRedis(redis =>
    {
        redis.Multiplexer(existingMultiplexer)
             .KeyPrefix("outbox:");
    });
}));

MongoDB​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseMongoDB(mongo =>
    {
        mongo.ConnectionString(connectionString)
             .DatabaseName("myapp");
    });
}));

Elasticsearch​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseElasticSearch(options =>
    {
        options.IndexName = "excalibur-outbox";
        options.DefaultBatchSize = 100;
    });
}));

Firestore​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseFirestore(options =>
    {
        options.ProjectId = "my-gcp-project";
        options.CollectionName = "outbox";
    });
}));

Cosmos DB​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseCosmosDb(cosmos =>
    {
        cosmos.ConnectionString(connectionString)
              .DatabaseName("myapp")
              .ContainerName("outbox");
    });
}));

DynamoDB​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseDynamoDb(options =>
    {
        options.Connection.Region = "us-east-1";
        options.TableName = "outbox";
    });
}));

Database Schema​

The MessageType column holds the event's declared [MessageName] — the same stable name the event store records and external subscribers see as the CloudEvents type. It is what your consumers route on, and it does not change when the CLR type moves. See stable message names.

SQL Server​

The SQL Server store does not auto-create tables — create the schema before starting the application. The IX_OutboxMessages_Claim index backs the atomic claim predicate (status + retry-visibility) and the partition-ordered delivery guarantee.

Excalibur.Outbox.SqlServer packs its DDL as numbered scripts under scripts/ in the NuGet package. Those files are the authority for this schema — the store's own statements are derived from them.

ScriptWhat it does
001_CreateOutboxSchema.sqlCreates OutboxMessages, OutboxMessageTransports, DeadLetterQueue and OutboxFence. Also carries a guarded additive step that converges a legacy table whose TenantId is still nullable.
# The scripts sit alongside lib/ in the package folder your restore populated.
ls ~/.nuget/packages/excalibur.outbox.sqlserver/<version>/scripts/
Provision from the scripts, not from a transcription of them

This page used to restate these CREATE TABLE statements inline. A restatement drifts from the store silently and carries no upgrade path; the packaged scripts are the single copy kept in step with the code, and the only copy that is numbered. Provision from them rather than from any restatement.

A database provisioned by an earlier prerelease

TenantId is declared NVARCHAR(64), and a create script skips any table that already exists, so a database provisioned by an earlier prerelease keeps whatever width it was given. There is no in-place upgrade path — re-provision it from the script above. Two tenant identifiers that differ only after the 64th character are distinct in a wider column and identical in this one, and TenantId is part of the dead-letter primary key.

Ordering and retry-backoff columns

PartitionKey / GroupKey / SequenceNumber persist the message ordering keys, and NextAttemptAt records the per-message backoff deadline. The background processor claims rows with WHERE Status IN (Staged, Failed, PartiallyFailed) AND (NextAttemptAt IS NULL OR NextAttemptAt <= @now) ORDER BY PartitionKey, SequenceNumber — so same-partition messages are delivered in ascending SequenceNumber, and a failed message's computed backoff genuinely throttles re-delivery. See Ordering and Retry Scheduling.

If you use multi-transport delivery (IsMultiTransport / per-transport tracking) or the dead-letter queue, create those tables too — the store does not auto-create them either:

Both are created by the same 001_CreateOutboxSchema.sql — there is no separate script to run.

Upgrading an existing outbox table​

If your OutboxMessages table was created while TenantId was nullable, tighten it before deploying. A nullable tenant lets an un-tenanted row exist, and no tenant-scoped read can ever match it — the row becomes unreachable rather than shared. Existing NULLs are backfilled to the reserved untenanted value first, so the change preserves every row and cannot fail on legacy data:

Run 001_CreateOutboxSchema.sql. Its tightening step is additive and guarded on is_nullable, so it converges a legacy table and does nothing to an already-correct one.

Run this before starting the upgraded application. The framework never writes NULL — a message staged with no tenant in scope carries the reserved __untenanted__ value — so the backfill and the application agree on what an untenanted row looks like.

Do not test a drained message for a null tenant

A message you stage with TenantId = null is drained carrying the reserved __untenanted__ value, on every store — the document and key-value providers as well as the relational ones. Untenanted is a value, not an absence, so a handler written as:

if (message.TenantId is null) { /* untenanted */ }   // never true after a drain

silently never takes that branch. Compare against the reserved value, or fold through the framework's total conversion, which accepts every spelling:

var partition = KeyedTenantPartition.FromStoredValue(message.TenantId);
if (partition.Equals(KeyedTenantPartition.Untenanted)) { /* untenanted */ }

If you implement your own outbox store, the published OutboxStoreConformanceTestKit enforces this: its UntenantedPartition_MustRoundTripItsOwnMessage arm fails a store that hands back a null as well as one that invents a tenant.

PostgreSQL​

The PostgreSQL store does not auto-create tables — create the schema before starting the application. The tenant_id column persists tenant isolation and the destination column persists the delivery target, both through enqueue → reserve → dispatch; staged messages fail with column "tenant_id" does not exist or column "destination" does not exist if either is missing.

-- Object names follow the PostgresOutboxStoreOptions defaults (SchemaName "public",
-- OutboxTableName "outbox", DeadLetterTableName "outbox_dead_letters"). If you override any of
-- them, rename the corresponding object here to match.
CREATE TABLE IF NOT EXISTS public.outbox (
    -- The message id IS the primary key. Every read, claim, delete and dead-letter path
    -- addresses a row by it, and no query selects a surrogate, so there is no id column.
    message_id          VARCHAR(100)  NOT NULL PRIMARY KEY,
    message_type        VARCHAR(500)  NOT NULL,
    message_metadata    TEXT,
    message_body        BYTEA         NOT NULL,
    -- Every row carries a tenant term; "no tenant" is the reserved '__untenanted__' value rather
    -- than the absence of one, so a scoped predicate and an unscoped one compare the same kind of
    -- thing. The staging path binds the term explicitly, so the DEFAULT is a backstop for
    -- hand-written INSERTs rather than something the store relies on.
    tenant_id           VARCHAR(64)   NOT NULL DEFAULT '__untenanted__',
    destination         VARCHAR(500),
    correlation_id      VARCHAR(255),
    causation_id        VARCHAR(255),
    priority            INT           NOT NULL DEFAULT 0,
    scheduled_at        TIMESTAMPTZ,
    partition_key       VARCHAR(255),
    group_key           VARCHAR(255),
    sequence_number     BIGINT        NOT NULL DEFAULT 0,
    target_transports   VARCHAR(500),
    is_multi_transport  BOOLEAN       NOT NULL DEFAULT FALSE,
    -- The caller's created-at, not the server clock: the staging path binds it explicitly so the
    -- value survives the stage-then-reload round trip.
    occurred_on         TIMESTAMPTZ   NOT NULL DEFAULT NOW(),
    attempts            INT           NOT NULL DEFAULT 0,
    error_message       TEXT,
    -- The claim token written by a drain that has reserved the row, and the instant that
    -- reservation lapses. Together they form the reservation lease.
    dispatcher_id       VARCHAR(100),
    dispatcher_timeout  TIMESTAMPTZ,
    -- The failure-anchored visibility floor. A message reported failed below the retry ceiling is
    -- deferred until this instant, so a failure cannot hot-loop the drain.
    next_attempt_at     TIMESTAMPTZ
);

-- All four timestamp columns are TIMESTAMPTZ, not TIMESTAMP. The store binds them as true
-- timestamptz values; a column created WITHOUT time zone is silently shifted by the session
-- timezone on reload, so a scheduled message becomes due at the wrong instant.

-- Supports the claim cursor's ordering. The drain selects eligible rows ordered by partition_key,
-- sequence_number, occurred_on, which is what preserves per-partition ordering across concurrent
-- drains. Do not narrow this index to a partial one: the drain's ordering needs every row.
CREATE INDEX IF NOT EXISTS ix_outbox_claim_order ON public.outbox (partition_key, sequence_number, occurred_on);

-- Supports releasing a reservation by dispatcher, which matches on dispatcher_id alone.
CREATE INDEX IF NOT EXISTS ix_outbox_dispatcher ON public.outbox (dispatcher_id);

-- Supports the scheduled-message sweep, which selects rows whose scheduled_at has come due.
CREATE INDEX IF NOT EXISTS ix_outbox_scheduled_at ON public.outbox (scheduled_at);

-- The move into this table DELETEs the outbox row, so this is the ONLY remaining record of the
-- message. Anything the move does not copy is destroyed, not merely unindexed.
CREATE TABLE IF NOT EXISTS public.outbox_dead_letters (
    message_id          VARCHAR(100)  NOT NULL,
    -- Originating tenant, carried as provenance so a replay re-enters the SAME tenant. NOT NULL and
    -- a component of the primary key. No DEFAULT, deliberately -- and this is the one place it
    -- differs from the outbox table. The move always copies the value from the outbox row, whose
    -- own column is total, so the value is always supplied. A default here would let a
    -- hand-written INSERT that forgot the column record a message as untenanted when its tenant was
    -- simply never named. Without one, that INSERT fails loudly.
    tenant_id           VARCHAR(64)   NOT NULL,
    message_type        VARCHAR(500)  NOT NULL,
    message_metadata    TEXT,
    message_body        BYTEA         NOT NULL,
    occurred_on         TIMESTAMPTZ   NOT NULL DEFAULT NOW(),
    attempts            INT           NOT NULL DEFAULT 0,
    error_message       TEXT,
    moved_on            TIMESTAMPTZ   NOT NULL DEFAULT NOW(),
    -- Composite on purpose, and NOT a pattern to copy onto the outbox table. The outbox is drained
    -- by a claim-then-mark protocol that addresses a row by its id alone, so widening that key
    -- there is a correctness defect. Nothing addresses a dead letter by id: the read path pages by
    -- age and attempts, and the statistics path counts.
    CONSTRAINT pk_outbox_dead_letters PRIMARY KEY (message_id, tenant_id)
);

-- Supports the dead-letter read path, which pages by occurred_on ascending.
CREATE INDEX IF NOT EXISTS ix_outbox_dead_letters_occurred_on ON public.outbox_dead_letters (occurred_on);
Run the shipped script instead

Excalibur.Outbox.Postgres/Scripts/001_CreateOutboxSchema.sql is the authority for this schema and is reproduced above. It is packed inside the NuGet package under scripts/.

It provisions and it upgrades. The table and index statements are CREATE ... IF NOT EXISTS, so they do nothing against objects that already exist; the script then runs a guarded ALTER TABLE ... ADD COLUMN IF NOT EXISTS for every outbox column, so a table created by an earlier revision gains whichever columns it is missing. Run the packaged script rather than hand-writing the DDL.

Two columns are deliberately left out of the additive step. message_type and message_body are NOT NULL with no default, so adding them to a table that already has rows would fail outright, and both have existed since this table's first revision — a table missing either is not an earlier outbox but a name collision with something else, and failing loudly is the right outcome.

If your table was created from an earlier revision of this page it has a SERIAL id column and a UNIQUE constraint on message_id instead of a primary key on it. That shape still works -- the store addresses rows by message_id, which is unique either way -- but the surrogate key is never read. A tenant_id declared VARCHAR(255) is also wider than the store will ever write. A database provisioned by an earlier prerelease has no in-place upgrade path onto the current shape; re-provision it from the packaged script.

Leader-elected (fenced) deployments need a fence control table

If you run the Postgres outbox single-active across instances (a leader-elected processor passing a fencing token), also create the fence control table. It holds one monotonic high-water mark per scope so a demoted leader's stale-token drain/mark is rejected — the guard that prevents a demoted leader from double-delivering (delivery remains at-least-once; this closes the split-brain window, it does not make it exactly-once). It is not needed for a single-instance outbox.

CREATE TABLE IF NOT EXISTS public.outbox_fence (
    scope_key           VARCHAR(600)  NOT NULL PRIMARY KEY,
    high_water_token    BIGINT        NOT NULL
);

The table name defaults to outbox_fence (override via PostgresOutboxStoreOptions.FenceTableName, qualified by SchemaName). The Oracle store uses an equivalent control table named via OracleOutboxStoreOptions.FenceTableName.

The SQL Server outbox uses a durable OutboxFence control table. Like every other table this store uses, it is not created at runtime — the store auto-creates nothing. The packaged schema script (001_CreateOutboxSchema.sql, shipped inside the NuGet package) creates it for you if you run that script; if you instead copy the DDL from the SQL Server schema section above, OutboxFence is included in that block and you must create it along with the others. It is required even for a single-instance deployment that never uses fencing, because the drain references it unconditionally and SQL Server binds object names before the runtime predicate can short-circuit. It holds one monotonic high-water mark per scope that outbox cleanup never touches, so a demoted leader's stale-token drain/mark is rejected even after the outbox has been cleaned up or drained empty. The table name defaults to OutboxFence (override via SqlServerOutboxOptions.Tables.FenceTableName, qualified by Tables.SchemaName). See Multi-Instance (Leader-Fenced) Processing.

A database provisioned by an earlier prerelease

There is no in-place upgrade path onto the current shape; re-provision from the packaged 001_CreateOutboxSchema.sql. tenant_id arrives NOT NULL DEFAULT '__untenanted__', so an untenanted message is stored as a value rather than as the absence of one — which is what lets a tenant predicate compare like with like.

If you run a leader-elected (fenced) outbox, add the outbox_fence control table shown above as well.

A dead-letter table provisioned by an earlier prerelease cannot be carried forward

outbox_dead_letters carries the producing tenant, and the move that fills it deletes the outbox row it copied from — so for an entry already sitting in a table that predates the tenant column, the tenant that produced it exists nowhere the store can reach. There is no in-place upgrade path that could recover it. Re-provision from the packaged 001_CreateOutboxSchema.sql, and do not redrive an entry carried over by hand: stamping it untenanted records "no tenant was captured" rather than evidence the message was untenanted, and a message that really belonged to a tenant would re-enter the wrong one.

If those entries matter to you, drain, redrive, or export them before upgrading, while their tenant is still implied by your own operational records.

Background Processing​

Hosted Service (Default)​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))
          .EnableBackgroundProcessing();
}));

EnableBackgroundProcessing() registers the outbox background service for you. If you prefer to register it as a separate step, services.AddOutboxHostedService() registers the same service.

Quartz Job (Scheduled Processing)​

For enterprise scheduling needs, use OutboxProcessorJob from Excalibur.Jobs:

// Install: dotnet add package Excalibur.Jobs

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))));

// Register the Quartz.NET outbox processor job
// Configure schedule in appsettings.json or via Quartz API

The OutboxProcessorJob integrates with Quartz.NET for scheduled outbox processing with built-in health checks and multi-database support.

Manual Processing​

For serverless environments (Azure Functions, AWS Lambda):

// Serverless: leave EnableBackgroundProcessing() off so no polling loop is registered,
// and drive the processor from the platform's own trigger instead.
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))
          .WithProcessing(processing => processing
              .BatchSize(50)
              .MaxRetryCount(3));
}));

// Process manually (e.g., Azure Function timer trigger)
public class OutboxProcessorFunction
{
    private readonly IOutboxProcessor _processor;

    [Function("ProcessOutbox")]
    public async Task Run([TimerTrigger("*/5 * * * * *")] TimerInfo timer)
    {
        await _processor.DispatchPendingMessagesAsync(CancellationToken.None);
    }
}

Multi-Instance (Leader-Fenced) Processing​

When more than one instance can drain the same outbox, register an ILeaderElection provider. Doing so makes the outbox drain single-active (leader-fenced) by default — only the elected leader drains, and every claim/mark carries a monotonic fencing token, checked against the store's recorded fencing high-water mark, so a superseded leader's claim or mark is rejected.

PostgreSQL, Oracle, MongoDB, and SQL Server record the high-water mark in a dedicated fence control table that outbox cleanup never touches, so the rejection survives cleanup — closing the split-brain window that would otherwise double-deliver under a leader handover. (Delivery is at-least-once either way; this removes a specific split-brain duplicate source.) SQL Server's OutboxFence table is advanced by a single serializable MERGE (a compare-and-advance under HOLDLOCK) that raises the mark monotonically and never lowers it, so a superseded leader's stale token is rejected fail-closed even after a cleanup has purged the sent, token-bearing rows. The per-message lease independently prevents two processors from claiming or completing the same message.

services.AddExcalibur(exc => exc
    .AddOutbox(outbox => outbox.UsePostgres(pg => pg.ConnectionString(connectionString)))
    // Registering a leader election is the multi-instance signal — the outbox drain is fenced automatically.
    .AddLeaderElection(le => le.UseSqlServer(sql => sql
        .ConnectionString(leaderConnectionString)
        .LockResource("MyApp.Outbox"))));

Registering leader election is the multi-instance signal, so no extra opt-in is required to fence the drain. (For a fencing-capable store you can still call outbox.WithLeaderElection() explicitly; it composes idempotently with the automatic wiring.)

Fail-fast on a store that cannot fence. If a leader election is registered and the configured store does not implement IFencedOutboxStore — so it cannot enforce a fencing high-water mark — the processor refuses to start, throwing an InvalidOperationException at startup rather than draining unfenced. Fencing-capable first-party stores are SQL Server, PostgreSQL, Oracle, MongoDB, and the in-memory store — all record the leadership high-water mark in a durable fence control table that survives outbox cleanup. Some stores (for example Elasticsearch) cannot express an atomic fencing high-water mark with their native primitives and therefore always require the opt-out below under a leader election.

Fail-fast on an ungated drain. The check above keys on the presence of the leader election, not on the presence of the gate that fences the drain — a guard that keyed on the gate could not detect the case it exists to detect, because a host with no gate would simply read as single-instance. So if a leader election is resolvable and nothing fences the drain, startup refuses with an InvalidOperationException naming both remedies, rather than letting every instance drain concurrently. You will normally never see this: the first-party registration paths wire the gate for you. It fires when an election reaches the container by some other route — a hand-registered ILeaderElection, or a provider from outside the framework. Wire the gate with outbox.WithLeaderElection(), or declare a genuine single-drainer topology with AsSingleWriter().

Opting out — single-active-writer topology. If exactly one process drains the outbox even though a leader election is registered for other resources (leases, scheduled jobs), assert that topology explicitly with AsSingleWriter():

services.AddExcalibur(exc => exc
    .AddOutbox(outbox => outbox
        .UseElasticSearch(es => es.NodeUri("http://localhost:9200"))
        .AsSingleWriter())   // "I am the single active writer" - fences out a demoted leader.
                         // Delivery is still at-least-once; consumers must be idempotent.
    .AddLeaderElection(le => le.UseSqlServer(sql => sql
        .ConnectionString(leaderConnectionString)
        .LockResource("MyApp.Leader"))));

AsSingleWriter() runs the drain unfenced and is deliberately observable: the outbox logs a startup warning (OutboxUnfencedBySingleWriterOptOut). Enable it only for a genuine single-active-writer deployment — using it in a multi-writer deployment reopens the split-brain window fencing exists to close. When no leader election is registered at all, the outbox runs single-instance and unfenced by design, emitting a one-time startup info log (OutboxRunningUnfenced).

Publisher Configuration​

Default Publisher​

The outbox uses the configured IOutboxPublisher to send messages. The default behavior dispatches through the registered message bus:

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))));

// Messages are dispatched through IDispatcher by default

Transport-Specific Publisher​

services.AddDispatch(dispatch =>
{
    dispatch.AddHandlersFromAssembly(typeof(Program).Assembly);
    dispatch.UseKafka(kafka =>
    {
        kafka.BootstrapServers("localhost:9092");
        kafka.DefaultTopic("dispatch.events");
    });
});

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))));

// Register Kafka publisher for outbox
services.AddSingleton<IOutboxPublisher, KafkaOutboxPublisher>();

Custom Publisher​

Implement IOutboxPublisher for custom message publishing:

public class WebhookOutboxPublisher : IOutboxPublisher
{
    private readonly HttpClient _httpClient;
    private readonly IOutboxStore _store;
    private int _publishedCount;
    private int _failedCount;

    public WebhookOutboxPublisher(HttpClient httpClient, IOutboxStore store)
    {
        _httpClient = httpClient;
        _store = store;
    }

    public async Task<OutboundMessage> PublishAsync(
        object message,
        string destination,
        DateTimeOffset? scheduledAt,
        CancellationToken cancellationToken)
    {
        // Create and stage outbound message
        var payload = JsonSerializer.SerializeToUtf8Bytes(message);
        var outbound = new OutboundMessage(
            message.GetType().Name,
            payload,
            destination) { ScheduledAt = scheduledAt };

        await _store.StageMessageAsync(outbound, cancellationToken);
        return outbound;
    }

    public async Task<PublishingResult> PublishPendingMessagesAsync(
        CancellationToken cancellationToken)
    {
        var messages = await _store.GetUnsentMessagesAsync(100, cancellationToken);
        var published = 0;
        var failed = 0;

        foreach (var message in messages)
        {
            try
            {
                await _httpClient.PostAsync(
                    $"/webhooks/{message.Destination}",
                    new ByteArrayContent(message.Payload),
                    cancellationToken);

                await _store.MarkSentAsync(message.Id, cancellationToken);
                published++;
            }
            catch (Exception ex)
            {
                await _store.MarkFailedAsync(message.Id, ex.Message, 1, cancellationToken);
                failed++;
            }
        }

        Interlocked.Add(ref _publishedCount, published);
        Interlocked.Add(ref _failedCount, failed);

        return new PublishingResult { SuccessCount = published, FailureCount = failed };
    }

    // Implement other required methods...
}

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))));
services.AddSingleton<IOutboxPublisher, WebhookOutboxPublisher>();

Error Handling​

Retry Configuration​

// Aggressive retries for critical workloads: 10 attempts, 15 minutes apart.
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))
          .WithProcessing(processing => processing
              .MaxRetryCount(10)
              .RetryDelay(TimeSpan.FromMinutes(15)));
}));

// Or a lighter policy: 7 attempts, 2 minutes apart.
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))
          .WithProcessing(processing => processing
              .MaxRetryCount(7)
              .RetryDelay(TimeSpan.FromMinutes(2)));
}));

Dead Letter Handling​

services.AddSqlServerOutboxStore(options => options.ConnectionString = connectionString);

// The dead-letter table is configured on the dead-letter queue itself, not on the
// outbox store. Its default table is DeadLetterQueue, which is the table the
// packaged schema script creates.
services.AddSqlServerDeadLetterQueue(opts =>
{
    opts.ConnectionString = connectionString;
    opts.TableName = "DeadLetterQueue";
});

Ordering and Retry Scheduling​

Partition-Ordered Delivery​

Each outbound message carries three ordering fields — PartitionKey, GroupKey, and a monotonically increasing SequenceNumber. The polling claim selects eligible rows in (PartitionKey, SequenceNumber) order, so messages that share a PartitionKey are delivered in ascending SequenceNumber (per-partition FIFO). Messages without a PartitionKey have no cross-message ordering guarantee. GroupKey is an independent label for logical grouping and does not affect claim order.

This is message-level ordering persisted on each row. It is distinct from Partitioned Outbox processing, which shards the processor loops for throughput.

Retry Backoff Is Applied​

When delivery fails, the processor computes an exponential backoff delay and records the absolute next-attempt time on the row's NextAttemptAt column. The claim predicate excludes the message until that time elapses (NextAttemptAt IS NULL OR NextAttemptAt <= @now), so the configured retry delay genuinely throttles re-delivery rather than re-claiming the message as soon as its lease expires.

A circuit-breaker-open short-circuit is treated differently: because no delivery was actually attempted, no backoff is applied — the message stays immediately eligible and retries as soon as the breaker closes.

Backoff scheduling requires a store that implements the optional IBackoffSchedulableOutboxStore capability (MarkFailedWithBackoffAsync). The SQL Server and PostgreSQL stores implement it (the Postgres implementation is signature-identical to SQL Server for cross-provider consistency). Other providers (Redis, MongoDB, Elasticsearch, DynamoDB, Cosmos DB) do not yet implement it and are unaffected: the processor falls back to the plain MarkFailedAsync path (immediate re-eligibility), so no store is broken — matching the fail-open pattern used by IDeadLetterableOutboxStore. The capability is forwarded transparently through the telemetry and encrypting outbox-store decorators, so it survives a decorated store chain.

Administrative operations reach every tenant​

The operations on IOutboxStoreAdmin are estate-wide, and their names say so: GetAllTenantsStatisticsAsync, GetAllTenantsFailedMessagesAsync, GetAllTenantsScheduledMessagesAsync and CleanupAllTenantsSentMessagesAsync. IInboxStoreAdmin follows the same convention with GetAllTenantsEntriesAsync, GetAllTenantsStatisticsAsync and CleanupAllTenantsProcessedEntriesAsync.

Estate-wide operations here are reached by name, never by passing a wildcard tenant value. A name cannot be arrived at accidentally from a value that flowed in from somewhere else; a wildcard can. If you are calling one of these from a request path that holds a single tenant's context, that is worth a second look — the operation will not confine itself to that tenant.

Statistics is deliberately estate-wide, and could not be otherwise. The operation takes no tenant argument, the statistics type it returns carries no tenant field, and the store reads no ambient tenant context — so a confined result would have no way to say which partition it described. A report that reaches rows by no key can only lose rows by filtering; it cannot gain another tenant's. Adding a tenant predicate would therefore not tighten it, it would silently under-report the backlog. Treat these as operator reports, and do not surface them to a tenant as though they described that tenant alone.

Cleanup​

What removes a sent message​

No preset schedules cleanup, and the framework runs no cleanup service for the outbox. What happens to a sent message depends entirely on the provider you registered:

  • SQL Server, PostgreSQL, Oracle, Marten, in-memory — nothing removes it. Run the cleanup operation yourself, as shown below.
  • Redis, MongoDB, Cosmos DB — the store expires sent messages natively, on by default (7 days).
  • DynamoDB — expires on the default path: with CreateTableIfNotExists left true, the store creates the table and enables TTL on it. If you provision the table yourself, enable TTL or nothing is deleted.
  • Firestore — writes an expireAt field but never creates the TTL policy. Nothing is deleted until you configure that policy on the field yourself.
  • Elasticsearch — its retention setting is not currently applied; treat it as unbounded.

Every provider exposes a cleanup operation you can call, including the ones that also expire natively. What differs is only whether anything happens if you never call it.

See Retention and cleanup for the two-property table.

Manual Cleanup​

public class OutboxCleanupJob
{
    private readonly IOutboxStore _store;

    public async Task CleanupAsync(CancellationToken ct)
    {
        var cutoff = DateTimeOffset.UtcNow.AddDays(-7);
        var deleted = await _store.CleanupAllTenantsSentMessagesAsync(cutoff, batchSize: 1000, ct);
        _logger.LogInformation("Deleted {Count} processed messages", deleted);
    }
}

Monitoring​

Health Checks​

services.AddHealthChecks()
    .AddOutboxHealthCheck(options =>
    {
        options.UnhealthyInactivityTimeout = TimeSpan.FromMinutes(5);
        options.DegradedInactivityTimeout = TimeSpan.FromMinutes(2);
        options.UnhealthyFailureRatePercent = 20.0;
        options.DegradedFailureRatePercent = 5.0;
    });

Metrics​

Outbox metrics are included in the core Dispatch metrics:

services.AddOpenTelemetry()
    .WithMetrics(metrics =>
    {
        metrics.AddDispatchMetrics();
        // Includes outbox-related metrics:
        // - dispatch.messages.processed
        // - dispatch.messages.published
        // - dispatch.messages.failed
        // - dispatch.messages.duration
    });

Validation Rules​

The preset-based API validates configuration at build time:

RuleError Message
BatchSize >= 1"BatchSize must be at least 1."
BatchSize <= 10000"BatchSize cannot exceed 10000."
PollingInterval >= 10ms"PollingInterval must be at least 10ms."
MaxRetryCount >= 0"MaxRetryCount cannot be negative."
MaxDegreeOfParallelism >= 1"MaxDegreeOfParallelism must be at least 1."
RetryDelay > 0"RetryDelay must be positive."
ProcessorId not empty"ProcessorId cannot be null or whitespace."

Best Practices​

PracticeRecommendation
Use presetsStart with Balanced, adjust only if needed
Transaction scopeKeep outbox add in same transaction as domain changes
Batch sizeUse preset defaults (HighThroughput: 1000, Balanced: 100, HighReliability: 10)
Processing intervalUse preset defaults; 100ms for real-time, 1-5s for standard
Retention7 days for most workloads, 30 days for compliance
MonitoringAlert on high pending count or age
Preset selectionHighReliability for financial, Balanced for most, HighThroughput for event streaming

Troubleshooting​

Messages Not Processing​

-- Check unprocessed messages
SELECT TOP 100 *
FROM [outbox].[OutboxMessages]
WHERE [ProcessedAt] IS NULL
ORDER BY [CreatedAt];

-- Check failed messages
SELECT *
FROM [outbox].[OutboxMessages]
WHERE [Error] IS NOT NULL;

High Latency​

  • Increase batch size
  • Reduce processing interval
  • Add database indexes
  • Scale out processors (with locking)

Event Sourcing Outbox Integration​

When using event sourcing, integration events can be staged to the unified outbox automatically during aggregate save. The EventSourcedRepository supports three staging strategies controlled by OutboxStagingStrategy:

Staging Strategies​

StrategyBehaviorTrade-off
Auto (default)Framework selects the best available strategyNo configuration needed
TransactionalStages events in the same DB transaction as the event appendZero message loss, adds save latency
EventuallyConsistentStages events after a successful append in a separate callMinimal latency, tiny loss window on crash
DeferredNo staging during save; a background service picks up events laterZero added latency, higher delivery delay

Configuration​

services.AddExcalibur(excalibur => excalibur.AddEventSourcing(es =>
{
    es.UseSqlServer(sql => sql.ConnectionString(connectionString));

    // Per-aggregate staging strategy
    es.AddRepository<Order>(id => new Order(id), opts =>
    {
        opts.OutboxStagingStrategy = OutboxStagingStrategy.Transactional;
    });
}));

// Register the unified outbox store (required for Transactional and EventuallyConsistent)
services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString))));

How Auto Resolution Works​

When OutboxStagingStrategy.Auto is configured (the default), the repository checks at save time:

  1. If an ITransactionalEventStore (a transactional event store) and an ITransactionalOutboxWriter are registered, uses Transactional
  2. If only IOutboxStore is registered, uses EventuallyConsistent
  3. If neither is registered, uses Deferred (no staging)

Selecting OutboxStagingStrategy.Transactional explicitly (rather than Auto) without both pieces of infrastructure fails fast at startup via a ValidateOnStart guard, naming exactly what is missing — it never silently degrades to non-atomic staging.

ITransactionalEventStore​

The event-store side of the atomic path. An event store provider backed by a transactional database (SQL Server, PostgreSQL) implements the optional ITransactionalEventStore extension of IEventStore to enable the Transactional strategy:

namespace Excalibur.EventSourcing;

public interface ITransactionalEventStore : IEventStore
{
    ValueTask<AppendResult> AppendWithOutboxStagingAsync(
        string aggregateId,
        string aggregateType,
        IEnumerable<IDomainEvent> events,
        long expectedVersion,
        Func<IDbTransaction, CancellationToken, ValueTask> stageOutbox,
        CancellationToken cancellationToken);
}

This is a store-owned unit of work: the store opens and owns a single connection and transaction, runs the optimistic-concurrency version check, appends the events, invokes your stageOutbox callback on that same transaction (only when the version check succeeds), then commits. On a concurrency conflict or any throw from stageOutbox, the whole transaction rolls back, so neither the events nor the outbox rows persist. Because the transaction never leaves the store, appending events and staging outbox rows on two different transactions is structurally impossible.

SqlServerEventStore implements ITransactionalEventStore. You do not call AppendWithOutboxStagingAsync directly — EventSourcedRepository invokes it on your behalf when the resolved strategy is Transactional, supplying a stageOutbox callback that enlists each integration event's outbox write through ITransactionalOutboxWriter on the store's transaction. NoSQL event stores do not implement this interface; use EventuallyConsistent or Deferred staging with them.

ITransactionalOutboxWriter​

Relational outbox providers (SQL Server, PostgreSQL) implement ITransactionalOutboxWriter to stage outbox rows on the event store's database transaction (the stageOutbox callback above calls into it):

public interface ITransactionalOutboxWriter
{
    ValueTask StageMessageAsync(
        OutboundMessage message,
        IDbTransaction transaction,
        CancellationToken cancellationToken);
}

NoSQL providers (CosmosDB, DynamoDB, MongoDB, etc.) do not implement this interface. Use EventuallyConsistent or Deferred staging with NoSQL event stores.

Standard Header Names​

The OutboxHeaderNames class provides well-known constants used in outbox message headers and event metadata:

ConstantValuePurpose
AggregateId"aggregate-id"Aggregate that produced the event
AggregateType"aggregate-type"Aggregate type name
TenantId"tenant-id"Multi-tenant routing
CorrelationId"correlation-id"Distributed tracing
CausationId"causation-id"Cause-effect linking

Partitioned Outbox​

At high event rates (100K+ events/sec), the single outbox table becomes a contention bottleneck. Partitioned outbox splits processing into multiple independent loops, each handling a subset of messages.

Enable Partitioned Processing​

services.AddExcalibur(excalibur => excalibur.AddOutbox(outbox =>
{
    outbox.UseSqlServer(sql => sql.ConnectionString(connectionString));

    outbox.UsePartitionedProcessing(opts =>
    {
        opts.Strategy = OutboxPartitionStrategy.ByTenantHash;
        opts.PartitionCount = 8;
        opts.ProcessorCountPerPartition = 1;
        opts.PollingInterval = TimeSpan.FromSeconds(1);
        opts.ErrorBackoffInterval = TimeSpan.FromSeconds(5);
    });
}));

Partitioning Strategies​

StrategyDescriptionUse Case
NoneSingle processor loop (default)Low-to-moderate throughput
PerShardOne partition per tenant shardWhen tenant sharding is active
ByTenantHashXxHash32(tenantId) % N partitionsHigh throughput without sharding infrastructure

How It Works​

Each partition runs an independent processor loop with its own error isolation. When ProcessorCountPerPartition > 1, multiple concurrent processors handle the same partition.

Configuration Options​

OptionDefaultDescription
StrategyNonePartitioning strategy
PartitionCount8Number of partitions (for ByTenantHash)
ProcessorCountPerPartition1Concurrent processor instances per partition
PollingInterval1sDelay when no messages are available
ErrorBackoffInterval5sDelay after a processing error
ShardIds[]Required shard IDs when Strategy is PerShard

Custom Partitioner​

Implement IOutboxPartitioner for custom routing logic:

public interface IOutboxPartitioner
{
    int GetPartition(string tenantId);
    int PartitionCount { get; }
}

Design Principles​

PrincipleDescription
Preset-based APIHighThroughput(), Balanced(), HighReliability(), Custom() factory methods
Immutable optionsOutboxOptions created via fluent IOutboxOptionsBuilder
Override supportPresets provide opinionated defaults; .With*() methods allow surgical overrides
Fail-fast validationValidation at Build() time, not at registration
API consistencyParallel InboxOptions presets for consistent experience

Next Steps​

See Also​