Table of Contents

Class DynamoDbOutboxStore

Namespace
Excalibur.Outbox.DynamoDb
Assembly
Excalibur.Outbox.DynamoDb.dll

AWS DynamoDB implementation of the cloud-native outbox store.

[SuppressMessage("Maintainability", "CA1506:Avoid excessive class coupling", Justification = "Cloud outbox implementations inherently couple with many SDK and abstraction types.")]
public sealed class DynamoDbOutboxStore : ICloudNativeOutboxStore, ICloudNativeOutboxStoreBatch, ICloudNativeOutboxStoreClaim, IAsyncDisposable, ITenantPartitionedStore
Inheritance
DynamoDbOutboxStore
Implements
Inherited Members

Constructors

DynamoDbOutboxStore(IOptions<DynamoDbOutboxOptions>, ILogger<DynamoDbOutboxStore>)

Initializes a new instance of the DynamoDbOutboxStore class.

public DynamoDbOutboxStore(IOptions<DynamoDbOutboxOptions> options, ILogger<DynamoDbOutboxStore> logger)

Parameters

options IOptions<DynamoDbOutboxOptions>

The DynamoDB outbox options.

logger ILogger<DynamoDbOutboxStore>

The logger instance.

Properties

ProviderType

Gets the underlying cloud provider type.

public CloudPersistenceProviderType ProviderType { get; }

Property Value

CloudPersistenceProviderType

Methods

AddAsync(CloudOutboxMessage, IPartitionKey, CancellationToken)

Adds a message to the outbox within a transactional batch.

public Task<CloudOperationResult<CloudOutboxMessage>> AddAsync(CloudOutboxMessage message, IPartitionKey partitionKey, CancellationToken cancellationToken)

Parameters

message CloudOutboxMessage

The outbox message to add.

partitionKey IPartitionKey

The partition key for the message.

cancellationToken CancellationToken

Cancellation token.

Returns

Task<CloudOperationResult<CloudOutboxMessage>>

The operation result with cost information.

AddBatchAsync(IEnumerable<CloudOutboxMessage>, IPartitionKey, CancellationToken)

Adds multiple messages in a transactional batch.

public Task<CloudBatchResult> AddBatchAsync(IEnumerable<CloudOutboxMessage> messages, IPartitionKey partitionKey, CancellationToken cancellationToken)

Parameters

messages IEnumerable<CloudOutboxMessage>
partitionKey IPartitionKey
cancellationToken CancellationToken

Returns

Task<CloudBatchResult>

ClaimPendingAsync(IPartitionKey, int, string, CancellationToken)

Atomically claims up to batchSize unpublished messages from a partition, stamping each with a lease held by claimantId.

public Task<CloudQueryResult<CloudOutboxMessage>> ClaimPendingAsync(IPartitionKey partitionKey, int batchSize, string claimantId, CancellationToken cancellationToken)

Parameters

partitionKey IPartitionKey

The partition to claim from.

batchSize int

The maximum number of messages to claim.

claimantId string

Identifies the claiming process. Recorded on each claimed message as the lease owner, so an operator can see which process holds a message and a stalled claimant can be attributed.

cancellationToken CancellationToken

Cancellation token.

Returns

Task<CloudQueryResult<CloudOutboxMessage>>

The messages this call won, each carrying the lease it was stamped with. Losing a race is the normal outcome under concurrency and is not an error: the contested message is simply absent from the result. An empty result therefore means "nothing was available to me", never "something went wrong". A call SHOULD return the messages it wins in creation order where the provider can support it — the same FIFO guarantee GetPendingAsync(IPartitionKey, int, CancellationToken) documents, including the same latency (never loss) caveat for a provider that must order candidates via a separately-maintained, eventually-consistent index rather than a strongly-consistent query.

Remarks

The atomic step is UpdateItem under a ConditionExpression naming the lease. DynamoDB evaluates the condition and applies the update as one operation on a single item, so of two claimants issuing it against the same item exactly one succeeds and the other is refused with ConditionalCheckFailedException. That refusal is the mechanism working, not a fault.

The query that precedes it only nominates candidates — its filter is evaluated on a read and excludes nobody. An unconditional UpdateItem here would be silently wrong: with no condition to fail, both claimants would succeed and both would publish.

CleanupOldMessagesAsync(IPartitionKey, TimeSpan, CancellationToken)

Deletes published messages older than the retention period.

public Task<CloudCleanupResult> CleanupOldMessagesAsync(IPartitionKey partitionKey, TimeSpan retentionPeriod, CancellationToken cancellationToken)

Parameters

partitionKey IPartitionKey
retentionPeriod TimeSpan
cancellationToken CancellationToken

Returns

Task<CloudCleanupResult>

DisposeAsync()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources asynchronously.

public ValueTask DisposeAsync()

Returns

ValueTask

A task that represents the asynchronous dispose operation.

GetPendingAsync(IPartitionKey, int, CancellationToken)

Gets pending (unpublished) messages from a partition in FIFO order.

public Task<CloudQueryResult<CloudOutboxMessage>> GetPendingAsync(IPartitionKey partitionKey, int batchSize, CancellationToken cancellationToken)

Parameters

partitionKey IPartitionKey

The partition key to query.

batchSize int

Maximum number of messages to retrieve.

cancellationToken CancellationToken

Cancellation token.

Returns

Task<CloudQueryResult<CloudOutboxMessage>>

Pending messages with cost information.

Remarks

This is a query, not a claim. It modifies nothing: no lease is taken, no message is marked, no row is written. Two callers running it concurrently against the same partition therefore receive the same messages and both will publish them. It does not divide work between processes, and no amount of care at the call site makes it do so.

Do not build a multi-process poller on it. Two models support concurrent publishing and this member is part of neither: the provider's change-feed trigger, which holds one lease per partition, and — where a self-managed poller is genuinely wanted — ClaimPendingAsync(IPartitionKey, int, string, CancellationToken), which takes a real per-message lease. Choose one and do not combine them: the trigger path does not observe the claim's lease, so running both reproduces the duplicate delivery the claim exists to prevent.

What this read is for: recovery. A change feed surfaces a document when it is written or updated, and not again on its own. A message whose publish failed, and whose failure was never recorded back onto the document, is consequently never surfaced a second time — the feed has already moved past it, and nothing else will mention it. This read is how such a message is found. The ordinary retry path does not need it: recording the failure through IncrementRetryCountAsync(string, IPartitionKey, string?, CancellationToken) rewrites the document, which puts it back on the feed. This read is the backstop for when that did not happen — a publisher that swallowed the error, or one that stopped between the failed publish and the record of it.

Run it from a single process — a scheduled sweep or an operator tool — and treat overlapping runs as producing duplicates, which handlers must tolerate in any case.

FIFO, and what "eventually" can mean for a provider that orders via a secondary index. A provider whose native query engine can order a strongly-consistent read (Cosmos DB's ORDER BY, Firestore's OrderBy) returns messages in exact creation order with no further caveat. A provider that must order via a separately-maintained, eventually-consistent index (for example DynamoDB, whose base-table query is physically ordered by a per-message key and therefore orders pending reads through a Global Secondary Index instead) can, for a brief interval after AddAsync(CloudOutboxMessage, IPartitionKey, CancellationToken) returns, omit a just-staged message from this read — a LATENCY property, not a loss one: the message is durably present on the provider's strongly-consistent primary store throughout, and a subsequent call (the next poll, or this recovery sweep) will see it. AWS documents GSI propagation as typically completing within a fraction of a second under normal conditions, with no formal upper bound (a failure scenario can extend it) — design a recovery sweep interval, never a single immediate read, around this guarantee. Messages this call DOES return are always in creation order.

IncrementRetryCountAsync(string, IPartitionKey, string?, CancellationToken)

Increments the retry count for a failed message.

public Task<CloudOperationResult> IncrementRetryCountAsync(string messageId, IPartitionKey partitionKey, string? errorMessage, CancellationToken cancellationToken)

Parameters

messageId string
partitionKey IPartitionKey
errorMessage string
cancellationToken CancellationToken

Returns

Task<CloudOperationResult>

InitializeAsync(CancellationToken)

Initializes the DynamoDB client and creates the table if needed.

public Task InitializeAsync(CancellationToken cancellationToken)

Parameters

cancellationToken CancellationToken

Cancellation token.

Returns

Task

MarkAsPublishedAsync(string, IPartitionKey, CancellationToken)

Marks a message as published.

public Task<CloudOperationResult> MarkAsPublishedAsync(string messageId, IPartitionKey partitionKey, CancellationToken cancellationToken)

Parameters

messageId string

The message identifier.

partitionKey IPartitionKey

The partition key for the message.

cancellationToken CancellationToken

Cancellation token.

Returns

Task<CloudOperationResult>

The operation result with cost information.

MarkBatchAsPublishedAsync(IEnumerable<string>, IPartitionKey, CancellationToken)

Marks multiple messages as published in a batch.

public Task<CloudBatchResult> MarkBatchAsPublishedAsync(IEnumerable<string> messageIds, IPartitionKey partitionKey, CancellationToken cancellationToken)

Parameters

messageIds IEnumerable<string>
partitionKey IPartitionKey
cancellationToken CancellationToken

Returns

Task<CloudBatchResult>

SubscribeToNewMessagesAsync(IChangeFeedOptions?, CancellationToken)

Creates a change feed subscription for outbox processing.

[SuppressMessage("Reliability", "CA2000:Dispose objects before losing scope", Justification = "Ownership of subscription transfers to caller on successful return; disposed on failure path.")]
public Task<IChangeFeedSubscription<CloudOutboxMessage>> SubscribeToNewMessagesAsync(IChangeFeedOptions? options, CancellationToken cancellationToken)

Parameters

options IChangeFeedOptions

Change feed options.

cancellationToken CancellationToken

Cancellation token.

Returns

Task<IChangeFeedSubscription<CloudOutboxMessage>>

A subscription that streams new outbox messages.