Skip to main content

Amazon DynamoDB Provider

The DynamoDB provider implements ICloudNativePersistenceProvider for AWS serverless workloads with hash/sort key support, DynamoDB Streams integration, and transactional batch operations.

Before You Start​

Installation​

dotnet add package Excalibur.Data.DynamoDb

Dependencies: Excalibur.Data.Abstractions, AWSSDK.DynamoDBv2

Quick Start​

using Microsoft.Extensions.DependencyInjection;

services.AddExcaliburDynamoDb(dynamo =>
{
    dynamo.Region(RegionEndpoint.USEast1)
          .TablePrefix("MyApp_");
});

Registration​

The builder API (IDynamoDBDataBuilder) supports 4 canonical connection overloads plus configuration binding:

// AWS Region (uses default credentials / IAM role)
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.Region(RegionEndpoint.USEast1).TableName("MyTable"));

// Local DynamoDB / LocalStack
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.ServiceUrl("http://localhost:8000").TableName("MyTable"));

// Existing IAmazonDynamoDB client
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.Client(existingClient).TableName("MyTable"));

// Client factory (for custom configuration)
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.ClientFactory(sp => new AmazonDynamoDBClient(credentials, region))
          .TableName("MyTable"));

// IConfiguration binding
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.BindConfiguration("DynamoDb"));

All registrations include ValidateOnStart for options validation.

The event store writes each event's declared [MessageName] -- not its CLR type name -- to the stored event-type field, and resolves that name back to a CLR type through the registered event-type registry on read. See Stable Message Names.

Projection Store​

Projections are stored flat at the document root. Framework metadata is isolated under a _projection nested object to avoid collisions with your projection properties. See Projections — Document Storage Format for details.

Subsystem Registration​

Each Excalibur subsystem supports DynamoDB via its own builder:

services.AddExcalibur(excalibur =>
{
    // Event Sourcing
    excalibur.AddEventSourcing(es =>
        es.UseDynamoDb(dynamo =>
            dynamo.Region(RegionEndpoint.USEast1)
                  .TableName("Events")
                  .TablePrefix("MyApp_")));
    // Configured by region, so a matching Streams client is built for you and the
    // change feed works with no further configuration. See "Streams Methods" below
    // for when you supply your own client.

    // Saga
    excalibur.AddSaga(saga =>
        saga.UseDynamoDb(dynamo =>
            dynamo.Region(RegionEndpoint.USEast1)
                  .TableName("Sagas")));

    // Inbox (idempotency)
    excalibur.AddInbox(inbox =>
        inbox.UseDynamoDb(dynamo =>
            dynamo.Region(RegionEndpoint.USEast1)
                  .TableName("Inbox")));

    // Outbox (reliable messaging)
    excalibur.AddOutbox(outbox =>
        outbox.UseDynamoDb(dynamo =>
            dynamo.Region(RegionEndpoint.USEast1)
                  .TableName("Outbox")));
});

Change Data Capture​

services.AddCdcProcessor(cdc =>
{
    cdc.UseDynamoDb(dynamo =>
    {
        dynamo.TableName("Orders")
              .StreamArn("arn:aws:dynamodb:us-east-1:123456789:table/Orders/stream/2026-01-01T00:00:00.000")
              .ProcessorName("order-processor")
              .WithStateStore(
                  sp => new AmazonDynamoDBClient(),
                  state => state.TableName("CdcState"));
    })
    .TrackTable("Orders", t => t.MapAll<OrderChangedEvent>())
    .EnableBackgroundProcessing();
});

Builder Methods​

Connection Methods​

All 5 non-CDC subsystem builders share the same connection methods:

MethodDescription
ServiceUrl(string)Local DynamoDB or LocalStack endpoint URL
Region(RegionEndpoint)AWS region (uses default credential chain)
Client(IAmazonDynamoDB)Pre-configured client instance
ClientFactory(Func<IServiceProvider, IAmazonDynamoDB>)Factory for custom client creation
BindConfiguration(string)Bind from IConfiguration section

Connection methods follow last-wins semantics — calling Client() after Region() replaces the region-based connection.

Streams Methods​

Available on the event sourcing builder (IDynamoDBEventSourcingBuilder) only.

MethodDescription
StreamsClient(IAmazonDynamoDBStreams)Pre-configured Streams client instance, enabling the change feed
StreamsClientFactory(Func<IServiceProvider, IAmazonDynamoDBStreams>)Factory resolving the Streams client from DI, enabling the change feed

You do not need either of these unless you consume a change feed. The event store appends, loads, and reads versions through the DynamoDB client alone; a store built without a Streams client is fully functional for those operations and simply reports the change feed as unavailable.

When you do consume one, whether you must supply a client depends on how the connection was configured:

Connection configured byStreams client
ServiceUrl(...) or Region(...)Built for you automatically — nothing to configure
Client(...) or ClientFactory(...)You must supply one, or the change feed is unavailable

The registration will not guess at the endpoint and credentials behind a client you supplied, so it builds no Streams client on those paths. Supplying your own also means the change feed runs under your own credentials, endpoint, and telemetry:

using Amazon.DynamoDBStreams;
using Amazon.DynamoDBv2;

services.AddExcalibur(excalibur => excalibur.AddEventSourcing(es =>
    es.UseDynamoDb(dynamo =>
        dynamo.Client(myDynamoDbClient)
              .StreamsClient(myStreamsClient)
              .TableName("Events"))));

Unlike the connection methods, these are orthogonal to the connection mode: selecting or changing a connection method neither sets nor clears the Streams client. StreamsClient and StreamsClientFactory are last-wins against each other.

See Event Sourcing — Change Feed for the full walkthrough.

Domain Methods​

MethodDescriptionAvailable On
TableName(string)DynamoDB table nameAll 6 builders
TablePrefix(string)Prefix for table names5 builders (not CDC)
StreamArn(string)DynamoDB Streams ARNCDC only
StreamsClient(...) / StreamsClientFactory(...)Streams client for the change feedEvent sourcing only
ProcessorName(string)CDC processor nameCDC only
WithStateStore(...)CDC state store configurationCDC only

Partition and Sort Keys​

DynamoDB uses hash keys and optional sort keys:

var key = new PartitionKey("USER#user-123", "/pk");
var result = await provider.GetByIdAsync<UserProfile>(
    "PROFILE#main", key,
    consistencyOptions: null,
    cancellationToken: ct);

Batch Operations​

Execute multiple operations atomically:

var batchResult = await provider.ExecuteBatchAsync(
    partitionKey,
    operations,
    cancellationToken: ct);

Atomic event-store append limit​

The DynamoDB event store appends a batch as a single all-or-nothing TransactWriteItems call. DynamoDB hard-caps TransactWriteItems at 100 items, and offers no atomic primitive beyond that. To guarantee the append contract (no torn event-stream prefix), the store rejects an atomic append of more than 100 events at the boundary, before any write:

  • More than 100 events → the append is rejected up front; split the batch into appends of at most 100 events. The exception carries the count you supplied and the limit, so the split is mechanical.
  • 2 to 100 events → one TransactWriteItems call, all-or-nothing.
  • A single event → one conditional PutItem, which is atomic on its own. It does not use a transaction, so it does not incur the doubled write cost or the additional IAM permission TransactWriteItems requires.

There is no opt-out that accepts a larger append. An earlier release offered one, and it bought throughput by writing items individually — which leaves a torn prefix if it fails partway, and no later read can tell that apart from a stream that was simply never written further.

Configuration Binding​

Bind DynamoDB options from appsettings.json:

{
  "DynamoDb": {
    "Connection": {
      "Region": "us-east-1",
      "ServiceUrl": null
    },
    "DefaultTableName": "MyTable",
    "DefaultPartitionKeyAttribute": "pk",
    "DefaultSortKeyAttribute": "sk",
    "UseConsistentReads": false
  }
}
services.AddExcaliburDynamoDb(dynamo =>
    dynamo.BindConfiguration("DynamoDb"));

See Also​