Table of Contents

Interface IStreamConsumerHandler<TDocument>

Namespace
Excalibur.Dispatch.Delivery
Assembly
Excalibur.Dispatch.Abstractions.dll

Defines a handler that consumes a stream of documents.

public interface IStreamConsumerHandler<in TDocument> where TDocument : IDispatchDocument

Type Parameters

TDocument

The type of documents in the stream.

Extension Methods

Examples

public class BatchImportHandler : IStreamConsumerHandler<DataRow>
{
    private readonly IDatabase _database;

    public BatchImportHandler(IDatabase database) => _database = database;

    public async Task HandleAsync(
        IAsyncEnumerable<DataRow> documents,
        CancellationToken cancellationToken)
    {
        var batch = new List<DataRow>();
        await foreach (var row in documents.WithCancellation(cancellationToken)
            .ConfigureAwait(false))
        {
            batch.Add(row);
            if (batch.Count >= 1000)
            {
                await _database.BulkInsertAsync(batch, cancellationToken)
                    .ConfigureAwait(false);
                batch.Clear();
            }
        }
        if (batch.Count > 0)
        {
            await _database.BulkInsertAsync(batch, cancellationToken)
                .ConfigureAwait(false);
        }
    }
}

Remarks

Implement this interface to process an incoming stream of documents. This enables memory-efficient batch processing where documents arrive incrementally. The handler controls consumption rate, providing natural backpressure. Common use cases include:

  • Batch imports: processing CSV/JSON rows as they are parsed
  • ETL sinks: writing transformed records to storage
  • Message queue consumers: processing message streams
  • Real-time data ingestion: handling continuous data feeds
  • Aggregation pipelines: computing statistics over streams

The contravariant in modifier on TDocument allows handlers to accept streams of derived document types.

Handlers are resolved from dependency injection and should be registered with scoped lifetime to maintain state within a single stream processing operation.

Methods

HandleAsync(IAsyncEnumerable<TDocument>, CancellationToken)

Handles the incoming stream of documents asynchronously.

Task HandleAsync(IAsyncEnumerable<in TDocument> documents, CancellationToken cancellationToken)

Parameters

documents IAsyncEnumerable<TDocument>

The stream of documents to process.

cancellationToken CancellationToken

The cancellation token to observe throughout processing.

Returns

Task

A task that completes when the stream has been fully processed.

Remarks

The handler controls consumption rate by the speed at which it iterates through documents. This provides natural backpressure to upstream producers.

Use WithCancellation(cancellationToken) when iterating to ensure proper cancellation propagation. Use ConfigureAwait(false) on all await expressions when implementing in library code.

The task completes when all documents have been processed. If cancellation is requested, the task should throw OperationCanceledException.