Table of Contents

Interface IStreamingDispatcher

Namespace
Excalibur.Dispatch
Assembly
Excalibur.Dispatch.Abstractions.dll

Provides streaming dispatch operations for document processing.

public interface IStreamingDispatcher
Extension Methods

Remarks

This interface is available as a sub-interface of the dispatcher, discoverable via DI or by casting an IDispatcher implementation that supports streaming.

Follows the ISP (Interface Segregation Principle) to keep the core IDispatcher focused on fire-and-forget and request-response patterns, while streaming operations are opt-in for consumers that need them.

Methods

DispatchStreamAsync<TDocument>(IAsyncEnumerable<TDocument>, IMessageContext, CancellationToken)

Dispatches a stream of documents to a consumer handler.

Task DispatchStreamAsync<TDocument>(IAsyncEnumerable<TDocument> documents, IMessageContext context, CancellationToken cancellationToken) where TDocument : IDispatchDocument

Parameters

documents IAsyncEnumerable<TDocument>

The stream of documents to process.

context IMessageContext

Context for the dispatch operation.

cancellationToken CancellationToken

Token used to cancel the operation.

Returns

Task

A task that completes when the stream has been fully processed.

Type Parameters

TDocument

Type of documents in the stream.

Examples

var rows = ParseCsvAsync(fileStream, cancellationToken);
await streamingDispatcher.DispatchStreamAsync<DataRow>(
    rows, context, cancellationToken);

Remarks

This method resolves an IStreamConsumerHandler<TDocument> from the service provider and invokes it with the document stream. The handler controls consumption rate, providing natural backpressure to upstream producers.

The task completes when the entire stream has been consumed. If cancellation is requested, an OperationCanceledException is thrown.

DispatchStreamingAsync<TDocument, TOutput>(TDocument, IMessageContext, CancellationToken)

Dispatches a document and returns a stream of outputs from the streaming handler.

IAsyncEnumerable<TOutput> DispatchStreamingAsync<TDocument, TOutput>(TDocument document, IMessageContext context, CancellationToken cancellationToken) where TDocument : IDispatchDocument

Parameters

document TDocument

The document to process.

context IMessageContext

Context for the dispatch operation.

cancellationToken CancellationToken

Token used to cancel the operation and streaming.

Returns

IAsyncEnumerable<TOutput>

An asynchronous stream of output items produced by the handler.

Type Parameters

TDocument

Type of document being dispatched.

TOutput

Type of output items produced by the handler.

Examples

await foreach (var row in streamingDispatcher.DispatchStreamingAsync<CsvDocument, DataRow>(
    document, context, cancellationToken))
{
    ProcessRow(row);
}

Remarks

This method resolves an IStreamingDocumentHandler<TDocument, TOutput> from the service provider and invokes it with the document. The returned stream should be consumed using await foreach to enable memory-efficient processing.

The cancellation token is propagated throughout the streaming operation. Consumers should use WithCancellation() when iterating to ensure responsive cancellation.

DispatchTransformStreamAsync<TInput, TOutput>(IAsyncEnumerable<TInput>, IMessageContext, CancellationToken)

Dispatches an input stream through a transform handler and returns the output stream.

IAsyncEnumerable<TOutput> DispatchTransformStreamAsync<TInput, TOutput>(IAsyncEnumerable<TInput> input, IMessageContext context, CancellationToken cancellationToken) where TInput : IDispatchDocument

Parameters

input IAsyncEnumerable<TInput>

The input stream of documents to transform.

context IMessageContext

Context for the dispatch operation.

cancellationToken CancellationToken

Token used to cancel the operation.

Returns

IAsyncEnumerable<TOutput>

An asynchronous stream of transformed output items.

Type Parameters

TInput

Type of input documents in the stream.

TOutput

Type of output items produced by the transformation.

Examples

var enriched = streamingDispatcher.DispatchTransformStreamAsync<CustomerRecord, EnrichedCustomer>(
    records, context, cancellationToken);
await foreach (var customer in enriched.WithCancellation(cancellationToken))
{
    await SaveAsync(customer);
}

Remarks

This method resolves an IStreamTransformHandler<TInput, TOutput> from the service provider and invokes it with the input stream. Transform handlers enable composable streaming pipelines for data enrichment, filtering, and format conversion.

The transformation is performed lazily as items are consumed from the output stream. This enables memory-efficient processing of large data sets without materializing intermediate collections.