Skip to main content

OpenSearch Provider

Full parity with the Elasticsearch provider, built on OpenSearch.Client. Covers projections, index lifecycle management (ISM), resilient client, health monitoring, dead letter handling, materialized views, and tenant sharding.

Before You Start​

  • .NET 10.0
  • An OpenSearch cluster (local, AWS OpenSearch Service, or self-hosted)
  • Familiarity with data access and projections

Installation​

dotnet add package Excalibur.Data.OpenSearch

Dependencies: Excalibur.Data.Abstractions, OpenSearch.Client

Quick Start​

// Register OpenSearch client + projection store
services.AddOpenSearchServices("https://opensearch.example.com:9200");

services.AddOpenSearchProjectionStore<OrderSummary>(opts =>
{
    opts.NodeUri = "https://opensearch.example.com:9200";
    opts.IndexName = "order-summaries";
});

Registration Options​

Client Registration​

// Single node
services.AddOpenSearchServices("https://opensearch.example.com:9200");

// Multi-node cluster
services.AddOpenSearchServices(new[]
{
    new Uri("https://node1.example.com:9200"),
    new Uri("https://node2.example.com:9200"),
    new Uri("https://node3.example.com:9200"),
});

// With custom connection settings
services.AddOpenSearchServices("https://opensearch.example.com:9200",
    configureSettings: settings =>
    {
        settings.BasicAuthentication("admin", "password");
        settings.DisableDirectStreaming();
    });

// With preconfigured client
var client = new OpenSearchClient(new ConnectionSettings(new Uri("https://...")));
services.AddOpenSearchServices(client);

Projection Store​

// Per-projection registration
services.AddOpenSearchProjectionStore<OrderSummary>(opts =>
{
    opts.NodeUri = "https://opensearch.example.com:9200";
    opts.IndexName = "order-summaries";
});

// With node URI shorthand
services.AddOpenSearchProjectionStore<OrderSummary>(
    "https://opensearch.example.com:9200");

// With shared client factory
services.AddOpenSearchProjectionStore<OrderSummary>(
    clientFactory: sp => sp.GetRequiredService<OpenSearchClient>(),
    configureOptions: opts => opts.IndexName = "order-summaries");

// Batch registration (multiple projections, shared node)
services.AddOpenSearchProjections("https://opensearch.example.com:9200", projections =>
{
    projections.Add<OrderSummary>();
    projections.Add<CustomerProfile>(opts => opts.IndexName = "customers");
    projections.Add<ProductCatalog>(opts => opts.IndexName = "products");
});

Resilient Client​

Adds retry and circuit breaker policies around OpenSearch operations:

services.Configure<OpenSearchResilienceOptions>(opts =>
{
    opts.MaxRetries = 3;
    opts.CircuitBreakerThreshold = 5;
});
Option TypeKey Settings
OpenSearchResilienceOptionsMaxRetries, CircuitBreakerThreshold
OpenSearchRetryPolicyOptionsMaxRetries, BaseDelay, MaxDelay
CircuitBreakerOptionsFailureThreshold, ResetTimeout, HalfOpenMaxAttempts
OpenSearchTimeoutOptionsRequestTimeout, ConnectionTimeout

Monitoring​

services.Configure<OpenSearchMonitoringOptions>(opts =>
{
    // Enable health monitoring, metrics, request logging
});

Includes:

  • Health monitoring via OpenSearchHealthMonitor (cluster green/yellow/red)
  • OTel metrics via OpenSearchMetrics (operations, latency, errors)
  • Activity tracing via OpenSearchActivitySource
  • Request logging via OpenSearchRequestLogger
  • Performance diagnostics via OpenSearchPerformanceDiagnostics

Health Checks​

services.AddHealthChecks()
    .AddOpenSearchHealthCheck();

Index Lifecycle Management (ISM)​

OpenSearch uses ISM (Index State Management) instead of Elasticsearch's ILM:

// Interfaces available via DI
// IIndexLifecycleManager -- ISM policy management
// IIndexTemplateManager -- Index template management
// IIndexOperationsManager -- Index CRUD, aliases, rollover
// IIndexAliasManager -- Alias management

Materialized Views​

services.AddOpenSearchMaterializedViews(opts =>
{
    opts.NodeUri = "https://opensearch.example.com:9200";
});

Dead Letter Handling​

Failed documents are captured in a dead letter index:

services.Configure<OpenSearchDeadLetterOptions>(opts =>
{
    opts.IndexName = "dead-letters";
    opts.MaxRetries = 3;
});

Tenant Sharding​

services.AddExcalibur(excalibur => excalibur.AddEventSourcing(builder =>
{
    builder.EnableTenantSharding(opts => opts.DefaultShardId = "shard-default");
    builder.UseOpenSearchTenantProjectionStore<OrderSummary>();
}));

Persistence Provider​

services.AddOpenSearchPersistence(opts =>
{
    opts.RefreshPolicy = OpenSearchRefreshPolicy.WaitFor;
});

Host Extensions​

Initialize indexes at application startup:

var host = builder.Build();
await host.InitializeOpenSearchIndexesAsync();
await host.RunAsync();

Elasticsearch vs OpenSearch Comparison​

FeatureElasticsearch PackageOpenSearch Package
Client libraryElastic.Clients.ElasticsearchOpenSearch.Client (NEST-based)
Index lifecycleILM (Index Lifecycle Management)ISM (Index State Management)
Projection storeAddElasticSearchProjectionStore<T>AddOpenSearchProjectionStore<T>
Batch registrationAddElasticSearchProjections()AddOpenSearchProjections()
Health checkAddElasticsearchHealthCheck()AddOpenSearchHealthCheck()
Resilient clientAddResilientElasticsearchServices()IResilientOpenSearchClient + options
Tenant shardingUseElasticSearchTenantProjectionStore<T>UseOpenSearchTenantProjectionStore<T>
Event ID range106000-106999108000-108999
Audit sinkAddElasticsearchAuditSink()AddOpenSearchAuditSink()
Audit exporterAddElasticsearchAuditExporter()AddOpenSearchAuditExporter()
Audit event IDs93471-9347593490-93505

The two packages provide feature parity. Choose based on your search engine.

Audit Sink​

A separate package provides an OpenSearch audit sink for real-time audit event indexing:

dotnet add package Excalibur.AuditLogging.OpenSearch
// With options callback
services.AddOpenSearchAuditSink(options =>
{
    // Single node
    options.OpenSearchUrl = "https://os.example.com:9200";

    // Or cluster (round-robin)
    options.NodeUrls = ["https://os1:9200", "https://os2:9200", "https://os3:9200"];

    options.IndexPrefix = "dispatch-audit";
    options.ApplicationName = "MyApp"; // fallback if AuditEvent.ApplicationName is null
});

// Or from IConfiguration
services.AddOpenSearchAuditSink(configuration.GetSection("AuditSink:OpenSearch"));
info

OpenSearch serves as a search/analytics sink, not a compliance-grade audit store. Use SQL Server for tamper-evident hash-chained storage. See ADR-290 and Audit Logging Providers.

See Also​