Implement in-process multi-dataset sync isolation across core, network, persistence, and tests
All checks were successful
NuGet Package Publish / nuget (push) Successful in 1m14s

This commit is contained in:
Joseph Doherty
2026-02-22 11:58:34 -05:00
parent c06b56172a
commit 8e97061ab8
60 changed files with 4519 additions and 559 deletions

View File

@@ -30,6 +30,13 @@ To optimize reconnection, each node maintains a **Snapshot** of the last known s
- If the chain hash matches, they only exchange the delta. - If the chain hash matches, they only exchange the delta.
- This avoids re-processing the entire operation history and ensures efficient gap recovery. - This avoids re-processing the entire operation history and ensures efficient gap recovery.
### Multi-Dataset Sync
CBDDC supports per-dataset sync pipelines in one process.
- Dataset identity (`datasetId`) is propagated in protocol and persistence records.
- Each dataset has independent oplog reads, confirmation state, and maintenance cadence.
- Legacy peers without dataset fields interoperate on `primary`.
### Peer-Confirmed Oplog Pruning ### Peer-Confirmed Oplog Pruning
CBDDC maintenance pruning now uses a two-cutoff model: CBDDC maintenance pruning now uses a two-cutoff model:

View File

@@ -8,6 +8,7 @@ This index tracks CBDDC major functionality. Each feature has one canonical docu
- [Peer-to-Peer Gossip Sync](peer-to-peer-gossip-sync.md) - [Peer-to-Peer Gossip Sync](peer-to-peer-gossip-sync.md)
- [Secure Peer Transport](secure-peer-transport.md) - [Secure Peer Transport](secure-peer-transport.md)
- [Peer-Confirmed Pruning](peer-confirmed-pruning.md) - [Peer-Confirmed Pruning](peer-confirmed-pruning.md)
- [Multi-Dataset Sync](multi-dataset-sync.md)
## Maintenance Rules ## Maintenance Rules

View File

@@ -0,0 +1,67 @@
# Multi-Dataset Sync
## Summary
CBDDC can run multiple sync pipelines inside one process by assigning each pipeline a `datasetId` (for example `primary`, `logs`, `timeseries`).
Each dataset pipeline has independent oplog state, vector-clock reads, peer confirmation watermarks, and maintenance scheduling.
## Why Use It
- Keep primary business data sync latency stable during high telemetry volume.
- Isolate append-only streams (`logs`, `timeseries`) from CRUD-heavy collections.
- Roll out incrementally using runtime flags and per-dataset enablement.
## Configuration
Register dataset options and enable the runtime coordinator:
```csharp
services.AddCBDDCSurrealEmbedded<SampleDocumentStore>(sp => options)
.AddCBDDCSurrealEmbeddedDataset("primary", o =>
{
o.InterestingCollections = ["Users", "TodoLists"];
})
.AddCBDDCSurrealEmbeddedDataset("logs", o =>
{
o.InterestingCollections = ["Logs"];
o.SyncLoopDelay = TimeSpan.FromMilliseconds(500);
})
.AddCBDDCSurrealEmbeddedDataset("timeseries", o =>
{
o.InterestingCollections = ["Timeseries"];
o.SyncLoopDelay = TimeSpan.FromMilliseconds(500);
})
.AddCBDDCNetwork<StaticPeerNodeConfigurationProvider>();
services.AddCBDDCMultiDataset(options =>
{
options.EnableMultiDatasetSync = true;
options.EnableDatasetPrimary = true;
options.EnableDatasetLogs = true;
options.EnableDatasetTimeseries = true;
});
```
## Wire and Storage Compatibility
- Protocol messages include optional `dataset_id` fields.
- Missing `dataset_id` is treated as `primary`.
- Surreal persistence records include `datasetId`; legacy rows without `datasetId` are read as `primary`.
## Operational Notes
- Each dataset runs its own `SyncOrchestrator` instance.
- Maintenance pruning is dataset-scoped (`datasetId` + cutoff).
- Snapshot APIs support dataset-scoped operations (`CreateSnapshotAsync(stream, datasetId)`).
## Migration
1. Deploy with `EnableMultiDatasetSync = false`.
2. Enable multi-dataset mode with only `primary` enabled.
3. Enable `logs`, verify primary sync SLO.
4. Enable `timeseries`, verify primary sync SLO again.
## Rollback
- Set `EnableDatasetLogs = false` and `EnableDatasetTimeseries = false` first.
- If needed, set `EnableMultiDatasetSync = false` to return to the single `primary` sync path.

View File

@@ -221,6 +221,14 @@ services.AddCBDDCCore()
}); });
``` ```
### Multi-Dataset Partitioning
Surreal persistence now stores `datasetId` on oplog, metadata, snapshot metadata, confirmation, and CDC checkpoint records.
- Composite indexes include `datasetId` to prevent cross-dataset reads.
- Legacy rows missing `datasetId` are interpreted as `primary` during reads.
- Dataset-scoped store APIs (`ExportAsync(datasetId)`, `GetOplogAfterAsync(..., datasetId, ...)`) enforce isolation.
### CDC Durability Notes ### CDC Durability Notes
1. **Checkpoint semantics**: each consumer id has an independent durable cursor (`timestamp + hash`). 1. **Checkpoint semantics**: each consumer id has an independent durable cursor (`timestamp + hash`).

View File

@@ -27,6 +27,15 @@ Capture these artifacts before remediation:
- Current runtime configuration (excluding secrets). - Current runtime configuration (excluding secrets).
- Most recent deployment identifier and change window. - Most recent deployment identifier and change window.
## Multi-Dataset Gates
Before enabling telemetry datasets in production:
1. Enable `primary` only and record baseline primary sync lag.
2. Enable `logs`; confirm primary lag remains within SLO.
3. Enable `timeseries`; confirm primary lag remains within SLO.
4. If primary SLO regresses, disable telemetry datasets first before broader rollback.
## Recovery Plays ## Recovery Plays
### Peer unreachable or lagging ### Peer unreachable or lagging

View File

@@ -112,6 +112,7 @@ public class ConsoleInteractiveService : BackgroundService
System.Console.WriteLine("Commands:"); System.Console.WriteLine("Commands:");
System.Console.WriteLine(" [p]ut, [g]et, [d]elete, [f]ind, [l]ist peers, [q]uit"); System.Console.WriteLine(" [p]ut, [g]et, [d]elete, [f]ind, [l]ist peers, [q]uit");
System.Console.WriteLine(" [n]ew (auto), [s]pam (5x), [c]ount, [t]odos"); System.Console.WriteLine(" [n]ew (auto), [s]pam (5x), [c]ount, [t]odos");
System.Console.WriteLine(" log [count], ts [count] (append telemetry load)");
System.Console.WriteLine(" [h]ealth, cac[h]e"); System.Console.WriteLine(" [h]ealth, cac[h]e");
System.Console.WriteLine(" [r]esolver [lww|merge], [demo] conflict"); System.Console.WriteLine(" [r]esolver [lww|merge], [demo] conflict");
} }
@@ -156,8 +157,12 @@ public class ConsoleInteractiveService : BackgroundService
{ {
int userCount = _db.Users.FindAll().Count(); int userCount = _db.Users.FindAll().Count();
int todoCount = _db.TodoLists.FindAll().Count(); int todoCount = _db.TodoLists.FindAll().Count();
int logCount = _db.Logs.FindAll().Count();
int timeseriesCount = _db.Timeseries.FindAll().Count();
System.Console.WriteLine($"Collection 'Users': {userCount} documents"); System.Console.WriteLine($"Collection 'Users': {userCount} documents");
System.Console.WriteLine($"Collection 'TodoLists': {todoCount} documents"); System.Console.WriteLine($"Collection 'TodoLists': {todoCount} documents");
System.Console.WriteLine($"Collection 'Logs': {logCount} documents");
System.Console.WriteLine($"Collection 'Timeseries': {timeseriesCount} documents");
} }
else if (input.StartsWith("p")) else if (input.StartsWith("p"))
{ {
@@ -212,6 +217,42 @@ public class ConsoleInteractiveService : BackgroundService
var results = _db.Users.Find(u => u.Age > 28); var results = _db.Users.Find(u => u.Age > 28);
foreach (var u in results) System.Console.WriteLine($"Found: {u.Name} ({u.Age})"); foreach (var u in results) System.Console.WriteLine($"Found: {u.Name} ({u.Age})");
} }
else if (input.StartsWith("log", StringComparison.OrdinalIgnoreCase))
{
int count = ParseCount(input, 100);
for (var i = 0; i < count; i++)
{
var entry = new TelemetryLogEntry
{
Id = Guid.NewGuid().ToString("N"),
Level = i % 25 == 0 ? "Warning" : "Information",
Message = $"sample-log-{DateTimeOffset.UtcNow:O}-{i}",
CreatedUtc = DateTime.UtcNow
};
await _db.Logs.InsertAsync(entry);
}
await _db.SaveChangesAsync();
System.Console.WriteLine($"Appended {count} log entries.");
}
else if (input.StartsWith("ts", StringComparison.OrdinalIgnoreCase))
{
int count = ParseCount(input, 100);
for (var i = 0; i < count; i++)
{
var point = new TimeseriesPoint
{
Id = Guid.NewGuid().ToString("N"),
Metric = i % 2 == 0 ? "cpu" : "latency",
Value = Random.Shared.NextDouble() * 100,
RecordedUtc = DateTime.UtcNow
};
await _db.Timeseries.InsertAsync(point);
}
await _db.SaveChangesAsync();
System.Console.WriteLine($"Appended {count} timeseries points.");
}
else if (input.StartsWith("h")) else if (input.StartsWith("h"))
{ {
var health = await _healthCheck.CheckAsync(); var health = await _healthCheck.CheckAsync();
@@ -283,6 +324,13 @@ public class ConsoleInteractiveService : BackgroundService
} }
} }
private static int ParseCount(string input, int fallback)
{
string[] parts = input.Split(' ', StringSplitOptions.RemoveEmptyEntries);
if (parts.Length < 2) return fallback;
return int.TryParse(parts[1], out int parsed) && parsed > 0 ? parsed : fallback;
}
private async Task RunConflictDemo() private async Task RunConflictDemo()
{ {
System.Console.WriteLine("\n=== Conflict Resolution Demo ==="); System.Console.WriteLine("\n=== Conflict Resolution Demo ===");

View File

@@ -4,6 +4,7 @@ using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using Serilog; using Serilog;
using System.Text.Json; using System.Text.Json;
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Network; using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Core.Storage; using ZB.MOM.WW.CBDDC.Core.Storage;
using ZB.MOM.WW.CBDDC.Core.Sync; using ZB.MOM.WW.CBDDC.Core.Sync;
@@ -62,11 +63,22 @@ internal class Program
Directory.CreateDirectory(dataPath); Directory.CreateDirectory(dataPath);
string databasePath = Path.Combine(dataPath, $"{nodeId}.rocksdb"); string databasePath = Path.Combine(dataPath, $"{nodeId}.rocksdb");
string surrealDatabase = nodeId.Replace("-", "_", StringComparison.Ordinal); string surrealDatabase = nodeId.Replace("-", "_", StringComparison.Ordinal);
var multiDatasetOptions = builder.Configuration
.GetSection("CBDDC:MultiDataset")
.Get<MultiDatasetRuntimeOptions>()
?? new MultiDatasetRuntimeOptions
{
EnableMultiDatasetSync = true,
EnableDatasetPrimary = true,
EnableDatasetLogs = true,
EnableDatasetTimeseries = true
};
// Register CBDDC services with embedded Surreal (RocksDB). // Register CBDDC services with embedded Surreal (RocksDB).
builder.Services.AddSingleton<ICBDDCSurrealSchemaInitializer, SampleSurrealSchemaInitializer>(); builder.Services.AddSingleton<ICBDDCSurrealSchemaInitializer, SampleSurrealSchemaInitializer>();
builder.Services.AddSingleton<SampleDbContext>(); builder.Services.AddSingleton<SampleDbContext>();
builder.Services.AddCBDDCCore() builder.Services
.AddCBDDCCore()
.AddCBDDCSurrealEmbedded<SampleDocumentStore>(_ => new CBDDCSurrealEmbeddedOptions .AddCBDDCSurrealEmbedded<SampleDocumentStore>(_ => new CBDDCSurrealEmbeddedOptions
{ {
Endpoint = "rocksdb://local", Endpoint = "rocksdb://local",
@@ -74,8 +86,30 @@ internal class Program
Namespace = "cbddc_sample", Namespace = "cbddc_sample",
Database = surrealDatabase Database = surrealDatabase
}) })
.AddCBDDCSurrealEmbeddedDataset(DatasetId.Primary, options =>
{
options.InterestingCollections = ["Users", "TodoLists"];
})
.AddCBDDCSurrealEmbeddedDataset(DatasetId.Logs, options =>
{
options.InterestingCollections = ["Logs"];
})
.AddCBDDCSurrealEmbeddedDataset(DatasetId.Timeseries, options =>
{
options.InterestingCollections = ["Timeseries"];
})
.AddCBDDCNetwork<StaticPeerNodeConfigurationProvider>(); // useHostedService = true by default .AddCBDDCNetwork<StaticPeerNodeConfigurationProvider>(); // useHostedService = true by default
if (multiDatasetOptions.EnableMultiDatasetSync)
builder.Services.AddCBDDCMultiDataset(options =>
{
options.EnableMultiDatasetSync = multiDatasetOptions.EnableMultiDatasetSync;
options.EnableDatasetPrimary = multiDatasetOptions.EnableDatasetPrimary;
options.EnableDatasetLogs = multiDatasetOptions.EnableDatasetLogs;
options.EnableDatasetTimeseries = multiDatasetOptions.EnableDatasetTimeseries;
options.AdditionalDatasets = multiDatasetOptions.AdditionalDatasets.ToList();
});
builder.Services.AddHostedService<ConsoleInteractiveService>(); // Runs the Input Loop builder.Services.AddHostedService<ConsoleInteractiveService>(); // Runs the Input Loop
var host = builder.Build(); var host = builder.Build();

View File

@@ -11,6 +11,8 @@ public class SampleDbContext : IDisposable
{ {
private const string UsersTable = "sample_users"; private const string UsersTable = "sample_users";
private const string TodoListsTable = "sample_todo_lists"; private const string TodoListsTable = "sample_todo_lists";
private const string LogsTable = "sample_logs";
private const string TimeseriesTable = "sample_timeseries";
private readonly bool _ownsClient; private readonly bool _ownsClient;
@@ -28,6 +30,8 @@ public class SampleDbContext : IDisposable
Users = new SampleSurrealCollection<User>(UsersTable, u => u.Id, SurrealEmbeddedClient, SchemaInitializer); Users = new SampleSurrealCollection<User>(UsersTable, u => u.Id, SurrealEmbeddedClient, SchemaInitializer);
TodoLists = new SampleSurrealCollection<TodoList>(TodoListsTable, t => t.Id, SurrealEmbeddedClient, SchemaInitializer); TodoLists = new SampleSurrealCollection<TodoList>(TodoListsTable, t => t.Id, SurrealEmbeddedClient, SchemaInitializer);
Logs = new SampleSurrealCollection<TelemetryLogEntry>(LogsTable, e => e.Id, SurrealEmbeddedClient, SchemaInitializer);
Timeseries = new SampleSurrealCollection<TimeseriesPoint>(TimeseriesTable, p => p.Id, SurrealEmbeddedClient, SchemaInitializer);
OplogEntries = new SampleSurrealReadOnlyCollection<SampleOplogEntry>( OplogEntries = new SampleSurrealReadOnlyCollection<SampleOplogEntry>(
CBDDCSurrealSchemaNames.OplogEntriesTable, CBDDCSurrealSchemaNames.OplogEntriesTable,
SurrealEmbeddedClient, SurrealEmbeddedClient,
@@ -57,6 +61,8 @@ public class SampleDbContext : IDisposable
Users = new SampleSurrealCollection<User>(UsersTable, u => u.Id, SurrealEmbeddedClient, SchemaInitializer); Users = new SampleSurrealCollection<User>(UsersTable, u => u.Id, SurrealEmbeddedClient, SchemaInitializer);
TodoLists = new SampleSurrealCollection<TodoList>(TodoListsTable, t => t.Id, SurrealEmbeddedClient, SchemaInitializer); TodoLists = new SampleSurrealCollection<TodoList>(TodoListsTable, t => t.Id, SurrealEmbeddedClient, SchemaInitializer);
Logs = new SampleSurrealCollection<TelemetryLogEntry>(LogsTable, e => e.Id, SurrealEmbeddedClient, SchemaInitializer);
Timeseries = new SampleSurrealCollection<TimeseriesPoint>(TimeseriesTable, p => p.Id, SurrealEmbeddedClient, SchemaInitializer);
OplogEntries = new SampleSurrealReadOnlyCollection<SampleOplogEntry>( OplogEntries = new SampleSurrealReadOnlyCollection<SampleOplogEntry>(
CBDDCSurrealSchemaNames.OplogEntriesTable, CBDDCSurrealSchemaNames.OplogEntriesTable,
SurrealEmbeddedClient, SurrealEmbeddedClient,
@@ -88,6 +94,16 @@ public class SampleDbContext : IDisposable
/// </summary> /// </summary>
public SampleSurrealReadOnlyCollection<SampleOplogEntry> OplogEntries { get; private set; } public SampleSurrealReadOnlyCollection<SampleOplogEntry> OplogEntries { get; private set; }
/// <summary>
/// Gets the append-only telemetry logs collection.
/// </summary>
public SampleSurrealCollection<TelemetryLogEntry> Logs { get; private set; }
/// <summary>
/// Gets the append-only timeseries collection.
/// </summary>
public SampleSurrealCollection<TimeseriesPoint> Timeseries { get; private set; }
/// <summary> /// <summary>
/// Ensures schema changes are applied before persisting updates. /// Ensures schema changes are applied before persisting updates.
/// </summary> /// </summary>
@@ -102,6 +118,8 @@ public class SampleDbContext : IDisposable
{ {
Users.Dispose(); Users.Dispose();
TodoLists.Dispose(); TodoLists.Dispose();
Logs.Dispose();
Timeseries.Dispose();
if (_ownsClient) SurrealEmbeddedClient.Dispose(); if (_ownsClient) SurrealEmbeddedClient.Dispose();
} }
@@ -126,6 +144,8 @@ public sealed class SampleSurrealSchemaInitializer : ICBDDCSurrealSchemaInitiali
private const string SampleSchemaSql = """ private const string SampleSchemaSql = """
DEFINE TABLE OVERWRITE sample_users SCHEMALESS CHANGEFEED 7d; DEFINE TABLE OVERWRITE sample_users SCHEMALESS CHANGEFEED 7d;
DEFINE TABLE OVERWRITE sample_todo_lists SCHEMALESS CHANGEFEED 7d; DEFINE TABLE OVERWRITE sample_todo_lists SCHEMALESS CHANGEFEED 7d;
DEFINE TABLE OVERWRITE sample_logs SCHEMALESS CHANGEFEED 7d;
DEFINE TABLE OVERWRITE sample_timeseries SCHEMALESS CHANGEFEED 7d;
"""; """;
private readonly ICBDDCSurrealEmbeddedClient _client; private readonly ICBDDCSurrealEmbeddedClient _client;
private int _initialized; private int _initialized;

View File

@@ -14,6 +14,8 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
{ {
private const string UsersCollection = "Users"; private const string UsersCollection = "Users";
private const string TodoListsCollection = "TodoLists"; private const string TodoListsCollection = "TodoLists";
private const string LogsCollection = "Logs";
private const string TimeseriesCollection = "Timeseries";
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="SampleDocumentStore"/> class. /// Initializes a new instance of the <see cref="SampleDocumentStore"/> class.
@@ -40,6 +42,8 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
{ {
WatchCollection(UsersCollection, context.Users, u => u.Id); WatchCollection(UsersCollection, context.Users, u => u.Id);
WatchCollection(TodoListsCollection, context.TodoLists, t => t.Id); WatchCollection(TodoListsCollection, context.TodoLists, t => t.Id);
WatchCollection(LogsCollection, context.Logs, entry => entry.Id);
WatchCollection(TimeseriesCollection, context.Timeseries, point => point.Id);
} }
/// <inheritdoc /> /// <inheritdoc />
@@ -71,6 +75,8 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
{ {
UsersCollection => SerializeEntity(await _context.Users.FindByIdAsync(key, cancellationToken)), UsersCollection => SerializeEntity(await _context.Users.FindByIdAsync(key, cancellationToken)),
TodoListsCollection => SerializeEntity(await _context.TodoLists.FindByIdAsync(key, cancellationToken)), TodoListsCollection => SerializeEntity(await _context.TodoLists.FindByIdAsync(key, cancellationToken)),
LogsCollection => SerializeEntity(await _context.Logs.FindByIdAsync(key, cancellationToken)),
TimeseriesCollection => SerializeEntity(await _context.Timeseries.FindByIdAsync(key, cancellationToken)),
_ => null _ => null
}; };
} }
@@ -106,6 +112,12 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
TodoListsCollection => (await _context.TodoLists.FindAllAsync(cancellationToken)) TodoListsCollection => (await _context.TodoLists.FindAllAsync(cancellationToken))
.Select(t => (t.Id, SerializeEntity(t)!.Value)) .Select(t => (t.Id, SerializeEntity(t)!.Value))
.ToList(), .ToList(),
LogsCollection => (await _context.Logs.FindAllAsync(cancellationToken))
.Select(entry => (entry.Id, SerializeEntity(entry)!.Value))
.ToList(),
TimeseriesCollection => (await _context.Timeseries.FindAllAsync(cancellationToken))
.Select(point => (point.Id, SerializeEntity(point)!.Value))
.ToList(),
_ => [] _ => []
}; };
} }
@@ -137,6 +149,26 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
await _context.TodoLists.UpdateAsync(todo, cancellationToken); await _context.TodoLists.UpdateAsync(todo, cancellationToken);
break; break;
case LogsCollection:
var logEntry = content.Deserialize<TelemetryLogEntry>() ??
throw new InvalidOperationException("Failed to deserialize telemetry log.");
logEntry.Id = key;
if (await _context.Logs.FindByIdAsync(key, cancellationToken) == null)
await _context.Logs.InsertAsync(logEntry, cancellationToken);
else
await _context.Logs.UpdateAsync(logEntry, cancellationToken);
break;
case TimeseriesCollection:
var point = content.Deserialize<TimeseriesPoint>() ??
throw new InvalidOperationException("Failed to deserialize timeseries point.");
point.Id = key;
if (await _context.Timeseries.FindByIdAsync(key, cancellationToken) == null)
await _context.Timeseries.InsertAsync(point, cancellationToken);
else
await _context.Timeseries.UpdateAsync(point, cancellationToken);
break;
default: default:
throw new NotSupportedException($"Collection '{collection}' is not supported for sync."); throw new NotSupportedException($"Collection '{collection}' is not supported for sync.");
} }
@@ -152,6 +184,12 @@ public class SampleDocumentStore : SurrealDocumentStore<SampleDbContext>
case TodoListsCollection: case TodoListsCollection:
await _context.TodoLists.DeleteAsync(key, cancellationToken); await _context.TodoLists.DeleteAsync(key, cancellationToken);
break; break;
case LogsCollection:
await _context.Logs.DeleteAsync(key, cancellationToken);
break;
case TimeseriesCollection:
await _context.Timeseries.DeleteAsync(key, cancellationToken);
break;
default: default:
_logger.LogWarning("Attempted to remove entity from unsupported collection: {Collection}", collection); _logger.LogWarning("Attempted to remove entity from unsupported collection: {Collection}", collection);
break; break;

View File

@@ -0,0 +1,57 @@
using System.ComponentModel.DataAnnotations;
namespace ZB.MOM.WW.CBDDC.Sample.Console;
/// <summary>
/// Append-only telemetry log entry used for high-volume sync scenarios.
/// </summary>
public class TelemetryLogEntry
{
/// <summary>
/// Gets or sets the unique log identifier.
/// </summary>
[Key]
public string Id { get; set; } = Guid.NewGuid().ToString("N");
/// <summary>
/// Gets or sets the log level.
/// </summary>
public string Level { get; set; } = "Information";
/// <summary>
/// Gets or sets the log message.
/// </summary>
public string Message { get; set; } = string.Empty;
/// <summary>
/// Gets or sets the UTC timestamp.
/// </summary>
public DateTime CreatedUtc { get; set; } = DateTime.UtcNow;
}
/// <summary>
/// Append-only timeseries metric point used for telemetry sync scenarios.
/// </summary>
public class TimeseriesPoint
{
/// <summary>
/// Gets or sets the unique metric point identifier.
/// </summary>
[Key]
public string Id { get; set; } = Guid.NewGuid().ToString("N");
/// <summary>
/// Gets or sets the metric name.
/// </summary>
public string Metric { get; set; } = "cpu";
/// <summary>
/// Gets or sets the metric value.
/// </summary>
public double Value { get; set; }
/// <summary>
/// Gets or sets the UTC timestamp.
/// </summary>
public DateTime RecordedUtc { get; set; } = DateTime.UtcNow;
}

View File

@@ -34,6 +34,12 @@
"EnableOfflineQueue": true, "EnableOfflineQueue": true,
"MaxQueueSize": 1000 "MaxQueueSize": 1000
}, },
"MultiDataset": {
"EnableMultiDatasetSync": true,
"EnableDatasetPrimary": true,
"EnableDatasetLogs": true,
"EnableDatasetTimeseries": true
},
"Logging": { "Logging": {
"LogLevel": "Information", "LogLevel": "Information",
"LogFilePath": "logs/cbddc.log", "LogFilePath": "logs/cbddc.log",

323
separate.md Normal file
View File

@@ -0,0 +1,323 @@
# In-Process Multi-Dataset Sync Plan (Worktree Execution)
## Goal
Add true in-process multi-dataset sync so primary business data can sync independently from high-volume append-only datasets (logs, timeseries), with separate state, scheduling, and backpressure behavior.
## Desired Outcome
1. Primary dataset sync throughput/latency is not materially impacted by telemetry dataset volume.
2. Log and timeseries datasets use independent sync pipelines in the same process.
3. Existing single-dataset apps continue to work with minimal/no code changes.
4. Test coverage explicitly verifies isolation and no cross-dataset leakage.
## Current Baseline (Why This Change Is Needed)
1. Current host wiring registers a single `IDocumentStore`, `IOplogStore`, and `ISyncOrchestrator` graph.
2. Collection filtering exists, but all collections still share one orchestrator/sync loop and one oplog/vector clock lifecycle.
3. Protocol filters by collection only; there is no dataset identity boundary.
4. Surreal schema objects are fixed names per configured namespace/database and are not dataset-aware by design.
## Proposed Target Architecture
## New Concepts
1. `DatasetId`:
- Stable identifier (`primary`, `logs`, `timeseries`, etc).
- Included in all sync-state-bearing entities and wire messages.
2. `DatasetSyncContext`:
- Encapsulates one dataset's services: document store adapter, oplog store, snapshot metadata, peer confirmation state, orchestrator configuration.
3. `IMultiDatasetSyncOrchestrator`:
- Host-level coordinator that starts/stops one `ISyncOrchestrator` per dataset.
4. `DatasetSyncOptions`:
- Per-dataset scheduling and limits (loop delay, max peers, optional bandwidth/entry caps, maintenance interval override).
## Isolation Model
1. Independent per-dataset oplog stream and vector clock.
2. Independent per-dataset peer confirmation watermarks for pruning.
3. Independent per-dataset transport filtering (handshake and pull/push include dataset id).
4. Independent per-dataset observability counters.
## Compatibility Strategy
1. Backward compatible wire changes:
- Add optional `dataset_id` fields; default to `"primary"` when absent.
2. Backward compatible storage:
- Add `datasetId` columns/fields where needed.
- Existing rows default to `"primary"` during migration/read fallback.
3. API defaults:
- Existing single-store registration maps to dataset `"primary"` with no functional change.
## Git Worktree Execution Plan
## 0. Worktree Preparation
1. Create worktree and branch:
- `git worktree add ../CBDDC-multidataset -b codex/multidataset-sync`
2. Build baseline in worktree:
- `dotnet build CBDDC.slnx`
3. Capture baseline tests (save output artifact in worktree):
- `dotnet test CBDDC.slnx`
Deliverable:
1. Clean baseline build/test result captured before changes.
## 1. Design and Contract Layer
### Code Changes
1. Add dataset contracts in `src/ZB.MOM.WW.CBDDC.Core`:
- `DatasetId` value object or constants.
- `DatasetSyncOptions`.
- `IDatasetSyncContext`/`IMultiDatasetSyncOrchestrator`.
2. Extend domain models where sync identity is required:
- `OplogEntry` add `DatasetId` (constructor defaults to `"primary"`).
- Any metadata types used for causal state/pruning that need dataset partitioning.
3. Extend store interfaces (minimally invasive):
- Keep existing methods as compatibility overloads.
- Add dataset-aware variants where cross-dataset ambiguity exists.
### Test Work
1. Add Core unit tests:
- `OplogEntry` hash stability with `DatasetId`.
- Defaulting behavior to `"primary"`.
- Equality/serialization behavior for dataset-aware records.
2. Update existing Core tests that construct `OplogEntry` directly.
Exit Criteria:
1. Core tests compile and pass with default dataset behavior unchanged.
## 2. Persistence Partitioning (Surreal)
### Code Changes
1. Add dataset partition key to persistence records:
- Oplog rows.
- Document metadata rows.
- Snapshot metadata rows (if used in dataset-scoped recoveries).
- Peer confirmation records.
- CDC checkpoints (consumer id should include dataset id or add dedicated field).
2. Update schema initializer:
- Add `datasetId` fields and composite indexes (`datasetId + existing key dimensions`).
3. Update queries in all Surreal stores:
- Enforce dataset filter in every select/update/delete path.
- Guard against full-table scans that omit dataset filter.
4. Add migration/read fallback:
- If `datasetId` missing on older records, treat as `"primary"` during transitional reads.
### Test Work
1. Extend `SurrealStoreContractTests`:
- Write records in two datasets and verify strict isolation.
- Verify prune/merge/export/import scoped by dataset.
2. Add regression tests:
- Legacy records without `datasetId` load as `"primary"` only.
3. Update durability tests:
- CDC checkpoints do not collide between datasets.
Exit Criteria:
1. Persistence tests prove no cross-dataset reads/writes.
## 3. Network Protocol Dataset Awareness
### Code Changes
1. Update `sync.proto` (backward compatible):
- Add `dataset_id` to `HandshakeRequest`, `HandshakeResponse`, `PullChangesRequest`, `PushChangesRequest`, and optionally snapshot requests.
2. Regenerate protocol classes and adapt transport handlers:
- `TcpPeerClient` sends dataset id for every dataset pipeline.
- `TcpSyncServer` routes requests to correct dataset context.
3. Defaulting rules:
- Missing/empty `dataset_id` => `"primary"`.
4. Add explicit rejection semantics:
- If remote peer does not support requested dataset, return accepted handshake but with dataset capability mismatch response path (or reject per dataset connection).
### Test Work
1. Add protocol-level unit tests:
- Message parse/serialize with and without dataset field.
2. Update network tests:
- Handshake stores remote interests per dataset.
- Pull/push operations do not cross datasets.
- Backward compatibility with no dataset id present.
Exit Criteria:
1. Network tests pass for both new and legacy message shapes.
## 4. Multi-Orchestrator Runtime and DI
### Code Changes
1. Add multi-dataset DI registration extensions:
- `AddCBDDCSurrealEmbeddedDataset(...)`
- `AddCBDDCMultiDataset(...)`
2. Build `MultiDatasetSyncOrchestrator`:
- Start/stop orchestrators for configured datasets.
- Isolated cancellation tokens, loops, and failure handling per dataset.
3. Ensure hosting services (`CBDDCNodeService`, `TcpSyncServerHostedService`) initialize dataset contexts deterministically.
4. Add per-dataset knobs:
- Sync interval, max entries per cycle, maintenance interval, optional parallelism limits.
### Test Work
1. Add Hosting tests:
- Multiple datasets register/start/stop cleanly.
- Failure in one dataset does not stop others.
2. Add orchestrator tests:
- Scheduling fairness and per-dataset failure backoff isolation.
3. Update `NoOp`/fallback tests for multi-dataset mode.
Exit Criteria:
1. Runtime starts N dataset pipelines with independent lifecycle behavior.
## 5. Snapshot and Recovery Semantics
### Code Changes
1. Define snapshot scope options:
- Per-dataset snapshot and full multi-dataset snapshot.
2. Update snapshot service APIs and implementations to support:
- Export/import/merge by dataset id.
3. Ensure emergency recovery paths in orchestrator are dataset-scoped.
### Test Work
1. Add snapshot tests:
- Replace/merge for one dataset leaves others untouched.
2. Update reconnect regression tests:
- Snapshot-required flow only affects targeted dataset pipeline.
Exit Criteria:
1. Recovery operations preserve dataset isolation.
## 6. Sample App and Developer Experience
### Code Changes
1. Add sample configuration for three datasets:
- `primary`, `logs`, `timeseries`.
2. Implement append-only sample stores for `logs` and `timeseries`.
3. Expose sample CLI commands to emit load independently per dataset.
### Test Work
1. Add sample integration tests:
- Heavy append load on logs/timeseries does not significantly delay primary data convergence.
2. Add benchmark harness cases:
- Single-dataset baseline vs multi-dataset under telemetry load.
Exit Criteria:
1. Demonstrable isolation in sample workload.
## 7. Documentation and Migration Guides
### Code/Docs Changes
1. New doc: `docs/features/multi-dataset-sync.md`.
2. Update:
- `docs/architecture.md`
- `docs/persistence-providers.md`
- `docs/runbook.md`
3. Add migration notes:
- From single pipeline to multi-dataset configuration.
- Backward compatibility and rollout toggles.
### Test Work
1. Doc examples compile check (if applicable).
2. Add config parsing tests for dataset option sections.
Exit Criteria:
1. Operators have explicit rollout and rollback steps.
## 8. Rollout Strategy (Safe Adoption)
1. Feature flags:
- `EnableMultiDatasetSync` (global).
- `EnableDatasetPrimary/Logs/Timeseries`.
2. Rollout sequence:
- Stage 1: Deploy with flag off.
- Stage 2: Enable `primary` only in new runtime path.
- Stage 3: Enable `logs`, then `timeseries`.
3. Observability gates:
- Primary sync latency SLO must remain within threshold before enabling telemetry datasets.
## 9. Test Plan (Comprehensive Coverage Matrix)
## Unit Tests
1. Core model defaults and hash behavior with dataset id.
2. Dataset routing logic in orchestrator dispatcher.
3. Protocol adapters default `dataset_id` to `"primary"` when absent.
4. Persistence query builders always include dataset predicate.
## Integration Tests
1. Surreal stores:
- Same key/collection in different datasets remains isolated.
2. Network:
- Pull/push with mixed datasets never cross-stream.
3. Hosting:
- Independent orchestrator lifecycle and failure isolation.
## E2E Tests
1. Multi-node cluster:
- Primary converges under heavy append-only telemetry load.
2. Snapshot/recovery:
- Dataset-scoped restore preserves other datasets.
3. Backward compatibility:
- Legacy node (no dataset id) interoperates on `"primary"`.
## Non-Functional Tests
1. Throughput and latency benchmarks:
- Compare primary p95 sync lag before/after.
2. Resource isolation:
- CPU/memory pressure from telemetry datasets should not break primary SLO.
## Test Update Checklist (Existing Tests to Modify)
1. `tests/ZB.MOM.WW.CBDDC.Core.Tests`:
- Update direct `OplogEntry` constructions.
2. `tests/ZB.MOM.WW.CBDDC.Network.Tests`:
- Handshake/connection/vector-clock tests for dataset-aware flows.
3. `tests/ZB.MOM.WW.CBDDC.Hosting.Tests`:
- Add multi-dataset startup/shutdown/failure cases.
4. `tests/ZB.MOM.WW.CBDDC.Sample.Console.Tests`:
- Extend Surreal contract and durability tests for dataset partitioning.
5. `tests/ZB.MOM.WW.CBDDC.E2E.Tests`:
- Add multi-dataset convergence + interference tests.
## Worktree Task Breakdown (Execution Order)
1. `Phase-A`: Contracts + Core model updates + unit tests.
2. `Phase-B`: Surreal schema/store partitioning + persistence tests.
3. `Phase-C`: Protocol and network routing + network tests.
4. `Phase-D`: Multi-orchestrator DI/runtime + hosting tests.
5. `Phase-E`: Snapshot/recovery updates + regression tests.
6. `Phase-F`: Sample/bench/docs + end-to-end verification.
Each phase should be committed separately in the worktree to keep reviewable deltas.
## Validation Commands (Run in Worktree)
1. `dotnet build /Users/dohertj2/Desktop/CBDDC/CBDDC.slnx`
2. `dotnet test /Users/dohertj2/Desktop/CBDDC/CBDDC.slnx`
3. Focused suites during implementation:
- `dotnet test /Users/dohertj2/Desktop/CBDDC/tests/ZB.MOM.WW.CBDDC.Core.Tests/ZB.MOM.WW.CBDDC.Core.Tests.csproj`
- `dotnet test /Users/dohertj2/Desktop/CBDDC/tests/ZB.MOM.WW.CBDDC.Network.Tests/ZB.MOM.WW.CBDDC.Network.Tests.csproj`
- `dotnet test /Users/dohertj2/Desktop/CBDDC/tests/ZB.MOM.WW.CBDDC.Hosting.Tests/ZB.MOM.WW.CBDDC.Hosting.Tests.csproj`
- `dotnet test /Users/dohertj2/Desktop/CBDDC/tests/ZB.MOM.WW.CBDDC.Sample.Console.Tests/ZB.MOM.WW.CBDDC.Sample.Console.Tests.csproj`
- `dotnet test /Users/dohertj2/Desktop/CBDDC/tests/ZB.MOM.WW.CBDDC.E2E.Tests/ZB.MOM.WW.CBDDC.E2E.Tests.csproj`
## Definition of Done
1. Multi-dataset mode runs `primary`, `logs`, and `timeseries` in one process with independent sync paths.
2. No cross-dataset data movement in persistence, protocol, or runtime.
3. Single-dataset existing usage still works via default `"primary"` dataset.
4. Added/updated unit, integration, and E2E tests pass in CI.
5. Docs include migration and operational guidance.

View File

@@ -0,0 +1,34 @@
namespace ZB.MOM.WW.CBDDC.Core;
/// <summary>
/// Provides well-known dataset identifiers and normalization helpers.
/// </summary>
public static class DatasetId
{
/// <summary>
/// The default dataset identifier used by legacy single-dataset deployments.
/// </summary>
public const string Primary = "primary";
/// <summary>
/// A high-volume append-only telemetry/log dataset identifier.
/// </summary>
public const string Logs = "logs";
/// <summary>
/// A high-volume append-only timeseries dataset identifier.
/// </summary>
public const string Timeseries = "timeseries";
/// <summary>
/// Normalizes a dataset identifier and applies the default when missing.
/// </summary>
/// <param name="datasetId">The raw dataset identifier.</param>
/// <returns>The normalized dataset identifier.</returns>
public static string Normalize(string? datasetId)
{
return string.IsNullOrWhiteSpace(datasetId)
? Primary
: datasetId.Trim().ToLowerInvariant();
}
}

View File

@@ -0,0 +1,45 @@
using System;
using System.Collections.Generic;
namespace ZB.MOM.WW.CBDDC.Core;
/// <summary>
/// Configures synchronization behavior for a single dataset pipeline.
/// </summary>
public sealed class DatasetSyncOptions
{
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary>
/// Gets or sets a value indicating whether this dataset pipeline is enabled.
/// </summary>
public bool Enabled { get; set; } = true;
/// <summary>
/// Gets or sets the delay between sync loop cycles.
/// </summary>
public TimeSpan SyncLoopDelay { get; set; } = TimeSpan.FromSeconds(2);
/// <summary>
/// Gets or sets the maximum number of peers selected per cycle.
/// </summary>
public int MaxPeersPerCycle { get; set; } = 3;
/// <summary>
/// Gets or sets the maximum number of oplog entries pushed or pulled in one cycle.
/// </summary>
public int? MaxEntriesPerCycle { get; set; }
/// <summary>
/// Gets or sets an optional maintenance interval override for this dataset.
/// </summary>
public TimeSpan? MaintenanceIntervalOverride { get; set; }
/// <summary>
/// Gets or sets collection interests for this dataset pipeline.
/// </summary>
public List<string> InterestingCollections { get; set; } = [];
}

View File

@@ -24,6 +24,8 @@ public static class OplogEntryExtensions
using var sha256 = SHA256.Create(); using var sha256 = SHA256.Create();
var sb = new StringBuilder(); var sb = new StringBuilder();
sb.Append(DatasetId.Normalize(entry.DatasetId));
sb.Append('|');
sb.Append(entry.Collection); sb.Append(entry.Collection);
sb.Append('|'); sb.Append('|');
sb.Append(entry.Key); sb.Append(entry.Key);
@@ -59,9 +61,11 @@ public class OplogEntry
/// <param name="timestamp">The logical timestamp.</param> /// <param name="timestamp">The logical timestamp.</param>
/// <param name="previousHash">The previous entry hash.</param> /// <param name="previousHash">The previous entry hash.</param>
/// <param name="hash">The current entry hash. If null, it is computed.</param> /// <param name="hash">The current entry hash. If null, it is computed.</param>
/// <param name="datasetId">The dataset identifier for this entry. Defaults to <c>primary</c>.</param>
public OplogEntry(string collection, string key, OperationType operation, JsonElement? payload, public OplogEntry(string collection, string key, OperationType operation, JsonElement? payload,
HlcTimestamp timestamp, string previousHash, string? hash = null) HlcTimestamp timestamp, string previousHash, string? hash = null, string? datasetId = null)
{ {
DatasetId = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Normalize(datasetId);
Collection = collection; Collection = collection;
Key = key; Key = key;
Operation = operation; Operation = operation;
@@ -71,6 +75,11 @@ public class OplogEntry
Hash = hash ?? this.ComputeHash(); Hash = hash ?? this.ComputeHash();
} }
/// <summary>
/// Gets the dataset identifier associated with this entry.
/// </summary>
public string DatasetId { get; }
/// <summary> /// <summary>
/// Gets the collection name associated with this entry. /// Gets the collection name associated with this entry.
/// </summary> /// </summary>

View File

@@ -7,6 +7,11 @@ namespace ZB.MOM.WW.CBDDC.Core;
/// </summary> /// </summary>
public class PeerOplogConfirmation public class PeerOplogConfirmation
{ {
/// <summary>
/// Gets or sets the dataset identifier associated with this confirmation record.
/// </summary>
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary> /// <summary>
/// Gets or sets the tracked peer node identifier. /// Gets or sets the tracked peer node identifier.
/// </summary> /// </summary>

View File

@@ -2,6 +2,11 @@ namespace ZB.MOM.WW.CBDDC.Core;
public class SnapshotMetadata public class SnapshotMetadata
{ {
/// <summary>
/// Gets or sets the dataset identifier associated with the snapshot metadata.
/// </summary>
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary> /// <summary>
/// Gets or sets the node identifier associated with the snapshot. /// Gets or sets the node identifier associated with the snapshot.
/// </summary> /// </summary>

View File

@@ -0,0 +1,42 @@
namespace ZB.MOM.WW.CBDDC.Core.Storage;
/// <summary>
/// Represents the storage and runtime contracts bound to a specific dataset pipeline.
/// </summary>
public interface IDatasetSyncContext
{
/// <summary>
/// Gets the dataset identifier represented by this context.
/// </summary>
string DatasetId { get; }
/// <summary>
/// Gets the per-dataset synchronization options.
/// </summary>
DatasetSyncOptions Options { get; }
/// <summary>
/// Gets the dataset-scoped document store.
/// </summary>
IDocumentStore DocumentStore { get; }
/// <summary>
/// Gets the dataset-scoped oplog store.
/// </summary>
IOplogStore OplogStore { get; }
/// <summary>
/// Gets the dataset-scoped snapshot metadata store.
/// </summary>
ISnapshotMetadataStore SnapshotMetadataStore { get; }
/// <summary>
/// Gets the dataset-scoped snapshot service.
/// </summary>
ISnapshotService SnapshotService { get; }
/// <summary>
/// Gets the optional dataset-scoped peer confirmation store.
/// </summary>
IPeerOplogConfirmationStore? PeerOplogConfirmationStore { get; }
}

View File

@@ -20,6 +20,23 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
Task<DocumentMetadata?> GetMetadataAsync(string collection, string key, Task<DocumentMetadata?> GetMetadataAsync(string collection, string key,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Gets metadata for a specific document within a dataset.
/// </summary>
/// <param name="collection">The collection name.</param>
/// <param name="key">The document key.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The matching metadata when found.</returns>
Task<DocumentMetadata?> GetMetadataAsync(
string collection,
string key,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetMetadataAsync(collection, key, cancellationToken);
}
/// <summary> /// <summary>
/// Gets metadata for all documents in a collection. /// Gets metadata for all documents in a collection.
/// </summary> /// </summary>
@@ -29,6 +46,21 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(string collection, Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(string collection,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Gets metadata for all documents in a collection for the specified dataset.
/// </summary>
/// <param name="collection">The collection name.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Enumerable of metadata rows.</returns>
Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(
string collection,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetMetadataByCollectionAsync(collection, cancellationToken);
}
/// <summary> /// <summary>
/// Upserts (inserts or updates) metadata for a document. /// Upserts (inserts or updates) metadata for a document.
/// </summary> /// </summary>
@@ -36,6 +68,20 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
/// <param name="cancellationToken">A cancellation token.</param> /// <param name="cancellationToken">A cancellation token.</param>
Task UpsertMetadataAsync(DocumentMetadata metadata, CancellationToken cancellationToken = default); Task UpsertMetadataAsync(DocumentMetadata metadata, CancellationToken cancellationToken = default);
/// <summary>
/// Upserts metadata for a specific dataset.
/// </summary>
/// <param name="metadata">The metadata to upsert.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task UpsertMetadataAsync(
DocumentMetadata metadata,
string datasetId,
CancellationToken cancellationToken = default)
{
return UpsertMetadataAsync(metadata, cancellationToken);
}
/// <summary> /// <summary>
/// Upserts metadata for multiple documents in batch. /// Upserts metadata for multiple documents in batch.
/// </summary> /// </summary>
@@ -44,6 +90,20 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
Task UpsertMetadataBatchAsync(IEnumerable<DocumentMetadata> metadatas, Task UpsertMetadataBatchAsync(IEnumerable<DocumentMetadata> metadatas,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Upserts metadata batch for a specific dataset.
/// </summary>
/// <param name="metadatas">The metadata items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task UpsertMetadataBatchAsync(
IEnumerable<DocumentMetadata> metadatas,
string datasetId,
CancellationToken cancellationToken = default)
{
return UpsertMetadataBatchAsync(metadatas, cancellationToken);
}
/// <summary> /// <summary>
/// Marks a document as deleted by setting IsDeleted=true and updating the timestamp. /// Marks a document as deleted by setting IsDeleted=true and updating the timestamp.
/// </summary> /// </summary>
@@ -54,6 +114,24 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
Task MarkDeletedAsync(string collection, string key, HlcTimestamp timestamp, Task MarkDeletedAsync(string collection, string key, HlcTimestamp timestamp,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Marks a document as deleted in a specific dataset.
/// </summary>
/// <param name="collection">The collection name.</param>
/// <param name="key">The document key.</param>
/// <param name="timestamp">The deletion timestamp.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task MarkDeletedAsync(
string collection,
string key,
HlcTimestamp timestamp,
string datasetId,
CancellationToken cancellationToken = default)
{
return MarkDeletedAsync(collection, key, timestamp, cancellationToken);
}
/// <summary> /// <summary>
/// Gets all document metadata with timestamps after the specified timestamp. /// Gets all document metadata with timestamps after the specified timestamp.
/// Used for incremental sync to find documents modified since last sync. /// Used for incremental sync to find documents modified since last sync.
@@ -64,6 +142,23 @@ public interface IDocumentMetadataStore : ISnapshotable<DocumentMetadata>
/// <returns>Documents modified after the specified timestamp.</returns> /// <returns>Documents modified after the specified timestamp.</returns>
Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(HlcTimestamp since, Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(HlcTimestamp since,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default); IEnumerable<string>? collections = null, CancellationToken cancellationToken = default);
/// <summary>
/// Gets document metadata modified after a timestamp for a specific dataset.
/// </summary>
/// <param name="since">The lower-bound timestamp.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="collections">Optional collection filter.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Documents modified after the specified timestamp.</returns>
Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(
HlcTimestamp since,
string datasetId,
IEnumerable<string>? collections = null,
CancellationToken cancellationToken = default)
{
return GetMetadataAfterAsync(since, collections, cancellationToken);
}
} }
/// <summary> /// <summary>
@@ -85,14 +180,22 @@ public class DocumentMetadata
/// <param name="key">The document key.</param> /// <param name="key">The document key.</param>
/// <param name="updatedAt">The last update timestamp.</param> /// <param name="updatedAt">The last update timestamp.</param>
/// <param name="isDeleted">Whether the document is marked as deleted.</param> /// <param name="isDeleted">Whether the document is marked as deleted.</param>
public DocumentMetadata(string collection, string key, HlcTimestamp updatedAt, bool isDeleted = false) /// <param name="datasetId">The dataset identifier. Defaults to <c>primary</c>.</param>
public DocumentMetadata(string collection, string key, HlcTimestamp updatedAt, bool isDeleted = false,
string? datasetId = null)
{ {
Collection = collection; Collection = collection;
Key = key; Key = key;
UpdatedAt = updatedAt; UpdatedAt = updatedAt;
IsDeleted = isDeleted; IsDeleted = isDeleted;
DatasetId = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Normalize(datasetId);
} }
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary> /// <summary>
/// Gets or sets the collection name. /// Gets or sets the collection name.
/// </summary> /// </summary>

View File

@@ -0,0 +1,25 @@
using System.Collections.Generic;
using System.Threading.Tasks;
namespace ZB.MOM.WW.CBDDC.Core.Storage;
/// <summary>
/// Coordinates the lifecycle of multiple dataset synchronization pipelines.
/// </summary>
public interface IMultiDatasetSyncOrchestrator
{
/// <summary>
/// Gets the registered dataset contexts.
/// </summary>
IReadOnlyCollection<IDatasetSyncContext> Contexts { get; }
/// <summary>
/// Starts synchronization for all configured datasets.
/// </summary>
Task Start();
/// <summary>
/// Stops synchronization for all configured datasets.
/// </summary>
Task Stop();
}

View File

@@ -23,6 +23,21 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task that represents the asynchronous append operation.</returns> /// <returns>A task that represents the asynchronous append operation.</returns>
Task AppendOplogEntryAsync(OplogEntry entry, CancellationToken cancellationToken = default); Task AppendOplogEntryAsync(OplogEntry entry, CancellationToken cancellationToken = default);
/// <summary>
/// Appends a new entry to the operation log asynchronously for a specific dataset.
/// </summary>
/// <param name="entry">The operation log entry to append.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous append operation.</returns>
Task AppendOplogEntryAsync(
OplogEntry entry,
string datasetId,
CancellationToken cancellationToken = default)
{
return AppendOplogEntryAsync(entry, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves all oplog entries that occurred after the specified timestamp. /// Asynchronously retrieves all oplog entries that occurred after the specified timestamp.
/// </summary> /// </summary>
@@ -33,6 +48,23 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(HlcTimestamp timestamp, IEnumerable<string>? collections = null, Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(HlcTimestamp timestamp, IEnumerable<string>? collections = null,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves oplog entries after the specified timestamp for a specific dataset.
/// </summary>
/// <param name="timestamp">The lower-bound timestamp.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="collections">Optional collection filter.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing matching oplog entries.</returns>
Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(
HlcTimestamp timestamp,
string datasetId,
IEnumerable<string>? collections = null,
CancellationToken cancellationToken = default)
{
return GetOplogAfterAsync(timestamp, collections, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves the latest observed hybrid logical clock (HLC) timestamp. /// Asynchronously retrieves the latest observed hybrid logical clock (HLC) timestamp.
/// </summary> /// </summary>
@@ -40,6 +72,17 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task that represents the asynchronous operation containing the latest HLC timestamp.</returns> /// <returns>A task that represents the asynchronous operation containing the latest HLC timestamp.</returns>
Task<HlcTimestamp> GetLatestTimestampAsync(CancellationToken cancellationToken = default); Task<HlcTimestamp> GetLatestTimestampAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves the latest observed timestamp for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing the latest timestamp.</returns>
Task<HlcTimestamp> GetLatestTimestampAsync(string datasetId, CancellationToken cancellationToken = default)
{
return GetLatestTimestampAsync(cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves the current vector clock representing the state of distributed events. /// Asynchronously retrieves the current vector clock representing the state of distributed events.
/// </summary> /// </summary>
@@ -47,6 +90,17 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task that represents the asynchronous operation containing the current vector clock.</returns> /// <returns>A task that represents the asynchronous operation containing the current vector clock.</returns>
Task<VectorClock> GetVectorClockAsync(CancellationToken cancellationToken = default); Task<VectorClock> GetVectorClockAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves the vector clock for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing the vector clock.</returns>
Task<VectorClock> GetVectorClockAsync(string datasetId, CancellationToken cancellationToken = default)
{
return GetVectorClockAsync(cancellationToken);
}
/// <summary> /// <summary>
/// Retrieves a collection of oplog entries for the specified node that occurred after the given timestamp. /// Retrieves a collection of oplog entries for the specified node that occurred after the given timestamp.
/// </summary> /// </summary>
@@ -58,6 +112,25 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(string nodeId, HlcTimestamp since, Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(string nodeId, HlcTimestamp since,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default); IEnumerable<string>? collections = null, CancellationToken cancellationToken = default);
/// <summary>
/// Retrieves oplog entries for the specified node and dataset after the provided timestamp.
/// </summary>
/// <param name="nodeId">The node identifier.</param>
/// <param name="since">The lower-bound timestamp.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="collections">Optional collection filter.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing matching oplog entries.</returns>
Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(
string nodeId,
HlcTimestamp since,
string datasetId,
IEnumerable<string>? collections = null,
CancellationToken cancellationToken = default)
{
return GetOplogForNodeAfterAsync(nodeId, since, collections, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves the hash of the most recent entry for the specified node. /// Asynchronously retrieves the hash of the most recent entry for the specified node.
/// </summary> /// </summary>
@@ -69,6 +142,18 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task that represents the asynchronous operation containing the hash string of the last entry or null.</returns> /// <returns>A task that represents the asynchronous operation containing the hash string of the last entry or null.</returns>
Task<string?> GetLastEntryHashAsync(string nodeId, CancellationToken cancellationToken = default); Task<string?> GetLastEntryHashAsync(string nodeId, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves the last entry hash for a node within a specific dataset.
/// </summary>
/// <param name="nodeId">The node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing the last hash or null.</returns>
Task<string?> GetLastEntryHashAsync(string nodeId, string datasetId, CancellationToken cancellationToken = default)
{
return GetLastEntryHashAsync(nodeId, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves a sequence of oplog entries representing the chain between the specified start and end /// Asynchronously retrieves a sequence of oplog entries representing the chain between the specified start and end
/// hashes. /// hashes.
@@ -80,6 +165,23 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
Task<IEnumerable<OplogEntry>> GetChainRangeAsync(string startHash, string endHash, Task<IEnumerable<OplogEntry>> GetChainRangeAsync(string startHash, string endHash,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves a chain range for a specific dataset.
/// </summary>
/// <param name="startHash">The start hash.</param>
/// <param name="endHash">The end hash.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing chain entries.</returns>
Task<IEnumerable<OplogEntry>> GetChainRangeAsync(
string startHash,
string endHash,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetChainRangeAsync(startHash, endHash, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves the oplog entry associated with the specified hash value. /// Asynchronously retrieves the oplog entry associated with the specified hash value.
/// </summary> /// </summary>
@@ -88,6 +190,18 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task representing the asynchronous operation containing the OplogEntry if found, otherwise null.</returns> /// <returns>A task representing the asynchronous operation containing the OplogEntry if found, otherwise null.</returns>
Task<OplogEntry?> GetEntryByHashAsync(string hash, CancellationToken cancellationToken = default); Task<OplogEntry?> GetEntryByHashAsync(string hash, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves an entry by hash for a specific dataset.
/// </summary>
/// <param name="hash">The entry hash.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task containing the entry when found.</returns>
Task<OplogEntry?> GetEntryByHashAsync(string hash, string datasetId, CancellationToken cancellationToken = default)
{
return GetEntryByHashAsync(hash, cancellationToken);
}
/// <summary> /// <summary>
/// Applies a batch of oplog entries asynchronously to the target data store. /// Applies a batch of oplog entries asynchronously to the target data store.
/// </summary> /// </summary>
@@ -96,6 +210,21 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <returns>A task that represents the asynchronous batch apply operation.</returns> /// <returns>A task that represents the asynchronous batch apply operation.</returns>
Task ApplyBatchAsync(IEnumerable<OplogEntry> oplogEntries, CancellationToken cancellationToken = default); Task ApplyBatchAsync(IEnumerable<OplogEntry> oplogEntries, CancellationToken cancellationToken = default);
/// <summary>
/// Applies a batch of oplog entries asynchronously for a specific dataset.
/// </summary>
/// <param name="oplogEntries">The entries to apply.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the apply operation.</returns>
Task ApplyBatchAsync(
IEnumerable<OplogEntry> oplogEntries,
string datasetId,
CancellationToken cancellationToken = default)
{
return ApplyBatchAsync(oplogEntries, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously removes entries from the oplog that are older than the specified cutoff timestamp. /// Asynchronously removes entries from the oplog that are older than the specified cutoff timestamp.
/// </summary> /// </summary>
@@ -103,4 +232,16 @@ public interface IOplogStore : ISnapshotable<OplogEntry>
/// <param name="cancellationToken">A cancellation token that can be used to cancel the prune operation.</param> /// <param name="cancellationToken">A cancellation token that can be used to cancel the prune operation.</param>
/// <returns>A task that represents the asynchronous prune operation.</returns> /// <returns>A task that represents the asynchronous prune operation.</returns>
Task PruneOplogAsync(HlcTimestamp cutoff, CancellationToken cancellationToken = default); Task PruneOplogAsync(HlcTimestamp cutoff, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously removes entries from the specified dataset oplog that are older than the cutoff.
/// </summary>
/// <param name="cutoff">The prune cutoff timestamp.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the prune operation.</returns>
Task PruneOplogAsync(HlcTimestamp cutoff, string datasetId, CancellationToken cancellationToken = default)
{
return PruneOplogAsync(cutoff, cancellationToken);
}
} }

View File

@@ -23,6 +23,24 @@ public interface IPeerOplogConfirmationStore : ISnapshotable<PeerOplogConfirmati
PeerType type, PeerType type,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Ensures the specified peer is tracked for a given dataset.
/// </summary>
/// <param name="peerNodeId">The peer node identifier.</param>
/// <param name="address">The peer address.</param>
/// <param name="type">The peer type.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task EnsurePeerRegisteredAsync(
string peerNodeId,
string address,
PeerType type,
string datasetId,
CancellationToken cancellationToken = default)
{
return EnsurePeerRegisteredAsync(peerNodeId, address, type, cancellationToken);
}
/// <summary> /// <summary>
/// Updates the confirmation watermark for a tracked peer and source node. /// Updates the confirmation watermark for a tracked peer and source node.
/// </summary> /// </summary>
@@ -38,6 +56,26 @@ public interface IPeerOplogConfirmationStore : ISnapshotable<PeerOplogConfirmati
string hash, string hash,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Updates the confirmation watermark for a tracked peer and dataset.
/// </summary>
/// <param name="peerNodeId">The tracked peer node identifier.</param>
/// <param name="sourceNodeId">The source node identifier.</param>
/// <param name="timestamp">The confirmed timestamp.</param>
/// <param name="hash">The confirmed hash.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task UpdateConfirmationAsync(
string peerNodeId,
string sourceNodeId,
HlcTimestamp timestamp,
string hash,
string datasetId,
CancellationToken cancellationToken = default)
{
return UpdateConfirmationAsync(peerNodeId, sourceNodeId, timestamp, hash, cancellationToken);
}
/// <summary> /// <summary>
/// Gets all persisted peer confirmations. /// Gets all persisted peer confirmations.
/// </summary> /// </summary>
@@ -45,6 +83,19 @@ public interface IPeerOplogConfirmationStore : ISnapshotable<PeerOplogConfirmati
/// <returns>All peer confirmations.</returns> /// <returns>All peer confirmations.</returns>
Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync(CancellationToken cancellationToken = default); Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Gets all persisted peer confirmations for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>All peer confirmations in the dataset.</returns>
Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
return GetConfirmationsAsync(cancellationToken);
}
/// <summary> /// <summary>
/// Gets persisted confirmations for a specific tracked peer. /// Gets persisted confirmations for a specific tracked peer.
/// </summary> /// </summary>
@@ -55,6 +106,21 @@ public interface IPeerOplogConfirmationStore : ISnapshotable<PeerOplogConfirmati
string peerNodeId, string peerNodeId,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Gets persisted confirmations for a specific peer and dataset.
/// </summary>
/// <param name="peerNodeId">The peer node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Peer confirmations for the requested peer and dataset.</returns>
Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsForPeerAsync(
string peerNodeId,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetConfirmationsForPeerAsync(peerNodeId, cancellationToken);
}
/// <summary> /// <summary>
/// Deactivates tracking for the specified peer. /// Deactivates tracking for the specified peer.
/// </summary> /// </summary>
@@ -62,10 +128,34 @@ public interface IPeerOplogConfirmationStore : ISnapshotable<PeerOplogConfirmati
/// <param name="cancellationToken">A cancellation token.</param> /// <param name="cancellationToken">A cancellation token.</param>
Task RemovePeerTrackingAsync(string peerNodeId, CancellationToken cancellationToken = default); Task RemovePeerTrackingAsync(string peerNodeId, CancellationToken cancellationToken = default);
/// <summary>
/// Deactivates tracking for a peer in a specific dataset.
/// </summary>
/// <param name="peerNodeId">The peer node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task RemovePeerTrackingAsync(string peerNodeId, string datasetId, CancellationToken cancellationToken = default)
{
return RemovePeerTrackingAsync(peerNodeId, cancellationToken);
}
/// <summary> /// <summary>
/// Gets all active tracked peer identifiers. /// Gets all active tracked peer identifiers.
/// </summary> /// </summary>
/// <param name="cancellationToken">A cancellation token.</param> /// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Distinct active tracked peer identifiers.</returns> /// <returns>Distinct active tracked peer identifiers.</returns>
Task<IEnumerable<string>> GetActiveTrackedPeersAsync(CancellationToken cancellationToken = default); Task<IEnumerable<string>> GetActiveTrackedPeersAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Gets active tracked peers for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Distinct active peer identifiers.</returns>
Task<IEnumerable<string>> GetActiveTrackedPeersAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
return GetActiveTrackedPeersAsync(cancellationToken);
}
} }

View File

@@ -20,6 +20,21 @@ public interface ISnapshotMetadataStore : ISnapshotable<SnapshotMetadata>
/// </returns> /// </returns>
Task<SnapshotMetadata?> GetSnapshotMetadataAsync(string nodeId, CancellationToken cancellationToken = default); Task<SnapshotMetadata?> GetSnapshotMetadataAsync(string nodeId, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves snapshot metadata for a node within a specific dataset.
/// </summary>
/// <param name="nodeId">The node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The snapshot metadata when available.</returns>
Task<SnapshotMetadata?> GetSnapshotMetadataAsync(
string nodeId,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetSnapshotMetadataAsync(nodeId, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously inserts the specified snapshot metadata into the data store. /// Asynchronously inserts the specified snapshot metadata into the data store.
/// </summary> /// </summary>
@@ -28,6 +43,20 @@ public interface ISnapshotMetadataStore : ISnapshotable<SnapshotMetadata>
/// <returns>A task that represents the asynchronous insert operation.</returns> /// <returns>A task that represents the asynchronous insert operation.</returns>
Task InsertSnapshotMetadataAsync(SnapshotMetadata metadata, CancellationToken cancellationToken = default); Task InsertSnapshotMetadataAsync(SnapshotMetadata metadata, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously inserts snapshot metadata for a specific dataset.
/// </summary>
/// <param name="metadata">The metadata to insert.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task InsertSnapshotMetadataAsync(
SnapshotMetadata metadata,
string datasetId,
CancellationToken cancellationToken = default)
{
return InsertSnapshotMetadataAsync(metadata, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously updates the metadata for an existing snapshot. /// Asynchronously updates the metadata for an existing snapshot.
/// </summary> /// </summary>
@@ -36,6 +65,20 @@ public interface ISnapshotMetadataStore : ISnapshotable<SnapshotMetadata>
/// <returns>A task that represents the asynchronous update operation.</returns> /// <returns>A task that represents the asynchronous update operation.</returns>
Task UpdateSnapshotMetadataAsync(SnapshotMetadata existingMeta, CancellationToken cancellationToken = default); Task UpdateSnapshotMetadataAsync(SnapshotMetadata existingMeta, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously updates snapshot metadata for a specific dataset.
/// </summary>
/// <param name="existingMeta">The metadata to update.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task UpdateSnapshotMetadataAsync(
SnapshotMetadata existingMeta,
string datasetId,
CancellationToken cancellationToken = default)
{
return UpdateSnapshotMetadataAsync(existingMeta, cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously retrieves the hash of the current snapshot for the specified node. /// Asynchronously retrieves the hash of the current snapshot for the specified node.
/// </summary> /// </summary>
@@ -44,10 +87,38 @@ public interface ISnapshotMetadataStore : ISnapshotable<SnapshotMetadata>
/// <returns>A task containing the snapshot hash as a string, or null if no snapshot is available.</returns> /// <returns>A task containing the snapshot hash as a string, or null if no snapshot is available.</returns>
Task<string?> GetSnapshotHashAsync(string nodeId, CancellationToken cancellationToken = default); Task<string?> GetSnapshotHashAsync(string nodeId, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously retrieves the snapshot hash for a node within a specific dataset.
/// </summary>
/// <param name="nodeId">The node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The snapshot hash when available.</returns>
Task<string?> GetSnapshotHashAsync(
string nodeId,
string datasetId,
CancellationToken cancellationToken = default)
{
return GetSnapshotHashAsync(nodeId, cancellationToken);
}
/// <summary> /// <summary>
/// Gets all snapshot metadata entries. Used for initializing VectorClock cache. /// Gets all snapshot metadata entries. Used for initializing VectorClock cache.
/// </summary> /// </summary>
/// <param name="cancellationToken">A cancellation token.</param> /// <param name="cancellationToken">A cancellation token.</param>
/// <returns>All snapshot metadata entries.</returns> /// <returns>All snapshot metadata entries.</returns>
Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync(CancellationToken cancellationToken = default); Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Gets all snapshot metadata entries for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>All snapshot metadata entries for the dataset.</returns>
Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
return GetAllSnapshotMetadataAsync(cancellationToken);
}
} }

View File

@@ -17,6 +17,18 @@ public interface ISnapshotService
/// <returns>A task that represents the asynchronous snapshot creation operation.</returns> /// <returns>A task that represents the asynchronous snapshot creation operation.</returns>
Task CreateSnapshotAsync(Stream destination, CancellationToken cancellationToken = default); Task CreateSnapshotAsync(Stream destination, CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously creates a snapshot scoped to a specific dataset and writes it to the destination stream.
/// </summary>
/// <param name="destination">The stream receiving serialized snapshot content.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous snapshot creation operation.</returns>
Task CreateSnapshotAsync(Stream destination, string datasetId, CancellationToken cancellationToken = default)
{
return CreateSnapshotAsync(destination, cancellationToken);
}
/// <summary> /// <summary>
/// Replaces the existing database with the contents provided in the specified stream asynchronously. /// Replaces the existing database with the contents provided in the specified stream asynchronously.
/// </summary> /// </summary>
@@ -25,6 +37,18 @@ public interface ISnapshotService
/// <returns>A task that represents the asynchronous database replacement operation.</returns> /// <returns>A task that represents the asynchronous database replacement operation.</returns>
Task ReplaceDatabaseAsync(Stream databaseStream, CancellationToken cancellationToken = default); Task ReplaceDatabaseAsync(Stream databaseStream, CancellationToken cancellationToken = default);
/// <summary>
/// Replaces data for a specific dataset with the contents provided in the stream.
/// </summary>
/// <param name="databaseStream">The stream containing replacement data.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous replace operation.</returns>
Task ReplaceDatabaseAsync(Stream databaseStream, string datasetId, CancellationToken cancellationToken = default)
{
return ReplaceDatabaseAsync(databaseStream, cancellationToken);
}
/// <summary> /// <summary>
/// Merges the provided snapshot stream into the current data store asynchronously. /// Merges the provided snapshot stream into the current data store asynchronously.
/// </summary> /// </summary>
@@ -32,4 +56,16 @@ public interface ISnapshotService
/// <param name="cancellationToken">A cancellation token that can be used to cancel the merge operation.</param> /// <param name="cancellationToken">A cancellation token that can be used to cancel the merge operation.</param>
/// <returns>A task that represents the asynchronous merge operation.</returns> /// <returns>A task that represents the asynchronous merge operation.</returns>
Task MergeSnapshotAsync(Stream snapshotStream, CancellationToken cancellationToken = default); Task MergeSnapshotAsync(Stream snapshotStream, CancellationToken cancellationToken = default);
/// <summary>
/// Merges a snapshot stream into a specific dataset asynchronously.
/// </summary>
/// <param name="snapshotStream">The stream containing snapshot data.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous merge operation.</returns>
Task MergeSnapshotAsync(Stream snapshotStream, string datasetId, CancellationToken cancellationToken = default)
{
return MergeSnapshotAsync(snapshotStream, cancellationToken);
}
} }

View File

@@ -17,6 +17,17 @@ public interface ISnapshotable<T>
/// <returns>A task that represents the asynchronous drop operation.</returns> /// <returns>A task that represents the asynchronous drop operation.</returns>
Task DropAsync(CancellationToken cancellationToken = default); Task DropAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously deletes data for the specified dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous drop operation.</returns>
Task DropAsync(string datasetId, CancellationToken cancellationToken = default)
{
return DropAsync(cancellationToken);
}
/// <summary> /// <summary>
/// Asynchronously exports a collection of items of type T. /// Asynchronously exports a collection of items of type T.
/// </summary> /// </summary>
@@ -27,6 +38,17 @@ public interface ISnapshotable<T>
/// </returns> /// </returns>
Task<IEnumerable<T>> ExportAsync(CancellationToken cancellationToken = default); Task<IEnumerable<T>> ExportAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Asynchronously exports items for the specified dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>An enumerable collection of exported items.</returns>
Task<IEnumerable<T>> ExportAsync(string datasetId, CancellationToken cancellationToken = default)
{
return ExportAsync(cancellationToken);
}
/// <summary> /// <summary>
/// Imports the specified collection of items asynchronously. /// Imports the specified collection of items asynchronously.
/// </summary> /// </summary>
@@ -35,6 +57,18 @@ public interface ISnapshotable<T>
/// <returns>A task that represents the asynchronous import operation.</returns> /// <returns>A task that represents the asynchronous import operation.</returns>
Task ImportAsync(IEnumerable<T> items, CancellationToken cancellationToken = default); Task ImportAsync(IEnumerable<T> items, CancellationToken cancellationToken = default);
/// <summary>
/// Imports items into the specified dataset asynchronously.
/// </summary>
/// <param name="items">The items to import.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous import operation.</returns>
Task ImportAsync(IEnumerable<T> items, string datasetId, CancellationToken cancellationToken = default)
{
return ImportAsync(items, cancellationToken);
}
/// <summary> /// <summary>
/// Merges the specified collection of items into the target data store asynchronously. /// Merges the specified collection of items into the target data store asynchronously.
/// </summary> /// </summary>
@@ -47,4 +81,16 @@ public interface ISnapshotable<T>
/// <param name="cancellationToken">A cancellation token that can be used to cancel the merge operation.</param> /// <param name="cancellationToken">A cancellation token that can be used to cancel the merge operation.</param>
/// <returns>A task that represents the asynchronous merge operation.</returns> /// <returns>A task that represents the asynchronous merge operation.</returns>
Task MergeAsync(IEnumerable<T> items, CancellationToken cancellationToken = default); Task MergeAsync(IEnumerable<T> items, CancellationToken cancellationToken = default);
/// <summary>
/// Merges items into the specified dataset asynchronously.
/// </summary>
/// <param name="items">The items to merge.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>A task that represents the asynchronous merge operation.</returns>
Task MergeAsync(IEnumerable<T> items, string datasetId, CancellationToken cancellationToken = default)
{
return MergeAsync(items, cancellationToken);
}
} }

View File

@@ -0,0 +1,67 @@
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Storage;
namespace ZB.MOM.WW.CBDDC.Network;
/// <summary>
/// Default dataset sync context implementation used by the runtime coordinator.
/// </summary>
public sealed class DatasetSyncContext : IDatasetSyncContext
{
/// <summary>
/// Initializes a new instance of the <see cref="DatasetSyncContext" /> class.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="options">The dataset options.</param>
/// <param name="documentStore">The document store.</param>
/// <param name="oplogStore">The oplog store.</param>
/// <param name="snapshotMetadataStore">The snapshot metadata store.</param>
/// <param name="snapshotService">The snapshot service.</param>
/// <param name="peerOplogConfirmationStore">The optional peer confirmation store.</param>
/// <param name="orchestrator">The dataset orchestrator.</param>
public DatasetSyncContext(
string datasetId,
DatasetSyncOptions options,
IDocumentStore documentStore,
IOplogStore oplogStore,
ISnapshotMetadataStore snapshotMetadataStore,
ISnapshotService snapshotService,
IPeerOplogConfirmationStore? peerOplogConfirmationStore,
ISyncOrchestrator orchestrator)
{
DatasetId = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Normalize(datasetId);
Options = options;
DocumentStore = documentStore;
OplogStore = oplogStore;
SnapshotMetadataStore = snapshotMetadataStore;
SnapshotService = snapshotService;
PeerOplogConfirmationStore = peerOplogConfirmationStore;
Orchestrator = orchestrator;
}
/// <inheritdoc />
public string DatasetId { get; }
/// <inheritdoc />
public DatasetSyncOptions Options { get; }
/// <inheritdoc />
public IDocumentStore DocumentStore { get; }
/// <inheritdoc />
public IOplogStore OplogStore { get; }
/// <inheritdoc />
public ISnapshotMetadataStore SnapshotMetadataStore { get; }
/// <inheritdoc />
public ISnapshotService SnapshotService { get; }
/// <inheritdoc />
public IPeerOplogConfirmationStore? PeerOplogConfirmationStore { get; }
/// <summary>
/// Gets the orchestrator instance for this dataset context.
/// </summary>
public ISyncOrchestrator Orchestrator { get; }
}

View File

@@ -0,0 +1,34 @@
using ZB.MOM.WW.CBDDC.Core;
namespace ZB.MOM.WW.CBDDC.Network;
/// <summary>
/// Feature flags and defaults for multi-dataset sync runtime activation.
/// </summary>
public sealed class MultiDatasetRuntimeOptions
{
/// <summary>
/// Gets or sets a value indicating whether multi-dataset sync runtime is enabled.
/// </summary>
public bool EnableMultiDatasetSync { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the primary dataset pipeline is enabled.
/// </summary>
public bool EnableDatasetPrimary { get; set; } = true;
/// <summary>
/// Gets or sets a value indicating whether the logs dataset pipeline is enabled.
/// </summary>
public bool EnableDatasetLogs { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the timeseries dataset pipeline is enabled.
/// </summary>
public bool EnableDatasetTimeseries { get; set; }
/// <summary>
/// Gets or sets additional dataset-specific runtime options.
/// </summary>
public List<DatasetSyncOptions> AdditionalDatasets { get; set; } = [];
}

View File

@@ -0,0 +1,199 @@
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Core.Storage;
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.CBDDC.Network.Security;
using ZB.MOM.WW.CBDDC.Network.Telemetry;
namespace ZB.MOM.WW.CBDDC.Network;
/// <summary>
/// Coordinates one sync orchestrator per dataset in-process.
/// </summary>
public sealed class MultiDatasetSyncOrchestrator : IMultiDatasetSyncOrchestrator
{
private readonly ILoggerFactory _loggerFactory;
private readonly ILogger<MultiDatasetSyncOrchestrator> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="MultiDatasetSyncOrchestrator" /> class.
/// </summary>
/// <param name="discovery">The discovery service.</param>
/// <param name="oplogStore">The shared oplog store supporting dataset partitioning.</param>
/// <param name="documentStore">The document store.</param>
/// <param name="snapshotMetadataStore">The snapshot metadata store.</param>
/// <param name="snapshotService">The snapshot service.</param>
/// <param name="peerNodeConfigurationProvider">The peer configuration provider.</param>
/// <param name="loggerFactory">The logger factory.</param>
/// <param name="registeredDatasetOptions">Registered dataset options from DI.</param>
/// <param name="runtimeOptions">Runtime feature flags.</param>
/// <param name="peerOplogConfirmationStore">Optional peer confirmation store.</param>
/// <param name="handshakeService">Optional handshake service.</param>
/// <param name="telemetry">Optional telemetry service.</param>
/// <param name="oplogPruneCutoffCalculator">Optional prune cutoff calculator.</param>
/// <param name="orchestratorFactory">Optional factory override for dataset orchestrators (used by tests).</param>
public MultiDatasetSyncOrchestrator(
IDiscoveryService discovery,
IOplogStore oplogStore,
IDocumentStore documentStore,
ISnapshotMetadataStore snapshotMetadataStore,
ISnapshotService snapshotService,
IPeerNodeConfigurationProvider peerNodeConfigurationProvider,
ILoggerFactory loggerFactory,
IEnumerable<DatasetSyncOptions> registeredDatasetOptions,
MultiDatasetRuntimeOptions runtimeOptions,
IPeerOplogConfirmationStore? peerOplogConfirmationStore = null,
IPeerHandshakeService? handshakeService = null,
INetworkTelemetryService? telemetry = null,
IOplogPruneCutoffCalculator? oplogPruneCutoffCalculator = null,
Func<DatasetSyncOptions, ISyncOrchestrator>? orchestratorFactory = null)
{
_loggerFactory = loggerFactory;
_logger = loggerFactory.CreateLogger<MultiDatasetSyncOrchestrator>();
List<DatasetSyncOptions> datasets = BuildDatasetList(registeredDatasetOptions, runtimeOptions);
Contexts = datasets
.Select(options =>
{
var orchestrator = orchestratorFactory?.Invoke(options) ?? new SyncOrchestrator(
discovery,
oplogStore,
documentStore,
snapshotMetadataStore,
snapshotService,
peerNodeConfigurationProvider,
loggerFactory,
peerOplogConfirmationStore,
handshakeService,
telemetry,
oplogPruneCutoffCalculator,
options);
return (IDatasetSyncContext)new DatasetSyncContext(
options.DatasetId,
options,
documentStore,
oplogStore,
snapshotMetadataStore,
snapshotService,
peerOplogConfirmationStore,
orchestrator);
})
.ToList();
}
/// <inheritdoc />
public IReadOnlyCollection<IDatasetSyncContext> Contexts { get; }
/// <inheritdoc />
public async Task Start()
{
foreach (var context in Contexts)
{
if (!context.Options.Enabled) continue;
try
{
var runtimeContext = (DatasetSyncContext)context;
await runtimeContext.Orchestrator.Start();
_logger.LogInformation("Started dataset orchestrator: {DatasetId}", context.DatasetId);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to start dataset orchestrator: {DatasetId}", context.DatasetId);
}
}
}
/// <inheritdoc />
public async Task Stop()
{
foreach (var context in Contexts)
try
{
var runtimeContext = (DatasetSyncContext)context;
await runtimeContext.Orchestrator.Stop();
_logger.LogInformation("Stopped dataset orchestrator: {DatasetId}", context.DatasetId);
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to stop dataset orchestrator: {DatasetId}", context.DatasetId);
}
}
private static List<DatasetSyncOptions> BuildDatasetList(
IEnumerable<DatasetSyncOptions> registeredDatasetOptions,
MultiDatasetRuntimeOptions runtimeOptions)
{
var optionMap = registeredDatasetOptions
.Select(Clone)
.GroupBy(o => DatasetId.Normalize(o.DatasetId))
.ToDictionary(g => g.Key, g => g.Last(), StringComparer.Ordinal);
foreach (var additional in runtimeOptions.AdditionalDatasets)
optionMap[DatasetId.Normalize(additional.DatasetId)] = Clone(additional);
if (!runtimeOptions.EnableMultiDatasetSync)
return
[
ResolveOrDefault(optionMap, DatasetId.Primary, enabled: true)
];
var result = new List<DatasetSyncOptions>();
if (runtimeOptions.EnableDatasetPrimary)
result.Add(ResolveOrDefault(optionMap, DatasetId.Primary, enabled: true));
if (runtimeOptions.EnableDatasetLogs)
result.Add(ResolveOrDefault(optionMap, DatasetId.Logs, enabled: true));
if (runtimeOptions.EnableDatasetTimeseries)
result.Add(ResolveOrDefault(optionMap, DatasetId.Timeseries, enabled: true));
foreach (var kvp in optionMap)
{
if (string.Equals(kvp.Key, DatasetId.Primary, StringComparison.Ordinal)) continue;
if (string.Equals(kvp.Key, DatasetId.Logs, StringComparison.Ordinal)) continue;
if (string.Equals(kvp.Key, DatasetId.Timeseries, StringComparison.Ordinal)) continue;
result.Add(kvp.Value);
}
return result
.GroupBy(o => DatasetId.Normalize(o.DatasetId), StringComparer.Ordinal)
.Select(g => g.Last())
.ToList();
}
private static DatasetSyncOptions ResolveOrDefault(
IReadOnlyDictionary<string, DatasetSyncOptions> optionMap,
string datasetId,
bool enabled)
{
var normalized = DatasetId.Normalize(datasetId);
if (optionMap.TryGetValue(normalized, out var existing))
{
existing.DatasetId = normalized;
existing.Enabled = enabled && existing.Enabled;
return existing;
}
return new DatasetSyncOptions
{
DatasetId = normalized,
Enabled = enabled
};
}
private static DatasetSyncOptions Clone(DatasetSyncOptions options)
{
return new DatasetSyncOptions
{
DatasetId = DatasetId.Normalize(options.DatasetId),
Enabled = options.Enabled,
SyncLoopDelay = options.SyncLoopDelay,
MaxPeersPerCycle = options.MaxPeersPerCycle,
MaxEntriesPerCycle = options.MaxEntriesPerCycle,
MaintenanceIntervalOverride = options.MaintenanceIntervalOverride,
InterestingCollections = options.InterestingCollections.ToList()
};
}
}

View File

@@ -0,0 +1,32 @@
using ZB.MOM.WW.CBDDC.Core.Storage;
namespace ZB.MOM.WW.CBDDC.Network;
/// <summary>
/// Adapts <see cref="IMultiDatasetSyncOrchestrator" /> to <see cref="ISyncOrchestrator" />.
/// </summary>
internal sealed class MultiDatasetSyncOrchestratorAdapter : ISyncOrchestrator
{
private readonly IMultiDatasetSyncOrchestrator _inner;
/// <summary>
/// Initializes a new instance of the <see cref="MultiDatasetSyncOrchestratorAdapter" /> class.
/// </summary>
/// <param name="inner">The multi-dataset orchestrator.</param>
public MultiDatasetSyncOrchestratorAdapter(IMultiDatasetSyncOrchestrator inner)
{
_inner = inner;
}
/// <inheritdoc />
public Task Start()
{
return _inner.Start();
}
/// <inheritdoc />
public Task Stop()
{
return _inner.Stop();
}
}

View File

@@ -1,6 +1,8 @@
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions; using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Storage;
using ZB.MOM.WW.CBDDC.Core.Network; using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Network.Security; using ZB.MOM.WW.CBDDC.Network.Security;
using ZB.MOM.WW.CBDDC.Network.Telemetry; using ZB.MOM.WW.CBDDC.Network.Telemetry;
@@ -52,4 +54,38 @@ public static class CBDDCNetworkExtensions
return services; return services;
} }
/// <summary>
/// Enables multi-dataset orchestration and replaces the single orchestrator with a dataset coordinator.
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="configure">Optional configuration for runtime feature flags and dataset defaults.</param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddCBDDCMultiDataset(
this IServiceCollection services,
Action<MultiDatasetRuntimeOptions>? configure = null)
{
var runtimeOptions = new MultiDatasetRuntimeOptions
{
EnableMultiDatasetSync = true,
EnableDatasetPrimary = true
};
configure?.Invoke(runtimeOptions);
services.AddSingleton(runtimeOptions);
// Ensure a primary dataset option exists for compatibility when callers did not register dataset options.
if (!services.Any(descriptor => descriptor.ServiceType == typeof(DatasetSyncOptions)))
services.AddSingleton(new DatasetSyncOptions
{
DatasetId = DatasetId.Primary,
Enabled = true
});
services.TryAddSingleton<IMultiDatasetSyncOrchestrator, MultiDatasetSyncOrchestrator>();
services.Replace(ServiceDescriptor.Singleton<ISyncOrchestrator>(sp =>
new MultiDatasetSyncOrchestratorAdapter(sp.GetRequiredService<IMultiDatasetSyncOrchestrator>())));
return services;
}
} }

View File

@@ -34,6 +34,8 @@ public class SyncOrchestrator : ISyncOrchestrator
private readonly ISnapshotService _snapshotService; private readonly ISnapshotService _snapshotService;
private readonly object _startStopLock = new(); private readonly object _startStopLock = new();
private readonly INetworkTelemetryService? _telemetry; private readonly INetworkTelemetryService? _telemetry;
private readonly DatasetSyncOptions _datasetSyncOptions;
private readonly string _datasetId;
private CancellationTokenSource? _cts; private CancellationTokenSource? _cts;
private DateTime _lastMaintenanceTime = DateTime.MinValue; private DateTime _lastMaintenanceTime = DateTime.MinValue;
@@ -52,6 +54,7 @@ public class SyncOrchestrator : ISyncOrchestrator
/// <param name="handshakeService">The optional peer handshake service.</param> /// <param name="handshakeService">The optional peer handshake service.</param>
/// <param name="telemetry">The optional network telemetry service.</param> /// <param name="telemetry">The optional network telemetry service.</param>
/// <param name="oplogPruneCutoffCalculator">The optional cutoff calculator for safe maintenance pruning.</param> /// <param name="oplogPruneCutoffCalculator">The optional cutoff calculator for safe maintenance pruning.</param>
/// <param name="datasetSyncOptions">The optional per-dataset synchronization options.</param>
public SyncOrchestrator( public SyncOrchestrator(
IDiscoveryService discovery, IDiscoveryService discovery,
IOplogStore oplogStore, IOplogStore oplogStore,
@@ -63,7 +66,8 @@ public class SyncOrchestrator : ISyncOrchestrator
IPeerOplogConfirmationStore? peerOplogConfirmationStore = null, IPeerOplogConfirmationStore? peerOplogConfirmationStore = null,
IPeerHandshakeService? handshakeService = null, IPeerHandshakeService? handshakeService = null,
INetworkTelemetryService? telemetry = null, INetworkTelemetryService? telemetry = null,
IOplogPruneCutoffCalculator? oplogPruneCutoffCalculator = null) IOplogPruneCutoffCalculator? oplogPruneCutoffCalculator = null,
DatasetSyncOptions? datasetSyncOptions = null)
{ {
_discovery = discovery; _discovery = discovery;
_oplogStore = oplogStore; _oplogStore = oplogStore;
@@ -77,6 +81,8 @@ public class SyncOrchestrator : ISyncOrchestrator
_logger = loggerFactory.CreateLogger<SyncOrchestrator>(); _logger = loggerFactory.CreateLogger<SyncOrchestrator>();
_handshakeService = handshakeService; _handshakeService = handshakeService;
_telemetry = telemetry; _telemetry = telemetry;
_datasetSyncOptions = datasetSyncOptions ?? new DatasetSyncOptions();
_datasetId = DatasetId.Normalize(_datasetSyncOptions.DatasetId);
} }
/// <summary> /// <summary>
@@ -166,7 +172,7 @@ public class SyncOrchestrator : ISyncOrchestrator
/// </summary> /// </summary>
private async Task SyncLoopAsync(CancellationToken token) private async Task SyncLoopAsync(CancellationToken token)
{ {
_logger.LogInformation("Sync Orchestrator Started (Parallel P2P)"); _logger.LogInformation("Sync Orchestrator Started (Parallel P2P, Dataset: {DatasetId})", _datasetId);
while (!token.IsCancellationRequested) while (!token.IsCancellationRequested)
{ {
var config = await _peerNodeConfigurationProvider.GetConfiguration(); var config = await _peerNodeConfigurationProvider.GetConfiguration();
@@ -193,11 +199,11 @@ public class SyncOrchestrator : ISyncOrchestrator
}).ToList(); }).ToList();
// Interest-Aware Gossip: Prioritize peers sharing interests with us // Interest-Aware Gossip: Prioritize peers sharing interests with us
var localInterests = _documentStore.InterestedCollection.ToList(); var localInterests = GetDatasetInterests();
var targets = eligiblePeers var targets = eligiblePeers
.OrderByDescending(p => p.InterestingCollections.Any(ci => localInterests.Contains(ci))) .OrderByDescending(p => p.InterestingCollections.Any(ci => localInterests.Contains(ci)))
.ThenBy(x => _random.Next()) .ThenBy(x => _random.Next())
.Take(3) .Take(Math.Max(1, _datasetSyncOptions.MaxPeersPerCycle))
.ToList(); .ToList();
// NetStandard 2.0 fallback: Use Task.WhenAll // NetStandard 2.0 fallback: Use Task.WhenAll
@@ -218,7 +224,7 @@ public class SyncOrchestrator : ISyncOrchestrator
try try
{ {
await Task.Delay(2000, token); await Task.Delay(_datasetSyncOptions.SyncLoopDelay, token);
} }
catch (OperationCanceledException) catch (OperationCanceledException)
{ {
@@ -236,7 +242,8 @@ public class SyncOrchestrator : ISyncOrchestrator
/// <returns>A task that represents the asynchronous maintenance operation.</returns> /// <returns>A task that represents the asynchronous maintenance operation.</returns>
internal async Task RunMaintenanceIfDueAsync(PeerNodeConfiguration config, DateTime now, CancellationToken token) internal async Task RunMaintenanceIfDueAsync(PeerNodeConfiguration config, DateTime now, CancellationToken token)
{ {
var maintenanceInterval = TimeSpan.FromMinutes(config.MaintenanceIntervalMinutes); var maintenanceInterval = _datasetSyncOptions.MaintenanceIntervalOverride ??
TimeSpan.FromMinutes(config.MaintenanceIntervalMinutes);
if (now - _lastMaintenanceTime < maintenanceInterval) return; if (now - _lastMaintenanceTime < maintenanceInterval) return;
_logger.LogInformation("Running periodic maintenance (Oplog pruning)..."); _logger.LogInformation("Running periodic maintenance (Oplog pruning)...");
@@ -253,7 +260,7 @@ public class SyncOrchestrator : ISyncOrchestrator
return; return;
} }
await _oplogStore.PruneOplogAsync(cutoffDecision.EffectiveCutoff.Value, token); await _oplogStore.PruneOplogAsync(cutoffDecision.EffectiveCutoff.Value, _datasetId, token);
_lastMaintenanceTime = now; _lastMaintenanceTime = now;
if (cutoffDecision.ConfirmationCutoff.HasValue) if (cutoffDecision.ConfirmationCutoff.HasValue)
@@ -312,6 +319,7 @@ public class SyncOrchestrator : ISyncOrchestrator
try try
{ {
var config = await _peerNodeConfigurationProvider.GetConfiguration(); var config = await _peerNodeConfigurationProvider.GetConfiguration();
var localInterests = GetDatasetInterests();
// Get or create persistent client // Get or create persistent client
client = _clients.GetOrAdd(peer.NodeId, id => new TcpPeerClient( client = _clients.GetOrAdd(peer.NodeId, id => new TcpPeerClient(
@@ -324,8 +332,8 @@ public class SyncOrchestrator : ISyncOrchestrator
if (!client.IsConnected) await client.ConnectAsync(token); if (!client.IsConnected) await client.ConnectAsync(token);
// Handshake (idempotent) // Handshake (idempotent)
if (!await client.HandshakeAsync(config.NodeId, config.AuthToken, _documentStore.InterestedCollection, if (!await client.HandshakeAsync(config.NodeId, config.AuthToken, localInterests,
token)) _datasetId, token))
{ {
_logger.LogWarning("Handshake rejected by {NodeId}", peer.NodeId); _logger.LogWarning("Handshake rejected by {NodeId}", peer.NodeId);
shouldRemoveClient = true; shouldRemoveClient = true;
@@ -334,7 +342,7 @@ public class SyncOrchestrator : ISyncOrchestrator
// 1. Exchange Vector Clocks // 1. Exchange Vector Clocks
var remoteVectorClock = await client.GetVectorClockAsync(token); var remoteVectorClock = await client.GetVectorClockAsync(token);
var localVectorClock = await _oplogStore.GetVectorClockAsync(token); var localVectorClock = await _oplogStore.GetVectorClockAsync(_datasetId, token);
_logger.LogDebug("Vector Clock - Local: {Local}, Remote: {Remote}", localVectorClock, remoteVectorClock); _logger.LogDebug("Vector Clock - Local: {Local}, Remote: {Remote}", localVectorClock, remoteVectorClock);
@@ -361,7 +369,7 @@ public class SyncOrchestrator : ISyncOrchestrator
// PASS LOCAL INTERESTS TO PULL // PASS LOCAL INTERESTS TO PULL
var changes = await client.PullChangesFromNodeAsync(nodeId, localTs, var changes = await client.PullChangesFromNodeAsync(nodeId, localTs,
_documentStore.InterestedCollection, token); localInterests, _datasetId, token);
if (changes != null && changes.Count > 0) if (changes != null && changes.Count > 0)
{ {
var result = await ProcessInboundBatchAsync(client, peer.NodeId, changes, token); var result = await ProcessInboundBatchAsync(client, peer.NodeId, changes, token);
@@ -390,13 +398,13 @@ public class SyncOrchestrator : ISyncOrchestrator
// PUSH FILTERING: Pass remote receiver's interests to oplogStore for efficient retrieval // PUSH FILTERING: Pass remote receiver's interests to oplogStore for efficient retrieval
var remoteInterests = client.RemoteInterests; var remoteInterests = client.RemoteInterests;
var changes = var changes =
(await _oplogStore.GetOplogForNodeAfterAsync(nodeId, remoteTs, remoteInterests, token)) (await _oplogStore.GetOplogForNodeAfterAsync(nodeId, remoteTs, _datasetId, remoteInterests, token))
.ToList(); .ToList();
if (changes.Any()) if (changes.Any())
{ {
_logger.LogDebug("Pushing {Count} filtered changes for Node {NodeId}", changes.Count, nodeId); _logger.LogDebug("Pushing {Count} filtered changes for Node {NodeId}", changes.Count, nodeId);
await client.PushChangesAsync(changes, token); await client.PushChangesAsync(changes, _datasetId, token);
await AdvanceConfirmationForPushedBatchAsync(peer.NodeId, nodeId, changes, token); await AdvanceConfirmationForPushedBatchAsync(peer.NodeId, nodeId, changes, token);
} }
} }
@@ -552,7 +560,7 @@ public class SyncOrchestrator : ISyncOrchestrator
try try
{ {
await _peerOplogConfirmationStore.EnsurePeerRegisteredAsync(peer.NodeId, peer.Address, peer.Type, await _peerOplogConfirmationStore.EnsurePeerRegisteredAsync(peer.NodeId, peer.Address, peer.Type,
token); _datasetId, token);
} }
catch (OperationCanceledException) when (token.IsCancellationRequested) catch (OperationCanceledException) when (token.IsCancellationRequested)
{ {
@@ -623,6 +631,7 @@ public class SyncOrchestrator : ISyncOrchestrator
sourceNodeId, sourceNodeId,
maxPushed.Timestamp, maxPushed.Timestamp,
maxPushed.Hash ?? string.Empty, maxPushed.Hash ?? string.Empty,
_datasetId,
token); token);
} }
catch (OperationCanceledException) when (token.IsCancellationRequested) catch (OperationCanceledException) when (token.IsCancellationRequested)
@@ -648,8 +657,14 @@ public class SyncOrchestrator : ISyncOrchestrator
try try
{ {
// Best-effort hash lookup: IOplogStore exposes latest hash per source node. // Best-effort hash lookup: IOplogStore exposes latest hash per source node.
string hash = await _oplogStore.GetLastEntryHashAsync(sourceNodeId, token) ?? string.Empty; string hash = await _oplogStore.GetLastEntryHashAsync(sourceNodeId, _datasetId, token) ?? string.Empty;
await _peerOplogConfirmationStore.UpdateConfirmationAsync(peerNodeId, sourceNodeId, timestamp, hash, token); await _peerOplogConfirmationStore.UpdateConfirmationAsync(
peerNodeId,
sourceNodeId,
timestamp,
hash,
_datasetId,
token);
} }
catch (OperationCanceledException) when (token.IsCancellationRequested) catch (OperationCanceledException) when (token.IsCancellationRequested)
{ {
@@ -716,7 +731,7 @@ public class SyncOrchestrator : ISyncOrchestrator
// Check linkage with Local State // Check linkage with Local State
var firstEntry = authorChain[0]; var firstEntry = authorChain[0];
string? localHeadHash = await _oplogStore.GetLastEntryHashAsync(authorNodeId, token); string? localHeadHash = await _oplogStore.GetLastEntryHashAsync(authorNodeId, _datasetId, token);
_logger.LogDebug( _logger.LogDebug(
"Processing chain for Node {AuthorId}: FirstEntry.PrevHash={PrevHash}, FirstEntry.Hash={Hash}, LocalHeadHash={LocalHead}", "Processing chain for Node {AuthorId}: FirstEntry.PrevHash={PrevHash}, FirstEntry.Hash={Hash}, LocalHeadHash={LocalHead}",
@@ -725,7 +740,7 @@ public class SyncOrchestrator : ISyncOrchestrator
if (localHeadHash != null && firstEntry.PreviousHash != localHeadHash) if (localHeadHash != null && firstEntry.PreviousHash != localHeadHash)
{ {
// Check if entry starts from snapshot boundary (valid case after pruning) // Check if entry starts from snapshot boundary (valid case after pruning)
string? snapshotHash = await _snapshotMetadataStore.GetSnapshotHashAsync(authorNodeId, token); string? snapshotHash = await _snapshotMetadataStore.GetSnapshotHashAsync(authorNodeId, _datasetId, token);
if (snapshotHash != null && firstEntry.PreviousHash == snapshotHash) if (snapshotHash != null && firstEntry.PreviousHash == snapshotHash)
{ {
@@ -748,7 +763,7 @@ public class SyncOrchestrator : ISyncOrchestrator
List<OplogEntry>? missingChain = null; List<OplogEntry>? missingChain = null;
try try
{ {
missingChain = await client.GetChainRangeAsync(localHeadHash, firstEntry.PreviousHash, token); missingChain = await client.GetChainRangeAsync(localHeadHash, firstEntry.PreviousHash, _datasetId, token);
} }
catch (SnapshotRequiredException) catch (SnapshotRequiredException)
{ {
@@ -779,7 +794,7 @@ public class SyncOrchestrator : ISyncOrchestrator
} }
// Apply Missing Chain First // Apply Missing Chain First
await _oplogStore.ApplyBatchAsync(missingChain, token); await _oplogStore.ApplyBatchAsync(missingChain, _datasetId, token);
_logger.LogInformation("Gap Recovery Applied Successfully."); _logger.LogInformation("Gap Recovery Applied Successfully.");
} }
else else
@@ -808,7 +823,7 @@ public class SyncOrchestrator : ISyncOrchestrator
} }
// Apply original batch (grouped by node for clarity, but oplogStore usually handles bulk) // Apply original batch (grouped by node for clarity, but oplogStore usually handles bulk)
await _oplogStore.ApplyBatchAsync(authorChain, token); await _oplogStore.ApplyBatchAsync(authorChain, _datasetId, token);
} }
return SyncBatchResult.Success; return SyncBatchResult.Success;
@@ -824,7 +839,7 @@ public class SyncOrchestrator : ISyncOrchestrator
_logger.LogInformation("Downloading snapshot to {TempFile}...", tempFile); _logger.LogInformation("Downloading snapshot to {TempFile}...", tempFile);
using (var fs = File.Create(tempFile)) using (var fs = File.Create(tempFile))
{ {
await client.GetSnapshotAsync(fs, token); await client.GetSnapshotAsync(fs, _datasetId, token);
} }
_logger.LogInformation("Snapshot Downloaded. applying to store..."); _logger.LogInformation("Snapshot Downloaded. applying to store...");
@@ -832,9 +847,9 @@ public class SyncOrchestrator : ISyncOrchestrator
using (var fs = File.OpenRead(tempFile)) using (var fs = File.OpenRead(tempFile))
{ {
if (mergeOnly) if (mergeOnly)
await _snapshotService.MergeSnapshotAsync(fs, token); await _snapshotService.MergeSnapshotAsync(fs, _datasetId, token);
else else
await _snapshotService.ReplaceDatabaseAsync(fs, token); await _snapshotService.ReplaceDatabaseAsync(fs, _datasetId, token);
} }
_logger.LogInformation("Snapshot applied successfully."); _logger.LogInformation("Snapshot applied successfully.");
@@ -858,6 +873,17 @@ public class SyncOrchestrator : ISyncOrchestrator
} }
} }
private List<string> GetDatasetInterests()
{
if (_datasetSyncOptions.InterestingCollections.Count == 0)
return _documentStore.InterestedCollection.ToList();
return _datasetSyncOptions.InterestingCollections
.Where(value => !string.IsNullOrWhiteSpace(value))
.Distinct(StringComparer.Ordinal)
.ToList();
}
private class PeerStatus private class PeerStatus
{ {
/// <summary> /// <summary>

View File

@@ -170,7 +170,7 @@ public class TcpPeerClient : IDisposable
/// <returns>True if handshake was accepted, false otherwise.</returns> /// <returns>True if handshake was accepted, false otherwise.</returns>
public async Task<bool> HandshakeAsync(string myNodeId, string authToken, CancellationToken token) public async Task<bool> HandshakeAsync(string myNodeId, string authToken, CancellationToken token)
{ {
return await HandshakeAsync(myNodeId, authToken, null, token); return await HandshakeAsync(myNodeId, authToken, null, DatasetId.Primary, token);
} }
/// <summary> /// <summary>
@@ -183,15 +183,36 @@ public class TcpPeerClient : IDisposable
/// <returns><see langword="true" /> if handshake was accepted; otherwise <see langword="false" />.</returns> /// <returns><see langword="true" /> if handshake was accepted; otherwise <see langword="false" />.</returns>
public async Task<bool> HandshakeAsync(string myNodeId, string authToken, public async Task<bool> HandshakeAsync(string myNodeId, string authToken,
IEnumerable<string>? interestingCollections, CancellationToken token) IEnumerable<string>? interestingCollections, CancellationToken token)
{
return await HandshakeAsync(myNodeId, authToken, interestingCollections, DatasetId.Primary, token);
}
/// <summary>
/// Performs authentication handshake with the remote peer for a specific dataset.
/// </summary>
/// <param name="myNodeId">The local node identifier.</param>
/// <param name="authToken">The authentication token.</param>
/// <param name="interestingCollections">Optional collection names this node is interested in receiving.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns><see langword="true" /> if handshake was accepted; otherwise <see langword="false" />.</returns>
public async Task<bool> HandshakeAsync(string myNodeId, string authToken,
IEnumerable<string>? interestingCollections, string datasetId, CancellationToken token)
{ {
if (HasHandshaked) return true; if (HasHandshaked) return true;
string normalizedDatasetId = DatasetId.Normalize(datasetId);
if (_handshakeService != null) if (_handshakeService != null)
// Perform secure handshake if service is available // Perform secure handshake if service is available
// We assume we are initiator here // We assume we are initiator here
_cipherState = await _handshakeService.HandshakeAsync(_stream!, true, myNodeId, token); _cipherState = await _handshakeService.HandshakeAsync(_stream!, true, myNodeId, token);
var req = new HandshakeRequest { NodeId = myNodeId, AuthToken = authToken ?? "" }; var req = new HandshakeRequest
{
NodeId = myNodeId,
AuthToken = authToken ?? "",
DatasetId = normalizedDatasetId
};
if (interestingCollections != null) if (interestingCollections != null)
foreach (string coll in interestingCollections) foreach (string coll in interestingCollections)
@@ -208,6 +229,13 @@ public class TcpPeerClient : IDisposable
if (type != MessageType.HandshakeRes) return false; if (type != MessageType.HandshakeRes) return false;
var res = HandshakeResponse.Parser.ParseFrom(payload); var res = HandshakeResponse.Parser.ParseFrom(payload);
string resolvedResponseDatasetId = DatasetId.Normalize(res.DatasetId);
if ((res.HasDatasetSupported && !res.DatasetSupported) ||
!string.Equals(resolvedResponseDatasetId, normalizedDatasetId, StringComparison.Ordinal))
{
HasHandshaked = false;
return false;
}
// Store remote interests // Store remote interests
_remoteInterests = res.InterestingCollections.ToList(); _remoteInterests = res.InterestingCollections.ToList();
@@ -276,7 +304,7 @@ public class TcpPeerClient : IDisposable
/// <returns>The list of oplog entries returned by the remote peer.</returns> /// <returns>The list of oplog entries returned by the remote peer.</returns>
public async Task<List<OplogEntry>> PullChangesAsync(HlcTimestamp since, CancellationToken token) public async Task<List<OplogEntry>> PullChangesAsync(HlcTimestamp since, CancellationToken token)
{ {
return await PullChangesAsync(since, null, token); return await PullChangesAsync(since, null, DatasetId.Primary, token);
} }
/// <summary> /// <summary>
@@ -289,12 +317,28 @@ public class TcpPeerClient : IDisposable
public async Task<List<OplogEntry>> PullChangesAsync(HlcTimestamp since, IEnumerable<string>? collections, public async Task<List<OplogEntry>> PullChangesAsync(HlcTimestamp since, IEnumerable<string>? collections,
CancellationToken token) CancellationToken token)
{ {
return await PullChangesAsync(since, collections, DatasetId.Primary, token);
}
/// <summary>
/// Pulls oplog changes from the remote peer for a specific dataset.
/// </summary>
/// <param name="since">The starting timestamp for requested changes.</param>
/// <param name="collections">Optional collection names used to filter the returned entries.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns>The list of oplog entries returned by the remote peer.</returns>
public async Task<List<OplogEntry>> PullChangesAsync(HlcTimestamp since, IEnumerable<string>? collections,
string datasetId, CancellationToken token)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
var req = new PullChangesRequest var req = new PullChangesRequest
{ {
SinceWall = since.PhysicalTime, SinceWall = since.PhysicalTime,
SinceLogic = since.LogicalCounter, SinceLogic = since.LogicalCounter,
// Empty SinceNode indicates a global pull (not source-node filtered). // Empty SinceNode indicates a global pull (not source-node filtered).
SinceNode = string.Empty SinceNode = string.Empty,
DatasetId = normalizedDatasetId
}; };
if (collections != null) if (collections != null)
foreach (string coll in collections) foreach (string coll in collections)
@@ -315,7 +359,8 @@ public class TcpPeerClient : IDisposable
string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData), string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData),
new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode), new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode),
e.PreviousHash, e.PreviousHash,
e.Hash // Pass the received hash to preserve integrity reference e.Hash, // Pass the received hash to preserve integrity reference
string.IsNullOrWhiteSpace(e.DatasetId) ? normalizedDatasetId : e.DatasetId
)).ToList(); )).ToList();
} }
@@ -329,7 +374,7 @@ public class TcpPeerClient : IDisposable
public async Task<List<OplogEntry>> PullChangesFromNodeAsync(string nodeId, HlcTimestamp since, public async Task<List<OplogEntry>> PullChangesFromNodeAsync(string nodeId, HlcTimestamp since,
CancellationToken token) CancellationToken token)
{ {
return await PullChangesFromNodeAsync(nodeId, since, null, token); return await PullChangesFromNodeAsync(nodeId, since, null, DatasetId.Primary, token);
} }
/// <summary> /// <summary>
@@ -344,11 +389,28 @@ public class TcpPeerClient : IDisposable
public async Task<List<OplogEntry>> PullChangesFromNodeAsync(string nodeId, HlcTimestamp since, public async Task<List<OplogEntry>> PullChangesFromNodeAsync(string nodeId, HlcTimestamp since,
IEnumerable<string>? collections, CancellationToken token) IEnumerable<string>? collections, CancellationToken token)
{ {
return await PullChangesFromNodeAsync(nodeId, since, collections, DatasetId.Primary, token);
}
/// <summary>
/// Pulls oplog changes for a specific node and dataset from the remote peer.
/// </summary>
/// <param name="nodeId">The node identifier to filter changes by.</param>
/// <param name="since">The starting timestamp for requested changes.</param>
/// <param name="collections">Optional collection names used to filter the returned entries.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns>The list of oplog entries returned by the remote peer.</returns>
public async Task<List<OplogEntry>> PullChangesFromNodeAsync(string nodeId, HlcTimestamp since,
IEnumerable<string>? collections, string datasetId, CancellationToken token)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
var req = new PullChangesRequest var req = new PullChangesRequest
{ {
SinceNode = nodeId, SinceNode = nodeId,
SinceWall = since.PhysicalTime, SinceWall = since.PhysicalTime,
SinceLogic = since.LogicalCounter SinceLogic = since.LogicalCounter,
DatasetId = normalizedDatasetId
}; };
if (collections != null) if (collections != null)
foreach (string coll in collections) foreach (string coll in collections)
@@ -369,7 +431,8 @@ public class TcpPeerClient : IDisposable
string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData), string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData),
new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode), new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode),
e.PreviousHash, e.PreviousHash,
e.Hash e.Hash,
string.IsNullOrWhiteSpace(e.DatasetId) ? normalizedDatasetId : e.DatasetId
)).ToList(); )).ToList();
} }
@@ -383,7 +446,27 @@ public class TcpPeerClient : IDisposable
public virtual async Task<List<OplogEntry>> GetChainRangeAsync(string startHash, string endHash, public virtual async Task<List<OplogEntry>> GetChainRangeAsync(string startHash, string endHash,
CancellationToken token) CancellationToken token)
{ {
var req = new GetChainRangeRequest { StartHash = startHash, EndHash = endHash }; return await GetChainRangeAsync(startHash, endHash, DatasetId.Primary, token);
}
/// <summary>
/// Retrieves a range of oplog entries connecting two hashes for a specific dataset.
/// </summary>
/// <param name="startHash">The starting hash in the chain.</param>
/// <param name="endHash">The ending hash in the chain.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns>The chain entries connecting the requested hash range.</returns>
public virtual async Task<List<OplogEntry>> GetChainRangeAsync(string startHash, string endHash,
string datasetId, CancellationToken token)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
var req = new GetChainRangeRequest
{
StartHash = startHash,
EndHash = endHash,
DatasetId = normalizedDatasetId
};
await _protocol.SendMessageAsync(_stream!, MessageType.GetChainRangeReq, req, _useCompression, _cipherState, await _protocol.SendMessageAsync(_stream!, MessageType.GetChainRangeReq, req, _useCompression, _cipherState,
token); token);
@@ -401,7 +484,8 @@ public class TcpPeerClient : IDisposable
string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData), string.IsNullOrEmpty(e.JsonData) ? default : JsonSerializer.Deserialize<JsonElement>(e.JsonData),
new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode), new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode),
e.PreviousHash, e.PreviousHash,
e.Hash e.Hash,
string.IsNullOrWhiteSpace(e.DatasetId) ? normalizedDatasetId : e.DatasetId
)).ToList(); )).ToList();
} }
@@ -413,7 +497,23 @@ public class TcpPeerClient : IDisposable
/// <returns>A task that represents the asynchronous push operation.</returns> /// <returns>A task that represents the asynchronous push operation.</returns>
public async Task PushChangesAsync(IEnumerable<OplogEntry> entries, CancellationToken token) public async Task PushChangesAsync(IEnumerable<OplogEntry> entries, CancellationToken token)
{ {
var req = new PushChangesRequest(); await PushChangesAsync(entries, DatasetId.Primary, token);
}
/// <summary>
/// Pushes local oplog changes to the remote peer for a specific dataset.
/// </summary>
/// <param name="entries">The oplog entries to push.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns>A task that represents the asynchronous push operation.</returns>
public async Task PushChangesAsync(IEnumerable<OplogEntry> entries, string datasetId, CancellationToken token)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
var req = new PushChangesRequest
{
DatasetId = normalizedDatasetId
};
var entryList = entries.ToList(); var entryList = entries.ToList();
if (entryList.Count == 0) return; if (entryList.Count == 0) return;
@@ -428,7 +528,8 @@ public class TcpPeerClient : IDisposable
HlcLogic = e.Timestamp.LogicalCounter, HlcLogic = e.Timestamp.LogicalCounter,
HlcNode = e.Timestamp.NodeId, HlcNode = e.Timestamp.NodeId,
Hash = e.Hash, Hash = e.Hash,
PreviousHash = e.PreviousHash PreviousHash = e.PreviousHash,
DatasetId = string.IsNullOrWhiteSpace(e.DatasetId) ? normalizedDatasetId : e.DatasetId
}); });
await _protocol.SendMessageAsync(_stream!, MessageType.PushChangesReq, req, _useCompression, _cipherState, await _protocol.SendMessageAsync(_stream!, MessageType.PushChangesReq, req, _useCompression, _cipherState,
@@ -455,7 +556,22 @@ public class TcpPeerClient : IDisposable
/// <returns>A task that represents the asynchronous snapshot transfer operation.</returns> /// <returns>A task that represents the asynchronous snapshot transfer operation.</returns>
public async Task GetSnapshotAsync(Stream destination, CancellationToken token) public async Task GetSnapshotAsync(Stream destination, CancellationToken token)
{ {
await _protocol.SendMessageAsync(_stream!, MessageType.GetSnapshotReq, new GetSnapshotRequest(), await GetSnapshotAsync(destination, DatasetId.Primary, token);
}
/// <summary>
/// Downloads a full snapshot for a specific dataset from the remote peer.
/// </summary>
/// <param name="destination">The stream that receives snapshot bytes.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="token">Cancellation token.</param>
/// <returns>A task that represents the asynchronous snapshot transfer operation.</returns>
public async Task GetSnapshotAsync(Stream destination, string datasetId, CancellationToken token)
{
await _protocol.SendMessageAsync(_stream!, MessageType.GetSnapshotReq, new GetSnapshotRequest
{
DatasetId = DatasetId.Normalize(datasetId)
},
_useCompression, _cipherState, token); _useCompression, _cipherState, token);
while (true) while (true)

View File

@@ -248,6 +248,7 @@ internal class TcpSyncServer : ISyncServer
var useCompression = false; var useCompression = false;
CipherState? cipherState = null; CipherState? cipherState = null;
List<string> remoteInterests = new(); List<string> remoteInterests = new();
string currentDatasetId = DatasetId.Primary;
// Perform Secure Handshake (if service is available) // Perform Secure Handshake (if service is available)
var config = await _configProvider.GetConfiguration(); var config = await _configProvider.GetConfiguration();
@@ -278,6 +279,8 @@ internal class TcpSyncServer : ISyncServer
{ {
var hReq = HandshakeRequest.Parser.ParseFrom(payload); var hReq = HandshakeRequest.Parser.ParseFrom(payload);
_logger.LogDebug("Received HandshakeReq from Node {NodeId}", hReq.NodeId); _logger.LogDebug("Received HandshakeReq from Node {NodeId}", hReq.NodeId);
string requestedDatasetId = DatasetId.Normalize(hReq.DatasetId);
currentDatasetId = requestedDatasetId;
// Track remote peer interests // Track remote peer interests
remoteInterests = hReq.InterestingCollections.ToList(); remoteInterests = hReq.InterestingCollections.ToList();
@@ -292,7 +295,13 @@ internal class TcpSyncServer : ISyncServer
return; return;
} }
var hRes = new HandshakeResponse { NodeId = config.NodeId, Accepted = true }; var hRes = new HandshakeResponse
{
NodeId = config.NodeId,
Accepted = true,
DatasetId = requestedDatasetId,
DatasetSupported = true
};
// Include local interests from IDocumentStore in response for push filtering // Include local interests from IDocumentStore in response for push filtering
foreach (string coll in _documentStore.InterestedCollection) foreach (string coll in _documentStore.InterestedCollection)
@@ -315,7 +324,7 @@ internal class TcpSyncServer : ISyncServer
switch (type) switch (type)
{ {
case MessageType.GetClockReq: case MessageType.GetClockReq:
var clock = await _oplogStore.GetLatestTimestampAsync(token); var clock = await _oplogStore.GetLatestTimestampAsync(currentDatasetId, token);
response = new ClockResponse response = new ClockResponse
{ {
HlcWall = clock.PhysicalTime, HlcWall = clock.PhysicalTime,
@@ -326,7 +335,7 @@ internal class TcpSyncServer : ISyncServer
break; break;
case MessageType.GetVectorClockReq: case MessageType.GetVectorClockReq:
var vectorClock = await _oplogStore.GetVectorClockAsync(token); var vectorClock = await _oplogStore.GetVectorClockAsync(currentDatasetId, token);
var vcRes = new VectorClockResponse(); var vcRes = new VectorClockResponse();
foreach (string nodeId in vectorClock.NodeIds) foreach (string nodeId in vectorClock.NodeIds)
{ {
@@ -345,13 +354,16 @@ internal class TcpSyncServer : ISyncServer
case MessageType.PullChangesReq: case MessageType.PullChangesReq:
var pReq = PullChangesRequest.Parser.ParseFrom(payload); var pReq = PullChangesRequest.Parser.ParseFrom(payload);
string pullDatasetId = string.IsNullOrWhiteSpace(pReq.DatasetId)
? currentDatasetId
: DatasetId.Normalize(pReq.DatasetId);
var since = new HlcTimestamp(pReq.SinceWall, pReq.SinceLogic, pReq.SinceNode); var since = new HlcTimestamp(pReq.SinceWall, pReq.SinceLogic, pReq.SinceNode);
// Use collection filter from request // Use collection filter from request
var filter = pReq.Collections.Any() ? pReq.Collections : null; var filter = pReq.Collections.Any() ? pReq.Collections : null;
var oplog = string.IsNullOrWhiteSpace(pReq.SinceNode) var oplog = string.IsNullOrWhiteSpace(pReq.SinceNode)
? await _oplogStore.GetOplogAfterAsync(since, filter, token) ? await _oplogStore.GetOplogAfterAsync(since, pullDatasetId, filter, token)
: await _oplogStore.GetOplogForNodeAfterAsync(pReq.SinceNode, since, filter, token); : await _oplogStore.GetOplogForNodeAfterAsync(pReq.SinceNode, since, pullDatasetId, filter, token);
var csRes = new ChangeSetResponse(); var csRes = new ChangeSetResponse();
foreach (var e in oplog) foreach (var e in oplog)
@@ -365,7 +377,8 @@ internal class TcpSyncServer : ISyncServer
HlcLogic = e.Timestamp.LogicalCounter, HlcLogic = e.Timestamp.LogicalCounter,
HlcNode = e.Timestamp.NodeId, HlcNode = e.Timestamp.NodeId,
Hash = e.Hash, Hash = e.Hash,
PreviousHash = e.PreviousHash PreviousHash = e.PreviousHash,
DatasetId = e.DatasetId
}); });
response = csRes; response = csRes;
resType = MessageType.ChangeSetRes; resType = MessageType.ChangeSetRes;
@@ -373,6 +386,9 @@ internal class TcpSyncServer : ISyncServer
case MessageType.PushChangesReq: case MessageType.PushChangesReq:
var pushReq = PushChangesRequest.Parser.ParseFrom(payload); var pushReq = PushChangesRequest.Parser.ParseFrom(payload);
string pushDatasetId = string.IsNullOrWhiteSpace(pushReq.DatasetId)
? currentDatasetId
: DatasetId.Normalize(pushReq.DatasetId);
var entries = pushReq.Entries.Select(e => new OplogEntry( var entries = pushReq.Entries.Select(e => new OplogEntry(
e.Collection, e.Collection,
e.Key, e.Key,
@@ -382,10 +398,11 @@ internal class TcpSyncServer : ISyncServer
: JsonSerializer.Deserialize<JsonElement>(e.JsonData), : JsonSerializer.Deserialize<JsonElement>(e.JsonData),
new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode), new HlcTimestamp(e.HlcWall, e.HlcLogic, e.HlcNode),
e.PreviousHash, // Restore PreviousHash e.PreviousHash, // Restore PreviousHash
e.Hash // Restore Hash e.Hash, // Restore Hash
string.IsNullOrWhiteSpace(e.DatasetId) ? pushDatasetId : e.DatasetId
)); ));
await _oplogStore.ApplyBatchAsync(entries, token); await _oplogStore.ApplyBatchAsync(entries, pushDatasetId, token);
response = new AckResponse { Success = true }; response = new AckResponse { Success = true };
resType = MessageType.AckRes; resType = MessageType.AckRes;
@@ -393,8 +410,15 @@ internal class TcpSyncServer : ISyncServer
case MessageType.GetChainRangeReq: case MessageType.GetChainRangeReq:
var rangeReq = GetChainRangeRequest.Parser.ParseFrom(payload); var rangeReq = GetChainRangeRequest.Parser.ParseFrom(payload);
string chainDatasetId = string.IsNullOrWhiteSpace(rangeReq.DatasetId)
? currentDatasetId
: DatasetId.Normalize(rangeReq.DatasetId);
var rangeEntries = var rangeEntries =
await _oplogStore.GetChainRangeAsync(rangeReq.StartHash, rangeReq.EndHash, token); await _oplogStore.GetChainRangeAsync(
rangeReq.StartHash,
rangeReq.EndHash,
chainDatasetId,
token);
var rangeRes = new ChainRangeResponse(); var rangeRes = new ChainRangeResponse();
if (!rangeEntries.Any() && rangeReq.StartHash != rangeReq.EndHash) if (!rangeEntries.Any() && rangeReq.StartHash != rangeReq.EndHash)
@@ -412,7 +436,8 @@ internal class TcpSyncServer : ISyncServer
HlcLogic = e.Timestamp.LogicalCounter, HlcLogic = e.Timestamp.LogicalCounter,
HlcNode = e.Timestamp.NodeId, HlcNode = e.Timestamp.NodeId,
Hash = e.Hash, Hash = e.Hash,
PreviousHash = e.PreviousHash PreviousHash = e.PreviousHash,
DatasetId = e.DatasetId
}); });
response = rangeRes; response = rangeRes;
@@ -420,6 +445,10 @@ internal class TcpSyncServer : ISyncServer
break; break;
case MessageType.GetSnapshotReq: case MessageType.GetSnapshotReq:
var snapshotRequest = GetSnapshotRequest.Parser.ParseFrom(payload);
string snapshotDatasetId = string.IsNullOrWhiteSpace(snapshotRequest.DatasetId)
? currentDatasetId
: DatasetId.Normalize(snapshotRequest.DatasetId);
_logger.LogInformation("Processing GetSnapshotReq from {Endpoint}", remoteEp); _logger.LogInformation("Processing GetSnapshotReq from {Endpoint}", remoteEp);
string tempFile = Path.GetTempFileName(); string tempFile = Path.GetTempFileName();
try try
@@ -427,7 +456,7 @@ internal class TcpSyncServer : ISyncServer
// Create backup // Create backup
using (var fs = File.Create(tempFile)) using (var fs = File.Create(tempFile))
{ {
await _snapshotStore.CreateSnapshotAsync(fs, token); await _snapshotStore.CreateSnapshotAsync(fs, snapshotDatasetId, token);
} }
using (var fs = File.OpenRead(tempFile)) using (var fs = File.OpenRead(tempFile))

View File

@@ -9,6 +9,7 @@ message HandshakeRequest {
string auth_token = 2; string auth_token = 2;
repeated string supported_compression = 3; // v4 repeated string supported_compression = 3; // v4
repeated string interesting_collections = 4; // v5 repeated string interesting_collections = 4; // v5
string dataset_id = 5; // v6
} }
message HandshakeResponse { message HandshakeResponse {
@@ -16,6 +17,8 @@ message HandshakeResponse {
bool accepted = 2; bool accepted = 2;
string selected_compression = 3; // v4 string selected_compression = 3; // v4
repeated string interesting_collections = 4; // v5 repeated string interesting_collections = 4; // v5
string dataset_id = 5; // v6
optional bool dataset_supported = 6; // v6
} }
message GetClockRequest { message GetClockRequest {
@@ -45,6 +48,7 @@ message PullChangesRequest {
int32 since_logic = 2; int32 since_logic = 2;
string since_node = 3; string since_node = 3;
repeated string collections = 4; // v5: Filter by collection repeated string collections = 4; // v5: Filter by collection
string dataset_id = 5; // v6
} }
message ChangeSetResponse { message ChangeSetResponse {
@@ -53,11 +57,13 @@ message ChangeSetResponse {
message PushChangesRequest { message PushChangesRequest {
repeated ProtoOplogEntry entries = 1; repeated ProtoOplogEntry entries = 1;
string dataset_id = 2; // v6
} }
message GetChainRangeRequest { message GetChainRangeRequest {
string start_hash = 1; string start_hash = 1;
string end_hash = 2; string end_hash = 2;
string dataset_id = 3; // v6
} }
message ChainRangeResponse { message ChainRangeResponse {
@@ -80,9 +86,11 @@ message ProtoOplogEntry {
string hlc_node = 7; string hlc_node = 7;
string hash = 8; string hash = 8;
string previous_hash = 9; string previous_hash = 9;
string dataset_id = 10; // v6
} }
message GetSnapshotRequest { message GetSnapshotRequest {
string dataset_id = 1; // v6
} }
message SnapshotChunk { message SnapshotChunk {

View File

@@ -20,6 +20,11 @@ public class SnapshotDto
/// </summary> /// </summary>
public string NodeId { get; set; } = ""; public string NodeId { get; set; } = "";
/// <summary>
/// Gets or sets the dataset identifier represented by this snapshot payload.
/// </summary>
public string DatasetId { get; set; } = "primary";
/// <summary> /// <summary>
/// Gets or sets the serialized document records. /// Gets or sets the serialized document records.
/// </summary> /// </summary>
@@ -48,6 +53,11 @@ public class SnapshotDto
public class DocumentDto public class DocumentDto
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = "primary";
/// <summary> /// <summary>
/// Gets or sets the document collection name. /// Gets or sets the document collection name.
/// </summary> /// </summary>
@@ -86,6 +96,11 @@ public class DocumentDto
public class OplogDto public class OplogDto
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = "primary";
/// <summary> /// <summary>
/// Gets or sets the collection associated with the operation. /// Gets or sets the collection associated with the operation.
/// </summary> /// </summary>
@@ -134,6 +149,11 @@ public class OplogDto
public class SnapshotMetadataDto public class SnapshotMetadataDto
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = "primary";
/// <summary> /// <summary>
/// Gets or sets the node identifier. /// Gets or sets the node identifier.
/// </summary> /// </summary>
@@ -180,6 +200,11 @@ public class RemotePeerDto
public class PeerOplogConfirmationDto public class PeerOplogConfirmationDto
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = "primary";
/// <summary> /// <summary>
/// Gets or sets the tracked peer node identifier. /// Gets or sets the tracked peer node identifier.
/// </summary> /// </summary>

View File

@@ -92,24 +92,43 @@ public class SnapshotStore : ISnapshotService
/// <inheritdoc /> /// <inheritdoc />
public async Task CreateSnapshotAsync(Stream destination, CancellationToken cancellationToken = default) public async Task CreateSnapshotAsync(Stream destination, CancellationToken cancellationToken = default)
{ {
await CreateSnapshotAsync(destination, DatasetId.Primary, cancellationToken);
}
/// <inheritdoc />
public async Task CreateSnapshotAsync(Stream destination, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
_logger.LogInformation("Creating snapshot..."); _logger.LogInformation("Creating snapshot...");
var documents = await _documentStore.ExportAsync(cancellationToken); var documents = await _documentStore.ExportAsync(cancellationToken);
var remotePeers = await _peerConfigurationStore.ExportAsync(cancellationToken); var remotePeers = await _peerConfigurationStore.ExportAsync(cancellationToken);
var oplogEntries = await _oplogStore.ExportAsync(cancellationToken); var oplogEntries = (await _oplogStore.ExportAsync(normalizedDatasetId, cancellationToken)).ToList();
var peerConfirmations = _peerOplogConfirmationStore == null var peerConfirmations = _peerOplogConfirmationStore == null
? [] ? []
: await _peerOplogConfirmationStore.ExportAsync(cancellationToken); : await _peerOplogConfirmationStore.ExportAsync(normalizedDatasetId, cancellationToken);
if (!string.Equals(normalizedDatasetId, DatasetId.Primary, StringComparison.Ordinal))
{
var datasetDocumentKeys = oplogEntries
.Select(o => (o.Collection, o.Key))
.ToHashSet();
documents = documents.Where(document => datasetDocumentKeys.Contains((document.Collection, document.Key)));
remotePeers = [];
}
var snapshot = new SnapshotDto var snapshot = new SnapshotDto
{ {
Version = "1.0", Version = "1.0",
CreatedAt = DateTime.UtcNow.ToString("O"), CreatedAt = DateTime.UtcNow.ToString("O"),
NodeId = "", // Will be set by caller if needed NodeId = "", // Will be set by caller if needed
DatasetId = normalizedDatasetId,
Documents = Documents =
[ [
.. documents.Select(d => new DocumentDto .. documents.Select(d => new DocumentDto
{ {
DatasetId = normalizedDatasetId,
Collection = d.Collection, Collection = d.Collection,
Key = d.Key, Key = d.Key,
JsonData = d.Content.GetRawText(), JsonData = d.Content.GetRawText(),
@@ -123,6 +142,7 @@ public class SnapshotStore : ISnapshotService
[ [
.. oplogEntries.Select(o => new OplogDto .. oplogEntries.Select(o => new OplogDto
{ {
DatasetId = o.DatasetId,
Collection = o.Collection, Collection = o.Collection,
Key = o.Key, Key = o.Key,
Operation = (int)o.Operation, Operation = (int)o.Operation,
@@ -149,6 +169,7 @@ public class SnapshotStore : ISnapshotService
[ [
.. peerConfirmations.Select(c => new PeerOplogConfirmationDto .. peerConfirmations.Select(c => new PeerOplogConfirmationDto
{ {
DatasetId = c.DatasetId,
PeerNodeId = c.PeerNodeId, PeerNodeId = c.PeerNodeId,
SourceNodeId = c.SourceNodeId, SourceNodeId = c.SourceNodeId,
ConfirmedWall = c.ConfirmedWall, ConfirmedWall = c.ConfirmedWall,
@@ -174,9 +195,17 @@ public class SnapshotStore : ISnapshotService
/// <inheritdoc /> /// <inheritdoc />
public async Task ReplaceDatabaseAsync(Stream databaseStream, CancellationToken cancellationToken = default) public async Task ReplaceDatabaseAsync(Stream databaseStream, CancellationToken cancellationToken = default)
{ {
await ReplaceDatabaseAsync(databaseStream, DatasetId.Primary, cancellationToken);
}
/// <inheritdoc />
public async Task ReplaceDatabaseAsync(Stream databaseStream, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
_logger.LogWarning("Replacing data from snapshot stream..."); _logger.LogWarning("Replacing data from snapshot stream...");
await ClearAllDataAsync(cancellationToken); await ClearAllDataAsync(normalizedDatasetId, cancellationToken);
var snapshot = var snapshot =
await JsonSerializer.DeserializeAsync<SnapshotDto>(databaseStream, cancellationToken: cancellationToken); await JsonSerializer.DeserializeAsync<SnapshotDto>(databaseStream, cancellationToken: cancellationToken);
@@ -198,7 +227,8 @@ public class SnapshotStore : ISnapshotService
: JsonSerializer.Deserialize<JsonElement>(o.JsonData), : JsonSerializer.Deserialize<JsonElement>(o.JsonData),
new HlcTimestamp(o.HlcWall, o.HlcLogic, o.HlcNode), new HlcTimestamp(o.HlcWall, o.HlcLogic, o.HlcNode),
o.PreviousHash ?? string.Empty, o.PreviousHash ?? string.Empty,
string.IsNullOrWhiteSpace(o.Hash) ? null : o.Hash)).ToList(); string.IsNullOrWhiteSpace(o.Hash) ? null : o.Hash,
string.IsNullOrWhiteSpace(o.DatasetId) ? normalizedDatasetId : o.DatasetId)).ToList();
var remotePeers = snapshot.RemotePeers.Select(p => new RemotePeerConfiguration var remotePeers = snapshot.RemotePeers.Select(p => new RemotePeerConfiguration
{ {
@@ -210,6 +240,7 @@ public class SnapshotStore : ISnapshotService
}).ToList(); }).ToList();
var peerConfirmations = (snapshot.PeerConfirmations ?? []).Select(c => new PeerOplogConfirmation var peerConfirmations = (snapshot.PeerConfirmations ?? []).Select(c => new PeerOplogConfirmation
{ {
DatasetId = string.IsNullOrWhiteSpace(c.DatasetId) ? normalizedDatasetId : c.DatasetId,
PeerNodeId = c.PeerNodeId, PeerNodeId = c.PeerNodeId,
SourceNodeId = c.SourceNodeId, SourceNodeId = c.SourceNodeId,
ConfirmedWall = c.ConfirmedWall, ConfirmedWall = c.ConfirmedWall,
@@ -219,11 +250,15 @@ public class SnapshotStore : ISnapshotService
IsActive = c.IsActive IsActive = c.IsActive
}).ToList(); }).ToList();
if (string.Equals(normalizedDatasetId, DatasetId.Primary, StringComparison.Ordinal))
await _documentStore.ImportAsync(documents, cancellationToken); await _documentStore.ImportAsync(documents, cancellationToken);
await _oplogStore.ImportAsync(oplogEntries, cancellationToken); else
await _documentStore.MergeAsync(documents, cancellationToken);
await _oplogStore.ImportAsync(oplogEntries, normalizedDatasetId, cancellationToken);
if (string.Equals(normalizedDatasetId, DatasetId.Primary, StringComparison.Ordinal))
await _peerConfigurationStore.ImportAsync(remotePeers, cancellationToken); await _peerConfigurationStore.ImportAsync(remotePeers, cancellationToken);
if (_peerOplogConfirmationStore != null) if (_peerOplogConfirmationStore != null)
await _peerOplogConfirmationStore.ImportAsync(peerConfirmations, cancellationToken); await _peerOplogConfirmationStore.ImportAsync(peerConfirmations, normalizedDatasetId, cancellationToken);
_logger.LogInformation("Database replaced successfully."); _logger.LogInformation("Database replaced successfully.");
} }
@@ -231,6 +266,14 @@ public class SnapshotStore : ISnapshotService
/// <inheritdoc /> /// <inheritdoc />
public async Task MergeSnapshotAsync(Stream snapshotStream, CancellationToken cancellationToken = default) public async Task MergeSnapshotAsync(Stream snapshotStream, CancellationToken cancellationToken = default)
{ {
await MergeSnapshotAsync(snapshotStream, DatasetId.Primary, cancellationToken);
}
/// <inheritdoc />
public async Task MergeSnapshotAsync(Stream snapshotStream, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = DatasetId.Normalize(datasetId);
_logger.LogInformation("Merging snapshot from stream..."); _logger.LogInformation("Merging snapshot from stream...");
var snapshot = var snapshot =
await JsonSerializer.DeserializeAsync<SnapshotDto>(snapshotStream, cancellationToken: cancellationToken); await JsonSerializer.DeserializeAsync<SnapshotDto>(snapshotStream, cancellationToken: cancellationToken);
@@ -250,7 +293,8 @@ public class SnapshotStore : ISnapshotService
: JsonSerializer.Deserialize<JsonElement>(o.JsonData), : JsonSerializer.Deserialize<JsonElement>(o.JsonData),
new HlcTimestamp(o.HlcWall, o.HlcLogic, o.HlcNode), new HlcTimestamp(o.HlcWall, o.HlcLogic, o.HlcNode),
o.PreviousHash ?? string.Empty, o.PreviousHash ?? string.Empty,
string.IsNullOrWhiteSpace(o.Hash) ? null : o.Hash)).ToList(); string.IsNullOrWhiteSpace(o.Hash) ? null : o.Hash,
string.IsNullOrWhiteSpace(o.DatasetId) ? normalizedDatasetId : o.DatasetId)).ToList();
var remotePeers = snapshot.RemotePeers.Select(p => new RemotePeerConfiguration var remotePeers = snapshot.RemotePeers.Select(p => new RemotePeerConfiguration
{ {
NodeId = p.NodeId, NodeId = p.NodeId,
@@ -261,6 +305,7 @@ public class SnapshotStore : ISnapshotService
}).ToList(); }).ToList();
var peerConfirmations = (snapshot.PeerConfirmations ?? []).Select(c => new PeerOplogConfirmation var peerConfirmations = (snapshot.PeerConfirmations ?? []).Select(c => new PeerOplogConfirmation
{ {
DatasetId = string.IsNullOrWhiteSpace(c.DatasetId) ? normalizedDatasetId : c.DatasetId,
PeerNodeId = c.PeerNodeId, PeerNodeId = c.PeerNodeId,
SourceNodeId = c.SourceNodeId, SourceNodeId = c.SourceNodeId,
ConfirmedWall = c.ConfirmedWall, ConfirmedWall = c.ConfirmedWall,
@@ -271,19 +316,24 @@ public class SnapshotStore : ISnapshotService
}).ToList(); }).ToList();
await _documentStore.MergeAsync(documents, cancellationToken); await _documentStore.MergeAsync(documents, cancellationToken);
await _oplogStore.MergeAsync(oplogEntries, cancellationToken); await _oplogStore.MergeAsync(oplogEntries, normalizedDatasetId, cancellationToken);
if (string.Equals(normalizedDatasetId, DatasetId.Primary, StringComparison.Ordinal))
await _peerConfigurationStore.MergeAsync(remotePeers, cancellationToken); await _peerConfigurationStore.MergeAsync(remotePeers, cancellationToken);
if (_peerOplogConfirmationStore != null) if (_peerOplogConfirmationStore != null)
await _peerOplogConfirmationStore.MergeAsync(peerConfirmations, cancellationToken); await _peerOplogConfirmationStore.MergeAsync(peerConfirmations, normalizedDatasetId, cancellationToken);
_logger.LogInformation("Snapshot merged successfully."); _logger.LogInformation("Snapshot merged successfully.");
} }
private async Task ClearAllDataAsync(CancellationToken cancellationToken = default) private async Task ClearAllDataAsync(string datasetId, CancellationToken cancellationToken = default)
{
await _oplogStore.DropAsync(datasetId, cancellationToken);
if (_peerOplogConfirmationStore != null) await _peerOplogConfirmationStore.DropAsync(datasetId, cancellationToken);
if (string.Equals(datasetId, DatasetId.Primary, StringComparison.Ordinal))
{ {
await _documentStore.DropAsync(cancellationToken); await _documentStore.DropAsync(cancellationToken);
await _peerConfigurationStore.DropAsync(cancellationToken); await _peerConfigurationStore.DropAsync(cancellationToken);
await _oplogStore.DropAsync(cancellationToken); }
if (_peerOplogConfirmationStore != null) await _peerOplogConfirmationStore.DropAsync(cancellationToken);
} }
} }

View File

@@ -1,5 +1,6 @@
using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions; using Microsoft.Extensions.DependencyInjection.Extensions;
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Network; using ZB.MOM.WW.CBDDC.Core.Network;
using SurrealDb.Net; using SurrealDb.Net;
using ZB.MOM.WW.CBDDC.Core.Storage; using ZB.MOM.WW.CBDDC.Core.Storage;
@@ -46,6 +47,62 @@ public static class CBDDCSurrealEmbeddedExtensions
return services; return services;
} }
/// <summary>
/// Registers dataset synchronization options for a Surreal-backed dataset pipeline.
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="configure">Optional per-dataset option overrides.</param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddCBDDCSurrealEmbeddedDataset(
this IServiceCollection services,
string datasetId,
Action<DatasetSyncOptions>? configure = null)
{
if (services == null) throw new ArgumentNullException(nameof(services));
var options = new DatasetSyncOptions
{
DatasetId = DatasetId.Normalize(datasetId),
Enabled = true
};
configure?.Invoke(options);
options.DatasetId = DatasetId.Normalize(options.DatasetId);
services.AddSingleton(options);
return services;
}
/// <summary>
/// Registers dataset synchronization options for a Surreal-backed dataset pipeline.
/// </summary>
/// <param name="services">The service collection.</param>
/// <param name="configure">Configuration delegate for dataset options.</param>
/// <returns>The service collection for chaining.</returns>
public static IServiceCollection AddCBDDCSurrealEmbeddedDataset(
this IServiceCollection services,
Action<DatasetSyncOptions> configure)
{
if (configure == null) throw new ArgumentNullException(nameof(configure));
var options = new DatasetSyncOptions
{
DatasetId = DatasetId.Primary,
Enabled = true
};
configure(options);
return services.AddCBDDCSurrealEmbeddedDataset(options.DatasetId, configured =>
{
configured.Enabled = options.Enabled;
configured.SyncLoopDelay = options.SyncLoopDelay;
configured.MaxPeersPerCycle = options.MaxPeersPerCycle;
configured.MaxEntriesPerCycle = options.MaxEntriesPerCycle;
configured.MaintenanceIntervalOverride = options.MaintenanceIntervalOverride;
configured.InterestingCollections = options.InterestingCollections.ToList();
});
}
private static void RegisterCoreServices( private static void RegisterCoreServices(
IServiceCollection services, IServiceCollection services,
Func<IServiceProvider, CBDDCSurrealEmbeddedOptions> optionsFactory) Func<IServiceProvider, CBDDCSurrealEmbeddedOptions> optionsFactory)

View File

@@ -67,31 +67,31 @@ public sealed class CBDDCSurrealSchemaInitializer : ICBDDCSurrealSchemaInitializ
{ {
return $""" return $"""
DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.OplogEntriesTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral}; DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.OplogEntriesTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral};
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogHashIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS hash UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogHashIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS datasetId, hash UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS timestampPhysicalTime, timestampLogicalCounter, timestampNodeId; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS datasetId, timestampPhysicalTime, timestampLogicalCounter, timestampNodeId;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogCollectionIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS collection; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.OplogCollectionIndex} ON TABLE {CBDDCSurrealSchemaNames.OplogEntriesTable} COLUMNS datasetId, collection;
DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral}; DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral};
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.SnapshotNodeIdIndex} ON TABLE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} COLUMNS nodeId UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.SnapshotNodeIdIndex} ON TABLE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} COLUMNS datasetId, nodeId UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.SnapshotHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} COLUMNS timestampPhysicalTime, timestampLogicalCounter; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.SnapshotHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.SnapshotMetadataTable} COLUMNS datasetId, timestampPhysicalTime, timestampLogicalCounter;
DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral}; DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral};
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerNodeIdIndex} ON TABLE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} COLUMNS nodeId UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerNodeIdIndex} ON TABLE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} COLUMNS nodeId UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerEnabledIndex} ON TABLE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} COLUMNS isEnabled; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerEnabledIndex} ON TABLE {CBDDCSurrealSchemaNames.RemotePeerConfigurationsTable} COLUMNS isEnabled;
DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral}; DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral};
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataCollectionKeyIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS collection, key UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataCollectionKeyIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS datasetId, collection, key UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS hlcPhysicalTime, hlcLogicalCounter, hlcNodeId; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS datasetId, hlcPhysicalTime, hlcLogicalCounter, hlcNodeId;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataCollectionIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS collection; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.DocumentMetadataCollectionIndex} ON TABLE {CBDDCSurrealSchemaNames.DocumentMetadataTable} COLUMNS datasetId, collection;
DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral}; DEFINE TABLE OVERWRITE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} SCHEMAFULL CHANGEFEED {_changefeedRetentionLiteral};
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationPairIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS peerNodeId, sourceNodeId UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationPairIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS datasetId, peerNodeId, sourceNodeId UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationActiveIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS isActive; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationActiveIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS datasetId, isActive;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationSourceHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS sourceNodeId, confirmedWall, confirmedLogic; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.PeerConfirmationSourceHlcIndex} ON TABLE {CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable} COLUMNS datasetId, sourceNodeId, confirmedWall, confirmedLogic;
DEFINE TABLE OVERWRITE {_checkpointTable} SCHEMAFULL; DEFINE TABLE OVERWRITE {_checkpointTable} SCHEMAFULL;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.CdcCheckpointConsumerIndex} ON TABLE {_checkpointTable} COLUMNS consumerId UNIQUE; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.CdcCheckpointConsumerIndex} ON TABLE {_checkpointTable} COLUMNS datasetId, consumerId UNIQUE;
DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.CdcCheckpointVersionstampIndex} ON TABLE {_checkpointTable} COLUMNS versionstampCursor; DEFINE INDEX OVERWRITE {CBDDCSurrealSchemaNames.CdcCheckpointVersionstampIndex} ON TABLE {_checkpointTable} COLUMNS datasetId, versionstampCursor;
"""; """;
} }

View File

@@ -7,6 +7,11 @@ namespace ZB.MOM.WW.CBDDC.Persistence.Surreal;
/// </summary> /// </summary>
public sealed class SurrealCdcCheckpoint public sealed class SurrealCdcCheckpoint
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary> /// <summary>
/// Gets or sets the logical consumer identifier. /// Gets or sets the logical consumer identifier.
/// </summary> /// </summary>
@@ -48,6 +53,21 @@ public interface ISurrealCdcCheckpointPersistence
string? consumerId = null, string? consumerId = null,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Reads the checkpoint for a consumer within a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="consumerId">Optional consumer id.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The checkpoint if found; otherwise <see langword="null" />.</returns>
Task<SurrealCdcCheckpoint?> GetCheckpointAsync(
string datasetId,
string? consumerId,
CancellationToken cancellationToken = default)
{
return GetCheckpointAsync(consumerId, cancellationToken);
}
/// <summary> /// <summary>
/// Upserts checkpoint progress for a consumer. /// Upserts checkpoint progress for a consumer.
/// </summary> /// </summary>
@@ -63,6 +83,26 @@ public interface ISurrealCdcCheckpointPersistence
CancellationToken cancellationToken = default, CancellationToken cancellationToken = default,
long? versionstampCursor = null); long? versionstampCursor = null);
/// <summary>
/// Upserts checkpoint progress for a consumer and dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="timestamp">The last processed timestamp.</param>
/// <param name="lastHash">The last processed hash.</param>
/// <param name="consumerId">Optional consumer id.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <param name="versionstampCursor">Optional changefeed versionstamp cursor.</param>
Task UpsertCheckpointAsync(
string datasetId,
HlcTimestamp timestamp,
string lastHash,
string? consumerId = null,
CancellationToken cancellationToken = default,
long? versionstampCursor = null)
{
return UpsertCheckpointAsync(timestamp, lastHash, consumerId, cancellationToken, versionstampCursor);
}
/// <summary> /// <summary>
/// Advances checkpoint progress from an oplog entry. /// Advances checkpoint progress from an oplog entry.
/// </summary> /// </summary>
@@ -73,4 +113,20 @@ public interface ISurrealCdcCheckpointPersistence
OplogEntry entry, OplogEntry entry,
string? consumerId = null, string? consumerId = null,
CancellationToken cancellationToken = default); CancellationToken cancellationToken = default);
/// <summary>
/// Advances checkpoint progress for a specific dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="entry">The oplog entry that was processed.</param>
/// <param name="consumerId">Optional consumer id.</param>
/// <param name="cancellationToken">A cancellation token.</param>
Task AdvanceCheckpointAsync(
string datasetId,
OplogEntry entry,
string? consumerId = null,
CancellationToken cancellationToken = default)
{
return AdvanceCheckpointAsync(entry, consumerId, cancellationToken);
}
} }

View File

@@ -49,11 +49,21 @@ public sealed class SurrealCdcCheckpointPersistence : ISurrealCdcCheckpointPersi
public async Task<SurrealCdcCheckpoint?> GetCheckpointAsync( public async Task<SurrealCdcCheckpoint?> GetCheckpointAsync(
string? consumerId = null, string? consumerId = null,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{
return await GetCheckpointAsync(DatasetId.Primary, consumerId, cancellationToken);
}
/// <inheritdoc />
public async Task<SurrealCdcCheckpoint?> GetCheckpointAsync(
string datasetId,
string? consumerId,
CancellationToken cancellationToken = default)
{ {
if (!_enabled) return null; if (!_enabled) return null;
string resolvedConsumerId = ResolveConsumerId(consumerId); string resolvedConsumerId = ResolveConsumerId(consumerId);
var existing = await FindByConsumerIdAsync(resolvedConsumerId, cancellationToken); string resolvedDatasetId = DatasetId.Normalize(datasetId);
var existing = await FindByConsumerIdAsync(resolvedDatasetId, resolvedConsumerId, cancellationToken);
return existing?.ToDomain(); return existing?.ToDomain();
} }
@@ -64,26 +74,47 @@ public sealed class SurrealCdcCheckpointPersistence : ISurrealCdcCheckpointPersi
string? consumerId = null, string? consumerId = null,
CancellationToken cancellationToken = default, CancellationToken cancellationToken = default,
long? versionstampCursor = null) long? versionstampCursor = null)
{
await UpsertCheckpointAsync(
DatasetId.Primary,
timestamp,
lastHash,
consumerId,
cancellationToken,
versionstampCursor);
}
/// <inheritdoc />
public async Task UpsertCheckpointAsync(
string datasetId,
HlcTimestamp timestamp,
string lastHash,
string? consumerId = null,
CancellationToken cancellationToken = default,
long? versionstampCursor = null)
{ {
if (!_enabled) return; if (!_enabled) return;
string resolvedConsumerId = ResolveConsumerId(consumerId); string resolvedConsumerId = ResolveConsumerId(consumerId);
string resolvedDatasetId = DatasetId.Normalize(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
long? effectiveVersionstampCursor = versionstampCursor; long? effectiveVersionstampCursor = versionstampCursor;
if (!effectiveVersionstampCursor.HasValue) if (!effectiveVersionstampCursor.HasValue)
{ {
var existing = await FindByConsumerIdAsync( var existing = await FindByConsumerIdAsync(
resolvedDatasetId,
resolvedConsumerId, resolvedConsumerId,
cancellationToken, cancellationToken,
ensureInitialized: false); ensureInitialized: false);
effectiveVersionstampCursor = existing?.VersionstampCursor; effectiveVersionstampCursor = existing?.VersionstampCursor;
} }
RecordId recordId = RecordId.From(_checkpointTable, ComputeConsumerKey(resolvedConsumerId)); RecordId recordId = RecordId.From(_checkpointTable, ComputeConsumerKey(resolvedDatasetId, resolvedConsumerId));
var record = new SurrealCdcCheckpointRecord var record = new SurrealCdcCheckpointRecord
{ {
DatasetId = resolvedDatasetId,
ConsumerId = resolvedConsumerId, ConsumerId = resolvedConsumerId,
TimestampPhysicalTime = timestamp.PhysicalTime, TimestampPhysicalTime = timestamp.PhysicalTime,
TimestampLogicalCounter = timestamp.LogicalCounter, TimestampLogicalCounter = timestamp.LogicalCounter,
@@ -106,7 +137,18 @@ public sealed class SurrealCdcCheckpointPersistence : ISurrealCdcCheckpointPersi
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
ArgumentNullException.ThrowIfNull(entry); ArgumentNullException.ThrowIfNull(entry);
return UpsertCheckpointAsync(entry.Timestamp, entry.Hash, consumerId, cancellationToken); return UpsertCheckpointAsync(entry.DatasetId, entry.Timestamp, entry.Hash, consumerId, cancellationToken);
}
/// <inheritdoc />
public Task AdvanceCheckpointAsync(
string datasetId,
OplogEntry entry,
string? consumerId = null,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(entry);
return UpsertCheckpointAsync(datasetId, entry.Timestamp, entry.Hash, consumerId, cancellationToken);
} }
private string ResolveConsumerId(string? consumerId) private string ResolveConsumerId(string? consumerId)
@@ -124,32 +166,44 @@ public sealed class SurrealCdcCheckpointPersistence : ISurrealCdcCheckpointPersi
} }
private async Task<SurrealCdcCheckpointRecord?> FindByConsumerIdAsync( private async Task<SurrealCdcCheckpointRecord?> FindByConsumerIdAsync(
string datasetId,
string consumerId, string consumerId,
CancellationToken cancellationToken, CancellationToken cancellationToken,
bool ensureInitialized = true) bool ensureInitialized = true)
{ {
string normalizedDatasetId = DatasetId.Normalize(datasetId);
if (ensureInitialized) await EnsureReadyAsync(cancellationToken); if (ensureInitialized) await EnsureReadyAsync(cancellationToken);
RecordId deterministicId = RecordId.From(_checkpointTable, ComputeConsumerKey(consumerId)); RecordId deterministicId = RecordId.From(_checkpointTable, ComputeConsumerKey(normalizedDatasetId, consumerId));
var deterministic = await _surrealClient.Select<SurrealCdcCheckpointRecord>(deterministicId, cancellationToken); var deterministic = await _surrealClient.Select<SurrealCdcCheckpointRecord>(deterministicId, cancellationToken);
if (deterministic != null && if (deterministic != null &&
string.Equals(deterministic.DatasetId, normalizedDatasetId, StringComparison.Ordinal) &&
string.Equals(deterministic.ConsumerId, consumerId, StringComparison.Ordinal)) string.Equals(deterministic.ConsumerId, consumerId, StringComparison.Ordinal))
return deterministic; return deterministic;
var all = await _surrealClient.Select<SurrealCdcCheckpointRecord>(_checkpointTable, cancellationToken); var all = await _surrealClient.Select<SurrealCdcCheckpointRecord>(_checkpointTable, cancellationToken);
return all?.FirstOrDefault(c => return all?.FirstOrDefault(c =>
(string.IsNullOrWhiteSpace(c.DatasetId)
? string.Equals(normalizedDatasetId, DatasetId.Primary, StringComparison.Ordinal)
: string.Equals(c.DatasetId, normalizedDatasetId, StringComparison.Ordinal)) &&
string.Equals(c.ConsumerId, consumerId, StringComparison.Ordinal)); string.Equals(c.ConsumerId, consumerId, StringComparison.Ordinal));
} }
private static string ComputeConsumerKey(string consumerId) private static string ComputeConsumerKey(string datasetId, string consumerId)
{ {
byte[] input = Encoding.UTF8.GetBytes(consumerId); byte[] input = Encoding.UTF8.GetBytes($"{datasetId}\n{consumerId}");
return Convert.ToHexString(SHA256.HashData(input)).ToLowerInvariant(); return Convert.ToHexString(SHA256.HashData(input)).ToLowerInvariant();
} }
} }
internal sealed class SurrealCdcCheckpointRecord : Record internal sealed class SurrealCdcCheckpointRecord : Record
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
[JsonPropertyName("datasetId")]
public string DatasetId { get; set; } = global::ZB.MOM.WW.CBDDC.Core.DatasetId.Primary;
/// <summary> /// <summary>
/// Gets or sets the CDC consumer identifier. /// Gets or sets the CDC consumer identifier.
/// </summary> /// </summary>
@@ -204,6 +258,7 @@ internal static class SurrealCdcCheckpointRecordMappers
{ {
return new SurrealCdcCheckpoint return new SurrealCdcCheckpoint
{ {
DatasetId = string.IsNullOrWhiteSpace(record.DatasetId) ? DatasetId.Primary : record.DatasetId,
ConsumerId = record.ConsumerId, ConsumerId = record.ConsumerId,
Timestamp = new HlcTimestamp( Timestamp = new HlcTimestamp(
record.TimestampPhysicalTime, record.TimestampPhysicalTime,

View File

@@ -9,6 +9,7 @@ namespace ZB.MOM.WW.CBDDC.Persistence.Surreal;
public class SurrealDocumentMetadataStore : DocumentMetadataStore public class SurrealDocumentMetadataStore : DocumentMetadataStore
{ {
private const string PrimaryDatasetId = DatasetId.Primary;
private readonly ILogger<SurrealDocumentMetadataStore> _logger; private readonly ILogger<SurrealDocumentMetadataStore> _logger;
private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer; private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer;
private readonly ISurrealDbClient _surrealClient; private readonly ISurrealDbClient _surrealClient;
@@ -30,11 +31,35 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
_logger = logger ?? NullLogger<SurrealDocumentMetadataStore>.Instance; _logger = logger ?? NullLogger<SurrealDocumentMetadataStore>.Instance;
} }
private static string NormalizeDatasetId(string? datasetId)
{
return DatasetId.Normalize(datasetId);
}
private static bool MatchesDataset(string? recordDatasetId, string datasetId)
{
if (string.IsNullOrWhiteSpace(recordDatasetId))
return string.Equals(datasetId, PrimaryDatasetId, StringComparison.Ordinal);
return string.Equals(NormalizeDatasetId(recordDatasetId), datasetId, StringComparison.Ordinal);
}
/// <inheritdoc /> /// <inheritdoc />
public override async Task<DocumentMetadata?> GetMetadataAsync(string collection, string key, public override async Task<DocumentMetadata?> GetMetadataAsync(string collection, string key,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var existing = await FindByCollectionKeyAsync(collection, key, cancellationToken); var existing = await FindByCollectionKeyAsync(collection, key, PrimaryDatasetId, cancellationToken);
return existing?.ToDomain();
}
/// <inheritdoc />
public async Task<DocumentMetadata?> GetMetadataAsync(
string collection,
string key,
string datasetId,
CancellationToken cancellationToken = default)
{
var existing = await FindByCollectionKeyAsync(collection, key, NormalizeDatasetId(datasetId), cancellationToken);
return existing?.ToDomain(); return existing?.ToDomain();
} }
@@ -42,7 +67,20 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
public override async Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(string collection, public override async Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(string collection,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
return all
.Where(m => string.Equals(m.Collection, collection, StringComparison.Ordinal))
.Select(m => m.ToDomain())
.ToList();
}
/// <inheritdoc />
public async Task<IEnumerable<DocumentMetadata>> GetMetadataByCollectionAsync(
string collection,
string datasetId,
CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all return all
.Where(m => string.Equals(m.Collection, collection, StringComparison.Ordinal)) .Where(m => string.Equals(m.Collection, collection, StringComparison.Ordinal))
.Select(m => m.ToDomain()) .Select(m => m.ToDomain())
@@ -53,10 +91,26 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
public override async Task UpsertMetadataAsync(DocumentMetadata metadata, public override async Task UpsertMetadataAsync(DocumentMetadata metadata,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await UpsertMetadataAsync(metadata, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task UpsertMetadataAsync(
DocumentMetadata metadata,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
metadata.DatasetId = normalizedDatasetId;
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
var existing = await FindByCollectionKeyAsync(metadata.Collection, metadata.Key, cancellationToken); var existing =
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.DocumentMetadata(metadata.Collection, metadata.Key); await FindByCollectionKeyAsync(metadata.Collection, metadata.Key, normalizedDatasetId, cancellationToken);
RecordId recordId = existing?.Id ??
SurrealStoreRecordIds.DocumentMetadata(
metadata.Collection,
metadata.Key,
normalizedDatasetId);
await _surrealClient.Upsert<SurrealDocumentMetadataRecord, SurrealDocumentMetadataRecord>( await _surrealClient.Upsert<SurrealDocumentMetadataRecord, SurrealDocumentMetadataRecord>(
recordId, recordId,
@@ -67,24 +121,46 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task UpsertMetadataBatchAsync(IEnumerable<DocumentMetadata> metadatas, public override async Task UpsertMetadataBatchAsync(IEnumerable<DocumentMetadata> metadatas,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{
await UpsertMetadataBatchAsync(metadatas, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task UpsertMetadataBatchAsync(IEnumerable<DocumentMetadata> metadatas,
string datasetId,
CancellationToken cancellationToken = default)
{ {
foreach (var metadata in metadatas) foreach (var metadata in metadatas)
await UpsertMetadataAsync(metadata, cancellationToken); await UpsertMetadataAsync(metadata, datasetId, cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task MarkDeletedAsync(string collection, string key, HlcTimestamp timestamp, public override async Task MarkDeletedAsync(string collection, string key, HlcTimestamp timestamp,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var metadata = new DocumentMetadata(collection, key, timestamp, true); await MarkDeletedAsync(collection, key, timestamp, PrimaryDatasetId, cancellationToken);
await UpsertMetadataAsync(metadata, cancellationToken); }
/// <inheritdoc />
public async Task MarkDeletedAsync(string collection, string key, HlcTimestamp timestamp, string datasetId,
CancellationToken cancellationToken = default)
{
var metadata = new DocumentMetadata(collection, key, timestamp, true, datasetId);
await UpsertMetadataAsync(metadata, datasetId, cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(HlcTimestamp since, public override async Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(HlcTimestamp since,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default) IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); return await GetMetadataAfterAsync(since, PrimaryDatasetId, collections, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<DocumentMetadata>> GetMetadataAfterAsync(HlcTimestamp since, string datasetId,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null; HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null;
return all return all
@@ -101,14 +177,45 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task DropAsync(CancellationToken cancellationToken = default) public override async Task DropAsync(CancellationToken cancellationToken = default)
{ {
await EnsureReadyAsync(cancellationToken); await DropAsync(PrimaryDatasetId, cancellationToken);
await _surrealClient.Delete(CBDDCSurrealSchemaNames.DocumentMetadataTable, cancellationToken); }
/// <summary>
/// Drops all metadata rows for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task DropAsync(string datasetId, CancellationToken cancellationToken = default)
{
var rows = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
foreach (var row in rows)
{
var recordId = row.Id ??
SurrealStoreRecordIds.DocumentMetadata(
row.Collection,
row.Key,
NormalizeDatasetId(datasetId));
await _surrealClient.Delete(recordId, cancellationToken);
}
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<DocumentMetadata>> ExportAsync(CancellationToken cancellationToken = default) public override async Task<IEnumerable<DocumentMetadata>> ExportAsync(CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
return all.Select(m => m.ToDomain()).ToList();
}
/// <summary>
/// Exports metadata rows for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The exported metadata rows.</returns>
public async Task<IEnumerable<DocumentMetadata>> ExportAsync(string datasetId,
CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all.Select(m => m.ToDomain()).ToList(); return all.Select(m => m.ToDomain()).ToList();
} }
@@ -116,16 +223,42 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
public override async Task ImportAsync(IEnumerable<DocumentMetadata> items, public override async Task ImportAsync(IEnumerable<DocumentMetadata> items,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
foreach (var item in items) await UpsertMetadataAsync(item, cancellationToken); await ImportAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Imports metadata rows into a dataset.
/// </summary>
/// <param name="items">The metadata items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task ImportAsync(IEnumerable<DocumentMetadata> items, string datasetId,
CancellationToken cancellationToken = default)
{
foreach (var item in items) await UpsertMetadataAsync(item, datasetId, cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task MergeAsync(IEnumerable<DocumentMetadata> items, public override async Task MergeAsync(IEnumerable<DocumentMetadata> items,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await MergeAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Merges metadata rows into a dataset.
/// </summary>
/// <param name="items">The metadata items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task MergeAsync(IEnumerable<DocumentMetadata> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByCollectionKeyAsync(item.Collection, item.Key, cancellationToken); item.DatasetId = normalizedDatasetId;
var existing = await FindByCollectionKeyAsync(item.Collection, item.Key, normalizedDatasetId, cancellationToken);
if (existing == null) if (existing == null)
{ {
@@ -138,7 +271,8 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
if (item.UpdatedAt.CompareTo(existingTimestamp) <= 0) continue; if (item.UpdatedAt.CompareTo(existingTimestamp) <= 0) continue;
RecordId recordId = existing.Id ?? SurrealStoreRecordIds.DocumentMetadata(item.Collection, item.Key); RecordId recordId =
existing.Id ?? SurrealStoreRecordIds.DocumentMetadata(item.Collection, item.Key, normalizedDatasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
await _surrealClient.Upsert<SurrealDocumentMetadataRecord, SurrealDocumentMetadataRecord>( await _surrealClient.Upsert<SurrealDocumentMetadataRecord, SurrealDocumentMetadataRecord>(
recordId, recordId,
@@ -152,27 +286,37 @@ public class SurrealDocumentMetadataStore : DocumentMetadataStore
await _schemaInitializer.EnsureInitializedAsync(cancellationToken); await _schemaInitializer.EnsureInitializedAsync(cancellationToken);
} }
private async Task<List<SurrealDocumentMetadataRecord>> SelectAllAsync(CancellationToken cancellationToken) private async Task<List<SurrealDocumentMetadataRecord>> SelectAllAsync(string datasetId,
CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
var rows = await _surrealClient.Select<SurrealDocumentMetadataRecord>( var rows = await _surrealClient.Select<SurrealDocumentMetadataRecord>(
CBDDCSurrealSchemaNames.DocumentMetadataTable, CBDDCSurrealSchemaNames.DocumentMetadataTable,
cancellationToken); cancellationToken);
return rows?.ToList() ?? []; return rows?
.Where(row => MatchesDataset(row.DatasetId, normalizedDatasetId))
.ToList()
?? [];
} }
private async Task<SurrealDocumentMetadataRecord?> FindByCollectionKeyAsync(string collection, string key, private async Task<SurrealDocumentMetadataRecord?> FindByCollectionKeyAsync(
string collection,
string key,
string datasetId,
CancellationToken cancellationToken) CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
RecordId deterministicId = SurrealStoreRecordIds.DocumentMetadata(collection, key); RecordId deterministicId = SurrealStoreRecordIds.DocumentMetadata(collection, key, normalizedDatasetId);
var deterministic = await _surrealClient.Select<SurrealDocumentMetadataRecord>(deterministicId, cancellationToken); var deterministic = await _surrealClient.Select<SurrealDocumentMetadataRecord>(deterministicId, cancellationToken);
if (deterministic != null && if (deterministic != null &&
MatchesDataset(deterministic.DatasetId, normalizedDatasetId) &&
string.Equals(deterministic.Collection, collection, StringComparison.Ordinal) && string.Equals(deterministic.Collection, collection, StringComparison.Ordinal) &&
string.Equals(deterministic.Key, key, StringComparison.Ordinal)) string.Equals(deterministic.Key, key, StringComparison.Ordinal))
return deterministic; return deterministic;
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
return all.FirstOrDefault(m => return all.FirstOrDefault(m =>
string.Equals(m.Collection, collection, StringComparison.Ordinal) && string.Equals(m.Collection, collection, StringComparison.Ordinal) &&
string.Equals(m.Key, key, StringComparison.Ordinal)); string.Equals(m.Key, key, StringComparison.Ordinal));

View File

@@ -1205,7 +1205,10 @@ public abstract class SurrealDocumentStore<TContext> : IDocumentStore, ISurrealC
{ {
["oplogRecordId"] = SurrealStoreRecordIds.Oplog(oplogEntry.Hash), ["oplogRecordId"] = SurrealStoreRecordIds.Oplog(oplogEntry.Hash),
["oplogRecord"] = oplogEntry.ToSurrealRecord(), ["oplogRecord"] = oplogEntry.ToSurrealRecord(),
["metadataRecordId"] = SurrealStoreRecordIds.DocumentMetadata(metadata.Collection, metadata.Key), ["metadataRecordId"] = SurrealStoreRecordIds.DocumentMetadata(
metadata.Collection,
metadata.Key,
metadata.DatasetId),
["metadataRecord"] = metadata.ToSurrealRecord() ["metadataRecord"] = metadata.ToSurrealRecord()
}; };
@@ -1261,10 +1264,12 @@ public abstract class SurrealDocumentStore<TContext> : IDocumentStore, ISurrealC
checkpointRecord = new Dictionary<string, object?>(); checkpointRecord = new Dictionary<string, object?>();
if (!TryGetCheckpointSettings(out string checkpointTable, out string consumerId)) return false; if (!TryGetCheckpointSettings(out string checkpointTable, out string consumerId)) return false;
string consumerKey = ComputeConsumerKey(consumerId); const string datasetId = DatasetId.Primary;
string consumerKey = ComputeConsumerKey(datasetId, consumerId);
checkpointRecordId = RecordId.From(checkpointTable, consumerKey); checkpointRecordId = RecordId.From(checkpointTable, consumerKey);
checkpointRecord = new Dictionary<string, object?> checkpointRecord = new Dictionary<string, object?>
{ {
["datasetId"] = datasetId,
["consumerId"] = consumerId, ["consumerId"] = consumerId,
["timestampPhysicalTime"] = oplogEntry.Timestamp.PhysicalTime, ["timestampPhysicalTime"] = oplogEntry.Timestamp.PhysicalTime,
["timestampLogicalCounter"] = oplogEntry.Timestamp.LogicalCounter, ["timestampLogicalCounter"] = oplogEntry.Timestamp.LogicalCounter,
@@ -1294,10 +1299,12 @@ public abstract class SurrealDocumentStore<TContext> : IDocumentStore, ISurrealC
? long.MaxValue ? long.MaxValue
: (long)pendingCursorCheckpoint.Value.Cursor; : (long)pendingCursorCheckpoint.Value.Cursor;
string consumerKey = ComputeConsumerKey(cursorConsumerId); const string datasetId = DatasetId.Primary;
string consumerKey = ComputeConsumerKey(datasetId, cursorConsumerId);
checkpointRecordId = RecordId.From(checkpointTable, consumerKey); checkpointRecordId = RecordId.From(checkpointTable, consumerKey);
checkpointRecord = new Dictionary<string, object?> checkpointRecord = new Dictionary<string, object?>
{ {
["datasetId"] = datasetId,
["consumerId"] = cursorConsumerId, ["consumerId"] = cursorConsumerId,
["timestampPhysicalTime"] = encodedCursor, ["timestampPhysicalTime"] = encodedCursor,
["timestampLogicalCounter"] = 0, ["timestampLogicalCounter"] = 0,
@@ -1329,9 +1336,9 @@ public abstract class SurrealDocumentStore<TContext> : IDocumentStore, ISurrealC
return true; return true;
} }
private static string ComputeConsumerKey(string consumerId) private static string ComputeConsumerKey(string datasetId, string consumerId)
{ {
byte[] bytes = Encoding.UTF8.GetBytes(consumerId); byte[] bytes = Encoding.UTF8.GetBytes($"{datasetId}\n{consumerId}");
return Convert.ToHexString(SHA256.HashData(bytes)).ToLowerInvariant(); return Convert.ToHexString(SHA256.HashData(bytes)).ToLowerInvariant();
} }

View File

@@ -10,6 +10,7 @@ namespace ZB.MOM.WW.CBDDC.Persistence.Surreal;
public class SurrealOplogStore : OplogStore public class SurrealOplogStore : OplogStore
{ {
private const string PrimaryDatasetId = DatasetId.Primary;
private readonly ILogger<SurrealOplogStore> _logger; private readonly ILogger<SurrealOplogStore> _logger;
private readonly ICBDDCSurrealSchemaInitializer? _schemaInitializer; private readonly ICBDDCSurrealSchemaInitializer? _schemaInitializer;
private readonly ISurrealDbClient? _surrealClient; private readonly ISurrealDbClient? _surrealClient;
@@ -46,17 +47,38 @@ public class SurrealOplogStore : OplogStore
InitializeVectorClock(); InitializeVectorClock();
} }
private static string NormalizeDatasetId(string? datasetId)
{
return DatasetId.Normalize(datasetId);
}
private static bool MatchesDataset(string? recordDatasetId, string datasetId)
{
if (string.IsNullOrWhiteSpace(recordDatasetId))
return string.Equals(datasetId, PrimaryDatasetId, StringComparison.Ordinal);
return string.Equals(NormalizeDatasetId(recordDatasetId), datasetId, StringComparison.Ordinal);
}
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<OplogEntry>> GetChainRangeAsync(string startHash, string endHash, public override async Task<IEnumerable<OplogEntry>> GetChainRangeAsync(string startHash, string endHash,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var startRow = await FindByHashAsync(startHash, cancellationToken); return await GetChainRangeAsync(startHash, endHash, PrimaryDatasetId, cancellationToken);
var endRow = await FindByHashAsync(endHash, cancellationToken); }
/// <inheritdoc />
public async Task<IEnumerable<OplogEntry>> GetChainRangeAsync(string startHash, string endHash, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var startRow = await FindByHashAsync(startHash, normalizedDatasetId, cancellationToken);
var endRow = await FindByHashAsync(endHash, normalizedDatasetId, cancellationToken);
if (startRow == null || endRow == null) return []; if (startRow == null || endRow == null) return [];
string nodeId = startRow.TimestampNodeId; string nodeId = startRow.TimestampNodeId;
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
return all return all
.Where(o => string.Equals(o.TimestampNodeId, nodeId, StringComparison.Ordinal) && .Where(o => string.Equals(o.TimestampNodeId, nodeId, StringComparison.Ordinal) &&
@@ -75,7 +97,16 @@ public class SurrealOplogStore : OplogStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task<OplogEntry?> GetEntryByHashAsync(string hash, CancellationToken cancellationToken = default) public override async Task<OplogEntry?> GetEntryByHashAsync(string hash, CancellationToken cancellationToken = default)
{ {
var existing = await FindByHashAsync(hash, cancellationToken); var existing = await FindByHashAsync(hash, PrimaryDatasetId, cancellationToken);
return existing?.ToDomain();
}
/// <inheritdoc />
public async Task<OplogEntry?> GetEntryByHashAsync(string hash, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var existing = await FindByHashAsync(hash, normalizedDatasetId, cancellationToken);
return existing?.ToDomain(); return existing?.ToDomain();
} }
@@ -83,7 +114,15 @@ public class SurrealOplogStore : OplogStore
public override async Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(HlcTimestamp timestamp, public override async Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(HlcTimestamp timestamp,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default) IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); return await GetOplogAfterAsync(timestamp, PrimaryDatasetId, collections, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<OplogEntry>> GetOplogAfterAsync(HlcTimestamp timestamp, string datasetId,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null; HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null;
return all return all
@@ -102,7 +141,15 @@ public class SurrealOplogStore : OplogStore
public override async Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(string nodeId, HlcTimestamp since, public override async Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(string nodeId, HlcTimestamp since,
IEnumerable<string>? collections = null, CancellationToken cancellationToken = default) IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); return await GetOplogForNodeAfterAsync(nodeId, since, PrimaryDatasetId, collections, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<OplogEntry>> GetOplogForNodeAfterAsync(string nodeId, HlcTimestamp since,
string datasetId, IEnumerable<string>? collections = null, CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null; HashSet<string>? collectionSet = collections != null ? new HashSet<string>(collections) : null;
return all return all
@@ -121,7 +168,15 @@ public class SurrealOplogStore : OplogStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task PruneOplogAsync(HlcTimestamp cutoff, CancellationToken cancellationToken = default) public override async Task PruneOplogAsync(HlcTimestamp cutoff, CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); await PruneOplogAsync(cutoff, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task PruneOplogAsync(HlcTimestamp cutoff, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
var toDelete = all var toDelete = all
.Where(o => o.TimestampPhysicalTime < cutoff.PhysicalTime || .Where(o => o.TimestampPhysicalTime < cutoff.PhysicalTime ||
(o.TimestampPhysicalTime == cutoff.PhysicalTime && (o.TimestampPhysicalTime == cutoff.PhysicalTime &&
@@ -139,38 +194,114 @@ public class SurrealOplogStore : OplogStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task DropAsync(CancellationToken cancellationToken = default) public override async Task DropAsync(CancellationToken cancellationToken = default)
{ {
await EnsureReadyAsync(cancellationToken); await DropAsync(PrimaryDatasetId, cancellationToken);
await _surrealClient!.Delete(CBDDCSurrealSchemaNames.OplogEntriesTable, cancellationToken);
_vectorClock.Invalidate(); _vectorClock.Invalidate();
} }
/// <summary>
/// Drops all oplog rows for the specified dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task DropAsync(string datasetId, CancellationToken cancellationToken = default)
{
var rows = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
foreach (var row in rows)
{
RecordId recordId = row.Id ?? SurrealStoreRecordIds.Oplog(row.Hash);
await EnsureReadyAsync(cancellationToken);
await _surrealClient!.Delete(recordId, cancellationToken);
}
}
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<OplogEntry>> ExportAsync(CancellationToken cancellationToken = default) public override async Task<IEnumerable<OplogEntry>> ExportAsync(CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
return all.Select(o => o.ToDomain()).ToList();
}
/// <summary>
/// Exports all oplog entries for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>All dataset-scoped oplog entries.</returns>
public async Task<IEnumerable<OplogEntry>> ExportAsync(string datasetId, CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all.Select(o => o.ToDomain()).ToList(); return all.Select(o => o.ToDomain()).ToList();
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task ImportAsync(IEnumerable<OplogEntry> items, CancellationToken cancellationToken = default) public override async Task ImportAsync(IEnumerable<OplogEntry> items, CancellationToken cancellationToken = default)
{ {
await ImportAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Imports oplog entries for the specified dataset.
/// </summary>
/// <param name="items">The entries to import.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task ImportAsync(IEnumerable<OplogEntry> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
Dictionary<string, RecordId> existingByHash =
await LoadOplogRecordIdsByHashAsync(normalizedDatasetId, cancellationToken);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByHashAsync(item.Hash, cancellationToken); var normalizedItem = new OplogEntry(
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.Oplog(item.Hash); item.Collection,
await UpsertAsync(item, recordId, cancellationToken); item.Key,
item.Operation,
item.Payload,
item.Timestamp,
item.PreviousHash,
item.Hash,
normalizedDatasetId);
RecordId recordId = existingByHash.TryGetValue(item.Hash, out RecordId? existingRecordId)
? existingRecordId
: SurrealStoreRecordIds.Oplog(item.Hash);
await UpsertAsync(normalizedItem, recordId, cancellationToken);
existingByHash[item.Hash] = recordId;
} }
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task MergeAsync(IEnumerable<OplogEntry> items, CancellationToken cancellationToken = default) public override async Task MergeAsync(IEnumerable<OplogEntry> items, CancellationToken cancellationToken = default)
{ {
await MergeAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Merges oplog entries into the specified dataset.
/// </summary>
/// <param name="items">The entries to merge.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task MergeAsync(IEnumerable<OplogEntry> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
HashSet<string> existingHashes = await LoadOplogHashesAsync(normalizedDatasetId, cancellationToken);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByHashAsync(item.Hash, cancellationToken); if (!existingHashes.Add(item.Hash))
if (existing != null) continue; continue;
await UpsertAsync(item, SurrealStoreRecordIds.Oplog(item.Hash), cancellationToken); var normalizedItem = new OplogEntry(
item.Collection,
item.Key,
item.Operation,
item.Payload,
item.Timestamp,
item.PreviousHash,
item.Hash,
normalizedDatasetId);
await UpsertAsync(normalizedItem, SurrealStoreRecordIds.Oplog(normalizedItem.Hash), cancellationToken);
} }
} }
@@ -190,6 +321,8 @@ public class SurrealOplogStore : OplogStore
{ {
var snapshots = _snapshotMetadataStore.GetAllSnapshotMetadataAsync().GetAwaiter().GetResult(); var snapshots = _snapshotMetadataStore.GetAllSnapshotMetadataAsync().GetAwaiter().GetResult();
foreach (var snapshot in snapshots) foreach (var snapshot in snapshots)
{
if (!MatchesDataset(snapshot.DatasetId, PrimaryDatasetId)) continue;
_vectorClock.UpdateNode( _vectorClock.UpdateNode(
snapshot.NodeId, snapshot.NodeId,
new HlcTimestamp( new HlcTimestamp(
@@ -198,6 +331,7 @@ public class SurrealOplogStore : OplogStore
snapshot.NodeId), snapshot.NodeId),
snapshot.Hash ?? ""); snapshot.Hash ?? "");
} }
}
catch catch
{ {
// Ignore snapshot bootstrap failures to keep oplog fallback behavior aligned. // Ignore snapshot bootstrap failures to keep oplog fallback behavior aligned.
@@ -209,6 +343,7 @@ public class SurrealOplogStore : OplogStore
?? []; ?? [];
var latestPerNode = all var latestPerNode = all
.Where(x => MatchesDataset(x.DatasetId, PrimaryDatasetId))
.Where(x => !string.IsNullOrWhiteSpace(x.TimestampNodeId)) .Where(x => !string.IsNullOrWhiteSpace(x.TimestampNodeId))
.GroupBy(x => x.TimestampNodeId) .GroupBy(x => x.TimestampNodeId)
.Select(g => g .Select(g => g
@@ -229,17 +364,27 @@ public class SurrealOplogStore : OplogStore
/// <inheritdoc /> /// <inheritdoc />
protected override async Task InsertOplogEntryAsync(OplogEntry entry, CancellationToken cancellationToken = default) protected override async Task InsertOplogEntryAsync(OplogEntry entry, CancellationToken cancellationToken = default)
{ {
var existing = await FindByHashAsync(entry.Hash, cancellationToken); string datasetId = NormalizeDatasetId(entry.DatasetId);
var normalizedEntry = new OplogEntry(
entry.Collection,
entry.Key,
entry.Operation,
entry.Payload,
entry.Timestamp,
entry.PreviousHash,
entry.Hash,
datasetId);
var existing = await FindByHashAsync(normalizedEntry.Hash, datasetId, cancellationToken);
if (existing != null) return; if (existing != null) return;
await UpsertAsync(entry, SurrealStoreRecordIds.Oplog(entry.Hash), cancellationToken); await UpsertAsync(normalizedEntry, SurrealStoreRecordIds.Oplog(normalizedEntry.Hash), cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
protected override async Task<string?> QueryLastHashForNodeAsync(string nodeId, protected override async Task<string?> QueryLastHashForNodeAsync(string nodeId,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
var lastEntry = all var lastEntry = all
.Where(o => string.Equals(o.TimestampNodeId, nodeId, StringComparison.Ordinal)) .Where(o => string.Equals(o.TimestampNodeId, nodeId, StringComparison.Ordinal))
.OrderByDescending(o => o.TimestampPhysicalTime) .OrderByDescending(o => o.TimestampPhysicalTime)
@@ -252,11 +397,106 @@ public class SurrealOplogStore : OplogStore
protected override async Task<(long Wall, int Logic)?> QueryLastHashTimestampFromOplogAsync(string hash, protected override async Task<(long Wall, int Logic)?> QueryLastHashTimestampFromOplogAsync(string hash,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var existing = await FindByHashAsync(hash, cancellationToken); var existing = await FindByHashAsync(hash, PrimaryDatasetId, cancellationToken);
if (existing == null) return null; if (existing == null) return null;
return (existing.TimestampPhysicalTime, existing.TimestampLogicalCounter); return (existing.TimestampPhysicalTime, existing.TimestampLogicalCounter);
} }
/// <inheritdoc />
public async Task AppendOplogEntryAsync(
OplogEntry entry,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var normalizedEntry = new OplogEntry(
entry.Collection,
entry.Key,
entry.Operation,
entry.Payload,
entry.Timestamp,
entry.PreviousHash,
entry.Hash,
normalizedDatasetId);
await AppendOplogEntryAsync(normalizedEntry, cancellationToken);
}
/// <inheritdoc />
public async Task<HlcTimestamp> GetLatestTimestampAsync(string datasetId, CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
var latest = all
.OrderByDescending(o => o.TimestampPhysicalTime)
.ThenByDescending(o => o.TimestampLogicalCounter)
.FirstOrDefault();
return latest == null
? new HlcTimestamp(0, 0, "")
: new HlcTimestamp(latest.TimestampPhysicalTime, latest.TimestampLogicalCounter, latest.TimestampNodeId);
}
/// <inheritdoc />
public async Task<VectorClock> GetVectorClockAsync(string datasetId, CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
var vectorClock = new VectorClock();
foreach (var latest in all
.Where(o => !string.IsNullOrWhiteSpace(o.TimestampNodeId))
.GroupBy(o => o.TimestampNodeId)
.Select(g => g.OrderByDescending(o => o.TimestampPhysicalTime)
.ThenByDescending(o => o.TimestampLogicalCounter)
.First()))
vectorClock.SetTimestamp(
latest.TimestampNodeId,
new HlcTimestamp(latest.TimestampPhysicalTime, latest.TimestampLogicalCounter, latest.TimestampNodeId));
return vectorClock;
}
/// <inheritdoc />
public async Task<string?> GetLastEntryHashAsync(string nodeId, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
var latest = all
.Where(o => string.Equals(o.TimestampNodeId, nodeId, StringComparison.Ordinal))
.OrderByDescending(o => o.TimestampPhysicalTime)
.ThenByDescending(o => o.TimestampLogicalCounter)
.FirstOrDefault();
if (latest != null) return latest.Hash;
if (_snapshotMetadataStore == null) return null;
var snapshotHash =
await _snapshotMetadataStore.GetSnapshotHashAsync(nodeId, normalizedDatasetId, cancellationToken);
return snapshotHash;
}
/// <inheritdoc />
public async Task ApplyBatchAsync(
IEnumerable<OplogEntry> oplogEntries,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
var normalizedEntries = oplogEntries
.Select(entry => new OplogEntry(
entry.Collection,
entry.Key,
entry.Operation,
entry.Payload,
entry.Timestamp,
entry.PreviousHash,
entry.Hash,
normalizedDatasetId))
.ToList();
await ApplyBatchAsync(normalizedEntries, cancellationToken);
}
private async Task UpsertAsync(OplogEntry entry, RecordId recordId, CancellationToken cancellationToken) private async Task UpsertAsync(OplogEntry entry, RecordId recordId, CancellationToken cancellationToken)
{ {
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
@@ -271,25 +511,56 @@ public class SurrealOplogStore : OplogStore
await _schemaInitializer!.EnsureInitializedAsync(cancellationToken); await _schemaInitializer!.EnsureInitializedAsync(cancellationToken);
} }
private async Task<List<SurrealOplogRecord>> SelectAllAsync(CancellationToken cancellationToken) private async Task<List<SurrealOplogRecord>> SelectAllAsync(string datasetId, CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
var rows = await _surrealClient!.Select<SurrealOplogRecord>( var rows = await _surrealClient!.Select<SurrealOplogRecord>(
CBDDCSurrealSchemaNames.OplogEntriesTable, CBDDCSurrealSchemaNames.OplogEntriesTable,
cancellationToken); cancellationToken);
return rows?.ToList() ?? []; return rows?
.Where(row => MatchesDataset(row.DatasetId, normalizedDatasetId))
.ToList()
?? [];
} }
private async Task<SurrealOplogRecord?> FindByHashAsync(string hash, CancellationToken cancellationToken) private async Task<SurrealOplogRecord?> FindByHashAsync(string hash, string datasetId, CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
RecordId deterministicId = SurrealStoreRecordIds.Oplog(hash); RecordId deterministicId = SurrealStoreRecordIds.Oplog(hash);
var deterministic = await _surrealClient!.Select<SurrealOplogRecord>(deterministicId, cancellationToken); var deterministic = await _surrealClient!.Select<SurrealOplogRecord>(deterministicId, cancellationToken);
if (deterministic != null && string.Equals(deterministic.Hash, hash, StringComparison.Ordinal)) if (deterministic != null &&
string.Equals(deterministic.Hash, hash, StringComparison.Ordinal) &&
MatchesDataset(deterministic.DatasetId, normalizedDatasetId))
return deterministic; return deterministic;
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
return all.FirstOrDefault(o => string.Equals(o.Hash, hash, StringComparison.Ordinal)); return all.FirstOrDefault(o => string.Equals(o.Hash, hash, StringComparison.Ordinal));
} }
private async Task<HashSet<string>> LoadOplogHashesAsync(string datasetId, CancellationToken cancellationToken)
{
var rows = await SelectAllAsync(datasetId, cancellationToken);
return rows
.Where(r => !string.IsNullOrWhiteSpace(r.Hash))
.Select(r => r.Hash)
.ToHashSet(StringComparer.Ordinal);
}
private async Task<Dictionary<string, RecordId>> LoadOplogRecordIdsByHashAsync(string datasetId,
CancellationToken cancellationToken)
{
var rows = await SelectAllAsync(datasetId, cancellationToken);
var records = new Dictionary<string, RecordId>(StringComparer.Ordinal);
foreach (var row in rows)
{
if (string.IsNullOrWhiteSpace(row.Hash)) continue;
records[row.Hash] = row.Id ?? SurrealStoreRecordIds.Oplog(row.Hash);
}
return records;
}
} }

View File

@@ -10,6 +10,7 @@ namespace ZB.MOM.WW.CBDDC.Persistence.Surreal;
public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
{ {
internal const string RegistrationSourceNodeId = "__peer_registration__"; internal const string RegistrationSourceNodeId = "__peer_registration__";
private const string PrimaryDatasetId = DatasetId.Primary;
private readonly ILogger<SurrealPeerOplogConfirmationStore> _logger; private readonly ILogger<SurrealPeerOplogConfirmationStore> _logger;
private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer; private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer;
@@ -32,6 +33,19 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
_logger = logger ?? NullLogger<SurrealPeerOplogConfirmationStore>.Instance; _logger = logger ?? NullLogger<SurrealPeerOplogConfirmationStore>.Instance;
} }
private static string NormalizeDatasetId(string? datasetId)
{
return DatasetId.Normalize(datasetId);
}
private static bool MatchesDataset(string? recordDatasetId, string datasetId)
{
if (string.IsNullOrWhiteSpace(recordDatasetId))
return string.Equals(datasetId, PrimaryDatasetId, StringComparison.Ordinal);
return string.Equals(NormalizeDatasetId(recordDatasetId), datasetId, StringComparison.Ordinal);
}
/// <inheritdoc /> /// <inheritdoc />
public override async Task EnsurePeerRegisteredAsync( public override async Task EnsurePeerRegisteredAsync(
string peerNodeId, string peerNodeId,
@@ -39,16 +53,29 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
PeerType type, PeerType type,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await EnsurePeerRegisteredAsync(peerNodeId, address, type, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task EnsurePeerRegisteredAsync(
string peerNodeId,
string address,
PeerType type,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
if (string.IsNullOrWhiteSpace(peerNodeId)) if (string.IsNullOrWhiteSpace(peerNodeId))
throw new ArgumentException("Peer node id is required.", nameof(peerNodeId)); throw new ArgumentException("Peer node id is required.", nameof(peerNodeId));
var existing = var existing =
await FindByPairAsync(peerNodeId, RegistrationSourceNodeId, cancellationToken); await FindByPairAsync(peerNodeId, RegistrationSourceNodeId, normalizedDatasetId, cancellationToken);
if (existing == null) if (existing == null)
{ {
var created = new PeerOplogConfirmation var created = new PeerOplogConfirmation
{ {
DatasetId = normalizedDatasetId,
PeerNodeId = peerNodeId, PeerNodeId = peerNodeId,
SourceNodeId = RegistrationSourceNodeId, SourceNodeId = RegistrationSourceNodeId,
ConfirmedWall = 0, ConfirmedWall = 0,
@@ -58,7 +85,9 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
IsActive = true IsActive = true
}; };
await UpsertAsync(created, SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, RegistrationSourceNodeId), await UpsertAsync(
created,
SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, RegistrationSourceNodeId, normalizedDatasetId),
cancellationToken); cancellationToken);
_logger.LogDebug("Registered peer confirmation tracking for {PeerNodeId} ({Address}, {Type}).", peerNodeId, _logger.LogDebug("Registered peer confirmation tracking for {PeerNodeId} ({Address}, {Type}).", peerNodeId,
@@ -71,7 +100,8 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
existing.IsActive = true; existing.IsActive = true;
existing.LastConfirmedUtcMs = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(); existing.LastConfirmedUtcMs = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
RecordId recordId = RecordId recordId =
existing.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, RegistrationSourceNodeId); existing.Id ??
SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, RegistrationSourceNodeId, normalizedDatasetId);
await UpsertAsync(existing, recordId, cancellationToken); await UpsertAsync(existing, recordId, cancellationToken);
} }
@@ -83,19 +113,33 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
string hash, string hash,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await UpdateConfirmationAsync(peerNodeId, sourceNodeId, timestamp, hash, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task UpdateConfirmationAsync(
string peerNodeId,
string sourceNodeId,
HlcTimestamp timestamp,
string hash,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
if (string.IsNullOrWhiteSpace(peerNodeId)) if (string.IsNullOrWhiteSpace(peerNodeId))
throw new ArgumentException("Peer node id is required.", nameof(peerNodeId)); throw new ArgumentException("Peer node id is required.", nameof(peerNodeId));
if (string.IsNullOrWhiteSpace(sourceNodeId)) if (string.IsNullOrWhiteSpace(sourceNodeId))
throw new ArgumentException("Source node id is required.", nameof(sourceNodeId)); throw new ArgumentException("Source node id is required.", nameof(sourceNodeId));
var existing = await FindByPairAsync(peerNodeId, sourceNodeId, cancellationToken); var existing = await FindByPairAsync(peerNodeId, sourceNodeId, normalizedDatasetId, cancellationToken);
long nowMs = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(); long nowMs = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
if (existing == null) if (existing == null)
{ {
var created = new PeerOplogConfirmation var created = new PeerOplogConfirmation
{ {
DatasetId = normalizedDatasetId,
PeerNodeId = peerNodeId, PeerNodeId = peerNodeId,
SourceNodeId = sourceNodeId, SourceNodeId = sourceNodeId,
ConfirmedWall = timestamp.PhysicalTime, ConfirmedWall = timestamp.PhysicalTime,
@@ -104,7 +148,7 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
LastConfirmedUtc = DateTimeOffset.FromUnixTimeMilliseconds(nowMs), LastConfirmedUtc = DateTimeOffset.FromUnixTimeMilliseconds(nowMs),
IsActive = true IsActive = true
}; };
await UpsertAsync(created, SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId), await UpsertAsync(created, SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId, normalizedDatasetId),
cancellationToken); cancellationToken);
return; return;
} }
@@ -122,7 +166,7 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
existing.LastConfirmedUtcMs = nowMs; existing.LastConfirmedUtcMs = nowMs;
existing.IsActive = true; existing.IsActive = true;
RecordId recordId = existing.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId); RecordId recordId = existing.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId, normalizedDatasetId);
await UpsertAsync(existing, recordId, cancellationToken); await UpsertAsync(existing, recordId, cancellationToken);
} }
@@ -130,7 +174,15 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
public override async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync( public override async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync(
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); return await GetConfirmationsAsync(PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all return all
.Where(c => !string.Equals(c.SourceNodeId, RegistrationSourceNodeId, StringComparison.Ordinal)) .Where(c => !string.Equals(c.SourceNodeId, RegistrationSourceNodeId, StringComparison.Ordinal))
.Select(c => c.ToDomain()) .Select(c => c.ToDomain())
@@ -141,11 +193,20 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
public override async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsForPeerAsync( public override async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsForPeerAsync(
string peerNodeId, string peerNodeId,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{
return await GetConfirmationsForPeerAsync(peerNodeId, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<PeerOplogConfirmation>> GetConfirmationsForPeerAsync(
string peerNodeId,
string datasetId,
CancellationToken cancellationToken = default)
{ {
if (string.IsNullOrWhiteSpace(peerNodeId)) if (string.IsNullOrWhiteSpace(peerNodeId))
throw new ArgumentException("Peer node id is required.", nameof(peerNodeId)); throw new ArgumentException("Peer node id is required.", nameof(peerNodeId));
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all return all
.Where(c => string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal) && .Where(c => string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal) &&
!string.Equals(c.SourceNodeId, RegistrationSourceNodeId, StringComparison.Ordinal)) !string.Equals(c.SourceNodeId, RegistrationSourceNodeId, StringComparison.Ordinal))
@@ -156,10 +217,18 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task RemovePeerTrackingAsync(string peerNodeId, CancellationToken cancellationToken = default) public override async Task RemovePeerTrackingAsync(string peerNodeId, CancellationToken cancellationToken = default)
{ {
await RemovePeerTrackingAsync(peerNodeId, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task RemovePeerTrackingAsync(string peerNodeId, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
if (string.IsNullOrWhiteSpace(peerNodeId)) if (string.IsNullOrWhiteSpace(peerNodeId))
throw new ArgumentException("Peer node id is required.", nameof(peerNodeId)); throw new ArgumentException("Peer node id is required.", nameof(peerNodeId));
var matches = (await SelectAllAsync(cancellationToken)) var matches = (await SelectAllAsync(normalizedDatasetId, cancellationToken))
.Where(c => string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal)) .Where(c => string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal))
.ToList(); .ToList();
@@ -173,7 +242,11 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
match.IsActive = false; match.IsActive = false;
match.LastConfirmedUtcMs = nowMs; match.LastConfirmedUtcMs = nowMs;
RecordId recordId = match.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(match.PeerNodeId, match.SourceNodeId); RecordId recordId = match.Id ??
SurrealStoreRecordIds.PeerOplogConfirmation(
match.PeerNodeId,
match.SourceNodeId,
normalizedDatasetId);
await UpsertAsync(match, recordId, cancellationToken); await UpsertAsync(match, recordId, cancellationToken);
} }
} }
@@ -182,7 +255,15 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
public override async Task<IEnumerable<string>> GetActiveTrackedPeersAsync( public override async Task<IEnumerable<string>> GetActiveTrackedPeersAsync(
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); return await GetActiveTrackedPeersAsync(PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<string>> GetActiveTrackedPeersAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all return all
.Where(c => c.IsActive) .Where(c => c.IsActive)
.Select(c => c.PeerNodeId) .Select(c => c.PeerNodeId)
@@ -193,14 +274,45 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
/// <inheritdoc /> /// <inheritdoc />
public override async Task DropAsync(CancellationToken cancellationToken = default) public override async Task DropAsync(CancellationToken cancellationToken = default)
{ {
await EnsureReadyAsync(cancellationToken); await DropAsync(PrimaryDatasetId, cancellationToken);
await _surrealClient.Delete(CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable, cancellationToken); }
/// <summary>
/// Drops all peer confirmation rows for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task DropAsync(string datasetId, CancellationToken cancellationToken = default)
{
var rows = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
foreach (var row in rows)
{
RecordId recordId = row.Id ??
SurrealStoreRecordIds.PeerOplogConfirmation(
row.PeerNodeId,
row.SourceNodeId,
NormalizeDatasetId(datasetId));
await _surrealClient.Delete(recordId, cancellationToken);
}
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<PeerOplogConfirmation>> ExportAsync(CancellationToken cancellationToken = default) public override async Task<IEnumerable<PeerOplogConfirmation>> ExportAsync(CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
return all.Select(c => c.ToDomain()).ToList();
}
/// <summary>
/// Exports peer confirmations for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>The exported confirmations.</returns>
public async Task<IEnumerable<PeerOplogConfirmation>> ExportAsync(string datasetId,
CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all.Select(c => c.ToDomain()).ToList(); return all.Select(c => c.ToDomain()).ToList();
} }
@@ -208,11 +320,25 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
public override async Task ImportAsync(IEnumerable<PeerOplogConfirmation> items, public override async Task ImportAsync(IEnumerable<PeerOplogConfirmation> items,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await ImportAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Imports peer confirmation rows into a dataset.
/// </summary>
/// <param name="items">The confirmation items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task ImportAsync(IEnumerable<PeerOplogConfirmation> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByPairAsync(item.PeerNodeId, item.SourceNodeId, cancellationToken); item.DatasetId = normalizedDatasetId;
var existing = await FindByPairAsync(item.PeerNodeId, item.SourceNodeId, normalizedDatasetId, cancellationToken);
RecordId recordId = RecordId recordId =
existing?.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(item.PeerNodeId, item.SourceNodeId); existing?.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(item.PeerNodeId, item.SourceNodeId, normalizedDatasetId);
await UpsertAsync(item, recordId, cancellationToken); await UpsertAsync(item, recordId, cancellationToken);
} }
} }
@@ -221,12 +347,26 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
public override async Task MergeAsync(IEnumerable<PeerOplogConfirmation> items, public override async Task MergeAsync(IEnumerable<PeerOplogConfirmation> items,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await MergeAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Merges peer confirmations into a dataset.
/// </summary>
/// <param name="items">The confirmation items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task MergeAsync(IEnumerable<PeerOplogConfirmation> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByPairAsync(item.PeerNodeId, item.SourceNodeId, cancellationToken); item.DatasetId = normalizedDatasetId;
var existing = await FindByPairAsync(item.PeerNodeId, item.SourceNodeId, normalizedDatasetId, cancellationToken);
if (existing == null) if (existing == null)
{ {
await UpsertAsync(item, SurrealStoreRecordIds.PeerOplogConfirmation(item.PeerNodeId, item.SourceNodeId), await UpsertAsync(item, SurrealStoreRecordIds.PeerOplogConfirmation(item.PeerNodeId, item.SourceNodeId, normalizedDatasetId),
cancellationToken); cancellationToken);
continue; continue;
} }
@@ -259,7 +399,11 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
if (!changed) continue; if (!changed) continue;
RecordId recordId = RecordId recordId =
existing.Id ?? SurrealStoreRecordIds.PeerOplogConfirmation(existing.PeerNodeId, existing.SourceNodeId); existing.Id ??
SurrealStoreRecordIds.PeerOplogConfirmation(
existing.PeerNodeId,
existing.SourceNodeId,
normalizedDatasetId);
await UpsertAsync(existing, recordId, cancellationToken); await UpsertAsync(existing, recordId, cancellationToken);
} }
} }
@@ -288,27 +432,37 @@ public class SurrealPeerOplogConfirmationStore : PeerOplogConfirmationStore
await _schemaInitializer.EnsureInitializedAsync(cancellationToken); await _schemaInitializer.EnsureInitializedAsync(cancellationToken);
} }
private async Task<List<SurrealPeerOplogConfirmationRecord>> SelectAllAsync(CancellationToken cancellationToken) private async Task<List<SurrealPeerOplogConfirmationRecord>> SelectAllAsync(string datasetId,
CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
var rows = await _surrealClient.Select<SurrealPeerOplogConfirmationRecord>( var rows = await _surrealClient.Select<SurrealPeerOplogConfirmationRecord>(
CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable, CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable,
cancellationToken); cancellationToken);
return rows?.ToList() ?? []; return rows?
.Where(row => MatchesDataset(row.DatasetId, normalizedDatasetId))
.ToList()
?? [];
} }
private async Task<SurrealPeerOplogConfirmationRecord?> FindByPairAsync(string peerNodeId, string sourceNodeId, private async Task<SurrealPeerOplogConfirmationRecord?> FindByPairAsync(
string peerNodeId,
string sourceNodeId,
string datasetId,
CancellationToken cancellationToken) CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
RecordId deterministicId = SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId); RecordId deterministicId = SurrealStoreRecordIds.PeerOplogConfirmation(peerNodeId, sourceNodeId, normalizedDatasetId);
var deterministic = await _surrealClient.Select<SurrealPeerOplogConfirmationRecord>(deterministicId, cancellationToken); var deterministic = await _surrealClient.Select<SurrealPeerOplogConfirmationRecord>(deterministicId, cancellationToken);
if (deterministic != null && if (deterministic != null &&
MatchesDataset(deterministic.DatasetId, normalizedDatasetId) &&
string.Equals(deterministic.PeerNodeId, peerNodeId, StringComparison.Ordinal) && string.Equals(deterministic.PeerNodeId, peerNodeId, StringComparison.Ordinal) &&
string.Equals(deterministic.SourceNodeId, sourceNodeId, StringComparison.Ordinal)) string.Equals(deterministic.SourceNodeId, sourceNodeId, StringComparison.Ordinal))
return deterministic; return deterministic;
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
return all.FirstOrDefault(c => return all.FirstOrDefault(c =>
string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal) && string.Equals(c.PeerNodeId, peerNodeId, StringComparison.Ordinal) &&
string.Equals(c.SourceNodeId, sourceNodeId, StringComparison.Ordinal)); string.Equals(c.SourceNodeId, sourceNodeId, StringComparison.Ordinal));

View File

@@ -8,6 +8,7 @@ namespace ZB.MOM.WW.CBDDC.Persistence.Surreal;
public class SurrealSnapshotMetadataStore : SnapshotMetadataStore public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
{ {
private const string PrimaryDatasetId = DatasetId.Primary;
private readonly ILogger<SurrealSnapshotMetadataStore> _logger; private readonly ILogger<SurrealSnapshotMetadataStore> _logger;
private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer; private readonly ICBDDCSurrealSchemaInitializer _schemaInitializer;
private readonly ISurrealDbClient _surrealClient; private readonly ISurrealDbClient _surrealClient;
@@ -29,17 +30,56 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
_logger = logger ?? NullLogger<SurrealSnapshotMetadataStore>.Instance; _logger = logger ?? NullLogger<SurrealSnapshotMetadataStore>.Instance;
} }
private static string NormalizeDatasetId(string? datasetId)
{
return DatasetId.Normalize(datasetId);
}
private static bool MatchesDataset(string? recordDatasetId, string datasetId)
{
if (string.IsNullOrWhiteSpace(recordDatasetId))
return string.Equals(datasetId, PrimaryDatasetId, StringComparison.Ordinal);
return string.Equals(NormalizeDatasetId(recordDatasetId), datasetId, StringComparison.Ordinal);
}
/// <inheritdoc /> /// <inheritdoc />
public override async Task DropAsync(CancellationToken cancellationToken = default) public override async Task DropAsync(CancellationToken cancellationToken = default)
{ {
await EnsureReadyAsync(cancellationToken); await DropAsync(PrimaryDatasetId, cancellationToken);
await _surrealClient.Delete(CBDDCSurrealSchemaNames.SnapshotMetadataTable, cancellationToken); }
/// <summary>
/// Drops snapshot metadata rows for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task DropAsync(string datasetId, CancellationToken cancellationToken = default)
{
var rows = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
foreach (var row in rows)
{
RecordId recordId = row.Id ?? SurrealStoreRecordIds.SnapshotMetadata(row.NodeId, NormalizeDatasetId(datasetId));
await _surrealClient.Delete(recordId, cancellationToken);
}
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task<IEnumerable<SnapshotMetadata>> ExportAsync(CancellationToken cancellationToken = default) public override async Task<IEnumerable<SnapshotMetadata>> ExportAsync(CancellationToken cancellationToken = default)
{ {
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(PrimaryDatasetId, cancellationToken);
return all.Select(m => m.ToDomain()).ToList();
}
/// <summary>
/// Exports snapshot metadata rows for a dataset.
/// </summary>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
/// <returns>Dataset-scoped snapshot metadata rows.</returns>
public async Task<IEnumerable<SnapshotMetadata>> ExportAsync(string datasetId, CancellationToken cancellationToken = default)
{
var all = await SelectAllAsync(NormalizeDatasetId(datasetId), cancellationToken);
return all.Select(m => m.ToDomain()).ToList(); return all.Select(m => m.ToDomain()).ToList();
} }
@@ -47,14 +87,32 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
public override async Task<SnapshotMetadata?> GetSnapshotMetadataAsync(string nodeId, public override async Task<SnapshotMetadata?> GetSnapshotMetadataAsync(string nodeId,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var existing = await FindByNodeIdAsync(nodeId, cancellationToken); var existing = await FindByNodeIdAsync(nodeId, PrimaryDatasetId, cancellationToken);
return existing?.ToDomain();
}
/// <inheritdoc />
public async Task<SnapshotMetadata?> GetSnapshotMetadataAsync(
string nodeId,
string datasetId,
CancellationToken cancellationToken = default)
{
var existing = await FindByNodeIdAsync(nodeId, NormalizeDatasetId(datasetId), cancellationToken);
return existing?.ToDomain(); return existing?.ToDomain();
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task<string?> GetSnapshotHashAsync(string nodeId, CancellationToken cancellationToken = default) public override async Task<string?> GetSnapshotHashAsync(string nodeId, CancellationToken cancellationToken = default)
{ {
var existing = await FindByNodeIdAsync(nodeId, cancellationToken); var existing = await FindByNodeIdAsync(nodeId, PrimaryDatasetId, cancellationToken);
return existing?.Hash;
}
/// <inheritdoc />
public async Task<string?> GetSnapshotHashAsync(string nodeId, string datasetId,
CancellationToken cancellationToken = default)
{
var existing = await FindByNodeIdAsync(nodeId, NormalizeDatasetId(datasetId), cancellationToken);
return existing?.Hash; return existing?.Hash;
} }
@@ -62,10 +120,24 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
public override async Task ImportAsync(IEnumerable<SnapshotMetadata> items, public override async Task ImportAsync(IEnumerable<SnapshotMetadata> items,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
await ImportAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Imports snapshot metadata rows into a dataset.
/// </summary>
/// <param name="items">Snapshot metadata items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task ImportAsync(IEnumerable<SnapshotMetadata> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
foreach (var item in items) foreach (var item in items)
{ {
var existing = await FindByNodeIdAsync(item.NodeId, cancellationToken); item.DatasetId = normalizedDatasetId;
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.SnapshotMetadata(item.NodeId); var existing = await FindByNodeIdAsync(item.NodeId, normalizedDatasetId, cancellationToken);
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.SnapshotMetadata(item.NodeId, normalizedDatasetId);
await UpsertAsync(item, recordId, cancellationToken); await UpsertAsync(item, recordId, cancellationToken);
} }
} }
@@ -74,20 +146,45 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
public override async Task InsertSnapshotMetadataAsync(SnapshotMetadata metadata, public override async Task InsertSnapshotMetadataAsync(SnapshotMetadata metadata,
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
var existing = await FindByNodeIdAsync(metadata.NodeId, cancellationToken); await InsertSnapshotMetadataAsync(metadata, PrimaryDatasetId, cancellationToken);
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId); }
/// <inheritdoc />
public async Task InsertSnapshotMetadataAsync(
SnapshotMetadata metadata,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
metadata.DatasetId = normalizedDatasetId;
var existing = await FindByNodeIdAsync(metadata.NodeId, normalizedDatasetId, cancellationToken);
RecordId recordId = existing?.Id ?? SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId, normalizedDatasetId);
await UpsertAsync(metadata, recordId, cancellationToken); await UpsertAsync(metadata, recordId, cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
public override async Task MergeAsync(IEnumerable<SnapshotMetadata> items, CancellationToken cancellationToken = default) public override async Task MergeAsync(IEnumerable<SnapshotMetadata> items, CancellationToken cancellationToken = default)
{ {
await MergeAsync(items, PrimaryDatasetId, cancellationToken);
}
/// <summary>
/// Merges snapshot metadata rows into a dataset.
/// </summary>
/// <param name="items">Snapshot metadata items.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public async Task MergeAsync(IEnumerable<SnapshotMetadata> items, string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
foreach (var metadata in items) foreach (var metadata in items)
{ {
var existing = await FindByNodeIdAsync(metadata.NodeId, cancellationToken); metadata.DatasetId = normalizedDatasetId;
var existing = await FindByNodeIdAsync(metadata.NodeId, normalizedDatasetId, cancellationToken);
if (existing == null) if (existing == null)
{ {
await UpsertAsync(metadata, SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId), cancellationToken); await UpsertAsync(metadata, SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId, normalizedDatasetId), cancellationToken);
continue; continue;
} }
@@ -96,7 +193,7 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
metadata.TimestampLogicalCounter <= existing.TimestampLogicalCounter)) metadata.TimestampLogicalCounter <= existing.TimestampLogicalCounter))
continue; continue;
RecordId recordId = existing.Id ?? SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId); RecordId recordId = existing.Id ?? SurrealStoreRecordIds.SnapshotMetadata(metadata.NodeId, normalizedDatasetId);
await UpsertAsync(metadata, recordId, cancellationToken); await UpsertAsync(metadata, recordId, cancellationToken);
} }
} }
@@ -105,10 +202,21 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
public override async Task UpdateSnapshotMetadataAsync(SnapshotMetadata existingMeta, public override async Task UpdateSnapshotMetadataAsync(SnapshotMetadata existingMeta,
CancellationToken cancellationToken) CancellationToken cancellationToken)
{ {
var existing = await FindByNodeIdAsync(existingMeta.NodeId, cancellationToken); await UpdateSnapshotMetadataAsync(existingMeta, PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task UpdateSnapshotMetadataAsync(
SnapshotMetadata existingMeta,
string datasetId,
CancellationToken cancellationToken = default)
{
string normalizedDatasetId = NormalizeDatasetId(datasetId);
existingMeta.DatasetId = normalizedDatasetId;
var existing = await FindByNodeIdAsync(existingMeta.NodeId, normalizedDatasetId, cancellationToken);
if (existing == null) return; if (existing == null) return;
RecordId recordId = existing.Id ?? SurrealStoreRecordIds.SnapshotMetadata(existingMeta.NodeId); RecordId recordId = existing.Id ?? SurrealStoreRecordIds.SnapshotMetadata(existingMeta.NodeId, normalizedDatasetId);
await UpsertAsync(existingMeta, recordId, cancellationToken); await UpsertAsync(existingMeta, recordId, cancellationToken);
} }
@@ -116,7 +224,15 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
public override async Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync( public override async Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync(
CancellationToken cancellationToken = default) CancellationToken cancellationToken = default)
{ {
return await ExportAsync(cancellationToken); return await ExportAsync(PrimaryDatasetId, cancellationToken);
}
/// <inheritdoc />
public async Task<IEnumerable<SnapshotMetadata>> GetAllSnapshotMetadataAsync(
string datasetId,
CancellationToken cancellationToken = default)
{
return await ExportAsync(datasetId, cancellationToken);
} }
private async Task UpsertAsync(SnapshotMetadata metadata, RecordId recordId, CancellationToken cancellationToken) private async Task UpsertAsync(SnapshotMetadata metadata, RecordId recordId, CancellationToken cancellationToken)
@@ -133,25 +249,33 @@ public class SurrealSnapshotMetadataStore : SnapshotMetadataStore
await _schemaInitializer.EnsureInitializedAsync(cancellationToken); await _schemaInitializer.EnsureInitializedAsync(cancellationToken);
} }
private async Task<List<SurrealSnapshotMetadataRecord>> SelectAllAsync(CancellationToken cancellationToken) private async Task<List<SurrealSnapshotMetadataRecord>> SelectAllAsync(string datasetId,
CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
var rows = await _surrealClient.Select<SurrealSnapshotMetadataRecord>( var rows = await _surrealClient.Select<SurrealSnapshotMetadataRecord>(
CBDDCSurrealSchemaNames.SnapshotMetadataTable, CBDDCSurrealSchemaNames.SnapshotMetadataTable,
cancellationToken); cancellationToken);
return rows?.ToList() ?? []; return rows?
.Where(row => MatchesDataset(row.DatasetId, normalizedDatasetId))
.ToList()
?? [];
} }
private async Task<SurrealSnapshotMetadataRecord?> FindByNodeIdAsync(string nodeId, CancellationToken cancellationToken) private async Task<SurrealSnapshotMetadataRecord?> FindByNodeIdAsync(string nodeId, string datasetId,
CancellationToken cancellationToken)
{ {
string normalizedDatasetId = NormalizeDatasetId(datasetId);
await EnsureReadyAsync(cancellationToken); await EnsureReadyAsync(cancellationToken);
RecordId deterministicId = SurrealStoreRecordIds.SnapshotMetadata(nodeId); RecordId deterministicId = SurrealStoreRecordIds.SnapshotMetadata(nodeId, normalizedDatasetId);
var deterministic = await _surrealClient.Select<SurrealSnapshotMetadataRecord>(deterministicId, cancellationToken); var deterministic = await _surrealClient.Select<SurrealSnapshotMetadataRecord>(deterministicId, cancellationToken);
if (deterministic != null && if (deterministic != null &&
MatchesDataset(deterministic.DatasetId, normalizedDatasetId) &&
string.Equals(deterministic.NodeId, nodeId, StringComparison.Ordinal)) string.Equals(deterministic.NodeId, nodeId, StringComparison.Ordinal))
return deterministic; return deterministic;
var all = await SelectAllAsync(cancellationToken); var all = await SelectAllAsync(normalizedDatasetId, cancellationToken);
return all.FirstOrDefault(m => string.Equals(m.NodeId, nodeId, StringComparison.Ordinal)); return all.FirstOrDefault(m => string.Equals(m.NodeId, nodeId, StringComparison.Ordinal));
} }
} }

View File

@@ -26,22 +26,28 @@ internal static class SurrealStoreRecordIds
/// </summary> /// </summary>
/// <param name="collection">The document collection name.</param> /// <param name="collection">The document collection name.</param>
/// <param name="key">The document key.</param> /// <param name="key">The document key.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <returns>The SurrealDB record identifier.</returns> /// <returns>The SurrealDB record identifier.</returns>
public static RecordId DocumentMetadata(string collection, string key) public static RecordId DocumentMetadata(string collection, string key, string datasetId)
{ {
string normalizedDatasetId = DatasetId.Normalize(datasetId);
return RecordId.From( return RecordId.From(
CBDDCSurrealSchemaNames.DocumentMetadataTable, CBDDCSurrealSchemaNames.DocumentMetadataTable,
CompositeKey("docmeta", collection, key)); CompositeKey("docmeta", normalizedDatasetId, collection, key));
} }
/// <summary> /// <summary>
/// Creates the record identifier for snapshot metadata. /// Creates the record identifier for snapshot metadata.
/// </summary> /// </summary>
/// <param name="nodeId">The node identifier.</param> /// <param name="nodeId">The node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <returns>The SurrealDB record identifier.</returns> /// <returns>The SurrealDB record identifier.</returns>
public static RecordId SnapshotMetadata(string nodeId) public static RecordId SnapshotMetadata(string nodeId, string datasetId)
{ {
return RecordId.From(CBDDCSurrealSchemaNames.SnapshotMetadataTable, nodeId); string normalizedDatasetId = DatasetId.Normalize(datasetId);
return RecordId.From(
CBDDCSurrealSchemaNames.SnapshotMetadataTable,
CompositeKey("snapshotmeta", normalizedDatasetId, nodeId));
} }
/// <summary> /// <summary>
@@ -59,12 +65,14 @@ internal static class SurrealStoreRecordIds
/// </summary> /// </summary>
/// <param name="peerNodeId">The peer node identifier.</param> /// <param name="peerNodeId">The peer node identifier.</param>
/// <param name="sourceNodeId">The source node identifier.</param> /// <param name="sourceNodeId">The source node identifier.</param>
/// <param name="datasetId">The dataset identifier.</param>
/// <returns>The SurrealDB record identifier.</returns> /// <returns>The SurrealDB record identifier.</returns>
public static RecordId PeerOplogConfirmation(string peerNodeId, string sourceNodeId) public static RecordId PeerOplogConfirmation(string peerNodeId, string sourceNodeId, string datasetId)
{ {
string normalizedDatasetId = DatasetId.Normalize(datasetId);
return RecordId.From( return RecordId.From(
CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable, CBDDCSurrealSchemaNames.PeerOplogConfirmationsTable,
CompositeKey("peerconfirm", peerNodeId, sourceNodeId)); CompositeKey("peerconfirm", normalizedDatasetId, peerNodeId, sourceNodeId));
} }
private static string CompositeKey(string prefix, string first, string second) private static string CompositeKey(string prefix, string first, string second)
@@ -72,10 +80,22 @@ internal static class SurrealStoreRecordIds
byte[] bytes = Encoding.UTF8.GetBytes($"{prefix}\n{first}\n{second}"); byte[] bytes = Encoding.UTF8.GetBytes($"{prefix}\n{first}\n{second}");
return Convert.ToHexString(SHA256.HashData(bytes)).ToLowerInvariant(); return Convert.ToHexString(SHA256.HashData(bytes)).ToLowerInvariant();
} }
private static string CompositeKey(string prefix, string first, string second, string third)
{
byte[] bytes = Encoding.UTF8.GetBytes($"{prefix}\n{first}\n{second}\n{third}");
return Convert.ToHexString(SHA256.HashData(bytes)).ToLowerInvariant();
}
} }
internal sealed class SurrealOplogRecord : Record internal sealed class SurrealOplogRecord : Record
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
[JsonPropertyName("datasetId")]
public string DatasetId { get; set; } = "";
/// <summary> /// <summary>
/// Gets or sets the collection name. /// Gets or sets the collection name.
/// </summary> /// </summary>
@@ -133,6 +153,12 @@ internal sealed class SurrealOplogRecord : Record
internal sealed class SurrealDocumentMetadataRecord : Record internal sealed class SurrealDocumentMetadataRecord : Record
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
[JsonPropertyName("datasetId")]
public string DatasetId { get; set; } = "";
/// <summary> /// <summary>
/// Gets or sets the collection name. /// Gets or sets the collection name.
/// </summary> /// </summary>
@@ -205,6 +231,12 @@ internal sealed class SurrealRemotePeerRecord : Record
internal sealed class SurrealPeerOplogConfirmationRecord : Record internal sealed class SurrealPeerOplogConfirmationRecord : Record
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
[JsonPropertyName("datasetId")]
public string DatasetId { get; set; } = "";
/// <summary> /// <summary>
/// Gets or sets the peer node identifier. /// Gets or sets the peer node identifier.
/// </summary> /// </summary>
@@ -250,6 +282,12 @@ internal sealed class SurrealPeerOplogConfirmationRecord : Record
internal sealed class SurrealSnapshotMetadataRecord : Record internal sealed class SurrealSnapshotMetadataRecord : Record
{ {
/// <summary>
/// Gets or sets the dataset identifier.
/// </summary>
[JsonPropertyName("datasetId")]
public string DatasetId { get; set; } = "";
/// <summary> /// <summary>
/// Gets or sets the node identifier. /// Gets or sets the node identifier.
/// </summary> /// </summary>
@@ -286,6 +324,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new SurrealOplogRecord return new SurrealOplogRecord
{ {
DatasetId = DatasetId.Normalize(entry.DatasetId),
Collection = entry.Collection, Collection = entry.Collection,
Key = entry.Key, Key = entry.Key,
Operation = (int)entry.Operation, Operation = (int)entry.Operation,
@@ -316,7 +355,8 @@ internal static class SurrealStoreRecordMappers
payload, payload,
new HlcTimestamp(record.TimestampPhysicalTime, record.TimestampLogicalCounter, record.TimestampNodeId), new HlcTimestamp(record.TimestampPhysicalTime, record.TimestampLogicalCounter, record.TimestampNodeId),
record.PreviousHash, record.PreviousHash,
record.Hash); record.Hash,
record.DatasetId);
} }
/// <summary> /// <summary>
@@ -328,6 +368,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new SurrealDocumentMetadataRecord return new SurrealDocumentMetadataRecord
{ {
DatasetId = DatasetId.Normalize(metadata.DatasetId),
Collection = metadata.Collection, Collection = metadata.Collection,
Key = metadata.Key, Key = metadata.Key,
HlcPhysicalTime = metadata.UpdatedAt.PhysicalTime, HlcPhysicalTime = metadata.UpdatedAt.PhysicalTime,
@@ -348,7 +389,8 @@ internal static class SurrealStoreRecordMappers
record.Collection, record.Collection,
record.Key, record.Key,
new HlcTimestamp(record.HlcPhysicalTime, record.HlcLogicalCounter, record.HlcNodeId), new HlcTimestamp(record.HlcPhysicalTime, record.HlcLogicalCounter, record.HlcNodeId),
record.IsDeleted); record.IsDeleted,
record.DatasetId);
} }
/// <summary> /// <summary>
@@ -401,6 +443,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new SurrealPeerOplogConfirmationRecord return new SurrealPeerOplogConfirmationRecord
{ {
DatasetId = DatasetId.Normalize(confirmation.DatasetId),
PeerNodeId = confirmation.PeerNodeId, PeerNodeId = confirmation.PeerNodeId,
SourceNodeId = confirmation.SourceNodeId, SourceNodeId = confirmation.SourceNodeId,
ConfirmedWall = confirmation.ConfirmedWall, ConfirmedWall = confirmation.ConfirmedWall,
@@ -420,6 +463,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new PeerOplogConfirmation return new PeerOplogConfirmation
{ {
DatasetId = DatasetId.Normalize(record.DatasetId),
PeerNodeId = record.PeerNodeId, PeerNodeId = record.PeerNodeId,
SourceNodeId = record.SourceNodeId, SourceNodeId = record.SourceNodeId,
ConfirmedWall = record.ConfirmedWall, ConfirmedWall = record.ConfirmedWall,
@@ -439,6 +483,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new SurrealSnapshotMetadataRecord return new SurrealSnapshotMetadataRecord
{ {
DatasetId = DatasetId.Normalize(metadata.DatasetId),
NodeId = metadata.NodeId, NodeId = metadata.NodeId,
TimestampPhysicalTime = metadata.TimestampPhysicalTime, TimestampPhysicalTime = metadata.TimestampPhysicalTime,
TimestampLogicalCounter = metadata.TimestampLogicalCounter, TimestampLogicalCounter = metadata.TimestampLogicalCounter,
@@ -455,6 +500,7 @@ internal static class SurrealStoreRecordMappers
{ {
return new SnapshotMetadata return new SnapshotMetadata
{ {
DatasetId = DatasetId.Normalize(record.DatasetId),
NodeId = record.NodeId, NodeId = record.NodeId,
TimestampPhysicalTime = record.TimestampPhysicalTime, TimestampPhysicalTime = record.TimestampPhysicalTime,
TimestampLogicalCounter = record.TimestampLogicalCounter, TimestampLogicalCounter = record.TimestampLogicalCounter,

View File

@@ -0,0 +1,43 @@
using System.Text.Json;
using ZB.MOM.WW.CBDDC.Core.Storage;
namespace ZB.MOM.WW.CBDDC.Core.Tests;
public class DatasetAwareModelTests
{
[Fact]
public void DocumentMetadata_ShouldDefaultDatasetId_ToPrimary()
{
var metadata = new DocumentMetadata("Users", "42", new HlcTimestamp(100, 0, "node"));
metadata.DatasetId.ShouldBe(DatasetId.Primary);
}
[Fact]
public void DocumentMetadata_SerializationRoundTrip_ShouldPreserveDatasetId()
{
var original = new DocumentMetadata("Users", "42", new HlcTimestamp(100, 0, "node"), false, "logs");
string json = JsonSerializer.Serialize(original);
var restored = JsonSerializer.Deserialize<DocumentMetadata>(json);
restored.ShouldNotBeNull();
restored.DatasetId.ShouldBe("logs");
}
[Fact]
public void SnapshotMetadata_ShouldDefaultDatasetId_ToPrimary()
{
var metadata = new SnapshotMetadata();
metadata.DatasetId.ShouldBe(DatasetId.Primary);
}
[Fact]
public void PeerOplogConfirmation_ShouldDefaultDatasetId_ToPrimary()
{
var confirmation = new PeerOplogConfirmation();
confirmation.DatasetId.ShouldBe(DatasetId.Primary);
}
}

View File

@@ -70,4 +70,28 @@ public class OplogEntryTests
entry.IsValid().ShouldBeTrue(); entry.IsValid().ShouldBeTrue();
} }
/// <summary>
/// Verifies that entries default to the primary dataset when dataset is omitted.
/// </summary>
[Fact]
public void Constructor_ShouldDefaultDatasetId_ToPrimary()
{
var entry = new OplogEntry("col", "key", OperationType.Put, null, new HlcTimestamp(1, 0, "node"), "prev");
entry.DatasetId.ShouldBe(DatasetId.Primary);
}
/// <summary>
/// Verifies that hash computation includes dataset identity to prevent cross-dataset collisions.
/// </summary>
[Fact]
public void ComputeHash_ShouldDiffer_WhenDatasetDiffers()
{
var timestamp = new HlcTimestamp(100, 0, "node-1");
var primary = new OplogEntry("col", "key", OperationType.Put, null, timestamp, "prev", datasetId: "primary");
var logs = new OplogEntry("col", "key", OperationType.Put, null, timestamp, "prev", datasetId: "logs");
logs.Hash.ShouldNotBe(primary.Hash);
}
} }

View File

@@ -0,0 +1,47 @@
using Microsoft.Extensions.DependencyInjection;
using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Core.Storage;
namespace ZB.MOM.WW.CBDDC.Network.Tests;
public class MultiDatasetRegistrationTests
{
[Fact]
public void AddCBDDCMultiDataset_ShouldRegisterCoordinatorAndReplaceSyncOrchestrator()
{
var services = new ServiceCollection();
services.AddCBDDCNetwork<TestPeerNodeConfigurationProvider>(useHostedService: false);
services.AddCBDDCMultiDataset(options =>
{
options.EnableMultiDatasetSync = true;
options.EnableDatasetPrimary = true;
options.EnableDatasetLogs = true;
options.EnableDatasetTimeseries = true;
});
services.Any(descriptor => descriptor.ServiceType == typeof(IMultiDatasetSyncOrchestrator)).ShouldBeTrue();
var syncDescriptor = services.Last(descriptor => descriptor.ServiceType == typeof(ISyncOrchestrator));
syncDescriptor.ImplementationFactory.ShouldNotBeNull();
}
private sealed class TestPeerNodeConfigurationProvider : IPeerNodeConfigurationProvider
{
public event PeerNodeConfigurationChangedEventHandler? ConfigurationChanged
{
add { }
remove { }
}
public Task<PeerNodeConfiguration> GetConfiguration()
{
return Task.FromResult(new PeerNodeConfiguration
{
NodeId = "node-test",
TcpPort = 9000,
AuthToken = "auth"
});
}
}
}

View File

@@ -0,0 +1,102 @@
using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Core.Storage;
namespace ZB.MOM.WW.CBDDC.Network.Tests;
public class MultiDatasetSyncOrchestratorTests
{
[Fact]
public void Constructor_WhenMultiDatasetDisabled_ShouldOnlyCreatePrimaryContext()
{
var sut = CreateSut(
[
new DatasetSyncOptions { DatasetId = DatasetId.Primary, Enabled = true },
new DatasetSyncOptions { DatasetId = DatasetId.Logs, Enabled = true }
],
new MultiDatasetRuntimeOptions
{
EnableMultiDatasetSync = false,
EnableDatasetPrimary = true,
EnableDatasetLogs = true
});
var datasetIds = sut.Contexts.Select(c => c.DatasetId).ToList();
datasetIds.Count.ShouldBe(1);
datasetIds[0].ShouldBe(DatasetId.Primary);
}
[Fact]
public async Task StartStop_WhenOneDatasetThrows_ShouldContinueOtherDatasets()
{
var orchestrators = new Dictionary<string, TrackingSyncOrchestrator>(StringComparer.Ordinal)
{
[DatasetId.Primary] = new TrackingSyncOrchestrator(),
[DatasetId.Logs] = new TrackingSyncOrchestrator(startException: new InvalidOperationException("boom")),
[DatasetId.Timeseries] = new TrackingSyncOrchestrator()
};
var sut = CreateSut(
[],
new MultiDatasetRuntimeOptions
{
EnableMultiDatasetSync = true,
EnableDatasetPrimary = true,
EnableDatasetLogs = true,
EnableDatasetTimeseries = true
},
options => orchestrators[DatasetId.Normalize(options.DatasetId)]);
await sut.Start();
await sut.Stop();
orchestrators[DatasetId.Primary].StartCalls.ShouldBe(1);
orchestrators[DatasetId.Primary].StopCalls.ShouldBe(1);
orchestrators[DatasetId.Logs].StartCalls.ShouldBe(1);
orchestrators[DatasetId.Logs].StopCalls.ShouldBe(1);
orchestrators[DatasetId.Timeseries].StartCalls.ShouldBe(1);
orchestrators[DatasetId.Timeseries].StopCalls.ShouldBe(1);
}
private static MultiDatasetSyncOrchestrator CreateSut(
IEnumerable<DatasetSyncOptions> datasetOptions,
MultiDatasetRuntimeOptions runtimeOptions,
Func<DatasetSyncOptions, ISyncOrchestrator>? orchestratorFactory = null)
{
return new MultiDatasetSyncOrchestrator(
Substitute.For<IDiscoveryService>(),
Substitute.For<IOplogStore>(),
Substitute.For<IDocumentStore>(),
Substitute.For<ISnapshotMetadataStore>(),
Substitute.For<ISnapshotService>(),
Substitute.For<IPeerNodeConfigurationProvider>(),
NullLoggerFactory.Instance,
datasetOptions,
runtimeOptions,
orchestratorFactory: orchestratorFactory);
}
private sealed class TrackingSyncOrchestrator(Exception? startException = null, Exception? stopException = null)
: ISyncOrchestrator
{
public int StartCalls { get; private set; }
public int StopCalls { get; private set; }
public Task Start()
{
StartCalls++;
if (startException != null) throw startException;
return Task.CompletedTask;
}
public Task Stop()
{
StopCalls++;
if (stopException != null) throw stopException;
return Task.CompletedTask;
}
}
}

View File

@@ -1,4 +1,6 @@
using Microsoft.Extensions.Logging.Abstractions; using Microsoft.Extensions.Logging.Abstractions;
using Google.Protobuf;
using ZB.MOM.WW.CBDDC.Core;
using ZB.MOM.WW.CBDDC.Network.Proto; using ZB.MOM.WW.CBDDC.Network.Proto;
using ZB.MOM.WW.CBDDC.Network.Protocol; using ZB.MOM.WW.CBDDC.Network.Protocol;
using ZB.MOM.WW.CBDDC.Network.Security; using ZB.MOM.WW.CBDDC.Network.Security;
@@ -145,6 +147,44 @@ public class ProtocolTests
decoded.NodeId.ShouldBe("fragmented"); decoded.NodeId.ShouldBe("fragmented");
} }
/// <summary>
/// Verifies that dataset-aware protocol fields are serialized and parsed correctly.
/// </summary>
[Fact]
public void DatasetAwareMessages_ShouldRoundTripDatasetFields()
{
var request = new PullChangesRequest
{
SinceWall = 10,
SinceLogic = 1,
SinceNode = "node-a",
DatasetId = "logs"
};
byte[] payload = request.ToByteArray();
var decoded = PullChangesRequest.Parser.ParseFrom(payload);
decoded.DatasetId.ShouldBe("logs");
}
/// <summary>
/// Verifies that legacy messages with no dataset id default to the primary dataset.
/// </summary>
[Fact]
public void DatasetAwareMessages_WhenMissingDataset_ShouldDefaultToPrimary()
{
var legacy = new HandshakeRequest
{
NodeId = "node-legacy",
AuthToken = "token"
};
byte[] payload = legacy.ToByteArray();
var decoded = HandshakeRequest.Parser.ParseFrom(payload);
DatasetId.Normalize(decoded.DatasetId).ShouldBe(DatasetId.Primary);
}
// Helper Stream for fragmentation test // Helper Stream for fragmentation test
private class FragmentedMemoryStream : MemoryStream private class FragmentedMemoryStream : MemoryStream
{ {

View File

@@ -16,6 +16,8 @@ public class SnapshotReconnectRegressionTests
.Returns((SnapshotMetadata?)null); .Returns((SnapshotMetadata?)null);
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()) snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns((string?)null); .Returns((string?)null);
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns((string?)null);
snapshotMetadataStore.GetAllSnapshotMetadataAsync(Arg.Any<CancellationToken>()) snapshotMetadataStore.GetAllSnapshotMetadataAsync(Arg.Any<CancellationToken>())
.Returns(Array.Empty<SnapshotMetadata>()); .Returns(Array.Empty<SnapshotMetadata>());
return snapshotMetadataStore; return snapshotMetadataStore;
@@ -30,6 +32,10 @@ public class SnapshotReconnectRegressionTests
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
snapshotService.MergeSnapshotAsync(Arg.Any<Stream>(), Arg.Any<CancellationToken>()) snapshotService.MergeSnapshotAsync(Arg.Any<Stream>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
snapshotService.ReplaceDatabaseAsync(Arg.Any<Stream>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
snapshotService.MergeSnapshotAsync(Arg.Any<Stream>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
return snapshotService; return snapshotService;
} }
@@ -69,8 +75,12 @@ public class SnapshotReconnectRegressionTests
var oplogStore = Substitute.For<IOplogStore>(); var oplogStore = Substitute.For<IOplogStore>();
oplogStore.GetLastEntryHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()) oplogStore.GetLastEntryHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(localHeadHash); .Returns(localHeadHash);
oplogStore.GetLastEntryHashAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(localHeadHash);
oplogStore.ApplyBatchAsync(Arg.Any<IEnumerable<OplogEntry>>(), Arg.Any<CancellationToken>()) oplogStore.ApplyBatchAsync(Arg.Any<IEnumerable<OplogEntry>>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
oplogStore.ApplyBatchAsync(Arg.Any<IEnumerable<OplogEntry>>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
return oplogStore; return oplogStore;
} }
@@ -84,6 +94,8 @@ public class SnapshotReconnectRegressionTests
null); null);
client.GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>()) client.GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(_ => Task.FromException<List<OplogEntry>>(new SnapshotRequiredException())); .Returns(_ => Task.FromException<List<OplogEntry>>(new SnapshotRequiredException()));
client.GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(_ => Task.FromException<List<OplogEntry>>(new SnapshotRequiredException()));
return client; return client;
} }
@@ -109,19 +121,38 @@ public class SnapshotReconnectRegressionTests
store.EnsurePeerRegisteredAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<PeerType>(), store.EnsurePeerRegisteredAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<PeerType>(),
Arg.Any<CancellationToken>()) Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
store.EnsurePeerRegisteredAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<PeerType>(), Arg.Any<string>(),
Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
store.UpdateConfirmationAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<HlcTimestamp>(), Arg.Any<string>(), store.UpdateConfirmationAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<HlcTimestamp>(), Arg.Any<string>(),
Arg.Any<CancellationToken>()) Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
store.UpdateConfirmationAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<HlcTimestamp>(), Arg.Any<string>(),
Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
store.GetConfirmationsAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<PeerOplogConfirmation>()); store.GetConfirmationsAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<PeerOplogConfirmation>());
store.GetConfirmationsAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Array.Empty<PeerOplogConfirmation>());
store.GetConfirmationsForPeerAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()) store.GetConfirmationsForPeerAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Array.Empty<PeerOplogConfirmation>()); .Returns(Array.Empty<PeerOplogConfirmation>());
store.GetConfirmationsForPeerAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Array.Empty<PeerOplogConfirmation>());
store.RemovePeerTrackingAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(Task.CompletedTask); store.RemovePeerTrackingAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(Task.CompletedTask);
store.RemovePeerTrackingAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
store.GetActiveTrackedPeersAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<string>()); store.GetActiveTrackedPeersAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<string>());
store.GetActiveTrackedPeersAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(Array.Empty<string>());
store.ExportAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<PeerOplogConfirmation>()); store.ExportAsync(Arg.Any<CancellationToken>()).Returns(Array.Empty<PeerOplogConfirmation>());
store.ExportAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Array.Empty<PeerOplogConfirmation>());
store.ImportAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<CancellationToken>()) store.ImportAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
store.ImportAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
store.MergeAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<CancellationToken>()) store.MergeAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask); .Returns(Task.CompletedTask);
store.MergeAsync(Arg.Any<IEnumerable<PeerOplogConfirmation>>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Task.CompletedTask);
return store; return store;
} }
@@ -136,6 +167,8 @@ public class SnapshotReconnectRegressionTests
var snapshotMetadataStore = CreateSnapshotMetadataStore(); var snapshotMetadataStore = CreateSnapshotMetadataStore();
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()) snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns("snapshot-boundary-hash"); .Returns("snapshot-boundary-hash");
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns("snapshot-boundary-hash");
var snapshotService = CreateSnapshotService(); var snapshotService = CreateSnapshotService();
var orch = new TestableSyncOrchestrator( var orch = new TestableSyncOrchestrator(
@@ -165,7 +198,7 @@ public class SnapshotReconnectRegressionTests
// Assert // Assert
result.ShouldBe("Success"); result.ShouldBe("Success");
await client.DidNotReceive() await client.DidNotReceive()
.GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>()); .GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>());
} }
/// <summary> /// <summary>
@@ -179,6 +212,8 @@ public class SnapshotReconnectRegressionTests
var snapshotMetadataStore = CreateSnapshotMetadataStore(); var snapshotMetadataStore = CreateSnapshotMetadataStore();
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>()) snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns("snapshot-boundary-hash"); .Returns("snapshot-boundary-hash");
snapshotMetadataStore.GetSnapshotHashAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns("snapshot-boundary-hash");
var snapshotService = CreateSnapshotService(); var snapshotService = CreateSnapshotService();
var orch = new TestableSyncOrchestrator( var orch = new TestableSyncOrchestrator(
@@ -208,7 +243,11 @@ public class SnapshotReconnectRegressionTests
await Should.ThrowAsync<SnapshotRequiredException>(async () => await Should.ThrowAsync<SnapshotRequiredException>(async () =>
await orch.TestProcessInboundBatchAsync(client, "remote-node", entries, CancellationToken.None)); await orch.TestProcessInboundBatchAsync(client, "remote-node", entries, CancellationToken.None));
await client.Received(1).GetChainRangeAsync(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<CancellationToken>()); await client.Received(1).GetChainRangeAsync(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>());
} }
// Subclass to expose private method // Subclass to expose private method

View File

@@ -39,18 +39,21 @@ public class SyncOrchestratorConfirmationTests
"peer-a", "peer-a",
"10.0.0.1:9000", "10.0.0.1:9000",
PeerType.LanDiscovered, PeerType.LanDiscovered,
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.Received(1).EnsurePeerRegisteredAsync( await confirmationStore.Received(1).EnsurePeerRegisteredAsync(
"peer-b", "peer-b",
"10.0.0.2:9010", "10.0.0.2:9010",
PeerType.StaticRemote, PeerType.StaticRemote,
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.DidNotReceive().EnsurePeerRegisteredAsync( await confirmationStore.DidNotReceive().EnsurePeerRegisteredAsync(
"local", "local",
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<PeerType>(), Arg.Any<PeerType>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
@@ -89,6 +92,7 @@ public class SyncOrchestratorConfirmationTests
"peer-new", "peer-new",
"10.0.0.25:9010", "10.0.0.25:9010",
PeerType.LanDiscovered, PeerType.LanDiscovered,
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
@@ -114,9 +118,9 @@ public class SyncOrchestratorConfirmationTests
remote.SetTimestamp("node-behind", new HlcTimestamp(299, 9, "node-behind")); remote.SetTimestamp("node-behind", new HlcTimestamp(299, 9, "node-behind"));
remote.SetTimestamp("node-remote-only", new HlcTimestamp(900, 0, "node-remote-only")); remote.SetTimestamp("node-remote-only", new HlcTimestamp(900, 0, "node-remote-only"));
oplogStore.GetLastEntryHashAsync("node-equal", Arg.Any<CancellationToken>()) oplogStore.GetLastEntryHashAsync("node-equal", DatasetId.Primary, Arg.Any<CancellationToken>())
.Returns("hash-equal"); .Returns("hash-equal");
oplogStore.GetLastEntryHashAsync("node-ahead", Arg.Any<CancellationToken>()) oplogStore.GetLastEntryHashAsync("node-ahead", DatasetId.Primary, Arg.Any<CancellationToken>())
.Returns((string?)null); .Returns((string?)null);
await orchestrator.AdvanceConfirmationsFromVectorClockAsync("peer-1", local, remote, CancellationToken.None); await orchestrator.AdvanceConfirmationsFromVectorClockAsync("peer-1", local, remote, CancellationToken.None);
@@ -126,6 +130,7 @@ public class SyncOrchestratorConfirmationTests
"node-equal", "node-equal",
new HlcTimestamp(100, 1, "node-equal"), new HlcTimestamp(100, 1, "node-equal"),
"hash-equal", "hash-equal",
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.Received(1).UpdateConfirmationAsync( await confirmationStore.Received(1).UpdateConfirmationAsync(
@@ -133,6 +138,7 @@ public class SyncOrchestratorConfirmationTests
"node-ahead", "node-ahead",
new HlcTimestamp(200, 0, "node-ahead"), new HlcTimestamp(200, 0, "node-ahead"),
string.Empty, string.Empty,
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.DidNotReceive().UpdateConfirmationAsync( await confirmationStore.DidNotReceive().UpdateConfirmationAsync(
@@ -140,6 +146,7 @@ public class SyncOrchestratorConfirmationTests
"node-behind", "node-behind",
Arg.Any<HlcTimestamp>(), Arg.Any<HlcTimestamp>(),
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.DidNotReceive().UpdateConfirmationAsync( await confirmationStore.DidNotReceive().UpdateConfirmationAsync(
@@ -147,6 +154,7 @@ public class SyncOrchestratorConfirmationTests
"node-local-only", "node-local-only",
Arg.Any<HlcTimestamp>(), Arg.Any<HlcTimestamp>(),
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
await confirmationStore.DidNotReceive().UpdateConfirmationAsync( await confirmationStore.DidNotReceive().UpdateConfirmationAsync(
@@ -154,6 +162,7 @@ public class SyncOrchestratorConfirmationTests
"node-remote-only", "node-remote-only",
Arg.Any<HlcTimestamp>(), Arg.Any<HlcTimestamp>(),
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
@@ -182,6 +191,7 @@ public class SyncOrchestratorConfirmationTests
"source-1", "source-1",
new HlcTimestamp(120, 1, "source-1"), new HlcTimestamp(120, 1, "source-1"),
"hash-120", "hash-120",
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
@@ -206,6 +216,7 @@ public class SyncOrchestratorConfirmationTests
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<HlcTimestamp>(), Arg.Any<HlcTimestamp>(),
Arg.Any<string>(), Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }

View File

@@ -138,7 +138,10 @@ public class SyncOrchestratorMaintenancePruningTests
await orchestrator.RunMaintenanceIfDueAsync(config, DateTime.UtcNow, CancellationToken.None); await orchestrator.RunMaintenanceIfDueAsync(config, DateTime.UtcNow, CancellationToken.None);
await oplogStore.DidNotReceive().PruneOplogAsync(Arg.Any<HlcTimestamp>(), Arg.Any<CancellationToken>()); await oplogStore.DidNotReceive().PruneOplogAsync(
Arg.Any<HlcTimestamp>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>());
} }
/// <summary> /// <summary>
@@ -187,6 +190,7 @@ public class SyncOrchestratorMaintenancePruningTests
timestamp.PhysicalTime == 100 && timestamp.PhysicalTime == 100 &&
timestamp.LogicalCounter == 0 && timestamp.LogicalCounter == 0 &&
string.Equals(timestamp.NodeId, "node-local", StringComparison.Ordinal)), string.Equals(timestamp.NodeId, "node-local", StringComparison.Ordinal)),
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
@@ -228,7 +232,10 @@ public class SyncOrchestratorMaintenancePruningTests
var now = DateTime.UtcNow; var now = DateTime.UtcNow;
await orchestrator.RunMaintenanceIfDueAsync(config, now, CancellationToken.None); await orchestrator.RunMaintenanceIfDueAsync(config, now, CancellationToken.None);
await oplogStore.DidNotReceive().PruneOplogAsync(Arg.Any<HlcTimestamp>(), Arg.Any<CancellationToken>()); await oplogStore.DidNotReceive().PruneOplogAsync(
Arg.Any<HlcTimestamp>(),
Arg.Any<string>(),
Arg.Any<CancellationToken>());
await orchestrator.RunMaintenanceIfDueAsync(config, now.AddMinutes(2), CancellationToken.None); await orchestrator.RunMaintenanceIfDueAsync(config, now.AddMinutes(2), CancellationToken.None);
@@ -237,6 +244,7 @@ public class SyncOrchestratorMaintenancePruningTests
timestamp.PhysicalTime == 100 && timestamp.PhysicalTime == 100 &&
timestamp.LogicalCounter == 0 && timestamp.LogicalCounter == 0 &&
string.Equals(timestamp.NodeId, "node-local", StringComparison.Ordinal)), string.Equals(timestamp.NodeId, "node-local", StringComparison.Ordinal)),
DatasetId.Primary,
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }

View File

@@ -0,0 +1,38 @@
using System.Text;
using Microsoft.Extensions.Configuration;
using ZB.MOM.WW.CBDDC.Network;
namespace ZB.MOM.WW.CBDDC.Sample.Console.Tests;
public class MultiDatasetConfigParsingTests
{
[Fact]
public void MultiDatasetSection_ShouldBindRuntimeOptions()
{
const string json = """
{
"CBDDC": {
"MultiDataset": {
"EnableMultiDatasetSync": true,
"EnableDatasetPrimary": true,
"EnableDatasetLogs": true,
"EnableDatasetTimeseries": false
}
}
}
""";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(json));
var config = new ConfigurationBuilder()
.AddJsonStream(stream)
.Build();
var options = config.GetSection("CBDDC:MultiDataset").Get<MultiDatasetRuntimeOptions>();
options.ShouldNotBeNull();
options.EnableMultiDatasetSync.ShouldBeTrue();
options.EnableDatasetPrimary.ShouldBeTrue();
options.EnableDatasetLogs.ShouldBeTrue();
options.EnableDatasetTimeseries.ShouldBeFalse();
}
}

View File

@@ -54,6 +54,68 @@ public class SurrealOplogStoreContractTests
(await store.ExportAsync()).ShouldBeEmpty(); (await store.ExportAsync()).ShouldBeEmpty();
} }
/// <summary>
/// Verifies oplog reads and writes are isolated by dataset identifier.
/// </summary>
[Fact]
public async Task OplogStore_DatasetIsolation_Works()
{
await using var harness = new SurrealTestHarness();
var store = harness.CreateOplogStore();
var primaryEntry = CreateOplogEntry("Users", "p1", "node-a", 100, 0, "");
var logsEntry = CreateOplogEntry("Users", "l1", "node-a", 100, 1, "");
await store.AppendOplogEntryAsync(primaryEntry, DatasetId.Primary);
await store.AppendOplogEntryAsync(logsEntry, DatasetId.Logs);
var primary = (await store.ExportAsync(DatasetId.Primary)).ToList();
var logs = (await store.ExportAsync(DatasetId.Logs)).ToList();
primary.Count.ShouldBe(1);
primary[0].DatasetId.ShouldBe(DatasetId.Primary);
logs.Count.ShouldBe(1);
logs[0].DatasetId.ShouldBe(DatasetId.Logs);
}
/// <summary>
/// Verifies legacy oplog rows without dataset id are treated as primary dataset.
/// </summary>
[Fact]
public async Task OplogStore_LegacyRowsWithoutDatasetId_MapToPrimaryOnly()
{
await using var harness = new SurrealTestHarness();
var store = harness.CreateOplogStore();
await harness.SurrealEmbeddedClient.InitializeAsync();
await harness.SurrealEmbeddedClient.RawQueryAsync(
"""
UPSERT type::thing($table, $id) CONTENT {
collection: "Users",
key: "legacy",
operation: 0,
payloadJson: "{}",
timestampPhysicalTime: 10,
timestampLogicalCounter: 0,
timestampNodeId: "node-legacy",
hash: "legacy-hash",
previousHash: ""
};
""",
new Dictionary<string, object?>
{
["table"] = CBDDCSurrealSchemaNames.OplogEntriesTable,
["id"] = "legacy-hash"
});
var primary = (await store.GetOplogAfterAsync(new HlcTimestamp(0, 0, ""), DatasetId.Primary)).ToList();
var logs = (await store.GetOplogAfterAsync(new HlcTimestamp(0, 0, ""), DatasetId.Logs)).ToList();
primary.Any(entry => entry.Hash == "legacy-hash").ShouldBeTrue();
logs.Any(entry => entry.Hash == "legacy-hash").ShouldBeFalse();
}
private static OplogEntry CreateOplogEntry( private static OplogEntry CreateOplogEntry(
string collection, string collection,
string key, string key,
@@ -110,6 +172,34 @@ public class SurrealDocumentMetadataStoreContractTests
var exported = (await store.ExportAsync()).ToList(); var exported = (await store.ExportAsync()).ToList();
exported.Count.ShouldBe(3); exported.Count.ShouldBe(3);
} }
/// <summary>
/// Verifies document metadata records do not leak across datasets.
/// </summary>
[Fact]
public async Task DocumentMetadataStore_DatasetIsolation_Works()
{
await using var harness = new SurrealTestHarness();
var store = harness.CreateDocumentMetadataStore();
await store.UpsertMetadataAsync(
new DocumentMetadata("Users", "doc-shared", new HlcTimestamp(100, 0, "node-a"), false, DatasetId.Primary),
DatasetId.Primary);
await store.UpsertMetadataAsync(
new DocumentMetadata("Users", "doc-shared", new HlcTimestamp(101, 0, "node-a"), false, DatasetId.Logs),
DatasetId.Logs);
var primary = await store.GetMetadataAsync("Users", "doc-shared", DatasetId.Primary);
var logs = await store.GetMetadataAsync("Users", "doc-shared", DatasetId.Logs);
primary.ShouldNotBeNull();
primary.DatasetId.ShouldBe(DatasetId.Primary);
primary.UpdatedAt.ShouldBe(new HlcTimestamp(100, 0, "node-a"));
logs.ShouldNotBeNull();
logs.DatasetId.ShouldBe(DatasetId.Logs);
logs.UpdatedAt.ShouldBe(new HlcTimestamp(101, 0, "node-a"));
}
} }
public class SurrealPeerConfigurationStoreContractTests public class SurrealPeerConfigurationStoreContractTests
@@ -206,6 +296,42 @@ public class SurrealPeerOplogConfirmationStoreContractTests
afterDeactivate.All(x => x.IsActive == false).ShouldBeTrue(); afterDeactivate.All(x => x.IsActive == false).ShouldBeTrue();
} }
/// <summary>
/// Verifies peer confirmations are isolated by dataset.
/// </summary>
[Fact]
public async Task PeerOplogConfirmationStore_DatasetIsolation_Works()
{
await using var harness = new SurrealTestHarness();
var store = harness.CreatePeerOplogConfirmationStore();
await store.EnsurePeerRegisteredAsync("peer-a", "10.0.0.10:5050", PeerType.StaticRemote, DatasetId.Primary);
await store.EnsurePeerRegisteredAsync("peer-a", "10.0.0.10:5050", PeerType.StaticRemote, DatasetId.Logs);
await store.UpdateConfirmationAsync(
"peer-a",
"source-1",
new HlcTimestamp(100, 1, "source-1"),
"hash-primary",
DatasetId.Primary);
await store.UpdateConfirmationAsync(
"peer-a",
"source-1",
new HlcTimestamp(200, 1, "source-1"),
"hash-logs",
DatasetId.Logs);
var primary = (await store.GetConfirmationsForPeerAsync("peer-a", DatasetId.Primary)).ToList();
var logs = (await store.GetConfirmationsForPeerAsync("peer-a", DatasetId.Logs)).ToList();
primary.Count.ShouldBe(1);
primary[0].ConfirmedHash.ShouldBe("hash-primary");
primary[0].DatasetId.ShouldBe(DatasetId.Primary);
logs.Count.ShouldBe(1);
logs[0].ConfirmedHash.ShouldBe("hash-logs");
logs[0].DatasetId.ShouldBe(DatasetId.Logs);
}
/// <summary> /// <summary>
/// Verifies merge semantics prefer newer confirmations and preserve active-state transitions. /// Verifies merge semantics prefer newer confirmations and preserve active-state transitions.
/// </summary> /// </summary>
@@ -343,6 +469,45 @@ public class SurrealSnapshotMetadataStoreContractTests
all[0].NodeId.ShouldBe("node-a"); all[0].NodeId.ShouldBe("node-a");
all[1].NodeId.ShouldBe("node-b"); all[1].NodeId.ShouldBe("node-b");
} }
/// <summary>
/// Verifies snapshot metadata rows are isolated by dataset.
/// </summary>
[Fact]
public async Task SnapshotMetadataStore_DatasetIsolation_Works()
{
await using var harness = new SurrealTestHarness();
var store = harness.CreateSnapshotMetadataStore();
await store.InsertSnapshotMetadataAsync(new SnapshotMetadata
{
DatasetId = DatasetId.Primary,
NodeId = "node-a",
TimestampPhysicalTime = 100,
TimestampLogicalCounter = 0,
Hash = "hash-primary"
}, DatasetId.Primary);
await store.InsertSnapshotMetadataAsync(new SnapshotMetadata
{
DatasetId = DatasetId.Logs,
NodeId = "node-a",
TimestampPhysicalTime = 200,
TimestampLogicalCounter = 0,
Hash = "hash-logs"
}, DatasetId.Logs);
var primary = await store.GetSnapshotMetadataAsync("node-a", DatasetId.Primary);
var logs = await store.GetSnapshotMetadataAsync("node-a", DatasetId.Logs);
primary.ShouldNotBeNull();
primary.Hash.ShouldBe("hash-primary");
primary.DatasetId.ShouldBe(DatasetId.Primary);
logs.ShouldNotBeNull();
logs.Hash.ShouldBe("hash-logs");
logs.DatasetId.ShouldBe(DatasetId.Logs);
}
} }
internal sealed class SurrealTestHarness : IAsyncDisposable internal sealed class SurrealTestHarness : IAsyncDisposable
@@ -431,6 +596,11 @@ internal sealed class SurrealTestHarness : IAsyncDisposable
NullLogger<SurrealSnapshotMetadataStore>.Instance); NullLogger<SurrealSnapshotMetadataStore>.Instance);
} }
/// <summary>
/// Gets the embedded Surreal client used by this harness.
/// </summary>
public ICBDDCSurrealEmbeddedClient SurrealEmbeddedClient => _client;
/// <inheritdoc /> /// <inheritdoc />
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {

View File

@@ -113,12 +113,7 @@ internal sealed class BenchmarkPeerNode : IAsyncDisposable
public async Task UpsertUserAsync(User user) public async Task UpsertUserAsync(User user)
{ {
User? existing = Context.Users.Find(u => u.Id == user.Id).FirstOrDefault();
if (existing == null)
await Context.Users.InsertAsync(user);
else
await Context.Users.UpdateAsync(user); await Context.Users.UpdateAsync(user);
await Context.SaveChangesAsync(); await Context.SaveChangesAsync();
} }
@@ -127,6 +122,11 @@ internal sealed class BenchmarkPeerNode : IAsyncDisposable
return Context.Users.Find(u => u.Id == userId).Any(); return Context.Users.Find(u => u.Id == userId).Any();
} }
public int CountUsersWithPrefix(string prefix)
{
return Context.Users.FindAll().Count(u => u.Id.StartsWith(prefix, StringComparison.Ordinal));
}
public async ValueTask DisposeAsync() public async ValueTask DisposeAsync()
{ {
try try

View File

@@ -0,0 +1,142 @@
using BenchmarkDotNet.Attributes;
using System.Net;
using System.Net.Sockets;
using ZB.MOM.WW.CBDDC.Core.Network;
using ZB.MOM.WW.CBDDC.Sample.Console;
namespace ZB.MOM.WW.CDBBC.E2E.Benchmark.Tests;
[MemoryDiagnoser]
[SimpleJob(launchCount: 1, warmupCount: 0, iterationCount: 1)]
public class OfflineResyncThroughputBenchmarks
{
private const int BacklogOperationCount = 10_000;
private BenchmarkPeerNode _onlineNode = null!;
private BenchmarkPeerNode _offlineNode = null!;
private int _runSequence;
private string _currentPrefix = string.Empty;
[GlobalSetup]
public Task GlobalSetupAsync()
{
return Task.CompletedTask;
}
[GlobalCleanup]
public Task GlobalCleanupAsync()
{
// Avoid explicit node disposal in BenchmarkDotNet child processes due Surreal embedded callback race.
// Process teardown releases resources after benchmark completion.
return Task.CompletedTask;
}
[IterationSetup(Target = nameof(OfflineBacklogWriteThroughput100k))]
public void SetupOfflineWriteThroughput()
{
_currentPrefix = $"offline-write-{Interlocked.Increment(ref _runSequence):D6}";
InitializeIterationNodesAsync().GetAwaiter().GetResult();
}
[Benchmark(Description = "Offline backlog write throughput (10K ops)", OperationsPerInvoke = BacklogOperationCount)]
public async Task OfflineBacklogWriteThroughput100k()
{
await WriteBatchAsync(_currentPrefix, BacklogOperationCount);
}
[IterationSetup(Target = nameof(OfflineNodeResyncDurationAfter100kBacklog))]
public void SetupOfflineResyncBenchmark()
{
_currentPrefix = $"offline-resync-{Interlocked.Increment(ref _runSequence):D6}";
InitializeIterationNodesAsync().GetAwaiter().GetResult();
WriteBatchAsync(_currentPrefix, BacklogOperationCount).GetAwaiter().GetResult();
}
[Benchmark(Description = "Offline node re-sync duration after 10K backlog")]
public async Task OfflineNodeResyncDurationAfter100kBacklog()
{
await _offlineNode.StartAsync();
await WaitForReplicationAsync(_currentPrefix, BacklogOperationCount, TimeSpan.FromMinutes(3));
}
private async Task WriteBatchAsync(string prefix, int count)
{
for (var i = 0; i < count; i++)
{
string userId = $"{prefix}-{i:D6}";
await _onlineNode.UpsertUserAsync(CreateUser(userId));
}
}
private async Task WaitForReplicationAsync(string prefix, int expectedCount, TimeSpan timeout)
{
DateTime deadline = DateTime.UtcNow.Add(timeout);
while (DateTime.UtcNow < deadline)
{
if (_offlineNode.CountUsersWithPrefix(prefix) >= expectedCount)
return;
await Task.Delay(250);
}
int replicatedCount = _offlineNode.CountUsersWithPrefix(prefix);
throw new TimeoutException(
$"Timed out waiting for re-sync. Expected {expectedCount}, replicated {replicatedCount}.");
}
private static User CreateUser(string userId)
{
return new User
{
Id = userId,
Name = $"user-{userId}",
Age = 30,
Address = new Address { City = "OfflineBenchmarkCity" }
};
}
private static int GetAvailableTcpPort()
{
using var listener = new TcpListener(IPAddress.Loopback, 0);
listener.Start();
return ((IPEndPoint)listener.LocalEndpoint).Port;
}
private async Task InitializeIterationNodesAsync()
{
int onlinePort = GetAvailableTcpPort();
int offlinePort = GetAvailableTcpPort();
while (offlinePort == onlinePort)
offlinePort = GetAvailableTcpPort();
string clusterToken = Guid.NewGuid().ToString("N");
_onlineNode = BenchmarkPeerNode.Create(
"offline-benchmark-online",
onlinePort,
clusterToken,
[
new KnownPeerConfiguration
{
NodeId = "offline-benchmark-offline",
Host = "127.0.0.1",
Port = offlinePort
}
]);
_offlineNode = BenchmarkPeerNode.Create(
"offline-benchmark-offline",
offlinePort,
clusterToken,
[
new KnownPeerConfiguration
{
NodeId = "offline-benchmark-online",
Host = "127.0.0.1",
Port = onlinePort
}
]);
await _onlineNode.StartAsync();
await Task.Delay(250);
}
}

View File

@@ -0,0 +1,50 @@
using System;
using System.Collections.Generic;
namespace ZB.MOM.WW.CDBBC.E2E.Benchmark.Tests;
/// <summary>
/// Represents a typical Serilog log entry.
/// </summary>
public sealed class SerilogLogEntry
{
/// <summary>
/// Timestamp when the log event was written.
/// </summary>
public DateTimeOffset Timestamp { get; set; }
/// <summary>
/// Log level (for example: Information, Warning, Error).
/// </summary>
public string Level { get; set; } = "Information";
/// <summary>
/// Name of the logger/category (typically the class name).
/// </summary>
public string? LoggerName { get; set; }
/// <summary>
/// Correlation context identifier used to tie log entries to a request.
/// </summary>
public string? ContextId { get; set; }
/// <summary>
/// Original message template used by Serilog.
/// </summary>
public string MessageTemplate { get; set; } = string.Empty;
/// <summary>
/// Fully rendered message text.
/// </summary>
public string? RenderedMessage { get; set; }
/// <summary>
/// Exception details if one was logged.
/// </summary>
public string? Exception { get; set; }
/// <summary>
/// Structured context values captured from Serilog context.
/// </summary>
public Dictionary<string, object?> ContextProperties { get; set; } = new(StringComparer.Ordinal);
}

View File

@@ -0,0 +1,440 @@
using System.Diagnostics;
using System.Text;
using BenchmarkDotNet.Attributes;
using SurrealDb.Net.Models;
using SurrealDb.Net.Models.Response;
using ZB.MOM.WW.CBDDC.Persistence.Surreal;
namespace ZB.MOM.WW.CDBBC.E2E.Benchmark.Tests;
[MemoryDiagnoser]
[SimpleJob(launchCount: 1, warmupCount: 1, iterationCount: 3)]
public class SurrealLogStorageBenchmarks
{
private const int LogRecordCount = 100_000;
private const int InsertBatchSize = 500;
private const string LogTable = "benchmark_log_entry";
private const string LogKvTable = "benchmark_log_kv";
private static readonly string[] LoggerNames =
[
"Api.RequestHandler",
"Api.AuthController",
"Api.OrderController",
"Api.InventoryController",
"Api.CustomerController",
"Workers.OutboxPublisher",
"Workers.NotificationDispatcher",
"Infrastructure.SqlRepository",
"Infrastructure.CacheService",
"Infrastructure.HttpClient",
"Domain.OrderService",
"Domain.PricingService"
];
private static readonly string[] TenantIds =
[
"tenant-01",
"tenant-02",
"tenant-03",
"tenant-04",
"tenant-05",
"tenant-06",
"tenant-07",
"tenant-08"
];
private CBDDCSurrealEmbeddedClient _surrealClient = null!;
private string _databasePath = string.Empty;
private string _workDir = string.Empty;
private DateTime _seedBaseUtc;
private DateTime _queryRangeStartUtc;
private DateTime _queryRangeEndUtc;
private string _contextIdQueryValue = string.Empty;
private string _loggerQueryValue = string.Empty;
private string _contextStringKeyQueryValue = string.Empty;
private string _contextStringValueQueryValue = string.Empty;
private string _contextNumericKeyQueryValue = string.Empty;
private int _contextNumericValueQueryValue;
[GlobalSetup]
public async Task GlobalSetupAsync()
{
_workDir = Path.Combine(Path.GetTempPath(), $"cbddc-serilog-benchmark-{Guid.NewGuid():N}");
Directory.CreateDirectory(_workDir);
_databasePath = Path.Combine(_workDir, "serilog.rocksdb");
_surrealClient = new CBDDCSurrealEmbeddedClient(
new CBDDCSurrealEmbeddedOptions
{
Endpoint = "rocksdb://local",
DatabasePath = _databasePath,
Namespace = "cbddc_benchmark",
Database = $"serilog_{Guid.NewGuid():N}",
Cdc = new CBDDCSurrealCdcOptions { Enabled = false }
});
await _surrealClient.InitializeAsync();
await DefineSchemaAndIndexesAsync();
_seedBaseUtc = DateTime.UtcNow.AddDays(-1);
_contextIdQueryValue = BuildContextId(LogRecordCount / 2);
_loggerQueryValue = LoggerNames[3];
_contextStringKeyQueryValue = "tenantId";
_contextStringValueQueryValue = TenantIds[5];
_contextNumericKeyQueryValue = "statusCode";
_contextNumericValueQueryValue = 500;
_queryRangeStartUtc = _seedBaseUtc.AddMinutes(6);
_queryRangeEndUtc = _queryRangeStartUtc.AddSeconds(30);
var seedTimer = Stopwatch.StartNew();
await InsertLogRecordsAsync();
seedTimer.Stop();
long sizeBytes = CalculatePathSizeBytes(_databasePath);
Console.WriteLine(
$"Seeded {LogRecordCount:N0} records in {seedTimer.Elapsed.TotalSeconds:F2}s. " +
$"RocksDB size: {sizeBytes / (1024d * 1024d):F2} MiB ({sizeBytes:N0} bytes). Path: {_databasePath}");
}
[GlobalCleanup]
public Task GlobalCleanupAsync()
{
// Avoid explicit Surreal embedded disposal in benchmark child processes due known native callback race.
return Task.CompletedTask;
}
[Benchmark(Description = "Query by contextId (latest 200 rows)")]
public async Task QueryByContextIdAsync()
{
await ExecuteQueryAsync(
$"""
SELECT * FROM {LogTable}
WHERE contextId = $contextId
ORDER BY timestamp DESC
LIMIT 200;
""",
new Dictionary<string, object?> { ["contextId"] = _contextIdQueryValue });
}
[Benchmark(Description = "Query by loggerName + timestamp range (latest 200 rows)")]
public async Task QueryByLoggerAndTimestampAsync()
{
await ExecuteQueryAsync(
$"""
SELECT * FROM {LogTable}
WHERE loggerName = $loggerName
AND timestamp >= $fromTs
AND timestamp <= $toTs
ORDER BY timestamp DESC
LIMIT 200;
""",
new Dictionary<string, object?>
{
["loggerName"] = _loggerQueryValue,
["fromTs"] = _queryRangeStartUtc,
["toTs"] = _queryRangeEndUtc
});
}
[Benchmark(Description = "Query by loggerName + timestamp + arbitrary context string key/value")]
public async Task QueryByLoggerTimestampAndContextKeyAsync()
{
await ExecuteQueryAsync(
$"""
LET $logIds = (
SELECT VALUE logId FROM {LogKvTable}
WHERE loggerName = $loggerName
AND key = $contextKey
AND valueStr = $contextValueStr
AND timestamp >= $fromTs
AND timestamp <= $toTs
ORDER BY timestamp DESC
LIMIT 200
);
SELECT * FROM {LogTable}
WHERE id INSIDE $logIds
ORDER BY timestamp DESC
LIMIT 200;
""",
new Dictionary<string, object?>
{
["loggerName"] = _loggerQueryValue,
["fromTs"] = _queryRangeStartUtc,
["toTs"] = _queryRangeEndUtc,
["contextKey"] = _contextStringKeyQueryValue,
["contextValueStr"] = _contextStringValueQueryValue
});
}
[Benchmark(Description = "Query by loggerName + timestamp + arbitrary context number key/value")]
public async Task QueryByLoggerTimestampAndNumericContextKeyAsync()
{
await ExecuteQueryAsync(
$"""
LET $logIds = (
SELECT VALUE logId FROM {LogKvTable}
WHERE loggerName = $loggerName
AND key = $contextKey
AND valueNum = $contextValueNum
AND timestamp >= $fromTs
AND timestamp <= $toTs
ORDER BY timestamp DESC
LIMIT 200
);
SELECT * FROM {LogTable}
WHERE id INSIDE $logIds
ORDER BY timestamp DESC
LIMIT 200;
""",
new Dictionary<string, object?>
{
["loggerName"] = _loggerQueryValue,
["fromTs"] = _queryRangeStartUtc,
["toTs"] = _queryRangeEndUtc,
["contextKey"] = _contextNumericKeyQueryValue,
["contextValueNum"] = _contextNumericValueQueryValue
});
}
[Benchmark(Description = "RocksDB size (bytes)")]
public long GetDatabaseFileSizeBytes()
{
return CalculatePathSizeBytes(_databasePath);
}
private async Task DefineSchemaAndIndexesAsync()
{
string schemaSql =
$"""
DEFINE TABLE OVERWRITE {LogTable} SCHEMAFULL;
DEFINE FIELD OVERWRITE timestamp ON TABLE {LogTable} TYPE datetime;
DEFINE FIELD OVERWRITE level ON TABLE {LogTable} TYPE string;
DEFINE FIELD OVERWRITE loggerName ON TABLE {LogTable} TYPE option<string>;
DEFINE FIELD OVERWRITE contextId ON TABLE {LogTable} TYPE option<string>;
DEFINE FIELD OVERWRITE messageTemplate ON TABLE {LogTable} TYPE string;
DEFINE FIELD OVERWRITE renderedMessage ON TABLE {LogTable} TYPE option<string>;
DEFINE FIELD OVERWRITE exception ON TABLE {LogTable} TYPE option<string>;
DEFINE FIELD OVERWRITE contextValues ON TABLE {LogTable} TYPE object FLEXIBLE;
DEFINE INDEX OVERWRITE idx_log_contextid_ts
ON TABLE {LogTable} COLUMNS contextId, timestamp;
DEFINE INDEX OVERWRITE idx_log_logger_ts
ON TABLE {LogTable} COLUMNS loggerName, timestamp;
DEFINE TABLE OVERWRITE {LogKvTable} SCHEMAFULL;
DEFINE FIELD OVERWRITE logId ON TABLE {LogKvTable} TYPE record<{LogTable}>;
DEFINE FIELD OVERWRITE loggerName ON TABLE {LogKvTable} TYPE string;
DEFINE FIELD OVERWRITE timestamp ON TABLE {LogKvTable} TYPE datetime;
DEFINE FIELD OVERWRITE key ON TABLE {LogKvTable} TYPE string;
DEFINE FIELD OVERWRITE valueStr ON TABLE {LogKvTable} TYPE option<string>;
DEFINE FIELD OVERWRITE valueNum ON TABLE {LogKvTable} TYPE option<number>;
DEFINE FIELD OVERWRITE valueBool ON TABLE {LogKvTable} TYPE option<bool>;
DEFINE INDEX OVERWRITE idx_logkv_logger_key_str_ts
ON TABLE {LogKvTable} COLUMNS loggerName, key, valueStr, timestamp;
DEFINE INDEX OVERWRITE idx_logkv_logger_key_num_ts
ON TABLE {LogKvTable} COLUMNS loggerName, key, valueNum, timestamp;
DEFINE INDEX OVERWRITE idx_logkv_logger_key_bool_ts
ON TABLE {LogKvTable} COLUMNS loggerName, key, valueBool, timestamp;
DEFINE INDEX OVERWRITE idx_logkv_logid
ON TABLE {LogKvTable} COLUMNS logId;
""";
var response = await _surrealClient.RawQueryAsync(schemaSql);
EnsureSuccessfulResponse(response, "Schema definition");
}
private async Task InsertLogRecordsAsync()
{
for (var batchStart = 0; batchStart < LogRecordCount; batchStart += InsertBatchSize)
{
int batchCount = Math.Min(InsertBatchSize, LogRecordCount - batchStart);
var sqlBuilder = new StringBuilder();
sqlBuilder.AppendLine("BEGIN TRANSACTION;");
var parameters = new Dictionary<string, object?>(batchCount * 2);
for (var offset = 0; offset < batchCount; offset++)
{
int sequence = batchStart + offset;
string idParameterName = $"id{offset}";
string recordParameterName = $"record{offset}";
string logId = $"log-{sequence:D8}";
RecordId logRecordId = RecordId.From(LogTable, logId);
IReadOnlyDictionary<string, object?> logRecord = CreateLogRecord(sequence);
parameters[idParameterName] = logRecordId;
parameters[recordParameterName] = logRecord;
sqlBuilder.Append("UPSERT $")
.Append(idParameterName)
.Append(" CONTENT $")
.Append(recordParameterName)
.AppendLine(";");
int kvOrdinal = 0;
foreach (IReadOnlyDictionary<string, object?> kvRow in CreateKvRows(logId, logRecordId, logRecord))
{
string kvIdParameterName = $"kvid{offset}_{kvOrdinal}";
string kvRecordParameterName = $"kvrecord{offset}_{kvOrdinal}";
parameters[kvIdParameterName] = RecordId.From(LogKvTable, $"{logId}-{kvOrdinal:D2}");
parameters[kvRecordParameterName] = kvRow;
sqlBuilder.Append("UPSERT $")
.Append(kvIdParameterName)
.Append(" CONTENT $")
.Append(kvRecordParameterName)
.AppendLine(";");
kvOrdinal++;
}
}
sqlBuilder.AppendLine("COMMIT TRANSACTION;");
var response = await _surrealClient.RawQueryAsync(sqlBuilder.ToString(), parameters);
EnsureSuccessfulResponse(response, $"Insert batch starting at row {batchStart}");
}
}
private IReadOnlyDictionary<string, object?> CreateLogRecord(int sequence)
{
DateTime timestamp = _seedBaseUtc.AddMilliseconds(sequence * 10L);
string loggerName = LoggerNames[sequence % LoggerNames.Length];
string tenantId = TenantIds[sequence % TenantIds.Length];
bool isBackground = sequence % 7 == 0;
string? exception = sequence % 2_500 == 0
? "System.InvalidOperationException: simulated benchmark exception."
: null;
var record = new Dictionary<string, object?>
{
["timestamp"] = timestamp,
["level"] = ResolveLogLevel(sequence),
["loggerName"] = loggerName,
["contextId"] = BuildContextId(sequence),
["messageTemplate"] = "Processed request {RequestId} for {Route}",
["renderedMessage"] = $"Processed request req-{sequence:D8} for /api/items/{sequence % 250}",
["contextValues"] = new Dictionary<string, object?>
{
["tenantId"] = tenantId,
["requestId"] = $"req-{sequence:D8}",
["route"] = $"/api/items/{sequence % 250}",
["statusCode"] = sequence % 20 == 0 ? 500 : 200,
["elapsedMs"] = 5 + (sequence % 200),
["nodeId"] = $"node-{sequence % 8:D2}",
["isBackground"] = isBackground
}
};
if (!string.IsNullOrEmpty(exception))
record["exception"] = exception;
return record;
}
private static IEnumerable<IReadOnlyDictionary<string, object?>> CreateKvRows(
string logId,
RecordId logRecordId,
IReadOnlyDictionary<string, object?> logRecord)
{
if (!logRecord.TryGetValue("loggerName", out object? loggerNameValue) || loggerNameValue is not string loggerName)
yield break;
if (!logRecord.TryGetValue("timestamp", out object? timestampValue) || timestampValue is not DateTime timestamp)
yield break;
if (!logRecord.TryGetValue("contextValues", out object? contextValuesObject) ||
contextValuesObject is not IReadOnlyDictionary<string, object?> contextValues)
yield break;
foreach ((string key, object? value) in contextValues)
{
if (value == null) continue;
var row = new Dictionary<string, object?>
{
["logId"] = logRecordId,
["loggerName"] = loggerName,
["timestamp"] = timestamp,
["key"] = key
};
switch (value)
{
case string stringValue:
row["valueStr"] = stringValue;
break;
case bool boolValue:
row["valueBool"] = boolValue;
break;
case sbyte or byte or short or ushort or int or uint or long or ulong or float or double or decimal:
row["valueNum"] = Convert.ToDouble(value);
break;
default:
row["valueStr"] = value.ToString();
break;
}
yield return row;
}
}
private async Task ExecuteQueryAsync(string query, IReadOnlyDictionary<string, object?> parameters)
{
var response = await _surrealClient.RawQueryAsync(query, parameters);
EnsureSuccessfulResponse(response, "Query execution");
}
private static string BuildContextId(int sequence)
{
return $"ctx-{sequence / 10:D6}";
}
private static string ResolveLogLevel(int sequence)
{
if (sequence % 2_500 == 0) return "Error";
if (sequence % 500 == 0) return "Warning";
return "Information";
}
private static long CalculatePathSizeBytes(string path)
{
if (File.Exists(path)) return new FileInfo(path).Length;
if (!Directory.Exists(path)) return 0;
long size = 0;
foreach (string file in Directory.EnumerateFiles(path, "*", SearchOption.AllDirectories))
size += new FileInfo(file).Length;
return size;
}
private static void EnsureSuccessfulResponse(SurrealDbResponse response, string operation)
{
if (!response.HasErrors) return;
string errorSummary = string.Join(
" | ",
response.Errors.Take(3).Select((error, index) => DescribeError(error, index)));
throw new InvalidOperationException(
$"{operation} failed with SurrealDB errors. Details: {errorSummary}");
}
private static string DescribeError(object error, int index)
{
Type errorType = error.GetType();
string[] fieldsToExtract = ["Status", "Details", "Description", "Information", "Code"];
var extracted = new List<string>();
foreach (string field in fieldsToExtract)
{
object? value = errorType.GetProperty(field)?.GetValue(error);
if (value != null) extracted.Add($"{field}={value}");
}
if (extracted.Count == 0) return $"error[{index}] type={errorType.Name}";
return $"error[{index}] type={errorType.Name} {string.Join(", ", extracted)}";
}
}