Add LMDB oplog migration path with dual-write cutover support
All checks were successful
NuGet Package Publish / nuget (push) Successful in 1m16s

Introduce LMDB oplog store, migration flags, telemetry/backfill tooling, and parity tests to enable staged Surreal-to-LMDB rollout with rollback coverage.
This commit is contained in:
Joseph Doherty
2026-02-22 17:44:57 -05:00
parent 3b9ff69adc
commit cce24fa8f3
16 changed files with 3601 additions and 6 deletions

View File

@@ -237,6 +237,83 @@ Surreal persistence now stores `datasetId` on oplog, metadata, snapshot metadata
4. **Delete durability**: deletes persist as oplog delete operations plus tombstone metadata.
5. **Remote apply behavior**: remote sync applies documents without generating local loopback CDC entries.
## LMDB Oplog Migration Mode
CBDDC now supports an LMDB-backed oplog provider for staged cutover from Surreal oplog tables.
### Registration
```csharp
services.AddCBDDCCore()
.AddCBDDCSurrealEmbedded<SampleDocumentStore>(optionsFactory)
.AddCBDDCLmdbOplog(
_ => new LmdbOplogOptions
{
EnvironmentPath = "/var/lib/cbddc/oplog-lmdb",
MapSizeBytes = 256L * 1024 * 1024,
MaxDatabases = 16,
PruneBatchSize = 512
},
flags =>
{
flags.UseLmdbOplog = true;
flags.DualWriteOplog = true;
flags.PreferLmdbReads = false;
});
```
### Feature Flags
- `UseLmdbOplog`: enables LMDB migration path.
- `DualWriteOplog`: mirrors writes to Surreal + LMDB.
- `PreferLmdbReads`: cuts reads over to LMDB.
- `EnableReadShadowValidation`: compares Surreal/LMDB read results and logs mismatches.
### Consistency Model
The initial migration model is eventual cross-engine atomicity (Option A):
- Surreal local CDC transactions remain authoritative for atomic document + metadata persistence.
- LMDB is backfilled/reconciled when LMDB reads are preferred and LMDB is missing recent Surreal writes.
- During rollout, keep dual-write enabled until mismatch logs remain stable.
### Backfill Utility
`LmdbOplogBackfillTool` performs Surreal -> LMDB oplog backfill and parity validation per dataset:
```csharp
var backfill = provider.GetRequiredService<LmdbOplogBackfillTool>();
LmdbOplogBackfillReport report = await backfill.BackfillOrThrowAsync(DatasetId.Primary);
```
Validation includes:
- total entry counts
- per-node entry counts
- latest hash per node
- hash spot checks
- chain-range spot checks
### Migration Telemetry
`FeatureFlagOplogStore` records migration counters through `OplogMigrationTelemetry`:
- shadow comparisons
- shadow mismatches
- LMDB preferred-read fallbacks to Surreal
- reconciliation runs and reconciled entry counts (global + per dataset)
You can resolve `OplogMigrationTelemetry` from DI or call `GetTelemetrySnapshot()` on `FeatureFlagOplogStore`.
### Rollback Path
To roll back read/write behavior to Surreal during migration:
- set `PreferLmdbReads = false`
- set `DualWriteOplog = false`
With `UseLmdbOplog = true`, this keeps LMDB services available while routing reads/writes to Surreal only.
If LMDB should be fully disabled, set `UseLmdbOplog = false`.
## Feature Comparison
| Feature | SQLite (Direct) | EF Core | PostgreSQL | Surreal Embedded |