All checks were successful
CI / verify (push) Successful in 2m33s
Add canonical operations/security/access/feature docs and fix path integrity to improve onboarding and incident readiness.
832 lines
27 KiB
Markdown
Executable File
832 lines
27 KiB
Markdown
Executable File
# CBDDC
|
|
|
|
<div align="center">
|
|
|
|
**Peer-to-Peer Data Synchronization Middleware for .NET**
|
|
|
|
[](https://dotnet.microsoft.com/)
|
|
|
|
## Status
|
|

|
|

|
|
|
|
CBDDC is not a database - it's a **sync layer** that plugs into your existing data store and enables automatic peer-to-peer replication across nodes in a mesh network.
|
|
|
|
[Architecture](#architecture) | [Quick Start](#quick-start) | [Integration Guide](#integrating-with-your-database) | [Documentation](#documentation)
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
- [Overview](#overview)
|
|
- [Ownership and Support](#ownership-and-support)
|
|
- [Architecture](#architecture)
|
|
- [Key Features](#key-features)
|
|
- [Installation](#installation)
|
|
- [Prerequisites](#prerequisites)
|
|
- [Configuration and Secrets](#configuration-and-secrets)
|
|
- [Build, Test, and Quality Gates](#build-test-and-quality-gates)
|
|
- [Deployment and Rollback](#deployment-and-rollback)
|
|
- [Operations and Incident Response](#operations-and-incident-response)
|
|
- [Security and Compliance Posture](#security-and-compliance-posture)
|
|
- [Troubleshooting](#troubleshooting)
|
|
- [Change Governance](#change-governance)
|
|
- [Quick Start](#quick-start)
|
|
- [Integrating with Your Database](#integrating-with-your-database)
|
|
- [Cloud Deployment](#cloud-deployment)
|
|
- [Production Features](#production-features)
|
|
- [Use Cases](#use-cases)
|
|
- [Documentation](#documentation)
|
|
- [Contributing](#contributing)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
**CBDDC** is a lightweight, embeddable **data synchronization middleware** for .NET. It observes changes in your database via **Change Data Capture (CDC)**, records them in an append-only **Oplog**, and replicates them across nodes connected via a **P2P mesh network**.
|
|
|
|
Your application continues to read and write to its database as usual. CBDDC works in the background.
|
|
|
|
> **[LAN] Designed for Local Area Networks (LAN)**
|
|
> Built for trusted environments: offices, retail stores, edge deployments. Cross-platform (Windows, Linux, macOS).
|
|
|
|
> **[Cloud] Cloud Ready**
|
|
> ASP.NET Core hosting for controlled, server-side deployments.
|
|
|
|
---
|
|
|
|
## Ownership and Support
|
|
|
|
- **Owning team**: CBDDC Core Maintainers
|
|
- **Issue reporting**: [GitHub Issues](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/issues)
|
|
- **Usage/support questions**: [GitHub Discussions](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/discussions)
|
|
- **Incident escalation for active deployments**: Follow your local on-call process first, then attach CBDDC diagnostics from `docs/runbook.md`
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
+---------------------------------------------------+
|
|
| Your Application |
|
|
| db.Users.InsertAsync(user) |
|
|
| db.Users.Find(u => u.Age > 18) |
|
|
+---------------------------------------------------+
|
|
| uses your DbContext directly
|
|
+---------------------------------------------------+
|
|
| Your Database (BLite) |
|
|
| +---------------------------------------------+ |
|
|
| | Users | Orders | Products | ... |
|
|
| +---------------------------------------------+ |
|
|
| | CDC (Change Data Capture) |
|
|
| | |
|
|
| +---------------------------------------------+ |
|
|
| | CBDDC Sync Engine | |
|
|
| | - Oplog (append-only hash-chained journal)| |
|
|
| | - Vector Clock (causal ordering) | |
|
|
| | - Conflict Resolution (LWW / Custom Merge)| |
|
|
| +---------------------------------------------+ |
|
|
+---------------------------------------------------+
|
|
| P2P Network (TCP + UDP Discovery)
|
|
+---------------------------------------------------+
|
|
| Other Nodes (same setup) |
|
|
| Node-A <-----> Node-B <-----> Node-C |
|
|
+---------------------------------------------------+
|
|
```
|
|
|
|
### Core Concepts
|
|
|
|
| Concept | Description |
|
|
|---------|-------------|
|
|
| **Oplog** | Append-only journal of changes, hash-chained per node for integrity |
|
|
| **Vector Clock** | Tracks causal ordering - knows who has what across the mesh |
|
|
| **CDC** | Change Data Capture - watches your registered collections for local writes |
|
|
| **Document Store** | Your bridge class - maps between your entities and the sync engine |
|
|
| **Conflict Resolution** | Pluggable strategy (Last-Write-Wins or custom recursive merge) |
|
|
| **VectorClockService** | Shared singleton keeping the Vector Clock in sync between CDC and OplogStore |
|
|
|
|
### Sync Flow
|
|
|
|
```
|
|
Local Write -> CDC Trigger -> OplogEntry Created -> VectorClock Updated
|
|
|
|
|
v
|
|
SyncOrchestrator
|
|
(gossip every 2s)
|
|
|
|
|
+---------+----------+
|
|
| |
|
|
Push changes Pull changes
|
|
to peers from peers
|
|
| |
|
|
v v
|
|
Remote node Apply to local
|
|
applies via OplogStore +
|
|
ApplyBatchAsync DocumentStore
|
|
```
|
|
|
|
---
|
|
|
|
## Key Features
|
|
|
|
### [Select] Selective Collection Sync
|
|
Only collections registered via `WatchCollection()` are tracked. Your database can have hundreds of tables - only the ones you opt-in participate in replication.
|
|
|
|
### [Gossip] Interest-Aware Gossip
|
|
Nodes advertise which collections they sync. The orchestrator prioritizes peers sharing common interests, reducing unnecessary traffic.
|
|
|
|
### [Offline] Offline First
|
|
- Read/write operations work offline - they're direct database operations
|
|
- Automatic sync when peers reconnect
|
|
- Oplog-based gap recovery and snapshot fallback
|
|
|
|
### [Secure] Secure Networking
|
|
- Noise Protocol handshake with ECDH key exchange
|
|
- AES-256 encryption for data in transit
|
|
- HMAC authentication
|
|
- Brotli compression for bandwidth efficiency
|
|
|
|
### [Conflict] Conflict Resolution
|
|
- **Last Write Wins (LWW)** - default, HLC timestamp-based
|
|
- **Recursive Merge** - deep JSON merge for concurrent edits
|
|
- **Custom** - implement `IConflictResolver` for your business logic
|
|
|
|
### [Cloud] Cloud Infrastructure
|
|
- ASP.NET Core hosting (single-cluster mode)
|
|
- BLite embedded persistence
|
|
- shared-token authentication
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
### Packages
|
|
|
|
| Package | Purpose |
|
|
|---------|---------|
|
|
| `ZB.MOM.WW.CBDDC.Core` | Interfaces, models, conflict resolution (.NET Standard 2.0+) |
|
|
| `ZB.MOM.WW.CBDDC.Persistence` | BLite persistence provider, OplogStore, VectorClockService (.NET 10+) |
|
|
| `CBDDC.Network` | TCP sync, UDP discovery, Protobuf protocol (.NET Standard 2.0+) |
|
|
|
|
```bash
|
|
# BLite (embedded document DB)
|
|
dotnet add package ZB.MOM.WW.CBDDC.Core
|
|
dotnet add package ZB.MOM.WW.CBDDC.Persistence
|
|
dotnet add package CBDDC.Network
|
|
```
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
Before onboarding a new node, verify:
|
|
|
|
- .NET SDK/runtime required by selected packages (recommended: .NET 8+)
|
|
- Network access for configured TCP/UDP sync ports
|
|
- Persistent storage path with write permissions for database and backups
|
|
- Access to deployment secrets (cluster `AuthToken`, connection strings, environment-specific config)
|
|
- Ability to run health checks (`/health` in hosted mode)
|
|
|
|
---
|
|
|
|
## Configuration and Secrets
|
|
|
|
CBDDC runtime behavior should be configured from environment-specific settings, not hardcoded values.
|
|
|
|
- Required runtime settings:
|
|
- `NodeId`
|
|
- `TcpPort` (and `UdpPort` when discovery is enabled)
|
|
- `AuthToken` (secret; do not commit)
|
|
- Persistence connection/database path
|
|
- Secret handling guidance:
|
|
- Keep secrets in a secret manager or protected environment variables
|
|
- Rotate tokens during maintenance windows and validate mesh convergence after rotation
|
|
- Avoid exposing tokens in logs, screenshots, or troubleshooting output
|
|
|
|
See `docs/security.md` and `docs/production-hardening.md` for detailed control guidance.
|
|
|
|
---
|
|
|
|
## Build, Test, and Quality Gates
|
|
|
|
Run these checks before merge or release:
|
|
|
|
```bash
|
|
dotnet restore
|
|
dotnet build
|
|
dotnet test
|
|
```
|
|
|
|
Recommended release gates:
|
|
|
|
- Unit/integration tests pass
|
|
- Health endpoint reports healthy in staging after deployment
|
|
- No unresolved high-severity issues in deployment or runbook checklists
|
|
|
|
---
|
|
|
|
## Deployment and Rollback
|
|
|
|
Canonical deployment procedures are documented in:
|
|
|
|
- `docs/deployment.md` (promotion flow, validation gates, rollback path)
|
|
- `docs/deployment-lan.md` (LAN deployment details)
|
|
- `docs/deployment-modes.md` (hosted mode behavior)
|
|
- `docs/production-hardening.md` (operational hardening controls)
|
|
|
|
---
|
|
|
|
## Operations and Incident Response
|
|
|
|
Use `docs/runbook.md` as the primary operational runbook. It includes:
|
|
|
|
- Alert triage sequence
|
|
- Escalation path and severity mapping
|
|
- Recovery and verification steps
|
|
- Post-incident follow-up expectations
|
|
|
|
For peer lifecycle incidents, use `docs/peer-deprecation-removal-runbook.md`.
|
|
|
|
---
|
|
|
|
## Security and Compliance Posture
|
|
|
|
- Primary protections: authenticated peer communication and encrypted transport options in CBDDC networking components
|
|
- Sensitive data handling: classify data by environment and apply least privilege on runtime access
|
|
- Operational controls: hardening, key rotation, and monitoring requirements are documented in `docs/security.md` and `docs/access.md`
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
Common issue patterns and step-by-step remediation are documented in `docs/troubleshooting.md`.
|
|
|
|
High-priority troubleshooting topics:
|
|
|
|
- Peer unreachable or lagging confirmation state
|
|
- Persistence faults and backup restore flow
|
|
- Authentication mismatch or configuration drift
|
|
|
|
---
|
|
|
|
## Change Governance
|
|
|
|
- Use short-lived branches and pull requests for every change
|
|
- Require review before merge to protected branches
|
|
- Run build/test/health validation before release
|
|
- Record release-impacting changes in `CHANGELOG.md` and link related deployment notes
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### 1. Define Your Database Context
|
|
|
|
```csharp
|
|
public class MyDbContext : CBDDCDocumentDbContext
|
|
{
|
|
public DocumentCollection<string, Customer> Customers { get; private set; }
|
|
public DocumentCollection<string, Order> Orders { get; private set; }
|
|
|
|
public MyDbContext(string dbPath) : base(dbPath) { }
|
|
}
|
|
```
|
|
|
|
### 2. Create Your Document Store (the Sync Bridge)
|
|
|
|
This is where you tell CBDDC **which collections to sync** and **how to map** between your entities and the sync engine:
|
|
|
|
```csharp
|
|
public class MyDocumentStore : BLiteDocumentStore<MyDbContext>
|
|
{
|
|
public MyDocumentStore(
|
|
MyDbContext context,
|
|
IPeerNodeConfigurationProvider configProvider,
|
|
IVectorClockService vectorClockService,
|
|
ILogger<MyDocumentStore>? logger = null)
|
|
: base(context, configProvider, vectorClockService, logger: logger)
|
|
{
|
|
// Register collections for CDC - only these will be synced
|
|
WatchCollection("Customers", context.Customers, c => c.Id);
|
|
WatchCollection("Orders", context.Orders, o => o.Id);
|
|
}
|
|
|
|
// Map incoming sync data back to your entities
|
|
protected override async Task ApplyContentToEntityAsync(
|
|
string collection, string key, JsonElement content, CancellationToken ct)
|
|
{
|
|
switch (collection)
|
|
{
|
|
case "Customers":
|
|
var customer = content.Deserialize<Customer>()!;
|
|
customer.Id = key;
|
|
var existing = _context.Customers
|
|
.Find(c => c.Id == key).FirstOrDefault();
|
|
if (existing != null) _context.Customers.Update(customer);
|
|
else _context.Customers.Insert(customer);
|
|
break;
|
|
case "Orders":
|
|
var order = content.Deserialize<Order>()!;
|
|
order.Id = key;
|
|
var existingOrder = _context.Orders
|
|
.Find(o => o.Id == key).FirstOrDefault();
|
|
if (existingOrder != null) _context.Orders.Update(order);
|
|
else _context.Orders.Insert(order);
|
|
break;
|
|
}
|
|
await _context.SaveChangesAsync(ct);
|
|
}
|
|
|
|
protected override Task<JsonElement?> GetEntityAsJsonAsync(
|
|
string collection, string key, CancellationToken ct)
|
|
{
|
|
object? entity = collection switch
|
|
{
|
|
"Customers" => _context.Customers.Find(c => c.Id == key).FirstOrDefault(),
|
|
"Orders" => _context.Orders.Find(o => o.Id == key).FirstOrDefault(),
|
|
_ => null
|
|
};
|
|
return Task.FromResult(entity != null
|
|
? (JsonElement?)JsonSerializer.SerializeToElement(entity) : null);
|
|
}
|
|
|
|
protected override async Task RemoveEntityAsync(
|
|
string collection, string key, CancellationToken ct)
|
|
{
|
|
switch (collection)
|
|
{
|
|
case "Customers": _context.Customers.Delete(key); break;
|
|
case "Orders": _context.Orders.Delete(key); break;
|
|
}
|
|
await _context.SaveChangesAsync(ct);
|
|
}
|
|
|
|
protected override Task<IEnumerable<(string Key, JsonElement Content)>>
|
|
GetAllEntitiesAsJsonAsync(string collection, CancellationToken ct)
|
|
{
|
|
IEnumerable<(string, JsonElement)> result = collection switch
|
|
{
|
|
"Customers" => _context.Customers.FindAll()
|
|
.Select(c => (c.Id, JsonSerializer.SerializeToElement(c))),
|
|
"Orders" => _context.Orders.FindAll()
|
|
.Select(o => (o.Id, JsonSerializer.SerializeToElement(o))),
|
|
_ => Enumerable.Empty<(string, JsonElement)>()
|
|
};
|
|
return Task.FromResult(result);
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. Wire It Up
|
|
|
|
```csharp
|
|
var builder = Host.CreateApplicationBuilder();
|
|
|
|
// Configure the node
|
|
builder.Services.AddSingleton<IPeerNodeConfigurationProvider>(
|
|
new StaticPeerNodeConfigurationProvider(new PeerNodeConfiguration
|
|
{
|
|
NodeId = "node-1",
|
|
TcpPort = 8580,
|
|
AuthToken = "my-cluster-secret"
|
|
}));
|
|
|
|
// Register CBDDC services
|
|
builder.Services
|
|
.AddCBDDCCore()
|
|
.AddCBDDCBLite<MyDbContext, MyDocumentStore>(
|
|
sp => new MyDbContext("mydata.blite"))
|
|
.AddCBDDCNetwork<StaticPeerNodeConfigurationProvider>();
|
|
|
|
await builder.Build().RunAsync();
|
|
```
|
|
|
|
### 4. Use Your Database Normally
|
|
|
|
```csharp
|
|
public class MyService
|
|
{
|
|
private readonly MyDbContext _db;
|
|
|
|
public MyService(MyDbContext db) => _db = db;
|
|
|
|
public async Task CreateCustomer(string name)
|
|
{
|
|
// Write directly - CBDDC handles sync automatically
|
|
await _db.Customers.InsertAsync(
|
|
new Customer { Id = Guid.NewGuid().ToString(), Name = name });
|
|
await _db.SaveChangesAsync();
|
|
|
|
// Changes are automatically:
|
|
// 1. Detected via CDC
|
|
// 2. Recorded in the Oplog with HLC timestamp + hash chain
|
|
// 3. Pushed to connected peers via gossip
|
|
// 4. Applied on remote nodes via conflict resolution
|
|
}
|
|
|
|
public async Task<List<Customer>> GetYoungCustomers()
|
|
{
|
|
// Read directly from your DB - no CBDDC API
|
|
return _db.Customers.Find(c => c.Age < 30).ToList();
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Integrating with Your Database
|
|
|
|
If you have an **existing database** and want to add P2P sync:
|
|
|
|
### Step 1 - Wrap your context
|
|
|
|
Create a `DbContext` extending `CBDDCDocumentDbContext`. This can wrap your existing collections/tables.
|
|
|
|
```csharp
|
|
public class MyExistingDbContext : CBDDCDocumentDbContext
|
|
{
|
|
// Your existing collections
|
|
public DocumentCollection<string, Product> Products { get; private set; }
|
|
public DocumentCollection<string, Inventory> Inventory { get; private set; }
|
|
|
|
public MyExistingDbContext(string dbPath) : base(dbPath) { }
|
|
}
|
|
```
|
|
|
|
### Step 2 - Create a DocumentStore
|
|
|
|
Extend `BLiteDocumentStore<T>`. This is the **bridge** between your data model and the sync engine.
|
|
|
|
```csharp
|
|
public class MyDocumentStore : BLiteDocumentStore<MyExistingDbContext>
|
|
{
|
|
public MyDocumentStore(MyExistingDbContext ctx,
|
|
IPeerNodeConfigurationProvider cfg,
|
|
IVectorClockService vc,
|
|
ILogger<MyDocumentStore>? log = null)
|
|
: base(ctx, cfg, vc, logger: log)
|
|
{
|
|
// Continue to next step...
|
|
}
|
|
|
|
// Implement abstract methods (see below)...
|
|
}
|
|
```
|
|
|
|
### Step 3 - Register only what you need
|
|
|
|
Call `WatchCollection()` in the constructor for each collection you want to replicate. Everything else is ignored by the sync engine.
|
|
|
|
```csharp
|
|
public MyDocumentStore(...)
|
|
: base(ctx, cfg, vc, logger: log)
|
|
{
|
|
// Only these 2 collections will be synced across the mesh
|
|
WatchCollection("Products", ctx.Products, p => p.Id);
|
|
WatchCollection("Inventory", ctx.Inventory, i => i.Id);
|
|
|
|
// All other collections in your DB are local-only
|
|
}
|
|
```
|
|
|
|
### Step 4 - Implement the mapping methods
|
|
|
|
CBDDC stores data as `JsonElement`. You provide four mapping methods:
|
|
|
|
| Method | Purpose |
|
|
|--------|---------|
|
|
| `ApplyContentToEntityAsync` | Write incoming sync data to your entities |
|
|
| `GetEntityAsJsonAsync` | Read your entities for outbound sync |
|
|
| `RemoveEntityAsync` | Handle remote deletes |
|
|
| `GetAllEntitiesAsJsonAsync` | Provide full collection for snapshot sync |
|
|
|
|
```csharp
|
|
protected override async Task ApplyContentToEntityAsync(
|
|
string collection, string key, JsonElement content, CancellationToken ct)
|
|
{
|
|
switch (collection)
|
|
{
|
|
case "Products":
|
|
var product = content.Deserialize<Product>()!;
|
|
product.Id = key;
|
|
var existing = _context.Products.Find(p => p.Id == key).FirstOrDefault();
|
|
if (existing != null) _context.Products.Update(product);
|
|
else _context.Products.Insert(product);
|
|
break;
|
|
case "Inventory":
|
|
var inv = content.Deserialize<Inventory>()!;
|
|
inv.Id = key;
|
|
var existingInv = _context.Inventory.Find(i => i.Id == key).FirstOrDefault();
|
|
if (existingInv != null) _context.Inventory.Update(inv);
|
|
else _context.Inventory.Insert(inv);
|
|
break;
|
|
}
|
|
await _context.SaveChangesAsync(ct);
|
|
}
|
|
|
|
protected override Task<JsonElement?> GetEntityAsJsonAsync(
|
|
string collection, string key, CancellationToken ct)
|
|
{
|
|
object? entity = collection switch
|
|
{
|
|
"Products" => _context.Products.Find(p => p.Id == key).FirstOrDefault(),
|
|
"Inventory" => _context.Inventory.Find(i => i.Id == key).FirstOrDefault(),
|
|
_ => null
|
|
};
|
|
return Task.FromResult(entity != null
|
|
? (JsonElement?)JsonSerializer.SerializeToElement(entity) : null);
|
|
}
|
|
|
|
protected override async Task RemoveEntityAsync(
|
|
string collection, string key, CancellationToken ct)
|
|
{
|
|
switch (collection)
|
|
{
|
|
case "Products": _context.Products.Delete(key); break;
|
|
case "Inventory": _context.Inventory.Delete(key); break;
|
|
}
|
|
await _context.SaveChangesAsync(ct);
|
|
}
|
|
|
|
protected override Task<IEnumerable<(string Key, JsonElement Content)>>
|
|
GetAllEntitiesAsJsonAsync(string collection, CancellationToken ct)
|
|
{
|
|
IEnumerable<(string, JsonElement)> result = collection switch
|
|
{
|
|
"Products" => _context.Products.FindAll()
|
|
.Select(p => (p.Id, JsonSerializer.SerializeToElement(p))),
|
|
"Inventory" => _context.Inventory.FindAll()
|
|
.Select(i => (i.Id, JsonSerializer.SerializeToElement(i))),
|
|
_ => Enumerable.Empty<(string, JsonElement)>()
|
|
};
|
|
return Task.FromResult(result);
|
|
}
|
|
|
|
// Optional: Batch operations for better performance
|
|
protected override async Task ApplyContentToEntitiesBatchAsync(
|
|
IEnumerable<(string Collection, string Key, JsonElement Content)> documents,
|
|
CancellationToken ct)
|
|
{
|
|
foreach (var (collection, key, content) in documents)
|
|
{
|
|
// Call the single-item method (you can optimize this further)
|
|
await ApplyContentToEntityAsync(collection, key, content, ct);
|
|
}
|
|
}
|
|
```
|
|
|
|
**Your existing CRUD code stays unchanged.** CBDDC plugs in alongside it.
|
|
|
|
### What Happens Under the Hood
|
|
|
|
```
|
|
Your Code: db.Users.InsertAsync(user)
|
|
|
|
|
v
|
|
BLite: SaveChangesAsync()
|
|
|
|
|
| CDC fires (WatchCollection observer)
|
|
DocumentStore: CreateOplogEntryAsync()
|
|
|
|
|
+-> OplogEntry written (hash-chained, HLC timestamped)
|
|
+-> VectorClockService.Update() -> sync sees it immediately
|
|
|
|
|
v
|
|
SyncOrchestrator (background, every 2s)
|
|
+-> Compare VectorClocks with peers
|
|
+-> Push local changes (interest-filtered)
|
|
+-> Pull remote changes -> ApplyBatchAsync
|
|
|
|
|
v
|
|
Remote DocumentStore: ApplyContentToEntityAsync()
|
|
|
|
|
v
|
|
Remote Database: Updated!
|
|
```
|
|
|
|
---
|
|
|
|
## Cloud Deployment
|
|
|
|
CBDDC supports ASP.NET Core hosting with BLite persistence for cloud deployments.
|
|
|
|
### Example: ASP.NET Core with BLite
|
|
|
|
```csharp
|
|
var builder = WebApplication.CreateBuilder(args);
|
|
|
|
builder.Services.AddCBDDCCore()
|
|
.AddCBDDCBLite<MyDbContext, MyDocumentStore>(sp => new MyDbContext("cbddc.blite"))
|
|
.AddCBDDCNetwork<MyPeerConfigProvider>();
|
|
|
|
builder.Services.AddCBDDCHostingSingleCluster(options =>
|
|
{
|
|
options.TcpPort = 5001;
|
|
});
|
|
|
|
var app = builder.Build();
|
|
app.MapHealthChecks("/health");
|
|
await app.RunAsync();
|
|
```
|
|
|
|
---
|
|
|
|
## Production Features
|
|
|
|
### Configuration
|
|
|
|
```json
|
|
{
|
|
"CBDDC": {
|
|
"KnownPeers": [
|
|
{
|
|
"NodeId": "gateway-1",
|
|
"Address": "192.168.1.10:5000",
|
|
"Type": "StaticRemote"
|
|
}
|
|
],
|
|
"RetentionHours": 24,
|
|
"SyncIntervalSeconds": 2
|
|
},
|
|
"Logging": {
|
|
"LogLevel": {
|
|
"Default": "Information",
|
|
"CBDDC": "Warning"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Health Monitoring
|
|
|
|
```csharp
|
|
var healthCheck = new CBDDCHealthCheck(store, syncTracker);
|
|
var status = await healthCheck.CheckAsync();
|
|
|
|
Console.WriteLine($"Database: {status.DatabaseHealthy}");
|
|
Console.WriteLine($"Network: {status.NetworkHealthy}");
|
|
Console.WriteLine($"Peers: {status.ConnectedPeers}");
|
|
```
|
|
|
|
### Resilience
|
|
|
|
- **Exponential Backoff**: Automatic retry for unreachable peers
|
|
- **Offline Queue**: Buffer local changes when network is down
|
|
- **Snapshot Recovery**: Fast catch-up after long disconnects
|
|
- **Hash Chain Validation**: Detect and recover from oplog gaps
|
|
|
|
### Performance
|
|
|
|
- **VectorClock Cache**: In-memory tracking of node states
|
|
- **Brotli Compression**: 70-80% bandwidth reduction
|
|
- **Batch Operations**: Group changes for efficient network transfer
|
|
- **Interest Filtering**: Only sync collections both peers care about
|
|
|
|
### Security
|
|
|
|
- **Noise Protocol Handshake**: XX pattern with ECDH key exchange
|
|
- **AES-256 Encryption**: Protect data in transit
|
|
- **Auth Tokens**: Shared secret validation
|
|
- **LAN Isolation**: Designed for trusted network environments
|
|
|
|
---
|
|
|
|
## Use Cases
|
|
|
|
### Ideal For
|
|
|
|
- **Retail POS Systems** - Terminals syncing inventory and sales across a store
|
|
- **Office Applications** - Shared task lists, calendars, CRM data on LAN
|
|
- **Edge Computing** - Distributed sensors and controllers at a facility
|
|
- **Offline-First Apps** - Work without internet, sync when connected
|
|
- **Multi-Site Replication** - Keep regional databases in sync (over VPN)
|
|
- **Existing Database Modernization** - Add P2P sync without rewriting your app
|
|
|
|
### Not Designed For
|
|
|
|
- **Public internet without HTTPS/VPN** (P2P mesh mode, use ASP.NET Core mode instead)
|
|
- **Sub-millisecond consistency requirements** (eventual consistency model, typical convergence < 5s)
|
|
- **Unstructured data** (designed for document collections with keys)
|
|
- **Append-only event logs** (oplog pruning after 24h retention)
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### Getting Started
|
|
|
|
- **[Sample Application](samples/ZB.MOM.WW.CBDDC.Sample.Console/)** - Complete two-node sync example with interactive CLI
|
|
- **[Quick Start Guide](#quick-start)** - 5-minute setup
|
|
- **[Integration Guide](#integrating-with-your-database)** - Add sync to existing DB
|
|
|
|
### Concepts
|
|
|
|
- **[Architecture & Concepts](docs/architecture.md)** - HLC, Gossip, Vector Clocks, Hash Chains
|
|
- **[Conflict Resolution](docs/conflict-resolution.md)** - LWW vs Recursive Merge
|
|
- **[Oplog & CDC Design](docs/database-sync-manager-design.md)** - How change tracking works
|
|
|
|
### Deployment
|
|
|
|
- **[Production Guide](docs/production-hardening.md)** - Configuration, monitoring, best practices
|
|
- **[Deployment Runbook](docs/deployment.md)** - Promotion flow, validation, rollback
|
|
- **[Deployment Modes](docs/deployment-modes.md)** - Single-cluster deployment strategy
|
|
- **[Operations Runbook](docs/runbook.md)** - Incident triage and recovery
|
|
|
|
### API
|
|
|
|
- **[API Reference](docs/api-reference.md)** - Complete API documentation
|
|
- **[Persistence Providers](docs/persistence-providers.md)** - BLite, custom
|
|
|
|
---
|
|
|
|
## Examples
|
|
|
|
See [`samples/ZB.MOM.WW.CBDDC.Sample.Console/`](samples/ZB.MOM.WW.CBDDC.Sample.Console/) for a complete working example with:
|
|
|
|
- **Two-node sync** simulation on different ports
|
|
- **Interactive CLI** for testing operations
|
|
- **Conflict resolution demo** (switchable LWW/Merge)
|
|
- **User and TodoList** entities with full CRUD
|
|
- **Health monitoring** and cache inspection
|
|
- **Automatic peer discovery** via UDP
|
|
|
|
Run two instances:
|
|
```bash
|
|
# Terminal 1
|
|
cd samples/ZB.MOM.WW.CBDDC.Sample.Console
|
|
dotnet run -- node-1 8580
|
|
|
|
# Terminal 2
|
|
dotnet run -- node-2 8581
|
|
|
|
# Create a user on node-1 with command "n"
|
|
# Watch it appear on node-2 automatically!
|
|
```
|
|
|
|
---
|
|
|
|
## Roadmap
|
|
|
|
- [x] Core P2P mesh networking (v0.1.0)
|
|
- [x] Secure networking (ECDH + AES-256) (v0.6.0)
|
|
- [x] Conflict resolution strategies (LWW, Recursive Merge) (v0.6.0)
|
|
- [x] Hash-chain sync with gap recovery (v0.7.0)
|
|
- [x] Brotli compression (v0.7.0)
|
|
- [x] Persistence snapshots (v0.8.6)
|
|
- [x] ASP.NET Core hosting (v0.8.0)
|
|
- [x] **VectorClockService refactor** (v1.0.0)
|
|
- [x] **CDC-aware sync** (v1.0.0)
|
|
- [ ] Query optimization & advanced indexing
|
|
- [ ] Admin UI / monitoring dashboard
|
|
- [ ] Mobile support (Xamarin/MAUI)
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
We welcome contributions! CBDDC is open-source and we'd love your help.
|
|
|
|
### How to Contribute
|
|
|
|
1. **Fork the repository**
|
|
2. **Create a feature branch** (`git checkout -b feature/amazing-feature`)
|
|
3. **Make your changes** with clear commit messages
|
|
4. **Add tests** for new functionality
|
|
5. **Ensure all tests pass** (`dotnet test`)
|
|
6. **Submit a Pull Request**
|
|
|
|
### Development Setup
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net.git
|
|
cd CBDDC.Net
|
|
|
|
# Restore dependencies
|
|
dotnet restore
|
|
|
|
# Build
|
|
dotnet build
|
|
|
|
# Run all tests (69 tests)
|
|
dotnet test
|
|
|
|
# Run sample
|
|
cd samples/ZB.MOM.WW.CBDDC.Sample.Console
|
|
dotnet run
|
|
```
|
|
|
|
### Areas We Need Help
|
|
|
|
- **[Bug] Bug Reports** - Found an issue? Let us know!
|
|
- **[Docs] Documentation** - Improve guides and examples
|
|
- **[Feature] Features** - Implement items from the roadmap
|
|
- **[Test] Testing** - Add integration and performance tests
|
|
- **[Sample] Samples** - Build example applications
|
|
|
|
### Code of Conduct
|
|
|
|
Be respectful, inclusive, and constructive. We're all here to learn and build great software together.
|