Initial import of the CBDDC codebase with docs and tests. Add a .NET-focused gitignore to keep generated artifacts out of source control.
Some checks failed
CI / verify (push) Has been cancelled
Some checks failed
CI / verify (push) Has been cancelled
This commit is contained in:
Joseph Doherty
73
docs/README.md
Executable file
73
docs/README.md
Executable file
@@ -0,0 +1,73 @@
|
||||
# CBDDC Documentation
|
||||
|
||||
This folder contains the official documentation for CBDDC, published as GitHub Pages.
|
||||
|
||||
## Documentation Structure (v0.9.0)
|
||||
|
||||
### Getting Started
|
||||
- **[Getting Started](getting-started.md)** - Installation, setup, and first steps with CBDDC
|
||||
|
||||
### Core Documentation
|
||||
- **[Architecture](architecture.md)** - Hybrid Logical Clocks, Gossip Protocol, mesh networking
|
||||
- **[API Reference](api-reference.md)** - Complete API documentation with examples
|
||||
- **[Querying](querying.md)** - Data querying patterns and LINQ support
|
||||
|
||||
### Persistence & Storage
|
||||
- **[Persistence Providers](persistence-providers.md)** - SQLite, EF Core, PostgreSQL comparison
|
||||
- **[Deployment Modes](deployment-modes.md)** - Single-cluster deployment strategy
|
||||
|
||||
### Networking & Security
|
||||
- **[Security](security.md)** - Encryption, authentication, secure networking
|
||||
- **[Conflict Resolution](conflict-resolution.md)** - LWW and Recursive Merge strategies
|
||||
- **[Network Telemetry](network-telemetry.md)** - Monitoring and diagnostics
|
||||
- **[Dynamic Reconfiguration](dynamic-reconfiguration.md)** - Runtime configuration changes
|
||||
- **[Remote Peer Configuration](remote-peer-configuration.md)** - Managing remote peers and tracking lifecycle
|
||||
- **[Upgrade: Peer-Confirmed Pruning](upgrade-peer-confirmed-pruning.md)** - Rollout notes and adoption checklist
|
||||
|
||||
### Deployment & Operations
|
||||
- **[Deployment (LAN)](deployment-lan.md)** - Platform-specific deployment guide
|
||||
- **[Production Hardening](production-hardening.md)** - Configuration, monitoring, best practices
|
||||
- **[Peer Deprecation & Removal Runbook](peer-deprecation-removal-runbook.md)** - Operational workflow for de-tracking and removal
|
||||
|
||||
## Building the Documentation
|
||||
|
||||
This documentation uses Jekyll with the Cayman theme and is automatically published via GitHub Pages.
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Install Jekyll (requires Ruby)
|
||||
gem install bundler jekyll
|
||||
|
||||
# Serve documentation locally
|
||||
cd docs
|
||||
jekyll serve
|
||||
|
||||
# Open http://localhost:4000
|
||||
```
|
||||
|
||||
### Site Configuration
|
||||
|
||||
- **_config.yml** - Jekyll configuration and site metadata
|
||||
- **_layouts/default.html** - Main page layout with navigation and styling
|
||||
- **_includes/nav.html** - Top navigation bar
|
||||
- **_data/navigation.yml** - Sidebar navigation structure per version
|
||||
|
||||
## Version History
|
||||
|
||||
The documentation supports multiple versions:
|
||||
- **v0.9** (current) - Latest stable release
|
||||
- **v0.8** - ASP.NET Core hosting, EF Core, PostgreSQL support
|
||||
- **v0.7** - Brotli compression, Protocol v4
|
||||
- **v0.6** - Secure networking, conflict resolution strategies
|
||||
|
||||
## Contributing to Documentation
|
||||
|
||||
1. **Fix typos or improve clarity** - Submit PRs directly
|
||||
2. **Add new guides** - Create a new .md file and update navigation.yml
|
||||
3. **Test locally** - Run Jekyll locally to preview changes before submitting
|
||||
|
||||
## Links
|
||||
|
||||
- [Main Repository](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net)
|
||||
- [NuGet Packages](https://www.nuget.org/packages?q=CBDDC)
|
||||
5
docs/_config.yml
Executable file
5
docs/_config.yml
Executable file
@@ -0,0 +1,5 @@
|
||||
title: "CBDDC Documentation"
|
||||
description: A decentralized, offline-first peer-to-peer database.
|
||||
theme: jekyll-theme-cayman
|
||||
plugins:
|
||||
- jekyll-seo-tag
|
||||
123
docs/_data/navigation.yml
Executable file
123
docs/_data/navigation.yml
Executable file
@@ -0,0 +1,123 @@
|
||||
v0.9:
|
||||
- title: Getting Started
|
||||
url: /getting-started.html
|
||||
- title: Architecture
|
||||
url: /architecture.html
|
||||
- title: Persistence Providers
|
||||
url: /persistence-providers.html
|
||||
- title: Deployment Modes
|
||||
url: /deployment-modes.html
|
||||
- title: Deployment (LAN)
|
||||
url: /deployment-lan.html
|
||||
- title: Security
|
||||
url: /security.html
|
||||
- title: Conflict Resolution
|
||||
url: /conflict-resolution.html
|
||||
- title: Querying
|
||||
url: /querying.html
|
||||
- title: Network Telemetry
|
||||
url: /network-telemetry.html
|
||||
- title: Dynamic Reconfiguration
|
||||
url: /dynamic-reconfiguration.html
|
||||
- title: Remote Peer Configuration
|
||||
url: /remote-peer-configuration.html
|
||||
- title: Upgrade: Peer-Confirmed Pruning
|
||||
url: /upgrade-peer-confirmed-pruning.html
|
||||
- title: Peer Deprecation & Removal Runbook
|
||||
url: /peer-deprecation-removal-runbook.html
|
||||
- title: Production Hardening
|
||||
url: /production-hardening.html
|
||||
- title: API Reference
|
||||
url: /api-reference.html
|
||||
|
||||
v0.8:
|
||||
- title: Getting Started
|
||||
url: /v0.8/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.8/architecture.html
|
||||
- title: Persistence Providers
|
||||
url: /v0.8/persistence-providers.html
|
||||
- title: Deployment Modes
|
||||
url: /v0.8/deployment-modes.html
|
||||
- title: Security
|
||||
url: /v0.8/security.html
|
||||
- title: Conflict Resolution
|
||||
url: /v0.8/conflict-resolution.html
|
||||
- title: Querying
|
||||
url: /v0.8/querying.html
|
||||
- title: Production Hardening
|
||||
url: /v0.8/production-hardening.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.8/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.8/api-reference.html
|
||||
|
||||
v0.7:
|
||||
- title: Getting Started
|
||||
url: /v0.7/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.7/architecture.html
|
||||
- title: Security
|
||||
url: /v0.7/security.html
|
||||
- title: Conflict Resolution
|
||||
url: /v0.7/conflict-resolution.html
|
||||
- title: Querying
|
||||
url: /v0.7/querying.html
|
||||
- title: Production Hardening
|
||||
url: /v0.7/production-hardening.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.7/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.7/api-reference.html
|
||||
|
||||
v0.6:
|
||||
- title: Getting Started
|
||||
url: /v0.6/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.6/architecture.html
|
||||
- title: Security
|
||||
url: /v0.6/security.html
|
||||
- title: Conflict Resolution
|
||||
url: /v0.6/conflict-resolution.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.6/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.6/api-reference.html
|
||||
- title: Production Hardening
|
||||
url: /v0.6/production-hardening.html
|
||||
|
||||
v0.5:
|
||||
- title: Getting Started
|
||||
url: /v0.5/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.5/architecture.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.5/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.5/api-reference.html
|
||||
- title: Production Hardening
|
||||
url: /v0.5/production-hardening.html
|
||||
|
||||
v0.4:
|
||||
- title: Getting Started
|
||||
url: /v0.4/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.4/architecture.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.4/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.4/api-reference.html
|
||||
- title: Production Hardening
|
||||
url: /v0.4/production-hardening.html
|
||||
|
||||
v0.2:
|
||||
- title: Getting Started
|
||||
url: /v0.2/getting-started.html
|
||||
- title: Architecture
|
||||
url: /v0.2/architecture.html
|
||||
- title: Deployment (LAN)
|
||||
url: /v0.2/deployment-lan.html
|
||||
- title: API Reference
|
||||
url: /v0.2/api-reference.html
|
||||
- title: Production Hardening
|
||||
url: /v0.2/production-hardening.html
|
||||
5
docs/_includes/nav.html
Executable file
5
docs/_includes/nav.html
Executable file
@@ -0,0 +1,5 @@
|
||||
<nav class="site-nav">
|
||||
<a href="{{ '/' | relative_url }}">Home</a>
|
||||
<a href="{{ '/getting-started.html' | relative_url }}">Docs (v0.9)</a>
|
||||
<a href="https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net" target="_blank">GitHub</a>
|
||||
</nav>
|
||||
390
docs/_layouts/default.html
Executable file
390
docs/_layouts/default.html
Executable file
@@ -0,0 +1,390 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="{{ site.lang | default: " en-US" }}">
|
||||
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="description" content="{{ site.description }}">
|
||||
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
|
||||
<title>{{ page.title | default: site.title }}</title>
|
||||
<style>
|
||||
/* Modern Color Palette */
|
||||
:root {
|
||||
--primary-color: #159957;
|
||||
--primary-dark: #0e7042;
|
||||
--primary-light: #1eb96a;
|
||||
--secondary-color: #155799;
|
||||
--background: #ffffff;
|
||||
--background-alt: #f8f9fa;
|
||||
--text-primary: #24292e;
|
||||
--text-secondary: #586069;
|
||||
--border-color: #e1e4e8;
|
||||
--shadow: 0 1px 3px rgba(0,0,0,0.12), 0 1px 2px rgba(0,0,0,0.24);
|
||||
--shadow-hover: 0 3px 6px rgba(0,0,0,0.15), 0 2px 4px rgba(0,0,0,0.12);
|
||||
}
|
||||
|
||||
/* Base Styles */
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
|
||||
color: var(--text-primary);
|
||||
line-height: 1.6;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/* Top Navigation */
|
||||
.site-nav {
|
||||
background: linear-gradient(135deg, var(--primary-color), var(--primary-light));
|
||||
padding: 1rem 0;
|
||||
text-align: center;
|
||||
box-shadow: var(--shadow);
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 1000;
|
||||
}
|
||||
|
||||
.site-nav a {
|
||||
color: rgba(255, 255, 255, 0.9);
|
||||
margin: 0 1.5rem;
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
font-size: 0.95rem;
|
||||
letter-spacing: 0.5px;
|
||||
transition: all 0.3s ease;
|
||||
padding: 0.5rem 1rem;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.site-nav a:hover,
|
||||
.site-nav a.active {
|
||||
color: white;
|
||||
background: rgba(255, 255, 255, 0.15);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
/* Header Section */
|
||||
.page-header {
|
||||
background: linear-gradient(135deg, var(--primary-color), var(--secondary-color));
|
||||
color: white;
|
||||
text-align: center;
|
||||
padding: 3rem 1rem;
|
||||
}
|
||||
|
||||
.project-name {
|
||||
margin: 0 0 0.5rem 0;
|
||||
font-size: 2.5rem;
|
||||
font-weight: 700;
|
||||
letter-spacing: -0.5px;
|
||||
}
|
||||
|
||||
.project-tagline {
|
||||
margin: 0 0 1.5rem 0;
|
||||
font-size: 1.2rem;
|
||||
font-weight: 400;
|
||||
opacity: 0.9;
|
||||
}
|
||||
|
||||
.btn {
|
||||
display: inline-block;
|
||||
margin: 0.5rem;
|
||||
padding: 0.75rem 2rem;
|
||||
background: rgba(255, 255, 255, 0.2);
|
||||
color: white;
|
||||
border: 2px solid rgba(255, 255, 255, 0.8);
|
||||
border-radius: 6px;
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
transition: all 0.3s ease;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
.btn:hover {
|
||||
background: white;
|
||||
color: var(--primary-color);
|
||||
transform: translateY(-2px);
|
||||
box-shadow: var(--shadow-hover);
|
||||
}
|
||||
|
||||
/* Layout Grid */
|
||||
.page-content {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
max-width: 1200px;
|
||||
margin: 2rem auto;
|
||||
padding: 0 1rem;
|
||||
gap: 2rem;
|
||||
}
|
||||
|
||||
@media (min-width: 64em) {
|
||||
.page-content {
|
||||
flex-direction: row;
|
||||
align-items: flex-start;
|
||||
padding: 0 2rem;
|
||||
}
|
||||
}
|
||||
|
||||
/* Sidebar */
|
||||
.sidebar {
|
||||
width: 100%;
|
||||
background: var(--background-alt);
|
||||
padding: 1.5rem;
|
||||
border-radius: 8px;
|
||||
border: 1px solid var(--border-color);
|
||||
box-shadow: var(--shadow);
|
||||
}
|
||||
|
||||
@media (min-width: 64em) {
|
||||
.sidebar {
|
||||
width: 280px;
|
||||
flex-shrink: 0;
|
||||
position: sticky;
|
||||
top: calc(4rem + 1rem);
|
||||
max-height: calc(100vh - 6rem);
|
||||
overflow-y: auto;
|
||||
}
|
||||
}
|
||||
|
||||
.sidebar h3 {
|
||||
margin: 0 0 1rem 0;
|
||||
font-size: 1.1rem;
|
||||
color: var(--text-primary);
|
||||
font-weight: 700;
|
||||
padding-bottom: 0.75rem;
|
||||
border-bottom: 2px solid var(--primary-color);
|
||||
}
|
||||
|
||||
.sidebar ul {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.sidebar li {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.sidebar a {
|
||||
color: var(--text-secondary);
|
||||
text-decoration: none;
|
||||
display: block;
|
||||
padding: 0.5rem 0.75rem;
|
||||
border-radius: 4px;
|
||||
transition: all 0.2s ease;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
.sidebar a:hover {
|
||||
color: var(--primary-color);
|
||||
background: rgba(21, 153, 87, 0.08);
|
||||
padding-left: 1rem;
|
||||
}
|
||||
|
||||
/* Main Content */
|
||||
.main-content-wrapper {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.main-content {
|
||||
background: white;
|
||||
padding: 2rem;
|
||||
border-radius: 8px;
|
||||
box-shadow: var(--shadow);
|
||||
}
|
||||
|
||||
.main-content h1 {
|
||||
color: var(--text-primary);
|
||||
font-size: 2rem;
|
||||
margin-top: 0;
|
||||
margin-bottom: 1.5rem;
|
||||
padding-bottom: 0.75rem;
|
||||
border-bottom: 3px solid var(--primary-color);
|
||||
}
|
||||
|
||||
.main-content h2 {
|
||||
color: var(--text-primary);
|
||||
font-size: 1.5rem;
|
||||
margin-top: 2rem;
|
||||
margin-bottom: 1rem;
|
||||
padding-bottom: 0.5rem;
|
||||
border-bottom: 2px solid var(--border-color);
|
||||
}
|
||||
|
||||
.main-content h3 {
|
||||
color: var(--text-primary);
|
||||
font-size: 1.25rem;
|
||||
margin-top: 1.5rem;
|
||||
margin-bottom: 0.75rem;
|
||||
}
|
||||
|
||||
.main-content code {
|
||||
background: var(--background-alt);
|
||||
padding: 0.2rem 0.4rem;
|
||||
border-radius: 3px;
|
||||
font-size: 0.9em;
|
||||
color: var(--secondary-color);
|
||||
}
|
||||
|
||||
.main-content pre {
|
||||
background: var(--background-alt);
|
||||
padding: 1rem;
|
||||
border-radius: 6px;
|
||||
overflow-x: auto;
|
||||
border: 1px solid var(--border-color);
|
||||
}
|
||||
|
||||
.main-content pre code {
|
||||
background: none;
|
||||
padding: 0;
|
||||
color: var(--text-primary);
|
||||
}
|
||||
|
||||
.main-content a {
|
||||
color: var(--primary-color);
|
||||
text-decoration: underline;
|
||||
text-decoration-color: rgba(21, 153, 87, 0.3);
|
||||
text-decoration-thickness: 1px;
|
||||
text-underline-offset: 2px;
|
||||
transition: all 0.2s ease;
|
||||
}
|
||||
|
||||
.main-content a:hover {
|
||||
color: var(--primary-dark);
|
||||
text-decoration-color: var(--primary-dark);
|
||||
text-decoration-thickness: 2px;
|
||||
}
|
||||
|
||||
.main-content table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin: 1rem 0;
|
||||
}
|
||||
|
||||
.main-content th,
|
||||
.main-content td {
|
||||
padding: 0.75rem;
|
||||
border: 1px solid var(--border-color);
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.main-content th {
|
||||
background: var(--background-alt);
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.main-content blockquote {
|
||||
margin: 1rem 0;
|
||||
padding: 1rem;
|
||||
border-left: 4px solid var(--primary-color);
|
||||
background: var(--background-alt);
|
||||
border-radius: 0 4px 4px 0;
|
||||
}
|
||||
|
||||
/* Footer */
|
||||
.site-footer {
|
||||
text-align: center;
|
||||
padding: 2rem;
|
||||
border-top: 1px solid var(--border-color);
|
||||
margin-top: 3rem;
|
||||
background: var(--background-alt);
|
||||
color: var(--text-secondary);
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.site-footer a {
|
||||
color: var(--primary-color);
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.site-footer a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Mobile Responsiveness */
|
||||
@media (max-width: 64em) {
|
||||
.project-name {
|
||||
font-size: 2rem;
|
||||
}
|
||||
|
||||
.page-header {
|
||||
padding: 2rem 1rem;
|
||||
}
|
||||
|
||||
.main-content {
|
||||
padding: 1.5rem;
|
||||
}
|
||||
|
||||
.sidebar {
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
}
|
||||
|
||||
/* Scrollbar Styling */
|
||||
.sidebar::-webkit-scrollbar {
|
||||
width: 8px;
|
||||
}
|
||||
|
||||
.sidebar::-webkit-scrollbar-track {
|
||||
background: var(--background-alt);
|
||||
}
|
||||
|
||||
.sidebar::-webkit-scrollbar-thumb {
|
||||
background: var(--border-color);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.sidebar::-webkit-scrollbar-thumb:hover {
|
||||
background: var(--text-secondary);
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
|
||||
<body>
|
||||
{% include nav.html %}
|
||||
|
||||
<section class="page-header">
|
||||
<h1 class="project-name">{{ site.title }}</h1>
|
||||
<h2 class="project-tagline">{{ site.description }}</h2>
|
||||
<a href="https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net" class="btn">View on GitHub</a>
|
||||
<a href="https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/releases" class="btn">Download</a>
|
||||
</section>
|
||||
|
||||
<div class="page-content">
|
||||
<!-- Sidebar Logic -->
|
||||
{% assign version = page.url | split: '/' %}
|
||||
{% assign current_version = version[1] %}
|
||||
|
||||
<!-- If current_version is empty or doesn't start with 'v', treat as v0.9 (root level) -->
|
||||
{% if current_version == '' or current_version contains '.html' or current_version contains '.md' %}
|
||||
{% assign current_version = 'v0.9' %}
|
||||
{% endif %}
|
||||
|
||||
{% if current_version == 'v0.9' or current_version == 'v0.8' or current_version == 'v0.7' or current_version == 'v0.6' or current_version == 'v0.5' or current_version ==
|
||||
'v0.4' or current_version ==
|
||||
'v0.2' %}
|
||||
<nav class="sidebar">
|
||||
<h3>Docs {{ current_version }}</h3>
|
||||
<ul>
|
||||
{% for item in site.data.navigation[current_version] %}
|
||||
<li><a href="{{ item.url | relative_url }}">{{ item.title }}</a></li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
</nav>
|
||||
{% endif %}
|
||||
|
||||
<div class="main-content-wrapper">
|
||||
<section class="main-content">
|
||||
{{ content }}
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<footer class="site-footer">
|
||||
<span class="site-footer-owner"><a href="https://cbddc.com">CBDDC</a> is maintained by <a
|
||||
href="https://github.com/mrdevrobot">MrDevRobot</a>.</span>
|
||||
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub
|
||||
Pages</a>.</span>
|
||||
</footer>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
142
docs/api-reference.md
Executable file
142
docs/api-reference.md
Executable file
@@ -0,0 +1,142 @@
|
||||
# API Reference
|
||||
|
||||
## PeerDatabase
|
||||
|
||||
The entry point for interacting with the database.
|
||||
|
||||
```csharp
|
||||
public class PeerDatabase : IPeerDatabase
|
||||
{
|
||||
public PeerDatabase(IPeerStore store, string nodeId = "local");
|
||||
|
||||
// Non-generic collection access
|
||||
public IPeerCollection Collection(string name);
|
||||
|
||||
// Generic collection access (type-safe)
|
||||
public IPeerCollection<T> Collection<T>(string? customName = null);
|
||||
}
|
||||
```
|
||||
|
||||
### Collection Naming Convention
|
||||
|
||||
When using `Collection<T>()`, the collection name defaults to `typeof(T).Name.ToLowerInvariant()`. You can change this globally using `CBDDCMapper`.
|
||||
|
||||
```csharp
|
||||
// Uses default collection name "user"
|
||||
var users = db.Collection<User>();
|
||||
|
||||
// Custom name override
|
||||
var users = db.Collection<User>("custom_users");
|
||||
```
|
||||
|
||||
## IPeerCollection<T>
|
||||
|
||||
Represents a collection of documents (like a Table or Container).
|
||||
|
||||
```csharp
|
||||
public interface IPeerCollection<T> : IPeerCollection
|
||||
{
|
||||
// Single Operations
|
||||
Task Put(string key, T document, CancellationToken cancellationToken = default);
|
||||
Task<T> Get(string key, CancellationToken cancellationToken = default);
|
||||
Task Delete(string key, CancellationToken cancellationToken = default);
|
||||
|
||||
// Batch Operations
|
||||
Task PutMany(IEnumerable<T> documents, CancellationToken cancellationToken = default);
|
||||
Task DeleteMany(IEnumerable<string> keys, CancellationToken cancellationToken = default);
|
||||
|
||||
// Queries
|
||||
Task<IEnumerable<T>> Find(Expression<Func<T, bool>> predicate, CancellationToken cancellationToken = default);
|
||||
Task<int> Count(Expression<Func<T, bool>>? predicate = null, CancellationToken cancellationToken = default);
|
||||
}
|
||||
```
|
||||
|
||||
### Batch Operations
|
||||
|
||||
For bulk inserts or deletes, use `PutMany` and `DeleteMany` for better performance and atomicity.
|
||||
|
||||
```csharp
|
||||
var users = new List<User> { ... };
|
||||
await db.Collection<User>().PutMany(users);
|
||||
```
|
||||
|
||||
### Count
|
||||
|
||||
Get the total number of documents matching a filter efficiently (database-side).
|
||||
|
||||
```csharp
|
||||
int total = await db.Collection<User>().Count();
|
||||
int active = await db.Collection<User>().Count(u => u.IsActive);
|
||||
```
|
||||
|
||||
## Global Configuration (CBDDCMapper)
|
||||
|
||||
You can configure entity mappings globally instead of using attributes.
|
||||
|
||||
```csharp
|
||||
CBDDCMapper.Global.Entity<Product>()
|
||||
.Collection("products_v2")
|
||||
.Index(p => p.Price)
|
||||
.Index(p => p.Category);
|
||||
```
|
||||
|
||||
## Primary Keys and Indexes
|
||||
|
||||
### Attributes
|
||||
|
||||
```csharp
|
||||
using ZB.MOM.WW.CBDDC.Core.Metadata;
|
||||
|
||||
public class User
|
||||
{
|
||||
[PrimaryKey(AutoGenerate = true)]
|
||||
public string Id { get; set; } = "";
|
||||
|
||||
[Indexed]
|
||||
public int Age { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
### Indexed Fields
|
||||
|
||||
Indexes improve query performance (e.g., `Find(u => u.Age > 18)`).
|
||||
You can define them via `[Indexed]` attribute or `CBDDCMapper`.
|
||||
|
||||
> **Note**: CBDDC automatically creates SQLite indexes for these properties.
|
||||
|
||||
## Exceptions
|
||||
|
||||
CBDDC throws specific exceptions for better error handling:
|
||||
|
||||
- `CBDDCException` (Base class)
|
||||
- `DocumentNotFoundException`
|
||||
- `CBDDCConcurrencyException`
|
||||
- `TimeoutException`
|
||||
- `NetworkException`
|
||||
- `PersistenceException`
|
||||
|
||||
```csharp
|
||||
try
|
||||
{
|
||||
await db.Collection<User>().Get("invalid-id");
|
||||
}
|
||||
catch (DocumentNotFoundException ex)
|
||||
{
|
||||
Console.WriteLine($"Document {ex.Key} not found in {ex.Collection}");
|
||||
}
|
||||
```
|
||||
|
||||
## Querying
|
||||
|
||||
Supported LINQ operators:
|
||||
- Equality: `==`, `!=`
|
||||
- Comparison: `>`, `<`, `>=`, `<=`
|
||||
- Boolean: `&&`, `||`
|
||||
- String: `Contains`, `StartsWith`, `EndsWith` (Mapped to SQL `LIKE`)
|
||||
- Collections: `Contains` (IN clause)
|
||||
|
||||
```csharp
|
||||
// Find users in list of IDs
|
||||
var ids = new[] { "1", "2" };
|
||||
await col.Find(u => ids.Contains(u.Id));
|
||||
```
|
||||
70
docs/architecture.md
Executable file
70
docs/architecture.md
Executable file
@@ -0,0 +1,70 @@
|
||||
# Architecture & Concepts
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
CBDDC is designed for **Local Area Networks (LAN)** and **Local-First** scenarios.
|
||||
It does not rely on a central master server. Every node is equal (Peer-to-Peer).
|
||||
|
||||
**Target Deployment**: Trusted LAN environments (offices, homes, private networks)
|
||||
**Cross-Platform**: Windows, Linux, macOS (.NET 8.0+, with .NET 6.0 and .NET Standard 2.0 support)
|
||||
|
||||
### HLC (Hybrid Logical Clock)
|
||||
To resolve conflicts without a central authority, we use **Hybrid Logical Clocks**.
|
||||
This allows us to determine the "happened-before" relationship between events even if system clocks are slightly skewed.
|
||||
In case of concurrent edits, the "Last Write Wins" (LWW) policy based on HLC is applied.
|
||||
|
||||
## Synchronization
|
||||
|
||||
### Anti-Entropy
|
||||
When two nodes connect, they exchange their latest HLC timestamps.
|
||||
- If Node A is ahead of Node B, Node B "pulls" the missing operations from Node A.
|
||||
- If Node B is ahead, Node A "pushes" its operations.
|
||||
|
||||
### Gossip Protocol
|
||||
Nodes discover each other via UDP Broadcast (LAN) and then form random TCP connections to gossip updates.
|
||||
This ensures that updates propagate exponentially through the network (Epidemic Algorithm).
|
||||
|
||||
### Snapshots & Fast Recovery
|
||||
To optimize reconnection, each node maintains a **Snapshot** of the last known state (Hash & Timestamp) for every peer.
|
||||
- When re-connecting, nodes compare their snapshot state.
|
||||
- If the chain hash matches, they only exchange the delta.
|
||||
- This avoids re-processing the entire operation history and ensures efficient gap recovery.
|
||||
|
||||
### Peer-Confirmed Oplog Pruning
|
||||
CBDDC maintenance pruning now uses a two-cutoff model:
|
||||
|
||||
- **Retention cutoff**: derived from `OplogRetentionHours` (time-based policy).
|
||||
- **Confirmation cutoff**: the oldest confirmation watermark across all active tracked peers and relevant source nodes.
|
||||
|
||||
The effective prune cutoff is:
|
||||
|
||||
- `min(retention cutoff, confirmation cutoff)` when confirmation data is complete.
|
||||
- retention cutoff only when confirmation tracking is not configured or when there are no active tracked peers.
|
||||
- **no prune** for that maintenance cycle when any active tracked peer is missing confirmation for any relevant source stream.
|
||||
|
||||
Key semantics:
|
||||
|
||||
- Relevant source streams are taken from the local non-default vector clock entries.
|
||||
- A peer only gates pruning while it is active in peer tracking.
|
||||
- Removing peer tracking excludes that peer from prune gating.
|
||||
- Maintenance logs include explicit skip reasons when pruning is blocked (for example,
|
||||
missing confirmation for a tracked peer/source pair).
|
||||
|
||||
## Security Disclaimer
|
||||
|
||||
::: warning FOR LAN USE ONLY
|
||||
**CBDDC is designed for trusted Local Area Networks.**
|
||||
:::
|
||||
|
||||
- **P2P Mesh Mode**: Designed for **LAN/VPN** use. Uses raw TCP and UDP broadcast. Not safe for public internet by default.
|
||||
- **ASP.NET Core Server**: Designed for controlled server deployments. Supports standard HTTPS and hosted sync endpoints.
|
||||
- **Transport**: Data is transmitted via raw TCP. There is **NO Encryption (TLS/SSL)** by default.
|
||||
- **Authentication**: A basic "Shared Key" mechanism is implemented. Nodes must share the same `AuthToken` to sync.
|
||||
- **Authorization**: Once authenticated, a node has full read/write access to all collections.
|
||||
|
||||
**Recommendation**:
|
||||
- Use only within trusted private networks (LAN, VPN, or localhost)
|
||||
- For internet deployment, implement TLS, proper authentication, and firewall rules
|
||||
- Consider the production hardening features for resilience on LAN
|
||||
|
||||
**Cross-Platform Support**: Runs on Windows, Linux, and macOS with .NET 8.0+ (also supports .NET 6.0 and .NET Standard 2.0).
|
||||
251
docs/conflict-resolution.md
Executable file
251
docs/conflict-resolution.md
Executable file
@@ -0,0 +1,251 @@
|
||||
# Conflict Resolution
|
||||
|
||||
**CBDDC v0.6.0** introduces **pluggable conflict resolution strategies** to handle concurrent updates to the same document across different nodes.
|
||||
|
||||
## Overview
|
||||
|
||||
When two nodes modify the same document offline and later sync, a conflict occurs. CBDDC provides two built-in strategies:
|
||||
|
||||
1. **Last Write Wins (LWW)** - Simple, timestamp-based resolution
|
||||
2. **Recursive Merge** - Intelligent JSON merging with array handling
|
||||
|
||||
## Conflict Resolution Strategies
|
||||
|
||||
### Last Write Wins (LWW)
|
||||
|
||||
**How it works:**
|
||||
- Each document has a Hybrid Logical Clock (HLC) timestamp
|
||||
- During sync, the document with the **highest timestamp wins**
|
||||
- Conflicts are resolved automatically with no merge attempt
|
||||
|
||||
**Pros:**
|
||||
- ✅ Simple and predictable
|
||||
- ✅ Fast (no merge computation)
|
||||
- ✅ No data corruption or invalid states
|
||||
|
||||
**Cons:**
|
||||
- ❌ Data loss - one change is discarded entirely
|
||||
- ❌ Not suitable for collaborative editing
|
||||
|
||||
**Use Cases:**
|
||||
- Configuration data with infrequent updates
|
||||
- Reference data (product catalogs, price lists)
|
||||
- Single-user scenarios with backup sync
|
||||
|
||||
#### Example
|
||||
|
||||
```csharp
|
||||
// Both nodes start with same document
|
||||
{ "id": "doc-1", "name": "Alice", "age": 25 }
|
||||
|
||||
// Node A updates (timestamp: 100)
|
||||
{ "id": "doc-1", "name": "Alice", "age": 26 }
|
||||
|
||||
// Node B updates (timestamp: 105)
|
||||
{ "id": "doc-1", "name": "Alicia", "age": 25 }
|
||||
|
||||
// After sync: Node B wins (higher timestamp)
|
||||
{ "id": "doc-1", "name": "Alicia", "age": 25 }
|
||||
// Node A's age change is LOST
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Recursive Merge
|
||||
|
||||
**How it works:**
|
||||
- Performs deep JSON merge of conflicting documents
|
||||
- Uses the **highest timestamp** for each individual field
|
||||
- Arrays with `id` or `_id` fields are merged by identity
|
||||
- Arrays without IDs are concatenated and deduplicated
|
||||
|
||||
**Pros:**
|
||||
- ✅ Preserves changes from both nodes
|
||||
- ✅ Suitable for collaborative scenarios
|
||||
- ✅ Intelligent array handling
|
||||
|
||||
**Cons:**
|
||||
- ❌ More complex logic
|
||||
- ❌ Slightly slower (~5-10ms overhead)
|
||||
- ❌ May produce unexpected results with complex nested structures
|
||||
|
||||
**Use Cases:**
|
||||
- TodoLists, shopping carts (demonstrated in samples)
|
||||
- Collaborative documents
|
||||
- Complex objects with independent fields
|
||||
- Data where every change matters
|
||||
|
||||
#### Example: Field-Level Merge
|
||||
|
||||
```csharp
|
||||
// Both nodes start with same document
|
||||
{ "id": "doc-1", "name": "Alice", "age": 25 }
|
||||
|
||||
// Node A updates (timestamp: 100)
|
||||
{ "id": "doc-1", "name": "Alice", "age": 26 }
|
||||
|
||||
// Node B updates (timestamp: 105)
|
||||
{ "id": "doc-1", "name": "Alicia", "age": 25 }
|
||||
|
||||
// After Recursive Merge:
|
||||
{ "id": "doc-1", "name": "Alicia", "age": 26 }
|
||||
// Uses latest timestamp for each field independently
|
||||
```
|
||||
|
||||
#### Example: Array Merging with IDs
|
||||
|
||||
```csharp
|
||||
// Initial TodoList
|
||||
{
|
||||
"id": "list-1",
|
||||
"name": "Shopping",
|
||||
"items": [
|
||||
{ "id": "1", "task": "Buy milk", "completed": false },
|
||||
{ "id": "2", "task": "Buy bread", "completed": false }
|
||||
]
|
||||
}
|
||||
|
||||
// Node A: Completes milk, adds eggs
|
||||
{
|
||||
"items": [
|
||||
{ "id": "1", "task": "Buy milk", "completed": true },
|
||||
{ "id": "2", "task": "Buy bread", "completed": false },
|
||||
{ "id": "3", "task": "Buy eggs", "completed": false }
|
||||
]
|
||||
}
|
||||
|
||||
// Node B: Completes bread, adds cheese
|
||||
{
|
||||
"items": [
|
||||
{ "id": "1", "task": "Buy milk", "completed": false },
|
||||
{ "id": "2", "task": "Buy bread", "completed": true },
|
||||
{ "id": "4", "task": "Buy cheese", "completed": false }
|
||||
]
|
||||
}
|
||||
|
||||
// After Recursive Merge: ALL changes preserved!
|
||||
{
|
||||
"items": [
|
||||
{ "id": "1", "task": "Buy milk", "completed": true }, // A's completion
|
||||
{ "id": "2", "task": "Buy bread", "completed": true }, // B's completion
|
||||
{ "id": "3", "task": "Buy eggs", "completed": false }, // A's addition
|
||||
{ "id": "4", "task": "Buy cheese", "completed": false } // B's addition
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Select Strategy at Startup
|
||||
|
||||
```csharp
|
||||
using ZB.MOM.WW.CBDDC.Core.Sync;
|
||||
|
||||
// Last Write Wins (default)
|
||||
services.AddSingleton<IConflictResolver, LastWriteWinsConflictResolver>();
|
||||
|
||||
// OR Recursive Merge
|
||||
services.AddSingleton<IConflictResolver, RecursiveNodeMergeConflictResolver>();
|
||||
```
|
||||
|
||||
### Console Sample
|
||||
|
||||
```bash
|
||||
# Use Recursive Merge
|
||||
dotnet run --merge
|
||||
|
||||
# Use Last Write Wins (default)
|
||||
dotnet run
|
||||
```
|
||||
|
||||
### UI Clients
|
||||
|
||||
Resolver selection can be exposed in your UI:
|
||||
1. Choose "Recursive Merge" or "Last Write Wins"
|
||||
2. Save configuration
|
||||
3. Restart the application if required by your host
|
||||
|
||||
## Interactive Demo
|
||||
|
||||
A UI demo can expose a **"Run Conflict Demo"** action that:
|
||||
1. Creates a TodoList with 2 items
|
||||
2. Simulates concurrent edits from two "nodes"
|
||||
3. Shows the merged result
|
||||
4. Compares LWW vs Recursive Merge behavior
|
||||
|
||||
**Try it yourself** to see the difference!
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Use LWW When:
|
||||
- Only one user/node typically writes
|
||||
- Data is reference/configuration
|
||||
- Simplicity is more important than preserving every change
|
||||
- Performance is critical
|
||||
|
||||
### Use Recursive Merge When:
|
||||
- Multiple users collaborate
|
||||
- Every change is valuable (e.g., TodoItems, cart items)
|
||||
- Data has independent fields that can conflict
|
||||
- Arrays have `id` fields for identity
|
||||
|
||||
### Avoid Conflicts Entirely:
|
||||
- Use **different collections** for different data types
|
||||
- Implement **optimistic locking** with version fields
|
||||
- Design data models to minimize overlapping writes
|
||||
|
||||
## Custom Resolvers
|
||||
|
||||
You can implement `IConflictResolver` for custom logic:
|
||||
|
||||
```csharp
|
||||
public class CustomResolver : IConflictResolver
|
||||
{
|
||||
public ValueTask<string> ResolveConflict(
|
||||
string localJson,
|
||||
string remoteJson,
|
||||
long localTimestamp,
|
||||
long remoteTimestamp)
|
||||
{
|
||||
// Your custom merge logic here
|
||||
return new ValueTask<string>(result);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Register your resolver:
|
||||
```csharp
|
||||
services.AddSingleton<IConflictResolver, CustomResolver>();
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
Benchmark results (1000 conflict resolutions):
|
||||
|
||||
| Strategy | Avg Time | Throughput |
|
||||
|----------|----------|------------|
|
||||
| Last Write Wins | 0.05ms | 20,000 ops/sec |
|
||||
| Recursive Merge | 0.15ms | 6,600 ops/sec |
|
||||
|
||||
**Recommendation**: Performance difference is negligible for most applications. Choose based on data preservation needs.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I switch resolvers at runtime?**
|
||||
A: No. The resolver is injected at startup. Changing requires application restart.
|
||||
|
||||
**Q: What happens if I change resolvers after data exists?**
|
||||
A: Existing data is unaffected. Only future conflicts use the new strategy.
|
||||
|
||||
**Q: Can different nodes use different resolvers?**
|
||||
A: Technically yes, but **not recommended**. All nodes should use the same strategy for consistency.
|
||||
|
||||
**Q: Does this handle schema changes?**
|
||||
A: No. Conflict resolution assumes both documents have compatible schemas.
|
||||
|
||||
---
|
||||
|
||||
**See Also:**
|
||||
- [Getting Started](getting-started.html)
|
||||
- [Architecture](architecture.html)
|
||||
- [Security](security.html)
|
||||
1270
docs/database-sync-manager-design.md
Executable file
1270
docs/database-sync-manager-design.md
Executable file
File diff suppressed because it is too large
Load Diff
146
docs/deployment-lan.md
Executable file
146
docs/deployment-lan.md
Executable file
@@ -0,0 +1,146 @@
|
||||
# CBDDC - Deployment Guide for LAN
|
||||
|
||||
## Target Environment
|
||||
|
||||
CBDDC is specifically designed for **Local Area Networks (LAN)** in trusted environments:
|
||||
|
||||
✅ **Ideal Use Cases**:
|
||||
- Office networks (employee workstations, kiosks)
|
||||
- Home automation systems
|
||||
- Retail point-of-sale systems (POS)
|
||||
- Edge computing deployments
|
||||
- Private industrial networks
|
||||
- Development/testing environments
|
||||
|
||||
❌ **NOT Recommended**:
|
||||
- Public internet deployment (without significant security enhancements)
|
||||
- Multi-tenant SaaS applications
|
||||
- Untrusted network environments
|
||||
|
||||
## Cross-Platform Support
|
||||
|
||||
CBDDC runs on all major operating systems:
|
||||
|
||||
| Platform | Support | Notes |
|
||||
|----------|---------|-------|
|
||||
| **Windows** | ✅ Full | Windows 10+, Server 2019+ |
|
||||
| **Linux** | ✅ Full | Ubuntu, Debian, RHEL, Alpine |
|
||||
| **macOS** | ✅ Full | macOS 11+ (Big Sur and later) |
|
||||
|
||||
**Requirements**: .NET 8.0+ Runtime (also compatible with .NET 6.0 and .NET Standard 2.0)
|
||||
|
||||
## LAN Deployment Checklist
|
||||
|
||||
### Network Configuration
|
||||
|
||||
- [ ] **Firewall Rules**: Open TCP port (default: 5000) and UDP port (default: 6000)
|
||||
- [ ] **Broadcast Domain**: Ensure nodes are in the same subnet for UDP discovery
|
||||
- [ ] **Network Stability**: LAN should have reasonable stability (WiFi or wired)
|
||||
- [ ] **Bandwidth**: Adequate for sync operations (typically low, < 1 Mbps)
|
||||
|
||||
### Security Configuration
|
||||
|
||||
- [ ] **Cluster Key**: Configure unique cluster authentication key
|
||||
- [ ] **Network Isolation**: Use VLANs or network segmentation
|
||||
- [ ] **Access Control**: Limit network access to authorized devices
|
||||
- [ ] **Monitoring**: Set up logging and health checks
|
||||
|
||||
### Application Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"CBDDC": {
|
||||
"Network": {
|
||||
"TcpPort": 5000,
|
||||
"UdpPort": 6000,
|
||||
"LocalhostOnly": false
|
||||
},
|
||||
"Persistence": {
|
||||
"DatabasePath": "/var/lib/cbddc/data.db",
|
||||
"EnableWalMode": true,
|
||||
"EnableAutoBackup": true,
|
||||
"BackupPath": "/var/lib/cbddc/backups"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Platform-Specific Considerations
|
||||
|
||||
#### Windows
|
||||
- Use Windows Services for background operation
|
||||
- Configure Windows Firewall rules
|
||||
- Consider SQLite file locking on network shares
|
||||
|
||||
#### Linux
|
||||
- Use systemd for service management
|
||||
- Set appropriate file permissions
|
||||
- Consider SELinux/AppArmor policies
|
||||
|
||||
#### macOS
|
||||
- Use launchd for background services
|
||||
- Configure macOS firewall
|
||||
- Handle macOS sleep/wake for laptops
|
||||
|
||||
## Example: Office Network Deployment
|
||||
|
||||
### Scenario
|
||||
10 workstations in an office need to sync product catalog data.
|
||||
|
||||
### Setup
|
||||
1. **Network**: All on 192.168.1.0/24 subnet
|
||||
2. **Nodes**: Each workstation runs CBDDC
|
||||
3. **Discovery**: UDP broadcast for automatic peer discovery
|
||||
4. **Sync**: TCP for data synchronization
|
||||
5. **Storage**: Local SQLite database per workstation
|
||||
|
||||
### Benefits
|
||||
- **No Internet Required**: Works during internet outages
|
||||
- **Low Latency**: Local network = fast reads/writes
|
||||
- **Resilient**: No single point of failure
|
||||
- **Offline Capable**: Each workstation works independently
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Nodes Not Discovering Each Other
|
||||
- Check firewall rules for UDP port
|
||||
- Verify nodes are on same broadcast domain
|
||||
- Check cluster key matches on all nodes
|
||||
|
||||
### Slow Synchronization
|
||||
- Check network bandwidth
|
||||
- Verify no packet loss
|
||||
- Review batch size configuration
|
||||
|
||||
### Database Corruption
|
||||
- Verify WAL mode is enabled
|
||||
- Check disk space
|
||||
- Review backup/restore procedures
|
||||
|
||||
## Security Best Practices for LAN
|
||||
|
||||
1. **Network Segmentation**: Isolate CBDDC network from public networks
|
||||
2. **Cluster Authentication**: Use strong cluster keys
|
||||
3. **Access Control**: Limit which devices can join the network
|
||||
4. **Monitoring**: Log all sync operations
|
||||
5. **Regular Backups**: Automated backup to separate storage
|
||||
6. **Update Policy**: Keep .NET runtime updated
|
||||
|
||||
## NOT Recommended for Internet
|
||||
|
||||
CBDDC **should NOT** be deployed on public internet without:
|
||||
- TLS/SSL encryption for TCP connections
|
||||
- Proper authentication beyond cluster key
|
||||
- Network firewalls and security groups
|
||||
- DDoS protection
|
||||
- Rate limiting
|
||||
- Intrusion detection
|
||||
|
||||
For internet deployment, consider traditional client-server databases instead.
|
||||
|
||||
## Support
|
||||
|
||||
For LAN deployment questions, see:
|
||||
- [Production Hardening Guide](production-hardening.md)
|
||||
- [API Reference](api-reference.md)
|
||||
- [Architecture Documentation](architecture.md)
|
||||
82
docs/deployment-modes.md
Executable file
82
docs/deployment-modes.md
Executable file
@@ -0,0 +1,82 @@
|
||||
# CBDDC Deployment Mode
|
||||
|
||||
ZB.MOM.WW.CBDDC.Hosting supports a single deployment mode.
|
||||
|
||||
## Single-Cluster Mode
|
||||
|
||||
**Best for:**
|
||||
- Dedicated database servers
|
||||
- Simple deployments
|
||||
- Development/testing environments
|
||||
- Small-to-medium applications
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────┐
|
||||
│ ASP.NET Server │
|
||||
│ │
|
||||
│ ┌───────────────┐ │
|
||||
│ │ CBDDC │ │
|
||||
│ │ (1 cluster) │ │
|
||||
│ └───────────────┘ │
|
||||
│ │ │
|
||||
│ [TCP:5001] │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
```csharp
|
||||
builder.Services.AddCBDDCHostingSingleCluster(options =>
|
||||
{
|
||||
options.NodeId = "server-01";
|
||||
options.TcpPort = 5001;
|
||||
});
|
||||
```
|
||||
|
||||
### Characteristics
|
||||
|
||||
- One database per server instance
|
||||
- Simple configuration
|
||||
- Easy to scale horizontally (multiple servers = multiple databases)
|
||||
- No port collision issues
|
||||
- Recommended for most use cases
|
||||
|
||||
## Server Behavior
|
||||
|
||||
Servers use respond-only operation:
|
||||
|
||||
- Accept incoming TCP sync connections
|
||||
- Respond to sync requests
|
||||
- Serve data to clients
|
||||
- Maintain persistence
|
||||
- Do not initiate outbound sync
|
||||
- Do not perform UDP discovery
|
||||
- Do not connect to other servers automatically
|
||||
|
||||
## Persistence Layer Compatibility
|
||||
|
||||
Single-cluster mode works with all persistence providers:
|
||||
|
||||
| Provider | Support | Notes |
|
||||
|----------|---------|-------|
|
||||
| SQLite (Direct) | Yes | One DB file per cluster |
|
||||
| EF Core | Yes | Flexible DB options |
|
||||
| PostgreSQL | Yes | Best for production |
|
||||
|
||||
## Scaling Strategy
|
||||
|
||||
### Horizontal Scaling
|
||||
|
||||
```
|
||||
Load Balancer
|
||||
│
|
||||
├─> Server 1 (Cluster A) [PostgreSQL A]
|
||||
├─> Server 2 (Cluster B) [PostgreSQL B]
|
||||
└─> Server 3 (Cluster C) [PostgreSQL C]
|
||||
```
|
||||
|
||||
## Recommendation
|
||||
|
||||
Use single-cluster mode for ASP.NET hosting. It is simpler, robust, and straightforward to scale.
|
||||
66
docs/dynamic-reconfiguration.md
Executable file
66
docs/dynamic-reconfiguration.md
Executable file
@@ -0,0 +1,66 @@
|
||||
---
|
||||
layout: default
|
||||
title: Dynamic Reconfiguration
|
||||
nav_order: 8
|
||||
---
|
||||
|
||||
# Dynamic Reconfiguration & Leader Election
|
||||
|
||||
CBDDC v0.8.0 introduces support for dynamic reconfiguration, allowing nodes to change their role, listening ports, and database identity without a full process restart. This is essential for containerized environments and long-running services.
|
||||
|
||||
## Configuration Provider
|
||||
|
||||
The core of this feature is the `IPeerNodeConfigurationProvider` interface (available in .Net, Kotlin, and NodeJs).
|
||||
|
||||
```typescript
|
||||
// NodeJs Example
|
||||
interface IPeerNodeConfigurationProvider {
|
||||
getConfiguration(): Promise<PeerNodeConfiguration>;
|
||||
onConfigurationChanged(callback: (config: PeerNodeConfiguration) => void): void;
|
||||
}
|
||||
```
|
||||
|
||||
Implementations can watch a file (e.g., `config.json`), a remote configuration server, or environment variables. When the configuration changes, the provider fires an event.
|
||||
|
||||
### Usage
|
||||
|
||||
Services like `TcpSyncServer` subscribe to these events.
|
||||
- **Port Change**: The server stops the listener and restarts on the new port.
|
||||
- **NodeId Change**: If the NodeID changes, it typically implies a change of identity. The persistence layer (SQLite) is bound to a specific NodeID (internal HLC history). Thus, a NodeID change requires closing the existing DB and opening/creating a new one matching the new ID.
|
||||
|
||||
## Known Peers (Static Remote)
|
||||
|
||||
You can now persist a list of "Known Peers" (Static Remote nodes) that survive restarts. This is useful for:
|
||||
- Configuring a fixed set of Cloud Gateways.
|
||||
- Connecting to specific peers outside the local broadcast domain.
|
||||
|
||||
### API
|
||||
|
||||
```typescript
|
||||
// Add a static peer
|
||||
await peerStore.saveRemotePeer({
|
||||
nodeId: "gateway-1",
|
||||
address: "192.168.1.50:25000",
|
||||
type: PeerType.StaticRemote,
|
||||
...
|
||||
});
|
||||
|
||||
// Retrieve
|
||||
const peers = await peerStore.getRemotePeers();
|
||||
```
|
||||
|
||||
## Leader Election (Bully Algorithm)
|
||||
|
||||
For Cloud Synchronization, usually one node in a LAN cluster acts as the "Gateway" to the cloud to avoid redundant traffic. CBDDC implements the **Bully Algorithm** for automatic leader election.
|
||||
|
||||
- **Rule**: Among all discovered peers (LAN or Static) that are eligible, the node with the **lexicographically smallest NodeID** declares itself the Leader.
|
||||
- **Mechanism**: Each node periodically checks the list of active peers. If it finds no other peer with a smaller ID than itself, it assumes leadership.
|
||||
- **Event**: The `LeaderElectionService` emits a `LeadershipChanged` event (boolean `isLeader`).
|
||||
- **Action**: The `SyncOrchestrator` uses this flag to decide whether to push/pull from Cloud Remotes.
|
||||
|
||||
### NodeJs Logic
|
||||
```typescript
|
||||
const amILeader = !activePeers.some(p => p.nodeId < myNodeId);
|
||||
```
|
||||
|
||||
This simple consensus mechanism works well for small-to-medium clusters on a low-latency LAN.
|
||||
188
docs/getting-started.md
Executable file
188
docs/getting-started.md
Executable file
@@ -0,0 +1,188 @@
|
||||
# Getting Started (v0.9.0)
|
||||
|
||||
## Installation
|
||||
|
||||
CBDDC is available as a set of NuGet packages for .NET 8.0, .NET 6.0, and .NET Standard 2.0.
|
||||
|
||||
```bash
|
||||
dotnet add package ZB.MOM.WW.CBDDC.Core
|
||||
dotnet add package CBDDC.Network
|
||||
dotnet add package ZB.MOM.WW.CBDDC.Persistence.Sqlite
|
||||
```
|
||||
|
||||
### Cloud & Enterprise Packages
|
||||
|
||||
For ASP.NET Core hosting and enterprise database support:
|
||||
|
||||
```bash
|
||||
# ASP.NET Core hosting
|
||||
dotnet add package ZB.MOM.WW.CBDDC.Hosting
|
||||
|
||||
# Entity Framework Core (SQL Server, MySQL, SQLite)
|
||||
dotnet add package ZB.MOM.WW.CBDDC.Persistence.EntityFramework
|
||||
|
||||
# PostgreSQL with JSONB optimization
|
||||
dotnet add package ZB.MOM.WW.CBDDC.Persistence.PostgreSQL
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- **.NET 8.0+ Runtime** (recommended) or .NET 6.0+
|
||||
- **SQLite** (included via Microsoft.Data.Sqlite)
|
||||
- **PostgreSQL 12+** (optional, for PostgreSQL persistence)
|
||||
- **SQL Server 2016+** (optional, for SQL Server persistence)
|
||||
|
||||
## Basic Usage
|
||||
|
||||
### 1. Initialize the Store
|
||||
Use `SqlitePeerStore` for persistence. Supported on Windows, Linux, and macOS.
|
||||
|
||||
```csharp
|
||||
using ZB.MOM.WW.CBDDC.Core;
|
||||
using ZB.MOM.WW.CBDDC.Core.Sync;
|
||||
using ZB.MOM.WW.CBDDC.Persistence.Sqlite;
|
||||
using CBDDC.Network.Security;
|
||||
|
||||
// Choose conflict resolver (v0.6.0+)
|
||||
var resolver = new RecursiveNodeMergeConflictResolver(); // OR LastWriteWinsConflictResolver()
|
||||
|
||||
var store = new SqlitePeerStore("Data Source=my-node.db", logger, resolver);
|
||||
// Automatically creates tables on first run
|
||||
```
|
||||
|
||||
### 2. Configure Networking (with Optional Security)
|
||||
Use `AddCBDDCNetwork` extension method to register services.
|
||||
|
||||
```csharp
|
||||
var services = new ServiceCollection();
|
||||
string myNodeId = "node-1";
|
||||
int port = 5001;
|
||||
string authToken = "my-secret-cluster-key";
|
||||
|
||||
services.AddSingleton<IPeerStore>(store);
|
||||
|
||||
// Optional: Enable encryption (v0.6.0+)
|
||||
services.AddSingleton<IPeerHandshakeService, SecureHandshakeService>();
|
||||
|
||||
services.AddCBDDCNetwork(myNodeId, port, authToken);
|
||||
```
|
||||
|
||||
### 3. Start the Node
|
||||
|
||||
```csharp
|
||||
var provider = services.BuildServiceProvider();
|
||||
var node = provider.GetRequiredService<CBDDCNode>();
|
||||
|
||||
node.Start();
|
||||
```
|
||||
|
||||
### 4. CRUD Operations
|
||||
Interact with data using `PeerDatabase`.
|
||||
|
||||
```csharp
|
||||
var db = new PeerDatabase(store, "my-node-id"); // Node ID used for HLC clock
|
||||
await db.InitializeAsync();
|
||||
|
||||
var users = db.Collection("users");
|
||||
|
||||
// Put
|
||||
await users.Put("user-1", new { Name = "Alice", Age = 30 });
|
||||
|
||||
// Get
|
||||
var user = await users.Get<User>("user-1");
|
||||
|
||||
// Query
|
||||
var results = await users.Find<User>(u => u.Age > 20);
|
||||
```
|
||||
|
||||
## ASP.NET Core Deployment (v0.8.0+)
|
||||
|
||||
### Single Cluster Mode (Recommended)
|
||||
|
||||
Perfect for production deployments with dedicated database servers:
|
||||
|
||||
```csharp
|
||||
// Program.cs
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
// Use PostgreSQL for production
|
||||
builder.Services.AddCBDDCPostgreSql(
|
||||
builder.Configuration.GetConnectionString("CBDDC"));
|
||||
|
||||
// Configure single cluster
|
||||
builder.Services.AddCBDDCHostingSingleCluster(options =>
|
||||
{
|
||||
options.NodeId = "server-01";
|
||||
options.TcpPort = 5001;
|
||||
});
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
app.MapHealthChecks("/health");
|
||||
app.Run();
|
||||
```
|
||||
|
||||
See [Deployment Modes](deployment-modes.md) for deployment details.
|
||||
|
||||
## What's New in v0.9.0
|
||||
|
||||
### 🚀 Production Enhancements
|
||||
- **Improved ASP.NET Core Sample**: Enhanced error handling and better examples
|
||||
- **EF Core Stability**: Fixed runtime issues for all persistence providers
|
||||
- **Sync Refinements**: More reliable synchronization across hosted deployments
|
||||
|
||||
### 📸 Snapshots (v0.8.6)
|
||||
- **Fast Reconnection**: Peers resume sync from the last known state
|
||||
- **Optimized Recovery**: Prevents re-processing of already applied operations
|
||||
- **Automatic Management**: Snapshot metadata tracked per peer
|
||||
|
||||
See [CHANGELOG](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/blob/main/CHANGELOG.md) for complete version history.
|
||||
|
||||
## What's New in v0.8.0
|
||||
|
||||
### ☁️ Cloud Infrastructure
|
||||
- **ASP.NET Core Hosting**: Single-cluster deployment mode
|
||||
- **Multi-Database Support**: SQL Server, PostgreSQL, MySQL, SQLite via EF Core
|
||||
- **PostgreSQL Optimization**: JSONB storage with GIN indexes
|
||||
- **Shared-token Authentication**: Secure controlled deployments
|
||||
- **Health Checks**: Production monitoring and observability
|
||||
|
||||
[Learn more about Cloud Deployment →](deployment-modes.md)
|
||||
|
||||
## What's New in v0.7.0
|
||||
|
||||
### 📦 Efficient Networking
|
||||
- **Brotli Compression**: Data is automatically compressed, significantly reducing bandwidth usage
|
||||
- **Protocol v4**: Enhanced framing and security negotiation
|
||||
|
||||
## What's New in v0.6.0
|
||||
|
||||
### 🔐 Secure Networking
|
||||
Protect your data in transit with:
|
||||
- **ECDH** key exchange
|
||||
- **AES-256-CBC** encryption
|
||||
- **HMAC-SHA256** authentication
|
||||
|
||||
[Learn more about Security →](security.md)
|
||||
|
||||
### 🔀 Advanced Conflict Resolution
|
||||
Choose your strategy:
|
||||
- **Last Write Wins** - Simple, fast, timestamp-based
|
||||
- **Recursive Merge** - Intelligent JSON merging with array ID detection
|
||||
|
||||
[Learn more about Conflict Resolution →](conflict-resolution.md)
|
||||
|
||||
### 🎯 Multi-Target Framework Support
|
||||
- `netstandard2.0` - Maximum compatibility
|
||||
- `net6.0` - Modern features
|
||||
- `net8.0` - Latest performance optimizations
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Architecture Overview](architecture.html) - Understand HLC, Gossip Protocol, and mesh networking
|
||||
- [Persistence Providers](persistence-providers.html) - Choose the right database for your deployment
|
||||
- [Deployment Modes](deployment-modes.html) - Single-cluster deployment strategy
|
||||
- [Security Configuration](security.html) - Encryption and authentication
|
||||
- [Conflict Resolution Strategies](conflict-resolution.html) - LWW vs Recursive Merge
|
||||
- [Production Hardening](production-hardening.html) - Best practices and monitoring
|
||||
- [API Reference](api-reference.html) - Complete API documentation
|
||||
57
docs/index.md
Executable file
57
docs/index.md
Executable file
@@ -0,0 +1,57 @@
|
||||
---
|
||||
layout: default
|
||||
---
|
||||
|
||||
# CBDDC Documentation
|
||||
|
||||
Welcome to the CBDDC documentation for **version 0.9.0**.
|
||||
|
||||
## What's New in v0.9.0
|
||||
|
||||
Version 0.9.0 brings enhanced ASP.NET Core support, improved EF Core runtime stability, and refined synchronization and persistence layers.
|
||||
|
||||
- **ASP.NET Core Enhancements**: Improved sample application with better error handling
|
||||
- **EF Core Stability**: Fixed runtime issues and improved persistence layer reliability
|
||||
- **Sync & Persistence**: Enhanced stability across all persistence providers
|
||||
- **Production Ready**: All features tested and stable for production use
|
||||
|
||||
See the [CHANGELOG](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/blob/main/CHANGELOG.md) for complete details.
|
||||
|
||||
## Documentation
|
||||
|
||||
### Getting Started
|
||||
* [**Getting Started**](getting-started.html) - Installation, basic setup, and your first CBDDC application
|
||||
|
||||
### Core Concepts
|
||||
* [Architecture](architecture.html) - Understanding HLC, Gossip Protocol, and P2P mesh networking
|
||||
* [API Reference](api-reference.html) - Complete API documentation and examples
|
||||
* [Querying](querying.html) - Data querying patterns and LINQ support
|
||||
|
||||
### Persistence & Storage
|
||||
* [Persistence Providers](persistence-providers.html) - SQLite, EF Core, PostgreSQL comparison and configuration
|
||||
* [Deployment Modes](deployment-modes.html) - Single-cluster deployment strategy
|
||||
|
||||
### Networking & Security
|
||||
* [Security](security.html) - Encryption, authentication, and secure networking
|
||||
* [Conflict Resolution](conflict-resolution.html) - LWW and Recursive Merge strategies
|
||||
* [Network Telemetry](network-telemetry.html) - Monitoring and diagnostics
|
||||
* [Dynamic Reconfiguration](dynamic-reconfiguration.html) - Runtime configuration and leader election
|
||||
* [Remote Peer Configuration](remote-peer-configuration.html) - Managing remote peers and tracking lifecycle
|
||||
* [Upgrade: Peer-Confirmed Pruning](upgrade-peer-confirmed-pruning.html) - Adoption and rollout notes
|
||||
|
||||
### Deployment & Operations
|
||||
* [Deployment (LAN)](deployment-lan.html) - Platform-specific deployment instructions
|
||||
* [Production Hardening](production-hardening.html) - Configuration, monitoring, and best practices
|
||||
* [Peer Deprecation & Removal Runbook](peer-deprecation-removal-runbook.html) - Operational deprecation/removal workflow
|
||||
|
||||
## Previous Versions
|
||||
|
||||
- [v0.8.x Documentation](v0.8/getting-started.html)
|
||||
- [v0.7.x Documentation](v0.7/getting-started.html)
|
||||
- [v0.6.x Documentation](v0.6/getting-started.html)
|
||||
|
||||
## Links
|
||||
|
||||
* [GitHub Repository](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net)
|
||||
* [NuGet Packages](https://www.nuget.org/packages?q=CBDDC)
|
||||
* [Changelog](https://github.com/CBDDC/ZB.MOM.WW.CBDDC.Net/blob/main/CHANGELOG.md)
|
||||
71
docs/network-telemetry.md
Executable file
71
docs/network-telemetry.md
Executable file
@@ -0,0 +1,71 @@
|
||||
---
|
||||
layout: default
|
||||
title: Network Telemetry
|
||||
nav_order: 9
|
||||
---
|
||||
|
||||
# Network Telemetry
|
||||
|
||||
Starting from v0.7.2, CBDDC includes a built-in telemetry system to monitor network performance, compression efficiency, and encryption overhead. This system is designed to have near-zero impact on runtime performance while providing valuable insights into peer-to-peer communication.
|
||||
|
||||
## Collected Metrics
|
||||
|
||||
The system collects the following metrics:
|
||||
|
||||
| Metric Type | Description | Unit |
|
||||
| :--- | :--- | :--- |
|
||||
| **CompressionRatio** | Ratio of compressed size to original size. Lower is better (e.g., 0.4 means 60% reduction). | Ratio (0.0 - 1.0) |
|
||||
| **EncryptionTime** | Time taken to encrypt a message payload. | Milliseconds (ms) |
|
||||
| **DecryptionTime** | Time taken to decrypt a received message. | Milliseconds (ms) |
|
||||
| **RoundTripTime** | Time taken for a `GetClock` request-response cycle (latency). | Milliseconds (ms) |
|
||||
|
||||
## Architecture
|
||||
|
||||
The telemetry service uses a high-performance, non-blocking architecture:
|
||||
1. **Capture**: Metrics are pushed to a `System.Threading.Channels` queue, ensuring the critical network path is never blocked.
|
||||
2. **Aggregation**: A background worker aggregates samples into **1-second buckets**.
|
||||
3. **Rolling Windows**: Averages are calculated on-the-fly for **1m, 5m, 10m, and 30m** windows.
|
||||
4. **Persistence**: Aggregated data is automatically persisted to a local binary file (`cbddc_metrics.bin`) every minute.
|
||||
|
||||
## Configuration
|
||||
|
||||
Telemetry is enabled by default when using the standard DI extensions.
|
||||
|
||||
### Dependency Injection
|
||||
|
||||
When using `AddCBDDCNetwork`, the `INetworkTelemetryService` is automatically registered as a singleton.
|
||||
|
||||
```csharp
|
||||
services.AddCBDDCCore()
|
||||
.AddCBDDCNetwork<MyNodeConfiguration>();
|
||||
```
|
||||
|
||||
Use `INetworkTelemetryService` to access metric data programmatically:
|
||||
|
||||
```csharp
|
||||
public class MyService
|
||||
{
|
||||
private readonly INetworkTelemetryService _telemetry;
|
||||
|
||||
public MyService(INetworkTelemetryService telemetry)
|
||||
{
|
||||
_telemetry = telemetry;
|
||||
}
|
||||
|
||||
public void PrintMetrics()
|
||||
{
|
||||
var snapshot = _telemetry.GetSnapshot();
|
||||
// Access snapshot[MetricType.CompressionRatio][60] for 1-minute average
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Security & Privacy
|
||||
|
||||
- Telemetry data is **stored locally** (on the device/server).
|
||||
- No data is sent to external servers or other peers.
|
||||
- Metric data does not contain any PII or payload content.
|
||||
|
||||
## Viewing Metrics
|
||||
|
||||
You can visualize these metrics in a custom dashboard (desktop or web) for real-time monitoring.
|
||||
66
docs/peer-deprecation-removal-runbook.md
Normal file
66
docs/peer-deprecation-removal-runbook.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Peer Deprecation & Removal Runbook
|
||||
|
||||
Operational workflow for safely deprecating or removing peers in clusters using
|
||||
peer-confirmed pruning.
|
||||
|
||||
## When to use this runbook
|
||||
|
||||
- A site is permanently decommissioned.
|
||||
- A peer has been unreachable long enough to block prune progress.
|
||||
- A peer is being replaced and should stop gating prune decisions.
|
||||
|
||||
## Decision matrix
|
||||
|
||||
| Scenario | Action |
|
||||
|------|-----------|
|
||||
| Peer is temporarily offline and expected to return soon | Keep tracking; monitor lag and confirmations. |
|
||||
| Peer should stay configured but must stop gating pruning | `RemovePeerTrackingAsync(nodeId, removeRemoteConfig: false)` |
|
||||
| Peer is permanently removed from topology | `RemoveRemotePeerAsync(nodeId)` |
|
||||
|
||||
## Procedure
|
||||
|
||||
1. Confirm peer intent (temporary outage vs. decommission).
|
||||
2. Inspect health payload:
|
||||
- `peersWithNoConfirmation`
|
||||
- `laggingPeers`
|
||||
- `lastSuccessfulConfirmationUpdateByPeer`
|
||||
3. If deprecating from prune gating only, run:
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemovePeerTrackingAsync(
|
||||
nodeId: "peer-to-deprecate",
|
||||
removeRemoteConfig: false,
|
||||
cancellationToken);
|
||||
```
|
||||
|
||||
4. If permanently removing, run:
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemoveRemotePeerAsync("peer-to-remove", cancellationToken);
|
||||
```
|
||||
|
||||
5. Re-check `/health` and verify status transition:
|
||||
- `Degraded`/`Unhealthy` should clear if the removed peer was the cause.
|
||||
6. Confirm maintenance logs no longer report prune blocking for that peer.
|
||||
|
||||
## Post-change verification
|
||||
|
||||
- Peer no longer appears in active tracked peers.
|
||||
- `maxLagMs` trends with only current active peers.
|
||||
- Pruning resumes with a valid effective cutoff (or a known non-peer reason).
|
||||
|
||||
## Emergency path
|
||||
|
||||
If pruning is blocked and storage pressure is high:
|
||||
|
||||
1. De-track the clearly retired peer first (`removeRemoteConfig: false`).
|
||||
2. Validate pruning resumes.
|
||||
3. Perform full peer removal after change-control approval.
|
||||
|
||||
## Re-activation path
|
||||
|
||||
If a deprecated peer returns and should gate pruning again:
|
||||
|
||||
1. Ensure peer config is enabled/available.
|
||||
2. Allow sync to re-register and refresh confirmations.
|
||||
3. Watch health payload until peer exits `peersWithNoConfirmation`.
|
||||
280
docs/persistence-providers.md
Executable file
280
docs/persistence-providers.md
Executable file
@@ -0,0 +1,280 @@
|
||||
# CBDDC Persistence Providers
|
||||
|
||||
CBDDC supports multiple persistence backends to suit different deployment scenarios.
|
||||
|
||||
## Overview
|
||||
|
||||
| Provider | Best For | Performance | Setup | Production Ready |
|
||||
|----------|----------|-------------|-------|------------------|
|
||||
| **SQLite (Direct)** | Embedded apps, single-node | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ✅ Yes |
|
||||
| **EF Core (Generic)** | Multi-DB support, migrations | ⭐⭐⭐ | ⭐⭐⭐ | ✅ Yes |
|
||||
| **PostgreSQL** | Production, high load, JSON queries | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ✅ Yes |
|
||||
|
||||
## SQLite (Direct)
|
||||
|
||||
**Package:** `ZB.MOM.WW.CBDDC.Persistence.Sqlite`
|
||||
|
||||
### Characteristics
|
||||
|
||||
- ✅ **Zero configuration**: Works out of the box
|
||||
- ✅ **Excellent performance**: Native SQL, no ORM overhead
|
||||
- ✅ **WAL mode**: Concurrent readers + writers
|
||||
- ✅ **Per-collection tables**: Optional for better isolation
|
||||
- ✅ **Snapshots**: Fast reconnection with `SnapshotMetadata`
|
||||
- ✅ **Portable**: Single file database
|
||||
- ❌ **Limited JSON queries**: Uses `json_extract()`
|
||||
|
||||
### When to Use
|
||||
|
||||
- Building single-node applications
|
||||
- Embedded scenarios (desktop, mobile)
|
||||
- Development/testing
|
||||
- Maximum simplicity required
|
||||
- File-based portability important
|
||||
|
||||
### Configuration
|
||||
|
||||
```csharp
|
||||
// Legacy mode (simple)
|
||||
services.AddCBDDCSqlite("Data Source=cbddc.db");
|
||||
|
||||
// New mode (per-collection tables)
|
||||
services.AddCBDDCSqlite(options =>
|
||||
{
|
||||
options.BasePath = "/var/lib/cbddc";
|
||||
options.DatabaseFilenameTemplate = "cbddc-{NodeId}.db";
|
||||
options.UsePerCollectionTables = true;
|
||||
});
|
||||
```
|
||||
|
||||
### Performance Tips
|
||||
|
||||
1. Enable WAL mode (done automatically)
|
||||
2. Use per-collection tables for large datasets
|
||||
3. Create indexes on frequently queried fields
|
||||
4. Keep database on fast storage (SSD)
|
||||
|
||||
## EF Core (Generic)
|
||||
|
||||
**Package:** `ZB.MOM.WW.CBDDC.Persistence.EntityFramework`
|
||||
|
||||
### Characteristics
|
||||
|
||||
- ✅ **Multi-database support**: SQL Server, MySQL, SQLite, PostgreSQL
|
||||
- ✅ **EF Core benefits**: Migrations, LINQ, change tracking
|
||||
- ✅ **Type-safe**: Strongly-typed entities
|
||||
- ⚠️ **Query limitation**: JSON queries evaluated in-memory
|
||||
- ⚠️ **ORM overhead**: Slightly slower than direct SQL
|
||||
|
||||
### When to Use
|
||||
|
||||
- Need to support multiple database backends
|
||||
- Team familiar with EF Core patterns
|
||||
- Want automated migrations
|
||||
- Building enterprise applications
|
||||
- Database portability is important
|
||||
|
||||
### Configuration
|
||||
|
||||
#### SQLite
|
||||
```csharp
|
||||
services.AddCBDDCEntityFrameworkSqlite("Data Source=cbddc.db");
|
||||
```
|
||||
|
||||
#### SQL Server
|
||||
```csharp
|
||||
services.AddCBDDCEntityFrameworkSqlServer(
|
||||
"Server=localhost;Database=CBDDC;Trusted_Connection=True;");
|
||||
```
|
||||
|
||||
#### PostgreSQL
|
||||
```csharp
|
||||
services.AddDbContext<CBDDCContext>(options =>
|
||||
options.UseNpgsql(connectionString));
|
||||
services.AddCBDDCEntityFramework();
|
||||
```
|
||||
|
||||
#### MySQL
|
||||
```csharp
|
||||
var serverVersion = ServerVersion.AutoDetect(connectionString);
|
||||
services.AddCBDDCEntityFrameworkMySql(connectionString, serverVersion);
|
||||
```
|
||||
|
||||
### Migrations
|
||||
|
||||
```bash
|
||||
# Add migration
|
||||
dotnet ef migrations add InitialCreate --context CBDDCContext
|
||||
|
||||
# Apply migration
|
||||
dotnet ef database update --context CBDDCContext
|
||||
```
|
||||
|
||||
## PostgreSQL
|
||||
|
||||
**Package:** `ZB.MOM.WW.CBDDC.Persistence.PostgreSQL`
|
||||
|
||||
### Characteristics
|
||||
|
||||
- ✅ **JSONB native storage**: Optimal JSON handling
|
||||
- ✅ **GIN indexes**: Fast JSON path queries
|
||||
- ✅ **High performance**: Production-grade
|
||||
- ✅ **Connection resilience**: Built-in retry logic
|
||||
- ✅ **Full ACID**: Strong consistency guarantees
|
||||
- ⚠️ **Future feature**: JSONB query translation (roadmap)
|
||||
|
||||
### When to Use
|
||||
|
||||
- Production deployments with high traffic
|
||||
- Need advanced JSON querying (future)
|
||||
- Require horizontal scalability
|
||||
- Want best-in-class reliability
|
||||
- Cloud deployments (AWS RDS, Azure Database, etc.)
|
||||
|
||||
### Configuration
|
||||
|
||||
```csharp
|
||||
services.AddCBDDCPostgreSql(
|
||||
"Host=localhost;Database=CBDDC;Username=user;Password=pass");
|
||||
|
||||
// With custom options
|
||||
services.AddCBDDCPostgreSql(connectionString, options =>
|
||||
{
|
||||
options.EnableSensitiveDataLogging(); // Dev only
|
||||
options.CommandTimeout(30);
|
||||
});
|
||||
```
|
||||
|
||||
### JSONB Indexes
|
||||
|
||||
For optimal performance, create GIN indexes via migrations:
|
||||
|
||||
```csharp
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.Sql(@"
|
||||
CREATE INDEX IF NOT EXISTS IX_Documents_ContentJson_gin
|
||||
ON ""Documents"" USING GIN (""ContentJson"" jsonb_path_ops);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS IX_Oplog_PayloadJson_gin
|
||||
ON ""Oplog"" USING GIN (""PayloadJson"" jsonb_path_ops);
|
||||
");
|
||||
}
|
||||
```
|
||||
|
||||
### Connection String Examples
|
||||
|
||||
#### Local Development
|
||||
```
|
||||
Host=localhost;Port=5432;Database=CBDDC;Username=admin;Password=secret
|
||||
```
|
||||
|
||||
#### Production with SSL
|
||||
```
|
||||
Host=prod-db.example.com;Database=CBDDC;Username=admin;Password=secret;SSL Mode=Require
|
||||
```
|
||||
|
||||
#### Connection Pooling
|
||||
```
|
||||
Host=localhost;Database=CBDDC;Username=admin;Password=secret;Pooling=true;Minimum Pool Size=5;Maximum Pool Size=100
|
||||
```
|
||||
|
||||
## Feature Comparison
|
||||
|
||||
| Feature | SQLite (Direct) | EF Core | PostgreSQL |
|
||||
|---------|----------------|---------|------------|
|
||||
| **Storage Format** | File-based | Varies | Server-based |
|
||||
| **JSON Storage** | TEXT | NVARCHAR/TEXT | JSONB |
|
||||
| **JSON Indexing** | Standard | Standard | GIN/GIST |
|
||||
| **JSON Queries** | `json_extract()` | In-Memory | Native (future) |
|
||||
| **Concurrent Writes** | Good (WAL) | Varies | Excellent |
|
||||
| **Horizontal Scaling** | No | Limited | Yes (replication) |
|
||||
| **Migrations** | Manual SQL | EF Migrations | EF Migrations |
|
||||
| **Connection Pooling** | N/A | Built-in | Built-in |
|
||||
| **Cloud Support** | N/A | Varies | Excellent |
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
_These are approximate figures for comparison:_
|
||||
|
||||
### Write Performance (docs/sec)
|
||||
|
||||
| Provider | Single Write | Bulk Insert (1000) |
|
||||
|----------|--------------|-------------------|
|
||||
| SQLite | 5,000 | 50,000 |
|
||||
| EF Core (SQL Server) | 3,000 | 30,000 |
|
||||
| PostgreSQL | 8,000 | 80,000 |
|
||||
|
||||
### Read Performance (docs/sec)
|
||||
|
||||
| Provider | Single Read | Query (100 results) |
|
||||
|----------|-------------|---------------------|
|
||||
| SQLite | 10,000 | 5,000 |
|
||||
| EF Core (SQL Server) | 8,000 | 4,000 |
|
||||
| PostgreSQL | 12,000 | 8,000 |
|
||||
|
||||
_*Benchmarks vary based on hardware, network, and configuration_
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### From SQLite to PostgreSQL
|
||||
|
||||
1. Export data from SQLite
|
||||
2. Set up PostgreSQL database
|
||||
3. Update connection configuration
|
||||
4. Import data to PostgreSQL
|
||||
5. Verify functionality
|
||||
|
||||
### From EF Core to PostgreSQL
|
||||
|
||||
1. Change NuGet package reference
|
||||
2. Update service registration
|
||||
3. Generate new migrations for PostgreSQL
|
||||
4. Apply migrations
|
||||
5. Test thoroughly
|
||||
|
||||
## Recommendations
|
||||
|
||||
### Development
|
||||
- **Use**: SQLite (Direct)
|
||||
- **Why**: Fast, simple, portable
|
||||
|
||||
### Testing
|
||||
- **Use**: SQLite (Direct) or EF Core with SQLite
|
||||
- **Why**: Disposable, fast test execution
|
||||
|
||||
### Production (Low-Medium Scale)
|
||||
- **Use**: SQLite (Direct) with per-collection tables
|
||||
- **Why**: Excellent performance, simple ops
|
||||
|
||||
### Production (High Scale)
|
||||
- **Use**: PostgreSQL
|
||||
- **Why**: Best performance, scalability, reliability
|
||||
|
||||
### Enterprise
|
||||
- **Use**: EF Core with SQL Server or PostgreSQL
|
||||
- **Why**: Enterprise support, compliance, familiarity
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### SQLite: "Database is locked"
|
||||
- Ensure WAL mode is enabled (automatic)
|
||||
- Increase busy timeout
|
||||
- Check for long-running transactions
|
||||
|
||||
### EF Core: "Query evaluated in-memory"
|
||||
- Expected for complex JSON queries
|
||||
- Consider PostgreSQL for better JSON support
|
||||
- Use indexes on frequently queried properties
|
||||
|
||||
### PostgreSQL: "Connection pool exhausted"
|
||||
- Increase `Maximum Pool Size`
|
||||
- Check for connection leaks
|
||||
- Consider connection pooler (PgBouncer)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- **JSONB Query Translation**: Native PostgreSQL JSON queries from QueryNode
|
||||
- **MongoDB Provider**: NoSQL option for document-heavy workloads
|
||||
- **Redis Cache Layer**: Hybrid persistence for high-read scenarios
|
||||
- **Multi-Master PostgreSQL**: Active-active replication support
|
||||
267
docs/production-hardening.md
Executable file
267
docs/production-hardening.md
Executable file
@@ -0,0 +1,267 @@
|
||||
# Production Hardening - Implementation Guide
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Configuration (appsettings.json)
|
||||
```json
|
||||
{
|
||||
"CBDDC": {
|
||||
"Network": {
|
||||
"TcpPort": 5000,
|
||||
"UdpPort": 6000,
|
||||
"RetryAttempts": 3
|
||||
},
|
||||
"Persistence": {
|
||||
"DatabasePath": "data/cbddc.db",
|
||||
"EnableWalMode": true,
|
||||
"CacheSizeMb": 50,
|
||||
"EnableAutoBackup": true,
|
||||
"BackupPath": "backups/"
|
||||
},
|
||||
"Sync": {
|
||||
"EnableOfflineQueue": true,
|
||||
"MaxQueueSize": 1000
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### DI Setup
|
||||
```csharp
|
||||
services.Configure<CBDDCOptions>(configuration.GetSection("CBDDC"));
|
||||
services.AddSingleton<RetryPolicy>();
|
||||
services.AddSingleton<DocumentCache>();
|
||||
services.AddSingleton<OfflineQueue>();
|
||||
services.AddSingleton<SyncStatusTracker>();
|
||||
services.AddCBDDCHosting(options =>
|
||||
{
|
||||
options.Cluster.NodeId = "server-01";
|
||||
options.Cluster.TcpPort = 5001;
|
||||
options.Cluster.PeerConfirmationLagThresholdMs = 30_000;
|
||||
options.Cluster.PeerConfirmationCriticalLagThresholdMs = 120_000;
|
||||
});
|
||||
```
|
||||
|
||||
### Health Check
|
||||
```csharp
|
||||
app.MapHealthChecks("/health");
|
||||
```
|
||||
|
||||
### Peer Confirmation Lag Thresholds
|
||||
|
||||
`CBDDCHealthCheck` evaluates tracked peers using confirmation lag thresholds:
|
||||
|
||||
- `PeerConfirmationLagThresholdMs` (default `30000`) marks peers as lagging and
|
||||
returns `Degraded`.
|
||||
- `PeerConfirmationCriticalLagThresholdMs` (default `120000`) marks critical lag and
|
||||
returns `Unhealthy`.
|
||||
|
||||
Thresholds are clamped so critical is never lower than lag.
|
||||
|
||||
### Health Status Interpretation
|
||||
|
||||
For the `cbddc` health check:
|
||||
|
||||
| Status | Meaning | Typical operator action |
|
||||
|------|-----------|-------------|
|
||||
| Healthy | All active tracked peers have confirmations and lag is within threshold. | No action required. |
|
||||
| Degraded | At least one tracked peer is lagging, or at least one tracked peer has no confirmation rows yet. | Investigate slow/unconfirmed peers, confirm whether any should be untracked/deprecated. |
|
||||
| Unhealthy | At least one tracked peer exceeds critical lag threshold, or persistence check throws. | Page on-call, verify storage/network path, and evaluate emergency peer de-tracking for permanently retired peers. |
|
||||
|
||||
Health payload fields:
|
||||
|
||||
- `trackedPeerCount`
|
||||
- `peersWithNoConfirmation`
|
||||
- `maxLagMs`
|
||||
- `laggingPeers`
|
||||
- `lastSuccessfulConfirmationUpdateByPeer`
|
||||
|
||||
Use these fields to distinguish temporary lag from stale peer registrations.
|
||||
|
||||
### Offline Queue
|
||||
```csharp
|
||||
// Enqueue during offline
|
||||
if (!isOnline)
|
||||
{
|
||||
offlineQueue.Enqueue(new PendingOperation
|
||||
{
|
||||
Type = "put",
|
||||
Collection = "users",
|
||||
Key = "user1",
|
||||
Data = user
|
||||
});
|
||||
}
|
||||
|
||||
// Flush when back online
|
||||
var (successful, failed) = await offlineQueue.FlushAsync(async op =>
|
||||
{
|
||||
var collection = database.Collection(op.Collection);
|
||||
if (op.Type == "put" && op.Data != null)
|
||||
await collection.Put(op.Key, op.Data);
|
||||
else if (op.Type == "delete")
|
||||
await collection.Delete(op.Key);
|
||||
});
|
||||
```
|
||||
|
||||
### Document Cache
|
||||
```csharp
|
||||
var cache = new DocumentCache(maxSizeMb: 50);
|
||||
|
||||
// Check cache first
|
||||
var cached = cache.Get("users", "user1");
|
||||
if (cached != null) return cached;
|
||||
|
||||
// Load from database
|
||||
var doc = await store.GetDocumentAsync("users", "user1");
|
||||
if (doc != null) cache.Set("users", "user1", doc);
|
||||
```
|
||||
|
||||
### SQLite Backup
|
||||
```csharp
|
||||
await store.BackupAsync("backups/backup-20260115.db");
|
||||
```
|
||||
|
||||
### Retry Policy
|
||||
```csharp
|
||||
var retry = new RetryPolicy(logger, maxAttempts: 3, delayMs: 1000);
|
||||
|
||||
await retry.ExecuteAsync(
|
||||
() => tcpClient.ConnectAsync(endpoint),
|
||||
"TCP Connect"
|
||||
);
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
Use specific exceptions for robust control flow:
|
||||
|
||||
```csharp
|
||||
try
|
||||
{
|
||||
await operation();
|
||||
}
|
||||
catch (DocumentNotFoundException ex)
|
||||
{
|
||||
// Handle specific document missing case
|
||||
logger.LogWarning("Document {Key} missing", ex.Key);
|
||||
}
|
||||
catch (CBDDCConcurrencyException ex)
|
||||
{
|
||||
// Handle conflict (though LWW usually resolves it automatically)
|
||||
logger.LogWarning("Concurrency conflict: {Message}", ex.Message);
|
||||
}
|
||||
catch (NetworkException ex)
|
||||
{
|
||||
logger.LogError(ex, "Network operation failed");
|
||||
syncTracker.RecordError(ex.Message, peerNodeId, ex.ErrorCode);
|
||||
}
|
||||
catch (PersistenceException ex) when (ex is DatabaseCorruptionException)
|
||||
{
|
||||
logger.LogCritical(ex, "Database corruption detected!");
|
||||
// Attempt recovery or alert admin
|
||||
}
|
||||
```
|
||||
|
||||
## Error Codes
|
||||
|
||||
| Code | Exception | Description |
|
||||
|------|-----------|-------------|
|
||||
| NETWORK_ERROR | NetworkException | Network operation failed |
|
||||
| PERSISTENCE_ERROR | PersistenceException | Database operation failed |
|
||||
| SYNC_ERROR | SyncException | Synchronization failed |
|
||||
| CONFIG_ERROR | ConfigurationException | Invalid configuration |
|
||||
| TIMEOUT_ERROR | TimeoutException | Operation timed out |
|
||||
|
||||
## Logging Levels
|
||||
|
||||
- **Trace**: Internal details (cache hits/misses)
|
||||
- **Debug**: Debugging info (sync operations)
|
||||
- **Information**: Normal events (peer discovered, backup created)
|
||||
- **Warning**: Recoverable errors (queue full, retry attempt, documents not found)
|
||||
- **Error**: Failures requiring attention (sync failed, corruption detected)
|
||||
- **Critical**: System failures (database initialization failed)
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always use structured logging**
|
||||
```csharp
|
||||
_logger.LogInformation("User {UserId} synced {Count} documents", userId, count);
|
||||
```
|
||||
|
||||
2. **Wrap network operations with retry policy**
|
||||
```csharp
|
||||
await _retryPolicy.ExecuteAsync(() => client.ConnectAsync(), "Connect");
|
||||
```
|
||||
|
||||
3. **Check cache before database**
|
||||
```csharp
|
||||
var doc = _cache.Get(collection, key) ?? await _store.GetDocumentAsync(collection, key);
|
||||
```
|
||||
|
||||
4. **Enable offline queue for LAN instability**
|
||||
```csharp
|
||||
if (options.Sync.EnableOfflineQueue && !isOnline)
|
||||
_offlineQueue.Enqueue(operation);
|
||||
```
|
||||
|
||||
5. **Periodic health checks**
|
||||
```csharp
|
||||
var timer = new Timer(async _ =>
|
||||
{
|
||||
var report = await healthCheckService.CheckHealthAsync(r => r.Name == "cbddc");
|
||||
var entry = report.Entries["cbddc"];
|
||||
if (entry.Status != HealthStatus.Healthy)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"CBDDC health is {Status}. LaggingPeers={LaggingPeers} UnconfirmedPeers={UnconfirmedPeers}",
|
||||
entry.Status,
|
||||
entry.Data["laggingPeers"],
|
||||
entry.Data["peersWithNoConfirmation"]);
|
||||
}
|
||||
}, null, TimeSpan.Zero, TimeSpan.FromMinutes(5));
|
||||
```
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
- [ ] Configuration file created (appsettings.json)
|
||||
- [ ] Log directory permissions set
|
||||
- [ ] Backup directory configured
|
||||
- [ ] Database file location specified
|
||||
- [ ] Network ports configured (firewall)
|
||||
- [ ] Health check endpoint tested
|
||||
- [ ] Offline queue tested
|
||||
- [ ] Backup/restore tested
|
||||
- [ ] Graceful shutdown tested
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Database corruption
|
||||
```csharp
|
||||
try
|
||||
{
|
||||
await store.CheckIntegrityAsync();
|
||||
}
|
||||
catch (DatabaseCorruptionException)
|
||||
{
|
||||
// Restore from backup
|
||||
File.Copy("backups/latest.db", options.Persistence.DatabasePath, overwrite: true);
|
||||
}
|
||||
```
|
||||
|
||||
### Network issues
|
||||
```
|
||||
Check sync tracker:
|
||||
- Last sync time
|
||||
- Active peers
|
||||
- Recent errors
|
||||
```
|
||||
|
||||
### Performance degradation
|
||||
```csharp
|
||||
var stats = cache.GetStatistics();
|
||||
if (stats.HitRate < 0.5)
|
||||
{
|
||||
// Consider increasing cache size
|
||||
options.Persistence.CacheSizeMb = 100;
|
||||
}
|
||||
```
|
||||
53
docs/querying.md
Executable file
53
docs/querying.md
Executable file
@@ -0,0 +1,53 @@
|
||||
---
|
||||
layout: default
|
||||
title: Querying
|
||||
---
|
||||
|
||||
# Querying
|
||||
|
||||
CBDDC allows querying local collections using a rich set of operators and idiomatic C# syntax, including LINQ support.
|
||||
|
||||
## Basic Querying
|
||||
|
||||
You can query documents using the `Find` method on a `PeerCollection`.
|
||||
|
||||
```csharp
|
||||
var users = await peerStore.GetCollection<User>("users");
|
||||
|
||||
// Precise match
|
||||
var fabio = await users.Find(u => u.FirstName == "Fabio");
|
||||
|
||||
// Comparisons
|
||||
var adults = await users.Find(u => u.Age >= 18);
|
||||
|
||||
// Logical Operators
|
||||
var activeAdmins = await users.Find(u => u.IsActive && u.Role == "Admin");
|
||||
```
|
||||
|
||||
## Serialization Consistency
|
||||
|
||||
CBDDC respects your configured serialization settings. If you use `snake_case` in your JSON serialization but standard C# PascalCase properties, the query translator automatically handles the mapping.
|
||||
|
||||
```csharp
|
||||
// Definition
|
||||
public class User
|
||||
{
|
||||
[JsonPropertyName("first_name")]
|
||||
public string FirstName { get; set; }
|
||||
}
|
||||
|
||||
// Query
|
||||
// This translates to a SQL query checking json_extract(data, '$.first_name')
|
||||
var result = await users.Find(u => u.FirstName == "Fabio");
|
||||
```
|
||||
|
||||
## Supported Operators
|
||||
|
||||
- `==` (Equal)
|
||||
- `!=` (Not Equal)
|
||||
- `>` (Greater Than)
|
||||
- `<` (Less Than)
|
||||
- `>=` (Greater Than or Equal)
|
||||
- `<=` (Less Than or Equal)
|
||||
- `&&` (AND)
|
||||
- `||` (OR)
|
||||
108
docs/remote-peer-configuration.md
Executable file
108
docs/remote-peer-configuration.md
Executable file
@@ -0,0 +1,108 @@
|
||||
# Remote Peer Configuration
|
||||
|
||||
Use `IPeerManagementService` to manage persistent remote peers and their pruning
|
||||
tracking lifecycle at runtime.
|
||||
|
||||
## Supported peer operations
|
||||
|
||||
- Add a static remote peer
|
||||
- Remove a configured peer and tracking
|
||||
- Remove tracking only (keep static peer config)
|
||||
- Enable/disable a peer
|
||||
- List all configured peers
|
||||
|
||||
## Add a static peer
|
||||
|
||||
```csharp
|
||||
await peerManagement.AddStaticPeerAsync(
|
||||
"branch-2",
|
||||
"branch2.example.com:9000",
|
||||
cancellationToken);
|
||||
```
|
||||
|
||||
## Peer lifecycle model
|
||||
|
||||
Peer management now has two independent dimensions:
|
||||
|
||||
- **Remote peer configuration**: whether a static peer exists in remote peer config.
|
||||
- **Peer confirmation tracking**: whether the peer participates in
|
||||
peer-confirmed prune gating.
|
||||
|
||||
Common states:
|
||||
|
||||
- **Enabled + tracked**: normal operation, peer is eligible for sync and prune gating.
|
||||
- **Disabled + tracked**: peer config retained, outbound static sync disabled, still
|
||||
counted as tracked until tracking is removed/deactivated.
|
||||
- **Enabled/disabled + untracked**: peer config may remain, but peer is excluded from
|
||||
prune confirmation gating.
|
||||
- **Removed**: peer config removed and tracking removed.
|
||||
|
||||
## Remove tracking only (deprecate peer from pruning)
|
||||
|
||||
Use this when a peer should no longer gate pruning but you are not removing static
|
||||
configuration yet.
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemovePeerTrackingAsync(
|
||||
nodeId: "branch-2",
|
||||
removeRemoteConfig: false,
|
||||
cancellationToken);
|
||||
```
|
||||
|
||||
## Remove a peer and tracking (full removal)
|
||||
|
||||
`RemoveRemotePeerAsync` performs full cleanup by removing both peer tracking and
|
||||
static peer configuration.
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemoveRemotePeerAsync("branch-2", cancellationToken);
|
||||
```
|
||||
|
||||
Equivalent explicit call:
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemovePeerTrackingAsync(
|
||||
nodeId: "branch-2",
|
||||
removeRemoteConfig: true,
|
||||
cancellationToken);
|
||||
```
|
||||
|
||||
## Enable or disable a peer
|
||||
|
||||
```csharp
|
||||
await peerManagement.DisablePeerAsync("branch-2", cancellationToken);
|
||||
await peerManagement.EnablePeerAsync("branch-2", cancellationToken);
|
||||
```
|
||||
|
||||
## List configured peers
|
||||
|
||||
```csharp
|
||||
var peers = await peerManagement.GetAllRemotePeersAsync(cancellationToken);
|
||||
|
||||
foreach (var peer in peers)
|
||||
{
|
||||
Console.WriteLine($"{peer.NodeId} @ {peer.Address} ({peer.Type}) Enabled={peer.IsEnabled}");
|
||||
}
|
||||
```
|
||||
|
||||
## Storage model
|
||||
|
||||
Remote peer configuration is persisted in the peer configuration store and synced across nodes as part of CBDDC metadata replication.
|
||||
|
||||
Fields stored per peer:
|
||||
- `NodeId`
|
||||
- `Address`
|
||||
- `Type`
|
||||
- `IsEnabled`
|
||||
- `InterestingCollections`
|
||||
|
||||
## Notes
|
||||
|
||||
- Authentication for sync handshakes is based on the cluster shared token (`AuthToken`) from node configuration.
|
||||
- Disabled peers remain persisted but are excluded from active sync.
|
||||
- Peer tracking removal is implemented as deactivation in the confirmation store,
|
||||
which removes the peer from active prune-gating.
|
||||
- Re-observed peers can be re-registered and become active for tracking again.
|
||||
- For rollout steps and production operations, see:
|
||||
- [Upgrade: Peer-Confirmed Pruning](upgrade-peer-confirmed-pruning.html)
|
||||
- [Peer Deprecation & Removal Runbook](peer-deprecation-removal-runbook.html)
|
||||
145
docs/security.md
Executable file
145
docs/security.md
Executable file
@@ -0,0 +1,145 @@
|
||||
# Network Security
|
||||
|
||||
**CBDDC v0.6.0** introduces optional **secure networking** to protect data in transit between peers using industry-standard cryptography.
|
||||
|
||||
## Overview
|
||||
|
||||
The security layer provides:
|
||||
- **ECDH (Elliptic Curve Diffie-Hellman)** key exchange for establishing shared secrets
|
||||
- **AES-256-CBC** encryption for all synchronized data
|
||||
- **HMAC-SHA256** authentication to prevent tampering
|
||||
- **Perfect Forward Secrecy** - each session uses unique ephemeral keys
|
||||
|
||||
## When to Use Secure Networking
|
||||
|
||||
### ✅ Recommended For:
|
||||
- **Sensitive data** (customer information, financial records, health data)
|
||||
- **Compliance requirements** (GDPR, HIPAA, PCI-DSS)
|
||||
- **Untrusted network segments** within your LAN
|
||||
- **Production deployments** where data confidentiality is important
|
||||
|
||||
### ⚠️ Consider Performance Impact:
|
||||
- Encryption adds ~5-10ms latency per message
|
||||
- CPU overhead for encryption/decryption
|
||||
- Not necessary for public/non-sensitive data
|
||||
|
||||
## How It Works
|
||||
|
||||
### Handshake Process
|
||||
|
||||
```
|
||||
Peer A Peer B
|
||||
| |
|
||||
| 1. Generate ECDH keypair |
|
||||
|------ Public Key A ---------------->| 2. Generate ECDH keypair
|
||||
| |
|
||||
|<----- Public Key B -----------------|
|
||||
| |
|
||||
3. Compute shared secret 3. Compute shared secret
|
||||
4. Derive encryption keys 4. Derive encryption keys
|
||||
| |
|
||||
|==== Encrypted Communication ====|
|
||||
```
|
||||
|
||||
### Encryption Details
|
||||
|
||||
1. **Key Exchange**: NIST P-256 elliptic curve (secp256r1)
|
||||
2. **Encryption**: AES-256 in CBC mode with random IV per message
|
||||
3. **Authentication**: HMAC-SHA256 over ciphertext
|
||||
4. **Message Format**: `IV (16 bytes) + Ciphertext + HMAC (32 bytes)`
|
||||
|
||||
## Usage
|
||||
|
||||
### Enable Security
|
||||
|
||||
```csharp
|
||||
using CBDDC.Network.Security;
|
||||
|
||||
// Register secure handshake service
|
||||
services.AddSingleton<IPeerHandshakeService, SecureHandshakeService>();
|
||||
|
||||
// Network will automatically use encryption if service is registered
|
||||
services.AddCBDDCNetwork(nodeId, tcpPort, authToken);
|
||||
```
|
||||
|
||||
### Console Sample (--secure flag)
|
||||
|
||||
```bash
|
||||
# Start with encryption enabled
|
||||
dotnet run --secure
|
||||
|
||||
# Or without encryption (faster, less secure)
|
||||
dotnet run
|
||||
```
|
||||
|
||||
### UI Clients
|
||||
|
||||
If you build a UI client, enable secure mode by default to avoid mixed-security clusters.
|
||||
|
||||
### Verify Security Status
|
||||
|
||||
UI clients should display security status:
|
||||
- **🔒 Encrypted** - Secure communication active
|
||||
- **🔓 Plaintext** - No encryption (Console sample default)
|
||||
|
||||
## Configuration
|
||||
|
||||
### Optional: Custom Security Settings
|
||||
|
||||
Currently, security uses default secure parameters. Future versions may support:
|
||||
- Custom curve selection
|
||||
- Cipher suite configuration
|
||||
- Certificate pinning
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### ✅ What This Protects Against:
|
||||
- **Eavesdropping**: Data is encrypted in transit
|
||||
- **Tampering**: HMAC prevents message modification
|
||||
- **Man-in-the-Middle**: ECDH provides authenticity
|
||||
|
||||
### ⚠️ Out of Scope:
|
||||
- **Data at Rest**: SQLite databases are NOT encrypted. Use OS-level encryption if needed.
|
||||
- **Authentication**: No peer identity verification beyond shared auth token.
|
||||
- **Public Internet**: Still designed for trusted LANs. Use VPN/firewall for internet exposure.
|
||||
|
||||
## Performance Impact
|
||||
|
||||
Benchmarks on typical workloads:
|
||||
|
||||
| Operation | Plaintext | Encrypted | Overhead |
|
||||
|-----------|-----------|-----------|----------|
|
||||
| Small sync (10 docs) | 15ms | 22ms | +47% |
|
||||
| Large sync (1000 docs) | 450ms | 520ms | +16% |
|
||||
| Handshake | N/A | 8ms | Initial |
|
||||
|
||||
**Recommendation**: Enable for production. Disable for development/testing if needed.
|
||||
|
||||
## Compatibility
|
||||
|
||||
- **Secure ↔ Secure**: ✅ Works
|
||||
- **Plaintext ↔ Plaintext**: ✅ Works
|
||||
- **Secure ↔ Plaintext**: ❌ Connection fails (by design)
|
||||
|
||||
All nodes in a network must use the **same security mode**.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I use this over the public internet?**
|
||||
A: While encryption helps, CBDDC is still designed for LANs. Use VPN, firewall rules, and consider TLS/SSL wrapping for internet exposure.
|
||||
|
||||
**Q: How do I encrypt the SQLite database?**
|
||||
A: Use SQLCipher or OS-level disk encryption (BitLocker, LUKS, FileVault).
|
||||
|
||||
**Q: Can different nodes use different keys?**
|
||||
A: No. All nodes in a mesh share the same `authToken`. Key rotation is not yet supported.
|
||||
|
||||
**Q: What about .NET Standard 2.0 support?**
|
||||
A: Security works on all target frameworks (netstandard2.0, net6.0, net8.0) with appropriate polyfills.
|
||||
|
||||
---
|
||||
|
||||
**See Also:**
|
||||
- [Getting Started](getting-started.html)
|
||||
- [Architecture](architecture.html)
|
||||
- [Production Hardening](production-hardening.html)
|
||||
69
docs/upgrade-peer-confirmed-pruning.md
Normal file
69
docs/upgrade-peer-confirmed-pruning.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# Upgrade Notes: Peer-Confirmed Pruning
|
||||
|
||||
This guide covers adopting peer-confirmed pruning semantics introduced across
|
||||
Phases 1-4.
|
||||
|
||||
## What changed
|
||||
|
||||
1. Oplog pruning now uses an **effective cutoff**:
|
||||
- `min(retention cutoff, confirmation cutoff)` when peer confirmations are complete.
|
||||
- prune is skipped for that cycle when an active tracked peer is missing required confirmation.
|
||||
2. Peer tracking is now a managed lifecycle:
|
||||
- `RemovePeerTrackingAsync(nodeId, removeRemoteConfig: false)` deprecates a peer from prune gating.
|
||||
- `RemoveRemotePeerAsync(nodeId)` removes both static peer config and tracking.
|
||||
3. Hosting health now includes confirmation lag semantics:
|
||||
- `Degraded` for lagging or unconfirmed tracked peers.
|
||||
- `Unhealthy` for critical lag or storage failures.
|
||||
|
||||
## Upgrade impact to expect
|
||||
|
||||
- During initial rollout, a peer may appear in `peersWithNoConfirmation` until the
|
||||
first successful confirmation update.
|
||||
- Any stale active tracked peer can block prune progress and/or keep health degraded.
|
||||
|
||||
## Recommended rollout sequence
|
||||
|
||||
1. Upgrade one node and validate health payload and pruning logs.
|
||||
2. Upgrade remaining nodes in the cluster.
|
||||
3. Audit peer inventory and remove/deprecate stale peers.
|
||||
4. Tune lag thresholds after observing normal confirmation latency.
|
||||
|
||||
## Peer inventory and cleanup
|
||||
|
||||
List configured peers:
|
||||
|
||||
```csharp
|
||||
var peers = await peerManagement.GetAllRemotePeersAsync(cancellationToken);
|
||||
```
|
||||
|
||||
Deprecate from pruning only:
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemovePeerTrackingAsync(
|
||||
nodeId: "retired-peer",
|
||||
removeRemoteConfig: false,
|
||||
cancellationToken);
|
||||
```
|
||||
|
||||
Fully remove peer + tracking:
|
||||
|
||||
```csharp
|
||||
await peerManagement.RemoveRemotePeerAsync("retired-peer", cancellationToken);
|
||||
```
|
||||
|
||||
## Validation checklist
|
||||
|
||||
- `/health` returns `Healthy` or expected transient `Degraded` during warm-up.
|
||||
- `laggingPeers` and `peersWithNoConfirmation` converge toward zero for active peers.
|
||||
- Maintenance logs no longer report prune skip reasons for retired peers.
|
||||
|
||||
## Rollback/mitigation
|
||||
|
||||
If rollout exposes unexpected persistent degradation:
|
||||
|
||||
1. Remove tracking for permanently retired peers.
|
||||
2. Temporarily raise lag thresholds to reduce alert noise while investigating.
|
||||
3. Keep full peer removal for nodes that are confirmed decommissioned.
|
||||
|
||||
For a detailed operator procedure, see
|
||||
[Peer Deprecation & Removal Runbook](peer-deprecation-removal-runbook.html).
|
||||
Reference in New Issue
Block a user