Joseph Doherty
cea3930b94
Baselining one table used to set a single per-DATABASE needs_snapshot bit, and a snapshot streams every registered table in full — so opting one table into replication late re-sent all of them, in both directions at once on a pair enabling replication late. On a database holding a high-volume table (an event log at its retention cap) that is the largest transfer this library performs. 0.2.1 documented the cost and deferred the fix as needing an on-disk change. Schema v3 adds __localdb_snapshot_state (table_name PK, needs_snapshot), created by the shared DDL with IF NOT EXISTS so an older file gains it on open; the v2->v3 transition is a version stamp only. RegisterReplicated(..., baselineExistingRows: true) now flags that table alone. The database-wide flag survives unchanged for the paths that genuinely reach every table — a cap prune or an impossible peer ack loses delta continuity for the oplog as a whole. DeregisterReplicated drops a table's pending flag with its ledger rows, since nothing would ever stream (and so clear) it afterwards. Wire compatibility is additive and negotiated by CAPABILITY, not version: lib_schema_version stays 1 because ValidateHandshake compares it fail-closed for equality, so bumping it would stop 0.1.x/0.2.x pairs syncing outright. Handshake gains supports_partial_snapshot (field 5) and SnapshotBegin gains a tables scope (field 2) where EMPTY means all tables — precisely what a pre-0.3.0 sender emits and what a pre-0.3.0 receiver assumes. A peer that does not advertise the capability is served a full snapshot: a scoped one would make it advance last_applied_remote_seq to the snapshot's as-of and silently discard every delta below it for the tables left out. For the same reason a partial snapshot moves no watermark on either side. The receiver keeps last_applied_remote_seq (carrying only the HLC across, which is MAX-guarded), and the sender keeps _sentThruSeq, so the deltas the snapshot overlaps are still pumped and merged by LWW. Crash safety: snapshot debt now clears on the receiver's new SnapshotAck rather than on enqueue, so an ungraceful exit part-way through re-streams the same scope on reconnect. The ack rides the unbounded CONTROL lane like DeltaAck, so it composes with 0.2.1's bounded 64-message inbox and its out-of-handshake snapshot. Against a pre-0.3.0 peer the 0.2.x clear-on-enqueue behaviour is kept — there is no ack to wait for — with the handshake's horizon gap check as the net. Tests 199/199 (was 188): single-table baseline streams only that table, mixed- version interop (peer without the capability gets a full snapshot), debt standing until SnapshotAck, partial snapshot not advancing the receiver's watermark, no SnapshotAck emitted to an old peer, end-to-end over two real databases, plus v2->v3 upgrade and per-table flag isolation. README, ILocalDb remarks (which named this as the follow-up) and the umbrella CLAUDE.md index updated.
scadaproj
Umbrella workspace for a family of related SCADA / OT (operational-technology) projects built around AVEVA System Platform (Wonderware), OPC UA, and Akka.NET.
This repository is part index — a map of the sister projects, which are their own repositories — and part home for cross-project normalization work and the shared code it produces.
Working in here with Claude Code? See
CLAUDE.mdfor the machine-oriented index and guidance. This README is the human-facing overview.
What's in here
| Path | What it is |
|---|---|
CLAUDE.md |
High-level index of the sister projects + working guidance |
components/ |
Component-normalization framework (per concern: target spec, current state, gaps) — 7 components |
docs/plans/ |
Implementation + design plans (incl. Galaxy Repository and LocalDb, which have no components/ entry) |
infra/glauth/ |
The shared GLAuth every app's dev/test LDAP points at (10.100.0.35:3893) |
ZB.MOM.WW.Auth/ |
Built shared library (.NET 10) — login / identity / API keys |
ZB.MOM.WW.Theme/ |
Built shared RCL (.NET 10) — Technical-Light design system |
ZB.MOM.WW.Audit/ |
Built shared library — canonical audit event + writer seam |
ZB.MOM.WW.Health/ |
Built shared library — readiness / liveness / active-node probes |
ZB.MOM.WW.Telemetry/ |
Built shared library — OTel metrics / traces + Serilog |
ZB.MOM.WW.Configuration/ |
Built shared library — options validation + startup preflight |
ZB.MOM.WW.Secrets/ |
Built shared library — AES-256-GCM secret store + ${secret:} resolution + CLI + SQL-Server / Akka cluster replication |
ZB.MOM.WW.GalaxyRepository/ |
Built shared library — Galaxy object-hierarchy SQL browse + gRPC service |
ZB.MOM.WW.LocalDb/ |
Built shared library — embedded SQLite cache + 2-node gRPC replication |
The sister projects
Independent repositories, coupled only at runtime over wire protocols (gRPC + OPC UA) — not by a shared build. scadaproj indexes them; it does not contain them.
- OtOpcUa (
lmxopcua) — OPC UA server (.NET 10) that republishes Wonderware Galaxy tags as an OPC UA address space. - MxAccessGateway (
mxaccessgw) — gRPC gateway giving modern x64 / .NET-10 clients full MXAccess parity without loading 32-bit COM. The linchpin both others depend on. - ScadaBridge — distributed SCADA platform on Akka.NET (hub-and-spoke: one central cluster
- N site clusters); its Data Connection Layer ingests OPC UA and MxAccess.
- HistorianGateway (
historiangw) — gRPC sidecar over the AVEVA Historian (read/write) plus read-only Galaxy browse; no COM, no x86 worker. OtOpcUa's sole historian backend.
Data flow:
Wonderware Galaxy ──gRPC──▶ mxaccessgw ──▶ OtOpcUa (republishes as OPC UA) ─┐
└────────────────────────────────────┴─▶ ScadaBridge clusters
(or ScadaBridge's MxGateway adapter, direct)
Full relationship map: CLAUDE.md.
Component normalization
The sister repos kept re-implementing the same cross-cutting concerns and drifting apart.
components/ captures, per component, the one target spec, each project's
code-verified current state, and a gaps / adoption backlog. Convention + workflow in
components/README.md.
Status verified 2026-07-18 against the feed + consumer branches (see Roadmap). "All four" = OtOpcUa, MxAccessGateway, ScadaBridge, HistorianGateway.
| Component | Feed | Adoption | Folder |
|---|---|---|---|
| Auth (login / identity / authz) | 0.1.0–0.1.5 |
mxaccessgw / ScadaBridge / HistorianGateway @ 0.1.5; OtOpcUa @ 0.1.1 |
components/auth/ |
| UI Theme (layout / tokens / components) | 0.2.0–0.3.1 |
all four @ 0.3.1 |
components/ui-theme/ |
| Audit (event model + writer seam) | 0.1.0 |
all four (DEEP) | components/audit/ |
| Config + validation | 0.1.0 |
all four | components/configuration/ |
| Observability (metrics / traces / logs) | 0.1.0 |
all four | components/observability/ |
| Health (readiness / liveness / active-node) | 0.1.0 |
all four | components/health/ |
Secrets (encrypted store + ${secret:}) |
0.1.0–0.2.0 |
all four @ 0.1.2 (0.1.3/0.2.0 published, not yet consumed) |
components/secrets/ |
| Galaxy Repository (SQL browse + gRPC) | 0.1.0, 0.2.0 |
HistorianGateway + mxaccessgw | (design in docs/plans/) |
| LocalDb (embedded cache + 2-node sync) | 0.1.0 |
none yet | (design in docs/plans/) |
ZB.MOM.WW.Theme — the shared UI kit
The UI-theme component, realized as a single-package .NET 10 Razor Class Library that replaced
each app's copy-pasted Technical-Light design system. Adopted by all four apps, currently on
0.3.1 (feed carries 0.2.0–0.3.1); adoption history in
components/ui-theme/GAPS.md.
| Asset / Component | Purpose | Used by |
|---|---|---|
_content/ZB.MOM.WW.Theme/css/theme.css |
Design tokens, IBM Plex typography, Bootstrap overrides | all |
_content/ZB.MOM.WW.Theme/css/layout.css |
Side-rail shell layout, nav CSS, chip/card helpers | all |
_content/ZB.MOM.WW.Theme/fonts/*.woff2 |
IBM Plex Sans 400/600 + Mono 500, vendored | all |
ThemeHead, ThemeShell, BrandBar |
Shell entry point and chassis components | all |
NavRailItem, NavRailSection |
Rail nav components | all |
StatusPill (StatusState) |
Inline status chip — replaces per-app StatusBadge |
all |
LoginCard |
Static form-POST sign-in card | OtOpcUa, ScadaBridge (MxGateway when login page added) |
TechButton, TechCard, TechField |
Common controls (Bootstrap 5 wrappers) | all |
Consumer matrix: all four apps consume the single ZB.MOM.WW.Theme package —
OtOpcUa AdminUI, MxAccessGateway Server, ScadaBridge Host + CentralUI, HistorianGateway Server.
Build & test
cd ZB.MOM.WW.Theme
dotnet build -c Release # 0 warnings (TreatWarningsAsErrors)
dotnet test # 48 bUnit tests (verified 2026-07-18)
./build/pack.sh # → ./artifacts/ZB.MOM.WW.Theme.<version>.nupkg (feed is at 0.3.1)
Stack: .NET 10 · Razor Class Library · bUnit · xUnit · central package management.
More detail: ZB.MOM.WW.Theme/README.md.
ZB.MOM.WW.Auth — the shared library
The auth component, realized as a four-package .NET 10 library that replaced each app's
re-implemented authentication. Adopted by all four apps — pins drift deliberately
(OtOpcUa 0.1.1, ScadaBridge 0.1.3, MxAccessGateway + HistorianGateway 0.1.4); history in
components/auth/GAPS.md.
| Package | Purpose | Used by |
|---|---|---|
ZB.MOM.WW.Auth.Abstractions |
Contracts, CanonicalRole, API-key types (zero deps) |
all |
ZB.MOM.WW.Auth.Ldap |
Bind-then-search LDAP authentication (fail-closed) | all |
ZB.MOM.WW.Auth.ApiKeys |
Peppered-HMAC API keys + SQLite store + verifier + admin commands | MxAccessGateway, ScadaBridge |
ZB.MOM.WW.Auth.AspNetCore |
Canonical claim types, cookie defaults, LDAP DI helpers | all (web UIs) |
Consumer matrix: OtOpcUa → Abstractions + Ldap + AspNetCore (no ApiKeys / SQLite);
MxAccessGateway, ScadaBridge & HistorianGateway → all four packages.
Identity (LDAP / GLAuth, the canonical role set) is normalized; each app keeps its own
authorization enforcement (OPC-UA permissions / gRPC scopes / roles + site-scoping) and maps
it onto the canonical roles. Design: components/auth/spec/SPEC.md
and CANONICAL-ROLES.md.
Build & test
cd ZB.MOM.WW.Auth
dotnet build -c Release # 0 warnings
dotnet test # 215 passing, 1 skipped (opt-in LDAP integration test) — verified 2026-07-18
dotnet pack -c Release -o ./artifacts # → 4 nupkgs (feed is at 0.1.4)
Stack: .NET 10 · xUnit · Novell.Directory.Ldap.NETStandard · Microsoft.Data.Sqlite · central
package management. The packages are libraries linked into each consumer — there is no
central auth service. More detail: ZB.MOM.WW.Auth/README.md.
The optional LDAP integration test is opt-in:
ZB_LDAP_IT=1 dotnet test # requires a reachable GLAuth (e.g. a sister repo's infra/glauth)
Roadmap
Verified 2026-07-18 against the Gitea feed (
nuget/query?q=ZB.MOM.WW) and the actualPackageReferences on each consumer's default branch — not from status claims in docs. Per-component detail in eachcomponents/<name>/GAPS.md.
Shipped
- ✅ Auth normalized +
ZB.MOM.WW.Authbuilt (feed:0.1.0–0.1.5) and adopted by all four apps.0.1.5(2026-07-18) pins patched SQLitePCLRaw; mxaccessgw, ScadaBridge and HistorianGateway bumped to it. OtOpcUa stays at0.1.1— it takes noApiKeys/SQLite dependency, so it was never exposed. - ✅ UI Theme normalized +
ZB.MOM.WW.Themebuilt (feed:0.2.0–0.3.1); all four apps on0.3.1. - ✅ Audit —
ZB.MOM.WW.Audit0.1.0on the feed; adopted by all four (DEEP: canonical record everywhere). - ✅ Configuration —
ZB.MOM.WW.Configuration0.1.0on the feed; adopted by all four. - ✅ Observability —
ZB.MOM.WW.Telemetry+.Serilog0.1.0on the feed; adopted by all four. - ✅ Health —
ZB.MOM.WW.Health(+.Akka,.EntityFrameworkCore)0.1.0on the feed; adopted by all four (MapZbHealthwired in each; previously documented as "not yet adopted" — that was stale). - ✅ Galaxy Repository —
ZB.MOM.WW.GalaxyRepository0.2.0on the feed; consumed by HistorianGateway and mxaccessgw (which wiresAddZbGalaxyRepositoryin its Server; previously documented as a pending follow-on — also stale). - ✅ Secrets —
ZB.MOM.WW.Secrets(+.Abstractions,.Ui) published through0.1.3(KEK rotation, 2026-07-18); all four apps still pinned at0.1.2. - ✅ Secrets G-7 clustered replication — built 2026-07-18 at
0.2.0, both fork options:ZB.MOM.WW.Secrets.Replicator.SqlServer(shared store and local-plus-hub) andZB.MOM.WW.Secrets.Replicator.AkkaDotNet(peer-to-peer, partition-tolerant). Verified against a real SQL Server 2022 and an in-process 2-node Akka cluster. Published to the feed 2026-07-18 (restore-verified). - ✅ LocalDb —
ZB.MOM.WW.LocalDb(+.Contracts,.Replication) built at0.1.0and published to the feed 2026-07-18 (restore-verified from a scratch consumer).
Open
- ⬜ Adopt a clustered secrets topology in ScadaBridge / OtOpcUa — config + a shared KEK, no code
change.
0.2.0is on the feed.components/secrets/GAPS.md. - ⬜ Bump the four Secrets consumers
0.1.2→0.1.3/0.2.0(KEK rotation + the same security fix). - ⬜ Clear the pre-existing
AngleSharpNU1902 build break in ScadaBridgeCentralUI.Testsand HistorianGatewayTests— both fail full-solution builds underTreatWarningsAsErrors. - ⬜ Adopt
ZB.MOM.WW.LocalDbin an app — no consumer references it yet. - ⬜ Give LocalDb a
components/localdb/entry; it is the one shared lib whose design lives only indocs/plans/2026-07-17-localdb-design.md. - ⬜ Converge the Auth version pins (OtOpcUa is three minors behind at
0.1.1). - ⬜ Normalize the next cross-cutting component.