Joseph Doherty 10d076a9da test(localdb): sustained concurrent write load through pooled connections
Closes criterion 5 of ScadaBridge/docs/known-issues/2026-07-20-localdb-disk-io-
error-under-load.md §9: Phase 1 shipped only correctness and convergence tests,
so nothing exercised many pooled per-operation connections writing a REGISTERED
table (capture trigger + zb_hlc_next UDF) concurrently — which is exactly the
shape the ScadaBridge site nodes run under load.

8 writers x 250 inserts, one fresh pooled connection per operation, with
concurrent readers churning the pool and the WAL index throughout. Asserts both
that every write lands and that the oplog captured each exactly once, so a
trigger that dropped or double-counted under contention fails it.

Written during the 2026-07-20 disk-I/O incident investigation and left
uncommitted; the incident itself was root-caused to host-side sqlite3 access on
a bind-mounted WAL database, not a library defect.

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
2026-07-20 05:01:57 -04:00

scadaproj

Umbrella workspace for a family of related SCADA / OT (operational-technology) projects built around AVEVA System Platform (Wonderware), OPC UA, and Akka.NET.

This repository is part index — a map of the sister projects, which are their own repositories — and part home for cross-project normalization work and the shared code it produces.

Working in here with Claude Code? See CLAUDE.md for the machine-oriented index and guidance. This README is the human-facing overview.

What's in here

Path What it is
CLAUDE.md High-level index of the sister projects + working guidance
components/ Component-normalization framework (per concern: target spec, current state, gaps) — 7 components
docs/plans/ Implementation + design plans (incl. Galaxy Repository and LocalDb, which have no components/ entry)
infra/glauth/ The shared GLAuth every app's dev/test LDAP points at (10.100.0.35:3893)
ZB.MOM.WW.Auth/ Built shared library (.NET 10) — login / identity / API keys
ZB.MOM.WW.Theme/ Built shared RCL (.NET 10) — Technical-Light design system
ZB.MOM.WW.Audit/ Built shared library — canonical audit event + writer seam
ZB.MOM.WW.Health/ Built shared library — readiness / liveness / active-node probes
ZB.MOM.WW.Telemetry/ Built shared library — OTel metrics / traces + Serilog
ZB.MOM.WW.Configuration/ Built shared library — options validation + startup preflight
ZB.MOM.WW.Secrets/ Built shared library — AES-256-GCM secret store + ${secret:} resolution + CLI + SQL-Server / Akka cluster replication
ZB.MOM.WW.GalaxyRepository/ Built shared library — Galaxy object-hierarchy SQL browse + gRPC service
ZB.MOM.WW.LocalDb/ Built shared library — embedded SQLite cache + 2-node gRPC replication

The sister projects

Independent repositories, coupled only at runtime over wire protocols (gRPC + OPC UA) — not by a shared build. scadaproj indexes them; it does not contain them.

  • OtOpcUa (lmxopcua) — OPC UA server (.NET 10) that republishes Wonderware Galaxy tags as an OPC UA address space.
  • MxAccessGateway (mxaccessgw) — gRPC gateway giving modern x64 / .NET-10 clients full MXAccess parity without loading 32-bit COM. The linchpin both others depend on.
  • ScadaBridge — distributed SCADA platform on Akka.NET (hub-and-spoke: one central cluster
    • N site clusters); its Data Connection Layer ingests OPC UA and MxAccess.
  • HistorianGateway (historiangw) — gRPC sidecar over the AVEVA Historian (read/write) plus read-only Galaxy browse; no COM, no x86 worker. OtOpcUa's sole historian backend.

Data flow:

Wonderware Galaxy ──gRPC──▶ mxaccessgw ──▶ OtOpcUa (republishes as OPC UA) ─┐
                                       └────────────────────────────────────┴─▶ ScadaBridge clusters
                                          (or ScadaBridge's MxGateway adapter, direct)

Full relationship map: CLAUDE.md.

Component normalization

The sister repos kept re-implementing the same cross-cutting concerns and drifting apart. components/ captures, per component, the one target spec, each project's code-verified current state, and a gaps / adoption backlog. Convention + workflow in components/README.md.

Status verified 2026-07-18 against the feed + consumer branches (see Roadmap). "All four" = OtOpcUa, MxAccessGateway, ScadaBridge, HistorianGateway.

Component Feed Adoption Folder
Auth (login / identity / authz) 0.1.00.1.5 mxaccessgw / ScadaBridge / HistorianGateway @ 0.1.5; OtOpcUa @ 0.1.1 components/auth/
UI Theme (layout / tokens / components) 0.2.00.3.1 all four @ 0.3.1 components/ui-theme/
Audit (event model + writer seam) 0.1.0 all four (DEEP) components/audit/
Config + validation 0.1.0 all four components/configuration/
Observability (metrics / traces / logs) 0.1.0 all four components/observability/
Health (readiness / liveness / active-node) 0.1.0 all four components/health/
Secrets (encrypted store + ${secret:}) 0.1.00.2.0 all four @ 0.1.2 (0.1.3/0.2.0 published, not yet consumed) components/secrets/
Galaxy Repository (SQL browse + gRPC) 0.1.0, 0.2.0 HistorianGateway + mxaccessgw (design in docs/plans/)
LocalDb (embedded cache + 2-node sync) 0.1.0 none yet (design in docs/plans/)

ZB.MOM.WW.Theme — the shared UI kit

The UI-theme component, realized as a single-package .NET 10 Razor Class Library that replaced each app's copy-pasted Technical-Light design system. Adopted by all four apps, currently on 0.3.1 (feed carries 0.2.00.3.1); adoption history in components/ui-theme/GAPS.md.

Asset / Component Purpose Used by
_content/ZB.MOM.WW.Theme/css/theme.css Design tokens, IBM Plex typography, Bootstrap overrides all
_content/ZB.MOM.WW.Theme/css/layout.css Side-rail shell layout, nav CSS, chip/card helpers all
_content/ZB.MOM.WW.Theme/fonts/*.woff2 IBM Plex Sans 400/600 + Mono 500, vendored all
ThemeHead, ThemeShell, BrandBar Shell entry point and chassis components all
NavRailItem, NavRailSection Rail nav components all
StatusPill (StatusState) Inline status chip — replaces per-app StatusBadge all
LoginCard Static form-POST sign-in card OtOpcUa, ScadaBridge (MxGateway when login page added)
TechButton, TechCard, TechField Common controls (Bootstrap 5 wrappers) all

Consumer matrix: all four apps consume the single ZB.MOM.WW.Theme package — OtOpcUa AdminUI, MxAccessGateway Server, ScadaBridge Host + CentralUI, HistorianGateway Server.

Build & test

cd ZB.MOM.WW.Theme
dotnet build -c Release   # 0 warnings (TreatWarningsAsErrors)
dotnet test               # 48 bUnit tests (verified 2026-07-18)
./build/pack.sh           # → ./artifacts/ZB.MOM.WW.Theme.<version>.nupkg (feed is at 0.3.1)

Stack: .NET 10 · Razor Class Library · bUnit · xUnit · central package management. More detail: ZB.MOM.WW.Theme/README.md.


ZB.MOM.WW.Auth — the shared library

The auth component, realized as a four-package .NET 10 library that replaced each app's re-implemented authentication. Adopted by all four apps — pins drift deliberately (OtOpcUa 0.1.1, ScadaBridge 0.1.3, MxAccessGateway + HistorianGateway 0.1.4); history in components/auth/GAPS.md.

Package Purpose Used by
ZB.MOM.WW.Auth.Abstractions Contracts, CanonicalRole, API-key types (zero deps) all
ZB.MOM.WW.Auth.Ldap Bind-then-search LDAP authentication (fail-closed) all
ZB.MOM.WW.Auth.ApiKeys Peppered-HMAC API keys + SQLite store + verifier + admin commands MxAccessGateway, ScadaBridge
ZB.MOM.WW.Auth.AspNetCore Canonical claim types, cookie defaults, LDAP DI helpers all (web UIs)

Consumer matrix: OtOpcUa → Abstractions + Ldap + AspNetCore (no ApiKeys / SQLite); MxAccessGateway, ScadaBridge & HistorianGateway → all four packages.

Identity (LDAP / GLAuth, the canonical role set) is normalized; each app keeps its own authorization enforcement (OPC-UA permissions / gRPC scopes / roles + site-scoping) and maps it onto the canonical roles. Design: components/auth/spec/SPEC.md and CANONICAL-ROLES.md.

Build & test

cd ZB.MOM.WW.Auth
dotnet build -c Release                 # 0 warnings
dotnet test                             # 215 passing, 1 skipped (opt-in LDAP integration test) — verified 2026-07-18
dotnet pack  -c Release -o ./artifacts  # → 4 nupkgs (feed is at 0.1.4)

Stack: .NET 10 · xUnit · Novell.Directory.Ldap.NETStandard · Microsoft.Data.Sqlite · central package management. The packages are libraries linked into each consumer — there is no central auth service. More detail: ZB.MOM.WW.Auth/README.md.

The optional LDAP integration test is opt-in:

ZB_LDAP_IT=1 dotnet test   # requires a reachable GLAuth (e.g. a sister repo's infra/glauth)

Roadmap

Verified 2026-07-18 against the Gitea feed (nuget/query?q=ZB.MOM.WW) and the actual PackageReferences on each consumer's default branch — not from status claims in docs. Per-component detail in each components/<name>/GAPS.md.

Shipped

  • Auth normalized + ZB.MOM.WW.Auth built (feed: 0.1.00.1.5) and adopted by all four apps. 0.1.5 (2026-07-18) pins patched SQLitePCLRaw; mxaccessgw, ScadaBridge and HistorianGateway bumped to it. OtOpcUa stays at 0.1.1 — it takes no ApiKeys/SQLite dependency, so it was never exposed.
  • UI Theme normalized + ZB.MOM.WW.Theme built (feed: 0.2.00.3.1); all four apps on 0.3.1.
  • Audit — ZB.MOM.WW.Audit 0.1.0 on the feed; adopted by all four (DEEP: canonical record everywhere).
  • Configuration — ZB.MOM.WW.Configuration 0.1.0 on the feed; adopted by all four.
  • Observability — ZB.MOM.WW.Telemetry + .Serilog 0.1.0 on the feed; adopted by all four.
  • Health — ZB.MOM.WW.Health (+ .Akka, .EntityFrameworkCore) 0.1.0 on the feed; adopted by all four (MapZbHealth wired in each; previously documented as "not yet adopted" — that was stale).
  • Galaxy Repository — ZB.MOM.WW.GalaxyRepository 0.2.0 on the feed; consumed by HistorianGateway and mxaccessgw (which wires AddZbGalaxyRepository in its Server; previously documented as a pending follow-on — also stale).
  • Secrets — ZB.MOM.WW.Secrets (+ .Abstractions, .Ui) published through 0.1.3 (KEK rotation, 2026-07-18); all four apps still pinned at 0.1.2.
  • Secrets G-7 clustered replication — built 2026-07-18 at 0.2.0, both fork options: ZB.MOM.WW.Secrets.Replicator.SqlServer (shared store and local-plus-hub) and ZB.MOM.WW.Secrets.Replicator.AkkaDotNet (peer-to-peer, partition-tolerant). Verified against a real SQL Server 2022 and an in-process 2-node Akka cluster. Published to the feed 2026-07-18 (restore-verified).
  • LocalDb — ZB.MOM.WW.LocalDb (+ .Contracts, .Replication) built at 0.1.0 and published to the feed 2026-07-18 (restore-verified from a scratch consumer).

Open

  • Adopt a clustered secrets topology in ScadaBridge / OtOpcUa — config + a shared KEK, no code change. 0.2.0 is on the feed. components/secrets/GAPS.md.
  • Bump the four Secrets consumers 0.1.20.1.3/0.2.0 (KEK rotation + the same security fix).
  • Clear the pre-existing AngleSharp NU1902 build break in ScadaBridge CentralUI.Tests and HistorianGateway Tests — both fail full-solution builds under TreatWarningsAsErrors.
  • Adopt ZB.MOM.WW.LocalDb in an app — no consumer references it yet.
  • Give LocalDb a components/localdb/ entry; it is the one shared lib whose design lives only in docs/plans/2026-07-17-localdb-design.md.
  • Converge the Auth version pins (OtOpcUa is three minors behind at 0.1.1).
  • Normalize the next cross-cutting component.
S
Description
No description provided
Readme 4.9 MiB
Languages
C# 92.9%
Python 3%
HTML 2.2%
CSS 1.4%
Shell 0.3%
Other 0.2%