dohertj2/lmxopcua
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(mesh): rig seed + Phase 1 documentation

Browse Source 
- docker-dev seed: AkkaPort = 4053 on all six ClusterNode rows, so a freshly
  seeded rig matches a migrated one. GrpcPort left null.
- config-db-schema.md: both columns, why AkkaPort is NOT NULL/4053 and GrpcPort
  is nullable, the unenforced duplication + its reconciler, and Enabled's new
  second meaning as the deploy path's expected-ack set.
- Configuration.md: Cluster:Port / PublicHostname now flag that they are stored
  twice, with the "update the row too" instruction and why the drift is silent.
- design doc §7: Phase 1 marked done, plus a "Phase 1 as shipped" note recording
  both deviations rather than leaving the sketch reading as what happened.
- program plan: Phase 1 marked done; AdminUI node edit explicitly deferred.
- CLAUDE.md: the deploy-path behaviour change and its three consequences.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-22 08:38:45 -04:00
parent 90c1302f71
commit 57e1d01766
6 changed files with 79 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Configuration.md
+4 -2
View File
@@ -101,13 +101,15 @@ The checked-in `appsettings*.json` files are deliberately thin: they carry only
|---|---|---|---|
| `SystemName` | string | `otopcua` | Akka actor-system name. |
| `Hostname` | string | `0.0.0.0` | Bind hostname. |
| `Port` | int | `4053` | Cluster transport port. |
| `PublicHostname` | string | `127.0.0.1` | Hostname advertised in cluster gossip; must be reachable by peers. |
| `Port` | int | `4053` | Cluster transport port. **Duplicated as `ClusterNode.AkkaPort` in the Config DB** — see the note below. |
| `PublicHostname` | string | `127.0.0.1` | Hostname advertised in cluster gossip; must be reachable by peers. **Duplicated as `ClusterNode.Host`** — see the note below. |
| `SeedNodes` | string[] | `[]` | Seed nodes for bootstrapping. **Ordered:** a node that appears in its own list must be entry 0 (only `seed-nodes[0]` can form a new cluster), or the host refuses to start — see [Redundancy.md § Bootstrap: self-first seed ordering](Redundancy.md#bootstrap-self-first-seed-ordering). |
| `Roles` | string[] | `[]` | Cluster roles for this node. When empty, falls back to `OTOPCUA_ROLES`. Allowed values: `admin`, `driver`, `dev`. |
> The full redundancy model (ServiceLevel tiers, split-brain, peer discovery) is in [`Redundancy.md`](Redundancy.md). The OPC UA peer-URI advertising lives in the `OpcUa:PeerApplicationUris` key above.
> **`Port` / `PublicHostname` are stored twice.** The node binds from these keys; the fleet's `ClusterNode` row records the same address as `AkkaPort` / `Host` so that **central can dial the node without sharing a gossip ring with it** — which is what [per-cluster mesh Phase 2](plans/2026-07-21-per-cluster-mesh-design.md) needs. Nothing in the schema makes the two agree, so `ClusterNodeAddressReconcilerActor` (admin-role singleton) compares them against live membership and logs an Error on mismatch. If you change `Cluster:Port` or `Cluster:PublicHostname` on a node, **update its `ClusterNode` row too** — a node binding 4054 with a row saying 4053 still gossips fine today, and fails silently in Phase 2. The row's `NodeId` is `host:port` and is also the deploy path's ack identity, so a drifted node's deployment acks stop matching as well. See [`config-db-schema.md` § `ClusterNode`](v2/config-db-schema.md#clusternode).
### `ConnectionStrings` → `ConfigDb`
- **Purpose:** the central Config DB connection string. **Required for every role** — `Program.cs` calls `AddOtOpcUaConfigDb` unconditionally.
docs/plans/2026-07-21-per-cluster-mesh-design.md
+20 -1
View File
@@ -287,7 +287,7 @@ Deliberately not a task plan — per-phase plans follow, one at a time.
|---|---|---|
| 0a | Downing strategy: keep-oldest cannot survive an oldest-node crash in a 2-node cluster (§6.2) | **Yes** — **DONE 2026-07-21**, live gate deferred to Phase 7 |
| 0b | Oldest-Up role derivation (§4) | **Yes** — **DONE 2026-07-21** |
| 1 | `ClusterNode` gains Akka + gRPC address columns; coordinator sources its expected-ack set from the DB | Yes |
| 1 | `ClusterNode` gains Akka + gRPC address columns; coordinator sources its expected-ack set from the DB | **Yes** — **DONE 2026-07-22** (see below) |
| 2 | Comm actors + receptionist registration; ClusterClient transport; deploy notify + acks across the boundary | No |
| 3 | Config fetch-and-cache: artifact served by central, driver nodes read LocalDb (§6.1) | No |
| 4 | Cut the driver-side ConfigDb connection: re-home `EfAlarmConditionStateStore`, resolve the `DbHealthProbeActor` ServiceLevel input, audit `OpcUaPublishActor` | No |
@@ -298,6 +298,25 @@ Deliberately not a task plan — per-phase plans follow, one at a time.
Phases 3 and 4 are the ones that change a running system's data path rather than its wiring, and each
deserves its own live gate.
### Phase 1 as shipped (2026-07-22)
Plan: `2026-07-21-per-cluster-mesh-phase1.md`. Two deviations from the sketch above, both decided
before implementing:
- **No cluster-scope filtering of the expected-ack set.** The plan called for it, but there is no
cluster-scoped deployment to filter on: `Deployment` has no `ClusterId`,
`ConfigComposer.SnapshotAndFlattenAsync` always snapshots the whole DB, and
`DeploymentArtifact.ResolveClusterScope` is *node-side* self-scoping of a fleet-wide artifact. The
expected set is every enabled `ClusterNode` row, which is what the membership rule produced too.
- **No per-node role column.** The membership rule filtered on the `driver` role and the DB has none.
Rather than add one — a second declaration of node roles, free to drift from `Cluster:Roles` — every
`ClusterNode` row is now *defined* to be a driver node. An admin-only node must not be given one.
Also shipped beyond the sketch: `ClusterNodeAddressReconciler` (Task 4), an admin singleton that
catches `AkkaPort` drifting from the node's own `Cluster:Port`. **Phase 2 must revisit it** — once the
meshes split, an admin node cannot see site members and every site row would report
`EnabledRowNotInCluster` forever.
## 8. Risks
- **LocalDb becomes load-bearing for configuration, not just resilient.** Phase 1 built it as a
docs/plans/2026-07-22-per-cluster-mesh-program.md
+7 -1
View File
@@ -82,7 +82,13 @@ per phase:** (1) invoke writing-plans in this repo to produce
references, exploring current code first; (2) execute it task-by-task; (3) run the phase's exit
gate; (4) update this file's status column and the design doc's §7 table.
### Phase 1 — `ClusterNode` address columns + DB-sourced ack set
### Phase 1 — `ClusterNode` address columns + DB-sourced ack set — **DONE 2026-07-22**
Shipped per `2026-07-21-per-cluster-mesh-phase1.md`; see that plan's Task 6 gate record and the
design doc's "Phase 1 as shipped" note for the two scope deviations (no cluster-scope filtering — no
such deployment exists; no per-node role column) and the one addition (`ClusterNodeAddressReconciler`).
**AdminUI node edit was deferred to Phase 2** — nothing reads the columns until then, and the
migration default plus the rig seed cover every node today.
**Scope:** `ClusterNode` gains Akka + gRPC address columns (mirroring ScadaBridge's `Site`
entity `NodeAAddress`/`GrpcNodeAAddress` pattern, but per-node rows); EF migration; AdminUI node
edit surfaces the fields; `ConfigPublishCoordinator` derives its expected-ack set from
docs/v2/config-db-schema.md
+28
View File
@@ -148,6 +148,8 @@ CREATE TABLE dbo.ClusterNode (
Host nvarchar(255) NOT NULL,
OpcUaPort int NOT NULL DEFAULT 4840,
DashboardPort int NOT NULL DEFAULT 8081,
AkkaPort int NOT NULL DEFAULT 4053,
GrpcPort int NULL,
ApplicationUri nvarchar(256) NOT NULL,
ServiceLevelBase tinyint NOT NULL DEFAULT 200,
DriverConfigOverridesJson nvarchar(max) NULL CHECK (DriverConfigOverridesJson IS NULL OR ISJSON(DriverConfigOverridesJson) = 1),
@@ -167,6 +169,32 @@ CREATE UNIQUE INDEX UX_ClusterNode_Primary_Per_Cluster
WHERE RedundancyRole = 'Primary';
```
#### `AkkaPort` / `GrpcPort` — central's dial targets (per-cluster mesh Phase 1)
These are **the addresses central dials**, not the node's own binding configuration. The node binds
from `Cluster:Port` in its own appsettings; these columns duplicate that value for a reader that
cannot see the node's config. Today central shares a gossip ring with every node and does not need
them — [per-cluster mesh Phase 2](../plans/2026-07-21-per-cluster-mesh-design.md) splits the fleet
into one mesh per `Cluster`, after which `Host` + `AkkaPort` is how central builds its ClusterClient
contact points, and `GrpcPort` is the Phase 5 telemetry stream.
`AkkaPort` is `NOT NULL DEFAULT 4053` because every node listens on a remoting port — `0` is never a
truthful value, and rows predating the column must migrate to something real. `GrpcPort` is nullable
with **no** default because nothing listens on it until Phase 5, and a non-null default would assert
a port that does not exist.
**The duplication against `Cluster:Port` is not enforced by the schema.** A node that binds 4054 while
its row says 4053 is unreachable from central in Phase 2, and the symptom there is a silent absence of
acks rather than an error. `ClusterNodeAddressReconcilerActor` (an admin-role singleton) compares the
rows against observed cluster membership and logs an Error on mismatch. It reads membership rather
than having each driver node assert its own row, because Phase 4 removes the driver nodes' ConfigDb
connection entirely.
`Enabled` also gained a second meaning in Phase 1: it is the **expected-ack set** for a deployment.
`ConfigPublishCoordinator` no longer derives that set from cluster membership, so an enabled row whose
node is down now fails the deployment at the apply deadline instead of letting it seal green without
that node. Set `Enabled = 0` for a node taken down for maintenance.
`DriverConfigOverridesJson` shape:
```jsonc