chore(docker-dev): site-b-2 is Cluster:Roles-only, site-a-2 pins a miscased Cluster:Roles entry (archreview 01/S-12, 01/C-9)

T11: site-b-2 drops OTOPCUA_ROLES entirely — Cluster:Roles is its sole role source, the rig's
only node that can exhibit the two-role-authority split (01/S-12). Expected to fail every
deploy on a pre-R3-04 build.

T12: site-a-2 pins Cluster__Roles__0="Driver" (capital D) as the standing 01/C-9 case-mismatch
pin, keeping cluster-SITE-A verbatim. Pre-fix this un-elects the SITE-A Primary.

Combined into one commit (deviation from the plan's two separate T11/T12 commits): the header
comment block documenting all six nodes' role sources interleaves both nodes on adjacent lines,
so a clean split was not practical. Recorded in the plan's Execution notes.

Compose edits only — the rig is NOT brought up as part of this change.
This commit is contained in:
Joseph Doherty
2026-08-20 10:32:18 -04:00
parent b8d2d50006
commit 985de0b6ad
3 changed files with 40 additions and 8 deletions
@@ -83,13 +83,13 @@
{ {
"id": "T11", "id": "T11",
"subject": "docker-dev: site-b-2 becomes Cluster:Roles-only so the rig can exhibit the two-authority split (theme 2 rig-fidelity)", "subject": "docker-dev: site-b-2 becomes Cluster:Roles-only so the rig can exhibit the two-authority split (theme 2 rig-fidelity)",
"status": "pending", "status": "completed",
"blockedBy": [] "blockedBy": []
}, },
{ {
"id": "T12", "id": "T12",
"subject": "docker-dev: site-a-2 pins a miscased Cluster__Roles__0 = 'Driver' as the standing 01/C-9 pin", "subject": "docker-dev: site-a-2 pins a miscased Cluster__Roles__0 = 'Driver' as the standing 01/C-9 pin",
"status": "pending", "status": "completed",
"blockedBy": [] "blockedBy": []
}, },
{ {
+21 -2
View File
@@ -27,17 +27,36 @@ Authentication uses the **shared GLAuth** on the Linux Docker host at `10.100.0.
| Service | Roles | Ports | | Service | Roles | Ports |
|---|---|---| |---|---|---|
| `site-a-1` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4842` → container `4840` | | `site-a-1` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4842` → container `4840` |
| `site-a-2` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4843` → container `4840` | | `site-a-2` | `OTOPCUA_ROLES=driver` **+** a deliberately miscased `Cluster__Roles__0=Driver` (see below) | host `4843` → container `4840` |
### Site B nodes — driver-only (SITE-B cluster) ### Site B nodes — driver-only (SITE-B cluster)
| Service | Roles | Ports | | Service | Roles | Ports |
|---|---|---| |---|---|---|
| `site-b-1` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4844` → container `4840` | | `site-b-1` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4844` → container `4840` |
| `site-b-2` | `OTOPCUA_ROLES=driver`, joins the single mesh | host `4845` → container `4840` | | `site-b-2` | **`Cluster:Roles`-ONLY** — no `OTOPCUA_ROLES` at all (see below) | host `4845` → container `4840` |
Site nodes serve no UI and authenticate no users. The central cluster manages and deploys to them over the shared Akka mesh. All six nodes bind Akka remoting to port `4053` inside their own network namespace; `PublicHostname` for each matches its Compose service name. Site nodes serve no UI and authenticate no users. The central cluster manages and deploys to them over the shared Akka mesh. All six nodes bind Akka remoting to port `4053` inside their own network namespace; `PublicHostname` for each matches its Compose service name.
### Role-authority standing pins (archreview R3-04)
Every node above sets **both** `OTOPCUA_ROLES` and the matching `Cluster__Roles__*` env vars,
consistently — which is exactly why neither R3-04 finding (01/S-12, 01/C-9) could reproduce on this
rig as originally configured. Two nodes deliberately deviate, as standing pins:
- **`site-b-2` is `Cluster:Roles`-only.** No `OTOPCUA_ROLES` is set at all; `Cluster__Roles__0=driver`
/ `Cluster__Roles__1=cluster-SITE-B` are its sole role source. **Expected to FAIL every deploy on a
pre-R3-04 build** (`Program.cs` read `OTOPCUA_ROLES` only, so this node's host-side conditional
wiring mounted no driver stack at all — no LocalDb, no OPC UA endpoint, no ConfigDb, the deny-all
`NullOpcUaUserAuthenticator` — while its Akka gossip still looked healthy) — the exact outage
01/S-12 describes. Post-fix it behaves identically to `site-b-1`.
- **`site-a-2` pins a miscased `Cluster__Roles__0=Driver`** (capital D; `cluster-SITE-A` keeps its
own spelling verbatim, by design). Post-fix `EffectiveRoles` normalizes it silently and this node
behaves identically to `site-a-1`. Pre-fix, all four role-scoped validators (`SplitTopologyTransportValidator`
and three siblings) compared roles case-sensitively, so three of the four silently skipped every
driver-scoped boot-time rule for this node, and `site-a` elected **no Primary at all**
(`RedundancyStateActor.SelectDriverPrimary` matches roles ordinally against gossip).
## Multi-tenancy ## Multi-tenancy
All six host nodes write to the same `OtOpcUa` ConfigDb. The `ServerCluster` table differentiates the three logical clusters: each maps to one row, and each `ClusterNode` row's `ClusterId` ties the runtime node back to its owning cluster scope. All six host nodes write to the same `OtOpcUa` ConfigDb. The `ServerCluster` table differentiates the three logical clusters: each maps to one row, and each `ClusterNode` row's `ClusterId` ties the runtime node back to its owning cluster scope.
+17 -4
View File
@@ -47,10 +47,15 @@
# Traefik). Both are Akka seed nodes for the MAIN mesh # Traefik). Both are Akka seed nodes for the MAIN mesh
# ONLY, each listing ITSELF first (2026-07-22) so either # ONLY, each listing ITSELF first (2026-07-22) so either
# can cold-start while the other is down. # can cold-start while the other is down.
# site-a-1, site-a-2 OTOPCUA_ROLES=driver,cluster-SITE-A — driver-only # site-a-1, site-a-2 OTOPCUA_ROLES=driver,cluster-SITE-A — driver-only.
# site-a-2 additionally pins a miscased Cluster__Roles__0=
# "Driver" (R3-04, archreview 01/C-9 standing pin).
# site-b-1, site-b-2 OTOPCUA_ROLES=driver,cluster-SITE-B — members of their # site-b-1, site-b-2 OTOPCUA_ROLES=driver,cluster-SITE-B — members of their
# OWN 2-node mesh (self-first seeded within the pair, NOT # OWN 2-node mesh (self-first seeded within the pair, NOT
# off central). They serve no UI, but they DO expose OPC UA # off central) — EXCEPT site-b-2, which sets NO OTOPCUA_ROLES
# at all and is Cluster:Roles-only (R3-04, archreview 01/S-12
# standing pin — the rig's one node that can exhibit the
# two-role-authority split). They serve no UI, but they DO expose OPC UA
# data-plane endpoints (4842-4845) and authenticate those # data-plane endpoints (4842-4845) and authenticate those
# users against the same shared GLAuth (via the *ldap-env # users against the same shared GLAuth (via the *ldap-env
# anchor); central manages + deploys to them over the # anchor); central manages + deploys to them over the
@@ -713,7 +718,13 @@ services:
# Bootstrap-guard enablement demo — see site-a-1. site-a-2 is the HIGHER address, so the guard # Bootstrap-guard enablement demo — see site-a-1. site-a-2 is the HIGHER address, so the guard
# makes it probe site-a-1 and join it (peer-first) rather than race-form its own cluster. # makes it probe site-a-1 and join it (peer-first) rather than race-form its own cluster.
Cluster__BootstrapGuard__Enabled: "true" Cluster__BootstrapGuard__Enabled: "true"
Cluster__Roles__0: "driver" # R3-04 (01/C-9) standing pin: 'Driver' (capital D) — a deliberately miscased plain role.
# EffectiveRoles normalizes it silently (Program.cs and AkkaClusterOptions.Roles both still
# gossip lower-case 'driver'), so post-fix this node behaves identically to site-a-1; pre-fix
# every one of the four role-scoped validators skipped its driver-scoped rules for this node,
# and site-a elected no Primary (RedundancyStateActor.SelectDriverPrimary matches ordinally).
# cluster-SITE-A keeps its own spelling verbatim by design — do not "fix" that half.
Cluster__Roles__0: "Driver"
Cluster__Roles__1: "cluster-SITE-A" Cluster__Roles__1: "cluster-SITE-A"
# Mesh command transport (per-cluster mesh Phase 2, made mandatory by Phase 6) — see # Mesh command transport (per-cluster mesh Phase 2, made mandatory by Phase 6) — see
# central-1/site-a-1. # central-1/site-a-1.
@@ -850,7 +861,9 @@ services:
# cluster on simultaneous start (split brain). # cluster on simultaneous start (split brain).
site-b-1: { condition: service_healthy } site-b-1: { condition: service_healthy }
environment: environment:
OTOPCUA_ROLES: "driver,cluster-SITE-B" # R3-04 (01/S-12) standing pin: this is the rig's ONLY Cluster:Roles-ONLY node — deliberately
# no OTOPCUA_ROLES at all (every other node sets both, which is why neither R3-04 finding could
# reproduce on the rig before this). Cluster__Roles__0/1 below are this node's sole role source.
# Per-cluster mesh Phase 4: no ConfigDb string on driver-only site nodes (see site-a-1). # Per-cluster mesh Phase 4: no ConfigDb string on driver-only site nodes (see site-a-1).
Cluster__Hostname: "0.0.0.0" Cluster__Hostname: "0.0.0.0"
Cluster__Port: "4053" Cluster__Port: "4053"