dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: one role authority — precedence, normalization, and the overlay invariant (R3-04)

Browse Source
This commit is contained in:
Joseph Doherty
2026-08-20 10:29:37 -04:00
parent 659d5f51e4
commit b8d2d50006
7 changed files with 75 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Configuration.md
+7 -5
View File
@@ -26,7 +26,7 @@ The Host (`Program.cs`) loads `appsettings.json`, then overlays a **per-role** f
- Both roles → `appsettings.admin-driver.json` (roles joined with `-`, ordinal-sorted).
- `appsettings.{ASPNETCORE_ENVIRONMENT}.json` (e.g. `appsettings.Development.json`) is layered on by the host builder.
All role overlays are **optional** — the base `appsettings.json` plus the Options-class C# defaults are enough to boot. The roles themselves come from the `OTOPCUA_ROLES` env var (see [`ServiceHosting.md`](ServiceHosting.md) and the table below).
All role overlays are **optional** — the base `appsettings.json` plus the Options-class C# defaults are enough to boot. The roles themselves come from `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`) — `Cluster:Roles`, falling back to `OTOPCUA_ROLES` when that is empty (see [`ServiceHosting.md`](ServiceHosting.md) and the table below).
The checked-in `appsettings*.json` files are deliberately thin: they carry only `Serilog` and the `Security:Ldap` overlay. Everything else (`OpcUa`, `Cluster`, `ConnectionStrings`/`ConfigDb`) binds from the Options-class defaults documented below unless an operator adds the section explicitly or supplies the corresponding environment variable.
@@ -121,11 +121,13 @@ The checked-in `appsettings*.json` files are deliberately thin: they carry only
| `Port` | int | `4053` | Cluster transport port. **Duplicated as `ClusterNode.AkkaPort` in the Config DB** — see the note below. |
| `PublicHostname` | string | `127.0.0.1` | Hostname advertised in cluster gossip; must be reachable by peers. **Duplicated as `ClusterNode.Host`** — see the note below. |
| `SeedNodes` | string[] | `[]` | Seed nodes for bootstrapping. **Ordered:** a node that appears in its own list must be entry 0 (only `seed-nodes[0]` can form a new cluster), or the host refuses to start — see [Redundancy.md § Bootstrap: self-first seed ordering](Redundancy.md#bootstrap-self-first-seed-ordering). **Per-pair since Phase 6:** each application `Cluster`'s pair is its own Akka mesh, so a node's `SeedNodes` lists only itself (first) and its own pair partner (second) — never a node from another cluster's pair. |
| `Roles` | string[] | `[]` | Cluster roles for this node. When empty, falls back to `OTOPCUA_ROLES` (see the note below). Allowed values: `admin`, `driver`, `dev`, plus a per-cluster `cluster-{ClusterId}` role (e.g. `cluster-MAIN`, `cluster-SITE-A`) — see [Per-cluster mesh split (Phase 6)](#per-cluster-mesh-split-phase-6-cluster-clusterid-roles--splittopologytransportvalidator) below. |
| `Roles` | string[] | `[]` | Cluster roles for this node. When empty, falls back to `OTOPCUA_ROLES` (see the note below). Values are trimmed and (for a plain role) lower-cased; a cluster role's ClusterId suffix keeps the operator's spelling. An unknown role from either source **fails host start**. Allowed values: `admin`, `driver`, `dev`, plus a per-cluster `cluster-{ClusterId}` role (e.g. `cluster-MAIN`, `cluster-SITE-A`) — see [Per-cluster mesh split (Phase 6)](#per-cluster-mesh-split-phase-6-cluster-clusterid-roles--splittopologytransportvalidator) below. |
> **`Cluster:Roles` and `OTOPCUA_ROLES` are two knobs for one thing, and only one of them is the Akka role list.** `OTOPCUA_ROLES` drives all of `Program.cs`'s conditional wiring (which subsystems mount); `Cluster:Roles` becomes the node's **gossiped Akka member roles**. Since Gitea #532 `AddOtOpcUaCluster` applies the documented fallback — an empty `Cluster:Roles` takes the env var — so setting only `OTOPCUA_ROLES` now works. Before that fix it did not, silently: the node joined the mesh carrying **no roles at all**, which reads as healthy locally and as *"no driver-role nodes are Up"* at central, and every deployment to it timed out ([`deployments/wonder-app-vd03-makino-z-34184.md`](deployments/wonder-app-vd03-makino-z-34184.md)).
> **One role authority, one precedence, both consumers (archreview R3-04).** `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`) is the SINGLE place a node's roles are decided: `Cluster:Roles` wins outright when non-empty, else `OTOPCUA_ROLES`, else empty. Both `Program.cs`'s conditional host wiring (which subsystems mount) AND the Akka `AkkaClusterOptions.Roles` gossiped to the cluster are the SAME `EffectiveRoles` instance's `Gossiped` list — not two separate derivations that happen to usually agree. The boot banner (`OtOpcUa.Host starting with roles=[...] source=...`) names which source won. Setting both sources to *different* normalized role sets logs a Warning at boot (still resolves from `Cluster:Roles`) — almost certainly not what an operator intended.
>
> Setting `Cluster:Roles` explicitly is still the recommendation, because the two sources normalize differently: `Cluster:Roles` binds **verbatim** while the env path is **lower-cased** by `RoleParser.Parse`. Akka compares role names ordinally, so a redundant pair configured through *different* sources with a mixed-case `cluster-{ClusterId}` (e.g. `cluster-SITE-A` vs `cluster-site-a`) would gossip two different role names and elect two singletons. Configure both nodes of a pair the same way.
> Before R3-04, `Program.cs` read `OTOPCUA_ROLES` only and never looked at `Cluster:Roles` at all — a node configured solely through `Cluster:Roles` (the shape Gitea #532's fix made viable on the Akka side) joined the mesh gossiping the right roles while `Program.cs`'s own conditional wiring mounted NONE of them, silently. See [`deployments/wonder-app-vd03-makino-z-34184.md`](deployments/wonder-app-vd03-makino-z-34184.md) for the inverse (env-only, pre-#532) incident this family of defect belongs to.
>
> **A role overlay may not itself set `Cluster:Roles`.** `Program.cs` resolves `EffectiveRoles` off the *pre-overlay* configuration — it needs the answer to pick the overlay's filename — so a role overlay (or a `${secret:}` expansion) that changed `Cluster:Roles` would leave the bound options disagreeing with every consumer holding the registered instance. `AkkaClusterOptionsValidator` fails host start if this ever happens, naming both views.
> The full redundancy model (ServiceLevel tiers, split-brain, peer discovery) is in [`Redundancy.md`](Redundancy.md). The OPC UA peer-URI advertising lives in the `OpcUa:PeerApplicationUris` key above.
@@ -446,7 +448,7 @@ All names are read in this repo's source via `Environment.GetEnvironmentVariable
| Variable | Read by | Effect / default |
|---|---|---|
| `OTOPCUA_ROLES` | `src/Server/ZB.MOM.WW.OtOpcUa.Host/Program.cs` (`RoleParser.Parse`) and `AddOtOpcUaCluster`'s `AkkaClusterOptions` post-configure (`RoleParser.ResolveEffectiveRoles`) | Comma-separated cluster roles for the node (`admin`, `driver`, `dev`, `cluster-{ClusterId}`). Drives the conditional wiring and the per-role appsettings overlay, and — when `Cluster:Roles` is empty — the node's Akka member roles too. See the note under `Cluster:Roles` for why the two sources are not interchangeable. |
| `OTOPCUA_ROLES` | `EffectiveRoles.Resolve` (`ZB.MOM.WW.OtOpcUa.Cluster`) — the single role authority both `Program.cs` and `AddOtOpcUaCluster` consume | Comma-separated cluster roles for the node (`admin`, `driver`, `dev`, `cluster-{ClusterId}`), read as the **fallback** only when `Cluster:Roles` is empty. Drives the conditional wiring, the per-role appsettings overlay, and the node's Akka member roles — all from the same resolved instance. See the note under `Cluster:Roles` above. |
| `OTOPCUA_CONFIG_CONNECTION` | `src/Core/ZB.MOM.WW.OtOpcUa.Configuration/DesignTimeDbContextFactory.cs` (design-time / `dotnet ef` only) | Read at **design time** by `DesignTimeDbContextFactory.cs` for `dotnet ef` migrations. At **runtime** the server resolves the connection string from `ConnectionStrings:ConfigDb` (env form: `ConnectionStrings__ConfigDb`) via `configuration.GetConnectionString("ConfigDb")` in `ServiceCollectionExtensions.cs` — `OTOPCUA_CONFIG_CONNECTION` appears there only as a hint in an error message, not via `GetEnvironmentVariable`. No credential is embedded in source. |
| `ASPNETCORE_ENVIRONMENT` | ASP.NET host builder (framework) | Selects `appsettings.{Environment}.json` (e.g. `Development`). |