Joseph Doherty
9af935f237
docs/Redundancy.md's bootstrap section is rewritten around the mechanism that is
actually in force: which of Akka's two bootstrap processes runs is decided by
whether seed-nodes[0] is this node's own address, so the ORDER of Cluster:SeedNodes
is the fix, not a timer. Adds the per-process table, the shipped self-first
example, why the validator is conditional (site nodes are not seeds) and why it
matches PublicHostname rather than the 0.0.0.0 bind address.
The retirement is documented rather than erased: a new subsection explains that a
watchdog outside the join handshake cannot tell "no seed answered" from "a seed
answered and the join is in flight", and the ea45ace1 live-gate finding is kept
verbatim as the evidence that produced that conclusion (InitJoinAck at 10:49:05,
old incarnation retired 10:49:06, watchdog fired 10:49:15, second cluster). The
watchdog's own earlier PASS is kept too — it did pass what it was gated on.
Also: a live gate for the NEW mechanism is registered as outstanding (not
re-drilled on the docker-dev rig since the swap), and the plan doc that shipped
the watchdog gets a superseded banner rather than an edit, so the execution record
and its Task 8 finding stay readable.
Ripple: CLAUDE.md cluster section, docs/Configuration.md + docs/v2/Cluster.md +
docs/ServiceHosting.md SeedNodes entries, the per-cluster-mesh design doc's two
"CLOSED by SelfFormAfter" notes, and mesh-program Phase 6 (which had this
convergence queued — now marked done early) + Phase 7's live-gate name.
99 lines
6.4 KiB
Markdown
99 lines
6.4 KiB
Markdown
# Service Hosting (v2)
|
|
|
|
## Overview
|
|
|
|
A production OtOpcUa deployment runs **one binary per node**. The historian backend is the external
|
|
`ZB.MOM.WW.HistorianGateway`, deployed separately (not installed by this repo's scripts):
|
|
|
|
| Process | Project | Runtime | Platform | Responsibility |
|
|
|---|---|---|---|---|
|
|
| **OtOpcUa Host** | `src/Server/ZB.MOM.WW.OtOpcUa.Host` | .NET 10 | AnyCPU | Single fused binary. `OTOPCUA_ROLES` env decides what to mount: `admin` (Blazor + auth + control-plane singletons), `driver` (OPC UA endpoint + per-driver actors), or both. |
|
|
| **ZB.MOM.WW.HistorianGateway** *(external — separate deployment)* | not in this repo | .NET 10 | — | The sole historian backend. OtOpcUa talks gRPC to it (via the `ZB.MOM.WW.HistorianGateway.Client` package) for HistoryRead, alarm `SendEvent`, and continuous `WriteLiveValues`. Must run `RuntimeDb:Enabled=true` + `RuntimeDb:EventReadsEnabled=true`; the API key must carry `historian:read` + `historian:write` + `historian:tags:write`. |
|
|
|
|
Galaxy access still uses the separately-installed **mxaccessgw** sidecar (see `docs/v2/Galaxy.ParityRig.md`); the gateway owns the MXAccess COM bitness constraint (its worker is x86 net48). Nothing in the OtOpcUa repo carries that constraint anymore. (The bespoke Wonderware historian sidecar this deployment used to ship was retired — see [drivers/Historian.Wonderware.md](drivers/Historian.Wonderware.md).)
|
|
|
|
> **v2 change.** v1's separate `OtOpcUa.Server` + `OtOpcUa.Admin` Windows services merged into a single role-gated `OtOpcUa.Host` binary. Two installers became one (with a `-Roles` parameter). The whole DI graph is composed in `OtOpcUa.Host/Program.cs`; per-role wiring is conditional on the env var.
|
|
|
|
## Role gating
|
|
|
|
`Program.cs` reads `OTOPCUA_ROLES`, parses it with `RoleParser`, and conditionally registers services:
|
|
|
|
| Role present | Wires |
|
|
|---|---|
|
|
| `admin` | `AddOtOpcUaAuth`, `AddAdminUI`, `AddSignalR`, `AddOtOpcUaAdminClients`, `MapOtOpcUaAuth`, `MapAdminUI<App>`, `MapOtOpcUaHubs`, `WithOtOpcUaControlPlaneSingletons` (5 admin singletons via `Akka.Hosting`) |
|
|
| `driver` | `WithOtOpcUaRuntimeActors` (DriverHostActor + DbHealthProbeActor) — and the OPC UA endpoint on port 4840 |
|
|
| Either / both | `AddOtOpcUaConfigDb`, `AddOtOpcUaCluster`, `AddOtOpcUaHealth` (`/health/ready`, `/health/active`, `/healthz`) |
|
|
|
|
Single-node dev: `OTOPCUA_ROLES=admin,driver`. Production: typically two admin nodes (HA pair) + N driver nodes.
|
|
|
|
### Per-role configuration overlays
|
|
|
|
`Program.cs:33-35` builds a role suffix by joining the parsed roles **alphabetically** with `-` and loads `appsettings.{roleSuffix}.json` as an optional overlay on top of base `appsettings.json`. Three overlays ship in `src/Server/ZB.MOM.WW.OtOpcUa.Host/`:
|
|
|
|
- `appsettings.admin.json` — admin-only nodes
|
|
- `appsettings.driver.json` — driver-only nodes
|
|
- `appsettings.admin-driver.json` — fused single-node dev / small deployments
|
|
|
|
All three carry Serilog log-level overrides + `Security:Ldap:DevStubMode = false`. Configuration loading order (lowest to highest precedence) is:
|
|
|
|
`appsettings.json` < `appsettings.{Environment}.json` < `appsettings.{role}.json` < environment variables < command-line args
|
|
|
|
The role overlay intentionally outranks `appsettings.{Environment}.json` so role-level security defaults (such as `DevStubMode = false`) cannot be silently overridden by a developer's local `appsettings.Development.json`; environment variables and command-line args still outrank everything for deployment-level overrides. Overlays are optional; the base file boots a node on its own.
|
|
|
|
## Akka cluster
|
|
|
|
The host joins an Akka.NET cluster bound to the address in `appsettings.json::Cluster`:
|
|
|
|
```json
|
|
{
|
|
"Cluster": {
|
|
"Hostname": "0.0.0.0",
|
|
"Port": 4053,
|
|
"PublicHostname": "node-a.lan",
|
|
"SeedNodes": ["akka.tcp://otopcua@node-a.lan:4053"],
|
|
"Roles": ["admin", "driver"]
|
|
}
|
|
}
|
|
```
|
|
|
|
- `SeedNodes` is **ordered**: a node listed in its own seed list must be entry 0, since Akka only lets `seed-nodes[0]` form a new cluster. `AkkaClusterOptionsValidator` fails startup otherwise — see [Redundancy.md § Bootstrap: self-first seed ordering](Redundancy.md#bootstrap-self-first-seed-ordering).
|
|
- `WithOtOpcUaClusterBootstrap` (in `OtOpcUa.Cluster`) loads the embedded HOCON (split-brain resolver, pinned dispatcher, failure detector tuning) and overlays remote endpoint + cluster options.
|
|
- All cluster singletons + per-node actors live on this single ActorSystem — there is no second Akka instance.
|
|
|
|
See [Redundancy.md](Redundancy.md) for the role-leader + ServiceLevel story.
|
|
|
|
## Health endpoints
|
|
|
|
Both admin and driver nodes expose:
|
|
|
|
| Path | Status meaning |
|
|
|---|---|
|
|
| `/healthz` | Process alive. |
|
|
| `/health/ready` | ConfigDb reachable + cluster member state is `Up`. |
|
|
| `/health/active` | Admin-role leader (the node Traefik or an HA LB should route traffic to). |
|
|
|
|
Used by Traefik for the active-leader-only routing pattern (see [Architecture-v2.md](v2/Architecture-v2.md)).
|
|
|
|
## Historian backend (HistorianGateway — external)
|
|
|
|
The historian backend is the external `ZB.MOM.WW.HistorianGateway`, deployed and operated separately (not
|
|
installed by `Install-Services.ps1`). OtOpcUa connects to it over gRPC via the
|
|
`ZB.MOM.WW.HistorianGateway.Client` package — configure the `ServerHistorian:Endpoint` (`https://host:5222`)
|
|
and supply `ServerHistorian__ApiKey` via the environment on the OtOpcUa host side. The gateway must run with
|
|
`RuntimeDb:Enabled=true` + `RuntimeDb:EventReadsEnabled=true` and an API key carrying `historian:read` +
|
|
`historian:write` + `historian:tags:write`. See [Historian.md](Historian.md) for the full config-key and
|
|
deployment-prerequisite reference. (The retired Wonderware TCP sidecar: [Historian.Wonderware.md](drivers/Historian.Wonderware.md).)
|
|
|
|
## Install / Uninstall
|
|
|
|
- `scripts/install/Install-Services.ps1 -Roles admin,driver` — installs `OtOpcUaHost`.
|
|
- `scripts/install/Uninstall-Services.ps1` — stops + removes the host service. (The historian backend is the external HistorianGateway — not installed/removed by these scripts.)
|
|
|
|
## Logging
|
|
|
|
Serilog with rolling-daily file sinks. Each host writes to `logs/otopcua-*.log` plus stdout (NSSM/systemd-friendly). Per-environment log level overrides go in `appsettings.{Environment}.json`.
|
|
|
|
## Depth reference
|
|
|
|
For the full host-architecture rationale (why fused vs. split, role-gating tradeoffs, multi-node deployment shapes), see `docs/plans/2026-05-26-akka-hosting-alignment-design.md` §3-4.
|