fix(redundancy): advertise ServiceLevel 0 until the node has a redundancy level (#885)
v2-ci / unit-tests (push) Has been cancelled

The SDK starts Server.ServiceLevel at 255 (full service), and OpcUaPublishActor publishes nothing
until the node's first redundancy snapshot arrives, so a restarting pair node advertised 255 for the
~10-13 s it took to join, out-ranking the real Primary (250) for any client that picks the highest
ServiceLevel.

OtOpcUaSdkServer now writes StartupServiceLevel = 0 in OnNodeManagerStartedAsync: after the SDK's
CreateServerObjectAsync wrote 255 and before the server goes Running, so no session can ever read
255. 0 is the design's existing Down / Detached tier (ServiceLevelCalculator returns 0 for a member
that is not Up/Joining, and a node with no snapshot has no role yet). It is below the SDK 2
session-headroom floor (200), so it stands through session churn before the #884 claim, and the
claim's bind replays any level already requested. Single-node deployments are unaffected beyond
the window: the first snapshot publishes 250 as before.

New Node_with_no_requested_level_advertises_the_startup_level_until_one_is_published: reads 0
before the bind, 0 after a bind with nothing to replay, then 250 once published. Without the
startup write it reads 255 on the first session (mutation-checked). docs/Redundancy.md updated for
#884 and #885.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-12 11:11:11 -05:00
parent 5e3fb4229d
commit 3fff439b11
3 changed files with 93 additions and 4 deletions
+12 -4
View File
@@ -16,7 +16,7 @@ The runtime pieces live in:
|---|---|---|
| `RedundancyStateActor` | `OtOpcUa.ControlPlane.Redundancy` | Admin-role cluster singleton; subscribes to cluster topology events, debounces 250ms, broadcasts `RedundancyStateChanged` on the `redundancy-state` DPS topic. |
| `OpcUaPublishActor` | `OtOpcUa.Runtime.OpcUa` | Per-driver-node; subscribes to the `redundancy-state` topic, computes a health-aware ServiceLevel byte via `ServiceLevelCalculator` (see below), and forwards it to `IServiceLevelPublisher`. |
| `IServiceLevelPublisher` / `SdkServiceLevelPublisher` | `OtOpcUa.Commons.OpcUa` / `OtOpcUa.OpcUaServer` | Writes the byte into the SDK's `Server.ServiceLevel` Variable. Production binds `DeferredServiceLevelPublisher`, which swaps in the real `SdkServiceLevelPublisher` once the SDK is up (it needs `IServerInternal`, available only after `StandardServer.Start`); until then writes route through `NullServiceLevelPublisher`. |
| `IServiceLevelPublisher` / `SdkServiceLevelPublisher` | `OtOpcUa.Commons.OpcUa` / `OtOpcUa.OpcUaServer` | Writes the byte into the SDK's `Server.ServiceLevel` Variable. Production binds `DeferredServiceLevelPublisher`, which swaps in the real `SdkServiceLevelPublisher` once the SDK is up (it needs `IServerInternal`, available only after `StandardServer.Start`); until then it retains the latest request and replays it on bind. Before any request the wire reads `OtOpcUaSdkServer.StartupServiceLevel` (0, #885). On SDK 2 the publisher claims the value so the SDK stops overwriting it (#884). |
| `ServiceLevelCalculator` | `OtOpcUa.Cluster.Redundancy` (`Core.Cluster`) | Pure function `(NodeHealthInputs) → byte` — the DB/probe-aware tiering (see truth table below). Covered by `ServiceLevelCalculatorTests`. **Now the live publish path** — `OpcUaPublishActor` calls it on every `HealthTick` and `RedundancyStateChanged` event. Moved to `Core.Cluster` so Runtime can reach it without a Runtime→ControlPlane reference. |
| `DbHealthProbeActor` | `OtOpcUa.Runtime.Health` | Runs `SELECT 1` against ConfigDb every 5s. Read by the health endpoint AND by `OpcUaPublishActor` (the `DbReachable` ServiceLevel input). **Spawned only when a ConfigDb is present** (admin-role or fused admin+driver nodes) — a driver-only node has no `IDbContextFactory<OtOpcUaConfigDbContext>` to probe, so this actor is not spawned there at all; see "DB-less nodes" below. |
| `PeerProbeSupervisor` | `OtOpcUa.Runtime.Health` | Per-node; subscribes to the `redundancy-state` topic and maintains one `PeerOpcUaProbeActor` child per OTHER driver-role peer (spawn on join, stop on departure), so every node is continuously probed by its peers. |
@@ -47,7 +47,7 @@ The resulting truth table (all tiers are now reachable at runtime):
| Tier | Byte | Condition |
|---|---|---|
| Down / Detached | 0 | Member status is not `Up` or `Joining` (leaving, removed, exiting), OR node has no `driver` role (Detached). Published immediately — a starting or detached node never leaves the SDK default 255. |
| Down / Detached | 0 | Member status is not `Up` or `Joining` (leaving, removed, exiting), OR node has no `driver` role (Detached). Published immediately. A node that has no redundancy snapshot yet (cold start, rejoin after a restart) also advertises **0**: `OtOpcUaSdkServer` replaces the SDK's built-in 255 before the endpoint accepts sessions (`StartupServiceLevel`, #885), so a joining node never out-ranks the real Primary. |
| Endpoint failed | 0 | This node's OPC UA SDK failed to start (`EndpointListening=false`). Wins over every other input, including the +10 Primary bonus; the legacy role-only seam applies it too. See [Endpoint start failure: Primary step-down](#endpoint-start-failure-primary-step-down). |
| Critically degraded | 100 | ConfigDb unreachable AND data is stale. |
| Stale | 200 | Data stale but ConfigDb reachable. |
@@ -93,8 +93,16 @@ A node with no `DbHealthStatus` wired (e.g. early bootstrap window before the
first `DbHealthProbeActor` reply) falls back to the old role-only mapping:
Primary-leader → 240, Primary → 200, Secondary → 100, Detached → 0. Once the
first `DbHealthStatus` arrives the calculator takes over. The first computed
ServiceLevel (even 0) is always published so no node lingers at the SDK default
255.
ServiceLevel (even 0) is always published. Before that first level the server
advertises 0, not the SDK's built-in 255 (#885).
> **SDK 2.0 ownership (#884).** SDK 2.0.1 recomputes `Server.ServiceLevel` from
> session headroom on every session create/close and overwrites any value ≥ 200
> (255 when lightly loaded) unless a provider has claimed the value.
> `SdkServiceLevelPublisher` claims it through
> `IServerServiceLevelControl.ClaimServiceLevelControl()` when the host binds it
> and writes through the returned writer, so 250 / 240 / 200 survive client
> sessions opening and closing. Values below 200 were never touched by the SDK.
Roles come from `RedundancyStateActor.BuildSnapshot`: a node with the `driver`
role is `Primary` when it is the **oldest Up member carrying the `driver` role**, otherwise