feat(dcl): ServiceLevel-aware OPC UA failover for primary/backup pairs (#138)

Prerequisite for OtOpcUa Warm redundancy (lmxopcua#717): a Warm standby keeps a
healthy session but serves BadOutOfService at a standby ServiceLevel (90/50, boot 1),
so a connection that lands there sat on Bad quality forever.

- OPC UA connections with a backup poll Server_ServiceLevel (i=2267) on the connected
  endpoint (IServiceLevelReadable, implemented by OpcUaDataConnection; never raises
  Disconnected). A level < 200 sustained for the degraded window (default 10 s), or
  the first sample after any (re)connect, triggers a probe of the other endpoint
  (short-lived session: connect, read, dispose). Switch only when it reports a
  Healthy (>= 200) higher level, and not within the minimum dwell (default 60 s) of
  the previous ServiceLevel switch. Both degraded, other unreachable or level
  unknown: stay put. 250/240 (today's OtOpcUa) never starts a streak.
- The switch reuses the failover path: SwitchToOtherEndpoint() now shared by all
  three triggers, then BecomeReconnecting(ReconnectCause.ServiceLevelFailover) -
  zero delay, no unstable accounting, ReSubscribeAll on the new endpoint.
- Config: OpcUaEndpointConfig.ServiceLevelFailover {enabled, degradedWindowSeconds,
  minDwellSeconds}, read from the PRIMARY endpoint's JSON only; null = defaults
  (enabled, 10, 60). Inside the encrypted protocol JSON, so no schema change or
  migration. Central UI: fields in the Backup card next to Failover Retry Count.
  Global: ScadaBridge:DataConnection:ServiceLevelPollInterval (2 s) and
  ServiceLevelProbeTimeout (15 s).
- Tests: actor tests per trigger/non-trigger with positive controls, an unsolicited
  promotion RefreshStart/RefreshEnd test at client and actor level, adapter tests,
  and a real in-process OPC UA server pair end to end. All 15 mutants killed.
- Docs: Component-DataConnectionLayer, components/DataConnectionLayer, Component-CLI.

Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
Joseph Doherty
2026-11-13 03:06:45 -05:00
parent 16209831ca
commit 712c5117ee
23 changed files with 1760 additions and 36 deletions
@@ -220,4 +220,30 @@ public class OpcUaEndpointConfigValidatorTests
var r = OpcUaEndpointConfigValidator.Validate(c, fieldPrefix: "Primary.");
Assert.Contains(r.Errors, e => e.EntityName == "Primary.UserIdentity.Username");
}
/// <summary>ScadaBridge#138: the ServiceLevel failover window and dwell are range-checked, prefixed.</summary>
/// <param name="window">Degraded window, seconds.</param>
/// <param name="dwell">Minimum dwell, seconds.</param>
/// <param name="badField">The field expected to fail, or null when valid.</param>
[Theory]
[InlineData(10, 60, null)]
[InlineData(1, 0, null)]
[InlineData(3600, 86400, null)]
[InlineData(0, 60, "Primary.ServiceLevelFailover.DegradedWindowSeconds")]
[InlineData(3601, 60, "Primary.ServiceLevelFailover.DegradedWindowSeconds")]
[InlineData(10, -1, "Primary.ServiceLevelFailover.MinDwellSeconds")]
[InlineData(10, 86401, "Primary.ServiceLevelFailover.MinDwellSeconds")]
public void Validate_ServiceLevelFailoverRanges(int window, int dwell, string? badField)
{
var c = Valid();
c.ServiceLevelFailover = new OpcUaServiceLevelFailoverConfig
{
DegradedWindowSeconds = window, MinDwellSeconds = dwell,
};
var r = OpcUaEndpointConfigValidator.Validate(c, fieldPrefix: "Primary.");
if (badField is null)
Assert.True(r.IsValid);
else
Assert.Contains(r.Errors, e => e.EntityName == badField);
}
}