feat(mqtt): Sparkplug primary-host STATE publishing, Primary-gated (#511)

`ActAsPrimaryHost` was accepted by config and by the AdminUI form and did nothing but
log a warning. It now delivers the whole three-part contract or none of it: a retained
`ONLINE` on `spBv1.0/STATE/{hostId}`, a matching retained `OFFLINE` armed as the MQTT
Last Will, and an explicit `OFFLINE` on a clean stand-down.

**The claim is a dedicated connection, not a message.** A Will can only be registered in
the CONNECT packet — it cannot be added to a live session, changed, or withdrawn. So a
claim that is acquired and released at runtime cannot ride the long-lived ingest session.
`SparkplugPrimaryHost` opens its own connection (client id `{clientId}-spbhost`, forced
clean-session) when the node gains Primary and closes it when the node loses Primary; the
connection's lifetime *is* the claim, so every leg follows from its lifecycle. Ingest is
untouched by a role change — no dropped subscriptions, no lost birth cache, no data gap at
the Secondary->Primary transition.

**Primary-gated** (owner decision, recorded on the issue). Both pair nodes subscribe
independently, so an ungated claim means both claim the same host id — and the sharp
failure is not the two fighting over the retained STATE, it is that the SECONDARY's death
fires its Will and publishes OFFLINE while the Primary is healthy, stopping every edge node
in the group. Registering the Will only on whichever node is Primary at connect time fails
the other way: a node that later gains Primary would publish ONLINE with no Will behind it.

New seam: `IDriverPrimaryRoleSource` (`Core.Abstractions`) carrying the *data-plane* verdict
(`PrimaryGatePolicy.ShouldServiceAsPrimary`), not the alarm-drain verdict — an unknown role
with a driver peer present must deny. Registered unconditionally by `AddOtOpcUaRuntime`,
deliberately NOT alongside `IRedundancyRoleView`, which `AddAlarmHistorian` registers only
when that section is enabled: piggybacking would leave a pair with no alarm historian
falling back to "claim unconditionally". Polled on a 1s timer rather than evented — a
process singleton raising events at per-instance drivers outlives every driver it holds.

Accepted trade: a brief edge-visible OFFLINE window during failover. That is a genuine
state transition, not a glitch to hide.

Live-gated 5/5 against the Mosquitto fixture, including the Will leg — `AbandonForTests`
drops the socket with no DISCONNECT, because a clean disconnect makes the broker discard
the Will by protocol rule, so a test driven through `StopAsync` would observe the
stand-down OFFLINE and pass against an implementation that never armed a Will. Falsified:
removing the Will turns that leg and only that leg red.

Also corrects two stale claims found en route: the AdminUI form still said MQTT tags are
read-only (#508 shipped), and the driver doc still said nothing consumes `IRediscoverable`.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-31 07:17:17 -04:00
parent 2a7d11674f
commit 7f2108dcf2
21 changed files with 1881 additions and 87 deletions
@@ -273,6 +273,19 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
/// today, the alarm store-and-forward drain. Null on nodes that do not wire one.</summary>
private readonly IRedundancyRoleView? _redundancyRoleView;
/// <summary>
/// Where the Primary <b>data-plane</b> verdict is published for drivers that must know it
/// directly (Gitea #511). Null on nodes that wire no redundancy manager.
/// </summary>
/// <remarks>
/// Note the different verdict from <see cref="_redundancyRoleView"/>: that one carries
/// <see cref="PrimaryGatePolicy.ShouldDrainAlarmHistory"/>, which OPENS on an unknown role. A
/// Sparkplug host-id claim is an exclusive assertion on a shared broker, so it takes the
/// data-plane verdict, which closes. Publishing the drain verdict here would let both nodes of
/// a pair claim the host id during every boot window.
/// </remarks>
private readonly IDriverPrimaryRoleSource? _primaryRoleSource;
/// <summary>
/// Host of this node's LocalDb replication partner — the only node that holds a copy of this
/// node's alarm queue, and therefore the only node it may ever stand down in favour of.
@@ -408,7 +421,8 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
bool fetchAndCacheMode = false,
IDeploymentArtifactFetcher? artifactFetcher = null,
IAlarmStateStore? alarmStateStore = null,
ITelemetryLocalHub? telemetryHub = null) =>
ITelemetryLocalHub? telemetryHub = null,
IDriverPrimaryRoleSource? primaryRoleSource = null) =>
// WARNING: this forwarding list is POSITIONAL, and Props.Create compiles it into an
// expression tree. Six IActorRef? parameters and several interface-typed ones mean a
// mis-ordered argument is usually type-compatible and therefore compiles clean, then binds
@@ -419,7 +433,7 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
healthPublisher, virtualTagEvaluator, historyWriter, virtualTagHostOverride,
loggerFactory, scriptRootLogger, scriptedAlarmHostOverride, invokerFactory, driverMemberCountProvider,
deploymentArtifactCache, redundancyRoleView, replicationPeerHost, fetchAndCacheMode, artifactFetcher,
alarmStateStore, telemetryHub));
alarmStateStore, telemetryHub, primaryRoleSource));
/// <summary>Initializes a new DriverHostActor with the specified dependencies.</summary>
/// <param name="dbFactory">Database context factory for configuration database access.</param>
@@ -460,6 +474,12 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
/// <param name="telemetryHub">Per-cluster mesh Phase 5: optional node-local live-telemetry hub each
/// Primary-gated native-alarm transition is also emitted into, and which is threaded into the spawned
/// VirtualTag + ScriptedAlarm hosts. Null (admin-only nodes / tests) skips the emit and passes null down.</param>
/// <param name="primaryRoleSource">Gitea #511: where the Primary <i>data-plane</i> verdict is published
/// for drivers that assert a standing exclusive identity on a shared external system — today the MQTT
/// driver's Sparkplug primary-host STATE claim. Deliberately a separate parameter from
/// <paramref name="redundancyRoleView"/> even though the production object implements both: a test that
/// supplies only a role-view double must not silently also satisfy this, or its primary-role assertions
/// pass without the publish ever happening. Null on nodes that wire no redundancy manager.</param>
public DriverHostActor(
IDbContextFactory<OtOpcUaConfigDbContext>? dbFactory,
CommonsNodeId localNode,
@@ -483,11 +503,13 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
bool fetchAndCacheMode = false,
IDeploymentArtifactFetcher? artifactFetcher = null,
IAlarmStateStore? alarmStateStore = null,
ITelemetryLocalHub? telemetryHub = null)
ITelemetryLocalHub? telemetryHub = null,
IDriverPrimaryRoleSource? primaryRoleSource = null)
{
_telemetryHub = telemetryHub;
_deploymentArtifactCache = deploymentArtifactCache;
_redundancyRoleView = redundancyRoleView;
_primaryRoleSource = primaryRoleSource;
_replicationPeerHost = replicationPeerHost;
_fetchAndCacheMode = fetchAndCacheMode;
_artifactFetcher = artifactFetcher;
@@ -1331,6 +1353,12 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
_redundancyRoleView?.Publish(
PrimaryGatePolicy.ShouldDrainAlarmHistory(_localRole, queueSharingPeerIsPrimary: peerIsPrimary));
// The DATA-PLANE verdict, deliberately — the same one HandleRouteNodeWrite gates on, not the
// drain verdict published immediately above. A driver reading this claims an exclusive identity
// on a shared external system (Gitea #511: the Sparkplug host id), where two claimants is the
// harm, so an unknown role with a driver peer present must deny.
_primaryRoleSource?.PublishPrimaryRole(ShouldServiceAsPrimary());
}
private void Stale()