Files
lmxopcua/src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/PrimaryGatePolicy.cs
T
Joseph Doherty 232c03ed1b fix(historian): #674 review follow-ups — HistoryUpdate boot-window gate, node-level v1 refusal, plan-before-gate, UTC skew compare
- HistoryUpdate Primary gate now reads IHistoryUpdateRoleView (PrimaryGatePolicy.ShouldServeHistoryUpdate):
  the data-plane verdict but CLOSED until this node's first redundancy snapshot, so both pair nodes cannot
  accept in the pre-join boot window. Single nodes open on their own singleton's first snapshot; admin-only
  nodes register no view => no gate. DriverHostActor publishes it at every Primary-verdict publish point.
- SqlServer v1: the updater throws NotSupportedException on every arm; the node manager maps it to a
  node-level BadHistoryOperationUnsupported (was per-item under a Good node for UpdateData/DeleteAtTime).
  HistoryWrite bit still advertised on v1 (not knowable at materialise time) — documented.
- The update plan is evaluated before the Primary gate: an unservable request (Remove, IsDeleteModified,
  not historized, no updater) is BadHistoryOperationUnsupported off-Primary, not BadInvalidState.
- Local-kind SourceTimestamp converted to UTC before the future-skew compare and sent as UTC.
- Hosted-service wiring extracted to BuildHistoryUpdatePosture and tested (gate source, skew, clock).
- Docs: Historian.md (boot window, ordering, node-level v1, HistoryWrite bit), CLAUDE.md.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-10-11 11:41:11 -04:00

164 lines
11 KiB
C#

using ZB.MOM.WW.OtOpcUa.Commons.Messages.Redundancy;
namespace ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
/// <summary>
/// Single decision point for the Primary data-plane gates (inbound device writes, native alarm acks,
/// fleet-wide alerts publishes). A KNOWN role wins outright; an UNKNOWN role (no redundancy snapshot
/// yet, or the snapshot never mentioned this node — the 03/S5 identity-mismatch shape) is resolved by
/// cluster membership: a single-driver cluster stays default-ALLOW (no peer exists — the
/// boot-window / single-node posture), a multi-driver cluster is default-DENY (a real Primary peer
/// exists; do not touch the shared field device until the redundancy snapshot proves this node is it).
/// Fixes archreview 03/S4 (primary gate defaulted to allow while role unknown → dual-primary window).
/// </summary>
public static class PrimaryGatePolicy
{
/// <summary>Decide whether this node should service a Primary-only data-plane operation.</summary>
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown
/// (no snapshot received yet, or the snapshot omitted this node).</param>
/// <param name="driverMemberCount">Count of Up cluster members carrying the <c>driver</c> role.</param>
/// <returns><c>true</c> to service as Primary; <c>false</c> to deny.</returns>
public static bool ShouldServiceAsPrimary(RedundancyRole? localRole, int driverMemberCount) =>
localRole switch
{
RedundancyRole.Primary => true,
RedundancyRole.Secondary or RedundancyRole.Detached => false,
_ => driverMemberCount <= 1, // unknown: allow only when no driver peer exists
};
/// <summary>
/// Gitea #674 review: decide whether this node may serve an OPC UA <b>HistoryUpdate</b>. The
/// <see cref="ShouldServiceAsPrimary"/> verdict, but CLOSED until this node has received its first
/// redundancy snapshot.
/// </summary>
/// <remarks>
/// <para>
/// <see cref="ShouldServiceAsPrimary"/> deliberately allows an unknown role while no driver peer is
/// visible, which covers the pre-join boot window: a node spawned before it has joined its mesh sees a
/// member count of 0. For a live data-plane operation (a device write, an ack) that is the right trade —
/// denying would stall a genuine single node during its own join. HistoryUpdate is not live: a client can
/// retry it, and both nodes of a pair accepting one concurrently during that window is exactly the
/// wrong-prior-value race the Primary gate exists for. So it waits for a real verdict.
/// </para>
/// <para>
/// "No snapshot yet" is told apart from "single node" by the snapshot itself: the redundancy singleton runs
/// on every driver node (one per mesh, a single node included) and publishes on every change and every
/// 10 s heartbeat, so a single node is named Primary within seconds of joining. Once a snapshot has
/// arrived this is exactly <see cref="ShouldServiceAsPrimary"/> (including its unknown-role member-count
/// rule for a snapshot that omits this node). A node with no redundancy manager at all (admin-only, or a
/// harness) wires no gate and so allows.
/// </para>
/// </remarks>
/// <param name="redundancySnapshotReceived">Whether any redundancy snapshot has reached this node yet.</param>
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
/// <param name="driverMemberCount">Count of Up cluster members carrying the <c>driver</c> role.</param>
/// <returns><c>true</c> to serve HistoryUpdate; <c>false</c> to refuse it (<c>BadInvalidState</c>).</returns>
public static bool ShouldServeHistoryUpdate(bool redundancySnapshotReceived, RedundancyRole? localRole, int driverMemberCount) =>
redundancySnapshotReceived && ShouldServiceAsPrimary(localRole, driverMemberCount);
/// <summary>
/// Decide whether this node should drain the replicated alarm store-and-forward queue.
/// </summary>
/// <remarks>
/// <para>
/// <b>Deliberately not <see cref="ShouldServiceAsPrimary"/>.</b> That gate protects a shared
/// field device, where two nodes acting at once is dangerous and irreversible, so an unknown
/// role with a peer present must deny. This gate protects an append-only historian, and the
/// delivery contract is already <i>at-least-once across a failover, by design</i> — identical
/// events even collapse to a single row, because ids hash the payload.
/// </para>
/// <para>
/// The two error costs are therefore asymmetric: a false allow costs a duplicate history row;
/// a false deny silently stops the alarm audit trail and eventually evicts it at the capacity
/// wall. So this decision keys on the role <b>alone</b> and opens when the role is unknown —
/// no membership tie-break, because cluster-wide driver count says nothing about whether
/// <i>this</i> node has a redundant partner.
/// </para>
/// <para>
/// Found by the LocalDb Phase 2 live gate: on a rig of four driver nodes in one cluster with
/// no <c>Redundancy</c> section, borrowing the write gate's verdict suspended the drain on
/// every node — including unpaired ones — so nothing drained anywhere.
/// </para>
/// </remarks>
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
/// <param name="queueSharingPeerIsPrimary">
/// Whether the node holding <see cref="RedundancyRole.Primary"/> in the same snapshot is a node
/// that <b>shares this node's queue</b> — i.e. its LocalDb replication partner.
/// <para>
/// Merely knowing that <i>a</i> Primary exists is not enough, because the redundancy role is
/// a CLUSTER-WIDE election while the alarm queue is PAIR-LOCAL. A fleet runs many pairs in
/// one cluster, so the elected driver Primary is usually in some other pair — and on the
/// docker-dev rig it is a central node, which carries the driver Akka role, replicates
/// nobody's LocalDb and does not even run the alarm historian. Deferring to it means this
/// node's events are delivered by no one, ever.
/// </para>
/// </param>
/// <returns><c>true</c> to drain; <c>false</c> only when a node holding these same rows is doing it.</returns>
public static bool ShouldDrainAlarmHistory(RedundancyRole? localRole, bool queueSharingPeerIsPrimary) =>
localRole switch
{
RedundancyRole.Primary => true,
RedundancyRole.Secondary or RedundancyRole.Detached => !queueSharingPeerIsPrimary,
_ => true, // unknown role ⇒ drain; a duplicate row beats a silent gap
};
/// <summary>
/// Decide whether this node should <b>capture</b> continuous-historization values into its
/// durable outbox (Gitea #537).
/// </summary>
/// <remarks>
/// <para>
/// <b>Same rule as <see cref="ShouldDrainAlarmHistory"/>, deliberately — and for the same
/// reason.</b> The historian is append-only and the error costs are asymmetric: a false
/// allow costs a duplicate sample, a false deny costs a permanent hole in the trend. So an
/// unknown role captures, and only an explicit Secondary/Detached stands down.
/// </para>
/// <para>
/// <b>But it gates the CAPTURE, not the drain — the opposite of the alarm sink.</b> That
/// choice follows from where the queue lives. The alarm queue is in LocalDb and
/// <i>replicates</i>, so the Secondary already holds the Primary's rows and gating the
/// drain loses nothing. The historization outbox is a node-local FasterLog that replicates
/// to nobody: gating only its drain would leave the standby appending forever to a queue no
/// one empties (unbounded disk, and a flood of stale duplicates the moment it is promoted),
/// while gating the capture leaves it holding nothing. Values already captured are still
/// drained by the node that captured them, so a demotion loses nothing that was recorded.
/// </para>
/// <para>
/// The residual cost is a capture gap across a failover, bounded by how long the newly
/// promoted node takes to see its first snapshot. That is inherent to any non-duplicating
/// design; the alternative — both nodes capturing — is what #537 was filed about.
/// </para>
/// <para>
/// Gitea #672: on a CRASH the newly promoted node may not see a snapshot until auto-down (the
/// redundancy singleton died with the old Primary), so the caller excludes a Primary the cluster
/// failure detector reports unreachable from <paramref name="addressSpaceSharingPeerIsPrimary"/> —
/// shrinking the gap to the failure-detector window. This method is unchanged by that; it only
/// sees the narrower input.
/// </para>
/// </remarks>
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
/// <param name="addressSpaceSharingPeerIsPrimary">
/// Whether the node holding <see cref="RedundancyRole.Primary"/> in the same snapshot is one
/// that serves <b>this node's address space</b>, and therefore captures the very same historized
/// refs — its per-cluster mesh partner.
/// <para>
/// The scoping matters for the same reason it does on the alarm queue, and the mistake is
/// easier to make here because it looks safe: the redundancy election reaches only as far as
/// the caller can prove, whereas historization is fleet-wide by tag name. A node that stands
/// down in favour of a Primary running some <i>other</i> cluster's configuration hands its
/// tags to a node that never records them, and nothing says so — the drain simply reports
/// success on an empty outbox forever. Callers that cannot establish the sharing pass
/// <c>false</c>, which never stands down; duplicates are recoverable, a silent gap is not.
/// </para>
/// </param>
/// <returns><c>true</c> to capture; <c>false</c> only when a node recording these same tags is doing it.</returns>
public static bool ShouldRecordHistorizedValues(
RedundancyRole? localRole, bool addressSpaceSharingPeerIsPrimary) =>
localRole switch
{
RedundancyRole.Primary => true,
RedundancyRole.Secondary or RedundancyRole.Detached => !addressSpaceSharingPeerIsPrimary,
_ => true, // unknown role ⇒ capture; a duplicate sample beats a hole in the trend
};
}