232c03ed1b
- HistoryUpdate Primary gate now reads IHistoryUpdateRoleView (PrimaryGatePolicy.ShouldServeHistoryUpdate): the data-plane verdict but CLOSED until this node's first redundancy snapshot, so both pair nodes cannot accept in the pre-join boot window. Single nodes open on their own singleton's first snapshot; admin-only nodes register no view => no gate. DriverHostActor publishes it at every Primary-verdict publish point. - SqlServer v1: the updater throws NotSupportedException on every arm; the node manager maps it to a node-level BadHistoryOperationUnsupported (was per-item under a Good node for UpdateData/DeleteAtTime). HistoryWrite bit still advertised on v1 (not knowable at materialise time) — documented. - The update plan is evaluated before the Primary gate: an unservable request (Remove, IsDeleteModified, not historized, no updater) is BadHistoryOperationUnsupported off-Primary, not BadInvalidState. - Local-kind SourceTimestamp converted to UTC before the future-skew compare and sent as UTC. - Hosted-service wiring extracted to BuildHistoryUpdatePosture and tested (gate source, skew, clock). - Docs: Historian.md (boot window, ordering, node-level v1, HistoryWrite bit), CLAUDE.md. Co-Authored-By: Claude <noreply@anthropic.com>
164 lines
11 KiB
C#
164 lines
11 KiB
C#
using ZB.MOM.WW.OtOpcUa.Commons.Messages.Redundancy;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
|
|
|
|
/// <summary>
|
|
/// Single decision point for the Primary data-plane gates (inbound device writes, native alarm acks,
|
|
/// fleet-wide alerts publishes). A KNOWN role wins outright; an UNKNOWN role (no redundancy snapshot
|
|
/// yet, or the snapshot never mentioned this node — the 03/S5 identity-mismatch shape) is resolved by
|
|
/// cluster membership: a single-driver cluster stays default-ALLOW (no peer exists — the
|
|
/// boot-window / single-node posture), a multi-driver cluster is default-DENY (a real Primary peer
|
|
/// exists; do not touch the shared field device until the redundancy snapshot proves this node is it).
|
|
/// Fixes archreview 03/S4 (primary gate defaulted to allow while role unknown → dual-primary window).
|
|
/// </summary>
|
|
public static class PrimaryGatePolicy
|
|
{
|
|
/// <summary>Decide whether this node should service a Primary-only data-plane operation.</summary>
|
|
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown
|
|
/// (no snapshot received yet, or the snapshot omitted this node).</param>
|
|
/// <param name="driverMemberCount">Count of Up cluster members carrying the <c>driver</c> role.</param>
|
|
/// <returns><c>true</c> to service as Primary; <c>false</c> to deny.</returns>
|
|
public static bool ShouldServiceAsPrimary(RedundancyRole? localRole, int driverMemberCount) =>
|
|
localRole switch
|
|
{
|
|
RedundancyRole.Primary => true,
|
|
RedundancyRole.Secondary or RedundancyRole.Detached => false,
|
|
_ => driverMemberCount <= 1, // unknown: allow only when no driver peer exists
|
|
};
|
|
|
|
/// <summary>
|
|
/// Gitea #674 review: decide whether this node may serve an OPC UA <b>HistoryUpdate</b>. The
|
|
/// <see cref="ShouldServiceAsPrimary"/> verdict, but CLOSED until this node has received its first
|
|
/// redundancy snapshot.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <see cref="ShouldServiceAsPrimary"/> deliberately allows an unknown role while no driver peer is
|
|
/// visible, which covers the pre-join boot window: a node spawned before it has joined its mesh sees a
|
|
/// member count of 0. For a live data-plane operation (a device write, an ack) that is the right trade —
|
|
/// denying would stall a genuine single node during its own join. HistoryUpdate is not live: a client can
|
|
/// retry it, and both nodes of a pair accepting one concurrently during that window is exactly the
|
|
/// wrong-prior-value race the Primary gate exists for. So it waits for a real verdict.
|
|
/// </para>
|
|
/// <para>
|
|
/// "No snapshot yet" is told apart from "single node" by the snapshot itself: the redundancy singleton runs
|
|
/// on every driver node (one per mesh, a single node included) and publishes on every change and every
|
|
/// 10 s heartbeat, so a single node is named Primary within seconds of joining. Once a snapshot has
|
|
/// arrived this is exactly <see cref="ShouldServiceAsPrimary"/> (including its unknown-role member-count
|
|
/// rule for a snapshot that omits this node). A node with no redundancy manager at all (admin-only, or a
|
|
/// harness) wires no gate and so allows.
|
|
/// </para>
|
|
/// </remarks>
|
|
/// <param name="redundancySnapshotReceived">Whether any redundancy snapshot has reached this node yet.</param>
|
|
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
|
|
/// <param name="driverMemberCount">Count of Up cluster members carrying the <c>driver</c> role.</param>
|
|
/// <returns><c>true</c> to serve HistoryUpdate; <c>false</c> to refuse it (<c>BadInvalidState</c>).</returns>
|
|
public static bool ShouldServeHistoryUpdate(bool redundancySnapshotReceived, RedundancyRole? localRole, int driverMemberCount) =>
|
|
redundancySnapshotReceived && ShouldServiceAsPrimary(localRole, driverMemberCount);
|
|
|
|
/// <summary>
|
|
/// Decide whether this node should drain the replicated alarm store-and-forward queue.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>Deliberately not <see cref="ShouldServiceAsPrimary"/>.</b> That gate protects a shared
|
|
/// field device, where two nodes acting at once is dangerous and irreversible, so an unknown
|
|
/// role with a peer present must deny. This gate protects an append-only historian, and the
|
|
/// delivery contract is already <i>at-least-once across a failover, by design</i> — identical
|
|
/// events even collapse to a single row, because ids hash the payload.
|
|
/// </para>
|
|
/// <para>
|
|
/// The two error costs are therefore asymmetric: a false allow costs a duplicate history row;
|
|
/// a false deny silently stops the alarm audit trail and eventually evicts it at the capacity
|
|
/// wall. So this decision keys on the role <b>alone</b> and opens when the role is unknown —
|
|
/// no membership tie-break, because cluster-wide driver count says nothing about whether
|
|
/// <i>this</i> node has a redundant partner.
|
|
/// </para>
|
|
/// <para>
|
|
/// Found by the LocalDb Phase 2 live gate: on a rig of four driver nodes in one cluster with
|
|
/// no <c>Redundancy</c> section, borrowing the write gate's verdict suspended the drain on
|
|
/// every node — including unpaired ones — so nothing drained anywhere.
|
|
/// </para>
|
|
/// </remarks>
|
|
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
|
|
/// <param name="queueSharingPeerIsPrimary">
|
|
/// Whether the node holding <see cref="RedundancyRole.Primary"/> in the same snapshot is a node
|
|
/// that <b>shares this node's queue</b> — i.e. its LocalDb replication partner.
|
|
/// <para>
|
|
/// Merely knowing that <i>a</i> Primary exists is not enough, because the redundancy role is
|
|
/// a CLUSTER-WIDE election while the alarm queue is PAIR-LOCAL. A fleet runs many pairs in
|
|
/// one cluster, so the elected driver Primary is usually in some other pair — and on the
|
|
/// docker-dev rig it is a central node, which carries the driver Akka role, replicates
|
|
/// nobody's LocalDb and does not even run the alarm historian. Deferring to it means this
|
|
/// node's events are delivered by no one, ever.
|
|
/// </para>
|
|
/// </param>
|
|
/// <returns><c>true</c> to drain; <c>false</c> only when a node holding these same rows is doing it.</returns>
|
|
public static bool ShouldDrainAlarmHistory(RedundancyRole? localRole, bool queueSharingPeerIsPrimary) =>
|
|
localRole switch
|
|
{
|
|
RedundancyRole.Primary => true,
|
|
RedundancyRole.Secondary or RedundancyRole.Detached => !queueSharingPeerIsPrimary,
|
|
_ => true, // unknown role ⇒ drain; a duplicate row beats a silent gap
|
|
};
|
|
|
|
/// <summary>
|
|
/// Decide whether this node should <b>capture</b> continuous-historization values into its
|
|
/// durable outbox (Gitea #537).
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// <b>Same rule as <see cref="ShouldDrainAlarmHistory"/>, deliberately — and for the same
|
|
/// reason.</b> The historian is append-only and the error costs are asymmetric: a false
|
|
/// allow costs a duplicate sample, a false deny costs a permanent hole in the trend. So an
|
|
/// unknown role captures, and only an explicit Secondary/Detached stands down.
|
|
/// </para>
|
|
/// <para>
|
|
/// <b>But it gates the CAPTURE, not the drain — the opposite of the alarm sink.</b> That
|
|
/// choice follows from where the queue lives. The alarm queue is in LocalDb and
|
|
/// <i>replicates</i>, so the Secondary already holds the Primary's rows and gating the
|
|
/// drain loses nothing. The historization outbox is a node-local FasterLog that replicates
|
|
/// to nobody: gating only its drain would leave the standby appending forever to a queue no
|
|
/// one empties (unbounded disk, and a flood of stale duplicates the moment it is promoted),
|
|
/// while gating the capture leaves it holding nothing. Values already captured are still
|
|
/// drained by the node that captured them, so a demotion loses nothing that was recorded.
|
|
/// </para>
|
|
/// <para>
|
|
/// The residual cost is a capture gap across a failover, bounded by how long the newly
|
|
/// promoted node takes to see its first snapshot. That is inherent to any non-duplicating
|
|
/// design; the alternative — both nodes capturing — is what #537 was filed about.
|
|
/// </para>
|
|
/// <para>
|
|
/// Gitea #672: on a CRASH the newly promoted node may not see a snapshot until auto-down (the
|
|
/// redundancy singleton died with the old Primary), so the caller excludes a Primary the cluster
|
|
/// failure detector reports unreachable from <paramref name="addressSpaceSharingPeerIsPrimary"/> —
|
|
/// shrinking the gap to the failure-detector window. This method is unchanged by that; it only
|
|
/// sees the narrower input.
|
|
/// </para>
|
|
/// </remarks>
|
|
/// <param name="localRole">This node's last-known redundancy role, or <c>null</c> when unknown.</param>
|
|
/// <param name="addressSpaceSharingPeerIsPrimary">
|
|
/// Whether the node holding <see cref="RedundancyRole.Primary"/> in the same snapshot is one
|
|
/// that serves <b>this node's address space</b>, and therefore captures the very same historized
|
|
/// refs — its per-cluster mesh partner.
|
|
/// <para>
|
|
/// The scoping matters for the same reason it does on the alarm queue, and the mistake is
|
|
/// easier to make here because it looks safe: the redundancy election reaches only as far as
|
|
/// the caller can prove, whereas historization is fleet-wide by tag name. A node that stands
|
|
/// down in favour of a Primary running some <i>other</i> cluster's configuration hands its
|
|
/// tags to a node that never records them, and nothing says so — the drain simply reports
|
|
/// success on an empty outbox forever. Callers that cannot establish the sharing pass
|
|
/// <c>false</c>, which never stands down; duplicates are recoverable, a silent gap is not.
|
|
/// </para>
|
|
/// </param>
|
|
/// <returns><c>true</c> to capture; <c>false</c> only when a node recording these same tags is doing it.</returns>
|
|
public static bool ShouldRecordHistorizedValues(
|
|
RedundancyRole? localRole, bool addressSpaceSharingPeerIsPrimary) =>
|
|
localRole switch
|
|
{
|
|
RedundancyRole.Primary => true,
|
|
RedundancyRole.Secondary or RedundancyRole.Detached => !addressSpaceSharingPeerIsPrimary,
|
|
_ => true, // unknown role ⇒ capture; a duplicate sample beats a hole in the trend
|
|
};
|
|
}
|