fix(historian): capture while the recording Primary is unreachable (#672)
A SIGKILLed Primary stays an unreachable-but-not-downed member until auto-down, and the redundancy singleton usually dies with it, so the standby's cached snapshot kept naming it Primary and nobody captured for ~30 s. DriverHostActor now holds its own ReachabilityEvent (+MemberUp) subscription, caches the last snapshot, and excludes unreachable peers from the historization peer-is-Primary predicate only, republishing just the capture verdict on a reachability change. Alarm drain and the Primary data-plane verdict are unchanged. Gated by ContinuousHistorization:CaptureWhilePrimaryUnreachable (default true). Tests: message-driven cases with real ReachabilityEvents (incl. a falsification control on the drain/data-plane verdicts) plus a real two-node cluster crash test with downing disabled.
This commit is contained in:
@@ -129,6 +129,25 @@ public sealed class ContinuousHistorizationOptions
|
||||
/// <summary>Cap (seconds) on the exponential retry backoff after repeated drain failures. Defaults to 30.</summary>
|
||||
public double MaxBackoffSeconds { get; init; } = 30;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #672: when <c>true</c> (the default), a redundant node whose capture is stood down for its
|
||||
/// mesh-partner Primary (Gitea #537) resumes capturing as soon as the cluster failure detector reports
|
||||
/// that Primary <b>unreachable</b>, instead of waiting for auto-down and a fresh redundancy snapshot.
|
||||
/// <para>
|
||||
/// A crashed Primary otherwise leaves a capture hole of failure-detection + auto-down (~25 s on the
|
||||
/// shipped <c>akka.conf</c>): the redundancy singleton usually died with it, so the standby's last
|
||||
/// snapshot keeps naming the dead node Primary. With this on, the hole shrinks to the failure-detector
|
||||
/// window (~10–12 s). The cost is duplicates: during a genuine partition both nodes capture from
|
||||
/// detection onward, and a briefly-unreachable-but-alive Primary overlaps the standby. SqlServer and
|
||||
/// QuestDB absorb a duplicate of a source-timestamped sample idempotently; a driver with no source
|
||||
/// timestamp (e.g. Modbus, stamped with each node's own poll time) writes a duplicate row on every
|
||||
/// backend, and the Gateway backend stores duplicates. <c>false</c> restores the pre-#672 behaviour.
|
||||
/// Only the historization capture verdict is affected — never the alarm drain or the Primary data-plane
|
||||
/// gate.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public bool CaptureWhilePrimaryUnreachable { get; init; } = true;
|
||||
|
||||
/// <summary>Returns operator-facing misconfiguration warnings for an <c>Enabled</c> recorder
|
||||
/// (empty when disabled or correctly configured). Pure — the registration logs each entry.</summary>
|
||||
/// <returns>Zero or more human-readable warning messages (never carrying secret values).</returns>
|
||||
|
||||
Reference in New Issue
Block a user