fix(historian): capture while the recording Primary is unreachable (#672)

A SIGKILLed Primary stays an unreachable-but-not-downed member until
auto-down, and the redundancy singleton usually dies with it, so the
standby's cached snapshot kept naming it Primary and nobody captured for
~30 s. DriverHostActor now holds its own ReachabilityEvent (+MemberUp)
subscription, caches the last snapshot, and excludes unreachable peers
from the historization peer-is-Primary predicate only, republishing just
the capture verdict on a reachability change. Alarm drain and the Primary
data-plane verdict are unchanged. Gated by
ContinuousHistorization:CaptureWhilePrimaryUnreachable (default true).

Tests: message-driven cases with real ReachabilityEvents (incl. a
falsification control on the drain/data-plane verdicts) plus a real
two-node cluster crash test with downing disabled.
This commit is contained in:
Joseph Doherty
2026-10-11 08:26:39 -04:00
parent db6a53036b
commit 1b2c36eb87
11 changed files with 573 additions and 21 deletions
@@ -625,6 +625,9 @@ public static class ServiceCollectionExtensions
// verdict, and how it decides which peer is worth standing down for.
historizationRoleView: resolver.GetService<IHistorizationRoleView>(),
sharesHistorizationScopeWithLocal: historizationScopePeer,
// Gitea #672: stop deferring the capture to a recording Primary the failure detector
// already reports unreachable. Unbound section ⇒ the option's default (true).
captureWhilePrimaryUnreachable: continuousOptions?.CaptureWhilePrimaryUnreachable ?? true,
// Review R6 MINOR-4: the passive half of a LocalDb pair has no PeerAddress, so it is told
// its queue-sharing peer as its mesh partner instead — otherwise it never stands down.
sharesAlarmQueueWithLocal: BuildAlarmQueuePeerPredicate(