feat(cluster): opt-in leader-auto-down strategy - in-repo port of Akka AutoDowning (ScadaBridge#106)

Adds LeaderAutoDowningProvider (Host/Actors), a line-for-line port of Akka.NET
1.5.62 AutoDowning/AutoDown/AutoDownBase read from the decompiled assembly: the
leader among the reachable members downs a peer unreachable for StableAfter; a
non-leader records it as pending and downs it on becoming leader; reachable-again
or removed members are forgotten; Down/Exiting members are skipped. Its window is
carried under scadabridge.cluster.leader-auto-down.unreachable-after, so no
deprecated auto-down-unreachable-after key is emitted and Akka's boot-time
"keep-majority will be used instead" warning goes away. DownRemovalMargin reads
akka.cluster.down-removal-margin exactly as AutoDowning does (default off).

New SplitBrainResolverStrategy value "leader-auto-down", wired through BuildHocon
and ClusterOptionsValidator. The default stays "auto-down"; auto-down and
keep-oldest HOCON is byte-identical to before. Placed in Host, not
ClusterInfrastructure, per the Code Placement rule (ClusterInfrastructure stays
Akka-free).

Tests: HOCON + validator units; LeaderAutoDownTests ports Akka's AutoDownSpec
cases (leader-only, pending-on-leader-change, losing leadership, reachable again,
removed, Down/Exiting skipped, single Down per member); LeaderAutoDownFailoverTests
clones SbrFailoverTests' oldest-crash and younger-crash cases on a real two-node
cluster plus a test that Akka actually loaded the provider and its downing actor.
Mutation-checked: a provider that never calls Cluster.Down fails both failover
tests; never-downs, leader-check-dropped, reachable-ignored and pending-not-flushed
mutations each fail the unit tests.

Remaining gate before switching the default: the docker failover drill
(DRILL_MODE=active and standby mode) on leader-auto-down.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 16:46:33 -05:00
parent dd99a9f5a5
commit ab561faa43
12 changed files with 805 additions and 10 deletions
@@ -1,6 +1,13 @@
# Replacing Akka `AutoDowning` before an Akka.NET upgrade removes it (plan, 2026-11-09)
**Status: PLAN ONLY (ScadaBridge#51, register row 70).** No config or code changes are made by this document.
**Update 2026-11-09 (ScadaBridge#106):** option B is implemented as the opt-in strategy `leader-auto-down`
(`LeaderAutoDowningProvider`, placed in **Host** rather than ClusterInfrastructure: the Code Placement rule in
`Component-ClusterInfrastructure.md` keeps ClusterInfrastructure Akka-free, and the provider is runtime wiring).
`DownRemovalMargin` matches `AutoDowning` exactly (`akka.cluster.down-removal-margin`, default off), not a `StableAfter`-derived value.
Unit, HOCON, validator and two-node real-cluster tests are green. Remaining gate before switching the default: the
docker failover drill (section 6) on the new value, then the per-environment rollout in section 5.
Related decision: [`2026-07-21-auto-down-availability-decision.md`](2026-07-21-auto-down-availability-decision.md).
## 1. Current state (read from the code)