feat(cluster)!: leader-auto-down is the shipped default downing strategy (#116)
SplitBrainResolverStrategy now defaults to leader-auto-down (the in-repo LeaderAutoDowningProvider, #106, with the #132 shut-out restart on). auto-down and keep-oldest stay supported. Drilled on the docker rig, docker-env2 and the QA pair (central + site, kill and stop-and-hold) before the flip. - ClusterOptions default + validator/HOCON comments updated. - Shipped Host appsettings.Central/Site.json and every docker/ and docker-env2/ node appsettings now say leader-auto-down. The leader-auto-down overlays are kept as no-ops (drill records reference them); new docker-compose.auto-down.yml overlays keep Akka AutoDowning available for comparison drills. - Tests: default asserts leader-auto-down; new HOCON test for an unset strategy; TwoNodeClusterFixture defaults to the production posture. - Docs: CLAUDE.md, Component-ClusterInfrastructure, components/Host + Cluster, README, failover procedures, Traefik, docker + env2 READMEs; register row 70 closed. No deploy needed: DEV, QA and both rigs set the key explicitly. Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
This commit is contained in:
@@ -8,7 +8,7 @@
|
||||
|
||||
1. Active central node becomes unreachable (process crash, network failure, hardware failure).
|
||||
2. Akka.NET failure detection triggers after ~10 seconds (2s heartbeat, 10s threshold).
|
||||
3. `auto-down`: the leader among the reachable members downs the unreachable node after 15 seconds of sustained unreachability.
|
||||
3. `leader-auto-down` (the default since 2026-11-15, ScadaBridge#116; same rule as the former default `auto-down`): the leader among the reachable members downs the unreachable node after 15 seconds of sustained unreachability. The survivor logs `[leader-auto-down] DOWNING <address>`.
|
||||
4. Standby node is promoted to active. Total time: ~25 seconds.
|
||||
5. Cluster singletons migrate to the new active node.
|
||||
6. Load balancer detects the failed node via `/health/ready` and routes traffic to the surviving node.
|
||||
@@ -82,7 +82,7 @@
|
||||
**Symptoms:** Both nodes believe they are the active node. Logs show "Cluster partition detected".
|
||||
|
||||
**How the system handles it:**
|
||||
- `auto-down` (decision 2026-07-21): each side downs the unreachable peer after 15 s and continues as a one-node cluster, so a real partition leaves BOTH nodes active until an operator restarts one side (accepted trade for ~25 s failover on either-node crash).
|
||||
- `leader-auto-down` (default since 2026-11-15, ScadaBridge#116) and `auto-down` (decision 2026-07-21; the default until then): each side downs the unreachable peer after 15 s and continues as a one-node cluster, so a real partition leaves BOTH nodes active (accepted trade for ~25 s failover on either-node crash). Under `leader-auto-down` the #132 shut-out restart then restarts the stale side when the partition heals (exit 75, the service manager restarts it); under `auto-down` an operator restarts one side.
|
||||
- `keep-oldest` (opt-in): the older node survives and the younger is downed; but a crash of the oldest is a total outage.
|
||||
- Stable-after (15s): the window of sustained unreachability before the downing decision.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user