feat(cluster)!: leader-auto-down is the shipped default downing strategy (#116)
SplitBrainResolverStrategy now defaults to leader-auto-down (the in-repo LeaderAutoDowningProvider, #106, with the #132 shut-out restart on). auto-down and keep-oldest stay supported. Drilled on the docker rig, docker-env2 and the QA pair (central + site, kill and stop-and-hold) before the flip. - ClusterOptions default + validator/HOCON comments updated. - Shipped Host appsettings.Central/Site.json and every docker/ and docker-env2/ node appsettings now say leader-auto-down. The leader-auto-down overlays are kept as no-ops (drill records reference them); new docker-compose.auto-down.yml overlays keep Akka AutoDowning available for comparison drills. - Tests: default asserts leader-auto-down; new HOCON test for an unset strategy; TwoNodeClusterFixture defaults to the production posture. - Docs: CLAUDE.md, Component-ClusterInfrastructure, components/Host + Cluster, README, failover procedures, Traefik, docker + env2 READMEs; register row 70 closed. No deploy needed: DEV, QA and both rigs set the key explicitly. Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
This commit is contained in:
+4
-2
@@ -400,14 +400,16 @@ DRILL_ACTION=pause DRILL_MODE=active bash docker/failover-drill.sh # freeze ins
|
||||
|
||||
The rejoin check is strict. The victim must report `/health/ready` 200 and `/health/active` 503, and the survivor must report `memberCount 2, unreachableCount 0`. After that, 10 one-second samples must each show exactly one active node.
|
||||
|
||||
**Trying `leader-auto-down` (ScadaBridge#106/#116).** The opt-in overlay `docker/docker-compose.leader-auto-down.yml` switches the central and site-a pairs to the in-repo provider through an environment override. The shipped default stays `auto-down`. Switch both pairs together, never one node at a time:
|
||||
**Downing provider (ScadaBridge#106/#116).** Since 2026-11-15 the shipped default is `leader-auto-down`, the in-repo `LeaderAutoDowningProvider`, and every node's mounted appsettings say so. It was drilled on this rig, on docker-env2 and on the QA pair before the flip. To compare against Akka's deprecated `AutoDowning` provider (the default from 2026-07-21 until 2026-11-15), the opt-in overlay `docker/docker-compose.auto-down.yml` switches the central and site-a pairs back through an environment override. Switch both pairs together, never one node at a time:
|
||||
|
||||
```bash
|
||||
docker compose -f docker/docker-compose.yml -f docker/docker-compose.leader-auto-down.yml \
|
||||
docker compose -f docker/docker-compose.yml -f docker/docker-compose.auto-down.yml \
|
||||
up -d --force-recreate --no-deps central-a central-b site-a-a site-a-b
|
||||
# back to the default: the same command without the overlay file
|
||||
```
|
||||
|
||||
`docker/docker-compose.leader-auto-down.yml` is now a no-op, kept because the #116 drill records reference it.
|
||||
|
||||
The scripted drill (`docker kill` = SIGKILL, the hard-crash path — a `docker stop` would take the graceful `CoordinatedShutdown` path and would not prove crash recovery) has **two modes**, and since the **auto-down decision (2026-07-21)** both expect recovery — the cluster runs Akka's `AutoDowning` provider (`auto-down-unreachable-after` = 15s), under which the leader among the *reachable* members downs the unreachable peer, so a crash of either node fails over:
|
||||
|
||||
- **`DRILL_MODE=standby` (default) — kills the STANDBY (younger) central node.** The active node is untouched: expected result is **no routing outage at all** (`/health/active` blips = 0) and member removal on the survivor within **~25s** (10s failure-detection threshold + 15s auto-down window; the 2s heartbeat interval is not additive). PASS = the survivor logs the downing/removal within `TIMEOUT_S` (default 90s) while routing stays up.
|
||||
|
||||
Reference in New Issue
Block a user