docs(cluster): per-direction failover truth — first-seed bootstrap constraint + operator recovery actions; drop the undeliverable ~25s active-crash promise (plan R2-01 T2)

docker/README.md

@@ -273,19 +273,25 @@ All test passwords are `password`. See `infra/glauth/config.toml` for the full l
 ### Automated failover drill (`failover-drill.sh`)
 
 ```bash
-bash docker/failover-drill.sh
+DRILL_MODE=standby bash docker/failover-drill.sh   # default — survivable younger-node crash
+DRILL_MODE=active  bash docker/failover-drill.sh    # oldest-node crash — measures the registered outage gap
 ```
 
-The scripted drill is the repeatable version of the manual steps below. It:
+The scripted drill (`docker kill` = SIGKILL, the hard-crash path — a `docker stop` would take the graceful `CoordinatedShutdown` path and would not prove crash recovery) has **two modes**, because under the unified oldest-member semantics the *active* node IS the oldest, i.e. the one crash two-node keep-oldest cannot survive:
 
-1. Finds the **active** central node (via each node's `/health/active`).
+- **`DRILL_MODE=standby` (default) — kills the STANDBY (younger) central node.** The survivable direction: SBR downs the crashed member and the active node keeps its singletons. Expected result: **no routing outage at all** (the active node is never touched, so `/health/active` blips = 0) and member removal on the survivor within **~25s** (10s failure-detection threshold + 15s stable-after; the 2s heartbeat interval is not additive). PASS = the survivor logs the member removal within `TIMEOUT_S` (default 90s) while routing stays up.
-2. **`docker kill`s it — SIGKILL, the hard-crash path.** This is the exact scenario arch-review 01 found had *no* automatic recovery before the SBR downing provider was enabled (`Akka.Cluster.SBR.SplitBrainResolverProvider` is now named explicitly in HOCON); a `docker stop` would take the graceful `CoordinatedShutdown` path instead and would not prove crash recovery.
+- **`DRILL_MODE=active` — kills the ACTIVE (oldest) central node.** Expected result: a **total central outage** until the victim container is restarted — this is the registered deferred keep-oldest decision (master tracker 2026-07-08): keep-oldest downs the partition *without* the oldest, so the younger survivor downs itself, and it cannot re-form a cluster alone (see the seed-node constraint below). The drill confirms the dark window, then recovery within ~2 min of restarting the victim. The mode exists to make the registered gap *observable*, not to pretend it is covered.
-3. Polls Traefik (`http://localhost:9000/health/active`, override with `TRAEFIK_URL`) until the survivor is routable, printing the observed failover time. Fails after `TIMEOUT_S` (default 90s).
-4. Dumps the survivor's singleton/oldest/downing log evidence, then restarts the victim so it **rejoins as a fresh incarnation** (Task 20's recovery loop).
 
-Expected failover: **~25s** (2s heartbeat + 10s detection + 15s stable-after) **plus the Traefik ~5s health-check interval**. The drill exercises S1 (SBR downing on hard crash), S3 (single active node routed through Traefik), and the Task 20 restart/rejoin contract. Requires a running cluster (`bash docker/deploy.sh`) and `curl` + `docker` on the host.
+The drill exercises S1 (SBR downing on hard crash), S3 (single active node routed through Traefik), and the Task 20 restart/rejoin contract. Requires a running cluster (`bash docker/deploy.sh`) and `curl` + `docker` on the host.
 
-> **Observed failover time:** _run after next deploy and record here_ (the script was validated with `bash -n`; no live cluster was up when it was committed).
+**Seed-node bootstrap constraint.** Only the FIRST seed in `Cluster:SeedNodes` may self-join to form a *new* cluster. Both central nodes list `scadabridge-central-a` first (`docker/central-node-a/appsettings.Central.json`, `docker/central-node-b/appsettings.Central.json`), so a lone restarted `central-b` (with `central-a` still down) loops on `InitJoin` forever — it never reaches `Up`, and `/health/active` never returns 200. Operator recovery actions: **(1)** restart the dead first-seed node (`central-a`) — preferred; or **(2)** restart the survivor with a self-first seed override (env `ScadaBridge__Cluster__SeedNodes__0=akka.tcp://scadabridge@<self-host>:8081`, `ScadaBridge__Cluster__SeedNodes__1=<peer>`). The repo deliberately does NOT ship self-first ordering per node: with *both* nodes self-first, a simultaneous cold start can let each self-join independently → two one-node clusters that never merge (the cold-start split-brain the identical-seed-order convention exists to prevent). The real remedy is the pending keep-oldest topology/strategy decision (deferred, owner: user).
+
+> **Observed results** (recorded by plan R2-01 T3, cluster @ commit <sha>):
+>
+> | Direction (`DRILL_MODE`) | Outcome | Measured |
+> |--------------------------|---------|----------|
+> | `standby` (younger-node crash) | _pending T3_ | _pending T3_ |
+> | `active` (oldest-node crash) | _pending T3_ | _pending T3_ |
 
 ### Central Failover
 

docs/requirements/Component-ClusterInfrastructure.md

@@ -108,9 +108,11 @@ When a node downs itself (via `down-if-alone`, or any other SBR decision), the r
 1. Self-down ⇒ `CoordinatedShutdown` ⇒ `ActorSystem` termination.
 2. The Host watches `ActorSystem.WhenTerminated`; a termination that is **not** the host's own `StopAsync` triggers `IHostApplicationLifetime.StopApplication()`, so **the process exits**.
 3. The service supervisor restarts it — docker `restart: unless-stopped`, or Windows service recovery actions (`sc.exe failure … restart/…`).
-4. The restarted process rejoins as a **fresh incarnation** (the keep-oldest resolver handles the rejoin cleanly; there is no stale membership to reconcile).
+4. The restarted process rejoins as a **fresh incarnation** (the keep-oldest resolver handles the rejoin cleanly; there is no stale membership to reconcile) — **but only while a peer still holding cluster state is reachable**. A lone restarted node that is *not* the first seed cannot re-form a cluster on its own (see the seed-node bootstrap constraint below); it waits for its peer.
 
-The docker failover drill exercises this loop end-to-end (kill the active node, assert singleton migration + clean rejoin).
+**Seed-node bootstrap constraint.** Only the FIRST seed listed in `Cluster:SeedNodes` may self-join to form a *new* cluster. All nodes list the same first seed (e.g. `scadabridge-central-a`), so a lone restarted non-first-seed node (with the first seed still down) loops on `InitJoin` forever — never `Up`, never routable. This is why the two-node keep-oldest **oldest/active-node crash is a total-outage gap**: after the oldest dies the younger survivor self-downs, and it cannot re-bootstrap alone. Recovery is operator-driven — either restart the dead first-seed node (preferred) or restart the survivor with a self-first seed override (`ScadaBridge__Cluster__SeedNodes__0` = self, `__1` = peer). The repo does not ship self-first ordering per node: with both nodes self-first a simultaneous cold start risks two independent one-node clusters that never merge. Removing the gap itself is the **registered deferred keep-oldest topology/strategy decision** (master tracker 2026-07-08, owner: user).
+
+The docker failover drill (`docker/failover-drill.sh`) exercises this per direction: `standby` mode proves SBR downing + singleton continuity on the oldest; `active` mode measures the registered total-outage gap and the recovery-on-restart path.
 
 ## Single-Node Operation