feat(cluster): enable the #33 bootstrap guard on the docker rig — live gate PASS

deploy.sh's simultaneous recreate split site pairs twice on 2026-08-01 with
the guard off (mutual InitJoinNack, each node forming its own 1-node
cluster; a per-pair coin flip compose depends_on does not prevent). Guard
enabled on all 8 rig nodes: two consecutive simultaneous-start trials (full
redeploy + full-topology compose restart) converged all four pairs
deterministically — founder self-first on every lower address, peer-first
join on every higher, zero splits. This closes the deferred issue-acceptance
live gate; the switch stays default-off everywhere else.

CLAUDE.md

@@ -177,7 +177,7 @@ spec for each is `docs/requirements/Component-<Name>.md`, and `README.md` carrie
 - Automatic dual-node recovery from persistent storage.
 - **Active/standby is decided by `ActiveNodeEvaluator.SelfIsOldestUp`, never by cluster leadership** — see the Architecture note above. `/health/active` is **central-only** (site nodes map no `/health/*` at all) and backs both Traefik's active-node routing and `IActiveNodeGate`, so the proxy and the Inbound API always agree on which node is active. Central reaches a site by dialling `GrpcNodeAAddress`/`GrpcNodeBAddress` explicitly and flipping on error; whichever node answers, the site-internal `ClusterSingletonProxy` lands the work on the active node for free, so central still never needs to track which *site* node is active.
 - **Seed-node ordering: every node lists ITSELF first (decision 2026-07-22) — the boot-alone gap is CLOSED.** Only `seed-nodes[0]` may self-join to form a new cluster (Akka runs `FirstSeedNodeProcess` for it, `JoinSeedNodeProcess` — which can never form one — for everyone else). All 14 shipped node appsettings now lead with the node's own address, so any node can cold-start alone and become operational unattended (~5s, `seed-node-timeout`); `StartupValidator` fails the boot if the ordering is broken (compares host AND port; Akka does no DNS canonicalisation). Two nodes cold-starting together while mutually reachable converge on ONE cluster via the `InitJoin` handshake — they split only under a genuine boot-time partition, the same class auto-down accepts. **An external self-form timer (`Cluster.Join(SelfAddress)` after a window) was implemented and REJECTED:** it sits outside the join handshake, so on a routine standby restart — where the peer is alive but the join is stalled behind removal of the node's own stale incarnation — it fires mid-join and permanently splits the pair (measured: still split after 90s). Regression tests: `SelfFirstSeedBootstrapTests`. The keep-oldest active-crash total outage was separately closed by the auto-down decision. See `docs/requirements/Component-ClusterInfrastructure.md` → Seed Node Ordering.
-- **Simultaneous-cold-start split-brain guard (opt-in, Gitea #33) — the residual self-first cost, closed.** Self-first-on-both means a *truly simultaneous* cold start (shared power/hypervisor event) races `FirstSeedNodeProcess` on BOTH nodes → two 1-node clusters that never merge (in-process loopback tests converge and hide it; real parallel VM starts don't — OtOpcUa reproduced it live). Ported from OtOpcUa (`lmxopcua` `d1dac87f`): a **dark switch `ScadaBridge:Cluster:BootstrapGuard:Enabled` (default OFF, guard-off byte-identical)**. On: `BuildHocon` emits an EMPTY seed list (Akka does not auto-join) and `ClusterBootstrapCoordinator` (`IHostedService`, registered in BOTH the Central and Site composition roots) issues ONE reachability-gated `JoinSeedNodes` — the pure `ClusterBootstrapGuard` core makes the lower canonical `host:port` the **founder** (self-first, forms immediately), the higher node TCP-probes the founder up to `PartnerProbeSeconds` (25s) and joins **peer-first** if reachable else **self-first** (cold-start-alone preserved). Decided BEFORE a single join, **never re-forms mid-handshake** (that was the rejected self-form-timer's flaw); case-insensitive founder tie-break; probe timings validated `>0` when enabled. Residual accepted trade: founder dies in the probe→join window → higher node hangs unjoined, coordinator WARNs, a restart recovers it. Tests: `ClusterBootstrapGuardTests` (pure) + `ClusterBootstrapCoordinatorTests` (real-ActorSystem, incl. both-cold-start-together-form-one-cluster). See `docs/requirements/Component-ClusterInfrastructure.md` → Simultaneous cold start.
+- **Simultaneous-cold-start split-brain guard (opt-in, Gitea #33) — the residual self-first cost, closed.** Self-first-on-both means a *truly simultaneous* cold start (shared power/hypervisor event) races `FirstSeedNodeProcess` on BOTH nodes → two 1-node clusters that never merge (in-process loopback tests converge and hide it; real parallel VM starts don't — OtOpcUa reproduced it live). Ported from OtOpcUa (`lmxopcua` `d1dac87f`): a **dark switch `ScadaBridge:Cluster:BootstrapGuard:Enabled` (default OFF, guard-off byte-identical)**. On: `BuildHocon` emits an EMPTY seed list (Akka does not auto-join) and `ClusterBootstrapCoordinator` (`IHostedService`, registered in BOTH the Central and Site composition roots) issues ONE reachability-gated `JoinSeedNodes` — the pure `ClusterBootstrapGuard` core makes the lower canonical `host:port` the **founder** (self-first, forms immediately), the higher node TCP-probes the founder up to `PartnerProbeSeconds` (25s) and joins **peer-first** if reachable else **self-first** (cold-start-alone preserved). Decided BEFORE a single join, **never re-forms mid-handshake** (that was the rejected self-form-timer's flaw); case-insensitive founder tie-break; probe timings validated `>0` when enabled. Residual accepted trade: founder dies in the probe→join window → higher node hangs unjoined, coordinator WARNs, a restart recovers it. Tests: `ClusterBootstrapGuardTests` (pure) + `ClusterBootstrapCoordinatorTests` (real-ActorSystem, incl. both-cold-start-together-form-one-cluster). **ENABLED on the docker rig (all 8 nodes) 2026-08-02** after `deploy.sh`'s simultaneous recreate split site pairs twice on 2026-08-01 with the guard off; live gate PASS — two consecutive simultaneous-start trials (a full redeploy + a full-topology `docker compose restart`), all four pairs converged (every lower-address node founded self-first, every higher node probed-then-joined peer-first), zero splits. Default remains OFF elsewhere (wonder-app-vd03, docker-env2, Host defaults). See `docs/requirements/Component-ClusterInfrastructure.md` → Simultaneous cold start.
 
 ### UI & Monitoring
 - Central UI: Blazor Server (ASP.NET Core + SignalR) with Bootstrap CSS. No third-party component frameworks (no Blazorise, MudBlazor, Radzen, etc.). Build custom Blazor components for tables, grids, forms, etc.

docker/central-node-a/appsettings.Central.json

@@ -15,7 +15,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       "_comment": "ConfigurationDb/MachineDataDb (which carry the dev SQL password) are supplied as ScadaBridge__Database__* whole-key env overrides in docker/docker-compose.yml (dev-only-insecure, mirroring the ApiKeyPepper convention). The same dev password already ships committed for the sibling scadabridge-mssql container (infra/docker-compose.yml MSSQL_SA_PASSWORD) — this is consolidation, not new exposure. Env overrides layer over JSON before the ${secret:} expander runs, so the dev cluster boots with no KEK/secret store. Real/prod config uses ${secret:} tokens in src/.../appsettings.Central.json (T4)."

docker/central-node-b/appsettings.Central.json

@@ -15,7 +15,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       "_comment": "ConfigurationDb/MachineDataDb (which carry the dev SQL password) are supplied as ScadaBridge__Database__* whole-key env overrides in docker/docker-compose.yml (dev-only-insecure, mirroring the ApiKeyPepper convention). The same dev password already ships committed for the sibling scadabridge-mssql container (infra/docker-compose.yml MSSQL_SA_PASSWORD) — this is consolidation, not new exposure. Env overrides layer over JSON before the ${secret:} expander runs, so the dev cluster boots with no KEK/secret store. Real/prod config uses ${secret:} tokens in src/.../appsettings.Central.json (T4)."

docker/site-a-node-a/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docker/site-a-node-b/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docker/site-b-node-a/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docker/site-b-node-b/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docker/site-c-node-a/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docker/site-c-node-b/appsettings.Site.json

@@ -18,7 +18,14 @@
       "StableAfter": "00:00:15",
       "HeartbeatInterval": "00:00:02",
       "FailureDetectionThreshold": "00:00:10",
-      "MinNrOfMembers": 1
+      "MinNrOfMembers": 1,
+      "_bootstrapGuard": "Gitea #33 guard ENABLED on the docker rig (2026-08-02): deploy.sh recreates all containers simultaneously, which twice split site pairs into two 1-node clusters on 2026-08-01. Lower host:port founds self-first; the higher node TCP-probes then joins peer-first.",
+      "BootstrapGuard": {
+        "Enabled": true,
+        "PartnerProbeSeconds": 25,
+        "PartnerProbeIntervalMs": 500,
+        "ProbeConnectTimeoutMs": 1000
+      }
     },
     "Database": {
       // Migration-only as of LocalDb Phase 2. The site config tables now live in the

docs/requirements/Component-ClusterInfrastructure.md

@@ -156,7 +156,9 @@ The **bootstrap guard** eliminates the split without giving up cold-start-alone.
 | `BootstrapGuard:PartnerProbeIntervalMs` | `500` | Interval between probes. Validated `> 0` when enabled. |
 | `BootstrapGuard:ProbeConnectTimeoutMs` | `1000` | Per-probe TCP connect timeout. Validated `> 0` when enabled. |
 
-Decision core (`ClusterBootstrapGuard`) is a pure, fully unit-tested function; the probe + `JoinSeedNodes` runtime (`ClusterBootstrapCoordinator`) is covered by real-ActorSystem tests including the load-bearing higher-node-cold-start-alone case and the headline both-cold-start-together-form-one-cluster case (`ClusterBootstrapCoordinatorTests`). The alternative is purely operational (stagger the two VMs' service-manager start / start the founder first); the docker rig's compose `depends_on` does that today, but that does not exist on the real co-located VMs — the guard is the production-faithful fix. Ported from OtOpcUa (`lmxopcua` commit `d1dac87f`); implementing it in both products keeps their failure/recovery model identical, since a shared power event hits both pairs at once.
+Decision core (`ClusterBootstrapGuard`) is a pure, fully unit-tested function; the probe + `JoinSeedNodes` runtime (`ClusterBootstrapCoordinator`) is covered by real-ActorSystem tests including the load-bearing higher-node-cold-start-alone case and the headline both-cold-start-together-form-one-cluster case (`ClusterBootstrapCoordinatorTests`). The alternative is purely operational (stagger the two VMs' service-manager start / start the founder first); compose `depends_on` was assumed to do that on the docker rig, but it demonstrably does not — see below. Ported from OtOpcUa (`lmxopcua` commit `d1dac87f`); implementing it in both products keeps their failure/recovery model identical, since a shared power event hits both pairs at once.
+
+**Live status (2026-08-02): ENABLED on the primary docker rig (all 8 nodes), live gate PASS.** With the guard off, `docker/deploy.sh`'s simultaneous container recreate reproduced the split naturally TWICE on 2026-08-01 — first run split both site-b and site-c (mutual `InitJoinNack`, each node "JOINING itself and forming a new cluster"), second run split site-c while the other pairs converged: the race is a per-pair coin flip, and compose `depends_on` does not serialize far enough to prevent it. With the guard enabled on every rig node, two consecutive simultaneous-start trials (a full redeploy plus a full-topology `docker compose restart`) converged all four pairs deterministically — every lower-address node logged founder/self-first, every higher node logged partner-REACHABLE/peer-first and received `Welcome from` the founder; zero splits, `/health/ready` green. This is the issue-acceptance live gate that was deferred at implementation time. The switch stays default-off in the Host appsettings and everywhere else (wonder-app-vd03, docker-env2).
 
 ### Manual Failover (admin-triggered)