docs(design): correct 6.2 — the keep-oldest gap is closed upstream and live here
I reported this gap as still open, citing a requirements doc line and the PR that made the failover drill honest about it. Both were stale: ScadaBridge closed it three hours earlier in cf3bd52f by switching to auto-down. My check missed it because I compared only one direction (main..origin/main) and never asked whether local main was AHEAD of origin, so two unpushed commits were invisible to me. The substance matters more than the correction. Their commit message records a live-proven finding: Akka.NET 1.5.62 KeepOldest.OldestDecision only lets down-if-alone rescue a side with >= 2 members, so in a 1-vs-1 split the survivor takes DownReachable and downs ITSELF. OtOpcUa runs precisely that configuration — akka.conf active-strategy keep-oldest with down-if-alone on, plus the matching typed KeepOldestOption — which means a two-node pair cannot fail over when the oldest node crashes. That is a total-outage defect inside the redundancy feature, independent of the mesh design, and it is now Phase 0a: most urgent, and needing a crash-the-oldest live gate because the failure only appears 1-vs-1. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -2,8 +2,8 @@
|
||||
|
||||
> **Status:** design for review. No implementation planned yet.
|
||||
> **Decisions settled 2026-07-21** — see §6: one central DB with driver nodes disconnected from it,
|
||||
> the two-node keep-oldest gap accepted as ScadaBridge accepts it, and their auth posture matched for
|
||||
> now.
|
||||
> the two-node keep-oldest gap CLOSED upstream and still live here (§6.2 — most urgent item), and
|
||||
> their auth posture matched for now.
|
||||
|
||||
## 1. What this changes
|
||||
|
||||
@@ -209,27 +209,33 @@ SHA-256 verification, newest-2 retention and pair replication all carry over unc
|
||||
under this model it stops existing for driver nodes. That is a behaviour change on a client-visible
|
||||
value, not just an internal refactor.
|
||||
|
||||
### 6.2 DECIDED — accept the two-node keep-oldest gap, as ScadaBridge does
|
||||
### 6.2 CORRECTED — the gap was closed in ScadaBridge, and OtOpcUa still has it
|
||||
|
||||
**Status corrected.** What was resolved is the *documentation and the drill*, not the gap. Gitea
|
||||
**PR #12** (closed) — <https://gitea.dohertylan.com/dohertj2/ScadaBridge/pulls/12> — "R2-01: Cluster,
|
||||
Host & Failover round-2 fixes" carried:
|
||||
**An earlier revision of this section was wrong.** It reported the two-node keep-oldest outage gap as
|
||||
still open, citing `Component-ClusterInfrastructure.md:113` and Gitea PR
|
||||
[#12](https://gitea.dohertylan.com/dohertj2/ScadaBridge/pulls/12) (which made the failover drill
|
||||
*honest* about the gap rather than closing it). That doc line was stale: ScadaBridge closed the gap
|
||||
in `cf3bd52f`, *"feat(cluster): auto-down downing strategy — either-node crash now fails over (owner
|
||||
decision 2026-07-21: availability over partition-safety)"*.
|
||||
|
||||
- **T1** (`badc97af`) — the failover drill now kills the STANDBY by default, and its `active` mode
|
||||
*"measures the registered keep-oldest outage instead of pretending recovery"*.
|
||||
- **T2** (`d5364506`) — per-direction failover docs, dropping what the commit message calls *"the
|
||||
undeliverable ~25s active-crash promise"*.
|
||||
Its commit message carries the analysis, live-proven on their rig:
|
||||
|
||||
So the closed work made the gap **honest**, not absent. `Component-ClusterInfrastructure.md:113`
|
||||
still reads: *"Removing the gap itself is the **registered deferred keep-oldest topology/strategy
|
||||
decision** (master tracker 2026-07-08, owner: user)."* There is no open Gitea issue for it — it is
|
||||
tracked on that master tracker, not in the issue list.
|
||||
> *"Two-node keep-oldest could NEVER survive a crash of the oldest/active node: Akka.NET 1.5.62
|
||||
> `KeepOldest.OldestDecision` only lets `down-if-alone` rescue a side with >= 2 members, so the 1-vs-1
|
||||
> survivor takes `DownReachable` and downs ITSELF — proven live on the rig ('SBR took decision …
|
||||
> including myself') before this change. `static-quorum(1)` is worse (`IsTooManyMembers` -> `DownAll`);
|
||||
> `keep-majority` just re-keys the fatal crash to the lowest address."*
|
||||
|
||||
OtOpcUa inherits the same gap and, for now, the same posture. Worth recording so it is a known
|
||||
accepted risk rather than a surprise: after the oldest node of a two-node mesh dies, the survivor
|
||||
self-downs and cannot re-bootstrap alone, because only the first-listed seed may self-join. Recovery
|
||||
is operator-driven — restart the dead first seed, or restart the survivor with a self-first seed
|
||||
override.
|
||||
**OtOpcUa is running that exact configuration today** —
|
||||
`Core/ZB.MOM.WW.OtOpcUa.Cluster/Resources/akka.conf:49-53`: `active-strategy = "keep-oldest"`,
|
||||
`keep-oldest { down-if-alone = on }`, `stable-after = 15s`, with
|
||||
`BuildClusterOptions` setting the matching typed `KeepOldestOption { DownIfAlone = true }`.
|
||||
|
||||
So a two-node OtOpcUa pair **cannot fail over when the oldest node crashes**: the survivor downs
|
||||
itself. That is a total-outage defect in the redundancy feature itself, it is independent of this
|
||||
design, and the sister project has already done the analysis and shipped the fix. **It should be
|
||||
Phase 0 alongside the oldest-Up role derivation**, and it needs its own live gate — a
|
||||
crash-the-oldest drill, since the failure only appears in the 1-vs-1 case.
|
||||
|
||||
### 6.3 DECIDED — match ScadaBridge's auth posture for now
|
||||
|
||||
@@ -244,7 +250,8 @@ Deliberately not a task plan — per-phase plans follow, one at a time.
|
||||
|
||||
| Phase | Content | Independent of the split? |
|
||||
|---|---|---|
|
||||
| 0 | Oldest-Up role derivation (§4) | **Yes** — ship first |
|
||||
| 0a | Downing strategy: keep-oldest cannot survive an oldest-node crash in a 2-node cluster (§6.2) | **Yes** — most urgent |
|
||||
| 0b | Oldest-Up role derivation (§4) | **Yes** |
|
||||
| 1 | `ClusterNode` gains Akka + gRPC address columns; coordinator sources its expected-ack set from the DB | Yes |
|
||||
| 2 | Comm actors + receptionist registration; ClusterClient transport; deploy notify + acks across the boundary | No |
|
||||
| 3 | Config fetch-and-cache: artifact served by central, driver nodes read LocalDb (§6.1) | No |
|
||||
|
||||
Reference in New Issue
Block a user