dohertj2/lmxopcua
1
0
Fork 0
Code Issues 8 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(adminui): manual failover control on the cluster redundancy page

Browse Source 
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-22 06:44:44 -04:00
parent 69b697bc3e
commit d8a85c3d89
4 changed files with 442 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Redundancy.md
+30 -2
View File
@@ -258,8 +258,36 @@ bootstrap and reading `akka.cluster.downing-provider-class` back off the running
the typed option or the akka.conf text is not sufficient, because several HOCON fragments compete and the
losing one still looks correct in the file.
There is no operator-driven role swap during a partition; failover is what the cluster and supervisor do
automatically.
Failover during a partition is what the cluster and supervisor do automatically. For a *planned* swap —
patching, draining a node — see [Manual failover](#manual-failover) below.
## Manual failover
The cluster redundancy page (`/clusters/{id}/redundancy`) carries a **Live redundancy** panel — the current
driver Primary and every Up `driver` member, read from live cluster state — and an admin-gated **Trigger
failover** button.
The mechanism is a graceful cluster **`Leave`** of the current Primary, never a `Down`:
`IManualFailoverService.FailOverDriverPrimaryAsync` selects the **oldest Up `driver` member** — the identical
query to `RedundancyStateActor.SelectDriverPrimary`, pinned together by
`ManualFailoverServiceTests.Parity_with_SelectDriverPrimary` so the node acted on is exactly the node the
election names. The leaving node hands its singletons over through the cluster-leave phases,
`CoordinatedShutdown` runs, `ActorSystemTerminationWatchdog` exits the process, the supervisor restarts it,
and it rejoins as the **youngest** member — so the survivor is now the oldest, becomes Primary, and
advertises `ServiceLevel` 250. OPC UA clients re-select.
| Rule | Behaviour |
|---|---|
| Authorization | `AdminUiPolicies.FleetAdmin` (**Administrator only**) — gated in markup *and* re-checked server-side before the call. Deliberately not `ConfigEditor`, which the neighbouring cluster pages use: this restarts a production node rather than editing config, and ConfigEditor also admits Designer. |
| Peer guard | Disabled, with the reason as its tooltip, when fewer than two Up driver members exist — with one node a "failover" is a shutdown. Re-evaluated inside the service against live state, so a stale page cannot bypass it. |
| Confirmation | A dialog naming the node, and what follows (restart, ServiceLevel 250 moves, clients re-select). |
| Audit | Written through the shared `ZB.MOM.WW.Audit.IAuditWriter` seam **before** the Leave — action `cluster.manual-failover`, actor, target. A *refused* failover changes nothing and writes nothing. |
> **Mesh-scope caveat.** Until the per-cluster mesh work lands
> (`docs/plans/2026-07-21-per-cluster-mesh-design.md`), the Primary is elected once per Akka mesh rather than
> per application `Cluster` row — so on a fleet running several clusters in one mesh this button acts on the
> whole mesh's Primary, which may belong to a different cluster than the page you are on. The panel says so.
> Phase 6 of the mesh program removes both the caveat and this notice.
> **Live gate outstanding.** The auto-down change is verified by unit tests against the effective
> configuration, and by the sister project's live drill on an equivalent Akka version and topology. It has