docs(mesh): Phase 6 — mark done, remove single-mesh caveats, document pair-local meshes
Redundancy.md: KNOWN-LIMITATION (per-Akka-cluster election) marked RESOLVED;
new 'Per-cluster meshes (Phase 6)' section + pair-local secrets note; manual-
failover mesh-scope caveat removed; auto-down 1-vs-1 note updated (every mesh is
now two nodes, drill deferred to Phase 7). IManualFailoverService + ClusterRedundancy
razor caveats updated to pair-local reality. Configuration.md documents the
cluster-{ClusterId} role, per-pair self-first seeds, SplitTopologyTransportValidator,
and the intentional Dps default. Program + design status tables + CLAUDE.md marked
Phase 6 DONE.
(Task 8 subagent completed the edits but died on an API error before committing;
reviewed and committed by the controller.)
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
+29
-2
@@ -103,13 +103,40 @@ The checked-in `appsettings*.json` files are deliberately thin: they carry only
|
||||
| `Hostname` | string | `0.0.0.0` | Bind hostname. |
|
||||
| `Port` | int | `4053` | Cluster transport port. **Duplicated as `ClusterNode.AkkaPort` in the Config DB** — see the note below. |
|
||||
| `PublicHostname` | string | `127.0.0.1` | Hostname advertised in cluster gossip; must be reachable by peers. **Duplicated as `ClusterNode.Host`** — see the note below. |
|
||||
| `SeedNodes` | string[] | `[]` | Seed nodes for bootstrapping. **Ordered:** a node that appears in its own list must be entry 0 (only `seed-nodes[0]` can form a new cluster), or the host refuses to start — see [Redundancy.md § Bootstrap: self-first seed ordering](Redundancy.md#bootstrap-self-first-seed-ordering). |
|
||||
| `Roles` | string[] | `[]` | Cluster roles for this node. When empty, falls back to `OTOPCUA_ROLES`. Allowed values: `admin`, `driver`, `dev`. |
|
||||
| `SeedNodes` | string[] | `[]` | Seed nodes for bootstrapping. **Ordered:** a node that appears in its own list must be entry 0 (only `seed-nodes[0]` can form a new cluster), or the host refuses to start — see [Redundancy.md § Bootstrap: self-first seed ordering](Redundancy.md#bootstrap-self-first-seed-ordering). **Per-pair since Phase 6:** each application `Cluster`'s pair is its own Akka mesh, so a node's `SeedNodes` lists only itself (first) and its own pair partner (second) — never a node from another cluster's pair. |
|
||||
| `Roles` | string[] | `[]` | Cluster roles for this node. When empty, falls back to `OTOPCUA_ROLES`. Allowed values: `admin`, `driver`, `dev`, plus a per-cluster `cluster-{ClusterId}` role (e.g. `cluster-MAIN`, `cluster-SITE-A`) — see [Per-cluster mesh split (Phase 6)](#per-cluster-mesh-split-phase-6-cluster-clusterid-roles--splittopologytransportvalidator) below. |
|
||||
|
||||
> The full redundancy model (ServiceLevel tiers, split-brain, peer discovery) is in [`Redundancy.md`](Redundancy.md). The OPC UA peer-URI advertising lives in the `OpcUa:PeerApplicationUris` key above.
|
||||
|
||||
> **`Port` / `PublicHostname` are stored twice.** The node binds from these keys; the fleet's `ClusterNode` row records the same address as `AkkaPort` / `Host` so that **central can dial the node without sharing a gossip ring with it** — which is what [per-cluster mesh Phase 2](plans/2026-07-21-per-cluster-mesh-design.md) needs. Nothing in the schema makes the two agree, so `ClusterNodeAddressReconcilerActor` (admin-role singleton) compares them against live membership and logs an Error on mismatch. If you change `Cluster:Port` or `Cluster:PublicHostname` on a node, **update its `ClusterNode` row too** — a node binding 4054 with a row saying 4053 still gossips fine today, and fails silently in Phase 2. The row's `NodeId` is `host:port` and is also the deploy path's ack identity, so a drifted node's deployment acks stop matching as well. See [`config-db-schema.md` § `ClusterNode`](v2/config-db-schema.md#clusternode).
|
||||
|
||||
### Per-cluster mesh split (Phase 6): `cluster-{ClusterId}` roles + `SplitTopologyTransportValidator`
|
||||
|
||||
- **Purpose:** [per-cluster mesh](plans/2026-07-22-per-cluster-mesh-program.md) **Phase 6** split the
|
||||
single fleet-wide Akka mesh into one independent 2-node mesh per application `Cluster`. A node's
|
||||
`Cluster:Roles` now carries a `cluster-{ClusterId}` role alongside `driver` (and `admin` for a fused
|
||||
central node) — e.g. `driver,cluster-SITE-A` — and its `Cluster:SeedNodes` lists only itself and its
|
||||
own pair partner (see the `Roles`/`SeedNodes` rows above). Nodes in different clusters never gossip
|
||||
with each other.
|
||||
- **`SplitTopologyTransportValidator`** (`src/Core/ZB.MOM.WW.OtOpcUa.Cluster/`, wired at
|
||||
`ValidateOnStart` alongside the other `Cluster`-family validators) fails host start on any node
|
||||
carrying a `cluster-{ClusterId}` role unless it is also configured for the split-safe transports:
|
||||
`MeshTransport:Mode = ClusterClient`, `Telemetry:Mode = Grpc` on a driver node, `TelemetryDial:Mode =
|
||||
Grpc` on an admin node. A DPS-mode transport on a cluster-role node would try to gossip across a
|
||||
partition boundary that no longer exists, so the validator fails closed rather than let commands or
|
||||
telemetry silently vanish. A legacy node with no `cluster-{ClusterId}` role is exempt and may stay on
|
||||
`Dps`.
|
||||
- **The compiled transport-mode defaults were deliberately NOT flipped.** `MeshTransport:Mode`,
|
||||
`Telemetry:Mode`, and `TelemetryDial:Mode` (documented below) all still default to `Dps` — a
|
||||
blast-radius assessment found that flipping the compiled default would break every integration test
|
||||
and every existing `appsettings.json` profile that leaves these keys unset. `SplitTopologyTransportValidator`
|
||||
plus explicit per-deployment configuration is the guardrail, not a default flip; every node that
|
||||
actually carries a `cluster-{ClusterId}` role must set the split-safe modes explicitly.
|
||||
- See [`Redundancy.md` § Per-cluster meshes (Phase 6)](Redundancy.md#per-cluster-meshes-phase-6) for the
|
||||
full runtime picture (one Primary per pair, the cluster-scoped `RedundancyStateActor` singleton,
|
||||
`ClusterNodeAddressReconciler` own-cluster-scoping, and central's one-`ClusterClient`-per-`Cluster`
|
||||
fan-out).
|
||||
|
||||
### `MeshTransport` (central↔node command transport)
|
||||
|
||||
- **Purpose:** selects how central's three command channels (`deployments`, `driver-control`, `alarm-commands`) and the node's `deployment-acks` reply reach the other side — over the mesh-wide DistributedPubSub, or over an Akka `ClusterClient` that does **not** require central and the node to share a gossip ring. The second option is what [per-cluster mesh](plans/2026-07-21-per-cluster-mesh-design.md) Phase 6 needs once the meshes split.
|
||||
|
||||
Reference in New Issue
Block a user