docs(mesh): sweep for ClusterClient deletion — Dps|Grpc only, CentralContactPoints retired

Task 3.2 of the ClusterClient->gRPC migration. Sweeps the docs that still
described the Akka ClusterClient transport as live, following its Phase 3
deletion (this branch): MeshTransport:Mode is now Dps (pair-local DPS) or
Grpc (cross-mesh unary calls) only, ClusterClient fails validation loudly,
CentralContactPoints is deleted (not superseded), and the split-topology
validator requires exactly Grpc rather than ClusterClient.

- docs/Configuration.md: rewrite the MeshTransport section — table,
  dark-switch/PSK-seeding prose, listener sharing (:4056 driver-role,
  :4055 admin-role), remove the "shipped dark, cutover pending" framing.
- docs/Redundancy.md: replace the Dps/ClusterClient transport table with
  Dps/Grpc, fix the Phase 6 CentralCommunicationActor + validator bullets.
- docs/Telemetry.md: fix stale ClusterClient cross-refs, add a note that
  the driver-role telemetry listener also hosts NodeCommandService.
- docs/v2/config-db-schema.md: one-line "(retired: ...)" annotation on the
  historical AkkaPort/GrpcPort note — left as a historical record otherwise.
- CLAUDE.md: rewrite "## Mesh command transport (MeshTransport)" for the
  post-deletion world (no ClusterClient/receptionist, per-node gRPC dial,
  fire-and-forget semantics unchanged) and fix two Phase 6 bullets.

deferment.md has no mesh-transport mentions; left untouched.

docs-only change; `dotnet build ZB.MOM.WW.OtOpcUa.slnx` stays green (0
errors) after the edit.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
This commit is contained in:
Joseph Doherty
2026-08-06 16:23:23 -04:00
parent cd7135570c
commit 406b1ae5ac
5 changed files with 126 additions and 85 deletions
+26 -16
View File
@@ -139,13 +139,16 @@ Consequences:
- **`ClusterNodeAddressReconciler` is own-cluster-scoped** — an admin node can only reconcile
`ClusterNode` rows belonging to its own cluster; it has no gossip visibility into another mesh to
reconcile against.
- **`CentralCommunicationActor` creates one `ClusterClient` per application `Cluster`** and fans
commands across them, rather than one fleet-wide client. Central reaches a site mesh over
`ClusterClient` (commands), gRPC (telemetry — see `docs/Telemetry.md`), and `ConfigServe` (config
fetch — see [`ConfigSource`/`ConfigServe`](../CLAUDE.md#config-source-configsource--configserve)),
never over gossip.
- **`CentralCommunicationActor` dials each node directly over gRPC** — one `NodeCommandService.Execute`
call per enabled, non-maintenance `ClusterNode` row carrying a `GrpcPort` — rather than fanning
through any client/receptionist object (the Akka `ClusterClient` transport this used to run on is
**deleted**; see [ClusterClient→gRPC migration](plans/2026-08-06-clusterclient-to-grpc-design.md)).
Central reaches a site mesh over gRPC for commands (`MeshTransport`), telemetry (see
`docs/Telemetry.md`), and config fetch (`ConfigServe` — see
[`ConfigSource`/`ConfigServe`](../CLAUDE.md#config-source-configsource--configserve)), never over
gossip.
- **`SplitTopologyTransportValidator` fails host start** on a node carrying a `cluster-{ClusterId}`
role unless it is also configured for the split-safe transports: `MeshTransport:Mode = ClusterClient`,
role unless it is also configured for the split-safe transports: `MeshTransport:Mode = Grpc`,
`Telemetry:Mode = Grpc` (driver nodes), `TelemetryDial:Mode = Grpc` (admin nodes). A DPS-mode transport
on a cluster-role node would silently try to gossip across a partition that no longer exists, so the
validator fails closed instead. Legacy/non-cluster-role nodes are exempt.
@@ -538,27 +541,34 @@ See [ScriptedAlarms.md](ScriptedAlarms.md) and [AlarmTracking.md](AlarmTracking.
Everything above rides one Akka mesh: central publishes on DistributedPubSub, every node
subscribes, and the two ends must be members of the same cluster. Per-cluster mesh **Phase 2**
(cross-mesh cutover: [ClusterClient→gRPC migration](plans/2026-08-06-clusterclient-to-grpc-design.md))
adds a second transport under `MeshTransport:Mode` (see
[Configuration.md § `MeshTransport`](Configuration.md#meshtransport-centralnode-command-transport))
that does not need shared membership — the prerequisite for splitting the fleet into one mesh per
application `Cluster`.
application `Cluster`. The Akka `ClusterClient` transport that originally filled this second slot is
**deleted**; `Grpc` is its replacement, not an alternative alongside it.
| | `Dps` (default) | `ClusterClient` |
| | `Dps` (default, pair-local) | `Grpc` (cross-mesh) |
|---|---|---|
| Central → node commands | `Publish` on `deployments` / `driver-control` / `alarm-commands` | `ClusterClient.SendToAll` to `/user/node-communication`, re-emitted on the node's **local** `EventStream` |
| Node → central acks | `Publish` on `deployment-acks` | `ClusterClient.Send` to `/user/central-communication`, forwarded to the deploy-coordinator singleton |
| Central → node commands | `Publish` on `deployments` / `driver-control` / `alarm-commands` | `NodeCommandService.Execute`, one unary call per node dialled directly from `ClusterNode.Host`+`GrpcPort`; decoded and handed to the node's `NodeCommunicationActor`, which re-emits it on the node's **local** `EventStream` exactly as the DPS path does |
| Node → central acks | `Publish` on `deployment-acks` | `CentralCommandService.ReportApply`, walking `MeshTransport:CentralGrpcEndpoints` in order and stopping at the first success, forwarded to the deploy-coordinator singleton |
| Requires shared cluster membership | **yes** | no |
| Auth | none (mesh membership is the boundary) | per-cluster PSK bearer, both directions, fail-closed |
Two properties are load-bearing and easy to break:
- **Both comm actors are registered per node, NOT as cluster singletons.** A `ClusterClient` rotates
across its contact points and must find a live comm actor at whichever node answers. As a
singleton the boundary would be reachable through exactly one node, and rotation would fail
silently against the others — acks landing or not depending on where the client happened to
settle. `akka.cluster.client.receptionist.role` is deliberately empty for the same reason.
- **The gRPC dialer fans node by node, not through any shared client or receptionist object.** There
is no rotation and no "whichever node answers" — each command is dialled at the specific node's own
endpoint, so a node left out of the `ClusterNode` dispatch set (no `GrpcPort`, disabled, or in
`MaintenanceMode`) silently keeps its old configuration while the deployment still seals green
around it, the same failure shape `ClusterClient`'s `SendToAll` had.
- **Inbound commands go to the node-local `EventStream`, never back onto a DPS topic.** DPS is
mesh-wide and central `SendToAll`s to *every* node, so a DPS re-publish would deliver N copies of
mesh-wide and central fans out to *every* node, so a DPS re-publish would deliver N copies of
every command to every subscriber.
- **Both transports are fire-and-forget, with no retry or queue.** A failed gRPC dispatch is logged
(naming the node and the gRPC status) and dropped in the dispatch second — the deploy path's
2-minute apply deadline and a node's boot-time replay (`DriverHostActor.Bootstrap()` re-applying an
orphan `Applying` row) are the only recovery mechanisms, unchanged from the DPS/`ClusterClient` era.
Phase 2 changes no redundancy semantics: the Primary gate, ServiceLevel, and the singleton layout
are untouched, and `redundancy-state` itself stays on DPS (it is bidirectional and built from