feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md). Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath. Deleted: - AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests) - ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it) - ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in AkkaHostedService; the RegisterCentralClient message + receive block - CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport coexistence flags; the CentralTransportMode/SiteTransportKind enums gRPC is now the only site↔central transport (site→central CentralControlService via GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default so TestKit command-dispatch suites still construct the site actor without a wired transport; production always injects GrpcCentralTransport. Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator rejects blank entries (role-agnostic), and StartupValidator requires a Site node to list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default, deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used). Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted the audit-push integration relay to an in-process bridge transport. Docs: Component-Communication/Host/StoreAndForward, components/Communication, topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired amendment), and CLAUDE.md transport decisions. Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
This commit is contained in:
Joseph Doherty
@@ -327,9 +327,9 @@ Critical path ≈ 1B: **~4–6 weeks total**, matching the design estimate.
|
||||
- [x] P3 Central `SiteTransport=Grpc` all sites; full UI command matrix per site; site-kill mid-command clean; no PSK noise; zero ClusterClient log activity on flipped paths — **PASS 2026-07-23** (live gate: `ExecuteQuery`/`ExecuteParked` all 3 sites + `ExecuteLifecycle` disable/enable on site-a, all 200 over `SiteCommandService`; active-site-node hard-kill mid-command → clean `TIMEOUT` at the 30s deadline, next call failed over to site-a-b automatically; 0 PermissionDenied / 0 ClusterClient-to-site on both central. `ExecuteOpcUa`/`ExecuteRoute`/`TriggerFailover` deferred — no OPC-bound instance / no CLI verb, unit-proven)
|
||||
|
||||
**Phase 4 — deletion**
|
||||
- [ ] Defaults flip to `Grpc` + soak; then delete Akka transports, ClusterClient creation, `DefaultSiteClientFactory`, receptionist registrations, `CentralContactPoints`, then the flags
|
||||
- [ ] Grep-gates pass (`clusterclient|receptionist` → historical docs only; `CentralContactPoints` → empty)
|
||||
- [ ] Docs updated: `grpc_streams.md`, `Component-Host.md`, `Component-StoreAndForward.md:137`, known-issues cross-ref
|
||||
- [x] gRPC made the only transport (flags DELETED, not flipped — end state is identical) + deleted `AkkaCentralTransport`/`AkkaSiteTransport`, ClusterClient creation (`AkkaHostedService`), `DefaultSiteClientFactory`+`ISiteClientFactory`, both receptionist registrations (`:436`/`:1001`), `CentralContactPoints` + the `CentralTransport`/`SiteTransport` flags + `CentralTransportMode`/`SiteTransportKind` enums. `NoOpCentralTransport` added as the fail-loud null-default; `CentralGrpcEndpoints` now unconditional (StartupValidator requires ≥1 on Site). Kept `Akka.Cluster.Tools` (ClusterSingleton). Rig configs (docker ×6, docker-env2 ×2, Host default, wonder-app-vd03) moved `CentralContactPoints`→`CentralGrpcEndpoints`. **Full solution build 0/0; Communication.Tests 640, Host.Tests + StartupValidator + SiteActorPath green, audit-push integration green** (2026-07-23)
|
||||
- [x] Grep-gates pass (`CentralContactPoints` → only "replaces the former" doc refs + plan trackers; deleted symbols → 0 live refs; remaining `clusterclient` in src = the misleadingly-named `ClusterClientSiteAuditClient` [transport-agnostic, works unchanged] + stale inline doc-comments, noted as follow-up)
|
||||
- [x] Docs updated: `Component-Communication.md`, `components/Communication.md`, `Component-Host.md`, `Component-StoreAndForward.md`, `topology-guide.md`, `grpc_streams.md`, known-issues frame-size amendment, **CLAUDE.md** transport decisions
|
||||
|
||||
**Phase 5 — live gate** (record in `2026-07-22-clusterclient-to-grpc-live-gate.md`)
|
||||
- [ ] 1 PSK negatives · [ ] 2 site→central matrix · [ ] 3 central→site matrix · [ ] 4 failover/failback both directions · [ ] 5 mid-drain kill · [ ] 6 >128 KB frame-class proof · [ ] 7 no cross-boundary Akka association · [ ] 8 full-rig restart clean
|
||||
|
||||
@@ -218,22 +218,24 @@
|
||||
"id": "P4.1",
|
||||
"phase": "4",
|
||||
"subject": "Flip both flag defaults to Grpc + soak; delete Akka transports, ClusterClient creation, DefaultSiteClientFactory, receptionist registrations, CentralContactPoints, then the flags",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"activeForm": "Deleting the ClusterClient transport",
|
||||
"blockedBy": [
|
||||
"P2",
|
||||
"P3"
|
||||
]
|
||||
],
|
||||
"notes": "Branch feat/grpc-phase4-deletion (2026-07-23). Flags DELETED rather than flipped — end state is gRPC-only, identical. Deleted: AkkaCentralTransport, AkkaSiteTransport, ISiteClientFactory+DefaultSiteClientFactory, CentralCommunicationActor legacy ctor + SelectTransport, ClusterClient creation + both ClusterClientReceptionist.RegisterService in AkkaHostedService, CommunicationOptions.CentralContactPoints + CentralTransport/SiteTransport flags + CentralTransportMode/SiteTransportKind enums, RegisterCentralClient message + receive block, AkkaCentralTransportTests + DefaultSiteClientFactoryTests. Added NoOpCentralTransport (fail-loud null-default so command-dispatch TestKit suites keep constructing without a wired transport; production always injects GrpcCentralTransport). CommunicationOptionsValidator: CentralGrpcEndpoints now unconditional (no-blank-entries, role-agnostic); StartupValidator: Site-only requires ≥1 CentralGrpcEndpoint (fail-fast, mirrors GrpcPsk). Host builds GrpcSiteTransport (central) + GrpcCentralTransport (site) unconditionally. Kept Akka.Cluster.Tools (ClusterSingleton). Test rework: deleted the ClusterClient.Send per-site-routing tests (covered by CentralCommunicationActorTransportTests + GrpcSiteTransport suites), swapped ISiteClientFactory→substitute ISiteCommandTransport across 5 files, converted SiteAuditPushFlow's ClusterClientRelay→BridgeCentralTransport, converted Heartbeat_StampsIsActive to a substitute ICentralTransport, repurposed HealthReportAck no-client test to the NoOp fail-loud path, removed SiteActors_CentralClusterClient_Exists. Rig: docker ×6 + docker-env2 ×2 + Host appsettings.Site.json + deploy/wonder-app-vd03 moved CentralContactPoints→CentralGrpcEndpoints. Full solution build 0/0; Communication.Tests 640, StartupValidator 59, SiteActorPath 5, audit-push integration 1 all green. DID NOT fold in the dead IntegrationCallRequest deletion (#32) — SiteEnvelope routing is transport-agnostic so it still compiles; user-owned behavioral decision, left OUT of this PR."
|
||||
},
|
||||
{
|
||||
"id": "P4.2",
|
||||
"phase": "4",
|
||||
"subject": "Grep-gates pass + docs updated (grpc_streams.md, Component-Host.md, Component-StoreAndForward.md, known-issues cross-ref)",
|
||||
"status": "pending",
|
||||
"status": "completed",
|
||||
"activeForm": "Running the deletion grep-gates and doc sweep",
|
||||
"blockedBy": [
|
||||
"P4.1"
|
||||
]
|
||||
],
|
||||
"notes": "Grep-gates: CentralContactPoints → only intentional 'replaces the former' doc refs + plan trackers + stale bin/ artifacts (regenerate on build); deleted symbols (DefaultSiteClientFactory/ISiteClientFactory/RegisterCentralClient/AkkaCentralTransport/AkkaSiteTransport/CentralTransportMode/SiteTransportKind/the flags) → 0 live refs. Residual 'clusterclient' in src = ClusterClientSiteAuditClient (misleadingly named but transport-agnostic — it Asks SiteCommunicationActor, holds no ClusterClient; works unchanged) + ~25 stale inline XML-doc comments across SiteRuntime/Commons/SiteCallAudit — NOT scrubbed (out of plan scope, no behavior impact); FOLLOW-UP noted. Docs updated (subagent + me): Component-Communication.md, components/Communication.md, Component-Host.md, Component-StoreAndForward.md, deployment/topology-guide.md, plans/grpc_streams.md (SUPERSEDED note + LoadSiteAddressesFromDb correction), known-issues/2026-06-26-deploy-config-exceeds-akka-frame-size.md (2026-07-23 frame-size-retired amendment), and CLAUDE.md (2 transport-decision passages). Phase 5 (live gate) is next and requires a rig redeploy."
|
||||
},
|
||||
{
|
||||
"id": "P5",
|
||||
|
||||
@@ -1,10 +1,21 @@
|
||||
# gRPC Streaming Channel: Site → Central Real-Time Data
|
||||
|
||||
> **Status update (2026-07-23) — the "ClusterClient keeps command/control" framing is SUPERSEDED.**
|
||||
> This plan introduced gRPC for *streaming only*, leaving command/control on Akka ClusterClient. That
|
||||
> split no longer holds: **Phase 4 of the ClusterClient→gRPC migration moved command/control to gRPC
|
||||
> too and deleted the ClusterClient site↔central transport entirely.** Command/control now rides gRPC
|
||||
> in both directions — site→central to the central-hosted `CentralControlService` (`GrpcCentralTransport`)
|
||||
> and central→site to the site-hosted `SiteCommandService` (`GrpcSiteTransport`) — alongside the
|
||||
> streaming `SiteStreamService` this plan describes. Wherever the text below says "ClusterClient handles
|
||||
> command/control", read it as "gRPC command/control (`CentralControlService` / `SiteCommandService`)".
|
||||
> See `docs/requirements/Component-Communication.md` and
|
||||
> `docs/plans/2026-07-22-clusterclient-to-grpc-plan.md`.
|
||||
|
||||
## Context
|
||||
|
||||
Debug streaming events currently flow through Akka.NET ClusterClient (`InstanceActor → SiteCommunicationActor → ClusterClient.Send → CentralCommunicationActor → bridge actor`). ClusterClient wasn't built for high-throughput value streaming — it's a cluster coordination tool with gossip-based routing. As we scale beyond debug view to health streaming, alarm feeds, or future live dashboards, pushing all real-time data through ClusterClient will become a bottleneck.
|
||||
|
||||
**Goal**: Add a dedicated gRPC server-streaming channel on each site node. Central subscribes to sites over gRPC for real-time data. ClusterClient continues to handle command/control (subscribe, unsubscribe, deploy, lifecycle) but all streaming values flow through the gRPC channel.
|
||||
**Goal**: Add a dedicated gRPC server-streaming channel on each site node. Central subscribes to sites over gRPC for real-time data. Command/control (subscribe, unsubscribe, deploy, lifecycle) and streaming values flow over gRPC. *(As originally written, command/control stayed on ClusterClient; Phase 4 later moved it to gRPC — see the status note above.)*
|
||||
|
||||
**Scope**: General-purpose site→central streaming transport. Debug view is the first consumer, but the proto and server are designed so future features (health streaming, alarm feeds, live dashboards) can subscribe with different event types and filters.
|
||||
|
||||
@@ -57,7 +68,7 @@ flowchart TD
|
||||
class PB dec
|
||||
```
|
||||
|
||||
**Key separation**: ClusterClient handles subscribe/unsubscribe/snapshot (request-response). gRPC handles the ongoing value stream (server-streaming).
|
||||
**Key separation**: command/control handles subscribe/unsubscribe/snapshot (request-response); the ongoing value stream is server-streaming. Both ride gRPC after Phase 4 (subscribe/unsubscribe/snapshot over `SiteCommandService`, the value stream over `SiteStreamService`); as originally drawn, the request-response leg was ClusterClient.
|
||||
|
||||
## Port & Address Configuration
|
||||
|
||||
@@ -164,7 +175,7 @@ Add corresponding columns to the sites list table. Wire `_formGrpcNodeAAddress`
|
||||
|
||||
### SiteStreamGrpcClientFactory
|
||||
|
||||
Reads `GrpcNodeAAddress` / `GrpcNodeBAddress` from the `Site` entity (loaded by `CentralCommunicationActor.LoadSiteAddressesFromDb()`) when creating per-site gRPC channels. Falls back to NodeB if NodeA connection fails (same pattern as ClusterClient dual-contact-point failover).
|
||||
Reads `GrpcNodeAAddress` / `GrpcNodeBAddress` from the `Site` entity when creating per-site gRPC channels. As of Phase 4, `CentralCommunicationActor.LoadSiteAddressesFromDb()` no longer builds Akka ClusterClient contacts at all — it builds a **per-site gRPC-endpoint cache** from those same `GrpcNodeAAddress` / `GrpcNodeBAddress` columns, feeding a `SitePairChannelProvider` that both this streaming factory and the command/control `GrpcSiteTransport` (`SiteCommandService`) dial. Falls back to NodeB if NodeA connection fails.
|
||||
|
||||
### Docker Compose Port Allocation
|
||||
|
||||
@@ -364,7 +375,7 @@ public async Task<StreamSubscription> SubscribeAsync(
|
||||
|
||||
### Client Factory
|
||||
|
||||
`SiteStreamGrpcClientFactory` caches per-site `GrpcChannel` instances (same pattern as `CentralCommunicationActor._siteClients` caching per-site ClusterClient instances).
|
||||
`SiteStreamGrpcClientFactory` caches per-site `GrpcChannel` instances. (As originally written this mirrored `CentralCommunicationActor._siteClients` caching per-site ClusterClient instances; after Phase 4 that dictionary caches per-site gRPC endpoints/channels, not ClusterClients.)
|
||||
|
||||
## Failover & Reconnection
|
||||
|
||||
|
||||
Reference in New Issue
Block a user