Files
ScadaBridge/docs/plans/2026-07-22-clusterclient-to-grpc-plan.md.tasks.json
T
Joseph Doherty 7fd5cb2b56 feat(comm): Phase 4 — delete Akka ClusterClient site↔central transport, gRPC-only
ClusterClient→gRPC migration Phase 4 (docs/plans/2026-07-22-clusterclient-to-grpc-plan.md).
Phases 2/3 proved both directions on gRPC; this removes the Akka transport underneath.

Deleted:
- AkkaCentralTransport, AkkaSiteTransport (+ their dedicated tests)
- ISiteClientFactory + DefaultSiteClientFactory; CentralCommunicationActor legacy
  ctor + SelectTransport (Host now builds GrpcSiteTransport and injects it)
- ClusterClient creation + both ClusterClientReceptionist.RegisterService calls in
  AkkaHostedService; the RegisterCentralClient message + receive block
- CommunicationOptions.CentralContactPoints; the CentralTransport/SiteTransport
  coexistence flags; the CentralTransportMode/SiteTransportKind enums

gRPC is now the only site↔central transport (site→central CentralControlService via
GrpcCentralTransport; central→site SiteCommandService via GrpcSiteTransport), both
built unconditionally by the Host. NoOpCentralTransport is the fail-loud null-default
so TestKit command-dispatch suites still construct the site actor without a wired
transport; production always injects GrpcCentralTransport.

Config: CentralGrpcEndpoints is now unconditional — CommunicationOptionsValidator
rejects blank entries (role-agnostic), and StartupValidator requires a Site node to
list >=1 endpoint (fail-fast, mirrors GrpcPsk). Rig configs moved
CentralContactPoints -> CentralGrpcEndpoints (docker x6, docker-env2 x2, Host default,
deploy/wonder-app-vd03). Kept Akka.Cluster.Tools (ClusterSingleton still used).

Tests: build 0/0; Communication.Tests 640, Host.Tests 421 green. Removed the
ClusterClient.Send per-site-routing tests (covered by the transport suites), swapped
the ISiteClientFactory-based ctors to a substitute ISiteCommandTransport, converted
the audit-push integration relay to an in-process bridge transport.

Docs: Component-Communication/Host/StoreAndForward, components/Communication,
topology-guide, grpc_streams (SUPERSEDED note), the frame-size known-issue (retired
amendment), and CLAUDE.md transport decisions.

Not included: the dead IntegrationCallRequest path (#32) is a separate user-owned
behavioral decision — SiteEnvelope routing is transport-agnostic so it still compiles.
2026-07-23 12:54:32 -04:00

251 lines
20 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"plan": "docs/plans/2026-07-22-clusterclient-to-grpc-plan.md",
"design": "~/Desktop/scadaproj/scadabridge_clusterclient_to_grpc.md",
"worktrees": {
"feat/grpc-phase0-psk": "/Users/dohertj2/Desktop/ScadaBridge-phase0"
},
"tasks": [
{
"id": "T0.1",
"phase": "0",
"subject": "Delete the vestigial /user/management receptionist registration",
"status": "completed",
"activeForm": "Deleting the /user/management receptionist registration",
"files": [
"src/ZB.MOM.WW.ScadaBridge.Host/Actors/AkkaHostedService.cs",
"docs/requirements/Component-Host.md",
"docs/requirements/Component-ManagementService.md",
"docs/requirements/Component-Communication.md",
"docs/components/ManagementService.md",
"docs/components/Communication.md",
"src/ZB.MOM.WW.ScadaBridge.CLI/README.md"
],
"notes": "Actor stays; only the ClusterClientReceptionist.RegisterService call goes. Verified: CLI is Akka-free, so 6 docs claiming the CLI reaches ManagementActor over ClusterClient are stale and must be corrected in the same change."
},
{
"id": "T0.2",
"phase": "0",
"subject": "Record the dead IntegrationCallRequest exclusion (28 of 29 commands migrate)",
"status": "completed",
"activeForm": "Recording the dead IntegrationCallRequest exclusion",
"files": [
"docs/known-issues/"
],
"notes": "Plan says 'file a Gitea issue' \u2014 outward-facing, handed to the user. In-repo half is a known-issues note + the exclusion record."
},
{
"id": "T0.3",
"phase": "0",
"subject": "ControlPlaneAuthInterceptor + CommunicationOptions.GrpcPsk + SitePskProvider; gate SiteStream; attach PSK on central's clients",
"status": "completed",
"activeForm": "Building the control-plane PSK auth",
"files": [
"src/ZB.MOM.WW.ScadaBridge.Host/ControlPlaneAuthInterceptor.cs",
"src/ZB.MOM.WW.ScadaBridge.Communication/CommunicationOptions.cs",
"src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/ISitePskProvider.cs",
"src/ZB.MOM.WW.ScadaBridge.Host/SitePskProvider.cs",
"src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcClient.cs",
"src/ZB.MOM.WW.ScadaBridge.Communication/Grpc/SiteStreamGrpcClientFactory.cs",
"src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullAuditEventsClient.cs",
"src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullSiteCallsClient.cs",
"src/ZB.MOM.WW.ScadaBridge.Host/Program.cs"
],
"notes": "Service prefix verified from sitestream.proto: /sitestream.SiteStreamService/. Grpc 2.76 -> CallCredentials.FromInterceptor + UnsafeUseInsecureChannelCallCredentials is the async-safe attach path on h2c."
},
{
"id": "T0.4",
"phase": "0",
"subject": "Rig dev keys (3 sites + central secret seeds) + interceptor/provider/wiring tests",
"status": "completed",
"activeForm": "Seeding rig dev keys and writing the auth tests",
"files": [
"docker/site-a-node-a/appsettings.Site.json",
"docker/site-a-node-b/appsettings.Site.json",
"docker/site-b-node-a/appsettings.Site.json",
"docker/site-b-node-b/appsettings.Site.json",
"docker/site-c-node-a/appsettings.Site.json",
"docker/site-c-node-b/appsettings.Site.json",
"tests/ZB.MOM.WW.ScadaBridge.Host.Tests/ControlPlaneAuthInterceptorTests.cs"
],
"notes": "Fail-closed from day one \u2014 every environment (incl. docker-env2 and the gitignored deploy/wonder-app-vd03 overlay) needs its key before upgrade. Ops item for the user."
},
{
"id": "P0.DoD",
"phase": "0",
"subject": "Phase 0 DoD: suite green; rig unauthenticated => PermissionDenied, authenticated paths work; PR merged",
"status": "completed",
"activeForm": "Verifying the Phase 0 DoD",
"blockedBy": [
"T0.1",
"T0.2",
"T0.3",
"T0.4"
],
"notes": "Live gate PASS 2026-07-22 (all 7 checks, docs/plans/2026-07-22-clusterclient-to-grpc-live-gate.md). Suite: 29 non-Playwright suites / 6872 tests / 0 failures. Playwright 170 pass / 2 fail / 1 skip of 173 - BOTH failures root-caused and PRE-EXISTING on main, unrelated to Phase 0 (branch touches no EF/CentralUI/Transport/ManagementService file): (1) TransportImportTests - REAL production bug, BundleImporter.cs:1298 user-initiated transaction + EnableRetryOnFailure => import broken on real MS SQL, hidden by the in-memory EF provider; (2) SmsNotificationE2ETests - stale fixture SID 'ACtest123' vs the ^AC[0-9a-fA-F]{32}$ guard added 2026-07-10 (40088a21); failing since then, which also silences its secret-non-leak assertion. The earlier 44-failure run is VOID (concurrent rig rebuild). REMAINING: PR + merge, HELD for the user per the plan's stop-at-DoD rule."
},
{
"id": "T1A.1",
"phase": "1A",
"subject": "central_control.proto (7 RPCs, checked-in codegen) + CentralControlDtoMapper + round-trip golden tests",
"status": "completed",
"activeForm": "Authoring central_control.proto and its mappers",
"blockedBy": [
"P0.DoD"
],
"notes": "d7455577 on feat/grpc-central-control. 7 RPCs; ingest RPCs reuse sitestream AuditEventBatch/CachedTelemetryBatch/IngestAck by import. 32 goldens, verified to have teeth by mutation. PLAN CORRECTIONS FOUND: (1) actor sends IngestAuditEventsCommand/-Reply (IReadOnlyList<Guid>), NOT the batch/IngestAck types the plan's table claims - mapper bridges; (2) CachedTelemetryEntry carries SiteCall not SiteCallOperational, needed a new SiteCallDtoMapper.ToDto(SiteCall); (3) SiteHealthReport is ~33 members and 5 are INIT-ONLY props not ctor params - FromDto needs an object initializer or they silently drop; (4) 3 collections are nullable with load-bearing null, proto3 cannot express presence on repeated/map => wrapper messages; (5) ConnectionHealth has no Unspecified member, so naive mapping puts Connected on proto3 zero - reserved 0 and unknown decodes to Error, never Connected."
},
{
"id": "T1A.2",
"phase": "1A",
"subject": "Central hosting: AddGrpc + per-site-PSK interceptor, CentralGrpcPort h2c listener, CentralControlGrpcService, readiness gate",
"status": "completed",
"activeForm": "Hosting CentralControlService on central",
"blockedBy": [
"T1A.1"
],
"notes": "780bb9c3 on feat/grpc-central-control. NEW class CentralControlAuthInterceptor (one public ctor, 3-arg internal, pinned by reflection test) - central verifies per-site PSK via ISitePskProvider keyed by required x-scadabridge-site header, fail-closed on missing/blank/unknown/mismatch. CentralControlGrpcService Asks existing CentralCommunicationActor (0 handler changes), readiness via SetReady mirror, heartbeat Tell/never-gated, ingest reuses AuditIngestAskTimeout. Central branch had NO AddGrpc/Kestrel before - added h2c listener on CentralGrpcPort default 8083, :5000 untouched. Rig ports 9013/9014:8083 published. PLAN GAPS: (1) mappers throw on unset WKT fields - test DTOs must carry a timestamp (no prod impact); (2) plan gave no deadline for Submit/QueryNotification - used NotificationForwardTimeout, check T1A.3 client sets same. 17 CentralControl tests, Host.Tests 384, Communication.Tests 356."
},
{
"id": "T1A.3",
"phase": "1A",
"subject": "ICentralTransport (Akka extract + Grpc impl), CentralChannelProvider, CentralTransport flag, CentralGrpcEndpoints option",
"status": "completed",
"activeForm": "Building the site->central transport seam",
"blockedBy": [
"T1A.1"
],
"notes": "33b15f10 on feat/grpc-central-control. ICentralTransport: 7 methods, actor delegates all 7; optional ctor param (null->actor self-builds AkkaCentralTransport in PreStart, byte-identical Akka path). CentralChannelProvider = sticky failover/failback, 1s-double-cap-60s backoff, per-site PSK not needed (site's own GrpcPsk + site header). Flag CentralTransport default Akka; on Grpc the ClusterClient is not created at all. Deadlines matched per-RPC to today's Ask timeouts (table in report). PLAN CORRECTION (important): connect-refused surfaces as StatusCode.Internal + 'Error starting gRPC call'/HttpRequestException, NOT Unavailable, in this Grpc.Net version - IsConnectFailure covers Unavailable OR that Internal class, still excludes DeadlineExceeded (verified in code + DeadlineExceeded_IsNotRetriedOnThePeer test). Above-seam suites (NotificationForwarder/HealthReportSender/SiteAuditTelemetry) pass UNMODIFIED. Communication.Tests 371, Host.Tests 391."
},
{
"id": "T1A.4",
"phase": "1A",
"subject": "Tests: actor-with-fake-transport x7, TestServer transport tests, S&F/audit/health suites pass unmodified",
"status": "completed",
"activeForm": "Testing the central control plane",
"blockedBy": [
"T1A.2",
"T1A.3"
],
"notes": "Tests shipped inside T1A.1/1A.2/1A.3 per the plan folding T1A.4 into each task. Actor-with-fake-transport x7, TestServer transport tests, above-seam suites pass unmodified."
},
{
"id": "P1A.DoD",
"phase": "1A",
"subject": "1A DoD: rig site-a on Grpc proves all 5 site->central paths while site-b/c stay Akka; PR merged before 1B",
"status": "pending",
"activeForm": "Verifying the 1A DoD",
"blockedBy": [
"T1A.4"
]
},
{
"id": "T1B.1",
"phase": "1B",
"subject": "site_command.proto (6 oneof RPCs / 28 commands) + SiteCommandDtoMapper + round-trip golden tests",
"status": "completed",
"activeForm": "Authoring site_command.proto and its mappers",
"blockedBy": [
"P0.DoD"
],
"notes": "a7481174 on feat/grpc-sitecommand. 6 oneof RPCs / 28 commands / 22 replies / 18 nested, verified by reflection guards not a hand list. PLAN CORRECTIONS: (1) generic Guid?->empty-string-means-null is UNSAFE for RouteToWaitForAttributeRequest.TargetValueEncoded, a nullable STRING where '' is a real wait target distinct from null - gave it a StringValue wrapper; (2) RouteToGetAttributesResponse.Values is NON-nullable dict but RouteToCallRequest.Parameters is nullable - two decode paths (absent->empty vs absent->null); (3) DeployArtifactsCommand has 6 NULLABLE artifact collections, proto3 repeated collapses null<->empty => wrapper messages (PLAN-05 T8 class); (4) AlarmStateChanged.Condition is derived-on-read over a nullable backing field - encoding unconditionally breaks record equality, mapper omits condition==computed default. Added seam types for T1B.2/3: SiteCommandGroup, GroupOf/GroupOfReply, UnsubscribeDebugViewAck marker (unsubscribe is Tell today, unary RPC must answer). REBASE NOTE: union-conflict expected in Communication.csproj commented Protobuf ItemGroup + docker/regen-proto.sh vs 1A."
},
{
"id": "T1B.2",
"phase": "1B",
"subject": "SiteCommandDispatcher refactor (actor + SiteCommandGrpcService share it; parked stays node-local)",
"status": "completed",
"activeForm": "Extracting the site command dispatcher",
"blockedBy": [
"T1B.1"
],
"notes": "cd6c20e1 on feat/grpc-sitecommand. SiteCommandDispatcher = pure routing (ResolveRoute -> Route{Disposition,Target,Reply}); actor delegates all 27 non-failover Receives to it, gRPC SiteCommandGrpcService calls the SAME dispatcher; shared via SetReady(dispatcher) hand-off like SiteStreamGrpcServer. Parked stays node-local (proven NotSame(proxy)). Interceptor: SiteCommandService added to DefaultGatedPrefixes via descriptor - NO new ctor, one-public-ctor test green. PLAN CORRECTION (important): existing actor code issues cluster Leave BEFORE the ack (fine over ClusterClient Tell, WRONG for gRPC ack-before-Leave). Solved via the pre-existing dryRun param on ClusterFailoverCoordinator.FailOverOldest: seam widened to Func<role,dryRun,addr?>, actor commits immediately (dryRun:false, byte-identical today), gRPC resolves dry-run then defers CommitLeave until after ack. Two ordering tests pin resolve-then-leave. Actor got an OPTIONAL dispatcher param so existing SiteCommunicationActorTests pass with ZERO edits. Server derives local-Ask timeout from ServerCallContext.Deadline (remaining), 2min fallback. Communication.Tests 574, Host.Tests 377."
},
{
"id": "T1B.3",
"phase": "1B",
"subject": "ISiteCommandTransport in CentralCommunicationActor (Akka extract + Grpc impl), SitePairChannelProvider, SiteTransport flag",
"status": "completed",
"activeForm": "Building the central->site transport seam",
"blockedBy": [
"T1B.1"
],
"notes": "3be85f19 on feat/grpc-sitecommand. ISiteCommandTransport injected into CentralCommunicationActor; HandleSiteEnvelope->_transport.Send(env,Sender), HandleSiteAddressCacheLoaded->_transport.ReconcileSites. AkkaSiteTransport extracted verbatim (incl no-client->drop path). GrpcSiteTransport + SitePairChannelProvider (A/B from GrpcNode*Address, per-site PSK via ISitePskProvider, Invalidate on removal). Rode the EXISTING LoadSiteAddressesFromDb loop (extended SiteAddressCacheLoaded to carry gRPC cols too) - ONE poll loop. Flag SiteTransport default Akka. PLAN CORRECTIONS: (1) LoadSiteAddressesFromDb DOES exist (known); (2) plan deadline table wrong TWICE - DeploymentStateQuery uses QueryTimeout not LifecycleTimeout, TriggerSiteFailover uses QueryTimeout not LifecycleTimeout - resolver is per-command-type not per-group, verified in code; (3) SiteCommandGroup XML doc's 'shared deadline class' claim is false; (4) THIRD SiteEnvelope producer DebugStreamBridgeActor (not just CommunicationService+SiteCallAudit) - reply plumbing routes to a real actor sender, not only Ask temp actors. RetryParkedOperation/DiscardParkedOperation->QueryTimeout(30s) keeps SiteCallAudit RelayTimeout(10s)<30s. Communication.Tests 607, gRPC Host.Tests 34. Existing suites: 1 trivial helper edit (SiteAddressCacheLoaded internal->public + new dict arg)."
},
{
"id": "T1B.4",
"phase": "1B",
"subject": "Tests: dispatcher routing x28, actor envelope/reply plumbing, TestServer service tests, existing suites green",
"status": "completed",
"activeForm": "Testing the site command plane",
"blockedBy": [
"T1B.2",
"T1B.3"
],
"notes": "Tests shipped inside T1B.1/1B.2/1B.3 per the plan folding T1B.4 into each task. Dispatcher routing x28, actor envelope/reply plumbing, TestServer service tests, deadline theory, failover/failback, existing Communication suites green with Akka default."
},
{
"id": "P1B.DoD",
"phase": "1B",
"subject": "1B DoD: rig central on Grpc for site-a proves full command matrix incl. standby parked retry; rebased on 1A; PR merged",
"status": "pending",
"activeForm": "Verifying the 1B DoD",
"blockedBy": [
"T1B.4",
"P1A.DoD"
]
},
{
"id": "P2",
"phase": "2",
"subject": "All sites CentralTransport=Grpc; central-kill S&F soak, failback observed, health sequences clean",
"status": "completed",
"activeForm": "Running the site->central cutover soak",
"blockedBy": [
"P1A.DoD"
]
},
{
"id": "P3",
"phase": "3",
"subject": "Central SiteTransport=Grpc all sites; full UI command matrix; site-kill mid-command clean; zero ClusterClient activity",
"status": "completed",
"activeForm": "Running the central->site cutover soak",
"blockedBy": [
"P1B.DoD"
]
},
{
"id": "P4.1",
"phase": "4",
"subject": "Flip both flag defaults to Grpc + soak; delete Akka transports, ClusterClient creation, DefaultSiteClientFactory, receptionist registrations, CentralContactPoints, then the flags",
"status": "completed",
"activeForm": "Deleting the ClusterClient transport",
"blockedBy": [
"P2",
"P3"
],
"notes": "Branch feat/grpc-phase4-deletion (2026-07-23). Flags DELETED rather than flipped — end state is gRPC-only, identical. Deleted: AkkaCentralTransport, AkkaSiteTransport, ISiteClientFactory+DefaultSiteClientFactory, CentralCommunicationActor legacy ctor + SelectTransport, ClusterClient creation + both ClusterClientReceptionist.RegisterService in AkkaHostedService, CommunicationOptions.CentralContactPoints + CentralTransport/SiteTransport flags + CentralTransportMode/SiteTransportKind enums, RegisterCentralClient message + receive block, AkkaCentralTransportTests + DefaultSiteClientFactoryTests. Added NoOpCentralTransport (fail-loud null-default so command-dispatch TestKit suites keep constructing without a wired transport; production always injects GrpcCentralTransport). CommunicationOptionsValidator: CentralGrpcEndpoints now unconditional (no-blank-entries, role-agnostic); StartupValidator: Site-only requires ≥1 CentralGrpcEndpoint (fail-fast, mirrors GrpcPsk). Host builds GrpcSiteTransport (central) + GrpcCentralTransport (site) unconditionally. Kept Akka.Cluster.Tools (ClusterSingleton). Test rework: deleted the ClusterClient.Send per-site-routing tests (covered by CentralCommunicationActorTransportTests + GrpcSiteTransport suites), swapped ISiteClientFactory→substitute ISiteCommandTransport across 5 files, converted SiteAuditPushFlow's ClusterClientRelay→BridgeCentralTransport, converted Heartbeat_StampsIsActive to a substitute ICentralTransport, repurposed HealthReportAck no-client test to the NoOp fail-loud path, removed SiteActors_CentralClusterClient_Exists. Rig: docker ×6 + docker-env2 ×2 + Host appsettings.Site.json + deploy/wonder-app-vd03 moved CentralContactPoints→CentralGrpcEndpoints. Full solution build 0/0; Communication.Tests 640, StartupValidator 59, SiteActorPath 5, audit-push integration 1 all green. DID NOT fold in the dead IntegrationCallRequest deletion (#32) — SiteEnvelope routing is transport-agnostic so it still compiles; user-owned behavioral decision, left OUT of this PR."
},
{
"id": "P4.2",
"phase": "4",
"subject": "Grep-gates pass + docs updated (grpc_streams.md, Component-Host.md, Component-StoreAndForward.md, known-issues cross-ref)",
"status": "completed",
"activeForm": "Running the deletion grep-gates and doc sweep",
"blockedBy": [
"P4.1"
],
"notes": "Grep-gates: CentralContactPoints → only intentional 'replaces the former' doc refs + plan trackers + stale bin/ artifacts (regenerate on build); deleted symbols (DefaultSiteClientFactory/ISiteClientFactory/RegisterCentralClient/AkkaCentralTransport/AkkaSiteTransport/CentralTransportMode/SiteTransportKind/the flags) → 0 live refs. Residual 'clusterclient' in src = ClusterClientSiteAuditClient (misleadingly named but transport-agnostic — it Asks SiteCommunicationActor, holds no ClusterClient; works unchanged) + ~25 stale inline XML-doc comments across SiteRuntime/Commons/SiteCallAudit — NOT scrubbed (out of plan scope, no behavior impact); FOLLOW-UP noted. Docs updated (subagent + me): Component-Communication.md, components/Communication.md, Component-Host.md, Component-StoreAndForward.md, deployment/topology-guide.md, plans/grpc_streams.md (SUPERSEDED note + LoadSiteAddressesFromDb correction), known-issues/2026-06-26-deploy-config-exceeds-akka-frame-size.md (2026-07-23 frame-size-retired amendment), and CLAUDE.md (2 transport-decision passages). Phase 5 (live gate) is next and requires a rig redeploy."
},
{
"id": "P5",
"phase": "5",
"subject": "Live gate, 8 checks, recorded in docs/plans/2026-07-22-clusterclient-to-grpc-live-gate.md",
"status": "pending",
"activeForm": "Running the live gate",
"blockedBy": [
"P4.2"
]
}
]
}