docs(akka): ClusterClient→gRPC migration design + chunking addendum

akka_msquic_transport.md gains §8a (Akka.Cluster.Chunking / Akka.Delivery —
the 1.5-era in-cluster large-message tool, why it can't cross the ClusterClient
boundary) and the Artery-now-in-development §9 fix.
scadabridge_clusterclient_to_grpc.md: full migration assessment — ClusterClient
inventory (two-actor choke points, 29 commands incl. one dead), gRPC-only
design (topology, oneof contracts, active-node analysis: singleton-routed so
transport needs only either-node failover+failback), PSK-from-Secrets auth,
frame-cap-raise tradeoffs, and §7 deep-dive corrections. Companion impl plan
lives in ScadaBridge/docs/plans.
This commit is contained in:
Joseph Doherty
2026-07-22 17:10:30 -04:00
parent 6dd7858619
commit 4d93d2580b
2 changed files with 404 additions and 3 deletions
+53 -3
View File
@@ -282,6 +282,52 @@ Mostly no — mapping the family's gRPC surfaces:
The "big messages so we don't need side channels" hope is really an Artery feature (dedicated
chunked large-message streams), not a QUIC-transport feature.
### 8a. The overlooked 1.5-era tool: Akka.Cluster.Chunking (added 2026-07-22)
One mechanism was missing from the original analysis — Petabridge's
**[Akka.Cluster.Chunking](https://github.com/petabridge/Akka.Cluster.Chunking)** (Apache-2.0),
which exists precisely to bridge the pre-Artery gap this report is about: sending large messages
(5 MB+) over Akka.Remote **without raising `maximum-frame-size` and without head-of-line
blocking**. It chunks at the application layer on top of `Akka.Delivery`'s reliable-delivery
machinery (ordered, ack'd, resend-on-loss), so every chunk is an ordinary sub-frame-cap remoting
message and heartbeats/system messages interleave between chunks — directly defusing the §2
guardrail concern with zero transport work.
Usage is pleasantly transparent: `ChunkingManager.For(system)` then
`chunker.DeliverChunked(message, recipient, sender)` (Akka.Hosting: `.AddChunkingManager()`);
config via `akka.cluster.chunking.chunk-size` (default 64 kB), `outbound-queue-capacity`
(default 20), `request-timeout` (default 5 s). Unlike raw `Akka.Delivery`, no per-flow
`ProducerController`/`ConsumerController` pair has to be hand-wired — the manager runs the
delivery channels per node pair.
The honest caveats for our context:
- **Maturity.** v0.4.0 (Aug 2025), and the README says plainly it is "currently an experiment and
is available on an as-is basis." For an OT product that's a real adoption bar; budget for
owning/forking it if Petabridge doesn't graduate it.
- **Acceptance ≠ delivery.** `DeliverChunked`'s task completes when the message is *accepted* for
delivery, not when the far side has it — callers needing end-to-end confirmation still build
their own ack on top.
- **Requires Akka.Cluster, in-cluster only.** It rides remoting between cluster members. It does
not cross the ScadaBridge site↔central boundary — two separate Akka clusters bridged by
ClusterClient/gRPC — which is exactly where the family's big payloads actually flow.
**Nor does ClusterClient itself chunk** (the tunneled payload + receptionist envelope must fit
one frame). It *could* be bolted on — ClusterClient replies carry real IActorRefs, so raw
`Akka.Delivery` could then run point-to-point over a direct cross-cluster remoting association —
but that reopens exactly the direct-remoting coupling (firewall surface, quarantine interplay)
the seed-partitioned boundary exists to avoid; large cross-boundary payloads stay on the gRPC
side-channels.
- **Correctness tool, not throughput tool** — shallow queues and the reliable-delivery handshake
make it slower than plain tells; it removes a ceiling, it doesn't add speed.
Re-mapping §8 with this in hand changes no conclusion: mxaccessgw/HistorianGateway peers aren't
cluster members; LocalDb sync is independent of cluster state *by design*; the ScadaBridge
bulk/artifact side channels are cross-cluster, out of reach. Where it **is** the right answer is
any *future within-cluster* large-message need (e.g. a big payload between the members of one
pair): reach for Akka.Cluster.Chunking (or raw `Akka.Delivery` with
`ProducerController.Settings.ChunkLargeMessagesBytes`, the in-box primitive it wraps) instead of
raising the frame cap — until 1.6's Artery large-message lanes make both workarounds unnecessary.
## 9. Options
**A. Don't build it; tune what we have; track 1.6 (recommended).** Best-practice fit:
@@ -289,8 +335,9 @@ transport-layer plumbing is exactly the code an OT product wants to own least, a
design explicitly targets the wishlist (large-message channels, quarantine-bypass control channel —
the latter directly relevant to the 2-node availability-first failover posture). Concretely: bump
1.5.62 → 1.5.70 (eight releases of remoting/streams fixes, including a cluster-wedging
consistent-hash-router bug), and if specific pains exist now, raise `maximum-frame-size` and/or
enable the new dot-netty mTLS. Subscribe to #7466; when real alphas ship, the pair-restart
consistent-hash-router bug), and if specific pains exist now, prefer Akka.Cluster.Chunking /
`Akka.Delivery` chunking (§8a) over raising `maximum-frame-size` for any within-cluster large
message, and/or enable the new dot-netty mTLS. Subscribe to #7466; when real alphas ship, the pair-restart
discipline makes adoption cheap.
**B. Build a classic-SPI QUIC transport now.** ~35 weeks to production quality per §3. Choose this
@@ -306,7 +353,8 @@ all get exercised) without owning it in production. A reasonable middle path.
**Recommendation: A**, with **C** if hands-on evidence is wanted — the honest summary is that for
this workload, an MsQuic transport on Akka 1.5 buys security and handshake ergonomics that are
mostly available from config today, while the features that would actually reduce the gRPC
side-channels only arrive with the Artery pipeline rewrite, which upstream hasn't started coding.
side-channels only arrive with the Artery pipeline rewrite — now in active development upstream
(see the §1 status update) — with Akka.Cluster.Chunking (§8a) as the interim in-cluster answer.
## Sources
@@ -317,3 +365,5 @@ side-channels only arrive with the Artery pipeline rewrite, which upstream hasn'
- [Akka.NET remoting transports documentation](https://getakka.net/articles/remoting/transports.html)
- [Akka.NET PR #7851 — mutual TLS for DotNetty transport](https://github.com/akkadotnet/akka.net/pull/7851)
- [QUIC support in .NET (System.Net.Quic)](https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/quic/quic-overview)
- [petabridge/Akka.Cluster.Chunking](https://github.com/petabridge/Akka.Cluster.Chunking) — v0.4.0, Apache-2.0, experimental
- [Akka.NET reliable delivery (Akka.Delivery)](https://getakka.net/articles/actors/reliable-delivery.html)