docs(akka): ClusterClient→gRPC migration design + chunking addendum
akka_msquic_transport.md gains §8a (Akka.Cluster.Chunking / Akka.Delivery — the 1.5-era in-cluster large-message tool, why it can't cross the ClusterClient boundary) and the Artery-now-in-development §9 fix. scadabridge_clusterclient_to_grpc.md: full migration assessment — ClusterClient inventory (two-actor choke points, 29 commands incl. one dead), gRPC-only design (topology, oneof contracts, active-node analysis: singleton-routed so transport needs only either-node failover+failback), PSK-from-Secrets auth, frame-cap-raise tradeoffs, and §7 deep-dive corrections. Companion impl plan lives in ScadaBridge/docs/plans.
This commit is contained in:
@@ -282,6 +282,52 @@ Mostly no — mapping the family's gRPC surfaces:
|
||||
The "big messages so we don't need side channels" hope is really an Artery feature (dedicated
|
||||
chunked large-message streams), not a QUIC-transport feature.
|
||||
|
||||
### 8a. The overlooked 1.5-era tool: Akka.Cluster.Chunking (added 2026-07-22)
|
||||
|
||||
One mechanism was missing from the original analysis — Petabridge's
|
||||
**[Akka.Cluster.Chunking](https://github.com/petabridge/Akka.Cluster.Chunking)** (Apache-2.0),
|
||||
which exists precisely to bridge the pre-Artery gap this report is about: sending large messages
|
||||
(5 MB+) over Akka.Remote **without raising `maximum-frame-size` and without head-of-line
|
||||
blocking**. It chunks at the application layer on top of `Akka.Delivery`'s reliable-delivery
|
||||
machinery (ordered, ack'd, resend-on-loss), so every chunk is an ordinary sub-frame-cap remoting
|
||||
message and heartbeats/system messages interleave between chunks — directly defusing the §2
|
||||
guardrail concern with zero transport work.
|
||||
|
||||
Usage is pleasantly transparent: `ChunkingManager.For(system)` then
|
||||
`chunker.DeliverChunked(message, recipient, sender)` (Akka.Hosting: `.AddChunkingManager()`);
|
||||
config via `akka.cluster.chunking.chunk-size` (default 64 kB), `outbound-queue-capacity`
|
||||
(default 20), `request-timeout` (default 5 s). Unlike raw `Akka.Delivery`, no per-flow
|
||||
`ProducerController`/`ConsumerController` pair has to be hand-wired — the manager runs the
|
||||
delivery channels per node pair.
|
||||
|
||||
The honest caveats for our context:
|
||||
|
||||
- **Maturity.** v0.4.0 (Aug 2025), and the README says plainly it is "currently an experiment and
|
||||
is available on an as-is basis." For an OT product that's a real adoption bar; budget for
|
||||
owning/forking it if Petabridge doesn't graduate it.
|
||||
- **Acceptance ≠ delivery.** `DeliverChunked`'s task completes when the message is *accepted* for
|
||||
delivery, not when the far side has it — callers needing end-to-end confirmation still build
|
||||
their own ack on top.
|
||||
- **Requires Akka.Cluster, in-cluster only.** It rides remoting between cluster members. It does
|
||||
not cross the ScadaBridge site↔central boundary — two separate Akka clusters bridged by
|
||||
ClusterClient/gRPC — which is exactly where the family's big payloads actually flow.
|
||||
**Nor does ClusterClient itself chunk** (the tunneled payload + receptionist envelope must fit
|
||||
one frame). It *could* be bolted on — ClusterClient replies carry real IActorRefs, so raw
|
||||
`Akka.Delivery` could then run point-to-point over a direct cross-cluster remoting association —
|
||||
but that reopens exactly the direct-remoting coupling (firewall surface, quarantine interplay)
|
||||
the seed-partitioned boundary exists to avoid; large cross-boundary payloads stay on the gRPC
|
||||
side-channels.
|
||||
- **Correctness tool, not throughput tool** — shallow queues and the reliable-delivery handshake
|
||||
make it slower than plain tells; it removes a ceiling, it doesn't add speed.
|
||||
|
||||
Re-mapping §8 with this in hand changes no conclusion: mxaccessgw/HistorianGateway peers aren't
|
||||
cluster members; LocalDb sync is independent of cluster state *by design*; the ScadaBridge
|
||||
bulk/artifact side channels are cross-cluster, out of reach. Where it **is** the right answer is
|
||||
any *future within-cluster* large-message need (e.g. a big payload between the members of one
|
||||
pair): reach for Akka.Cluster.Chunking (or raw `Akka.Delivery` with
|
||||
`ProducerController.Settings.ChunkLargeMessagesBytes`, the in-box primitive it wraps) instead of
|
||||
raising the frame cap — until 1.6's Artery large-message lanes make both workarounds unnecessary.
|
||||
|
||||
## 9. Options
|
||||
|
||||
**A. Don't build it; tune what we have; track 1.6 (recommended).** Best-practice fit:
|
||||
@@ -289,8 +335,9 @@ transport-layer plumbing is exactly the code an OT product wants to own least, a
|
||||
design explicitly targets the wishlist (large-message channels, quarantine-bypass control channel —
|
||||
the latter directly relevant to the 2-node availability-first failover posture). Concretely: bump
|
||||
1.5.62 → 1.5.70 (eight releases of remoting/streams fixes, including a cluster-wedging
|
||||
consistent-hash-router bug), and if specific pains exist now, raise `maximum-frame-size` and/or
|
||||
enable the new dot-netty mTLS. Subscribe to #7466; when real alphas ship, the pair-restart
|
||||
consistent-hash-router bug), and if specific pains exist now, prefer Akka.Cluster.Chunking /
|
||||
`Akka.Delivery` chunking (§8a) over raising `maximum-frame-size` for any within-cluster large
|
||||
message, and/or enable the new dot-netty mTLS. Subscribe to #7466; when real alphas ship, the pair-restart
|
||||
discipline makes adoption cheap.
|
||||
|
||||
**B. Build a classic-SPI QUIC transport now.** ~3–5 weeks to production quality per §3. Choose this
|
||||
@@ -306,7 +353,8 @@ all get exercised) without owning it in production. A reasonable middle path.
|
||||
**Recommendation: A**, with **C** if hands-on evidence is wanted — the honest summary is that for
|
||||
this workload, an MsQuic transport on Akka 1.5 buys security and handshake ergonomics that are
|
||||
mostly available from config today, while the features that would actually reduce the gRPC
|
||||
side-channels only arrive with the Artery pipeline rewrite, which upstream hasn't started coding.
|
||||
side-channels only arrive with the Artery pipeline rewrite — now in active development upstream
|
||||
(see the §1 status update) — with Akka.Cluster.Chunking (§8a) as the interim in-cluster answer.
|
||||
|
||||
## Sources
|
||||
|
||||
@@ -317,3 +365,5 @@ side-channels only arrive with the Artery pipeline rewrite, which upstream hasn'
|
||||
- [Akka.NET remoting transports documentation](https://getakka.net/articles/remoting/transports.html)
|
||||
- [Akka.NET PR #7851 — mutual TLS for DotNetty transport](https://github.com/akkadotnet/akka.net/pull/7851)
|
||||
- [QUIC support in .NET (System.Net.Quic)](https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/quic/quic-overview)
|
||||
- [petabridge/Akka.Cluster.Chunking](https://github.com/petabridge/Akka.Cluster.Chunking) — v0.4.0, Apache-2.0, experimental
|
||||
- [Akka.NET reliable delivery (Akka.Delivery)](https://getakka.net/articles/actors/reliable-delivery.html)
|
||||
|
||||
Reference in New Issue
Block a user