feat(cluster): opt-in Akka.Remote TLS block, default off (ScadaBridge#64)

ScadaBridge:Cluster:RemotingTls (ClusterRemotingTlsOptions) renders
akka.remote.dot-netty.tcp enable-ssl + ssl { certificate, mutual auth,
hostname validation } only when Enabled. Chain validation is always on
(no suppress-validation switch). Certificate is a .pfx path + password
(${secret:} reference) or an OS-store thumbprint.

ClusterOptionsValidator, TLS on only: exactly one certificate source, a
legal store location, and akka.ssl.tcp:// on every seed - Akka.Remote
advertises akka.ssl.tcp addresses once SSL is on, so an akka.tcp seed is
never answered and the join retries forever with no startup error.

ClusterBootstrapGuard.SeedPattern accepted only one scheme segment, so
akka.ssl.tcp seeds parsed as nothing and the guard would silently never
engage on a TLS pair; it now accepts any number.

Default HOCON is byte-identical: 12 rendered documents (3 downing
strategies x guard on/off x Central/Site) captured before the change and
diffed after, all identical; a permanent test pins that a disabled block
with every field set renders the default document. A real ActorSystem
booted on the emitted TLS document comes up as akka.ssl.tcp (mutation
check: forcing enable-ssl=false turns it red).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV

src/ZB.MOM.WW.ScadaBridge.ClusterInfrastructure/ClusterBootstrapGuard.cs

@@ -45,8 +45,12 @@ namespace ZB.MOM.WW.ScadaBridge.ClusterInfrastructure;
 /// </remarks>
 public static class ClusterBootstrapGuard
 {
+    // Any number of dot segments after "akka": akka.tcp AND akka.ssl.tcp (Akka.Remote's scheme
+    // once dot-netty TLS is on, ScadaBridge#64). The old single-segment form did not match
+    // akka.ssl.tcp, so a TLS pair would have parsed no seeds and the guard would silently never
+    // engage.
     private static readonly Regex SeedPattern =
-        new(@"^akka(?:\.[a-z0-9]+)?://[^@/]+@(?<host>[^:/]+):(?<port>\d+)", RegexOptions.IgnoreCase | RegexOptions.Compiled);
+        new(@"^akka(?:\.[a-z0-9]+)*://[^@/]+@(?<host>[^:/]+):(?<port>\d+)", RegexOptions.IgnoreCase | RegexOptions.Compiled);
 
     /// <summary>The analyzed bootstrap role of this node within its seed list.</summary>
     /// <param name="Applies">True only when the guard engages: exactly two seeds, one of them this node.</param>

src/ZB.MOM.WW.ScadaBridge.ClusterInfrastructure/ClusterOptions.cs

@@ -135,6 +135,81 @@ public class ClusterOptions
     /// decision logic and <c>ClusterBootstrapCoordinator</c> for the runtime.
     /// </summary>
     public ClusterBootstrapGuardOptions BootstrapGuard { get; set; } = new();
+
+    /// <summary>
+    /// TLS for Akka.Remote within a pair (<c>ScadaBridge:Cluster:RemotingTls</c>, ScadaBridge#64).
+    /// Default OFF — a dark switch: with it off <c>AkkaHostedService.BuildHocon</c> emits no SSL
+    /// keys at all, so the remoting document is byte-identical to before. See
+    /// <see cref="ClusterRemotingTlsOptions"/> and
+    /// <c>docs/plans/2026-11-09-tls-production-profile-design.md</c>.
+    /// </summary>
+    public ClusterRemotingTlsOptions RemotingTls { get; set; } = new();
+}
+
+/// <summary>
+/// Opt-in TLS for the Akka.Remote DotNetty transport (ScadaBridge#64). Maps onto
+/// <c>akka.remote.dot-netty.tcp.enable-ssl</c> and its <c>ssl</c> block. Each node carries its
+/// OWN certificate (a cert per node); trust is the chain to an issuing CA in the node's OS trust
+/// store, which is how Akka.Remote validates the peer.
+/// </summary>
+/// <remarks>
+/// <para>
+/// <b>Turning it on changes every node address.</b> Akka.Remote's scheme becomes
+/// <c>akka.ssl.tcp</c>, so every seed node must be written <c>akka.ssl.tcp://scadabridge@host:port</c>
+/// — <see cref="ClusterOptionsValidator"/> refuses a TLS-enabled config whose seeds still say
+/// <c>akka.tcp</c>. Both nodes of a pair must flip together: a TLS node and a plaintext node cannot
+/// talk to each other.
+/// </para>
+/// <para>
+/// <b>Deliberately no <c>suppress-validation</c> switch.</b> Akka's own knob disables chain
+/// validation entirely; exposing it would make "TLS on" mean nothing. A lab uses a lab CA instead.
+/// </para>
+/// </remarks>
+public sealed class ClusterRemotingTlsOptions
+{
+    /// <summary>Whether Akka.Remote runs over TLS. Default <see langword="false"/>.</summary>
+    public bool Enabled { get; set; }
+
+    /// <summary>
+    /// Path to this node's PKCS#12 (<c>.pfx</c>) certificate with its private key. Set this OR
+    /// <see cref="CertificateThumbprint"/>, never both.
+    /// </summary>
+    public string? CertificatePath { get; set; }
+
+    /// <summary>
+    /// Password for <see cref="CertificatePath"/>. Write it as a <c>${secret:NAME}</c> reference so the
+    /// pre-host expander resolves it from the Secrets store; never commit a literal.
+    /// </summary>
+    public string? CertificatePassword { get; set; }
+
+    /// <summary>
+    /// Thumbprint of this node's certificate in the OS certificate store (the Windows production
+    /// path). Set this OR <see cref="CertificatePath"/>, never both.
+    /// </summary>
+    public string? CertificateThumbprint { get; set; }
+
+    /// <summary>Certificate store name for <see cref="CertificateThumbprint"/>. Default <c>My</c> (Personal).</summary>
+    public string StoreName { get; set; } = "My";
+
+    /// <summary>
+    /// Certificate store location for <see cref="CertificateThumbprint"/>: <c>local-machine</c>
+    /// (default — a Windows service account reads it) or <c>current-user</c>.
+    /// </summary>
+    public string StoreLocation { get; set; } = "local-machine";
+
+    /// <summary>
+    /// Whether both ends must present a certificate (mutual TLS). Default <see langword="true"/>,
+    /// matching Akka's own default. With it on, an unauthenticated host can no longer join or
+    /// send to the pair — which is the point, since Akka.Remote has no other authentication.
+    /// </summary>
+    public bool RequireMutualAuthentication { get; set; } = true;
+
+    /// <summary>
+    /// Whether the peer certificate's CN/SAN must match the host being dialled. Default
+    /// <see langword="false"/>, matching Akka's default; the trust anchor is the issuing CA.
+    /// Turn on only when every seed host name is a SAN on the peer's certificate.
+    /// </summary>
+    public bool ValidateCertificateHostname { get; set; }
 }
 
 /// <summary>

src/ZB.MOM.WW.ScadaBridge.ClusterInfrastructure/ClusterOptionsValidator.cs

@@ -34,6 +34,7 @@ public sealed class ClusterOptionsValidator : OptionsValidatorBase<ClusterOption
     protected override void Validate(ValidationBuilder builder, ClusterOptions options)
     {
         ValidateBootstrapGuard(builder, options.BootstrapGuard);
+        ValidateRemotingTls(builder, options.RemotingTls, options.SeedNodes);
 
         // The design doc states "both nodes are seed nodes — each node lists
         // both itself and its partner" so a properly-configured deployment lists
@@ -84,6 +85,47 @@ public sealed class ClusterOptionsValidator : OptionsValidatorBase<ClusterOption
             + "younger node forms its own, producing two live clusters.");
     }
 
+    /// <summary>
+    /// When Akka.Remote TLS is enabled (ScadaBridge#64): exactly one certificate source, a legal
+    /// store location, and every seed node on the <c>akka.ssl.tcp</c> scheme. Akka.Remote advertises
+    /// <c>akka.ssl.tcp://</c> addresses once SSL is on, so an <c>akka.tcp://</c> seed names an
+    /// address no node listens as — the join would retry forever with no startup error. Nothing is
+    /// checked when TLS is off, so the default configuration can never be blocked by this.
+    /// </summary>
+    private static void ValidateRemotingTls(
+        ValidationBuilder builder, ClusterRemotingTlsOptions? tls, IReadOnlyList<string>? seedNodes)
+    {
+        if (tls is null || !tls.Enabled)
+        {
+            return;
+        }
+
+        var hasPath = !string.IsNullOrWhiteSpace(tls.CertificatePath);
+        var hasThumbprint = !string.IsNullOrWhiteSpace(tls.CertificateThumbprint);
+        builder.RequireThat(hasPath ^ hasThumbprint,
+            "ClusterOptions.RemotingTls requires exactly one certificate source when Enabled: "
+            + "CertificatePath (a .pfx) or CertificateThumbprint (OS certificate store), not both and not neither.");
+
+        if (hasThumbprint)
+        {
+            builder.RequireThat(!string.IsNullOrWhiteSpace(tls.StoreName),
+                "ClusterOptions.RemotingTls.StoreName must be set when CertificateThumbprint is used (e.g. 'My').");
+            builder.RequireThat(
+                string.Equals(tls.StoreLocation, "local-machine", StringComparison.Ordinal)
+                || string.Equals(tls.StoreLocation, "current-user", StringComparison.Ordinal),
+                $"ClusterOptions.RemotingTls.StoreLocation must be 'local-machine' or 'current-user' (was '{tls.StoreLocation}').");
+        }
+
+        var nonTlsSeeds = (seedNodes ?? Array.Empty<string>())
+            .Where(seed => !(seed ?? string.Empty).TrimStart()
+                .StartsWith("akka.ssl.tcp://", StringComparison.OrdinalIgnoreCase))
+            .ToList();
+        builder.RequireThat(nonTlsSeeds.Count == 0,
+            "ClusterOptions.SeedNodes must use the 'akka.ssl.tcp://' scheme when RemotingTls is enabled "
+            + "(Akka.Remote advertises akka.ssl.tcp addresses over TLS, so an akka.tcp seed is never answered); "
+            + $"offending: {string.Join(", ", nonTlsSeeds)}.");
+    }
+
     /// <summary>
     /// When the bootstrap guard is enabled, its timing knobs must be positive. A
     /// zero/negative <see cref="ClusterBootstrapGuardOptions.PartnerProbeSeconds"/> in particular

src/ZB.MOM.WW.ScadaBridge.Host/Actors/AkkaHostedService.cs

@@ -311,6 +311,10 @@ public class AkkaHostedService : IHostedService
             ? $"{LeaderAutoDowningProvider.UnreachableAfterPath} = {DurationHocon(clusterOptions.StableAfter)}\n"
             : string.Empty;
 
+        // Akka.Remote TLS (ScadaBridge#64, opt-in). Emitted inside dot-netty.tcp only when
+        // ScadaBridge:Cluster:RemotingTls:Enabled, so a TLS-off document is byte-identical to before.
+        var remotingTlsBlock = BuildRemotingTlsBlock(clusterOptions.RemotingTls);
+
         return $@"
 {leaderAutoDownBlock}audit-telemetry-dispatcher {{
     type = ForkJoinDispatcher
@@ -329,7 +333,7 @@ akka {{
     remote {{
         dot-netty.tcp {{
             hostname = {QuoteHocon(nodeOptions.NodeHostname)}
-            port = {nodeOptions.RemotingPort}
+            port = {nodeOptions.RemotingPort}{remotingTlsBlock}
         }}
         transport-failure-detector {{
             heartbeat-interval = {DurationHocon(transportHeartbeat)}
@@ -354,6 +358,47 @@ akka {{
 }}";
     }
 
+    /// <summary>
+    /// Renders the <c>enable-ssl</c> + <c>ssl</c> keys for <c>akka.remote.dot-netty.tcp</c>
+    /// (ScadaBridge#64), or an empty string when TLS is off. The leading newline is part of the
+    /// block so the disabled case adds nothing — not even whitespace — to the document.
+    /// </summary>
+    /// <remarks>
+    /// <c>suppress-validation</c> is always emitted <c>false</c> on purpose: there is no option to
+    /// turn chain validation off (see <see cref="ClusterRemotingTlsOptions"/>). Exactly one of the
+    /// two certificate sources is rendered; <see cref="ClusterOptionsValidator"/> has already
+    /// refused a config carrying both or neither, and a thumbprint wins here if it ever got through.
+    /// </remarks>
+    /// <param name="tls">The bound TLS options; null or disabled renders nothing.</param>
+    /// <returns>The HOCON fragment, or <see cref="string.Empty"/>.</returns>
+    internal static string BuildRemotingTlsBlock(ClusterRemotingTlsOptions? tls)
+    {
+        if (tls is null || !tls.Enabled)
+        {
+            return string.Empty;
+        }
+
+        var certificate = !string.IsNullOrWhiteSpace(tls.CertificateThumbprint)
+            ? $@"use-thumbprint-over-file = true
+                    thumbprint = {QuoteHocon(tls.CertificateThumbprint)}
+                    store-name = {QuoteHocon(tls.StoreName)}
+                    store-location = {QuoteHocon(tls.StoreLocation)}"
+            : $@"use-thumbprint-over-file = false
+                    path = {QuoteHocon(tls.CertificatePath)}
+                    password = {QuoteHocon(tls.CertificatePassword)}";
+
+        return $@"
+            enable-ssl = true
+            ssl {{
+                certificate {{
+                    {certificate}
+                }}
+                suppress-validation = false
+                require-mutual-authentication = {(tls.RequireMutualAuthentication ? "on" : "off")}
+                validate-certificate-hostname = {(tls.ValidateCertificateHostname ? "on" : "off")}
+            }}";
+    }
+
     /// <summary>
     /// Renders a <see cref="TimeSpan"/> as a HOCON duration in milliseconds. Akka's
     /// HOCON parser accepts a <c>ms</c> suffix, so emitting whole milliseconds