fix(mesh): resolve dialer + PSK provider lazily — central boots again
Phase 1A live gate defect 4. Neither central node booted on this branch,
in ANY transport mode:
Akka.Hosting.MissingActorRegistryEntryException: No actor registered
for key ...ControlPlane.AuditWriterActorKey
at AdminUI.Audit.ActorAuditWriter..ctor(ActorRegistry registry)
at ControlPlane.ServiceCollectionExtensions.<WithOtOpcUaControlPlaneSingletons>
WithOtOpcUaControlPlaneSingletons' first WithActors starter resolved
IMeshCommandDialer and IMeshPskProvider eagerly for the Props. Their
graph reaches back into the registry — dialer -> IMeshPskProvider ->
ISecretResolver -> AddZbSecrets -> IAuditWriter -> ActorAuditWriter,
whose ctor calls registry.Get<AuditWriterActorKey>() — and
WithSingleton<AuditWriterActorKey> is registered ~60 lines LATER in the
same builder. Boot died before the entry existed. Mode-independent: the
resolve happened before anything consulted MeshTransport:Mode, so Dps
(the default, and the rollback target) died exactly like Grpc.
Both dependencies now arrive as Func<> and are invoked in PreStart, by
which point the host is fully built — the same shape, for the same
class of reason, as the coordinator proxy Func one argument earlier.
PreStart re-resolves on restart rather than caching across one; both
are container singletons, so a restart gets the same instances. A
resolution throw is caught and logged rather than propagated: a throw
out of PreStart is an actor failure, the supervisor answers it with a
restart, and a broken DI graph is not transient — that is an unbounded
restart loop that would take the transport-independent ack path down
with it.
Regression pin: ControlPlaneStarterOrderTests boots a REAL admin host
through WithOtOpcUaClusterBootstrap + WithOtOpcUaControlPlaneSingletons
in production starter order, with the production graph from
IMeshCommandDialer down to a registry-bound IAuditWriter, and asserts
StartAsync itself — the failure occurs while the builder's starters
execute, so building a ServiceProvider and resolving by hand cannot
reproduce it. Verified RED against the eager resolve: 3/3 modes fail
with MissingActorRegistryEntryException. A second fact pins that the
deferral did not become 'never resolved' — the actor must resolve the
provider from the container and drive it through the contact refresh,
signalled by a TaskCompletionSource (PreStart runs on the actor thread
and the contact load is a piped DB read, so no synchronous assertion
after StartAsync is correct).
Why nothing caught this: the dialer's container-activation test resolves
from a collection with no secrets registration, so its graph stops one
hop in; the actor facts inject fakes and never touch DI; and the
existing whole-host boots register no mesh provider, so the eager
lookups returned null and walked nothing.
Two aggravators recorded by the gate, no code here: the failed process
did not exit (docker Up while /health/ready 503s — Phase 2 readiness
concern, tracked), and this is the third instance of the same
eager-resolve-during-AddAkka class after scadaproj#1 and mesh Phase 6's
IClusterRoleInfo boot crash.
ControlPlane.Tests 236 -> 240, Host.Tests 92, Cluster.Tests 184,
0 warnings in every touched project.
This commit is contained in:
+86
-17
@@ -64,8 +64,15 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
|
|||||||
private readonly MeshTransportOptions _options;
|
private readonly MeshTransportOptions _options;
|
||||||
private readonly IMeshClusterClientFactory _clientFactory;
|
private readonly IMeshClusterClientFactory _clientFactory;
|
||||||
private readonly Func<IActorRef?> _coordinator;
|
private readonly Func<IActorRef?> _coordinator;
|
||||||
private readonly IMeshCommandDialer? _dialer;
|
|
||||||
private readonly IMeshPskProvider? _pskProvider;
|
// Resolved in PreStart, never at construction. See the Props doc: the DI graph behind these two
|
||||||
|
// reaches actor-registry entries that do not exist yet while the Akka builder's starters are
|
||||||
|
// still running, and touching it there is what stopped every central node from booting.
|
||||||
|
private readonly Func<IMeshCommandDialer?>? _dialerFactory;
|
||||||
|
private readonly Func<IMeshPskProvider?>? _pskProviderFactory;
|
||||||
|
private IMeshCommandDialer? _dialer;
|
||||||
|
private IMeshPskProvider? _pskProvider;
|
||||||
|
|
||||||
private readonly ILoggingAdapter _log = Context.GetLogger();
|
private readonly ILoggingAdapter _log = Context.GetLogger();
|
||||||
private readonly bool _useClusterClient;
|
private readonly bool _useClusterClient;
|
||||||
private readonly bool _useGrpc;
|
private readonly bool _useGrpc;
|
||||||
@@ -100,24 +107,45 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
|
|||||||
/// time — it resolves on first use and caches.
|
/// time — it resolves on first use and caches.
|
||||||
/// </param>
|
/// </param>
|
||||||
/// <param name="dialer">
|
/// <param name="dialer">
|
||||||
/// Places one command call on one node, under <see cref="MeshTransportOptions.ModeGrpc"/>.
|
/// Resolves the thing that places one command call on one node, under
|
||||||
/// Optional so the two older transports need nothing new wired; a <see langword="null"/>
|
/// <see cref="MeshTransportOptions.ModeGrpc"/>. Optional so the two older transports need
|
||||||
/// dialer under <c>Grpc</c> is a mis-wiring and is reported loudly rather than silently
|
/// nothing new wired; a <see langword="null"/> dialer under <c>Grpc</c> is a mis-wiring and
|
||||||
/// dropping every command.
|
/// is reported loudly rather than silently dropping every command.
|
||||||
/// </param>
|
/// </param>
|
||||||
/// <param name="pskProvider">
|
/// <param name="pskProvider">
|
||||||
/// Central's per-cluster key lookup, told on every contact refresh which clusters exist (the
|
/// Resolves central's per-cluster key lookup, told on every contact refresh which clusters
|
||||||
/// known-cluster gate) and to drop its cache (so a rotated key takes effect within one
|
/// exist (the known-cluster gate) and to drop its cache (so a rotated key takes effect within
|
||||||
/// refresh). Optional for the same reason as <paramref name="dialer"/>.
|
/// one refresh). Optional for the same reason as <paramref name="dialer"/>.
|
||||||
/// </param>
|
/// </param>
|
||||||
/// <returns>The props.</returns>
|
/// <returns>The props.</returns>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// <b>Both dependencies arrive as <see cref="Func{TResult}"/>, and that is not a style
|
||||||
|
/// choice.</b> Passing instances meant the caller resolved them while the Akka builder's
|
||||||
|
/// starters were still running, and the DI graph behind them reaches back into the actor
|
||||||
|
/// registry: dialer → <c>IMeshPskProvider</c> → <c>ISecretResolver</c> → <c>AddZbSecrets</c>
|
||||||
|
/// → <c>IAuditWriter</c> → <c>ActorAuditWriter</c>, whose constructor calls
|
||||||
|
/// <c>registry.Get<AuditWriterActorKey>()</c> — a singleton registered dozens of
|
||||||
|
/// lines LATER in the very same builder. Every central node failed to boot, in every
|
||||||
|
/// transport mode, on a <c>MissingActorRegistryEntryException</c>. Deferring to
|
||||||
|
/// <see cref="PreStart"/> moves the first resolve to a point where the host is fully
|
||||||
|
/// built, and matches the <see cref="Func{TResult}"/> the coordinator proxy already uses
|
||||||
|
/// one argument earlier for the same class of reason.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// This is the third instance of the same failure mode in this family — eager resolution
|
||||||
|
/// inside an <c>AddAkka</c> configurator, previously <c>scadaproj#1</c> (a DI cycle that
|
||||||
|
/// deadlocked) and mesh Phase 6's <c>IClusterRoleInfo</c> boot <c>StackOverflow</c>. The
|
||||||
|
/// rule it keeps re-teaching: inside an Akka builder, resolve nothing you can defer.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
public static Props Props(
|
public static Props Props(
|
||||||
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
|
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
|
||||||
MeshTransportOptions options,
|
MeshTransportOptions options,
|
||||||
IMeshClusterClientFactory clientFactory,
|
IMeshClusterClientFactory clientFactory,
|
||||||
Func<IActorRef?> coordinator,
|
Func<IActorRef?> coordinator,
|
||||||
IMeshCommandDialer? dialer = null,
|
Func<IMeshCommandDialer?>? dialer = null,
|
||||||
IMeshPskProvider? pskProvider = null) =>
|
Func<IMeshPskProvider?>? pskProvider = null) =>
|
||||||
Akka.Actor.Props.Create(() =>
|
Akka.Actor.Props.Create(() =>
|
||||||
new CentralCommunicationActor(
|
new CentralCommunicationActor(
|
||||||
dbFactory, options, clientFactory, coordinator, dialer, pskProvider));
|
dbFactory, options, clientFactory, coordinator, dialer, pskProvider));
|
||||||
@@ -127,22 +155,22 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
|
|||||||
/// <param name="options">The bound mesh-transport options.</param>
|
/// <param name="options">The bound mesh-transport options.</param>
|
||||||
/// <param name="clientFactory">Creates the ClusterClient.</param>
|
/// <param name="clientFactory">Creates the ClusterClient.</param>
|
||||||
/// <param name="coordinator">Lazily resolves the deploy-coordinator singleton proxy.</param>
|
/// <param name="coordinator">Lazily resolves the deploy-coordinator singleton proxy.</param>
|
||||||
/// <param name="dialer">Places one gRPC command call on one node; <c>Grpc</c> mode only.</param>
|
/// <param name="dialer">Resolves the gRPC command dialer; invoked in <see cref="PreStart"/>.</param>
|
||||||
/// <param name="pskProvider">Central's per-cluster key lookup; <c>Grpc</c> mode only.</param>
|
/// <param name="pskProvider">Resolves the PSK provider; invoked in <see cref="PreStart"/>.</param>
|
||||||
public CentralCommunicationActor(
|
public CentralCommunicationActor(
|
||||||
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
|
IDbContextFactory<OtOpcUaConfigDbContext> dbFactory,
|
||||||
MeshTransportOptions options,
|
MeshTransportOptions options,
|
||||||
IMeshClusterClientFactory clientFactory,
|
IMeshClusterClientFactory clientFactory,
|
||||||
Func<IActorRef?> coordinator,
|
Func<IActorRef?> coordinator,
|
||||||
IMeshCommandDialer? dialer = null,
|
Func<IMeshCommandDialer?>? dialer = null,
|
||||||
IMeshPskProvider? pskProvider = null)
|
Func<IMeshPskProvider?>? pskProvider = null)
|
||||||
{
|
{
|
||||||
_dbFactory = dbFactory;
|
_dbFactory = dbFactory;
|
||||||
_options = options;
|
_options = options;
|
||||||
_clientFactory = clientFactory;
|
_clientFactory = clientFactory;
|
||||||
_coordinator = coordinator;
|
_coordinator = coordinator;
|
||||||
_dialer = dialer;
|
_dialerFactory = dialer;
|
||||||
_pskProvider = pskProvider;
|
_pskProviderFactory = pskProvider;
|
||||||
_useClusterClient = string.Equals(
|
_useClusterClient = string.Equals(
|
||||||
options.Mode, MeshTransportOptions.ModeClusterClient, StringComparison.OrdinalIgnoreCase);
|
options.Mode, MeshTransportOptions.ModeClusterClient, StringComparison.OrdinalIgnoreCase);
|
||||||
_useGrpc = string.Equals(
|
_useGrpc = string.Equals(
|
||||||
@@ -183,6 +211,12 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
|
|||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
protected override void PreStart()
|
protected override void PreStart()
|
||||||
{
|
{
|
||||||
|
// First point at which the host is fully built — see the Props remark. Re-runs on every
|
||||||
|
// actor restart, which re-reads the container rather than caching a stale reference; both
|
||||||
|
// dependencies are container singletons, so a restart gets the same instances back.
|
||||||
|
_dialer = Resolve(_dialerFactory, nameof(IMeshCommandDialer));
|
||||||
|
_pskProvider = Resolve(_pskProviderFactory, nameof(IMeshPskProvider));
|
||||||
|
|
||||||
if (!_useClusterClient && !_useGrpc)
|
if (!_useClusterClient && !_useGrpc)
|
||||||
{
|
{
|
||||||
_log.Info(
|
_log.Info(
|
||||||
@@ -229,6 +263,41 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
|
|||||||
TimeSpan.FromSeconds(_options.ContactRefreshSeconds));
|
TimeSpan.FromSeconds(_options.ContactRefreshSeconds));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Invokes a dependency factory, converting a resolution failure into a logged absence.
|
||||||
|
/// </summary>
|
||||||
|
/// <typeparam name="T">The dependency's type.</typeparam>
|
||||||
|
/// <param name="factory">The factory, or <see langword="null"/> when nothing was supplied.</param>
|
||||||
|
/// <param name="name">The dependency's name, for the log.</param>
|
||||||
|
/// <returns>The resolved dependency, or <see langword="null"/>.</returns>
|
||||||
|
/// <remarks>
|
||||||
|
/// Catching is the point. A throw out of <see cref="PreStart"/> is an actor failure, and the
|
||||||
|
/// supervisor's answer to that is to restart — which calls <see cref="PreStart"/> again, and
|
||||||
|
/// a broken DI graph is not a transient condition. That is an unbounded restart loop on the
|
||||||
|
/// node's command boundary, taking the transport-independent ack path down with it. Logging
|
||||||
|
/// and continuing leaves the ack path serving and hands the two null guards below a genuine
|
||||||
|
/// absence to report, which is the same outcome as never having been wired.
|
||||||
|
/// </remarks>
|
||||||
|
private T? Resolve<T>(Func<T?>? factory, string name)
|
||||||
|
where T : class
|
||||||
|
{
|
||||||
|
if (factory is null)
|
||||||
|
return null;
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
return factory();
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_log.Error(
|
||||||
|
ex,
|
||||||
|
"Failed to resolve {Dependency} for the mesh command boundary; continuing without "
|
||||||
|
+ "it. The guards below report what that costs", name);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// <inheritdoc />
|
/// <inheritdoc />
|
||||||
protected override void PostStop()
|
protected override void PostStop()
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -66,14 +66,20 @@ public static class ServiceCollectionExtensions
|
|||||||
var dbFactory = resolver.GetService<IDbContextFactory<OtOpcUaConfigDbContext>>();
|
var dbFactory = resolver.GetService<IDbContextFactory<OtOpcUaConfigDbContext>>();
|
||||||
var options = resolver.GetService<IOptions<MeshTransportOptions>>().Value;
|
var options = resolver.GetService<IOptions<MeshTransportOptions>>().Value;
|
||||||
|
|
||||||
|
// Captured, NOT resolved. Resolving the mesh dialer or PSK provider here made every
|
||||||
|
// central node fail to boot in every transport mode: their graph is dialer →
|
||||||
|
// IMeshPskProvider → ISecretResolver → AddZbSecrets → IAuditWriter → ActorAuditWriter,
|
||||||
|
// whose constructor calls registry.Get<AuditWriterActorKey>() — and WithSingleton
|
||||||
|
// <AuditWriterActorKey> is registered further down THIS builder, so the entry does not
|
||||||
|
// exist while this starter runs. The actor invokes these in PreStart instead, by which
|
||||||
|
// point the host is built. Same shape, same reason, as the coordinator Func below.
|
||||||
|
//
|
||||||
// Through IServiceProvider rather than the Akka resolver, deliberately: these two are
|
// Through IServiceProvider rather than the Akka resolver, deliberately: these two are
|
||||||
// OPTIONAL (Grpc mode only, and admin-role only within that), and GetService's
|
// OPTIONAL (Grpc mode only, and admin-role only within that), and GetService's
|
||||||
// null-for-absent contract is the thing being relied on. The Akka resolver's
|
// null-for-absent contract is the thing being relied on. The Akka resolver's
|
||||||
// GetService<T> makes no such promise, and this same block runs in test harnesses whose
|
// GetService<T> makes no such promise, and this same block runs in test harnesses whose
|
||||||
// service collections register neither.
|
// service collections register neither.
|
||||||
var serviceProvider = resolver.GetService<IServiceProvider>();
|
var serviceProvider = resolver.GetService<IServiceProvider>();
|
||||||
var dialer = serviceProvider.GetService<IMeshCommandDialer>();
|
|
||||||
var pskProvider = serviceProvider.GetService<IMeshPskProvider>();
|
|
||||||
|
|
||||||
var actor = system.ActorOf(
|
var actor = system.ActorOf(
|
||||||
CentralCommunicationActor.Props(
|
CentralCommunicationActor.Props(
|
||||||
@@ -85,8 +91,8 @@ public static class ServiceCollectionExtensions
|
|||||||
() => registry.TryGet<ConfigPublishCoordinatorKey>(out var coordinator)
|
() => registry.TryGet<ConfigPublishCoordinatorKey>(out var coordinator)
|
||||||
? coordinator
|
? coordinator
|
||||||
: null,
|
: null,
|
||||||
dialer,
|
() => serviceProvider.GetService<IMeshCommandDialer>(),
|
||||||
pskProvider),
|
() => serviceProvider.GetService<IMeshPskProvider>()),
|
||||||
MeshPaths.CentralCommunicationName);
|
MeshPaths.CentralCommunicationName);
|
||||||
|
|
||||||
// Registered unconditionally, in both transport modes. Under Dps nothing dials it, and
|
// Registered unconditionally, in both transport modes. Under Dps nothing dials it, and
|
||||||
|
|||||||
+4
-4
@@ -742,8 +742,8 @@ public class CentralCommunicationActorTests : ControlPlaneActorTestBase
|
|||||||
new MeshTransportOptions { Mode = mode, ContactRefreshSeconds = 60 },
|
new MeshTransportOptions { Mode = mode, ContactRefreshSeconds = 60 },
|
||||||
clients,
|
clients,
|
||||||
() => null,
|
() => null,
|
||||||
new RecordingDialer(),
|
() => new RecordingDialer(),
|
||||||
psk));
|
() => psk));
|
||||||
|
|
||||||
AwaitCondition(
|
AwaitCondition(
|
||||||
() => psk.KnownClusterPublications.Count >= 1, TimeSpan.FromSeconds(5));
|
() => psk.KnownClusterPublications.Count >= 1, TimeSpan.FromSeconds(5));
|
||||||
@@ -855,8 +855,8 @@ public class CentralCommunicationActorTests : ControlPlaneActorTestBase
|
|||||||
GrpcMode(),
|
GrpcMode(),
|
||||||
clientFactory ?? new RecordingClientFactory(() => CreateTestProbe()),
|
clientFactory ?? new RecordingClientFactory(() => CreateTestProbe()),
|
||||||
() => coordinator,
|
() => coordinator,
|
||||||
dialer,
|
() => dialer,
|
||||||
psk));
|
() => psk));
|
||||||
return new GrpcRig(actor, dialer, psk);
|
return new GrpcRig(actor, dialer, psk);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+233
@@ -0,0 +1,233 @@
|
|||||||
|
using System.Diagnostics.CodeAnalysis;
|
||||||
|
using Akka.Actor;
|
||||||
|
using Akka.Cluster.Hosting;
|
||||||
|
using Akka.Hosting;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.Extensions.DependencyInjection;
|
||||||
|
using Microsoft.Extensions.Hosting;
|
||||||
|
using Microsoft.Extensions.Options;
|
||||||
|
using Shouldly;
|
||||||
|
using Xunit;
|
||||||
|
using ZB.MOM.WW.Audit;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Cluster;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Commons.Interfaces;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Commons.Types;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.Configuration;
|
||||||
|
using ZB.MOM.WW.OtOpcUa.ControlPlane.Communication;
|
||||||
|
using ZB.MOM.WW.Secrets.Abstractions;
|
||||||
|
|
||||||
|
namespace ZB.MOM.WW.OtOpcUa.ControlPlane.Tests.Communication;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Guards the one thing every other test in this project structurally cannot see: whether an
|
||||||
|
/// admin node <b>boots</b> with the real registration graph behind the mesh command boundary.
|
||||||
|
/// </summary>
|
||||||
|
/// <remarks>
|
||||||
|
/// <para>
|
||||||
|
/// The Phase 1A live gate failed here. <c>WithOtOpcUaControlPlaneSingletons</c>'s first
|
||||||
|
/// starter resolved <c>IMeshCommandDialer</c> and <c>IMeshPskProvider</c> eagerly, and their
|
||||||
|
/// graph reaches back into the actor registry — dialer → <c>IMeshPskProvider</c> →
|
||||||
|
/// <c>ISecretResolver</c> → <c>AddZbSecrets</c> → <c>IAuditWriter</c> →
|
||||||
|
/// <c>ActorAuditWriter</c>, whose constructor calls
|
||||||
|
/// <c>registry.Get<AuditWriterActorKey>()</c> for a singleton registered further down
|
||||||
|
/// the SAME builder. Every central node died on
|
||||||
|
/// <c>MissingActorRegistryEntryException</c>, in every transport mode, before serving
|
||||||
|
/// anything.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// <b>Why the existing tests were blind.</b> <c>MeshGrpcCommandDialerTests</c> activates the
|
||||||
|
/// dialer from a <c>ServiceCollection</c> that has no secrets registration at all, so its
|
||||||
|
/// graph stops one hop in; the <c>CentralCommunicationActorTests</c> facts inject fakes
|
||||||
|
/// directly and never touch DI; and the existing whole-host boots register no mesh provider,
|
||||||
|
/// so the eager lookups returned <see langword="null"/> and resolved nothing. Every one of
|
||||||
|
/// them passed while no central node could start.
|
||||||
|
/// </para>
|
||||||
|
/// <para>
|
||||||
|
/// <b>So the assertion is <c>StartAsync</c> itself.</b> The failure happens while the Akka
|
||||||
|
/// builder's starters execute, which is inside <c>StartAsync</c> — building the
|
||||||
|
/// <c>ServiceProvider</c> and resolving from it by hand would not reproduce it. The graph
|
||||||
|
/// below is the production one from <c>IMeshCommandDialer</c> down to the registry-bound
|
||||||
|
/// audit writer; only the secret store itself is a stand-in, because a real one would add a
|
||||||
|
/// KEK and a SQLite file to a test whose subject is starter ORDER.
|
||||||
|
/// </para>
|
||||||
|
/// </remarks>
|
||||||
|
public sealed class ControlPlaneStarterOrderTests
|
||||||
|
{
|
||||||
|
[Theory]
|
||||||
|
[InlineData(MeshTransportOptions.ModeDps)]
|
||||||
|
[InlineData(MeshTransportOptions.ModeClusterClient)]
|
||||||
|
[InlineData(MeshTransportOptions.ModeGrpc)]
|
||||||
|
public async Task An_admin_node_boots_with_the_real_mesh_dependency_graph(string mode)
|
||||||
|
{
|
||||||
|
// Every mode, because the eager resolve was mode-independent: the starter resolved both
|
||||||
|
// dependencies before anything looked at MeshTransport:Mode, so Dps — the DEFAULT, and the
|
||||||
|
// rollback target — died exactly like Grpc did.
|
||||||
|
using var host = BuildAdminHost(mode);
|
||||||
|
|
||||||
|
await Should.NotThrowAsync(() => host.StartAsync());
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var registry = host.Services.GetRequiredService<ActorRegistry>();
|
||||||
|
registry.TryGet<CentralCommunicationActorKey>(out _).ShouldBeTrue(
|
||||||
|
"the command boundary must be registered on a booted admin node");
|
||||||
|
registry.TryGet<AuditWriterActorKey>(out _).ShouldBeTrue(
|
||||||
|
"the audit singleton the eager resolve reached for too early must exist by now");
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
await host.StopAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task The_deferred_dependencies_are_resolved_and_used_once_the_host_is_up()
|
||||||
|
{
|
||||||
|
// Deferring must not become "never resolved" — that would trade a loud boot failure for a
|
||||||
|
// silent one, with the known-cluster gate permanently unpublished and the dialer permanently
|
||||||
|
// absent. So this asserts the far end of the deferral: the actor resolved the PSK provider
|
||||||
|
// FROM THE CONTAINER and drove it, through the ordinary contact-refresh path.
|
||||||
|
//
|
||||||
|
// The signal is a TaskCompletionSource rather than a counter, because PreStart runs on the
|
||||||
|
// actor's own thread and the contact load is a piped DB read: there is no instant after
|
||||||
|
// StartAsync at which a synchronous assertion is correct.
|
||||||
|
var signal = new SignallingPskProvider();
|
||||||
|
using var host = BuildAdminHost(MeshTransportOptions.ModeGrpc, signal);
|
||||||
|
|
||||||
|
await host.StartAsync();
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await signal.Published.WaitAsync(TimeSpan.FromSeconds(20));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
await host.StopAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds an admin host through the production cluster bootstrap and the production control-plane
|
||||||
|
/// starter registration, with the mesh dependency graph wired as it is on a real central node.
|
||||||
|
/// </summary>
|
||||||
|
/// <param name="mode">The mesh transport mode to boot under.</param>
|
||||||
|
/// <param name="pskProvider">
|
||||||
|
/// Substitute for the real <see cref="MeshPskProvider"/>. Only the resolution test supplies
|
||||||
|
/// one; the boot test deliberately keeps the production type so the graph it walks is the
|
||||||
|
/// production graph.
|
||||||
|
/// </param>
|
||||||
|
/// <returns>The unstarted host.</returns>
|
||||||
|
private static IHost BuildAdminHost(string mode, IMeshPskProvider? pskProvider = null)
|
||||||
|
{
|
||||||
|
var clusterOptions = new AkkaClusterOptions
|
||||||
|
{
|
||||||
|
Port = 0,
|
||||||
|
Hostname = "127.0.0.1",
|
||||||
|
PublicHostname = "127.0.0.1",
|
||||||
|
SeedNodes = [],
|
||||||
|
Roles = ["admin"],
|
||||||
|
};
|
||||||
|
|
||||||
|
return Host.CreateDefaultBuilder()
|
||||||
|
.ConfigureServices(services =>
|
||||||
|
{
|
||||||
|
services.AddSingleton<IOptions<AkkaClusterOptions>>(Options.Create(clusterOptions));
|
||||||
|
services.AddSingleton<IOptions<MeshTransportOptions>>(Options.Create(
|
||||||
|
new MeshTransportOptions { Mode = mode, DispatchTimeoutSeconds = 10 }));
|
||||||
|
services.AddDbContextFactory<OtOpcUaConfigDbContext>(
|
||||||
|
o => o.UseInMemoryDatabase($"starter-order-{Guid.NewGuid():N}"));
|
||||||
|
services.AddSingleton<IClusterRoleInfo>(new StubClusterRoleInfo());
|
||||||
|
|
||||||
|
// THE GRAPH. Production types from the dialer down to the registry-bound writer; the
|
||||||
|
// order of these registrations is deliberately irrelevant, because the defect was
|
||||||
|
// never about registration order — it was about WHEN the graph is first walked.
|
||||||
|
services.AddSingleton<IAuditWriter, RegistryBoundAuditWriter>();
|
||||||
|
services.AddSingleton<ISecretResolver, AuditingSecretResolver>();
|
||||||
|
if (pskProvider is null)
|
||||||
|
services.AddSingleton<IMeshPskProvider, MeshPskProvider>();
|
||||||
|
else
|
||||||
|
services.AddSingleton(pskProvider);
|
||||||
|
services.AddSingleton<IMeshCommandDialer, MeshGrpcCommandDialer>();
|
||||||
|
|
||||||
|
services.AddAkka("otopcua-starter-order-test", (ab, sp) =>
|
||||||
|
{
|
||||||
|
ab.WithOtOpcUaClusterBootstrap(sp);
|
||||||
|
ab.WithOtOpcUaControlPlaneSingletons();
|
||||||
|
});
|
||||||
|
})
|
||||||
|
.Build();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Stands in for <c>ActorAuditWriter</c>: an <see cref="IAuditWriter"/> that resolves a
|
||||||
|
/// cluster singleton from the <see cref="ActorRegistry"/> <b>in its constructor</b>. That
|
||||||
|
/// constructor is the tripwire — it throws
|
||||||
|
/// <c>MissingActorRegistryEntryException</c> for exactly as long as the audit singleton has
|
||||||
|
/// not been registered yet.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class RegistryBoundAuditWriter : IAuditWriter
|
||||||
|
{
|
||||||
|
private readonly IActorRef _proxy;
|
||||||
|
|
||||||
|
public RegistryBoundAuditWriter(ActorRegistry registry) => _proxy = registry.Get<AuditWriterActorKey>();
|
||||||
|
|
||||||
|
public Task WriteAsync(AuditEvent evt, CancellationToken ct = default)
|
||||||
|
{
|
||||||
|
_proxy.Tell(evt);
|
||||||
|
return Task.CompletedTask;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// An <see cref="IMeshPskProvider"/> that completes a task the first time the actor publishes
|
||||||
|
/// the known-cluster set. Takes <see cref="ISecretResolver"/> it never uses, purely so that
|
||||||
|
/// resolving it still walks the same registry-bound chain the real provider does.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class SignallingPskProvider : IMeshPskProvider
|
||||||
|
{
|
||||||
|
private readonly TaskCompletionSource _published =
|
||||||
|
new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
|
|
||||||
|
/// <summary>Completes once <see cref="SetKnownClusters"/> has been called.</summary>
|
||||||
|
public Task Published => _published.Task;
|
||||||
|
|
||||||
|
public bool TryGetPsk(string clusterId, [NotNullWhen(true)] out string? psk)
|
||||||
|
{
|
||||||
|
psk = null;
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
public bool Matches(string clusterId, string presented) => false;
|
||||||
|
|
||||||
|
public void SetKnownClusters(IReadOnlyCollection<string> clusterIds) => _published.TrySetResult();
|
||||||
|
|
||||||
|
public void Invalidate(string clusterId) { }
|
||||||
|
|
||||||
|
public void InvalidateAll() { }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Stands in for the shared secret store's resolver. Its only load-bearing property is the
|
||||||
|
/// one that broke boot: it depends on <see cref="IAuditWriter"/>, exactly as
|
||||||
|
/// <c>AddZbSecrets</c>' resolver registration does.
|
||||||
|
/// </summary>
|
||||||
|
private sealed class AuditingSecretResolver : ISecretResolver
|
||||||
|
{
|
||||||
|
public AuditingSecretResolver(IAuditWriter auditWriter) => ArgumentNullException.ThrowIfNull(auditWriter);
|
||||||
|
|
||||||
|
public Task<string?> GetAsync(SecretName name, CancellationToken ct) => Task.FromResult<string?>(null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Minimal <see cref="IClusterRoleInfo"/>; the admin singleton factories read it.</summary>
|
||||||
|
private sealed class StubClusterRoleInfo : IClusterRoleInfo
|
||||||
|
{
|
||||||
|
public string? ClusterRole => null;
|
||||||
|
public string? ClusterId => null;
|
||||||
|
public NodeId LocalNode => NodeId.Parse("127.0.0.1:0");
|
||||||
|
public IReadOnlySet<string> LocalRoles => new HashSet<string>();
|
||||||
|
public bool HasRole(string role) => false;
|
||||||
|
public IReadOnlyList<NodeId> MembersWithRole(string role) => [];
|
||||||
|
public NodeId? RoleLeader(string role) => null;
|
||||||
|
public event EventHandler<RoleLeaderChangedEventArgs>? RoleLeaderChanged { add { } remove { } }
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user