bacea1a44f
Remediates RESILIENCE-DISPATCH-GAP (surfaced by the 07/C-1 analyzer): the Phase 6.1 CapabilityInvoker resilience pipeline (retry / breaker / bulkhead / telemetry) was constructed only in tests — the production dispatch layer called driver-capability methods directly, bypassing it entirely. The filed plan (thread CapabilityInvoker directly into DriverInstanceActor) is INFEASIBLE: Runtime is deliberately Polly-free (references Core.Abstractions, not Core — the same boundary IDriverFactory documents). So this introduces a seam, mirroring IDriverFactory exactly: - Core.Abstractions: IDriverCapabilityInvoker (+ NullDriverCapabilityInvoker pass-through) and IDriverCapabilityInvokerFactory (+ null factory). - Core: CapabilityInvoker now implements the interface; new DriverCapabilityInvokerFactory builds a per-instance invoker over the process-singleton pipeline builder + status tracker + tier resolver. - Runtime: DriverInstanceActor takes an IDriverCapabilityInvoker (default = pass-through) and routes all 6 dispatch sites (write, alarm-ack, subscribe, unsubscribe, alarm-subscribe, discover) through it; per-host key resolved via IPerCallHostResolver for single-ref calls, driver-instance key for bulk calls. DriverHostActor builds + injects the real invoker per spawned driver. - Host: DriverFactoryBootstrap registers the tracker, pipeline builder, and concrete factory (needs DriverFactoryRegistry.GetTier). Runtime SCE resolves the factory from DI like IDriverFactory; pass-through on nodes without it bound. - Analyzer: IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync are now recognized wrapper homes (interface-typed invoker calls must suppress OTOPCUA0001 too). All 6 RESILIENCE-DISPATCH-GAP pragmas removed — the analyzer (an error in Runtime) is now the standing regression guard. Also adds retry / breaker-open / breaker-close LOGGING to the pipeline builder — the operator-facing observability surface (the Admin /hosts reader, Phase 6.1 Stream E.2/E.3, was never built, so the pipeline otherwise runs silently). Scope notes: - Tier-DEFAULT policy only: DriverInstanceSpec (the deploy artifact) does not carry the per-instance ResilienceConfig JSON, so overrides aren't applied yet (tracked follow-up: plumb ResilienceConfig through the composer/artifact). - GenericDriverNodeManager's call is NOT wired: that class is test-only scaffolding (zero production references — the Server has its own address-space path); its pragma is re-annotated accordingly, not left as a "tracked gap". Verification (unit + analyzer; live behavioral gate still pending — see FOLLOWUP-10): - Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001. - New Runtime guard (recording invoker): write routes via ExecuteWriteAsync with the IPerCallHostResolver host; subscribe via ExecuteAsync — proves runtime routing. - New analyzer test: the interface is a valid wrapper home. - New pipeline-builder test: retry events are logged. - Full solution builds clean (0 errors); Runtime.Tests 357, Core.Tests 238, Analyzers.Tests 32 all green; pass-through default keeps existing dispatch tests byte-for-byte unchanged.
139 lines
7.1 KiB
C#
139 lines
7.1 KiB
C#
using System.Collections.Concurrent;
|
|
using Akka.Actor;
|
|
using Shouldly;
|
|
using Xunit;
|
|
using ZB.MOM.WW.OtOpcUa.Commons.Types;
|
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
|
using ZB.MOM.WW.OtOpcUa.Runtime.Drivers;
|
|
using ZB.MOM.WW.OtOpcUa.Runtime.Tests.Harness;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Runtime.Tests.Drivers;
|
|
|
|
/// <summary>
|
|
/// Runtime guard for the Phase 6.1 resilience wiring (arch-review #10 / RESILIENCE-DISPATCH-GAP):
|
|
/// proves the <see cref="DriverInstanceActor"/> routes each driver-capability call through the
|
|
/// injected <see cref="IDriverCapabilityInvoker"/> at RUNTIME, with the correct
|
|
/// <see cref="DriverCapability"/> and resolved host key — not just that the calls compile wrapped
|
|
/// (the <c>OTOPCUA0001</c> analyzer is the compile-time guard; this is the behavioural one).
|
|
/// A recording invoker stands in for the real <c>CapabilityInvoker</c> so the test stays in the
|
|
/// Polly-free Runtime layer.
|
|
/// </summary>
|
|
public sealed class DriverInstanceActorResilienceWiringTests : RuntimeActorTestBase
|
|
{
|
|
/// <summary>
|
|
/// A bulk subscribe issued on connect (via <c>ResubscribeDesired</c>) routes through the invoker
|
|
/// with <see cref="DriverCapability.Subscribe"/> keyed on the driver-instance id (single-host
|
|
/// fallback for a set that may span hosts). The driver's <c>SubscribeAsync</c> only runs from
|
|
/// inside the invoker's call-site, so <c>SubscribeCount >= 1</c> already proves the route.
|
|
/// </summary>
|
|
[Fact]
|
|
public async Task Subscribe_routes_through_the_capability_invoker()
|
|
{
|
|
var invoker = new RecordingInvoker();
|
|
var driver = new SubscribableStubDriver();
|
|
var actor = Sys.ActorOf(DriverInstanceActor.Props(driver, invoker: invoker));
|
|
|
|
actor.Tell(new DriverInstanceActor.SetDesiredSubscriptions(
|
|
new[] { "tag-a", "tag-b" }, TimeSpan.FromMilliseconds(100)));
|
|
actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
|
|
AwaitCondition(() => driver.SubscribeCount >= 1, TimeSpan.FromSeconds(3));
|
|
|
|
// Barrier: round-trip a Connected-only Ask so the async subscribe handler has fully recorded.
|
|
await actor.Ask<DriverInstanceActor.SubscriptionEstablished>(
|
|
new DriverInstanceActor.Subscribe(new[] { "tag-a" }, TimeSpan.FromMilliseconds(100)),
|
|
TimeSpan.FromSeconds(3));
|
|
|
|
invoker.Calls.ShouldContain(c => c.Capability == DriverCapability.Subscribe && c.Host == driver.DriverInstanceId);
|
|
}
|
|
|
|
/// <summary>
|
|
/// A write routes through the invoker's <c>ExecuteWriteAsync</c> path (recorded as
|
|
/// <see cref="DriverCapability.Write"/>) with the host resolved via
|
|
/// <see cref="IPerCallHostResolver"/> — proving per-host breaker keying is wired, not the
|
|
/// single-host fallback. A write that arrives before <c>Connected</c> fast-fails WITHOUT
|
|
/// touching the invoker, so the loop drives to Connected and the recorded call is the real one.
|
|
/// </summary>
|
|
[Fact]
|
|
public async Task Write_routes_through_the_invoker_with_resolved_host()
|
|
{
|
|
var invoker = new RecordingInvoker();
|
|
var driver = new HostResolvingWritableDriver(); // ResolveHost("tag-1") => "plc-7"
|
|
var actor = Sys.ActorOf(DriverInstanceActor.Props(driver, invoker: invoker));
|
|
actor.Tell(new DriverInstanceActor.InitializeRequested("{}"));
|
|
|
|
// Retry until the write succeeds (i.e. the actor reached Connected). Connecting-state writes
|
|
// fast-fail on a synchronous path that never reaches the invoker, so they don't pollute Calls.
|
|
DriverInstanceActor.WriteAttributeResult? reply = null;
|
|
for (var i = 0; i < 60 && reply is not { Success: true }; i++)
|
|
{
|
|
reply = await actor.Ask<DriverInstanceActor.WriteAttributeResult>(
|
|
new DriverInstanceActor.WriteAttribute("tag-1", 42), TimeSpan.FromSeconds(1));
|
|
if (reply is not { Success: true }) await Task.Delay(25);
|
|
}
|
|
|
|
reply.ShouldNotBeNull();
|
|
reply!.Success.ShouldBeTrue();
|
|
driver.Writes.Count.ShouldBeGreaterThanOrEqualTo(1); // the invoker's call-site actually ran
|
|
invoker.Calls.ShouldContain(c => c.Capability == DriverCapability.Write && c.Host == "plc-7");
|
|
}
|
|
|
|
/// <summary>Records every capability + resolved-host key routed through it, then delegates to the
|
|
/// real call-site so the driver still runs. Stands in for <c>CapabilityInvoker</c> without pulling
|
|
/// Polly / Core into the Runtime test.</summary>
|
|
private sealed class RecordingInvoker : IDriverCapabilityInvoker
|
|
{
|
|
public ConcurrentQueue<(DriverCapability Capability, string Host)> Calls { get; } = new();
|
|
|
|
public ValueTask<TResult> ExecuteAsync<TResult>(
|
|
DriverCapability capability, string hostName,
|
|
Func<CancellationToken, ValueTask<TResult>> callSite, CancellationToken cancellationToken)
|
|
{
|
|
Calls.Enqueue((capability, hostName));
|
|
return callSite(cancellationToken);
|
|
}
|
|
|
|
public ValueTask ExecuteAsync(
|
|
DriverCapability capability, string hostName,
|
|
Func<CancellationToken, ValueTask> callSite, CancellationToken cancellationToken)
|
|
{
|
|
Calls.Enqueue((capability, hostName));
|
|
return callSite(cancellationToken);
|
|
}
|
|
|
|
public ValueTask<TResult> ExecuteWriteAsync<TResult>(
|
|
string hostName, bool isIdempotent,
|
|
Func<CancellationToken, ValueTask<TResult>> callSite, CancellationToken cancellationToken)
|
|
{
|
|
Calls.Enqueue((DriverCapability.Write, hostName));
|
|
return callSite(cancellationToken);
|
|
}
|
|
}
|
|
|
|
/// <summary>A writable driver that also resolves a per-tag host, so the write path exercises
|
|
/// <see cref="DriverInstanceActor"/>'s <c>ResolveHost</c> (via <see cref="IPerCallHostResolver"/>).</summary>
|
|
private sealed class HostResolvingWritableDriver : IDriver, IWritable, IPerCallHostResolver
|
|
{
|
|
public List<WriteRequest> Writes { get; } = new();
|
|
|
|
public string DriverInstanceId => "host-resolving-driver";
|
|
public string DriverType => "Stub";
|
|
|
|
public string ResolveHost(string fullReference) => "plc-7";
|
|
|
|
public Task InitializeAsync(string driverConfigJson, CancellationToken cancellationToken) => Task.CompletedTask;
|
|
public Task ReinitializeAsync(string driverConfigJson, CancellationToken cancellationToken) => Task.CompletedTask;
|
|
public Task ShutdownAsync(CancellationToken cancellationToken) => Task.CompletedTask;
|
|
public DriverHealth GetHealth() => new(DriverState.Healthy, DateTime.UtcNow, null);
|
|
public long GetMemoryFootprint() => 0;
|
|
public Task FlushOptionalCachesAsync(CancellationToken cancellationToken) => Task.CompletedTask;
|
|
|
|
public Task<IReadOnlyList<WriteResult>> WriteAsync(
|
|
IReadOnlyList<WriteRequest> writes, CancellationToken cancellationToken)
|
|
{
|
|
Writes.AddRange(writes);
|
|
IReadOnlyList<WriteResult> results = writes.Select(_ => new WriteResult(0u)).ToList();
|
|
return Task.FromResult(results);
|
|
}
|
|
}
|
|
}
|