fix(archreview #10): wire CapabilityInvoker into prod dispatch via a Polly-free seam
Remediates RESILIENCE-DISPATCH-GAP (surfaced by the 07/C-1 analyzer): the Phase 6.1 CapabilityInvoker resilience pipeline (retry / breaker / bulkhead / telemetry) was constructed only in tests — the production dispatch layer called driver-capability methods directly, bypassing it entirely. The filed plan (thread CapabilityInvoker directly into DriverInstanceActor) is INFEASIBLE: Runtime is deliberately Polly-free (references Core.Abstractions, not Core — the same boundary IDriverFactory documents). So this introduces a seam, mirroring IDriverFactory exactly: - Core.Abstractions: IDriverCapabilityInvoker (+ NullDriverCapabilityInvoker pass-through) and IDriverCapabilityInvokerFactory (+ null factory). - Core: CapabilityInvoker now implements the interface; new DriverCapabilityInvokerFactory builds a per-instance invoker over the process-singleton pipeline builder + status tracker + tier resolver. - Runtime: DriverInstanceActor takes an IDriverCapabilityInvoker (default = pass-through) and routes all 6 dispatch sites (write, alarm-ack, subscribe, unsubscribe, alarm-subscribe, discover) through it; per-host key resolved via IPerCallHostResolver for single-ref calls, driver-instance key for bulk calls. DriverHostActor builds + injects the real invoker per spawned driver. - Host: DriverFactoryBootstrap registers the tracker, pipeline builder, and concrete factory (needs DriverFactoryRegistry.GetTier). Runtime SCE resolves the factory from DI like IDriverFactory; pass-through on nodes without it bound. - Analyzer: IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync are now recognized wrapper homes (interface-typed invoker calls must suppress OTOPCUA0001 too). All 6 RESILIENCE-DISPATCH-GAP pragmas removed — the analyzer (an error in Runtime) is now the standing regression guard. Also adds retry / breaker-open / breaker-close LOGGING to the pipeline builder — the operator-facing observability surface (the Admin /hosts reader, Phase 6.1 Stream E.2/E.3, was never built, so the pipeline otherwise runs silently). Scope notes: - Tier-DEFAULT policy only: DriverInstanceSpec (the deploy artifact) does not carry the per-instance ResilienceConfig JSON, so overrides aren't applied yet (tracked follow-up: plumb ResilienceConfig through the composer/artifact). - GenericDriverNodeManager's call is NOT wired: that class is test-only scaffolding (zero production references — the Server has its own address-space path); its pragma is re-annotated accordingly, not left as a "tracked gap". Verification (unit + analyzer; live behavioral gate still pending — see FOLLOWUP-10): - Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001. - New Runtime guard (recording invoker): write routes via ExecuteWriteAsync with the IPerCallHostResolver host; subscribe via ExecuteAsync — proves runtime routing. - New analyzer test: the interface is a valid wrapper home. - New pipeline-builder test: retry events are logged. - Full solution builds clean (0 errors); Runtime.Tests 357, Core.Tests 238, Analyzers.Tests 32 all green; pass-through default keeps existing dispatch tests byte-for-byte unchanged.
This commit is contained in:
Joseph Doherty
@@ -137,6 +137,15 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
public static readonly TimeSpan HealthPollInterval = TimeSpan.FromSeconds(30);
|
||||
|
||||
private readonly IDriver _driver;
|
||||
|
||||
/// <summary>Phase 6.1 resilience seam: every guarded driver-capability call this actor makes is
|
||||
/// routed through this invoker (retry / breaker / bulkhead / tracker telemetry). Defaults to
|
||||
/// <see cref="NullDriverCapabilityInvoker"/> (a genuine pass-through) when none is supplied — so
|
||||
/// unit tests + the F7 skeleton path behave exactly as a raw driver call. The fused Host builds a
|
||||
/// real per-instance invoker via <see cref="IDriverCapabilityInvokerFactory"/> and injects it at
|
||||
/// spawn. Consumed through the <see cref="IDriverCapabilityInvoker"/> abstraction because Runtime
|
||||
/// is deliberately Polly-free (it does not reference <c>ZB.MOM.WW.OtOpcUa.Core</c>).</summary>
|
||||
private readonly IDriverCapabilityInvoker _invoker;
|
||||
private readonly string _driverInstanceId;
|
||||
private readonly string _clusterId;
|
||||
private readonly IDriverHealthPublisher _healthPublisher;
|
||||
@@ -212,6 +221,8 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
/// <param name="rediscoverInterval">Optional interval between post-connect re-discovery passes; defaults to 2 seconds.</param>
|
||||
/// <param name="rediscoverMaxAttempts">Optional cap on re-discovery passes; defaults to 15.</param>
|
||||
/// <param name="rediscoverDiscoverTimeout">Optional per-pass timeout for <see cref="ITagDiscovery.DiscoverAsync"/>; defaults to 30 seconds.</param>
|
||||
/// <param name="invoker">Optional Phase 6.1 resilience invoker wrapping this driver's capability
|
||||
/// calls; defaults to <see cref="NullDriverCapabilityInvoker"/> (pass-through) when not supplied.</param>
|
||||
/// <returns>A <see cref="Akka.Actor.Props"/> instance configured to create the wrapped <see cref="DriverInstanceActor"/>.</returns>
|
||||
public static Props Props(
|
||||
IDriver driver,
|
||||
@@ -221,7 +232,8 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
string? clusterId = null,
|
||||
TimeSpan? rediscoverInterval = null,
|
||||
int rediscoverMaxAttempts = DefaultRediscoverMaxAttempts,
|
||||
TimeSpan? rediscoverDiscoverTimeout = null) =>
|
||||
TimeSpan? rediscoverDiscoverTimeout = null,
|
||||
IDriverCapabilityInvoker? invoker = null) =>
|
||||
Akka.Actor.Props.Create(() => new DriverInstanceActor(
|
||||
driver,
|
||||
reconnectInterval ?? DefaultReconnectInterval,
|
||||
@@ -230,7 +242,8 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
clusterId ?? string.Empty,
|
||||
rediscoverInterval,
|
||||
rediscoverMaxAttempts,
|
||||
rediscoverDiscoverTimeout));
|
||||
rediscoverDiscoverTimeout,
|
||||
invoker));
|
||||
|
||||
/// <summary>
|
||||
/// Returns true when the driver should boot in DEV-STUB mode based on host platform and
|
||||
@@ -262,6 +275,8 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
/// <param name="rediscoverInterval">Interval between post-connect re-discovery passes; defaults to 2 seconds.</param>
|
||||
/// <param name="rediscoverMaxAttempts">Cap on the number of re-discovery passes; defaults to 15.</param>
|
||||
/// <param name="rediscoverDiscoverTimeout">Per-pass timeout for <see cref="ITagDiscovery.DiscoverAsync"/>; defaults to 30 seconds.</param>
|
||||
/// <param name="invoker">Phase 6.1 resilience invoker wrapping this driver's capability calls;
|
||||
/// defaults to <see cref="NullDriverCapabilityInvoker"/> (pass-through) when null.</param>
|
||||
public DriverInstanceActor(
|
||||
IDriver driver,
|
||||
TimeSpan reconnectInterval,
|
||||
@@ -270,9 +285,11 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
string? clusterId = null,
|
||||
TimeSpan? rediscoverInterval = null,
|
||||
int rediscoverMaxAttempts = DefaultRediscoverMaxAttempts,
|
||||
TimeSpan? rediscoverDiscoverTimeout = null)
|
||||
TimeSpan? rediscoverDiscoverTimeout = null,
|
||||
IDriverCapabilityInvoker? invoker = null)
|
||||
{
|
||||
_driver = driver;
|
||||
_invoker = invoker ?? NullDriverCapabilityInvoker.Instance;
|
||||
_driverInstanceId = driver.DriverInstanceId;
|
||||
_clusterId = clusterId ?? string.Empty;
|
||||
_healthPublisher = healthPublisher ?? NullDriverHealthPublisher.Instance;
|
||||
@@ -572,9 +589,16 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
|
||||
try
|
||||
{
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
var results = await writable.WriteAsync(request, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
// Route through the resilience invoker (Write pipeline: timeout, no retry by default,
|
||||
// per-host breaker). An OPC UA attribute set is idempotent (writing value X twice == once),
|
||||
// so a configured Write retry is safe to apply; the Write tier default is RetryCount=0, so
|
||||
// this stays behavior-neutral until an operator opts in via ResilienceConfig. The outer cts
|
||||
// is retained as a hard backstop above the (typically tighter) tier timeout.
|
||||
var results = await _invoker.ExecuteWriteAsync(
|
||||
ResolveHost(msg.TagId),
|
||||
isIdempotent: true,
|
||||
async ct => await writable.WriteAsync(request, ct),
|
||||
cts.Token);
|
||||
if (results is { Count: 1 } && IsGoodStatus(results[0].StatusCode))
|
||||
{
|
||||
replyTo.Tell(new WriteAttributeResult(true, null));
|
||||
@@ -622,9 +646,12 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
|
||||
try
|
||||
{
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
await src.AcknowledgeAsync(request, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
// Ack is a write-shaped operation — the AlarmAcknowledge tier policy never retries.
|
||||
await _invoker.ExecuteAsync(
|
||||
DriverCapability.AlarmAcknowledge,
|
||||
ResolveHost(msg.ConditionId),
|
||||
async ct => await src.AcknowledgeAsync(request, ct),
|
||||
cts.Token);
|
||||
_log.Info("DriverInstance {Id}: acknowledged native condition {Cond} by {User}",
|
||||
_driverInstanceId, msg.ConditionId, msg.OperatorUser);
|
||||
}
|
||||
@@ -661,10 +688,13 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
subscribable.OnDataChange += _dataChangeHandler;
|
||||
|
||||
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
_subscriptionHandle = await subscribable
|
||||
.SubscribeAsync(msg.FullReferences, msg.PublishingInterval, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
// A bulk subscribe spans potentially many hosts, so key the pipeline on the driver
|
||||
// instance (single-host fallback) rather than any one reference's host.
|
||||
_subscriptionHandle = await _invoker.ExecuteAsync(
|
||||
DriverCapability.Subscribe,
|
||||
_driverInstanceId,
|
||||
async ct => await subscribable.SubscribeAsync(msg.FullReferences, msg.PublishingInterval, ct),
|
||||
cts.Token);
|
||||
|
||||
replyTo.Tell(new SubscriptionEstablished(_subscriptionHandle.DiagnosticId, msg.FullReferences.Count));
|
||||
_log.Info("DriverInstance {Id}: subscribed to {Count} refs ({Diag})",
|
||||
@@ -688,9 +718,12 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
try
|
||||
{
|
||||
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
await subscribable.UnsubscribeAsync(_subscriptionHandle, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
var handle = _subscriptionHandle; // non-null per the guard above; capture before the async lambda
|
||||
await _invoker.ExecuteAsync(
|
||||
DriverCapability.Subscribe,
|
||||
_driverInstanceId,
|
||||
async ct => await subscribable.UnsubscribeAsync(handle, ct),
|
||||
cts.Token);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
@@ -767,9 +800,11 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
|
||||
try
|
||||
{
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
_alarmSubscriptionHandle = await src.SubscribeAlarmsAsync(refs, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
_alarmSubscriptionHandle = await _invoker.ExecuteAsync(
|
||||
DriverCapability.AlarmSubscribe,
|
||||
_driverInstanceId,
|
||||
async ct => await src.SubscribeAlarmsAsync(refs, ct),
|
||||
cts.Token);
|
||||
_log.Info("DriverInstance {Id}: native-alarm subscription established for {Count} alarm ref(s) ({Diag})",
|
||||
_driverInstanceId, refs.Count, _alarmSubscriptionHandle.DiagnosticId);
|
||||
}
|
||||
@@ -824,13 +859,17 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
// Bound the browse — ReceiveAsync suspends the mailbox for the whole handler, so an unbounded
|
||||
// DiscoverAsync would block DisconnectObserved / ForceReconnect / writes / health-poll behind it.
|
||||
using var cts = new CancellationTokenSource(_rediscoverDiscoverTimeout);
|
||||
// NO ConfigureAwait(false): a genuinely-async DiscoverAsync (Galaxy / OpcUaClient / TwinCAT) must
|
||||
// resume on the actor task scheduler so the Context.Parent.Tell + Timers calls below run with a
|
||||
// live ActorContext. ConfigureAwait(false) would resume off-context and throw
|
||||
// NotSupportedException("no active ActorContext") — see the same warning on HandleSubscribeAsync.
|
||||
#pragma warning disable OTOPCUA0001 // RESILIENCE-DISPATCH-GAP (07/C-1): dispatch bypasses CapabilityInvoker — Phase 6.1 pipeline built but never wired into DriverInstanceActor; tracked, NOT intentional.
|
||||
await discovery.DiscoverAsync(builder, cts.Token);
|
||||
#pragma warning restore OTOPCUA0001
|
||||
// NO ConfigureAwait(false) on this outer await: a genuinely-async DiscoverAsync (Galaxy /
|
||||
// OpcUaClient / TwinCAT) must resume on the actor task scheduler so the Context.Parent.Tell +
|
||||
// Timers calls below run with a live ActorContext. ConfigureAwait(false) would resume
|
||||
// off-context and throw NotSupportedException("no active ActorContext"). The invoker's own
|
||||
// internal ConfigureAwait(false) does NOT propagate to this caller's await — the actor
|
||||
// continuation still resumes on the captured actor scheduler. (Discover retries per tier.)
|
||||
await _invoker.ExecuteAsync(
|
||||
DriverCapability.Discover,
|
||||
_driverInstanceId,
|
||||
async ct => await discovery.DiscoverAsync(builder, ct),
|
||||
cts.Token);
|
||||
nodes = builder.Nodes.ToArray(); // immutable snapshot — never hand the builder's live list across actors
|
||||
}
|
||||
catch (Exception ex)
|
||||
@@ -910,6 +949,14 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
|
||||
|
||||
private static bool IsGoodStatus(uint statusCode) => (statusCode >> 30) == 0;
|
||||
|
||||
/// <summary>Resolve the resilience-pipeline host key for a single driver-side reference. Multi-host
|
||||
/// drivers (e.g. Modbus across N PLCs) implement <see cref="IPerCallHostResolver"/> so one dead host
|
||||
/// trips only its own breaker; single-host drivers fall back to the driver-instance id. Used for the
|
||||
/// single-reference capability calls (write, alarm ack); bulk/whole-driver calls key on the instance
|
||||
/// id directly.</summary>
|
||||
private string ResolveHost(string reference) =>
|
||||
_driver is IPerCallHostResolver resolver ? resolver.ResolveHost(reference) : _driverInstanceId;
|
||||
|
||||
/// <summary>Records a transition into a Faulted / error state for the 5-minute sliding counter.</summary>
|
||||
private void RecordFault()
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user