26ab6f8f09
P2-3. ConnectResult was the last piped completion with no adapter generation on
it, so a connect launched against generation N and answered after the adapter had
been swapped (failover, or the row-97 in-place reconfigure) was applied to its
REPLACEMENT. A stale success drove ReSubscribeAll + BecomeConnected against an
adapter that had never connected, and the replacement's own result then landed in
the Connected state where nothing handles it. With a bad new endpoint the batch
subscribes failed at connection level with a log line only, no AdapterDisconnected
ever fired, and the idempotent-reconfigure guard refused a re-send of the same
configuration — only a restart recovered the connection. A stale FAILURE was as
wrong the other way: it counted toward failover and re-armed the reconnect timer
underneath the attempt genuinely in flight.
HandleAttemptConnect now stamps the generation and both handlers open with the
same guard the subscribe leg got in c4d3c8b8, dropping a mismatched result whole.
The adapter instance is deliberately NOT carried: unlike a subscribe completion a
connect result holds no adapter-scoped handles to release, and SwapAdapter has
already disposed the outgoing adapter. Dropping cannot wedge the machine — every
generation bump (CountFailureAndMaybeFailover, BecomeReconnecting's failover
branch, HandleReconfigure through it) arms an AttemptConnect for the new
generation, and the state stays Connecting/Reconnecting until that answers.
P2-4. CreateSubscriptionsCoreAsync registers each monitored item as it places it
(_callbacks, _itemShard, shard budget, the SDK subscription's item list) and only
then issues ApplyChanges, so a throw there propagated with every registration
still in place and nothing holding a handle to release them. The actor reads the
throw as a whole-chunk fault and re-issues the SAME batch on its backoff timer, so
each probe grew the bookkeeping again. Placement + apply moved to
PlaceAndApplyAsync behind a try/catch that unwinds via the existing
ReleaseFailedItem (per-item try, so one failure neither stops the rest nor
replaces the original fault); no ApplyChanges is re-issued, and items a partial
apply did reach the server with are reclaimed when the shard and session are torn
down on reconnect.
That reconnect is the other half: IsConnectionLevelFailure knew nothing about
ServiceResultException, which is how the SDK reports a dead session — so a dead
session classified as a per-tag resolution failure for the whole chunk. It now
also matches a narrow set of session/channel status codes (BadSessionIdInvalid,
BadSessionClosed, BadSessionNotActivated, BadSecureChannelClosed,
BadSecureChannelIdInvalid, BadServerNotConnected, BadConnectionClosed,
BadCommunicationError, BadNotConnected). Per-node codes such as BadNodeIdUnknown
stay per-tag — that is exactly the device-still-booting case the resolution timer
exists for.
Tests: DataConnectionStaleConnectResultTests (stale success neither connects nor
re-subscribes against the new adapter; stale failure does not disturb the current
attempt; the current generation's result is still processed after a stale drop) —
all 3 fail with the guards removed. DataConnectionOpcUaFaultClassificationTests
(bad-session drives reconnect — fails unfixed; bad-node-id stays per-tag).
RealOpcUaClientBatchRollbackTests (no residual registrations after an ApplyChanges
throw; repeated failed batches do not grow the bookkeeping) — both fail with the
rollback removed. Full DCL suite 342/342 green.
110 lines
5.1 KiB
C#
110 lines
5.1 KiB
C#
using Akka.Actor;
|
|
using Akka.TestKit.Xunit2;
|
|
using NSubstitute;
|
|
using Opc.Ua;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Protocol;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Messages.DataConnection;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
|
|
using ZB.MOM.WW.ScadaBridge.DataConnectionLayer.Actors;
|
|
using ZB.MOM.WW.ScadaBridge.HealthMonitoring;
|
|
|
|
namespace ZB.MOM.WW.ScadaBridge.DataConnectionLayer.Tests.Actors;
|
|
|
|
/// <summary>
|
|
/// Arch-review 2026-08-19 <b>P2-4</b> (classification half) — how a thrown
|
|
/// <see cref="ServiceResultException"/> out of a batch subscribe is classified.
|
|
///
|
|
/// <para>
|
|
/// <b>The gap these pin.</b> <c>IsConnectionLevelFailure</c> knew only about
|
|
/// <c>InvalidOperationException</c> / socket / timeout / IO faults, so the OPC UA SDK's own way of
|
|
/// reporting a dead session — a <c>ServiceResultException</c> carrying a bad-session status —
|
|
/// was treated as a per-tag resolution failure for the WHOLE chunk. Every tag went unresolved and
|
|
/// the backoff probe re-issued the same batch against the same dead session forever, growing the
|
|
/// adapter's monitored-item bookkeeping each round, instead of reconnecting once.
|
|
/// </para>
|
|
///
|
|
/// <para>
|
|
/// The classification must stay narrow: a per-node status (<c>BadNodeIdUnknown</c>) is exactly the
|
|
/// "device still booting / wrong tag path" case the resolution timer exists for, and must NOT put
|
|
/// the connection into a reconnect loop.
|
|
/// </para>
|
|
/// </summary>
|
|
public class DataConnectionOpcUaFaultClassificationTests : TestKit
|
|
{
|
|
private readonly ISiteHealthCollector _health = Substitute.For<ISiteHealthCollector>();
|
|
private readonly IDataConnectionFactory _factory = Substitute.For<IDataConnectionFactory>();
|
|
|
|
private static DataConnectionOptions Options() => new()
|
|
{
|
|
ReconnectInterval = TimeSpan.FromMilliseconds(100),
|
|
TagResolutionRetryInterval = TimeSpan.FromMilliseconds(150),
|
|
TagResolutionRetryMaxInterval = TimeSpan.FromMilliseconds(600),
|
|
WriteTimeout = TimeSpan.FromSeconds(5),
|
|
SeedReadMaxAttempts = 1,
|
|
SeedReadRetryDelay = TimeSpan.FromMilliseconds(10),
|
|
SubscribeBatchSize = 5,
|
|
SubscribeBatchDelay = TimeSpan.FromMilliseconds(50),
|
|
SeedReadBatchSize = 4,
|
|
SeedReadMaxParallelism = 2,
|
|
SeedOverallTimeout = TimeSpan.FromSeconds(30),
|
|
QualityFlushInterval = TimeSpan.FromMilliseconds(200)
|
|
};
|
|
|
|
private IActorRef CreateActor(FakeBatchDataConnection adapter, string name) =>
|
|
Sys.ActorOf(Props.Create(() => new DataConnectionActor(
|
|
name, adapter, Options(), _health, _factory, "OpcUa")), name);
|
|
|
|
private int HealthTransitions(string connectionName, ConnectionHealth health) =>
|
|
_health.ReceivedCalls().Count(c =>
|
|
c.GetMethodInfo().Name == nameof(ISiteHealthCollector.UpdateConnectionHealth) &&
|
|
c.GetArguments() is [string name, ConnectionHealth reported, ..] &&
|
|
name == connectionName && reported == health);
|
|
|
|
/// <summary>
|
|
/// A bad-session fault is a CONNECTION fault: the request fails and the actor enters
|
|
/// Reconnecting (which pushes bad quality for the connection).
|
|
///
|
|
/// <para><b>Fails before the fix:</b> the chunk was treated per-tag, so the request was acked
|
|
/// Success and the connection stayed nominally up on a dead session.</para>
|
|
/// </summary>
|
|
[Fact]
|
|
public void BadSessionServiceResultException_IsConnectionLevel_AndDrivesReconnect()
|
|
{
|
|
const string name = "opcua-bad-session";
|
|
var adapter = new FakeBatchDataConnection
|
|
{
|
|
BatchSubscribeThrows = () =>
|
|
new ServiceResultException(StatusCodes.BadSessionIdInvalid, "session is no longer valid")
|
|
};
|
|
var actor = CreateActor(adapter, name);
|
|
|
|
actor.Tell(new SubscribeTagsRequest("c1", "inst1", name, ["tag1", "tag2"], DateTimeOffset.UtcNow));
|
|
|
|
ExpectMsg<SubscribeTagsResponse>(m => !m.Success && m.ErrorMessage != null, TimeSpan.FromSeconds(5));
|
|
ExpectMsg<ConnectionQualityChanged>(q => q.Quality == QualityCode.Bad, TimeSpan.FromSeconds(5));
|
|
}
|
|
|
|
/// <summary>
|
|
/// The narrowness control. <c>BadNodeIdUnknown</c> is a per-node fault: the tags go unresolved
|
|
/// (bad quality, retried on the backoff timer) and the connection is NOT torn down.
|
|
/// </summary>
|
|
[Fact]
|
|
public void BadNodeIdServiceResultException_StaysPerTag_AndDoesNotTearDownTheConnection()
|
|
{
|
|
const string name = "opcua-bad-nodeid";
|
|
var adapter = new FakeBatchDataConnection
|
|
{
|
|
BatchSubscribeThrows = () =>
|
|
new ServiceResultException(StatusCodes.BadNodeIdUnknown, "no such node")
|
|
};
|
|
var actor = CreateActor(adapter, name);
|
|
|
|
actor.Tell(new SubscribeTagsRequest("c1", "inst1", name, ["tag1", "tag2"], DateTimeOffset.UtcNow));
|
|
|
|
// Per-tag treatment: bad quality per tag, and the request itself is still acked Success
|
|
// (the tags are registered and retried, exactly as an unresolved tag path is).
|
|
FishForMessage<SubscribeTagsResponse>(m => m.Success, TimeSpan.FromSeconds(5));
|
|
Assert.Equal(0, HealthTransitions(name, ConnectionHealth.Disconnected));
|
|
}
|
|
}
|