fix(runtime): a changed alarm set re-subscribes instead of latching (review F1)

The alarm subscribe gated on 'a handle exists', which is correct for an idempotent
re-push and wrong for a deploy that widens/narrows the set. Combined with the
spawn-time desired set, a respawning deploy could latch the PREVIOUS revision's
alarm refs for the life of the connection; sourceNodeIds-filtering drivers
(OpcUaClient/AbCip/Focas) would never deliver the new alarm.

Seen RED: LastSubscribedAlarmRefs stayed [alarmX] after a deploy widening to
[alarmX, alarmY].
This commit is contained in:
Joseph Doherty
2026-08-20 10:24:52 -04:00
parent c1ad9e24d2
commit edd01fcc34
3 changed files with 135 additions and 6 deletions
@@ -271,6 +271,15 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// idempotent across repeated <see cref="SetDesiredSubscriptions"/> pushes.</summary>
private IAlarmSubscriptionHandle? _alarmSubscriptionHandle;
/// <summary>The alarm references the live <see cref="_alarmSubscriptionHandle"/> was actually established
/// with, so a CHANGED desired set can be told apart from a repeat of the same one.
/// <para>Without this the subscription latched: both <see cref="SubscribeDesiredAlarms"/> and
/// <see cref="HandleSubscribeAlarmsAsync"/> gate on "a handle already exists", which is the right guard
/// for an idempotent re-push and the WRONG one for a deploy that widens or narrows the alarm set —
/// drivers that filter on <c>sourceNodeIds</c> (OPC UA Client, AB CIP, FOCAS) would then never deliver a
/// newly-added alarm for the life of the connection. Empty whenever no handle is held.</para></summary>
private IReadOnlyList<string> _subscribedAlarmRefs = Array.Empty<string>();
/// <summary>
/// Gets or sets the timer scheduler for scheduling reconnection attempts.
/// </summary>
@@ -521,10 +530,14 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
// Native-alarm analogue: un-gate the IAlarmSource feed when alarm tags are (now) present. The
// common live path — a deploy delivers SetDesiredSubscriptions while the driver is already
// Connected — flows through HERE, so the alarm subscribe must happen on this message, not only
// on Connected entry. Unlike the value path above, there is deliberately no unsubscribe-on-empty:
// a removed alarm tag's condition node is torn down on the address-space rebuild and
// DriverHostActor.ForwardNativeAlarm drops any transition whose ConditionId no longer maps, so a
// lingering session-less alarm subscription can never surface a removed alarm.
// on Connected entry. A CHANGED set re-subscribes (release-then-subscribe inside
// HandleSubscribeAlarmsAsync): unlike the value path, "a subscription already exists" is NOT
// sufficient, because several drivers filter their feed on the subscribed sourceNodeIds, so a
// widened set that never re-subscribed would silently never deliver the new alarm.
// Still deliberately no unsubscribe-on-EMPTY: a removed alarm tag's condition node is torn down
// on the address-space rebuild and DriverHostActor.ForwardNativeAlarm drops any transition whose
// ConditionId no longer maps, so a lingering session-less alarm subscription can never surface a
// removed alarm.
SubscribeDesiredAlarms();
});
ReceiveAsync<SubscribeAlarms>(HandleSubscribeAlarmsAsync);
@@ -1167,6 +1180,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
// (GalaxyDriver.SubscribeAlarmsAsync clears the set before adding), so handles no longer accumulate
// across reconnects. The gate (Count > 0) and the one-shot fan-out are unchanged.
_alarmSubscriptionHandle = null;
_subscribedAlarmRefs = Array.Empty<string>();
}
/// <summary>Establish the native-alarm subscription that un-gates an <see cref="IAlarmSource"/> driver's
@@ -1176,10 +1190,17 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// desired, and no subscription is yet established.</summary>
private void SubscribeDesiredAlarms()
{
if (_driver is IAlarmSource && _desiredAlarmRefs.Count > 0 && _alarmSubscriptionHandle is null)
if (_driver is IAlarmSource && _desiredAlarmRefs.Count > 0 && AlarmSubscriptionIsStale())
Self.Tell(new SubscribeAlarms());
}
/// <summary>True when the desired alarm refs are not the ones the live handle was established with —
/// either because there is no handle yet, or because a deploy changed the set. The second case is the
/// one a bare "handle is null" check misses; see <see cref="_subscribedAlarmRefs"/>.</summary>
private bool AlarmSubscriptionIsStale() =>
_alarmSubscriptionHandle is null ||
!_subscribedAlarmRefs.SequenceEqual(_desiredAlarmRefs, StringComparer.Ordinal);
/// <summary>Calls the driver's <see cref="IAlarmSource.SubscribeAlarmsAsync"/> (bounded) to register the
/// alarm subscription that un-gates its native feed, caching the returned handle. Re-checks the guard
/// (the desired set may have cleared, or another SubscribeAlarms may have already established a handle,
@@ -1187,10 +1208,36 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
/// stays gated until then.</summary>
private async Task HandleSubscribeAlarmsAsync(SubscribeAlarms _)
{
if (_driver is not IAlarmSource src || _desiredAlarmRefs.Count == 0 || _alarmSubscriptionHandle is not null)
if (_driver is not IAlarmSource src || _desiredAlarmRefs.Count == 0 || !AlarmSubscriptionIsStale())
return;
var refs = _desiredAlarmRefs;
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10));
// A handle held against a DIFFERENT set must be released before re-subscribing, or a
// sourceNodeIds-filtering driver accumulates two overlapping subscriptions. Best-effort and
// deliberately non-fatal: the re-subscribe below is what actually matters, and a driver whose
// unsubscribe throws must not be left on the stale set because of it. Cleared FIRST so a throw
// cannot leave a handle that the staleness check would keep treating as live.
if (_alarmSubscriptionHandle is { } stale)
{
_alarmSubscriptionHandle = null;
_subscribedAlarmRefs = Array.Empty<string>();
try
{
await _invoker.ExecuteAsync(
DriverCapability.AlarmSubscribe,
_driverInstanceId,
async ct => await src.UnsubscribeAlarmsAsync(stale, ct),
cts.Token);
}
catch (Exception ex)
{
_log.Warning(ex,
"DriverInstance {Id}: releasing the previous native-alarm subscription ({Diag}) threw; re-subscribing anyway",
_driverInstanceId, stale.DiagnosticId);
}
}
try
{
_alarmSubscriptionHandle = await _invoker.ExecuteAsync(
@@ -1198,6 +1245,7 @@ public sealed class DriverInstanceActor : ReceiveActor, IWithTimers
_driverInstanceId,
async ct => await src.SubscribeAlarmsAsync(refs, ct),
cts.Token);
_subscribedAlarmRefs = refs;
_log.Info("DriverInstance {Id}: native-alarm subscription established for {Count} alarm ref(s) ({Diag})",
_driverInstanceId, refs.Count, _alarmSubscriptionHandle.DiagnosticId);
}