fix(twincat): book a refused registration's backoff before releasing ConnectGate (lmxopcua#773)

RegisterNotificationAsync published a refused registration into
NativeRegistrations under ConnectGate with NextRetryUtc = MinValue; the
SubscribeAsync caller booked the refusal backoff (NoteDeadFailure refused:
true) only after the gate was released and after its WARN. A probe tick in
that window took the gate, saw the registration as due, and retried it at
once - the CI flake in A_rearm_during_an_in_flight_retry_is_not_lost
("AttemptsFor(\"MAIN.T\") should be 1 but was 2").

RegisterNotificationAsync now books the failure (refusal backoff, or the
comms-class every-tick retry) before it publishes the dead registration, still
under the gate. The caller no longer books the refusal a second time (that
would double the backoff); its comms-class booking is idempotent and stays.

Tests: TwinCATCapturingLogger gains an OnWarning hook. A logger that holds
the refusal WARN for ten probe intervals forces the window open, so
A_rearm_during_an_in_flight_retry_is_not_lost exercises it on every run, and
the new A_refused_registration_is_published_with_its_backoff_already_set
pins the backoff (booked once, not doubled).

Pass rate of A_rearm_during_an_in_flight_retry_is_not_lost:
- before, unmodified test: 40/40 runs, plus 200/200 in-process repeats under
  CPU load (the natural window is a few instructions wide; the flake is rare);
- before, window held open: 0/32 (deterministic failure, "should be 1 but
  was 2");
- after, window held open: 40/40.
Full TwinCAT suite 663/663 at AKKA_TEST_TIMEFACTOR 1 and 3.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 15:06:25 -05:00
parent 17d83e4d16
commit f0992394b6
3 changed files with 67 additions and 5 deletions
@@ -964,10 +964,11 @@ public sealed class TwinCATDriver : IDriver, IReadable, IWritable, ITagDiscovery
// snapshot carrying the refusal's status, no health write (a refusal is not a comms
// fault). Its intent stays on the device with no live handle, so the probe-tick retry
// re-checks it after a PLC program change.
// Its backoff was booked under ConnectGate by RegisterNotificationAsync (lmxopcua#773);
// booking it again here would double it.
_logger.LogWarning(
"TwinCAT driver '{DriverInstanceId}' refused native notification for '{Reference}': {Reason}",
_driverInstanceId, reference, refused.Message);
NoteDeadFailure(reg, refused.StatusCode, refused: true, DateTime.UtcNow);
RaiseDataChange(handle, reference, new DataValueSnapshot(
null, refused.StatusCode, null, DateTime.UtcNow));
}
@@ -990,6 +991,7 @@ public sealed class TwinCATDriver : IDriver, IReadable, IWritable, ITagDiscovery
"'{Reference}'; it will be retried on the next healthy probe tick",
_driverInstanceId, reference);
MarkCommsFault(ex.Message, device);
// Idempotent for a comms-class failure (RegisterNotificationAsync already booked it).
NoteDeadFailure(reg, TwinCATStatusMapper.BadCommunicationError, refused: false, DateTime.UtcNow);
RaiseDataChange(handle, reference, new DataValueSnapshot(
null, TwinCATStatusMapper.BadCommunicationError, null, DateTime.UtcNow));
@@ -1196,10 +1198,16 @@ public sealed class TwinCATDriver : IDriver, IReadable, IWritable, ITagDiscovery
_options.NotificationMaxDelayMs, reg.OnChange, ct).ConfigureAwait(false);
}
catch (OperationCanceledException) { throw; }
catch
catch (Exception ex)
{
// Keep the intent with no live handle (a "dead" registration) so the probe-tick
// retry re-checks it and UnsubscribeAsync still removes it; the caller isolates it.
// Book the failure (a refusal's backoff) BEFORE publishing it, still under the gate: the
// probe tick takes the same gate, and a refusal it saw with NextRetryUtc = MinValue was
// retried at once instead of waiting out its backoff (lmxopcua#773).
NoteDeadFailure(reg,
(ex as TwinCATNotificationRefusedException)?.StatusCode ?? TwinCATStatusMapper.BadCommunicationError,
refused: ex is TwinCATNotificationRefusedException, DateTime.UtcNow);
device.NativeRegistrations[reg.Id] = reg;
throw;
}