fix(s7): bound async wire ops with a wall-clock deadline (R2-01 read leg)
v2-ci / build (pull_request) Successful in 5m5s
v2-ci / unit-tests (pull_request) Failing after 13m50s

The R2-01 live gate (S7_1500ConnectTimeoutOutageTests, docker-pause blackhole)
surfaced a real gap the offline fakes couldn't: S7.Net's ReadTimeout/WriteTimeout
map to the TcpClient's ReceiveTimeout/SendTimeout, which govern only SYNCHRONOUS
socket calls — the async read/write paths S7.Net uses ignore them. On an
established-but-frozen peer (frozen PLC / firewall DROP / cable pulled mid-flow,
TCP session still open) a read blocked until the OS TCP stack gave up (minutes),
silently wedging the poll loop: no Bad tick, no reconnect. STAB-14 fixed only the
CONNECT leg (EnsureConnectedAsync CancelAfter); this is its READ-leg sibling.

- New S7OperationDeadline: bounds every data-plane wire op with a wall-clock
  ceiling (= _options.Timeout), surfacing an overrun as TimeoutException. Applied
  in S7PlcAdapter to Read/ReadBytes/Write/WriteBytes/ReadStatus. OpenAsync is left
  to EnsureConnectedAsync's own CancelAfter (not double-bounded).
- IsS7ConnectionFatal now classifies TimeoutException fatal → handle marked dead →
  next EnsureConnectedAsync reopens (connect-timeout fix takes over from there).
- Tests: 5 S7OperationDeadline unit tests (deadline / token-honouring / caller-
  cancel-passthrough / resultless), 1 driver-reaction test (read TimeoutException
  → reopen). Driver.S7.Tests 260/260.
- Live gate S7_1500ConnectTimeoutOutageTests now GREEN against the real snap7 sim
  (8s: baseline Good -> pause blackhole -> Bad tick -> unpause -> recovered Good).
This commit is contained in:
Joseph Doherty
2026-07-15 07:11:42 -04:00
parent 152a5645ad
commit 88e0977ae6
5 changed files with 239 additions and 8 deletions
@@ -63,6 +63,40 @@ public sealed class S7DriverReconnectTests
factory.Created[1].Disposed.ShouldBeFalse();
}
/// <summary>
/// R2-01 read-leg gate. A <see cref="TimeoutException"/> — what <c>S7OperationDeadline</c>
/// raises when a wire read blows its wall-clock ceiling on an established-but-frozen peer
/// (the async read S7.Net's ignored socket <c>ReadTimeout</c> can't bound) — is classified
/// connection-fatal: the handle is marked dead and the NEXT read reopens a fresh connection,
/// exactly like a socket drop. Before the fix a read-timeout was NOT fatal, so the wedged
/// handle was reused and the poll loop never recovered (<c>Created.Count</c> pinned at 1).
/// </summary>
[Fact]
public async Task Read_reopens_after_a_wire_operation_timeout()
{
var factory = new FakeS7PlcFactory();
using var drv = new S7Driver(Options(), "s7-read-timeout-fatal", factory);
await drv.InitializeAsync("{}", TestContext.Current.CancellationToken);
factory.Created.Count.ShouldBe(1);
// The next read blows its deadline — surfaces as a TimeoutException, the shape
// S7OperationDeadline raises for a frozen-but-established peer.
factory.Created[0].NextReadThrows = new TimeoutException("S7 read of 'DB1.DBW0' did not complete within 250 ms.");
// Read #1 — timeout → Degraded, handle marked dead, no reopen yet.
var bad = await drv.ReadAsync(["W0"], TestContext.Current.CancellationToken);
bad[0].StatusCode.ShouldNotBe(0u);
drv.GetHealth().State.ShouldBe(DriverState.Degraded);
factory.Created.Count.ShouldBe(1);
// Read #2 — disposes the dead handle, opens a SECOND connection, reads Good, recovers.
var recovered = await drv.ReadAsync(["W0"], TestContext.Current.CancellationToken);
recovered[0].StatusCode.ShouldBe(0u);
drv.GetHealth().State.ShouldBe(DriverState.Healthy);
factory.Created.Count.ShouldBe(2);
factory.Created[0].Disposed.ShouldBeTrue();
}
/// <summary>
/// A protocol / data-address error (S7.Net <see cref="ErrorCode.ReadData"/>) is NOT a
/// connection loss — the driver keeps the same connection and does not churn a reopen.
@@ -0,0 +1,95 @@
using System.Diagnostics;
using Shouldly;
using Xunit;
namespace ZB.MOM.WW.OtOpcUa.Driver.S7.Tests;
/// <summary>
/// Unit tests for <see cref="S7OperationDeadline"/> — the wall-clock ceiling that bounds every
/// S7 async wire op (R2-01 read-leg gate). Async S7.Net reads/writes ignore the socket
/// ReadTimeout/WriteTimeout, so a frozen-but-established peer would otherwise block for minutes;
/// these pin the deadline behaviour without a live PLC.
/// </summary>
[Trait("Category", "Unit")]
public sealed class S7OperationDeadlineTests
{
private static readonly TimeSpan ShortTimeout = TimeSpan.FromMilliseconds(150);
/// <summary>An operation that completes within the deadline returns its value untouched.</summary>
[Fact]
public async Task Completes_within_deadline_returns_value()
{
var result = await S7OperationDeadline.RunAsync(
_ => Task.FromResult(42), ShortTimeout, "read", TestContext.Current.CancellationToken);
result.ShouldBe(42);
}
/// <summary>
/// An operation that ignores its token and never completes is cut at the deadline and
/// surfaces a <see cref="TimeoutException"/> — the frozen-established-peer shape. Uses a bare
/// <see cref="TaskCompletionSource"/> that never completes, so the token is genuinely ignored.
/// </summary>
[Fact]
public async Task Token_ignoring_hang_times_out()
{
var neverCompletes = new TaskCompletionSource<int>().Task;
var sw = Stopwatch.StartNew();
var ex = await Should.ThrowAsync<TimeoutException>(async () =>
await S7OperationDeadline.RunAsync(
_ => neverCompletes, ShortTimeout, "read of 'DB1.DBW0'", TestContext.Current.CancellationToken));
sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(5));
ex.Message.ShouldContain("did not complete within 150 ms");
}
/// <summary>
/// An operation that honours its token (the well-behaved async-read case) is cancelled at the
/// deadline; the resulting cancellation is normalised to a <see cref="TimeoutException"/>,
/// NOT surfaced as an <see cref="OperationCanceledException"/>.
/// </summary>
[Fact]
public async Task Token_honouring_hang_times_out_as_timeout_not_cancel()
{
await Should.ThrowAsync<TimeoutException>(async () =>
await S7OperationDeadline.RunAsync(
async token =>
{
await Task.Delay(Timeout.Infinite, token);
return 0;
},
ShortTimeout, "read", TestContext.Current.CancellationToken));
}
/// <summary>
/// Caller cancellation (as opposed to the deadline firing) propagates as an
/// <see cref="OperationCanceledException"/> — the deadline wrapper must not swallow genuine
/// cancellation into a timeout.
/// </summary>
[Fact]
public async Task Caller_cancellation_propagates_as_operation_cancelled()
{
using var cts = new CancellationTokenSource();
cts.CancelAfter(50);
// A long deadline so the CALLER token fires first, not the internal deadline.
await Should.ThrowAsync<OperationCanceledException>(async () =>
await S7OperationDeadline.RunAsync(
async token =>
{
await Task.Delay(Timeout.Infinite, token);
return 0;
},
TimeSpan.FromSeconds(30), "read", cts.Token));
}
/// <summary>The result-less overload also bounds a hang and surfaces a <see cref="TimeoutException"/>.</summary>
[Fact]
public async Task Resultless_overload_times_out()
{
var neverCompletes = new TaskCompletionSource().Task;
await Should.ThrowAsync<TimeoutException>(async () =>
await S7OperationDeadline.RunAsync(
_ => neverCompletes, ShortTimeout, "write of 'DB1.DBW0'", TestContext.Current.CancellationToken));
}
}