fix(mqtt): gate DisposeAsync against in-flight lifecycle ops (C1) + rebuild-branch tests (I1)

C1 (Critical) — DisposeAsync went straight to TeardownAsync with no lifecycle-gate
wait, reopening the orphaned-connection class. Interleaving: ReinitializeAsync's
rebuild branch holds the gate mid-InitializeCoreAsync, past its own teardown but
before `_connection = connection`; an ungated dispose sees a null connection, does
nothing, and sets `_disposed`; the rebuild then publishes a live connection plus a
host-probe loop that every later dispose short-circuits past. Not proven reachable
today (DriverInstanceActor.PostStop calls the gated ShutdownAsync), but MqttDriver
publicly implements IAsyncDisposable, which invites `await using`.

- DisposeAsync now takes the gate with the same bounded wait + fallback teardown
  ShutdownAsync uses, and sets `_disposed` BEFORE the wait.
- InitializeCoreAsync re-checks `_disposed` after the connect and disposes the
  connection it just built rather than publishing it — this closes the residual
  window on the bounded-timeout fallback path.
- The doc comment no longer claims parity with ShutdownAsync it did not have, and
  stops conflating "don't Dispose() the semaphore object" with "don't WaitAsync".

I1 (Important) — the SameSession == false rebuild branch had no test driving it.
Adds three: an endpoint-changing delta rebuilds and Faults on a refused connect;
an ingest-only delta (MaxPayloadBytes) ALSO rebuilds, pinning that IngestIdentity
is nested inside SessionIdentity; and DisposeAsync serializes behind a lifecycle
operation parked at a new internal BeforeConnectHookForTests seam (mirroring
MqttConnection's AfterConnectHookForTests, which exists for the same race class).

Minors: comments recording that IngestIdentity names no Sparkplug field and must
grow one in P2 (tasks 21/22), and why _options/_subscriptions/_authoredRawPaths
get looser memory discipline than _health/_hostState.

Falsifiability: reverting DisposeAsync to the ungated form reddens exactly the new
serialization test ("Shouldly.ShouldAssertException : raced"); forcing SameSession
to always-true reddens exactly the two new rebuild tests. 240/240 MQTT tests pass;
forced rebuild of the driver project is 0 warnings under TreatWarningsAsErrors.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-24 17:31:04 -04:00
parent f79d13e2d8
commit a13ae926a8
2 changed files with 203 additions and 6 deletions
@@ -28,6 +28,33 @@ public sealed class MqttDriverDiscoveryTests
private static MqttDriver PlainDriver(params RawTagEntry[] tags)
=> new(new MqttDriverOptions { Mode = MqttMode.Plain, RawTags = tags }, "d", null);
/// <summary>
/// Options aimed at a definitely-closed port so a connect is <b>refused immediately</b> rather
/// than hanging — the tests that must prove a rebuild actually dials need the dial to fail
/// fast, not to burn a connect deadline.
/// </summary>
private static MqttDriverOptions ClosedPortOptions(params RawTagEntry[] tags) => new()
{
Mode = MqttMode.Plain,
Host = "127.0.0.1",
Port = 1,
UseTls = false,
ConnectTimeoutSeconds = 2,
RawTags = tags,
};
private static MqttDriver ClosedPortDriver(params RawTagEntry[] tags)
=> new(ClosedPortOptions(tags), "d", null);
/// <summary>
/// Serializes a full options record as the reinitialize blob. Round-tripping the whole record
/// (rather than hand-writing a partial JSON object) guarantees that only the field the caller
/// changed differs — a hand-written delta silently omitting a session field would take the
/// rebuild branch for the wrong reason and the test would pass vacuously.
/// </summary>
private static string DeltaJson(MqttDriverOptions options)
=> System.Text.Json.JsonSerializer.Serialize(options, MqttJson.Options);
/// <summary>
/// Plain mode replays ONLY the authored tag set: one variable per authored raw tag, a
/// single discovery pass (<see cref="DiscoveryRediscoverPolicy.Once"/>), and no online
@@ -192,6 +219,86 @@ public sealed class MqttDriverDiscoveryTests
.ShouldBe(["Plant/Mqtt/dev1/Flow", "Plant/Mqtt/dev1/Pressure"]);
}
/// <summary>
/// A delta that changes the broker endpoint takes the <b>rebuild</b> branch: it must actually
/// dial the new endpoint (proven here by the refused connect), and a rebuild whose connect
/// fails must land <see cref="DriverState.Faulted"/> rather than letting the failure escape
/// with the health surface still claiming Healthy.
/// </summary>
[Fact]
public async Task ReinitializeAsync_SessionChangingDelta_Rebuilds_AndFaultsOnUnreachableBroker()
{
var driver = ClosedPortDriver(Tag("Plant/Mqtt/dev1/Temp", "f/t"));
// Only Host differs from the ctor options — everything else round-trips identically.
var delta = DeltaJson(ClosedPortOptions() with { Host = "127.0.0.2" });
await Should.ThrowAsync<Exception>(() => driver.ReinitializeAsync(delta, CancellationToken.None));
driver.GetHealth().State.ShouldBe(DriverState.Faulted);
}
/// <summary>
/// Pins that <c>IngestIdentity</c> is nested inside <c>SessionIdentity</c>: a delta touching
/// ONLY an ingest setting still rebuilds. Without the nesting it would be judged "same
/// session", applied in place, and the subscription manager that actually reads
/// <c>MaxPayloadBytes</c> would never be rebuilt — a config change that silently does nothing.
/// </summary>
[Fact]
public async Task ReinitializeAsync_IngestOnlyDelta_AlsoRebuilds()
{
var driver = ClosedPortDriver(Tag("Plant/Mqtt/dev1/Temp", "f/t"));
var delta = DeltaJson(ClosedPortOptions() with { MaxPayloadBytes = 4096 });
// Reaching the (refused) connect at all is the proof the rebuild branch was taken; the
// tag-only test above is the control that shows an in-place delta never dials.
await Should.ThrowAsync<Exception>(() => driver.ReinitializeAsync(delta, CancellationToken.None));
}
/// <summary>
/// The falsifiability pin for the orphaned-connection class: <see cref="MqttDriver.DisposeAsync"/>
/// must serialize behind an in-flight lifecycle operation, not race past it.
/// </summary>
/// <remarks>
/// An ungated dispose running while a rebuild holds the gate observes <c>_connection == null</c>
/// (the rebuild has torn the old session down but not yet assigned the new one), does nothing,
/// and sets the disposed flag — after which the rebuild assigns a live connection plus a
/// host-probe loop that no later dispose can reach. Here the driver is parked inside
/// <c>InitializeCoreAsync</c> at the pre-connect hook; the assertion is that dispose does not
/// complete while it is parked, and does complete once it is released.
/// </remarks>
[Fact]
public async Task DisposeAsync_SerializesBehindAnInFlightLifecycleOperation()
{
var driver = ClosedPortDriver(Tag("Plant/Mqtt/dev1/Temp", "f/t"));
var entered = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
driver.BeforeConnectHookForTests = async _ =>
{
entered.TrySetResult();
await release.Task;
};
var initialize = Task.Run(() => driver.InitializeAsync("", CancellationToken.None));
await entered.Task.WaitAsync(TimeSpan.FromSeconds(5));
var dispose = driver.DisposeAsync().AsTask();
// The gate is held by the parked initialize. Dispose must still be waiting. (The gate wait is
// bounded by ConnectTimeoutSeconds = 2 s, so this 300 ms window is comfortably inside it —
// an ungated dispose returns essentially instantly.)
var raced = await Task.WhenAny(dispose, Task.Delay(TimeSpan.FromMilliseconds(300)));
raced.ShouldNotBe(dispose, "DisposeAsync returned while a lifecycle operation held the gate");
release.SetResult();
await Should.ThrowAsync<Exception>(() => initialize); // connect refused on the closed port
await dispose.WaitAsync(TimeSpan.FromSeconds(10));
dispose.IsCompletedSuccessfully.ShouldBeTrue();
}
/// <summary>Identity is fixed at construction and must match the persisted <c>DriverInstance.DriverType</c>.</summary>
[Fact]
public void Identity_IsMqtt()