test: replace wall-clock sleeps used as synchronization — register row 40
Six sleeps across four test files stood in for an observable edge. Each preceded an
ABSENCE assertion (or "settled" before one), so they failed toward false-pass rather than
flake: the assertion could not distinguish "the mechanism ran and correctly stayed quiet"
from "the mechanism had not run yet".
ScriptDeadlineAtEnqueueTests (two sleeps, the worst offender)
- :105 Thread.Sleep(2s) — now two explicit edges: the run must be observably QUEUED
(scheduler.QueueDepth >= 1, never checked before), then the wait polls until the 1 s
budget has elapsed, bailing early if the run leaves the queue, and asserts it is STILL
queued afterwards. The test now proves the design claim it was named for — queue wait
burns the run's budget — instead of assuming it.
- :128 Thread.Sleep(2s) + Assert.Equal(0, DetachedThreadCount) — replaced by a positive
control: a second script that genuinely wedges a worker, on the same scheduler, same
1 s timeout + 1 s grace, armed strictly later. Its stuck report is the edge; once it
lands, every watchdog armed at or before it has provably finished its grace delay. The
absence is then asserted per script name, and the detach counter is pinned at 1 (the
wedged run) rather than 0 — which a counter that never moves also satisfied.
DeploymentWarmThenGateTests (:117 plus two sibling Task.Delay(500) startup waits)
- The manager calls SetInstanceCounts unconditionally when its startup config load
completes, so a report counter on the test collector is the startup edge. The
empty-site zero is now the manager's own observation instead of an untouched field.
SiteAlarmLiveCacheServiceTests (:195, :216)
- Both sampled IsLive once after a fixed sleep. Now they poll continuously across three
linger windows on a new teardown observable (the mock client counts subscription
cancellations), so a stop at any instant inside the window is caught, and each closes
with a positive control proving that counter moves.
DataConnectionActorTests (:435)
- A Task.Delay(200) "settle" after an AwaitCondition on a connect counter. The counter
only proves ConnectAsync was CALLED; the actor is still Connecting until it processes
the piped result. Now waits for the Connected health publication that BecomeConnected
emits, so the subscribe under test provably runs against Connected rather than being
silently stashed. (The stash path keeps its own coverage in
WP6_ConnectingState_StashesSubscribeRequests.)
Negative controls (product broken in the working tree, old vs new test compared):
- Stuck-script watchdog disabled entirely: OLD test passed 2/2, NEW test failed. The
whole watchdog could be deleted and the old assertions were satisfied.
- Startup reporting a phantom instance count and taking longer than 500 ms: OLD passed
2/2, NEW failed (expected 0, actual 1).
- Linger cancellation disabled + the remaining-viewer guard dropped, so the stream is
torn down under a live viewer while IsLive keeps reporting true:
Resubscribe_Within_Linger OLD passed 2/2, NEW failed. (Last_Viewer_Leaving caught this
particular break in the old form too, via a later assertion — its strengthening is the
teardown observable and the positive control, not a false-pass fix.)
- DataConnectionActorTests :435 has no false-pass demo: the sleep guarded no assertion
there, and the change is determinism plus proving the claimed state.
Also fixes a latent race these timing changes exposed in
InstanceActorChildAttributeRaceTests: it resolved child actors single-shot immediately
after construction, but ResolveOne throws until the child's cell is published on the
parent AND the cell's Actor property stays null until the child's constructor has run on
its own mailbox thread. Neither is ordered against the test thread, so it failed both ways
under full-suite load (~1 run in 6 once the sleeps above stopped padding the schedule;
14/14 green on the unmodified tree, so it was latent, not new). Resolution now retries to
a bounded deadline; the assertion is unchanged.
Verification: SiteRuntime.Tests 8/8 green (605 tests), Communication.Tests 3/3 (717),
DataConnectionLayer.Tests 3/3 (319). Test-only change — no src/ modifications.
This commit is contained in:
@@ -41,6 +41,17 @@ public class SiteAlarmLiveCacheServiceTests : TestKit
|
||||
/// <summary>Graceful-completion callbacks captured from every subscribe.</summary>
|
||||
public List<Action> Completions { get { lock (_lock) { return _completions.ToList(); } } }
|
||||
|
||||
private int _streamEnds;
|
||||
|
||||
/// <summary>
|
||||
/// Register row 40: how many site-wide subscriptions have been torn down (their
|
||||
/// cancellation token fired). This is the aggregator-stop observable — the linger
|
||||
/// stop cancels the aggregator's token, which ends the hanging subscribe below.
|
||||
/// Tests that must show an aggregator was NOT stopped assert on this rather than
|
||||
/// sampling <c>IsLive</c> once after a fixed sleep.
|
||||
/// </summary>
|
||||
public int StreamEndCount => Volatile.Read(ref _streamEnds);
|
||||
|
||||
public override Task SubscribeSiteAsync(
|
||||
string correlationId, Action<Commons.Messages.Streaming.AlarmStateChanged> onAlarmEvent,
|
||||
Action<Exception> onError, Action onCompleted, CancellationToken ct,
|
||||
@@ -55,7 +66,11 @@ public class SiteAlarmLiveCacheServiceTests : TestKit
|
||||
// subscribe), which is what makes the aggregator report the stream live.
|
||||
if (AutoConnect) onConnected?.Invoke();
|
||||
var tcs = new TaskCompletionSource();
|
||||
ct.Register(() => tcs.TrySetResult());
|
||||
ct.Register(() =>
|
||||
{
|
||||
Interlocked.Increment(ref _streamEnds);
|
||||
tcs.TrySetResult();
|
||||
});
|
||||
return tcs.Task;
|
||||
}
|
||||
public override void Unsubscribe(string correlationId) { }
|
||||
@@ -184,27 +199,47 @@ public class SiteAlarmLiveCacheServiceTests : TestKit
|
||||
[Fact]
|
||||
public void Last_Viewer_Leaving_Stops_Aggregator_After_Linger()
|
||||
{
|
||||
var service = CreateService(TimeSpan.FromMilliseconds(200), out _);
|
||||
var linger = TimeSpan.FromMilliseconds(200);
|
||||
var service = CreateService(linger, out var factory);
|
||||
|
||||
var sub1 = service.Subscribe(SiteId, () => { });
|
||||
var sub2 = service.Subscribe(SiteId, () => { });
|
||||
AwaitCondition(() => service.IsLive(SiteId), TimeSpan.FromSeconds(5));
|
||||
|
||||
sub1.Dispose();
|
||||
// One viewer remains — still live, not stopped.
|
||||
Thread.Sleep(300);
|
||||
Assert.True(service.IsLive(SiteId));
|
||||
|
||||
// Register row 40: one viewer remains, so the aggregator must NOT be stopped. The old
|
||||
// test slept 300 ms and sampled IsLive once — a teardown that happened at any other
|
||||
// instant (or one that stopped and restarted the stream) was invisible to it. Poll
|
||||
// CONTINUOUSLY across three linger windows instead, bailing out the moment either
|
||||
// signal says the aggregator went away.
|
||||
var held = System.Diagnostics.Stopwatch.StartNew();
|
||||
AwaitCondition(
|
||||
() => held.Elapsed > linger * 3
|
||||
|| !service.IsLive(SiteId)
|
||||
|| factory.Client.StreamEndCount > 0,
|
||||
TimeSpan.FromSeconds(5), TimeSpan.FromMilliseconds(10));
|
||||
Assert.True(service.IsLive(SiteId),
|
||||
"the aggregator was reported not-live while a viewer was still subscribed");
|
||||
Assert.Equal(0, factory.Client.StreamEndCount);
|
||||
Assert.Equal(1, factory.GetOrCreateCount);
|
||||
|
||||
sub2.Dispose();
|
||||
// Last viewer left → after the linger the aggregator is torn down and the entry cleared.
|
||||
AwaitCondition(() => !service.IsLive(SiteId), TimeSpan.FromSeconds(3));
|
||||
|
||||
// Positive control on the SAME observable the negative above rests on: the site-wide
|
||||
// stream really is torn down now, so the zero above was a live signal rather than a
|
||||
// counter that never moves.
|
||||
AwaitCondition(() => factory.Client.StreamEndCount > 0, TimeSpan.FromSeconds(3));
|
||||
Assert.Empty(service.GetCurrentAlarms(SiteId));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Resubscribe_Within_Linger_Cancels_The_Pending_Stop()
|
||||
{
|
||||
var service = CreateService(TimeSpan.FromMilliseconds(400), out var factory);
|
||||
var linger = TimeSpan.FromMilliseconds(400);
|
||||
var service = CreateService(linger, out var factory);
|
||||
|
||||
var sub1 = service.Subscribe(SiteId, () => { });
|
||||
AwaitCondition(() => service.IsLive(SiteId), TimeSpan.FromSeconds(5));
|
||||
@@ -212,13 +247,28 @@ public class SiteAlarmLiveCacheServiceTests : TestKit
|
||||
sub1.Dispose(); // schedules linger stop
|
||||
var sub2 = service.Subscribe(SiteId, () => { }); // returns within the linger window → cancels it
|
||||
|
||||
// Wait comfortably past the original linger; the aggregator must still be alive.
|
||||
Thread.Sleep(700);
|
||||
Assert.True(service.IsLive(SiteId));
|
||||
// Register row 40: the pending stop must be CANCELLED, not merely outrun. The old test
|
||||
// slept 700 ms and sampled once — it could not see a stream that was torn down and
|
||||
// silently replaced inside that window, because GetOrCreateCount is checked only at the
|
||||
// end and the mock hands back the same client object either way. Poll continuously past
|
||||
// the original linger on the teardown observable itself.
|
||||
var kept = System.Diagnostics.Stopwatch.StartNew();
|
||||
AwaitCondition(
|
||||
() => kept.Elapsed > linger * 3
|
||||
|| !service.IsLive(SiteId)
|
||||
|| factory.Client.StreamEndCount > 0,
|
||||
TimeSpan.FromSeconds(5), TimeSpan.FromMilliseconds(10));
|
||||
Assert.True(service.IsLive(SiteId),
|
||||
"the re-subscribe did not cancel the pending linger stop");
|
||||
Assert.Equal(0, factory.Client.StreamEndCount);
|
||||
// No second aggregator/stream was opened — the same one was kept warm.
|
||||
Assert.Equal(1, factory.GetOrCreateCount);
|
||||
|
||||
sub2.Dispose();
|
||||
|
||||
// Positive control: with the last viewer gone the linger stop is allowed to run, so the
|
||||
// teardown counter this test's zero rests on does in fact move.
|
||||
AwaitCondition(() => factory.Client.StreamEndCount > 0, TimeSpan.FromSeconds(5));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
|
||||
Reference in New Issue
Block a user