test(controlplane): #544 de-flake deadline-timeout test — seed acks before spawn, un-hittable wall deadline

DeadlineElapsed_for_current_deployment_marks_TimedOut carried two races,
both test-side. ConfigPublishCoordinator itself is untouched.

(a) The NodeDeploymentState row was inserted AFTER Sys.ActorOf. The
coordinator's PreStart reads NodeDeploymentStates to recover the
expected-ack set, and it runs as soon as the actor is scheduled — so if
PreStart won, the expected-ack set was empty and the coordinator could
short-circuit to Sealed instead of ever reaching TimedOut. The row is now
seeded before the spawn.

(b) The applyDeadline was 150 ms of real wall clock, and it arms a real
Akka timer. Under load that timer could fire on its own and race the
test's explicit DeadlineElapsed tell — so the test was intermittently
exercising the timer rather than the message handler it names. The
deadline is now TimeSpan.FromMinutes(5), matching the other two tests in
this class, which makes it un-hittable inside the test's own 3 s
AwaitAssert bound and leaves the explicit tell as the only deadline
signal the coordinator ever sees. That tell, and the AwaitAssert bound,
are unchanged.

Verified: the single test 5/5 green in a row, then the whole class 3/3.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
This commit is contained in:
Joseph Doherty
2026-08-06 17:15:42 -04:00
parent f10f4c3754
commit 722a222db5
@@ -22,11 +22,11 @@ public sealed class ConfigPublishCoordinatorTimeoutTests : ControlPlaneActorTest
var dbFactory = NewInMemoryDbFactory();
var deploymentId = SeedDispatchingDeployment(dbFactory);
// Short deadline so we can drive it deterministically in the test.
var actor = Sys.ActorOf(ConfigPublishCoordinator.Props(dbFactory, TimeSpan.FromMilliseconds(150)));
// Seed a NodeDeploymentState row so the coordinator doesn't see "zero expected acks"
// and short-circuit to Sealed. We pretend a driver node exists in the cluster.
// Seed the NodeDeploymentState row BEFORE the actor is spawned. PreStart reads
// NodeDeploymentStates to recover the expected-ack set, and it runs as soon as Sys.ActorOf
// schedules the actor — so seeding afterwards is a race: if PreStart wins, the coordinator
// sees zero expected acks and can short-circuit straight to Sealed, and the assertion below
// never sees TimedOut. We pretend a driver node exists in the cluster.
using (var db = dbFactory.CreateDbContext())
{
db.NodeDeploymentStates.Add(new Configuration.Entities.NodeDeploymentState
@@ -38,6 +38,13 @@ public sealed class ConfigPublishCoordinatorTimeoutTests : ControlPlaneActorTest
db.SaveChanges();
}
// Deliberately un-hittable wall-clock deadline (matching the other tests in this class).
// The apply deadline is a real Akka timer: a short one can self-fire under load and race
// the explicit DeadlineElapsed below, so the test would sometimes be measuring the timer
// rather than the message handler. Five minutes guarantees the ONLY deadline signal the
// coordinator ever receives is the one this test sends.
var actor = Sys.ActorOf(ConfigPublishCoordinator.Props(dbFactory, TimeSpan.FromMinutes(5)));
// Drive the deadline ourselves rather than waiting for the cluster's empty driver-set
// bypass. Tell the actor the deadline elapsed for this id.
actor.Tell(new ConfigPublishCoordinator.DeadlineElapsed(deploymentId));