test(mesh): close review findings on the PreStart-race logging fix — non-vacuous once-per-incarnation pin, broken-dialer symmetry, comment idiom

Follow-up to a94eb8a3, addressing CHANGES REQUESTED on code review. Production
logic was confirmed correct on all spec points; this commit fixes the review's
test/comment findings only.

IMPORTANT-1: A_permanently_unresolvable_pskProvider_is_reported_once_per_incarnation_not_once_per_refresh
did not prove the second refresh ran — its Expect(0) window gated on a dialer
dispatch landing, but LoadContactsFromDb is fire-and-forget and the first
block's AwaitCondition already re-Tells MeshCommand per poll, so the dialer
call count could reach the threshold before PublishKnownClusters was ever
re-invoked by the second load. Fixed by having the PSK factory count its own
invocations and gating both AwaitConditions on that count directly (pskCalls
>= 2 proves the first retry; pskCalls > pskCallsAfterFirstLoad, awaited INSIDE
the Expect(0) window, proves the second one). Bite-checked: with
_pskProviderAbsenceReported's guard temporarily removed, the test now
correctly fails ("Received 1 message too many... received 1 that matched
filter"); guard restored and reverified green.

MINOR-2: reworded four comments (CentralCommunicationActor.cs ~109, ~259,
~305-309, ~853) from narrating the change ("this fix removes", "already
existed, unchanged", "added by this fix", "the old two-call-site shape") to
stating the timeless invariant.

MINOR-3: added A_supplied_dialer_factory_that_always_throws_still_drops_each_command_loudly_while_PreStart_stays_quiet
— the symmetric case to the existing never-supplied-dialer test, exercising
Resolve's catch + firstUse:true path (a factory that WAS supplied and always
throws) rather than the null-factory path. Asserts PreStart plus the first
contact load stay silent and DispatchOverGrpc's existing per-dispatch Error
still fires on every dispatch, unguarded.

MINOR-4: "Check the ControlPlane registration wires IMeshPskProvider" ->
"Check that the ControlPlane registration wires up IMeshPskProvider"; no test
pinned the old substring, so no test update was needed for this one.

Skipped item 5 (restart-re-arms pin) per instruction — out of scope.

Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
This commit is contained in:
Joseph Doherty
2026-08-06 10:30:20 -04:00
parent 2e06dfe52d
commit d7c9ead64e
2 changed files with 87 additions and 41 deletions
@@ -104,10 +104,9 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
// Set once PublishKnownClusters has reported a genuinely-absent PSK provider (a factory WAS
// supplied and still resolves null after the first-use retry). PublishKnownClusters runs on
// the contact-refresh timer forever, so without this an actor with a permanently broken
// provider would repeat the Error every ContactRefreshSeconds for its whole life — the same
// per-refresh noise this fix removes, just relocated. Same warn-once shape as
// _warnedPortlessNodes; actor thread only.
// the contact-refresh timer for the actor's whole life, so an unguarded Error there would
// repeat every ContactRefreshSeconds forever. Same warn-once shape as _warnedPortlessNodes;
// actor thread only.
private bool _pskProviderAbsenceReported;
// The gRPC dispatch set, replaced wholesale on every contact load. Actor thread only; the fan
@@ -257,14 +256,12 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
}
// Deliberately NO null guard for either dependency here. At this point a still-null value
// is indistinguishable from the starter-ordering race Resolve just logged quietly above —
// reporting it loudly here would be the exact false alarm this fix removes, on nearly
// every boot. A retry that ALSO fails — the genuinely permanent absence — is reported
// loudly where it actually costs something instead: DispatchOverGrpc for the dialer (every
// dropped command, unchanged — see the EventFilter pin on that message), and
// PublishKnownClusters for the PSK provider (once per incarnation, not once per refresh
// forever, since that handler runs on the contact-refresh timer for the actor's whole
// life).
// is indistinguishable from the starter-ordering race Resolve just logged quietly above,
// so a loud report here cannot tell a lost race from a genuine absence — that decision
// belongs where a still-null result after the memoized properties' first-use retry
// actually costs something: DispatchOverGrpc for the dialer (every dropped command), and
// PublishKnownClusters for the PSK provider (once per incarnation, since that handler runs
// on the contact-refresh timer for the actor's whole life).
// Started in EVERY mode. The refresh is not transport state: it is central's ClusterNode
// inventory, and the known-cluster gate it feeds protects an endpoint mapped by PORT, not by
@@ -308,10 +305,10 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
/// only the caller knows whether a value came back after the retry the memoized
/// properties perform. So the loud, operator-facing signal is deliberately NOT here: it
/// is at the two places that consume a still-null result after that retry —
/// <see cref="DispatchOverGrpc"/> for the dialer (already existed, unchanged) and
/// <see cref="PublishKnownClusters"/> for the PSK provider (added by this fix, guarded
/// to once per incarnation because that call site runs on a timer forever, where
/// <see cref="DispatchOverGrpc"/> only runs once per command).
/// <see cref="DispatchOverGrpc"/> for the dialer (fires every dropped command) and
/// <see cref="PublishKnownClusters"/> for the PSK provider (fires once per incarnation,
/// because that call site runs on a timer forever, where <see cref="DispatchOverGrpc"/>
/// only runs once per command).
/// </para>
/// </remarks>
private T? Resolve<T>(Func<T?>? factory, string name, bool firstUse)
@@ -854,13 +851,13 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
/// </para>
/// <para>
/// <b>This is the PSK provider's "first use" call site.</b> <see cref="PskProvider"/> is
/// evaluated exactly ONCE per call (not once per statement — the old two-call-site shape
/// doubled the memoized property's retry rate for no benefit) and its result reused for
/// both calls below. A still-null result here means the property's own retry, on top of
/// PreStart's original attempt, has ALSO failed — the genuinely permanent absence — so
/// this is where that gets reported loudly, guarded to once per incarnation because this
/// handler runs on the contact-refresh timer for the actor's entire life and an unguarded
/// Error here would repeat every <c>ContactRefreshSeconds</c> forever.
/// evaluated exactly once per call and its result reused below; evaluating it
/// per-statement would double the memoized property's retry rate for no benefit. A
/// still-null result here means the property's own retry, on top of PreStart's original
/// attempt, has ALSO failed — the genuinely permanent absence — so this is where that
/// gets reported loudly, guarded to once per incarnation because this handler runs on
/// the contact-refresh timer for the actor's entire life and an unguarded Error here
/// would repeat every <c>ContactRefreshSeconds</c> forever.
/// </para>
/// </remarks>
private void PublishKnownClusters(ContactsLoaded msg)
@@ -879,7 +876,7 @@ public sealed class CentralCommunicationActor : ReceiveActor, IWithTimers
+ "the known-cluster gate will NEVER be published and cached per-cluster keys "
+ "will NEVER be dropped. Inbound mesh acks stay exposed to the "
+ "pre-authentication secret-store cost the gate removes, and a rotated key "
+ "needs a central restart. Check the ControlPlane registration wires "
+ "needs a central restart. Check that the ControlPlane registration wires up "
+ "IMeshPskProvider (this will not be logged again this incarnation)");
}