P1-1 — the row-67 ensure-started self-heal re-read the durable row off-thread but
captured no _storeWriteGeneration and never ran IsStoreSnapshotStale, unlike the boot
load and the reconcile sweep. A Disable or Delete committing inside that window left the
result carrying an enabled config that passed every guard (neither command sets any of
the three pipeline markers ShouldAttemptEnsureStart re-checks), so the heal started an
instance the operator had just stopped — PERMANENTLY, since the sweep iterates store rows
only and has no runtime→store stop leg. The generation is now captured at dispatch,
threaded on EnsureInstanceStartedResult and checked on landing, and the comment claiming
the stored row already covered this is corrected. The heal reads through _configLoader,
the same seam the boot load and sweep use (identical production call), so a test can hold
the read open across a committed disable/delete.
P1-2 — the "static overrides reset on redeployment" invariant was racy in both
directions. ApplyDeployment created the successor Instance Actor first and only then
dispatched the clear, queued behind a full config-JSON write, so the successor's PreStart
override load routinely won and applied the PREDECESSOR's overrides — keys absent from
the new config included — to in-memory state for the actor's lifetime while the disk
ended cleared; a later restart then silently flipped the values. The clear is now
dispatched BEFORE CreateInstanceActor and handed to the actor as a gate its override load
chains behind; the persistence task joins the same task, so a failed clear still fails the
deploy exactly as before. Leg 2: the predecessor's PostStop flush was fire-and-forget and
could commit up to one coalescing window AFTER the clear (the Deployment Manager gates the
clear on Terminated), durably resurrecting overrides — PostStop now waits for it, bounded
at 5s with a Warning on overrun.
Tests: new RedeployStaticOverrideResetTests (5) pins both legs plus the gate mechanism and
first-deploy behaviour; three new cases in ReconcileStartsInstanceTests pin the discarded
disable/delete heals and that an unraced heal still starts. InstanceActorTests'
StaticOverride_ResetOnRedeployment comment claimed to simulate an ordering the production
path did not have — corrected.
Joseph Doherty
e63ffd90ee