fix(siteruntime): guard the DCL arbitration overlay against stale store snapshots — register row 101
An off-thread snapshot read racing an artifact apply could land after the apply populated the overlay; its stale absences then tombstoned and stopped just-applied LIVE connections, which nothing re-creates until an instance redeploy (the removal evicts the hash slot the overlay push is gated on). Each read now captures a definitions-write generation at dispatch; a stale result is discarded whole (additions too — a stale row would reconfigure a live connection backward). Deterministic repro fails before the fix; a fresh-generation guard test pins that row-90 disappearance still fires.
This commit is contained in:
@@ -387,3 +387,28 @@ tests plus SiteRuntime-010, renamed `…_ReissuesReconfigureCommand`).
|
||||
|
||||
§6 residuals **2** (a removal is not durable across a restart) and **3** (a delete is undetectable on
|
||||
a node that never adopted the row) are unchanged and still open. §6 residual **4** was never a gap.
|
||||
|
||||
## 10. Postscript — the stale-snapshot race (register row 101, 2026-08-17)
|
||||
|
||||
The sweep design read the store off the actor thread (§5) and compared the result against the
|
||||
overlay only when the piped message landed. That gap was a TOCTOU the original memo did not name:
|
||||
a read starting before an artifact apply's store writes could land AFTER
|
||||
`HandleApplyArtifactDataConnectionsToDcl` populated the overlay, and its stale snapshot would then
|
||||
"prove" the just-applied names disappeared. The disappearance detection (§5) tombstoned them and
|
||||
stopped the just-created live connection actors — and because the removal also evicts the hash
|
||||
slot the overlay's push is gated on, the next fresh sweep clears the tombstone but re-creates
|
||||
nothing: **the connection stays dead until an instance redeploy**. Surfaced as a one-off test flake
|
||||
(`ApplyingArtifacts_RemovedDataConnection_StopsLiveDclConnection` received a removal for `PlcA`
|
||||
where `PlcGone` was expected — overlay iteration order removing BOTH names off a stale empty
|
||||
snapshot), first suspected to be test-fixture-local; verification showed the mechanism is real in
|
||||
production whenever an artifact deploy lands while a sweep read is in flight.
|
||||
|
||||
**Fix: a definitions-write generation.** `_definitionsWriteGeneration` is bumped on the actor
|
||||
thread by every artifact apply; each snapshot read captures it at dispatch and a result carrying a
|
||||
stale generation is discarded WHOLE — additions included, since a stale row pushed at a live
|
||||
connection would reconfigure it backward and flip-flop with the next fresh read. Discarding is
|
||||
safe: the apply already updated the overlay in the same pass (§5), and the next tick reads fresh.
|
||||
Applies to both snapshot consumers (boot load and sweep). Deterministic reproduction + a guard
|
||||
test that the legitimate row-90 disappearance still fires on a FRESH empty snapshot:
|
||||
`StaleSweepSnapshot_RacingAnArtifactApply_DoesNotRemoveJustAppliedConnections` (fails before the
|
||||
fix) and `FreshEmptySweepSnapshot_StillRemovesDisappearedConnections`.
|
||||
|
||||
Reference in New Issue
Block a user