docs(register): Wave 6 close-out — close rows 97 (in-place DCL reconfigure, live-probed round trip), 98 (fail-closed scope gate, 5 sites not 2 — id-enumeration oracle closed), 99 (strict audit filters, 7 gates + lax-server premise corrected, live-probed), 100 (compile cache instance-scoped, Clear() deleted — zero production callers); row-88 postscript (ordinal guard); add row 101 (overlay-sweep race hypothesis)
This commit is contained in:
@@ -80,7 +80,7 @@ All 7 fix-now items landed via PLAN-04/05/06/07/08 (verified in review 08 round
|
||||
| 85 | ~~**DCL connection actors are not spawned for connection configs that arrive via LocalDb replication — row 78's gap, one layer down** *(Medium-Low, availability; proven live 2026-08-16 by the row-78 close-out probe)*~~ **RESOLVED 2026-08-17 (`8f2c2813`) — with a premise correction AND a widened blast radius.** The row's mechanism was wrong: `data_connection_definitions` has NO production reader (its getter is test-only) — connection actors are created solely from the `Connections` map in an instance's own flattened config via `EnsureDclConnections`, so there was never a store→actor path to repair. The real defect: that ensure was the CALLER'S job and only 2 of 4 instance-start paths did it (boot batch ✅, `ApplyDeployment` ✅, `StartStoredInstance` ❌ — both the row-67 ensure-started and the row-78 sweep — and `HandleEnableResult` ❌; the two misses are exactly the two paths added later). So the bug was NOT replication-specific: an ordinary unreplicated site that boots with an instance disabled and enables it later hit the same dead end, because the boot load reads only enabled configs. Fix is structural: `EnsureDclConnections` moved INTO `CreateInstanceActor` (the documented single creation funnel), before the already-exists guard so the two working paths keep exact semantics; reuses the per-name hash guard so steady state is silent; a fifth path cannot forget. Store-level sweep pass explicitly REJECTED (memo §7.4): it wouldn't have fixed this (the table is empty on deploy-only sites) and the unguarded form ping-pongs the hash slot against the deploy path, dropping live subscriptions every tick. Fail-before by behavioral revert: 4 of 7 new tests fail (sweep-started, reconcile-ensure-started, enable-after-disabled-boot, exactly-once-across-sweeps). SiteRuntime 633/633, DCL 320/320. Sibling gaps found during the fix recorded as rows 89/90. | Proven live 2026-08-16 by the row-78 close-out probe (sweep started `rig-probe-1`/`soak-notify-1` but every tag subscribe failed for 6+ min with `Unknown connection: OPC PLC Simulator — will retry`; a second pair restart healed it) | Closed 2026-08-17. **Live-proven on the rig same day** in the exact original failure sequence (pair stopped, node-a `site-localdb.db` quarantined as `.probe85-quarantine-2026-08-17`, pair restarted): reconcile "0 fetched" at 08:29:59, then at the 08:30:54 sweep tick `Created DCL connection OPC PLC Simulator` fired ALONGSIDE the two self-heal Warnings (pre-fix they appeared alone), zero `No connection actor`/`Unknown connection` lines on either node, all three `rig-probe-1` tags Good with live values ~2 min after restart — NO second restart; steady-state silence confirmed (0 Created/Updated lines across subsequent ticks); replication reseeded (`localDbReplicationConnected: true`, backlog 0), site-b/c still correctly omit the fields. | — |
|
||||
| 86 | ~~**CLI and UI disagree on deployment site-scoping for Administrators**~~ **RESOLVED 2026-08-17 (merge `ad0100f2`, agent commit `9f67147e`) — Administrator = system-wide is the intended semantic; the UI was the defect.** The rule was already documented on BOTH sides (`ManagementActor.cs` ~:522 "Users with the Administrator role, or system-wide Deployer, are not restricted"; `SiteScopeService`'s own XML doc "any Admin/Design user — is system-wide") and enforced at 6+ ManagementActor sites — only `SiteScopeService.ResolveAsync` ignored it. It now treats an Administrator as system-wide regardless of `SiteId` claims, read via explicit `FindAll(RoleClaimType)` + `OrdinalIgnoreCase` (NOT `IsInRole`, which compares ordinally and needs a `roleType` the harness identities don't set); an Administrator's site claims aren't collected at all, preserving *system-wide ⇒ empty permitted-id set*. All 15 UI consumer call sites funnel through `ResolveAsync` (surveyed). Fail-before verified: 6/6 new Administrator pins failed with only `SiteScopeService.cs` reverted; `DesignerWithSiteClaims_IsStillScoped` pins that the bypass was NOT widened to Designer (the XML doc's "Design user" claim was inaccurate — corrected). | Found by the 2026-08-17 Low-tail sweep; fixed with the row-37 consolidation | Closed 2026-08-17. | — |
|
||||
| 87 | ~~**CLI deployment scoping is non-transactional and unbounded**~~ **RESOLVED 2026-08-17 (merge `ad0100f2`, agent commit `9f67147e`), with the row-37 consolidation.** Site grants now ride as `SiteIdScope` resolved inside the query's join — the whole-table `GetAllInstancesAsync` read is gone (pinned by `..._ReadsNoInstancesAtAll`, a strictly stronger replacement for the old "bulk load exactly once" pin, consciously migrated). An unparseable permitted-site id is *dropped* (narrows the scope) rather than discarding the grant (which would widen to fleet-wide). `PageSize <= 0` no longer means unlimited: it takes the CLI's own default 50, capped at `MaxPageSize` 1000. | Found by the 2026-08-17 Low-tail sweep; fixed with the row-37 consolidation | Closed 2026-08-17. | — |
|
||||
| 88 | ~~**Unparseable CLI `--status` silently means "no filter"**~~ **RESOLVED 2026-08-17 (merge `ad0100f2`, agent commit `9f67147e`).** An unrecognized non-empty status now produces a `ManagementError` naming the valid values (row-46 principle: loud failure over silent empty/full results) — enforced actor-side so HTTP API callers get it too. Case-insensitive parse retained; numeric strings rejected via `Enum.IsDefined` (`Enum.TryParse` accepts `"7"`). CLI help text documents the valid set; deliberately NOT `AcceptOnlyFromAmong`, whose case-sensitive matching is a live minor defect on `audit query --status` (row 99). **Live-probed on the rig:** `deploy status --status Sucess` → `Unknown deployment status 'Sucess'. Valid values: Pending, InProgress, Success, Failed.` | Found by the 2026-08-17 Low-tail sweep | Closed 2026-08-17. | — |
|
||||
| 88 | ~~**Unparseable CLI `--status` silently means "no filter"**~~ **RESOLVED 2026-08-17 (merge `ad0100f2`, agent commit `9f67147e`).** An unrecognized non-empty status now produces a `ManagementError` naming the valid values (row-46 principle: loud failure over silent empty/full results) — enforced actor-side so HTTP API callers get it too. Case-insensitive parse retained; numeric strings rejected via `Enum.IsDefined` (`Enum.TryParse` accepts `"7"`). CLI help text documents the valid set; deliberately NOT `AcceptOnlyFromAmong`, whose case-sensitive matching is a live minor defect on `audit query --status` (row 99). **Live-probed on the rig:** `deploy status --status Sucess` → `Unknown deployment status 'Sucess'. Valid values: Pending, InProgress, Success, Failed.` **Postscript 2026-08-17 (Wave 6, `16056f8e`): the "numeric strings rejected via `Enum.IsDefined`" claim was WRONG** — `Enum.IsDefined(parsed)` is true for any valid ordinal, so `--status 2` silently resolved to `InProgress`. Fixed with the same `LooksNumeric` first-character guard the row-99 strict parser uses; live-probed: `deploy status --status 2` now fails loud naming the valid values. | Found by the 2026-08-17 Low-tail sweep | Closed 2026-08-17. | — |
|
||||
| 89 | ~~**Connection-only artifact deploy does not reach the peer node's DCL**~~ **RESOLVED 2026-08-17 (merge `6cfb6a79`, agent commit `c57e74e6`) — premise corrected TWICE, then fixed.** (1) The standby's DCL is **EMPTY**, not "serving the old endpoint": `DeploymentManagerActor` (role-scoped singleton) is the sole sender of `CreateConnectionCommand` and only populates its own node's `dcl-manager`. The real defect was **create-time and not replication-specific**: `_createdConnections` is per singleton *incarnation*, and `data_connection_definitions` had NO production reader — so any restart/failover rebuilt every connection from the flattened-config snapshots and silently discarded newer connection-only artifact deploys (identical loss on an unreplicated single-node restart). (2) Arbitration decided and shipped: **the definitions store wins wherever it has an opinion; flattened config is authoritative only for names the store never owned** (deploy-only sites unchanged — memo §7.4's objection). A `_connectionDefinitions` overlay refreshes from `ReadStoreSnapshotAsync` at boot AND every sweep tick, applied before instance creation; ping-pong killed by construction (every ensure path resolves the *winner* before hashing, so one config per hash slot). Fail-before: 5/8 new `DclConnectionArbitrationTests` fail with the overlay inert. Decision memo: `docs/plans/2026-08-17-dcl-connection-arbitration-decision.md` (records the accepted residuals: re-adds land at next instance start, not instantly — mirrors the removal's own "bad quality until redeploy" semantic, pinned by test). The discovery that a *live* connection actor is never reconfigured in place on ANY node is a separate pre-existing defect → row 97. **Live rig probe at wave close-out (post-redeploy):** the singleton node logged exactly ONE boot-time `Created DCL connection OPC PLC Simulator` with zero update churn across later sweep ticks (no ping-pong from the overlay), the standby stayed silent (topology as corrected), all three rig-probe-1 tags Good, site-a replication connected / backlog 0. | Found by the 2026-08-17 row-85 fix session | Closed 2026-08-17. | — |
|
||||
| 90 | ~~**No removal path for a connection deleted on the peer**~~ **RESOLVED 2026-08-17 (merge `6cfb6a79`, agent commit `c57e74e6`), with row 89.** Both removal paths now share `RemoveConnectionCentrally` + an in-memory tombstone that `EnsureDclConnections` honors — which also closed a pre-existing bug where an artifact removal was **undone by the very next instance start**. The sweep detects a previously-definition-sourced row's disappearance via its own overlay bookkeeping. Accepted residuals, documented in the decision memo: tombstones are not durable across a restart (by decision — matches the long-standing restart-resurrection semantic), and a delete is undetectable on a node that never adopted the row (the stronger "non-empty table ⇒ complete set" inference would stop live connections on a guess; deliberately not taken). | Found by the 2026-08-17 row-85 fix session | Closed 2026-08-17. | — |
|
||||
| 91 | ~~**`SiteScriptCompileCacheTests.GetOrAdd_SameCodeAndGlobals_ComputesOnce` is parallel-load fragile**~~ **RESOLVED 2026-08-17 (merge `106e467d`, agent commit `26e3a1b9`) — row-74 pattern applied (exemplar `98c94588`), and wider than the named test.** Two habits: a per-test-instance guid key token (no other class can collide with or serve this test's entries, making `factoryCalls`/`Assert.Same` exact and owned) + one-sided bounds on the process-global counters (`>= baseline + n`; safe because all `Clear()` callers share the serialized collection). Four assertions changed, two got STRONGER (`GetOrAdd_SameCode_DifferentGlobals` had a second absolute pair; `Overflow_StaysWithinTheBound` gained an eviction probe over its own keys — a slack bound is not proof a sweep ran; its `ForeignEntrySlack = 512` is a judgement call, the eviction probe is the load-bearing assertion). Sibling absolutes in the same class and in `DeploymentWarmThenGateTests` fixed too. SiteRuntime.Tests run twice at full parallelism, 633/633 both. The underlying process-global `Clear()` cross-talk hazard remains → row 100. | Found by the 2026-08-17 row-85 fix session | Closed 2026-08-17. | — |
|
||||
@@ -89,10 +89,11 @@ All 7 fix-now items landed via PLAN-04/05/06/07/08 (verified in review 08 round
|
||||
| 94 | **`SqliteCapabilityTests` is a deliberate tripwire — do not "fix" it by weakening the assertion** *(Low, standing note)* | Row-81 fix session (2026-08-17) | The test FAILS BY DESIGN if a future SQLite bump ships `dbstat`/`sqlite_dbpage` support. The correct response at that point is to REPLACE the sampled page-layout model with exact `dbstat` measurement (retiring rows 92/93 with it), not to relax the test. The test's own comment says the same. | `SqliteCapabilityTests` fails after a SQLite/provider bump |
|
||||
| 95 | ~~**The leaked 100 ms-tick test-actor pattern is systemic — three more classes carry it**~~ **RESOLVED 2026-08-17 (merge `106e467d`, agent commit `34a8079d`).** All three classes live in AuditLog.Tests (premise correction: none in SiteCallAudit.Tests). `Watch`/`Stop`/`ExpectTerminated` replicated from `da23f2dc` at every creation site: 11 in `AuditLogPurgeActorTests`, 10 in `SiteAuditReconciliationActorTests`, 2 in `OutageReconciliationTests`. Second premise correction: `OutageReconciliationTests` DID call `Sys.Stop` — but fire-and-forget AND from the last line of the `await using` block, so an in-flight tick could still be writing to MSSQL during read-back and any earlier assertion failure skipped the stop entirely; stops moved before the read-backs, plus the `AuditLogIngestActor` that test 2 was discarding outright (holding a disposed provider) is now stopped too. Known lesser sibling habit left in place: EventStream probes are never unsubscribed (bounded by the per-test ActorSystem — noted, not a row). AuditLog.Tests 383/383. | Row-82 fix session (2026-08-17) | Closed 2026-08-17. | — |
|
||||
| 96 | ~~**xunit parallelism is uncapped almost everywhere**~~ **RESOLVED 2026-08-17 (merge `106e467d`, agent commit `d2d4db49`) — policy made and shipped: per-suite caps for the resource-heavy suites, pure unit suites stay uncapped.** The four MSSQL-fixture users (`ConfigurationDatabase.Tests`, `AuditLog.Tests`, `SiteCallAudit.Tests`, `Host.Tests` — verified before capping) get `xunit.runner.json` with `maxParallelThreads: 4`, **deliberately matching row-82's `sp_getapplock` slot default** so the in-process and cross-process limits agree; no collections serialized. `IntegrationTests` (already fully serialized) and `CentralUI.PlaywrightTests` (already `maxParallelThreads: 1`, stricter than needed) left alone. Cap verified live (216% CPU uncapped vs 145% capped on ConfigurationDatabase.Tests); cost is nil — these suites are latency-bound on MSSQL/sockets (14–25% CPU), durations unchanged within noise. Policy documented in `docs/test_infra/test_infra.md` → *Test Parallelism Policy* (two-axis model, per-suite table, default for new projects), cross-linked from `test_infra_db.md`. Found-not-fixed: `StoreAndForward.Tests/xunit.runner.json` is a no-op (pins the default `parallelizeTestCollections: true`) — harmless, tidy on next touch. | Row-82 fix session (2026-08-17) | Closed 2026-08-17. | — |
|
||||
| 97 | **A live DCL connection actor is never reconfigured in place — on ANY node** *(Medium, availability)* | Found by the 2026-08-17 rows-89/90 fix session; documented in `docs/plans/2026-08-17-dcl-connection-arbitration-decision.md` | `DataConnectionManagerActor.HandleCreateConnection` is create-only (`if (_connectionActors.ContainsKey(name)) { warn; return; }`), so `EnsureDclConnection`'s documented "re-issues the create so the DCL adopts the new configuration" has NEVER been true — including on the deploy-handling node — while the hash cache recorded the change as applied. The rows-89/90 fix made this *announced* (Warning, once per changed config) instead of silent, but a changed endpoint still only takes effect at instance redeploy or singleton restart. Fix sketch (from the memo): `ReconfigureConnectionCommand` handled *inside* `DataConnectionActor`, reusing the primary→backup failover adapter swap (dispose → `_factory.Create` → bump `_adapterGeneration` → re-enter the connect cycle, which re-issues from `_subscriptionsByInstance`). The naive stop-and-recreate is DESTRUCTIVE: subscriptions live in `DataConnectionActor` and `InstanceActor` cancels its subscribe-retry timer on first success, so nothing re-subscribes — it would trade "stale endpoint, data flowing" for "correct endpoint, no data until redeploy". | Next DataConnectionLayer session, or first field report that an artifact-deployed endpoint change didn't take effect on a live connection |
|
||||
| 98 | **`EnforceSiteScopeForInstance` is fail-open for nonexistent instance ids** *(Low, authorization shape)* | Found by the 2026-08-17 rows-37/86-88 fix session (Agent A survey) | A site-scoped user querying a nonexistent instance id passes the scope gate rather than being denied (`ManagementActor.cs` ~:540); `EnforceSiteScopeForIdentifier` shares the shape. Harmless for the deployment read (no records reference a missing instance), but fail-open is the wrong default for an authorization gate — a future caller reusing it for something with side effects inherits the hole. | Next ManagementActor authorization touch, or any new caller of either enforce helper |
|
||||
| 99 | **CLI `audit query --status` client-side validation is case-sensitive** *(Low, UX)* | Found by the 2026-08-17 rows-37/86-88 fix session | `AuditCommands` uses `AcceptOnlyFromAmong` with capitalised values, and that check matches case-*sensitively* — `audit query --status delivered` fails client-side while the server would accept it. The row-88 fix deliberately avoided introducing the same trap on `deploy status` (documented valid set in help text + case-insensitive server-side check instead). Fix: same treatment for the audit command group. | Next CLI audit-command touch |
|
||||
| 100 | **`SiteScriptCompileCache.Clear()` is process-global — test classes wipe each other's entries** *(Low, test-infra hazard)* | Found by the 2026-08-17 row-91 fix session | Any test calling `Clear()` evicts entries other parallel classes just inserted. Harmless to correctness now that row 91 removed the absolute-count assertions (all `Clear()` callers share a serialized collection), but it remains latent cross-talk: a future test outside that collection calling `Clear()` mid-run degrades other classes to recompiles (perf noise) and re-opens the row-91 class if anyone reintroduces absolute counts. A `ClearKeys(prefix)` or an instance-scoped cache seam would remove the last shared mutable state. | Next `SiteScriptCompileCache` API touch, or a flake with the row-91 fingerprint from a class outside the serialized collection |
|
||||
| 97 | ~~**A live DCL connection actor is never reconfigured in place — on ANY node**~~ **RESOLVED 2026-08-17 (merge `9dbda4a2`, agent commit `7c1268d8`) — the memo's §6 sketch shipped, plus two things the sketch missed.** `EnsureDclConnection` now has three outcomes off its one hash slot: unchanged → nothing; new → create; **changed → `ReconfigureConnectionCommand`** (arbitration/tombstones byte-for-byte untouched). The manager forwards to the owning actor (unknown name falls back to create); `DataConnectionActor` handles it in **all three** lifecycle states and never stashes it, via a new shared `SwapAdapter` helper now also used by both failover sites. The sketch's two gaps, found in implementation: **endpoint selection must be re-armed to Primary** (a connection sitting on its old *backup* would otherwise dial the new backup first), and the deliberate link drop must **suppress the unstable-disconnect accounting** (else the swap itself could trip a failover). Also a premise-sharpening: "reconfigure while mid-connect" is the *primary* case, not an edge — a wrong endpoint loops in `Connecting` forever and never reaches `Connected`. Fail-before: 5/7 new `DataConnectionReconfigureTests` + 3 SiteRuntime ensure-path tests fail on the reverted behavior. **Live rig probe (post-redeploy, round trip):** connection 1's config changed (operationTimeoutMs 15000→16000→back) fired exactly TWO in-place reconfigures on the active node — `Reconfiguring DataConnectionActor for OPC PLC Simulator in place` → `Reconnected successfully on Primary endpoint` in 45 ms → `Re-subscribing 2 tags` — with all rig-probe-1 tags Good after each, the standby silent (0 reconfigure lines, topology as designed), and the old dead-end "has a new configuration" Warning never appearing. Rig config restored to original. | Found by the 2026-08-17 rows-89/90 fix session; memo `docs/plans/2026-08-17-dcl-connection-arbitration-decision.md` (§9 postscript appended) | Closed 2026-08-17. | — |
|
||||
| 98 | ~~**`EnforceSiteScopeForInstance` is fail-open for nonexistent instance ids**~~ **RESOLVED 2026-08-17 (merge `152ec85e`, agent commit `f33f25a1`) — premise corrected: the defect had FIVE instances, not two, and the worst three were worse than "harmless".** Beyond the two named helpers, the same `if (target != null) EnforceSiteScope(...)` shape sat **inline** in `HandleGetInstance` (:879), `HandleGetSite` (:1702) and `HandleGetDataConnection` (:1844) — and those returned `ManagementSuccess{JsonData=null}` for a nonexistent id vs `ManagementUnauthorized` for an out-of-scope one, i.e. a working **id-enumeration oracle across the whole fleet** for a site-scoped caller. All five now route through one shared `DenyUnresolvedTargetIfScoped`: a scoped caller's unresolvable target is DENIED, indistinguishable from out-of-scope (no existence leak); system-wide callers (Administrator / system-wide Deployer) are untouched. Also checked: `Instance.SiteId` is non-nullable `int`, so no secondary fail-open path through `EnforceSiteScope(null)`. Fail-before: 5 of the 16 new ManagementActorTests fail on the old code. | Found by the 2026-08-17 rows-37/86-88 fix session (Agent A survey) | Closed 2026-08-17. | — |
|
||||
| 99 | ~~**CLI `audit query --status` client-side validation is case-sensitive**~~ **RESOLVED 2026-08-17 (merge `152ec85e`, agent commit `f33f25a1`) — premise corrected: "the server would accept it" was FALSE, and removing the client gate alone would have been a regression.** The server-side parser (`AuditQueryParamParsers.ParseEnumList`) is documented-and-tested **lax** — it silently drops unparseable values and collapses to null (filter unconstrained), i.e. exactly the row-46/88 silent-wrong-answer trap. Fix shipped both halves: **seven** `AcceptOnlyFromAmong` gates removed across `audit query`/`audit export` (`--channel`/`--kind`/`--status` on both, plus `export --format`, which rejected `CSV` the server lowercases anyway), valid sets moved to help text, and a new **strict** `ParseEnumListStrict` (throws naming the valid values; `LooksNumeric` guard rejects ordinals) wired into both ManagementService audit endpoints. The lax parser deliberately survives for the CentralUI drill-in/export query strings (bookmarked fragments must render, not fault — documented as non-reconcilable). Left alone with reasons in code: `--direction up|down` (no server parse to defer to), `--format json|table` (CLI-local), NotificationCommands' literals (unsurveyed). Four pre-existing tests were passing *vacuously* (no-credentials exit codes) and now pin the real contract; fail-before 21/26. **Found-and-fixed follow-on (orchestrator, `16056f8e`):** row 88's own `ParseDeploymentStatus` accepted ordinals (`--status 2` → InProgress) — same `LooksNumeric` guard applied + 3 tests. **Live rig probes (post-redeploy):** `audit query --status delivered` returns rows; `--status Nonsense` and `--status 3` both fail loud naming the 8 valid values; `deploy status --status 2` fails loud naming the 4 valid values. | Found by the 2026-08-17 rows-37/86-88 fix session | Closed 2026-08-17. | — |
|
||||
| 100 | ~~**`SiteScriptCompileCache.Clear()` is process-global — test classes wipe each other's entries**~~ **RESOLVED 2026-08-17 (merge `8a9a8d29`, agent commit `84b405ff`) — premise corrected: there were ZERO production callers of `Clear()`** (the only production entry point is `ScriptCompilationService.CompileCore` → `GetOrAdd`; `Clear()` existed solely for tests), so neither register option was taken as written — `Clear()` was **deleted outright** and the cache became an instance-scoped `sealed class` owned by `ScriptCompilationService`. The N1 "one compile per body per process" property now rests on `AddSiteRuntime` registering that service as a **DI singleton** — a lifetime that became load-bearing, so new `SiteRuntimeServiceRegistrationTests` pins it (a Scoped/Transient downgrade would reintroduce WP3.1's recompile storms with no behavioral test failing). The 37 test sites doing `new ScriptCompilationService(logger)` each silently get their own cache — isolation for the whole assembly, not just the row-91 classes. Row-91 workarounds RETIRED, not preserved: serialized collection removed from all three classes, exact `Hits`/`Count` assertions restored, guid key token gone, `ForeignEntrySlack` 512→8; new `SeparateCaches_DoNotShareEntriesOrCounters` pins the scoping. SiteRuntime.Tests green three consecutive full-parallel runs. | Found by the 2026-08-17 row-91 fix session | Closed 2026-08-17. | — |
|
||||
| 101 | **`ApplyingArtifacts_RemovedDataConnection_StopsLiveDclConnection` flaked once — suspected overlay-sweep read racing the artifact-apply write** *(Low, flake + possible production race — UNVERIFIED hypothesis)* | Observed by the 2026-08-17 row-100 fix session (one failure in its first SiteRuntime.Tests run, on the pristine `dcfcf822` code path; passed in isolation and in 3 subsequent full runs + this wave's full suite) | The test received `RemoveConnectionCommand("PlcA")` where `"PlcGone"` was expected — and `RemoveConnectionCommand("PlcA")` can only come from `ApplyConnectionDefinitionOverlay`'s disappearance detection, i.e. the periodic sweep's `ReadStoreSnapshotAsync` racing the artifact-apply write and momentarily not seeing `PlcA`'s definitions row. If that mechanism is real it is not test-only: in production the same race would transiently remove a LIVE connection on a sweep tick that straddles an artifact apply (self-healing at the next tick via re-create, but a needless data gap). Verify the read/write ordering inside `DeploymentManagerActor` first — if the store read and the apply are on the same actor's timeline the hypothesis is wrong and this is a pure test-fixture race. | Next SiteRuntime/DeploymentManagerActor session, or a second occurrence of this fingerprint |
|
||||
| 68 | ~~**`deploy.sh` run from a git worktree silently rehomes every node's bind-mount data dir**~~ **RESOLVED 2026-08-15 (guard added in the same close-out).** The compose files' bind mounts are relative (`./site-a-node-a/data`), so a deploy from a worktree checkout mounts the worktree's copies — the rig keeps running and looks healthy, but its SQLite stores split from the main checkout's and everything written there is orphaned by the next deploy from main. Bit the rows-46–53 close-out live: the seeding agent's `rig-probe-1` deploy persisted into worktree-homed dirs and "vanished" after the orchestrator's redeploy (no tombstone, no product bug — LWW/tombstone forensics ruled out deletion; the row sat in `.claude/worktrees/…/docker/site-a-node-a/data/site-localdb.db` the whole time). Both `docker/deploy.sh` and `docker-env2/deploy.sh` now refuse to run from a `.claude/worktrees/` path (override: `SCADABRIDGE_ALLOW_WORKTREE_DEPLOY=1`). | Found during the rows-46–53 close-out | Closed 2026-08-15. | — |
|
||||
|
||||
## Resolved (verified against the code 2026-07-10)
|
||||
|
||||
Reference in New Issue
Block a user