Files
lmxopcua/docs/plans/2026-08-17-perf-followups-r3.md
T

510 lines
28 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Perf Follow-ups Round 3 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development
> to implement this plan task-by-task (per the standing preference: Opus implementers,
> classification-driven review chains).
**Goal:** Close the actionable items in the round-2 follow-up register
(`docs/plans/2026-08-17-perf-followups-r2.md` §"Follow-up register") and run the
still-unexercised local live gates (FetchAndCache legs, alarm-bearing surgical ratchet,
ACL-enabled enforcement, docker-dev first-POST-404 catch), on branch `perf/followups-r3`
off master merge `49d05bc2`.
**Architecture:** Two surgical code fixes in already-reviewed code (the ACL cache's
single-shot CAS becomes a monotonic-max loop; S7 read-pass health becomes worst-state-wins
aggregation), one doc-staleness fix found during exploration, and four live gates on the
local docker-dev rig (OrbStack) sequenced on one rig session. No new subsystems.
**Tech Stack:** .NET 10 / C#, xUnit + Shouldly, docker-dev rig (OrbStack), Traefik v3.1
access log, Client.CLI, Chrome browser automation for AdminUI authoring.
**Out of scope (recorded, deliberately not here):**
- **TM/CT decode at a non-zero device number** — needs a real S7 CPU; the test exists and
is env-gated on `S7_TIMER_NUMBER_ADDRESSING=1`. Nothing to do until hardware exists.
- **Cross-process plant-vs-reap race on the shared SQL server** — accepted in r2; the 24 h
age floor is the mitigation. No work.
- **Modbus per-tag health last-writer-wins** — Task 2 fixes S7 only (the recorded item).
Modbus has the same shape (`ModbusDriver.ReadAsync:493/511/518` + the coalesced path);
fixing it here doubles the blast radius of a watchdog-adjacent change. Recorded in this
round's register as a deliberate asymmetry.
- **S7 write-path per-item health** (`S7Driver.cs:1575/1633/1645/1654`) — same
last-writer-wins shape on `WriteAsync`; out of scope for the same reason, recorded.
- Historian end-to-end live gate — needs the `wonder-sql-vd03` VPN (user-driven).
- HistorianGateway batch `WriteLiveValues` RPC — cross-repo.
- Load-scale throughput — needs a load harness; not defined enough to gate.
**Execution rules (unchanged from prior rounds):** shared checkout — explicit-pathspec
`git add`/`git commit -- <files>` only (never a bare commit); never checkout/switch/reset/
stash/clean; retry on `index.lock` and CS2012/MSB3030 build races; reviewers judge commits
via `git show`; falsification mutations applied AND reverted via Edit (never git), run
against a freshly built binary (never `--no-build` after a mutation), and labeled honestly
("pin, not red-first" where the behaviour predates the test; "probabilistic red" where the
mutation trips a race). Live-gate rig work is serial — one rig session, Tasks 4→5→6→7 in
that order, images built from the branch tip AFTER Tasks 1–3 land so the gates also
regression the new code.
---
### Task 1: ACL `InvalidateIfStale` — monotonic-max CAS loop (publication never deferred)
**Classification:** high-risk
**Estimated implement time:** ~4 min
**Parallelizable with:** Tasks 2, 3
**Files:**
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/SessionAclDecisionCache.cs:453-461`
(+ the XML remarks block `:430-452`)
- Test: `tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/SessionAclDecisionCacheTests.cs`
**Context.** The r2 register records: the single-shot CAS in `InvalidateIfStale` can defer
stamp publication by one call. Two callers read the same `seen` baseline; gen-N wins the
CAS, gen-N+1's CAS fails against the moved stamp and falls out with **no retry** — N+1 is
not published on that call, so its `TryGet` misses and its `Set` is dropped at `:223`.
Benign (verdict safety lives in the per-shard generation gate), but it costs a lost store
exactly in the post-deploy window. Current code:
```csharp
private void InvalidateIfStale(long generation)
{
var seen = Interlocked.Read(ref _generation);
if (generation <= seen) return;
if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen) ClearAll();
}
```
**Step 1: Write the racing pin (red against current code — probabilistic red, say so).**
```csharp
[Fact]
public void Concurrent_lookups_at_mixed_generations_always_publish_the_highest()
{
// The single-shot CAS defect: when gen-1 wins the shared-baseline race, gen-2's CAS
// fails and is never retried, leaving Generation == 1. Red is probabilistic per
// round; 200 barrier-synchronised rounds make it effectively certain.
for (var round = 0; round < 200; round++)
{
var cache = /* fresh cache, generation seed -1 */;
using var barrier = new Barrier(2);
var low = Task.Run(() => { barrier.SignalAndWait(); /* TryGet at generation 1 */ });
var high = Task.Run(() => { barrier.SignalAndWait(); /* TryGet at generation 2 */ });
Task.WaitAll(low, high);
cache.Generation.ShouldBe(2L,
$"round {round}: the newer generation must publish even when it loses the CAS race");
}
}
```
Use the ordinary `TryGet` lookup path (as the straggler test at `:286` does), NOT
`AdvanceGenerationForTest` — the defect lives in the lookup path's invalidation.
**Step 2: Run it, observe red.** `dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~Concurrent_lookups_at_mixed_generations"` — expect FAIL within a few rounds. If 200 rounds pass green against unmodified code, the test is not exercising the race — fix the test, do not proceed.
**Step 3: Implement the monotonic-max loop.**
```csharp
private void InvalidateIfStale(long generation)
{
while (true)
{
var seen = Interlocked.Read(ref _generation);
if (generation <= seen) return; // guard re-checked per attempt
if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen)
{
ClearAll();
return;
}
}
}
```
Invariants (update the XML remarks `:430-452` to say all of this):
- The `<=` guard is INSIDE the loop against a fresh read — a straggler that lost a CAS
re-reads the newer stamp and exits; hoisting the guard reintroduces the 98d6a190 defect.
- Only the thread whose CAS moved the stamp clears (elected clear preserved). A rollover
now legitimately clears once per distinct generation (N then N+1) — bounded by
generations, not request volume.
- The loop self-terminates: each iteration exits via the guard or CASes against a strictly
larger `seen`, and `_generation` only rises.
- The `-1` seed still admits generation 0.
**Step 4: Run the new test + the full cache suite.** All 18 green (17 existing + 1 new),
especially `A_straggler_from_an_older_generation_neither_clears_nor_rolls_the_stamp_back`
(`:286`) and `No_hit_ever_returns_a_verdict_from_another_generation` (`:363`).
**Step 5: Falsify.** Via Edit, revert the loop to the single-shot form (fresh build, not
`--no-build`) → the new test must go red (probabilistic; report the failing round). Revert
via Edit. Separately, move the `<=` guard outside the loop → the straggler test at `:286`
must go red. Revert.
**Step 6: Commit.**
```bash
git add -- src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/SessionAclDecisionCache.cs \
tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/SessionAclDecisionCacheTests.cs
git commit -m "fix(acl): make InvalidateIfStale a monotonic-max CAS loop — stamp publication is never deferred" -- <same paths>
```
---
### Task 2: S7 read-pass health — worst-state-wins aggregation
**Classification:** high-risk
**Estimated implement time:** ~5 min
**Parallelizable with:** Tasks 1, 3
**Files:**
- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs` — per-item health sites
`:647` (per-tag loop), `:828` (multi-var decode loop), `:941` (span member loop),
`RecordReadFailure:1010-1046`, plus one end-of-pass write in `ReadAsync`
- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7SpanReadParityTests.cs`
(flip the `:423` pin; doc-comment `:389-393` rewritten; new tests)
**Context.** The r2 register records: per-item decode health is last-writer-wins — a mixed
chunk reports whatever the LAST item's outcome was, so `Healthy` can be reported for a
pass that had failures (pinned at `:396-423`), and the `:1023` access-denied `Faulted` can
be overwritten by a later good item. Fix = aggregate one outcome per `ReadAsync` pass.
**Watchdog contract (MUST hold — CLAUDE.md comms-loss section):** `Degraded` means
connected-but-some-bad; `LastSuccessfulRead` advances on ANY success and is preserved on
failure. The aggregate write must advance `LastSuccessfulRead` when ANY item succeeded,
even when the state lands `Degraded`/`Faulted` — gating it on `Healthy` turns the fix into
a false-positive reconnect storm.
**Step 1: Red tests first** (the flip IS red-first; the rest are new):
1. Flip `:423`: `One_undecodable_multi_var_item...` now expects `Degraded` (rename to say
worst-state-wins; rewrite doc-comment `:389-393` — the last-writer-wins pin is being
deliberately retired).
2. New: a mixed multi-var pass reports `Degraded` AND `LastSuccessfulRead` is advanced
(non-null / newer than the pre-pass value).
3. New: span path — one bad member decode then a good member ⇒ `Degraded` (today: Healthy).
4. New: an access-denied `PlcException` on one tag followed by a success ⇒ state stays
`Faulted` (ordering `Healthy < Degraded < Faulted`), `LastSuccessfulRead` advanced.
Use the per-tag path if the multi-var fake can't produce a per-item PlcException.
5. Must-stay-green: `:326` (`A_failing_multi_var_chunk_falls_back...`, asserts Healthy at
`:346`) and `:432` (short answer, Healthy at `:447`) — a multi-var REQUEST fault whose
tags are re-served Good by the per-tag tail is NOT a failed outcome. The chunk-fault
catch (`:806-814`) stays outside the aggregate.
**Step 2: Run, observe red on 1–4** (each for its own reason; report per-test).
**Step 3: Implement.** A small per-pass accumulator threaded through the pass:
```csharp
private sealed class ReadPassOutcome
{
public bool AnySuccess;
public DriverState WorstFailure = DriverState.Healthy; // Healthy = none recorded
public string? FirstError; // first failure is causal
public void RecordSuccess() => AnySuccess = true;
public void RecordFailure(DriverState state, string message)
{
if (Rank(state) > Rank(WorstFailure)) WorstFailure = state;
FirstError ??= message;
static int Rank(DriverState s) => s switch
{
DriverState.Faulted => 2, DriverState.Degraded => 1, _ => 0
};
}
}
```
- The three success sites (`:647`, `:828`, `:941`) call `outcome.RecordSuccess()` instead
of assigning `_health`.
- `RecordReadFailure` takes the outcome, records `(Degraded|Faulted, message)` into it
instead of assigning `_health`, and KEEPS its `_plcDead` / `MarkConnectionDeadIfFatal`
side effects immediate. It still returns the per-item status code.
- One `_health` write at the end of the `ReadAsync` try body (NOT `finally` — the OCE
arms at `:649`/`:801`/`:920` rethrow and must skip it):
```csharp
_health = new DriverHealth(
outcome.WorstFailure == DriverState.Healthy
? DriverState.Healthy : outcome.WorstFailure,
outcome.AnySuccess ? now : _health.LastSuccessfulRead,
outcome.WorstFailure == DriverState.Healthy ? null : outcome.FirstError);
```
- Untouched: the `:567` connect-failure early return; `HandlePollError` (`:1980`); the
entire write path; init/shutdown sites. `ReadSpanAsync` / `ReadLeftoversByMultiVarAsync`
gain an outcome parameter — signature changes, no logic movement.
**Step 4: Run the full S7 suite** (`dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests`)
— all green including the parity anchor `:202` (all-members-fail ⇒ Degraded both paths).
**Step 5: Falsify.** Via Edit: (a) make the end-of-pass write use the LAST recorded state
instead of the worst → tests 1/3/4 red; (b) gate `LastSuccessfulRead` advancement on the
state being Healthy → test 2 red. Fresh builds. Revert via Edit.
**Step 6: Commit** (explicit pathspec, message
`fix(s7): read-pass health is worst-state-wins — a mixed pass can no longer report Healthy`).
---
### Task 3: Fix stale `OTOPCUA_CONFIG_MODE` docs
**Classification:** trivial
**Estimated implement time:** ~2 min
**Parallelizable with:** Tasks 1, 2
**Files:**
- Modify: `docs/Configuration.md` (~line 253) — "Flip only the site nodes with
`OTOPCUA_CONFIG_MODE=FetchAndCache` at `docker compose up`" is stale
- Modify: `CLAUDE.md` — same sentence in the `ConfigSource`/`ConfigServe` section
- Check-and-fix if matched: `grep -rn "OTOPCUA_CONFIG_MODE" docs/ docker-dev/ CLAUDE.md`
**Spec.** Phase 4 made `FetchAndCache` mandatory on driver-only nodes
(`ConfigSourceOptionsValidator` fails host start on `Direct`), and the rig's four site
nodes are hardcoded `ConfigSource__Mode: "FetchAndCache"` in
`docker-dev/docker-compose.yml:572-579/691-696/787-792/858-863`. The env var appears
nowhere in the compose file; setting it is a no-op. Rewrite both sentences to state: the
site nodes run `FetchAndCache` unconditionally (mandatory for driver-only nodes since
Phase 4); central stays `Direct` via its admin role. No review (doc-only). Commit with
explicit pathspec, message `docs(config): OTOPCUA_CONFIG_MODE is gone — site nodes are hardcoded FetchAndCache (Phase 4)`.
---
### Task 4: Live gate — docker-dev first-POST-404 catch + no-body POST binding
**Classification:** live-gate (rig session step 1 — FRESH BOOT window)
**Estimated implement time:** ~10 min wall (mostly rig boot)
**Parallelizable with:** none (rig serial; runs after Tasks 1–3 land, images built from branch tip)
**Files:**
- Possibly modify: `docker-dev/deploy.sh` (the header NOTE about no-body binding),
`docker-dev/README.md` (§"First-POST 404" — mechanism note if caught)
- Evidence: write to the scratchpad; Task 8 folds it into the plan's Outcomes
**Procedure:**
1. Fresh rig: `docker compose -f docker-dev/docker-compose.yml down -v` then
`up -d --build` (build mandatory — the branch tip must be in the images).
2. IMMEDIATELY run `./docker-dev/deploy.sh` (no body). Its 2 s `/health/ready` poll rides
through Traefik during the boot window — every poll is access-logged with
`defaultmode=keep`, and an unmatched request shows an EMPTY `RouterName`.
3. After the deploy completes (or fails), capture
`docker compose -f docker-dev/docker-compose.yml logs traefik` and extract every
non-2xx access-log line from the boot window. **If any 404 appears:** name the
mechanism from `RouterName`/`ServiceName`/timing (the r2 elimination says: AdminUI
cannot 404 that path; backend-down is 503; so 404 = no router matched at that instant
— presumably the window before the dynamic-config file provider loads or before the
health-checked service has a server). Update `docker-dev/README.md` §"First-POST 404"
with the observed mechanism. **If it does not reproduce:** record that honestly —
"did not reproduce under the readiness gate; register item stays open" — do NOT
invent a mechanism.
4. No-body POST binding: with the rig ready, run `./docker-dev/deploy.sh` bare and record
the HTTP status + response body. Determine whether the minimal-API `DeployRequest?`
binds null (and what the endpoint then does) or 400s. Update the header NOTE in
`deploy.sh` from "unverified" to the observed behaviour, and `README.md` if wrong.
5. Leave the rig UP for Task 5.
Commit any `deploy.sh`/`README.md` edits (explicit pathspec). Report: the access-log
excerpt, the no-body verdict, and whether the 404 reproduced.
---
### Task 5: Live gate — FetchAndCache legs on the running rig
**Classification:** live-gate (rig session step 2)
**Estimated implement time:** ~8 min wall
**Parallelizable with:** none (rig serial, after Task 4)
**Procedure** (rig already up from Task 4; a deploy already ran):
1. **Fetch + verify:** `docker compose -f docker-dev/docker-compose.yml logs site-a-1 |
grep -E "Fetched \+ verified artifact"` — expect
`Fetched + verified artifact for {DeploymentId} from http://central-… ({Bytes} bytes).`
2. **Served (central side):** `logs central-1 central-2 | grep "Served artifact"` —
`Served artifact for {DeploymentId} ({Bytes} bytes, {Chunks} chunks)`.
3. **Applied-from-fetch + cached:** `logs site-a-1 | grep -E "applied fetched deployment|cached applied deployment"` — both lines must appear (the `fetched` variant, NOT the
Direct `applied deployment` form).
4. **Boot-from-cache:** `docker compose restart site-a-1`, wait for boot, then
`logs site-a-1 | grep "FetchAndCache boot"` — expect
`restored served state for deployment {Id} (rev {Rev}) from the LocalDb pointer.`
5. **Absence proof:** on site-a-1 logs, assert NO occurrence of `entering Stale`,
`ConfigDb back`, or the Direct-form `applied deployment {Id} (rev` (without
`fetched`). A FetchAndCache node must never reach Stale.
6. Evidence excerpts to the scratchpad for Task 8. Rig stays UP for Task 6.
No commit expected. If any leg fails, STOP and report — that is a real regression, not a
gate formality.
---
### Task 6: Live gate — alarm-bearing surgical ratchet
**Classification:** live-gate (rig session step 3)
**Estimated implement time:** ~12 min wall
**Parallelizable with:** none (rig serial, after Task 5)
**Context.** The surgical-deploys classifier routes an alarm-bearing raw-tag delta
(`Alarm != null` on EITHER side) to `Rebuild`; a whitelisted non-alarm attribute delta
(`Writable` among them) is surgical. Never observed live. The ONLY seam is the applier
summary line (`AddressSpaceApplier.cs:322-326`):
`applied plan (kind={Kind}, …, surgicalRawTags={SurgicalRaw}, …, rebuild={Rebuild})`.
**Procedure:**
1. Pick TWO alarm-free raw tags in the rig's seeded config (browse `/raw` via AdminUI at
`http://localhost:9200`, login disabled — or read the ConfigDb `Tag` rows). Call them
TagS (surgical lever) and TagA (alarm lever).
2. **Surgical control first:** toggle TagS's Access level (Read ↔ ReadWrite) — via the
RawTagModal in the browser, or a direct ConfigDb UPDATE of the tag row (the rig's DB
on `10.100.0.35,14330`; acceptable for a classifier gate, note which route was used).
Deploy via `./docker-dev/deploy.sh`. Expect on central-1:
`kind=AttributeOnly, … surgicalRawTags=1, … rebuild=False`.
3. **Alarm add ⇒ rebuild:** add `"alarm": {"alarmType": "OffNormalAlarm", "severity": 500}`
to TagA's `TagConfig` JSON (raw-JSON textarea — available because the seeded device is
a Galaxy driver, which has no typed editor — or the same direct-DB route). Deploy.
Expect `kind=Rebuild, … surgicalRawTags=0, … rebuild=True`.
4. **Alarm remove ⇒ rebuild:** remove the `alarm` object, deploy, expect `rebuild=True`
again (Alarm != null on the PREVIOUS side also disqualifies).
5. **Restore:** revert TagS's access level, deploy once more (surgical again — also
re-proves the ratchet is per-deploy, not latched).
6. Evidence: the four applier lines (grep `docker compose logs central-1 site-a-1 |
grep "applied plan (kind="` — note which nodes applied). Scratchpad for Task 8. Rig
stays UP for Task 7.
No commit expected. Failure of the expected routing = STOP and report.
---
### Task 7: Live gate — ACL-enabled enforcement on site-a (re-shaped cache)
**Classification:** live-gate (rig session step 4)
**Estimated implement time:** ~20 min wall
**Parallelizable with:** none (rig serial, after Task 6)
**Context.** `Security:Acl:Enabled` has never been exercised live since the r1/r2 cache
re-shape (per-key shards, generation stamp, Task 1's CAS loop). Site-a is the env-gated
demo pair: `OTOPCUA_ACL_ENABLED=true` reaches site-a-1/-2 only
(`docker-compose.yml:585/:702`); site-b stays off as the A/B control. Site-a OPC UA:
host ports 4842/4843; site-b: 4844/4845. Keys/grants reach driver-only site nodes ONLY
via the deployment artifact — every authoring step needs a deploy after it.
**Procedure:**
1. Restart site-a gated: `OTOPCUA_ACL_ENABLED=true docker compose -f docker-dev/docker-compose.yml up -d site-a-1 site-a-2`. Confirm the boot Warning on both:
`node ACL enforcement is ENABLED — EVERY data-plane operation … requires a matching NodeAcl grant`.
2. Mint a key: AdminUI `/apikeys` (Chrome automation; login disabled) — Name
`r3-acl-gate`, Principal `r3gate`. COPY the one-time `otua_…` credential immediately.
3. Author a grant: `/clusters/{site-a cluster id}/acls/new` — Principal `r3gate`, scope
picked via the AclScopePickerModal on a WRITABLE raw tag (or its device — inheritance
covers the tag), permissions: the Operator bundle (must include Browse/Read/
WriteOperate). Save; expect no "authorizes nobody" warning.
4. Deploy (`./docker-dev/deploy.sh`) so the key + grant ride the artifact to site-a.
Confirm on site-a-1: `data-plane authority generation {G} installed (… keys=…, grants=…)`.
5. **The load-bearing ALLOW:** Client.CLI write through the grant —
`dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- write -u opc.tcp://localhost:4842 -n "<granted raw tag NodeId>" -v <value> -U apikey -P otua_…`
Expect success (the session carries NO server-wide role; the grant alone authorizes —
a deny-only gate would pass against a build where the ACL is never consulted).
Note: the rig's Galaxy driver has no live gateway, so the DEVICE write may fail
downstream — the gate is that the request passes the ACL (not `BadUserAccessDenied`);
any downstream Bad status is fine and should be reported as such.
6. **Deny controls:** (a) same write WITHOUT credentials (anonymous) ⇒ denied; (b) a
read/browse on a node OUTSIDE the granted scope with the key ⇒ withheld (grep
site-a-1 for `AclGate: denied {Operation} on {NodeId} for principal r3gate`).
7. **Revocation reaches the artifact path:** disable the key on `/apikeys`, deploy,
confirm a NEW `data-plane authority generation {G+1} installed` on site-a-1, and the
same CLI write now fails auth (`ApiKeyAuth: denied — no enabled, unexpired key`).
8. **Teardown:** restore site-a ungated (`docker compose up -d site-a-1 site-a-2` without
the env), optionally delete the gate key/grant, and `docker compose down` the rig
unless Task 8 needs it. Evidence excerpts to the scratchpad.
No commit expected. If Chrome automation of the scope picker proves impractical, direct
ConfigDb inserts for `ApiKey`/`NodeAcl` rows are acceptable ONLY as a fallback — say so,
and keep the key's peppered HMAC consistent with the rig pepper
(`docker-dev-apikey-pepper-not-for-production`); minting via the UI is strongly preferred
because it exercises the real authoring path.
---
### Task 9: Roslyn evaluator racing test — remove the timing assumption
**Classification:** standard
**Estimated implement time:** ~5 min
**Parallelizable with:** Task 10 (disjoint files); independent of the rig tasks
**Origin:** ScadaBridge deferred-work register row 38, relayed cross-session 2026-08-18
(coordinated by the ScadaBridge session on Joe's behalf).
**Files:**
- Modify: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/RoslynVirtualTagEvaluatorTests.cs`
(`Evaluate_racing_ClearCompiledScripts_never_fails_with_disposed`, ~:391)
**Spec.** The test is load-fragile: it fails under full-solution parallel test load but
passes 3/3 in isolation. Its design assumes at most ONE `ClearCompiledScripts` lands
inside a single evaluation (so the product's one-shot retry suffices); under contention
two can land and the retry legitimately exhausts. Test-design issue, NOT a product bug.
Fix by removing the timing assumption while keeping the original intent (no
disposed-script failure ever ESCAPES the retry): either bound the clear cadence relative
to observed evaluation latency, or assert on retry-exhaustion behaviour instead of zero
failures — implementer picks whichever preserves intent with less machinery, documents
the choice in the test doc-comment, and demonstrates the test survives a
full-parallel-load run (run the containing suite alongside another heavy suite, or
repeat-run the suite ≥5×; report what was actually run).
---
### Task 10: LocalDb conditional registration + baseline + deregister self-heal
**Classification:** high-risk
**Estimated implement time:** ~8 min
**Parallelizable with:** Task 9 (disjoint files)
**Origin:** ScadaBridge deferred-work register row 39, relayed cross-session 2026-08-18.
ScadaBridge closed the same class with conditional registration + boot-time self-heal
(`Host/SiteLocalDbSetup.cs`, arch-review WP1.3/WP3.3); port that pattern here.
**Files:**
- Modify: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LocalDbSetup.cs` (`OnReady`,
registration at :63-66)
- Read for the predicate: `src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LocalDbRegistration.cs`
(`LocalDb:SyncListenPort`, `LocalDb:Replication:*` keys)
- Modify: `tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/LocalDbSetupTests.cs`
(the registered-table-set + ordering pins — update DELIBERATELY, say so in the diff)
- Modify: `CLAUDE.md` LocalDb section — the two bullets recording unconditional
registration as deliberate ("DeregisterReplicated exists and this repo deliberately
does not call it" / "Consequently, baselineExistingRows is not used either") must be
rewritten to record the new conditional shape and why it changed.
**Spec.**
- "Replication configured" = `PeerAddress` OR `ApiKey` present (OR, because only the
dialling half sets PeerAddress while the passive half carries just the key + listen
port — verify the exact key names against `LocalDbRegistration`).
- Configured ⇒ `RegisterReplicated(table, baselineExistingRows: true)` per table, in the
existing DDL→register order (the ordering is load-bearing and pinned) — seeds
row-versions at the LWW floor and flags snapshot resync, so late-enabled replication
on an established node converges on EXISTING rows, not just subsequent writes.
- NOT configured ⇒ `DeregisterReplicated` on every table at boot — idempotent self-heal:
drops stale triggers, prunes oplog/row-version rows, logs once at Information when it
actually cleaned something (and stays silent when there was nothing to clean).
- Registration stays per-table in a loop (0.3.0 made the snapshot flag per-table; the
loop preserves that granularity if the pin ever moves).
- **Pin note:** this repo is on `ZB.MOM.WW.LocalDb 0.2.1`, which has both
`baselineExistingRows` and `DeregisterReplicated`. Do NOT bump the pin unless a needed
API is genuinely absent at 0.2.1 — if it is, stop and report before bumping.
- **Traps (both hit by ScadaBridge):** the registered-table set is pinned by
`LocalDbSetupTests` — update the pins deliberately with the behaviour change named in
the test doc-comment; and deregistration must be pair-symmetric — the sync handshake
compares registered-table digests fail-closed, so flipping registration is a
both-nodes-together change. Record the pair-symmetry consequence in CLAUDE.md and in
the rollout note (docker-dev site-a has replication ON and was enabled from fresh, so
current impact is low; the change is correctness-before-someone-enables-late).
- Tests: pin the new predicate both ways (configured ⇒ registered with baseline;
unconfigured ⇒ deregistered/self-healed) and keep the DDL→register ordering pin green.
---
### Task 8: Close-out — sweep + Outcomes + register annotations
**Classification:** close-out (blockedBy: all)
**Estimated implement time:** ~15 min
1. Full build (`dotnet build ZB.MOM.WW.OtOpcUa.slnx`) — 0 errors; warning count vs r2's
904 explained.
2. Full unit sweep, every `*.Tests` project, per-project; reconcile the passed-count
delta against Tasks 1–2's added/changed tests exactly. DB suites must RUN (AdminUI
`0 skipped`, Configuration `0 skipped`).
3. Write `## Outcomes` on this plan: sweep record, per-task table, all four live-gate
evidence excerpts, falsification notes, and this round's register (Modbus + S7-write
asymmetry, TM/CT real-CPU, cross-process reap race carried; whatever Tasks 4–7 leave
open).
4. Annotate the r2 register in `2026-08-17-perf-followups-r2.md` append-only with
"Closed by <sha> (r3)" / "Gated live (r3)" sub-bullets.
5. Update the `.tasks.json` ledger; commit docs with explicit pathspec.