dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
549d2d68b3bbb839e467779634ccd2d0fdcef392
lmxopcua/docs/plans/2026-08-17-perf-followups-r3.md
T

56 KiB
Raw Blame History

Perf Follow-ups Round 3 Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development to implement this plan task-by-task (per the standing preference: Opus implementers, classification-driven review chains).

Goal: Close the actionable items in the round-2 follow-up register (docs/plans/2026-08-17-perf-followups-r2.md §"Follow-up register") and run the still-unexercised local live gates (FetchAndCache legs, alarm-bearing surgical ratchet, ACL-enabled enforcement, docker-dev first-POST-404 catch), on branch perf/followups-r3 off master merge 49d05bc2.

Architecture: Two surgical code fixes in already-reviewed code (the ACL cache's single-shot CAS becomes a monotonic-max loop; S7 read-pass health becomes worst-state-wins aggregation), one doc-staleness fix found during exploration, and four live gates on the local docker-dev rig (OrbStack) sequenced on one rig session. No new subsystems.

Tech Stack: .NET 10 / C#, xUnit + Shouldly, docker-dev rig (OrbStack), Traefik v3.1 access log, Client.CLI, Chrome browser automation for AdminUI authoring.

Out of scope (recorded, deliberately not here):

  • TM/CT decode at a non-zero device number — needs a real S7 CPU; the test exists and is env-gated on S7_TIMER_NUMBER_ADDRESSING=1. Nothing to do until hardware exists.
  • Cross-process plant-vs-reap race on the shared SQL server — accepted in r2; the 24 h age floor is the mitigation. No work.
  • Modbus per-tag health last-writer-wins — Task 2 fixes S7 only (the recorded item). Modbus has the same shape (ModbusDriver.ReadAsync:493/511/518 + the coalesced path); fixing it here doubles the blast radius of a watchdog-adjacent change. Recorded in this round's register as a deliberate asymmetry.
  • S7 write-path per-item health (S7Driver.cs:1575/1633/1645/1654) — same last-writer-wins shape on WriteAsync; out of scope for the same reason, recorded.
  • Historian end-to-end live gate — needs the wonder-sql-vd03 VPN (user-driven).
  • HistorianGateway batch WriteLiveValues RPC — cross-repo.
  • Load-scale throughput — needs a load harness; not defined enough to gate.

Execution rules (unchanged from prior rounds): shared checkout — explicit-pathspec git add/git commit -- <files> only (never a bare commit); never checkout/switch/reset/ stash/clean; retry on index.lock and CS2012/MSB3030 build races; reviewers judge commits via git show; falsification mutations applied AND reverted via Edit (never git), run against a freshly built binary (never --no-build after a mutation), and labeled honestly ("pin, not red-first" where the behaviour predates the test; "probabilistic red" where the mutation trips a race). Live-gate rig work is serial — one rig session, Tasks 4→5→6→7 in that order, images built from the branch tip AFTER Tasks 1–3 land so the gates also regression the new code.

Task 1: ACL InvalidateIfStale — monotonic-max CAS loop (publication never deferred)

Classification: high-risk Estimated implement time: ~4 min Parallelizable with: Tasks 2, 3

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/SessionAclDecisionCache.cs:453-461 (+ the XML remarks block :430-452)
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/SessionAclDecisionCacheTests.cs

Context. The r2 register records: the single-shot CAS in InvalidateIfStale can defer stamp publication by one call. Two callers read the same seen baseline; gen-N wins the CAS, gen-N+1's CAS fails against the moved stamp and falls out with no retry — N+1 is not published on that call, so its TryGet misses and its Set is dropped at :223. Benign (verdict safety lives in the per-shard generation gate), but it costs a lost store exactly in the post-deploy window. Current code:

private void InvalidateIfStale(long generation)
{
    var seen = Interlocked.Read(ref _generation);
    if (generation <= seen) return;
    if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen) ClearAll();
}

Step 1: Write the racing pin (red against current code — probabilistic red, say so).

[Fact]
public void Concurrent_lookups_at_mixed_generations_always_publish_the_highest()
{
    // The single-shot CAS defect: when gen-1 wins the shared-baseline race, gen-2's CAS
    // fails and is never retried, leaving Generation == 1. Red is probabilistic per
    // round; 200 barrier-synchronised rounds make it effectively certain.
    for (var round = 0; round < 200; round++)
    {
        var cache = /* fresh cache, generation seed -1 */;
        using var barrier = new Barrier(2);
        var low  = Task.Run(() => { barrier.SignalAndWait(); /* TryGet at generation 1 */ });
        var high = Task.Run(() => { barrier.SignalAndWait(); /* TryGet at generation 2 */ });
        Task.WaitAll(low, high);
        cache.Generation.ShouldBe(2L,
            $"round {round}: the newer generation must publish even when it loses the CAS race");
    }
}

Use the ordinary TryGet lookup path (as the straggler test at :286 does), NOT AdvanceGenerationForTest — the defect lives in the lookup path's invalidation.

Step 2: Run it, observe red. dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~Concurrent_lookups_at_mixed_generations" — expect FAIL within a few rounds. If 200 rounds pass green against unmodified code, the test is not exercising the race — fix the test, do not proceed.

Step 3: Implement the monotonic-max loop.

private void InvalidateIfStale(long generation)
{
    while (true)
    {
        var seen = Interlocked.Read(ref _generation);
        if (generation <= seen) return;                 // guard re-checked per attempt
        if (Interlocked.CompareExchange(ref _generation, generation, seen) == seen)
        {
            ClearAll();
            return;
        }
    }
}

Invariants (update the XML remarks :430-452 to say all of this):

  • The <= guard is INSIDE the loop against a fresh read — a straggler that lost a CAS re-reads the newer stamp and exits; hoisting the guard reintroduces the 98d6a190 defect.
  • Only the thread whose CAS moved the stamp clears (elected clear preserved). A rollover now legitimately clears once per distinct generation (N then N+1) — bounded by generations, not request volume.
  • The loop self-terminates: each iteration exits via the guard or CASes against a strictly larger seen, and _generation only rises.
  • The -1 seed still admits generation 0.

Step 4: Run the new test + the full cache suite. All 18 green (17 existing + 1 new), especially A_straggler_from_an_older_generation_neither_clears_nor_rolls_the_stamp_back (:286) and No_hit_ever_returns_a_verdict_from_another_generation (:363).

Step 5: Falsify. Via Edit, revert the loop to the single-shot form (fresh build, not --no-build) → the new test must go red (probabilistic; report the failing round). Revert via Edit. Separately, move the <= guard outside the loop → the straggler test at :286 must go red. Revert.

Step 6: Commit.

git add -- src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Security/SessionAclDecisionCache.cs \
  tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/SessionAclDecisionCacheTests.cs
git commit -m "fix(acl): make InvalidateIfStale a monotonic-max CAS loop — stamp publication is never deferred" -- <same paths>

Task 2: S7 read-pass health — worst-state-wins aggregation

Classification: high-risk Estimated implement time: ~5 min Parallelizable with: Tasks 1, 3

Files:

  • Modify: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs — per-item health sites :647 (per-tag loop), :828 (multi-var decode loop), :941 (span member loop), RecordReadFailure:1010-1046, plus one end-of-pass write in ReadAsync
  • Test: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/S7SpanReadParityTests.cs (flip the :423 pin; doc-comment :389-393 rewritten; new tests)

Context. The r2 register records: per-item decode health is last-writer-wins — a mixed chunk reports whatever the LAST item's outcome was, so Healthy can be reported for a pass that had failures (pinned at :396-423), and the :1023 access-denied Faulted can be overwritten by a later good item. Fix = aggregate one outcome per ReadAsync pass.

Watchdog contract (MUST hold — CLAUDE.md comms-loss section): Degraded means connected-but-some-bad; LastSuccessfulRead advances on ANY success and is preserved on failure. The aggregate write must advance LastSuccessfulRead when ANY item succeeded, even when the state lands Degraded/Faulted — gating it on Healthy turns the fix into a false-positive reconnect storm.

Step 1: Red tests first (the flip IS red-first; the rest are new):

  1. Flip :423: One_undecodable_multi_var_item... now expects Degraded (rename to say worst-state-wins; rewrite doc-comment :389-393 — the last-writer-wins pin is being deliberately retired).
  2. New: a mixed multi-var pass reports Degraded AND LastSuccessfulRead is advanced (non-null / newer than the pre-pass value).
  3. New: span path — one bad member decode then a good member ⇒ Degraded (today: Healthy).
  4. New: an access-denied PlcException on one tag followed by a success ⇒ state stays Faulted (ordering Healthy < Degraded < Faulted), LastSuccessfulRead advanced. Use the per-tag path if the multi-var fake can't produce a per-item PlcException.
  5. Must-stay-green: :326 (A_failing_multi_var_chunk_falls_back..., asserts Healthy at :346) and :432 (short answer, Healthy at :447) — a multi-var REQUEST fault whose tags are re-served Good by the per-tag tail is NOT a failed outcome. The chunk-fault catch (:806-814) stays outside the aggregate.

Step 2: Run, observe red on 1–4 (each for its own reason; report per-test).

Step 3: Implement. A small per-pass accumulator threaded through the pass:

private sealed class ReadPassOutcome
{
    public bool AnySuccess;
    public DriverState WorstFailure = DriverState.Healthy;   // Healthy = none recorded
    public string? FirstError;                                // first failure is causal
    public void RecordSuccess() => AnySuccess = true;
    public void RecordFailure(DriverState state, string message)
    {
        if (Rank(state) > Rank(WorstFailure)) WorstFailure = state;
        FirstError ??= message;
        static int Rank(DriverState s) => s switch
        {
            DriverState.Faulted => 2, DriverState.Degraded => 1, _ => 0
        };
    }
}
  • The three success sites (:647, :828, :941) call outcome.RecordSuccess() instead of assigning _health.
  • RecordReadFailure takes the outcome, records (Degraded|Faulted, message) into it instead of assigning _health, and KEEPS its _plcDead / MarkConnectionDeadIfFatal side effects immediate. It still returns the per-item status code.
  • One _health write at the end of the ReadAsync try body (NOT finally — the OCE arms at :649/:801/:920 rethrow and must skip it):
_health = new DriverHealth(
    outcome.WorstFailure == DriverState.Healthy
        ? DriverState.Healthy : outcome.WorstFailure,
    outcome.AnySuccess ? now : _health.LastSuccessfulRead,
    outcome.WorstFailure == DriverState.Healthy ? null : outcome.FirstError);
  • Untouched: the :567 connect-failure early return; HandlePollError (:1980); the entire write path; init/shutdown sites. ReadSpanAsync / ReadLeftoversByMultiVarAsync gain an outcome parameter — signature changes, no logic movement.

Step 4: Run the full S7 suite (dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests) — all green including the parity anchor :202 (all-members-fail ⇒ Degraded both paths).

Step 5: Falsify. Via Edit: (a) make the end-of-pass write use the LAST recorded state instead of the worst → tests 1/3/4 red; (b) gate LastSuccessfulRead advancement on the state being Healthy → test 2 red. Fresh builds. Revert via Edit.

Step 6: Commit (explicit pathspec, message fix(s7): read-pass health is worst-state-wins — a mixed pass can no longer report Healthy).

Task 3: Fix stale OTOPCUA_CONFIG_MODE docs

Classification: trivial Estimated implement time: ~2 min Parallelizable with: Tasks 1, 2

Files:

  • Modify: docs/Configuration.md (~line 253) — "Flip only the site nodes with OTOPCUA_CONFIG_MODE=FetchAndCache at docker compose up" is stale
  • Modify: CLAUDE.md — same sentence in the ConfigSource/ConfigServe section
  • Check-and-fix if matched: grep -rn "OTOPCUA_CONFIG_MODE" docs/ docker-dev/ CLAUDE.md

Spec. Phase 4 made FetchAndCache mandatory on driver-only nodes (ConfigSourceOptionsValidator fails host start on Direct), and the rig's four site nodes are hardcoded ConfigSource__Mode: "FetchAndCache" in docker-dev/docker-compose.yml:572-579/691-696/787-792/858-863. The env var appears nowhere in the compose file; setting it is a no-op. Rewrite both sentences to state: the site nodes run FetchAndCache unconditionally (mandatory for driver-only nodes since Phase 4); central stays Direct via its admin role. No review (doc-only). Commit with explicit pathspec, message docs(config): OTOPCUA_CONFIG_MODE is gone — site nodes are hardcoded FetchAndCache (Phase 4).

Task 4: Live gate — docker-dev first-POST-404 catch + no-body POST binding

Classification: live-gate (rig session step 1 — FRESH BOOT window) Estimated implement time: ~10 min wall (mostly rig boot) Parallelizable with: none (rig serial; runs after Tasks 1–3 land, images built from branch tip)

Files:

  • Possibly modify: docker-dev/deploy.sh (the header NOTE about no-body binding), docker-dev/README.md (§"First-POST 404" — mechanism note if caught)
  • Evidence: write to the scratchpad; Task 8 folds it into the plan's Outcomes

Procedure:

  1. Fresh rig: docker compose -f docker-dev/docker-compose.yml down -v then up -d --build (build mandatory — the branch tip must be in the images).
  2. IMMEDIATELY run ./docker-dev/deploy.sh (no body). Its 2 s /health/ready poll rides through Traefik during the boot window — every poll is access-logged with defaultmode=keep, and an unmatched request shows an EMPTY RouterName.
  3. After the deploy completes (or fails), capture docker compose -f docker-dev/docker-compose.yml logs traefik and extract every non-2xx access-log line from the boot window. If any 404 appears: name the mechanism from RouterName/ServiceName/timing (the r2 elimination says: AdminUI cannot 404 that path; backend-down is 503; so 404 = no router matched at that instant — presumably the window before the dynamic-config file provider loads or before the health-checked service has a server). Update docker-dev/README.md §"First-POST 404" with the observed mechanism. If it does not reproduce: record that honestly — "did not reproduce under the readiness gate; register item stays open" — do NOT invent a mechanism.
  4. No-body POST binding: with the rig ready, run ./docker-dev/deploy.sh bare and record the HTTP status + response body. Determine whether the minimal-API DeployRequest? binds null (and what the endpoint then does) or 400s. Update the header NOTE in deploy.sh from "unverified" to the observed behaviour, and README.md if wrong.
  5. Leave the rig UP for Task 5.

Commit any deploy.sh/README.md edits (explicit pathspec). Report: the access-log excerpt, the no-body verdict, and whether the 404 reproduced.

Task 5: Live gate — FetchAndCache legs on the running rig

Classification: live-gate (rig session step 2) Estimated implement time: ~8 min wall Parallelizable with: none (rig serial, after Task 4)

Procedure (rig already up from Task 4; a deploy already ran):

  1. Fetch + verify: docker compose -f docker-dev/docker-compose.yml logs site-a-1 | grep -E "Fetched \+ verified artifact" — expect Fetched + verified artifact for {DeploymentId} from http://central-… ({Bytes} bytes).
  2. Served (central side): logs central-1 central-2 | grep "Served artifact" — Served artifact for {DeploymentId} ({Bytes} bytes, {Chunks} chunks).
  3. Applied-from-fetch + cached: logs site-a-1 | grep -E "applied fetched deployment|cached applied deployment" — both lines must appear (the fetched variant, NOT the Direct applied deployment form).
  4. Boot-from-cache: docker compose restart site-a-1, wait for boot, then logs site-a-1 | grep "FetchAndCache boot" — expect restored served state for deployment {Id} (rev {Rev}) from the LocalDb pointer.
  5. Absence proof: on site-a-1 logs, assert NO occurrence of entering Stale, ConfigDb back, or the Direct-form applied deployment {Id} (rev (without fetched). A FetchAndCache node must never reach Stale.
  6. Evidence excerpts to the scratchpad for Task 8. Rig stays UP for Task 6.

No commit expected. If any leg fails, STOP and report — that is a real regression, not a gate formality.

Task 6: Live gate — alarm-bearing surgical ratchet

Classification: live-gate (rig session step 3) Estimated implement time: ~12 min wall Parallelizable with: none (rig serial, after Task 5)

Context. The surgical-deploys classifier routes an alarm-bearing raw-tag delta (Alarm != null on EITHER side) to Rebuild; a whitelisted non-alarm attribute delta (Writable among them) is surgical. Never observed live. The ONLY seam is the applier summary line (AddressSpaceApplier.cs:322-326): applied plan (kind={Kind}, …, surgicalRawTags={SurgicalRaw}, …, rebuild={Rebuild}).

Procedure:

  1. Pick TWO alarm-free raw tags in the rig's seeded config (browse /raw via AdminUI at http://localhost:9200, login disabled — or read the ConfigDb Tag rows). Call them TagS (surgical lever) and TagA (alarm lever).
  2. Surgical control first: toggle TagS's Access level (Read ↔ ReadWrite) — via the RawTagModal in the browser, or a direct ConfigDb UPDATE of the tag row (the rig's DB on 10.100.0.35,14330; acceptable for a classifier gate, note which route was used). Deploy via ./docker-dev/deploy.sh. Expect on central-1: kind=AttributeOnly, … surgicalRawTags=1, … rebuild=False.
  3. Alarm add ⇒ rebuild: add "alarm": {"alarmType": "OffNormalAlarm", "severity": 500} to TagA's TagConfig JSON (raw-JSON textarea — available because the seeded device is a Galaxy driver, which has no typed editor — or the same direct-DB route). Deploy. Expect kind=Rebuild, … surgicalRawTags=0, … rebuild=True.
  4. Alarm remove ⇒ rebuild: remove the alarm object, deploy, expect rebuild=True again (Alarm != null on the PREVIOUS side also disqualifies).
  5. Restore: revert TagS's access level, deploy once more (surgical again — also re-proves the ratchet is per-deploy, not latched).
  6. Evidence: the four applier lines (grep docker compose logs central-1 site-a-1 | grep "applied plan (kind=" — note which nodes applied). Scratchpad for Task 8. Rig stays UP for Task 7.

No commit expected. Failure of the expected routing = STOP and report.

Task 7: Live gate — ACL-enabled enforcement on site-a (re-shaped cache)

Classification: live-gate (rig session step 4) Estimated implement time: ~20 min wall Parallelizable with: none (rig serial, after Task 6)

Context. Security:Acl:Enabled has never been exercised live since the r1/r2 cache re-shape (per-key shards, generation stamp, Task 1's CAS loop). Site-a is the env-gated demo pair: OTOPCUA_ACL_ENABLED=true reaches site-a-1/-2 only (docker-compose.yml:585/:702); site-b stays off as the A/B control. Site-a OPC UA: host ports 4842/4843; site-b: 4844/4845. Keys/grants reach driver-only site nodes ONLY via the deployment artifact — every authoring step needs a deploy after it.

Procedure:

  1. Restart site-a gated: OTOPCUA_ACL_ENABLED=true docker compose -f docker-dev/docker-compose.yml up -d site-a-1 site-a-2. Confirm the boot Warning on both: node ACL enforcement is ENABLED — EVERY data-plane operation … requires a matching NodeAcl grant.
  2. Mint a key: AdminUI /apikeys (Chrome automation; login disabled) — Name r3-acl-gate, Principal r3gate. COPY the one-time otua_… credential immediately.
  3. Author a grant: /clusters/{site-a cluster id}/acls/new — Principal r3gate, scope picked via the AclScopePickerModal on a WRITABLE raw tag (or its device — inheritance covers the tag), permissions: the Operator bundle (must include Browse/Read/ WriteOperate). Save; expect no "authorizes nobody" warning.
  4. Deploy (./docker-dev/deploy.sh) so the key + grant ride the artifact to site-a. Confirm on site-a-1: data-plane authority generation {G} installed (… keys=…, grants=…).
  5. The load-bearing ALLOW: Client.CLI write through the grant — dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- write -u opc.tcp://localhost:4842 -n "<granted raw tag NodeId>" -v <value> -U apikey -P otua_… Expect success (the session carries NO server-wide role; the grant alone authorizes — a deny-only gate would pass against a build where the ACL is never consulted). Note: the rig's Galaxy driver has no live gateway, so the DEVICE write may fail downstream — the gate is that the request passes the ACL (not BadUserAccessDenied); any downstream Bad status is fine and should be reported as such.
  6. Deny controls: (a) same write WITHOUT credentials (anonymous) ⇒ denied; (b) a read/browse on a node OUTSIDE the granted scope with the key ⇒ withheld (grep site-a-1 for AclGate: denied {Operation} on {NodeId} for principal r3gate).
  7. Revocation reaches the artifact path: disable the key on /apikeys, deploy, confirm a NEW data-plane authority generation {G+1} installed on site-a-1, and the same CLI write now fails auth (ApiKeyAuth: denied — no enabled, unexpired key).
  8. Teardown: restore site-a ungated (docker compose up -d site-a-1 site-a-2 without the env), optionally delete the gate key/grant, and docker compose down the rig unless Task 8 needs it. Evidence excerpts to the scratchpad.

No commit expected. If Chrome automation of the scope picker proves impractical, direct ConfigDb inserts for ApiKey/NodeAcl rows are acceptable ONLY as a fallback — say so, and keep the key's peppered HMAC consistent with the rig pepper (docker-dev-apikey-pepper-not-for-production); minting via the UI is strongly preferred because it exercises the real authoring path.

Task 9: Roslyn evaluator racing test — remove the timing assumption

Classification: standard Estimated implement time: ~5 min Parallelizable with: Task 10 (disjoint files); independent of the rig tasks

Origin: ScadaBridge deferred-work register row 38, relayed cross-session 2026-08-18 (coordinated by the ScadaBridge session on Joe's behalf).

Files:

  • Modify: tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/RoslynVirtualTagEvaluatorTests.cs (Evaluate_racing_ClearCompiledScripts_never_fails_with_disposed, ~:391 — renamed by the implementation to Evaluate_racing_ClearCompiledScripts_never_leaks_a_raw_disposed_failure)

Spec. The test is load-fragile: it fails under full-solution parallel test load but passes 3/3 in isolation. Its design assumes at most ONE ClearCompiledScripts lands inside a single evaluation (so the product's one-shot retry suffices); under contention two can land and the retry legitimately exhausts. Test-design issue, NOT a product bug. Fix by removing the timing assumption while keeping the original intent (no disposed-script failure ever ESCAPES the retry): either bound the clear cadence relative to observed evaluation latency, or assert on retry-exhaustion behaviour instead of zero failures — implementer picks whichever preserves intent with less machinery, documents the choice in the test doc-comment, and demonstrates the test survives a full-parallel-load run (run the containing suite alongside another heavy suite, or repeat-run the suite ≥5×; report what was actually run).

Task 10: LocalDb conditional registration + baseline + deregister self-heal

Classification: high-risk Estimated implement time: ~8 min Parallelizable with: Task 9 (disjoint files)

Origin: ScadaBridge deferred-work register row 39, relayed cross-session 2026-08-18. ScadaBridge closed the same class with conditional registration + boot-time self-heal (Host/SiteLocalDbSetup.cs, arch-review WP1.3/WP3.3); port that pattern here.

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LocalDbSetup.cs (OnReady, registration at :63-66)
  • Read for the predicate: src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/LocalDbRegistration.cs (LocalDb:SyncListenPort, LocalDb:Replication:* keys)
  • Modify: tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/LocalDbSetupTests.cs (the registered-table-set + ordering pins — update DELIBERATELY, say so in the diff)
  • Modify: CLAUDE.md LocalDb section — the two bullets recording unconditional registration as deliberate ("DeregisterReplicated exists and this repo deliberately does not call it" / "Consequently, baselineExistingRows is not used either") must be rewritten to record the new conditional shape and why it changed.

Spec.

  • "Replication configured" = PeerAddress OR ApiKey present (OR, because only the dialling half sets PeerAddress while the passive half carries just the key + listen port — verify the exact key names against LocalDbRegistration).
  • Configured ⇒ RegisterReplicated(table, baselineExistingRows: true) per table, in the existing DDL→register order (the ordering is load-bearing and pinned) — seeds row-versions at the LWW floor and flags snapshot resync, so late-enabled replication on an established node converges on EXISTING rows, not just subsequent writes.
  • NOT configured ⇒ DeregisterReplicated on every table at boot — idempotent self-heal: drops stale triggers, prunes oplog/row-version rows, logs once at Information when it actually cleaned something (and stays silent when there was nothing to clean).
  • Registration stays per-table in a loop (0.3.0 made the snapshot flag per-table; the loop preserves that granularity if the pin ever moves).
  • Pin note: this repo is on ZB.MOM.WW.LocalDb 0.2.1, which has both baselineExistingRows and DeregisterReplicated. Do NOT bump the pin unless a needed API is genuinely absent at 0.2.1 — if it is, stop and report before bumping.
  • Traps (both hit by ScadaBridge): the registered-table set is pinned by LocalDbSetupTests — update the pins deliberately with the behaviour change named in the test doc-comment; and deregistration must be pair-symmetric — the sync handshake compares registered-table digests fail-closed, so flipping registration is a both-nodes-together change. Record the pair-symmetry consequence in CLAUDE.md and in the rollout note (docker-dev site-a has replication ON and was enabled from fresh, so current impact is low; the change is correctness-before-someone-enables-late).
  • Tests: pin the new predicate both ways (configured ⇒ registered with baseline; unconfigured ⇒ deregistered/self-healed) and keep the DDL→register ordering pin green.

Task 8: Close-out — sweep + Outcomes + register annotations

Classification: close-out (blockedBy: all) Estimated implement time: ~15 min

  1. Full build (dotnet build ZB.MOM.WW.OtOpcUa.slnx) — 0 errors; warning count vs r2's 904 explained.
  2. Full unit sweep, every *.Tests project, per-project; reconcile the passed-count delta against Tasks 1–2's added/changed tests exactly. DB suites must RUN (AdminUI 0 skipped, Configuration 0 skipped).
  3. Write ## Outcomes on this plan: sweep record, per-task table, all four live-gate evidence excerpts, falsification notes, and this round's register (Modbus + S7-write asymmetry, TM/CT real-CPU, cross-process reap race carried; whatever Tasks 4–7 leave open).
  4. Annotate the r2 register in 2026-08-17-perf-followups-r2.md append-only with "Closed by (r3)" / "Gated live (r3)" sub-bullets.
  5. Update the .tasks.json ledger; commit docs with explicit pathspec.

Outcomes

Written at close-out, 2026-08-18, on branch perf/followups-r3. All 10 tasks complete (8 planned + Tasks 9/10 joined mid-round by amendment 088a1a1e). This round is a follow-up round: every code item it closes was opened by the round-2 register in 2026-08-17-perf-followups-r2.md, and that register is annotated in place with the closing SHAs. Tasks 9 and 10 are cross-repo — rows 38 and 39 of the ScadaBridge deferred-work register, relayed on Joe's behalf and closed here.

Sweep record

The docker-dev rig was torn down BEFORE the sweep (docker compose -f docker-dev/docker-compose.yml down, 0 otopcua-dev-* containers remaining). This is not housekeeping — a running rig un-skips env-gated tests and would have made the skip count incomparable to the r2 baseline. It did: Host.IntegrationTests reports 229 / 0 / 2 with the rig up and 226 / 0 / 5 with it down, the same 231 total.

dotnet build ZB.MOM.WW.OtOpcUa.slnx --no-incremental → 0 errors, 904 warnings — identical to r2's 904, not merely close. The non-incremental flag matters: an incremental build reported 896 because up-to-date projects re-emit nothing. The two pre-existing families are unchanged in kind and in count (OTOPCUA0001 CapabilityInvoker analyzer on test call sites 1012, xUnit1051 cancellation-token advice 696, plus the small CS tail). No delta to attribute — this round's new test call sites happened to land in files already carrying the same families, so nothing moved.

Full unit sweep — every *.Tests project, run per-project with --no-build after the single solution build, against the tree at 837d8a17. *.IntegrationTests are build-only by convention (as in r2); Host.IntegrationTests is recorded separately below because r2's baseline did not include it.

43 / 43 test projects green.

Round 2 This round Δ
Passed 10 370 10 377 +7
Failed 0 0 —
Skipped 19 19 —
Total 10 389 10 396 +7

The +7 reconciles exactly against the two suites this round added tests to, with nothing unaccounted for — confirmed at class granularity, not just project granularity:

Project Round 2 This round Δ Source
OpcUaServer.Tests 481 482 +1 SessionAclDecisionCacheTests 17 → 18 (T1)
Driver.S7.Tests 401 407 +6 S7SpanReadParityTests 13 → 19 (T2)

⚠️ The S7 +6 is the sum of three separate commits, not one addition — 4c161181 (+3, the worst-state-wins pins), b35e423c (+1, the observe-nothing guard the spec review found), and 1765e300 (+2, the code-review round's severity-escalation and OCE-flush pins). T3 (docs) and T4–T7 (live gates) added no tests by design.

All 19 skips are the pre-existing env-gated ones, project for project — Core.Tests (1), Driver.Galaxy.Tests (8), Driver.Historian.Gateway.Tests (6, the LiveIntegration category needing the wonder-sql-vd03 VPN), OpcUaServer.Tests (1), Runtime.Tests (3). This round added no skips, and Runtime.Tests is unchanged at 637 / 3 as expected.

The DB-backed suites ran against a reachable 10.100.0.35,14330 — executed, not skipped: AdminUI.Tests 1540 passed / 0 skipped, Configuration.Tests 190 passed / 0 skipped.

Host.IntegrationTests (T9 + T10) — 226 passed / 0 failed / 5 skipped (231 total)

Not part of the r2 baseline, so there is no measured predecessor. Derived from the diff against the merge-base 49d05bc2, the expected delta is +9, all plain [Fact] (no theories, so attribute count equals test count): LocalDbSetupTests 7 → 15 (+8) and LocalDbWiringTests 6 → 7 (+1); RoslynVirtualTagEvaluatorTests stays 20 → 20 because T9 reworked its racing test in place. ⚠️ The task brief's "LocalDbSetupTests 13 → 15" was an intermediate reading — 13 was the count after 3e62f161 and before the df533916 review round; against the merge-base it is 7 → 15. The 5 skips are the rig/fixture-gated DriverTestConnectE2eTests (2, Modbus fixture) and DriverProbeHandshakeE2eTests (3) — three of which run when the rig is up, which is exactly the teardown effect noted above.

⚠️ One flake, isolated and cleared — Client.CLI.Tests

The first sweep pass reported Client.CLI.Tests 1 failed / 103 passed: LoggerLifecycleTests.ConfigureLogging_DisposesPreviousLogger_BeforeReassigning, asserting trackerSink.Disposed was False. Isolation re-runs cleared it: the single test alone 1/1 green, and the whole project green twice more (104 / 104 both times).

The mechanism is a pre-existing intra-assembly race on the process-global Serilog.Log.Logger static, not a regression: the test assigns Log.Logger at LoggerLifecycleTests.cs:26 and then asserts, at :41, that the command's ConfigureLogging disposed that instance — but the class carries no [Collection] and no DisableTestParallelization, so any concurrently-scheduled test whose command path also calls ConfigureLogging (which does Log.CloseAndFlush() then reassigns) can swap the global out from under it in that window. No commit in this round touches src/Client or tests/Client (git log 49d05bc2..HEAD -- src/Client tests/Client is empty). Recorded here rather than in the register, which the close-out brief scoped explicitly; it is a worthwhile one-line fix (pin the class to a serial collection) whenever someone is next in that file.

Per-task outcomes

# Task Commit(s) Outcome + deviation
1 ACL InvalidateIfStale — monotonic-max CAS loop 441b34a3, f4b91dbb, e29fab19 Done; closes the r2 register's deferred-publication item. ⚠️ The plan's own 200-round Barrier harness was measured GREEN-ON-DEFECT (~1-in-19k hit rate — Barrier's post-phase release hands one racer a head start), so it was rebuilt as long-lived racers with a third-thread release store: 25–50 % of rounds red, falsification red at rounds 1–13. 10 000 rounds kept over the reviewer's suggested 2 000 after measuring a 1-in-3 vacuous pass at 2 000 under full-suite parallelism (misses burst-correlate). 18/18.
2 S7 read-pass health — worst-state-wins 4c161181, b35e423c, 1765e300 Done; closes the r2 register's last-writer-wins item for the S7 READ path only (asymmetry recorded below). The spec review found a second defect in-round — an empty pass that observed nothing must report nothing, or it clears a live Faulted. The code review then found that the OCE arms skipped the end-of-pass write and froze LastSuccessfulRead; the coordinator ruled the OCE path must flush the recorded pass evidence via finally, which is what protects the watchdog contract. Five falsifications, each red on its own test. 19/19.
3 Stale OTOPCUA_CONFIG_MODE docs e9ba1741 Done as specified; compose facts verified before editing. Historical plan docs deliberately untouched. Trivial ⇒ no review.
4 Live gate — first-POST 404 + no-body POST 3f04a4d4 PASSED — and the 404 mechanism is now NAMED, closing an r2 register item the gate was only expected to instrument. Detail below.
5 Live gate — FetchAndCache legs (none — gate only) PASSED, all six legs. Detail below.
6 Live gate — alarm-bearing surgical ratchet (none — gate only) PASSED, all four routings. Detail below.
7 Live gate — ACL-enabled enforcement (none — gate only) PASSED, with a topology adaptation and two non-blocking findings. Detail below.
9 Roslyn evaluator racing test — remove the timing assumption 34829ea0, 837d8a17 Done. Cross-repo: ScadaBridge deferred-work register row 38, relayed 2026-08-18. Detail below.
10 LocalDb conditional registration + baseline + deregister self-heal 3e62f161, df533916 Done. Cross-repo: ScadaBridge deferred-work register row 39. No pin bump. The scadaproj umbrella index was updated and pushed to main as 1145df6. Detail below.
8 Close-out — sweep + Outcomes + register annotations this commit Done. Rig torn down first, full build + sweep, this Outcomes section, and the append-only r2 register annotations.

Task 4 — the first-POST 404, mechanism CONFIRMED

2026-08-18, local OrbStack docker-dev rig built from the branch tip. Two results, and the second is the one that matters.

  • The prescribed fresh-boot run did NOT reproduce the 404 — the entire Traefik access log for the boot window was 3 lines, all 2xx, every one carrying RouterName = otopcua-admin@file. ⚠️ And deploy.sh under up -d can never reproduce it: Compose blocks on the depends_on: service_healthy chain, so up -d returned ~30 s after Traefik started and the first poll landed long after the window closed.
  • So the window was exercised directly — rapid-poll (~10 ms) across docker restart otopcua-dev-traefik-1, twice. Result: 000 (port unbound) → 404 for ~65–160 ms → 200. Body is exactly 404 page not found (Go's http.NotFound), and the access-log signature is the discriminator the r2 round asked for: RouterName = "-", ServiceName = "-", 0ms — no router matched, no backend dialled.
  • NAMED MECHANISM: the window between Traefik binding the web entrypoint and loading the dynamic file-provider config from /etc/traefik/dynamic.yml. The listener accepts connections before the router table exists.
  • The competing candidate is excluded on evidence, not argument — a health-checked service with no healthy server yet would be a matched router answering 503; there are zero 503s anywhere in the session's access log.
  • ⚠️ Caveat now recorded in both deploy.sh and the README: not all 404s are access-logged. Probe 1's seven 404s produced zero log lines; probe 2's produced 4 of 7. The accesslog handler is itself part of the dynamic chain that has not been built yet, so an empty access log does NOT prove the 404 did not happen.
  • No-body POST binding — VERIFIED, four request shapes (script stdin, no data flag at all, explicit empty body, valid JSON), all reaching the handler. The minimal-API DeployRequest? binds null; it does not 400 and does not 415. A clean post-seal run returned HTTP 202 Accepted, and the DB confirms the null took the fallback: CreatedBy = rest-api.

Task 5 — FetchAndCache, all six legs

PASSED. Two independent full cycles (deploy 1e990163 rev 0b9289fb, then the revert deploy da56012f rev 315de861), both sealed acks=6.

  • Fetch + SHA-verify: all four site nodes — Fetched + verified artifact for 1e99016312004ceca5f73aaba2c899b1 from http://central-1:4055 (1584 bytes). The template only logs after the SHA-256 == RevisionHash check, so its presence is the verify proof. The byte count tracked the composition (1584 → 1468 on revert).
  • Serve (central): four Served artifact for … (1584 bytes, 1 chunks) on central-1.
  • Applied-from-fetch + cached: the applied fetched deployment variant on every site node — and the two Direct central nodes emitted the plain applied deployment form on the same deploy, which is the falsification control that the two templates are distinguishable.
  • Boot-from-cache: after restart site-a-1, FetchAndCache boot — restored served state for deployment 1e9901… (rev 0b9289fb…) from the LocalDb pointer. — the wanted variant, not the empty-cache or failed-pointer ones, matching what the node cached 52 s earlier.
  • Absence proof over the node's whole 302-line lifetime: entering Stale 0, any stale 0, ConfigDb back 0, Direct-form apply 0, and [ERR]/[WRN]/[FTL] 0. ⚠️ The Direct-form check must anchor on ": applied deployment " — a naive grep "applied deployment" matches the cache template and reports a false positive.
  • Bonus: a post-restart browse showed both namespaces live (ns=2 raw + ns=3 UNS) including the temporary gate area — proof the node materialised the address space from the artifact it fetched and restored, not from a stale cache.
  • ⚠️ Method note worth carrying: same-revision deploys short-circuit the fetch path entirely (DriverHostActor.HandleDispatchFromSteady immediate-ACK). A bare re-run of this gate on unchanged config shows zero fetch lines and that is not a failure — force a new revision first. The gate did so with a temporary UnsArea row, then reverted; the revert deploy came back byte-identical to the pre-gate revision.

Task 6 — the alarm-bearing surgical ratchet, four deploys

PASSED. All four applier lines emitted identically on central-1 and central-2, every deploy sealed 6/6:

Step Lever Applier line Verdict
1 AccessLevel ReadWrite → Read kind=AttributeOnly, changed=1, surgicalRawTags=1, rebuild=False surgical ✅
2 alarm ADD kind=Rebuild, changed=1, surgicalRawTags=0, rebuild=True rebuild ✅ (Alarm != null current side)
3 alarm REMOVE kind=Rebuild, changed=1, surgicalRawTags=0, rebuild=True rebuild ✅ (previous side also disqualifies)
4 AccessLevel Read → ReadWrite kind=AttributeOnly, changed=1, surgicalRawTags=1, rebuild=False surgical ✅ — the ratchet is per-deploy, not latched
  • Deviation, and why it is sound: the rig's ConfigDb holds exactly one Tag row, so TagS and TagA are the same tag exercised sequentially. Each step is an independent deploy producing exactly one raw-tag delta over orthogonal fields, so every verdict is unambiguous; a second tag would have added an added-node delta for no gain.
  • Proven at the artifact seam, not just the log: each Deployment.ArtifactBlob was pulled, gunzipped and semantically diffed — each step's artifact differs by exactly the intended lever and nothing else.
  • Only the cluster-MAIN nodes emit an applier line. Site nodes acked every deploy but logged nothing: DeploymentArtifact.ResolveClusterScope self-scopes the fleet-wide artifact, the site plan is empty, and Apply early-returns on plan.IsEmpty before the summary log. Confirmed they did process each artifact via their authority-generation lines.
  • ⚠️ A live-gate fact worth knowing for every future rig gate: the revision hash can never return to its baseline after any UPDATE. ConfigComposer snapshots the SQL Server rowversion column into the artifact, and rowversion increments on every UPDATE including one that writes back an identical value. A byte-diff confirmed RowVersion was the only difference in the entire 4922-byte artifact. Semantic restore was verified instead (config hash back to 7F75AB45…, rowcount unchanged).

Task 7 — ACL-enabled enforcement, first live run since the cache re-shape

PASSED, with a forced topology adaptation and two findings.

  • ⚠️ The planned gated node was not executable. site-a serves containers only — the rig's single tag belongs to cluster MAIN, so site-a-1's authority generation always reports keys=0, grants=0, resolvedScopes=0 and there is no writable value node to authorize. The gate moved to central-1 (fused admin,driver,cluster-MAIN, serving ns=2;s=pymodbus/plc/HR200) via a scratchpad-only compose overlay — the repo compose file was never edited — with central-2 as the ungated A/B control. Consequence recorded: central-1 is a ConfigDb node so key revocation is immediate rather than next-deployment; grants still arrive only via the artifact, so the code actually under test is exercised as intended.
  • Authoring fell back to direct ConfigDb INSERT (the Chrome extension was not connected). ApiKeySecret.Generate was reproduced from source and the pepper read live off the container; the construction is proven correct because the server accepted the key.
  • The ALLOW leg is genuinely load-bearing, because the ungated baseline was captured first. The identical key and call on the ungated node returned 0x801F0000 — the Tag-scoped grant confers no role, so ApiKeyInterimRoleMapper leaves the session with roles [] and the server-wide gate denies. With the gate on, the same call returned Write successful. That is the live demonstration that the ACL decision REPLACES the role check rather than chaining with it — a deny-only gate would have passed against a build that never consults the ACL.
  • Denials fail closed: anonymous write and read both 0x801F0000 with AclGate: Read withheld 1 of 1 node(s) for principal anonymous; out-of-scope browse refused; the ungated control node still open on the same NodeId.
  • Revocation was split into its two distinct code paths, and only the first tests this round's fix. Grant removal → generation 1 → 2, grants=0, resolvedScopes=0, a deliberately primed decision cache discarded, and the denial arrives while ApiKeyAuth: granted still logs — that is the per-key-shard/CAS-loop path. Key disable → 0x80210000 BadIdentityTokenRejected and ApiKeyAuth: denied — no enabled, unexpired key with id … — a different status, because the session never activates and never reaches the gate.
  • Restore verified by hash: the post-task revision returned to the exact pre-task a3a9061bbf50…, with 0 ApiKey / 0 NodeAcl rows and the ENABLED warning gone.

Task 9 — the racing test, and an honest disclosure

Renamed to Evaluate_racing_ClearCompiledScripts_never_leaks_a_raw_disposed_failure and re-shaped around three-bucket classification (leaked ⇒ fail on first; exhausted and starved each bounded at max(evals/10, 8)) with duration-bounded loops (10 s or 400 iterations).

  • ⚠️ The real load failure was not the one the ticket described. It was the test's own 45 s WaitAsync budget expiring under recompile pressure, not the retry legitimately exhausting. Old shape: RED 4/4 at 4-way concurrency. New shape: GREEN 4/4, then 8/8.
  • The review found a further flake vector — TimedScriptEvaluator's 5 s clock starts at enqueue, so a starved evaluation can time out without any product fault. Closed and falsified with a 12 ms-budget probe (old classification RED, new GREEN at a 0.11 % starve rate).
  • Failure-reason literals were promoted to a shared ScriptEvalFailureReasons in Commons, used by both evaluators and the test.
  • ⚠️ Disclosed honestly: the disposed race does not reproduce on this arm64 macOS box at all. The raw-escape assertion is the load-independent guarantee; the exhaustion/starvation bounds are calibrated loose (see the register).

Task 10 — LocalDb registration now follows replication config

Ported from ScadaBridge's WP1.3/WP3.3 pattern. Predicate = PeerAddress OR ApiKey (SyncListenPort deliberately excluded — the passive endpoint fails closed on the key). Configured ⇒ RegisterReplicated(table, baselineExistingRows: true) per table in the pinned DDL→register order; not configured ⇒ DeregisterReplicated on every table at boot as an idempotent self-heal. The CLAUDE.md bullets that recorded the unconditional shape as deliberate were rewritten in the same commit.

  • No pin bump. The 0.2.1 APIs were verified by reflection and a live probe (baseline seeds hlc=0; DeregisterReplicated returns bool) rather than assumed from the changelog.
  • Two vacuous-fixture bugs were surfaced and fixed in passing (LocalDbPairHarness, and an AlarmSfLegacyMigrator two-config split) — fixtures that would have let the new predicate pass without being exercised.
  • Pair-symmetry is the operational trap, recorded in CLAUDE.md and the runbook: the sync handshake compares registered-table digests fail-closed, so flipping registration is a both-nodes-together change.
  • The scadaproj umbrella index was updated and pushed to main as 1145df6, per the standing cross-repo rule.

Review chain

9 implementers; 12 review verdicts across the 4 reviewed code tasks (T1 spec + code + delta re-review; T2 spec + code + delta re-review; T9 spec + code + delta re-review; T10 spec + code + delta re-review); 4 fix rounds; 2 CHANGES REQUESTED (T2, T9). T3 was trivial (doc-only ⇒ no review) and T4–T7 are live gates with no code to review. Zero findings papered over; one coordinator ruling (T2's OCE flush).

Three evidence-hygiene lessons from this round, all worth carrying forward:

  • ⚠️ A race harness can be green-on-defect, and you must measure its hit rate to know. T1's plan-supplied Barrier harness "passed" at ~1-in-19 000 — it looked like a working test. Only measuring the hit rate exposed that Barrier's post-phase release hands one racer a head start. Round count is a function of the measured hit rate under full-suite parallelism, not a round number: 2 000 rounds was measured vacuous 1 time in 3 because misses burst-correlate.
  • ⚠️ Tear the rig down before a comparison sweep. A running rig silently un-skips env-gated tests; the r2 skip count is only comparable because the rig was down for both.
  • ⚠️ Name the version of a count you are quoting. The "13 → 15" in this round's own close-out brief was a mid-round reading; against the merge-base it is 7 → 15. Intermediate counts drift the moment a review round lands.

Follow-up register (opened by this round)

Only genuinely new or still-open items. Everything this round closed is annotated on the round-2 register. None of these blocks the branch.

ACL

  • ⚠️ Browse transit stops below the realm root — a grant-holding client cannot browse down to its own granted tag. With the gate on, the key browses ns=2;s=pymodbus and ns=2;s=pymodbus/plc fine but is refused 0x801F0000 on ns=2;s=OtOpcUa and on a recursive browse from the Objects root. PermissionTrieNode.BrowseTransitPrincipals marks the ancestors lying on the tag's RawPath; the synthetic realm root sits above the RawPath space and is not in the trie at all. It fails closed (not a security hole) and blocked no leg, but it would make an ACL-enabled deployment look broken to any browse-driven client — including the Client.CLI's own default recursive browse. Discovered by the T7 live gate. RECOMMEND filing a Gitea issue before anyone enables Security:Acl:Enabled in production. Likely fix: mark the realm-root container browse-transit whenever any grant resolves beneath it.
    • Closed by 89630ee4 + ff9d617a (branch fix/acl-anchor-browse-transit, 2026-08-18) — no Gitea issue needed; fixed directly. The actual mechanism was one step off the guess above: the trie's zero-segment realm-root branch already existed and was correct — nothing ever produced that scope, because the shared anchor's identifier (s=OtOpcUa, now V3NodeIds.SharedRootFolderId) split into a one-segment scope no grant path contains. DataPlaneAuthorizer.BuildScope now maps the anchor (realm-qualified: Raw namespace only) onto the zero-segment scope, and Evaluate adds a Browse-only second walk against the zero-segment Uns scope — the single anchor fronts both trees, so a UNS-only grant holder must also get through. 11 new tests (3 trie pins, 7 authorizer incl. falsification of the OR-leg and of the realm qualification, 1 e2e browse-succeeds-and-still- filters through a booted server). Full high-risk chain: spec ✅, code CHANGES REQUESTED (stale "two namespace root folders" XML remark) → fixed → delta re-APPROVED.
  • ns=0 standard nodes are ungated — the boot warning overstates. With the gate on, both the key and anonymous read i=2256 and browse i=2253 successfully: those are served by the SDK's CoreNodeManager and never reach DataPlaneAuthorizer. Defensible (standard server metadata, no process data), but "A principal with no grant can connect and see nothing" is not literally true. Informational — a doc softening or a deliberate decision to gate ns=0.

S7

  • _health remains a plain field, versus Modbus's Volatile.Read/Volatile.Write idiom. Pre-existing, but more of an outlier now that the read pass consolidates to a single end-of-pass write — the one place an alignment would be cheap and obvious. Follow-up alignment, not a correctness claim.
  • Worst-state-wins landed for the S7 READ pass only. Modbus (all paths) and the S7 WRITE path keep last-writer-wins. Deliberate scope — both were named out of scope in this plan because fixing them here doubles the blast radius of a watchdog-adjacent change. Recorded as a known asymmetry so the next reader does not mistake it for an oversight.

LocalDb

  • ZB.MOM.WW.LocalDb 0.3.1 is published to the Gitea feed (per-table snapshot flags + a sync fault-teardown fix). This repo deliberately stays on 0.2.1 — the 0.2.1 APIs cover everything Task 10 needed, verified by probe. The per-database snapshot caveat is documented; 0.3.x makes it per-table. ⚠️ A bump is a both-nodes-together change (bookkeeping schema v2 → v3 is one-way), so it wants a planned window, not an incidental pin edit.

FetchAndCache

  • central-2's serve path was never exercised live. All four site nodes resolved central-1:4055 first and stopped there — the documented walk-in-order, stop-at-first-success behaviour, so central-2 served nothing. Not a fault; simply an untested half of the pair.

Scripting / test calibration

  • The disposed race does not reproduce on this arm64 macOS box at all. The raw-escape assertion is the load-independent guarantee and holds regardless; the exhaustion and starvation bounds are calibrated loose on a machine that cannot produce the phenomenon they bound. They would benefit from a Linux-CI observation to tighten.

Test reliability

  • Client.CLI.Tests LoggerLifecycleTests.ConfigureLogging_DisposesPreviousLogger_BeforeReassigning is flaky under intra-assembly parallelism — observed once in the r3 close-out sweep, green in isolation (1/1) and on two whole-project re-runs (104/104). Mechanism: a pre-existing race on the process-global Serilog.Log.Logger — the class carries no [Collection], so a concurrent test's ConfigureLogging (CloseAndFlush + reassign) can swap the global between arrange and assert. No r3 commit touches src/Client or tests/Client. One-line fix when picked up: pin the class to a serial collection.

Carried from round 2, still open

  • TM/CT decode at a non-zero device number still needs a real S7 CPU. The test exists and the env var (S7_TIMER_NUMBER_ADDRESSING=1) is ready; python-snap7 resolves TM/CT as a BIT address and cannot serve it.
  • The cross-process plant-vs-reap race on the shared SQL server is accepted. The 24 h age floor is the mitigation; a lock table on a shared dev server costs more than it buys.

Round-2 register items CLOSED by this round

  • ACL single-shot CAS deferred publication → closed by 441b34a3 (T1).
  • S7 per-item decode health last-writer-wins → closed by 4c161181 + b35e423c + 1765e300 (T2), for the read path; the write-path and Modbus asymmetry is recorded above.
  • docker-dev first-POST 404 → mechanism NAMED by 3f04a4d4 (T4), and the no-body POST binding VERIFIED by the same commit.
Reference in New Issue View Git Blame Copy Permalink