From 73d4061b771b5c9043ee687bad4adf9d7028a190 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 13 Jul 2026 09:52:12 -0400 Subject: [PATCH] chore(r2-12): untrack/archive root planning files; honor pending.md's never-stage rule (07/U-4) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - pending.md, current.md: git rm --cached (untrack, keep on disk) + root-anchored .gitignore entries — making the files' own declared 'never stage' rule true. - looseends.md: deleted (2026-05-18 snapshot; sole open item + all follow-ups target the retired Wonderware historian sidecar rig, moot since the gateway cutover; recoverable from history). - stillpending.md -> docs/plans/2026-06-18-stillpending-backlog-audit.md (git mv; referenced by docs/Historian.md + memory index, preserved under dated-plan convention). - HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md -> docs/plans/2026-06-27-historian-gateway-integration-issues.md (git mv). - docs/Historian.md §'Closed backlog §2 items' repointed at the archived path. Historical docs/plans/* references to the bare stillpending.md filename are frozen point-in-time records, left untouched per the documentation-audit convention. --- .gitignore | 4 + .../R2-12-repo-hygiene-plan.md.tasks.json | 4 +- current.md | 53 -------- docs/Historian.md | 5 +- .../2026-06-18-stillpending-backlog-audit.md | 0 ...27-historian-gateway-integration-issues.md | 0 looseends.md | 54 -------- pending.md | 122 ------------------ 8 files changed, 9 insertions(+), 233 deletions(-) delete mode 100644 current.md rename stillpending.md => docs/plans/2026-06-18-stillpending-backlog-audit.md (100%) rename HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md => docs/plans/2026-06-27-historian-gateway-integration-issues.md (100%) delete mode 100644 looseends.md delete mode 100644 pending.md diff --git a/.gitignore b/.gitignore index d0accbef..cabcf58f 100644 --- a/.gitignore +++ b/.gitignore @@ -54,3 +54,7 @@ src/Server/ZB.MOM.WW.OtOpcUa.Host/pki/ # Vendor binaries — never commit (license risk; see R2-12/U-5) lib/ + +# Session working notes — never stage (pending.md's own hard rule; R2-12/U-4) +/pending.md +/current.md diff --git a/archreview/plans/R2-12-repo-hygiene-plan.md.tasks.json b/archreview/plans/R2-12-repo-hygiene-plan.md.tasks.json index 358c6e76..a4366e71 100644 --- a/archreview/plans/R2-12-repo-hygiene-plan.md.tasks.json +++ b/archreview/plans/R2-12-repo-hygiene-plan.md.tasks.json @@ -5,8 +5,8 @@ { "id": 1, "subject": "T2: U-5 — git rm -r lib/ + .gitignore lib/ guard + rebuild (Commit 1; MAINTAINER SIGN-OFF)", "status": "completed", "blockedBy": [0] }, { "id": 2, "subject": "T3a: U-6 proof — husk dirs untracked (git ls-files = 0) + absent from slnx", "status": "completed", "blockedBy": [] }, { "id": 3, "subject": "T3b: U-6 — rm -rf the 5 Driver.Historian.Wonderware* husk dirs (disk-only, no commit)", "status": "completed", "blockedBy": [2] }, - { "id": 4, "subject": "T5a: U-4 proof — 5 planning files tracked; re-read looseends.md; present per-file table for sign-off", "status": "pending", "blockedBy": [] }, - { "id": 5, "subject": "T5b: U-4 — untrack pending/current + gitignore, archive stillpending + HISTORIAN-GATEWAY to docs/plans/, delete looseends.md, repoint docs/Historian.md (Commit 2; SIGN-OFF for the delete)", "status": "pending", "blockedBy": [4] }, + { "id": 4, "subject": "T5a: U-4 proof — 5 planning files tracked; re-read looseends.md; present per-file table for sign-off", "status": "completed", "blockedBy": [] }, + { "id": 5, "subject": "T5b: U-4 — untrack pending/current + gitignore, archive stillpending + HISTORIAN-GATEWAY to docs/plans/, delete looseends.md, repoint docs/Historian.md (Commit 2; SIGN-OFF for the delete)", "status": "completed", "blockedBy": [4] }, { "id": 6, "subject": "T6: S-3 — git rm .github/workflows/v2-e2e.yml (zero Category=E2E proven; SIGN-OFF); record E2ETests+zero-match-guard follow-up", "status": "pending", "blockedBy": [] }, { "id": 7, "subject": "T7: P-5 — delete redundant integration matrix job from v2-ci.yml + fix header comments; YAML-validate (Commit 3 with T6)", "status": "pending", "blockedBy": [6] }, { "id": 8, "subject": "T8: U-3 — add ack/confirm/shelve/enable/disable sections to docs/Client.CLI.md; correct command count to 13", "status": "pending", "blockedBy": [] }, diff --git a/current.md b/current.md deleted file mode 100644 index 9f7bc98c..00000000 --- a/current.md +++ /dev/null @@ -1,53 +0,0 @@ -# Current status — Galaxy standard-driver effort - -_Updated 2026-06-13._ - -## Where we are - -**Phase A (Galaxy = standard Equipment-kind driver) is DONE, merged to master (`c3c56172`), pushed, and live-verified.** - -- `GalaxyMxGateway` is now an ordinary **Equipment-kind** driver. Retired: the `SystemPlatform` `NamespaceKind` + its mirror (`Phase7Applier.MaterialiseGalaxyTags`, the composer/artifact `GalaxyTags` contract + `GalaxyTagPlan`), the byte-parity `|| DriverType=="GalaxyMxGateway"` exception clauses, the authz `NodeHierarchyKind.SystemPlatform` + `PermissionTrie.WalkSystemPlatform`, and the **entire alias-tag/relay AdminUI machinery**. -- A Galaxy point is now a plain equipment `Tag{EquipmentId set, DriverInstanceId=GalaxyMxGateway, TagConfig={"FullName":"tag.attr"}}`, authored through the standard `TagModal` + the "Browse Galaxy" picker. -- Forward-only data-only migration `CleanupSystemPlatformNamespaces` deletes orphaned SystemPlatform config + `NodeAcl` grants and releases `ExternalIdReservation`s. Kept `UX_Namespace_Cluster_Kind` (Simulated kind still reserved). -- Gate: 1009 unit tests green + full docker-dev live `/run` (migration applied clean, server boots on migrated DB, alias UI gone, namespace picker drops SystemPlatform, Galaxy driver created **under Equipment**, Galaxy tag authored via the new picker). - -Design: `docs/plans/2026-06-12-galaxy-standard-driver-design.md` (3-phase). Phase A plan: `docs/plans/2026-06-12-galaxy-standard-driver-phase-a.md` (+ `.tasks.json`). - ---- - -## The rest of the plan (all NOT started) - -### ✅ Milestone 1 — Equipment-tag live-value DELIVERY (the `FullName→NodeId` router) — DONE, merged `c4435e4f`, pushed - -Design `docs/plans/2026-06-13-equipment-tag-live-values-design.md`; plan `…-plan.md` (+ `.tasks.json`). Mirrored the proven `VirtualTagHostActor._nodeIdByVtag` pattern for DRIVER values: -- NEW `ZB.MOM.WW.OtOpcUa.Commons.OpcUa.EquipmentNodeIds` = single source of truth for the folder-scoped NodeId formula (`Variable(eq,fp,name)`); repointed `Phase7Applier` + `VirtualTagHostActor` to it (byte-identical, kills the drift risk). -- `DriverInstanceId` added to `AttributeValuePublished` (the router key — FullName is unique only WITHIN a driver instance). -- `DriverHostActor` builds a `(DriverInstanceId, FullName) → NodeId[]` map each apply (in `PushDesiredSubscriptions`, via `EquipmentNodeIds.Variable`) and resolves it in `ForwardToMux`, emitting the existing `AttributeValueUpdate` per resolved NodeId (1→many handled; no-match drops+debug-logs). NO `OpcUaPublishActor` change. -- Gate: build 0 errors; **322 tests** (incl. 3 Akka.TestKit tests driving the full router through a real deploy/apply); final integration review = READY TO MERGE (end-to-end NodeId coherence confirmed). Live `/run`: the deploy applied + the variable **materialised at the exact router NodeId** + SubscribeBulk pushed refs — the materialisation + router wiring is confirmed live. - -### 🔴 Milestone 1b — Per-driver "actually PUBLISH a value" gaps (NOT started — the router has nothing to route until these land; live value could NOT be shown in dev for this reason) - -The router is correct, but **no dev driver currently publishes an equipment-tag value**, so a variable still won't show data until the owning driver's publish path is wired. Three orthogonal, independent gaps (each its own follow-up): -- **OpcUaClient — MISSING FACTORY (real bug).** `src/Server/.../Host/Drivers/DriverFactoryBootstrap.cs:98-107` registers AbCip/AbLegacy/FOCAS/Galaxy/Modbus/S7/TwinCAT but **OMITS OpcUaClient** — and the `OpcUaClient` driver project has **no `…FactoryExtensions` class at all** (only `OpcUaClientDriver.cs` + `OpcUaClientDriverProbe.cs`). So OpcUaClient is **always stubbed** (`"no factory for driver type OpcUaClient … falling back to stub"`) — completely non-functional. Fix: add an `OpcUaClientDriverFactoryExtensions` (CreateInstance parsing `driverConfigJson`→`OpcUaClientDriverOptions` + `new OpcUaClientDriver(options, id, …)`, mirroring `ModbusDriverFactoryExtensions`) + register it in the bootstrap. **This is the cheapest path to a live value** (OpcUaClient is direct-ref: `TagConfig.FullName="ns=3;s=FastUInt1"` → subscribes to opc-plc → publishes → router delivers). opc-plc sim is up at `opc.tcp://10.100.0.35:50000` (`ns=3;s=FastUInt1`). -- **Protocol drivers (Modbus/S7/AbCip/…) — equipment-tag↔driver-subscription linkage.** The Modbus driver subscribes by **tag NAME from `DriverConfig.Tags`** (`_tagsByName`, keyed by `t.Name`), but an equipment tag's `TagConfig` (region/address, via `ModbusTagConfigModel`) carries **no top-level `FullName`**, so `ExtractTagFullName` falls back to the raw JSON blob → never matches a `DriverConfig.Tags` name → `SubscribeAsync` silently skips it (`if (!_tagsByName.TryGetValue(...)) continue;`). The equipment tag's register config is never connected to the driver. This is the bigger "equipment tags work for protocol drivers" gap (needs the deploy to build the driver's tag table from equipment tags, or the equipment-tag FullName to address a register directly). -- **Galaxy — needs a reachable mxaccessgw.** Direct-ref (FullName = MxAccess ref), so the router would deliver — but the gateway must be up + `ApiKeySecretRef` resolvable (dev `MX_API_KEY` unset). - -> Recommended next: build the OpcUaClient factory (small, fixes a real bug, gives the first live value). Then the protocol-driver linkage (larger). - -### 🟡 Phase B — Native alarms on the equipment-tag path (NOT started) - -Galaxy implements `IAlarmSource` (native MXAccess alarms), but that wiring lived **only** on the retired SystemPlatform-mirror path (`GenericDriverNodeManager` registers an alarm sink per `MarkAsAlarmCondition` variable keyed by `FullName`, routes `OnAlarmEvent.SourceNodeId` to it). After Phase A, **native Galaxy alarms have no materialization path** until this is ported. - -**Work:** teach `MaterialiseEquipmentTags`/`OtOpcUaNodeManager` to register an alarm-condition sink for `IsAlarm` equipment tags (keyed by `FullName`) and subscribe the owning driver's `IAlarmSource`, routing `OnAlarmEvent.SourceNodeId == FullName` → sink. Reuse the existing Part 9 `AlarmConditionState` materialization + `alarm-commands`/ack plumbing (scripted alarms already use it). Extract `GenericDriverNodeManager`'s forwarder so both paths share it. Mirror its unsubscribe-before-rewalk for `IRediscoverable` redeploys. - -### 🟢 Phase C — Server-side historian / `HistoryRead` (NOT started) - -Driver-agnostic server-side history backend over the existing Wonderware Historian reader (`Historian.Wonderware.Client` already implements `IHistoryProvider`). No mxaccessgw history RPC needed. - -**Work:** add an OPC UA `HistoryRead` service override in `OtOpcUaNodeManager` (none exists today) → a router that, for an `IsHistorized` node, resolves a historian tagname (default = the tag's driver `FullName`, optional override) and calls the registered `IHistorianDataSource`. Apply `DriverAttributeInfo.IsHistorized` → UA `Historizing` + `AccessLevel.HistoryRead` at materialization (currently hardcoded `false`). Config: a server `Historian` section (Null default; Wonderware when configured), mirroring the existing `AlarmHistorian` pattern. Works for any `IsHistorized` tag, not just Galaxy. - ---- - -## Dev rig state - -docker-dev runs **locally** (Docker 29.4.0 on this Mac); login disabled in dev (`Security__Auth__DisableLogin: "true"`); central UI at `http://localhost:9200`, SQL at `localhost:14330` (`sa`/`OtOpcUa!Dev123`). Currently on the **Phase-A build** with the clean-break migration applied. Verification artifacts left in place: `MAIN-galaxy-eq` Galaxy driver + `GalaxyTestTag` under the `nw-uns` Equipment namespace. Prior 1,391-tag Galaxy config is restorable from `OtOpcUa-prePhaseA-20260612-224908.bak` (in the SQL volume `/var/opt/mssql/backup/`). Rebuild central from a branch: `docker compose -f docker-dev/docker-compose.yml up -d --build migrator central-1 central-2`. diff --git a/docs/Historian.md b/docs/Historian.md index e97e9ba3..e33ab8c0 100644 --- a/docs/Historian.md +++ b/docs/Historian.md @@ -394,9 +394,10 @@ Array writes, multi-dimensional arrays (ValueRank>1), and per-element historizat --- -## Closed stillpending.md §2 items (Phase 4 / Phase 4b) +## Closed backlog §2 items (Phase 4 / Phase 4b) -The following items from the `stillpending.md` backlog §2 were closed by earlier +The following items from the backlog §2 (archived at +`docs/plans/2026-06-18-stillpending-backlog-audit.md`) were closed by earlier phases and are recorded here so future audits don't re-flag them. | Item | Closed by | Commit | diff --git a/stillpending.md b/docs/plans/2026-06-18-stillpending-backlog-audit.md similarity index 100% rename from stillpending.md rename to docs/plans/2026-06-18-stillpending-backlog-audit.md diff --git a/HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md b/docs/plans/2026-06-27-historian-gateway-integration-issues.md similarity index 100% rename from HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md rename to docs/plans/2026-06-27-historian-gateway-integration-issues.md diff --git a/looseends.md b/looseends.md deleted file mode 100644 index 277cd4f4..00000000 --- a/looseends.md +++ /dev/null @@ -1,54 +0,0 @@ -# Loose ends - -State as of 2026-05-18, after the #9–#29 task-list run. Everything on the -formal task list is shipped except #20; the items below are what genuinely -remains, plus follow-ups surfaced during the run. - -## Open task - -- **#20 — D.1 dev-rig rollout smoke.** A full 3-service deployment - (gateway + worker + server + Wonderware historian sidecar): deploy the - refreshed binaries, run `scripts/install/Refresh-Services.ps1`, exercise - alarms end-to-end, and capture the rollout artifact. The code blockers - were cleared by #18; the act itself needs the physical AVEVA dev rig and - cannot be produced from a dev box. Runbook context in - `docs/plans/alarms-worker-wiring-plan.md`. - -## Follow-ups surfaced during the run - -- **~~C.1 live SDK binding.~~** DONE (code). `SdkAlarmHistorianWriteBackend` - (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Wonderware/Backend/`) now - writes via the real entry point `HistorianAccess.AddStreamedValue(HistorianEvent, - out error)` in `aahClientManaged`. Two plan corrections found while pinning it: - (a) `ArchestrAAlarmsAndEvents.SDK` has no writer — it's a WCF query proxy; - (b) writes need their own `ReadOnly=false` connection, not the shared read - pool. Remaining: the live-rig write smoke (the `Live_*` tests are still - `Skip`-gated) — folds into #20 / D.1. - -- **~~#24 Shelve-method routing.~~** DONE. Acknowledge / Confirm already - routed; OneShotShelve / TimedShelve / Unshelve now route via the native - `AlarmConditionState.OnShelve` / `OnTimedUnshelve` hooks wired in - `DriverNodeManager.MarkAsAlarmCondition` (scripted alarms get a shelvable - `ShelvedStateMachine` subtree created before `alarm.Create`). The three - per-instance shelve method NodeIds are indexed so the Call gate resolves - them to `OpcUaOperation.AlarmShelve`. `AddComment` also now routes to the - engine (gated at the `AlarmAcknowledge` tier) — `phase-7-status.md` Gap 1 - is fully closed. Remaining: address-space materialisation of the shelve - method nodes is best confirmed by a live OPC UA browse (pairs with the - G6 / D.1 rig steps). - -- **mxaccessgw alarm epic branch.** The alarm subsystem work (A.2/A.3/A.4 - + the two production-gap fixes from #18) lives on the mxaccessgw branch - `docs/alarm-client-wm-app-finding`. It is NOT merged to mxaccessgw's main. - Whether/when to merge the alarm epic to main is an open release decision. - -- **#15 operator/lab GA gates.** Two v2 GA gates are manual lab steps, not - automatable here: the OPC UA CTT (Compliance Test Tool) pass and the - deployment-checklist signoff. Documented in - `docs/plans/v2-ga-lab-gates-plan.md`. - -## Done — for reference - -The 5 Phase 7 gaps discovered mid-run (#24–#28) were all completed and -merged; no Phase 7 gaps remain open. Add any new follow-ups above as they -are spun out. diff --git a/pending.md b/pending.md deleted file mode 100644 index d1f5defd..00000000 --- a/pending.md +++ /dev/null @@ -1,122 +0,0 @@ -# Pending — open follow-ups & deferrals - -As of 2026-06-16. master HEAD `1164d423` (synced with origin; feature branch `feat/stillpending-phase-5-probes` ff-merged + deleted). Working tree is clean except the expected DISK-ONLY files: `docker-dev/docker-compose.yml` (M — uncommitted rig config, never staged) and `pending.md` (M — these notes, never staged), plus two untracked pre-existing `docs/plans/2026-06-14-write-outcome-self-correction-plan.md*` docs. HARD RULE: never `git add .`; never stage `pending.md` / `current.md` / `docker-dev/docker-compose.yml` / `sql_login.txt` / `src/Server/ZB.MOM.WW.OtOpcUa.Host/pki/`; never commit secrets. - -**CERT-ACTION AUDIT LOGGING + OpcUaProbeResult DEAD-LETTER TIDY MERGED+PUSHED master `3eb370d4` (2026-06-19, ff, 7 commits incl. 2 review-polish).** Branch `feat/cert-audit-and-probe-tidy` ff-merged + deleted. Design `docs/plans/2026-06-19-cert-audit-and-probe-deadletter-tidy-design.md` + plan/`.tasks.json`. Two independent items, subagent-driven (T1 cert-core ∥ T3 probe-drops dispatched concurrently — disjoint projects; T2 wire-in after T1; T1 spec+code review, T3 code review, final integration review = SHIP). **(1) Cert-action audit logging (was stillpending §A.7 deferred):** the FIRST live structured `AuditEvent` emit site. New `ActorAuditWriter : IAuditWriter` (`AdminUI/Audit/`, resolves the `AuditWriterActor` cluster singleton from `ActorRegistry` + fire-and-forget `Tell`, mirrors `AdminOperationsClient`) + pure `CertAuditEvents.Build` factory (`Category:"Certificate"`, `Action` Trust/Untrust/Delete, success rows OMIT the `error` key); `CertificateStoreManager` audits at the choke point on EVERY return path (success + all failures, via a `DeleteCore` refactor); `Certificates.razor` sources the actor from `AuthenticationState`. `AuditEvent` is a `ZB.MOM.WW.Audit` record (init-setters; `AuditOutcome.Success/Failure/Denied`; `CorrelationId` `Guid?`); package ships `NoOpAuditWriter.Instance` (test default). NO EF migration (`ConfigAuditLog` table + `AuditWriterActor` pre-existed), NO Commons/proto change. **(2) OpcUaProbeResult dead-letter tidy (Phase-4 follow-up (i)):** added `Receive(_=>{})` drop to `DriverHostActor` (all 3 behaviours: Steady/Applying/Stale), `HistorianAdapterActor`, `ScriptedAlarmHostActor` (mirrors `PeerProbeSupervisor`); guard test fails-without/passes-with. **Build clean; AdminUI.Tests 510/510, Runtime.Tests 273/273.** **LIVE `/run` PASS (agent-driven, login disabled):** rebuilt central-1/central-2 → both boot healthy (DI resolves the new `CertificateStoreManager(IConfiguration, IAuditWriter)` + `ActorAuditWriter`→singleton — the integration review's #1 risk), `AddressSpaceApplier: applied plan (added=91)`, ZERO `OpcUaProbeResult` dead-letters on both nodes (before AND after the run). **CERT-AUDIT END-TO-END PROVEN:** seeded a throwaway rejected cert, clicked Trust on the Certificates page (:9200) → `ConfigAuditLog` row `multi-role-test | Certificate:Trust | Success | | {"store":"rejected→trusted","thumbprint":…}` (no error key — the success-omit polish), through the full live cluster seam (razor→manager→ActorAuditWriter→AuditWriterActor singleton→shared SQL); seed cert cleaned up. **Session also reconciled #400 (within-timestamp tie-cluster paging — already SHIPPED `2e6c6d3a`/`0f929ae6`, stale pending banners corrected) + RECLASSIFIED H2 (HistoryUpdate service) as INFRA-GATED/DEFERRED** (no historian backend can durably insert/replace/delete — same boundary as H5b; see stillpending §A.1). DEFERRED follow-up (unchanged): cert actions still also want to be reachable via an audit-query UI, but the DB rows now exist. **MINOR review nits (non-arch, from the final integration review — recorded so they're not lost):** (a) two UNREACHABLE `Certificates.razor` fallthrough arms (`"cannot delete from {Kind}"` / `"unknown action"`) build a `CertActionResult.Fail` INSIDE the razor switch, bypassing `CertificateStoreManager` → those (unreachable) failures aren't audited; add a comment or route through the manager; (b) `OpcUa:PkiStoreRoot` is read in BOTH `Certificates.razor:130` and `CertificateStoreManager` ctor — centralize on the manager. Both Minor, no rush. - -**STILLPENDING BACKLOG — PHASE 5 (Test-Connect protocol handshakes) MERGED+PUSHED master `1164d423` (2026-06-16, ff, 15 commits) + LIVE `/run` 9/11 PASS.** Branch `feat/stillpending-phase-5-probes` ff-merged + deleted. Design `…-phase-5-probes-design.md` (`1f2d32ac`) + plan `…-probes.md`/`.tasks.json` (`2f7c6bf9`). **Scope locked = ALL 8 TCP-only probes get real handshakes (best-effort, AskUserQuestion).** Replaced bare `socket.ConnectAsync` with a protocol handshake after a TCP preflight, so a live-but-rejecting device reads RED not false-green; **Historian.Wonderware probe was ALREADY a real Hello/HelloAck handshake → untouched** (the §2 "task 9" note was stale). New 3-way contract: TCP-fail→`"Connect failed: {err}"` (unchanged) · handshake-ok→`Ok=true`+latency+descriptive msg · TCP-ok-but-rejected→`"Reachable at {host}:{port} but {proto} handshake failed: …"` (the new behavior) · timeout→`"Probe timed out"` (unchanged). `IDriverProbe`/`DriverProbeResult`/DI/csproj UNCHANGED; NO migration. Per-probe (no shared scaffold — keeps the 8 driver projects disjoint/parallelizable; subagent-driven 2 waves of 4 concurrent, Wave-1 bundle review + Wave-2 bundle review + final integration review = **SHIP**, all 5 invariants pass). **Handshakes:** Modbus FC03 via `ModbusTcpTransport` (exception PDU still=real device) `9b909002`; OpcUaClient `DiscoveryClient.GetEndpointsAsync` (no session/cert) `957a63cf`; S7 `Plc.OpenAsync` COTP+S7-setup `9a8336ff`; AbCip/AbLegacy `libplctag` Tag init (CIP-level error=reachable) `0c08b152`/`21f3e8fe`; TwinCAT `AdsClient.ReadStateAsync` DEGRADE-guarded `b663ae6e`; FOCAS `cnc_allclibhndl3` via a NEW direct `[DllImport("fwlib32")]` DEGRADE-guarded `5ed0276f`; Galaxy gRPC `GalaxyRepository.TestConnection` `2d688c2a`. **DEGRADE principle (TwinCAT/FOCAS): an env that can't ATTEMPT the handshake → `Ok=true` "TCP reachability only" (never worse than TCP); only a real device rejection is RED** — both empirically proven on this headless box (AdsServerException / DllNotFoundException → degrade). Code-review fixes: timeout-CTS bound on OpcUaClient/AbCip/AbLegacy + IPv4 preflight + sw.Stop (`9bfbbb0f`/`5df3c732`). **LIVE `/run` (agent-driven, sandbox-off so the rig is reachable; brought up local central-1 @ :4840 as a real OPC UA server): 9 PASS / 2 SKIP / 0 FAIL** — Modbus green vs sim `10.100.0.35:5020` + red vs non-Modbus; OpcUaClient green vs opc-plc `:50000` AND vs the LOCAL OtOpcUa server (3 endpoints) + red vs SQL/non-OPC-UA; Galaxy green vs gateway `10.100.0.48:5120` + red vs non-gRPC. **S7/AbCip GREEN SKIPPED — fixtures down** (no `lmxopcua-fix` on the Mac; unit-proven + reviewed). **LIVE `/run` CAUGHT + FIXED 2 REAL GALAXY BUGS unit-tests+2-reviews MISSED (`1164d423`): (1) `Retry={MaxAttempts=1}` → invalid Polly RetryStrategyOptions → the probe FAILED on EVERY live gateway (removed it; TCP-preflight+deadline already fail-fast); (2) a rejected key surfaces as a typed `MxGatewayAuthenticationException`, NOT a raw `RpcException`, so "auth-rejection=reachable" was bypassed → catch the typed auth/authorization exceptions → `Ok=true`.** Galaxy UseTls must match the endpoint scheme (`UseTls:false` for `http://` h2c — same constraint the driver enforces). New `DriverProbeHandshakeE2eTests` (skip-gated cross-protocol green/red discrimination). Driver tests all green (Modbus 277, OpcUaClient 122, S7 121, AbCip 276, AbLegacy 168, TwinCAT 145, FOCAS 196, Galaxy 272+3skip). Docs `docs/drivers/TestConnectProbes.md`. **NEW FOLLOW-UPS:** (i) S7+AbCip happy-path live-verify (bring up `lmxopcua-fix up s7 s7_1500`/`up abcip controllogix` from the Windows VM); (ii) AbLegacy/TwinCAT/FOCAS happy-path live-verify (no PLC5/SLC sim, no ADS target, no CNC+FWLIB). **Backlog Phases 6–8 + Phase 4b remain.** - -**STILLPENDING BACKLOG — PHASE 4 (data-type tier: driver data-type & robustness coverage) MERGED+PUSHED master `050f5c4b` (2026-06-16, ff, 10 commits) + LIVE deploy-regression-clean.** Branch `feat/stillpending-phase-4-driver-datatypes` ff-merged + deleted. Design `…-phase-4-driver-datatypes-design.md` (`57d9f1b3`) + plan `…-driver-datatypes.md`/`.tasks.json` (`97c087f2`). **SCOPE NARROWED FROM ROADMAP after grounding** (AskUserQuestion locked both): **S7 wide types DEFERRED to Phase 4b** (driver reads via S7.Net address-strings → wide types need an `S7.Net.Types` byte-buffer read-path refactor; python-snap7 dev sim's wide-type support unconfirmed) and **ALL array work DEFERRED to a dedicated slice** (the live equipment-node materialiser `EnsureVariable` hard-wires `ValueRank=Scalar` with NO array params + `EquipmentTagPlan` carries no array metadata + S7/AbLegacy have no array READ support → surfacing arrays end-to-end is a coherent cross-cutting feature: plan-metadata + `EnsureVariable` contract change + per-driver read + Phase-6 UI, NOT a per-driver flag flip). Shipped 5 changes across 3 disjoint projects (subagent-driven, T1/T2/T4/T5 implementers dispatched CONCURRENTLY — disjoint projects, each commits by path — T3 after T2 same-file; every task spec/code reviewed + final integration review = **SHIP**): **T1 Modbus Int64/UInt64 node type** (`bd8fee61`): `ModbusDriver.MapDataType:1517` split `Int64⇒DriverDataType.Int64`/`UInt64⇒UInt64` (both ALREADY existed in the enum + `ResolveBuiltInDataType` ALREADY mapped `"Int64"`/`"UInt64"` → the gap was purely the driver lying Int32; codec round-trips long/ulong UNCHANGED; removed stale `Driver.Modbus-007` doc) — 271/271. **T2 FOCAS fail-fast factory** (`5c5aaef6`): new `IFocasClientFactory.EnsureUsable()` (no-op Wire/Fake, throws Unimplemented) called early in `InitializeAsync` → `unimplemented`/`none`/`stub` backend now FAULTS at init not first-read (operator footgun) — 191/191. **T3 FOCAS position scaling** (`49730752`+`88d9e3b1` stale-comment review fix): per-device `PositionDecimalPlaces` config (default 0 = unchanged) divides the 4 position fields by `10^dp` at `PublishAxisSnapshot`; widened `LastFixedSnapshots` `int→double` (the 6 refs through it are all already `Float64`; Int32 fixed-tree vars resolve from separate struct caches → no node-type change); `cnc_getfigure` auto-fetch stays deferred. **T4 Historian Total aggregate** (`5e27b5f7`): `ReadProcessedAsync` Total → query Average + post-multiply each bucket by `interval.TotalSeconds` (time-integral identity; Wonderware AnalogSummary has no Total column); null/bad buckets carried through → BadNoData — 35/35. **T5 Historian poison dead-letter cap** (`fcb38014`): `SqliteStoreAndForwardSink` `maxAttempts` ctor knob (default 10, guard >0) + `QueueRow`/`ReadBatch` carry `AttemptCount` (column pre-existed, NO migration) → RetryPlease dead-letters at `AttemptCount+1>=maxAttempts` (counts row leaving queue, no drift); threaded `AlarmHistorianOptions.MaxAttempts` + `AddAlarmHistorian` DI + `Validate()` warning — 27/27 + Runtime 267/267. **NO contract / `DriverDataType` / `IOpcUaAddressSpaceSink` / EF-migration change.** **Build clean (0 err).** **LIVE `/run`:** rebuilt central-1/central-2 to the branch → both healthy + `Phase7Applier: applied plan (added=90, rebuild=True)` + all materialisation + OPC UA serving, NO fatal/unhandled exception (the new driver DLLs + `AddAlarmHistorian maxAttempts` DI load+deploy clean). **Modbus-Int64 end-to-end node advertisement: UNIT-PROVEN** (271 tests + the equipment-node `"Int64"→DataTypeIds.Int64` resolver) — full live authoring DEFERRED (docker-dev seed has NO Modbus driver; sim `10.100.0.35:5020` reachable from the Mac but not seeded). **NEW FOLLOW-UPS:** (i) Phase-2 `OpcUaProbeResult` INFO dead-letters at `driver-host`/`historian-adapter`/`scripted-alarm-host` (those 3 actors subscribe the redundancy topic but don't drop the probe result like `OpcUaPublishActor` does — benign, capped at 10 then 5-min-suppressed, NOT a Phase-4 change — tidy by adding the same `Receive(_=>{})` drop); (ii) full Modbus-Int64 live authoring; (iii) **Phase 4b** = S7 wide types + Timer/Counter + the dedicated array slice + remaining §2 structural (AbCip/TwinCAT UDT, AbLegacy/TwinCAT bit-index RMW, Galaxy nesting, OpcUaClient event-history, FOCAS cnc_getfigure). **Backlog Phases 5–8 remain.** - -**STILLPENDING BACKLOG — PHASE 3 (OPC UA standards completeness: H4 alarm Enable/Disable + H2 HistoryUpdate permission-bit + H6 native-ack→AVEVA) MERGED+PUSHED master `7eeb9fb0` (2026-06-15, ff, 16 commits) + LIVE deploy-regression-clean.** Branch `feat/stillpending-phase-3-opcua-standards` ff-merged + deleted. Design `…-phase-3-opcua-standards-design.md` (`40b883ef`) + plan `…-standards.md`/`.tasks.json` (`6ab3d863`). Subagent-driven, every task spec+code reviewed (serial for high-risk node-manager/driver/contract tasks) + final integration review = **SHIP** (all 7 cross-cutting checks PASS, incl. the load-bearing native-ack round-trip + node-id seam). **H2-bit** (`c236263e`): `NodePermissions.HistoryUpdate = 1<<12` (int-backed flags enum, NO migration; excluded from all composites — service stays deferred) + `TriePermissionEvaluator:86` maps HistoryUpdate→its own bit (was HistoryRead — closed the read⇒update hole). **H4** (`328bd1b9`+`be6858ba`): wired the missing node-manager `OnEnableDisable` (SDK `ConditionEnableEventHandler(ctx,cond,bool enabling)`, verified via reflection) → scripted = AlarmAck-gated `HandleAlarmCommand("Enable"/"Disable")`→engine (already handled it); native→`BadNotSupported`; native-check via lock-guarded `IsNativeAlarmNode`. **H6**: `418663b3` thread `bool isNative` through the SINGLE shared `MaterialiseAlarmCondition` (Phase7Applier true@:204/false@:295 → `_nativeAlarmNodeIds`); `ed941c51` `AlarmAcknowledgeRequest.OperatorUser` (additive) + GalaxyDriver forwards + ScriptedAlarmSource honors-else-`opcua-client`; `a6d9de09`+`87dd65b9` native `OnAcknowledge`→NEW `NativeAlarmAckRouter` (AlarmAck-gated `HandleNativeAlarmAck`, principal=DisplayName; scripted ack unchanged→engine); `93d9160d` `DriverHostActor` inverse map `conditionNodeId→(driver,FullName)` (lock-step w/ `_alarmNodeIdByDriverRef`, both keyed by `EquipmentNodeIds.Variable(EquipmentId,FolderPath,Name)`) + primary-gated `RouteNativeAlarmAck`→`DriverInstanceActor`→`driver.AcknowledgeAsync` (fire-and-forget, Connected-only); `30315185` host DI maps `NativeAlarmAck→RouteNativeAlarmAck` (Runtime has NO OpcUaServer dep). GOTCHAS: native-check MUST use lock-guarded `IsNativeAlarmNode` (raw set read races a concurrent RebuildAddressSpace — OnTimedUnshelve bypasses the Call-service lock); node-id seam airtight because BOTH sides derive from the same `EquipmentNodeIds.Variable` over the same composition (null/empty FolderPath normalizes identically). NO EF migration; only additive contracts. Also fixed a PRE-EXISTING Phase-0 miss (`8899d6e0`): 2 Galaxy guard tests asserted "PR 4.4"/"PR 4.W" text that Phase 0 `b4af9e7f` removed → updated to current guard text. **Build clean; Core 237/237, OpcUaServer 201/201, Runtime 267/267, Galaxy 264/264(+3 live-gw skip).** **LIVE `/run`:** rebuilt central-1/central-2 to the branch → healthy startup + `Phase7Applier: applied plan (added=90, rebuild=True)` + `scripted alarms materialised (alarms=1)` + OPC UA serving (ServiceLevel 250) — substantial node-manager/DriverHostActor/host changes deploy+materialise+serve with NO regression. **H4 client-Disable BLOCKED: Client.CLI has no enable/disable command** (plan assumed one) → H4 fully unit-proven; CLI enable/disable = a FOLLOW-UP. **H6 full native-ack→AVEVA OPERATOR-GATED**: needs a Galaxy NATIVE alarm (dev rig's 1 alarm is scripted) + the real gateway/AVEVA on `10.100.0.48`; round-trip wiring unit+integration-proven. Phases 4–8 remain. - -**STILLPENDING BACKLOG — PHASE 2 (H3 health-aware redundancy ServiceLevel) MERGED+PUSHED master `4af8e65a` (2026-06-15, ff, 14 commits) + LIVE-PROVEN on the docker-dev 2-node rig.** Branch `feat/stillpending-phase-2-servicelevel` ff-merged + deleted. Design `docs/plans/2026-06-15-stillpending-phase-2-servicelevel-design.md` (`05283533`) + plan `…-servicelevel.md`/`.tasks.json` (`7605f4d8`). **LIVE `/run` ACCEPTANCE PASSED (agent-driven, login disabled):** rebuilt central-1+central-2 to the branch (`docker compose -f docker-dev/docker-compose.yml up -d --build central-1 central-2`); read `Server.ServiceLevel` (NodeId `i=2267`) via Client.CLI on :4840 (leader) + :4841 (follower). **BEFORE (old code): 240 / 100** (legacy role-only). **AFTER (branch, healthy): 250 / 240** (calculator path LIVE — the follower 100→240 + leader 240→250 is the discriminator vs legacy; proves dbHealthProbe Ask delivery + calculator invocation + NodeId self-match + the +10 leader bonus all live). **DB-down (stopped shared `otopcua-dev-sql-1`): 110 / 100** (`DbHealthProbe: probe failed` → `Stale=!DbReachable` → 100-tier, leader +10=110 — health-aware demotion). **Recovered (SQL restarted): 250 / 240.** Rig left healthy. (NOTE: stopping the SHARED SQL demotes BOTH nodes; single-node asymmetric isolation would need a per-node DB network block the rig doesn't cleanly support, but the both-demote + recover proof + the steady-state 250/240-vs-legacy-240/100 discriminator fully validate the wiring.) Wires the existing-but-never-invoked `ServiceLevelCalculator` into the live per-node publish path so a DB-unreachable / OPC-UA-unreachable node drops below its role level. **Subagent-driven, every task spec+code reviewed (serial for the high-risk actor tasks) + a final integration review = SHIP.** Commits: `ff0f62db` move `ServiceLevelCalculator`+`NodeHealthInputs` → `Core.Cluster` (`ZB.MOM.WW.OtOpcUa.Cluster.Redundancy`, cycle-free — Runtime already refs Core.Cluster, AVOIDS the deliberate no-Runtime→ControlPlane boundary); `3e609a2b`+`a9ff1a64` `OpcUaPublishActor.RecomputeServiceLevel` calls the calculator (DB+Stale+leader) with a **back-compat legacy seam** (no `dbHealthProbe` wired → old role-only 240/200/100/0, keeps the 2 pre-existing tests green) + Detached/no-entry guard → 0 + always-publish-first-even-0; `cf278035`+`5382eea9` `OpcUaProbeOk` from **peer-probes-me** (consume `OpcUaProbeResult` where NodeId==local; absent/stale verdict→true, only a RECENT observed false demotes); `37b32a56` periodic `HealthTick` Ask→PipeTo the local `DbHealthProbeActor` (Ask-timeout→Reachable=false fail-safe-demote); `f41e957e`+`5a064e08` NEW `PeerProbeSupervisor` spawns one `PeerOpcUaProbeActor` per non-self non-Detached driver peer (monotonic gen-suffix child names + Watch/Terminated-by-ref prune — stale Terminated can't evict a fresh same-peer child); `70e6d3d2`+`393b746d` docs/Redundancy.md WIRED; `4c78dcd3` DI wiring (`dbHealthProbe: dbHealth` into `OpcUaPublishActor.Props` + spawn `PeerProbeSupervisor` + `PeerProbeSupervisorKey`); `4af8e65a` supervisor explicitly drops co-mingled `OpcUaProbeResult` (no unhandled-msg noise). **Tiers (documented, now reachable): 0 down/Detached · 100 DB-down⇒stale · 200 DB-ok-stale · 240 healthy follower · 250 healthy leader. Behavior change: healthy Secondary 100→240** (both healthy nodes 240/250, leader preferred by +10 — clients still pick primary). `Stale = !DbReachable || (now-dbAsOf)>30s || (now-snapshotAsOf)>30s`. **Integration review's #1 risk (NodeId canonical form across the probe→publish seam) = PASS**: both `RedundancyStateActor.ToNodeId(member.Address)` + `ClusterRoleInfo.LocalNode` derive `host:port` from the same `PublicHostname`/`Port` that configures Akka remoting (same class as [[project_redundancy_state_delivery]], consistent here). NO EF migration, NO Commons contract change. **Build clean; ControlPlane.Tests 59/59, Runtime.Tests 264/264.** **LIVE `/run` GATE (Task 8, high-risk acceptance) PENDING:** on docker-dev MAIN (central-1/central-2 2-node warm), read `Server.ServiceLevel` on :4840/:4841 → leader 250 / follower 240; isolate ONE central node from SQL (docker network disconnect — do NOT stop the shared SQL container) → its ServiceLevel drops to 100 (below the healthy peer's 240); reconnect → climbs back. Agent drives docker-dev directly (login disabled). Phases 3–8 of the backlog remain (each gets its own plan when reached). - -**STILLPENDING BACKLOG — PHASE 0 + PHASE 1 (H1+H5) MERGED+PUSHED master `c6a543d1` (ff, 15 commits) 2026-06-15; H1 LIVE-PROVEN on docker-dev.** Then `907005d2` (CLAUDE.md: local rig login-disabled → run /run yourself) + `4bd7180e` (fix docker-dev seed: stop seeding retired SystemPlatform namespace). **LIVE VERIFY:** rebuilt local rig to master, drove AdminUI `localhost:9200` (login disabled), toggled `layer0-logcheck` (EQ-55297329838d) Historize → Deploy → `Phase7Applier: applied plan (added=0, removed=0, changed=1, rebuild=True)` + `sealed (acks=6)` — a PURE changed-only deploy now rebuilds (pre-H1 = `rebuild=False`); reverted = 2nd proof; config pristine. **Blocker found+fixed:** docker-dev seed still inserted `Kind='SystemPlatform'` (retired) → 500s /deployments + fails publish; ran sanctioned CleanupSystemPlatformNamespaces SQL on dev DB + fixed seed. H5 threading flows through real deploy; durable IHistoryWriter.Record silent (NullHistoryWriter infra-gated). (Historical IN-PROGRESS detail below.) Driven by the approved roadmap `docs/plans/2026-06-15-stillpending-backlog-design.md` + plan `…-phase-0-1.md` (full backlog audit = `stillpending.md`). DONE on the branch (subagent-driven, each task spec+code reviewed, all green): **Phase 0 hygiene** (`b4af9e7f` 7 stale comments; `a9d267c9` security.md write-outcome + benign DraftSnapshot/LeaderChanged notes; `50a2fdf3` marked 18 shipped `.tasks.json` completed). **H1 — changed-only deploys now take effect** (`1e95856b` Phase7Applier `needsRebuild` includes `Changed*` counts, ChangedDrivers excluded by design; `1dc71369` count removed tags/vtags in RemovedNodes [review follow-up]; `ada01e1a` VirtualTagHostActor stop+respawns a child whose plan changed in place; `ebf2f1dd` prune `_planByVtag` on termination + crash-then-change test [review follow-up]). **H5 — vtag Historize honored at runtime** (`fc8121cb` `EquipmentVirtualTagPlan.Historize` + composer reads `v.Historize`, in Equals/GetHashCode; `9c5a0913` artifact decode reads `Historize` byte-parity; `83d3b9f7` planner Historize-toggle test + doc nit; `0c6d4c54` VirtualTagHostActor invokes injected `IHistoryWriter` on historized results [+ Runtime→Core.VirtualTags ProjectReference]; `2f30c54d` thread-safe CapturingHistoryWriter [review follow-up]; `4501f12` thread IHistoryWriter through DriverHostActor + DI `TryAddSingleton` Null default). **DURABLE AVEVA vtag-history sink = INFRA-GATED** (no sidecar live-data WriteDataValues RPC) → seam wired w/ NullHistoryWriter default, flagged on the design defer list. NO EF migration (Historize column + ConfigComposer whole-entity serialization already carry it). REMAINING on branch: Task 10 docs (this commit) + Task 11 final build/test/integration-review + finishing (merge is the user's call). LIVE `/run` gate (user-driven): edit equipment/alarm/vtag → confirm it takes effect without restart; toggle a vtag Historize → host respawns + writer invoked. - -**#401 — AbCip + FOCAS ALARM-PROJECTION POLL-LOOP LEAK FIXED + MERGED + PUSHED master `151b7165` (2026-06-15, ff, 2 commits `6ba59f9d`+`151b7165`).** SIBLING of #399 found by auditing all `IAlarmSource` drivers, but WORSE: `AbCipAlarmProjection` + `FocasAlarmProjection` each `Task.Run` a poll loop per `SubscribeAsync` and survive in-place reconnect, so the actor's per-Connected re-subscribe leaked a LIVE never-cancelled poll loop each reconnect → CPU growth + DUPLICATE OnAlarmEvent transitions (functional, gated behind opt-in alarm projection). Fix (both projections, symmetric): `SubscribeAsync` snapshots existing subs under `_subsLock`, `_subs.Clear()`, adds the new sub, starts its loop, then `_ = RetireAsync(old)` fire-and-forget (Cancel→await loop→Dispose CTS). Snapshot+clear under the same lock `DisposeAsync` uses ⇒ no double-own/double-dispose (reviewer-confirmed airtight for all interleavings). Implemented via opus subagent (TDD fail→pass: subscribe twice → 1 device raise → exactly ONE event, was 2). AbCip 271/271, FOCAS 186/186. Code-reviewed (sonnet) — correct+safe; applied its 2 nice-to-haves (documented the at-most-ONE-duplicate-per-reconnect one-tick overlap residual in RetireAsync XML doc + guarded Cts.Dispose). **AUDIT RESULT: all 4 IAlarmSource drivers now correct — Galaxy (#399), AbCip+FOCAS (#401), OpcUaClient already safe (session-bound + clears `_alarmSubscriptions` on disconnect).** - -**#399 — GALAXY ALARM-SUBSCRIPTION HANDLE LEAK FIXED + MERGED + PUSHED master `01388226` (2026-06-15, ff, 1 commit).** The session-less Galaxy `StreamAlarms` feed survives an in-place reconnect, so `DriverInstanceActor` re-subscribed on every Connected re-entry (dropping its cached handle without an Unsubscribe — sync teardown) and `GalaxyDriver.SubscribeAlarmsAsync` grew `_alarmSubscriptions` by one untracked handle per reconnect (slow unbounded leak; functionally harmless — gate is Count>0, fan-out reads [0] once). Fix: `SubscribeAlarmsAsync` `Clear()`s the set before `Add` (under the existing `_alarmHandlersLock`, atomic w.r.t. the fan-out reader), collapsing to ONE live handle — faithful because there's exactly one consumer per driver instance (factory-per-actor lifecycle). Chosen over async/fire-and-forget actor-side Unsubscribe (disproportionate). NEW regression test `Re_subscribe_collapses_to_a_single_handle_no_accumulation` (TDD-VERIFIED fail-without-fix: releasing the latest handle leaves the feed open if stale handles remain). Galaxy 263 pass/3 skip, Runtime native-alarm 24 pass. Code-reviewed (sonnet) — correct+complete, both nice-to-haves applied (structural-invariant comment + distinct-handle assert). - -**#400 — within-timestamp cursor paging for oversized tie clusters: SHIPPED to master `2e6c6d3a` + review fix `0f929ae6` (2026-06-17).** No longer a documented follow-up. `HistoryPaging.SliceTieCluster` over-fetches the whole tie cluster (a bounded `start==end` read), carves out the next `cap` ties at `skip`, advances the cursor WITHIN the timestamp, and steps off it (`T + 1 tick`) once drained — wired into `OtOpcUaNodeManager.ServeRawPaged` (stall detected when a resume page's last sample is still at the boundary timestamp). New config `ServerHistorian:MaxTieClusterOverfetch` (default 65536) bounds the in-memory buffer with a `Validate()` warning; 6 `SliceTieCluster` unit tests + integration tests `Raw_oversized_tie_cluster_pages_within_the_timestamp` / `Raw_tie_cluster_beyond_overfetch_bound_fails_loudly`; `docs/Historian.md` "within-timestamp paging" section + config-table row updated. DELIBERATE residual backstop: a cluster strictly larger than `MaxTieClusterOverfetch` ties at one identical SourceTimestamp still loud-fails `BadHistoryOperationUnsupported` (a severe data anomaly — bounded on purpose to cap memory; raise the knob or `NumValuesPerNode` to cover it). - -**#398 — RESTART-DRIVER "actor name is not unique" BUG FIXED + MERGED + PUSHED master `c9643f68` (2026-06-15, ff, 1 commit).** `DriverHostActor.HandleRestartDriver` stopped + respawned the child in ONE synchronous handler reusing the base actor name `drv-`; `Context.Stop` is async so the old child was ALWAYS still registered → `InvalidActorNameException` deterministically on every AdminUI Restart press (crashed+restarted the host). Fix: a monotonic `_childSpawnGeneration` counter feeds a `-g` suffix on every spawned child name (children tracked by the `_children` dict via IActorRef, never by path, so the suffix is invisible). Also closes the same-shaped LATENT race in the reconcile path (removed-then-readded instance + a driver-type-change `ToStop+ToSpawn` in one plan — confirmed by review). NEW regression test `RestartDriver_respawns_the_child_without_an_actor_name_collision` (TDD-VERIFIED: fails on the old code with the exact `InvalidActorNameException` via `EventFilter.Exception<…>().Expect(0,…)`, passes with the fix). Runtime.Tests 238/238. Code-reviewed (sonnet) — correct+complete, no Critical/Important. `HandleReconnectDriver` confirmed bug-free (only `Tell`s ForceReconnect). - -**⚠️ OUT-OF-BAND WORKING-TREE CHANGE I DID NOT MAKE (surfaced to user 2026-06-15, NOT touched):** `Directory.Packages.props` is modified (`ZB.MOM.WW.MxGateway.Client` + `…Contracts` bumped `0.1.0`→`0.1.1`) and two new `nuget-packages/ZB.MOM.WW.MxGateway.{Client,Contracts}.0.1.1.nupkg` (dated 2026-06-15 05:39) appeared during this session. These are EXTERNAL (likely a sibling-repo publish / parallel action) — NOT part of my B2 or #398 commits (both clean, verified). Left untouched per the "didn't-create-it → surface, don't act" rule; awaiting user decision whether to commit, revert, or ignore. `nuget-packages/` is NOT gitignored (tracked dir). - -**B4 — docker-dev rig cleanup RESOLVED 2026-06-15 as "LEAVE-AS-IS" (user decision).** The rig keeps this session's verification artifacts on purpose: the historized dev `GalaxyTestTag` (= `TestMachine_002.TestHistoryValue`) + the local-only `docker-dev/docker-compose.yml` env block (`ServerHistorian__*` with `SharedSecret` via `${OTOPCUA_HISTORIAN_SECRET:-}`, + the data-plane `Security__Ldap__GroupToRole__*` map) so A2 (historian) + A3 (native alarms) + B1 (write-outcome) stay re-verifiable. `docker-compose.yml` stays an uncommitted DISK-ONLY file (never staged). No teardown, no commit — matches the earlier "leave the Galaxy verification artifacts as-is" decision. **All buckets (A2/A3 live, B1/B2/B3 code, B4 operational) are now CLOSED.** Remaining are logged follow-ups only: #398 (pre-existing Restart actor-name-not-unique), #399 (minor session-less alarm handle leak), #400 (within-timestamp cursor paging — now SHIPPED `2e6c6d3a`/`0f929ae6` 2026-06-17). - -**B2 — HISTORIAN HISTORYREAD CONTINUATION-POINT PAGING (Raw) MERGED + PUSHED to master `aa1e21f5` (2026-06-15, fast-forward, 3 commits: `94c3ca60` feature, `bea0b482` review fixes, `aa1e21f5` doc).** Server-side time-based paging for the only count-capped arm (Raw): full page (count==NumValuesPerNode>0) ⇒ opaque 16-byte CP + tie-safe resume cursor (`HistoryPaging.ComputeResumeCursor`/`TrimBoundaryDuplicates` — resume from boundary timestamp INCLUSIVE, drop already-emitted head ties → no dup/skip); short page / `==0` (all values) ⇒ no CP. CPs bound to the OPC UA session via the SDK store (`SessionHistoryContinuationStore`, capped by `ServerConfiguration.MaxHistoryContinuationPoints`=100 oldest-evict, disposed on session close); tests inject `InMemoryHistoryContinuationStore` (in-process harness is session-less). `HistoryReleaseContinuationPoints` override frees cursors; unknown/evicted/released ⇒ `BadContinuationPointInvalid`. Processed/AtTime stay single-shot (no client count cap → no full-page signal; spec-conformant). NEW: `HistoryPaging.cs`, `HistoryContinuationStore.cs` + `HistoryPagingTests.cs` (15) + `NodeManagerHistoryReadPagingTests.cs` (7, boot a real SDK server). **182/182 OpcUaServer tests pass, full solution builds clean.** Code-reviewed (sonnet subagent): **C1 (critical) FIXED** — an oversized boundary tie cluster (more ties at one SourceTimestamp than NumValuesPerNode) could silently truncate a resume to GoodNoData and DROP the un-emitted ties (the (timestamp,skip) cursor can't advance past a single timestamp the fixed-(start,end,cap) backend keeps re-returning); now fails LOUDLY per node with `BadHistoryOperationUnsupported` + a log + `docs/Historian.md` "Paging limitation" note + regression test `Raw_tie_cluster_larger_than_page_fails_loudly_not_silently`. Also I3 (Save after HistoryData build → no orphaned cursor), N1 (dropped never-produced `HistoryReadKind` enum + Processed-only state fields — only Raw pages), N3 (ComputeResumeCursor empty-page guard), I1 (documented test-store eventual-consistency). Full within-timestamp cursor paging = task #400, SHIPPED `2e6c6d3a`/`0f929ae6` (2026-06-17) — `HistoryPaging.SliceTieCluster` + `MaxTieClusterOverfetch`-bounded over-fetch; only clusters beyond the bound still loud-fail by design. **Modified-value history (`HistoryReadModified`) is INFRA-GATED, not a code gap**: the AVEVA Wonderware backend (`IHistorianDataSource` TCP sidecar) has no modified-read path; server cleanly rejects per node + `IsReadModified` honoured; doc note added. **B2 COMPLETE.** (B1 write-outcome residuals + B3 GroupToRole doc already on master `a5c0c826`/`063d004f`.) - -**GALAXY LIVE-GW INTEGRATION SMOKES (reopen + write-outcome) MERGED + PUSHED to master `d19deb9b` (2026-06-14, fast-forward, 2 commits).** Closed the two items the user re-opened with a live gateway: the harden-1b **Galaxy-reopen** integration test + the **write-outcome** Galaxy write-commit/persist proof. NEW skip-gated `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/Runtime/GatewayGalaxyLiveReopenAndWriteTests.cs` (gated on `MXGW_ENDPOINT` + `GALAXY_MXGW_API_KEY`, like `GatewayGalaxyAlarmFeedLiveTests`; **skips clean in CI** — verified). **LIVE-PROVEN 2/2 against `http://10.100.0.48:5120` (2026-06-14):** (1) `Live_reopen_recreates_session_and_re_establishes_write_handles` — a live write seeds the writer's item-handle + supervisory-advise caches, then the EXACT `GalaxyDriver.ReopenAsync` primitive pair runs (`GalaxyMxSession.RecreateAsync` → `GatewayGalaxyDataWriter.InvalidateHandleCaches`), both caches drop to 0, and a post-reopen write re-establishes them + returns Good — proving `RecreateAsync` yields a working live session (the un-fakeable part the unit suite can't reach); (2) `Live_supervisory_write_commits_and_persists_to_galaxy` — writes a value distinct from current, then reads it back through a BRAND-NEW session — proving the no-login (`WriteUserId=0`) write COMMITS + PERSISTS only because of `AdviseSupervisory` (the `f05b5d79` fix). Code-review APPROVED (refinement applied: read-back completes an explicit `TaskCompletionSource` from the stream pump; `bufferedUpdateIntervalMs:0`). **DURABLE RUN RECIPE:** the gw `10.100.0.48:5120` IS reachable from this Mac; source the key WITHOUT echoing it — `KEY=$(docker exec otopcua-dev-central-1-1 printenv GALAXY_MXGW_API_KEY); MXGW_ENDPOINT=http://10.100.0.48:5120 GALAXY_MXGW_API_KEY="$KEY" dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests --filter FullyQualifiedName~GatewayGalaxyLiveReopenAndWrite` (central-1's container env carries a valid key while its galaxy session is connected; check with `docker exec … sh -c 'test -n "$GALAXY_MXGW_API_KEY" && echo SET'`). Writes only `TestMachine_002.TestFloat` (the established write-demo attr). - -**RESIDUAL FOLLOW-UPS CLEANUP MERGED + PUSHED to master `fa2388ea` (2026-06-14, fast-forward, 7 commits = 1 plan doc + 6 test/doc).** Closed the offline-doable, non-blocking residuals from open-item #3 (write-pipeline / Phase B / Galaxy-nits sections) — **TESTS + DOCS only, ZERO production change.** Plan `docs/plans/2026-06-14-residual-followups-cleanup-plan.md`. **KEY RECONCILIATION FINDING:** a grounding sweep (3 Explore agents + direct verify) proved MOST of the listed residuals were ALREADY implemented in the harden-1b / write-outcome / `f05b5d79` work — pending.md's residual list was STALE. **Verified already-done (no work needed, file:line evidence):** WP(a) stale/reconnecting writes fast-fail (`DriverHostActor.cs:661` `Receive`, `DriverInstanceActor.cs:229,316` `Receive`); WP(b) `ExecuteSynchronously` already dropped (`DriverHostActor.cs:630` = `TaskContinuationOptions.None`) + `driverIds` forward-map already a `HashSet`; WP(c) FOCAS address cached via `GetOrAdd` (`FocasDriver.cs:247`); WP(d) raw-protocol-blob write test exists (`Primary_routes_write_for_raw_protocol_blob_tag`); WP(e) `[InlineData(2,false)]` future-enum trap already in the parity test; Galaxy(1) `_itemHandles`+`_supervisedHandles` cleared on reconnect (`GatewayGalaxyDataWriter.InvalidateHandleCaches()` ← `GalaxyDriver.ReopenAsync`); Galaxy(2) `SubscriptionEstablished` self-Tell handled (debug-swallow) in all 3 `DriverInstanceActor` states (`:253,:301,:336`). **ACTUALLY DONE this pass (4 items):** Phase B(a) regression test that a `NativeAlarmRaised` during `Reconnecting` is DROPPED not dead-lettered (uses an `AllDeadLetters` EventStream probe so it genuinely guards removal of the `DriverInstanceActor.cs:345` drop handler — the first cut used `parent.ExpectNoMsg` only, which a code review caught as a false-pass gap); Phase B(b) test that `OperatorComment`→`AlarmTransitionEvent.Comment` (and `User=="device"`) through `DriverHostActor.ForwardNativeAlarm`; Phase B(c) severity-bucket doc note in `docs/ScriptedAlarms.md` (each transition snaps to 200/500/700/900 via `NativeAlarmProjector.MapSeverity`, overriding the authored seed, then SDK `EventSeverity` brackets via `OtOpcUaNodeManager.MapSeverity`); Galaxy(3) extracted shared `tests/.../Runtime.Tests/Drivers/StubDrivers.cs` harness (de-dup of the superset `StubDriver`/`WritableStubDriver`/`SubscribableStubDriver`/`StubHandle` from `DriverInstanceActorTests` + `DriverInstanceActorWriteAndSubscribeTests`; `BlockingInitDriver` left local). Built via subagent-driven dev (1 implementer bundle → code-review APPROVED w/ 1 Important = the dead-letter-probe gap, fixed). **Build clean; Runtime.Tests 236/0.** (The harden-1b Galaxy-reopen integration test noted here as deferred is **NOW DONE** — see the Galaxy live-gw smokes banner above, `d19deb9b`.) - -**GALAXY PHASE C — SERVER-SIDE OPC UA HistoryRead MERGED + PUSHED to master `c24abc8a` (2026-06-14, fast-forward, 14 commits = 2 design/plan docs + 12 feature/test/doc).** The server now answers OPC UA **HistoryRead** (Raw / Processed / AtTime over historized variable nodes; **Events** over alarm-owning equipment-folder event-notifier nodes) for any equipment tag flagged historized, driver-agnostically, by dispatching to the registered `IHistorianDataSource` (the Wonderware historian TCP client, which already implemented that interface). **NO EF migration** — the flag rides in the existing `TagConfig` JSON blob (`{"FullName":"…","isHistorized":true,"historianTagname":"…"?}`, the Phase-B `alarm`-object carrier), `historianTagname` defaults to the tag's driver `FullName`. Design/plan `docs/plans/2026-06-14-galaxy-phase-c-historian-*.md`; guide `docs/Historian.md`. Pipeline: `Phase7Composer.ExtractTagHistorize` + byte-parity `DeploymentArtifact.ExtractTagHistorize` → `EquipmentTagPlan.{IsHistorized,HistorianTagname}` → `Phase7Applier` resolves `IsHistorized ? (HistorianTagname ?? FullName) : null` → sink seam (`IOpcUaAddressSpaceSink.EnsureVariable` +`string? historianTagname`) → `OtOpcUaNodeManager.EnsureVariable` sets `Historizing`+`AccessLevels.HistoryRead`+registers `_historizedTagnames`; the node manager overrides the four `CustomNodeManager2` HistoryRead virtuals, block-bridging to the `HistorianDataSource` property (volatile, default `NullHistorianDataSource.Instance`). DI mirrors `AlarmHistorian`: `AddServerHistorian` (config-gated, `ServerHistorian` appsettings section, Null default via `TryAddSingleton`) + `OtOpcUaSdkServer.SetHistorianDataSource` + Host `Program.cs`/`OtOpcUaServerHostedService` Start/Stop wiring. **Graceful degrade:** historized node + Null/unconfigured source → `Good_NoData` (empty), non-historized node → `BadHistoryOperationUnsupported`. **KEY FACTS/GOTCHAS:** the SDK base filters event reads by the `EventNotifier.HistoryRead` bit (variable nodes never reach the events arm); the SDK master propagates `errors[i].Code → results[i].StatusCode` (confirmed by decompiling `MasterNodeManager.HistoryReadAsync`), so the override signals per-node status via `errors[handle.Index]`; the two `HistoryReadResult` types (SDK `Opc.Ua.HistoryReadResult` vs Core.Abstractions DTO) are aliased `SdkHistoryReadResult`/`HistorianRead`; `ReadRawModifiedDetails.IsReadModified` defaults TRUE (Initialize() sets it) so a plain raw read must clear it — modified-history is unsupported; the events arm registers folder→sourceName (= equipment id) only when a non-Null historian is wired at promotion time (Host wires the source at StartAsync before any deployment materialises, so normal boot ordering is correct). Built via subagent-driven dev (T1/T2/T5 standard parallel review, T3/T4 high-risk serial spec→code, + a final integration review READY-TO-MERGE). **Build clean (0 errors); OpcUaServer.Tests 152/0, Runtime.Tests 234/0, Core.Abstractions.Tests 88/0.** **LIVE `/run` GATE (T7) DEFERRED — operator-driven: it needs the Wonderware sidecar + AVEVA Historian on the WW Historian VM `10.100.0.48`, which is NOT on the local docker-dev rig.** When run: author a historized Galaxy tag (`TagConfig` `"isHistorized":true`), set `ServerHistorian:Enabled=true` → sidecar (Host/Port/SharedSecret/TLS), deploy on `MAIN-galaxy-eq`, then `Client.CLI historyread -n "ns=2;s=/" --start … --end …` → samples; a non-historized tag → `BadHistoryOperationUnsupported`. KNOWN follow-ups (non-blocking, documented): single-shot reads only (no server-managed continuation-point paging yet); no modified-value history; no explicit timeout at the block-bridge call site (bounded by the `WonderwareHistorianClient` 30s `CallTimeout`); the StopAsync→DisposeAsync warm-shutdown window returns Good_NoData (mirrors the `SetNodeWriteGateway` pattern). - -**DRIVER-RECONFIGURE-WHILE-FAULTED (#7) MERGED + PUSHED to master `56f73e49` (2026-06-14, fast-forward, 5 commits = 2 docs + fix + review-nit + task-status).** A `DriverInstanceActor` stuck `Connecting`/`Reconnecting` now **adopts a corrected config delivered via `ApplyDelta`** and re-initialises with it, instead of dead-lettering the message and retrying the stale config forever (old workaround = restart node). Design/plan `docs/plans/2026-06-14-driver-reconfigure-while-faulted-*.md`. Mechanism (approach B): a monotonic `_initGeneration` tags each `InitializeAsync`; `InitializeSucceeded(int Generation)`/`InitializeFailed(string Reason, int Generation)` carry it; the `Connecting`/`Reconnecting` result handlers **drop superseded (stale-generation) results** so a corrected config always wins against an old init still in flight; a new `AdoptConfigDuringInit` (wired into both not-connected states) calls `InitializeAsync(newConfig)` (swaps `_currentConfigJson`, bumps the generation, retries immediately) + replies `ApplyResult(true,…)`. Contained to ONE file (`src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs`) + its test — NO host/contract/EF change; `Connected`/`Stubbed` `ApplyDelta` paths untouched; the two result records have ZERO external consumers (grep-verified) so the shape change is fully contained. Built via subagent-driven dev (high-risk full chain: spec ✅ · code ✅ · final integration ✅ — the integration pass traced the no-strand lifecycle in both states, host↔child contract, subscription/desired-refs/alarm safety, health/redundancy, double-adopt, and test fidelity, all clean). **Build clean (0 errors); Runtime 224/224.** **LIVE `/run` GATE DEFERRED per user ("skip the live test part").** When run: put `MAIN-opcua-eq` into a faulted/`Reconnecting` state via a bad `DriverConfig`, deploy a corrected config (`POST http://localhost:9200/api/deployments`, `X-Api-Key: docker-dev-deploy-key`), and confirm from central-1 logs that the driver adopts the new config + connects WITHOUT a node restart. - -**GALAXY PHASE B — NATIVE ALARMS ON THE EQUIPMENT-TAG PATH MERGED + PUSHED to master `f9be3843` (2026-06-14, fast-forward, 12 feature + 2 doc commits).** A Galaxy equipment `Tag` marked as a native alarm via its `TagConfig.alarm` object — `{"FullName":"tag.attr","alarm":{"alarmType":"OffNormalAlarm","severity":700}}`, **NO EF migration** — now materialises a real OPC UA Part 9 `AlarmConditionState` under its equipment folder, driven live by the driver's `IAlarmSource.OnAlarmEvent`; transitions fan out to `/alerts` + the historian (Primary-gated). Design/plan `docs/plans/2026-06-14-galaxy-phase-b-native-alarms-*.md`. NEW seam (mirrors the scripted-alarm seam, reuses the condition sink UNCHANGED): `AlarmEventArgs.Kind` (additive contract; Galaxy populates it) → `DriverInstanceActor` subscribes `OnAlarmEvent` → `AttributeAlarmPublished` → `DriverHostActor._alarmNodeIdByDriverRef` + `NativeAlarmProjector` (transition→`AlarmConditionSnapshot`) → `OpcUaPublishActor.AlarmStateUpdate` → reused `OtOpcUaNodeManager.WriteAlarmCondition`. Built via subagent-driven dev (full per-task review chain). **Build clean; Core.Abstractions 81/0, OpcUaServer 118/0, Runtime 222/0, Galaxy 262/0(+1 live-gw skip).** **THE FINAL INTEGRATION REVIEW CAUGHT A CRITICAL SEAM BUG every unit test missed (fixed in `f9be3843`):** the alarm map is keyed by the dotted `FullName`, but `GalaxyDriver` puts the BARE owning object in `AlarmEventArgs.SourceNodeId` and the DOTTED alarm ref in `ConditionId` (`GalaxyDriver.cs:1148-1149`; `AlarmFullReference`) — so `ForwardNativeAlarm` MUST resolve on `msg.Args.ConditionId` (= `AlarmFullReference` = the authored `FullName`), NOT `SourceNodeId`. The unit test had masked it by setting `SourceNodeId==FullName` (never true in prod); it is now production-shaped (`SourceNodeId="Temp"` ≠ `ConditionId="Temp.HiHi"=FullName`) so it genuinely guards the seam. **LIVE `/run` GATE (T9) NOT YET DONE — user-driven, deferred at merge (user choice "merge now").** When run: author a Galaxy alarm equipment tag whose `FullName` EXACTLY matches the gateway's `AlarmFullReference` (discoverable via the Galaxy picker/a probe), deploy on `MAIN-galaxy-eq`, trip the alarm → Part 9 condition goes active under the equipment + the `/alerts` row appears; clear → inactive. RESIDUAL non-blocking follow-ups (review-surfaced): (a) `DetachSubscription` alarm-coupling doc note + a dead-letter-during-reconnect regression test (WS-4b); (b) ack/comment-path test + assert `evt.Comment` (WS-5); (c) a `docs/ScriptedAlarms.md` note that authored severity (1..1000, seeds the condition at materialise) snaps to the projector's 4-bucket value (200/500/700/900) on the first transition; (d) DEFERRED by design: inbound device-ack (client Ack → `IAlarmSource.AcknowledgeAsync` → AVEVA), driving `SubscribeAlarmsAsync` from the materialised alarm-ref set (Galaxy doesn't need it), AdminUI Galaxy-picker `alarm` pre-fill, carrying raw OPC UA severity end-to-end. Phase **C** (server `HistoryRead`) **DONE — merged `c24abc8a` (2026-06-14); see the Phase C banner at top.** - -**WRITE-OUTCOME SELF-CORRECTION (#5) MERGED + PUSHED to master `1d797c1c` (2026-06-14, fast-forward, 6 commits).** A failed inbound device write now reverts the node to its real pre-write value (compare-and-revert) instead of leaving the optimistic-`Good` phantom. Design `docs/plans/2026-06-14-write-outcome-self-correction-design.md`; plan `…-plan.md`. NEW `IOpcUaNodeWriteGateway`/`NodeWriteOutcome` (Commons) + `ActorNodeWriteGateway` (Runtime, Asks `RouteNodeWrite`, returns the outcome) replace the fire-and-forget `Action` router; `OnEquipmentTagWrite` captures the prior value + fires an off-Lock (`RunContinuationsAsynchronously`) continuation that reverts on a failed outcome IF the node still holds the optimistic value (`ShouldRevert`). Build clean; Commons 39/0, Runtime 201/0, OpcUaServer 111/0. High-risk review verified the prior-value capture against the actual UA-.NETStandard source (`OnWriteValue` fires before `m_value=value`). **LIVE-PROVEN end-to-end** via a local Modbus exception-injector (FC06 reject on HR[20]): authorized failing write → gateway-logged `0x808B0000` reject → node reverts 99→20; authorized success (HR200=7777) stays; anon → BadUserAccessDenied. KEY FINDING: the **Galaxy gateway worker's `ExecuteWrite` is fire-and-forget** (returns OK without awaiting the MXAccess commit), so Galaxy writes ALWAYS return Success at the OPC UA layer and can NEVER surface a device-write failure to this revert — only protocol drivers (which await + return real status) can; this is the same gateway-side limitation noted under "optimistic-write phantom" (out of our scope). Two MINOR deferred follow-ups remain: a Bad-quality blip / OPC UA AuditWriteUpdateEvent on failure, and synchronous structural fail-fast (both explicitly out of scope per the chosen mechanism). - -**HARDEN MILESTONE 1b cluster MERGED + PUSHED to master `945c2380` (2026-06-14, fast-forward, 9 commits).** Follow-ups #3 (data-plane role docs), #4 (write-pipeline review nits), and #6 (Galaxy driver nits) below are CLOSED. Plan: `docs/plans/2026-06-14-harden-milestone-1b-plan.md`. Build clean; Runtime 197/0, FOCAS 185/0, Galaxy 257/0(+1 pre-existing skip); final integration review READY-TO-MERGE. Two MINOR residual follow-ups surfaced by review (non-blocking): (a) a *driver-level* regression test that `GalaxyDriver.ReopenAsync` actually calls `InvalidateHandleCaches` — STILL DEFERRED, needs a live gw (`RecreateAsync` can't be faked), so it's an integration test; (b) stub-driver test-class duplication between `DriverInstanceActorTests` + `DriverInstanceActorWriteAndSubscribeTests` — **DONE `fa2388ea`** (shared `StubDrivers.cs`). - -## STATE SUMMARY (post-compaction pickup) - -**ALL feature work is SHIPPED + PUSHED to master `d19deb9b` (synced with origin). Nothing is blocking.** Milestone 1b (equipment-tag live values: live READ + authorized inbound WRITE across OpcUaClient / the 6 protocol drivers / Galaxy, via the `FullName→NodeId` router) is COMPLETE, and all three Galaxy phases shipped: **A** standard Equipment driver `c3c56172`, **B** native alarms `f9be3843`, **C** server-side HistoryRead `c24abc8a`. The session's cluster-harden / write-outcome-self-correction / driver-reconfigure-while-faulted follow-ups merged too (`945c2380` / `1d797c1c` / `56f73e49`). The six banners above carry each feature's mechanism + gotchas + deferred live gate; the closed open-follow-ups #1–#7 (Phase B, Phase C, data-plane role docs, write-pipeline nits, write-outcome, Galaxy driver nits, reconfigure-while-faulted) are all DONE at those SHAs. - -**The ONLY genuinely open items (all user-driven / deferred — pick up here):** - -1. **User-driven live `/run` gates** — the agent does NOT sign in; all code is merged + unit-verified; these are the operator's end-to-end confirmations: - - **Phase C HistoryRead** (T7) — needs the Wonderware sidecar + AVEVA on the WW Historian VM `10.100.0.48` (NOT on the local docker-dev rig). Recipe: Phase C banner + `docs/Historian.md`. - - **Phase B native alarms** (T9) — author a Galaxy alarm tag whose `FullName` == the gateway `AlarmFullReference`, deploy on `MAIN-galaxy-eq`, trip → Part 9 condition + `/alerts` row. Recipe: Phase B banner. - - **Driver-reconfigure-while-faulted** — fault `MAIN-opcua-eq` with a bad config, deploy a corrected one (`POST http://localhost:9200/api/deployments`, `X-Api-Key: docker-dev-deploy-key`), confirm it adopts WITHOUT a node restart. Recipe: that banner. -2. **Rig cleanups** (operational, user-deferred) — see "Operational deferral" at the bottom. -3. **Minor non-blocking residual follow-ups** — MOSTLY CLOSED by the residual-followups-cleanup merge (`fa2388ea`; see the top banner for full reconciliation). DONE this pass: Phase B residuals (a)/(b)/(c) + the stub-driver de-dup. VERIFIED already-implemented (were stale in this list): all of write-pipeline (a)–(e), Galaxy `_itemHandles`/`_supervisedHandles` clear-on-reconnect, and the `SubscriptionEstablished` self-dead-letter handler. **STILL OPEN (deferred, none gate anything):** write-outcome residuals (Bad-quality blip / AuditWriteUpdateEvent / synchronous fail-fast — design-deferred OPC UA surface, NOT the Galaxy write-commit proof which is now live-tested `d19deb9b`); Phase C documented follow-ups (no continuation-point paging, no modified-value history, block-bridge timeout bounded only by the client's 30s `CallTimeout`); Phase B residual (d) (inbound device-ack etc., DEFERRED by design); the data-plane `GroupToRole` production-default note. - -**No queued feature remains** — Milestone 1b + Galaxy A/B/C were the headline deliverables and are all done. Future directions (NOT requested): the Phase C HistoryRead follow-ups above, or new driver/UNS work. - ---- - -The **six historian code follow-ups** (HistorizeToAveva opt-out, drain/capacity/retention config -knobs, SharedSecret/DatabasePath/non-positive-knob startup validation, operator-recording for -shelve/enable/disable, and the `SqliteStoreAndForwardSink` thread-safety nits) were **all resolved** -on branch `feat/alarm-historian-followups` (plan: `docs/plans/2026-06-11-alarm-historian-followups.md`). -They are no longer listed here. - -## Equipment-tag live values — MILESTONE 1b COMPLETE (2026-06-13) - -The Galaxy standard-driver effort shipped Phase A (`c3c56172`) + the **`FullName→NodeId` live-value ROUTER** (`c4435e4f`, both pushed). The router is done + verified (322 tests + integration review READY-TO-MERGE). **All three driver-publish gaps are now CLOSED** — an equipment tag bound to OpcUaClient, any protocol driver, OR Galaxy publishes a live value delivered by the router (full detail in `current.md` "Milestone 1b" + `docs/plans/2026-06-13-equipment-tag-live-values-design.md`): - -1. ~~**OpcUaClient has NO factory (real bug — always stubbed).**~~ **DONE — SHIPPED+PUSHED master `22d553af` 2026-06-13.** Added `OpcUaClientDriverFactoryExtensions` (mirror Modbus) + registered it in `DriverFactoryBootstrap`. **First live equipment-tag value PROVEN end-to-end:** OpcUaClient driver `MAIN-opcua-eq` spawns `stub=False`, connects to opc-plc, subscribes to `ns=3;s=FastUInt1`; the `FullName→NodeId` router (`c4435e4f`) delivers it to the materialised variable `ns=2;s=EQ-55297329838d/FastUInt1`, which reads a live **changing** value (10135→10141, Good) via Client.CLI. Design/plan `docs/plans/2026-06-13-opcuaclient-factory-*.md`. Two incidental findings while live-verifying (see below). -2. ~~**Protocol drivers (Modbus/S7/AbCip/…) — equipment-tag↔driver tag-table linkage unbuilt.**~~ **DONE — SHIPPED+PUSHED master `8d8c05f5` 2026-06-13 (+ full inbound operator WRITE pipeline).** Approach B (driver-side direct-ref): a shared `EquipmentTagRefResolver` (Core.Abstractions) resolves an equipment-tag ref (the raw `TagConfig` JSON blob the router already keys on) into a transient driver tag-def on a `_tagsByName` miss — wired into READ + WRITE for **all six** drivers (Modbus/S7/AbCip/AbLegacy/TwinCAT/FOCAS), each with a hardened never-throw `EquipmentTagParser`. **Part B (write-through):** writable nodes (`Tag.AccessLevel==ReadWrite`→`CurrentReadWrite`, byte-parity in Phase7Composer+DeploymentArtifact), an `OnWriteValue` gate on the `WriteOperate` data-plane role (mirrors the alarm-ack bridge; fire-and-forget dispatch since the SDK holds the node-manager Lock during `OnWriteValue`), a `NodeWriteRouter` on the node manager, and `DriverHostActor.RouteNodeWrite` (NodeId→driver reverse map, primary-gated). **LIVE-PROVEN end-to-end:** Modbus equipment tag (HR[100]) reads a live changing value; an authorized write (`opc-writeop`/WriteOperate) to HR[200] changes the register + persists; an anonymous write → BadUserAccessDenied. Design/plan `docs/plans/2026-06-13-protocol-equipment-tag-linkage-*.md`. Findings + rig artifacts below. -3. ~~**Galaxy — needs a reachable mxaccessgw.**~~ **DONE — LIVE-PROVEN 2026-06-13 (no code change; config-only).** The code-investigation confirmed Galaxy was already fully wired: `GalaxyDriverFactoryExtensions` IS registered in `DriverFactoryBootstrap.cs:103` (not the missing-factory bug OpcUaClient had), and the Galaxy driver keys subscriptions on the FullReference (`tag_name.AttributeName`) DIRECTLY (no `_tagsByName` miss). gap (c) was purely a misconfigured dev driver-instance + placeholder tag ref + unset key — ALL data in existing columns, NO EF/schema change. Fixes applied to the dev rig (`otopcua-dev-sql-1`/`OtOpcUa`): `MAIN-galaxy-eq` `DriverConfig` `gateway.endpoint` `https://10.100.0.35:5001`→`http://10.100.0.48:5120`, `useTls` `true`→`false`, `apiKeySecretRef` `env:MX_API_KEY`(unset)→`env:GALAXY_MXGW_API_KEY` (the var the compose already wires on every node); `GalaxyTestTag` `TagConfig.FullName` `TestMachine_002.SomeAttr`(placeholder)→`TestMachine_002.TestDuration` (a real galaxy Float attr). The gateway API key was injected via **ephemeral shell env** at `docker compose up -d --no-deps --force-recreate central-1 central-2` time (NEVER written to a tracked file; the compose's `${GALAXY_MXGW_API_KEY:-stale-default}` substitution picks it up — the running containers carry the real key only until the next recreate-without-the-env-var). **Live (central-1 logs):** `spawned GalaxyMxGateway driver MAIN-galaxy-eq (stub=False)` → `GalaxyMxSession connected — clientName=OtOpcUa` (auth OK) → `initialized — endpoint=http://10.100.0.48:5120` → `subscribed to 1 refs (galaxy-sub-1)` (TestMachine_002.TestDuration accepted, no BadNodeIdUnknown). **Value:** `Client.CLI read ns=2;s=EQ-55297329838d/GalaxyTestTag` → Value `0`, Status `0x00000000` (Good), Source Time `2026-05-07T07:14:26Z` (a real galaxy timestamp — a genuine attribute snapshot, NOT BadWaitingForInitialData; static because that attr isn't actively moving). Restore-the-rig SQL saved at `/tmp/galaxy-gapc-snapshot.sql`. **Milestone 1b is now COMPLETE — all three gaps closed.** Findings/follow-ups below. - -Then: Phase **B** = native `IAlarmSource` alarms on the equipment-tag path **(DONE — `f9be3843`)**; Phase **C** = server-side `HistoryRead` backend over the Wonderware reader **(DONE — `c24abc8a`, 2026-06-14; design sections in `docs/plans/2026-06-12-galaxy-standard-driver-design.md` + the dedicated `docs/plans/2026-06-14-galaxy-phase-c-historian-*.md`)**. - -### Findings + follow-ups from the Galaxy gap-(c) live-verify (2026-06-13) - -- **Benign dead-letter (minor, pre-existing in the Galaxy driver — NOT introduced here).** On subscribe the driver logs: `Message [SubscriptionEstablished] from drv-MAIN-galaxy-eq to drv-MAIN-galaxy-eq was unhandled. [N] dead letters`. The Galaxy `DriverInstanceActor`/driver sends itself a `SubscriptionEstablished` message that has no `Receive<>` handler. Harmless (the subscription IS established + delivering values), but noisy — add a handler (or stop self-Telling it). Cosmetic. -- **CHANGING-value read PROVEN (2026-06-13).** Repointed `GalaxyTestTag.FullName`→`TestMachine_002.TestChangingInt` (a script-driven Integer, `sec=Operate`): three Client.CLI reads returned **810 → 787 → 764** with **real galaxy source timestamps advancing ~7s each** (`02:28:41`/`:48`/`:55`) — a genuine live moving Galaxy value through the router (not optimistic/phantom). The dev rig is now left with `GalaxyTestTag` pointing here (DataType `Int32`, AccessLevel `Read`). Discovery was done with a throwaway probe (now deleted) using `GalaxyDriverBrowser.OpenAsync`→`session.AttributesAsync("TestMachine_002")`, which lists every attribute's `SecurityClass` (`ViewOnly`=read-only; `FreeAccess`/`Operate`/`Tune`/`Configure`=writable). Useful attrs on `TestMachine_002`: `TestFloat`(Float,Operate), `TestDouble`(Double,Operate), `TestChangingInt`(Integer,Operate,**moves**), `TestDuration`(**ElapsedTime**,Operate), `AlarmInhibit`(Boolean,FreeAccess). -- **GALAXY WRITE-THROUGH — FIXED + MERGED to master `f05b5d79` (`AdviseSupervisory` before raw `Write`).** Symptom was every Galaxy operator write returning `MxaccessFailure "ArgumentException: HRESULT 0x80070057"` (`E_INVALIDARG`). **TWO-LAYER root cause (debugged by SSH-reading the gateway source on 10.100.0.48 `C:\Users\dohertj2\Desktop\mxaccessgw` — there is NO live gateway file log: console-only/uncaptured, NSSM `stdout.log` stale, dashboard :5130 is Blazor/no-REST):** (1) the writer `AddItem`'d an UN-advised handle → MXAccess `Write` threw E_INVALIDARG (worker chain `ExecuteWrite`→`MxAccessSession.Write`→`MxAccessComServer.Write`→`AsProxyServer().Write(...)`). (2) DEEPER — a plain `Write` runs with **no user login** (`WriteUserId`=0), and MXAccess only **COMMITS** such a write when the item is advised in **SUPERVISORY** mode; a *regular* `Advise` removed the E_INVALIDARG but never committed (proven by a persistence check: read-back showed the value, but a `--force-recreate`+fresh-resubscribe reverted to the original `0 @ 2026-05-07`; the worker's `ExecuteWrite` is fire-and-forget, returns OK without awaiting `OnWriteComplete`). Confirmed against the sister **ScadaBridge** driver (`~/Desktop/ScadaBridge/.../RealMxGatewayClient.cs`): it commits the OTHER way — a configured **non-zero `WriteUserId`** + regular `Advise` + `WriteBulk`. We have no galaxy login → supervisory context. **FIX:** `GatewayGalaxyDataWriter` calls `AdviseSupervisory` (raw `MxCommand{Kind=AdviseSupervisory, AdviseSupervisory=new AdviseSupervisoryCommand{ServerHandle,ItemHandle}}` via `session.InvokeAsync`, mirroring `InvokeWriteSecuredAsync`; idempotent per handle via `_supervisedHandles`) before each raw `WriteRawAsync`; `SecuredWrite`/`VerifiedWrite` tags keep their own user-identity path (`NeedsSecuredWrite` unchanged — WriteSecured is ONLY for those special-security tags). The dead-end "reuse the subscription's advised handle" resolver attempt was reverted. **LIVE-PROVEN:** authorized write (`opc-writeop`/WriteOperate) of `TestMachine_002.TestFloat`=1234.5 then 8888.25 COMMITS + **PERSISTS across recreate/re-subscribe** (galaxy-sourced timestamp); anonymous → `BadUserAccessDenied`. 254 Galaxy tests green; central `--build` clean. **OPEN follow-ups from this:** (a) the worker's fire-and-forget `ExecuteWrite` can't surface an async write failure — with supervisory advise the write commits, but only a read-back confirms a *specific* write (gateway-side; out of our scope). (b) `_itemHandles`/`_supervisedHandles` caches aren't cleared on reconnect (pre-existing for `_itemHandles`) — a write right after a reconnect could use a stale handle; minor. -- **OPTIMISTIC-WRITE PHANTOM (open follow-up — surface real write status to the client).** The inbound write dispatch is fire-and-forget: it returns optimistic `Good` before the driver result (required — `OnWriteValue` runs under the node-manager Lock), and the SDK applies the written value to the node locally. So a write whose DEVICE write FAILS still returns `Good`, and for a STATIC attribute that never re-pushes, the wrong value LINGERS (a phantom the device never accepted). The pipeline already computes the real status in `NodeWriteResult.Success/Reason` but only LOGS it — consider surfacing it to the client. (How it was caught live: a failed Galaxy write showed the written value on read-back with a SERVER-clock source timestamp + a `rejected` driver log; a committed write shows a GALAXY-clock timestamp + no rejection, and persists across a re-subscribe.) -- **Dev-rig Galaxy config is CORRECT + WORKING (left in place).** The `MAIN-galaxy-eq` driver-instance is deployed and connecting to the live gateway `http://10.100.0.48:5120`. `GalaxyTestTag` (on `EQ-55297329838d`/filler-02, `nw-uns` namespace) is currently `{"FullName":"TestMachine_002.TestFloat"}`, DataType `Float`, AccessLevel `ReadWrite` (the write demo; galaxy now holds the last written value `8888.25`). Other useful `TestMachine_002` attrs (from the discovery probe): `TestChangingInt`(Integer,Operate,**moves on its own** — the live-changing READ demo), `TestDouble`(Double,Operate), `TestDuration`(**ElapsedTime**,Operate — reads as Float but a Float write is a type-mismatch), `AlarmInhibit`(Boolean,FreeAccess). To restore the original placeholder tag: `/tmp/galaxy-gapc-snapshot.sql`. The base seed `docker-dev/seed/seed-clusters.sql` still seeds the *legacy* SystemPlatform-namespace Galaxy driver (`MAIN-galaxy-mxgw`, tags `TestMachine_001.TestAlarm001..003`) — pre-Phase-A model, untouched/separate. **The injected gateway key is EPHEMERAL** — key=`mxgw_otopcuakey2_so0…` is supplied via shell env `GALAXY_MXGW_API_KEY='…'` at `docker compose up --no-deps --force-recreate central-1 central-2`; a recreate WITHOUT it re-exported falls back to the compose's stale default and Galaxy auth fails. ORDER on a redeploy: POST deploy FIRST, THEN recreate (a faulted driver ignores `ApplyDelta`). - -### Findings + follow-ups from the protocol-linkage + write-through work (2026-06-13) - -- **DATA-PLANE ROLE CONFIG REQUIREMENT (important, deployment-facing).** The OPC UA session's roles come from two sources unioned: the **DB `LdapGroupRoleMapping`** (its `Role` column is the **`AdminRole` enum** — Administrator/Designer/Viewer only, for the AdminUI) AND the **appsettings `Security:Ldap:GroupToRole`** baseline (free-form `string→string`). The OPC UA **data-plane** gates (`WriteOperate`, `AlarmAck`, …) read literal role STRINGS that the AdminRole-typed DB mapping **cannot** produce — so a deployment MUST map its LDAP data-plane groups → data-plane role strings via `GroupToRole`, or write-through (and scripted-alarm OPC UA ack) is inert (every write → `BadUserAccessDenied`). The shared dev GLAuth already has dedicated groups+users (group `WriteOperate`, user `opc-writeop`, `multi-role` in all; `opc-readonly`); the dev rig just never seeded the `GroupToRole`. **Consider a docs note (and/or a documented default) so production deployments wire this.** (Same latent requirement applies to the pre-existing alarm-ack gate.) -- **Write-pipeline review follow-ups (non-blocking, from the final integration review):** (a) `DriverHostActor.Stale` (and `DriverInstanceActor.Connecting`/`Reconnecting`) have **no `RouteNodeWrite`/`WriteAttribute` handler** → an operator write while stale/reconnecting dead-letters and the 10s Ask times out with a generic log (client got optimistic Good). Add fast-fail handlers returning a clear status. (b) Drop `TaskContinuationOptions.ExecuteSynchronously` on the router `ContinueWith`; `List.Contains`→`HashSet` in the forward-map build (micro). (c) FOCAS re-parses the address on every equipment-tag write (`_parsedAddressesByTagName` miss; perf only, rare). (d) `DriverHostActorWriteRoutingTests` seeds a Galaxy-style `{"FullName":...}` artifact, not a raw protocol-driver TagConfig blob — add a raw-blob case for belt-and-suspenders (runtime path is identical + live-verified). (e) Task-9 parity test is a faithful simulation of `ConfigComposer` (`ToSnapshot` casts AccessLevel to int) not a through-the-real-serializer proof; add an `InlineData(2,false)` future-enum trap. - -### Dev-rig artifacts created for the protocol-linkage live-verify (left in place, NOT committed) - -- **`docker-dev/docker-compose.yml`** gained `Security__Ldap__GroupToRole__{ReadOnly,WriteOperate,WriteTune,WriteConfigure,AlarmAck}` identity entries on both central nodes (needed for data-plane roles — see above). **Uncommitted** (rig config; the file was already modified at session start). -- **DB seeds** on `otopcua-dev-sql-1`/`OtOpcUa`: driver `MAIN-modbus-eq` (DriverType=Modbus, `{"Host":"10.100.0.35","Port":5020,"UnitId":1,"Tags":[]}`, namespace `nw-uns`, cluster MAIN) + tags `tag-modbus-hr100` (HR[100] auto-increment, Read — read demo) and `tag-modbus-hr200` (HR[200] scratch, ReadWrite — write demo), both on equipment `EQ-55297329838d` (filler-02). The pymodbus `standard` sim (`10.100.0.35:5020`) serves HR[0..31]=addr-as-value, HR[100]=auto-increment, HR[200..209]=writable scratch. - -### Incidental findings from the OpcUaClient live-verify (2026-06-13) - -- **Driver-reconfigure-while-faulted gap — FIXED + MERGED `56f73e49` (2026-06-14).** (Was: a `DriverInstanceActor` stuck in `Reconnecting`/`Connecting` had **no `ApplyDelta` handler**, so a corrected config dead-lettered and the actor retried the OLD `_currentConfigJson` forever; workaround = restart the node.) Now `Connecting`/`Reconnecting` handle `ApplyDelta` via `AdoptConfigDuringInit`, re-initialising with the new config; a monotonic `_initGeneration` guard supersedes the in-flight old init so the corrected config always wins. See the top banner + `docs/plans/2026-06-14-driver-reconfigure-while-faulted-*.md`. Live `/run` gate deferred (user choice). -- **Dev-rig config edit applied directly in DB.** The `MAIN-opcua-eq` `DriverConfig.targetNamespaceKind` was `0` (Equipment, which requires a `UnsMappingTable` → `InitializeAsync` rejected it). Set to `1` (SystemPlatform — the direct-ref mode the equipment-tag model wants; empty `unsMappingTable:{}` passes validation) via a direct `JSON_MODIFY` UPDATE on `otopcua-dev-sql-1` (DB `OtOpcUa`, `SET QUOTED_IDENTIFIER ON` required for JSON fns; `sqlcmd -h` and `-y 0` are mutually exclusive — pick one). The AdminUI driver-edit combobox for "Target namespace kind" did **not** persist the change (suspected live-only Blazor binding bug — unverified; the DB edit sidestepped it). Deploy snapshots the **live** config DB directly (`AdminOperationsActor` → `DraftSnapshotFactory.FromConfigDbAsync` + `ConfigComposer.SnapshotAndFlattenAsync`), so a DB edit flows through on the next `POST /api/deployments` (new revisionHash). - -## Operational deferral (user choice) - -1. **docker-dev rig cleanup (round-1 T9) deferred.** The local docker-dev rig still has the - live-verify seed artifacts deployed: the `t12-overheat` scripted alarm, the - `SC-ba675b168a85` predicate script, the `layer0-logcheck` vtag/script, and filler-02's - modified `cycle-time-s` line. Left as-is to inspect the working double-emit fix. **To clean - up:** delete those artifacts in the AdminUI (or DB), revert filler-02's `cycle-time-s` to - `return ctx.GetTag("TestMachine_002.TestDuration").Value;`, then redeploy - (`POST http://localhost:9200/api/deployments`, header `X-Api-Key: docker-dev-deploy-key`). - -2. **Equipment-tag live-value verify artifacts (left in place — all now FUNCTIONAL).** The docker-dev - rig carries verify artifacts under the `nw-uns` Equipment namespace on `EQ-55297329838d` - (filler-02), all three now working: `MAIN-galaxy-eq` (GalaxyMxGateway → live gateway, `GalaxyTestTag` - = `TestMachine_002.TestFloat` RW, write-proven; needs the ephemeral `GALAXY_MXGW_API_KEY` re-exported - on recreate — see Galaxy dev-rig note above); `MAIN-opcua-eq` (OpcUaClient, factory shipped `22d553af`) - + `FastUInt1` tag (`{"FullName":"ns=3;s=FastUInt1"}`); `MAIN-modbus-eq` (Modbus) + `tag-modbus-hr100` - (read) / `tag-modbus-hr200` (RW write demo). docker-dev is **LOCAL on this Mac** (OrbStack); central-1 - @ `localhost:4840`/AdminUI+deploy @ `localhost:9200`, sql @ `localhost:14330` (sa/`OtOpcUa!Dev123`), - login disabled. Sims for the protocol/opcua verifies run on the docker host `10.100.0.35` - (`otopcua-pymodbus-standard` :5020, `otopcua-opc-plc` :50000) — leave up or `docker compose down` per - `/opt/otopcua-modbus` + `/opt/otopcua-opcuaclient`. Phase A backup - `OtOpcUa-prePhaseA-20260612-224908.bak` is in the SQL volume `/var/opt/mssql/backup/`.