diff --git a/.claude/skills/sister-project-history/otopcua.md b/.claude/skills/sister-project-history/otopcua.md index 498ae72..ed1494c 100644 --- a/.claude/skills/sister-project-history/otopcua.md +++ b/.claude/skills/sister-project-history/otopcua.md @@ -416,3 +416,9 @@ AddRoute of the dev Mac (`oee-tagdump-mac`, `10.181.206.245.1.1`) on the two tes - **#796** test-oracle only (1.5.378.182 PercentGood counted Uncertain as Good); HistoryRead unaffected. **#809 (owner decision):** live AggregateFilter PercentGood/DurationGood still count Uncertain as Good, unlike HistoryRead — ScadaBridge uses neither. - **#806** AdminUI OPC UA browser keeps the configured host/port behind NAT (shared `OpcUaEndpointUrlRewriter` with the runtime driver); live-gated on docker-dev. - **#798** ScriptWorker shadow test made causal (barrier, not stopwatch). CLAUDE.md corrected: `Scripting:Runner` Shadow is implemented. + +- **2026-11-11 issue batch #803/#804/#808/#810–#814 resolved (master `5bbe1e3c`).** Pushed and closed. Follow-ups #815–#833. + - **#808 — ScadaBridge-visible authorization change (ACL on only), completes #797.** The authority generation now carries `UnsFolderScopes` (opaque Area/Line/Equipment folder id → name path + folder-id chain); every data-plane decision places UNS folders, virtual tags (`s=EQ-…/Name`), scripted conditions + children and reference variables against it. **Area/Line/Equipment-scoped keys can now browse/read/subscribe/HistoryRead their subtree, receive scripted and native alarm events on i=2253 and folder subscriptions, and Ack/Confirm/AddComment with the EventId — a Cluster grant is no longer needed to ack.** Fail-closed: a moved equipment/alarm is denied under old and new line grants (and briefly to Cluster keys) until the rebuild lands; NUL ids rejected; removed opaque ids decided only by realm-wide grants; narrow grantees no longer get NodeDeleted model-change events for removed UNS folders/VTs/conditions. No NodeId/wire change; ACL-off unchanged. Independent review (4 gaps fixed) + docker-dev live gate PASS (Equipment-only key: own subtree only, sibling denied, i=2253 events, ack by EventId, move leg). Residual #826 (per-folder event items during the stale window). + - **#803 — new driver: Heidenhain LSV2 (read-only, native TCP 19000).** Payload-aware outbound allowlist (logins + reads only), no writable nodes, credentials only `secret:heidenhain/…` / `env:OTOPCUA_HEIDENHAIN_…` (closes a Test-Connect credential-exfiltration path; same class for Galaxy/OpcUaClient → #815). Simulator + Java golden vectors only; real-control commissioning #829. The docker-dev gate caught an eager `ISecretResolver` resolution in the probe that crash-looped every central node at boot — fixed in `5bbe1e3c` before push. + - **#804** MTConnect parity vs the ignitionoee Java driver: 9 gaps closed, notably cppagent 2.x HTTP 400/404 error documents were treated as network failures (buffer-overrun recovery never ran), silent Agent-restart adoption, unbounded XML recursion (process stack overflow). `IWritable` writes only the Agent's AllowPut surface (no controller path). + - **#812–#814** Client CLI/UI share the endpoint rewriter; shared rewriter no longer un-escapes percent-encoded paths. NAT opc-plc fixture on 10.100.0.35:50010 (`/opt/otopcua-opcuaclient-nat`), CI wired. **#810** Uncertain/partial conformance data (no backend defects; new SDK TimeAverage/Total divergence #821, relates to #809). **#811** shadow-test flake root-caused (worker soft deadline under load; test budget 2 s). diff --git a/CLAUDE.md b/CLAUDE.md index 336b88e..3cca162 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -36,7 +36,7 @@ own `CLAUDE.md` for the full picture. See [Refreshing this index](#refreshing-th | Project | Location | Stack | Repo | Summary | |---|---|---|---|---| -| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **two-subtree address space (v3.0, `master` since 2026-07-16)** — `https://zb.com/otopcua/raw` (device tree Folder→Driver→Device→TagGroup→Tag, NodeId `s=`) and `https://zb.com/otopcua/uns` (equipment tree Area→Line→Equipment→signal); every value has one source fanned to both NodeIds. **This is the wire contract ScadaBridge binds against** (ScadaBridge consumes v3 end-to-end; its #14 is closed). Drivers: Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient, `Sql` (SQL Server / PostgreSQL / Oracle, read-only poll), `Mqtt` (Plain + Sparkplug B, writable where a tag declares a destination), `MTConnect` (read-only), Calculation, and AVEVA Galaxy as a standard Equipment-kind driver via in-process `GalaxyDriver` → gRPC → **mxaccessgw**. Historian backend is pluggable (`ServerHistorian:Backend`): **HistorianGateway** (default), **SqlServer** or **QuestDb** — the latter two add typed values with real status and OPC UA HistoryUpdate (QuestDB: soft delete + TTL retention). Address-space availability guarantee: an unreadable deployment artifact is *no answer*, never an empty configuration — nodes hold last-known-good, and a driver-only node boots and serves from its LocalDb cache with central SQL down (ServiceLevel stays 240/250). Redundancy: one independent 2-node Akka mesh per application `Cluster`; Primary = oldest Up `driver` member (ServiceLevel 250) + one Secondary (240) **per `Cluster`**; cross-mesh commands ride gRPC with per-cluster PSK auth (Akka `ClusterClient` fully deleted 2026-08-06). Data plane is machine-to-machine: OPC UA UserName `apikey` + `otua__` minted on AdminUI `/api-keys`; `OpcUa:AllowAnonymous` defaults `true` (anonymous is a normal grantable principal); LDAP is AdminUI-login only. Node ACLs are enforced **only** behind `Security:Acl:Enabled` (default `false`); with it on, scoped `NodeAcl` grants decide **every** data-plane operation incl. browse/read/subscribe, fail closed, and reach open sessions. Every driver's authoring path now consults the driver's own runtime parse, so what authoring accepts is what the driver serves (tracker zeroed 2026-08-10, origin `d06e92c6`). **`master` @ `efb9197a` (2026-11-10, not deployed):** opt-in out-of-process script runner (`Scripting:Runner`, #780 — sends `BadResourceUnavailable`, ScadaBridge#112 handles it), VT→VT dependencies now flow + `VirtualTagDependencyCycle` deploy error (#791), disabled VT keeps its node as `BadOutOfService` (#787), Sql write block Insert/Upsert/Procedure (#781), OPC SDK 1.5.378.182 (stricter CA KeyUsage — check DEV/QA certs, #782). **Full change history, live-gate records and the per-issue consumer-impact record for ScadaBridge: `.claude/skills/sister-project-history/otopcua.md` (skill `sister-project-history`).** | +| **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **two-subtree address space (v3.0, `master` since 2026-07-16)** — `https://zb.com/otopcua/raw` (device tree Folder→Driver→Device→TagGroup→Tag, NodeId `s=`) and `https://zb.com/otopcua/uns` (equipment tree Area→Line→Equipment→signal); every value has one source fanned to both NodeIds. **This is the wire contract ScadaBridge binds against** (ScadaBridge consumes v3 end-to-end; its #14 is closed). Drivers: Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient, `Sql` (SQL Server / PostgreSQL / Oracle, read-only poll), `Mqtt` (Plain + Sparkplug B, writable where a tag declares a destination), `MTConnect` (read-only), `Heidenhain` (LSV2, read-only, #803), Calculation, and AVEVA Galaxy as a standard Equipment-kind driver via in-process `GalaxyDriver` → gRPC → **mxaccessgw**. Historian backend is pluggable (`ServerHistorian:Backend`): **HistorianGateway** (default), **SqlServer** or **QuestDb** — the latter two add typed values with real status and OPC UA HistoryUpdate (QuestDB: soft delete + TTL retention). Address-space availability guarantee: an unreadable deployment artifact is *no answer*, never an empty configuration — nodes hold last-known-good, and a driver-only node boots and serves from its LocalDb cache with central SQL down (ServiceLevel stays 240/250). Redundancy: one independent 2-node Akka mesh per application `Cluster`; Primary = oldest Up `driver` member (ServiceLevel 250) + one Secondary (240) **per `Cluster`**; cross-mesh commands ride gRPC with per-cluster PSK auth (Akka `ClusterClient` fully deleted 2026-08-06). Data plane is machine-to-machine: OPC UA UserName `apikey` + `otua__` minted on AdminUI `/api-keys`; `OpcUa:AllowAnonymous` defaults `true` (anonymous is a normal grantable principal); LDAP is AdminUI-login only. Node ACLs are enforced **only** behind `Security:Acl:Enabled` (default `false`); with it on, scoped `NodeAcl` grants decide **every** data-plane operation incl. browse/read/subscribe, fail closed, and reach open sessions. Every driver's authoring path now consults the driver's own runtime parse, so what authoring accepts is what the driver serves (tracker zeroed 2026-08-10, origin `d06e92c6`). **`master` @ `5bbe1e3c` (2026-11-11, not deployed; adds narrow-grant UNS access + scripted-alarm ack for Area/Line/Equipment-scoped keys with ACLs on, #808):** opt-in out-of-process script runner (`Scripting:Runner`, #780 — sends `BadResourceUnavailable`, ScadaBridge#112 handles it), VT→VT dependencies now flow + `VirtualTagDependencyCycle` deploy error (#791), disabled VT keeps its node as `BadOutOfService` (#787), Sql write block Insert/Upsert/Procedure (#781), OPC SDK 1.5.378.182 (stricter CA KeyUsage — check DEV/QA certs, #782). **Full change history, live-gate records and the per-issue consumer-impact record for ScadaBridge: `.claude/skills/sister-project-history/otopcua.md` (skill `sister-project-history`).** | | **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** Alarm ack (main `488649c`, 2026-10-20) authenticates as the operator and writes `.AckMsg` so the Historian records the real operator; `AcknowledgeAlarmRequest` gained `operator_password`/`operator_domain` (proto change, published as Contracts/Client **`0.3.0`** on the feed 2026-11-06, mxaccessgw#214; OtOpcUa's `GatewayGalaxyAlarmAcknowledger` must send the password before it deploys against this — lmxopcua#749). Main `e5e7f18` (2026-11-06) also locks the worker pipe to the gateway's account and redacts secured bulk writes in logs. | | **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | `gitea.dohertylan.com/dohertj2/ScadaBridge` | **Current `main`: `c810bd0f` (2026-11-10 issue waves, NOT deployed: AuditLog clustered-key flip + off-box chain-head export + maintenance login + archive retention — four migrations ship together with both centrals stopped (#107); opt-in `leader-auto-down` downing provider (#106, drill #116); derived templates survive bundles (#118); bundle-signing policy + runbook (#110); Flow Debugger Node log + Akka→Serilog (#56/#120); OPC UA Good-subcode + BadResourceUnavailable handling (#117/#112)). Previous tip `3808b49f` (2026-10-22: 2026-10-21 eight-slice system review remediated — ~107 findings, bearer-token CLI auth, `MachineDataDb` removed, config-audit summaries + retention, migration `ReviewRemediationIndexes` (QA/prod not applied); Topology area rename fix, first-layer preload and area-subtree **Deploy all** / CLI `deploy area` (Gitea #38–#40); record `docs/plans/2026-10-21-system-review-remediation.md`). Previous tip `3a217b94` (2026-09-29: stale per-node secret-mirror status rows are now pruned; 2026-09-28: site-secrets follow-ups — a central secret-store outage fails only the sites that reference a secret, removal-only deploys are audited, each site node's mirror failures (standby included) ride the health report via a 12th replicated table `secret_mirror_status`, artifact-deploy CLI commands exit 1 on a failed site; rig gate 9/9). Previous tip `23a139a4` (2026-09-28: site secrets ride the artifact deploy and the gRPC secrets hub is retired — see below). Previous tip `5cc09b08` (2026-09-28: site scripts gain a read-only, allow-listed `Files` global — `ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots`, off by default — used by the in-memory MES/Delmia/alarm-API test machines SBTEST01/02 (`docker/seed-mes-test-machine.sh`) to parse downloaded recipe files; DEV vd03 and QA vq03/vq04 on `b3021e7c` since 2026-09-28, migrations 57; register row 126 closed) — adds site artifact staleness (`Site.ArtifactRevisionHash` stamped on every successful artifact deploy, migration `AddSiteArtifactRevision`; Sites page *Artifacts current / out of date / never deployed* badge), a rail-footer **Pending updates** indicator over a process-wide `PendingUpdatesCache` (stale instances + stale-artifact sites, Deployer/Admin), and a scrollable rail nav; design `docs/plans/2026-09-14-pending-updates-design.md`. Previous tip `9d529b82` (2026-09-11) — adds the Gitea #36 reset-aware handshake (`WriteBatchAndWaitAsync` reset-echo overload), the computed-alarm `IsMesReportable` flag (template checkbox + nullable instance override, `Alarms.MesReportableAsync()`, migration `AddAlarmMesReportable`; native alarms stay on the 900–999 band) and a nav fix nesting Flow Debugger in the Deployment section. DEPLOYED: `b3021e7c` on DEV vd03 and the QA pair vq03/vq04 (2026-09-28, migrations 55→57; records `docs/deployment/deployment-records/2026-09-28-wonder-app-{vd03-fe89363b,vq03-vq04-b3021e7c}.md`).** **Merged `23a139a4` (2026-09-28): site secrets now ride the artifact deploy and the pull-only gRPC secrets hub is retired in ScadaBridge** (design `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md`; central shares one secret store via `Secrets:Store:Provider=SqlServer`; rig live gate PASS 8/8, vd03/QA NOT deployed — rollout tracked as ScadaBridge register row 127 — see `ZB.MOM.WW.Secrets/CLAUDE.md` for the unpublished `0.6.3` hub-logging work this superseded for ScadaBridge). Management/API/CLI parity complete (`docs/plans/2026-09-10-management-api-cli-parity-review.md`): all 149 eligible contracts exposed in 20 CLI groups, 19 missing operations added, deadline/registry/outcome gaps fixed; CLI documentation checked against all 159 leaf commands; 9554 tests verified and primary 8-node rig rebuilt with read-only smoke checks. Previously completed 2026-09-09 whole-system arch review remediation (`docs/plans/2026-09-09-arch-review.md`: 92 findings; all 13 High + 33 Medium fixed; all 46 Low findings also committed and pushed (§11); 9478 solution tests and deployed browser smoke pass, primary 8-node rig rebuilt). Adds the `ZB.MOM.WW.ScadaBridge.CentralUI.SandboxWorker` child-process project and a coordinated central-pair migration that encrypts DataConnection protocol config at rest (old binaries cannot read protected rows). Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. **Inbound API keys live on the shared SQL Server store on a central pair** (design 2026-09-02, ScadaBridge register row 108): ScadaBridge is Auth's first consumer of the new `ZB.MOM.WW.Auth.ApiKeys.SqlServer` package at `0.3.0`, pointing both central nodes at one `zbauth` schema in the configuration database instead of each node's own per-node SQLite file. **Flow Debugger (including the instance picker and v1.4 step logs, 2026-09-09):** Central UI + CLI trace of one inbound-API method or value-change flow across Central / Site / Tags / External swimlanes, Live or Replay, built on `ScriptTrace` audit rows and the outbound-call audit rows; arch-reviewed and remediated the same day (F1–F13 — redacted + byte-capped inline trace copy, route-time site scope on Live runs, Replay open to `OperationalAudit` roles under `RequireFlowDebugger`; `docs/plans/2026-09-06-flow-debugger-arch-review.md` + `-remediation.md`); script entry (site → instance → script, 2026-09-08). | | **HistorianGateway** | `~/Desktop/HistorianGateway` | .NET 10 x64, gRPC, Blazor | `gitea.dohertylan.com/dohertj2/historiangw` | Single-process gRPC sidecar exposing (1) a full read/write API to the AVEVA Historian (5 gRPC services; 15 retrieval modes; historical/backfill writes; tag-config lifecycle; SQL live-value path; store-forward + redundancy resilience; all default-disabled) and (2) read-only Galaxy object-hierarchy browse via the shared `ZB.MOM.WW.GalaxyRepository` lib. No COM, no x86 worker. **`main` @ `860bf99` (2026-11-10, not deployed): redundancy + store-forward compose (F1b, #72 — keep both off on boxes until live gate #115), deterministic server-op faults (#112), versioned outbox frames + size limits + expiry cap (#113), stream keepalive guard on by default (#59), ReadEvents EventCount pass-down (#81). Earlier: `99b4a7d` (2026-11-06, historiangw#55): gRPC-only Historian client at `src/Historian.Client`** — `histsdk/` and the whole SQL path (RuntimeDb, SQL live-value writer, SQL `ReadEvents`) are removed; `Transport` must be `RemoteGrpc`, ReadBlocks removed from the contract (#62, 2026-11-10); RenameTags restored over StartJob (#56); UInt1 tags created as UInt2, Int8/UInt8/FileTime/Guid/Structure/Event rejected (lossless-cast-or-reject); empty event ids get a gateway-minted GUID (#28); AddHistoricalValues backfills string tags via `string_value` (#19). merged to main 2026-11-10 (PRs #116, #117 → `733eb25`); released as **Contracts/Client 0.4.0** (2026-11-10, PR #118 → `dde0c28`, tag `v0.4.0`, #79 closed — breaking: ReadBlocks removed; OtOpcUa consumes 0.4.0 on master `16111a5e` (historiangw#119 closed; no deployment)), then **1.0.0** (2026-11-10, PR #132 → `ea9f21c`, tag `v1.0.0` — breaking: secure-by-default client TLS, `AllowUntrustedServerCertificate` replaces `RequireCertificateValidation` (#58); also #25 WriteLiveValues hardening + #126; wire unchanged `historian_gateway.v1`; OtOpcUa master `230cef4e` consumes1.0.0 with the adapter rename verified; historiangw#131 closed). Deployed boxes still run `d8d890c` with the old config (redeploy needs a config change). Dev: dashboard `:5220` (HTTP/1.1) + gRPC h2c `:5221`; production: one `Kestrel:Endpoints:Https` with `Protocols: Http1AndHttp2` (ALPN). Default-on leased session pools amortize the historian handshake (v6 pool + a separate v8 event-session pool). Client lib `ZB.MOM.WW.HistorianGateway.Client` + Contracts are on the Gitea feed at `1.0.0` (OtOpcUa pinned to `1.0.0`, master `230cef4e`; historiangw#131 closed, no deployment); **OtOpcUa is the first consumer** (its sole historian backend since PR #423); ScadaBridge has no coupling. Native event reads now return rows through the gRPC-only client (2026-10-16 live run; the old 0-row C2 gate and its SQL `ReadEvents` workaround are gone — historiangw#24 closed 2026-11-09; the deployed `d8d890c` boxes still use the SQL reader until #78). Known 2023 R2 server-side gates: SysTimeSec raw reads still time out (#29, blocks lmxopcua#694/#700), `Int1`/`UInt1` analog-tag creation is degenerate (OtOpcUa maps Boolean → `Int2`), string historical writes non-convergent (follow-ups #18–#25). Only remaining archreview item: the secure-by-default TLS flip, deferred to 1.0.0. A Java client, `com.zb.mom.ww.historiangw:zb-mom-ww-historiangw-client:0.2.0` (Gitea Maven feed, `clients/java/`; 0.2.0 = the 0.4.0 contract, 2026-11-10, historiangw#120 — the consumer is still on 0.1.0), is consumed by the ignitionoee repo's `modules/historian` Ignition module. **Full change history (ArchReview cycles 1–2, pending.md items, PR list): `.claude/skills/sister-project-history/historiangateway.md`.** `HistorianStatus.GetOperationalDiagnostics` (22 RPCs) and the historical Int1 serializer are **merged to `main`** (Int1 persisted readback still unverified, #22/#29). Offline suite on `main`: 1311 passed / 54 skipped / 0 failed. New gap work targets native Historian gRPC only. |