33915c9e4d
docs/Raw.md (the /raw project tree: lazy tree, endpoint→DeviceConfig split, manual/CSV/browse authoring, Calculation driver, routed-flow retirement); CLAUDE.md AdminUI section notes /raw + DriverTypeNames rewire + Calculation driver; PR body with the 7-item live-gate evidence. Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
394 lines
32 KiB
Markdown
394 lines
32 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project Goal
|
||
|
||
Build an OPC UA server (.NET 10) that exposes industrial data sources —
|
||
including AVEVA System Platform (Wonderware) Galaxy — under a unified
|
||
Equipment-based address space. Galaxy access flows through the in-process
|
||
`GalaxyDriver` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/`) talking gRPC
|
||
to a separately installed **mxaccessgw** gateway process. The gateway owns the
|
||
MXAccess COM bitness constraint (its worker is x86 net48); everything in this
|
||
repo is .NET 10. PR 7.2 retired the legacy in-process `Galaxy.Host` /
|
||
`Galaxy.Proxy` / `Galaxy.Shared` projects + the `OtOpcUaGalaxyHost` Windows
|
||
service.
|
||
|
||
See `docs/v2/Galaxy.Performance.md` for the runtime perf surface
|
||
(tracing, metrics, soak harness).
|
||
|
||
## Related Projects (scadaproj umbrella)
|
||
|
||
This repo is one of a family of related SCADA / OT / Wonderware / OPC UA
|
||
**sister projects** that live as sibling directories under `~/Desktop/`. They are
|
||
**separate repos and separate processes**, coupled at runtime over wire protocols
|
||
(gRPC + OPC UA) — not by project/compile references — and share the
|
||
`ZB.MOM.WW.*` product namespace. The common subject is AVEVA System Platform
|
||
(Wonderware) "Galaxy" data, and `mxaccessgw` is the linchpin the others connect
|
||
through.
|
||
|
||
- **`~/Desktop/scadaproj`** ([`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)) —
|
||
the umbrella/index workspace that aggregates the whole family (a high-level scan
|
||
of each sister project: purpose, location, stack, primary commands). It also hosts
|
||
the shared `ZB.MOM.WW.*` libraries — `ZB.MOM.WW.Auth`, `ZB.MOM.WW.Theme`,
|
||
`ZB.MOM.WW.Health`, `ZB.MOM.WW.Telemetry`, `ZB.MOM.WW.Configuration`, and
|
||
`ZB.MOM.WW.GalaxyRepository`. **This repo is indexed there** — see the OtOpcUa
|
||
entry in `../scadaproj/CLAUDE.md`.
|
||
- **`~/Desktop/MxAccessGateway`** (`mxaccessgw`,
|
||
[`../MxAccessGateway/CLAUDE.md`](../MxAccessGateway/CLAUDE.md)) — the gRPC gateway
|
||
that owns the MXAccess 32-bit COM bitness + STA pump (gateway x64 .NET 10 + per-session
|
||
x86 net48 worker). **OtOpcUa's `GalaxyDriver` depends on this** for live Galaxy
|
||
read/write/subscribe and Galaxy Repository browse.
|
||
- **`~/Desktop/HistorianGateway`**
|
||
([`../HistorianGateway/CLAUDE.md`](../HistorianGateway/CLAUDE.md)) — single-process
|
||
gRPC sidecar exposing the AVEVA Historian read/write API + read-only Galaxy browse
|
||
(via the shared `ZB.MOM.WW.GalaxyRepository` lib). Patterned on `mxaccessgw` but with
|
||
no COM / no x86 worker. **OtOpcUa's historian backend** (`Driver.Historian.Gateway`)
|
||
consumes it as the sole historian path.
|
||
- **`~/Desktop/ScadaBridge`**
|
||
([`../ScadaBridge/CLAUDE.md`](../ScadaBridge/CLAUDE.md)) — the distributed SCADA
|
||
platform (Akka.NET hub-and-spoke) whose Data Connection Layer consumes this server's
|
||
OPC UA address space.
|
||
|
||
**Propagate cross-repo changes to the umbrella index.** When a fact the index records
|
||
about OtOpcUa changes here — remote/push status, the driver set, the Galaxy data flow,
|
||
shared-lib consumption, or per-project commands — update the **OtOpcUa entry in
|
||
`../scadaproj/CLAUDE.md`** in the same change so the index never drifts from this repo.
|
||
|
||
## Architecture Overview
|
||
|
||
### Data Flow
|
||
|
||
1. **Galaxy Repository DB (ZB)** — SQL Server database holding the
|
||
deployed object hierarchy and attribute definitions. The
|
||
mxaccessgw's `GalaxyRepositoryClient` queries it via gRPC; the
|
||
driver consumes the materialised hierarchy through
|
||
`IGalaxyHierarchySource`.
|
||
2. **MXAccess (via mxaccessgw)** — Live read/write/subscribe over a
|
||
gRPC session. The gateway owns the COM apartment + STA pump
|
||
server-side; the driver speaks `MxCommand` / `MxEvent` protos
|
||
exclusively.
|
||
3. **OPC UA Server** — Exposes authored equipment tags as variable nodes.
|
||
Galaxy tags are bound by `TagConfig.FullName` (`tag_name.AttributeName`);
|
||
reads/writes/subscriptions are translated to that reference for MXAccess.
|
||
|
||
### Key Concept: Tag Name and FullName
|
||
|
||
Galaxy objects have a **tag_name** — a globally unique system name used for
|
||
MXAccess read/write — and attributes are referenced as `tag_name.AttributeName`
|
||
(e.g. `DelmiaReceiver_001.DownloadPath`). This dot-separated reference is what
|
||
`TagConfig.FullName` stores for a Galaxy equipment tag. The Galaxy address
|
||
picker browses the live hierarchy using contained names for navigation, then
|
||
resolves the selection to the `tag_name.AttributeName` form.
|
||
|
||
**Galaxy is a standard Equipment-kind driver.** Galaxy points are ordinary
|
||
equipment `Tag`s bound to `GalaxyMxGateway` via `TagConfig.FullName`
|
||
(`tag_name.AttributeName`), authored through the standard Tag modal + Galaxy
|
||
address picker on the equipment page's Tags tab — the same flow as Modbus or
|
||
S7. There is no alias machinery, no `SystemPlatform` namespace kind, and no
|
||
relay→alias converter. See
|
||
`docs/plans/2026-06-12-galaxy-standard-driver-design.md` for the full design.
|
||
|
||
### Data Type Mapping
|
||
|
||
Galaxy `mx_data_type` values map to OPC UA types (Boolean, Int32, Float, Double, String, DateTime, etc.). Array attributes use ValueRank=1 with ArrayDimensions from the Galaxy attribute definition. The driver-side mapping lives in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Browse/DataTypeMap.cs`.
|
||
|
||
### Change Detection
|
||
|
||
`DeployWatcher` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Browse/DeployWatcher.cs`) polls the gateway's deploy-event signal and raises `IRediscoverable.OnRediscoveryNeeded` when the Galaxy redeploys. The server's `DriverHost` consumes the signal and rebuilds the address space.
|
||
|
||
## mxaccessgw
|
||
|
||
The gateway lives in a sibling repo at `~/Desktop/MxAccessGateway/` (`mxaccessgw` — see [Related Projects](#related-projects-scadaproj-umbrella)). See `docs/v2/Galaxy.ParityRig.md` for the gw setup recipe (build, API key provisioning via `apikey create-key`, env-var overrides for HTTP/2 cleartext + worker path). The gw's MXAccess Toolkit reference (its `gateway.md`) is the canonical MxAccess API doc; the standalone `mxaccess_documentation.md` previously kept in this repo retired in PR 7.3.
|
||
|
||
## Build Commands
|
||
|
||
```bash
|
||
dotnet restore ZB.MOM.WW.OtOpcUa.slnx
|
||
dotnet build ZB.MOM.WW.OtOpcUa.slnx
|
||
dotnet test ZB.MOM.WW.OtOpcUa.slnx # all tests
|
||
dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests # a single test project
|
||
dotnet test --filter "FullyQualifiedName~MyTestClass.MyMethod" # a single test
|
||
```
|
||
|
||
CVE-2025-6965 / GHSA-2m69-gcr7-jv3q (transitive native `SQLitePCLRaw.lib.e_sqlite3`)
|
||
is **resolved** — the temporary `NuGetAuditSuppress` was removed once the patched bundle
|
||
shipped. `Core.AlarmHistorian` now carries a surgical direct pin of
|
||
`SQLitePCLRaw.bundle_e_sqlite3` to `2.1.12` (the first fixed native bundle, embedding the
|
||
SQLite 3.50.2+ patch), overriding the `2.1.11` that `Microsoft.Data.Sqlite` pulls
|
||
transitively. `dotnet restore`/`build` is clean with the audit active.
|
||
|
||
Test projects live under `tests/<module>/` (Core, Server, Drivers,
|
||
Drivers/Cli, Client, Tooling) — there is no single unit-test project.
|
||
Unit suites are named `*.Tests`; integration suites are `*.IntegrationTests`
|
||
and need their Docker fixture up (see Docker Workflow). DB-backed tests in
|
||
`*.Configuration.Tests`, `*.Admin.Tests`, and `*.Server.Tests` require the
|
||
central SQL Server.
|
||
|
||
## Docker Workflow (driver fixtures + central SQL Server)
|
||
|
||
> **Infra index:** [`infra/README.md`](infra/README.md) is the canonical index of all testing +
|
||
> deployment infrastructure — the Docker host layout, driver fixtures, the env-gated **live-gate**
|
||
> recipes (S7 blackhole, GLAuth outage, HistorianGateway LiveIntegration), the integration-test
|
||
> harness, the docker-dev rig, and the deploy setup. Start there; the sections below + `docs/v2/dev-environment.md` carry the detail.
|
||
|
||
> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. The driver fixture compose files under `tests/.../Docker/docker-compose.yml` now carry a `project: lmxopcua` label on every service. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28).
|
||
|
||
Docker workloads run on a shared Linux host at **`10.100.0.35`** — not on this VM. Stacks live at `/opt/otopcua-<driver>/` on the host and carry the `project=lmxopcua` label so they're discoverable via `docker ps --filter label=project=lmxopcua`.
|
||
|
||
**`docker -H ssh://...` does NOT work from this VM.** Windows OpenSSH ↔ docker.exe stdio bridging hangs (`docker system dial-stdio` runs server-side but no API data flows). Use the helper below — it SSHes into the docker host and runs `docker compose` server-side.
|
||
|
||
**Use `lmxopcua-fix.ps1` (in `~/bin`) to control fixtures from this VM:**
|
||
|
||
```powershell
|
||
lmxopcua-fix ls # list all lmxopcua-tagged containers on the host
|
||
lmxopcua-fix up modbus standard # bring a profile up
|
||
lmxopcua-fix up abcip controllogix
|
||
lmxopcua-fix up s7 s7_1500
|
||
lmxopcua-fix up opcuaclient # single-service stack, no profile arg
|
||
lmxopcua-fix down modbus # tear stack down
|
||
lmxopcua-fix logs modbus
|
||
lmxopcua-fix sync modbus # rsync this repo's tests/.../Docker/ → /opt/otopcua-modbus/
|
||
```
|
||
|
||
**`sync` is the deployment step.** When you edit a fixture's compose file or Dockerfile under `tests/.../Docker/`, run `lmxopcua-fix sync <driver>` to push the changes to the docker host before bringing the stack up. The repo files are the source of truth; `/opt/otopcua-<driver>/` is a mirrored deployment.
|
||
|
||
**Endpoints (defaults already point at the docker host):**
|
||
- SQL Server (always-on): `10.100.0.35,14330` — used by `appsettings.json` for `ConfigDb`.
|
||
- Modbus: `10.100.0.35:5020` (`MODBUS_SIM_ENDPOINT`)
|
||
- AB CIP: `10.100.0.35:44818` (`AB_SERVER_ENDPOINT`)
|
||
- S7: `10.100.0.35:1102` (`S7_SIM_ENDPOINT`)
|
||
- OPC UA reference (opc-plc): `opc.tcp://10.100.0.35:50000` (`OPCUA_SIM_ENDPOINT`)
|
||
|
||
Override any endpoint via the env var to point at a real PLC. The local OtOpcUa server runs on this VM at `opc.tcp://localhost:4840` — **that's not on the docker host**.
|
||
|
||
**Local docker-dev rig — login is DISABLED, so do live `/run` verification yourself (don't wait for the user to sign in).** The local `docker-dev/docker-compose.yml` stack (AdminUI at `http://localhost:9200` via Traefik; OPC UA `opc.tcp://localhost:4840` central-1 / `:4841` central-2) runs the AdminUI with `Security__Auth__DisableLogin: "true"` — **no sign-in form; it's auto-authenticated as a full-access admin.** So AdminUI / Razor `/run` verification (deploy a config, drive a page, confirm behavior — e.g. via the Chrome browser-automation tools against `http://localhost:9200`) does **not** require the user to log in. Run it yourself; do not defer it as "user-driven sign-in required." (Caveat: OPC UA *data-plane* auth is still real LDAP against the shared GLAuth on `10.100.0.35:3893` — that only gates Client.CLI read/write **role** operations, e.g. binding a `multi-role` / `opc-writeop` user, and is independent of the AdminUI login. Things genuinely outside the local rig — real PLCs, or the AVEVA Historian reached via the `ZB.MOM.WW.HistorianGateway` sidecar — still need the user.)
|
||
|
||
See `docs/v2/dev-environment.md` for the full inventory and rationale.
|
||
|
||
## Build & Runtime Constraints
|
||
|
||
- Language: C#, .NET 10, AnyCPU. The MXAccess COM bitness constraint
|
||
is owned by the mxaccessgw worker (x86 net48), not by anything in
|
||
this repo.
|
||
- The gateway's MXAccess worker requires a deployed ArchestrA Platform
|
||
on the machine running the gateway. The OtOpcUa server itself does
|
||
not.
|
||
|
||
## Transport Security
|
||
|
||
The server supports configurable OPC UA transport security via the `OpcUa:EnabledSecurityProfiles` list in `appsettings.json`. Phase 1 profiles (the `OpcUaSecurityProfile` enum members): `None` (default), `Basic256Sha256Sign`, `Basic256Sha256SignAndEncrypt`. Security policies are built from the enabled profiles by `BuildSecurityPolicies` at startup. The server certificate is always created even for `None`-only deployments because `UserName` token encryption depends on it. See `docs/security.md` for the full guide.
|
||
|
||
## Redundancy
|
||
|
||
The server supports non-transparent warm/hot redundancy via the `Redundancy` section in `appsettings.json`. Two instances share the same Galaxy DB and the same mxaccessgw (under distinct `MxAccess.ClientName` values) but have unique `ApplicationUri` values. Each exposes `RedundancySupport`, `ServerUriArray`, and a dynamic `ServiceLevel` based on role and runtime health. The primary advertises a higher ServiceLevel than the secondary. See `docs/Redundancy.md` for the full guide.
|
||
|
||
## LDAP Authentication
|
||
|
||
The server uses LDAP-based user authentication via the `Security:Ldap` section in `appsettings.json`. When enabled, credentials are validated by LDAP bind against a GLAuth server, and LDAP group membership maps to OPC UA permissions: `ReadOnly` (browse/read), `WriteOperate` (write FreeAccess/Operate attributes), `WriteTune` (write Tune attributes), `WriteConfigure` (write Configure attributes), `AlarmAck` (alarm acknowledgment). `LdapOpcUaUserAuthenticator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/LdapOpcUaUserAuthenticator.cs`) implements `IOpcUaUserAuthenticator`, delegating the LDAP bind + group lookup to `OtOpcUaLdapAuthService` (`src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/OtOpcUaLdapAuthService.cs`, an `ILdapAuthService`). See `docs/security.md` for the full guide.
|
||
|
||
Dev/test LDAP is the **shared GLAuth** running on the Linux Docker host at `10.100.0.35:3893` (baseDN `dc=zb,dc=local`, plaintext/`Transport=None`). It is managed via `scadaproj/infra/glauth/` (source of truth + deploy runbook). Single bind account `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123`; all test users password `password`. The docker-dev compose binds this shared instance directly — `DevStubMode` is no longer used. The per-VM NSSM GLAuth at `C:\publish\glauth\` and the old base DNs `dc=lmxopcua,dc=local` / `dc=otopcua,dc=local` are obsolete. (The integration-test harness under `tests/.../Host.IntegrationTests/` runs its own ephemeral **GLAuth** container on port 3894 — seeded by `tests/.../Host.IntegrationTests/glauth/config.toml` with dev users alice/bob + the `cn=serviceaccount` bind account — for the opt-in `OTOPCUA_HARNESS_USE_LDAP=1` real-bind mode; distinct from the shared dev GLAuth on 3893. It replaced the retired `bitnami/openldap:2.6` in PR #451, 2026-07-15.)
|
||
|
||
## Library Preferences
|
||
|
||
- **Logging**: Serilog with rolling daily file sink
|
||
- **Unit tests**: xUnit + Shouldly for assertions
|
||
- **Service hosting (Server, Admin)**: .NET generic host with `AddWindowsService` (decision #30 — replaced TopShelf in v2; see `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs`)
|
||
- **OPC UA**: OPC Foundation UA .NET Standard stack (https://github.com/opcfoundation/ua-.netstandard) — NuGet: `OPCFoundation.NetStandard.Opc.Ua.Server`
|
||
|
||
## OPC UA .NET Standard Documentation
|
||
|
||
Use the DeepWiki MCP (`mcp__deepwiki`) to query documentation for the OPC UA .NET Standard stack: `https://deepwiki.com/OPCFoundation/UA-.NETStandard`. Tools: `read_wiki_structure`, `read_wiki_contents`, and `ask_question` with repo `OPCFoundation/UA-.NETStandard`.
|
||
|
||
## Testing
|
||
|
||
Use the Client CLI at `src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/` for manual testing against the running OPC UA server. Supports connect, read, write, browse, subscribe, historyread, alarms, and redundancy commands. See `docs/Client.CLI.md` for full documentation.
|
||
|
||
```bash
|
||
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- connect -u opc.tcp://localhost:4840
|
||
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- browse -u opc.tcp://localhost:4840 -r -d 3
|
||
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- read -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode"
|
||
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- subscribe -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -i 500
|
||
```
|
||
|
||
Address pickers in AdminUI support live browse for OpcUaClient and Galaxy drivers — see `docs/plans/2026-05-28-driver-browsers-design.md`.
|
||
|
||
The AdminUI's global **UNS** page (`/uns`) is the single surface for managing the unified namespace fleet-wide (Area → Line → Equipment → Tag/VirtualTag), replacing the old per-cluster UNS/Equipment/Tags tabs. See `docs/Uns.md`.
|
||
|
||
**v3 (Batch 2): device I/O is authored in the `/raw` project tree** (`GlobalRaw.razor` → `RawTree.razor`, `IRawTreeService`) — a cluster-rooted `Folder → Driver → Device → TagGroup → Tag` hierarchy with per-node context menus (the reusable `ContextMenu`), lazy loading, driver/device config modals (endpoint moved to `Device.DeviceConfig`; Test-connect probes the `DriverDeviceConfigMerger`-merged config), manual-entry + CSV import/export (RFC-4180, staged review grid, all-or-nothing), and browse-commit of discovered leaves into raw tags. The routed `/clusters/{id}/drivers` flow (`DriverTypePicker`, `DriverEditRouter`, the 8 `*DriverPage` shells, the Drivers list) is **retired** — its form bodies live on inside the `/raw` modals. The `DriverType` dispatch maps are keyed off the `DriverTypeNames` constants (fixing the `TwinCat`/`Focas` drift). A `Calculation` pseudo-driver (`DriverTypeNames.Calculation`, `IDependencyConsumer`, deploy-time `scriptId`-existence + Tarjan cycle gates) computes signal-level tags from other tags' RawPaths. See `docs/Raw.md`.
|
||
|
||
The `/uns` **TagModal** uses **driver-typed tag-config editors**: it dispatches by the bound driver's `DriverType` to a per-driver editor (Modbus/S7/AbCip/AbLegacy/TwinCAT/Focas/OpcUaClient) via `TagConfigEditorMap`, with client-side validation via `TagConfigValidator`; unmapped drivers (only Galaxy) fall back to the generic raw-`TagConfig`-JSON textarea. Each editor is a thin razor shell over a pure `<Driver>TagConfigModel` (`FromJson`/`ToJson`/`Validate`, preserves unknown keys). To add a driver's editor, copy the Modbus template under `Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`, reusing the driver's enums + camelCase JSON property names, and register it in `TagConfigEditorMap` + `TagConfigValidator`. See `docs/plans/2026-06-09-driver-typed-tag-editors-design.md`.
|
||
|
||
## Scripting / Script Editor
|
||
|
||
C# virtual-tag scripts are authored in a **Roslyn-backed Monaco editor** on the
|
||
ScriptEdit page (`/scripts/{id}`) and inline inside the virtual-tag modal on the
|
||
`/uns` page. The editor provides completions, live diagnostics, hover, signature
|
||
help, document formatting, and tag-path completions inside `ctx.GetTag("…")` /
|
||
`ctx.SetVirtualTag("…")` literals — all backed by the same compiler context the
|
||
runtime publish gate uses, so what the editor accepts/rejects matches publish
|
||
exactly. The backend lives in
|
||
`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ScriptAnalysis/` (six minimal-API
|
||
endpoints under `/api/script-analysis/*`, gated by the `ConfigEditor` policy
|
||
(Administrator or Designer) via `RequireAuthorization` in
|
||
`ScriptAnalysisEndpoints.MapScriptAnalysisEndpoints`).
|
||
See `docs/ScriptEditor.md` for the full guide. Scripts may use the `{{equip}}` token for equipment-relative tag paths that resolve per-equipment at deploy time — see the "Equipment-relative tag paths" section in `docs/ScriptEditor.md`.
|
||
|
||
## Scripted Alarm Ack/Shelve
|
||
|
||
Inbound operator acknowledge/shelve for scripted alarms is fully implemented. Two surfaces converge on the `alarm-commands` DPS topic: (1) OPC UA Part 9 condition methods (Acknowledge / Confirm / AddComment / OneShotShelve / TimedShelve / Unshelve) wired in `OtOpcUaNodeManager`, gated on the `AlarmAck` LDAP role via `RoleCarryingUserIdentity`; (2) AdminUI `/alerts` per-row buttons routed through the `AdminOperationsActor` singleton (gated by `DriverOperator` policy). `ScriptedAlarmHostActor` dispatches from the topic to the engine. Client.CLI supports `ack`, `confirm`, `shelve` commands. See `docs/ScriptedAlarms.md` §"Inbound operator ack/shelve" and `docs/AlarmTracking.md`.
|
||
|
||
## Historian / HistoryRead
|
||
|
||
**Backend: HistorianGateway (sole historian backend).** As of the gateway-integration cutover, the
|
||
historian read, alarm-write, and continuous-historization paths are all served by the
|
||
**`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
|
||
**`ZB.MOM.WW.HistorianGateway.Client`** gRPC package (`historian_gateway.v1`) behind a thin
|
||
`IHistorianGatewayClient` seam in `ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway`. **The bespoke
|
||
Wonderware TCP/ArchestrA sidecar projects and the vestigial `Historian.Wonderware` driver type were
|
||
retired** — there is no Wonderware backend in the tree anymore (see `docs/drivers/Historian.Wonderware.md`,
|
||
now a retired stub).
|
||
|
||
A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blob (authored in the
|
||
raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default
|
||
historian tagname, which is the tag's driver `FullName`.
|
||
|
||
### Read path (`ServerHistorian` section)
|
||
|
||
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
|
||
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
|
||
(historized nodes return `GoodNoData`, never an error). Supported variants: Raw, Processed
|
||
(Average/Minimum/Maximum/Total/Count aggregates), and AtTime over historized variable nodes; Events over
|
||
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
|
||
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
|
||
|
||
`ServerHistorian` appsettings keys (`ServerHistorianOptions`; `Enabled` defaults to `false`):
|
||
|
||
| Key | Default | Notes |
|
||
|---|---|---|
|
||
| `Enabled` | `false` | `true` registers the gateway read client; `false` keeps `NullHistorianDataSource` |
|
||
| `Endpoint` | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c) |
|
||
| `ApiKey` | `""` | Peppered-HMAC key `histgw_<id>_<secret>` sent as `Authorization: Bearer`. **Supply via env `ServerHistorian__ApiKey` — never commit** |
|
||
| `UseTls` | `true` | Connect over TLS; must match the `Endpoint` scheme |
|
||
| `AllowUntrustedServerCertificate` | `false` | Accept a self-signed / untrusted server cert (dev / on-prem only) |
|
||
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
|
||
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
|
||
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
|
||
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
|
||
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
|
||
| `HistoryReadDeadline` | `00:01:00` | Per-request HistoryRead wall-clock deadline (03/S3); a node still in flight surfaces `BadTimeout`. Non-positive = unbounded |
|
||
|
||
### Alarm-history path (`AlarmHistorian` section)
|
||
|
||
Alarm events are written through `GatewayAlarmHistorianWriter` (the gateway **`SendEvent`** path) behind
|
||
the durable **`SqliteStoreAndForwardSink`** — `AlarmHistorian:Enabled=true` swaps the `NullAlarmHistorianSink`
|
||
default for the SQLite store-and-forward queue, whose drain worker forwards batches to the gateway and uses
|
||
per-event outcomes to decide retry vs. dead-letter (never throws). The `AlarmHistorian` section carries
|
||
only the `Enabled` gate + the SQLite knobs (`DatabasePath`, `DrainIntervalSeconds`, `Capacity`,
|
||
`DeadLetterRetentionDays`, `BatchSize`, `MaxAttempts`) — the downstream gateway connection
|
||
(endpoint/key/TLS) is sourced from the `ServerHistorian` section. **Alarm-history `ReadEvents` requires the
|
||
target gateway deployed with `RuntimeDb:EventReadsEnabled=true`** (the C2 SQL event-read workaround).
|
||
|
||
### Continuous historization (`ContinuousHistorization` section)
|
||
|
||
When `ContinuousHistorization:Enabled=true` **and** `ServerHistorian` is configured, the Host builds a
|
||
durable, crash-safe **FasterLog** outbox (`FasterLogHistorizationOutbox`) + a gateway-backed
|
||
`IHistorianValueWriter`, and `WithOtOpcUaRuntimeActors` spawns the `ContinuousHistorizationRecorder`. The
|
||
recorder taps the per-node dependency-mux value fan-out, appends each numeric value to the outbox (the
|
||
crash boundary), and drains the outbox to the gateway's SQL live-value write path (**`WriteLiveValues`**)
|
||
with exponential backoff. The gateway connection is sourced from `ServerHistorian`; this section carries
|
||
only the recorder + outbox knobs:
|
||
|
||
| Key | Default | Notes |
|
||
|---|---|---|
|
||
| `Enabled` | `false` | `true` (with `ServerHistorian` configured) wires + spawns the recorder |
|
||
| `OutboxPath` | `""` (required when enabled) | **Directory** holding the FasterLog segment + commit files. In production set an **absolute** path on durable storage |
|
||
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before each append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs` |
|
||
| `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
|
||
| `DrainBatchSize` | `64` | Entries peeked + written per drain pass |
|
||
| `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
|
||
| `Capacity` | `0` | Max un-acked outbox entries before drop-oldest; `0` = unbounded |
|
||
| `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
|
||
| `MaxBackoffSeconds` | `30` | Cap on the exponential retry backoff |
|
||
|
||
### Tag auto-provisioning (`IHistorianProvisioning` EnsureTags hook)
|
||
|
||
`AddressSpaceApplier.Apply()` fires a **non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync`
|
||
hook for added historized value tags — the gateway-backed `GatewayTagProvisioner` calls the gateway's
|
||
`EnsureTags` so a brand-new historized tag exists in the historian before the recorder's `WriteLiveValues`
|
||
land. The hook is wrapped so a faulted/synchronously-throwing provisioner can **never** block or fail a
|
||
deploy. Non-numeric (`String`/`DateTime`/`Reference`) data types are skipped (not provisioned); the
|
||
recorder likewise drops + meters non-numeric values. Continuous historization is **numeric-analog only** in
|
||
v1 (`UInt16→UInt4` is a documented fallback).
|
||
|
||
The provisioner is registered by the Host's `AddHistorianProvisioning` (`Runtime.ServiceCollectionExtensions`),
|
||
built via `GatewayHistorian.CreateProvisioner`, and **gated on the same `ServerHistorian:Enabled` flag as the
|
||
read path** — `WithOtOpcUaRuntimeActors` resolves `IHistorianProvisioning` and passes it into the applier.
|
||
When the section is disabled the applier binds the no-op `NullHistorianProvisioning` and provisioning does not
|
||
run. Every dispatch logs a tally (`dispatched/requested/ensured/skipped/failed`); `dispatched=N` with
|
||
`requested=0` flags the dormant no-op. The API key must carry `historian:tags:write` for `EnsureTags`. (PR
|
||
#423 shipped `GatewayTagProvisioner` but left this wiring out, so provisioning was dormant; restored 2026-06-27.)
|
||
**DataType mapping (issue #441):** `EnsureTags` is an **upsert** (create-or-update, not skip-if-exists).
|
||
`HistorianTypeMapper` maps **`Boolean → Int2`** (Boolean historizes as a small 0/1 integer) because the
|
||
historian's `Int1` analog-tag creation path is **server-degenerate** — `EnsureTags(Int1)` stores an unusable
|
||
stub (live-probed; gateway-side resolution in HistorianGateway #11 / PR #16). Both Float and Int2 are
|
||
supported analog types, so a pre-existing Float Boolean tag retypes cleanly to Int2 on its next deploy — see
|
||
the "Boolean historization maps to Int2" section in `docs/Historian.md`.
|
||
|
||
### Gateway-side prerequisites
|
||
|
||
The target HistorianGateway OtOpcUa points at **must** run with:
|
||
|
||
- `RuntimeDb:Enabled=true` — enables the `WriteLiveValues` SQL live path (continuous historization).
|
||
- `RuntimeDb:EventReadsEnabled=true` — enables `ReadEvents` from `Runtime.dbo.Events` (alarm-history reads).
|
||
- An API key carrying scopes **`historian:read`**, **`historian:write`**, **`historian:tags:write`**.
|
||
|
||
### Migration note (deployments upgrading from the Wonderware backend)
|
||
|
||
The `ServerHistorian` section changed shape. Rename the old Wonderware keys and supply the key via env:
|
||
|
||
| Old (Wonderware) key | New (gateway) key |
|
||
|---|---|
|
||
| `ServerHistorian:Host` + `:Port` | `ServerHistorian:Endpoint` (`https://host:5222`) |
|
||
| `ServerHistorian:SharedSecret` | `ServerHistorian:ApiKey` (**env `ServerHistorian__ApiKey`**) |
|
||
| `ServerHistorian:ServerCertThumbprint` | `ServerHistorian:CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`) |
|
||
|
||
The `AlarmHistorian` section's old Wonderware connection keys (`Host`/`Port`/`UseTls`/`ServerCertThumbprint`/`SharedSecret`)
|
||
were pruned — remove them; the SQLite knobs are retained and the downstream connection now comes from
|
||
`ServerHistorian`. See `docs/Historian.md` for the full guide.
|
||
|
||
### KNOWN LIMITATION 1 — live-validation gate (do before merging/trusting the cutover)
|
||
|
||
The cutover is code-complete but **must be live-validated against a real gateway** (VPN to
|
||
`wonder-sql-vd03`, gateway running the prerequisites above) before it is merged or trusted. Run the
|
||
env-gated suite:
|
||
|
||
```bash
|
||
export HISTGW_GATEWAY_ENDPOINT=https://wonder-sql-vd03:5222 # absolute gateway URI; absent ⇒ all live tests skip
|
||
export HISTGW_GATEWAY_APIKEY=histgw_<id>_<secret> # must carry historian:read + historian:write (+ tags:write) scopes
|
||
export HISTGW_TEST_TAG=<existing-tag> # read round-trip
|
||
export HISTGW_WRITE_SANDBOX_TAG=<writable-float-tag> # e.g. HistGW.LiveTest.Sandbox — write round-trip (EnsureTags + write)
|
||
export HISTGW_BOOL_SANDBOX_TAG=<writable-bool-tag> # e.g. HistGW.LiveTest.BoolSandbox — Boolean→Int2 EnsureTags leg + retype probe (issue #441)
|
||
export HISTGW_ALARM_SOURCE=<source-name> # alarm SendEvent → ReadEvents round-trip
|
||
dotnet test --filter "Category=LiveIntegration"
|
||
```
|
||
|
||
The live suite **skips cleanly** when these env vars are absent (safe to run offline on macOS). It is the
|
||
gate the operator runs on the VPN before trusting the cutover.
|
||
|
||
### KNOWN LIMITATION 2 — continuous-historization value-capture wired; live end-to-end verification still pending
|
||
|
||
The `ContinuousHistorizationRecorder` is fully wired (actor + FasterLog outbox + gateway value-writer +
|
||
meters). It is spawned with an **empty *initial* ref set** (`Array.Empty<string>()` in
|
||
`WithOtOpcUaRuntimeActors`) because the deployed address space — and thus the historized-ref set — is built
|
||
later at deploy time, not at actor-spawn time. **The ref-feed gap is now closed:** on every deploy the
|
||
`AddressSpaceApplier.FeedHistorizedRefs` hook computes the added/removed historized-ref delta and posts it
|
||
through `ActorHistorizedTagSubscriptionSink` → the recorder's `UpdateHistorizedRefs` message, and the
|
||
recorder keeps the full set and re-registers its dependency-mux interest. So the recorder converges to the
|
||
deployed set of historized value tags from the first deploy onward (the feed is a non-blocking, deploy-safe
|
||
`Tell`; a faulting feed can never break a deploy). **Reads and alarm-writes work today, and the value-capture
|
||
path is code-complete.** What remains is the **live gate**: an end-to-end verification against a real gateway
|
||
(deploy → dependency-mux value delta → outbox append → `WriteLiveValues`) plus a restart-convergence test
|
||
proving the recorder re-registers the deployed set after a process restart. Until that live verification runs,
|
||
treat continuous value-capture as unproven-in-production rather than absent.
|