Files
lmxopcua/CLAUDE.md
T
Joseph Doherty 33915c9e4d
v2-ci / build (pull_request) Successful in 3m25s
v2-ci / unit-tests (pull_request) Failing after 8m37s
docs(v3): Batch 2 — /raw authoring tree guide + CLAUDE.md + PR description
docs/Raw.md (the /raw project tree: lazy tree, endpoint→DeviceConfig split, manual/CSV/browse
authoring, Calculation driver, routed-flow retirement); CLAUDE.md AdminUI section notes /raw +
DriverTypeNames rewire + Calculation driver; PR body with the 7-item live-gate evidence.

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 05:27:31 -04:00

394 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Goal
Build an OPC UA server (.NET 10) that exposes industrial data sources —
including AVEVA System Platform (Wonderware) Galaxy — under a unified
Equipment-based address space. Galaxy access flows through the in-process
`GalaxyDriver` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/`) talking gRPC
to a separately installed **mxaccessgw** gateway process. The gateway owns the
MXAccess COM bitness constraint (its worker is x86 net48); everything in this
repo is .NET 10. PR 7.2 retired the legacy in-process `Galaxy.Host` /
`Galaxy.Proxy` / `Galaxy.Shared` projects + the `OtOpcUaGalaxyHost` Windows
service.
See `docs/v2/Galaxy.Performance.md` for the runtime perf surface
(tracing, metrics, soak harness).
## Related Projects (scadaproj umbrella)
This repo is one of a family of related SCADA / OT / Wonderware / OPC UA
**sister projects** that live as sibling directories under `~/Desktop/`. They are
**separate repos and separate processes**, coupled at runtime over wire protocols
(gRPC + OPC UA) — not by project/compile references — and share the
`ZB.MOM.WW.*` product namespace. The common subject is AVEVA System Platform
(Wonderware) "Galaxy" data, and `mxaccessgw` is the linchpin the others connect
through.
- **`~/Desktop/scadaproj`** ([`../scadaproj/CLAUDE.md`](../scadaproj/CLAUDE.md)) —
the umbrella/index workspace that aggregates the whole family (a high-level scan
of each sister project: purpose, location, stack, primary commands). It also hosts
the shared `ZB.MOM.WW.*` libraries — `ZB.MOM.WW.Auth`, `ZB.MOM.WW.Theme`,
`ZB.MOM.WW.Health`, `ZB.MOM.WW.Telemetry`, `ZB.MOM.WW.Configuration`, and
`ZB.MOM.WW.GalaxyRepository`. **This repo is indexed there** — see the OtOpcUa
entry in `../scadaproj/CLAUDE.md`.
- **`~/Desktop/MxAccessGateway`** (`mxaccessgw`,
[`../MxAccessGateway/CLAUDE.md`](../MxAccessGateway/CLAUDE.md)) — the gRPC gateway
that owns the MXAccess 32-bit COM bitness + STA pump (gateway x64 .NET 10 + per-session
x86 net48 worker). **OtOpcUa's `GalaxyDriver` depends on this** for live Galaxy
read/write/subscribe and Galaxy Repository browse.
- **`~/Desktop/HistorianGateway`**
([`../HistorianGateway/CLAUDE.md`](../HistorianGateway/CLAUDE.md)) — single-process
gRPC sidecar exposing the AVEVA Historian read/write API + read-only Galaxy browse
(via the shared `ZB.MOM.WW.GalaxyRepository` lib). Patterned on `mxaccessgw` but with
no COM / no x86 worker. **OtOpcUa's historian backend** (`Driver.Historian.Gateway`)
consumes it as the sole historian path.
- **`~/Desktop/ScadaBridge`**
([`../ScadaBridge/CLAUDE.md`](../ScadaBridge/CLAUDE.md)) — the distributed SCADA
platform (Akka.NET hub-and-spoke) whose Data Connection Layer consumes this server's
OPC UA address space.
**Propagate cross-repo changes to the umbrella index.** When a fact the index records
about OtOpcUa changes here — remote/push status, the driver set, the Galaxy data flow,
shared-lib consumption, or per-project commands — update the **OtOpcUa entry in
`../scadaproj/CLAUDE.md`** in the same change so the index never drifts from this repo.
## Architecture Overview
### Data Flow
1. **Galaxy Repository DB (ZB)** — SQL Server database holding the
deployed object hierarchy and attribute definitions. The
mxaccessgw's `GalaxyRepositoryClient` queries it via gRPC; the
driver consumes the materialised hierarchy through
`IGalaxyHierarchySource`.
2. **MXAccess (via mxaccessgw)** — Live read/write/subscribe over a
gRPC session. The gateway owns the COM apartment + STA pump
server-side; the driver speaks `MxCommand` / `MxEvent` protos
exclusively.
3. **OPC UA Server** — Exposes authored equipment tags as variable nodes.
Galaxy tags are bound by `TagConfig.FullName` (`tag_name.AttributeName`);
reads/writes/subscriptions are translated to that reference for MXAccess.
### Key Concept: Tag Name and FullName
Galaxy objects have a **tag_name** — a globally unique system name used for
MXAccess read/write — and attributes are referenced as `tag_name.AttributeName`
(e.g. `DelmiaReceiver_001.DownloadPath`). This dot-separated reference is what
`TagConfig.FullName` stores for a Galaxy equipment tag. The Galaxy address
picker browses the live hierarchy using contained names for navigation, then
resolves the selection to the `tag_name.AttributeName` form.
**Galaxy is a standard Equipment-kind driver.** Galaxy points are ordinary
equipment `Tag`s bound to `GalaxyMxGateway` via `TagConfig.FullName`
(`tag_name.AttributeName`), authored through the standard Tag modal + Galaxy
address picker on the equipment page's Tags tab — the same flow as Modbus or
S7. There is no alias machinery, no `SystemPlatform` namespace kind, and no
relay→alias converter. See
`docs/plans/2026-06-12-galaxy-standard-driver-design.md` for the full design.
### Data Type Mapping
Galaxy `mx_data_type` values map to OPC UA types (Boolean, Int32, Float, Double, String, DateTime, etc.). Array attributes use ValueRank=1 with ArrayDimensions from the Galaxy attribute definition. The driver-side mapping lives in `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Browse/DataTypeMap.cs`.
### Change Detection
`DeployWatcher` (`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/Browse/DeployWatcher.cs`) polls the gateway's deploy-event signal and raises `IRediscoverable.OnRediscoveryNeeded` when the Galaxy redeploys. The server's `DriverHost` consumes the signal and rebuilds the address space.
## mxaccessgw
The gateway lives in a sibling repo at `~/Desktop/MxAccessGateway/` (`mxaccessgw` — see [Related Projects](#related-projects-scadaproj-umbrella)). See `docs/v2/Galaxy.ParityRig.md` for the gw setup recipe (build, API key provisioning via `apikey create-key`, env-var overrides for HTTP/2 cleartext + worker path). The gw's MXAccess Toolkit reference (its `gateway.md`) is the canonical MxAccess API doc; the standalone `mxaccess_documentation.md` previously kept in this repo retired in PR 7.3.
## Build Commands
```bash
dotnet restore ZB.MOM.WW.OtOpcUa.slnx
dotnet build ZB.MOM.WW.OtOpcUa.slnx
dotnet test ZB.MOM.WW.OtOpcUa.slnx # all tests
dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Core.Tests # a single test project
dotnet test --filter "FullyQualifiedName~MyTestClass.MyMethod" # a single test
```
CVE-2025-6965 / GHSA-2m69-gcr7-jv3q (transitive native `SQLitePCLRaw.lib.e_sqlite3`)
is **resolved** — the temporary `NuGetAuditSuppress` was removed once the patched bundle
shipped. `Core.AlarmHistorian` now carries a surgical direct pin of
`SQLitePCLRaw.bundle_e_sqlite3` to `2.1.12` (the first fixed native bundle, embedding the
SQLite 3.50.2+ patch), overriding the `2.1.11` that `Microsoft.Data.Sqlite` pulls
transitively. `dotnet restore`/`build` is clean with the audit active.
Test projects live under `tests/<module>/` (Core, Server, Drivers,
Drivers/Cli, Client, Tooling) — there is no single unit-test project.
Unit suites are named `*.Tests`; integration suites are `*.IntegrationTests`
and need their Docker fixture up (see Docker Workflow). DB-backed tests in
`*.Configuration.Tests`, `*.Admin.Tests`, and `*.Server.Tests` require the
central SQL Server.
## Docker Workflow (driver fixtures + central SQL Server)
> **Infra index:** [`infra/README.md`](infra/README.md) is the canonical index of all testing +
> deployment infrastructure — the Docker host layout, driver fixtures, the env-gated **live-gate**
> recipes (S7 blackhole, GLAuth outage, HistorianGateway LiveIntegration), the integration-test
> harness, the docker-dev rig, and the deploy setup. Start there; the sections below + `docs/v2/dev-environment.md` carry the detail.
> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. The driver fixture compose files under `tests/.../Docker/docker-compose.yml` now carry a `project: lmxopcua` label on every service. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28).
Docker workloads run on a shared Linux host at **`10.100.0.35`** — not on this VM. Stacks live at `/opt/otopcua-<driver>/` on the host and carry the `project=lmxopcua` label so they're discoverable via `docker ps --filter label=project=lmxopcua`.
**`docker -H ssh://...` does NOT work from this VM.** Windows OpenSSH ↔ docker.exe stdio bridging hangs (`docker system dial-stdio` runs server-side but no API data flows). Use the helper below — it SSHes into the docker host and runs `docker compose` server-side.
**Use `lmxopcua-fix.ps1` (in `~/bin`) to control fixtures from this VM:**
```powershell
lmxopcua-fix ls # list all lmxopcua-tagged containers on the host
lmxopcua-fix up modbus standard # bring a profile up
lmxopcua-fix up abcip controllogix
lmxopcua-fix up s7 s7_1500
lmxopcua-fix up opcuaclient # single-service stack, no profile arg
lmxopcua-fix down modbus # tear stack down
lmxopcua-fix logs modbus
lmxopcua-fix sync modbus # rsync this repo's tests/.../Docker/ → /opt/otopcua-modbus/
```
**`sync` is the deployment step.** When you edit a fixture's compose file or Dockerfile under `tests/.../Docker/`, run `lmxopcua-fix sync <driver>` to push the changes to the docker host before bringing the stack up. The repo files are the source of truth; `/opt/otopcua-<driver>/` is a mirrored deployment.
**Endpoints (defaults already point at the docker host):**
- SQL Server (always-on): `10.100.0.35,14330` — used by `appsettings.json` for `ConfigDb`.
- Modbus: `10.100.0.35:5020` (`MODBUS_SIM_ENDPOINT`)
- AB CIP: `10.100.0.35:44818` (`AB_SERVER_ENDPOINT`)
- S7: `10.100.0.35:1102` (`S7_SIM_ENDPOINT`)
- OPC UA reference (opc-plc): `opc.tcp://10.100.0.35:50000` (`OPCUA_SIM_ENDPOINT`)
Override any endpoint via the env var to point at a real PLC. The local OtOpcUa server runs on this VM at `opc.tcp://localhost:4840`**that's not on the docker host**.
**Local docker-dev rig — login is DISABLED, so do live `/run` verification yourself (don't wait for the user to sign in).** The local `docker-dev/docker-compose.yml` stack (AdminUI at `http://localhost:9200` via Traefik; OPC UA `opc.tcp://localhost:4840` central-1 / `:4841` central-2) runs the AdminUI with `Security__Auth__DisableLogin: "true"`**no sign-in form; it's auto-authenticated as a full-access admin.** So AdminUI / Razor `/run` verification (deploy a config, drive a page, confirm behavior — e.g. via the Chrome browser-automation tools against `http://localhost:9200`) does **not** require the user to log in. Run it yourself; do not defer it as "user-driven sign-in required." (Caveat: OPC UA *data-plane* auth is still real LDAP against the shared GLAuth on `10.100.0.35:3893` — that only gates Client.CLI read/write **role** operations, e.g. binding a `multi-role` / `opc-writeop` user, and is independent of the AdminUI login. Things genuinely outside the local rig — real PLCs, or the AVEVA Historian reached via the `ZB.MOM.WW.HistorianGateway` sidecar — still need the user.)
See `docs/v2/dev-environment.md` for the full inventory and rationale.
## Build & Runtime Constraints
- Language: C#, .NET 10, AnyCPU. The MXAccess COM bitness constraint
is owned by the mxaccessgw worker (x86 net48), not by anything in
this repo.
- The gateway's MXAccess worker requires a deployed ArchestrA Platform
on the machine running the gateway. The OtOpcUa server itself does
not.
## Transport Security
The server supports configurable OPC UA transport security via the `OpcUa:EnabledSecurityProfiles` list in `appsettings.json`. Phase 1 profiles (the `OpcUaSecurityProfile` enum members): `None` (default), `Basic256Sha256Sign`, `Basic256Sha256SignAndEncrypt`. Security policies are built from the enabled profiles by `BuildSecurityPolicies` at startup. The server certificate is always created even for `None`-only deployments because `UserName` token encryption depends on it. See `docs/security.md` for the full guide.
## Redundancy
The server supports non-transparent warm/hot redundancy via the `Redundancy` section in `appsettings.json`. Two instances share the same Galaxy DB and the same mxaccessgw (under distinct `MxAccess.ClientName` values) but have unique `ApplicationUri` values. Each exposes `RedundancySupport`, `ServerUriArray`, and a dynamic `ServiceLevel` based on role and runtime health. The primary advertises a higher ServiceLevel than the secondary. See `docs/Redundancy.md` for the full guide.
## LDAP Authentication
The server uses LDAP-based user authentication via the `Security:Ldap` section in `appsettings.json`. When enabled, credentials are validated by LDAP bind against a GLAuth server, and LDAP group membership maps to OPC UA permissions: `ReadOnly` (browse/read), `WriteOperate` (write FreeAccess/Operate attributes), `WriteTune` (write Tune attributes), `WriteConfigure` (write Configure attributes), `AlarmAck` (alarm acknowledgment). `LdapOpcUaUserAuthenticator` (`src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/LdapOpcUaUserAuthenticator.cs`) implements `IOpcUaUserAuthenticator`, delegating the LDAP bind + group lookup to `OtOpcUaLdapAuthService` (`src/Server/ZB.MOM.WW.OtOpcUa.Security/Ldap/OtOpcUaLdapAuthService.cs`, an `ILdapAuthService`). See `docs/security.md` for the full guide.
Dev/test LDAP is the **shared GLAuth** running on the Linux Docker host at `10.100.0.35:3893` (baseDN `dc=zb,dc=local`, plaintext/`Transport=None`). It is managed via `scadaproj/infra/glauth/` (source of truth + deploy runbook). Single bind account `cn=serviceaccount,dc=zb,dc=local` / `serviceaccount123`; all test users password `password`. The docker-dev compose binds this shared instance directly — `DevStubMode` is no longer used. The per-VM NSSM GLAuth at `C:\publish\glauth\` and the old base DNs `dc=lmxopcua,dc=local` / `dc=otopcua,dc=local` are obsolete. (The integration-test harness under `tests/.../Host.IntegrationTests/` runs its own ephemeral **GLAuth** container on port 3894 — seeded by `tests/.../Host.IntegrationTests/glauth/config.toml` with dev users alice/bob + the `cn=serviceaccount` bind account — for the opt-in `OTOPCUA_HARNESS_USE_LDAP=1` real-bind mode; distinct from the shared dev GLAuth on 3893. It replaced the retired `bitnami/openldap:2.6` in PR #451, 2026-07-15.)
## Library Preferences
- **Logging**: Serilog with rolling daily file sink
- **Unit tests**: xUnit + Shouldly for assertions
- **Service hosting (Server, Admin)**: .NET generic host with `AddWindowsService` (decision #30 — replaced TopShelf in v2; see `src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.cs`)
- **OPC UA**: OPC Foundation UA .NET Standard stack (https://github.com/opcfoundation/ua-.netstandard) — NuGet: `OPCFoundation.NetStandard.Opc.Ua.Server`
## OPC UA .NET Standard Documentation
Use the DeepWiki MCP (`mcp__deepwiki`) to query documentation for the OPC UA .NET Standard stack: `https://deepwiki.com/OPCFoundation/UA-.NETStandard`. Tools: `read_wiki_structure`, `read_wiki_contents`, and `ask_question` with repo `OPCFoundation/UA-.NETStandard`.
## Testing
Use the Client CLI at `src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI/` for manual testing against the running OPC UA server. Supports connect, read, write, browse, subscribe, historyread, alarms, and redundancy commands. See `docs/Client.CLI.md` for full documentation.
```bash
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- connect -u opc.tcp://localhost:4840
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- browse -u opc.tcp://localhost:4840 -r -d 3
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- read -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode"
dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- subscribe -u opc.tcp://localhost:4840 -n "ns=2;s=SomeNode" -i 500
```
Address pickers in AdminUI support live browse for OpcUaClient and Galaxy drivers — see `docs/plans/2026-05-28-driver-browsers-design.md`.
The AdminUI's global **UNS** page (`/uns`) is the single surface for managing the unified namespace fleet-wide (Area → Line → Equipment → Tag/VirtualTag), replacing the old per-cluster UNS/Equipment/Tags tabs. See `docs/Uns.md`.
**v3 (Batch 2): device I/O is authored in the `/raw` project tree** (`GlobalRaw.razor``RawTree.razor`, `IRawTreeService`) — a cluster-rooted `Folder → Driver → Device → TagGroup → Tag` hierarchy with per-node context menus (the reusable `ContextMenu`), lazy loading, driver/device config modals (endpoint moved to `Device.DeviceConfig`; Test-connect probes the `DriverDeviceConfigMerger`-merged config), manual-entry + CSV import/export (RFC-4180, staged review grid, all-or-nothing), and browse-commit of discovered leaves into raw tags. The routed `/clusters/{id}/drivers` flow (`DriverTypePicker`, `DriverEditRouter`, the 8 `*DriverPage` shells, the Drivers list) is **retired** — its form bodies live on inside the `/raw` modals. The `DriverType` dispatch maps are keyed off the `DriverTypeNames` constants (fixing the `TwinCat`/`Focas` drift). A `Calculation` pseudo-driver (`DriverTypeNames.Calculation`, `IDependencyConsumer`, deploy-time `scriptId`-existence + Tarjan cycle gates) computes signal-level tags from other tags' RawPaths. See `docs/Raw.md`.
The `/uns` **TagModal** uses **driver-typed tag-config editors**: it dispatches by the bound driver's `DriverType` to a per-driver editor (Modbus/S7/AbCip/AbLegacy/TwinCAT/Focas/OpcUaClient) via `TagConfigEditorMap`, with client-side validation via `TagConfigValidator`; unmapped drivers (only Galaxy) fall back to the generic raw-`TagConfig`-JSON textarea. Each editor is a thin razor shell over a pure `<Driver>TagConfigModel` (`FromJson`/`ToJson`/`Validate`, preserves unknown keys). To add a driver's editor, copy the Modbus template under `Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`, reusing the driver's enums + camelCase JSON property names, and register it in `TagConfigEditorMap` + `TagConfigValidator`. See `docs/plans/2026-06-09-driver-typed-tag-editors-design.md`.
## Scripting / Script Editor
C# virtual-tag scripts are authored in a **Roslyn-backed Monaco editor** on the
ScriptEdit page (`/scripts/{id}`) and inline inside the virtual-tag modal on the
`/uns` page. The editor provides completions, live diagnostics, hover, signature
help, document formatting, and tag-path completions inside `ctx.GetTag("…")` /
`ctx.SetVirtualTag("…")` literals — all backed by the same compiler context the
runtime publish gate uses, so what the editor accepts/rejects matches publish
exactly. The backend lives in
`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ScriptAnalysis/` (six minimal-API
endpoints under `/api/script-analysis/*`, gated by the `ConfigEditor` policy
(Administrator or Designer) via `RequireAuthorization` in
`ScriptAnalysisEndpoints.MapScriptAnalysisEndpoints`).
See `docs/ScriptEditor.md` for the full guide. Scripts may use the `{{equip}}` token for equipment-relative tag paths that resolve per-equipment at deploy time — see the "Equipment-relative tag paths" section in `docs/ScriptEditor.md`.
## Scripted Alarm Ack/Shelve
Inbound operator acknowledge/shelve for scripted alarms is fully implemented. Two surfaces converge on the `alarm-commands` DPS topic: (1) OPC UA Part 9 condition methods (Acknowledge / Confirm / AddComment / OneShotShelve / TimedShelve / Unshelve) wired in `OtOpcUaNodeManager`, gated on the `AlarmAck` LDAP role via `RoleCarryingUserIdentity`; (2) AdminUI `/alerts` per-row buttons routed through the `AdminOperationsActor` singleton (gated by `DriverOperator` policy). `ScriptedAlarmHostActor` dispatches from the topic to the engine. Client.CLI supports `ack`, `confirm`, `shelve` commands. See `docs/ScriptedAlarms.md` §"Inbound operator ack/shelve" and `docs/AlarmTracking.md`.
## Historian / HistoryRead
**Backend: HistorianGateway (sole historian backend).** As of the gateway-integration cutover, the
historian read, alarm-write, and continuous-historization paths are all served by the
**`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
**`ZB.MOM.WW.HistorianGateway.Client`** gRPC package (`historian_gateway.v1`) behind a thin
`IHistorianGatewayClient` seam in `ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway`. **The bespoke
Wonderware TCP/ArchestrA sidecar projects and the vestigial `Historian.Wonderware` driver type were
retired** — there is no Wonderware backend in the tree anymore (see `docs/drivers/Historian.Wonderware.md`,
now a retired stub).
A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blob (authored in the
raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default
historian tagname, which is the tag's driver `FullName`.
### Read path (`ServerHistorian` section)
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
(historized nodes return `GoodNoData`, never an error). Supported variants: Raw, Processed
(Average/Minimum/Maximum/Total/Count aggregates), and AtTime over historized variable nodes; Events over
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
`ServerHistorian` appsettings keys (`ServerHistorianOptions`; `Enabled` defaults to `false`):
| Key | Default | Notes |
|---|---|---|
| `Enabled` | `false` | `true` registers the gateway read client; `false` keeps `NullHistorianDataSource` |
| `Endpoint` | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c) |
| `ApiKey` | `""` | Peppered-HMAC key `histgw_<id>_<secret>` sent as `Authorization: Bearer`. **Supply via env `ServerHistorian__ApiKey` — never commit** |
| `UseTls` | `true` | Connect over TLS; must match the `Endpoint` scheme |
| `AllowUntrustedServerCertificate` | `false` | Accept a self-signed / untrusted server cert (dev / on-prem only) |
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
| `HistoryReadBatchParallelism` | `4` | Max nodes served concurrently within one HistoryRead batch (03/S3); `⌈N/P⌉ × CallTimeout` worst case instead of sequential |
| `MaxConcurrentHistoryReadBatches` | `16` | Process-wide concurrent HistoryRead batch cap (03/S3); flood degrades to `BadTooManyOperations`. Total in-flight reads capped at 16 × 4 = 64 |
| `HistoryReadDeadline` | `00:01:00` | Per-request HistoryRead wall-clock deadline (03/S3); a node still in flight surfaces `BadTimeout`. Non-positive = unbounded |
### Alarm-history path (`AlarmHistorian` section)
Alarm events are written through `GatewayAlarmHistorianWriter` (the gateway **`SendEvent`** path) behind
the durable **`SqliteStoreAndForwardSink`** — `AlarmHistorian:Enabled=true` swaps the `NullAlarmHistorianSink`
default for the SQLite store-and-forward queue, whose drain worker forwards batches to the gateway and uses
per-event outcomes to decide retry vs. dead-letter (never throws). The `AlarmHistorian` section carries
only the `Enabled` gate + the SQLite knobs (`DatabasePath`, `DrainIntervalSeconds`, `Capacity`,
`DeadLetterRetentionDays`, `BatchSize`, `MaxAttempts`) — the downstream gateway connection
(endpoint/key/TLS) is sourced from the `ServerHistorian` section. **Alarm-history `ReadEvents` requires the
target gateway deployed with `RuntimeDb:EventReadsEnabled=true`** (the C2 SQL event-read workaround).
### Continuous historization (`ContinuousHistorization` section)
When `ContinuousHistorization:Enabled=true` **and** `ServerHistorian` is configured, the Host builds a
durable, crash-safe **FasterLog** outbox (`FasterLogHistorizationOutbox`) + a gateway-backed
`IHistorianValueWriter`, and `WithOtOpcUaRuntimeActors` spawns the `ContinuousHistorizationRecorder`. The
recorder taps the per-node dependency-mux value fan-out, appends each numeric value to the outbox (the
crash boundary), and drains the outbox to the gateway's SQL live-value write path (**`WriteLiveValues`**)
with exponential backoff. The gateway connection is sourced from `ServerHistorian`; this section carries
only the recorder + outbox knobs:
| Key | Default | Notes |
|---|---|---|
| `Enabled` | `false` | `true` (with `ServerHistorian` configured) wires + spawns the recorder |
| `OutboxPath` | `""` (required when enabled) | **Directory** holding the FasterLog segment + commit files. In production set an **absolute** path on durable storage |
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before each append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs` |
| `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
| `DrainBatchSize` | `64` | Entries peeked + written per drain pass |
| `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
| `Capacity` | `0` | Max un-acked outbox entries before drop-oldest; `0` = unbounded |
| `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
| `MaxBackoffSeconds` | `30` | Cap on the exponential retry backoff |
### Tag auto-provisioning (`IHistorianProvisioning` EnsureTags hook)
`AddressSpaceApplier.Apply()` fires a **non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync`
hook for added historized value tags — the gateway-backed `GatewayTagProvisioner` calls the gateway's
`EnsureTags` so a brand-new historized tag exists in the historian before the recorder's `WriteLiveValues`
land. The hook is wrapped so a faulted/synchronously-throwing provisioner can **never** block or fail a
deploy. Non-numeric (`String`/`DateTime`/`Reference`) data types are skipped (not provisioned); the
recorder likewise drops + meters non-numeric values. Continuous historization is **numeric-analog only** in
v1 (`UInt16→UInt4` is a documented fallback).
The provisioner is registered by the Host's `AddHistorianProvisioning` (`Runtime.ServiceCollectionExtensions`),
built via `GatewayHistorian.CreateProvisioner`, and **gated on the same `ServerHistorian:Enabled` flag as the
read path** — `WithOtOpcUaRuntimeActors` resolves `IHistorianProvisioning` and passes it into the applier.
When the section is disabled the applier binds the no-op `NullHistorianProvisioning` and provisioning does not
run. Every dispatch logs a tally (`dispatched/requested/ensured/skipped/failed`); `dispatched=N` with
`requested=0` flags the dormant no-op. The API key must carry `historian:tags:write` for `EnsureTags`. (PR
#423 shipped `GatewayTagProvisioner` but left this wiring out, so provisioning was dormant; restored 2026-06-27.)
**DataType mapping (issue #441):** `EnsureTags` is an **upsert** (create-or-update, not skip-if-exists).
`HistorianTypeMapper` maps **`Boolean → Int2`** (Boolean historizes as a small 0/1 integer) because the
historian's `Int1` analog-tag creation path is **server-degenerate**`EnsureTags(Int1)` stores an unusable
stub (live-probed; gateway-side resolution in HistorianGateway #11 / PR #16). Both Float and Int2 are
supported analog types, so a pre-existing Float Boolean tag retypes cleanly to Int2 on its next deploy — see
the "Boolean historization maps to Int2" section in `docs/Historian.md`.
### Gateway-side prerequisites
The target HistorianGateway OtOpcUa points at **must** run with:
- `RuntimeDb:Enabled=true` — enables the `WriteLiveValues` SQL live path (continuous historization).
- `RuntimeDb:EventReadsEnabled=true` — enables `ReadEvents` from `Runtime.dbo.Events` (alarm-history reads).
- An API key carrying scopes **`historian:read`**, **`historian:write`**, **`historian:tags:write`**.
### Migration note (deployments upgrading from the Wonderware backend)
The `ServerHistorian` section changed shape. Rename the old Wonderware keys and supply the key via env:
| Old (Wonderware) key | New (gateway) key |
|---|---|
| `ServerHistorian:Host` + `:Port` | `ServerHistorian:Endpoint` (`https://host:5222`) |
| `ServerHistorian:SharedSecret` | `ServerHistorian:ApiKey` (**env `ServerHistorian__ApiKey`**) |
| `ServerHistorian:ServerCertThumbprint` | `ServerHistorian:CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`) |
The `AlarmHistorian` section's old Wonderware connection keys (`Host`/`Port`/`UseTls`/`ServerCertThumbprint`/`SharedSecret`)
were pruned — remove them; the SQLite knobs are retained and the downstream connection now comes from
`ServerHistorian`. See `docs/Historian.md` for the full guide.
### KNOWN LIMITATION 1 — live-validation gate (do before merging/trusting the cutover)
The cutover is code-complete but **must be live-validated against a real gateway** (VPN to
`wonder-sql-vd03`, gateway running the prerequisites above) before it is merged or trusted. Run the
env-gated suite:
```bash
export HISTGW_GATEWAY_ENDPOINT=https://wonder-sql-vd03:5222 # absolute gateway URI; absent ⇒ all live tests skip
export HISTGW_GATEWAY_APIKEY=histgw_<id>_<secret> # must carry historian:read + historian:write (+ tags:write) scopes
export HISTGW_TEST_TAG=<existing-tag> # read round-trip
export HISTGW_WRITE_SANDBOX_TAG=<writable-float-tag> # e.g. HistGW.LiveTest.Sandbox — write round-trip (EnsureTags + write)
export HISTGW_BOOL_SANDBOX_TAG=<writable-bool-tag> # e.g. HistGW.LiveTest.BoolSandbox — Boolean→Int2 EnsureTags leg + retype probe (issue #441)
export HISTGW_ALARM_SOURCE=<source-name> # alarm SendEvent → ReadEvents round-trip
dotnet test --filter "Category=LiveIntegration"
```
The live suite **skips cleanly** when these env vars are absent (safe to run offline on macOS). It is the
gate the operator runs on the VPN before trusting the cutover.
### KNOWN LIMITATION 2 — continuous-historization value-capture wired; live end-to-end verification still pending
The `ContinuousHistorizationRecorder` is fully wired (actor + FasterLog outbox + gateway value-writer +
meters). It is spawned with an **empty *initial* ref set** (`Array.Empty<string>()` in
`WithOtOpcUaRuntimeActors`) because the deployed address space — and thus the historized-ref set — is built
later at deploy time, not at actor-spawn time. **The ref-feed gap is now closed:** on every deploy the
`AddressSpaceApplier.FeedHistorizedRefs` hook computes the added/removed historized-ref delta and posts it
through `ActorHistorizedTagSubscriptionSink` → the recorder's `UpdateHistorizedRefs` message, and the
recorder keeps the full set and re-registers its dependency-mux interest. So the recorder converges to the
deployed set of historized value tags from the first deploy onward (the feed is a non-blocking, deploy-safe
`Tell`; a faulting feed can never break a deploy). **Reads and alarm-writes work today, and the value-capture
path is code-complete.** What remains is the **live gate**: an end-to-end verification against a real gateway
(deploy → dependency-mux value delta → outbox append → `WriteLiveValues`) plus a restart-convergence test
proving the recorder re-registers the deployed set after a process restart. Until that live verification runs,
treat continuous value-capture as unproven-in-production rather than absent.