docs(historian-gateway): document gateway backend, config keys, EnsureTags hook, known gates; retire Wonderware from docs
v2-ci / build (pull_request) Failing after 38s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (pull_request) Has been skipped
v2-ci / build (pull_request) Failing after 38s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (pull_request) Has been skipped
HistorianGateway is now the sole historian backend (read + alarm SendEvent + continuous WriteLiveValues). Document the final state and retire the Wonderware sidecar from the docs/config/labels: - CLAUDE.md: rewrite the Historian section — ServerHistorian / ContinuousHistorization / AlarmHistorian config keys, the IHistorianProvisioning EnsureTags hook, the GatewayAlarmHistorianWriter SendEvent path + ReadEvents dependency on gateway RuntimeDb:EventReadsEnabled=true, gateway-side prerequisites (RuntimeDb flags + historian:read/write/tags:write scopes), migration note, and two KNOWN-LIMITATION callouts (live-validation gate + empty historized-ref-set recorder follow-on). - appsettings.json: fix the stale ServerHistorian block (Host/Port/SharedSecret/ ServerCertThumbprint -> Endpoint/ApiKey/UseTls/AllowUntrustedServerCertificate/ CaCertificatePath/CallTimeout, keep MaxTieClusterOverfetch); add a disabled ContinuousHistorization block; prune the orphaned Wonderware keys from AlarmHistorian (keep the SQLite knobs). ApiKey env-supplied via ServerHistorian__ApiKey (commented; valid strict JSON via _comment keys). - README.md + docs (Historian.md, AlarmHistorian.md, Configuration.md, ServiceHosting.md, DriverLifecycle.md, drivers/README.md, Uns.md, VirtualTags.md, AlarmTracking.md, Client.UI.md, README.md, TestConnectProbes.md): retire the Wonderware historian backend from current-backend descriptions; fix the stale ServerHistorian/AlarmHistorian config tables (now gateway shape); convert drivers/Historian.Wonderware.md to a retired stub pointing at the gateway. - Source/UI labels (descriptive text only, no behavior change): OtOpcUaServerHostedService.cs, HistoryPaging.cs, OtOpcUaSdkServer.cs, HistorianAdapterActor.cs, VirtualTagModal.razor, ScriptedAlarmModal.razor, AlarmsHistorian.razor now name the HistorianGateway backend. Build clean (0 errors); AdminUI.Tests green (514 passed). Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
This commit is contained in:
@@ -112,7 +112,7 @@ lmxopcua-fix sync modbus # rsync this repo's tests/.../Docker/
|
||||
|
||||
Override any endpoint via the env var to point at a real PLC. The local OtOpcUa server runs on this VM at `opc.tcp://localhost:4840` — **that's not on the docker host**.
|
||||
|
||||
**Local docker-dev rig — login is DISABLED, so do live `/run` verification yourself (don't wait for the user to sign in).** The local `docker-dev/docker-compose.yml` stack (AdminUI at `http://localhost:9200` via Traefik; OPC UA `opc.tcp://localhost:4840` central-1 / `:4841` central-2) runs the AdminUI with `Security__Auth__DisableLogin: "true"` — **no sign-in form; it's auto-authenticated as a full-access admin.** So AdminUI / Razor `/run` verification (deploy a config, drive a page, confirm behavior — e.g. via the Chrome browser-automation tools against `http://localhost:9200`) does **not** require the user to log in. Run it yourself; do not defer it as "user-driven sign-in required." (Caveat: OPC UA *data-plane* auth is still real LDAP against the shared GLAuth on `10.100.0.35:3893` — that only gates Client.CLI read/write **role** operations, e.g. binding a `multi-role` / `opc-writeop` user, and is independent of the AdminUI login. Things genuinely outside the local rig — real PLCs, or the AVEVA Historian + Wonderware sidecar on `10.100.0.48` — still need the user.)
|
||||
**Local docker-dev rig — login is DISABLED, so do live `/run` verification yourself (don't wait for the user to sign in).** The local `docker-dev/docker-compose.yml` stack (AdminUI at `http://localhost:9200` via Traefik; OPC UA `opc.tcp://localhost:4840` central-1 / `:4841` central-2) runs the AdminUI with `Security__Auth__DisableLogin: "true"` — **no sign-in form; it's auto-authenticated as a full-access admin.** So AdminUI / Razor `/run` verification (deploy a config, drive a page, confirm behavior — e.g. via the Chrome browser-automation tools against `http://localhost:9200`) does **not** require the user to log in. Run it yourself; do not defer it as "user-driven sign-in required." (Caveat: OPC UA *data-plane* auth is still real LDAP against the shared GLAuth on `10.100.0.35:3893` — that only gates Client.CLI read/write **role** operations, e.g. binding a `multi-role` / `opc-writeop` user, and is independent of the AdminUI login. Things genuinely outside the local rig — real PLCs, or the AVEVA Historian reached via the `ZB.MOM.WW.HistorianGateway` sidecar — still need the user.)
|
||||
|
||||
See `docs/v2/dev-environment.md` for the full inventory and rationale.
|
||||
|
||||
@@ -165,7 +165,7 @@ Address pickers in AdminUI support live browse for OpcUaClient and Galaxy driver
|
||||
|
||||
The AdminUI's global **UNS** page (`/uns`) is the single surface for managing the unified namespace fleet-wide (Area → Line → Equipment → Tag/VirtualTag), replacing the old per-cluster UNS/Equipment/Tags tabs. See `docs/Uns.md`.
|
||||
|
||||
The `/uns` **TagModal** uses **driver-typed tag-config editors**: it dispatches by the bound driver's `DriverType` to a per-driver editor (Modbus/S7/AbCip/AbLegacy/TwinCAT/Focas) via `TagConfigEditorMap`, with client-side validation via `TagConfigValidator`; unmapped drivers (OpcUaClient/Galaxy/Historian.Wonderware) fall back to the generic raw-`TagConfig`-JSON textarea. Each editor is a thin razor shell over a pure `<Driver>TagConfigModel` (`FromJson`/`ToJson`/`Validate`, preserves unknown keys). To add a driver's editor, copy the Modbus template under `Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`, reusing the driver's enums + camelCase JSON property names, and register it in `TagConfigEditorMap` + `TagConfigValidator`. See `docs/plans/2026-06-09-driver-typed-tag-editors-design.md`.
|
||||
The `/uns` **TagModal** uses **driver-typed tag-config editors**: it dispatches by the bound driver's `DriverType` to a per-driver editor (Modbus/S7/AbCip/AbLegacy/TwinCAT/Focas) via `TagConfigEditorMap`, with client-side validation via `TagConfigValidator`; unmapped drivers (OpcUaClient/Galaxy) fall back to the generic raw-`TagConfig`-JSON textarea. Each editor is a thin razor shell over a pure `<Driver>TagConfigModel` (`FromJson`/`ToJson`/`Validate`, preserves unknown keys). To add a driver's editor, copy the Modbus template under `Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`, reusing the driver's enums + camelCase JSON property names, and register it in `TagConfigEditorMap` + `TagConfigValidator`. See `docs/plans/2026-06-09-driver-typed-tag-editors-design.md`.
|
||||
|
||||
## Scripting / Script Editor
|
||||
|
||||
@@ -186,4 +186,131 @@ Inbound operator acknowledge/shelve for scripted alarms is fully implemented. Tw
|
||||
|
||||
## Historian / HistoryRead
|
||||
|
||||
Server-side OPC UA HistoryRead for historized equipment tags is implemented driver-agnostically in Phase C. A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blob (authored in the raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default historian tagname, which is the tag's driver `FullName`. The server dispatches all history reads to the registered `IHistorianDataSource` (Wonderware historian TCP client) via the `ServerHistorian` appsettings section (`Enabled` defaults to `false`; when disabled, a `NullHistorianDataSource` is used and historized nodes return `GoodNoData` rather than an error). Supported variants: Raw, Processed (Average/Minimum/Maximum/Total/Count aggregates), and AtTime over historized variable nodes; Events over alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node); authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization. See `docs/Historian.md` for the full guide.
|
||||
**Backend: HistorianGateway (sole historian backend).** As of the gateway-integration cutover, the
|
||||
historian read, alarm-write, and continuous-historization paths are all served by the
|
||||
**`ZB.MOM.WW.HistorianGateway`** sidecar, consumed as the Gitea-feed
|
||||
**`ZB.MOM.WW.HistorianGateway.Client`** gRPC package (`historian_gateway.v1`) behind a thin
|
||||
`IHistorianGatewayClient` seam in `ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway`. **The bespoke
|
||||
Wonderware TCP/ArchestrA sidecar projects and the vestigial `Historian.Wonderware` driver type were
|
||||
retired** — there is no Wonderware backend in the tree anymore (see `docs/drivers/Historian.Wonderware.md`,
|
||||
now a retired stub).
|
||||
|
||||
A tag is historized by adding `"isHistorized": true` to its `TagConfig` JSON blob (authored in the
|
||||
raw-JSON textarea on the `/uns` TagModal); an optional `"historianTagname"` field overrides the default
|
||||
historian tagname, which is the tag's driver `FullName`.
|
||||
|
||||
### Read path (`ServerHistorian` section)
|
||||
|
||||
The server dispatches all OPC UA HistoryRead to the registered `IHistorianDataSource` — the
|
||||
`GatewayHistorianDataSource` read client when enabled, else the `NullHistorianDataSource` default
|
||||
(historized nodes return `GoodNoData`, never an error). Supported variants: Raw, Processed
|
||||
(Average/Minimum/Maximum/Total/Count aggregates), and AtTime over historized variable nodes; Events over
|
||||
alarm-owning equipment-folder event-notifier nodes. Reads are ungated (served from any redundancy node);
|
||||
authorization uses the standard `AccessLevels.HistoryRead` bit set at materialization.
|
||||
|
||||
`ServerHistorian` appsettings keys (`ServerHistorianOptions`; `Enabled` defaults to `false`):
|
||||
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `Enabled` | `false` | `true` registers the gateway read client; `false` keeps `NullHistorianDataSource` |
|
||||
| `Endpoint` | `""` | Absolute gateway URI, e.g. `https://host:5222`. Scheme selects transport (`https://` = TLS, `http://` = h2c) |
|
||||
| `ApiKey` | `""` | Peppered-HMAC key `histgw_<id>_<secret>` sent as `Authorization: Bearer`. **Supply via env `ServerHistorian__ApiKey` — never commit** |
|
||||
| `UseTls` | `true` | Connect over TLS; must match the `Endpoint` scheme |
|
||||
| `AllowUntrustedServerCertificate` | `false` | Accept a self-signed / untrusted server cert (dev / on-prem only) |
|
||||
| `CaCertificatePath` | `null` | PEM CA file pinning the gateway TLS chain; null/empty uses the OS trust store |
|
||||
| `CallTimeout` | `00:00:30` | Per-call deadline for each unary gateway read |
|
||||
| `MaxTieClusterOverfetch` | `65536` | Bounded over-fetch the HistoryRead-Raw paging uses to page within an oversized same-timestamp tie cluster (retained from the prior backend) |
|
||||
|
||||
### Alarm-history path (`AlarmHistorian` section)
|
||||
|
||||
Alarm events are written through `GatewayAlarmHistorianWriter` (the gateway **`SendEvent`** path) behind
|
||||
the durable **`SqliteStoreAndForwardSink`** — `AlarmHistorian:Enabled=true` swaps the `NullAlarmHistorianSink`
|
||||
default for the SQLite store-and-forward queue, whose drain worker forwards batches to the gateway and uses
|
||||
per-event outcomes to decide retry vs. dead-letter (never throws). The `AlarmHistorian` section carries
|
||||
only the `Enabled` gate + the SQLite knobs (`DatabasePath`, `DrainIntervalSeconds`, `Capacity`,
|
||||
`DeadLetterRetentionDays`, `BatchSize`, `MaxAttempts`) — the downstream gateway connection
|
||||
(endpoint/key/TLS) is sourced from the `ServerHistorian` section. **Alarm-history `ReadEvents` requires the
|
||||
target gateway deployed with `RuntimeDb:EventReadsEnabled=true`** (the C2 SQL event-read workaround).
|
||||
|
||||
### Continuous historization (`ContinuousHistorization` section)
|
||||
|
||||
When `ContinuousHistorization:Enabled=true` **and** `ServerHistorian` is configured, the Host builds a
|
||||
durable, crash-safe **FasterLog** outbox (`FasterLogHistorizationOutbox`) + a gateway-backed
|
||||
`IHistorianValueWriter`, and `WithOtOpcUaRuntimeActors` spawns the `ContinuousHistorizationRecorder`. The
|
||||
recorder taps the per-node dependency-mux value fan-out, appends each numeric value to the outbox (the
|
||||
crash boundary), and drains the outbox to the gateway's SQL live-value write path (**`WriteLiveValues`**)
|
||||
with exponential backoff. The gateway connection is sourced from `ServerHistorian`; this section carries
|
||||
only the recorder + outbox knobs:
|
||||
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `Enabled` | `false` | `true` (with `ServerHistorian` configured) wires + spawns the recorder |
|
||||
| `OutboxPath` | `""` (required when enabled) | **Directory** holding the FasterLog segment + commit files. In production set an **absolute** path on durable storage |
|
||||
| `CommitMode` | `PerEntry` | `PerEntry` = fsync before each append returns (no loss window); `Periodic` = batched commits every `CommitIntervalMs` |
|
||||
| `CommitIntervalMs` | `100` | Periodic-mode commit cadence; required `> 0` only under `Periodic` |
|
||||
| `DrainBatchSize` | `64` | Entries peeked + written per drain pass |
|
||||
| `DrainIntervalSeconds` | `2` | Steady drain cadence (and post-success reschedule) |
|
||||
| `Capacity` | `0` | Max un-acked outbox entries before drop-oldest; `0` = unbounded |
|
||||
| `MinBackoffSeconds` | `1` | Initial retry backoff after a failed drain pass |
|
||||
| `MaxBackoffSeconds` | `30` | Cap on the exponential retry backoff |
|
||||
|
||||
### Tag auto-provisioning (`IHistorianProvisioning` EnsureTags hook)
|
||||
|
||||
`AddressSpaceApplier.Apply()` fires a **non-blocking, fire-and-forget** `IHistorianProvisioning.EnsureTagsAsync`
|
||||
hook for added historized value tags — the gateway-backed `GatewayTagProvisioner` calls the gateway's
|
||||
`EnsureTags` so a brand-new historized tag exists in the historian before the recorder's `WriteLiveValues`
|
||||
land. The hook is wrapped so a faulted/synchronously-throwing provisioner can **never** block or fail a
|
||||
deploy. Non-numeric (`String`/`DateTime`/`Reference`) data types are skipped (not provisioned); the
|
||||
recorder likewise drops + meters non-numeric values. Continuous historization is **numeric-analog only** in
|
||||
v1 (`UInt16→UInt4` is a documented fallback).
|
||||
|
||||
### Gateway-side prerequisites
|
||||
|
||||
The target HistorianGateway OtOpcUa points at **must** run with:
|
||||
|
||||
- `RuntimeDb:Enabled=true` — enables the `WriteLiveValues` SQL live path (continuous historization).
|
||||
- `RuntimeDb:EventReadsEnabled=true` — enables `ReadEvents` from `Runtime.dbo.Events` (alarm-history reads).
|
||||
- An API key carrying scopes **`historian:read`**, **`historian:write`**, **`historian:tags:write`**.
|
||||
|
||||
### Migration note (deployments upgrading from the Wonderware backend)
|
||||
|
||||
The `ServerHistorian` section changed shape. Rename the old Wonderware keys and supply the key via env:
|
||||
|
||||
| Old (Wonderware) key | New (gateway) key |
|
||||
|---|---|
|
||||
| `ServerHistorian:Host` + `:Port` | `ServerHistorian:Endpoint` (`https://host:5222`) |
|
||||
| `ServerHistorian:SharedSecret` | `ServerHistorian:ApiKey` (**env `ServerHistorian__ApiKey`**) |
|
||||
| `ServerHistorian:ServerCertThumbprint` | `ServerHistorian:CaCertificatePath` (+ `UseTls` / `AllowUntrustedServerCertificate`) |
|
||||
|
||||
The `AlarmHistorian` section's old Wonderware connection keys (`Host`/`Port`/`UseTls`/`ServerCertThumbprint`/`SharedSecret`)
|
||||
were pruned — remove them; the SQLite knobs are retained and the downstream connection now comes from
|
||||
`ServerHistorian`. See `docs/Historian.md` for the full guide.
|
||||
|
||||
### KNOWN LIMITATION 1 — live-validation gate (do before merging/trusting the cutover)
|
||||
|
||||
The cutover is code-complete but **must be live-validated against a real gateway** (VPN to
|
||||
`wonder-sql-vd03`, gateway running the prerequisites above) before it is merged or trusted. Run the
|
||||
env-gated suite:
|
||||
|
||||
```bash
|
||||
export HISTGW_GATEWAY_ENDPOINT=https://wonder-sql-vd03:5222 # absolute gateway URI; absent ⇒ all live tests skip
|
||||
export HISTGW_GATEWAY_APIKEY=histgw_<id>_<secret> # must carry historian:read + historian:write (+ tags:write) scopes
|
||||
export HISTGW_TEST_TAG=<existing-tag> # read round-trip
|
||||
export HISTGW_WRITE_SANDBOX_TAG=<writable-float-tag> # e.g. HistGW.LiveTest.Sandbox — write round-trip (EnsureTags + write)
|
||||
export HISTGW_ALARM_SOURCE=<source-name> # alarm SendEvent → ReadEvents round-trip
|
||||
dotnet test --filter "Category=LiveIntegration"
|
||||
```
|
||||
|
||||
The live suite **skips cleanly** when these env vars are absent (safe to run offline on macOS). It is the
|
||||
gate the operator runs on the VPN before trusting the cutover.
|
||||
|
||||
### KNOWN LIMITATION 2 — continuous-historization value-capture is not yet live
|
||||
|
||||
The `ContinuousHistorizationRecorder` is fully wired (actor + FasterLog outbox + gateway value-writer +
|
||||
meters) but is currently spawned with an **EMPTY historized-ref set** (`Array.Empty<string>()` in
|
||||
`WithOtOpcUaRuntimeActors`): the deployed address space — and thus the set of historized tag refs — is built
|
||||
later at deploy time, not at actor-spawn time, so there is no clean ref set to resolve at wiring time. With
|
||||
an empty set the recorder **registers interest in nothing and historizes nothing**. **Reads and alarm-writes
|
||||
work today**; the recorder's value-capture is the remaining gap, blocked on a `SetHistorizedRefs`-style feed
|
||||
driven off the deployed composition (a tracked follow-on). Until that feed lands, continuous historization
|
||||
records no values.
|
||||
|
||||
Reference in New Issue
Block a user