21e0fdd4cd
Audit of docs/ against src/ surfaced shipped features without current-reference coverage (FOCAS CLI, Core.Scripting+VirtualTags, Core.ScriptedAlarms, Core.AlarmHistorian), an out-of-date driver count + capability matrix, ADR-002's virtual-tag dispatch not reflected in data-path docs, broken cross-references, and OpcUaServerReqs declaring OPC-020..022 that were never scoped. This commit closes all of those so operators + integrators can stay inside docs/ without falling back to v2/implementation/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
130 lines
12 KiB
Markdown
130 lines
12 KiB
Markdown
# Historical Data Access
|
|
|
|
OPC UA HistoryRead is a **per-driver optional capability** in OtOpcUa. The Core dispatches HistoryRead service calls to the owning driver through the `IHistoryProvider` capability interface (`src/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IHistoryProvider.cs`). Drivers that don't implement the interface return `BadHistoryOperationUnsupported` for every history call on their nodes; that is the expected behavior for protocol drivers (Modbus, S7, AB CIP, AB Legacy, TwinCAT, FOCAS) whose wire protocols carry no time-series data.
|
|
|
|
Historian integration is no longer a separate bolt-on assembly, as it was in v1 (`ZB.MOM.WW.LmxOpcUa.Historian.Aveva` plugin). It is now one optional capability any driver can implement. The first implementation is the Galaxy driver's Wonderware Historian integration; OPC UA Client forwards HistoryRead to the upstream server. Every other driver leaves the capability unimplemented and the Core short-circuits history calls on nodes that belong to those drivers.
|
|
|
|
## `IHistoryProvider`
|
|
|
|
Four methods, mapping onto the four OPC UA HistoryRead service variants:
|
|
|
|
| Method | OPC UA service | Notes |
|
|
|--------|----------------|-------|
|
|
| `ReadRawAsync` | HistoryReadRawModified (raw subset) | Returns `HistoryReadResult { Samples, ContinuationPoint? }`. The Core handles `ContinuationPoint` pagination. |
|
|
| `ReadProcessedAsync` | HistoryReadProcessed | Takes a `HistoryAggregateType` (Average / Minimum / Maximum / Total / Count) and a bucket `interval`. Drivers that can't express an aggregate throw `NotSupportedException`; the Core translates that into `BadAggregateNotSupported`. |
|
|
| `ReadAtTimeAsync` | HistoryReadAtTime | Default implementation throws `NotSupportedException` — drivers without interpolation / prior-boundary support leave the default. |
|
|
| `ReadEventsAsync` | HistoryReadEvents | Historical alarm/event rows, distinct from the live `IAlarmSource` stream. Default throws; only drivers with an event historian (Galaxy's A&E log) override. |
|
|
|
|
Supporting DTOs live alongside the interface in `Core.Abstractions`:
|
|
|
|
- `HistoryReadResult(IReadOnlyList<DataValueSnapshot> Samples, byte[]? ContinuationPoint)`
|
|
- `HistoryAggregateType` — enum `{ Average, Minimum, Maximum, Total, Count }`
|
|
- `HistoricalEvent(EventId, SourceName?, EventTimeUtc, ReceivedTimeUtc, Message?, Severity)`
|
|
- `HistoricalEventsResult(IReadOnlyList<HistoricalEvent> Events, byte[]? ContinuationPoint)`
|
|
|
|
## Alarm event history vs. `IHistoryProvider`
|
|
|
|
`IHistoryProvider.ReadEventsAsync` is the **pull** path: an OPC UA client calls `HistoryReadEvents` against a notifier node and the driver walks its own backend event store to satisfy the request. The Galaxy driver's implementation reads from AVEVA Historian's event schema via `aahClientManaged`; every other driver leaves the default `NotSupportedException` in place.
|
|
|
|
There is also a separate **push** path for persisting alarm transitions from any `IAlarmSource` (and the Phase 7 scripted-alarm engine) into a durable event log, independent of any client HistoryRead call. That path is covered by `IAlarmHistorianSink` + `SqliteStoreAndForwardSink` in `src/ZB.MOM.WW.OtOpcUa.Core.AlarmHistorian/` and is documented in [AlarmTracking.md#alarm-historian-sink](AlarmTracking.md#alarm-historian-sink). The two paths are complementary — the sink populates an external historian's alarm schema; `ReadEventsAsync` reads from whatever event store the driver owns — and share neither interface nor dispatch.
|
|
|
|
## Dispatch through `CapabilityInvoker`
|
|
|
|
All four HistoryRead surfaces are wrapped by `CapabilityInvoker` (`Core/Resilience/CapabilityInvoker.cs`) with `DriverCapability.HistoryRead`. The Polly pipeline keyed on `(DriverInstanceId, HostName, DriverCapability.HistoryRead)` provides timeout, circuit-breaker, and bulkhead defaults per the driver's stability tier (see [docs/v2/driver-stability.md](v2/driver-stability.md)).
|
|
|
|
The dispatch point is `DriverNodeManager` in `ZB.MOM.WW.OtOpcUa.Server`. When the OPC UA stack calls `HistoryRead`, the node manager:
|
|
|
|
1. Resolves the target `NodeHandle` to a `(DriverInstanceId, fullReference)` pair.
|
|
2. Checks the owning driver's `DriverTypeMetadata` to see if the type may advertise history at all (fast reject for types that never implement `IHistoryProvider`).
|
|
3. If the driver instance implements `IHistoryProvider`, wraps the `ReadRawAsync` / `ReadProcessedAsync` / `ReadAtTimeAsync` / `ReadEventsAsync` call in `CapabilityInvoker.InvokeAsync(... DriverCapability.HistoryRead ...)`.
|
|
4. Translates the `HistoryReadResult` into an OPC UA `HistoryData` + `ExtensionObject`.
|
|
5. Manages the continuation point via `HistoryContinuationPointManager` so clients can page through large result sets.
|
|
|
|
Driver-level history code never sees the continuation-point protocol or the OPC UA stack types — those stay in the Core.
|
|
|
|
## Driver coverage
|
|
|
|
| Driver | Implements `IHistoryProvider`? | Source |
|
|
|--------|:------------------------------:|--------|
|
|
| Galaxy | Yes — raw, processed, at-time, events | `aahClientManaged` SDK (Wonderware Historian) on the Host side, forwarded through the Proxy's IPC |
|
|
| OPC UA Client | Yes — raw, processed, at-time, events (forwarded to upstream) | `Opc.Ua.Client.Session.HistoryRead` against the remote server |
|
|
| Modbus | No | Wire protocol has no time-series concept |
|
|
| Siemens S7 | No | S7comm has no time-series concept |
|
|
| AB CIP | No | CIP has no time-series concept |
|
|
| AB Legacy | No | PCCC has no time-series concept |
|
|
| TwinCAT | No | ADS symbol reads are point-in-time; archiving is an external concern |
|
|
| FOCAS | No | Default — FOCAS has no general-purpose historian API |
|
|
|
|
## Galaxy — Wonderware Historian (`aahClientManaged`)
|
|
|
|
The Galaxy driver's `IHistoryProvider` implementation lives on the Host side (`.NET 4.8 x86`) in `src/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host/Backend/Historian/`. The Proxy's `GalaxyProxyDriver.ReadRawAsync` / `ReadProcessedAsync` / `ReadAtTimeAsync` / `ReadEventsAsync` each serializes a `HistoryRead*Request` and awaits the matching `HistoryRead*Response` over the named pipe (see [drivers/Galaxy.md](drivers/Galaxy.md#ipc-transport)).
|
|
|
|
Host-side, `HistorianDataSource` uses the AVEVA Historian managed SDK (`aahClientManaged.dll`) to query historical data via a cursor-based API through `ArchestrA.HistorianAccess`:
|
|
|
|
- **`HistoryQuery`** — raw historical samples (timestamp, value, OPC quality)
|
|
- **`AnalogSummaryQuery`** — pre-computed aggregates (Average, Minimum, Maximum, ValueCount, First, Last, StdDev)
|
|
|
|
The SDK DLLs are pulled into the Galaxy.Host project at build time; the Server and every other driver project remain SDK-free.
|
|
|
|
> **Gap / status note.** The raw SDK wrapper (`HistorianDataSource`, `HistorianClusterEndpointPicker`, `HistorianHealthSnapshot`, etc.) has been ported from the v1 `ZB.MOM.WW.LmxOpcUa.Historian.Aveva` plugin into `Driver.Galaxy.Host/Backend/Historian/`. The **IPC wire-up** — `HistoryReadRequest` / `HistoryReadResponse` message kinds, Proxy-side `ReadRawAsync` / `ReadProcessedAsync` / `ReadAtTimeAsync` / `ReadEventsAsync` forwarding — is in place on `GalaxyProxyDriver`. What remains to close on a given branch is Host-side **mapping of `HistoryAggregateType` onto the `AnalogSummaryQuery` column names** (done in `GalaxyProxyDriver.MapAggregateToColumn`; the Host side must mirror it) and the **end-to-end integration test** that was held by the v1 plugin suite. Until those land on a given driver branch, history calls against Galaxy may surface `GalaxyIpcException { Code = "not-implemented" }` or backend-specific errors rather than populated `HistoryReadResult`s. Track the remaining work against the Phase 2 Galaxy out-of-process gate in `docs/v2/plan.md`.
|
|
|
|
### Aggregate function mapping
|
|
|
|
`GalaxyProxyDriver.MapAggregateToColumn` (Proxy-side) translates the OPC UA Part 13 standard aggregate enum onto `AnalogSummaryQuery` column names consumed by `HistorianDataSource.ReadAggregateAsync`:
|
|
|
|
| `HistoryAggregateType` | Result Property |
|
|
|------------------------|-----------------|
|
|
| `Average` | `Average` |
|
|
| `Minimum` | `Minimum` |
|
|
| `Maximum` | `Maximum` |
|
|
| `Count` | `ValueCount` |
|
|
|
|
`HistoryAggregateType.Total` is **not supported** by Wonderware `AnalogSummary` and raises `NotSupportedException`, which the Core translates to `BadAggregateNotSupported`. Additional OPC UA aggregates (`Start`, `End`, `StandardDeviationPopulation`) sit on the Historian columns `First`, `Last`, `StdDev` and can be exposed by extending the enum + mapping together.
|
|
|
|
### Read-only cluster failover
|
|
|
|
`HistorianConfiguration.ServerNames` accepts an ordered list of cluster nodes. `HistorianClusterEndpointPicker` iterates the list in configuration order, marks failed nodes with a `FailureCooldownSeconds` window, and re-admits them when the cooldown elapses. One picker instance is shared by the process-values connection and the event-history connection (two SDK silos), so a node failure on one silo immediately benches it for the other. `FailureCooldownSeconds = 0` disables the cooldown — the SDK's own retry semantics are the sole gate.
|
|
|
|
Host-side cluster health is surfaced via `HistorianHealthSnapshot { NodeCount, HealthyNodeCount, ActiveProcessNode, ActiveEventNode, Nodes }` and forwarded to the Proxy so the Admin UI Historian panel can render a per-node table. `HealthCheckService` flips overall service health to `Degraded` when `HealthyNodeCount < NodeCount`.
|
|
|
|
### Runtime health counters
|
|
|
|
`HistorianDataSource` maintains per-read counters — `TotalQueries`, `TotalSuccesses`, `TotalFailures`, `ConsecutiveFailures`, `LastSuccessTime`, `LastFailureTime`, `LastError`, `ProcessConnectionOpen`, `EventConnectionOpen` — so the dashboard can distinguish "backend loaded but never queried" from "backend loaded and queries are failing". `LastError` is prefixed with the read path (`raw:`, `aggregate:`, `at-time:`, `events:`) so operators can tell which silo is broken. `HealthCheckService` degrades at `ConsecutiveFailures >= 3`.
|
|
|
|
### Quality mapping
|
|
|
|
The Historian SDK returns standard OPC DA quality values in `QueryResult.OpcQuality` (UInt16). The low byte flows through the shared `QualityMapper` pipeline (`MapFromMxAccessQuality` → `MapToOpcUaStatusCode`):
|
|
|
|
| OPC Quality Byte | OPC DA Family | OPC UA StatusCode |
|
|
|------------------|---------------|-------------------|
|
|
| 0-63 | Bad | `Bad` (with sub-code when an exact enum match exists) |
|
|
| 64-191 | Uncertain | `Uncertain` (with sub-code when an exact enum match exists) |
|
|
| 192+ | Good | `Good` (with sub-code when an exact enum match exists) |
|
|
|
|
See `Domain/QualityMapper.cs` and `Domain/Quality.cs` in `Driver.Galaxy.Host` for the full table.
|
|
|
|
## OPC UA Client — upstream forwarding
|
|
|
|
The OPC UA Client driver (`Driver.OpcUaClient`) implements `IHistoryProvider` by forwarding each call to the upstream server via `Session.HistoryRead`. Raw / processed / at-time / events map onto the stack's native HistoryRead details types. Continuation points are passed through — the Core's `HistoryContinuationPointManager` treats the driver as an opaque pager.
|
|
|
|
## Historizing flag and AccessLevel
|
|
|
|
During variable node creation, drivers that advertise history set:
|
|
|
|
```csharp
|
|
if (attr.IsHistorized)
|
|
accessLevel |= AccessLevels.HistoryRead;
|
|
variable.Historizing = attr.IsHistorized;
|
|
```
|
|
|
|
- **`Historizing = true`** — tells OPC UA clients that the node has historical data available.
|
|
- **`AccessLevels.HistoryRead`** — enables the `HistoryRead` access bit. The OPC UA stack checks this bit before routing history requests to the Core dispatcher; nodes without it are rejected before reaching `IHistoryProvider`.
|
|
|
|
The `IsHistorized` flag originates in the driver's discovery output. For Galaxy it comes from the repository query detecting a `HistoryExtension` primitive (see [drivers/Galaxy-Repository.md](drivers/Galaxy-Repository.md)). For OPC UA Client it is copied from the upstream server's `Historizing` property.
|
|
|
|
## Configuration
|
|
|
|
Driver-specific historian config lives in each driver's `DriverConfig` JSON blob, validated against the driver type's `DriverConfigJsonSchema` in `DriverTypeRegistry`. The Galaxy driver's historian section carries the fields exercised by `HistorianConfiguration` — `ServerName` / `ServerNames`, `FailureCooldownSeconds`, `IntegratedSecurity` / `UserName` / `Password`, `Port` (default `32568`), `CommandTimeoutSeconds`, `RequestTimeoutSeconds`, `MaxValuesPerRead`. The OPC UA Client driver inherits its timeouts from the upstream session.
|
|
|
|
See [Configuration.md](Configuration.md) for the schema shape and validation path.
|