Joseph Doherty
c51c59987b
Reverses the conservative call from the previous commit at the user's direction, and they were right: "surface it on a counter first" left the wire lying. An MXAccess event can carry Good quality while its payload is a kind MxValueDecoder has no mapping for, so the old behaviour published null-under-Good — which a client cannot tell apart from a genuine null reading. A value the gateway DID send would be consumed as a value it did not, with the only evidence in a log nobody is tailing. Bad data is the honest wire representation of lost data. BadTypeMismatch is an existing StatusCodeMap constant, deliberately reused rather than written inline: StatusCodeParityTests finds status constants by reflecting over Good/Uncertain/Bad-prefixed uint fields and checking them against the pinned SDK, so an inline literal would be invisible to that guard (Gitea #497, where 16 wrong constants had accumulated across 6 drivers exactly this way). Logging is now once per DISTINCT value kind rather than once per pump. The set is bounded by the MxValue oneof enum so a broken producer still cannot flood, but a second unrepresentable kind appearing later gets its own line — warn-once-per-pump would have silently swallowed it, which is the same class of bug as the null this commit fixes. The test's fixture matters more than its assertion: the undecodable event carries Quality = 192 (OPC DA Good), so a decoder that gave up silently would publish Good and fail. Paired with a positive control — a decodable value at the same quality must still publish Good — so it cannot be satisfied by a pump that marks everything Bad. Falsified by reverting the status to Good: the test reds, and only that test. Verified: 331 Galaxy tests green, Core.Abstractions 298 (the parity guard), solution builds 0 errors. Docs corrected in the two places that recorded the old decision as deliberate.
116 lines
14 KiB
Markdown
116 lines
14 KiB
Markdown
# Galaxy Driver
|
|
|
|
The Galaxy driver bridges OtOpcUa to AVEVA System Platform (Wonderware) Galaxies. It is a **Tier-A in-process driver** that runs in the OtOpcUa server's .NET 10 AnyCPU process and speaks gRPC to a separately installed `mxaccessgw` server (sibling repo at `c:\Users\dohertj2\Desktop\mxaccessgw\`). The gateway owns the MXAccess COM apartment, the STA + Win32 message pump, the Galaxy Repository SQL reader, and the Historian SDK — all the bits that need x86 / .NET Framework 4.8 / COM interop. The driver itself is platform-agnostic and contains no COM, no STA thread, and no x86 bitness constraint.
|
|
|
|
For the driver spec (capability surface, config shape, addressing), see [docs/v2/driver-specs.md §1](../v2/driver-specs.md). For the gateway setup recipe, see [docs/v2/Galaxy.ParityRig.md](../v2/Galaxy.ParityRig.md). For tracing, metrics, and soak profile, see [docs/v2/Galaxy.Performance.md](../v2/Galaxy.Performance.md).
|
|
|
|
> **Note**: the related docs [`Galaxy-Repository.md`](../v1/drivers/Galaxy-Repository.md) and [`Galaxy-Test-Fixture.md`](../v1/drivers/Galaxy-Test-Fixture.md) describe the previous v1 / out-of-process topology and now live under `docs/v1/drivers/`. For current testing use [`Galaxy.ParityRig.md`](../v2/Galaxy.ParityRig.md) and the `mxaccessgw` repo.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
+---------------------------------------+
|
|
| OtOpcUa.Server (.NET 10 AnyCPU) |
|
|
| GalaxyDriver (in-process) |
|
|
| ITagDiscovery / IReadable / |
|
|
| IWritable / ISubscribable / |
|
|
| IRediscoverable / |
|
|
| IHostConnectivityProbe / |
|
|
| IAlarmSource |
|
|
+-------------------+-------------------+
|
|
|
|
|
gRPC (default http://localhost:5120)
|
|
|
|
|
v
|
|
+---------------------------------------+
|
|
| mxaccessgw (sibling repo) |
|
|
| +-------------------------------+ |
|
|
| | MxGateway.Worker (x86 net48) | |
|
|
| | STA + WM_APP pump | |
|
|
| | ArchestrA.MxAccess COM | |
|
|
| | Galaxy Repository SQL | |
|
|
| | Wonderware Historian SDK | |
|
|
| +-------------------------------+ |
|
|
+---------------------------------------+
|
|
```
|
|
|
|
History reads moved server-side in PR 7.2 (`IHistoryRouter`). Galaxy no longer implements `IHistoryProvider` of its own.
|
|
|
|
`IAlarmSource` was retired with PR 7.2 and **restored in PR B.2** of the
|
|
alarms-over-gateway epic ([docs/plans/alarms-over-gateway.md](../plans/alarms-over-gateway.md)).
|
|
Alarm transitions arrive on the same gateway `StreamEvents` channel as
|
|
data-change events under the new `MX_EVENT_FAMILY_ON_ALARM_TRANSITION`
|
|
family; acknowledgements route through the gateway's
|
|
`AcknowledgeAlarm` RPC. The previous value-driven sub-attribute path
|
|
remains as a fallback for Galaxy templates without `$Alarm*`
|
|
extensions — the server-side `AlarmConditionService` dedups when both
|
|
paths fire on the same condition. See [docs/AlarmTracking.md](../AlarmTracking.md)
|
|
for the v2-final architecture.
|
|
|
|
## Project Layout
|
|
|
|
The driver ships as a single project: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/` (.NET 10, AnyCPU). Sub-folders:
|
|
|
|
| Folder | Role |
|
|
|--------|------|
|
|
| `Browse/` | Static-side discovery: `GalaxyDiscoverer` walks the gateway's hierarchy + attribute-set RPCs, `DataTypeMap` and `SecurityMap` translate Galaxy types and security classifications into OPC UA equivalents, `AlarmRefBuilder` extracts alarm-bearing attribute references for the server-layer alarm engine. `IGalaxyHierarchySource` + `GatewayGalaxyHierarchySource` + `TracedGalaxyHierarchySource` decorate the gateway browse RPC; `IGalaxyDeployWatchSource` + `GatewayGalaxyDeployWatchSource` + `DeployWatcher` drive `IRediscoverable`. |
|
|
| `Runtime/` | Live data path: `EventPump` runs the gateway's `StreamEvents` RPC and fans out to subscribers via a bounded channel; `GalaxyMxSession` is the read-side handle; `GatewayGalaxySubscriber` + `GatewayGalaxyDataWriter` (each with a `Traced*` decorator) implement `ISubscribable` / `IWritable`; `SubscriptionRegistry` tracks subscription state for replay; `ReconnectSupervisor` owns the backoff loop and triggers `ReplaySubscriptions` on session loss; `StatusCodeMap` translates gateway StatusCodes to OPC UA; `MxValueDecoder` / `MxValueEncoder` handle scalar + array marshalling; `GalaxyTelemetry` + `GalaxySubscriptionHandle` round out the surface. |
|
|
| `Health/` | `HostStatusAggregator` rolls per-platform probe state into the driver's `IHostConnectivityProbe` view; `PerPlatformProbeWatcher` listens on the gateway's per-host status stream; `HostConnectivityForwarder` pushes transitions out to the server's connectivity bus. |
|
|
| `Config/` | `GalaxyDriverOptions` and the four nested option records (`GalaxyGatewayOptions`, `GalaxyMxAccessOptions`, `GalaxyRepositoryOptions`, `GalaxyReconnectOptions`). |
|
|
|
|
Project root files:
|
|
|
|
- `GalaxyDriver.cs` — `IDriver` + capability-interface implementation; composes the Browse / Runtime / Health collaborators.
|
|
- `GalaxyDriverFactoryExtensions.cs` — DI registration helper used by the server's driver bootstrap.
|
|
|
|
## Capability Surface
|
|
|
|
`GalaxyDriver : IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, IDisposable, IAsyncDisposable`.
|
|
|
|
| Capability | Implementation entry point |
|
|
|------------|---------------------------|
|
|
| `ITagDiscovery` | `Browse/GalaxyDiscoverer.cs` |
|
|
| `IRediscoverable` | `Browse/DeployWatcher.cs` — consumed as an **operator prompt**: a Galaxy redeploy raises a "re-browse" chip on `/hosts`. ⚠️ It does **not** rebuild the address space — re-browse the device under `/raw` and commit ([#518](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/518)) |
|
|
| `IReadable` | `Runtime/GalaxyMxSession.cs` |
|
|
| `IWritable` | `Runtime/GatewayGalaxyDataWriter.cs` |
|
|
| `ISubscribable` | `Runtime/GatewayGalaxySubscriber.cs` (driven by `EventPump`) |
|
|
| `IHostConnectivityProbe` | `Health/HostStatusAggregator.cs` |
|
|
| `IAlarmSource` | `Runtime/GatewayGalaxyAlarmFeed.cs` (transitions) + `Runtime/GatewayGalaxyAlarmAcknowledger.cs` (acks) |
|
|
|
|
## Configuration
|
|
|
|
`DriverConfig` JSON binds to `Config/GalaxyDriverOptions.cs`. The four sections are:
|
|
|
|
- **`Gateway`** — endpoint, API key secret ref, TLS knobs, connect/call/stream timeouts. `StreamTimeoutSeconds = 0` keeps the long-lived `StreamEvents` RPC open for the driver's lifetime.
|
|
- **`MxAccess`** — `ClientName` (must be unique per OtOpcUa instance — redundancy pairs enforce uniqueness at install time), `PublishingIntervalMs` (forwarded as `buffered_update_interval_ms` on subscribe), `WriteUserId` for ArchestrA secured-write, `EventPumpChannelCapacity` (default 50_000 — one second of headroom at 50k tags / 1Hz; tune via the `galaxy.events.dropped` metric).
|
|
- **`Repository`** — `DiscoverPageSize`, `WatchDeployEvents`.
|
|
- **`Reconnect`** — `InitialBackoffMs`, `MaxBackoffMs`, `ReplayOnSessionLost` (calls the gateway's `ReplaySubscriptions` RPC after reconnect rather than re-issuing subscribe-bulk for every tag).
|
|
|
|
Full per-field descriptions live in `Config/GalaxyDriverOptions.cs`. The full JSON skeleton is reproduced in [docs/v2/driver-specs.md §1](../v2/driver-specs.md).
|
|
|
|
## Reconnect + Replay
|
|
|
|
`ReconnectSupervisor` owns an exponential-backoff loop bounded by `Reconnect.InitialBackoffMs` / `MaxBackoffMs`. On session loss it calls `GalaxyDriver.ReopenAsync`, which invokes `GalaxyMxSession.RecreateAsync` to dispose the stale/faulted session and client before rebuilding (`OpenSessionAsync` + `RegisterAsync`). Previously `ConnectAsync` was a no-op when a stale session handle was still present, so the reopen supervisor looped forever without recovering. After a successful reopen — when `ReplayOnSessionLost = true` — the supervisor calls the gateway's `ReplaySubscriptions` RPC with the cached subscription set from `SubscriptionRegistry` instead of re-subscribing tag-by-tag. The gateway's worker then re-issues `AdviseSupervisory` server-side under the apartment lock.
|
|
|
|
The session-less alarm feed (`GatewayGalaxyAlarmFeed`) and alarm acknowledger (`GatewayGalaxyAlarmAcknowledger`) run on a separate always-on `_ownedMxClient` that is intentionally **not** recreated when `ReconnectSupervisor` rebuilds the worker session. The feed has its own re-invoke loop (~5 s backoff) that reopens `StreamAlarms` after any stream fault, and gRPC.NET's channel auto-reconnect recovers the underlying HTTP/2 connection after a gateway restart — so both the alarm stream and individual ack calls recover without client replacement. The acknowledger is completely stateless between calls: each `AcknowledgeAsync` issues a single unary RPC and returns; no dead-client state is latched on failure. Channel-level keepalive hardening (TCP keep-alive intervals, gRPC ping frames) would require exposing additional knobs on `MxGatewayClientOptions` in the sibling `mxaccessgw` repo — a future option if flaky long-lived connections are observed in production.
|
|
|
|
## Testing
|
|
|
|
- **Unit tests**: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/` — fakes the gateway gRPC surface; covers Browse, Runtime, Health, and Config in isolation.
|
|
- **Parity rig + dev-rig walkthrough**: see [docs/v2/Galaxy.ParityRig.md](../v2/Galaxy.ParityRig.md). The rig stands up a real `mxaccessgw` against a live Galaxy and exercises the full read / write / subscribe / rediscover path.
|
|
- **Performance + soak**: see [docs/v2/Galaxy.Performance.md](../v2/Galaxy.Performance.md).
|
|
|
|
## Browse Tree Shape
|
|
|
|
`GalaxyDiscoverer` builds the OPC UA address space as a **true nested tree** by `parent_gobject_id`: each gobject folder is parented under its Galaxy parent folder rather than being emitted flat under the driver root. Parentage falls back gracefully — a gobject with `parent_gobject_id` of 0, a parent not present in the returned set, a self-referencing parent, or a cycle attaches directly to the driver root. This matches the Galaxy `ArchestrA` object hierarchy as seen in the IDE.
|
|
|
|
## Operational Notes
|
|
|
|
- **MXAccess `ClientName` collisions**: two OtOpcUa instances sharing a `ClientName` cause the older Wonderware session to lose subscription state. Redundancy pairs (decision #149) enforce uniqueness via install scripts.
|
|
- **Channel saturation**: `galaxy.events.dropped > 0` indicates `EventPump` is back-pressured. Raise `EventPumpChannelCapacity` or investigate downstream slowness in the server-side fan-out.
|
|
- **Connectivity surface**: per-platform probe state is exposed through `IHostConnectivityProbe` and aggregated by the server's connectivity bus — there is no driver-private dashboard surface anymore. The Admin UI's Host Status panel is the consumer.
|
|
- **A stream the gateway ends cleanly is treated as a fault.** A closed or evicted gateway session completes the `StreamEvents` RPC *without an error*, so nothing throws. `EventPump` reports that to the `ReconnectSupervisor` exactly as it reports a transport fault. Before this it exited silently: the supervisor was never told, `GetHealth()` kept reporting `Healthy`, and because the [comms-loss watchdog](../DriverLifecycle.md) requires a **non-Healthy** state to trip, it never fired either — the driver went permanently deaf with no signal anywhere. Nothing else in the system can observe this, so do not remove the notification on the grounds that "a clean end isn't an error".
|
|
- **Replay gaps** (`galaxy.events.replay_gaps`): the gateway signals that events were evicted from its replay ring before we could read them. The driver responds by re-issuing `SubscribeBulk` for every tracked subscription — MXAccess delivers an initial value per item on subscribe, so re-subscribing *is* the re-snapshot the contract asks for. It does **not** restart the event pump (the stream carrying the sentinel is healthy) and it honours `Reconnect:ReplayOnSessionLost=false` by logging and standing down, on the grounds that an operator who delegated recovery to the gateway did not ask for a re-subscribe on a different trigger.
|
|
⚠️ This cannot currently fire: the sentinel is only emitted on a stream opened with `after_worker_sequence > 0`, and the driver opens with `0`, which the gateway treats as "fresh stream, not a resume". Enabling resume is a bigger change than it looks — the sequence is only meaningful **within one gateway session**, and the reconnect path recreates the session, so a remembered sequence replayed against a fresh session would set the server's live watermark *above* every event it is about to send, silencing the stream rather than recovering it.
|
|
- **Undecodable values** (`galaxy.events.undecodable`): a value kind this driver has no CLR representation for is published as **`BadTypeMismatch` with a null value**, and the first occurrence of each distinct kind is logged. It is *not* published as a null carrying the event's own quality: an MXAccess event can be Good-quality while the payload is one we cannot map, and a good null is indistinguishable from a genuine null reading — a client would consume a value it never received as a real one, with the failure visible only in a log. Non-zero here means the gateway is sending a wire type `MxValueDecoder` has no mapping for and those readings are being lost; the fix is a decoder mapping, not a quality change.
|