lmxopcua/docs/drivers/Galaxy.md

Galaxy Driver

The Galaxy driver bridges OtOpcUa to AVEVA System Platform (Wonderware) Galaxies. It is a Tier-A in-process driver that runs in the OtOpcUa server's .NET 10 AnyCPU process and speaks gRPC to a separately installed mxaccessgw server (sibling repo at c:\Users\dohertj2\Desktop\mxaccessgw\). The gateway owns the MXAccess COM apartment, the STA + Win32 message pump, the Galaxy Repository SQL reader, and the Historian SDK — all the bits that need x86 / .NET Framework 4.8 / COM interop. The driver itself is platform-agnostic and contains no COM, no STA thread, and no x86 bitness constraint.

For the driver spec (capability surface, config shape, addressing), see docs/v2/driver-specs.md §1. For the gateway setup recipe, see docs/v2/Galaxy.ParityRig.md. For tracing, metrics, and soak profile, see docs/v2/Galaxy.Performance.md.

Note

: the related docs Galaxy-Repository.md and Galaxy-Test-Fixture.md describe the previous v1 / out-of-process topology and now live under docs/v1/drivers/. For current testing use Galaxy.ParityRig.md and the mxaccessgw repo.

Architecture

                    +---------------------------------------+
                    |  OtOpcUa.Server (.NET 10 AnyCPU)      |
                    |   GalaxyDriver (in-process)           |
                    |    ITagDiscovery / IReadable /        |
                    |    IWritable / ISubscribable /        |
                    |    IRediscoverable /                  |
                    |    IHostConnectivityProbe /           |
                    |    IAlarmSource                       |
                    +-------------------+-------------------+
                                        |
                                       gRPC (default http://localhost:5120)
                                        |
                                        v
                    +---------------------------------------+
                    |  mxaccessgw  (sibling repo)           |
                    |    +-------------------------------+  |
                    |    | MxGateway.Worker (x86 net48)  |  |
                    |    |   STA + WM_APP pump           |  |
                    |    |   ArchestrA.MxAccess COM      |  |
                    |    |   Galaxy Repository SQL       |  |
                    |    |   Wonderware Historian SDK    |  |
                    |    +-------------------------------+  |
                    +---------------------------------------+

History reads are served server-side by IHistorianDataSource (HistorianGateway); Galaxy has no history capability of its own.

IAlarmSource was retired with PR 7.2 and restored in PR B.2 of the alarms-over-gateway epic (docs/plans/alarms-over-gateway.md). Alarm transitions arrive on the same gateway StreamEvents channel as data-change events under the new MX_EVENT_FAMILY_ON_ALARM_TRANSITION family; acknowledgements route through the gateway's AcknowledgeAlarm RPC. The previous value-driven sub-attribute path remains as a fallback for Galaxy templates without $Alarm* extensions — the server-side AlarmConditionService dedups when both paths fire on the same condition. See docs/AlarmTracking.md for the v2-final architecture.

Project Layout

The driver ships as a single project: src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy/ (.NET 10, AnyCPU). Sub-folders:

Folder	Role
Browse/	Static-side discovery: GalaxyDiscoverer walks the gateway's hierarchy + attribute-set RPCs, DataTypeMap and SecurityMap translate Galaxy types and security classifications into OPC UA equivalents, AlarmRefBuilder extracts alarm-bearing attribute references for the server-layer alarm engine. IGalaxyHierarchySource + GatewayGalaxyHierarchySource + TracedGalaxyHierarchySource decorate the gateway browse RPC; IGalaxyDeployWatchSource + GatewayGalaxyDeployWatchSource + DeployWatcher drive IRediscoverable.
Runtime/	Live data path: EventPump runs the gateway's StreamEvents RPC and fans out to subscribers via a bounded channel; GalaxyMxSession is the read-side handle; GatewayGalaxySubscriber + GatewayGalaxyDataWriter (each with a Traced* decorator) implement ISubscribable / IWritable; SubscriptionRegistry tracks subscription state for replay; ReconnectSupervisor owns the backoff loop and triggers ReplaySubscriptions on session loss; StatusCodeMap translates gateway StatusCodes to OPC UA; MxValueDecoder / MxValueEncoder handle scalar + array marshalling; GalaxyTelemetry + GalaxySubscriptionHandle round out the surface.
Health/	HostStatusAggregator rolls per-platform probe state into the driver's IHostConnectivityProbe view; PerPlatformProbeWatcher listens on the gateway's per-host status stream; HostConnectivityForwarder pushes transitions out to the server's connectivity bus.
Config/	GalaxyDriverOptions and the four nested option records (GalaxyGatewayOptions, GalaxyMxAccessOptions, GalaxyRepositoryOptions, GalaxyReconnectOptions).

Project root files:

• GalaxyDriver.cs — IDriver + capability-interface implementation; composes the Browse / Runtime / Health collaborators.
• GalaxyDriverFactoryExtensions.cs — DI registration helper used by the server's driver bootstrap.

Capability Surface

GalaxyDriver : IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IRediscoverable, IHostConnectivityProbe, IAlarmSource, IDisposable, IAsyncDisposable.

Capability	Implementation entry point
ITagDiscovery	Browse/GalaxyDiscoverer.cs
IRediscoverable	Browse/DeployWatcher.cs — consumed as an operator prompt: a Galaxy redeploy raises a "re-browse" chip on /hosts. ⚠️ It does not rebuild the address space — re-browse the device under /raw and commit (#518)
IReadable	Runtime/GalaxyMxSession.cs
IWritable	Runtime/GatewayGalaxyDataWriter.cs
ISubscribable	Runtime/GatewayGalaxySubscriber.cs (driven by EventPump)
IHostConnectivityProbe	Health/HostStatusAggregator.cs
IAlarmSource	Runtime/GatewayGalaxyAlarmFeed.cs (transitions) + Runtime/GatewayGalaxyAlarmAcknowledger.cs (acks)

Configuration

DriverConfig JSON binds to Config/GalaxyDriverOptions.cs. The four sections are:

• Gateway — endpoint, API key secret ref, TLS knobs, connect/call/stream timeouts. StreamTimeoutSeconds = 0 keeps the long-lived StreamEvents RPC open for the driver's lifetime.
• MxAccess — ClientName (must be unique per OtOpcUa instance — redundancy pairs enforce uniqueness at install time), PublishingIntervalMs (forwarded as buffered_update_interval_ms on subscribe), WriteUserId for ArchestrA secured-write, EventPumpChannelCapacity (default 50_000 — one second of headroom at 50k tags / 1Hz; tune via the galaxy.events.dropped metric).
• Repository — DiscoverPageSize, WatchDeployEvents.
• Reconnect — InitialBackoffMs, MaxBackoffMs, ReplayOnSessionLost (calls the gateway's ReplaySubscriptions RPC after reconnect rather than re-issuing subscribe-bulk for every tag).
• AlarmAck (optional, lmxopcua#749) — FallbackUser, FallbackDomain, FallbackPasswordSecretRef. See Alarm acknowledge credential.

Alarm acknowledge credential

mxaccessgw 488649c+ (client/contracts 0.3.0) acknowledges a native alarm by running AuthenticateUser as the operator and writing the comment to <alarm>.AckMsg, so the Historian ACK_ALM row records a real Galaxy account. AcknowledgeAlarmRequest therefore carries operator_user + operator_domain + operator_password, and the gateway rejects an empty operator_user. The OPC UA data plane is machine-to-machine (API key or anonymous) and carries no Galaxy password, so GalaxyAckCredentialResolver picks the account per acknowledge:

1. The API key's own Galaxy ack identity — set on AdminUI /apikeys ("Ack identity"). Stored only in the encrypted Secrets store as otopcua/apikeys/<keyId>/galaxy-ack-user (DOMAIN\user or user) and otopcua/apikeys/<keyId>/galaxy-ack-password (GalaxyAckIdentitySecrets) — never in the config DB or the deployment artifact. A key with a user but no password secret is refused, not silently downgraded to the fallback.
2. The driver's fallback account — AlarmAck.FallbackUser / FallbackDomain / FallbackPasswordSecretRef in DriverConfig, used by keys without an identity and by anonymous sessions. The password ref must be secret:NAME / ${secret:NAME} (production), env:NAME, file:PATH or dev:VALUE; a bare literal is refused so a password never sits in DriverConfig.
3. Neither → the driver throws DriverOperationRefusedException; nothing is sent to the gateway. The refusal is excluded from the resilience breaker (it says nothing about gateway health). ⚠️ The native ack path is fire-and-forget after the Part 9 method returns, so the OPC UA call itself still returns Good and the refusal surfaces as a DriverInstance warning naming the fix — a synchronous status / revert is #760.

The comment written to AckMsg is suffixed with the OPC UA identity — "<comment> [OPC UA: <principal>, key <keyId>]" or "[OPC UA: anonymous]" — so the client is not lost when the Historian's Operator is a shared fallback account. The password is never logged: the acknowledger scrubs it from every gateway message it echoes and GalaxyAckCredential.ToString() omits it.

⚠️ Secrets are node-local (optionally Akka-replicated within one mesh). The per-key secrets and the fallback secret must exist on the node that serves the OPC UA session and hosts the Galaxy driver; on a split admin/driver topology create them on the driver nodes too (/admin/secrets or the secret CLI) under exactly those names.

Full per-field descriptions live in Config/GalaxyDriverOptions.cs. The full JSON skeleton is reproduced in docs/v2/driver-specs.md §1.

Reconnect + Replay

ReconnectSupervisor owns an exponential-backoff loop bounded by Reconnect.InitialBackoffMs / MaxBackoffMs. On session loss it calls GalaxyDriver.ReopenAsync, which invokes GalaxyMxSession.RecreateAsync to dispose the stale/faulted session and client before rebuilding (OpenSessionAsync + RegisterAsync). Previously ConnectAsync was a no-op when a stale session handle was still present, so the reopen supervisor looped forever without recovering. After a successful reopen — when ReplayOnSessionLost = true — the supervisor calls the gateway's ReplaySubscriptions RPC with the cached subscription set from SubscriptionRegistry instead of re-subscribing tag-by-tag. The gateway's worker then re-issues AdviseSupervisory server-side under the apartment lock.

The session-less alarm feed (GatewayGalaxyAlarmFeed) and alarm acknowledger (GatewayGalaxyAlarmAcknowledger) run on a separate always-on _ownedMxClient that is intentionally not recreated when ReconnectSupervisor rebuilds the worker session. The feed has its own re-invoke loop (~5 s backoff) that reopens StreamAlarms after any stream fault, and gRPC.NET's channel auto-reconnect recovers the underlying HTTP/2 connection after a gateway restart — so both the alarm stream and individual ack calls recover without client replacement. The acknowledger is completely stateless between calls: each AcknowledgeAsync issues a single unary RPC and returns; no dead-client state is latched on failure. Channel-level keepalive hardening (TCP keep-alive intervals, gRPC ping frames) would require exposing additional knobs on MxGatewayClientOptions in the sibling mxaccessgw repo — a future option if flaky long-lived connections are observed in production.

Every (re)open of StreamAlarms replays the gateway's active-alarm snapshot. GatewayGalaxyAlarmFeed maps each entry to a transition flagged IsRestatement (#656): Active → Retrigger (upstream's unack is authoritative), ActiveAcked → ActiveAcknowledged (one atomic active+acked projection — the old Acknowledge mapping projected an acked alarm inactive on a cold host), Inactive → Clear. The host writes no /alerts row for a replayed entry that changes nothing the condition shows, so a gateway reconnect no longer duplicates history; live Transition messages are never flagged. The gateway folds UnackRtn and AckRtn into Inactive, so an inactive-unacked alarm cannot be distinguished at reconnect (gateway contract limit).

Testing

• Unit tests: tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Tests/ — fakes the gateway gRPC surface; covers Browse, Runtime, Health, and Config in isolation.
• Parity rig + dev-rig walkthrough: see docs/v2/Galaxy.ParityRig.md. The rig stands up a real mxaccessgw against a live Galaxy and exercises the full read / write / subscribe / rediscover path.
• Performance + soak: see docs/v2/Galaxy.Performance.md.

Browse Tree Shape

GalaxyDiscoverer builds the OPC UA address space as a true nested tree by parent_gobject_id: each gobject folder is parented under its Galaxy parent folder rather than being emitted flat under the driver root. Parentage falls back gracefully — a gobject with parent_gobject_id of 0, a parent not present in the returned set, a self-referencing parent, or a cycle attaches directly to the driver root. This matches the Galaxy ArchestrA object hierarchy as seen in the IDE.

Operational Notes

• MXAccess ClientName collisions: two OtOpcUa instances sharing a ClientName cause the older Wonderware session to lose subscription state. Redundancy pairs (decision #149) enforce uniqueness via install scripts.
• Channel saturation: galaxy.events.dropped > 0 indicates EventPump is back-pressured. Raise EventPumpChannelCapacity or investigate downstream slowness in the server-side fan-out.
• Connectivity surface: per-platform probe state is exposed through IHostConnectivityProbe and aggregated by the server's connectivity bus — there is no driver-private dashboard surface anymore. The Admin UI's Host Status panel is the consumer.
• A stream the gateway ends cleanly is treated as a fault. A closed or evicted gateway session completes the StreamEvents RPC without an error, so nothing throws. EventPump reports that to the ReconnectSupervisor exactly as it reports a transport fault. Before this it exited silently: the supervisor was never told, GetHealth() kept reporting Healthy, and because the comms-loss watchdog requires a non-Healthy state to trip, it never fired either — the driver went permanently deaf with no signal anywhere. Nothing else in the system can observe this, so do not remove the notification on the grounds that "a clean end isn't an error".
• Replay gaps (galaxy.events.replay_gaps): the gateway signals that events were evicted from its replay ring before we could read them. The driver responds by re-issuing SubscribeBulk for every tracked subscription — MXAccess delivers an initial value per item on subscribe, so re-subscribing is the re-snapshot the contract asks for. It does not restart the event pump (the stream carrying the sentinel is healthy) and it honours Reconnect:ReplayOnSessionLost=false by logging and standing down, on the grounds that an operator who delegated recovery to the gateway did not ask for a re-subscribe on a different trigger. ⚠️ This cannot currently fire: the sentinel is only emitted on a stream opened with after_worker_sequence > 0, and the driver opens with 0, which the gateway treats as "fresh stream, not a resume". Enabling resume is a bigger change than it looks — the sequence is only meaningful within one gateway session, and the reconnect path recreates the session, so a remembered sequence replayed against a fresh session would set the server's live watermark above every event it is about to send, silencing the stream rather than recovering it.
• Undecodable values (galaxy.events.undecodable): a value kind this driver has no CLR representation for is published as BadTypeMismatch with a null value, and the first occurrence of each distinct kind is logged. It is not published as a null carrying the event's own quality: an MXAccess event can be Good-quality while the payload is one we cannot map, and a good null is indistinguishable from a genuine null reading — a client would consume a value it never received as a real one, with the failure visible only in a log. Non-zero here means the gateway is sending a wire type MxValueDecoder has no mapping for and those readings are being lost; the fix is a decoder mapping, not a quality change.
• Duplicate attributeRef (#605 / 06-S-13): two authored RawPaths dialling the same Galaxy attribute are both dialled, but the reverse map is first-wins, so value fan-out and alarm ConditionId surface under the first RawPath only. The loser is logged at Warning and recorded in the skipped-tag ledger with the winner path in the reason.
• Subscribe-once reads are bounded (#605 / 06-S-3): ReadAsync on the production path (SubscribeBulk, first event, Unsubscribe) carries an internal 30 s deadline in addition to the caller token; slots that never publish come back BadTimeout, so CancellationToken.None can no longer hang on a silent tag.
• Supervisory advise is retried after a thrown failure (#605 / 06-S-12): the writer marks an item handle advised before the call; a thrown advise (not just a non-OK status) now unmarks it, fails the write closed and re-advises on the next write.
• Array writes (#610): a Galaxy array tag is client-writable only for the MxValueEncoder array types (OPC UA Boolean/Int32/Int64/Float/Double/String/DateTime) and only with a declared arrayLength — with none the node is unbounded, and a short whole-array write would resize the attribute. The element type must match the node or be a numeric type that converts to it exactly — a numeric array of another element type is converted, checked, to the node's type in the node manager (an Int32 list to an Int64 array arrives as long[]), and any unrepresentable element refuses the write BadTypeMismatch inline. An array written to a scalar Galaxy node (a browse-committed array attribute without isArray) is still dispatched, as before #610. Separately, MxValueEncoder itself refuses uint[]/ulong[]: CLR array covariance let them match the int[]/long[] arms and be written bit-reinterpreted. Encoder-level only — Galaxy array writes have no live gate yet. See Uns.md §Array writes, including the upgrade note.