Files
lmxopcua/docs/drivers
Joseph Doherty c74ccc55b5 feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):

  Template  → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
  DataSet   → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
  PropertySet → node metadata, never tags — it describes a metric rather than being one

Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.

**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.

**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).

**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.

The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.

Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.

Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.

Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 08:14:04 -04:00
..

Drivers

OtOpcUa is a multi-driver OPC UA server. The Core (ZB.MOM.WW.OtOpcUa.Core + Core.Abstractions + Server) owns the OPC UA stack, address space, session/security/subscription machinery, resilience pipeline, and the two v3 OPC UA namespaces (Raw + UNS — see V3NodeIds / AddressSpaceRealm). Drivers plug in through capability interfaces defined in src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/:

  • IDriver — lifecycle (InitializeAsync, ReinitializeAsync, ShutdownAsync, GetHealth)
  • IReadable / IWritable — one-shot reads and writes
  • ITagDiscovery — address-space enumeration
  • ISubscribable — driver-pushed data-change streams
  • IHostConnectivityProbe — per-host reachability events
  • IPerCallHostResolver — multi-host drivers that route each call to a target endpoint at dispatch time
  • IAlarmSource — driver-emitted OPC UA A&C events
  • IHistoryProvider — driver-side raw / processed / at-time / events HistoryRead (see HistoricalDataAccess.md)
  • IRediscoverable — driver-initiated address-space rebuild notifications
  • IHistorianDataSource — server-side historian read backend registration (the HistorianGateway backend), distinct from the driver-side IHistoryProvider HistoryRead path

Each driver opts into only the capabilities it supports. Every async capability call at the Server dispatch layer goes through CapabilityInvoker (Core/Resilience/CapabilityInvoker.cs), which wraps it in a Polly pipeline keyed on (DriverInstanceId, HostName, DriverCapability). The OTOPCUA0001 analyzer enforces the wrap at build time. Drivers themselves never depend on Polly; they just implement the capability interface and let the Core wrap it.

Driver factories are registered at startup in DriverFactoryRegistry (src/Core/ZB.MOM.WW.OtOpcUa.Core/Hosting/DriverFactoryRegistry.cs) — all 12 in one place, src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs:147-164. Type names are the DriverTypeNames constants (src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs), guarded bidirectionally by DriverTypeNamesGuardTests.

⚠️ DriverTypeRegistry / DriverTypeMetadata were DELETED (Gitea #522, 2026-07-30). They were vestigial — no production code constructed or read them, only their own unit tests did — and AllowedNamespaceKinds had already been orphaned by the retirement of the v3 Namespace entity. Older docs and plans that name them as the driver-metadata or tier authority describe something that no longer exists.

Stability tier never came from that registry anyway: it is the DriverTier tier = DriverTier.A parameter on DriverFactoryRegistry.cs:46, and no factory in src/Drivers/ passes a tier, so every shipped driver runs Tier A. The Tier-C recycle protections that docs/v2/driver-stability.md describes — MemoryTracking, MemoryRecycle, ScheduledRecycleScheduler, IDriverSupervisor — are gone, not merely dormant: they required an out-of-process driver Host that no longer exists anywhere (Galaxy is gRPC-to-mxaccessgw, FOCAS is in-process), IDriverSupervisor had no implementations, and the trackers were only ever constructed in tests. The Tier column below is therefore uniformly A, and the tier's remaining effect is DriverResilienceOptions.GetTierDefaults — the per-capability timeout / retry / breaker policy set, which is live.

Ground-truth driver list

Driver Project path Tier Wire / library Capabilities Notable quirk
Galaxy Driver.Galaxy (+ .Browser, .Contracts) A gRPC to the external mxaccessgw gateway (the gateway owns MXAccess COM + the Galaxy Repository SQL reader) IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IAlarmSource, IRediscoverable, IHostConnectivityProbe In-process .NET 10 driver — the COM bitness constraint lives in the gateway's x86 net48 worker, not here. PR 7.2 retired the legacy in-process Galaxy.{Shared, Host, Proxy} + named-pipe Windows service. Native MxAccess alarms work end-to-end
Modbus TCP / RTU-over-TCP Driver.Modbus (+ .Addressing, .Contracts) A NModbus-derived in-house client; Transport selects MBAP (TCP) or RTU CRC-16 framing over a socket IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver Polled subscriptions via the shared PollGroupEngine. DL205 PLCs are covered by AddressFormat=DL205 (octal V/X/Y/C/T/CT translation) — no separate driver. Transport=RtuOverTcp (ModbusTransportMode, default Tcp) targets serial→Ethernet gateways via ModbusRtuOverTcpTransport + ModbusRtuFraming + ModbusCrc, with ModbusTransportDesyncException carrying distinct CrcMismatch/UnitMismatch reasons; direct-serial transport is descoped. Merged 0f38f486 (#495)
Siemens S7 Driver.S7 A S7netplus IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe Single S7netplus Plc instance per PLC serialized with SemaphoreSlim — the S7 CPU's comm mailbox is scanned at most once per cycle, so parallel reads don't help
AB CIP Driver.AbCip A libplctag CIP IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IAlarmSource ControlLogix / CompactLogix. Tag discovery uses the @tags walker to enumerate controller-scoped + program-scoped symbols; UDT member resolution via the UDT template reader
AB Legacy Driver.AbLegacy A libplctag PCCC IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver SLC 500 / MicroLogix. File-based addressing (N7:0, F8:0) — no symbol table, tag list is user-authored in the config DB
TwinCAT Driver.TwinCAT B Beckhoff TwinCAT.Ads (TcAdsClient) IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IRediscoverable The only native-notification driver outside Galaxy — ADS delivers ValueChangedCallback events the driver forwards straight to ISubscribable.OnDataChange without polling. Symbol tree uploaded via SymbolLoaderFactory
FOCAS Driver.FOCAS A Pure-managed FocasWireClient — FOCAS/2 Ethernet binary protocol on TCP:8193, inlined into the driver assembly IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IAlarmSource IWritable is implemented but read-only by design — WriteAsync returns BadNotWritable for every point. CNC-shaped data model (axes, spindle, PMC, macros, alarms) not a flat tag map. Previously Tier-C (Host + P/Invoke + shim DLL); retired in the 2026-04-24 migration when the managed wire client landed
OPC UA Client Driver.OpcUaClient B OPCFoundation Opc.Ua.Client IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe Gateway/aggregation driver — the only driver implementing driver-side IHistoryProvider (forwards HistoryRead to the upstream server). Opens a single Session against a remote OPC UA server and re-exposes its address space. Owns its own ApplicationConfiguration (distinct from Client.Shared) because it's always-on with keep-alive + TransferSubscriptions across SDK reconnect, not an interactive CLI
MQTT Driver.Mqtt (+ .Browser, .Contracts) A MQTTnet 5 IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable Push, not poll, and read-only — the broker delivers PUBLISH frames the driver forwards straight to ISubscribable.OnDataChange; IReadable serves the last retained/received value from cache, and there is no IWritable (nothing publishes back). Subscriptions are indexed by topic filter, not by topic, so wildcard tags survive a reconnect. A rejected CONNACK throws MqttConnectRejectedException: unrecoverable codes → Faulted + supervisor stop, transient → retry. Two modes: Plain binds a tag to a concrete topic + JSON path; SparkplugB decodes vendored Eclipse-Tahu protobuf under one spBv1.0/{groupId}/# subscription and binds by the group/edgeNode/device?/metric tuple (birth/alias/seq-gap state machine, death→STALE, scoped rebirth NCMD). IRediscoverable fires on a DBIRTH but is inert platform-side — see #507
MTConnect Driver.MTConnect (+ .Contracts) — Hand-rolled System.Xml.Linq HTTP/XML client against a vendor-neutral MTConnect Agent (/probe, /current, /sample) — no TrakHound dependency (dropped; see the driver doc) IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable Read-only by design (no IWritable). Production data plane is entirely ISubscribable/OnDataChange — IReadable is CLI/test-only. Browse is free via the Wave-0 universal discovery browser, no bespoke browser project. IRediscoverable/IHostConnectivityProbe have no server-side consumer yet — a pre-existing fleet-wide gap, not MTConnect-specific
Sql Driver.Sql (+ .Browser, .Contracts) A Microsoft.Data.SqlClient behind an ISqlDialect seam (SqlServerDialect shipped; Postgres/ODBC deferred) IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe Read-only DB-poll driver with a bespoke schema browser. Credentials are never persisted — the deploy gate rejects a stored connectionString (#498) and catalog identifiers are validated against the live catalog's own spelling (#496). SupportsOnlineDiscovery => false by design (the bespoke browser wins). SqlTagModel.Query is deferred to P3 and rejected end-to-end by parser, validator, planner and editor. Merged 4ad54037
Calculation Driver.Calculation A none — computes from other tags' RawPaths IDriver, IDependencyConsumer, ISubscribable, IReadable Pseudo-driver, not a wire protocol: signal-level tags computed from other tags via a script, with deploy-time scriptId-existence + Tarjan cycle gates. Its probe is parse-only (no backend to reach). runTimeoutMs is authorable through CalculationDriverForm (the "no typed config form" gap — G-1 — was closed 2026-07-27)
Historian.Gateway Driver.Historian.Gateway — ZB.MOM.WW.HistorianGateway.Client gRPC (historian_gateway.v1) IHistorianDataSource (server-side read backend) + alarm SendEvent writer + WriteLiveValues recorder + IHistorianProvisioning Not a tag driver — the sole historian backend. Registers GatewayHistorianDataSource : IHistorianDataSource for HistoryRead and serves alarm-write + continuous historization through the gateway. No IDriver/ITagDiscovery surface. (The retired Wonderware sidecar backend it replaced is documented at Historian.Wonderware.md.)

Per-driver documentation

  • Galaxy has its own docs in this folder because the gRPC-to-gateway architecture + MXAccess rules (owned by the gateway) + Galaxy Repository SQL + Historian + runtime probe manager don't fit a single table row:

    • Galaxy.md — gateway gRPC bridge, hierarchy source, runtime probes
    • Galaxy-Repository.md — ZB SQL reader, LocalPlatform scope filter, change detection (v1 archive)
  • FOCAS has a short getting-started doc because the backend-selection env var + alarm projection opt-in need explaining up front:

    • FOCAS.md — deployment, config, capability surface, alarm projection, troubleshooting
  • Modbus TCP, AB CIP, AB Legacy, Siemens S7, TwinCAT, OPC UA Client, and MQTT each have a per-driver overview page:

    • Modbus.md — in-process Modbus-TCP driver: address formats, polled subscription model, DL205 octal mapping
    • AbCip.md — AB CIP / EtherNet-IP driver (ControlLogix / CompactLogix / Micro800 / GuardLogix): tag discovery, UDT resolution, alarm source
    • AbLegacy.md — AB Legacy PCCC driver (SLC 500 / MicroLogix / PLC-5): file-based addressing, user-authored tag list
    • S7.md — Siemens S7 driver (S7-300/400/1200/1500 + S7-200): getting started, config, data-block addressing, serialized single-connection model
    • TwinCAT.md — Beckhoff TwinCAT (ADS) driver: getting started, native-notification subscription, symbol-tree upload
    • OpcUaClient.md — OPC UA Client (gateway/aggregation) driver: remote-server session, driver-side HistoryRead forwarding, reconnect behaviour
    • Mqtt.md — MQTT broker-subscribe driver: broker/TLS/auth config, topic + JSON-path tag binding, retained-value seeding, #-observation browser, CONNACK-rejection semantics, Sparkplug B (birth/alias/seq/rebirth state machine, birth-driven browse picker, death→STALE), and the Known gaps table
  • MTConnect has a short getting-started doc because the hand-rolled-vs-TrakHound divergence, the data-plane precedence rules, and a non-trivial known-limitations list need explaining up front:

    • MTConnect.md — Agent config, FullName==DataItem id, UNAVAILABLE→BadNoCommunication, CONDITION-as-String, browse-via-universal-browser, fixture recipe, deferred write-back
  • Historian.Gateway (server-side historian backend, not a tag driver) is documented in the main guide:

    • ../Historian.md — HistorianGateway backend: read-path registration, HistoryRead dispatch, alarm store-and-forward (SendEvent), continuous historization (WriteLiveValues), EnsureTags provisioning, config keys, deployment prerequisites. (The retired Wonderware sidecar backend it replaced: Historian.Wonderware.md.)
  • The full per-field spec (capability surface, config schema, addressing, data-type maps, connection settings, quirks for every driver) lives in docs/v2/driver-specs.md. The overview pages above are the short path; that file is the authoritative per-driver reference.

Test-fixture coverage maps

Each driver has a dedicated fixture doc that lays out what the integration / unit harness actually covers vs. what's trusted from field deployments. Read the relevant one before claiming "green suite = production-ready" for a driver.

  • AB CIP — Dockerized ab_server (multi-stage build from libplctag source); atomic-read smoke across 4 families; UDT / ALMD / family quirks unit-only
  • Modbus — Dockerized pymodbus + per-family JSON profiles (4 compose profiles); best-covered driver, gaps are error-path-shaped
  • Siemens S7 — Dockerized python-snap7 server; DB/MB read + write round-trip verified end-to-end on :1102
  • AB Legacy — Dockerized ab_server PCCC mode across SLC500 / MicroLogix / PLC-5 profiles (task #224); N/F/L-file round-trip verified end-to-end. /1,0 cip-path required for the Docker fixture; real hardware uses empty. Residual gap: bit-file writes (B3:0/5) still surface BadState — real HW / RSEmulate 500 for those
  • TwinCAT — XAR-VM integration scaffolding (task #221); three smoke tests skip when VM unreachable. Unit via FakeTwinCATClient with native-notification harness
  • FOCAS — no integration fixture, unit-only via FakeFocasClient; Tier C out-of-process isolation scoped but not shipped
  • OPC UA Client — Dockerized opc-plc integration suite (task #215): real Secure Channel + Session, read + subscribe verified end-to-end; write not yet exercised in the integration suite; exhaustive capability matrix (reconnect, failover, cert-auth, history, alarms) via unit suite with mocked Session
  • MQTT — Dockerized eclipse-mosquitto with TLS + real auth on both listeners (no anonymous fallback) + a JSON-publisher sidecar; env-gated live suite (15 tests) — 7 plain (TLS/auth connect, CA pinning both ways, subscribe→value, retained seeding, wrong-password rejection) + 8 Sparkplug against a project-owned C# edge-node simulator on the sparkplug compose profile (birth→alias bind, DDATA, NDEATH→STALE, rebirth NCMD round-trip). Births are never retained, so docker restart otopcua-sparkplug-sim is how you force a fresh one
  • Galaxy — richest harness: gateway E2E + ZB SQL live-smoke + MXAccess opt-in (v1 archive)
  • MTConnect — Dockerized mtconnect/agent + stdlib SHDR adapter (two services, the Agent alone reports everything UNAVAILABLE); 12/12 against the real Agent, canned XML unit fixtures cover the bulk (491/491)
  • HistoricalDataAccess.md — IHistoryProvider dispatch, aggregate mapping, continuation points. The OPC UA Client driver is the only driver that implements driver-side IHistoryProvider (it forwards HistoryRead to the upstream server); the AVEVA Historian path is served server-side by the HistorianGateway-backed IHistorianDataSource instead. Other drivers do not implement the interface and return BadHistoryOperationUnsupported.
  • AlarmTracking.md — IAlarmSource event model and filtering. Implemented by Galaxy (native MxAccess alarms, working end-to-end), OPC UA Client, AB CIP, and FOCAS; AB Legacy, Modbus, S7, TwinCAT, and MQTT have no alarm source.
  • Subscriptions.md — how the Server multiplexes subscriptions onto ISubscribable.OnDataChange.
  • docs/v2/driver-stability.md — tier system (A / B / C), shared CapabilityPolicy defaults per tier × capability, MemoryTracking hybrid formula, and process-level recycle rules.
  • docs/v2/plan.md — authoritative vision, architecture decisions, migration strategy.