From 8fc147d8d4f99ccca8e82eedcd28c6c71e2f5c42 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Wed, 15 Jul 2026 16:40:36 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20driver-expansion=20program=20=E2=80=94?= =?UTF-8?q?=208=20research=20reports=20+=207=20design=20docs,=20parallel-r?= =?UTF-8?q?eviewed?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the driver-expansion program design (umbrella: universal Discover-backed browser + MTConnect, MQTT/Sparkplug B, BACnet/IP, SQL poll, Omron, Modbus RTU; MELSEC deferred) plus the per-driver research reports. All docs went through a 7-agent parallel review against the codebase before this commit. Highlights fixed in review: - universal browser: FOCAS FixedTree fills post-connect -> UntilStable settle + FixedTree.Enabled patch; MQTT reconciled to bespoke (was contradicting the program doc's SupportsOnlineDiscovery=false verdict) - modbus-rtu: SerialPort.ReadTimeout doesn't bound async BaseStream reads -> linked-CTS per-op deadline (R2-01 class); BCL enum reuse would leak System.IO.Ports into Contracts - bacnet: DiscoveryRediscoverPolicy enum name; UDP 47808 contention; live suite rewritten around unicast Who-Is + BBMD (broadcast doesn't cross VMs) - sql-poll: real tier registration via DriverFactoryRegistry.Register; blackhole gate must not docker-pause the shared central SQL Server - mqtt: Sparkplug v3.0 STATE topic form; first-in-repo proto codegen noted - omron: host hardcodes isIdempotent:false today (retry seam unshipped); v1 scopes UDTs to dotted-leaf access - mtconnect: SecurityClassification.ViewOnly; factory ParseEnum pattern - program doc: both valid enum-serialization patterns; IRediscoverable is change-signal-gated; RTU P2 adds System.IO.Ports; label is host-side --- .../2026-07-15-bacnet-ip-driver-design.md | 556 ++++++++++++++ ...6-07-15-driver-expansion-program-design.md | 211 ++++++ .../2026-07-15-modbus-rtu-driver-design.md | 380 ++++++++++ ...2026-07-15-mqtt-sparkplug-driver-design.md | 412 +++++++++++ .../2026-07-15-mtconnect-driver-design.md | 182 +++++ docs/plans/2026-07-15-omron-driver-design.md | 524 ++++++++++++++ .../2026-07-15-sql-poll-driver-design.md | 685 ++++++++++++++++++ ...7-15-universal-discovery-browser-design.md | 296 ++++++++ .../00-existing-driver-browse-audit.md | 294 ++++++++ docs/research/drivers/README.md | 151 ++++ docs/research/drivers/bacnet-ip.md | 356 +++++++++ docs/research/drivers/melsec-slmp.md | 408 +++++++++++ docs/research/drivers/modbus-rtu.md | 422 +++++++++++ docs/research/drivers/mqtt-sparkplug.md | 292 ++++++++ docs/research/drivers/mtconnect-agent.md | 408 +++++++++++ docs/research/drivers/omron.md | 361 +++++++++ docs/research/drivers/sql-poll.md | 470 ++++++++++++ 17 files changed, 6408 insertions(+) create mode 100644 docs/plans/2026-07-15-bacnet-ip-driver-design.md create mode 100644 docs/plans/2026-07-15-driver-expansion-program-design.md create mode 100644 docs/plans/2026-07-15-modbus-rtu-driver-design.md create mode 100644 docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md create mode 100644 docs/plans/2026-07-15-mtconnect-driver-design.md create mode 100644 docs/plans/2026-07-15-omron-driver-design.md create mode 100644 docs/plans/2026-07-15-sql-poll-driver-design.md create mode 100644 docs/plans/2026-07-15-universal-discovery-browser-design.md create mode 100644 docs/research/drivers/00-existing-driver-browse-audit.md create mode 100644 docs/research/drivers/README.md create mode 100644 docs/research/drivers/bacnet-ip.md create mode 100644 docs/research/drivers/melsec-slmp.md create mode 100644 docs/research/drivers/modbus-rtu.md create mode 100644 docs/research/drivers/mqtt-sparkplug.md create mode 100644 docs/research/drivers/mtconnect-agent.md create mode 100644 docs/research/drivers/omron.md create mode 100644 docs/research/drivers/sql-poll.md diff --git a/docs/plans/2026-07-15-bacnet-ip-driver-design.md b/docs/plans/2026-07-15-bacnet-ip-driver-design.md new file mode 100644 index 00000000..59cd232d --- /dev/null +++ b/docs/plans/2026-07-15-bacnet-ip-driver-design.md @@ -0,0 +1,556 @@ +# BACnet/IP driver — executable implementation design + +**Status:** design, build-ready. 2026-07-15. +**Supersedes/derives from:** the research report `docs/research/drivers/bacnet-ip.md` (protocol +survey, library selection, browse verdict). This document turns that research into a concrete, +file-by-file build plan for `ZB.MOM.WW.OtOpcUa.Driver.Bacnet`. +**Reconciles with:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (BACnet browse is +delivered by the **universal discovery browser** in v1, not a bespoke `IBrowseSession`). + +--- + +## 1. Motivation + scope + +BACnet/IP (ASHRAE 135 / ISO 16484-5) is the dominant building-automation protocol — HVAC, lighting, +energy metering, access control. A BACnet driver lets the OtOpcUa server pull **facility / HVAC / +energy** points (space temperatures, setpoints, kW/kWh meters, occupancy, equipment status) into the +unified namespace alongside process data (Modbus/S7/Galaxy), so a plant's OT and facility signals live +under one OPC UA address space. + +BACnet is the closest analog to the **OpcUaClient** driver in this repo: it has native **device +discovery** (Who-Is/I-Am) and native **change-of-value push** (SubscribeCOV), so it is a first-class +*browseable* + *subscribe-capable* Equipment-kind driver, not a bare poller. + +**Scope (this driver is a standard Equipment-kind driver.)** BACnet points are ordinary equipment +`Tag`s bound to the BACnet driver via `TagConfig.FullName` (a JSON addressing blob — see §5), authored +through the standard `/uns` TagModal + address picker, exactly like Modbus / S7 / OpcUaClient. No alias +machinery, no bespoke namespace kind. + +**v1 scope: Connect + Read + Discover (+ universal browse) + BBMD/foreign-device. COV push is P2, Write +is P3.** See the phasing in §10. + +Research reference: `docs/research/drivers/bacnet-ip.md`. + +--- + +## 2. Project layout + +Two new projects, mirroring the Modbus / OpcUaClient split (transport project + a lean Contracts project +the AdminUI can reference without dragging in the UDP transport): + +| Project | Contents | References | +|---|---|---| +| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet`** | `BacnetDriver` (IDriver + capability surfaces), the async adapter over the ela-compil client (`BacnetClientAdapter`), `BacnetDriverProbe`, `BacnetDriverFactoryExtensions`, data-type map | `Core.Abstractions`, `Core` (the `Core.Hosting` namespace — `DriverFactoryRegistry` — lives inside the `Core` project), `.Bacnet.Contracts`, NuGet `BACnet` | +| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts`** | `BacnetDriverOptions` (+ nested `ForeignDeviceOptions`/`DiscoveryOptions`/`CovOptions`/`ProbeOptions`), the `BacnetObjectType` / `BacnetPropertyId` enums the AdminUI editor binds, `BacnetTagAddress` record, `BacnetEquipmentTagParser` | `Core.Abstractions` only (zero transport deps — same discipline as `Modbus.Contracts` / `Modbus.Addressing`) | + +**No `.Browser` project in v1.** BACnet browse comes *for free* from the universal discovery browser +(§4). A bespoke lazy `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser` is a documented **P-later** graduation for +large multi-device sites (§4.2), not v1 work. + +**Contracts csproj** — copy `ZB.MOM.WW.OtOpcUa.Driver.Modbus.Contracts.csproj` minus its +`Modbus.Addressing` reference (BACnet has no separate addressing project): `net10.0`, +`Nullable`/`ImplicitUsings` enable, `TreatWarningsAsErrors`, `ProjectReference` to `Core.Abstractions` +only (it owns the shared `TagConfigJson` field readers used by the parser). + +**NuGet.** Central package management: pin the version in `Directory.Packages.props` and reference +(version-less) from the transport project only: + +```xml + + + + +``` + +- **Library: `System.IO.BACnet`, ela-compil fork, NuGet package id `BACnet`, MIT license.** The engine + inside YABE; explicitly multi-targets **`net10.0`** (no TFM friction); covers everything we need — + Who-Is/I-Am, ReadProperty, ReadPropertyMultiple, WriteProperty, SubscribeCOV/SubscribeCOVProperty + + COV notifications, and BACnet/IP with BBMD + foreign-device registration. +- **The async wrinkle (§4).** The library API is **event/callback-driven and blocking-with-callbacks**, + not `async`/`Task`-first: `BacnetClient.WhoIs()` fires `OnIam` events; `ReadPropertyRequest` has a + blocking overload + a begin/end pair. The driver **must** wrap it behind a thin async adapter + (`TaskCompletionSource` + timeout + cancellation) exactly as `OpcUaClientDriver` wraps the OPC UA SDK. + Budget a dedicated `BacnetClientAdapter` layer for this. +- **One `BacnetClient` per driver instance**, not per device — a single shared + `BacnetIpUdpProtocolTransport` UDP socket multiplexes every device on the network (mirrors "single + `Session` per OpcUaClient driver"). **Port-sharing caveat:** receiving *broadcast* traffic requires + binding UDP 47808, and only one exclusive binder per host interface gets it — so **two BACnet + `DriverInstance`s deployed to the same Host node contend for the port** (shared-socket binds make + datagram delivery nondeterministic). Constrain to one BACnet driver instance per node/interface, or + give additional instances distinct `port` values (fine for sims; real devices always speak 47808). + Unicast flows (directed Who-Is, RP/RPM, COV notifications — devices reply to the request's source + address:port) work from any local port, which is what makes the probe's ephemeral-port bind (§7) safe. + +--- + +## 3. Capability mapping (`IDriver` + capability interfaces) + +Recommended surface, landing across phases: + +```csharp +public sealed class BacnetDriver + : IDriver, ITagDiscovery, IReadable, IHostConnectivityProbe, // P1 + ISubscribable, // P2 + IWritable, // P3 + IDisposable, IAsyncDisposable +``` + +`DriverType => "BACnet"` (see §7 for the exact casing decision). + +| OtOpcUa capability | BACnet mechanism | Phase | +|---|---|---| +| `IDriver.InitializeAsync` | Bind the UDP socket on the configured interface/port; if `ForeignDevice` is configured, `Register-Foreign-Device` to the BBMD **and start the TTL-renew timer** (renew at ~TTL/2); optionally fire one warm-up `Who-Is`. Health → Healthy once the socket is bound (BACnet has no session; liveness is per-device via read/COV, surfaced through the probe). | P1 | +| `ITagDiscovery.DiscoverAsync` | Bounded `Who-Is` collect → per-device Device `object-list` (RPM, segmented) → per-object `object-name` + `units` + `object-type` → stream folders/variables into `IAddressSpaceBuilder`. `SupportsOnlineDiscovery => true`; `RediscoverPolicy => DiscoveryRediscoverPolicy.Once`. | P1 | +| `IReadable.ReadAsync` | Group the batch's references by device-instance; issue **ReadPropertyMultiple** per device for `present-value` (+ `status-flags` for quality). Fall back to per-object **ReadProperty** for devices that reported no RPM/segmentation support in their I-Am. Map result → `DataValueSnapshot`. | P1 | +| `IHostConnectivityProbe` | Per-device liveness: a cheap RP of each known device's `system-status` (or a directed Who-Is). Running↔Stopped transitions raise `OnHostStatusChanged`, scoping Bad-quality fan-out to that device's subtree. Foreign-device mode: probe also asserts "BBMD registration alive." | P1 | +| `ISubscribable.SubscribeAsync` | **SubscribeCOV** per object (native push — the headline win). `OnCOVNotification` handler fans changes to `OnDataChange`. **Lifetime-renew timer** re-subscribes before expiry. Devices that reject COV **degrade transparently to a poll loop** at `publishingInterval`, so the subscribe surface always works. | P2 | +| `IWritable.WriteAsync` | **WriteProperty** `present-value` at a configured **priority** (1–16), or `null` to relinquish. **Verdict: NOT in v1 (§3.3).** | P3 | +| `GetHealth` / `GetMemoryFootprint` / `FlushOptionalCachesAsync` | Standard. Footprint ≈ discovered-object count × ~512 B (mirror `OpcUaClientDriver.GetMemoryFootprint`). Flush drops the device/object browse cache + resets the footprint counter. | P1 | + +### 3.1 `IReadable` shape (concrete) + +Follow `OpcUaClientDriver.ReadAsync` structure: return one `DataValueSnapshot` per requested reference in +order; report **per-reference** failures via the snapshot's `StatusCode` (Bad-coded), throw only if the +whole client is unreachable. BACnet specifics: + +1. Parse each `fullReference` with `BacnetEquipmentTagParser.TryParse` (§5); a malformed blob → + `BadNodeIdUnknown` snapshot without touching the wire. +2. Group parsed addresses by `deviceInstance`. Resolve device network address from the discovered device + table (cached from Who-Is/I-Am); a device not in the table → directed Who-Is with a short deadline, + else `BadCommunicationError`. +3. Per device: one **RPM** for all `(object, present-value + status-flags)` pairs, under the per-request + APDU deadline (§8). Non-RPM devices → sequential **RP** per object. +4. Map `BacnetValue` → `DataValueSnapshot(value, statusCode, sourceTs=null, serverTs=now)`. + `status-flags` `out-of-service`/`fault` ⇒ Uncertain/Bad; else Good (see §3.4). + +### 3.2 `ISubscribable` shape (P2 — the real advantage over poll drivers) + +- `SubscribeAsync` issues **SubscribeCOV** (`preferConfirmed` from config) per object, tracking a + `subscriberProcessIdentifier` per subscription. Fire the OPC UA initial-data callback with a current + read so clients get a value immediately. +- **Lifetime-renew timer.** Each subscription carries `lifetimeSeconds`; a per-driver timer re-issues + SubscribeCOV at ~`lifetime/2`. This is load-bearing — an un-renewed subscription silently expires and + the node goes stale-Good with no error. (Same class of bug as the redundancy-heartbeat and + continuous-historization renewal traps in this repo's memory — treat renewal as first-class.) +- **Transparent poll fallback.** On a SubscribeCOV reject (error-class `services`/`object`, or a device + whose I-Am advertised no COV), the driver **silently** switches that object to an internal poll loop at + `cov.fallbackPollMs` and still raises `OnDataChange`. The operator sees data either way; the + diagnostic id records which mode each object is in. +- `UnsubscribeAsync` sends SubscribeCOV with lifetime 0 (cancel) and stops any fallback poll. + +### 3.3 `IWritable` — verdict: NOT v1 (P3), and why + +BACnet writes target a **priority array**: a commandable output (AO/BO/MSV with a `priority-array` +property) resolves its effective value from the highest-priority non-null slot, with `relinquish-default` +as the floor. Writing `present-value` actually writes a *slot* (priority 1–16); writing the wrong slot, +or writing an absolute value where the site's BMS control logic expects to own a slot, **fights the +building's own control loops** — a genuine safety concern (e.g. commanding a damper the BMS is +sequencing). Correctly modelling commandable-vs-non-commandable objects, slot selection, and relinquish +semantics is a correctness minefield that deserves its own phase with explicit validation. So: + +- **v1: read-only.** Every tag's parsed address carries `writable=false`; the driver does **not** + implement `IWritable` in P1/P2. +- **P3: `IWritable`** with an explicit per-tag `writePriority` (default 8 "Manual Operator", never 1), a + commandable-object guard (reject writes to non-commandable object types), and a `null`-relinquish path. + Mirror `OpcUaClientDriver.WriteAsync` honesty: a timeout after dispatch → `BadTimeout` (outcome + unknown), a pre-wire failure → `BadCommunicationError`. + +### 3.4 Data-type mapping (`BacnetObjectType`/property → `DriverDataType`) + +Driver-side map lives in `Driver.Bacnet/BacnetDataTypeMap.cs` (parallel to Galaxy's `DataTypeMap.cs`). +`DriverDataType` members available: `Boolean, Int16/32/64, UInt16/32/64, Float32, Float64, String, +DateTime, Reference`. + +| BACnet present-value / property type | `DriverDataType` | Notes | +|---|---|---| +| REAL (AI/AO/AV present-value) | `Float32` | the common analog case | +| DOUBLE | `Float64` | | +| BOOLEAN | `Boolean` | | +| Enumerated Active/Inactive (BI/BO/BV) | `Boolean` | 0=inactive, 1=active | +| Enumerated multistate (MSI/MSO/MSV) | `UInt16` | present-value is a 1-based state index; `state-text` array surfaced as metadata later | +| Unsigned Integer | `UInt32` | | +| Signed Integer | `Int32` | | +| CharacterString (`object-name`, `description`) | `String` | | +| BACnetDateTime / Date / Time | `DateTime` | Schedules, Trend Log timestamps | +| ObjectIdentifier / property reference | `String` | rarely a tag target | +| Array property (element index N) | element type + `IsArray=true` | ValueRank=1, ArrayDimensions from the array length | + +**Engineering Units.** The BACnet `units` enum (`degrees-celsius`, `kilowatt-hours`, `percent`, …) is +captured at discovery/browse time and surfaced as OPC UA `EngineeringUnits` (EUInformation) node metadata +via `IAddressSpaceBuilder.AddProperty`, so UNS nodes carry proper units. + +**Quality from `status-flags`.** `out-of-service` or `fault` ⇒ Bad/Uncertain `StatusCode`; else Good. +Read RPM should always co-fetch `status-flags` alongside `present-value`. + +--- + +## 4. The async wrinkle + how discovery/browse are delivered + +### 4.1 The callback→async adapter + +`BacnetClientAdapter` (new, in the transport project) is the single seam that turns the ela-compil +callback/blocking API into the `Task`-returning calls the driver's capability methods need. Pattern +(identical intent to how `OpcUaClientDriver` wraps the SDK): + +- **Who-Is is a broadcast + asynchronous I-Am collection — there is no "the response".** The adapter + exposes `Task> WhoIsAsync(range, TimeSpan window, ct)`: subscribe + `OnIam`, `client.WhoIs(low, high)`, accumulate callbacks into a concurrent set, then `Task.Delay(window, + ct)` and return the snapshot. The `window` (`discovery.whoIsWindowMs`, default 4 s) is a **hard latency + floor** on discovery — bound it, show a spinner in the picker. +- **RP/RPM/WP/SubscribeCOV** each get a `TaskCompletionSource` + `CancellationTokenSource.CancelAfter` + wrapper around the library's begin/end (or blocking) call, so per-request APDU deadlines (§8) are + enforced independently of any socket timeout — the R2-01 frozen-peer lesson (an async call must not + outlive its own wall-clock deadline just because the transport didn't time out). +- COV notifications arrive on the library's callback thread; the adapter marshals them onto the driver's + `OnDataChange` event. + +### 4.2 Browse: universal in v1, bespoke-lazy P-later + +**Reconciliation with `2026-07-15-universal-discovery-browser-design.md`:** BACnet browse **is** discovery +(Who-Is → object-list → properties), and `DiscoverAsync` already streams exactly the tree the picker +wants into `IAddressSpaceBuilder`. So for **small/medium sites, v1 plugs straight into the universal +`DiscoveryDriverBrowser`** — **no bespoke browser code, no `.Browser` project.** The requirements: + +1. Add the opt-in flag on `ITagDiscovery` (the universal-browser design specifies this default member; + it is **not yet in `ITagDiscovery.cs`** — adding it is part of that design's P1, and BACnet consumes + it): + ```csharp + bool SupportsOnlineDiscovery => false; // add to ITagDiscovery + ``` + `BacnetDriver` overrides it to `true`. +2. BACnet needs **no `PatchForBrowse` entry** — discovery is unconditional (unlike AbCip/TwinCAT which + are config-gated). The picker's authoring config is sufficient to open + discover. +3. The universal browser runs `InitializeAsync → DiscoverAsync → ShutdownAsync` under its open-timeout and + serves the captured tree from memory. BACnet's Who-Is window fits inside that open-timeout (default + 60 s ≫ 4 s window). + +The universal browser's eager, whole-tree, one-shot capture is fine for a handful of devices with modest +`object-list`s. **For large multi-device sites (dozens/hundreds of controllers, thousands of objects) the +eager one-shot is too heavy** — it Who-Ises the whole network and RPMs every device's full object-list at +open. That is the documented **P-later graduation** to a bespoke lazy browser: + +> **P-later — bespoke `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser`.** A hand-written `IBrowseSession` +> (registered `IDriverBrowser` with `DriverType="BACnet"`, which *overrides* the universal fallback per +> §3.1 of the universal design) with three lazy levels: `RootAsync` = bounded Who-Is → devices; +> `ExpandAsync("dev:")` = that one device's `object-list` on demand; `AttributesAsync(...)` = RPM the +> object's key properties. Only build this when a real large site makes the eager capture painful. + +**v1 uses the universal browser. State this clearly in the PR: BACnet ships with `SupportsOnlineDiscovery += true` and zero browser code.** + +--- + +## 5. TagConfig + driver-config JSON + +Two layers, matching every Equipment-kind driver. + +### 5.1 Driver-level config (`BacnetDriverOptions`, in `.Contracts`) + +```jsonc +{ + "localEndpoint": "0.0.0.0", // interface to bind; "0.0.0.0" = all + "port": 47808, // 0xBAC0 default + "localDeviceInstance": 4194302, // our own Device instance id (unique on the net) + "foreignDevice": { // null when on-segment / a BBMD is local + "bbmdAddress": "10.20.0.1", + "bbmdPort": 47808, + "ttlSeconds": 900 // registration TTL; driver auto-renews at ~TTL/2 + }, + "apduTimeoutMs": 3000, // per-request APDU deadline (§8) + "apduRetries": 3, + "maxSegmentsAccepted": 16, // segmentation window we advertise — do NOT set too low + "discovery": { + "whoIsWindowMs": 4000, // I-Am collect window (the time-bounded wrinkle, §4.1) + "deviceInstanceLow": 0, // optional Who-Is range filter + "deviceInstanceHigh": 4194303 + }, + "cov": { // P2 + "preferConfirmed": true, + "lifetimeSeconds": 300, // per-subscription lifetime; renewed before expiry + "fallbackPollMs": 1000 // poll interval for devices that reject COV + }, + "probe": { "enabled": true, "intervalMs": 5000 } +} +``` + +`BacnetDriverOptions` is a plain record/class with `[JsonStringEnumConverter]`-friendly nested option +types; the factory and probe deserialize it identically (§6). Enum knobs authored as **string names**. + +### 5.2 Per-tag `TagConfig` (authored on the TagModal / emitted by the browse picker) + +A BACnet tag address is fully described by **device-instance + object-type + object-instance + property-id +(+ optional array index)**. `present-value` is the default property. + +```jsonc +{ + "deviceInstance": 100, // target device (from I-Am) + "objectType": "AnalogInput", // BacnetObjectType enum (string name) + "objectInstance": 3, // object-instance number + "propertyId": "PresentValue", // BacnetPropertyId enum; default PresentValue + "arrayIndex": null, // non-null → element of an array property + "dataType": "Float32", // DriverDataType hint (from browse-time units/type) + "writable": false, // v1: always false; P3 enables WriteProperty + "writePriority": 8 // P3 only: BACnet command priority 1..16 +} +``` + +### 5.3 `DriverAttributeInfo.FullName` encoding + the parser + +**`FullName` is the verbatim TagConfig JSON blob** (the §5.2 object serialized to a compact string) — +exactly the Modbus convention (`ModbusEquipmentTagParser`: a leading `{` marks an equipment-tag blob). +This is what the picker commits as `TagConfig.FullName`, what discovery writes into each +`DriverAttributeInfo.FullName`, and what the driver publishes values back under, so the runtime +forward-router resolves them. There is **no separate `deviceInstance.objectType:objectInstance` string +form** — the JSON blob is the single canonical reference. + +`BacnetEquipmentTagParser.TryParse(string reference, out BacnetTagAddress addr)` in `.Contracts` (copy the +Modbus parser's shape): + +- Leading `{` gate; `JsonDocument.Parse`; require `deviceInstance` (number), `objectType` (enum), + `objectInstance` (number). +- **Strict enum reads** via `TagConfigJson.TryReadEnumStrict` for `objectType` / `propertyId` — a + present-but-typo'd enum **rejects** the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a + wrong object type (the R2-11 strictness lesson the Modbus parser encodes). +- `propertyId` defaults to `PresentValue` when absent; `arrayIndex` optional; `writable` defaults false + in v1. +- Produce `BacnetTagAddress(Reference: reference, DeviceInstance, ObjectType, ObjectInstance, PropertyId, + ArrayIndex, DataType)` where `Reference == reference` (the def identity, so publish keys match). +- Provide an `Inspect(reference)` warnings method (deploy-time surfacing of invalid enums / unparseable + blobs), parallel to `ModbusEquipmentTagParser.Inspect`. + +--- + +## 6. Typed editor + validator + the enum-serialization trap + +Add a driver-typed tag editor so the TagModal dispatches by `DriverType="BACnet"` instead of falling back +to the raw-JSON textarea. Copy the Modbus template pair (`Components/Shared/Uns/TagEditors/` razor shell ++ `Uns/TagEditors/*Model.cs` pure model). + +1. **`BacnetTagConfigModel`** (`src/Server/.../AdminUI/Uns/TagEditors/BacnetTagConfigModel.cs`) — pure + `FromJson`/`ToJson`/`Validate`, preserving unknown keys via the `JsonObject` bag (mirror + `ModbusTagConfigModel`). Fields: `DeviceInstance:int`, `ObjectType:BacnetObjectType`, + `ObjectInstance:int`, `PropertyId:BacnetPropertyId`, `ArrayIndex:int?`, `DataType:DriverDataType`, + `Writable:bool?`, `WritePriority:int?`. `Validate()`: `deviceInstance` in 0..4194303, `objectInstance` + ≥ 0, `writePriority` in 1..16 when present. +2. **`BacnetTagConfigEditor.razor`** (`Components/Shared/Uns/TagEditors/`) — thin shell binding the model; + enum dropdowns for `ObjectType` / `PropertyId` from the `.Contracts` enums. +3. **Register in both maps:** + ```csharp + // TagConfigEditorMap.cs + ["BACnet"] = typeof(Components.Shared.Uns.TagEditors.BacnetTagConfigEditor), + // TagConfigValidator.cs + ["BACnet"] = j => BacnetTagConfigModel.FromJson(j).Validate(), + ``` + (Both dictionaries are `OrdinalIgnoreCase`, so casing of the key is forgiving, but keep it consistent + with `DriverType` — see §7.) + +**The `JsonStringEnumConverter` enum-serialization trap (must-fix, systemic in this repo).** AdminUI +pages/models that serialize enums **numerically** while the driver factory/probe DTOs are **string-typed** +produce configs that fault the driver at runtime (documented systemic bug: driver enum-serialization). +Guard rails: + +- The `BacnetTagConfigModel.ToJson` writes enums as **name strings** — use the repo's `TagConfigJson.Set` + helper (it emits enum names, as `ModbusTagConfigModel` does), never `JsonSerializer` with default enum + handling. +- `BacnetDriverFactoryExtensions.JsonOptions` **and** `BacnetDriverProbe`'s options **must** carry + `Converters = { new JsonStringEnumConverter() }` + `PropertyNameCaseInsensitive = true` + + `UnmappedMemberHandling.Skip` — identical options in both, so factory and probe parse a given config + byte-for-byte the same (copy `OpcUaClientDriverFactoryExtensions.JsonOptions`). +- Any AdminUI **driver-config** page for BACnet (the driver-level options form, when it lands) must also + register `JsonStringEnumConverter` so `ObjectType`/`PropertyId`/nested option enums round-trip as + strings. + +--- + +## 7. Factory + registration + +**`DriverType = "BACnet"`.** (Chosen casing; the editor/validator/probe/registry keys all use `"BACnet"`. +`DriverType` string comparisons in the tag-editor maps are `OrdinalIgnoreCase`, but keep every registration +site consistent to avoid the Modbus `"ModbusTcp"` vs `"Modbus"` drift that once dropped a driver to the +raw-JSON editor.) + +**`BacnetDriverFactoryExtensions`** (copy `OpcUaClientDriverFactoryExtensions`): + +```csharp +public static class BacnetDriverFactoryExtensions +{ + public const string DriverTypeName = "BACnet"; + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip, + Converters = { new JsonStringEnumConverter() }, + }; + public static void Register(DriverFactoryRegistry registry, ILoggerFactory? lf = null) + { + ArgumentNullException.ThrowIfNull(registry); + registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, lf)); + } + public static BacnetDriver CreateInstance(string id, string json, ILoggerFactory? lf = null) { /* deserialize BacnetDriverOptions, new BacnetDriver(...) */ } +} +``` + +**Wire into the two Host bootstrap sites** (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs`): + +```csharp +// alongside the other Driver.*.Register(registry, loggerFactory) calls (~line 128-135): +Driver.Bacnet.BacnetDriverFactoryExtensions.Register(registry, loggerFactory); + +// alongside the IDriverProbe TryAddEnumerable block (~line 108-114): +services.TryAddEnumerable(ServiceDescriptor.Singleton()); +``` + +**`BacnetDriverProbe`** (`IDriverProbe`, `DriverType => "BACnet"`): deserialize `BacnetDriverOptions` with +the *same* `JsonOptions`; the "cheap connection" is a **bind + directed Who-Is** — bind the UDP socket +**on an ephemeral port** (never the configured 47808 — the runtime driver instance may already hold it +on the same node, and unicast replies come back to the probe's source port anyway; see the §2 +port-sharing caveat), in foreign-device mode `Register-Foreign-Device` first, then broadcast/directed +Who-Is bounded by the probe `timeout`, return `Ok=true` with latency if ≥1 I-Am arrives, else a clear +"no BACnet devices answered / BBMD registration failed" message. Never throw — return `Ok=false` + +message (per `IDriverProbe` contract). Note the UDP-broadcast-on-CI caveat still applies to the probe +(§9). + +**Probe on admin nodes.** Per the repo's split-role gotcha, `IDriverProbe` must be registered where the +AdminUI resolves `IEnumerable` (admin-pinned), not only on driver nodes — the +`TryAddEnumerable` line above is in the shared bootstrap, which covers it, but verify on the split-role +MAIN cluster. + +--- + +## 8. Resilience / timeout + +- **Per-request APDU deadline (R2-01 frozen-peer lesson).** Every RP/RPM/WP/SubscribeCOV call runs under + `apduTimeoutMs` enforced by the adapter's `CancellationTokenSource.CancelAfter` (§4.1), **independent** + of any socket-level timeout. This is the exact class of bug R2-01 found in S7 (async read ignored socket + ReadTimeout → frozen-peer poll wedge): a BACnet device that ACKs at the BVLC layer but never returns the + APDU must surface `BadTimeout`/`BadCommunicationError` within the deadline, not wedge the read loop. + `apduRetries` bounds retry attempts for **idempotent** reads only (never writes — §3.3). +- **COV lifetime/renewal (P2).** The renew timer at ~`lifetime/2` is mandatory (§3.2). A **watchdog** on + the non-COV fallback path: if a fallback-polled object produces no successful read within N intervals, + mark it Bad and log, so a silently-dead poll doesn't masquerade as stale-Good. +- **Foreign-device TTL renewal.** In foreign-device mode a TTL-renew timer re-registers with the BBMD at + ~TTL/2; a failed re-registration transitions the driver `Degraded` and the probe surfaces it. Losing the + registration silently is the top field failure mode (§2 / research §6 risk 1). +- **Segmentation.** Large `object-list` / RPM results segment across APDUs; the ela-compil client handles + windowed segment ack, but `maxSegmentsAccepted` must **not** be set too low or a big device's + object-list truncates. Default 16; document that lowering it risks truncated discovery. +- **Per-device fault scoping.** `IHostConnectivityProbe` reports per-device Running/Stopped so one dead + controller Bad-quality-fans-out only its own subtree, not the whole driver namespace (mirror Galaxy's + per-host model). + +--- + +## 9. Test fixtures + +BACnet has good open simulators; all are UDP servers we containerize on the shared docker host +(`10.100.0.35`) with a `project=lmxopcua` label, driven by `lmxopcua-fix up bacnet` (add a +`tests/.../Docker/docker-compose.yml` under a new BACnet test project, then `lmxopcua-fix sync bacnet`). + +1. **`bacnet-stack` demo server (`bacserv`)** — the reference C stack's demo; configurable + device-instance + object set via env; answers Who-Is/I-Am, RP/RPM, WP, SubscribeCOV. The canonical + target. Also `fh1ch/bacstack-compliance-docker` (a ready-made DockerHub device-sim image) for the + lowest-friction single device. +2. **`mnp/bacnet-docker` (or `desolat/bacnet-docker`)** — docker-compose framework to stand up **multiple** + BACnet/IP servers + a BBMD on one host: the **multi-device Who-Is** discovery test and the + **BBMD/foreign-device topology** test (BBMD + device on one compose network; register the driver as a + foreign device across it). +3. **ela-compil `BACnet.Examples/BasicServer`** — a pure-.NET device sim embeddable **directly in the + integration-test process** (no docker): deterministic COV-notification + RP/RPM tests via **directed + (unicast) Who-Is to a known instance/address**, sidestepping broadcast entirely. Pair with + `BasicAdviseCOV` as an in-proc client oracle. This is the hermetic-unit path. +4. **YABE** (Windows GUI, same ela-compil library) — manual apples-to-apples oracle for cross-checking + reads/COV against a known-good client. + +**The UDP-broadcast-on-CI gotcha.** BACnet Who-Is relies on subnet **broadcast**; bridged docker networks +and macOS-hosted CI don't broadcast cleanly. Two mitigations, exactly mirroring the existing S7/historian +live-gate discipline: + +- **Hermetic unit/integration:** embedded `BasicServer` + **directed unicast Who-Is** to a known + address — no broadcast, runs on macOS/CI. +- **Broadcast / BBMD / multi-device:** an **env-gated live-integration suite** (e.g. + `BACNET_FIXTURE_ENDPOINT` present ⇒ run, absent ⇒ skip cleanly), fixture on the Linux docker host with + `network_mode: host` (or a dedicated bridge). Category `LiveIntegration` like the historian gate. + **Cross-VM reachability — be explicit about what routes:** this repo's tests execute on the dev + machine, *not* on `10.100.0.35`, and a subnet-broadcast Who-Is from an off-subnet peer **does not + route** (and bridged docker networks don't forward subnet broadcasts into containers either). So the + suite as run from the dev machine has exactly two working paths: (a) **directed unicast Who-Is** at + the fixture's host IP:port (host-mode `bacserv` answers a unicast Who-Is with a unicast I-Am to the + request's source address — verify this against the pinned `bacserv` build early, since spec-strict + stacks may broadcast the I-Am, which would never arrive); and (b) the **foreign-device/BBMD leg** — + register with a BBMD container on the docker host; BVLC `Forwarded-NPDU` relay is *unicast* and routes + across subnets, so this leg exercises real broadcast distribution cross-VM and is exactly the field + topology. A genuinely broadcast-only leg (plain Who-Is with no BBMD) can only run **on the docker host + itself** (ssh, containers sharing a compose network) — keep it a separate opt-in job. Never gate CI on + broadcast. + +**Parser + map unit tests** (fully hermetic, no UDP): `BacnetEquipmentTagParser` round-trip + strictness +(typo'd `objectType` rejects), `BacnetDataTypeMap` coverage, `BacnetTagConfigModel` FromJson/ToJson +unknown-key preservation + enum-as-string emission, `TagConfigValidator["BACnet"]` bounds. Live-verify the +typed editor on docker-dev `/uns` (Razor binding bugs pass unit tests — the repo's standing rule). + +--- + +## 10. Phasing + effort + +**Overall effort: Medium-Large** — bigger than Modbus (no discovery/push there), roughly on par with or +slightly above OpcUaClient, driven by the callback→async adapter + COV lifetime machinery + BBMD. + +Each phase is independently shippable (mirror how OpcUaClient landed across PRs): + +- **P1 — Connect + Read + Discover (+ universal browse) + BBMD. (MVP, delivers real value.)** + `.Contracts` (`BacnetDriverOptions`, enums, `BacnetTagAddress`, `BacnetEquipmentTagParser`); + `BacnetClientAdapter` (async wrap); `BacnetDriver : IDriver, ITagDiscovery, IReadable, + IHostConnectivityProbe` with `SupportsOnlineDiscovery=true` + `RediscoverPolicy => + DiscoveryRediscoverPolicy.Once`; RP/RPM read; + Who-Is/object-list discovery streamed to `IAddressSpaceBuilder`; **foreign-device/BBMD registration + + TTL renewal included here** (without it the driver can't reach most real sites); factory + probe + + Host registration; typed tag editor + validator. Browse works via the universal browser — **no browser + code**. +- **P2 — SubscribeCOV push.** `ISubscribable` via SubscribeCOV + lifetime-renew timer + transparent poll + fallback + fallback watchdog. The headline feature over poll-only drivers. +- **P3 — Write.** `IWritable` via WriteProperty with priority-array semantics + commandable-object + validation + relinquish. Deferred deliberately (§3.3 safety minefield). +- **P4 — History / Alarms.** `IHistoryProvider` over **Trend Log** objects (`ReadRange`); `IAlarmSource` + over intrinsic/algorithmic **event enrollment** + COV of `status-flags` → Part 9 conditions + (`DriverAttributeInfo.IsAlarm` + `MarkAsAlarmCondition`). Nice-to-have, large surface. + +### Top risks (from research §6, carried forward) + +1. **BBMD / foreign-device registration + UDP broadcast reachability (HIGH — the top risk).** The server + is usually not on the controllers' L2 segment, so correct foreign-device registration + TTL renewal is + load-bearing and is exactly what's hardest to reproduce in docker/CI. Mitigation: build BBMD in from + P1, test explicitly with the `bacnet-docker` multi-network compose, gate a live suite against a real + site BBMD. +2. **COV lifetime/renewal + non-COV fallback (HIGH, P2).** Subscriptions silently expire if not renewed; + some devices cap or lack COV and must degrade to polling without an operator-visible gap. Won't be + caught by unit tests — needs the live/sim COV soak (mirror the continuous-historization live-gate + discipline). + +Secondary: the callback/blocking library needs a careful TCS+timeout+cancellation adapter; write +priority-array semantics (why P3 is deferred); segmentation for large object-lists (`maxSegmentsAccepted` +not too low). + +--- + +## Appendix A — new/changed files checklist (P1) + +**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts/`:** `BacnetDriverOptions.cs` (+ nested +option records), `BacnetObjectType.cs`, `BacnetPropertyId.cs`, `BacnetTagAddress.cs`, +`BacnetEquipmentTagParser.cs`, `.csproj` (Core.Abstractions ref only). + +**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet/`:** `BacnetDriver.cs`, `BacnetClientAdapter.cs`, +`BacnetDataTypeMap.cs`, `BacnetDriverProbe.cs`, `BacnetDriverFactoryExtensions.cs`, `.csproj` +(Core.Abstractions + Core + .Contracts + NuGet `BACnet` — mirror the Modbus/OpcUaClient transport +csproj reference set). + +**New — AdminUI:** `Uns/TagEditors/BacnetTagConfigModel.cs`, +`Components/Shared/Uns/TagEditors/BacnetTagConfigEditor.razor`. + +**Changed:** `Directory.Packages.props` (+ `BACnet` version); the `.slnx` (add both projects); +`ITagDiscovery.cs` (+ `SupportsOnlineDiscovery` default member — shared with the universal-browser work); +`DriverFactoryBootstrap.cs` (Register + probe TryAddEnumerable); `TagConfigEditorMap.cs` + +`TagConfigValidator.cs` (`"BACnet"` entries). + +**Tests:** `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Tests/` (parser, datatype map, adapter with fake +client), `...Driver.Bacnet.IntegrationTests/` (embedded `BasicServer` hermetic + env-gated +`LiveIntegration` for broadcast/BBMD/COV), plus AdminUI `BacnetTagConfigModel` + validator tests. diff --git a/docs/plans/2026-07-15-driver-expansion-program-design.md b/docs/plans/2026-07-15-driver-expansion-program-design.md new file mode 100644 index 00000000..a55f892d --- /dev/null +++ b/docs/plans/2026-07-15-driver-expansion-program-design.md @@ -0,0 +1,211 @@ +# Driver-expansion program — overall design + +**Status:** program design, 2026-07-15. Umbrella for the next-driver effort: one universal +browse mechanism + six new/extended drivers (MELSEC deferred). This is the authoritative index +and the single statement of the effort's shared architecture, dependency graph, and build order. +Per-driver detail lives in the linked design docs; cross-cutting rules are stated **once here** +so the per-driver docs don't re-derive them. + +## 1. Goal & scope + +Extend the OtOpcUa OPC UA server's driver fleet under the existing Equipment-kind address-space +model — no new COM gateway, all in-process .NET 10 — and make more of the fleet **browsable** +from the AdminUI `/uns` address picker. + +**Curated set (user, 2026-07-15):** MTConnect Agent · MQTT/Sparkplug B · MELSEC SLMP · Omron · +BACnet/IP · SQL poll · Modbus RTU. **MELSEC is deferred** (commercial-license-forced hand-rolled +framer + the hardest addressing model on the roadmap; revisit on a Mitsubishi-shop need). The +other six are designed and scheduled below. + +Alongside the new drivers, a **universal Discover-backed browser** retrofits browse to the +already-shipped fleet (AbCip, TwinCAT, FOCAS) and is the foundation every browsable new driver +builds on. + +Existing fleet for context: Modbus(TCP), S7, AbCip, AbLegacy, TwinCAT, FOCAS, Galaxy, +OpcUaClient, Historian.Gateway (`src/Drivers/`). + +## 2. Document map + +**Program:** this file. + +**Research reports** (`docs/research/drivers/`, index: [`README.md`](../research/drivers/README.md)): +- [`mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) · + [`mqtt-sparkplug.md`](../research/drivers/mqtt-sparkplug.md) · + [`melsec-slmp.md`](../research/drivers/melsec-slmp.md) *(deferred)* · + [`omron.md`](../research/drivers/omron.md) · + [`bacnet-ip.md`](../research/drivers/bacnet-ip.md) · + [`sql-poll.md`](../research/drivers/sql-poll.md) · + [`modbus-rtu.md`](../research/drivers/modbus-rtu.md) · + [`00-existing-driver-browse-audit.md`](../research/drivers/00-existing-driver-browse-audit.md) + +**Design docs** (`docs/plans/2026-07-15-*`): +- [`universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md) — **Wave 0** +- [`mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md) +- [`bacnet-ip-driver-design.md`](2026-07-15-bacnet-ip-driver-design.md) +- [`mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md) +- [`sql-poll-driver-design.md`](2026-07-15-sql-poll-driver-design.md) +- [`omron-driver-design.md`](2026-07-15-omron-driver-design.md) +- [`modbus-rtu-driver-design.md`](2026-07-15-modbus-rtu-driver-design.md) + +Prior art the designs build on: [`2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md) +(bespoke `IBrowseSession` pattern), [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md) +(standard Equipment-kind driver), and the driver-typed tag editors design. + +## 3. Shared architecture (the contract every driver in this program follows) + +Every driver here is a **standard Equipment-kind driver** — an `IDriver` (lifecycle: `Initialize` +/ `Reinitialize` / `Shutdown` / `GetHealth` / footprint) composing only the capability interfaces +its backend supports, from `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`: + +| Capability | Interface | This program's drivers | +|---|---|---| +| Discover | `ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder, ct)` | all (device-enumerated or authored-only) | +| Read | `IReadable` | all | +| Subscribe | `ISubscribable` | all (poll-based, except the push drivers: MQTT, BACnet COV, MTConnect's `/sample` stream) | +| Write | `IWritable` | Omron, Modbus-RTU (v1); MTConnect/MQTT/BACnet/SQL read-only in v1 | +| Probe | `IHostConnectivityProbe` | all | +| Re-discover | `IRediscoverable` | only drivers with a native backend change signal (per its doc contract) — e.g. MQTT rebirth; BACnet/SQL are discovery-capable but have no change signal and correctly omit it | + +**Per-driver build pattern** (identical across the fleet — see any existing driver + the +driver-typed-editors design): +1. `ZB.MOM.WW.OtOpcUa.Driver..Contracts` — options + tag DTOs + parser + enums (no backend + NuGet dep). +2. `ZB.MOM.WW.OtOpcUa.Driver.` — the runtime `IDriver` + capabilities. +3. `.Browser` project **only when bespoke browse is required** (see §4). +4. Factory + `DriverType` string + `Register()`, wired in the Host's `DriverFactoryBootstrap` + (factory + probe on admin nodes via `TryAddEnumerable`). +5. Typed tag editor + validator: a `TagConfigModel` (`FromJson`/`ToJson`/`Validate`) + + `TagConfigEditorMap` + `TagConfigValidator` entries. +6. Docker fixture under `tests/.../Docker/`, deployed to `10.100.0.35` via `lmxopcua-fix sync` + (the `project=lmxopcua` label is a deployment convention applied host-side — no checked-in + compose file carries a `labels:` entry today). + +### 3.1 Cross-cutting rules (mandatory, stated once) + +- **Enum-serialization trap** — every enum on a config/tag surface MUST serialize as **names, + never numbers**, on both sides (AdminUI page serializer **and** probe carry + `JsonStringEnumConverter`). Two shipped factory-side patterns both satisfy this — Modbus keeps + DTO fields `string?` + parses via `ParseEnum` (no converter in the factory); OpcUaClient + uses enum-typed DTOs with the converter in factory **and** probe. Either is fine; a mismatch + between the authoring side and the factory side **faults the driver** at deploy. This is a + proven systemic bug — see the driver enum-serialization memory. Non-negotiable for every + driver here. +- **Per-op deadline (R2-01 frozen-peer lesson)** — every network call (read/write/discover/probe/ + subscribe/agent-HTTP/SQL-query) MUST have a bounded deadline (linked-CTS / socket ReadTimeout / + `CommandTimeout`). A frozen peer must never wedge a poll. No unbounded waits anywhere. +- **`WriteIdempotent`** — writable drivers flag only tags whose replay is safe. Note the invoker + seam (`ExecuteWriteAsync(…, isIdempotent, …)`) exists but the host's write dispatch currently + hardcodes `isIdempotent: false` (the arch-review "hardcoded isIdempotent" Medium) — so **no + write auto-retries today**; drivers carry the per-tag flag now and retry activates when that + fleet plumbing lands. +- **Secrets** — connection strings / API keys / broker creds come from env/secret refs (mirror + `ServerHistorian__ApiKey`), never committed or logged. +- **Live-verify discipline** — Razor binding + deploy-inertness bugs pass unit tests and review; + every driver's picker/editor and deploy path gets a docker-dev `/run` live-verify before it's + trusted. + +## 4. Browse architecture — the two-tier decision + +The program's central browse decision: instead of a bespoke browser per driver, a **universal +Discover-backed browser** (`DiscoveryDriverBrowser`, Wave 0) captures any driver's +`ITagDiscovery.DiscoverAsync` output and re-presents it to the picker. Bespoke `IBrowseSession` +browsers are reserved for the cases the universal one genuinely can't serve. + +**The gate:** a new default-interface member `ITagDiscovery.SupportsOnlineDiscovery => false` +(mirrors the existing `RediscoverPolicy` default member). A driver opts in when its `DiscoverAsync` +**enumerates from the device**; flat-address drivers leave it false and stay manual-entry. This +member is **added in the universal browser's P1** — **every browsable driver in this program +depends on it**, which is why the universal browser is the Wave-0 gate. + +| Driver | Browse mechanism | Verdict | +|---|---|---| +| **MTConnect** | `/probe` device model via `DiscoverAsync` | **Universal** (`SupportsOnlineDiscovery=true`) | +| **BACnet/IP** | Who-Is + object-list via `DiscoverAsync` | **Universal** for small sites; bespoke-lazy `IBrowseSession` a documented P-later for large multi-device sites | +| **AbCip** *(existing)* | CIP Symbol 0x6B/0x6C in `DiscoverAsync` | **Universal** (+ `PatchForBrowse` `EnableControllerBrowse:true`) | +| **TwinCAT** *(existing)* | ADS symbol upload in `DiscoverAsync` | **Universal** (+ patch); bespoke lazy only if symbol set too big | +| **FOCAS** *(existing)* | FixedTree curated discovery | **Universal** (curated; + patch `FixedTree.Enabled:true` + an `UntilStable` settle — its tree fills post-connect, so a one-shot capture would be empty) | +| **MQTT/Sparkplug** | passive topic/birth observation window | **Bespoke** `MqttBrowseSession` — runtime discovery is authored-only, so universal would replay only authored tags | +| **SQL poll** | `INFORMATION_SCHEMA` schema walk | **Bespoke** `SqlBrowseSession` — schema walk ≠ `DiscoverAsync` | +| **Omron** | libplctag `@tags` unsupported on NJ/NX | **None live** — offline Sysmac tag-export importer (P-later); FINS never browsable | +| **Modbus RTU** | flat registers, no discovery | **None** — not browsable | +| S7, AbLegacy, Modbus(TCP) *(existing)* | flat / no on-wire symbols | **None** — documented in the audit | + +Bespoke browsers **override** the universal fallback simply by being registered for their +`DriverType` (`BrowserSessionService` resolves bespoke-first). See the universal-browser design §3 +for the required `BrowserSessionService` fallback change (the `ToDictionary` dup-key constraint +means the universal browser is a **separate `IUniversalDriverBrowser`**, not one of the injected +`IDriverBrowser` set). + +## 5. Library & license decisions + +| Driver | Library | License | Note | +|---|---|---|---| +| MTConnect | TrakHound MTConnect.NET (`-Common` + `-HTTP`) | MIT | netstd2.0 → loads on net10; pin + license-review; hand-roll fallback | +| BACnet/IP | System.IO.BACnet (ela-compil fork, `BACnet`) | MIT | multi-targets net10; YABE engine; callback API → TCS adapter | +| MQTT/Sparkplug | MQTTnet v5 + **hand-rolled Tahu protobuf** | MIT | **Not SparkplugNet** — its MQTTnet-4.x transitive pin collides with the repo's deliberately-OFF transitive pinning (Roslyn-split constraint) | +| Omron (CIP) | libplctag.NET | MPL-2.0 | same dep AbCip already ships (`plc=omron-njnx`); FINS hand-rolled | +| SQL poll | Microsoft.Data.SqlClient | — | already in-repo at 6.1.1 — zero new deps for P1 | +| Modbus RTU | extend existing Modbus | — | serial impls behind the **existing** `IModbusTransport` seam; P1 (RTU-over-TCP) zero new deps, P2 direct-serial adds `System.IO.Ports` | +| ~~MELSEC~~ | ~~hand-rolled 3E~~ | — | deferred; HslCommunication is a commercial-license blocker | + +**Avoid HslCommunication** (commercial license; the "MIT" claim is a stale abandoned fork) for both +MELSEC and Omron. + +## 6. Dependency graph & build order + +``` +Wave 0 ─ Universal browser ── adds ITagDiscovery.SupportsOnlineDiscovery ──┐ + │ (DiscoveryDriverBrowser + CapturingAddressSpaceBuilder + │ every browsable + │ BrowserSessionService fallback) │ driver depends + │ │ on this member + ├── immediately lights up AbCip / TwinCAT / FOCAS pickers (existing) │ + ▼ │ +Wave 1 ─ Modbus RTU (extend, lowest effort) · SQL poll (bespoke schema browser) ◄┘ + ▼ +Wave 2 ─ MTConnect (universal browse) · MQTT/Sparkplug (bespoke observation browser) + ▼ +Wave 3 ─ BACnet/IP (universal browse + COV) · Omron (CIP-first; last — no NJ/NX CIP sim) +``` + +| Wave | Item | Browse | Write v1 | Effort | +|---|---|---|---|---| +| 0 | Universal `DiscoveryDriverBrowser` | — | — | S–M (~3–5 d) | +| 1 | Modbus RTU | none | Yes | **S (lowest)** | +| 1 | SQL poll | bespoke schema | No | S–M | +| 2 | MTConnect Agent | universal | No | S–M | +| 2 | MQTT/Sparkplug B | bespoke observation | No | M–L | +| 3 | BACnet/IP | universal (+ bespoke-lazy P-later) | No | M–L | +| 3 | Omron | offline import | Yes | M–L | + +**Sequencing rationale:** Wave 0 first because the `SupportsOnlineDiscovery` seam gates every +browsable driver and it retrofits browse to three shipped drivers for near-zero marginal cost. +Wave 1 is the low-effort/high-leverage pair (RTU extends an existing driver; SQL adds zero deps). +Wave 2 is the strategic telemetry/UNS pair. Wave 3 is the heavier/test-gated pair — Omron last +because there is **no free NJ/NX CIP simulator**, so its wire correctness is a live-hardware gate. +MELSEC is out of the sequence (deferred). + +## 7. Testing & fixtures + +- Fixtures live under `tests/.../Docker/`, deployed to the shared host `10.100.0.35` via + `lmxopcua-fix sync` (which owns the `project=lmxopcua` labelling host-side). Per-driver sims: `mtconnect/cppagent`, Mosquitto/EMQX + + Sparkplug simulator, `bacnet-stack`/ela-compil `BasicServer`, central SQL Server + (`10.100.0.35,14330`) + seeded table, `rtu_over_tcp` pymodbus profile + socat serial pair. +- **Env-gated live suites** for what a sim can't cover: BACnet Who-Is/BBMD UDP broadcast, Omron CIP + wire (live hardware), MQTT broker soak. +- Per driver: unit (framer/parser/type-map with golden vectors + fakes) → integration (docker + fixture) → **live `/run`** on docker-dev (picker + editor + deploy path). + +## 8. Deferred / out of scope + +- **MELSEC SLMP** — deferred (§1). Research complete (`melsec-slmp.md`); revisit on demand. +- **Write-back** for MTConnect (Interfaces), MQTT (NCMD/DCMD), BACnet (WriteProperty priority + array), SQL (parameterized UPSERT) — all documented as later phases in the respective designs. +- **Bespoke lazy browsers** for large BACnet sites and large ControlLogix/TwinCAT symbol sets — + graduate from universal only when eager one-shot discovery proves too heavy. +- Shared `Cip.Core` extraction (AbCip ↔ Omron CIP) — a follow-up, not a gate; keep Omron + self-contained in v1. + +## 9. Status + +Research: **done** (8 reports). Designs: **done** (7 design docs). Code: **not started.** +Next action: execute Wave 0 (universal browser), then Wave 1. diff --git a/docs/plans/2026-07-15-modbus-rtu-driver-design.md b/docs/plans/2026-07-15-modbus-rtu-driver-design.md new file mode 100644 index 00000000..cc0d3bd4 --- /dev/null +++ b/docs/plans/2026-07-15-modbus-rtu-driver-design.md @@ -0,0 +1,380 @@ +# Modbus RTU (serial + RTU-over-TCP) — implementation design + +**Status:** Design / build-ready. Not implemented. +**Date:** 2026-07-15 +**Scope:** Add Modbus **RTU** transport modes (direct serial + RTU-over-TCP) to the +existing `ModbusDriver`. +**Research input:** [`docs/research/drivers/modbus-rtu.md`](../research/drivers/modbus-rtu.md). +**Related:** [`docs/drivers/Modbus.md`](../drivers/Modbus.md), +[`docs/v2/modbus-addressing.md`](../v2/modbus-addressing.md), +[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md). + +--- + +## 1. Motivation + extend-vs-new verdict — **EXTEND** + +Add Modbus RTU as **two new `IModbusTransport` implementations behind the existing +transport seam + driver-level config plumbing to select them.** Do **not** create a +sibling driver. The Modbus application protocol above the wire (register model, function +codes FC01–06/15/16, exception-PDU convention, data-type codecs, byte order, arrays, +strings, BCD, bit-in-register, read planner + coalescing, auto-prohibit, deadband, +write path, connectivity probe, OPC-UA materialisation, HistoryRead) is **byte-for-byte +identical** across TCP and RTU — only the wire framing and the physical link differ. A +second driver would clone that entire surface for zero benefit. + +### The exact seam this plugs into + +The driver already splits the *PDU* (function code + data) from the *transport* +(framing + link I/O) behind one interface: + +`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs` + +```csharp +public interface IModbusTransport : IAsyncDisposable +{ + Task ConnectAsync(CancellationToken ct); + Task SendAsync(byte unitId, byte[] pdu, CancellationToken ct); // pdu = [fc, ...data], returns response PDU +} +``` + +- **`ModbusDriver`** builds every request as a raw PDU `byte[]` and calls + `transport.SendAsync(ResolveUnitId(tag), pdu, ct)` (e.g. `ModbusDriver.cs` lines + ~1014/1035/1052/1069/1142/1154). It never touches MBAP, sockets, or CRC. +- **`ModbusTcpTransport`** is the *only* place MBAP framing, the transaction-id counter + (`_nextTx`), and `TcpClient`/`NetworkStream` I/O live — specifically `SendOnceAsync` + (`ModbusTcpTransport.cs` lines 231–296) wraps the PDU as + `[TxId(2)][Proto=0(2)][Length(2)][UnitId(1)] + PDU`, single-flights via `SemaphoreSlim + _gate`, and owns reconnect/keepalive/idle-disconnect. +- **The transport is injected**: `ModbusDriver`'s ctor takes + `Func? transportFactory` + (`ModbusDriver.cs` lines 98–114); the default closure builds `ModbusTcpTransport`. + Tests already substitute in-memory fakes through this hook. + +So the work is **purely additive**: two new transport classes + a CRC-16 helper + +config/factory selection + one AdminUI panel. **Zero** changes to codecs, planner, +coalescing, health, materialisation, HistoryRead, or the address parser. + +### New/changed files at a glance + +| File | Change | +|---|---| +| `…/Driver.Modbus/ModbusCrc.cs` | **new** — CRC-16 (poly `0xA001`) helper | +| `…/Driver.Modbus/ModbusRtuFraming.cs` | **new** — shared ADU frame/deframe + FC-aware response sizing | +| `…/Driver.Modbus/ModbusRtuTransport.cs` | **new** — `System.IO.Ports.SerialPort` transport | +| `…/Driver.Modbus/ModbusRtuOverTcpTransport.cs` | **new** — RTU framing over a socket | +| `…/Driver.Modbus/ModbusSocketLifecycle.cs` | **new** (refactor) — socket connect/reconnect/keepalive/idle extracted from `ModbusTcpTransport`, shared by TCP + RtuOverTcp | +| `…/Driver.Modbus/ModbusTransportFactory.cs` | **new** — `Create(ModbusDriverOptions)` switch on `Transport`; used by the driver default closure **and** the probe | +| `…/Driver.Modbus.Contracts/ModbusDriverOptions.cs` | add `Transport` + serial fields (+ two enums) | +| `…/Driver.Modbus/ModbusDriverFactoryExtensions.cs` | DTO fields + wire the default closure to `ModbusTransportFactory.Create` | +| `…/Driver.Modbus/ModbusDriverProbe.cs` | build the transport via `ModbusTransportFactory` (currently hardcodes `new ModbusTcpTransport`, line 77) | +| `…/Driver.Modbus/ModbusDriver.cs` | `BuildSlaveHostName` endpoint string reflects the transport (COM/gateway vs host:port) | +| `…/AdminUI/…/Drivers/ModbusDriverPage.razor` | serial-parameter panel shown when `Transport != Tcp` | + +`System.IO.Ports` (10.0.x for .NET 10) becomes a new PackageReference on +`ZB.MOM.WW.OtOpcUa.Driver.Modbus`. + +--- + +## 2. Transport implementations + +Both new transports produce the **RTU ADU**: `[slaveAddress(1)][PDU][CRC-lo][CRC-hi]` — +no MBAP header, no transaction id. CRC-16 (poly `0xA001` reflected, appended +**low byte first**) replaces TCP's transport-level integrity. The RTU framing/deframing +logic is identical between the two; factor it into `ModbusRtuFraming` so the serial and +socket transports share it and differ only in the byte-stream they read/write. + +### 2a. `ModbusRtuFraming` (shared) + +- `byte[] BuildAdu(byte unitId, ReadOnlySpan pdu)` → `[unit][pdu][crcLo][crcHi]`. +- `int ExpectedResponseLength(byte functionCode)` / a streaming reader that determines + response length **by parsing the function code** (RTU frames are length-less — see + §7). Read `addr(1)` + `fc(1)` first, then: + - **Exception** (`fc & 0x80`): read `excCode(1) + CRC(2)` → 5-byte ADU total; throw + `ModbusException(fc & 0x7F, excCode, …)` after CRC-validating. + - **Read responses** FC01/02/03/04: read `byteCount(1)`, then `byteCount + CRC(2)`. + - **Write echoes** FC05/06/15/16: fixed `addr + qty/value(4) + CRC(2)` → read 6 more. +- After the full ADU is in hand: validate the trailing CRC-16 (mismatch → + `ModbusTransportDesyncException(DesyncReason.…)` so it maps onto the existing + `BadCommunicationError` handling), strip the address byte + CRC, and return the bare + response PDU `[fc, ...data]` — exactly what `SendAsync`'s callers already decode. + +### 2b. `ModbusRtuTransport : IModbusTransport` (direct serial) + +- Wraps a `System.IO.Ports.SerialPort`. `ConnectAsync` opens the port with the + configured `PortName`/`BaudRate`/`DataBits`/`Parity`/`StopBits`; sets + `ReadTimeout`/`WriteTimeout` from `ModbusDriverOptions.Timeout`. +- **Per-op deadline (R2-01): `SerialPort.ReadTimeout` only governs the synchronous read + API — async reads over `SerialPort.BaseStream` ignore `ReadTimeout`/`WriteTimeout` + entirely** (the exact wall-clock gap class R2-01 found in the S7 driver, where async + socket reads ignored `ReadTimeout` and a frozen peer wedged the poll). Every + transaction MUST therefore run under a linked-CTS + `CancellationTokenSource.CancelAfter(Options.Timeout)` deadline, mirroring + `ModbusTcpTransport.SendOnceAsync` — timeout ⇒ close the port + surface + `ModbusTransportDesyncException(DesyncReason.Timeout)`, distinct from caller + cancellation (no teardown). +- `SendAsync`: single-flight via the same `SemaphoreSlim _gate` pattern (**mandatory** on + RTU — no TxId means at most one transaction on the bus at a time). Enforce ≥3.5-char + inter-frame idle before transmit (computed from baud/word-length, or the + `InterFrameDelayMs` override), write the ADU, then read the response using + `ModbusRtuFraming`'s FC-aware sizing, backstopped by the linked-CTS per-op deadline. +- Failure model is **simpler than TCP** — a COM port doesn't "drop" the way a NAT'd + socket does. On an I/O error or CRC desync, close + reopen the port once and retry + (reuse the single-retry shape from `ModbusTcpTransport.SendAsync`); no keepalive, no + idle-disconnect, no geometric socket backoff. + +### 2c. `ModbusRtuOverTcpTransport : IModbusTransport` (RTU tunnelled over a socket) + +- **Identical RTU framing** (`ModbusRtuFraming`) but the byte stream rides a + `TcpClient`/`NetworkStream` to a serial→Ethernet gateway instead of a COM port. +- **Reuses the hardened socket lifecycle** — extract the connect (IPv4-preference DNS), + `SO_KEEPALIVE`, idle-disconnect, and reconnect-with-backoff machinery from + `ModbusTcpTransport` into `ModbusSocketLifecycle` and have **both** TCP variants + compose it. The *only* delta from `ModbusTcpTransport` is: CRC framing instead of MBAP, + and no transaction id. This is the lowest-risk of the two RTU transports to build and + test (see §9). + +**Modbus ASCII is out of scope** — a third, rarely-used framing (`:`-delimited hex, +LRC instead of CRC). It would be another `IModbusTransport` behind the same seam if ever +needed; not planned. + +--- + +## 3. Cross-platform serial reality + +- **`System.IO.Ports.SerialPort` is cross-platform** on .NET 10 (`System.IO.Ports` + 10.0.x): Windows + Linux (`/dev/tty*` via termios). **macOS is the weak platform** + (baud quirks, `MacCatalyst` unsupported) — this repo's dev machine is macOS, so + **direct-serial cannot be exercised on the dev Mac**; use fakes / RTU-over-TCP there + and run the real-serial suite on the Linux docker host / CI (§8). +- **Containers add a device-mapping hurdle:** a serial device must be explicitly passed + in (`docker run --device=/dev/ttyUSB0` or a compose `devices:` entry), and USB-serial + adapters re-enumerate (`ttyUSB0` ↔ `ttyUSB1`) across replug — pin a stable + `/dev/serial/by-id/...` path. Windows-container COM passthrough is unreliable. + +**RTU-over-TCP is the recommended primary path.** OtOpcUa deploys as a containerised +Linux server (docker-dev rig; docker host `10.100.0.35`) that is generally **not** +attached to an RS-485 bus. The idiomatic topology is a **serial→Ethernet gateway** (Moxa +NPort, Digi One, Lantronix, USR-TCP232) on the RS-485 multidrop, exposed over TCP. The +server talks **RTU-over-TCP** to it — a plain socket, **zero host-device mapping, no +`System.IO.Ports` dependency at runtime, no udev fragility**, and it reuses the already- +hardened socket lifecycle. Direct serial ships too, for bare-metal Windows/Linux installs +with a local/adapter COM port, but RTU-over-TCP is what most deployments will use. + +--- + +## 4. Config JSON shape + +**Per-tag `TagConfig` is unchanged.** `ModbusTagDefinition` / `ModbusTagDto` already carry +everything RTU needs — including the per-tag **`UnitId`** override that makes an RS-485 +multi-drop bus "just work" (the read planner already refuses to coalesce across UnitIds; +`ResolveUnitId` + `BuildSlaveHostName` already key per-slave resilience by unit). The +`ModbusTagConfigEditor` in `TagConfigEditorMap` needs **no change**. Additions are +**driver-level only**. + +New fields on `ModbusDriverOptions` (and the matching optional fields on +`ModbusDriverConfigDto`): + +| Field | Type | Applies to | Notes | +|---|---|---|---| +| `Transport` | `ModbusTransportMode` enum (`Tcp`\|`Rtu`\|`RtuOverTcp`) | all | **Default `Tcp`** (back-compat: existing configs omit it) | +| `SerialPort` | string | Rtu | `"COM3"` / `"/dev/ttyUSB0"` / `by-id` path | +| `BaudRate` | int | Rtu | 9600 / 19200 / 38400 / 115200 | +| `DataBits` | int | Rtu | usually 8 | +| `Parity` | `Parity` enum (`None`\|`Even`\|`Odd`) | Rtu | spec default **Even**; many devices use None | +| `StopBits` | `StopBits` enum (`One`\|`Two`) | Rtu | 1 with parity, 2 without | +| `InterFrameDelayMs` | int? | Rtu | optional override of computed T3.5 for slow/RF links | +| `Host` / `Port` | reused | Tcp, **RtuOverTcp** | the gateway's socket for RtuOverTcp | + +`Host`/`Port`/`UnitId`/`TimeoutMs`/`MaxRegistersPerRead`/keepalive/reconnect all stay. +Serial fields are ignored when `Transport=Tcp`; `Host`/`Port` are ignored when +`Transport=Rtu`. Keepalive/idle/reconnect apply to `Tcp` + `RtuOverTcp` only. + +> Serialization note: `Parity`/`StopBits` names collide with +> `System.IO.Ports.Parity`/`StopBits` (member names match the table), but **define local +> enums in `Driver.Modbus.Contracts`** and map them to the BCL enums inside +> `ModbusRtuTransport`. Reusing the BCL enums directly would put the `System.IO.Ports` +> package on the Contracts project (and transitively on the AdminUI, which deserializes +> `ModbusDriverOptions`) — contradicting §1's plan to add the dependency to +> `Driver.Modbus` only, and the program-doc rule that Contracts carries no backend +> NuGet dep. + +### Example A — direct serial RTU (multi-drop) + +```json +{ + "transport": "Rtu", + "serialPort": "/dev/ttyUSB0", + "baudRate": 19200, + "dataBits": 8, + "parity": "Even", + "stopBits": "One", + "unitId": 1, + "timeoutMs": 1000, + "tags": [ + { "name": "Flow", "addressString": "40001:F:ABCD", "writable": false }, + { "name": "Setpt", "addressString": "40010:F", "writable": true }, + { "name": "Pump2Run", "region": "Coils", "address": 0, "dataType": "Bool", + "writable": true, "unitId": 2 } + ] +} +``` + +`Pump2Run` is a second drop slave (UnitId 2) on the same bus — no extra transport config, +just the per-tag `unitId` override the driver already honours. + +### Example B — RTU-over-TCP to a serial→Ethernet gateway + +```json +{ + "transport": "RtuOverTcp", + "host": "10.20.0.50", + "port": 4001, + "unitId": 1, + "timeoutMs": 1500, + "tags": [ + { "name": "Temp", "addressString": "30001:I" }, + { "name": "Alarm", "region": "DiscreteInputs", "address": 5, "dataType": "Bool" } + ] +} +``` + +Same `host`/`port` shape as Modbus TCP, but the wire frames are raw RTU (address + CRC, +no MBAP). The `Transport` discriminator **must match the gateway's configured mode**: +`"Tcp"` for a gateway doing Modbus/TCP translation, `"RtuOverTcp"` for +transparent/RTU-passthrough. The two wire formats are mutually unparseable. + +--- + +## 5. The `JsonStringEnumConverter` trap — **must handle** + +Per the project-wide enum-serialization bug (memory: *Driver enum-serialization bug*), +driver pages serialize enums but factory DTOs are string-typed. The new +`Transport`/`Parity`/`StopBits` enums **must round-trip as strings**, or an AdminUI- +authored RTU config faults the driver at deploy. + +Good news — the plumbing is **already correct on both ends** and just needs the new fields: + +- `ModbusDriverPage.razor`'s serializer (`_jsonOpts`, line 328–334) already has + `Converters = { new JsonStringEnumConverter() }` + camelCase. It serializes a + `ModbusDriverOptions` directly, so a `ModbusTransportMode`/`Parity`/`StopBits` on + `ModbusDriverOptions` emits as `"Rtu"`/`"Even"`/`"One"` automatically. +- `ModbusDriverFactoryExtensions` DTO fields are typed `string?` and parsed via the + existing `ParseEnum` helper (case-insensitive) — mirror how `Family` / + `MelsecSubFamily` are already handled. **Do not** type the DTO fields as the enum. +- `ModbusDriverProbe._opts` (line 19–24) already carries `JsonStringEnumConverter`. + +Add a driver-page/factory round-trip unit test asserting `"transport":"Rtu"` (string, not +`1`) — the same guard that caught S7/Modbus previously. + +--- + +## 6. Capability mapping — identical to TCP, four deltas + +Same register model, function codes, data types, read/write semantics, coalescing, +deadband, `WriteOnChangeOnly`, connectivity probe (FC03@0). **Read + write both fully +supported.** The only deltas are below the seam: + +| Delta | TCP (today) | RTU (added) | +|---|---|---| +| Physical link | `TcpClient` | `SerialPort` (Rtu) / `TcpClient` (RtuOverTcp) | +| Framing | 7-byte MBAP + TxId | `[addr][PDU][CRC-16]`, no TxId | +| Integrity | TCP guarantees | app-level CRC-16 (`0xA001`) | +| Unit/slave id | often 1 | **central** — one bus, many drops by unit id (already supported per-tag) | +| Response length | MBAP `Length` field | **no length field** → FC-aware sizing (§7) | +| Timing | none | ≥3.5-char inter-frame silence; single-flight mandatory | + +`ResolveHost` / `BuildSlaveHostName` (`ModbusDriver.cs` line 162, +`"{host}:{port}/unit{n}"`) should format the per-slave resilience key from the active +endpoint — `COMx/unit{n}` (Rtu) or `gatewayHost:port/unit{n}` (RtuOverTcp) — so +per-slave breakers stay distinct. + +### Browseability — **NO** (reconcile w/ universal browser) + +Modbus RTU is **not browsable** — a flat, untyped register space with no discovery +protocol, identical to Modbus TCP. `SupportsOnlineDiscovery=false`; the driver keeps +`RediscoverPolicy = Once` and materialises the authored tag list into a flat folder. No +browser (universal or bespoke). The AdminUI `ModbusAddressPickerBody`/`ModbusAddressBuilder` +is an address *builder* (grammar helper), not a live browser, and stays valid for RTU. + +--- + +## 7. Resilience / timeout + +- **Per-op deadline + T3.5 framing.** The linked-CTS `CancelAfter(Options.Timeout)` + per-op deadline (§2b — **not** `SerialPort.ReadTimeout`, which async reads ignore) is + the hard backstop; FC-aware sizing is the primary length signal, + with the inter-frame idle gap as the delimiter fallback. Above 19200 baud the spec + fixes T3.5≈1.75 ms / T1.5≈750 µs rather than scaling further; below it, + T3.5 ≈ 3.5 × (bits-per-char / baud) (e.g. 9600 8-N-1 ≈ 3.6 ms). `InterFrameDelayMs` + overrides for slow/long RS-485 or RF links. +- **The one genuinely new correctness risk: RTU response sizing without a length field.** + Get the FC-aware calculation + exception-PDU short-frame detection right (§2a), or the + read hangs to timeout / mis-frames. Cover exhaustively with fake-stream unit tests + (below). A CRC mismatch or truncated frame maps to + `ModbusTransportDesyncException` → the existing single reconnect-retry + status mapping. +- **RtuOverTcp reuses the socket lifecycle** — keepalive, idle-disconnect, reconnect + backoff, IPv4-preference connect — unchanged from `ModbusTcpTransport` via the extracted + `ModbusSocketLifecycle`. **Direct serial does not** use any of that (a COM port has no + NAT reaping); it uses the simpler close/reopen-once-on-error model. +- **Single-flight is mandatory on RTU** (no TxId to correlate an interleaved response) — + the existing `_gate` semaphore already provides it; keep it in both new transports. + +--- + +## 8. Test fixtures + +1. **RTU-over-TCP against pymodbus (highest ROI, no serial hardware).** Add an + `rtu_over_tcp` profile to the existing fixture + `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/docker-compose.yml` + (alongside `standard`/`dl205`/`mitsubishi`/`s7_1500`/`exception_injection`), running + `pymodbus.simulator` in **RTU-framed-TCP** mode. The existing profiles all share host + port `:5020` and are mutually exclusive by design; bind `rtu_over_tcp` to a fresh port + so it can co-run with `standard` (and sidestep the known shared-port stale-container + trap). Add the `project=lmxopcua` label per the program-doc fixture convention (the + checked-in compose services carry no `labels:` entries today). Drive it via + `lmxopcua-fix up modbus rtu_over_tcp` + `lmxopcua-fix sync modbus`. This exercises the + real CRC + FC-aware framing end-to-end with **no serial anything** — the only + genuinely new logic. +2. **Virtual serial pair for the direct-serial transport.** `socat -d -d pty,raw,echo=0 + pty,raw,echo=0` creates a linked `/dev/pts/N` ↔ `/dev/pts/M` pair; point a pymodbus + **RTU serial** slave at one end and `ModbusRtuTransport` at the other. (`com0com` is + the Windows equivalent.) Runs on the Linux docker host / CI, **not** the dev Mac. + Defer to an **env-gated live suite** (like the other driver live gates). +3. **`diagslave`** (serial + TCP RTU) for manual/soak against a virtual pair or a real + USB-serial adapter — eventual live gate, not unit CI. + +**Unit-level (no PLC):** +- `ModbusCrc` table-driven test against known Modbus CRC vectors. +- `ModbusRtuFraming` / `ModbusRtuTransport` / `ModbusRtuOverTcpTransport` against an + **in-memory duplex stream fake** (fake one level below the existing `IModbusTransport` + fakes — the byte stream) asserting ADU build, CRC append/validate, FC-aware response + parse for each FC group, and exception-PDU short-frame handling. +- Driver-page/factory round-trip test for the `JsonStringEnumConverter` guard (§5). + +**Recommended CI shape:** unit (CRC + framing + config round-trip) + the `rtu_over_tcp` +pymodbus profile for integration; real-serial (socat / hardware) env-gated live. + +--- + +## 9. Phasing + effort + +**Effort: LOW — likely the lowest-effort item on the driver roadmap.** The seam is +purpose-built; net-new code is small and localised. + +- **P0 — shared plumbing:** `ModbusCrc` (~30 lines) + `ModbusRtuFraming` (FC-aware sizing) + + extract `ModbusSocketLifecycle` from `ModbusTcpTransport` (mechanical refactor; + `ModbusTcpTransport` keeps behaviour) + `ModbusTransportFactory.Create` + config fields + on `ModbusDriverOptions`/DTO + factory closure + probe wiring. Unit tests for CRC + + framing + config round-trip. +- **P1 — `ModbusRtuOverTcpTransport`** (reuses the socket lifecycle; delta = CRC framing + + FC-aware length, no TxId) + `rtu_over_tcp` pymodbus docker profile + AdminUI serial panel + wired for the RtuOverTcp subset. **Ship this first** — no host-device dependency, + testable on the existing pymodbus harness, reuses hardened socket code. +- **P2 — `ModbusRtuTransport`** (direct serial) + `System.IO.Ports` PackageReference + + socat/diagslave env-gated live suite. For bare-metal installs. + +**Top risk:** serial framing/timing on Linux/in-containers (best-effort T1.5/T3.5 gating; +device enumeration; the length-less-frame response-sizing correctness). Mitigated by +leading with RTU-over-TCP (P1), exhaustive fake-stream framing tests, and the +`InterFrameDelayMs` override for slow links. Direct-serial (P2) is the residual-risk +piece and is deferred behind the env-gated live suite. diff --git a/docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md b/docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md new file mode 100644 index 00000000..53312af4 --- /dev/null +++ b/docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md @@ -0,0 +1,412 @@ +# MQTT / Sparkplug B driver — executable implementation design + +**Status:** design, build-ready. 2026-07-15. +**Research input:** `docs/research/drivers/mqtt-sparkplug.md` (verdicts + protocol/library survey — this doc turns that into a build plan; it does not re-litigate the research). +**Related:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (why MQTT needs a *bespoke* browser despite the universal one — §4), `docs/plans/2026-06-12-galaxy-standard-driver-design.md` (standard Equipment-kind driver shape + fire-and-forget write precedent), `docs/plans/2026-05-28-driver-browsers-design.md` (`IDriverBrowser`/`IBrowseSession` pattern reused by the observation browser). + +--- + +## 1. Motivation + scope + +Build a standard **Equipment-kind** driver — same composable-`IDriver` shape as Modbus / S7 / AbCip / TwinCAT / FOCAS / OpcUaClient — that ingests **plain MQTT** and **Sparkplug B** into the OtOpcUa OPC UA / UNS address space. + +- OtOpcUa is a **Unified-Namespace product**, and **Sparkplug B is the de-facto UNS-over-MQTT payload standard** (mandated topic namespace + protobuf schema + birth-certificate discovery + stateful sessions). Sparkplug alignment is therefore the strategic headline; **plain MQTT/JSON** is the broad-compatibility fallback for brokers that carry raw JSON topics. +- This is a **subscribe-first** driver. Unlike the polled drivers (Modbus/S7/AbCip/FOCAS with `PollGroupEngine`), it holds a **live broker connection** for its whole lifetime and raises `OnDataChange` from the MQTT receive callback. The closest existing analog is the native-push side of `OpcUaClientDriver`. +- **In scope (this design):** connect/TLS/auth/reconnect; topic + Sparkplug subscribe → `OnDataChange`; last-value `IReadable`; authored-only `ITagDiscovery`; `IRediscoverable`; `IHostConnectivityProbe`; a **bespoke observation browser**; typed AdminUI tag editor + validator; Docker fixtures + env-gated live suite. +- **Out of scope for v1:** write-through (`IWritable` — Sparkplug NCMD/DCMD + plain publish) is deferred to phase 3 (§3.4, §10); `DataSet`/`Template` Sparkplug metrics; `Bytes`/`File` metrics beyond a raw-String fallback. + +Canonical **DriverType string: `"Mqtt"`** — one string used identically across the AdminUI driver page, probe, factory, editor map, validator, and browser. (Heed the Modbus `"ModbusTcp"` vs `"Modbus"` mismatch lesson: a divergent string silently drops tags to the raw-JSON editor / faults resolution. Pick `"Mqtt"` and grep to enforce it.) + +--- + +## 2. Project layout + +Three new projects under `src/Drivers/`, mirroring the OpcUaClient triad (`.Driver` + `.Contracts` + `.Browser`). All `net10.0`, `Nullable`+`ImplicitUsings` enabled, `TreatWarningsAsErrors=true` opted in per-csproj (mandatory for new projects — `Directory.Build.props` deliberately does **not** set it globally because legacy test projects would fail the build). + +``` +src/Drivers/ + ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ # options + tag DTOs + parser — zero transport deps + MqttDriverOptions.cs # driver-config DTO (broker conn + mode + sparkplug/plain sub-objects) + MqttMode.cs, MqttPayloadFormat.cs # enums (Plain|SparkplugB ; Json|Raw|Scalar) + MqttTagDefinition.cs # parsed per-tag descriptor (plain OR sparkplug variant) + MqttEquipmentTagParser.cs # TagConfig JSON → MqttTagDefinition (mirrors ModbusEquipmentTagParser) + SparkplugDataType.cs # Sparkplug metric-datatype enum + → DriverDataType map + + ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ # runtime driver — references .Contracts + Core.Abstractions + Core + MqttDriver.cs # IDriver, ITagDiscovery, ISubscribable, IReadable, + # IHostConnectivityProbe, IRediscoverable (+ IWritable phase 3) + MqttDriverProbe.cs # IDriverProbe — broker CONNECT handshake + MqttDriverFactoryExtensions.cs # DriverTypeName="Mqtt"; Register(registry, loggerFactory) + MqttConnection.cs # MQTTnet client wrapper: connect/TLS/auth/reconnect/backoff + LastValueCache.cs # FullReference → last DataValueSnapshot (seeds IReadable) + Sparkplug/ + SparkplugCodec.cs # Tahu-proto decode (Payload/Metric); encode NCMD (phase 3) + AliasTable.cs # per edge-node/device alias→(name,datatype); rebuilt each birth + BirthCache.cs # NBIRTH/DBIRTH metric catalog (name/alias/datatype/last value) + SequenceTracker.cs # seq (0-255 wrap) gap detection; bdSeq death→birth tie + RebirthRequester.cs # publishes NCMD Node Control/Rebirth=true + SparkplugTopic.cs # parse/format spBv1.0/{group}/{type}/{node}[/{device}] + + ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser/ # bespoke observation browser — references .Contracts + Commons(.Browsing) + MQTTnet + MqttDriverBrowser.cs # IDriverBrowser (DriverType="Mqtt"); OpenAsync → MqttBrowseSession + MqttBrowseSession.cs # IBrowseSession over an accumulating observed tree + +src/Server/.../AdminUI/ + Uns/TagEditors/MqttTagConfigModel.cs # typed working model (FromJson/ToJson/Validate; preserves unknown keys) + Uns/TagEditors/TagConfigEditorMap.cs # + ["Mqtt"] = MqttTagConfigEditor + Uns/TagEditors/TagConfigValidator.cs # + ["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate() + Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor + EndpointRouteBuilderExtensions.cs # + AddSingleton() + +src/Server/.../Host/Drivers/DriverFactoryBootstrap.cs + # + MqttDriverFactoryExtensions.Register(registry, loggerFactory) + # + TryAddEnumerable(IDriverProbe, MqttDriverProbe) + +tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests/ # unit — codec/alias/seq/parser (net10, macOS-safe) +tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/ # env-gated live suite (MQTT_FIXTURE_ENDPOINT) +tests/Drivers/.../Docker/docker-compose.yml # mosquitto + sparkplug-sim + json-publisher (project=lmxopcua) +``` + +**Why the split `.Contracts` project:** it carries `MqttDriverOptions` + `MqttTagDefinition` + `MqttEquipmentTagParser` + the enums with **zero transport (MQTTnet) dependency**, exactly like `Driver.Modbus.Contracts` depends only on `Modbus.Addressing` + `Core.Abstractions`. This lets the AdminUI typed editor (and any tooling) reference the DTO shape/enums without dragging MQTTnet into AdminUI. The runtime `.Driver` and `.Browser` both reference `.Contracts` and add MQTTnet. + +### 2.1 Libraries + the pinning caveat + +| Concern | Choice | Package | +|---|---|---| +| **MQTT transport (both modes)** | **MQTTnet v5** — MIT, **.NET Foundation** (`dotnet/MQTTnet`), the standard .NET MQTT client. v5 explicitly ships a `net10.0` TFM. Client + TLS + MQTT 3.1.1/5.0. | `MQTTnet` (add `PackageVersion Include="MQTTnet" Version="5.x"` to `Directory.Packages.props`) | +| **Sparkplug decode** | **Hand-rolled Tahu protobuf** (recommended) — generate C# from Tahu's `sparkplug_b.proto` with `Google.Protobuf` (`Grpc.Tools`/`protoc`), decode over a single MQTTnet-5 client. Small, stable schema; we own the alias/seq/rebirth state machine exactly as we want it. | `Google.Protobuf` + `Grpc.Tools` (build-time) | +| **Alternative Sparkplug decode** | **SparkplugNet** — MIT, provides `Application`/`Node`/`Device` base classes, saves the birth/alias plumbing. **Rejected as primary** for two repo-specific reasons (below). | `SparkplugNet` (only if the risk clears) | + +**Pinning caveat (load-bearing — this is why hand-roll is recommended).** This repo uses **central package management** (`Directory.Packages.props`, `ManagePackageVersionsCentrally=true`) and deliberately **leaves `CentralPackageTransitivePinningEnabled` OFF** because turning it on breaks the Roslyn 5.0.0/4.12.0 split (see the "Transitive pinning breaks Roslyn build" memory + `Directory.Packages.props:103` comment). SparkplugNet **pins MQTTnet 4.3.x transitively and has no explicit `net10.0` TFM** yet. A direct MQTTnet-5 reference (needed for the plain-mode transport + net10) will **coexist in the graph with SparkplugNet's transitive 4.x**, and because transitive pinning is off we cannot cleanly force-align the two. Net: SparkplugNet risks a hard-to-resolve MQTTnet 4/5 diamond in exactly the pinning configuration this repo can't change. + +**Decision: hand-roll Tahu-proto decode over one MQTTnet-5 client.** It removes the diamond and the net10-TFM concern entirely, and the Sparkplug payload schema (`Payload { seq, timestamp, Metric[], body }`, `Metric { name, alias, timestamp, datatype, value, is_null, ... }`) is ~90 lines of `.proto` — a bounded, well-understood effort. Phase 1 (plain MQTT) references **only MQTTnet-5**, so we validate MQTTnet-v5-on-net10 before ever touching Sparkplug. (If a future need for SparkplugNet arises, the phase-2 codec seam `SparkplugCodec` is the single swap point.) + +**Vendor the proto:** commit Tahu `sparkplug_b.proto` into `Driver.Mqtt.Contracts/Protos/` (pinned to a known Tahu commit, with a provenance comment) and generate at build via `Grpc.Tools`. Don't fetch at build time. + +**Existing protobuf tooling, honestly:** `Google.Protobuf` is **already centrally pinned** (`Directory.Packages.props:31`, 3.34.1) and consumed by `Driver.Galaxy` — but the Galaxy proto types ship **pre-generated** inside the `ZB.MOM.WW.MxGateway.*` NuGets; there is **no `.proto` file or `Grpc.Tools` protoc codegen anywhere in this repo today**. So `Grpc.Tools` is a new (build-time-only, `PrivateAssets=all`) package and this is the repo's first in-repo proto codegen step. If that step proves objectionable, the fallback is checking in the generated C# (the schema is small and stable) with the same provenance comment. + +--- + +## 3. Capability mapping + +`MqttDriver : IDriver, ITagDiscovery, ISubscribable, IReadable, IHostConnectivityProbe, IRediscoverable` (+ `IWritable` in phase 3). **Not** `IHistoryProvider` (historization is the server-side `isHistorized` path, unchanged). **Not** a poller — no `PollGroupEngine`. + +### 3.0 Lifecycle (`IDriver`) + +- `InitializeAsync(configJson, ct)` — deserialize `MqttDriverOptions` (shared `JsonSerializerOptions` with `JsonStringEnumConverter`, §6); build `MqttConnection`; connect to broker under a bounded connect deadline (§8); set LWT/clean-session; subscribe to the mode-appropriate filter (§3.1). Publish `STATE` birth if `actAsPrimaryHost` (Sparkplug). +- `ReinitializeAsync` — apply a config delta in place: if only tag membership changed, re-subscribe; if broker connection changed, tear down + reconnect. Never crash the process (marks Faulted → nodes go Bad, per driver-stability Tier A/B). +- `ShutdownAsync` — publish `STATE OFFLINE` (if primary host), disconnect the MQTTnet client, dispose. +- `GetHealth` — Connected/Reconnecting/Faulted + last-message-age; drives ServiceLevel + status dashboard. +- `GetMemoryFootprint`/`FlushOptionalCachesAsync` — footprint = last-value cache + birth caches + alias tables. The runtime driver has **no optional caches to flush**: birth/alias tables are required-for-correctness (flushing them mis-routes aliased DDATA — `IDriver.FlushOptionalCachesAsync` forbids flushing correctness state) and the last-value cache is what `IReadable` serves. Flush is a no-op; all three structures are bounded by the authored tag set + observed births. (The observation browser's accumulating tree lives in the AdminUI-side `MqttBrowseSession`, not in the runtime driver.) + +### 3.1 Subscribe (`ISubscribable` — primary) + +- **Plain mode:** subscribe to each authored tag's `topic` at its `qos` (dedupe identical topics; coalesce shared wildcards where the operator authored one). On message: match topic → authored tag(s), extract value at `jsonPath` (or raw/scalar), raise `OnDataChange(handle, FullReference, snapshot)`. Seed initial value from the broker's **retained** message replayed on subscribe (OPC UA initial-data convention). +- **Sparkplug mode:** subscribe `spBv1.0/{groupId}/#` (+ the host-state topic when acting as primary host — **`spBv1.0/STATE/{hostId}` with a JSON `{online, timestamp}` payload in spec v3.0**; pre-3.0 peers used top-level `STATE/{hostId}` with `ONLINE`/`OFFLINE` text — target the v3.0 form, tolerate the legacy one on receive). Maintain a **`BirthCache` + `AliasTable` per (edgeNode, device)** from NBIRTH/DBIRTH. On NDATA/DDATA: resolve each metric by **alias → (name, datatype)** via the alias table, map to the authored tag's `FullReference` **by stable metric name**, raise `OnDataChange`. On NDEATH/DDEATH: emit **STALE / Bad-quality** snapshots for that node's/device's metrics. On missed birth / unknown alias / seq gap: issue a **rebirth NCMD** (§3.6). +- `publishingInterval` is **advisory** (MQTT/Sparkplug are event-driven). Use it only for optional server-side coalescing/deadband, mirroring Modbus's `ShouldPublish`. `SubscribeAsync` returns an `ISubscriptionHandle` whose `DiagnosticId` names the mode+filter; `OnDataChange` fires from the MQTT receive handler for the whole driver lifetime. + +`SubscribeAsync` maps the requested `FullReference`s onto the live receive stream: it registers them in a `FullReference → MqttTagDefinition` resolver (parsed once, cached — reuse the shared `EquipmentTagRefResolver` from `Core.Abstractions`, exactly as Modbus does), so an inbound message/metric routes to the right node. + +### 3.2 Read (`IReadable`) + +MQTT has no request/response read. `ReadAsync(fullReferences)` returns the **last-value cache** (`LastValueCache`): last message seen on the topic (plain) or last metric value from data/birth (Sparkplug). Seed from the retained message at connect (plain) and from birth values (Sparkplug). If nothing has been observed yet, return a `GoodNoData`/uncertain snapshot **per reference** (never throw the batch — per `IReadable` contract, per-reference status is the snapshot's `StatusCode`). This satisfies OPC UA reads + HistoryRead-AtTime without a live round-trip. + +### 3.3 Discover (`ITagDiscovery`) — authored-only + +**`DiscoverAsync` replays authored tags only, both modes** — it does **not** auto-provision from wildcard/birth observation. Rationale (from research §2.2): a chatty broker or a large Sparkplug plant would produce an unbounded auto-provisioned address space. Wildcard/birth observation is a **browse-time** concern (§4), not a runtime-provisioning one. + +- The operator authors tags (topic+jsonPath, or group/node/device/metric) via the typed editor / observation browser. `DiscoverAsync` streams exactly those authored tags into the `IAddressSpaceBuilder`, each as a `Variable(browseName, displayName, DriverAttributeInfo{ FullName, DriverDataType, ... })`. +- **`RediscoverPolicy`:** + - **Plain:** `Once` (authored set is static per published config — like Modbus). + - **Sparkplug:** `UntilStable` (metric *datatypes* fill in asynchronously as births arrive after connect; keep re-streaming until the authored tags' resolved datatypes stop changing — like FOCAS's FixedTree). This lets a tag authored before its first birth pick up its true datatype once the birth lands. +- **`SupportsOnlineDiscovery => false`** (the default-member from the universal-browser design). This is deliberate and **decoupled from browseability** (§4): runtime discovery is authored-only, so the universal Discover-backed browser would only replay authored tags — useless for discovering *new* topics/metrics. Leaving the flag false keeps MQTT off the universal fallback; the bespoke observation browser (§4) is what backs the picker. + +### 3.4 `IRediscoverable` + +Sparkplug fires `OnRediscoveryNeeded` when a **new DBIRTH** (a device joins) or a **rebirth with a changed metric set** is observed — the Core re-runs `DiscoverAsync` + diffs. `ScopeHint` = the edge-node/device folder path so the rebuild is surgical. Plain mode never fires it (no backend change signal — a config republish is the only tag-set change), so plain-only drivers effectively no-op this interface. + +### 3.5 Data-type mapping + +Sparkplug metric datatype (or inferred/authored plain type) → `DriverDataType` (`Boolean, Int16, Int32, Int64, UInt16, UInt32, UInt64, Float32, Float64, String, DateTime, Reference`): + +| Sparkplug `DataType` | `DriverDataType` | OPC UA | +|---|---|---| +| Int8, Int16 | `Int16` (Int8 widens) | Int16 | +| Int32 | `Int32` | Int32 | +| UInt8, UInt16 | `UInt16` (UInt8 widens) | UInt16 | +| UInt32 | `UInt32` | UInt32 | +| Int64 | `Int64` | Int64 | +| UInt64 | `UInt64` | UInt64 | +| Float | `Float32` | Float | +| Double | `Float64` | Double | +| Boolean | `Boolean` | Boolean | +| String, Text, UUID | `String` | String | +| DateTime | `DateTime` | DateTime | +| Bytes, File | `String` (base64/raw fallback, v1) | String | +| DataSet, Template | **unsupported v1** — skip + warn, or expose raw-JSON as `String` | — | +| `*Array` variants | element type + `IsArray=true`, `ArrayDim` from birth | ValueRank=1 | + +Live map in `SparkplugDataType.ToDriverDataType()` (`.Contracts`). **Plain mode:** honour an explicit per-tag `dataType` (preferred — inference is brittle); fall back to inferring from the JSON token at `jsonPath` (number→`Float64`/`Int64`, bool→`Boolean`, string→`String`). Int8/UInt8 widen (no OPC UA signed-byte distinction needed). + +### 3.6 Sparkplug state-machine correctness (the #1 risk) + +The alias→metric map, seq-gap detection, late-join rebirth, and death→STALE are load-bearing and easy to get subtly wrong. **Invariants the implementation must hold:** + +1. **Bind tags by stable metric NAME, never by alias.** The authored `TagConfig` stores `metricName`; the alias is a **per-birth cache only**. An alias can be reused across a rebirth for a *different* metric — binding by alias silently mis-routes data to the wrong tag. +2. **Rebuild the alias table on every (re)birth.** NBIRTH/DBIRTH replaces the `(edgeNode,device)` `AliasTable` wholesale; never merge. +3. **Track `seq` (0–255, wraps) per edge-node.** A gap (or an unknown alias, or a data message before any birth) ⇒ request rebirth (`RebirthRequester` publishes `NCMD` writing `Node Control/Rebirth = true`), gated on `requestRebirthOnGap`. `bdSeq` ties an NDEATH to its NBIRTH. +4. **Death → STALE.** NDEATH/DDEATH (LWT-driven) emits Bad/uncertain-quality snapshots for all affected metrics; the next birth restores Good. +5. **Late join.** On connect the driver may have missed births — proactively request rebirth (or wait for the retained STATE + natural rebirth) so metadata is recovered. This is the same primitive the browser uses (§4). + +Unit-test these explicitly against a controllable simulator: rebirth, missed-birth, seq-gap, alias-reuse-across-rebirth, death→stale→rebirth. + +### 3.7 Write (`IWritable`) — **verdict: NOT in v1; phase 3, fire-and-forget optimistic-Good** + +Writes are possible but deferred (research §2.4). When shipped in phase 3: +- **Sparkplug:** publish `NCMD`/`DCMD` to `spBv1.0/{group}/NCMD/{node}[/{device}]` with the target metric (by name+alias) + value. **No synchronous write ack** — success is only observable when the edge echoes the change in the next N/DDATA. This mirrors the **Galaxy gateway's fire-and-forget write** ("Galaxy can never surface a write failure"): return `Good` optimistically; the value self-corrects on the next data message. Respect `WriteIdempotent` — Sparkplug commands (rebirth, pulse) are frequently **non-idempotent**, so default retry off. +- **Plain:** publish to a configured command/write topic with a formatted payload. + +v1 ships **read/subscribe/discover only** — a genuinely useful UNS-ingest driver. `MqttDriver` does **not** implement `IWritable` in v1 (the interface is composable; its absence means nodes materialize read-only). + +--- + +## 4. Bespoke observation browser + +### Decision — reconciling with the universal Discover-backed browser + +The universal browser (`docs/plans/2026-07-15-universal-discovery-browser-design.md`) captures whatever `DiscoverAsync` streams and re-presents it, gated on `SupportsOnlineDiscovery`. **MQTT/Sparkplug is the one roadmap driver that needs a bespoke browser despite the universal one**, because: + +- Runtime `DiscoverAsync` is **authored-only** (§3.3). A universal Discover-backed browser would replay only tags the operator already authored — it would **not discover new topics/metrics**, which is the entire point of a picker. +- MQTT/Sparkplug discovery is **observational**: you learn what exists only by *listening* for a bounded window. That's a fundamentally different session shape than "connect → one-shot enumerate → disconnect" — the tree **fills in over time**, and Sparkplug can be *accelerated* by forcing a rebirth NCMD. + +**Considered + rejected:** feeding the Sparkplug `BirthCache` into `DiscoverAsync` device-enumeration to plug into universal. Rejected because it would make runtime discovery auto-provision from births (the unbounded-address-space failure §3.3 explicitly avoids) and still wouldn't cover plain MQTT. The birth cache stays a browse-time + subscribe-time structure, not a discovery source. + +**Decision:** keep `SupportsOnlineDiscovery=false` (authored-only runtime discovery) **and** build a bespoke `IDriverBrowser` (`MqttDriverBrowser`) with an **observation-window `IBrowseSession`** (`MqttBrowseSession`). Registering the bespoke browser for `DriverType="Mqtt"` **overrides the universal fallback** for MQTT (the universal browser resolves bespoke-first — `BrowserSessionService` §3.1 of that doc). + +### 4.1 `MqttDriverBrowser : IDriverBrowser` + +`DriverType => "Mqtt"`. `OpenAsync(configJson, ct)`: +1. Parse the **same `MqttDriverOptions` JSON** the runtime driver consumes (shared `JsonSerializerOptions`), but connect with a **distinct client-id suffix** (`{clientId}-browse-{guid8}`) so the browse session never collides with the running driver's MQTT session / clean-session state. +2. Connect an MQTTnet client under a bounded connect budget (clamp like OpcUaClient's `PerEndpointConnectTimeout`, 5–30 s). +3. Subscribe to the discovery filter: `spBv1.0/{groupId}/#` (Sparkplug) or `#`/`{topicPrefix}#` (plain). +4. **Kick off the observation window immediately** and, in Sparkplug mode, optionally publish a **rebirth NCMD** to force every edge node to re-announce (turns the passive wait into a near-synchronous enumeration). +5. Return an `MqttBrowseSession` whose internal tree **fills in over the window and keeps filling** while the session lives. Because `IBrowseSession` methods are async and the session lands in the AdminUI `BrowseSessionRegistry` (TTL-reaped, per-call 20 s timeout), the observation cost is amortised across the user's clicks in the picker. + +### 4.2 `MqttBrowseSession : IBrowseSession` + +Serves from an **accumulating observed tree** (thread-safe; the MQTT receive handler mutates it, browse calls read it). Refreshes `LastUsedUtc` on each call. + +- **`RootAsync`** — Sparkplug: observed **groups** (or edge nodes under the configured group). Plain: top-level topic segments. `BrowseNode { Kind=Folder, HasChildrenHint=true }`. If the window hasn't yielded yet, return what's accumulated (possibly empty) and let the user re-expand; in Sparkplug mode trigger a rebirth on first `RootAsync` to populate fast. +- **`ExpandAsync(nodeId)`** — one level down the accumulated tree. Sparkplug: **Group → EdgeNode → Device → Metric**. Plain: next topic segment (split on `/`). Metrics / leaf-topics are `Kind=Leaf`. +- **`AttributesAsync(nodeId)`** — Sparkplug: the metric's `AttributeInfo { Name, DriverDataType (from birth), IsArray, SecurityClass }` — **self-describing**, so the picker side-panel is populated (unlike the OPC UA browser which returns empty). Plain: a single synthetic attribute carrying the leaf topic's **inferred type + last-seen payload snippet**. +- **`DisposeAsync`** — disconnect the MQTTnet client, best-effort (registry reaper may race), same pattern as `OpcUaClientBrowseSession`. + +### 4.3 Picked node → TagConfig + +On commit the picker projects the selected `BrowseNode`/`AttributeInfo` into the §5 TagConfig JSON: +- **Sparkplug:** the `NodeId` encodes `group/node/device:metric` → split into `groupId`/`edgeNodeId`/`deviceId`/`metricName`; `dataType` from `AttributeInfo`. +- **Plain:** the `NodeId` is the topic path → `topic` + a default `jsonPath` (`$`, operator-editable) + inferred `dataType`. + +### 4.4 Addressing the passive/time-bounded wrinkle explicitly + +1. Picker shows a live **"listening… N nodes/topics discovered"** affordance so the operator understands coverage is accumulating, not instantaneous. +2. Sparkplug mode **actively forces a rebirth** to convert the passive wait into a fast near-complete enumeration. +3. The browse session stays alive (registry TTL) so re-expanding later reflects newly observed topics without reconnecting. +4. **Manual entry** of a topic/metric via the typed editor (§6) is always available as an escape hatch for a tag that's silent during the window. + +### 4.5 Registration + +```csharp +// EndpointRouteBuilderExtensions.cs — alongside the existing two (near line 49-50) +services.AddSingleton(); +services.AddSingleton(); +services.AddSingleton(); // NEW — overrides universal fallback for "Mqtt" +``` + +--- + +## 5. TagConfig + driver-config JSON + +Two layers, mirroring every other driver: **driver options** (broker connection, one per driver instance, parsed by the factory) and **per-tag config** (`TagConfig` JSON authored on the `/uns` Equipment → Tags tab; the driver's `FullName` is the driver-side reference the router resolves). + +### 5.1 Driver options — `MqttDriverOptions` (`.Contracts`) + +```jsonc +{ + "host": "10.100.0.35", + "port": 8883, + "clientId": "otopcua-uns-1", + "useTls": true, + "allowUntrustedServerCertificate": false, // dev/on-prem only + "caCertificatePath": null, // PEM CA pin; null ⇒ OS trust store + "username": "otopcua", + "password": "", // supply via env, NEVER commit + "protocolVersion": "V500", // "V311" | "V500" + "cleanSession": true, + "keepAliveSeconds": 30, + "connectTimeoutSeconds": 15, // bounded connect deadline (§8) + "reconnectMinBackoffSeconds": 1, + "reconnectMaxBackoffSeconds": 30, + + "mode": "SparkplugB", // "Plain" | "SparkplugB" + + // ---- Sparkplug-only (null in Plain mode) ---- + "sparkplug": { + "groupId": "Plant1", // subscribe spBv1.0/Plant1/# + "hostId": "otopcua-host-1", // spBv1.0/STATE/{hostId} identity (v3.0) + "actAsPrimaryHost": false, // at most ONE primary host per host-id per broker + "requestRebirthOnGap": true, + "birthObservationWindowSeconds": 15 // browse/discovery: how long to collect births + }, + + // ---- Plain-only (null in SparkplugB mode) ---- + "plain": { + "topicPrefix": "factory/", + "defaultQos": 1 + } +} +``` + +### 5.2 Per-tag — Sparkplug + +```jsonc +{ + "FullName": "Plant1/Line3EdgeNode/Filler1:Temperature/degC", // PascalCase key (composer/walker contract) + "groupId": "Plant1", + "edgeNodeId": "Line3EdgeNode", + "deviceId": "Filler1", // omit/null for node-level metrics + "metricName": "Temperature/degC", // STABLE identity across rebirths — bind by this, not alias + "dataType": "Float" // from DBIRTH; explicit for safety +} +``` + +`FullName` convention: `"{group}/{node}[/{device}]:{metric}"`. The driver parses `FullName` (or the descriptor fields) once, caches it, and resolves inbound metrics by **`(group,node,device,metricName)`**. + +### 5.3 Per-tag — Plain MQTT + +```jsonc +{ + "FullName": "factory/line3/oven/temp#$.value", // topic#jsonPath — stable per-tag key + "topic": "factory/line3/oven/temp", + "payloadFormat": "Json", // "Json" | "Raw" | "Scalar" + "jsonPath": "$.value", // JSONPath into payload (Json only) + "dataType": "Double", // explicit — inference is the fallback + "qos": 1, + "retainSeed": true // seed last-value from retained msg on subscribe +} +``` + +`FullName` convention: `"{topic}#{jsonPath}"` (Json) or just `"{topic}"` (Raw/Scalar). `MqttEquipmentTagParser.TryParse(reference, out def)` follows `ModbusEquipmentTagParser`: a leading `{` marks a TagConfig blob; parse it, use **strict enum reads** (`TagConfigJson.TryReadEnumStrict`) so a typo'd `payloadFormat`/`dataType` rejects the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a wrong type; the def's `Name` = the reference string so published values key the forward router. Add an `Inspect(reference)` deploy-time warning pass (like Modbus) for present-but-invalid enums / unparseable blobs. + +**Historization:** the server-side `isHistorized` / `historianTagname` keys are owned by the TagModal-merge seam (`TagHistorizeConfig`) and survive load→save of the typed model as preserved unknown keys — the driver does not model them (identical to `OpcUaClientTagConfigModel`). + +--- + +## 6. Typed editor + validator + +### 6.1 `MqttTagConfigModel` (`.AdminUI/Uns/TagEditors/`) + +Thin typed working model over a preserved `JsonObject` key bag, per the `OpcUaClientTagConfigModel` template: `FromJson(json)` / `ToJson()` / `Validate()`, **preserving unknown keys** across a load→save (so history keys + fields this editor doesn't expose survive). It carries **both** mode shapes — a `Mode` field (Plain|SparkplugB, defaulted from which fields are present) selects which sub-shape the editor renders + validates. + +- `FromJson` reads the descriptor fields for the active mode plus the always-present `FullName` (PascalCase, case-sensitive — the composer/artifact-decoder/walker read it via case-sensitive `TryGetProperty("FullName", …)`). +- `ToJson` writes `FullName` PascalCase and re-derives it from the descriptor fields (`{group}/{node}[/{device}]:{metric}` or `{topic}#{jsonPath}`) so a hand-edited descriptor stays consistent with the routed key. +- `Validate()`: + - **Sparkplug:** `groupId`, `edgeNodeId`, `metricName` required; `deviceId` optional; `dataType` must be a known Sparkplug type. + - **Plain:** `topic` required and non-wildcard (a subscription topic for a *tag* must be concrete — reject `+`/`#`); `payloadFormat=Json` ⇒ `jsonPath` required; `dataType` a known `DriverDataType`. + +### 6.2 `MqttTagConfigEditor.razor` + +Copy the Modbus editor template (`Components/Shared/Uns/TagEditors/`): a razor shell over the pure model, mode-switch at top, per-mode field group below, client-side validation via `Validate()`. Register: + +```csharp +// TagConfigEditorMap.cs +["Mqtt"] = typeof(Components.Shared.Uns.TagEditors.MqttTagConfigEditor), +// TagConfigValidator.cs +["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate(), +``` + +### 6.3 The `JsonStringEnumConverter` enum-serialization trap + +**Every** JSON seam that (de)serializes `MqttDriverOptions` / tag configs — factory, probe, browser, **and the AdminUI driver-config page + probe DTO** — MUST use a `JsonSerializerOptions` that includes `new JsonStringEnumConverter()` (plus `PropertyNameCaseInsensitive=true`, `UnmappedMemberHandling=Skip`). The repo's SYSTEMIC bug (memory: "Driver enum-serialization bug"): AdminUI pages that serialize enums **numerically** while factory DTOs are **string-typed** fault the driver on any enum field. `MqttMode`, `MqttPayloadFormat`, `protocolVersion` are all enums — the AdminUI Mqtt driver page and its probe must serialize them as **strings** (mirror the OpcUaClient page). `JsonStringEnumConverter` also accepts numeric ordinals, so existing numeric configs still load. Keep one shared `JsonSerializerOptions` definition referenced by factory + probe + browser (as OpcUaClient does). + +--- + +## 7. Factory + registration + +`MqttDriverFactoryExtensions` (mirror `OpcUaClientDriverFactoryExtensions`): + +```csharp +public static class MqttDriverFactoryExtensions +{ + public const string DriverTypeName = "Mqtt"; + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip, + Converters = { new JsonStringEnumConverter() }, + }; + + public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null) + { + ArgumentNullException.ThrowIfNull(registry); + registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory)); + } + + public static MqttDriver CreateInstance(string driverInstanceId, string driverConfigJson, ILoggerFactory? lf = null) + { + var options = JsonSerializer.Deserialize(driverConfigJson, JsonOptions) + ?? throw new InvalidOperationException($"Mqtt driver config for '{driverInstanceId}' deserialised to null"); + return new MqttDriver(options, driverInstanceId, lf?.CreateLogger()); + } +} +``` + +Wire in the two host bootstrap sites: + +```csharp +// Host/Drivers/DriverFactoryBootstrap.cs — factory registration (near line 133) +Driver.Mqtt.MqttDriverFactoryExtensions.Register(registry, loggerFactory); +// … and probe registration in AddOtOpcUaDriverProbes (near line 114) +services.TryAddEnumerable(ServiceDescriptor.Singleton()); +``` + +**`MqttDriverProbe : IDriverProbe`** (`DriverType => "Mqtt"`): parse the config with the **shared** `JsonSerializerOptions`, open an MQTTnet CONNECT to `host:port` under the timeout, return green + latency on CONNACK-accepted, or a targeted error (connection refused / TLS failure / auth failure / timeout). Analogous to `ModbusDriverProbe`'s TCP+FC03 handshake — CONNACK is the MQTT "device is answering" proof. **Probes are admin-pinned** (the `admin-operations` singleton dispatches by DriverType) — the `TryAddEnumerable` line must be on admin nodes (it is, via `AddOtOpcUaDriverProbes` in `Program.cs`'s `hasAdmin` block). + +--- + +## 8. Resilience / timeout / security + +- **Bounded connect.** `InitializeAsync` connects under `connectTimeoutSeconds` (default 15) via a linked CTS — **no unbounded wait** on a dead broker (heed the S7/FOCAS read-timeout lessons: an async network call that ignores its deadline wedges the whole driver). +- **Reconnect with backoff.** MQTTnet's managed reconnect (or a hand-rolled loop) with exponential backoff `reconnectMinBackoffSeconds → reconnectMaxBackoffSeconds`. On disconnect: `GetHealth` → Reconnecting; on reconnect: re-subscribe, and in Sparkplug mode **request rebirth** to recover metadata (late-join, §3.6). Nodes go Bad-quality while disconnected via the standard fan-out. +- **Subscribe deadline.** `SubscribeAsync` completes under a bounded deadline; SUBACK failure surfaces as a per-reference Bad, not a hang. +- **No poll loop** — the receive handler must be non-blocking (offload heavy decode off the MQTT callback thread; never block the MQTTnet dispatcher). +- **Broker security — enforce, never ship public-broker defaults.** Default `useTls=true`; real auth (username/password or client-cert). `allowUntrustedServerCertificate` + `caCertificatePath` mirror the ServerHistorian TLS knobs (dev/on-prem escape hatch, off by default). **Never** put Sparkplug production-shaped data on public brokers (`test.mosquitto.org` etc. — manual smoke only). Credentials come from env (`password` blank in committed JSON), same posture as `ServerHistorian__ApiKey`. +- **Primary-host guard.** At most **one** primary-host client per `hostId` per broker (Sparkplug MUST). `actAsPrimaryHost=true` on two OtOpcUa instances sharing a `hostId` (e.g. a redundancy pair) is a misconfiguration — the redundancy pair should use **distinct** host-ids (mirrors the distinct `MxAccess.ClientName` redundancy rule). + +--- + +## 9. Test fixtures + +Follow the driver-fixture pattern: `tests/Drivers/.../Docker/docker-compose.yml` with the `project: lmxopcua` label, deployed to `10.100.0.35` via `lmxopcua-fix sync/up` (`CLAUDE.md` Docker Workflow). Repo files are source-of-truth; `/opt/otopcua-mqtt/` is the mirrored deployment. + +- **Broker:** **Eclipse Mosquitto** (`eclipse-mosquitto`, tiny, carries both plain + Sparkplug unchanged) as the default fixture; **EMQX** (`emqx/emqx`, Apache-2.0 CE) as the alternative when a dashboard / built-in Sparkplug tooling helps. Configure with **auth + TLS** so the fixtures exercise the real security path (not anonymous). +- **Sparkplug edge-node simulator:** prefer a **project-owned C# simulator** using the same MQTTnet + Tahu-proto path the driver uses — validates encode/decode symmetrically and gives full control of rebirth/seq-gap/death scenarios (the §3.6 test matrix). Alternatives: Eclipse Tahu reference (Java/Python) edge-node script; Bevywise MQTT/IoT Simulator (purpose-built Sparkplug B). The simulator must publish NBIRTH/DBIRTH → periodic NDATA/DDATA, honour a rebirth NCMD, and be able to inject a seq gap / alias reuse / N-D-DEATH on demand. +- **Plain-MQTT fixture:** a tiny publisher container (`mosquitto_pub` loop or a `paho-mqtt` script) emitting JSON on a handful of topics with `retain=true` so the last-value/read path **and** the `#`-observation browse are testable. +- **Env-gated live suite:** `Driver.Mqtt.IntegrationTests`, category `LiveIntegration` (à la HistorianGateway), skips cleanly when **`MQTT_FIXTURE_ENDPOINT`** is unset → macOS-offline-safe. Covers: connect/TLS/auth; plain subscribe+retained-read; Sparkplug birth→data→alias-resolve→`OnDataChange`; rebirth recovery; death→STALE; the browser observation window (birth-driven tree + rebirth-forced enumeration). +- **Live-verify (docker-dev):** author an Mqtt driver + tag via the `/uns` picker on `:9200`, deploy, confirm the node carries live broker values and the typed editor renders both modes (the usual live-`/run` discipline — Razor binding bugs pass unit tests). + +Add the fixture endpoint to `CLAUDE.md`'s endpoint list once landed (e.g. `10.100.0.35:1883`/`8883`, `MQTT_FIXTURE_ENDPOINT`). + +--- + +## 10. Phasing + effort + +Order: **plain MQTT first, then Sparkplug B, then write.** Plain MQTT is the dependency-light slice that stands up the whole skeleton and **validates MQTTnet-v5-on-net10 before any Sparkplug/proto work** — front-loading the reusable plumbing and de-risking the library decision. + +| Phase | Scope | Rel. effort | +|---|---|---| +| **P1 — Plain MQTT** | 3 projects; MQTTnet-5 connect/TLS/auth/reconnect+backoff (`MqttConnection`); `ISubscribable` (topic subscribe → `OnDataChange`); `IReadable` last-value (retained seed); authored-tag `ITagDiscovery` (`Once`); `IHostConnectivityProbe`; `MqttDriverProbe`; factory + host wiring; `#`-observation browser (`MqttDriverBrowser`/`MqttBrowseSession`); typed editor + validator + AdminUI driver page; Mosquitto + JSON-publisher fixtures + env-gated suite. **Validate MQTTnet-5/net10 here.** | **M** | +| **P2 — Sparkplug B ingest** | Vendored Tahu proto + `Google.Protobuf` codegen; `SparkplugCodec` decode; `spBv1.0/{group}/#` subscribe; N/DBIRTH → `BirthCache`/`AliasTable`; bind-by-name + seq-gap detect + **rebirth NCMD**; N/DDEATH → STALE; STATE/primary-host option; `ITagDiscovery` `UntilStable` + `IRediscoverable`; birth-driven browser tree + `AttributesAsync`; Sparkplug datatype map; C# edge-node simulator fixture + §3.6 test matrix. | **L** | +| **P3 — Write-through (optional)** | `IWritable`: Sparkplug NCMD/DCMD + plain publish; optimistic-Good + self-correct on echo; `WriteIdempotent` respect (default off for non-idempotent Sparkplug commands); write-topic config. | **M** | + +**Top risks:** (1) **Sparkplug state-machine correctness** (aliases + rebirth + seq — §3.6; bind-by-name is the mitigation, test against a controllable simulator); (2) **library/dependency friction** — SparkplugNet's MQTTnet-4.x transitive pin ✕ this repo's fragile central pinning (transitive pinning off), mitigated by **hand-rolling the Tahu proto over one MQTTnet-5 client** (§2.1). Secondary: passive-browse coverage gaps (rebirth-forcing + manual entry), unbounded address space (authored-only runtime discovery), write-has-no-ack (accepted, optimistic-Good like Galaxy), broker security (TLS + real auth enforced). diff --git a/docs/plans/2026-07-15-mtconnect-driver-design.md b/docs/plans/2026-07-15-mtconnect-driver-design.md new file mode 100644 index 00000000..fede66b2 --- /dev/null +++ b/docs/plans/2026-07-15-mtconnect-driver-design.md @@ -0,0 +1,182 @@ +# MTConnect (Agent-first) driver — executable implementation design + +**Status:** design, build-ready. 2026-07-15. +**Research input:** [`docs/research/drivers/mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) — this doc turns that research into a build spec; it does **not** re-argue the protocol findings. +**House-style references:** [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md), [`2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md). + +--- + +## 1. Motivation + scope + +MTConnect is the dominant open read-only telemetry standard for machine tools; adding a `DriverType = "MTConnect"` Equipment-kind driver lets OtOpcUa surface a machine's self-describing device model (positions, spindle speed, execution state, availability, conditions) alongside — and complementing — the lower-level **FOCAS** driver (FOCAS reaches the Fanuc CNC directly; MTConnect reaches the vendor-neutral Agent that often front-ends that same machine). v1 is **agent-first, read-only** (Discover + Read + Subscribe, no Write) because the mainstream MTConnect surface (`/probe`, `/current`, `/sample`) is read-only by design. Full rationale, protocol details, library survey, and risk register live in the [research report](../research/drivers/mtconnect-agent.md) §1–§8; this doc assumes them. + +## 2. Project layout + +Two new projects, mirroring the Modbus split (contracts DTOs isolated from runtime so the AdminUI probe/editor can reference config shapes without the driver's NuGet deps). **No `.Browser` project** — see §4. + +``` +src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ + MTConnectDriverOptions.cs // strongly-typed runtime options (like ModbusDriverOptions) + MTConnectTagDefinition.cs // one record per authored tag (FullName=dataItemId + mt* metadata) + MTConnectDataTypeInference.cs // pure category/type/units → DriverDataType table (§3.3) — shared by driver, factory, editor, browser-commit + ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj // refs Core.Abstractions only + +src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/ + MTConnectDriver.cs // IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable + MTConnectAgentClient.cs // thin seam over MTConnectHttpClient (probe/current/sample); testable via IMTConnectAgentClient + IMTConnectAgentClient.cs // seam interface — canned-XML fake in unit tests + MTConnectObservationIndex.cs // dataItemId → latest DataValueSnapshot, updated by /current + /sample + MTConnectDriverFactoryExtensions.cs // Register(registry, loggerFactory) + CreateInstance + MTConnectDriverProbe.cs // IDriverProbe: one-shot /probe reachability + ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj // refs Core, Core.Abstractions, .Contracts + TrakHound pkgs + +tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ + Fixtures/*.xml // canned probe/current/sample docs (§8) + ...Tests.cs +``` + +`.csproj` boilerplate copies Modbus verbatim: `net10.0`, `Nullable=enable`, `TreatWarningsAsErrors=true`, `GenerateDocumentationFile`, `InternalsVisibleTo` the Tests project. + +### NuGet dependency + +Consume **TrakHound MTConnect.NET client packages (MIT)** — pin an exact version: + +```xml + + + +``` + +These target `netstandard2.0` (load cleanly on .NET 10) and give `MTConnectHttpClient` (probe/current/sample with polling **and** the multipart long-poll stream, gzip, XML+JSON) plus the model types (`IDevice`/`IComponent`/`IDataItem`/`IObservation`) — removing the multipart-framing / version-negotiation / buffer-overflow-rebaseline grind. + +> **License-pin caveat (from research §1.4):** older TrakHound releases carried mixed MIT / Apache-2.0 / "all rights reserved" strings. Before merge, confirm the embedded `LICENSE` of the *pinned* version is MIT (a legal-review checkbox, not a blocker). **Hand-roll fallback** if review fails: `HttpClient` + `System.Xml.Linq` for probe/current, a `multipart/x-mixed-replace` boundary reader for sample (~300–500 LoC behind the same `IMTConnectAgentClient` seam — the driver above the seam is unchanged, so the fallback is a drop-in of one class). + +## 3. Capability mapping (concrete seam wiring) + +`MTConnectDriver` implements `IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable` — **not** `IWritable` (§3.5). One `MTConnectAgentClient` + one `MTConnectObservationIndex` per driver instance; one shared sample stream (the Agent streams the whole device model, so per-tag streams would be wasteful). + +### 3.1 `IDriver` + +- `InitializeAsync(json, ct)`: deserialize `MTConnectDriverOptions` (via factory), construct `MTConnectAgentClient` from `AgentUri`, run **one `/probe`** under a per-call deadline (§7). Cache the parsed `IDevice[]` model + `Header.instanceId`. Prime the `MTConnectObservationIndex` with one `/current`. Set `DriverState.Healthy` on success; `Faulted` (rethrow) on failure — the actor marks the instance Faulted, nodes go Bad, process stays up. +- `ReinitializeAsync`: tear down the sample stream, re-run Initialize. Config-only change (no address-space rebuild) unless `AgentUri`/`DeviceName` changed. +- `ShutdownAsync`: stop the sample stream, dispose the HTTP client. +- `GetHealth`: `DriverHealth(State, LastSuccessfulRead, LastError)` — `LastSuccessfulRead` = last `/current` or `/sample` chunk time. +- `GetMemoryFootprint` / `FlushOptionalCachesAsync`: footprint ≈ cached probe model + observation index; flush drops the browse/probe-model cache (re-fetchable), keeps the observation index (correctness state). + +### 3.2 `ITagDiscovery` — plugs into the universal browser (§4) + +```csharp +public bool SupportsOnlineDiscovery => true; // /probe enumerates from the device +public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once; // probe is synchronous + complete +public async Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken ct) +``` + +`DiscoverAsync` fetches (or reuses the Initialize-cached) `/probe` model and **streams the full Device→Component→DataItem tree into the builder**: + +- each `Device` → `builder.Folder(device.Name, device.Name)` → child builder; +- each nested `Component` → recursive `Folder(component.Name, component.DisplayName)` on the parent's child builder (component nesting becomes folder nesting — the builder-graph *is* the tree); +- each `DataItem` → `child.Variable(browseName, displayName, attr)` where + `attr = new DriverAttributeInfo(FullName: dataItem.Id, DriverDataType: MTConnectDataTypeInference.Infer(category, type, units), IsArray: representation==TIME_SERIES, ArrayDim: null, SecurityClass: SecurityClassification.ViewOnly, IsHistorized: false, IsAlarm: category==CONDITION)`. + (`ViewOnly` is the read-only-from-OPC-UA tier — `SecurityClassification` has no `ReadOnly` member.) + `browseName` = `dataItem.Name ?? dataItem.Id` (dataItem `name` is optional in MTConnect; fall back to `id`, which is guaranteed unique). + +**Critical:** `DriverAttributeInfo.FullName = dataItem.Id`. This is (a) the value the universal browser commits as `TagConfig.FullName`, and (b) the key the read/subscribe paths resolve against — the two align by construction (research §3). If `DeviceName` scopes to one device, only that device's subtree is streamed. + +`DeviceName` scoping and the whole-model-in-one-call nature make this a natural fit for **eager one-shot** discovery — exactly what the universal browser assumes. + +### 3.3 Data-type mapping (`MTConnectDataTypeInference.Infer`) + +Pure function in `.Contracts` (shared by driver, browser-commit, and the typed editor so all three agree). Weak wire typing means this is a heuristic, stored per-tag and **author-overridable** (§5): + +| MTConnect | `DriverDataType` | +|---|---| +| `SAMPLE` numeric (has `units`) | `Float64` | +| `SAMPLE` `representation=TIME_SERIES` | `Float64` array (`IsArray=true`) | +| `EVENT` numeric type (`PartCount`, `Line`, …) | `Int64` | +| `EVENT` controlled-vocab (`Execution`, `ControllerMode`, `Availability`, …) | `String` | +| `EVENT` free text (`Program`, `Block`, `Message`) | `String` | +| `CONDITION` | `String` (state word; §3.6) | + +**Quality mapping (`DataValueSnapshot.StatusCode`):** an observation value of `UNAVAILABLE` (MTConnect's explicit no-data sentinel) → OPC UA **`Bad`/`Uncertain`**, not the literal string. Missing dataItem / empty condition → `Bad`. `observation.timestamp` → `SourceTimestamp` (not a separate node). + +### 3.4 `IReadable` — `/current` + +`ReadAsync(fullReferences, ct)`: issue one `/current` under a per-call deadline, index the returned `MTConnectStreams` observations by `dataItemId` into `MTConnectObservationIndex`, then return **one `DataValueSnapshot` per requested ref in order**. A ref absent from the response → a `Bad`-coded snapshot (not a throw). Reads are idempotent (matches `IReadable` contract; `DriverCapability.Read` auto-retries). `/current` is the whole-device snapshot regardless of how many refs are requested, so batch reads cost one round-trip. + +### 3.5 `ISubscribable` — `/sample` multipart long-poll + +- `SubscribeAsync(fullReferences, publishingInterval, ct)`: on the **first** subscription, start the shared `MTConnectAgentClient` sample stream (`/sample?from=&interval=&count=`). Record the subscribed ref set. Immediately fire `OnDataChange` for each subscribed ref from the primed `/current` values (OPC UA initial-data convention). Return an `ISubscriptionHandle` (monotonic id + `DiagnosticId`). +- Stream pump: each received `MTConnectStreams` chunk → for each observation whose `dataItemId` ∈ subscribed set, update the index and raise `OnDataChange(handle, dataItemId, snapshot)`. Advance `from = Header.nextSequence` for the next request (contiguous, gap-free). +- **Ring-buffer overflow:** if the Agent reports a sequence gap / `from` older than `firstSequence`, re-`/current` to re-baseline, then resume the stream from the new `nextSequence`. (TrakHound handles this internally; the hand-roll fallback must replicate it — unit-tested via a fixture with a forced gap.) +- `UnsubscribeAsync(handle, ct)`: drop that handle's refs from the subscribed set; stop the shared stream when the set empties. +- `publishingInterval` maps to the `/sample` `interval` when it is the only/first subscription; the driver polls at the finest requested interval and fans out (one stream per instance, not per tag). + +### 3.6 `IWritable` — **not implemented (v1)** + +Justified per research §2.1: the MTConnect Agent surface is read-only by design; write-back exists only via optional, rarely-deployed MTConnect *Interfaces* (a request/response state-machine handshake, not a "set value" — modelling it as `IWritable` would mislead). Capability interfaces are composable, so omitting `IWritable` is idiomatic (Galaxy's write path is even fire-and-forget). Nodes materialize **without** the `AccessLevels.CurrentWrite` bit automatically. Revisit Interfaces only if a concrete deployment needs it. + +### 3.7 `IHostConnectivityProbe` + `IRediscoverable` + +- `IHostConnectivityProbe`: expose one `HostConnectivityStatus` per Agent (HostName = `AgentUri`). A cheap periodic `/probe` (or reuse the sample-stream heartbeat) flips `Running ↔ Stopped`; raise `OnHostStatusChanged` on transition. Enable/interval from `MTConnectDriverOptions.Probe` (mirrors `ModbusProbeOptions`). +- `IRediscoverable`: watch the Agent `Header.instanceId` on every `/current`/`/sample` chunk. A change means the Agent restarted / its model changed → raise `OnRediscoveryNeeded` so `DriverHost` rebuilds the address space (mirrors Galaxy's `DeployWatcher`). This is why `RediscoverPolicy = Once` is safe: instanceId change, not polling, drives re-discovery. + +### 3.8 CONDITION modelling + +**v1 simple (this design):** each `CONDITION` DataItem is a `String` variable whose value is the current state word (`Normal`/`Warning`/`Fault`/`Unavailable`), optionally suffixed with `nativeCode`. Zero alarm plumbing. `IsAlarm=true` is still set on the `DriverAttributeInfo` so the browser side-panel flags it and a future upgrade needn't re-author tags. +**v1.5 (fast-follow, not in this build):** implement `IAlarmSource` + emit a TagConfig `alarm` object so Fault/Warning become native OPC UA Part 9 alarms, routing on the authored dotted `ConditionId` per the Galaxy/Phase-B native-alarm pattern. + +## 4. Browse — plugs into the universal browser (reconciliation) + +**The research report (§4.2) proposed a bespoke `ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser` project with a hand-written `MTConnectBrowseSession`. This design supersedes that: build NO bespoke browser.** With the [universal discovery browser](2026-07-15-universal-discovery-browser-design.md) in place (the Wave-0 gate, which lands before this Wave-2 driver), MTConnect browse is covered for free by the generic `DiscoveryDriverBrowser`, because **MTConnect browse comes from discovery** (`/probe` → the device model → the same `DiscoverAsync` stream). Concretely, the universal browser already does exactly what the proposed bespoke session would have done: + +- `DiscoveryDriverBrowser.CanBrowse("MTConnect", cfg)` returns true because `TryCreate` succeeds and the instance is `ITagDiscovery { SupportsOnlineDiscovery: true }` (§3.2) → the AdminUI renders the **Browse** button. +- `OpenAsync` constructs the driver, `InitializeAsync` (the `/probe` connect), runs `DiscoverAsync` into a `CapturingAddressSpaceBuilder`, then `ShutdownAsync` — the captured tree (Device folders → Component folders → DataItem leaves) is served by `CapturedTreeBrowseSession`. Each leaf's `NodeId == DriverAttributeInfo.FullName == dataItemId`, committed directly as `TagConfig.FullName`; `IsAlarm`/`DriverDataType` flow to the side-panel and pre-fill the typed editor. + +**No `PatchForBrowse` entry is needed** — MTConnect discovery is unconditional (not config-gated like AbCip/TwinCAT's `EnableControllerBrowse`); it belongs in the "no patch" set alongside OpcUaClient/BACnet. **No `IUniversalDriverBrowser`/`BrowserSessionService` change is needed** — MTConnect lights up purely by setting `SupportsOnlineDiscovery => true`. The single required action to enable browse is that one default-member override in §3.2. + +The only limit (universal-browser §8) is that discovery is eager one-shot; `/probe` is already a whole-model single call, so eager is the right fit and graduating to bespoke is unlikely. + +## 5. Typed tag editor + validator (AdminUI) + +Avoid the raw-JSON fallback by adding a typed editor (mirrors the Modbus template exactly). + +- `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs` — pure `FromJson`/`ToJson`/`Validate`, **preserves unknown keys** via the `TagConfigJson` bag helper (copy `ModbusTagConfigModel`). Fields: `FullName` (dataItemId, required), `MtCategory`/`MtType`/`MtSubType` (enums / strings, from probe — read-only in UI), `DataType` (`DriverDataType` override dropdown), `Units`, `MtDevice`/`MtComponent` (author context). `Validate()` returns an error if `FullName` is blank. +- `.../Components/Shared/Uns/TagEditors/MTConnectTagConfigEditor.razor` — thin shell: `FullName` text, read-only `mtCategory/mtType`, a `DataType` override `