dohertj2/lmxopcua
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: driver-expansion program — 8 research reports + 7 design docs, parallel-reviewed
v2-ci / build (push) Successful in 3m14s
Details
v2-ci / unit-tests (push) Failing after 9m13s
Details

Browse Source 
Adds the driver-expansion program design (umbrella: universal Discover-backed
browser + MTConnect, MQTT/Sparkplug B, BACnet/IP, SQL poll, Omron, Modbus RTU;
MELSEC deferred) plus the per-driver research reports.

All docs went through a 7-agent parallel review against the codebase before
this commit. Highlights fixed in review:

- universal browser: FOCAS FixedTree fills post-connect -> UntilStable settle
  + FixedTree.Enabled patch; MQTT reconciled to bespoke (was contradicting the
  program doc's SupportsOnlineDiscovery=false verdict)
- modbus-rtu: SerialPort.ReadTimeout doesn't bound async BaseStream reads ->
  linked-CTS per-op deadline (R2-01 class); BCL enum reuse would leak
  System.IO.Ports into Contracts
- bacnet: DiscoveryRediscoverPolicy enum name; UDP 47808 contention; live
  suite rewritten around unicast Who-Is + BBMD (broadcast doesn't cross VMs)
- sql-poll: real tier registration via DriverFactoryRegistry.Register;
  blackhole gate must not docker-pause the shared central SQL Server
- mqtt: Sparkplug v3.0 STATE topic form; first-in-repo proto codegen noted
- omron: host hardcodes isIdempotent:false today (retry seam unshipped);
  v1 scopes UDTs to dotted-leaf access
- mtconnect: SecurityClassification.ViewOnly; factory ParseEnum<T> pattern
- program doc: both valid enum-serialization patterns; IRediscoverable is
  change-signal-gated; RTU P2 adds System.IO.Ports; label is host-side
This commit is contained in:
Joseph Doherty
2026-07-15 16:40:36 -04:00
parent 37cdbef7a5
commit 8fc147d8d4
17 changed files with 6408 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-07-15-bacnet-ip-driver-design.md
+556
View File
@@ -0,0 +1,556 @@
# BACnet/IP driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Supersedes/derives from:** the research report `docs/research/drivers/bacnet-ip.md` (protocol
survey, library selection, browse verdict). This document turns that research into a concrete,
file-by-file build plan for `ZB.MOM.WW.OtOpcUa.Driver.Bacnet`.
**Reconciles with:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (BACnet browse is
delivered by the **universal discovery browser** in v1, not a bespoke `IBrowseSession`).
---
## 1. Motivation + scope
BACnet/IP (ASHRAE 135 / ISO 16484-5) is the dominant building-automation protocol — HVAC, lighting,
energy metering, access control. A BACnet driver lets the OtOpcUa server pull **facility / HVAC /
energy** points (space temperatures, setpoints, kW/kWh meters, occupancy, equipment status) into the
unified namespace alongside process data (Modbus/S7/Galaxy), so a plant's OT and facility signals live
under one OPC UA address space.
BACnet is the closest analog to the **OpcUaClient** driver in this repo: it has native **device
discovery** (Who-Is/I-Am) and native **change-of-value push** (SubscribeCOV), so it is a first-class
*browseable* + *subscribe-capable* Equipment-kind driver, not a bare poller.
**Scope (this driver is a standard Equipment-kind driver.)** BACnet points are ordinary equipment
`Tag`s bound to the BACnet driver via `TagConfig.FullName` (a JSON addressing blob — see §5), authored
through the standard `/uns` TagModal + address picker, exactly like Modbus / S7 / OpcUaClient. No alias
machinery, no bespoke namespace kind.
**v1 scope: Connect + Read + Discover (+ universal browse) + BBMD/foreign-device. COV push is P2, Write
is P3.** See the phasing in §10.
Research reference: `docs/research/drivers/bacnet-ip.md`.
---
## 2. Project layout
Two new projects, mirroring the Modbus / OpcUaClient split (transport project + a lean Contracts project
the AdminUI can reference without dragging in the UDP transport):
| Project | Contents | References |
|---|---|---|
| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet`** | `BacnetDriver` (IDriver + capability surfaces), the async adapter over the ela-compil client (`BacnetClientAdapter`), `BacnetDriverProbe`, `BacnetDriverFactoryExtensions`, data-type map | `Core.Abstractions`, `Core` (the `Core.Hosting` namespace — `DriverFactoryRegistry` — lives inside the `Core` project), `.Bacnet.Contracts`, NuGet `BACnet` |
| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts`** | `BacnetDriverOptions` (+ nested `ForeignDeviceOptions`/`DiscoveryOptions`/`CovOptions`/`ProbeOptions`), the `BacnetObjectType` / `BacnetPropertyId` enums the AdminUI editor binds, `BacnetTagAddress` record, `BacnetEquipmentTagParser` | `Core.Abstractions` only (zero transport deps — same discipline as `Modbus.Contracts` / `Modbus.Addressing`) |
**No `.Browser` project in v1.** BACnet browse comes *for free* from the universal discovery browser
(§4). A bespoke lazy `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser` is a documented **P-later** graduation for
large multi-device sites (§4.2), not v1 work.
**Contracts csproj** — copy `ZB.MOM.WW.OtOpcUa.Driver.Modbus.Contracts.csproj` minus its
`Modbus.Addressing` reference (BACnet has no separate addressing project): `net10.0`,
`Nullable`/`ImplicitUsings` enable, `TreatWarningsAsErrors`, `ProjectReference` to `Core.Abstractions`
only (it owns the shared `TagConfigJson` field readers used by the parser).
**NuGet.** Central package management: pin the version in `Directory.Packages.props` and reference
(version-less) from the transport project only:
```xml
<!-- Directory.Packages.props -->
<PackageVersion Include="BACnet" Version="4.0.0" /> <!-- confirm the exact current release when implementing -->
<!-- Driver.Bacnet.csproj -->
<PackageReference Include="BACnet" />
```
- **Library: `System.IO.BACnet`, ela-compil fork, NuGet package id `BACnet`, MIT license.** The engine
inside YABE; explicitly multi-targets **`net10.0`** (no TFM friction); covers everything we need —
Who-Is/I-Am, ReadProperty, ReadPropertyMultiple, WriteProperty, SubscribeCOV/SubscribeCOVProperty +
COV notifications, and BACnet/IP with BBMD + foreign-device registration.
- **The async wrinkle (§4).** The library API is **event/callback-driven and blocking-with-callbacks**,
not `async`/`Task`-first: `BacnetClient.WhoIs()` fires `OnIam` events; `ReadPropertyRequest` has a
blocking overload + a begin/end pair. The driver **must** wrap it behind a thin async adapter
(`TaskCompletionSource` + timeout + cancellation) exactly as `OpcUaClientDriver` wraps the OPC UA SDK.
Budget a dedicated `BacnetClientAdapter` layer for this.
- **One `BacnetClient` per driver instance**, not per device — a single shared
`BacnetIpUdpProtocolTransport` UDP socket multiplexes every device on the network (mirrors "single
`Session` per OpcUaClient driver"). **Port-sharing caveat:** receiving *broadcast* traffic requires
binding UDP 47808, and only one exclusive binder per host interface gets it — so **two BACnet
`DriverInstance`s deployed to the same Host node contend for the port** (shared-socket binds make
datagram delivery nondeterministic). Constrain to one BACnet driver instance per node/interface, or
give additional instances distinct `port` values (fine for sims; real devices always speak 47808).
Unicast flows (directed Who-Is, RP/RPM, COV notifications — devices reply to the request's source
address:port) work from any local port, which is what makes the probe's ephemeral-port bind (§7) safe.
---
## 3. Capability mapping (`IDriver` + capability interfaces)
Recommended surface, landing across phases:
```csharp
public sealed class BacnetDriver
: IDriver, ITagDiscovery, IReadable, IHostConnectivityProbe, // P1
ISubscribable, // P2
IWritable, // P3
IDisposable, IAsyncDisposable
```
`DriverType => "BACnet"` (see §7 for the exact casing decision).
| OtOpcUa capability | BACnet mechanism | Phase |
|---|---|---|
| `IDriver.InitializeAsync` | Bind the UDP socket on the configured interface/port; if `ForeignDevice` is configured, `Register-Foreign-Device` to the BBMD **and start the TTL-renew timer** (renew at ~TTL/2); optionally fire one warm-up `Who-Is`. Health → Healthy once the socket is bound (BACnet has no session; liveness is per-device via read/COV, surfaced through the probe). | P1 |
| `ITagDiscovery.DiscoverAsync` | Bounded `Who-Is` collect → per-device Device `object-list` (RPM, segmented) → per-object `object-name` + `units` + `object-type` → stream folders/variables into `IAddressSpaceBuilder`. `SupportsOnlineDiscovery => true`; `RediscoverPolicy => DiscoveryRediscoverPolicy.Once`. | P1 |
| `IReadable.ReadAsync` | Group the batch's references by device-instance; issue **ReadPropertyMultiple** per device for `present-value` (+ `status-flags` for quality). Fall back to per-object **ReadProperty** for devices that reported no RPM/segmentation support in their I-Am. Map result → `DataValueSnapshot`. | P1 |
| `IHostConnectivityProbe` | Per-device liveness: a cheap RP of each known device's `system-status` (or a directed Who-Is). Running↔Stopped transitions raise `OnHostStatusChanged`, scoping Bad-quality fan-out to that device's subtree. Foreign-device mode: probe also asserts "BBMD registration alive." | P1 |
| `ISubscribable.SubscribeAsync` | **SubscribeCOV** per object (native push — the headline win). `OnCOVNotification` handler fans changes to `OnDataChange`. **Lifetime-renew timer** re-subscribes before expiry. Devices that reject COV **degrade transparently to a poll loop** at `publishingInterval`, so the subscribe surface always works. | P2 |
| `IWritable.WriteAsync` | **WriteProperty** `present-value` at a configured **priority** (1–16), or `null` to relinquish. **Verdict: NOT in v1 (§3.3).** | P3 |
| `GetHealth` / `GetMemoryFootprint` / `FlushOptionalCachesAsync` | Standard. Footprint ≈ discovered-object count × ~512 B (mirror `OpcUaClientDriver.GetMemoryFootprint`). Flush drops the device/object browse cache + resets the footprint counter. | P1 |
### 3.1 `IReadable` shape (concrete)
Follow `OpcUaClientDriver.ReadAsync` structure: return one `DataValueSnapshot` per requested reference in
order; report **per-reference** failures via the snapshot's `StatusCode` (Bad-coded), throw only if the
whole client is unreachable. BACnet specifics:
1. Parse each `fullReference` with `BacnetEquipmentTagParser.TryParse` (§5); a malformed blob →
`BadNodeIdUnknown` snapshot without touching the wire.
2. Group parsed addresses by `deviceInstance`. Resolve device network address from the discovered device
table (cached from Who-Is/I-Am); a device not in the table → directed Who-Is with a short deadline,
else `BadCommunicationError`.
3. Per device: one **RPM** for all `(object, present-value + status-flags)` pairs, under the per-request
APDU deadline (§8). Non-RPM devices → sequential **RP** per object.
4. Map `BacnetValue` → `DataValueSnapshot(value, statusCode, sourceTs=null, serverTs=now)`.
`status-flags` `out-of-service`/`fault` ⇒ Uncertain/Bad; else Good (see §3.4).
### 3.2 `ISubscribable` shape (P2 — the real advantage over poll drivers)
- `SubscribeAsync` issues **SubscribeCOV** (`preferConfirmed` from config) per object, tracking a
`subscriberProcessIdentifier` per subscription. Fire the OPC UA initial-data callback with a current
read so clients get a value immediately.
- **Lifetime-renew timer.** Each subscription carries `lifetimeSeconds`; a per-driver timer re-issues
SubscribeCOV at ~`lifetime/2`. This is load-bearing — an un-renewed subscription silently expires and
the node goes stale-Good with no error. (Same class of bug as the redundancy-heartbeat and
continuous-historization renewal traps in this repo's memory — treat renewal as first-class.)
- **Transparent poll fallback.** On a SubscribeCOV reject (error-class `services`/`object`, or a device
whose I-Am advertised no COV), the driver **silently** switches that object to an internal poll loop at
`cov.fallbackPollMs` and still raises `OnDataChange`. The operator sees data either way; the
diagnostic id records which mode each object is in.
- `UnsubscribeAsync` sends SubscribeCOV with lifetime 0 (cancel) and stops any fallback poll.
### 3.3 `IWritable` — verdict: NOT v1 (P3), and why
BACnet writes target a **priority array**: a commandable output (AO/BO/MSV with a `priority-array`
property) resolves its effective value from the highest-priority non-null slot, with `relinquish-default`
as the floor. Writing `present-value` actually writes a *slot* (priority 1–16); writing the wrong slot,
or writing an absolute value where the site's BMS control logic expects to own a slot, **fights the
building's own control loops** — a genuine safety concern (e.g. commanding a damper the BMS is
sequencing). Correctly modelling commandable-vs-non-commandable objects, slot selection, and relinquish
semantics is a correctness minefield that deserves its own phase with explicit validation. So:
- **v1: read-only.** Every tag's parsed address carries `writable=false`; the driver does **not**
implement `IWritable` in P1/P2.
- **P3: `IWritable`** with an explicit per-tag `writePriority` (default 8 "Manual Operator", never 1), a
commandable-object guard (reject writes to non-commandable object types), and a `null`-relinquish path.
Mirror `OpcUaClientDriver.WriteAsync` honesty: a timeout after dispatch → `BadTimeout` (outcome
unknown), a pre-wire failure → `BadCommunicationError`.
### 3.4 Data-type mapping (`BacnetObjectType`/property → `DriverDataType`)
Driver-side map lives in `Driver.Bacnet/BacnetDataTypeMap.cs` (parallel to Galaxy's `DataTypeMap.cs`).
`DriverDataType` members available: `Boolean, Int16/32/64, UInt16/32/64, Float32, Float64, String,
DateTime, Reference`.
| BACnet present-value / property type | `DriverDataType` | Notes |
|---|---|---|
| REAL (AI/AO/AV present-value) | `Float32` | the common analog case |
| DOUBLE | `Float64` | |
| BOOLEAN | `Boolean` | |
| Enumerated Active/Inactive (BI/BO/BV) | `Boolean` | 0=inactive, 1=active |
| Enumerated multistate (MSI/MSO/MSV) | `UInt16` | present-value is a 1-based state index; `state-text` array surfaced as metadata later |
| Unsigned Integer | `UInt32` | |
| Signed Integer | `Int32` | |
| CharacterString (`object-name`, `description`) | `String` | |
| BACnetDateTime / Date / Time | `DateTime` | Schedules, Trend Log timestamps |
| ObjectIdentifier / property reference | `String` | rarely a tag target |
| Array property (element index N) | element type + `IsArray=true` | ValueRank=1, ArrayDimensions from the array length |
**Engineering Units.** The BACnet `units` enum (`degrees-celsius`, `kilowatt-hours`, `percent`, …) is
captured at discovery/browse time and surfaced as OPC UA `EngineeringUnits` (EUInformation) node metadata
via `IAddressSpaceBuilder.AddProperty`, so UNS nodes carry proper units.
**Quality from `status-flags`.** `out-of-service` or `fault` ⇒ Bad/Uncertain `StatusCode`; else Good.
Read RPM should always co-fetch `status-flags` alongside `present-value`.
---
## 4. The async wrinkle + how discovery/browse are delivered
### 4.1 The callback→async adapter
`BacnetClientAdapter` (new, in the transport project) is the single seam that turns the ela-compil
callback/blocking API into the `Task`-returning calls the driver's capability methods need. Pattern
(identical intent to how `OpcUaClientDriver` wraps the SDK):
- **Who-Is is a broadcast + asynchronous I-Am collection — there is no "the response".** The adapter
exposes `Task<IReadOnlyList<DiscoveredDevice>> WhoIsAsync(range, TimeSpan window, ct)`: subscribe
`OnIam`, `client.WhoIs(low, high)`, accumulate callbacks into a concurrent set, then `Task.Delay(window,
ct)` and return the snapshot. The `window` (`discovery.whoIsWindowMs`, default 4 s) is a **hard latency
floor** on discovery — bound it, show a spinner in the picker.
- **RP/RPM/WP/SubscribeCOV** each get a `TaskCompletionSource<T>` + `CancellationTokenSource.CancelAfter`
wrapper around the library's begin/end (or blocking) call, so per-request APDU deadlines (§8) are
enforced independently of any socket timeout — the R2-01 frozen-peer lesson (an async call must not
outlive its own wall-clock deadline just because the transport didn't time out).
- COV notifications arrive on the library's callback thread; the adapter marshals them onto the driver's
`OnDataChange` event.
### 4.2 Browse: universal in v1, bespoke-lazy P-later
**Reconciliation with `2026-07-15-universal-discovery-browser-design.md`:** BACnet browse **is** discovery
(Who-Is → object-list → properties), and `DiscoverAsync` already streams exactly the tree the picker
wants into `IAddressSpaceBuilder`. So for **small/medium sites, v1 plugs straight into the universal
`DiscoveryDriverBrowser`** — **no bespoke browser code, no `.Browser` project.** The requirements:
1. Add the opt-in flag on `ITagDiscovery` (the universal-browser design specifies this default member;
it is **not yet in `ITagDiscovery.cs`** — adding it is part of that design's P1, and BACnet consumes
it):
```csharp
bool SupportsOnlineDiscovery => false; // add to ITagDiscovery
```
`BacnetDriver` overrides it to `true`.
2. BACnet needs **no `PatchForBrowse` entry** — discovery is unconditional (unlike AbCip/TwinCAT which
are config-gated). The picker's authoring config is sufficient to open + discover.
3. The universal browser runs `InitializeAsync → DiscoverAsync → ShutdownAsync` under its open-timeout and
serves the captured tree from memory. BACnet's Who-Is window fits inside that open-timeout (default
60 s ≫ 4 s window).
The universal browser's eager, whole-tree, one-shot capture is fine for a handful of devices with modest
`object-list`s. **For large multi-device sites (dozens/hundreds of controllers, thousands of objects) the
eager one-shot is too heavy** — it Who-Ises the whole network and RPMs every device's full object-list at
open. That is the documented **P-later graduation** to a bespoke lazy browser:
> **P-later — bespoke `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser`.** A hand-written `IBrowseSession`
> (registered `IDriverBrowser` with `DriverType="BACnet"`, which *overrides* the universal fallback per
> §3.1 of the universal design) with three lazy levels: `RootAsync` = bounded Who-Is → devices;
> `ExpandAsync("dev:<n>")` = that one device's `object-list` on demand; `AttributesAsync(...)` = RPM the
> object's key properties. Only build this when a real large site makes the eager capture painful.
**v1 uses the universal browser. State this clearly in the PR: BACnet ships with `SupportsOnlineDiscovery
= true` and zero browser code.**
---
## 5. TagConfig + driver-config JSON
Two layers, matching every Equipment-kind driver.
### 5.1 Driver-level config (`BacnetDriverOptions`, in `.Contracts`)
```jsonc
{
"localEndpoint": "0.0.0.0", // interface to bind; "0.0.0.0" = all
"port": 47808, // 0xBAC0 default
"localDeviceInstance": 4194302, // our own Device instance id (unique on the net)
"foreignDevice": { // null when on-segment / a BBMD is local
"bbmdAddress": "10.20.0.1",
"bbmdPort": 47808,
"ttlSeconds": 900 // registration TTL; driver auto-renews at ~TTL/2
},
"apduTimeoutMs": 3000, // per-request APDU deadline (§8)
"apduRetries": 3,
"maxSegmentsAccepted": 16, // segmentation window we advertise — do NOT set too low
"discovery": {
"whoIsWindowMs": 4000, // I-Am collect window (the time-bounded wrinkle, §4.1)
"deviceInstanceLow": 0, // optional Who-Is range filter
"deviceInstanceHigh": 4194303
},
"cov": { // P2
"preferConfirmed": true,
"lifetimeSeconds": 300, // per-subscription lifetime; renewed before expiry
"fallbackPollMs": 1000 // poll interval for devices that reject COV
},
"probe": { "enabled": true, "intervalMs": 5000 }
}
```
`BacnetDriverOptions` is a plain record/class with `[JsonStringEnumConverter]`-friendly nested option
types; the factory and probe deserialize it identically (§6). Enum knobs authored as **string names**.
### 5.2 Per-tag `TagConfig` (authored on the TagModal / emitted by the browse picker)
A BACnet tag address is fully described by **device-instance + object-type + object-instance + property-id
(+ optional array index)**. `present-value` is the default property.
```jsonc
{
"deviceInstance": 100, // target device (from I-Am)
"objectType": "AnalogInput", // BacnetObjectType enum (string name)
"objectInstance": 3, // object-instance number
"propertyId": "PresentValue", // BacnetPropertyId enum; default PresentValue
"arrayIndex": null, // non-null → element of an array property
"dataType": "Float32", // DriverDataType hint (from browse-time units/type)
"writable": false, // v1: always false; P3 enables WriteProperty
"writePriority": 8 // P3 only: BACnet command priority 1..16
}
```
### 5.3 `DriverAttributeInfo.FullName` encoding + the parser
**`FullName` is the verbatim TagConfig JSON blob** (the §5.2 object serialized to a compact string) —
exactly the Modbus convention (`ModbusEquipmentTagParser`: a leading `{` marks an equipment-tag blob).
This is what the picker commits as `TagConfig.FullName`, what discovery writes into each
`DriverAttributeInfo.FullName`, and what the driver publishes values back under, so the runtime
forward-router resolves them. There is **no separate `deviceInstance.objectType:objectInstance` string
form** — the JSON blob is the single canonical reference.
`BacnetEquipmentTagParser.TryParse(string reference, out BacnetTagAddress addr)` in `.Contracts` (copy the
Modbus parser's shape):
- Leading `{` gate; `JsonDocument.Parse`; require `deviceInstance` (number), `objectType` (enum),
`objectInstance` (number).
- **Strict enum reads** via `TagConfigJson.TryReadEnumStrict` for `objectType` / `propertyId` — a
present-but-typo'd enum **rejects** the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a
wrong object type (the R2-11 strictness lesson the Modbus parser encodes).
- `propertyId` defaults to `PresentValue` when absent; `arrayIndex` optional; `writable` defaults false
in v1.
- Produce `BacnetTagAddress(Reference: reference, DeviceInstance, ObjectType, ObjectInstance, PropertyId,
ArrayIndex, DataType)` where `Reference == reference` (the def identity, so publish keys match).
- Provide an `Inspect(reference)` warnings method (deploy-time surfacing of invalid enums / unparseable
blobs), parallel to `ModbusEquipmentTagParser.Inspect`.
---
## 6. Typed editor + validator + the enum-serialization trap
Add a driver-typed tag editor so the TagModal dispatches by `DriverType="BACnet"` instead of falling back
to the raw-JSON textarea. Copy the Modbus template pair (`Components/Shared/Uns/TagEditors/` razor shell
+ `Uns/TagEditors/*Model.cs` pure model).
1. **`BacnetTagConfigModel`** (`src/Server/.../AdminUI/Uns/TagEditors/BacnetTagConfigModel.cs`) — pure
`FromJson`/`ToJson`/`Validate`, preserving unknown keys via the `JsonObject` bag (mirror
`ModbusTagConfigModel`). Fields: `DeviceInstance:int`, `ObjectType:BacnetObjectType`,
`ObjectInstance:int`, `PropertyId:BacnetPropertyId`, `ArrayIndex:int?`, `DataType:DriverDataType`,
`Writable:bool?`, `WritePriority:int?`. `Validate()`: `deviceInstance` in 0..4194303, `objectInstance`
≥ 0, `writePriority` in 1..16 when present.
2. **`BacnetTagConfigEditor.razor`** (`Components/Shared/Uns/TagEditors/`) — thin shell binding the model;
enum dropdowns for `ObjectType` / `PropertyId` from the `.Contracts` enums.
3. **Register in both maps:**
```csharp
// TagConfigEditorMap.cs
["BACnet"] = typeof(Components.Shared.Uns.TagEditors.BacnetTagConfigEditor),
// TagConfigValidator.cs
["BACnet"] = j => BacnetTagConfigModel.FromJson(j).Validate(),
```
(Both dictionaries are `OrdinalIgnoreCase`, so casing of the key is forgiving, but keep it consistent
with `DriverType` — see §7.)
**The `JsonStringEnumConverter` enum-serialization trap (must-fix, systemic in this repo).** AdminUI
pages/models that serialize enums **numerically** while the driver factory/probe DTOs are **string-typed**
produce configs that fault the driver at runtime (documented systemic bug: driver enum-serialization).
Guard rails:
- The `BacnetTagConfigModel.ToJson` writes enums as **name strings** — use the repo's `TagConfigJson.Set`
helper (it emits enum names, as `ModbusTagConfigModel` does), never `JsonSerializer` with default enum
handling.
- `BacnetDriverFactoryExtensions.JsonOptions` **and** `BacnetDriverProbe`'s options **must** carry
`Converters = { new JsonStringEnumConverter() }` + `PropertyNameCaseInsensitive = true` +
`UnmappedMemberHandling.Skip` — identical options in both, so factory and probe parse a given config
byte-for-byte the same (copy `OpcUaClientDriverFactoryExtensions.JsonOptions`).
- Any AdminUI **driver-config** page for BACnet (the driver-level options form, when it lands) must also
register `JsonStringEnumConverter` so `ObjectType`/`PropertyId`/nested option enums round-trip as
strings.
---
## 7. Factory + registration
**`DriverType = "BACnet"`.** (Chosen casing; the editor/validator/probe/registry keys all use `"BACnet"`.
`DriverType` string comparisons in the tag-editor maps are `OrdinalIgnoreCase`, but keep every registration
site consistent to avoid the Modbus `"ModbusTcp"` vs `"Modbus"` drift that once dropped a driver to the
raw-JSON editor.)
**`BacnetDriverFactoryExtensions`** (copy `OpcUaClientDriverFactoryExtensions`):
```csharp
public static class BacnetDriverFactoryExtensions
{
public const string DriverTypeName = "BACnet";
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip,
Converters = { new JsonStringEnumConverter() },
};
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? lf = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, lf));
}
public static BacnetDriver CreateInstance(string id, string json, ILoggerFactory? lf = null) { /* deserialize BacnetDriverOptions, new BacnetDriver(...) */ }
}
```
**Wire into the two Host bootstrap sites** (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs`):
```csharp
// alongside the other Driver.*.Register(registry, loggerFactory) calls (~line 128-135):
Driver.Bacnet.BacnetDriverFactoryExtensions.Register(registry, loggerFactory);
// alongside the IDriverProbe TryAddEnumerable block (~line 108-114):
services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, BacnetDriverProbe>());
```
**`BacnetDriverProbe`** (`IDriverProbe`, `DriverType => "BACnet"`): deserialize `BacnetDriverOptions` with
the *same* `JsonOptions`; the "cheap connection" is a **bind + directed Who-Is** — bind the UDP socket
**on an ephemeral port** (never the configured 47808 — the runtime driver instance may already hold it
on the same node, and unicast replies come back to the probe's source port anyway; see the §2
port-sharing caveat), in foreign-device mode `Register-Foreign-Device` first, then broadcast/directed
Who-Is bounded by the probe `timeout`, return `Ok=true` with latency if ≥1 I-Am arrives, else a clear
"no BACnet devices answered / BBMD registration failed" message. Never throw — return `Ok=false` +
message (per `IDriverProbe` contract). Note the UDP-broadcast-on-CI caveat still applies to the probe
(§9).
**Probe on admin nodes.** Per the repo's split-role gotcha, `IDriverProbe` must be registered where the
AdminUI resolves `IEnumerable<IDriverProbe>` (admin-pinned), not only on driver nodes — the
`TryAddEnumerable` line above is in the shared bootstrap, which covers it, but verify on the split-role
MAIN cluster.
---
## 8. Resilience / timeout
- **Per-request APDU deadline (R2-01 frozen-peer lesson).** Every RP/RPM/WP/SubscribeCOV call runs under
`apduTimeoutMs` enforced by the adapter's `CancellationTokenSource.CancelAfter` (§4.1), **independent**
of any socket-level timeout. This is the exact class of bug R2-01 found in S7 (async read ignored socket
ReadTimeout → frozen-peer poll wedge): a BACnet device that ACKs at the BVLC layer but never returns the
APDU must surface `BadTimeout`/`BadCommunicationError` within the deadline, not wedge the read loop.
`apduRetries` bounds retry attempts for **idempotent** reads only (never writes — §3.3).
- **COV lifetime/renewal (P2).** The renew timer at ~`lifetime/2` is mandatory (§3.2). A **watchdog** on
the non-COV fallback path: if a fallback-polled object produces no successful read within N intervals,
mark it Bad and log, so a silently-dead poll doesn't masquerade as stale-Good.
- **Foreign-device TTL renewal.** In foreign-device mode a TTL-renew timer re-registers with the BBMD at
~TTL/2; a failed re-registration transitions the driver `Degraded` and the probe surfaces it. Losing the
registration silently is the top field failure mode (§2 / research §6 risk 1).
- **Segmentation.** Large `object-list` / RPM results segment across APDUs; the ela-compil client handles
windowed segment ack, but `maxSegmentsAccepted` must **not** be set too low or a big device's
object-list truncates. Default 16; document that lowering it risks truncated discovery.
- **Per-device fault scoping.** `IHostConnectivityProbe` reports per-device Running/Stopped so one dead
controller Bad-quality-fans-out only its own subtree, not the whole driver namespace (mirror Galaxy's
per-host model).
---
## 9. Test fixtures
BACnet has good open simulators; all are UDP servers we containerize on the shared docker host
(`10.100.0.35`) with a `project=lmxopcua` label, driven by `lmxopcua-fix up bacnet` (add a
`tests/.../Docker/docker-compose.yml` under a new BACnet test project, then `lmxopcua-fix sync bacnet`).
1. **`bacnet-stack` demo server (`bacserv`)** — the reference C stack's demo; configurable
device-instance + object set via env; answers Who-Is/I-Am, RP/RPM, WP, SubscribeCOV. The canonical
target. Also `fh1ch/bacstack-compliance-docker` (a ready-made DockerHub device-sim image) for the
lowest-friction single device.
2. **`mnp/bacnet-docker` (or `desolat/bacnet-docker`)** — docker-compose framework to stand up **multiple**
BACnet/IP servers + a BBMD on one host: the **multi-device Who-Is** discovery test and the
**BBMD/foreign-device topology** test (BBMD + device on one compose network; register the driver as a
foreign device across it).
3. **ela-compil `BACnet.Examples/BasicServer`** — a pure-.NET device sim embeddable **directly in the
integration-test process** (no docker): deterministic COV-notification + RP/RPM tests via **directed
(unicast) Who-Is to a known instance/address**, sidestepping broadcast entirely. Pair with
`BasicAdviseCOV` as an in-proc client oracle. This is the hermetic-unit path.
4. **YABE** (Windows GUI, same ela-compil library) — manual apples-to-apples oracle for cross-checking
reads/COV against a known-good client.
**The UDP-broadcast-on-CI gotcha.** BACnet Who-Is relies on subnet **broadcast**; bridged docker networks
and macOS-hosted CI don't broadcast cleanly. Two mitigations, exactly mirroring the existing S7/historian
live-gate discipline:
- **Hermetic unit/integration:** embedded `BasicServer` + **directed unicast Who-Is** to a known
address — no broadcast, runs on macOS/CI.
- **Broadcast / BBMD / multi-device:** an **env-gated live-integration suite** (e.g.
`BACNET_FIXTURE_ENDPOINT` present ⇒ run, absent ⇒ skip cleanly), fixture on the Linux docker host with
`network_mode: host` (or a dedicated bridge). Category `LiveIntegration` like the historian gate.
**Cross-VM reachability — be explicit about what routes:** this repo's tests execute on the dev
machine, *not* on `10.100.0.35`, and a subnet-broadcast Who-Is from an off-subnet peer **does not
route** (and bridged docker networks don't forward subnet broadcasts into containers either). So the
suite as run from the dev machine has exactly two working paths: (a) **directed unicast Who-Is** at
the fixture's host IP:port (host-mode `bacserv` answers a unicast Who-Is with a unicast I-Am to the
request's source address — verify this against the pinned `bacserv` build early, since spec-strict
stacks may broadcast the I-Am, which would never arrive); and (b) the **foreign-device/BBMD leg** —
register with a BBMD container on the docker host; BVLC `Forwarded-NPDU` relay is *unicast* and routes
across subnets, so this leg exercises real broadcast distribution cross-VM and is exactly the field
topology. A genuinely broadcast-only leg (plain Who-Is with no BBMD) can only run **on the docker host
itself** (ssh, containers sharing a compose network) — keep it a separate opt-in job. Never gate CI on
broadcast.
**Parser + map unit tests** (fully hermetic, no UDP): `BacnetEquipmentTagParser` round-trip + strictness
(typo'd `objectType` rejects), `BacnetDataTypeMap` coverage, `BacnetTagConfigModel` FromJson/ToJson
unknown-key preservation + enum-as-string emission, `TagConfigValidator["BACnet"]` bounds. Live-verify the
typed editor on docker-dev `/uns` (Razor binding bugs pass unit tests — the repo's standing rule).
---
## 10. Phasing + effort
**Overall effort: Medium-Large** — bigger than Modbus (no discovery/push there), roughly on par with or
slightly above OpcUaClient, driven by the callback→async adapter + COV lifetime machinery + BBMD.
Each phase is independently shippable (mirror how OpcUaClient landed across PRs):
- **P1 — Connect + Read + Discover (+ universal browse) + BBMD. (MVP, delivers real value.)**
`.Contracts` (`BacnetDriverOptions`, enums, `BacnetTagAddress`, `BacnetEquipmentTagParser`);
`BacnetClientAdapter` (async wrap); `BacnetDriver : IDriver, ITagDiscovery, IReadable,
IHostConnectivityProbe` with `SupportsOnlineDiscovery=true` + `RediscoverPolicy =>
DiscoveryRediscoverPolicy.Once`; RP/RPM read;
Who-Is/object-list discovery streamed to `IAddressSpaceBuilder`; **foreign-device/BBMD registration +
TTL renewal included here** (without it the driver can't reach most real sites); factory + probe +
Host registration; typed tag editor + validator. Browse works via the universal browser — **no browser
code**.
- **P2 — SubscribeCOV push.** `ISubscribable` via SubscribeCOV + lifetime-renew timer + transparent poll
fallback + fallback watchdog. The headline feature over poll-only drivers.
- **P3 — Write.** `IWritable` via WriteProperty with priority-array semantics + commandable-object
validation + relinquish. Deferred deliberately (§3.3 safety minefield).
- **P4 — History / Alarms.** `IHistoryProvider` over **Trend Log** objects (`ReadRange`); `IAlarmSource`
over intrinsic/algorithmic **event enrollment** + COV of `status-flags` → Part 9 conditions
(`DriverAttributeInfo.IsAlarm` + `MarkAsAlarmCondition`). Nice-to-have, large surface.
### Top risks (from research §6, carried forward)
1. **BBMD / foreign-device registration + UDP broadcast reachability (HIGH — the top risk).** The server
is usually not on the controllers' L2 segment, so correct foreign-device registration + TTL renewal is
load-bearing and is exactly what's hardest to reproduce in docker/CI. Mitigation: build BBMD in from
P1, test explicitly with the `bacnet-docker` multi-network compose, gate a live suite against a real
site BBMD.
2. **COV lifetime/renewal + non-COV fallback (HIGH, P2).** Subscriptions silently expire if not renewed;
some devices cap or lack COV and must degrade to polling without an operator-visible gap. Won't be
caught by unit tests — needs the live/sim COV soak (mirror the continuous-historization live-gate
discipline).
Secondary: the callback/blocking library needs a careful TCS+timeout+cancellation adapter; write
priority-array semantics (why P3 is deferred); segmentation for large object-lists (`maxSegmentsAccepted`
not too low).
---
## Appendix A — new/changed files checklist (P1)
**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts/`:** `BacnetDriverOptions.cs` (+ nested
option records), `BacnetObjectType.cs`, `BacnetPropertyId.cs`, `BacnetTagAddress.cs`,
`BacnetEquipmentTagParser.cs`, `.csproj` (Core.Abstractions ref only).
**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet/`:** `BacnetDriver.cs`, `BacnetClientAdapter.cs`,
`BacnetDataTypeMap.cs`, `BacnetDriverProbe.cs`, `BacnetDriverFactoryExtensions.cs`, `.csproj`
(Core.Abstractions + Core + .Contracts + NuGet `BACnet` — mirror the Modbus/OpcUaClient transport
csproj reference set).
**New — AdminUI:** `Uns/TagEditors/BacnetTagConfigModel.cs`,
`Components/Shared/Uns/TagEditors/BacnetTagConfigEditor.razor`.
**Changed:** `Directory.Packages.props` (+ `BACnet` version); the `.slnx` (add both projects);
`ITagDiscovery.cs` (+ `SupportsOnlineDiscovery` default member — shared with the universal-browser work);
`DriverFactoryBootstrap.cs` (Register + probe TryAddEnumerable); `TagConfigEditorMap.cs` +
`TagConfigValidator.cs` (`"BACnet"` entries).
**Tests:** `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Tests/` (parser, datatype map, adapter with fake
client), `...Driver.Bacnet.IntegrationTests/` (embedded `BasicServer` hermetic + env-gated
`LiveIntegration` for broadcast/BBMD/COV), plus AdminUI `BacnetTagConfigModel` + validator tests.
docs/plans/2026-07-15-driver-expansion-program-design.md
+211
View File
@@ -0,0 +1,211 @@
# Driver-expansion program — overall design
**Status:** program design, 2026-07-15. Umbrella for the next-driver effort: one universal
browse mechanism + six new/extended drivers (MELSEC deferred). This is the authoritative index
and the single statement of the effort's shared architecture, dependency graph, and build order.
Per-driver detail lives in the linked design docs; cross-cutting rules are stated **once here**
so the per-driver docs don't re-derive them.
## 1. Goal & scope
Extend the OtOpcUa OPC UA server's driver fleet under the existing Equipment-kind address-space
model — no new COM gateway, all in-process .NET 10 — and make more of the fleet **browsable**
from the AdminUI `/uns` address picker.
**Curated set (user, 2026-07-15):** MTConnect Agent · MQTT/Sparkplug B · MELSEC SLMP · Omron ·
BACnet/IP · SQL poll · Modbus RTU. **MELSEC is deferred** (commercial-license-forced hand-rolled
framer + the hardest addressing model on the roadmap; revisit on a Mitsubishi-shop need). The
other six are designed and scheduled below.
Alongside the new drivers, a **universal Discover-backed browser** retrofits browse to the
already-shipped fleet (AbCip, TwinCAT, FOCAS) and is the foundation every browsable new driver
builds on.
Existing fleet for context: Modbus(TCP), S7, AbCip, AbLegacy, TwinCAT, FOCAS, Galaxy,
OpcUaClient, Historian.Gateway (`src/Drivers/`).
## 2. Document map
**Program:** this file.
**Research reports** (`docs/research/drivers/`, index: [`README.md`](../research/drivers/README.md)):
- [`mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) ·
[`mqtt-sparkplug.md`](../research/drivers/mqtt-sparkplug.md) ·
[`melsec-slmp.md`](../research/drivers/melsec-slmp.md) *(deferred)* ·
[`omron.md`](../research/drivers/omron.md) ·
[`bacnet-ip.md`](../research/drivers/bacnet-ip.md) ·
[`sql-poll.md`](../research/drivers/sql-poll.md) ·
[`modbus-rtu.md`](../research/drivers/modbus-rtu.md) ·
[`00-existing-driver-browse-audit.md`](../research/drivers/00-existing-driver-browse-audit.md)
**Design docs** (`docs/plans/2026-07-15-*`):
- [`universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md) — **Wave 0**
- [`mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md)
- [`bacnet-ip-driver-design.md`](2026-07-15-bacnet-ip-driver-design.md)
- [`mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md)
- [`sql-poll-driver-design.md`](2026-07-15-sql-poll-driver-design.md)
- [`omron-driver-design.md`](2026-07-15-omron-driver-design.md)
- [`modbus-rtu-driver-design.md`](2026-07-15-modbus-rtu-driver-design.md)
Prior art the designs build on: [`2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md)
(bespoke `IBrowseSession` pattern), [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md)
(standard Equipment-kind driver), and the driver-typed tag editors design.
## 3. Shared architecture (the contract every driver in this program follows)
Every driver here is a **standard Equipment-kind driver** — an `IDriver` (lifecycle: `Initialize`
/ `Reinitialize` / `Shutdown` / `GetHealth` / footprint) composing only the capability interfaces
its backend supports, from `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`:
| Capability | Interface | This program's drivers |
|---|---|---|
| Discover | `ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder, ct)` | all (device-enumerated or authored-only) |
| Read | `IReadable` | all |
| Subscribe | `ISubscribable` | all (poll-based, except the push drivers: MQTT, BACnet COV, MTConnect's `/sample` stream) |
| Write | `IWritable` | Omron, Modbus-RTU (v1); MTConnect/MQTT/BACnet/SQL read-only in v1 |
| Probe | `IHostConnectivityProbe` | all |
| Re-discover | `IRediscoverable` | only drivers with a native backend change signal (per its doc contract) — e.g. MQTT rebirth; BACnet/SQL are discovery-capable but have no change signal and correctly omit it |
**Per-driver build pattern** (identical across the fleet — see any existing driver + the
driver-typed-editors design):
1. `ZB.MOM.WW.OtOpcUa.Driver.<Name>.Contracts` — options + tag DTOs + parser + enums (no backend
NuGet dep).
2. `ZB.MOM.WW.OtOpcUa.Driver.<Name>` — the runtime `IDriver` + capabilities.
3. `.Browser` project **only when bespoke browse is required** (see §4).
4. Factory + `DriverType` string + `Register()`, wired in the Host's `DriverFactoryBootstrap`
(factory + probe on admin nodes via `TryAddEnumerable`).
5. Typed tag editor + validator: a `<Name>TagConfigModel` (`FromJson`/`ToJson`/`Validate`) +
`TagConfigEditorMap` + `TagConfigValidator` entries.
6. Docker fixture under `tests/.../Docker/`, deployed to `10.100.0.35` via `lmxopcua-fix sync`
(the `project=lmxopcua` label is a deployment convention applied host-side — no checked-in
compose file carries a `labels:` entry today).
### 3.1 Cross-cutting rules (mandatory, stated once)
- **Enum-serialization trap** — every enum on a config/tag surface MUST serialize as **names,
never numbers**, on both sides (AdminUI page serializer **and** probe carry
`JsonStringEnumConverter`). Two shipped factory-side patterns both satisfy this — Modbus keeps
DTO fields `string?` + parses via `ParseEnum<T>` (no converter in the factory); OpcUaClient
uses enum-typed DTOs with the converter in factory **and** probe. Either is fine; a mismatch
between the authoring side and the factory side **faults the driver** at deploy. This is a
proven systemic bug — see the driver enum-serialization memory. Non-negotiable for every
driver here.
- **Per-op deadline (R2-01 frozen-peer lesson)** — every network call (read/write/discover/probe/
subscribe/agent-HTTP/SQL-query) MUST have a bounded deadline (linked-CTS / socket ReadTimeout /
`CommandTimeout`). A frozen peer must never wedge a poll. No unbounded waits anywhere.
- **`WriteIdempotent`** — writable drivers flag only tags whose replay is safe. Note the invoker
seam (`ExecuteWriteAsync(…, isIdempotent, …)`) exists but the host's write dispatch currently
hardcodes `isIdempotent: false` (the arch-review "hardcoded isIdempotent" Medium) — so **no
write auto-retries today**; drivers carry the per-tag flag now and retry activates when that
fleet plumbing lands.
- **Secrets** — connection strings / API keys / broker creds come from env/secret refs (mirror
`ServerHistorian__ApiKey`), never committed or logged.
- **Live-verify discipline** — Razor binding + deploy-inertness bugs pass unit tests and review;
every driver's picker/editor and deploy path gets a docker-dev `/run` live-verify before it's
trusted.
## 4. Browse architecture — the two-tier decision
The program's central browse decision: instead of a bespoke browser per driver, a **universal
Discover-backed browser** (`DiscoveryDriverBrowser`, Wave 0) captures any driver's
`ITagDiscovery.DiscoverAsync` output and re-presents it to the picker. Bespoke `IBrowseSession`
browsers are reserved for the cases the universal one genuinely can't serve.
**The gate:** a new default-interface member `ITagDiscovery.SupportsOnlineDiscovery => false`
(mirrors the existing `RediscoverPolicy` default member). A driver opts in when its `DiscoverAsync`
**enumerates from the device**; flat-address drivers leave it false and stay manual-entry. This
member is **added in the universal browser's P1** — **every browsable driver in this program
depends on it**, which is why the universal browser is the Wave-0 gate.
| Driver | Browse mechanism | Verdict |
|---|---|---|
| **MTConnect** | `/probe` device model via `DiscoverAsync` | **Universal** (`SupportsOnlineDiscovery=true`) |
| **BACnet/IP** | Who-Is + object-list via `DiscoverAsync` | **Universal** for small sites; bespoke-lazy `IBrowseSession` a documented P-later for large multi-device sites |
| **AbCip** *(existing)* | CIP Symbol 0x6B/0x6C in `DiscoverAsync` | **Universal** (+ `PatchForBrowse` `EnableControllerBrowse:true`) |
| **TwinCAT** *(existing)* | ADS symbol upload in `DiscoverAsync` | **Universal** (+ patch); bespoke lazy only if symbol set too big |
| **FOCAS** *(existing)* | FixedTree curated discovery | **Universal** (curated; + patch `FixedTree.Enabled:true` + an `UntilStable` settle — its tree fills post-connect, so a one-shot capture would be empty) |
| **MQTT/Sparkplug** | passive topic/birth observation window | **Bespoke** `MqttBrowseSession` — runtime discovery is authored-only, so universal would replay only authored tags |
| **SQL poll** | `INFORMATION_SCHEMA` schema walk | **Bespoke** `SqlBrowseSession` — schema walk ≠ `DiscoverAsync` |
| **Omron** | libplctag `@tags` unsupported on NJ/NX | **None live** — offline Sysmac tag-export importer (P-later); FINS never browsable |
| **Modbus RTU** | flat registers, no discovery | **None** — not browsable |
| S7, AbLegacy, Modbus(TCP) *(existing)* | flat / no on-wire symbols | **None** — documented in the audit |
Bespoke browsers **override** the universal fallback simply by being registered for their
`DriverType` (`BrowserSessionService` resolves bespoke-first). See the universal-browser design §3
for the required `BrowserSessionService` fallback change (the `ToDictionary` dup-key constraint
means the universal browser is a **separate `IUniversalDriverBrowser`**, not one of the injected
`IDriverBrowser` set).
## 5. Library & license decisions
| Driver | Library | License | Note |
|---|---|---|---|
| MTConnect | TrakHound MTConnect.NET (`-Common` + `-HTTP`) | MIT | netstd2.0 → loads on net10; pin + license-review; hand-roll fallback |
| BACnet/IP | System.IO.BACnet (ela-compil fork, `BACnet`) | MIT | multi-targets net10; YABE engine; callback API → TCS adapter |
| MQTT/Sparkplug | MQTTnet v5 + **hand-rolled Tahu protobuf** | MIT | **Not SparkplugNet** — its MQTTnet-4.x transitive pin collides with the repo's deliberately-OFF transitive pinning (Roslyn-split constraint) |
| Omron (CIP) | libplctag.NET | MPL-2.0 | same dep AbCip already ships (`plc=omron-njnx`); FINS hand-rolled |
| SQL poll | Microsoft.Data.SqlClient | — | already in-repo at 6.1.1 — zero new deps for P1 |
| Modbus RTU | extend existing Modbus | — | serial impls behind the **existing** `IModbusTransport` seam; P1 (RTU-over-TCP) zero new deps, P2 direct-serial adds `System.IO.Ports` |
| ~~MELSEC~~ | ~~hand-rolled 3E~~ | — | deferred; HslCommunication is a commercial-license blocker |
**Avoid HslCommunication** (commercial license; the "MIT" claim is a stale abandoned fork) for both
MELSEC and Omron.
## 6. Dependency graph & build order
```
Wave 0 ─ Universal browser ── adds ITagDiscovery.SupportsOnlineDiscovery ──┐
│ (DiscoveryDriverBrowser + CapturingAddressSpaceBuilder + │ every browsable
│ BrowserSessionService fallback) │ driver depends
│ │ on this member
├── immediately lights up AbCip / TwinCAT / FOCAS pickers (existing) │
▼ │
Wave 1 ─ Modbus RTU (extend, lowest effort) · SQL poll (bespoke schema browser) ◄┘
▼
Wave 2 ─ MTConnect (universal browse) · MQTT/Sparkplug (bespoke observation browser)
▼
Wave 3 ─ BACnet/IP (universal browse + COV) · Omron (CIP-first; last — no NJ/NX CIP sim)
```
| Wave | Item | Browse | Write v1 | Effort |
|---|---|---|---|---|
| 0 | Universal `DiscoveryDriverBrowser` | — | — | S–M (~3–5 d) |
| 1 | Modbus RTU | none | Yes | **S (lowest)** |
| 1 | SQL poll | bespoke schema | No | S–M |
| 2 | MTConnect Agent | universal | No | S–M |
| 2 | MQTT/Sparkplug B | bespoke observation | No | M–L |
| 3 | BACnet/IP | universal (+ bespoke-lazy P-later) | No | M–L |
| 3 | Omron | offline import | Yes | M–L |
**Sequencing rationale:** Wave 0 first because the `SupportsOnlineDiscovery` seam gates every
browsable driver and it retrofits browse to three shipped drivers for near-zero marginal cost.
Wave 1 is the low-effort/high-leverage pair (RTU extends an existing driver; SQL adds zero deps).
Wave 2 is the strategic telemetry/UNS pair. Wave 3 is the heavier/test-gated pair — Omron last
because there is **no free NJ/NX CIP simulator**, so its wire correctness is a live-hardware gate.
MELSEC is out of the sequence (deferred).
## 7. Testing & fixtures
- Fixtures live under `tests/.../Docker/`, deployed to the shared host `10.100.0.35` via
`lmxopcua-fix sync` (which owns the `project=lmxopcua` labelling host-side). Per-driver sims: `mtconnect/cppagent`, Mosquitto/EMQX +
Sparkplug simulator, `bacnet-stack`/ela-compil `BasicServer`, central SQL Server
(`10.100.0.35,14330`) + seeded table, `rtu_over_tcp` pymodbus profile + socat serial pair.
- **Env-gated live suites** for what a sim can't cover: BACnet Who-Is/BBMD UDP broadcast, Omron CIP
wire (live hardware), MQTT broker soak.
- Per driver: unit (framer/parser/type-map with golden vectors + fakes) → integration (docker
fixture) → **live `/run`** on docker-dev (picker + editor + deploy path).
## 8. Deferred / out of scope
- **MELSEC SLMP** — deferred (§1). Research complete (`melsec-slmp.md`); revisit on demand.
- **Write-back** for MTConnect (Interfaces), MQTT (NCMD/DCMD), BACnet (WriteProperty priority
array), SQL (parameterized UPSERT) — all documented as later phases in the respective designs.
- **Bespoke lazy browsers** for large BACnet sites and large ControlLogix/TwinCAT symbol sets —
graduate from universal only when eager one-shot discovery proves too heavy.
- Shared `Cip.Core` extraction (AbCip ↔ Omron CIP) — a follow-up, not a gate; keep Omron
self-contained in v1.
## 9. Status
Research: **done** (8 reports). Designs: **done** (7 design docs). Code: **not started.**
Next action: execute Wave 0 (universal browser), then Wave 1.
docs/plans/2026-07-15-modbus-rtu-driver-design.md
+380
View File
@@ -0,0 +1,380 @@
# Modbus RTU (serial + RTU-over-TCP) — implementation design
**Status:** Design / build-ready. Not implemented.
**Date:** 2026-07-15
**Scope:** Add Modbus **RTU** transport modes (direct serial + RTU-over-TCP) to the
existing `ModbusDriver`.
**Research input:** [`docs/research/drivers/modbus-rtu.md`](../research/drivers/modbus-rtu.md).
**Related:** [`docs/drivers/Modbus.md`](../drivers/Modbus.md),
[`docs/v2/modbus-addressing.md`](../v2/modbus-addressing.md),
[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
---
## 1. Motivation + extend-vs-new verdict — **EXTEND**
Add Modbus RTU as **two new `IModbusTransport` implementations behind the existing
transport seam + driver-level config plumbing to select them.** Do **not** create a
sibling driver. The Modbus application protocol above the wire (register model, function
codes FC01–06/15/16, exception-PDU convention, data-type codecs, byte order, arrays,
strings, BCD, bit-in-register, read planner + coalescing, auto-prohibit, deadband,
write path, connectivity probe, OPC-UA materialisation, HistoryRead) is **byte-for-byte
identical** across TCP and RTU — only the wire framing and the physical link differ. A
second driver would clone that entire surface for zero benefit.
### The exact seam this plugs into
The driver already splits the *PDU* (function code + data) from the *transport*
(framing + link I/O) behind one interface:
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs`
```csharp
public interface IModbusTransport : IAsyncDisposable
{
Task ConnectAsync(CancellationToken ct);
Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct); // pdu = [fc, ...data], returns response PDU
}
```
- **`ModbusDriver`** builds every request as a raw PDU `byte[]` and calls
`transport.SendAsync(ResolveUnitId(tag), pdu, ct)` (e.g. `ModbusDriver.cs` lines
~1014/1035/1052/1069/1142/1154). It never touches MBAP, sockets, or CRC.
- **`ModbusTcpTransport`** is the *only* place MBAP framing, the transaction-id counter
(`_nextTx`), and `TcpClient`/`NetworkStream` I/O live — specifically `SendOnceAsync`
(`ModbusTcpTransport.cs` lines 231–296) wraps the PDU as
`[TxId(2)][Proto=0(2)][Length(2)][UnitId(1)] + PDU`, single-flights via `SemaphoreSlim
_gate`, and owns reconnect/keepalive/idle-disconnect.
- **The transport is injected**: `ModbusDriver`'s ctor takes
`Func<ModbusDriverOptions, IModbusTransport>? transportFactory`
(`ModbusDriver.cs` lines 98–114); the default closure builds `ModbusTcpTransport`.
Tests already substitute in-memory fakes through this hook.
So the work is **purely additive**: two new transport classes + a CRC-16 helper +
config/factory selection + one AdminUI panel. **Zero** changes to codecs, planner,
coalescing, health, materialisation, HistoryRead, or the address parser.
### New/changed files at a glance
| File | Change |
|---|---|
| `…/Driver.Modbus/ModbusCrc.cs` | **new** — CRC-16 (poly `0xA001`) helper |
| `…/Driver.Modbus/ModbusRtuFraming.cs` | **new** — shared ADU frame/deframe + FC-aware response sizing |
| `…/Driver.Modbus/ModbusRtuTransport.cs` | **new** — `System.IO.Ports.SerialPort` transport |
| `…/Driver.Modbus/ModbusRtuOverTcpTransport.cs` | **new** — RTU framing over a socket |
| `…/Driver.Modbus/ModbusSocketLifecycle.cs` | **new** (refactor) — socket connect/reconnect/keepalive/idle extracted from `ModbusTcpTransport`, shared by TCP + RtuOverTcp |
| `…/Driver.Modbus/ModbusTransportFactory.cs` | **new** — `Create(ModbusDriverOptions)` switch on `Transport`; used by the driver default closure **and** the probe |
| `…/Driver.Modbus.Contracts/ModbusDriverOptions.cs` | add `Transport` + serial fields (+ two enums) |
| `…/Driver.Modbus/ModbusDriverFactoryExtensions.cs` | DTO fields + wire the default closure to `ModbusTransportFactory.Create` |
| `…/Driver.Modbus/ModbusDriverProbe.cs` | build the transport via `ModbusTransportFactory` (currently hardcodes `new ModbusTcpTransport`, line 77) |
| `…/Driver.Modbus/ModbusDriver.cs` | `BuildSlaveHostName` endpoint string reflects the transport (COM/gateway vs host:port) |
| `…/AdminUI/…/Drivers/ModbusDriverPage.razor` | serial-parameter panel shown when `Transport != Tcp` |
`System.IO.Ports` (10.0.x for .NET 10) becomes a new PackageReference on
`ZB.MOM.WW.OtOpcUa.Driver.Modbus`.
---
## 2. Transport implementations
Both new transports produce the **RTU ADU**: `[slaveAddress(1)][PDU][CRC-lo][CRC-hi]` —
no MBAP header, no transaction id. CRC-16 (poly `0xA001` reflected, appended
**low byte first**) replaces TCP's transport-level integrity. The RTU framing/deframing
logic is identical between the two; factor it into `ModbusRtuFraming` so the serial and
socket transports share it and differ only in the byte-stream they read/write.
### 2a. `ModbusRtuFraming` (shared)
- `byte[] BuildAdu(byte unitId, ReadOnlySpan<byte> pdu)` → `[unit][pdu][crcLo][crcHi]`.
- `int ExpectedResponseLength(byte functionCode)` / a streaming reader that determines
response length **by parsing the function code** (RTU frames are length-less — see
§7). Read `addr(1)` + `fc(1)` first, then:
- **Exception** (`fc & 0x80`): read `excCode(1) + CRC(2)` → 5-byte ADU total; throw
`ModbusException(fc & 0x7F, excCode, …)` after CRC-validating.
- **Read responses** FC01/02/03/04: read `byteCount(1)`, then `byteCount + CRC(2)`.
- **Write echoes** FC05/06/15/16: fixed `addr + qty/value(4) + CRC(2)` → read 6 more.
- After the full ADU is in hand: validate the trailing CRC-16 (mismatch →
`ModbusTransportDesyncException(DesyncReason.…)` so it maps onto the existing
`BadCommunicationError` handling), strip the address byte + CRC, and return the bare
response PDU `[fc, ...data]` — exactly what `SendAsync`'s callers already decode.
### 2b. `ModbusRtuTransport : IModbusTransport` (direct serial)
- Wraps a `System.IO.Ports.SerialPort`. `ConnectAsync` opens the port with the
configured `PortName`/`BaudRate`/`DataBits`/`Parity`/`StopBits`; sets
`ReadTimeout`/`WriteTimeout` from `ModbusDriverOptions.Timeout`.
- **Per-op deadline (R2-01): `SerialPort.ReadTimeout` only governs the synchronous read
API — async reads over `SerialPort.BaseStream` ignore `ReadTimeout`/`WriteTimeout`
entirely** (the exact wall-clock gap class R2-01 found in the S7 driver, where async
socket reads ignored `ReadTimeout` and a frozen peer wedged the poll). Every
transaction MUST therefore run under a linked-CTS
`CancellationTokenSource.CancelAfter(Options.Timeout)` deadline, mirroring
`ModbusTcpTransport.SendOnceAsync` — timeout ⇒ close the port + surface
`ModbusTransportDesyncException(DesyncReason.Timeout)`, distinct from caller
cancellation (no teardown).
- `SendAsync`: single-flight via the same `SemaphoreSlim _gate` pattern (**mandatory** on
RTU — no TxId means at most one transaction on the bus at a time). Enforce ≥3.5-char
inter-frame idle before transmit (computed from baud/word-length, or the
`InterFrameDelayMs` override), write the ADU, then read the response using
`ModbusRtuFraming`'s FC-aware sizing, backstopped by the linked-CTS per-op deadline.
- Failure model is **simpler than TCP** — a COM port doesn't "drop" the way a NAT'd
socket does. On an I/O error or CRC desync, close + reopen the port once and retry
(reuse the single-retry shape from `ModbusTcpTransport.SendAsync`); no keepalive, no
idle-disconnect, no geometric socket backoff.
### 2c. `ModbusRtuOverTcpTransport : IModbusTransport` (RTU tunnelled over a socket)
- **Identical RTU framing** (`ModbusRtuFraming`) but the byte stream rides a
`TcpClient`/`NetworkStream` to a serial→Ethernet gateway instead of a COM port.
- **Reuses the hardened socket lifecycle** — extract the connect (IPv4-preference DNS),
`SO_KEEPALIVE`, idle-disconnect, and reconnect-with-backoff machinery from
`ModbusTcpTransport` into `ModbusSocketLifecycle` and have **both** TCP variants
compose it. The *only* delta from `ModbusTcpTransport` is: CRC framing instead of MBAP,
and no transaction id. This is the lowest-risk of the two RTU transports to build and
test (see §9).
**Modbus ASCII is out of scope** — a third, rarely-used framing (`:`-delimited hex,
LRC instead of CRC). It would be another `IModbusTransport` behind the same seam if ever
needed; not planned.
---
## 3. Cross-platform serial reality
- **`System.IO.Ports.SerialPort` is cross-platform** on .NET 10 (`System.IO.Ports`
10.0.x): Windows + Linux (`/dev/tty*` via termios). **macOS is the weak platform**
(baud quirks, `MacCatalyst` unsupported) — this repo's dev machine is macOS, so
**direct-serial cannot be exercised on the dev Mac**; use fakes / RTU-over-TCP there
and run the real-serial suite on the Linux docker host / CI (§8).
- **Containers add a device-mapping hurdle:** a serial device must be explicitly passed
in (`docker run --device=/dev/ttyUSB0` or a compose `devices:` entry), and USB-serial
adapters re-enumerate (`ttyUSB0` ↔ `ttyUSB1`) across replug — pin a stable
`/dev/serial/by-id/...` path. Windows-container COM passthrough is unreliable.
**RTU-over-TCP is the recommended primary path.** OtOpcUa deploys as a containerised
Linux server (docker-dev rig; docker host `10.100.0.35`) that is generally **not**
attached to an RS-485 bus. The idiomatic topology is a **serial→Ethernet gateway** (Moxa
NPort, Digi One, Lantronix, USR-TCP232) on the RS-485 multidrop, exposed over TCP. The
server talks **RTU-over-TCP** to it — a plain socket, **zero host-device mapping, no
`System.IO.Ports` dependency at runtime, no udev fragility**, and it reuses the already-
hardened socket lifecycle. Direct serial ships too, for bare-metal Windows/Linux installs
with a local/adapter COM port, but RTU-over-TCP is what most deployments will use.
---
## 4. Config JSON shape
**Per-tag `TagConfig` is unchanged.** `ModbusTagDefinition` / `ModbusTagDto` already carry
everything RTU needs — including the per-tag **`UnitId`** override that makes an RS-485
multi-drop bus "just work" (the read planner already refuses to coalesce across UnitIds;
`ResolveUnitId` + `BuildSlaveHostName` already key per-slave resilience by unit). The
`ModbusTagConfigEditor` in `TagConfigEditorMap` needs **no change**. Additions are
**driver-level only**.
New fields on `ModbusDriverOptions` (and the matching optional fields on
`ModbusDriverConfigDto`):
| Field | Type | Applies to | Notes |
|---|---|---|---|
| `Transport` | `ModbusTransportMode` enum (`Tcp`\|`Rtu`\|`RtuOverTcp`) | all | **Default `Tcp`** (back-compat: existing configs omit it) |
| `SerialPort` | string | Rtu | `"COM3"` / `"/dev/ttyUSB0"` / `by-id` path |
| `BaudRate` | int | Rtu | 9600 / 19200 / 38400 / 115200 |
| `DataBits` | int | Rtu | usually 8 |
| `Parity` | `Parity` enum (`None`\|`Even`\|`Odd`) | Rtu | spec default **Even**; many devices use None |
| `StopBits` | `StopBits` enum (`One`\|`Two`) | Rtu | 1 with parity, 2 without |
| `InterFrameDelayMs` | int? | Rtu | optional override of computed T3.5 for slow/RF links |
| `Host` / `Port` | reused | Tcp, **RtuOverTcp** | the gateway's socket for RtuOverTcp |
`Host`/`Port`/`UnitId`/`TimeoutMs`/`MaxRegistersPerRead`/keepalive/reconnect all stay.
Serial fields are ignored when `Transport=Tcp`; `Host`/`Port` are ignored when
`Transport=Rtu`. Keepalive/idle/reconnect apply to `Tcp` + `RtuOverTcp` only.
> Serialization note: `Parity`/`StopBits` names collide with
> `System.IO.Ports.Parity`/`StopBits` (member names match the table), but **define local
> enums in `Driver.Modbus.Contracts`** and map them to the BCL enums inside
> `ModbusRtuTransport`. Reusing the BCL enums directly would put the `System.IO.Ports`
> package on the Contracts project (and transitively on the AdminUI, which deserializes
> `ModbusDriverOptions`) — contradicting §1's plan to add the dependency to
> `Driver.Modbus` only, and the program-doc rule that Contracts carries no backend
> NuGet dep.
### Example A — direct serial RTU (multi-drop)
```json
{
"transport": "Rtu",
"serialPort": "/dev/ttyUSB0",
"baudRate": 19200,
"dataBits": 8,
"parity": "Even",
"stopBits": "One",
"unitId": 1,
"timeoutMs": 1000,
"tags": [
{ "name": "Flow", "addressString": "40001:F:ABCD", "writable": false },
{ "name": "Setpt", "addressString": "40010:F", "writable": true },
{ "name": "Pump2Run", "region": "Coils", "address": 0, "dataType": "Bool",
"writable": true, "unitId": 2 }
]
}
```
`Pump2Run` is a second drop slave (UnitId 2) on the same bus — no extra transport config,
just the per-tag `unitId` override the driver already honours.
### Example B — RTU-over-TCP to a serial→Ethernet gateway
```json
{
"transport": "RtuOverTcp",
"host": "10.20.0.50",
"port": 4001,
"unitId": 1,
"timeoutMs": 1500,
"tags": [
{ "name": "Temp", "addressString": "30001:I" },
{ "name": "Alarm", "region": "DiscreteInputs", "address": 5, "dataType": "Bool" }
]
}
```
Same `host`/`port` shape as Modbus TCP, but the wire frames are raw RTU (address + CRC,
no MBAP). The `Transport` discriminator **must match the gateway's configured mode**:
`"Tcp"` for a gateway doing Modbus/TCP translation, `"RtuOverTcp"` for
transparent/RTU-passthrough. The two wire formats are mutually unparseable.
---
## 5. The `JsonStringEnumConverter` trap — **must handle**
Per the project-wide enum-serialization bug (memory: *Driver enum-serialization bug*),
driver pages serialize enums but factory DTOs are string-typed. The new
`Transport`/`Parity`/`StopBits` enums **must round-trip as strings**, or an AdminUI-
authored RTU config faults the driver at deploy.
Good news — the plumbing is **already correct on both ends** and just needs the new fields:
- `ModbusDriverPage.razor`'s serializer (`_jsonOpts`, line 328–334) already has
`Converters = { new JsonStringEnumConverter() }` + camelCase. It serializes a
`ModbusDriverOptions` directly, so a `ModbusTransportMode`/`Parity`/`StopBits` on
`ModbusDriverOptions` emits as `"Rtu"`/`"Even"`/`"One"` automatically.
- `ModbusDriverFactoryExtensions` DTO fields are typed `string?` and parsed via the
existing `ParseEnum<T>` helper (case-insensitive) — mirror how `Family` /
`MelsecSubFamily` are already handled. **Do not** type the DTO fields as the enum.
- `ModbusDriverProbe._opts` (line 19–24) already carries `JsonStringEnumConverter`.
Add a driver-page/factory round-trip unit test asserting `"transport":"Rtu"` (string, not
`1`) — the same guard that caught S7/Modbus previously.
---
## 6. Capability mapping — identical to TCP, four deltas
Same register model, function codes, data types, read/write semantics, coalescing,
deadband, `WriteOnChangeOnly`, connectivity probe (FC03@0). **Read + write both fully
supported.** The only deltas are below the seam:
| Delta | TCP (today) | RTU (added) |
|---|---|---|
| Physical link | `TcpClient` | `SerialPort` (Rtu) / `TcpClient` (RtuOverTcp) |
| Framing | 7-byte MBAP + TxId | `[addr][PDU][CRC-16]`, no TxId |
| Integrity | TCP guarantees | app-level CRC-16 (`0xA001`) |
| Unit/slave id | often 1 | **central** — one bus, many drops by unit id (already supported per-tag) |
| Response length | MBAP `Length` field | **no length field** → FC-aware sizing (§7) |
| Timing | none | ≥3.5-char inter-frame silence; single-flight mandatory |
`ResolveHost` / `BuildSlaveHostName` (`ModbusDriver.cs` line 162,
`"{host}:{port}/unit{n}"`) should format the per-slave resilience key from the active
endpoint — `COMx/unit{n}` (Rtu) or `gatewayHost:port/unit{n}` (RtuOverTcp) — so
per-slave breakers stay distinct.
### Browseability — **NO** (reconcile w/ universal browser)
Modbus RTU is **not browsable** — a flat, untyped register space with no discovery
protocol, identical to Modbus TCP. `SupportsOnlineDiscovery=false`; the driver keeps
`RediscoverPolicy = Once` and materialises the authored tag list into a flat folder. No
browser (universal or bespoke). The AdminUI `ModbusAddressPickerBody`/`ModbusAddressBuilder`
is an address *builder* (grammar helper), not a live browser, and stays valid for RTU.
---
## 7. Resilience / timeout
- **Per-op deadline + T3.5 framing.** The linked-CTS `CancelAfter(Options.Timeout)`
per-op deadline (§2b — **not** `SerialPort.ReadTimeout`, which async reads ignore) is
the hard backstop; FC-aware sizing is the primary length signal,
with the inter-frame idle gap as the delimiter fallback. Above 19200 baud the spec
fixes T3.5≈1.75 ms / T1.5≈750 µs rather than scaling further; below it,
T3.5 ≈ 3.5 × (bits-per-char / baud) (e.g. 9600 8-N-1 ≈ 3.6 ms). `InterFrameDelayMs`
overrides for slow/long RS-485 or RF links.
- **The one genuinely new correctness risk: RTU response sizing without a length field.**
Get the FC-aware calculation + exception-PDU short-frame detection right (§2a), or the
read hangs to timeout / mis-frames. Cover exhaustively with fake-stream unit tests
(below). A CRC mismatch or truncated frame maps to
`ModbusTransportDesyncException` → the existing single reconnect-retry + status mapping.
- **RtuOverTcp reuses the socket lifecycle** — keepalive, idle-disconnect, reconnect
backoff, IPv4-preference connect — unchanged from `ModbusTcpTransport` via the extracted
`ModbusSocketLifecycle`. **Direct serial does not** use any of that (a COM port has no
NAT reaping); it uses the simpler close/reopen-once-on-error model.
- **Single-flight is mandatory on RTU** (no TxId to correlate an interleaved response) —
the existing `_gate` semaphore already provides it; keep it in both new transports.
---
## 8. Test fixtures
1. **RTU-over-TCP against pymodbus (highest ROI, no serial hardware).** Add an
`rtu_over_tcp` profile to the existing fixture
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/docker-compose.yml`
(alongside `standard`/`dl205`/`mitsubishi`/`s7_1500`/`exception_injection`), running
`pymodbus.simulator` in **RTU-framed-TCP** mode. The existing profiles all share host
port `:5020` and are mutually exclusive by design; bind `rtu_over_tcp` to a fresh port
so it can co-run with `standard` (and sidestep the known shared-port stale-container
trap). Add the `project=lmxopcua` label per the program-doc fixture convention (the
checked-in compose services carry no `labels:` entries today). Drive it via
`lmxopcua-fix up modbus rtu_over_tcp` + `lmxopcua-fix sync modbus`. This exercises the
real CRC + FC-aware framing end-to-end with **no serial anything** — the only
genuinely new logic.
2. **Virtual serial pair for the direct-serial transport.** `socat -d -d pty,raw,echo=0
pty,raw,echo=0` creates a linked `/dev/pts/N` ↔ `/dev/pts/M` pair; point a pymodbus
**RTU serial** slave at one end and `ModbusRtuTransport` at the other. (`com0com` is
the Windows equivalent.) Runs on the Linux docker host / CI, **not** the dev Mac.
Defer to an **env-gated live suite** (like the other driver live gates).
3. **`diagslave`** (serial + TCP RTU) for manual/soak against a virtual pair or a real
USB-serial adapter — eventual live gate, not unit CI.
**Unit-level (no PLC):**
- `ModbusCrc` table-driven test against known Modbus CRC vectors.
- `ModbusRtuFraming` / `ModbusRtuTransport` / `ModbusRtuOverTcpTransport` against an
**in-memory duplex stream fake** (fake one level below the existing `IModbusTransport`
fakes — the byte stream) asserting ADU build, CRC append/validate, FC-aware response
parse for each FC group, and exception-PDU short-frame handling.
- Driver-page/factory round-trip test for the `JsonStringEnumConverter` guard (§5).
**Recommended CI shape:** unit (CRC + framing + config round-trip) + the `rtu_over_tcp`
pymodbus profile for integration; real-serial (socat / hardware) env-gated live.
---
## 9. Phasing + effort
**Effort: LOW — likely the lowest-effort item on the driver roadmap.** The seam is
purpose-built; net-new code is small and localised.
- **P0 — shared plumbing:** `ModbusCrc` (~30 lines) + `ModbusRtuFraming` (FC-aware sizing)
+ extract `ModbusSocketLifecycle` from `ModbusTcpTransport` (mechanical refactor;
`ModbusTcpTransport` keeps behaviour) + `ModbusTransportFactory.Create` + config fields
on `ModbusDriverOptions`/DTO + factory closure + probe wiring. Unit tests for CRC +
framing + config round-trip.
- **P1 — `ModbusRtuOverTcpTransport`** (reuses the socket lifecycle; delta = CRC framing +
FC-aware length, no TxId) + `rtu_over_tcp` pymodbus docker profile + AdminUI serial panel
wired for the RtuOverTcp subset. **Ship this first** — no host-device dependency,
testable on the existing pymodbus harness, reuses hardened socket code.
- **P2 — `ModbusRtuTransport`** (direct serial) + `System.IO.Ports` PackageReference +
socat/diagslave env-gated live suite. For bare-metal installs.
**Top risk:** serial framing/timing on Linux/in-containers (best-effort T1.5/T3.5 gating;
device enumeration; the length-less-frame response-sizing correctness). Mitigated by
leading with RTU-over-TCP (P1), exhaustive fake-stream framing tests, and the
`InterFrameDelayMs` override for slow links. Direct-serial (P2) is the residual-risk
piece and is deferred behind the env-gated live suite.
docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md
+412
View File
@@ -0,0 +1,412 @@
# MQTT / Sparkplug B driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Research input:** `docs/research/drivers/mqtt-sparkplug.md` (verdicts + protocol/library survey — this doc turns that into a build plan; it does not re-litigate the research).
**Related:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (why MQTT needs a *bespoke* browser despite the universal one — §4), `docs/plans/2026-06-12-galaxy-standard-driver-design.md` (standard Equipment-kind driver shape + fire-and-forget write precedent), `docs/plans/2026-05-28-driver-browsers-design.md` (`IDriverBrowser`/`IBrowseSession` pattern reused by the observation browser).
---
## 1. Motivation + scope
Build a standard **Equipment-kind** driver — same composable-`IDriver` shape as Modbus / S7 / AbCip / TwinCAT / FOCAS / OpcUaClient — that ingests **plain MQTT** and **Sparkplug B** into the OtOpcUa OPC UA / UNS address space.
- OtOpcUa is a **Unified-Namespace product**, and **Sparkplug B is the de-facto UNS-over-MQTT payload standard** (mandated topic namespace + protobuf schema + birth-certificate discovery + stateful sessions). Sparkplug alignment is therefore the strategic headline; **plain MQTT/JSON** is the broad-compatibility fallback for brokers that carry raw JSON topics.
- This is a **subscribe-first** driver. Unlike the polled drivers (Modbus/S7/AbCip/FOCAS with `PollGroupEngine`), it holds a **live broker connection** for its whole lifetime and raises `OnDataChange` from the MQTT receive callback. The closest existing analog is the native-push side of `OpcUaClientDriver`.
- **In scope (this design):** connect/TLS/auth/reconnect; topic + Sparkplug subscribe → `OnDataChange`; last-value `IReadable`; authored-only `ITagDiscovery`; `IRediscoverable`; `IHostConnectivityProbe`; a **bespoke observation browser**; typed AdminUI tag editor + validator; Docker fixtures + env-gated live suite.
- **Out of scope for v1:** write-through (`IWritable` — Sparkplug NCMD/DCMD + plain publish) is deferred to phase 3 (§3.4, §10); `DataSet`/`Template` Sparkplug metrics; `Bytes`/`File` metrics beyond a raw-String fallback.
Canonical **DriverType string: `"Mqtt"`** — one string used identically across the AdminUI driver page, probe, factory, editor map, validator, and browser. (Heed the Modbus `"ModbusTcp"` vs `"Modbus"` mismatch lesson: a divergent string silently drops tags to the raw-JSON editor / faults resolution. Pick `"Mqtt"` and grep to enforce it.)
---
## 2. Project layout
Three new projects under `src/Drivers/`, mirroring the OpcUaClient triad (`.Driver` + `.Contracts` + `.Browser`). All `net10.0`, `Nullable`+`ImplicitUsings` enabled, `TreatWarningsAsErrors=true` opted in per-csproj (mandatory for new projects — `Directory.Build.props` deliberately does **not** set it globally because legacy test projects would fail the build).
```
src/Drivers/
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ # options + tag DTOs + parser — zero transport deps
MqttDriverOptions.cs # driver-config DTO (broker conn + mode + sparkplug/plain sub-objects)
MqttMode.cs, MqttPayloadFormat.cs # enums (Plain|SparkplugB ; Json|Raw|Scalar)
MqttTagDefinition.cs # parsed per-tag descriptor (plain OR sparkplug variant)
MqttEquipmentTagParser.cs # TagConfig JSON → MqttTagDefinition (mirrors ModbusEquipmentTagParser)
SparkplugDataType.cs # Sparkplug metric-datatype enum + → DriverDataType map
ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ # runtime driver — references .Contracts + Core.Abstractions + Core
MqttDriver.cs # IDriver, ITagDiscovery, ISubscribable, IReadable,
# IHostConnectivityProbe, IRediscoverable (+ IWritable phase 3)
MqttDriverProbe.cs # IDriverProbe — broker CONNECT handshake
MqttDriverFactoryExtensions.cs # DriverTypeName="Mqtt"; Register(registry, loggerFactory)
MqttConnection.cs # MQTTnet client wrapper: connect/TLS/auth/reconnect/backoff
LastValueCache.cs # FullReference → last DataValueSnapshot (seeds IReadable)
Sparkplug/
SparkplugCodec.cs # Tahu-proto decode (Payload/Metric); encode NCMD (phase 3)
AliasTable.cs # per edge-node/device alias→(name,datatype); rebuilt each birth
BirthCache.cs # NBIRTH/DBIRTH metric catalog (name/alias/datatype/last value)
SequenceTracker.cs # seq (0-255 wrap) gap detection; bdSeq death→birth tie
RebirthRequester.cs # publishes NCMD Node Control/Rebirth=true
SparkplugTopic.cs # parse/format spBv1.0/{group}/{type}/{node}[/{device}]
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser/ # bespoke observation browser — references .Contracts + Commons(.Browsing) + MQTTnet
MqttDriverBrowser.cs # IDriverBrowser (DriverType="Mqtt"); OpenAsync → MqttBrowseSession
MqttBrowseSession.cs # IBrowseSession over an accumulating observed tree
src/Server/.../AdminUI/
Uns/TagEditors/MqttTagConfigModel.cs # typed working model (FromJson/ToJson/Validate; preserves unknown keys)
Uns/TagEditors/TagConfigEditorMap.cs # + ["Mqtt"] = MqttTagConfigEditor
Uns/TagEditors/TagConfigValidator.cs # + ["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate()
Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor
EndpointRouteBuilderExtensions.cs # + AddSingleton<IDriverBrowser, MqttDriverBrowser>()
src/Server/.../Host/Drivers/DriverFactoryBootstrap.cs
# + MqttDriverFactoryExtensions.Register(registry, loggerFactory)
# + TryAddEnumerable(IDriverProbe, MqttDriverProbe)
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests/ # unit — codec/alias/seq/parser (net10, macOS-safe)
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/ # env-gated live suite (MQTT_FIXTURE_ENDPOINT)
tests/Drivers/.../Docker/docker-compose.yml # mosquitto + sparkplug-sim + json-publisher (project=lmxopcua)
```
**Why the split `.Contracts` project:** it carries `MqttDriverOptions` + `MqttTagDefinition` + `MqttEquipmentTagParser` + the enums with **zero transport (MQTTnet) dependency**, exactly like `Driver.Modbus.Contracts` depends only on `Modbus.Addressing` + `Core.Abstractions`. This lets the AdminUI typed editor (and any tooling) reference the DTO shape/enums without dragging MQTTnet into AdminUI. The runtime `.Driver` and `.Browser` both reference `.Contracts` and add MQTTnet.
### 2.1 Libraries + the pinning caveat
| Concern | Choice | Package |
|---|---|---|
| **MQTT transport (both modes)** | **MQTTnet v5** — MIT, **.NET Foundation** (`dotnet/MQTTnet`), the standard .NET MQTT client. v5 explicitly ships a `net10.0` TFM. Client + TLS + MQTT 3.1.1/5.0. | `MQTTnet` (add `PackageVersion Include="MQTTnet" Version="5.x"` to `Directory.Packages.props`) |
| **Sparkplug decode** | **Hand-rolled Tahu protobuf** (recommended) — generate C# from Tahu's `sparkplug_b.proto` with `Google.Protobuf` (`Grpc.Tools`/`protoc`), decode over a single MQTTnet-5 client. Small, stable schema; we own the alias/seq/rebirth state machine exactly as we want it. | `Google.Protobuf` + `Grpc.Tools` (build-time) |
| **Alternative Sparkplug decode** | **SparkplugNet** — MIT, provides `Application`/`Node`/`Device` base classes, saves the birth/alias plumbing. **Rejected as primary** for two repo-specific reasons (below). | `SparkplugNet` (only if the risk clears) |
**Pinning caveat (load-bearing — this is why hand-roll is recommended).** This repo uses **central package management** (`Directory.Packages.props`, `ManagePackageVersionsCentrally=true`) and deliberately **leaves `CentralPackageTransitivePinningEnabled` OFF** because turning it on breaks the Roslyn 5.0.0/4.12.0 split (see the "Transitive pinning breaks Roslyn build" memory + `Directory.Packages.props:103` comment). SparkplugNet **pins MQTTnet 4.3.x transitively and has no explicit `net10.0` TFM** yet. A direct MQTTnet-5 reference (needed for the plain-mode transport + net10) will **coexist in the graph with SparkplugNet's transitive 4.x**, and because transitive pinning is off we cannot cleanly force-align the two. Net: SparkplugNet risks a hard-to-resolve MQTTnet 4/5 diamond in exactly the pinning configuration this repo can't change.
**Decision: hand-roll Tahu-proto decode over one MQTTnet-5 client.** It removes the diamond and the net10-TFM concern entirely, and the Sparkplug payload schema (`Payload { seq, timestamp, Metric[], body }`, `Metric { name, alias, timestamp, datatype, value, is_null, ... }`) is ~90 lines of `.proto` — a bounded, well-understood effort. Phase 1 (plain MQTT) references **only MQTTnet-5**, so we validate MQTTnet-v5-on-net10 before ever touching Sparkplug. (If a future need for SparkplugNet arises, the phase-2 codec seam `SparkplugCodec` is the single swap point.)
**Vendor the proto:** commit Tahu `sparkplug_b.proto` into `Driver.Mqtt.Contracts/Protos/` (pinned to a known Tahu commit, with a provenance comment) and generate at build via `Grpc.Tools`. Don't fetch at build time.
**Existing protobuf tooling, honestly:** `Google.Protobuf` is **already centrally pinned** (`Directory.Packages.props:31`, 3.34.1) and consumed by `Driver.Galaxy` — but the Galaxy proto types ship **pre-generated** inside the `ZB.MOM.WW.MxGateway.*` NuGets; there is **no `.proto` file or `Grpc.Tools` protoc codegen anywhere in this repo today**. So `Grpc.Tools` is a new (build-time-only, `PrivateAssets=all`) package and this is the repo's first in-repo proto codegen step. If that step proves objectionable, the fallback is checking in the generated C# (the schema is small and stable) with the same provenance comment.
---
## 3. Capability mapping
`MqttDriver : IDriver, ITagDiscovery, ISubscribable, IReadable, IHostConnectivityProbe, IRediscoverable` (+ `IWritable` in phase 3). **Not** `IHistoryProvider` (historization is the server-side `isHistorized` path, unchanged). **Not** a poller — no `PollGroupEngine`.
### 3.0 Lifecycle (`IDriver`)
- `InitializeAsync(configJson, ct)` — deserialize `MqttDriverOptions` (shared `JsonSerializerOptions` with `JsonStringEnumConverter`, §6); build `MqttConnection`; connect to broker under a bounded connect deadline (§8); set LWT/clean-session; subscribe to the mode-appropriate filter (§3.1). Publish `STATE` birth if `actAsPrimaryHost` (Sparkplug).
- `ReinitializeAsync` — apply a config delta in place: if only tag membership changed, re-subscribe; if broker connection changed, tear down + reconnect. Never crash the process (marks Faulted → nodes go Bad, per driver-stability Tier A/B).
- `ShutdownAsync` — publish `STATE OFFLINE` (if primary host), disconnect the MQTTnet client, dispose.
- `GetHealth` — Connected/Reconnecting/Faulted + last-message-age; drives ServiceLevel + status dashboard.
- `GetMemoryFootprint`/`FlushOptionalCachesAsync` — footprint = last-value cache + birth caches + alias tables. The runtime driver has **no optional caches to flush**: birth/alias tables are required-for-correctness (flushing them mis-routes aliased DDATA — `IDriver.FlushOptionalCachesAsync` forbids flushing correctness state) and the last-value cache is what `IReadable` serves. Flush is a no-op; all three structures are bounded by the authored tag set + observed births. (The observation browser's accumulating tree lives in the AdminUI-side `MqttBrowseSession`, not in the runtime driver.)
### 3.1 Subscribe (`ISubscribable` — primary)
- **Plain mode:** subscribe to each authored tag's `topic` at its `qos` (dedupe identical topics; coalesce shared wildcards where the operator authored one). On message: match topic → authored tag(s), extract value at `jsonPath` (or raw/scalar), raise `OnDataChange(handle, FullReference, snapshot)`. Seed initial value from the broker's **retained** message replayed on subscribe (OPC UA initial-data convention).
- **Sparkplug mode:** subscribe `spBv1.0/{groupId}/#` (+ the host-state topic when acting as primary host — **`spBv1.0/STATE/{hostId}` with a JSON `{online, timestamp}` payload in spec v3.0**; pre-3.0 peers used top-level `STATE/{hostId}` with `ONLINE`/`OFFLINE` text — target the v3.0 form, tolerate the legacy one on receive). Maintain a **`BirthCache` + `AliasTable` per (edgeNode, device)** from NBIRTH/DBIRTH. On NDATA/DDATA: resolve each metric by **alias → (name, datatype)** via the alias table, map to the authored tag's `FullReference` **by stable metric name**, raise `OnDataChange`. On NDEATH/DDEATH: emit **STALE / Bad-quality** snapshots for that node's/device's metrics. On missed birth / unknown alias / seq gap: issue a **rebirth NCMD** (§3.6).
- `publishingInterval` is **advisory** (MQTT/Sparkplug are event-driven). Use it only for optional server-side coalescing/deadband, mirroring Modbus's `ShouldPublish`. `SubscribeAsync` returns an `ISubscriptionHandle` whose `DiagnosticId` names the mode+filter; `OnDataChange` fires from the MQTT receive handler for the whole driver lifetime.
`SubscribeAsync` maps the requested `FullReference`s onto the live receive stream: it registers them in a `FullReference → MqttTagDefinition` resolver (parsed once, cached — reuse the shared `EquipmentTagRefResolver<TDef>` from `Core.Abstractions`, exactly as Modbus does), so an inbound message/metric routes to the right node.
### 3.2 Read (`IReadable`)
MQTT has no request/response read. `ReadAsync(fullReferences)` returns the **last-value cache** (`LastValueCache`): last message seen on the topic (plain) or last metric value from data/birth (Sparkplug). Seed from the retained message at connect (plain) and from birth values (Sparkplug). If nothing has been observed yet, return a `GoodNoData`/uncertain snapshot **per reference** (never throw the batch — per `IReadable` contract, per-reference status is the snapshot's `StatusCode`). This satisfies OPC UA reads + HistoryRead-AtTime without a live round-trip.
### 3.3 Discover (`ITagDiscovery`) — authored-only
**`DiscoverAsync` replays authored tags only, both modes** — it does **not** auto-provision from wildcard/birth observation. Rationale (from research §2.2): a chatty broker or a large Sparkplug plant would produce an unbounded auto-provisioned address space. Wildcard/birth observation is a **browse-time** concern (§4), not a runtime-provisioning one.
- The operator authors tags (topic+jsonPath, or group/node/device/metric) via the typed editor / observation browser. `DiscoverAsync` streams exactly those authored tags into the `IAddressSpaceBuilder`, each as a `Variable(browseName, displayName, DriverAttributeInfo{ FullName, DriverDataType, ... })`.
- **`RediscoverPolicy`:**
- **Plain:** `Once` (authored set is static per published config — like Modbus).
- **Sparkplug:** `UntilStable` (metric *datatypes* fill in asynchronously as births arrive after connect; keep re-streaming until the authored tags' resolved datatypes stop changing — like FOCAS's FixedTree). This lets a tag authored before its first birth pick up its true datatype once the birth lands.
- **`SupportsOnlineDiscovery => false`** (the default-member from the universal-browser design). This is deliberate and **decoupled from browseability** (§4): runtime discovery is authored-only, so the universal Discover-backed browser would only replay authored tags — useless for discovering *new* topics/metrics. Leaving the flag false keeps MQTT off the universal fallback; the bespoke observation browser (§4) is what backs the picker.
### 3.4 `IRediscoverable`
Sparkplug fires `OnRediscoveryNeeded` when a **new DBIRTH** (a device joins) or a **rebirth with a changed metric set** is observed — the Core re-runs `DiscoverAsync` + diffs. `ScopeHint` = the edge-node/device folder path so the rebuild is surgical. Plain mode never fires it (no backend change signal — a config republish is the only tag-set change), so plain-only drivers effectively no-op this interface.
### 3.5 Data-type mapping
Sparkplug metric datatype (or inferred/authored plain type) → `DriverDataType` (`Boolean, Int16, Int32, Int64, UInt16, UInt32, UInt64, Float32, Float64, String, DateTime, Reference`):
| Sparkplug `DataType` | `DriverDataType` | OPC UA |
|---|---|---|
| Int8, Int16 | `Int16` (Int8 widens) | Int16 |
| Int32 | `Int32` | Int32 |
| UInt8, UInt16 | `UInt16` (UInt8 widens) | UInt16 |
| UInt32 | `UInt32` | UInt32 |
| Int64 | `Int64` | Int64 |
| UInt64 | `UInt64` | UInt64 |
| Float | `Float32` | Float |
| Double | `Float64` | Double |
| Boolean | `Boolean` | Boolean |
| String, Text, UUID | `String` | String |
| DateTime | `DateTime` | DateTime |
| Bytes, File | `String` (base64/raw fallback, v1) | String |
| DataSet, Template | **unsupported v1** — skip + warn, or expose raw-JSON as `String` | — |
| `*Array` variants | element type + `IsArray=true`, `ArrayDim` from birth | ValueRank=1 |
Live map in `SparkplugDataType.ToDriverDataType()` (`.Contracts`). **Plain mode:** honour an explicit per-tag `dataType` (preferred — inference is brittle); fall back to inferring from the JSON token at `jsonPath` (number→`Float64`/`Int64`, bool→`Boolean`, string→`String`). Int8/UInt8 widen (no OPC UA signed-byte distinction needed).
### 3.6 Sparkplug state-machine correctness (the #1 risk)
The alias→metric map, seq-gap detection, late-join rebirth, and death→STALE are load-bearing and easy to get subtly wrong. **Invariants the implementation must hold:**
1. **Bind tags by stable metric NAME, never by alias.** The authored `TagConfig` stores `metricName`; the alias is a **per-birth cache only**. An alias can be reused across a rebirth for a *different* metric — binding by alias silently mis-routes data to the wrong tag.
2. **Rebuild the alias table on every (re)birth.** NBIRTH/DBIRTH replaces the `(edgeNode,device)` `AliasTable` wholesale; never merge.
3. **Track `seq` (0–255, wraps) per edge-node.** A gap (or an unknown alias, or a data message before any birth) ⇒ request rebirth (`RebirthRequester` publishes `NCMD` writing `Node Control/Rebirth = true`), gated on `requestRebirthOnGap`. `bdSeq` ties an NDEATH to its NBIRTH.
4. **Death → STALE.** NDEATH/DDEATH (LWT-driven) emits Bad/uncertain-quality snapshots for all affected metrics; the next birth restores Good.
5. **Late join.** On connect the driver may have missed births — proactively request rebirth (or wait for the retained STATE + natural rebirth) so metadata is recovered. This is the same primitive the browser uses (§4).
Unit-test these explicitly against a controllable simulator: rebirth, missed-birth, seq-gap, alias-reuse-across-rebirth, death→stale→rebirth.
### 3.7 Write (`IWritable`) — **verdict: NOT in v1; phase 3, fire-and-forget optimistic-Good**
Writes are possible but deferred (research §2.4). When shipped in phase 3:
- **Sparkplug:** publish `NCMD`/`DCMD` to `spBv1.0/{group}/NCMD/{node}[/{device}]` with the target metric (by name+alias) + value. **No synchronous write ack** — success is only observable when the edge echoes the change in the next N/DDATA. This mirrors the **Galaxy gateway's fire-and-forget write** ("Galaxy can never surface a write failure"): return `Good` optimistically; the value self-corrects on the next data message. Respect `WriteIdempotent` — Sparkplug commands (rebirth, pulse) are frequently **non-idempotent**, so default retry off.
- **Plain:** publish to a configured command/write topic with a formatted payload.
v1 ships **read/subscribe/discover only** — a genuinely useful UNS-ingest driver. `MqttDriver` does **not** implement `IWritable` in v1 (the interface is composable; its absence means nodes materialize read-only).
---
## 4. Bespoke observation browser
### Decision — reconciling with the universal Discover-backed browser
The universal browser (`docs/plans/2026-07-15-universal-discovery-browser-design.md`) captures whatever `DiscoverAsync` streams and re-presents it, gated on `SupportsOnlineDiscovery`. **MQTT/Sparkplug is the one roadmap driver that needs a bespoke browser despite the universal one**, because:
- Runtime `DiscoverAsync` is **authored-only** (§3.3). A universal Discover-backed browser would replay only tags the operator already authored — it would **not discover new topics/metrics**, which is the entire point of a picker.
- MQTT/Sparkplug discovery is **observational**: you learn what exists only by *listening* for a bounded window. That's a fundamentally different session shape than "connect → one-shot enumerate → disconnect" — the tree **fills in over time**, and Sparkplug can be *accelerated* by forcing a rebirth NCMD.
**Considered + rejected:** feeding the Sparkplug `BirthCache` into `DiscoverAsync` device-enumeration to plug into universal. Rejected because it would make runtime discovery auto-provision from births (the unbounded-address-space failure §3.3 explicitly avoids) and still wouldn't cover plain MQTT. The birth cache stays a browse-time + subscribe-time structure, not a discovery source.
**Decision:** keep `SupportsOnlineDiscovery=false` (authored-only runtime discovery) **and** build a bespoke `IDriverBrowser` (`MqttDriverBrowser`) with an **observation-window `IBrowseSession`** (`MqttBrowseSession`). Registering the bespoke browser for `DriverType="Mqtt"` **overrides the universal fallback** for MQTT (the universal browser resolves bespoke-first — `BrowserSessionService` §3.1 of that doc).
### 4.1 `MqttDriverBrowser : IDriverBrowser`
`DriverType => "Mqtt"`. `OpenAsync(configJson, ct)`:
1. Parse the **same `MqttDriverOptions` JSON** the runtime driver consumes (shared `JsonSerializerOptions`), but connect with a **distinct client-id suffix** (`{clientId}-browse-{guid8}`) so the browse session never collides with the running driver's MQTT session / clean-session state.
2. Connect an MQTTnet client under a bounded connect budget (clamp like OpcUaClient's `PerEndpointConnectTimeout`, 5–30 s).
3. Subscribe to the discovery filter: `spBv1.0/{groupId}/#` (Sparkplug) or `#`/`{topicPrefix}#` (plain).
4. **Kick off the observation window immediately** and, in Sparkplug mode, optionally publish a **rebirth NCMD** to force every edge node to re-announce (turns the passive wait into a near-synchronous enumeration).
5. Return an `MqttBrowseSession` whose internal tree **fills in over the window and keeps filling** while the session lives. Because `IBrowseSession` methods are async and the session lands in the AdminUI `BrowseSessionRegistry` (TTL-reaped, per-call 20 s timeout), the observation cost is amortised across the user's clicks in the picker.
### 4.2 `MqttBrowseSession : IBrowseSession`
Serves from an **accumulating observed tree** (thread-safe; the MQTT receive handler mutates it, browse calls read it). Refreshes `LastUsedUtc` on each call.
- **`RootAsync`** — Sparkplug: observed **groups** (or edge nodes under the configured group). Plain: top-level topic segments. `BrowseNode { Kind=Folder, HasChildrenHint=true }`. If the window hasn't yielded yet, return what's accumulated (possibly empty) and let the user re-expand; in Sparkplug mode trigger a rebirth on first `RootAsync` to populate fast.
- **`ExpandAsync(nodeId)`** — one level down the accumulated tree. Sparkplug: **Group → EdgeNode → Device → Metric**. Plain: next topic segment (split on `/`). Metrics / leaf-topics are `Kind=Leaf`.
- **`AttributesAsync(nodeId)`** — Sparkplug: the metric's `AttributeInfo { Name, DriverDataType (from birth), IsArray, SecurityClass }` — **self-describing**, so the picker side-panel is populated (unlike the OPC UA browser which returns empty). Plain: a single synthetic attribute carrying the leaf topic's **inferred type + last-seen payload snippet**.
- **`DisposeAsync`** — disconnect the MQTTnet client, best-effort (registry reaper may race), same pattern as `OpcUaClientBrowseSession`.
### 4.3 Picked node → TagConfig
On commit the picker projects the selected `BrowseNode`/`AttributeInfo` into the §5 TagConfig JSON:
- **Sparkplug:** the `NodeId` encodes `group/node/device:metric` → split into `groupId`/`edgeNodeId`/`deviceId`/`metricName`; `dataType` from `AttributeInfo`.
- **Plain:** the `NodeId` is the topic path → `topic` + a default `jsonPath` (`$`, operator-editable) + inferred `dataType`.
### 4.4 Addressing the passive/time-bounded wrinkle explicitly
1. Picker shows a live **"listening… N nodes/topics discovered"** affordance so the operator understands coverage is accumulating, not instantaneous.
2. Sparkplug mode **actively forces a rebirth** to convert the passive wait into a fast near-complete enumeration.
3. The browse session stays alive (registry TTL) so re-expanding later reflects newly observed topics without reconnecting.
4. **Manual entry** of a topic/metric via the typed editor (§6) is always available as an escape hatch for a tag that's silent during the window.
### 4.5 Registration
```csharp
// EndpointRouteBuilderExtensions.cs — alongside the existing two (near line 49-50)
services.AddSingleton<IDriverBrowser, OpcUaClientDriverBrowser>();
services.AddSingleton<IDriverBrowser, GalaxyDriverBrowser>();
services.AddSingleton<IDriverBrowser, MqttDriverBrowser>(); // NEW — overrides universal fallback for "Mqtt"
```
---
## 5. TagConfig + driver-config JSON
Two layers, mirroring every other driver: **driver options** (broker connection, one per driver instance, parsed by the factory) and **per-tag config** (`TagConfig` JSON authored on the `/uns` Equipment → Tags tab; the driver's `FullName` is the driver-side reference the router resolves).
### 5.1 Driver options — `MqttDriverOptions` (`.Contracts`)
```jsonc
{
"host": "10.100.0.35",
"port": 8883,
"clientId": "otopcua-uns-1",
"useTls": true,
"allowUntrustedServerCertificate": false, // dev/on-prem only
"caCertificatePath": null, // PEM CA pin; null ⇒ OS trust store
"username": "otopcua",
"password": "", // supply via env, NEVER commit
"protocolVersion": "V500", // "V311" | "V500"
"cleanSession": true,
"keepAliveSeconds": 30,
"connectTimeoutSeconds": 15, // bounded connect deadline (§8)
"reconnectMinBackoffSeconds": 1,
"reconnectMaxBackoffSeconds": 30,
"mode": "SparkplugB", // "Plain" | "SparkplugB"
// ---- Sparkplug-only (null in Plain mode) ----
"sparkplug": {
"groupId": "Plant1", // subscribe spBv1.0/Plant1/#
"hostId": "otopcua-host-1", // spBv1.0/STATE/{hostId} identity (v3.0)
"actAsPrimaryHost": false, // at most ONE primary host per host-id per broker
"requestRebirthOnGap": true,
"birthObservationWindowSeconds": 15 // browse/discovery: how long to collect births
},
// ---- Plain-only (null in SparkplugB mode) ----
"plain": {
"topicPrefix": "factory/",
"defaultQos": 1
}
}
```
### 5.2 Per-tag — Sparkplug
```jsonc
{
"FullName": "Plant1/Line3EdgeNode/Filler1:Temperature/degC", // PascalCase key (composer/walker contract)
"groupId": "Plant1",
"edgeNodeId": "Line3EdgeNode",
"deviceId": "Filler1", // omit/null for node-level metrics
"metricName": "Temperature/degC", // STABLE identity across rebirths — bind by this, not alias
"dataType": "Float" // from DBIRTH; explicit for safety
}
```
`FullName` convention: `"{group}/{node}[/{device}]:{metric}"`. The driver parses `FullName` (or the descriptor fields) once, caches it, and resolves inbound metrics by **`(group,node,device,metricName)`**.
### 5.3 Per-tag — Plain MQTT
```jsonc
{
"FullName": "factory/line3/oven/temp#$.value", // topic#jsonPath — stable per-tag key
"topic": "factory/line3/oven/temp",
"payloadFormat": "Json", // "Json" | "Raw" | "Scalar"
"jsonPath": "$.value", // JSONPath into payload (Json only)
"dataType": "Double", // explicit — inference is the fallback
"qos": 1,
"retainSeed": true // seed last-value from retained msg on subscribe
}
```
`FullName` convention: `"{topic}#{jsonPath}"` (Json) or just `"{topic}"` (Raw/Scalar). `MqttEquipmentTagParser.TryParse(reference, out def)` follows `ModbusEquipmentTagParser`: a leading `{` marks a TagConfig blob; parse it, use **strict enum reads** (`TagConfigJson.TryReadEnumStrict`) so a typo'd `payloadFormat`/`dataType` rejects the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a wrong type; the def's `Name` = the reference string so published values key the forward router. Add an `Inspect(reference)` deploy-time warning pass (like Modbus) for present-but-invalid enums / unparseable blobs.
**Historization:** the server-side `isHistorized` / `historianTagname` keys are owned by the TagModal-merge seam (`TagHistorizeConfig`) and survive load→save of the typed model as preserved unknown keys — the driver does not model them (identical to `OpcUaClientTagConfigModel`).
---
## 6. Typed editor + validator
### 6.1 `MqttTagConfigModel` (`.AdminUI/Uns/TagEditors/`)
Thin typed working model over a preserved `JsonObject` key bag, per the `OpcUaClientTagConfigModel` template: `FromJson(json)` / `ToJson()` / `Validate()`, **preserving unknown keys** across a load→save (so history keys + fields this editor doesn't expose survive). It carries **both** mode shapes — a `Mode` field (Plain|SparkplugB, defaulted from which fields are present) selects which sub-shape the editor renders + validates.
- `FromJson` reads the descriptor fields for the active mode plus the always-present `FullName` (PascalCase, case-sensitive — the composer/artifact-decoder/walker read it via case-sensitive `TryGetProperty("FullName", …)`).
- `ToJson` writes `FullName` PascalCase and re-derives it from the descriptor fields (`{group}/{node}[/{device}]:{metric}` or `{topic}#{jsonPath}`) so a hand-edited descriptor stays consistent with the routed key.
- `Validate()`:
- **Sparkplug:** `groupId`, `edgeNodeId`, `metricName` required; `deviceId` optional; `dataType` must be a known Sparkplug type.
- **Plain:** `topic` required and non-wildcard (a subscription topic for a *tag* must be concrete — reject `+`/`#`); `payloadFormat=Json` ⇒ `jsonPath` required; `dataType` a known `DriverDataType`.
### 6.2 `MqttTagConfigEditor.razor`
Copy the Modbus editor template (`Components/Shared/Uns/TagEditors/`): a razor shell over the pure model, mode-switch at top, per-mode field group below, client-side validation via `Validate()`. Register:
```csharp
// TagConfigEditorMap.cs
["Mqtt"] = typeof(Components.Shared.Uns.TagEditors.MqttTagConfigEditor),
// TagConfigValidator.cs
["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate(),
```
### 6.3 The `JsonStringEnumConverter` enum-serialization trap
**Every** JSON seam that (de)serializes `MqttDriverOptions` / tag configs — factory, probe, browser, **and the AdminUI driver-config page + probe DTO** — MUST use a `JsonSerializerOptions` that includes `new JsonStringEnumConverter()` (plus `PropertyNameCaseInsensitive=true`, `UnmappedMemberHandling=Skip`). The repo's SYSTEMIC bug (memory: "Driver enum-serialization bug"): AdminUI pages that serialize enums **numerically** while factory DTOs are **string-typed** fault the driver on any enum field. `MqttMode`, `MqttPayloadFormat`, `protocolVersion` are all enums — the AdminUI Mqtt driver page and its probe must serialize them as **strings** (mirror the OpcUaClient page). `JsonStringEnumConverter` also accepts numeric ordinals, so existing numeric configs still load. Keep one shared `JsonSerializerOptions` definition referenced by factory + probe + browser (as OpcUaClient does).
---
## 7. Factory + registration
`MqttDriverFactoryExtensions` (mirror `OpcUaClientDriverFactoryExtensions`):
```csharp
public static class MqttDriverFactoryExtensions
{
public const string DriverTypeName = "Mqtt";
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip,
Converters = { new JsonStringEnumConverter() },
};
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory));
}
public static MqttDriver CreateInstance(string driverInstanceId, string driverConfigJson, ILoggerFactory? lf = null)
{
var options = JsonSerializer.Deserialize<MqttDriverOptions>(driverConfigJson, JsonOptions)
?? throw new InvalidOperationException($"Mqtt driver config for '{driverInstanceId}' deserialised to null");
return new MqttDriver(options, driverInstanceId, lf?.CreateLogger<MqttDriver>());
}
}
```
Wire in the two host bootstrap sites:
```csharp
// Host/Drivers/DriverFactoryBootstrap.cs — factory registration (near line 133)
Driver.Mqtt.MqttDriverFactoryExtensions.Register(registry, loggerFactory);
// … and probe registration in AddOtOpcUaDriverProbes (near line 114)
services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, Driver.Mqtt.MqttDriverProbe>());
```
**`MqttDriverProbe : IDriverProbe`** (`DriverType => "Mqtt"`): parse the config with the **shared** `JsonSerializerOptions`, open an MQTTnet CONNECT to `host:port` under the timeout, return green + latency on CONNACK-accepted, or a targeted error (connection refused / TLS failure / auth failure / timeout). Analogous to `ModbusDriverProbe`'s TCP+FC03 handshake — CONNACK is the MQTT "device is answering" proof. **Probes are admin-pinned** (the `admin-operations` singleton dispatches by DriverType) — the `TryAddEnumerable` line must be on admin nodes (it is, via `AddOtOpcUaDriverProbes` in `Program.cs`'s `hasAdmin` block).
---
## 8. Resilience / timeout / security
- **Bounded connect.** `InitializeAsync` connects under `connectTimeoutSeconds` (default 15) via a linked CTS — **no unbounded wait** on a dead broker (heed the S7/FOCAS read-timeout lessons: an async network call that ignores its deadline wedges the whole driver).
- **Reconnect with backoff.** MQTTnet's managed reconnect (or a hand-rolled loop) with exponential backoff `reconnectMinBackoffSeconds → reconnectMaxBackoffSeconds`. On disconnect: `GetHealth` → Reconnecting; on reconnect: re-subscribe, and in Sparkplug mode **request rebirth** to recover metadata (late-join, §3.6). Nodes go Bad-quality while disconnected via the standard fan-out.
- **Subscribe deadline.** `SubscribeAsync` completes under a bounded deadline; SUBACK failure surfaces as a per-reference Bad, not a hang.
- **No poll loop** — the receive handler must be non-blocking (offload heavy decode off the MQTT callback thread; never block the MQTTnet dispatcher).
- **Broker security — enforce, never ship public-broker defaults.** Default `useTls=true`; real auth (username/password or client-cert). `allowUntrustedServerCertificate` + `caCertificatePath` mirror the ServerHistorian TLS knobs (dev/on-prem escape hatch, off by default). **Never** put Sparkplug production-shaped data on public brokers (`test.mosquitto.org` etc. — manual smoke only). Credentials come from env (`password` blank in committed JSON), same posture as `ServerHistorian__ApiKey`.
- **Primary-host guard.** At most **one** primary-host client per `hostId` per broker (Sparkplug MUST). `actAsPrimaryHost=true` on two OtOpcUa instances sharing a `hostId` (e.g. a redundancy pair) is a misconfiguration — the redundancy pair should use **distinct** host-ids (mirrors the distinct `MxAccess.ClientName` redundancy rule).
---
## 9. Test fixtures
Follow the driver-fixture pattern: `tests/Drivers/.../Docker/docker-compose.yml` with the `project: lmxopcua` label, deployed to `10.100.0.35` via `lmxopcua-fix sync/up` (`CLAUDE.md` Docker Workflow). Repo files are source-of-truth; `/opt/otopcua-mqtt/` is the mirrored deployment.
- **Broker:** **Eclipse Mosquitto** (`eclipse-mosquitto`, tiny, carries both plain + Sparkplug unchanged) as the default fixture; **EMQX** (`emqx/emqx`, Apache-2.0 CE) as the alternative when a dashboard / built-in Sparkplug tooling helps. Configure with **auth + TLS** so the fixtures exercise the real security path (not anonymous).
- **Sparkplug edge-node simulator:** prefer a **project-owned C# simulator** using the same MQTTnet + Tahu-proto path the driver uses — validates encode/decode symmetrically and gives full control of rebirth/seq-gap/death scenarios (the §3.6 test matrix). Alternatives: Eclipse Tahu reference (Java/Python) edge-node script; Bevywise MQTT/IoT Simulator (purpose-built Sparkplug B). The simulator must publish NBIRTH/DBIRTH → periodic NDATA/DDATA, honour a rebirth NCMD, and be able to inject a seq gap / alias reuse / N-D-DEATH on demand.
- **Plain-MQTT fixture:** a tiny publisher container (`mosquitto_pub` loop or a `paho-mqtt` script) emitting JSON on a handful of topics with `retain=true` so the last-value/read path **and** the `#`-observation browse are testable.
- **Env-gated live suite:** `Driver.Mqtt.IntegrationTests`, category `LiveIntegration` (à la HistorianGateway), skips cleanly when **`MQTT_FIXTURE_ENDPOINT`** is unset → macOS-offline-safe. Covers: connect/TLS/auth; plain subscribe+retained-read; Sparkplug birth→data→alias-resolve→`OnDataChange`; rebirth recovery; death→STALE; the browser observation window (birth-driven tree + rebirth-forced enumeration).
- **Live-verify (docker-dev):** author an Mqtt driver + tag via the `/uns` picker on `:9200`, deploy, confirm the node carries live broker values and the typed editor renders both modes (the usual live-`/run` discipline — Razor binding bugs pass unit tests).
Add the fixture endpoint to `CLAUDE.md`'s endpoint list once landed (e.g. `10.100.0.35:1883`/`8883`, `MQTT_FIXTURE_ENDPOINT`).
---
## 10. Phasing + effort
Order: **plain MQTT first, then Sparkplug B, then write.** Plain MQTT is the dependency-light slice that stands up the whole skeleton and **validates MQTTnet-v5-on-net10 before any Sparkplug/proto work** — front-loading the reusable plumbing and de-risking the library decision.
| Phase | Scope | Rel. effort |
|---|---|---|
| **P1 — Plain MQTT** | 3 projects; MQTTnet-5 connect/TLS/auth/reconnect+backoff (`MqttConnection`); `ISubscribable` (topic subscribe → `OnDataChange`); `IReadable` last-value (retained seed); authored-tag `ITagDiscovery` (`Once`); `IHostConnectivityProbe`; `MqttDriverProbe`; factory + host wiring; `#`-observation browser (`MqttDriverBrowser`/`MqttBrowseSession`); typed editor + validator + AdminUI driver page; Mosquitto + JSON-publisher fixtures + env-gated suite. **Validate MQTTnet-5/net10 here.** | **M** |
| **P2 — Sparkplug B ingest** | Vendored Tahu proto + `Google.Protobuf` codegen; `SparkplugCodec` decode; `spBv1.0/{group}/#` subscribe; N/DBIRTH → `BirthCache`/`AliasTable`; bind-by-name + seq-gap detect + **rebirth NCMD**; N/DDEATH → STALE; STATE/primary-host option; `ITagDiscovery` `UntilStable` + `IRediscoverable`; birth-driven browser tree + `AttributesAsync`; Sparkplug datatype map; C# edge-node simulator fixture + §3.6 test matrix. | **L** |
| **P3 — Write-through (optional)** | `IWritable`: Sparkplug NCMD/DCMD + plain publish; optimistic-Good + self-correct on echo; `WriteIdempotent` respect (default off for non-idempotent Sparkplug commands); write-topic config. | **M** |
**Top risks:** (1) **Sparkplug state-machine correctness** (aliases + rebirth + seq — §3.6; bind-by-name is the mitigation, test against a controllable simulator); (2) **library/dependency friction** — SparkplugNet's MQTTnet-4.x transitive pin ✕ this repo's fragile central pinning (transitive pinning off), mitigated by **hand-rolling the Tahu proto over one MQTTnet-5 client** (§2.1). Secondary: passive-browse coverage gaps (rebirth-forcing + manual entry), unbounded address space (authored-only runtime discovery), write-has-no-ack (accepted, optimistic-Good like Galaxy), broker security (TLS + real auth enforced).
docs/plans/2026-07-15-mtconnect-driver-design.md
+182
View File
@@ -0,0 +1,182 @@
# MTConnect (Agent-first) driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Research input:** [`docs/research/drivers/mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) — this doc turns that research into a build spec; it does **not** re-argue the protocol findings.
**House-style references:** [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md), [`2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
---
## 1. Motivation + scope
MTConnect is the dominant open read-only telemetry standard for machine tools; adding a `DriverType = "MTConnect"` Equipment-kind driver lets OtOpcUa surface a machine's self-describing device model (positions, spindle speed, execution state, availability, conditions) alongside — and complementing — the lower-level **FOCAS** driver (FOCAS reaches the Fanuc CNC directly; MTConnect reaches the vendor-neutral Agent that often front-ends that same machine). v1 is **agent-first, read-only** (Discover + Read + Subscribe, no Write) because the mainstream MTConnect surface (`/probe`, `/current`, `/sample`) is read-only by design. Full rationale, protocol details, library survey, and risk register live in the [research report](../research/drivers/mtconnect-agent.md) §1–§8; this doc assumes them.
## 2. Project layout
Two new projects, mirroring the Modbus split (contracts DTOs isolated from runtime so the AdminUI probe/editor can reference config shapes without the driver's NuGet deps). **No `.Browser` project** — see §4.
```
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/
MTConnectDriverOptions.cs // strongly-typed runtime options (like ModbusDriverOptions)
MTConnectTagDefinition.cs // one record per authored tag (FullName=dataItemId + mt* metadata)
MTConnectDataTypeInference.cs // pure category/type/units → DriverDataType table (§3.3) — shared by driver, factory, editor, browser-commit
ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj // refs Core.Abstractions only
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/
MTConnectDriver.cs // IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable
MTConnectAgentClient.cs // thin seam over MTConnectHttpClient (probe/current/sample); testable via IMTConnectAgentClient
IMTConnectAgentClient.cs // seam interface — canned-XML fake in unit tests
MTConnectObservationIndex.cs // dataItemId → latest DataValueSnapshot, updated by /current + /sample
MTConnectDriverFactoryExtensions.cs // Register(registry, loggerFactory) + CreateInstance
MTConnectDriverProbe.cs // IDriverProbe: one-shot /probe reachability
ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj // refs Core, Core.Abstractions, .Contracts + TrakHound pkgs
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/
Fixtures/*.xml // canned probe/current/sample docs (§8)
...Tests.cs
```
`.csproj` boilerplate copies Modbus verbatim: `net10.0`, `Nullable=enable`, `TreatWarningsAsErrors=true`, `GenerateDocumentationFile`, `InternalsVisibleTo` the Tests project.
### NuGet dependency
Consume **TrakHound MTConnect.NET client packages (MIT)** — pin an exact version:
```xml
<PackageReference Include="MTConnect.NET-Common" Version="6.9.0.2" />
<PackageReference Include="MTConnect.NET-HTTP" Version="6.9.0.2" />
<!-- MTConnect.NET-XML / -JSON pulled transitively for serialization -->
```
These target `netstandard2.0` (load cleanly on .NET 10) and give `MTConnectHttpClient` (probe/current/sample with polling **and** the multipart long-poll stream, gzip, XML+JSON) plus the model types (`IDevice`/`IComponent`/`IDataItem`/`IObservation`) — removing the multipart-framing / version-negotiation / buffer-overflow-rebaseline grind.
> **License-pin caveat (from research §1.4):** older TrakHound releases carried mixed MIT / Apache-2.0 / "all rights reserved" strings. Before merge, confirm the embedded `LICENSE` of the *pinned* version is MIT (a legal-review checkbox, not a blocker). **Hand-roll fallback** if review fails: `HttpClient` + `System.Xml.Linq` for probe/current, a `multipart/x-mixed-replace` boundary reader for sample (~300–500 LoC behind the same `IMTConnectAgentClient` seam — the driver above the seam is unchanged, so the fallback is a drop-in of one class).
## 3. Capability mapping (concrete seam wiring)
`MTConnectDriver` implements `IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable` — **not** `IWritable` (§3.5). One `MTConnectAgentClient` + one `MTConnectObservationIndex` per driver instance; one shared sample stream (the Agent streams the whole device model, so per-tag streams would be wasteful).
### 3.1 `IDriver`
- `InitializeAsync(json, ct)`: deserialize `MTConnectDriverOptions` (via factory), construct `MTConnectAgentClient` from `AgentUri`, run **one `/probe`** under a per-call deadline (§7). Cache the parsed `IDevice[]` model + `Header.instanceId`. Prime the `MTConnectObservationIndex` with one `/current`. Set `DriverState.Healthy` on success; `Faulted` (rethrow) on failure — the actor marks the instance Faulted, nodes go Bad, process stays up.
- `ReinitializeAsync`: tear down the sample stream, re-run Initialize. Config-only change (no address-space rebuild) unless `AgentUri`/`DeviceName` changed.
- `ShutdownAsync`: stop the sample stream, dispose the HTTP client.
- `GetHealth`: `DriverHealth(State, LastSuccessfulRead, LastError)` — `LastSuccessfulRead` = last `/current` or `/sample` chunk time.
- `GetMemoryFootprint` / `FlushOptionalCachesAsync`: footprint ≈ cached probe model + observation index; flush drops the browse/probe-model cache (re-fetchable), keeps the observation index (correctness state).
### 3.2 `ITagDiscovery` — plugs into the universal browser (§4)
```csharp
public bool SupportsOnlineDiscovery => true; // /probe enumerates from the device
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once; // probe is synchronous + complete
public async Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken ct)
```
`DiscoverAsync` fetches (or reuses the Initialize-cached) `/probe` model and **streams the full Device→Component→DataItem tree into the builder**:
- each `Device` → `builder.Folder(device.Name, device.Name)` → child builder;
- each nested `Component` → recursive `Folder(component.Name, component.DisplayName)` on the parent's child builder (component nesting becomes folder nesting — the builder-graph *is* the tree);
- each `DataItem` → `child.Variable(browseName, displayName, attr)` where
`attr = new DriverAttributeInfo(FullName: dataItem.Id, DriverDataType: MTConnectDataTypeInference.Infer(category, type, units), IsArray: representation==TIME_SERIES, ArrayDim: null, SecurityClass: SecurityClassification.ViewOnly, IsHistorized: false, IsAlarm: category==CONDITION)`.
(`ViewOnly` is the read-only-from-OPC-UA tier — `SecurityClassification` has no `ReadOnly` member.)
`browseName` = `dataItem.Name ?? dataItem.Id` (dataItem `name` is optional in MTConnect; fall back to `id`, which is guaranteed unique).
**Critical:** `DriverAttributeInfo.FullName = dataItem.Id`. This is (a) the value the universal browser commits as `TagConfig.FullName`, and (b) the key the read/subscribe paths resolve against — the two align by construction (research §3). If `DeviceName` scopes to one device, only that device's subtree is streamed.
`DeviceName` scoping and the whole-model-in-one-call nature make this a natural fit for **eager one-shot** discovery — exactly what the universal browser assumes.
### 3.3 Data-type mapping (`MTConnectDataTypeInference.Infer`)
Pure function in `.Contracts` (shared by driver, browser-commit, and the typed editor so all three agree). Weak wire typing means this is a heuristic, stored per-tag and **author-overridable** (§5):
| MTConnect | `DriverDataType` |
|---|---|
| `SAMPLE` numeric (has `units`) | `Float64` |
| `SAMPLE` `representation=TIME_SERIES` | `Float64` array (`IsArray=true`) |
| `EVENT` numeric type (`PartCount`, `Line`, …) | `Int64` |
| `EVENT` controlled-vocab (`Execution`, `ControllerMode`, `Availability`, …) | `String` |
| `EVENT` free text (`Program`, `Block`, `Message`) | `String` |
| `CONDITION` | `String` (state word; §3.6) |
**Quality mapping (`DataValueSnapshot.StatusCode`):** an observation value of `UNAVAILABLE` (MTConnect's explicit no-data sentinel) → OPC UA **`Bad`/`Uncertain`**, not the literal string. Missing dataItem / empty condition → `Bad`. `observation.timestamp` → `SourceTimestamp` (not a separate node).
### 3.4 `IReadable` — `/current`
`ReadAsync(fullReferences, ct)`: issue one `/current` under a per-call deadline, index the returned `MTConnectStreams` observations by `dataItemId` into `MTConnectObservationIndex`, then return **one `DataValueSnapshot` per requested ref in order**. A ref absent from the response → a `Bad`-coded snapshot (not a throw). Reads are idempotent (matches `IReadable` contract; `DriverCapability.Read` auto-retries). `/current` is the whole-device snapshot regardless of how many refs are requested, so batch reads cost one round-trip.
### 3.5 `ISubscribable` — `/sample` multipart long-poll
- `SubscribeAsync(fullReferences, publishingInterval, ct)`: on the **first** subscription, start the shared `MTConnectAgentClient` sample stream (`/sample?from=<nextSequence>&interval=<SampleIntervalMs>&count=<SampleCount>`). Record the subscribed ref set. Immediately fire `OnDataChange` for each subscribed ref from the primed `/current` values (OPC UA initial-data convention). Return an `ISubscriptionHandle` (monotonic id + `DiagnosticId`).
- Stream pump: each received `MTConnectStreams` chunk → for each observation whose `dataItemId` ∈ subscribed set, update the index and raise `OnDataChange(handle, dataItemId, snapshot)`. Advance `from = Header.nextSequence` for the next request (contiguous, gap-free).
- **Ring-buffer overflow:** if the Agent reports a sequence gap / `from` older than `firstSequence`, re-`/current` to re-baseline, then resume the stream from the new `nextSequence`. (TrakHound handles this internally; the hand-roll fallback must replicate it — unit-tested via a fixture with a forced gap.)
- `UnsubscribeAsync(handle, ct)`: drop that handle's refs from the subscribed set; stop the shared stream when the set empties.
- `publishingInterval` maps to the `/sample` `interval` when it is the only/first subscription; the driver polls at the finest requested interval and fans out (one stream per instance, not per tag).
### 3.6 `IWritable` — **not implemented (v1)**
Justified per research §2.1: the MTConnect Agent surface is read-only by design; write-back exists only via optional, rarely-deployed MTConnect *Interfaces* (a request/response state-machine handshake, not a "set value" — modelling it as `IWritable` would mislead). Capability interfaces are composable, so omitting `IWritable` is idiomatic (Galaxy's write path is even fire-and-forget). Nodes materialize **without** the `AccessLevels.CurrentWrite` bit automatically. Revisit Interfaces only if a concrete deployment needs it.
### 3.7 `IHostConnectivityProbe` + `IRediscoverable`
- `IHostConnectivityProbe`: expose one `HostConnectivityStatus` per Agent (HostName = `AgentUri`). A cheap periodic `/probe` (or reuse the sample-stream heartbeat) flips `Running ↔ Stopped`; raise `OnHostStatusChanged` on transition. Enable/interval from `MTConnectDriverOptions.Probe` (mirrors `ModbusProbeOptions`).
- `IRediscoverable`: watch the Agent `Header.instanceId` on every `/current`/`/sample` chunk. A change means the Agent restarted / its model changed → raise `OnRediscoveryNeeded` so `DriverHost` rebuilds the address space (mirrors Galaxy's `DeployWatcher`). This is why `RediscoverPolicy = Once` is safe: instanceId change, not polling, drives re-discovery.
### 3.8 CONDITION modelling
**v1 simple (this design):** each `CONDITION` DataItem is a `String` variable whose value is the current state word (`Normal`/`Warning`/`Fault`/`Unavailable`), optionally suffixed with `nativeCode`. Zero alarm plumbing. `IsAlarm=true` is still set on the `DriverAttributeInfo` so the browser side-panel flags it and a future upgrade needn't re-author tags.
**v1.5 (fast-follow, not in this build):** implement `IAlarmSource` + emit a TagConfig `alarm` object so Fault/Warning become native OPC UA Part 9 alarms, routing on the authored dotted `ConditionId` per the Galaxy/Phase-B native-alarm pattern.
## 4. Browse — plugs into the universal browser (reconciliation)
**The research report (§4.2) proposed a bespoke `ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser` project with a hand-written `MTConnectBrowseSession`. This design supersedes that: build NO bespoke browser.** With the [universal discovery browser](2026-07-15-universal-discovery-browser-design.md) in place (the Wave-0 gate, which lands before this Wave-2 driver), MTConnect browse is covered for free by the generic `DiscoveryDriverBrowser`, because **MTConnect browse comes from discovery** (`/probe` → the device model → the same `DiscoverAsync` stream). Concretely, the universal browser already does exactly what the proposed bespoke session would have done:
- `DiscoveryDriverBrowser.CanBrowse("MTConnect", cfg)` returns true because `TryCreate` succeeds and the instance is `ITagDiscovery { SupportsOnlineDiscovery: true }` (§3.2) → the AdminUI renders the **Browse** button.
- `OpenAsync` constructs the driver, `InitializeAsync` (the `/probe` connect), runs `DiscoverAsync` into a `CapturingAddressSpaceBuilder`, then `ShutdownAsync` — the captured tree (Device folders → Component folders → DataItem leaves) is served by `CapturedTreeBrowseSession`. Each leaf's `NodeId == DriverAttributeInfo.FullName == dataItemId`, committed directly as `TagConfig.FullName`; `IsAlarm`/`DriverDataType` flow to the side-panel and pre-fill the typed editor.
**No `PatchForBrowse` entry is needed** — MTConnect discovery is unconditional (not config-gated like AbCip/TwinCAT's `EnableControllerBrowse`); it belongs in the "no patch" set alongside OpcUaClient/BACnet. **No `IUniversalDriverBrowser`/`BrowserSessionService` change is needed** — MTConnect lights up purely by setting `SupportsOnlineDiscovery => true`. The single required action to enable browse is that one default-member override in §3.2.
The only limit (universal-browser §8) is that discovery is eager one-shot; `/probe` is already a whole-model single call, so eager is the right fit and graduating to bespoke is unlikely.
## 5. Typed tag editor + validator (AdminUI)
Avoid the raw-JSON fallback by adding a typed editor (mirrors the Modbus template exactly).
- `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs` — pure `FromJson`/`ToJson`/`Validate`, **preserves unknown keys** via the `TagConfigJson` bag helper (copy `ModbusTagConfigModel`). Fields: `FullName` (dataItemId, required), `MtCategory`/`MtType`/`MtSubType` (enums / strings, from probe — read-only in UI), `DataType` (`DriverDataType` override dropdown), `Units`, `MtDevice`/`MtComponent` (author context). `Validate()` returns an error if `FullName` is blank.
- `.../Components/Shared/Uns/TagEditors/MTConnectTagConfigEditor.razor` — thin shell: `FullName` text, read-only `mtCategory/mtType`, a `DataType` override `<select>`. Because most tags arrive via the browse picker (which fills the model), the editor is mostly a confirm/override surface.
- Register `["MTConnect"] = typeof(Components.Shared.Uns.TagEditors.MTConnectTagConfigEditor)` in `Uns/TagEditors/TagConfigEditorMap.cs`, and `["MTConnect"] = j => MTConnectTagConfigModel.FromJson(j).Validate()` in `TagConfigValidator.cs`.
> **`JsonStringEnumConverter` enum-serialization trap (project-wide gotcha — MEMORY).** Every driver's AdminUI serialization path must round-trip enums as **name strings**, never numerics. The factory/probe DTOs are string-typed (`ParseEnum<T>`), so a numerically-serialized enum FAULTS the driver at deploy. Two concrete requirements: (1) `MTConnectTagConfigModel.ToJson` writes `MtCategory`/`DataType` via `TagConfigJson.Set(bag, "dataType", DataType)` which emits the enum **name** (the Modbus helper already does this — do not hand-roll `JsonSerializer` without `JsonStringEnumConverter`); (2) `MTConnectDriverProbe` parses with `new JsonStringEnumConverter()` in its `JsonSerializerOptions` (copy `ModbusDriverProbe._opts`); the factory keeps enum-carrying DTO fields `string?` and parses them via `ParseEnum<T>` — the Modbus factory pattern (its `JsonOptions` carries no enum converter). The `.Contracts` `MTConnectDataTypeInference` returns a `DriverDataType` enum; when it's written to JSON it must be the string.
## 6. Factory + registration
- `MTConnectDriverFactoryExtensions` (copy Modbus): `public const string DriverTypeName = "MTConnect";` `Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)` → `registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory))`. `CreateInstance` deserializes a `MTConnectDriverConfigDto` (nullable-init DTO with `Tags`, `Probe`), validates `AgentUri` present, builds `MTConnectDriverOptions`, returns `new MTConnectDriver(options, id, agentClientFactory: null, logger)`. Copy the Modbus factory `JsonOptions` (`PropertyNameCaseInsensitive`, `ReadCommentHandling=Skip`, `AllowTrailingCommas` — no enum converter: enum-carrying DTO fields stay `string?` and go through `ParseEnum<T>`, per §5's gotcha).
- **Host wiring — 3 one-line edits in `src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs`:**
1. `Register(...)` body: add `Driver.MTConnect.MTConnectDriverFactoryExtensions.Register(registry, loggerFactory);`
2. add `using MTConnectProbe = Driver.MTConnect.MTConnectDriverProbe;`
3. `AddOtOpcUaDriverProbes`: add `services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, MTConnectProbe>());`
The probe MUST be in `AddOtOpcUaDriverProbes` (not only the factory path) so it reaches **admin-only nodes** — the `admin-operations` singleton that backs Test-Connect is admin-role-pinned (the MEMORY "driver probes on admin nodes" gotcha). `TryAddEnumerable` keeps a fused admin,driver node from double-registering (a dup makes the singleton's `ToDictionary(p=>p.DriverType)` throw).
- `MTConnectDriverProbe : IDriverProbe`, `DriverType => "MTConnect"`: parse the config DTO, one-shot `GET {AgentUri}/probe` under `timeout`, `Ok=true`+latency on any valid `MTConnectDevices` response, `Ok=false`+message on TCP/HTTP/timeout failure. **Never throw** (per `IDriverProbe` contract).
- **No AdminUI `IDriverBrowser` DI line** — browse is the universal browser (§4), already registered.
## 7. Resilience / timeout (the R2-01 frozen-peer lesson)
Every agent HTTP request MUST carry a **per-call deadline** — never an unbounded wait (the R2-01 S7 finding: an async read that ignores its socket timeout wedges the poll loop against a frozen peer). Concretely:
- **`/probe` and `/current`:** wrap each call in a `CancellationTokenSource(RequestTimeoutMs)` linked to the caller's `ct`; a timeout surfaces as a `Bad`-coded snapshot (read) or a Faulted init (probe), never a hang. Set `HttpClient.Timeout` **and** a linked CTS (belt-and-suspenders — `HttpClient.Timeout` doesn't cover the response-body read of a streamed multipart).
- **`/sample` long-poll stream watchdog:** the stream is intentionally long-lived, so `HttpClient.Timeout` cannot bound it. Instead run a **watchdog**: if no chunk **and** no keep-alive heartbeat arrives within `HeartbeatMs × N` (e.g. 3× the Agent heartbeat), treat the stream as dead → cancel it, transition `HostState.Stopped`, and reconnect. The Agent's own `heartbeat` on the multipart boundary is the liveness signal; absence past the watchdog window is the frozen-peer detector.
- **Reconnect / backoff:** geometric backoff (`MinBackoffMs`→`MaxBackoffMs`) on stream drop or connect failure, mirroring `ModbusReconnectOptions`. On reconnect, re-baseline via `/current` then resume `/sample` from the fresh `nextSequence`.
- All capability calls flow through the existing Phase-6.1 `IDriverCapabilityInvoker` resilience pipeline (Read/Discover/Subscribe/Probe auto-retry); the per-call deadlines above are the driver-internal floor beneath that pipeline.
## 8. Test fixtures
1. **Canned XML unit fixtures (bulk of coverage, no network).** Capture one `MTConnectDevices` (probe) + a couple of `MTConnectStreams` (current + sample, incl. one with a forced `nextSequence` gap) from a demo agent into `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/`. Drive through the `IMTConnectAgentClient` fake to pin: `DiscoverAsync` tree shape + leaf `FullName==dataItemId`, `MTConnectDataTypeInference` table, observation indexing, `UNAVAILABLE→Bad`, CONDITION→state-word, multipart chunk framing, and ring-buffer re-baseline paging.
2. **Dockerized `mtconnect/cppagent` (reproducible integration fixture).** Add `tests/.../Docker/docker-compose.yml` with the **`project: lmxopcua`** label on every service, seeded with a canned `Devices.xml` (+ an SHDR simulator or the built-in adapter), exposed on the shared docker host `10.100.0.35`; drive it via `lmxopcua-fix up mtconnect` + `sync`. Env-gated integration suite (`*.IntegrationTests`), skips cleanly when the fixture is down — the analog of the Modbus/S7 sims.
3. **Public demo agent (manual live smoke only).** `https://demo.mtconnect.org/` (or NIST `smstestbed` `Devices.xml`) for a real-shape browse/read/stream smoke. Internet-dependent, not in CI.
Live-verify the browse picker on docker-dev per the universal-browser discipline: open the `/uns` TagModal picker for an MTConnect driver, confirm the Device→Component→DataItem tree renders and a picked leaf commits `TagConfig.FullName = <dataItemId>` (Razor binding bugs pass unit tests — always `/run` it).
## 9. Phasing + effort
- **P1 (this design) — Agent MVP, ~1–1.5 wk with TrakHound (~2.5–3 wk hand-rolled):** `.Contracts` + `Driver.MTConnect` (`IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+`IHostConnectivityProbe`+`IRediscoverable`), `MTConnectDriverProbe`, `SupportsOnlineDiscovery=true` (browse is free via universal browser — no browser code), typed editor + map/validator entries, 3-line Host registration, canned-XML unit suite + cppagent fixture. CONDITION as `String`.
- **P1.5 fast-follow:** CONDITION → native Part-9 alarms via `IAlarmSource` (Galaxy pattern); `TIME_SERIES` SAMPLE arrays; EVENT controlled-vocab → OPC UA enumerations.
- **P2 (on demand) — SHDR adapter ingest:** add `SourceMode: "Agent" | "Shdr"`; SHDR opens the raw pipe-delimited TCP socket (`:7878`). **Loses auto-discovery** (no device model → `SupportsOnlineDiscovery` must return `false` in SHDR mode → picker falls back to manual entry; tags authored like Modbus). Niche.
Detailed risk register (weak typing, CONDITION analog, TrakHound license, demo-agent CI flakiness, ring-buffer overflow, netstandard2.0-on-.NET10) is in [research §7](../research/drivers/mtconnect-agent.md#7-effort--risk--phasing) — not repeated here.
docs/plans/2026-07-15-omron-driver-design.md
+524
View File
@@ -0,0 +1,524 @@
# Omron PLC Driver — Executable Implementation Design
**Status:** Build-ready design, 2026-07-15. Turns the research report
[`docs/research/drivers/omron.md`](../research/drivers/omron.md) into a concrete implementation plan.
**Kind:** Standard **Equipment-kind driver** (same shape as Modbus / S7 / AbCip / TwinCAT / FOCAS).
Points are ordinary equipment `Tag`s bound to the driver via `TagConfig.FullName`, authored on the
`/uns` Tags tab. No alias machinery, no bespoke namespace kind.
> **Primary reuse target:** `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/` (+ `.AbCip.Contracts/`).
> Omron NJ/NX speak CIP, so the CIP path reuses ~70–80% of AbCip's libplctag body. Read this doc
> alongside `AbCipDriver.cs`, `AbCipDriverFactoryExtensions.cs`, `AbCipEquipmentTagParser.cs`, and
> `IModbusTransport.cs` (the hand-rolled-transport template for FINS).
---
## 1. Motivation + transport decision
Omron exposes **two entirely different Ethernet protocols** depending on controller generation, and
they cover **disjoint hardware** — neither subsumes the other. We support **both, CIP-first.**
| Transport | Port / framing | Controller families | Addressing model | Online-browsable? |
|---|---|---|---|---|
| **EtherNet/IP CIP** | TCP/UDP **44818**, CIP explicit messaging | **NJ / NX / NY (Sysmac)** | **Named variables** (symbolic tags), like Rockwell Logix | **No** (libplctag `@tags` returns `ErrorUnsupported` — §4) |
| **FINS** | FINS/TCP + FINS/UDP on **9600** | **CJ / CS / CP / CV**, NSJ | **Memory areas** (CIO/WR/HR/AR/DM/EM) + word/bit offset | **No** (flat memory, no symbol table) |
**Decision — ship CIP first, FINS second:**
1. CIP targets the modern/current Omron line (NJ/NX/NY) most likely in new installs.
2. The CIP path **reuses the already-hardened AbCip stack** (libplctag, `PollGroupEngine`,
`EquipmentTagRefResolver`, resilience seam, status mapping) — fastest path to a working
read/write driver. Omron NJ/NX "variables" are CIP symbolic tags reached exactly like
ControlLogix, which is why libplctag already speaks it via `plc=omron-njnx`.
3. FINS is simpler as a protocol but is a **net-new transport codec** with zero reuse, targeting
legacy hardware.
Full protocol background + sources: [`docs/research/drivers/omron.md`](../research/drivers/omron.md) §1.
---
## 2. Project layout
Two projects, mirroring the AbCip split (`Driver` + zero-dependency `Contracts` leaf). **Both
transports live in one driver assembly** behind a single `DriverType = "Omron"`, discriminated by a
`transport` field in config — the two transports share the `IDriver` shell, poll wiring, resilience
seam, and `EquipmentTagRefResolver`, and only diverge at the wire codec.
```
src/Drivers/
ZB.MOM.WW.OtOpcUa.Driver.Omron.Contracts/ # options, enums, equipment-tag parser (NO package refs)
OmronTransport.cs # enum { Cip, Fins }
OmronDriverOptions.cs # OmronDriverOptions + OmronDeviceOptions + OmronTagDefinition + OmronProbeOptions
OmronDataType.cs # enum (Bool/SInt/Int/DInt/LInt/USInt/UInt/UDInt/ULInt/Real/LReal/String/Dt)
OmronMemoryArea.cs # enum { Cio, Wr, Hr, Ar, Dm, Em } (FINS)
OmronEquipmentTagParser.cs # TagConfig-JSON → OmronTagDefinition (mirror AbCipEquipmentTagParser)
ZB.MOM.WW.OtOpcUa.Driver.Omron/ # driver body
OmronDriver.cs # IDriver + capability interfaces; dispatches to a transport
OmronDriverFactoryExtensions.cs # Register() + ParseOptions() + config DTOs
OmronDriverProbe.cs # IDriverProbe (Test Connect) — transport-aware
OmronStatusMapper.cs # wire status → OPC UA StatusCode (shared)
OmronHostAddress.cs # omron://gw/path and fins://ip:port/net.node.unit parsers
Cip/
IOmronCipRuntime.cs # tag-runtime seam (mirror IAbCipTagRuntime) — the CI test seam
LibplctagOmronRuntime.cs # libplctag.NET impl, plc=omron-njnx
OmronCipTransport.cs # per-device tag-handle cache, read/write/probe
Fins/
IFinsTransport.cs # framed request/response seam (mirror IModbusTransport)
FinsTcpTransport.cs # hand-rolled FINS/TCP + node-address handshake
FinsUdpTransport.cs # hand-rolled FINS/UDP (no handshake)
FinsFrame.cs # header build/parse, 0x0101 read / 0x0102 write, area-code table
FinsAreaCodeTable.cs # (OmronMemoryArea, bit?) → area code, CPU-family-parameterised
```
### Library + license
| Path | Library | License | Notes |
|---|---|---|---|
| **CIP** | **`libplctag.NET`** (`<PackageReference Include="libplctag"/>`), attribute string `plc=omron-njnx` | native core dual MPL-2.0 / LGPL-2.1; .NET wrapper **MPL-2.0** | **Zero new license review** — the identical dependency AbCip already ships. |
| **FINS** | **Hand-rolled** (no third-party dep) | our own code | FINS framing is ~200 lines, fully documented (W227 + Wireshark dissector); mirrors `ModbusTcpTransport`. |
**Avoid `HslCommunication`.** The maintained builds are **commercial-paid**; the "MIT" claim in
casual sources refers to a stale, abandoned community fork
([`HslCommunication-Community`](https://github.com/HslCommunication-Community/HslCommunication-Community)),
not current releases. Use it only as a *protocol reference* to cross-check the hand-rolled FINS
framing — never as a dependency.
`Driver.Omron.csproj` mirrors `Driver.AbCip.csproj`: `net10.0`, `TreatWarningsAsErrors`,
`ProjectReference` to `Driver.Omron.Contracts`, `Core.Abstractions`, `Core`; a single
`<PackageReference Include="libplctag"/>`; `InternalsVisibleTo` the test project.
`Driver.Omron.Contracts.csproj` has **NO package references** — `ProjectReference` only to the
zero-dependency `Core.Abstractions` leaf (for the shared `TagConfigJson` readers), exactly like
`AbCip.Contracts`.
### Shared CIP core — defer
The research flags ~70–80% AbCip reuse and asks whether to extract a shared `Cip.Core`. **Decision:
keep Omron self-contained in v1** (copy AbCip's tag-runtime/poll/status patterns into
`Driver.Omron/Cip/` and specialise). Rationale: (a) a premature shared-core extraction couples two
drivers before we know Omron's real wire deltas (string encoding, array quirks, Network-Publish); (b)
Omron's tag-list enumerator is *not* shareable with Rockwell anyway (§4). **Revisit a shared
`ZB.MOM.WW.OtOpcUa.Driver.Cip.Core` extraction only if/when a third CIP driver lands or the Omron
CIP body proves byte-identical to AbCip's after the live gate** — track as a follow-up, not a v1 gate.
The deltas from AbCip are small and localized: the `plc=omron-njnx` attribute string, Omron's data-type
code set (§3), string encoding, and the browse story.
---
## 3. Capability mapping
`OmronDriver` implements the same capability set as `AbCipDriver`:
```csharp
public sealed class OmronDriver : IDriver, IReadable, IWritable, ITagDiscovery, ISubscribable,
IHostConnectivityProbe, IPerCallHostResolver, IDisposable, IAsyncDisposable
```
(No `IAlarmSource` in v1 — Omron has no analog to AbCip's ALMD projection; alarms can be authored as
scripted alarms on the equipment page like any flat-address driver.)
`InitializeAsync` reads the parsed `OmronDriverOptions`, and **per device** selects a transport
implementation by `device.Transport`:
- `Cip` → `OmronCipTransport` (libplctag tag-handle cache).
- `Fins` → `FinsTcpTransport` / `FinsUdpTransport` (framed socket codec).
Both transports expose the same internal seam the driver body calls
(`ReadAsync` / `WriteAsync` / `ProbeReadAsync`), so `OmronDriver`'s `IReadable`/`IWritable`/poll code
is transport-agnostic and routes by the resolved tag's `Transport`.
| Capability | CIP (NJ/NX) | FINS (CJ/CS/CP) |
|---|---|---|
| **`InitializeAsync`** (connect) | libplctag Forward-Open per device; attribute `protocol=ab-eip&gateway=<ip>&path=<cip-path>&plc=omron-njnx&name=<tag>` — via the .NET wrapper this is typed `Tag` properties (`Protocol.ab_eip` + `PlcType.Omron`), and AbCip's `LibplctagTagRuntime.MapPlcType` **already maps** `"omron-njnx" → PlcType.Omron` on the shipped libplctag 1.5.2 | open TCP `:9600` + FINS/TCP node-address handshake (cache assigned client node); UDP variant skips the handshake |
| **`IReadable`** | reuse AbCip per-tag runtime: create handle → `ReadAsync` → `GetStatus` → decode | build `0x0101` Memory Area Read (area code + 3-byte address + word count), parse 2-byte end-code + payload; multi-word reads for 32/64-bit |
| **`IWritable`** | full r/w; BOOL-in-word + array via AbCip's `EncodeValue`/bit-RMW paths | `0x0102` Memory Area Write; bit writes use bit-access area codes (no RMW needed — bit codes address bits directly) |
| **`ISubscribable`** | **poll-based** via shared `PollGroupEngine` (no native push) | **poll-based** via `PollGroupEngine` (FINS has no subscription) |
| **`ITagDiscovery`** | **authored-only** — emit pre-declared tags; `SupportsOnlineDiscovery=false` (§4). Sysmac tag-export importer feeds these tags offline (§4) | **authored-only**; `SupportsOnlineDiscovery=false` |
| **`IHostConnectivityProbe`** | reuse AbCip probe-loop: cheap tag read at interval → `HostState` transitions | cheap DM-word read at interval → `HostState` transitions |
The poll/subscribe wiring copies AbCip verbatim:
```csharp
_poll = new PollGroupEngine(
reader: ReadAsync,
onChange: (h, r, snap) => OnDataChange?.Invoke(this, new DataChangeEventArgs(h, r, snap)),
onError: HandlePollError,
backoffCap: TimeSpan.FromSeconds(30)); // 05/STAB-8 fleet-wide cap
```
`ResolveHost` routes each reference to its device host via `EquipmentTagRefResolver` (per-host
resilience isolation — a broken device B must not trip device A's breaker), copied from
`AbCipDriver.ResolveHost`.
### 3.1 Data-type mapping
Omron CIP (NJ/NX) uses IEC 61131 types that align with the existing `OmronDataType` (modelled on
`AbCipDataType`); FINS is raw words the TagConfig must type explicitly.
| `OmronDataType` | CIP (NJ/NX) native | FINS (word interpretation) | `DriverDataType` / OPC UA |
|---|---|---|---|
| `Bool` | BOOL | bit-area read, or bit N of a word | Boolean |
| `SInt` / `USInt` | 1-byte | low byte of a word | SByte / Byte |
| `Int` / `UInt` | 2-byte | 1 word | Int16 / UInt16 |
| `DInt` / `UDInt` | 4-byte | 2 words (word-order matters) | Int32 / UInt32 |
| `LInt` / `ULInt` | 8-byte | 4 words | Int64 / UInt64 |
| `Real` | IEEE-754 32 | 2 words | Float |
| `LReal` | IEEE-754 64 | 4 words | Double |
| `String` | CIP string | word-packed ASCII, fixed length | String |
| `Dt` | Sysmac DATE_AND_TIME/TIME | banked words | DateTime (best-effort) |
Arrays: `ValueRank=1`, `ArrayDim` from `arrayLength`, mirroring AbCip's explicit `IsArray` flag (a
1-element array is still an array — `ElementCount` alone can't carry the signal).
**UDT/structure scoping (CIP, v1):** structure *members* are addressed leaf-wise via a dotted
`tagName` (e.g. `Conveyor.Speed` — a Sysmac structure-member path resolved symbolically by
libplctag), each authored as its own typed tag. **Whole-structure reads are out of scope in v1** —
`OmronDataType` deliberately has no `Structure` marker (unlike `AbCipDataType`), because AbCip's
whole-UDT path rides on the Logix Template Object decode (`AbCipTemplateCache` /
`CipTemplateObjectDecoder`), which is Rockwell-specific and not known to apply to Omron's structure
metadata. Whether dotted member access holds for all NJ/NX structure shapes is a live-gate item
(§9).
**Word/byte-order caveat (FINS only):** Omron stores multi-word values with a specific word order
that differs from naïve concatenation. The FINS codec exposes a **per-tag `wordSwap`** option (same
class of concern S7/Modbus already handle). CIP via libplctag handles endianness internally, so no
`wordSwap` on the CIP path.
**Network Publish gate (CIP):** On NJ/NX a variable is only reachable over the network if the
programmer set its **Network Publish** attribute (`Publish Only`) in **Sysmac Studio**. Tags without
it are invisible to any external client — including our reads. Document this loudly in
`docs/drivers/Omron.md`: a CIP read of an unpublished variable fails at the wire, not a config bug.
---
## 4. Browse story — reconciliation with the universal browser
This is the single most important finding, and it must be stated honestly against the universal
discovery-browser design (`docs/plans/2026-07-15-universal-discovery-browser-design.md`).
### 4.1 Both transports set `SupportsOnlineDiscovery = false`
The universal `DiscoveryDriverBrowser` (Tier 1) works by running a driver's `DiscoverAsync` against a
`CapturingAddressSpaceBuilder` and re-presenting the captured tree. It is gated on
`ITagDiscovery.SupportsOnlineDiscovery` — a default-`false` interface member that does **not exist
yet**; the Wave-0 universal browser adds it (today `ITagDiscovery` carries only the
`RediscoverPolicy` default member). That gate exactly encodes
"does `DiscoverAsync` enumerate from the *device*, or merely replay pre-declared tags?"
**For Omron, `DiscoverAsync` can only replay pre-declared/authored tags on BOTH transports:**
- **CIP (NJ/NX):** libplctag's `@tags` walker (CIP Symbol Object class `0x6B`) — the exact mechanism
AbCip uses for controller-tag enumeration — returns **`ErrorUnsupported` / `PLCTAG_ERR_NOT_FOUND`
on Omron NJ/NX**. Omron exposes its published-variable list through a *different* CIP object/service
than Rockwell, which libplctag has not implemented (upstream
[libplctag #466](https://github.com/libplctag/libplctag/issues/466),
[libplctag.NET #371](https://github.com/libplctag/libplctag.NET/issues/371)). So online CIP
enumeration is **not available** the way it is on Rockwell.
- **FINS:** flat memory banks, no symbol/metadata table on the wire — nothing to enumerate, ever.
Therefore Omron **does not** opt into `SupportsOnlineDiscovery`. Concretely, `OmronDriver` inherits
the default:
```csharp
// OmronDriver: no override of SupportsOnlineDiscovery → stays false (authored replay).
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once;
```
Consequence per the universal-browser design §5/§7: Omron sits in the **"No — not browsable"** row
alongside Modbus/S7/MELSEC/AbLegacy. The universal browser's `CanBrowse` returns false, the AdminUI
renders **manual entry** (no Browse button), and there is **no bespoke `IBrowseSession` in v1.** This
is the honest reconciliation: the universal browser does **not** help Omron, because Omron cannot
device-enumerate.
### 4.2 The v1 browse experience — offline Sysmac tag-export importer (CIP)
The lowest-risk, highest-value browse story for CIP is **offline import**, not a live browser:
- NJ/NX projects export the published global-variable table from **Sysmac Studio** (CSV / tag file).
- Build an **AdminUI importer** that parses this export into candidate `OmronTagDefinition`s (name +
data type + array shape) and authors them as equipment tags (or fills a driver-config `Tags` list).
- This needs **no online CIP enumeration** and dodges the entire libplctag-can't-list-NJ/NX problem.
Placement: a small importer under the AdminUI Tags/driver surface (parse → preview → author), NOT an
`IDriverBrowser`. Scoped as **Phase 3** (below). FINS has no equivalent — **manual entry only**
(memory-area dropdown + word/bit offset in the typed editor).
### 4.3 Future live CIP browse (explicitly out of scope for v1)
If libplctag adds Omron NJ/NX variable listing, *or* we hand-write the Omron CIP tag-list service
(Rockwell-different, research-grade), a bespoke `Driver.Omron.Browser` implementing
`IDriverBrowser` (`DriverType="Omron"`, CIP config only) could be added later — it would **override**
the universal fallback per the two-tier resolution. It would reuse the `BrowseNode`/`IBrowseSession`
plumbing but **not** the wire enumerator (Omron's tag-list service differs from Rockwell's). Not built
in v1. Even then, only Network-Published variables would be visible.
**The driver is fully usable without any browser** — pre-declared/imported tags always work, and
unmapped types fall back to the raw-JSON TagConfig editor. Browse is convenience, not correctness.
---
## 5. TagConfig + driver-config JSON
Follows the AbCip equipment-tag convention (`AbCipEquipmentTagParser`): a leading `{` marks a
TagConfig blob; camelCase property names; strict enum reads via `TagConfigJson.TryReadEnumStrict` (a
typo'd enum **rejects** the tag → `BadNodeIdUnknown`, never a silently-wrong Good). A `transport`
discriminator distinguishes the two shapes.
### 5.1 Driver config (bound to `DriverConfig` at `DriverHost.RegisterAsync`)
```jsonc
{
"timeoutMs": 2000,
"devices": [
{ "transport": "Cip", "hostAddress": "omron://10.100.0.40/1,0", "deviceName": "NJ501-Line1" },
{ "transport": "Fins", "hostAddress": "fins://10.100.0.41:9600/0.1.0", "cpuFamily": "CJ2", "deviceName": "CJ2M-Legacy" }
],
"probe": { "enabled": true, "intervalMs": 5000, "timeoutMs": 2000, "probeReference": "Conveyor.Heartbeat" },
"tags": [ /* optional pre-declared tags; equipment tags are authored per-tag instead */ ]
}
```
- `OmronDeviceOptions`: `Transport`, `HostAddress`, optional `DeviceName`, `CpuFamily` (FINS
area-code table selector). Per-device routing keyed on `HostAddress` (via `IPerCallHostResolver`),
exactly like `AbCipDeviceOptions`.
- `OmronHostAddress.TryParse` accepts **both** forms: `omron://gateway[:port]/cip-path` (CIP) and
`fins://ip[:port]/net.node.unit` (FINS). Malformed → device fails init (never silently connects to
nothing), matching AbCip.
### 5.2 Per-tag TagConfig — CIP (NJ/NX)
```json
{
"transport": "Cip",
"deviceHostAddress": "omron://10.100.0.40/1,0",
"tagName": "Conveyor.Speed",
"dataType": "Real",
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
`tagName` is the Sysmac global-variable name (near-identical to AbCip's `tagPath`; CIP path reuses
AbCip's `isArray`/`arrayLength`/`writable`/`dataType` semantics verbatim).
### 5.3 Per-tag TagConfig — FINS
```json
{
"transport": "Fins",
"deviceHostAddress": "fins://10.100.0.41:9600/0.1.0",
"memoryArea": "DM",
"address": 100,
"bit": null,
"dataType": "Int",
"wordSwap": false,
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
- `memoryArea` ∈ `{Cio, Wr, Hr, Ar, Dm, Em}` (+ optional `emBank` for extended memory). The codec
maps `(memoryArea, bit==null ? word : bit)` to the correct FINS area code via
`FinsAreaCodeTable` — a **CPU-family-parameterised** table (word vs bit codes are distinct; codes
vary slightly per CPU family, so the table is keyed on `device.CpuFamily`). Representative
word/bit codes: CIO `0xB0`/`0x30`, WR `0xB1`/`0x31`, HR `0xB2`/`0x32`, AR `0xB3`/`0x33`,
DM `0x82`/`0x02`, EM `0xA0+bank`/`0x20+bank`.
- `address` = word offset; `bit` = 0–15 for bit access (`null` ⇒ word access).
- `deviceHostAddress`'s `net.node.unit` triple is the FINS destination.
- `wordSwap` handles Omron multi-word ordering for 32/64-bit types.
### 5.4 `DriverAttributeInfo.FullName` per transport
`FullName` is the value the picker/importer commits as `TagConfig.FullName` and the driver resolves
via `EquipmentTagRefResolver`. For an **equipment tag**, `FullName` **is the raw TagConfig JSON blob**
(exactly as AbCip does — `AbCipEquipmentTagParser` sets `Name: reference`). The parser reads the
`transport` discriminator first, then the transport-specific fields. For a **pre-declared tag**,
`FullName` is the tag's `Name`. `OmronDataType.ToDriverDataType()` (extension in the driver, mirroring
`AbCipDataTypeExtensions`) maps to `DriverDataType`; `SecurityClass` = `Operate` when `writable` else
`ViewOnly`.
---
## 6. Typed editor + validator
Add **one typed editor per transport-aware model**, registered by the `"Omron"` DriverType:
- **Model:** `src/Server/.../AdminUI/Uns/TagEditors/OmronTagConfigModel.cs` — pure
`FromJson`/`ToJson`/`Validate`, preserving unknown keys via `TagConfigJson` (copy
`AbCipTagConfigModel`). It holds a `Transport` enum plus the union of CIP + FINS fields; the razor
shell shows CIP fields or FINS fields based on the selected `Transport`. `Validate()`:
CIP ⇒ `tagName` required; FINS ⇒ `memoryArea` set + `address >= 0`.
- **Editor:** `src/Server/.../AdminUI/Components/Shared/Uns/TagEditors/OmronTagConfigEditor.razor`
— thin razor shell over the model (copy `AbCipTagConfigEditor`), with a transport toggle that
swaps the field set (CIP: tagName + dataType + array; FINS: memoryArea + address + bit + dataType +
wordSwap + array).
- **Register in both maps** (one line each):
- `TagConfigEditorMap.cs`: `["Omron"] = typeof(Components.Shared.Uns.TagEditors.OmronTagConfigEditor),`
- `TagConfigValidator.cs`: `["Omron"] = j => OmronTagConfigModel.FromJson(j).Validate(),`
### The `JsonStringEnumConverter` enum-serialization trap (systemic — do NOT skip)
Per the known systemic AdminUI bug: **all driver pages serialize enums NUMERICALLY by default, but
the driver-side DTOs are string-typed** (`ParseEnum`/`TryReadEnumStrict` read enum *names*). An
AdminUI-authored config with any enum field (`transport`, `dataType`, `memoryArea`, `cpuFamily`)
serialized as an integer will **fault the driver** at parse. **Every Omron surface that serializes
config JSON must emit enum names, not numbers:** the `OmronDriverPage.razor` config serializer and
`OmronDriverProbe` attach `new JsonStringEnumConverter()` (mirror `AbCipDriverPage.razor`'s
serializer options and `AbCipDriverProbe._opts`, which both already set
`Converters = { new JsonStringEnumConverter() }`); `OmronTagConfigModel.ToJson` writes enum names
via the AdminUI `TagConfigJson.Set` helper, exactly as `AbCipTagConfigModel.ToJson` does (no
converter needed on that path). Enums serialize as their **name string**, matching
the driver's strict-name enum reads. Confirm this in the live-`/run` verify — unit tests do not catch
it (Blazor binding + numeric-enum serialization pass all unit tests).
Also add Omron to `DriverTypePicker.razor` (`_types` list) so the New-driver card grid shows it, and
add an `OmronDriverPage.razor` under `Components/Pages/Clusters/Drivers/` (copy `AbCipDriverPage.razor`)
plus the `DriverEditRouter` mapping and `DriverIdentitySection`/`EquipmentTagConfigInspector` "Omron"
entries where AbCip appears.
---
## 7. Factory + registration
`OmronDriverFactoryExtensions` mirrors `AbCipDriverFactoryExtensions`:
```csharp
public static class OmronDriverFactoryExtensions
{
public const string DriverTypeName = "Omron";
public static void Register(DriverFactoryRegistry registry)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, CreateInstance); // Tier defaults to A, matching AbCip
}
internal static OmronDriver CreateInstance(string driverInstanceId, string driverConfigJson)
=> new(ParseOptions(driverInstanceId, driverConfigJson), driverInstanceId);
internal static OmronDriverOptions ParseOptions(string driverInstanceId, string driverConfigJson) { /* … */ }
}
```
`ParseOptions` deserializes into config DTOs and maps to `OmronDriverOptions` — copy AbCip's
`ParseEnum<T>` (name-based, throws on unknown) and `PositiveTimeoutOrDefault` (clamps
`timeoutMs: 0` to the default so a misconfigured zero can't fault every op — the R2-01 sibling
hardening). `InitializeAsync` re-parses the config JSON on reinit so a changed config takes effect
(copy AbCip's re-parse-in-`InitializeAsync` pattern).
**Wiring (one line each):**
- `DriverFactoryBootstrap.Register(...)`: `Driver.Omron.OmronDriverFactoryExtensions.Register(registry);`
- `DriverFactoryBootstrap.AddOtOpcUaDriverProbes(...)`:
`services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, OmronProbe>());` (+ a
`using OmronProbe = Driver.Omron.OmronDriverProbe;` alias). Probes MUST be wired on admin nodes —
Test Connect is an admin-pinned singleton.
`OmronDriverProbe` (`IDriverProbe`, `DriverType="Omron"`) is transport-aware: for a CIP device do the
two-phase AbCip probe (bare TCP `:44818` preflight → libplctag `plc=omron-njnx` Forward-Open, treat
tag-not-found statuses as *reachable*); for a FINS device do a TCP `:9600` connect + a single
`0x0101` DM-word read (or the node-address handshake) to confirm the endpoint speaks FINS.
---
## 8. Resilience / timeout
Inherit the fleet-hardened patterns; the R2-01 frozen-peer lesson is non-negotiable for **both**
transports:
- **Per-op deadline on every read and write.** A frozen peer that accepts TCP but never answers must
not wedge a poll. CIP: libplctag `Tag.Timeout` from `options.Timeout` (`PositiveTimeoutOrDefault`
clamps a zero, as AbCip does — the libplctag setter throws on non-positive). FINS: an explicit
per-call `CancellationTokenSource.CancelAfter(options.Timeout)` around every socket
read/write, and a socket `ReceiveTimeout`, so an async FINS read cannot ignore the deadline
(the exact gap the S7 R2-01 fix closed — async reads that ignored the socket timeout).
- **Evict-on-failure parity with AbCip.** A non-zero wire status or transport exception **evicts** the
cached handle/socket so the next call re-creates it (copy `EvictRuntime`). FINS closes + reopens the
socket; CIP disposes + recreates the libplctag `Tag`.
- **Per-device connection backoff** (`ConnectionBackoff`, 05/STAB-8): inside an open backoff window a
data-path caller fails fast (no connect), reset on success — copy AbCip's `device.Backoff` usage.
- **Poll backoff cap** 30 s fleet-wide; `HandlePollError` degrades health to `Degraded` preserving
`LastSuccessfulRead`, never downgrading `Faulted`.
- **`WriteIdempotent` flagging.** Default `false` (a pulse/counter-advance/recipe-step write must not
auto-replay on a write timeout). Operators flag only tags whose semantics make replay safe
(level-set holding values, analog set-points) via the tag's `writeIdempotent` field → carried into
`DriverAttributeInfo.WriteIdempotent` (exactly as AbCip's pre-declared tags carry it). **Honest
status of the retry gate:** the invoker seam exists
(`IDriverCapabilityInvoker.ExecuteWriteAsync(hostName, isIdempotent, …)` — the non-idempotent arm
forces `RetryCount = 0`), but the host's sole write dispatch
(`DriverInstanceActor.HandleWriteAsync`) currently hardcodes `isIdempotent: false`; per-tag opt-in
is the documented-but-unshipped forward path (see the `WriteIdempotentAttribute` remarks). So Omron
carries the flag from day one, and flagged tags gain retry when that fleet-wide plumbing lands —
same position as AbCip today. FINS bit/word writes to a set-point are the typical idempotent case;
a FINS write to a command register is not.
Health surface: `GetHealth()` returns `DriverHealth`; transitions on read/write success/failure
exactly as AbCip.
---
## 9. Test fixtures
The honest gap: **there is no free NJ/NX CIP simulator.**
### CIP (NJ/NX)
- **Primary CI surface — hand-rolled `IOmronCipRuntime` fake** (the same seam pattern AbCip uses via
`IAbCipTagRuntime`/factory injection). Drives read/write/decode/array/BOOL-in-word/status-mapping
unit tests with **no hardware and no container**. This is where correctness of the driver *body* is
proven.
- **Live gate — env-gated `Category=LiveIntegration`** against real NJ/NX hardware, skipping cleanly
when the env var is absent (the pattern the HistorianGateway live suite uses). This is the **only**
way to validate the real `plc=omron-njnx` libplctag wire path, Network-Publish behavior, and Omron
string/array/UDT quirks. Env vars e.g. `OMRON_CIP_ENDPOINT` / `OMRON_CIP_TEST_TAG` /
`OMRON_CIP_WRITE_SANDBOX_TAG`. **Flag as an infra-gated known limitation** in the driver doc + this
design until an NJ/NX is on the bench. (Omron's own CX-Simulator is Windows-only + license-gated in
CX-One — not a CI fixture.)
### FINS
- **Primary — hand-rolled byte-level `IFinsTransport` fake** (mirror the Modbus `IModbusTransport`
fake): deterministic, no container, the most valuable surface. Covers `0x0101`/`0x0102` framing,
area-code table, word-swap, end-code parsing, and the FINS/TCP handshake.
- **Integration fixture — open-source FINS simulators** wrapped in a Dockerfile under
`tests/Drivers/.../Omron/Docker/` with the `project=lmxopcua` label, deployed to the Linux Docker
host `10.100.0.35` via `lmxopcua-fix sync omron` (repo compose file is the source of truth). Both
known OSS sims — [`hiroeorz/omron-fins-simulator`](https://github.com/hiroeorz/omron-fins-simulator)
and [`ahmadfarisfs/fins_simulator_omron`](https://github.com/ahmadfarisfs/fins_simulator_omron) —
are UDP-oriented and ship no image, so wrap them; FINS/TCP-handshake coverage stays on the
hand-rolled fake.
Contracts-level unit tests for `OmronEquipmentTagParser` + `OmronTagConfigModel` round-trips
(strict-enum reject, unknown-key preservation, both transport shapes) run everywhere, no fixture.
---
## 10. Phasing + effort
**Omron is scheduled LAST of the active driver set** — the no-free-CIP-sim gap means CIP correctness
can't be proven in CI, so it should land when live NJ/NX hardware is available for the gate.
| Phase | Scope | Effort | Gate |
|---|---|---|---|
| **P1 — Omron CIP r/w (NJ/NX)** | `Driver.Omron` + `.Contracts`, `DriverType="Omron"`, `OmronCipTransport` on libplctag `plc=omron-njnx`; Connect/Read/Write/Subscribe(poll)/Probe; authored + equipment tags; typed editor + validator; factory + registration | Low–Medium (70–80% AbCip reuse) | hand-rolled runtime fake (CI) + env-gated live gate (hardware) |
| **P2 — Omron FINS r/w (CJ/CS/CP)** | `IFinsTransport` codec (TCP+UDP, `0x0101`/`0x0102`, area-code table, word-swap) behind the same `IDriver` shell; memory-area TagConfig + editor fields (no browse) | Medium (net-new codec, no dep/license risk) | byte-level fake (CI) + OSS FINS sim container |
| **P3 — Sysmac tag-export importer (CIP)** | AdminUI offline importer: parse Sysmac Studio variable export → author `OmronTagDefinition`s. **Not** a live browser. | Medium | parser unit tests on real export samples |
### Top risks
1. **No free NJ/NX CIP simulator** (the headline). CIP wire correctness (string/array/UDT/
Network-Publish) is provable only on live hardware behind the env-gated `LiveIntegration` suite →
**infra-gated known limitation** until an NJ/NX is on the bench. The hand-rolled fake proves the
driver *body*, not the *wire*.
2. **CIP browse is not free reuse.** libplctag's `@tags` walker returns `ErrorUnsupported` on Omron
NJ/NX; `SupportsOnlineDiscovery=false`, the universal browser doesn't apply, and browse ships as
an offline Sysmac-export importer (P3), with online browse deferred to a later research phase.
3. **FINS per-family variance** in memory-area codes + multi-word ordering — contained by the
CPU-family-parameterised `FinsAreaCodeTable` + per-tag `wordSwap`.
4. **Enum-serialization trap** — every config JSON surface must attach `JsonStringEnumConverter`, or
AdminUI-authored Omron configs fault the driver. Live-`/run` verify catches it; unit tests don't.
---
## Sources
See [`docs/research/drivers/omron.md`](../research/drivers/omron.md) §Sources for the full list
(Omron W506/W227/W596 manuals, libplctag #466 / libplctag.NET #371, Wireshark FINS dissector,
HslCommunication license situation, OSS FINS simulators).
</content>
</invoke>
docs/plans/2026-07-15-sql-poll-driver-design.md
+685
View File
@@ -0,0 +1,685 @@
# SQL poll driver (`Sql`) — executable implementation design
> **Status:** design, build-ready. Author: driver design sweep, 2026-07-15.
> **Supersedes/derives from research:** [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
> **Grounding read:** `CLAUDE.md` (Equipment-kind driver + `TagConfig.FullName`);
> `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/` (+ `.Contracts`) as the poll-driver template;
> `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/` capability seams
> (`IDriver`, `ITagDiscovery`, `IReadable`, `ISubscribable`, `IHostConnectivityProbe`,
> `PollGroupEngine`, `EquipmentTagRefResolver`, `DataValueSnapshot`, `DriverAttributeInfo`);
> [`docs/plans/2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md)
> (bespoke `IBrowseSession` house style);
> [`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md)
> (universal browser — see §4.0 reconciliation).
The driver type string is **`Sql`** (short, consistent with `Modbus`/`S7`/`Galaxy`). This doc
uses `Sql` throughout; the research report's working name `SqlPoll` maps 1:1.
---
## 0. TL;DR — decisions locked for the build
| Question | Decision |
|---|---|
| **Provider ship-first** | **SQL Server** via `Microsoft.Data.SqlClient` (already at `6.1.1` in `Directory.Packages.props:53`) behind an ADO.NET `DbProviderFactory` + a `SqlProvider` enum + an `ISqlDialect` seam. **Zero new NuGet deps for P1.** Postgres (`Npgsql`) + ODBC P2; MySQL/Oracle later. |
| **Browse** | **Bespoke schema-walk `IBrowseSession`** in a `Driver.Sql.Browser` project (`INFORMATION_SCHEMA`, dialect-abstracted). The **universal Discover-backed browser does NOT fit** — see §4.0. `ITagDiscovery.SupportsOnlineDiscovery` stays **`false`**. |
| **Write in v1** | **NO — read-only v1.** `IWritable` deferred to P4 as an opt-in, triple-gated, parameterized-UPSERT mode (design in §3.4). |
| **Capabilities v1** | `IDriver`, `ITagDiscovery` (authored replay only), `IReadable`, `ISubscribable` (poll via `PollGroupEngine`), `IHostConnectivityProbe`. |
| **Top 2 risks** | (1) **SQL injection** — values are always bound `DbParameter`s; identifiers only from `INFORMATION_SCHEMA`-validated + dialect-quoted names, never string-concatenated tag input. (2) **Secrets** — connection strings resolved from env/secret store via `connectionStringRef`, never committed/logged (mirror `ServerHistorian__ApiKey`). |
| **Effort** | Small–Medium. P1 ≈ Modbus minus wire-codec plus the dialect seam + result-set slicing; most infra (`PollGroupEngine`, `EquipmentTagRefResolver<TDef>`, health state machine, factory shape) is reused. |
---
## 1. Motivation + scope
Many plants expose process/production data only as **SQL rows** — MES/ERP staging tables, LIMS,
custom app databases, historian summary tables. Today OtOpcUa has no way to surface those under the
unified Equipment address space; an operator has to stand up a bespoke bridge. The `Sql` driver
closes that gap: it polls arbitrary SQL tables/views on an interval and publishes selected
columns/rows as ordinary OPC UA variable nodes — the same authoring flow (equipment page → Tags tab
→ address picker) as Modbus or S7.
**Scope v1: read-only.** Writing back into MES/ERP staging is operationally risky (a stray
`UPDATE`/`INSERT` can corrupt a batch record, double-post a transaction, or fire a trigger with side
effects). Read-only removes an entire risk class from the first ship. An opt-in write mode is
designed in §3.4 for a later phase but is **not** built in v1.
**Out of scope (v1):** `IWritable`, `IAlarmSource`, `IHistoryProvider` (alarms/history aren't
expressible from a plain poll query). SQL-poll data can still be *historized* by the server via the
standard `TagConfig.isHistorized` flag — that's the server historian path, not a driver capability.
Full research + verdicts: [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
---
## 2. Project layout
Three new projects, mirroring the Modbus split (runtime / contracts / browser), plus test projects.
All `net10.0`, `Nullable`/`ImplicitUsings` enabled, `TreatWarningsAsErrors`, matching the repo
`.csproj` conventions.
| Path | Role |
|---|---|
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts/` | Options DTOs, `SqlProvider` enum, tag-model records, `SqlEquipmentTagParser`. **Zero transport deps** (refs only `Core.Abstractions`) so AdminUI + browser can reference it without dragging `Microsoft.Data.SqlClient`. |
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql/` | `SqlDriver` (runtime capabilities), `ISqlDialect` + `SqlServerDialect`, `SqlPollReader`, factory extensions. **Owns the `Microsoft.Data.SqlClient` package ref.** |
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser/` | `SqlDriverBrowser : IDriverBrowser`, `SqlBrowseSession : IBrowseSession` (schema walk). Refs `Commons` + `Sql.Contracts` + **`Driver.Sql`** (for `ISqlDialect`/`SqlServerDialect` — the dialect's catalog SQL *is* the browse engine, so it must be shared, not duplicated; `Microsoft.Data.SqlClient` comes transitively). This runtime-project ref is a deliberate deviation from `OpcUaClient.Browser` (which refs only its Contracts and owns the OPC UA client packages itself): the browser still opens its own transient connection. |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests/` | Unit suite (SQLite fixture, §9). |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/` | Env-gated integration suite (central SQL Server, §9). |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser.Tests/` | Browser unit tests (SQLite `PRAGMA` dialect). |
Add all six to `ZB.MOM.WW.OtOpcUa.slnx`.
### 2.1 `Driver.Sql.csproj` (sketch)
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<NoWarn>$(NoWarn);CS1591</NoWarn>
<RootNamespace>ZB.MOM.WW.OtOpcUa.Driver.Sql</RootNamespace>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core\ZB.MOM.WW.OtOpcUa.Core.csproj"/>
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts\ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts.csproj"/>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" /> <!-- version from Directory.Packages.props -->
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests"/>
</ItemGroup>
</Project>
```
The capability seams + `PollGroupEngine` + `EquipmentTagRefResolver` all live in
`Core.Abstractions`, but the `Core.csproj` reference is **also** needed (Modbus carries it too):
the factory extensions (§7) take `DriverFactoryRegistry`, which lives in
`Core/Hosting/DriverFactoryRegistry.cs` (`ZB.MOM.WW.OtOpcUa.Core.Hosting`). The
SQLite unit-test project pulls `Microsoft.Data.Sqlite` (already `10.0.7` in the graph) — no product
dependency on SQLite.
### 2.2 Provider strategy — `DbProviderFactory` behind `ISqlDialect`
Use ADO.NET's provider-agnostic base types (`System.Data.Common`: `DbConnection`, `DbCommand`,
`DbParameter`, `DbDataReader`) obtained through a `DbProviderFactory`. A small `SqlProvider` enum
(not open-ended invariant strings) maps to a dialect that owns the two things the base types **can't**
abstract: identifier quoting and the metadata-catalog SQL.
```csharp
// Driver.Sql.Contracts
public enum SqlProvider { SqlServer, Postgres, MySql, Odbc, Oracle }
// Driver.Sql (public — the .Browser project consumes it; only implementations touch provider packages)
public interface ISqlDialect
{
SqlProvider Provider { get; }
DbProviderFactory Factory { get; } // SqlClientFactory.Instance (P1)
string QuoteIdentifier(string ident); // [x] / "x" / `x` (rejects/escapes embedded quote chars)
string LivenessSql { get; } // "SELECT 1" (Oracle: "SELECT 1 FROM DUAL")
string ListSchemasSql { get; } // browse Root
string ListTablesSql { get; } // browse Expand(schema) — parameterized by @schema
string ListColumnsSql { get; } // browse Expand(table)/Attributes — parameterized by @schema,@table
DriverDataType MapColumnType(string sqlDataType); // INFORMATION_SCHEMA DATA_TYPE → DriverDataType
}
```
**Ship-first: `SqlServerDialect` only** (`Microsoft.Data.SqlClient`, `SqlClientFactory.Instance`,
`[bracket]` quoting, `INFORMATION_SCHEMA` catalog). P2 adds `PostgresDialect` (`Npgsql`) + a SQLite
dialect for the browser tests (`PRAGMA table_info`, `sqlite_schema`) + `OdbcDialect`. `Npgsql` /
`System.Data.Odbc` / `Oracle.ManagedDataAccess.Core` are each a **new** `PackageVersion` gated behind
their phase — the P1 SQL-Server slice adds none. In .NET there is no `machine.config` provider
registry: factories come from each provider's static `Instance` singleton
(`SqlClientFactory.Instance`, `NpgsqlFactory.Instance`, `OdbcFactory.Instance`).
---
## 3. Capability mapping
`SqlDriver : IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe`.
### 3.1 `InitializeAsync` — open + validate
1. Deserialize `SqlDriverConfigDto` from `driverConfigJson` (via `SqlDriverFactoryExtensions`,
mirroring `ModbusDriverFactoryExtensions.CreateInstance`), using a shared `JsonSerializerOptions`
with **`JsonStringEnumConverter`** (§6 — the enum-serialization trap) and
`UnmappedMemberHandling.Skip`.
2. **Resolve the connection string** from `connectionStringRef` against the process config/env
(§8.2) — never from the committed JSON.
3. Select `ISqlDialect` from `options.Provider` (P1: only `SqlServer` is constructible; others throw
a clean "provider not available in this build").
4. **Validate**: open one `DbConnection` from `dialect.Factory`, run `dialect.LivenessSql` under
`CommandTimeout` + a linked CTS (§8.3), dispose. Success → `DriverState.Healthy`; failure →
`DriverState.Faulted` with `LastError` (same state machine as `ModbusDriver.InitializeAsync`).
5. Build the `PollGroupEngine` (reader delegate = §3.2) with `minInterval = 100ms`, an `onError`
sink routing to the health surface, and a `backoffCap` (Modbus/S7 precedent, 05/STAB-8).
**Pooling:** rely on ADO.NET built-in pooling (keyed by exact connection-string text). Do **not**
hold a long-lived connection — **open-use-dispose per poll pass** (`await using var conn = …`) so the
pool manages lifetime and a dropped/restarted server recovers transparently. Expose `Max Pool Size`
etc. only through the connection string.
**`ReinitializeAsync`** = teardown + init (Modbus pattern); no live sockets to preserve.
**`GetMemoryFootprint`** ≈ the cached parsed-tag map + last-value dictionaries (small). No symbol
tables to flush; `FlushOptionalCachesAsync` is a no-op returning `Task.CompletedTask`.
### 3.2 `IReadable` / `ISubscribable` — one query per group, not per tag
The efficiency story is **batching by query-group**. A naive "one SELECT per tag" hammers the DB;
instead the driver coalesces tags that share a source query, issues **one round-trip per group per
poll**, and slices the result back to per-tag snapshots — structurally identical to
`ModbusDriver.ReadCoalescedAsync` (group → single PDU → slice back).
```csharp
public async Task<IReadOnlyList<DataValueSnapshot>> ReadAsync(
IReadOnlyList<string> fullReferences, CancellationToken ct)
{
// 1. Resolve each ref to a SqlTagDefinition via EquipmentTagRefResolver<SqlTagDefinition>.
// 2. Group by GroupKey (model-dependent, §3.6).
// 3. Per group: open connection, build ONE parameterized command, ExecuteReaderAsync(ct),
// index rows, map each tag's cell → DataValueSnapshot.
// 4. Reassemble snapshots in INPUT order (PollGroupEngine contract: N in → N out).
// Per-tag failure = a Bad-coded snapshot, NOT an exception (IReadable contract).
// Whole call throws only if the driver itself is unreachable (→ engine backoff).
}
```
`ISubscribable.SubscribeAsync` delegates to `PollGroupEngine.Subscribe(refs, publishingInterval)`
with `ReadAsync` as the reader delegate — the engine owns the loop, the 100 ms interval floor,
capped-exponential failure backoff, and change-diffing; `OnDataChange` is raised from the engine's
`onChange`. `UnsubscribeAsync` → `engine.Unsubscribe(handle)`.
**Poll interval** is driver-level (`defaultPollInterval`, default 5 s) with optional per-tag/per-group
override; the OPC UA subscription's requested publishing interval is clamped up to the group's floor
by `PollGroupEngine`.
**Per-call deadline (mandatory — R2-01 frozen-peer lesson).** Every `DbCommand` sets
`CommandTimeout` (= `commandTimeout` option, seconds) **and** runs `ExecuteReaderAsync(linkedCt)`
where the linked CTS is bounded by `operationTimeout` (wall-clock). A frozen DB (partition, lock
wait, slow query) surfaces `BadTimeout` per group and lets the poll loop back off — it must **not**
wedge the poll thread. Both are required: the linked token gives real client-side cancellation;
`CommandTimeout` is the server-side backstop.
### 3.3 `ITagDiscovery` — authored replay only (NOT schema enumeration)
`DiscoverAsync` materializes exactly the authored tags — the `tags` array in options **plus** the
equipment tags contributed at deploy time (whose reference is their raw `TagConfig` JSON). It does
**not** enumerate the DB schema: schema enumeration is a browse-time concern (§4). Each authored tag
becomes an `IAddressSpaceBuilder.Variable(browseName, displayName, DriverAttributeInfo)` with
`FullName` = the tag reference string, `DriverDataType` from the tag's declared/inferred type,
`SecurityClass = ViewOnly` (read-only v1). `RediscoverPolicy => Once`.
**`SupportsOnlineDiscovery => false`** (default member — see §4.0).
### 3.4 `IWritable` — verdict: NOT in v1
**v1 does not implement `IWritable`.** Rationale in §1. The P4 opt-in design: a tag may carry a
`write` block describing a **parameterized** statement (never string-concatenated):
```jsonc
"write": {
"mode": "Update", // Update | Insert | Upsert | StoredProc
"sql": "UPDATE dbo.Setpoints SET value=@value, ts=@ts WHERE tag_name=@key",
"keyParam": "@key", "keyValue": "Line1.Speed", "valueParam": "@value"
}
```
`WriteAsync` binds the OPC UA write value to `@value` (typed via the tag's declared type) and executes
the tag's parameterized command. **Triple-gated:** (a) the node's `WriteOperate` role (standard
`NodeWriteRouter`), (b) a driver-level `allowWrites` master switch (default `false`), (c)
`TagConfig.writable`. Not auto-retried unless the tag is `WriteIdempotent`
(`DriverAttributeInfo.WriteIdempotent`): an `UPDATE … WHERE key=` set-value is idempotent; an
`INSERT` / counter `UPDATE x=x+1` is not. Because a failed inbound device write reverts the node to
its prior value (write-outcome self-correction, master `1d797c1c`), SQL writes — unlike Galaxy's
fire-and-forget — *can* surface a genuine failure, which is the honest behaviour.
### 3.5 `IHostConnectivityProbe`
One logical host (the DB server host:port from the connection string). `GetHostStatuses()` returns a
single `HostConnectivityStatus(server, HostState.Running|Stopped|Faulted, lastChangedUtc)` derived
from the last poll/liveness outcome; `OnHostStatusChanged` fires on Running↔Stopped transitions. This
lets the Core scope Bad-quality fan-out to the driver's subtree on a DB outage (same pattern the
Galaxy driver uses for Platform/AppEngine). Optional but cheap — recommended for v1.
### 3.6 Tag→value mapping models
Three models; **(a) and (b) ship in P1**, (c) is P3.
**(a) Key-value / EAV table** *(primary).* A `(tagname, value, timestamp)` table; a tag names a
`table`, `keyColumn`+`keyValue`, `valueColumn`, optional `timestampColumn`.
**GroupKey = `(table, keyColumn, valueColumn, timestampColumn)`.** All such tags fold into one
`SELECT <keyColumn>, <valueColumn>, <timestampColumn> FROM <table> WHERE <keyColumn> IN (@k0,@k1,…)`
per poll; index rows by key, slice back. Values in the `IN` list are **bound parameters**, one per
key — never interpolated.
**(b) Wide row** *(cheap).* One "latest status" row, many columns → many tags. A tag names a `table`,
a `columnName`, and a `rowSelector` (`{whereColumn, whereValue}` or `{topByTimestamp: <col>}`).
**GroupKey = `(table, rowSelector)`.** All wide-row tags on the same row fold into one
`SELECT <col_a>, <col_b>, … FROM <table> WHERE <whereColumn>=@w` (or `ORDER BY <col> DESC` +
`TOP 1`/`LIMIT 1` via dialect); map each selected column to its tag.
**(c) Named parameterized SELECT** *(escape hatch, P3).* Named query defs at driver level
(`queries: { "q1": { sql, params } }`); a tag references a query by name + a projection (`rowKey`
selecting the row, `column` selecting the value). **GroupKey = the named query.** Covers joins/views/
computed projections. The SQL is authored by a `ConfigEditor`-privileged operator and stored
server-side — it is *config*, not runtime user input — but it still binds only parameters.
**Group execution shape (recommended):** for each distinct GroupKey the driver holds a compiled
`SqlQueryPlan { GroupKey, SqlText, ParameterBinder, RowIndexer, IReadOnlyList<SqlTagDefinition> members }`
built lazily and cached; a poll pass executes each plan's command once. Cap concurrent group
execution (per-driver semaphore, Modbus/S7 precedent) so a driver spanning many tables never opens an
unbounded number of pooled connections.
### 3.7 Type mapping (SQL → OPC UA) + timestamps
Map the column's provider metadata (`DbColumn.DataType` from `DbDataReader.GetColumnSchema()`, or the
`INFORMATION_SCHEMA` `DATA_TYPE`) to `DriverDataType` (`Core.Abstractions/DriverDataType.cs`), which
the address-space layer maps to OPC UA built-ins.
| SQL type (family) | CLR (`DbDataReader`) | `DriverDataType` |
|---|---|---|
| `bit` / `boolean` | `bool` | `Boolean` |
| `tinyint` / `smallint` | `byte`/`short` | `Int16` |
| `int` / `integer` | `int` | `Int32` |
| `bigint` | `long` | `Int64` |
| `real` / `float(24)` | `float` | `Float32` |
| `float` / `double` / `decimal` / `numeric` / `money` | `double`/`decimal`→`double` | `Float64` |
| `char` / `varchar` / `nvarchar` / `text` | `string` | `String` |
| `date` / `datetime` / `datetime2` / `timestamptz` | `DateTime`/`DateTimeOffset` | `DateTime` |
| `uniqueidentifier` / `uuid` | `Guid` | `String` |
`decimal`/`numeric` collapse to `Float64` (v1) with a documented precision-loss caveat. A tag may
**override** the inferred type explicitly (`"type": "Int32"`) — the operator sometimes knows better
than column metadata (e.g. a `varchar` column holding a number); the driver then coerces the read
cell to the declared type. `NULL` → `DataValueSnapshot { Value = null }` with `Good`/`Uncertain` per
a `nullIsBad` option (default: treat as `Uncertain`, not `Bad`).
**Source timestamp:** if a tag declares `timestampColumn`, that column's value becomes
`DataValueSnapshot.SourceTimestampUtc`; otherwise the poll-read wall-clock is used for both source and
server timestamps (the Modbus behaviour). A source timestamp is what makes SQL-poll data honest about
staleness — strongly recommend authoring one wherever the staging table has it.
---
## 4. Bespoke schema browser (`Driver.Sql.Browser`)
### 4.0 Reconciliation with the universal Discover-backed browser — **honest statement**
The [universal browser design](2026-07-15-universal-discovery-browser-design.md) proposes one generic
`DiscoveryDriverBrowser` that runs a driver's `ITagDiscovery.DiscoverAsync` against a capturing
builder and re-presents the streamed tree. **That mechanism does not fit the `Sql` driver, and here is
exactly why:**
> The universal browser only ever shows what `DiscoverAsync` streams. For `Sql`, `DiscoverAsync` is
> **authored/config-driven** (§3.3) — it replays the *already-authored* tags, it does **not** enumerate
> the database schema. A universal Discover-backed browser would therefore replay the tags the operator
> already typed in, which is useless for *discovering* new ones. Schema enumeration
> (databases/schemas → tables/views → columns) is a **browse-time** concern that lives in
> `INFORMATION_SCHEMA`, not in the runtime read path.
The universal design's own capability gate encodes this distinction (`SupportsOnlineDiscovery`:
"does `DiscoverAsync` enumerate from the backend, or merely replay pre-declared tags?"). `Sql` is in
the **replay** bucket alongside Modbus/S7. So:
- **`Sql` leaves `ITagDiscovery.SupportsOnlineDiscovery = false`** (the default member) — the universal
browser's `CanBrowse` returns false for `Sql`, and it never offers the universal fallback.
- **`Sql` ships a bespoke `IBrowseSession`** (this section) that walks the *live schema catalog*,
which is the thing an operator actually wants to browse. A registered bespoke `IDriverBrowser` for
`DriverType="Sql"` takes precedence over the universal fallback by construction
(`BrowserSessionService` resolves bespoke-first).
This is the same reason OpcUaClient/Galaxy keep bespoke browsers: the browse tree is a *different
shape* from the discovered tag set. For `Sql` it's even starker — the discovered set is a strict
subset (only what's authored), while the schema catalog is the full browsable universe.
### 4.1 The schema walk
Relational DBs are self-describing. The picker walks the catalog three levels deep, each level a
dialect-abstracted catalog query:
- **`RootAsync`** → schemas (Folder nodes). SQL Server: `SELECT DISTINCT TABLE_SCHEMA FROM INFORMATION_SCHEMA.TABLES` (optionally databases first for multi-DB servers). SQLite (test dialect): the single main DB.
- **`ExpandAsync(schemaNodeId)`** → tables + views in that schema (Folder nodes): `… TABLE_NAME, TABLE_TYPE FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA=@schema`.
- **`ExpandAsync(tableNodeId)`** → columns (Leaf nodes): `… COLUMN_NAME, DATA_TYPE, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=@schema AND TABLE_NAME=@table`.
- **`AttributesAsync(columnNodeId)`** → column data type + nullability side-panel (mirrors Galaxy's two-stage attribute pick): `AttributeInfo(Name: column, DriverDataType: dialect.MapColumnType(dataType).ToString(), IsArray: false, SecurityClass: "ViewOnly")`.
`@schema`/`@table` are **bound parameters** in the catalog queries — even here, no interpolation.
### 4.2 Portability caveat — the catalog query is dialect-specific
`INFORMATION_SCHEMA` is the SQL-92 standard catalog, supported by **SQL Server, PostgreSQL, and
MySQL/MariaDB** — so the queries above are portable across those three. But **two backends don't
implement it:**
- **Oracle** uses `ALL_TABLES` / `ALL_TAB_COLUMNS` (`USER_`/`DBA_` variants).
- **SQLite** uses `sqlite_schema`/`sqlite_master` + `PRAGMA table_info(<table>)`.
So the catalog SQL **belongs on `ISqlDialect`** (`ListSchemasSql`/`ListTablesSql`/`ListColumnsSql` +
`MapColumnType`), not hardcoded. P1 `SqlServerDialect` uses `INFORMATION_SCHEMA`; the SQLite test
dialect uses `PRAGMA`; Postgres/MySQL reuse `INFORMATION_SCHEMA`; Oracle (later) uses
`ALL_TAB_COLUMNS`. (Alternative for the *column* level only: `DbDataReader.GetColumnSchema()` on a
`SELECT * … WHERE 1=0` `SchemaOnly` command — but the table list still needs catalog SQL, so the
dialect method stays regardless.)
### 4.3 Picked-column → `TagConfig`
On column pick the browser emits `BrowseNode.NodeId` encoding `schema.table|column`; the picker body
composes the `TagConfig` blob (§5.2/5.3): DisplayName defaults to `table.column`, the inferred
`DriverDataType` from `AttributesAsync` prefills `type`, and the operator chooses the model (key-value
vs wide-row) + fills the key/row selector. `SecurityClass = ViewOnly` in read-only v1.
### 4.4 Project + registration (mirrors `OpcUaClient.Browser`)
| Path | Purpose |
|---|---|
| `Driver.Sql.Browser/SqlDriverBrowser.cs` | `IDriverBrowser`, `DriverType => "Sql"`; `OpenAsync(configJson, ct)` deserializes options, resolves the connection string from the form JSON (same trust boundary as `TestDriverConnect` + the OpcUaClient browser — transient, never cached), opens a `DbConnection`, returns a `SqlBrowseSession`. |
| `Driver.Sql.Browser/SqlBrowseSession.cs` | `IBrowseSession`; `RootAsync`/`ExpandAsync`/`AttributesAsync` over the dialect catalog queries; a `SemaphoreSlim _gate` (one ADO.NET connection is not concurrent). Per-call work is bounded by the AdminUI's existing 20 s per-call timeout. |
| AdminUI DI (`EndpointRouteBuilderExtensions.cs`, beside `:49-50`) | `services.AddSingleton<IDriverBrowser, SqlDriverBrowser>();` |
| `AdminUI/Uns/TagEditors/TagConfigEditorMap.cs` | `["Sql"] = typeof(Components.Shared.Uns.TagEditors.SqlTagConfigEditor)` (§6). |
| A new `SqlAddressPickerBody.razor` picker body | Browse tree (reuse `DriverBrowseTree.razor`) + attribute side-panel + model/selector fields; gated by the existing `DriverOperator` policy; manual entry retained. |
The browser reuses the AdminUI `BrowseSessionRegistry` + reaper + 2-min idle TTL + per-call timeout
unchanged. **The universal browser DI line is NOT added for `Sql`** — its `CanBrowse` is false anyway,
and the bespoke registration wins.
---
## 5. TagConfig + driver-config JSON
### 5.1 Driver config (the `DriverConfig` blob)
```jsonc
{
"provider": "SqlServer",
// Connection string is resolved at Initialize from an env/secret ref — NOT stored here.
"connectionStringRef": "MesStaging", // => env Sql__ConnectionStrings__MesStaging
"defaultPollInterval": "00:00:05",
"operationTimeout": "00:00:15", // per-query wall-clock deadline (→ BadTimeout)
"commandTimeout": "00:00:10", // server-side CommandTimeout backstop (seconds granularity)
"maxConcurrentGroups": 4, // cap on concurrent group queries (pool guard)
"nullIsBad": false, // NULL cell → Uncertain (default) vs Bad
"allowWrites": false, // master write kill-switch (v1: always false)
"probe": { "enabled": true, "interval": "00:00:10" },
// Optional pre-declared tag table (analogous to Modbus "Tags"); equipment tags may also
// contribute their address as raw TagConfig JSON at deploy time.
"tags": [ /* see 5.2 / 5.3 */ ],
// Optional named queries for the arbitrary-SELECT model (P3).
"queries": {
"activeBatch": {
"sql": "SELECT line, speed, temp, ts FROM dbo.LineStatus WHERE active=@active",
"params": { "@active": true }
}
}
}
```
### 5.2 Per-tag `TagConfig` — key-value model
```jsonc
{
"driver": "Sql",
"model": "KeyValue",
"table": "dbo.TagValues",
"keyColumn": "tag_name",
"keyValue": "Line1.Speed",
"valueColumn": "num_value",
"timestampColumn": "sample_ts",
"type": "Float64", // optional explicit override of inferred type
"writable": false
}
```
GroupKey for batching = `("dbo.TagValues", "tag_name", "num_value", "sample_ts")`; all such tags fold
into one `… WHERE tag_name IN (@k0,@k1,…)` per poll.
### 5.3 Per-tag `TagConfig` — wide-row model
```jsonc
{
"driver": "Sql",
"model": "WideRow",
"table": "dbo.LatestStatus",
"columnName": "oven_temp",
"rowSelector": { "whereColumn": "station_id", "whereValue": 7 },
// or "rowSelector": { "topByTimestamp": "sample_ts" } // newest row
"type": "Float64",
"writable": false
}
```
GroupKey = `("dbo.LatestStatus", rowSelector)`; every wide-row tag on the same row folds into one
`SELECT oven_temp, <other cols>, … FROM dbo.LatestStatus WHERE station_id=@id`.
### 5.4 Per-tag `TagConfig` — named-query model (P3)
```jsonc
{
"driver": "Sql",
"model": "Query",
"query": "activeBatch",
"rowKey": { "column": "line", "value": "Line1" }, // which row this tag reads
"column": "speed", // which projected column is the value
"timestampColumn": "ts"
}
```
### 5.5 Equipment-tag parser
`SqlEquipmentTagParser.TryParse(reference, out SqlTagDefinition def)` (in `.Contracts`) mirrors
`ModbusEquipmentTagParser`: recognizes the blob by a leading `{` + a `"driver":"Sql"` (or a `"model"`
discriminator), reads the model-specific fields, uses **`TagConfigJson.TryReadEnumStrict`** for the
`model`/`type` enums (a present-but-invalid enum rejects the tag → `BadNodeIdUnknown`, per R2-11), and
maps to a transient `SqlTagDefinition` whose `Name == reference` so the value the driver publishes
keys the forward router correctly. `EquipmentTagRefResolver<SqlTagDefinition>` bridges the two
authoring models (an authored `tags`-table entry by name **or** an equipment tag whose reference is
its raw `TagConfig` JSON, parsed once and cached) — same as Modbus.
---
## 6. Typed editor + validator + the enum-serialization trap
Add `["Sql"] = typeof(…SqlTagConfigEditor)` to `TagConfigEditorMap` and a matching entry to
`TagConfigValidator`. The editor is a thin Razor shell over a pure `SqlTagConfigModel`
(`FromJson`/`ToJson`/`Validate`, preserves unknown keys) — copy the Modbus template under
`Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`. The model exposes: `Model` (KeyValue/WideRow/
Query), `Table`, `KeyColumn`/`KeyValue`/`ValueColumn`/`TimestampColumn` (key-value), `ColumnName` +
`RowSelector` (wide-row), `Query`/`RowKey`/`Column` (named), and `Type` (`DriverDataType?` override).
Without a typed editor `Sql` would fall to the generic raw-JSON textarea (like Galaxy today) — the
typed editor is worth building because the model discriminator + selector shape is error-prone by hand.
**The `JsonStringEnumConverter` enum-serialization trap (MEMORY: systemic).** Every AdminUI driver
page that serializes an enum **numerically** while the factory DTO is **string-typed** produces a
config that faults the driver at parse time (proven e2e for S7 + Modbus). **All** enum fields here —
`provider` (`SqlProvider`), `model`, `type` (`DriverDataType`) — MUST round-trip as **strings** on
**both** sides:
- The AdminUI editor/model `ToJson` and the driver-config page serializer attach
`JsonStringEnumConverter` (mirror the OpcUaClient page).
- The factory (`SqlDriverFactoryExtensions`) and `SqlEquipmentTagParser` deserialize with the same
string converter (and `TryReadEnumStrict` for tag enums).
- The `TestDriverConnect` probe uses the same options.
Add a unit test asserting `provider`/`model`/`type` serialize as `"SqlServer"`/`"KeyValue"`/`"Float64"`
(quoted strings), not `0`/`1`/`8`.
---
## 7. Factory + registration
`SqlDriverFactoryExtensions` mirrors `ModbusDriverFactoryExtensions`:
```csharp
public static class SqlDriverFactoryExtensions
{
public const string DriverTypeName = "Sql";
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory));
}
public static SqlDriver CreateInstance(string driverInstanceId, string driverConfigJson, ILoggerFactory? loggerFactory)
{ /* deserialize DTO (string enum converter), validate connectionStringRef present, build options+dialect */ }
}
```
Wire-up (one line each):
- **Factory:** add `Driver.Sql.SqlDriverFactoryExtensions.Register(registry, loggerFactory);` to
`src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs` `Register(...)` (beside the
eight existing driver `Register` lines).
- **Tier:** declared on the factory registration itself — the optional `tier` parameter of
`DriverFactoryRegistry.Register(driverType, factory, tier)`
(`Core/Hosting/DriverFactoryRegistry.cs`), which the resilience pipeline reads via
`DriverFactoryRegistry.GetTier`. The default `DriverTier.A` (managed SDK, in-process) is correct
for `Sql` — no explicit argument needed, same as every existing driver. (Do **not** hunt for a
`DriverTypeMetadata`/`DriverTypeRegistry` registration site: that registry in `Core.Abstractions`
currently has no production population site — no existing driver registers metadata there.)
- **Probe:** if a lightweight `SqlProbe : IDriverProbe` is added for the AdminUI "Test Connect"
button, register it in `AddOtOpcUaDriverProbes` via `TryAddEnumerable` (a bare `SELECT 1` liveness
check — recommended, cheap, and admin-node registration matters per the split-role gotcha).
- **Browser:** the `AddSingleton<IDriverBrowser, SqlDriverBrowser>()` line (§4.4).
`ShouldStub()` in `DriverInstanceActor` is platform/role-driven and needs no `Sql`-specific change —
SQL Server client works cross-platform, so `Sql` runs on macOS dev too (unlike Galaxy).
---
## 8. Resilience / security / timeout
### 8.1 SQL injection (critical, risk #1)
- **Every value is a bound `DbParameter`.** The `IN (@k0,@k1,…)` key list, `WHERE col=@w` selectors,
named-query `params`, and (P4) write values are all parameters. **Zero** runtime tag input is ever
concatenated into SQL text.
- **Identifiers** (schema/table/column) that must appear as SQL text come **only** from
`INFORMATION_SCHEMA`-validated names or a browse-derived allow-list, and are emitted through
`dialect.QuoteIdentifier` (which escapes/rejects embedded quote characters). A table/column string
in a `TagConfig` is validated against the live catalog (or an allow-list) before it's ever quoted
into text; an unknown identifier rejects the tag (→ `BadNodeIdUnknown`) rather than executing.
- **Treat any code path that builds SQL by string-concatenating a tag field as a defect** — enforce
with a review checklist item + a unit test that feeds a malicious `keyValue`/`table`
(`'; DROP TABLE …`) and asserts it either binds harmlessly (value) or is rejected (identifier),
never executes.
- The named-query model stores operator-authored SQL as *config* (`ConfigEditor`-gated), and even
there only parameters bind runtime values.
### 8.2 Secrets (critical, risk #2)
Mirror `ServerHistorian:ApiKey` (env `ServerHistorian__ApiKey`, appsettings carries only a reminder
comment) and the env-overridable ConfigDb connection string:
- `DriverConfig` JSON stores a **`connectionStringRef`** (a logical name), **not** the connection
string. The driver resolves it at `InitializeAsync` by reading the environment variable directly
(`Sql__ConnectionStrings__<ref>` — the flat env-var spelling of the config key), so the secret
lives in the process environment / secret store — never in the config DB row or any committed
file. Direct env read (not `IConfiguration`) is deliberate: the factory registry materializes
drivers via a static `(id, json)` closure (Modbus pattern) with no `IConfiguration` in reach, so
this keeps the factory shape unchanged.
- If an inline `connectionString` is ever permitted (dev convenience only), the AdminUI **redacts** it
in display/logging and flags it dev-only.
- **Never log the resolved connection string.** Log only provider + server host + database name.
- Prefer integrated/managed auth where the estate supports it (`Integrated Security=true` / Azure AD /
Kerberos) so no password transits config at all.
- The `.Browser` path receives the secret in form JSON over the authenticated Blazor circuit (same
boundary as every other driver browser + `TestDriverConnect`); it builds one transient connection
and releases it — **no `_lastConfigJson` cached field anywhere.**
### 8.3 Timeout / frozen-peer (high, risk #3)
Per §3.2: `CommandTimeout` (server-side backstop) **and** `ExecuteReaderAsync(linkedCt)` bounded by
`operationTimeout` (client-side, real cancellation) on **every** command — the SQL analogue of the
R2-01 S7 blackhole finding. A frozen DB surfaces `BadTimeout` per group + degrades the driver + backs
off the poll loop; it must never wedge a poll thread. Asserted by the blackhole live-gate (§9).
### 8.4 Connection pooling / exhaustion (medium)
Open-use-dispose per poll (`await using` on both `DbConnection` and `DbDataReader` — never leak a
reader). Cap concurrent group execution via `maxConcurrentGroups` (a `SemaphoreSlim`, Modbus/S7
precedent) so a driver spanning many tables never opens an unbounded number of pooled connections.
Pool sizing is expressed only through the connection string (`Max Pool Size`).
### 8.5 Provider portability (medium)
`INFORMATION_SCHEMA` covers SQL Server/Postgres/MySQL but not Oracle/SQLite; parameter markers and
identifier quoting differ. **All of it lives behind `ISqlDialect`** — no dialect assumption leaks into
the driver core (the reader, grouping, and poll engine are provider-agnostic).
### 8.6 Type / precision (low-medium)
`decimal`→`Float64` precision loss; `NULL` semantics (`nullIsBad`); `datetime` vs `datetimeoffset`
timezone handling (normalize to UTC in `SourceTimestampUtc`). Documented + per-tag `type` override.
---
## 9. Test fixtures
- **SQLite unit fixture (primary).** `Microsoft.Data.Sqlite` (already `10.0.7` in-graph, no
container). Seed a `(tag_name, num_value, sample_ts)` key-value table and a wide-row `LatestStatus`
table in-process. Covers read/subscribe mapping, type mapping, group-batch slicing, the poll-engine
wiring, `nullIsBad`, and the browser over `PRAGMA table_info` (SQLite dialect). Fast, offline,
CI-safe on macOS. Requires a `SqliteDialect` (test-scoped is fine, or ship it as the P2 SQLite
browser dialect). Note: SQLite's dynamic typing means the dialect's `MapColumnType` maps declared
affinity, not runtime type — keep the seed columns explicitly typed.
- **Central SQL Server integration fixture.** `10.100.0.35,14330` (always-on per `CLAUDE.md`). A
seeded `SqlPollFixture` database with the same two sample tables validates the real
`Microsoft.Data.SqlClient` path, `INFORMATION_SCHEMA` browse, `CommandTimeout`/cancellation, and
pooling. **Env-gated** (`SQL_TEST_ENDPOINT` / a full `Sql__ConnectionStrings__Fixture`) so it skips
cleanly offline, like the other `*.IntegrationTests`. Deploy the seed via the docker rig; label the
stack `project=lmxopcua`.
- **Blackhole / timeout live-gate (most important integration test).** Mirror R2-01 S7: `docker pause`
a SQL Server container mid-poll and assert the read surfaces `BadTimeout`, the driver degrades, and
the poll loop backs off — **not** a wedged thread. This gate runs against a **dedicated disposable
`mssql` container** (its own compose stack under `tests/.../Docker/`, `project=lmxopcua` label) —
**never** pause the shared central SQL Server on `,14330`, which also hosts `ConfigDb` for the whole
rig. This is the single highest-value integration test given the frozen-peer risk class.
- **Injection regression test.** Feed a malicious `keyValue`/`table` and assert bind-harmless (value)
or reject (identifier), never execute (§8.1).
- **Postgres/ODBC fixtures** land with P2/P3 (a `postgres:16` container for Npgsql; an ODBC DSN
against the same SQL Server).
---
## 10. Phasing + effort
1. **P1 — SQL Server read-only, key-value + wide-row.** `.Contracts` (options DTO, `SqlProvider` enum,
tag models, `SqlEquipmentTagParser`); `SqlDriver` (`IDriver`/`ITagDiscovery`/`IReadable`/
`ISubscribable` via `PollGroupEngine` + `IHostConnectivityProbe`); `SqlServerDialect`; grouping +
result-set slicing; factory + type-metadata + probe registration; `SqlTagConfigEditor` + validator
(+ the enum-serialization test); SQLite unit fixture; env-gated SQL-Server integration fixture +
blackhole live-gate + injection test. **No new NuGet deps.**
2. **P2 — Bespoke schema browser.** `Driver.Sql.Browser` (`SqlDriverBrowser`/`SqlBrowseSession` over
`INFORMATION_SCHEMA` + a SQLite `PRAGMA` dialect for tests), `SqlAddressPickerBody.razor` + tree +
attribute side-panel, DI wire-up, column→`TagConfig` commit. Near-mechanical clone of
`OpcUaClient.Browser`. **Small.**
3. **P3 — Named-query model + PostgreSQL + ODBC.** `PostgresDialect`/`OdbcDialect`; add `Npgsql` +
`System.Data.Odbc` `PackageVersion`s; named-query grouping; Postgres container fixture.
4. **P4 (optional) — Write mode.** `IWritable` with per-tag parameterized UPSERT/StoredProc,
triple-gated (`WriteOperate` + `allowWrites` + `TagConfig.writable`), `WriteIdempotent` retry
policy (write-failure revert already provided by the runtime).
5. **P5 (demand-driven) — MySQL/MariaDB (`MySqlConnector`) + native Oracle (`ALL_TAB_COLUMNS`
dialect).**
**Effort:** P1 **Medium** (Modbus minus wire-codec, plus dialect seam + slicing; poll engine,
resolver, health machine, factory shape all reused). P2 **Small**. The provider/dialect long tail
(P3+) is where portability effort lives — keeping it behind `ISqlDialect` from day one is what makes
the P1 SQL-Server slice cheap and the rest additive.
---
## Sources / grounding
- Research report: [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
- In-repo seams: `Core.Abstractions/{IDriver,ITagDiscovery,IReadable,ISubscribable,IHostConnectivityProbe,PollGroupEngine,EquipmentTagRefResolver,DataValueSnapshot,DriverAttributeInfo,DriverDataType,DriverHealth,TagConfigJson}.cs`.
- Templates: `Driver.Modbus/{ModbusDriver,ModbusDriverFactoryExtensions}.cs`, `Driver.Modbus.Contracts/ModbusEquipmentTagParser.cs`, `Driver.OpcUaClient.Browser/*`.
- Wire-up sites: `Host/Drivers/DriverFactoryBootstrap.cs` (`Register` + `AddOtOpcUaDriverProbes`), `AdminUI/EndpointRouteBuilderExtensions.cs:49-50` (browser DI), `AdminUI/Uns/TagEditors/{TagConfigEditorMap,TagConfigValidator}.cs`.
- Packages: `Directory.Packages.props:53-54` (`Microsoft.Data.SqlClient 6.1.1`, `Microsoft.Data.Sqlite 10.0.7`).
- Browse house style: [`docs/plans/2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md); universal browser reconciliation: [`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
```
docs/plans/2026-07-15-universal-discovery-browser-design.md
+296
View File
@@ -0,0 +1,296 @@
# Universal Discover-backed browser (`DiscoveryDriverBrowser`) — design
**Status:** draft, 2026-07-15. Foundational Wave-0 item of the next-driver roadmap
(`docs/research/drivers/README.md`).
## 1. Motivation
Today the AdminUI address picker is backed by one **bespoke** `IDriverBrowser` per driver
type — only **OpcUaClient** and **Galaxy** exist, each a full hand-written
`IBrowseSession` (connect + lazy level-at-a-time browse). Every other driver's picker is a
static manual-entry stub.
But the existing-driver browse audit
(`docs/research/drivers/00-existing-driver-browse-audit.md`) found that AbCip, TwinCAT, and
FOCAS **already contain complete, unit-tested symbol enumeration** — it lives in their
runtime `ITagDiscovery.DiscoverAsync`, streamed into an `IAddressSpaceBuilder`, and is
simply never surfaced to the picker. The same is true of every future discovery-capable
driver (MTConnect `/probe`, BACnet Who-Is/object-list).
The insight: **`DiscoverAsync` already produces exactly the tree the picker wants.** Instead
of writing N bespoke browsers, write **one** generic browser that captures whatever a driver
streams into the builder and re-presents it. This is the highest-leverage single piece of
browse work on the roadmap.
## 2. The seam that makes it work
`ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder builder, ct)` does not return a tree — it
**streams structured nodes into a builder** (`Core.Abstractions/IAddressSpaceBuilder.cs`):
```csharp
IAddressSpaceBuilder Folder(string browseName, string displayName); // → child builder
IVariableHandle Variable(string browseName, string displayName, DriverAttributeInfo a);
void AddProperty(string browseName, DriverDataType t, object? v);
```
Two facts close the loop:
- `Folder(...)` returns a **child builder scoped inside the folder** → the driver's hierarchy
is expressed as the builder-nesting graph. Capture the nesting and you inherit the driver's
tree (folders, UDT structure, component nesting) — this is **not** a flat dump.
- `DriverAttributeInfo.FullName` **is the driver-side full reference** for read/write
addressing — i.e. exactly the value the picker commits as `TagConfig.FullName`
(`DriverAttributeInfo.cs:13`). No per-driver commit logic needed.
So a **capturing `IAddressSpaceBuilder`** run against any driver's `DiscoverAsync` yields a
`BrowseNode` tree whose leaves already carry the commit value and the datatype metadata.
## 3. Architecture — two-tier browse
```
BrowserSessionService.OpenAsync(driverType, configJson)
│
├─ bespoke IDriverBrowser registered for driverType? ── yes ──► use it (Tier 2)
│ (OpcUaClient, Galaxy, later: large ControlLogix/BACnet)
│
└─ no bespoke browser ───► DiscoveryDriverBrowser (Tier 1, universal fallback)
gated on online-discovery capability
```
- **Tier 1 — universal `DiscoveryDriverBrowser`** (this doc): the default. Covers any driver
whose `DiscoverAsync` enumerates from the *device*.
- **Tier 2 — bespoke `IBrowseSession`** (existing pattern, `2026-05-28-driver-browsers-design.md`):
reserved for large / structured / lazy address spaces where eager one-shot discovery is too
heavy. A bespoke browser **overrides** the universal one simply by being registered for that
`DriverType`.
### 3.1 Resolution change in `BrowserSessionService` (required)
`BrowserSessionService` currently indexes browsers with
`browsers.ToDictionary(b => b.DriverType)` — **one browser per type, and a duplicate
`DriverType` key throws at construction.** Therefore the universal browser **cannot** be one
of the injected `IEnumerable<IDriverBrowser>` (it has no single `DriverType`, and registering
it under every type would collide). Instead:
- Introduce `IUniversalDriverBrowser` with
`Task<IBrowseSession> OpenAsync(string driverType, string configJson, CancellationToken ct)`
and `bool CanBrowse(string driverType, string configJson)`.
- Inject it into `BrowserSessionService` as a **separate optional dependency** (not part of the
`IDriverBrowser` set). `OpenAsync` resolves bespoke-first, falls back to the universal browser
when no bespoke browser matches **and** `CanBrowse` is true; otherwise returns the existing
"no browser registered" result (→ picker shows manual entry).
- `RootAsync`/`ExpandAsync`/`AttributesAsync`/`CloseAsync` are unchanged — the universal
browser returns an ordinary `IBrowseSession` that lands in the same `BrowseSessionRegistry`
(TTL reaper, 20 s per-call timeout) as today.
## 4. Components
### 4.1 `CapturingAddressSpaceBuilder` (the core mechanism)
An in-memory `IAddressSpaceBuilder` that records the streamed tree instead of materializing
OPC UA nodes. New home: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/` (Commons already owns
`BrowseNode`/`IBrowseSession` and is referenced by AdminUI; `IAddressSpaceBuilder`/
`DriverAttributeInfo` live in `Core.Abstractions`, which Commons can reference).
| Builder call | Captured as | Serves |
|---|---|---|
| `Folder(bn, dn)` | node `{ id = parentPath + "/" + bn, display = dn, Folder, children[] }`; returns a child capturing builder scoped to it | `BrowseNode(id, dn, Folder, HasChildrenHint=true)` |
| `Variable(bn, dn, attr)` | leaf `{ id = attr.FullName, display = dn, Leaf, attr }`; returns a stub `IVariableHandle{ FullReference = attr.FullName }` | `BrowseNode(attr.FullName, dn, Leaf, false)` + `AttributeInfo` |
| `AddProperty(...)` | attached to the current node's property bag (optional; feeds the side-panel) | `AttributesAsync` |
| `IVariableHandle.MarkAsAlarmCondition(info)` | records `IsAlarm=true` on the leaf; returns a **no-op `IAlarmConditionSink`** (browse never delivers live transitions) | `AttributeInfo.IsAlarm=true` (picker pre-fills a default `alarm` block, per existing Galaxy behaviour) |
Leaf → `AttributeInfo` mapping (drives the side-panel + commit):
`AttributeInfo(Name: bn, DriverDataType: attr.DriverDataType.ToString(), IsArray: attr.IsArray,
SecurityClass: attr.SecurityClass.ToString(), IsAlarm: attr.IsAlarm)`. Commit `NodeId` =
`attr.FullName` → `TagConfig.FullName`; `attr.ArrayDim`/`IsHistorized`/`WriteIdempotent` are
available to pre-fill the typed tag editor.
The builder enforces a **node-count cap** (config, default e.g. 50 000): once exceeded it stops
recording and marks the tree `truncated=true` so the session can surface "results truncated —
narrow the driver config or use manual entry" rather than OOM on a 100 k-node backend.
### 4.2 `DiscoveryDriverBrowser : IUniversalDriverBrowser`
`OpenAsync(driverType, configJson, ct)`:
1. `driver = _driverFactory.TryCreate(driverType, ephemeralId, PatchForBrowse(driverType, configJson))`
— `null` ⇒ throw a clean "driver type not available in this host" (see §6 wiring).
2. `if (driver is not ITagDiscovery disc || !disc.SupportsOnlineDiscovery)` ⇒ throw
"driver has no online discovery — author tags manually" (should not happen — `CanBrowse`
already gated it, but defend).
3. `await driver.InitializeAsync(patchedConfigJson, openCts.Token)` — **real device connect**,
under an **open-timeout** (config, default 60 s; separate from the 20 s per-call browse
timeout, which only covers Root/Expand/Attributes serving from memory).
4. `var capture = new CapturingAddressSpaceBuilder(nodeCap); await disc.DiscoverAsync(capture, openCts.Token);`
— eager, whole-tree, one-shot. **Except when `disc.RediscoverPolicy == UntilStable`**
(FOCAS): its `FixedTreeCache` is filled by a background loop a couple of seconds *after*
connect, so an immediate single pass captures an empty device folder (the comment above
`FocasDriver.DiscoverAsync` states this explicitly). For `UntilStable` drivers, re-run the
capture on a short interval until the node set is non-empty and stable across two passes,
bounded by the open-timeout — the same contract `DriverInstanceActor` honours at deploy.
5. `await driver.ShutdownAsync(...)` in a `finally` — the tree is fully captured; hold nothing
live. (No lazy expand ⇒ no need to keep the connection.)
6. return `new CapturedTreeBrowseSession(capture.Root)`.
`CanBrowse(driverType, configJson)` = `TryCreate` (cheap, no connect) succeeds **and** the
instance is `ITagDiscovery { SupportsOnlineDiscovery: true }`. Used by the AdminUI to decide
whether to render the **Browse** button vs. manual entry, before any connect.
### 4.3 `CapturedTreeBrowseSession : IBrowseSession`
Serves entirely from the in-memory captured tree (the shape the MTConnect research report
proposed for its cached-`/probe` session — now subsumed by this generic component): `RootAsync` = top-level nodes; `ExpandAsync(nodeId)` = the captured children of that
node; `AttributesAsync(nodeId)` = the leaf's `AttributeInfo` (+ any captured properties).
`Token`/`LastUsedUtc` per the interface; `DisposeAsync` drops the tree. No I/O after open.
## 5. The online-discovery capability gate
The distinction the gate encodes: does `DiscoverAsync` enumerate from the **device** (browsable)
or merely replay **pre-declared** tags (flat-address drivers — Modbus/S7/MELSEC/AbLegacy/
Omron-FINS, where it would show only what you already authored)?
**Add a default-interface property** on `ITagDiscovery` (mirrors the existing
`RediscoverPolicy => UntilStable` default member on the same interface — zero churn for drivers
that don't opt in):
```csharp
/// <summary>True when DiscoverAsync enumerates the tag set from the live backend
/// (browsable), rather than replaying pre-declared/authored tags. Default false.</summary>
bool SupportsOnlineDiscovery => false;
```
- **Return true (device-enumerated):** OpcUaClient, BACnet, MTConnect, and — **config-gated**
— AbCip/TwinCAT/FOCAS (their device enumeration sits behind a config flag; the browse patch
guarantees it at open time, see §5.1).
- **Leave false (authored replay):** Modbus, Modbus-RTU, S7, MELSEC, AbLegacy,
Omron (CIP + FINS) — their pickers stay manual-entry (Omron gets an offline
tag-export importer later); and **MQTT/Sparkplug + SQL poll**, whose runtime
`DiscoverAsync` is also deliberately authored-only — they get **bespoke** browsers instead
(observation-window / `INFORMATION_SCHEMA` walk — see the program doc §4), because a
universal replay of authored tags would discover nothing new.
(Galaxy/OpcUaClient keep their bespoke browsers regardless — the flag only governs the
*universal fallback*.)
### 5.1 Per-driver browse-config patch (`PatchForBrowse`) — the one wrinkle
For AbCip/TwinCAT/FOCAS, device enumeration is **config-gated**: AbCip/TwinCAT's
`DiscoverAsync` only walks the controller when `EnableControllerBrowse = true`, and FOCAS only
emits its FixedTree when `FixedTree.Enabled = true` (default `false`,
`FocasFixedTreeOptions.Enabled`) — but the picker holds the *authoring* config (often
`false`). The universal browser therefore merges a tiny, **declarative** per-type JSON
patch before constructing the driver:
```csharp
static readonly IReadOnlyDictionary<string, string> BrowsePatches = new Dictionary<string,string>(OrdinalIgnoreCase) {
["AbCip"] = """{ "EnableControllerBrowse": true }""",
["TwinCAT"] = """{ "EnableControllerBrowse": true }""",
["FOCAS"] = """{ "FixedTree": { "Enabled": true } }""",
// OpcUaClient/BACnet/MTConnect: no patch — discovery is unconditional
};
```
This is a dictionary entry, not code — it keeps the "zero per-driver *browser*" property while
being honest that browse-mode activation is one declarative fact per config-gated driver.
`SupportsOnlineDiscovery` for AbCip/TwinCAT/FOCAS returns true because the patch guarantees
browse mode at open time. (FOCAS additionally needs the §4.2 `UntilStable` settle — the patch
turns the FixedTree on; the settle waits for its cache to fill.)
## 6. Wiring & deployment
- Register in AdminUI DI (`EndpointRouteBuilderExtensions.AddAdminUI`):
`services.AddSingleton<IUniversalDriverBrowser, DiscoveryDriverBrowser>();` — **one line**,
no per-driver registration (contrast the two bespoke lines at
`EndpointRouteBuilderExtensions.cs:49-50`).
- Depends on `IDriverFactory` (Core.Abstractions). In the **fused Host** this is the real
`DriverFactoryRegistryAdapter` (all `Driver.*.Register()` ran) → every discovery-capable type
is constructible. On a **standalone AdminUI** with no driver assemblies it's
`NullDriverFactory` → `TryCreate` returns null → `CanBrowse` false → universal browse simply
unavailable (graceful; picker falls back to manual entry). No new failure mode.
- **Network/credentials posture is unchanged from the existing browsers**: OpcUaClient/Galaxy
browsers already connect to live backends *from the AdminUI process* using credentials in
`configJson`. The universal browser just extends that to more driver types — same role gating
(picker is behind the existing `ConfigEditor`/browse authorization), same
credentials-in-JSON handling, same `BrowseSessionRegistry` TTL reaping.
## 7. Where universal applies vs. bespoke
| Driver | Universal covers it? | Bespoke later? |
|---|---|---|
| **AbCip** | Yes (patch → `EnableControllerBrowse`) | Only if controller symbol set too large for eager one-shot (UDT drill-down) |
| **TwinCAT** | Yes (patch) | Only for very large symbol sets / Flat-mode caveats |
| **FOCAS** | Yes (patch → `FixedTree.Enabled` + §4.2 UntilStable settle) | Unlikely — surface is small/curated |
| **MTConnect** | Yes (`/probe`) | Optional — `/probe` is already whole-model; eager is fine |
| **BACnet** | Yes for small sites | **Yes** for large multi-device sites (lazy per-device object-list) |
| **MQTT/Sparkplug** | **No** (`SupportsOnlineDiscovery=false` — runtime discovery is authored-only by design) | **Yes** — bespoke observation-window `MqttBrowseSession` (its design doc §4) |
| **SQL poll** | **No** (authored replay) | **Yes** — bespoke `INFORMATION_SCHEMA` schema browser (its design doc) |
| **OpcUaClient** | (bespoke already) | Keep bespoke (100k-node lazy browse) |
| **Galaxy** | (bespoke already) | Keep bespoke (two-stage attribute pick) |
| Modbus/RTU, S7, MELSEC, AbLegacy, Omron | **No** (`SupportsOnlineDiscovery=false`) | N/A — not browsable (Omron: offline importer P-later) |
## 8. Limits (why bespoke browsers still exist)
1. **Eager, not lazy.** `DiscoverAsync` runs the whole enumeration at open. For huge trees
(ControlLogix 10 k+ tags, OPC UA 100 k nodes) the node-cap truncates and the UX is worse than
a bespoke per-click `ExpandAsync`. Graduate those to Tier 2.
2. **Runs the real driver in-process** (construct + connect + one-shot discover + shutdown) —
heavier than the lightweight ad-hoc bespoke session, and requires the driver assemblies in the
host (fused Host: yes).
3. **No incrementally-fetched attribute panel** beyond what `DriverAttributeInfo` +
`AddProperty` carry (fine for the picker's needs; Galaxy's richer two-stage pick stays bespoke).
## 9. Testing
- **Unit — `CapturingAddressSpaceBuilder`:** drive it with a fake `ITagDiscovery` that streams a
known Folder/Variable/AddProperty/alarm graph; assert the `BrowseNode` tree, leaf `NodeId ==
FullName`, `AttributeInfo` mapping, node-cap truncation, and the no-op alarm sink.
- **Unit — `DiscoveryDriverBrowser`:** fake `IDriverFactory` returning a fake driver; assert
Initialize→Discover→Shutdown ordering, open-timeout, `PatchForBrowse` merge, `CanBrowse` gate
(true for `SupportsOnlineDiscovery`, false otherwise / `TryCreate`→null), and the
`UntilStable` settle (a fake `UntilStable` driver whose tree only fills on the second
discovery pass must yield the full tree; a never-stable one must stop at the open-timeout).
- **Unit — `BrowserSessionService` resolution:** bespoke-first, universal-fallback, and
manual-entry-when-neither; confirm no `ToDictionary` collision.
- **Integration (fixture-gated):** point the universal browser at the existing driver fixtures on
`10.100.0.35` — AbCip (ControlLogix sim) and the OPC UA reference server — and assert a real
captured tree; reuse the driver fixtures already in `tests/`.
- **Live-verify:** on docker-dev, open the `/uns` TagModal picker for an AbCip driver and confirm
the controller tag tree renders and a picked leaf commits `TagConfig.FullName` (the usual
live-`/run` discipline — Razor binding bugs pass unit tests).
## 10. Phasing & effort
- **P1 — universal browser end-to-end:** `CapturingAddressSpaceBuilder` +
`CapturedTreeBrowseSession` + `DiscoveryDriverBrowser` + `IUniversalDriverBrowser` +
`BrowserSessionService` fallback + `ITagDiscovery.SupportsOnlineDiscovery` default member +
the AbCip/TwinCAT/FOCAS patches + the `UntilStable` settle loop (§4.2) + DI line + AdminUI
`CanBrowse`→Browse-button gate. Set `SupportsOnlineDiscovery=true` on
OpcUaClient(n/a—bespoke), AbCip, TwinCAT, FOCAS. **~3–5 days.**
- **P2 — light up new drivers for free:** as MTConnect/BACnet land, each sets
`SupportsOnlineDiscovery=true` (+ patch if config-gated) and gets a working picker with no
browser code. (MQTT/Sparkplug and SQL poll ship bespoke browsers instead — program doc §4.)
- **P3 — graduate to bespoke** only where §8.1 bites (large BACnet sites, ControlLogix UDT
drill-down): add a bespoke `IBrowseSession` that overrides the universal fallback for that type.
Net: one ~3–5 day piece replaces the previously-planned per-driver browser builds for AbCip +
TwinCAT (and pre-covers FOCAS + every future discovery driver), with bespoke lazy browsers
reserved for the genuinely-large trees.
## 11. v3 forward note (Raw/UNS two-subtree — lands after this)
This browser ships **before** v3 (`docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md`)
against the current `/uns` TagModal picker. When v3's Raw tree lands, three things about the
*consumer* change while the session/tree layer here (`CapturingAddressSpaceBuilder`,
`CapturedTreeBrowseSession`, `IUniversalDriverBrowser`, `BrowserSessionService` resolution)
carries over unchanged:
1. **Commit contract:** §2's "commits `TagConfig.FullName`" describes the pre-v3 picker. In
v3, `TagConfig` no longer carries identity (the tag's identity is its RawPath); the picked
leaf's `attr.FullName` is written into the driver-typed `TagConfig` as an ordinary
**address field**, and the commit creates **raw `Tag` rows** (name from browse name,
datatype from `DriverAttributeInfo`) under a Device/TagGroup — optionally mirroring the
captured folder nesting as `TagGroup`s.
2. **Config input:** v3 moves endpoint/connection settings from `DriverConfig` into
`DeviceConfig`. `OpenAsync(driverType, configJson, ct)`'s `configJson` becomes the
**merged Driver + Device config** composed by the caller (the `/raw` browse modal);
`PatchForBrowse` merges on top as today.
3. **Browse-button gate:** unchanged mechanism (`CanBrowse` / bespoke registration), hosted
in the `/raw` tree's "Add tags ▸ Browse device…" action instead of the equipment TagModal.
docs/research/drivers/00-existing-driver-browse-audit.md
+294
View File
@@ -0,0 +1,294 @@
# Existing-driver browse audit — which of the 6 no-browser drivers should get an `IDriverBrowser`
> **Status:** research complete, 2026-07-15. Author: browse-audit sweep.
> **Scope:** the 6 shipped drivers with **no** `IDriverBrowser` today — AbCip, TwinCAT,
> AbLegacy, FOCAS, S7, Modbus. (OpcUaClient + Galaxy already have browsers — see
> `docs/plans/2026-05-28-driver-browsers-design.md`.)
> **Question answered:** which protocols support live online discovery / symbol enumeration,
> what the browse would return, and how a picked node maps to that driver's `TagConfig`.
## TL;DR
The single most important finding: **AbCip and TwinCAT already contain a complete, working
symbol-enumeration stack** — but it is buried in the *runtime* driver's `DiscoverAsync`
(gated by an `EnableControllerBrowse` option) and is **not** exposed as the interactive,
per-click `IDriverBrowser` the AdminUI address picker consumes. Building their browsers is
therefore mostly a *wrapping* job over code that already ships and is already unit-tested,
not a from-scratch protocol implementation. That makes them high-value / low-effort and the
clear top priority.
---
## 1. Verdict table
| Driver | Browseable? | Mechanism (protocol-native discovery) | Existing code to reuse | Recommended action |
|---|---|---|---|---|
| **AbCip** (ControlLogix EtherNet/IP CIP) | **Yes** | CIP **Symbol Object class 0x6B** enumerated via libplctag's `@tags` pseudo-tag; UDT members via **Template Object class 0x6C** | `IAbCipTagEnumerator` / `LibplctagTagEnumerator` + `CipSymbolObjectDecoder` + `CipTemplateObjectDecoder` + `AbCipTemplateCache` + `AbCipUdtMemberLayout` **already exist** | **Add browser — priority 1** |
| **TwinCAT** (Beckhoff ADS) | **Yes** | ADS **symbol upload** (`ADSIGRP_SYM_UPLOAD`) via `SymbolLoaderFactory` (flat mode) | `AdsTwinCATClient.BrowseSymbolsAsync` + `TwinCATSymbolExpander` **already exist** | **Add browser — priority 2** |
| **FOCAS** (Fanuc CNC) | **Borderline** (curated, not free-form) | Fixed FOCAS API surface: live-enumerated axes/spindles/identity + per-series valid Macro/Parameter/PMC ranges | `FocasDriver.DiscoverAsync` FixedTree + `FixedTreeCache` + `FocasCapabilityMatrix` **already exist** | **Add curated browser — priority 3 (defer)** |
| **AbLegacy** (Rockwell PCCC / PLC-5 / SLC) | **No** | PCCC data-table is numeric (`N7:0`), no symbolic catalog; libplctag exposes no file-directory list | none (discovery = static re-advertisement of configured tags) | **Skip** |
| **S7** (Siemens S7comm) | **No** | Symbols live in the TIA/STEP7 project, not the CPU; S7netplus exposes no symbol/block-list API | none | **Skip** |
| **Modbus** | **No** | Flat register space; no discovery in the Modbus protocol | none | **Skip** |
---
## 2. Protocol-browseability evidence (sources)
- **CIP Symbol Object 0x6B** — the CIP object used to *browse* controller symbol addresses;
Rockwell's `Get Instance Attribute List` (service 0x55) enumerates the tag table, and the
ControlLogix tag database is accessed via ANSI-symbolic Data-Table-Read against the returned
tag names. Some firmware/devices don't implement 0x6B, so a browse must degrade gracefully.
(AdvancedHMI *Generic EtherNet/IP and CIP Technical Information*; scadaprotocols.com *CIP
Object Model*; PLCS.net tag-read threads.)
- **ADS symbol upload** — Beckhoff's documented flow reads `ADSIGRP_SYM_UPLOADINFO2` then
`ADSIGRP_SYM_UPLOAD` (+ `ADSIGRP_SYM_DT_UPLOAD` for datatypes); the .NET `SymbolLoaderFactory`
wraps this and yields a `DynamicSymbolsCollection` you enumerate by `InstancePath`, descending
`SubSymbols` for struct/UDT/FB members. (Beckhoff InfoSys *Access Data via Symbol Loader* /
*SymbolLoaderFactory.Create*; fisothemes *TwinCAT ADS in .NET* guide.)
- **S7comm has no on-wire symbol table** — S7-300/400/1200/1500 CPUs store only compiled code;
symbol tables + comments live in the offline TIA/STEP7 project. S7-1200/1500 *optimized* blocks
are addressable only symbolically and only via the PLC's **OPC UA server** — classic S7comm
GET/PUT (what S7netplus speaks) cannot browse or even read them by name. (FlowFuse *Read S7
optimized datablocks*; Industrial Monitor Direct *S7-1500 optimized block access* + *Siemens S7
upload missing symbols*.)
- **PCCC is numeric** — PLC-5/SLC access is `Protected Typed Logical Read/Write` against numeric
data-table addresses (`N7:0`, length N). There are no symbolic names on the wire and no
standard "list the files" catalog service. (Rockwell 1785-6.8.5 *Connecting PLC-5/SLC*; Ipesoft
*EtherNet/IP and encapsulated PCCC*.)
---
## 3. Browse designs for the **Yes / Borderline** drivers
The two existing browsers are the template: a sibling `*.Browser` project holding an
`IDriverBrowser` factory + an `IBrowseSession`, registered one line in
`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs` (`AddAdminUI`). The
picker razor bodies already exist per-driver under
`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Pickers/` — today
`AbCipAddressPickerBody.razor`, `TwinCATAddressPickerBody.razor`, and `FOCASAddressPickerBody.razor`
are **static manual-entry stubs**; only `OpcUaClientAddressPickerBody` and
`GalaxyAddressPickerBody` embed the shared `DriverBrowseTree`. So "wiring the picker" is the
same delta that was applied to those two.
> **Architectural note that differs from OpcUaClient/Galaxy:** for those two, the enumeration
> logic lives *inside* the `.Browser` project. For AbCip and TwinCAT the enumeration already
> lives in the *runtime* `Driver.*` project. The design doc's decision #4 ("keep AdminUI from
> pulling runtime `Driver.*` SDK chains") was about not dragging heavy SDKs transitively — but
> the AbCip/TwinCAT **browsers must connect with those exact SDKs anyway** (`libplctag`,
> `Beckhoff.TwinCAT.Ads`), so that concern is moot here. Two clean options:
> **(a)** the `.Browser` project references the runtime `Driver.AbCip` / `Driver.TwinCAT`
> project directly and calls the existing enumerator (least work); or
> **(b)** extract the enumerator + decoders into a small shared core lib both the runtime driver
> and the browser consume (cleaner separation, more churn). Recommendation: **(a)** for the
> first cut — the enumerator interfaces (`IAbCipTagEnumerator`, `ITwinCATClient.BrowseSymbolsAsync`)
> are already the seam, so a later extraction to (b) is mechanical.
### 3.1 AbCip browser (`DriverType = "AbCip"`)
**Mechanism.** Open a libplctag connection from the form's `AbCipDriverOptions` JSON (the same
shape the runtime factory + `AbCipDriverProbe` consume). For each configured device
(`AbCipDeviceOptions.HostAddress`), read the `@tags` pseudo-tag and decode with the existing
`CipSymbolObjectDecoder` → a stream of `AbCipDiscoveredTag { Name, ProgramScope, DataType,
ReadOnly, IsSystemTag, ElementCount, IsArray, TemplateInstanceId }`. For a `Structure`-typed
tag, resolve UDT members through the existing `AbCipTemplateCache` /
`CipTemplateObjectDecoder` (class 0x6C) → `AbCipUdtShape` member list.
**Session semantics:**
- `RootAsync` → one `Folder` node per device, plus a two-way split of the symbol table into
`Controller/` (controller-scope tags, `ProgramScope == null`) and `Programs/<prog>/`
(program-scope tags), driven off the already-decoded `ProgramScope`. Apply
`AbCipSystemTagFilter` so `@`-prefixed infrastructure tags don't clutter the tree (parity
with the runtime discovery filter).
- `ExpandAsync(nodeId)` →
- device / scope folder → its atomic tags as `Leaf` and its UDT tags as `Folder`;
- a UDT `Folder` → its members (via the Template Object) as `Leaf`, recursing for nested
structs (member `nodeId = "{tagPath}.{member}"`).
- `AttributesAsync` → **empty** (AbCip has no Galaxy-style attribute side-panel; the leaf *is*
the address). Kind is the terminating signal, same as OpcUaClient.
**Node → `TagConfig` mapping.** The picker commits the leaf's `NodeId` (the Logix symbolic
path) into the AbCip TagConfig JSON authored by `AbCipTagConfigModel` / parsed by
`AbCipEquipmentTagParser.TryParse`:
```json
{ "tagPath": "Program:Main.MyTag", "dataType": "DInt",
"deviceHostAddress": "ab://10.0.0.5/1,0",
"isArray": false, "arrayLength": 1, "writable": true }
```
The browser already knows `dataType` (from `AbCipDiscoveredTag.DataType`), `isArray` +
`arrayLength` (from `IsArray` + `ElementCount`), and can pre-fill `writable = !ReadOnly` — so
the pick can populate the whole config, not just the path (a nicer UX than OpcUaClient's
path-only commit). `deviceHostAddress` is the device the tree node came from.
**Project layout** (mirrors `Driver.OpcUaClient.Browser`):
```
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Browser/
AbCipDriverBrowser.cs (IDriverBrowser; DriverType="AbCip"; opens libplctag conn)
AbCipBrowseSession.cs (IBrowseSession over IAbCipTagEnumerator + template cache)
ZB.MOM.WW.OtOpcUa.Driver.AbCip.Browser.csproj
→ refs Commons + Driver.AbCip (option a) [+ libplctag transitively]
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip.Browser.Tests/ (fake enumerator, decoder golden)
```
**DI line** to add in `EndpointRouteBuilderExtensions.AddAdminUI`:
```csharp
services.AddSingleton<IDriverBrowser, AbCipDriverBrowser>();
```
**Effort: ~2–3 days.** The protocol-hard part (0x6B decode, 0x6C template decode, system-tag
filter, UDT layout) is done and tested. Work is: JSON-open path (copy the probe's connect),
the two-level `RootAsync`/`ExpandAsync` projection, and the picker razor (copy the OpcUaClient
tree wiring). Live-gate against the `abcip`/`controllogix` docker fixture on `10.100.0.35:44818`.
### 3.2 TwinCAT browser (`DriverType = "TwinCAT"`)
**Mechanism.** Open an `AdsClient` from the form's `TwinCATDriverOptions` JSON (same shape the
runtime consumes), then call the **already-implemented**
`AdsTwinCATClient.BrowseSymbolsAsync` → `IAsyncEnumerable<TwinCATDiscoveredSymbol>`. That method
builds a `SymbolLoaderFactory.Create(_client, SymbolsLoadMode.Flat)` loader and runs the pure
`TwinCATSymbolExpander.ExpandLeaves` to descend struct/UDT/FB instances to atomic members
(deduped by `InstancePath`). Each yielded symbol already carries the mapped `TwinCATDataType`,
optional array length, and read-only flag.
**Session semantics:**
- `RootAsync` → one `Folder` per device; under it the top-level PLC scopes as folders
(`Constants`, `MAIN`, `GVL`, `Global_Version`, `TwinCAT_SystemInfoVarList`, …), synthesised
by splitting each symbol's dotted `InstancePath` on the first segment. Apply
`TwinCATSystemSymbolFilter` to hide the infrastructure symbol lists (parity with runtime
discovery).
- `ExpandAsync(nodeId)` → children at the next `InstancePath` depth: atomic symbols as `Leaf`,
structs/FB instances as `Folder`. Because `BrowseSymbolsAsync` yields the fully-expanded leaf
set, the session can build the whole path→children index once on open and serve expands from
memory (no per-click wire call — the symbol blob is downloaded once by the loader).
- `AttributesAsync` → **empty** (leaf is the address).
**Node → `TagConfig` mapping.** Commit the leaf's `InstancePath` as `symbolPath` in the TwinCAT
TagConfig JSON (`TwinCATTagDefinition`: `symbolPath`, `dataType`, `deviceHostAddress`,
`arrayLength`, `writable`). The browser knows `dataType` + `arrayLength` + `writable`
(`!ReadOnly`) from the discovered symbol, so it pre-fills the full config.
**Two live caveats (already flagged in `AdsTwinCATClient`):**
1. **AMS router.** The AdminUI host process needs a reachable AMS router to browse — on Windows
from TwinCAT XAR, elsewhere from the `Beckhoff.TwinCAT.Ads.TcpRouter` package hosted in-proc.
`Beckhoff.TwinCAT.Ads` **7.0.172** is cross-platform-capable (netstandard), so this runs on
the Linux/macOS AdminUI host *if* a router is wired — but that is a deployment prerequisite to
document, not a code change.
2. **Flat-mode `SubSymbols`.** The code comment warns that on some TC3 firmware, `Flat` mode may
not populate `SubSymbols` for struct/UDT/FB symbols, silently dropping members; the documented
fallback is `SymbolsLoadMode.VirtualTree`. This must be **live-gated on a real TC3 target**
before trusting struct expansion.
**Project layout:** `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Browser/`
(`TwinCATDriverBrowser` + `TwinCATBrowseSession`, refs Commons + Driver.TwinCAT).
**DI line:** `services.AddSingleton<IDriverBrowser, TwinCATDriverBrowser>();`
**Effort: ~3–4 days** + a live-gate session on a real TC3 PLC for the `SubSymbols`/router risk.
The enumeration + expansion + type-mapping are done and unit-tested; work is the JSON-open
path, the path→tree projection, the picker razor, and the live validation.
### 3.3 FOCAS browser (`DriverType = "FOCAS"`) — curated, borderline
FOCAS has **no free-form symbol tree** — it's a fixed API surface. But the driver already does a
**live protocol-native discovery**: `FocasDriver.DiscoverAsync` (FixedTree) live-reads
`cnc_sysinfo` / `cnc_rdaxisname` / `cnc_rdspdlname` at session init (cached in `FixedTreeCache`)
to enumerate the machine's actual axes, spindles, identity, timers, program info, and status.
Separately, `FocasCapabilityMatrix` encodes the *valid ranges* of Macro variables, CNC
Parameters, and PMC letter-areas per `FocasCncSeries` (static tables, not live-queried).
**A curated browse would return:**
- `RootAsync` → one `Folder` per device, then category folders: `Identity`, `Axes`, `Spindles`,
`Program`, `Timers`, `Status` (populated live from `FixedTreeCache`), plus `Macros`,
`Parameters`, `PMC` (populated from `FocasCapabilityMatrix` for the device's declared series).
- `ExpandAsync` → e.g. `Axes/` → the live axis names as folders → per-axis `Leaf`s
(`AbsolutePosition`, `MachinePosition`, `ServoLoad`, …); `Macros/` → a `Leaf` per valid macro
number in the series range (or a paged range picker for the large ranges); `PMC/R` → valid R
addresses.
- `AttributesAsync` → **empty** (leaf is the address).
**Node → `TagConfig` mapping.** Commit the canonical FOCAS address string (`X0.0`, `R100`,
`PARAM:1815/0`, `MACRO:500`) as `address` in the FOCAS TagConfig, with `dataType` pre-filled
from the category (positions → Float64, macros → Float64, PMC bit → Bit, etc.).
**Effort: ~3–5 days**, medium value (curated list, not the "discover unknown tags" win that
AbCip/TwinCAT give). Reuses `FixedTreeCache` + `FocasCapabilityMatrix`. **Defer** behind AbCip
and TwinCAT.
---
## 4. Justification for the **No** drivers
### AbLegacy (PCCC / PLC-5 / SLC) — No
PCCC addresses are opaque numeric data-table references (`N7:0`, `F8:0`, `B3:0/5`) with **no
symbolic names on the wire** and **no standard catalog service**. libplctag (the driver's client,
`libplctag` 1.5.2) exposes no PCCC data-table-file directory listing. The driver's `DiscoverAsync`
merely re-advertises the statically configured `_options.Tags` — zero live reads. The only
"browse" one could build is blind range-probing (try reading `N7:0`, `N9:0`, …, infer existence
from status codes) which is slow, unreliable, and enumerates addresses not meaningful names — no
better than the existing manual `N7:0`-style entry. **Skip.**
### S7 (S7comm) — No
The Siemens CPU holds only compiled code; the symbol table lives exclusively in the offline
TIA/STEP7 project. Classic S7comm (GET/PUT — what `S7netplus` 0.20.0 speaks) has no symbol-browse
service, and `S7netplus` exposes no block-list/SZL API. S7-1200/1500 *optimized* blocks are
reachable **only** symbolically via the CPU's own OPC UA server — which is precisely the
`OpcUaClient` driver's job, not S7comm's. (An operator who wants to browse an S7-1500 optimized
DB should point the **OpcUaClient** driver at the PLC's OPC UA server, where the existing
OpcUaClient browser already works.) The driver's own `S7DriverOptions.Tags` doc-comment states it
plainly: "S7 has a symbol-table protocol but S7.Net does not expose it, so the driver operates off
a static tag list." **Skip.**
### Modbus — No
Modbus is a flat, typeless register/coil space (`Coils`, `DiscreteInputs`, `InputRegisters`,
`HoldingRegisters`) with **no discovery mechanism in the protocol at all** — a device cannot tell
you which registers exist or what they mean. The transport is hand-rolled (`ModbusTcpTransport`,
no NuGet). The driver's own option doc-comment says it: "Modbus has no discovery protocol — the
driver returns exactly these." **Skip.** (The existing `ModbusAddressPickerBody` region+address
builder is the right and only tool here.)
---
## 5. Priority ordering, shareable code, effort
**Build order (value × effort):**
| Priority | Browser | Value | Effort | Why this order |
|---|---|---|---|---|
| **1** | **AbCip** | High (ControlLogix ubiquity; symbolic tag browse is the flagship UX) | **~2–3 days** | Full 0x6B + 0x6C decode stack already ships and is tested; lowest effort, highest payoff |
| **2** | **TwinCAT** | High (ADS symbol upload is complete + rich) | **~3–4 days** (+ TC3 live-gate) | `BrowseSymbolsAsync` already ships; small residual risk (Flat `SubSymbols`, AMS router) needs live validation |
| **3** | **FOCAS** | Medium (curated, not free-form) | **~3–5 days** | Reuses FixedTree + capability matrix, but lower discovery value; defer |
| — | AbLegacy / S7 / Modbus | — | — | No browser (documented above) |
**Shareable-code opportunities:**
- **AbCip → future Omron / generic-EtherNet/IP browser.** The CIP browse core
(`CipSymbolObjectDecoder` 0x6B + `CipTemplateObjectDecoder` 0x6C + `AbCipSystemTagFilter` +
`AbCipUdtMemberLayout`) is protocol-CIP, not Rockwell-specific. If a second CIP-family driver
ever lands (Omron NX/NJ also speak CIP tag access), extracting that core into a shared
`ZB.MOM.WW.OtOpcUa.Cip.Core` lib (design-doc option **b**) would let both browsers share the
decode path. Not worth doing pre-emptively for one consumer — flag it as the natural refactor
point when/if the second CIP driver appears.
- **AbCip + TwinCAT both already put their enumerator behind an interface**
(`IAbCipTagEnumerator`, `ITwinCATClient.BrowseSymbolsAsync`) — that interface *is* the reuse
seam between runtime `DiscoverAsync` and the new browser session. No new abstraction needed.
- Nothing shareable across the No drivers (they have no discovery to share).
**Net recommendation:** build **AbCip** then **TwinCAT** browsers now (both are wrap-existing-code
jobs, ~1 week combined); **defer FOCAS** (curated, medium value); **do not build** AbLegacy, S7,
or Modbus browsers — their protocols have no online discovery and the existing manual-entry
pickers are the correct surface.
---
## Appendix — key source paths
- Contracts: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/{IDriverBrowser,IBrowseSession,BrowseNode}.cs`
- Templates: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser/`, `…Driver.Galaxy.Browser/`
- DI: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs` (`AddAdminUI`, lines 44–50)
- Picker bodies: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Pickers/*AddressPickerBody.razor`
- **AbCip** enumeration: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/{IAbCipTagEnumerator,LibplctagTagEnumerator,CipSymbolObjectDecoder,CipTemplateObjectDecoder,AbCipTemplateCache,AbCipUdtMemberLayout,AbCipSystemTagFilter}.cs`; TagConfig: `…Driver.AbCip.Contracts/{AbCipDriverOptions,AbCipEquipmentTagParser}.cs`; browse gate `AbCipDriverOptions.EnableControllerBrowse`
- **TwinCAT** enumeration: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT/{AdsTwinCATClient (BrowseSymbolsAsync),TwinCATSymbolExpander,TwinCATSystemSymbolFilter}.cs`; TagConfig: `…Driver.TwinCAT.Contracts/{TwinCATDriverOptions,TwinCATEquipmentTagParser}.cs`; browse gate `TwinCATDriverOptions.EnableControllerBrowse`
- **FOCAS**: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/{FocasDriver (DiscoverAsync/FixedTree),FocasCapabilityMatrix,IFocasClient}.cs`; TagConfig: `…Driver.FOCAS.Contracts/FocasDriverOptions.cs`
- **AbLegacy**: `…Driver.AbLegacy.Contracts/AbLegacyDriverOptions.cs`; `…Driver.AbLegacy/AbLegacyDriver.cs` (`DiscoverAsync` = static re-advertise)
- **S7**: `…Driver.S7.Contracts/S7DriverOptions.cs` (see `Tags` doc-comment)
- **Modbus**: `…Driver.Modbus.Contracts/ModbusDriverOptions.cs` (see `Tags` doc-comment); `…Driver.Modbus/ModbusTcpTransport.cs`
- Package versions (`Directory.Packages.props`): `libplctag` 1.5.2, `Beckhoff.TwinCAT.Ads` 7.0.172, `S7netplus` 0.20.0
docs/research/drivers/README.md
+151
View File
@@ -0,0 +1,151 @@
# Next-driver research — index & consolidated roadmap
> **Overall program design:** [`docs/plans/2026-07-15-driver-expansion-program-design.md`](../../plans/2026-07-15-driver-expansion-program-design.md)
> is the authoritative umbrella — shared architecture, cross-cutting rules, the browse-tier
> decision, the dependency graph, and the wave build order. Start there; this file is the
> research/design index it points back to.
Research sweep run 2026-07-15 (8 parallel agents). Each per-driver report covers
protocol + .NET library options (with license analysis), capability mapping to the
Equipment-kind driver seams, `TagConfig` JSON shape, a **browseability verdict +
`IDriverBrowser`/`IBrowseSession` design where browsable**, test-fixture strategy,
and effort/phasing. This index synthesizes them.
Scope decided with the user (2026-07-15): 7 new drivers + a browse-support audit of
the existing driver fleet. See `[[project_next_driver_roadmap]]` memory.
## Reports
| Report | Driver |
|---|---|
| [`mtconnect-agent.md`](mtconnect-agent.md) | MTConnect (Agent-first) |
| [`mqtt-sparkplug.md`](mqtt-sparkplug.md) | MQTT / Sparkplug B |
| [`melsec-slmp.md`](melsec-slmp.md) | Mitsubishi MELSEC SLMP / MC Protocol |
| [`omron.md`](omron.md) | Omron (CIP + FINS) |
| [`bacnet-ip.md`](bacnet-ip.md) | BACnet/IP |
| [`sql-poll.md`](sql-poll.md) | SQL poll |
| [`modbus-rtu.md`](modbus-rtu.md) | Modbus RTU (extend existing Modbus) |
| [`00-existing-driver-browse-audit.md`](00-existing-driver-browse-audit.md) | Browse audit of the 6 shipped browser-less drivers |
## Design docs (executable — built from the reports 2026-07-15)
Each non-deferred driver now has a build-ready design under `docs/plans/`:
| Driver | Design doc | Browse integration |
|---|---|---|
| Universal browser (Wave 0) | [`2026-07-15-universal-discovery-browser-design.md`](../../plans/2026-07-15-universal-discovery-browser-design.md) | — (the mechanism itself) |
| MTConnect Agent | [`2026-07-15-mtconnect-driver-design.md`](../../plans/2026-07-15-mtconnect-driver-design.md) | **Universal** (`SupportsOnlineDiscovery`) |
| BACnet/IP | [`2026-07-15-bacnet-ip-driver-design.md`](../../plans/2026-07-15-bacnet-ip-driver-design.md) | **Universal** (small sites); bespoke-lazy P-later |
| MQTT/Sparkplug B | [`2026-07-15-mqtt-sparkplug-driver-design.md`](../../plans/2026-07-15-mqtt-sparkplug-driver-design.md) | **Bespoke** observation browser |
| SQL poll | [`2026-07-15-sql-poll-driver-design.md`](../../plans/2026-07-15-sql-poll-driver-design.md) | **Bespoke** schema browser |
| Omron | [`2026-07-15-omron-driver-design.md`](../../plans/2026-07-15-omron-driver-design.md) | None — offline Sysmac tag-export import |
| Modbus RTU | [`2026-07-15-modbus-rtu-driver-design.md`](../../plans/2026-07-15-modbus-rtu-driver-design.md) | None (flat registers) — extends existing Modbus |
Browse split confirmed by the designs: **universal** covers MTConnect + BACnet (from `DiscoverAsync`);
**bespoke** required for MQTT (passive observation window) and SQL (schema walk ≠ discovery);
**Omron/Modbus-RTU** don't browse. All designs consume the universal browser's P1 addition of
`ITagDiscovery.SupportsOnlineDiscovery` — build the universal browser first.
## New drivers — at a glance
| Driver | Library (license) | Read/Write | Browsable | Write in v1 | Effort |
|---|---|---|---|---|---|
| **MTConnect Agent** | TrakHound MTConnect.NET (MIT) | Read-only | **Yes** — `/probe` device model | No (telemetry std) | S–M (~1–1.5 wk) |
| **MQTT / Sparkplug B** | MQTTnet v5 (MIT) + SparkplugNet (MIT, net10 risk) or hand-rolled Tahu | Read + subscribe | **Yes** — birth certs / topic tree (passive, time-bounded) | No | M |
| **MELSEC SLMP** _(deferred)_ | Hand-roll 3E-binary framer (HslCommunication is a commercial-license blocker) | Read/Write | No — flat device memory | Yes | M (addressing) |
| **Omron** | libplctag.NET (MPL-2.0) for CIP + hand-rolled FINS | Read/Write | CIP conditional (net-new) / FINS no | Yes | M–L (no free sim) |
| **BACnet/IP** | System.IO.BACnet / ela-compil (MIT, net10 TFM) | Read + COV subscribe | **Yes** — Who-Is + object-list | No (write P3) | M–L |
| **SQL poll** | Microsoft.Data.SqlClient (already in-repo) | Read-only | **Yes** — INFORMATION_SCHEMA | No (write later) | S–M |
| **Modbus RTU** | Extend existing Modbus (new `IModbusTransport`) | Read/Write | No — flat registers | Yes | **S (lowest)** |
Notes:
- **All fit the existing all-.NET-10 Equipment-kind model** — no COM gateway.
- **MELSEC**: `HslCommunication` is a hard commercial-license blocker; the clean-license
alternatives (McpX/MIT, McProtocol/LGPL) are too immature → hand-roll, patterned on S7.
Repo already has `MelsecAddress`/`MelsecFamily` (from the Modbus-path work) to port.
- **Omron CIP** reuses ~70–80% of AbCip's libplctag body, **but** libplctag's `@tags`
walker returns `ErrorUnsupported` on Omron NJ/NX — CIP browse is net-new Omron work
gated on the Sysmac "Network Publish" attribute, not free reuse.
- **MQTT/Sparkplug** & **BACnet** discovery is **passive/time-bounded** (listen/broadcast
over a window), unlike a synchronous OPC UA browse — the browse session handles this
with an observation window (+ Sparkplug rebirth NCMD to force near-synchronous enum).
- Every new driver needs a typed tag editor (`TagConfigEditorMap`+`TagConfigValidator`)
and must apply `JsonStringEnumConverter` on enum config fields (the known
`[[project_driver_enum_serialization_bug]]` trap).
## Existing-driver browse audit — verdicts
| Driver | Browsable? | Mechanism | Action |
|---|---|---|---|
| **AbCip** (ControlLogix EtherNet/IP) | **Yes** | CIP Symbol Object 0x6B + Template 0x6C — decode stack already ships | **Add browser** (~2–3 d) |
| **TwinCAT** (Beckhoff ADS) | **Yes** | ADS symbol upload (`SymbolLoaderFactory`) — enumerator already ships | **Add browser** (~3–4 d) |
| **FOCAS** (Fanuc CNC) | Borderline | Curated, not free-form (axes/spindles + macro/param/PMC ranges) | Curated browser, **deferred** (~3–5 d) |
| **AbLegacy** (PCCC) | No | Numeric addresses (`N7:0`), no symbolic catalog | Skip (documented) |
| **S7** (S7comm) | No | Symbols live in the TIA project, not the CPU | Skip (use OpcUaClient for optimized DBs) |
| **Modbus** (TCP/RTU) | No | Flat register space, no discovery | Skip (documented) |
**Key finding:** AbCip & TwinCAT already contain complete, unit-tested symbol-enumeration
logic buried in the runtime driver's `DiscoverAsync` (gated by `EnableControllerBrowse` /
Flat-mode) and simply not surfaced as the interactive `IDriverBrowser` the AdminUI picker
consumes. Their browsers are a **wrapping job over existing code**, not new protocol work —
the razor picker bodies already exist as static manual-entry stubs.
**Shareable code:** the CIP browse core (0x6B/0x6C decoders + system-tag filter + UDT
layout) is protocol-CIP, not Rockwell-specific — extract a shared `Cip.Core` lib **only
if/when** Omron CIP browse lands (don't pre-abstract). Both AbCip & TwinCAT already expose
their enumerator behind an interface = the ready-made reuse seam.
## Unified browse-support picture
**Browsers to build (net + existing), by protocol richness:**
- Synchronous, model-rich: **AbCip**, **TwinCAT**, **MTConnect** (`/probe`), **SQL** (schema), **BACnet** (object-list), **Omron-CIP**.
- Passive/time-bounded: **MQTT/Sparkplug** (observation window / rebirth).
- Curated/fixed surface: **FOCAS** (deferred).
**Not browsable — documented, decision recorded so it isn't revisited:**
MELSEC SLMP · Modbus TCP · Modbus RTU · S7 · AbLegacy · Omron-FINS (all flat address spaces / no on-wire symbol table).
## Recommended build order (value × effort)
**Wave 0 — universal Discover-backed browser (the foundational browse piece):**
1. **Universal `DiscoveryDriverBrowser`** — ONE generic `IDriverBrowser` that runs any
discovery-capable driver's `ITagDiscovery.DiscoverAsync` against a capturing
`IAddressSpaceBuilder` and serves the captured tree back through the picker. Lights up
AbCip, TwinCAT, FOCAS, MTConnect, BACnet (small sites), authored-MQTT, and every future
discovery driver's picker with **zero per-driver browser code**. Gated on a new
online-discovery capability flag so flat-address drivers stay manual-entry. Design:
[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](../../plans/2026-07-15-universal-discovery-browser-design.md).
2. **AbCip / TwinCAT** — covered *immediately* by the universal browser (their enumeration
already lives in `DiscoverAsync`). Graduate each to a **bespoke lazy `IBrowseSession`**
only if a real controller's symbol set is too large for the universal browser's eager
one-shot discovery (UDT drill-down / per-click expand). Not up-front work anymore.
**Wave 1 — low-effort, high-leverage new drivers:**
3. **Modbus RTU** — lowest-effort new capability; extends the existing driver via one new `IModbusTransport` (lead with RTU-over-TCP for the containerized server).
4. **SQL poll** — zero new NuGet deps (SqlClient already in-repo), broad MES/ERP glue value, browsable.
**Wave 2 — strategic new drivers:**
5. **MTConnect Agent** — MIT library, clean browse, complements FOCAS with vendor-neutral machine-tool telemetry.
6. **MQTT / Sparkplug B** — UNS-strategic; plain-MQTT first to stand up the skeleton + validate MQTTnet/net10, then Sparkplug birth-driven discovery.
**Wave 3 — heavier / test-gated:**
7. **BACnet/IP** — opens facilities/HVAC/energy; COV + discovery + BBMD are the meaty parts (live-gate the broadcast reachability).
8. **Omron** — CIP-first (libplctag) then FINS; last of the active set because there's **no free NJ/NX CIP simulator**, so real-wire correctness is a live-hardware `LiveIntegration` gate.
**Deferred (research complete, not scheduled):**
- **MELSEC SLMP** — deferred (2026-07-15). The mature library is a commercial-license
blocker so it's a full hand-rolled 3E-binary framer with the hardest addressing model
on the roadmap (device-code × bit/word × hex/octal/decimal × word order); revisit when
a Mitsubishi-shop customer need makes it worth the build. Report stays valid as the
starting point.
FOCAS browser slots opportunistically whenever the CNC surface is being touched.
## Per-driver add pattern (reference)
`*.Contracts` project + driver impl (Connect/Discover/Read/Subscribe[/Write]) + factory +
`DriverType` string + typed tag editor (`TagConfigEditorMap`+`TagConfigValidator`) +
optional `*.Browser` project (register one line at
`EndpointRouteBuilderExtensions` AddAdminUI DI) + docker fixture (`project=lmxopcua`
label). Watch the enum-serialization trap and add a per-query/-op timeout deadline
(the R2-01 frozen-peer lesson) on every network call.
docs/research/drivers/bacnet-ip.md
+356
View File
@@ -0,0 +1,356 @@
# BACnet/IP driver — research & implementation design
**Status:** Research / design proposal (not yet implemented)
**Author:** research sweep, 2026-07-15
**Scope:** A new **Equipment-kind** driver `ZB.MOM.WW.OtOpcUa.Driver.Bacnet` exposing
BACnet/IP facility/HVAC/energy data under the unified namespace, following the same shape as
Modbus / S7 / AbCip / OpcUaClient. BACnet is the closest analog to the **OpcUaClient** driver:
it has native **device discovery** and native **change-of-value push subscriptions**, so it is a
first-class *browseable* + *subscribe-capable* driver, not a bare poller.
---
## 1. Protocol summary + .NET library options
### 1.1 BACnet/IP in one screen
BACnet (ASHRAE Standard 135 / ISO 16484-5) is the dominant building-automation protocol
(HVAC, lighting, energy metering, access control). **BACnet/IP** carries BACnet over UDP,
default port **47808 (0xBAC0)**, with a layered framing:
- **BVLC** (BACnet Virtual Link Control) — the UDP-facing header. Distinguishes unicast,
broadcast, and the **BBMD / foreign-device** distribution messages (see §1.3).
- **NPDU** (Network layer) — source/destination network numbers + hop count; enables routing
across BACnet networks (e.g. an IP↔MS/TP router).
- **APDU** (Application layer) — the actual service request/response
(ReadProperty, WriteProperty, SubscribeCOV, Who-Is/I-Am, …), plus **segmentation** for
payloads larger than a single frame (a large `object-list` or `ReadPropertyMultiple` result
is segmented across several APDUs with windowed ack).
**Object model.** A BACnet device is a `Device` object plus a collection of **objects**, each
with a well-known **object-type** and an **object-instance** number. Common object types:
| Object type | Typical meaning | Present-Value type |
|---|---|---|
| Analog Input / Output / Value (AI/AO/AV) | sensor / setpoint / calc'd analog | REAL (float) |
| Binary Input / Output / Value (BI/BO/BV) | contact / relay / flag | enumerated (0/1, active/inactive) |
| Multistate Input / Output / Value (MSI/MSO/MSV) | discrete state (Off/Low/High) | unsigned int |
| Device | identity, object-list, vendor, APDU limits | — |
| Schedule, Calendar | time-based control | — |
| Trend Log, Trend Log Multiple | historical samples | — |
| Loop, Notification Class, File, … | control / infra | — |
Every object exposes **properties** addressed by a **property-identifier** (an enum). The ones
that matter for a tag driver:
- **`present-value` (PROP_PRESENT_VALUE, 85)** — the live value. The read/subscribe target.
- **`object-name` (77)**, **`description` (28)** — labels for browse.
- **`units` (117)** — an *Engineering-Units* enum (degrees-C, kW, percent, …) on analog objects.
- **`status-flags` (111)** — in-alarm / fault / overridden / out-of-service → maps to OPC UA quality.
- **`object-list` (76)** — on the Device object: the array of every object the device holds
(the enumeration backbone for discovery + browse).
- **`priority-array` (87)** + **`relinquish-default` (104)** — for commandable outputs; writes
target a priority slot (see §2 Write).
**Services used by a client/gateway:**
- **Who-Is / I-Am** — discovery. Client broadcasts `Who-Is` (optionally with a device-instance
low/high range); each device answers `I-Am` carrying its device-instance, max-APDU,
segmentation support, and vendor-id. *This is asynchronous and time-bounded* (broadcast, then
collect I-Ams over a window) — the key wrinkle for the browse seam (§4).
- **ReadProperty (RP)** — read one property of one object.
- **ReadPropertyMultiple (RPM)** — read many properties across many objects in one request; the
efficient bulk-read path (subject to segmentation for big results).
- **WriteProperty (WP) / WritePropertyMultiple** — write, usually `present-value` at a priority.
- **SubscribeCOV (Change-Of-Value)** — subscribe to an object; the device **pushes**
`COVNotification` when `present-value` (or `status-flags`) changes beyond the COV increment.
Subscriptions carry a **lifetime** (seconds) and must be **renewed** before expiry, or made
*confirmed* vs *unconfirmed*. `SubscribeCOVProperty` targets a specific property.
### 1.3 Cross-subnet: BBMD & foreign-device registration
UDP broadcasts (Who-Is, I-Am, unconfirmed-COV) don't cross IP routers. BACnet solves this with
**BBMDs** (BACnet Broadcast Management Devices): one BBMD per subnet forwards broadcasts to peer
BBMDs via a **Broadcast Distribution Table (BDT)**. A client on a subnet *without* a BBMD (or on
a different network, e.g. the OtOpcUa server VM) registers as a **Foreign Device** with a remote
BBMD (`Register-Foreign-Device`, with a TTL that must be renewed). Once registered, the BBMD
relays broadcasts to it. **For OtOpcUa this is the common case** — the server usually is not on
the same L2 segment as the controllers, so foreign-device registration to a site BBMD is the
realistic connect path. This is a first-class config knob (§3), not an afterthought.
### 1.2 .NET library options
| Library | Package / repo | License | Frameworks | Maturity | Verdict |
|---|---|---|---|---|---|
| **System.IO.BACnet (ela-compil fork)** | NuGet **`BACnet`** — [github.com/ela-compil/BACnet](https://github.com/ela-compil/BACnet) | **MIT** | net48, netstandard2.0, **net8.0, net10.0** | ~247★ / 45 releases / v4.0; core library used by **YABE** (Yet Another BACnet Explorer) | **RECOMMENDED** |
| bacnet-stack (C) | [github.com/bacnet-stack/bacnet-stack](https://github.com/bacnet-stack/bacnet-stack) | GPL/BSD (dual) | C, P/Invoke only | Reference impl; used by the docker sims | No — C interop, not a .NET API |
| BACsharp / gecambridge / sxul forks | various | MIT | older TFMs | Stale forks of the same SVN origin | No — ela-compil is the live one |
**Chosen library: `System.IO.BACnet` (ela-compil fork), NuGet `BACnet`, MIT.**
Why: it is the *actively maintained* fork of the canonical Morten Kvistgaard / F. Chaxel
`System.IO.BACnet` port of Steve Karg's bacnet-stack, **it explicitly multi-targets `net10.0`**
(so no TFM friction with this repo), MIT license fits, and it is the engine inside YABE so the
protocol coverage is battle-tested. It explicitly supports everything we need:
**Who-Is/I-Am, ReadProperty, ReadPropertyMultiple, WriteProperty, SubscribeCOV /
SubscribeCOVProperty + COV notifications, and BACnet/IP with BBMD + foreign-device registration**
(confirmed on the repo's feature list and in `ela-compil/BACnet.Examples`:
`BasicReadWrite`, `ObjectBrowseSample`, `BasicAdviseCOV`, `DemoBBMD`).
Honest caveats:
- The API is **event/callback-driven and largely synchronous-with-callbacks**, not
`async`/`Task`-first. `BacnetClient.WhoIs()` fires `OnIam` events; `ReadPropertyRequest` has a
blocking overload + a begin/end async pattern. The driver must **wrap it behind an async seam**
(`TaskCompletionSource` + timeout), exactly the way the OpcUaClient driver already wraps the
OPC UA SDK's session calls. Budget for a thin adapter layer.
- BACnet values come back as `BacnetValue` (a tagged union / `IList<BacnetValue>` for arrays);
the driver owns the BACnet-type → `DriverDataType` mapping.
- Transport is a single shared UDP socket (`BacnetIpUdpProtocolTransport`) per client — all
devices on the network multiplex over it. One `BacnetClient` per driver instance (per network
interface / BBMD), **not** one per device — mirrors "single Session per OpcUaClient driver."
---
## 2. Capability mapping (IDriver + capability interfaces)
The driver implements `IDriver` plus the capability interfaces it can honor. Recommended set:
`IDriver, ITagDiscovery, IReadable, ISubscribable, IWritable, IHostConnectivityProbe`
(and later `IHistoryProvider` via Trend Log, `IAlarmSource` via COV/event enrollment — phase 2+).
| OtOpcUa capability | BACnet mechanism |
|---|---|
| **`IDriver.InitializeAsync`** | Bind the UDP socket on the configured interface/port; if `ForeignDevice` configured, `Register-Foreign-Device` to the BBMD (+ start TTL-renew timer). Optionally fire an initial `Who-Is` to warm the device table. Health = Healthy once the socket is bound (BACnet has no "connection"; liveness is per-device via COV/read). |
| **`ITagDiscovery.DiscoverAsync`** | `Who-Is` (bounded collect window) → for each `I-Am`, read Device `object-list` (RPM, segmented) → for each object read `object-name` + `units` + `object-type` → register a variable node per (device, object[, property]). `RediscoverPolicy = Once` (like OpcUaClient); redeploy re-runs it. Note: for large sites, discovery is authored via the **browse picker** (§4) rather than a blind full-network discover — same as OpcUaClient where tags are usually hand-picked. |
| **`IReadable.ReadAsync`** | Group the batch's `FullReference`s by device; issue **ReadPropertyMultiple** per device for `present-value` (+ `status-flags` for quality). Fall back to per-object ReadProperty for devices that report no RPM support in their `I-Am`. Map BACnet result → `DataValueSnapshot` (value + StatusCode from status-flags + timestamp=now). |
| **`ISubscribable.SubscribeAsync`** | **SubscribeCOV** per object (the real win — native push). Register an `OnCOVNotification` handler that fans changes to `OnDataChange`. Maintain a **lifetime-renew timer** (re-subscribe before expiry). Devices that reject COV (return error) **degrade to a poll loop** at `publishingInterval` transparently — so the subscribe surface always works. |
| **`IWritable.WriteAsync`** | **WriteProperty** `present-value` at a configurable **priority** (1–16; default 8 "Manual Operator" or 16 "lowest"), or `null` to relinquish. Non-idempotent → surface `BadTimeout` on ambiguous outcome (mirror OpcUaClient's write-result honesty). **Verdict: ship read-only in v1**, add write in phase 2 (priority-array semantics + commandable-vs-non-commandable validation are a correctness minefield; see §6). |
| **`IHostConnectivityProbe`** | Periodic `Who-Is` to a specific device-instance (or a cheap RP of the Device object's `system-status`); Running↔Stopped transitions raise `OnHostStatusChanged`. For foreign-device mode, probe = "is the BBMD registration alive." |
| **`GetHealth` / `GetMemoryFootprint` / `FlushOptionalCachesAsync`** | Standard. Footprint ≈ discovered-object count × constant (mirror OpcUaClient). Flush drops the device/object browse cache. |
### 2.1 Data-type mapping (BACnet → DriverDataType)
`DriverDataType` members available: `Boolean, Int16/32/64, UInt16/32/64, Float32, Float64,
String, DateTime, Reference`.
| BACnet present-value / property type | `DriverDataType` | Notes |
|---|---|---|
| REAL (analog present-value) | `Float32` | AI/AO/AV |
| DOUBLE | `Float64` | |
| BOOLEAN | `Boolean` | |
| Enumerated: binary Active/Inactive (BI/BO/BV) | `Boolean` | 0=inactive,1=active |
| Enumerated: multistate (MSI/MSO/MSV) | `UInt16` (or `UInt32`) | present-value is 1-based state index; expose `state-text` array as metadata later |
| Unsigned Integer | `UInt32` | |
| Signed Integer | `Int32` | |
| CharacterString (`object-name`, `description`) | `String` | |
| BACnetDateTime / Date / Time | `DateTime` | Schedules, Trend Log timestamps |
| ObjectIdentifier / property references | `Reference` / `String` | rarely a tag target |
| Array property (element index N) | element type + `IsArray` | ValueRank=1 with ArrayDimensions |
**Engineering Units** (the BACnet `units` enum: `degrees-celsius`, `kilowatt-hours`, `percent`,
…) map to the OPC UA node's **EngineeringUnits** (EUInformation) — surface it as node metadata
(the address-space builder's variable metadata), captured at discovery/browse time so the UNS
node carries proper units. **`status-flags`** (in-alarm/fault/overridden/out-of-service) →
OPC UA StatusCode: `out-of-service` or `fault` ⇒ Bad/Uncertain quality; else Good.
---
## 3. TagConfig JSON shape
Two layers, matching every other Equipment-kind driver: a **driver-level config** (the
`DriverConfig` blob bound in `InitializeAsync`, analogous to `ModbusDriverOptions`) and a
**per-tag `TagConfig`** blob (authored on the `/uns` TagModal; the driver's equipment-tag parser
turns it into an addressing record — mirror `ModbusEquipmentTagParser`).
### 3.1 Driver-level config (`BacnetDriverOptions`)
```jsonc
{
"localEndpoint": "0.0.0.0", // interface to bind; "0.0.0.0" = all
"port": 47808, // 0xBAC0 default
"localDeviceInstance": 4194302, // our own Device instance id (must be unique on the net)
"foreignDevice": { // null when on-segment / a BBMD is local
"bbmdAddress": "10.20.0.1",
"bbmdPort": 47808,
"ttlSeconds": 900 // registration TTL; driver auto-renews at ~TTL/2
},
"apduTimeoutMs": 3000,
"apduRetries": 3,
"maxSegmentsAccepted": 16, // segmentation window we advertise
"discovery": {
"whoIsWindowMs": 4000, // collect I-Am for this long (the time-bounded wrinkle)
"deviceInstanceLow": 0, // optional Who-Is range filter
"deviceInstanceHigh": 4194303
},
"cov": {
"preferConfirmed": true, // confirmed vs unconfirmed COV
"lifetimeSeconds": 300, // per-subscription lifetime; renewed before expiry
"fallbackPollMs": 1000 // poll interval for devices that reject COV
},
"probe": { "enabled": true, "deviceInstance": 100, "intervalMs": 5000 }
}
```
### 3.2 Per-tag `TagConfig` (authored on the TagModal / emitted by the browse picker)
A BACnet tag address is fully described by **device-instance + object-type + object-instance +
property-id (+ optional array index)**. `present-value` is the default property.
```jsonc
{
"deviceInstance": 100, // the target device (from I-Am)
"objectType": "AnalogInput", // enum: AnalogInput/AnalogOutput/AnalogValue/
// BinaryInput/.../MultistateValue/...
"objectInstance": 3, // object-instance number
"propertyId": "PresentValue", // default; may be StatusFlags, Units, etc.
"arrayIndex": null, // non-null → element of an array property
"dataType": "Float32", // driver data-type hint (from browse-time units/type)
"writable": false, // v1: always false (read-only); v2: enables WriteProperty
"writePriority": 8 // v2 only: BACnet command priority 1..16
}
```
This blob is stored verbatim as the equipment tag's `TagConfig.FullName` reference; the driver's
`BacnetEquipmentTagParser.TryParse` (leading `{` ⇒ TagConfig blob, same convention as
`ModbusEquipmentTagParser`) turns it into the internal address record used for RP/RPM/COV keys.
The parser publishes values back keyed by the same reference string so the runtime forward-router
resolves them — identical to the Modbus contract.
---
## 4. BROWSEABILITY VERDICT + browse design
### VERDICT: **YES — strongly browseable.** BACnet is arguably *more* browseable than most
drivers: Who-Is enumerates devices, `object-list` enumerates a device's objects, and per-object
property reads give names/units/types. This is a first-class `IDriverBrowser`, on par with the
OpcUaClient browser.
New project **`ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser`**, mirroring
`ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser`: one `IDriverBrowser` (`DriverType = "Bacnet"`)
that `OpenAsync(configJson)` → binds a transient `BacnetClient` (+ BBMD reg if configured) and
returns a `BacnetBrowseSession : IBrowseSession`. Registered in
`AdminUI/EndpointRouteBuilderExtensions.cs` alongside the other two:
`services.AddSingleton<IDriverBrowser, BacnetDriverBrowser>();`.
Three-level tree mapped onto `RootAsync / ExpandAsync / AttributesAsync`:
| Seam | BACnet action | Returns |
|---|---|---|
| **`RootAsync`** | Broadcast `Who-Is` (range from config), **collect `I-Am` over `whoIsWindowMs`**, dedupe by device-instance | one `BrowseNode` per device (`NodeId = "dev:<instance>"`, `Kind=Folder`, `HasChildrenHint=true`, DisplayName = device `object-name` if a quick RP resolves, else `Device <instance>`) |
| **`ExpandAsync("dev:<instance>")`** | Read the device's `object-list` (RPM, segmented); optionally batch-read each object's `object-name` | one `BrowseNode` per object (`NodeId = "dev:<instance>/<objtype>:<objinst>"`, `Kind=Leaf` — an object commits to a tag, `DisplayName = object-name`) |
| **`AttributesAsync("dev:.../<objtype>:<objinst>")`** | RPM the object's key properties: `present-value, units, status-flags, description` (+ `state-text` for multistate) | `AttributeInfo[]` for the side-panel — surfaces the resolved `DriverDataType`, units, writability (present iff commandable object type), so the picker pre-fills the TagConfig |
**Commit mapping.** Selecting an object (a Leaf) + `present-value` → the picker emits the §3.2
TagConfig blob: `deviceInstance` from the device node, `objectType`/`objectInstance` from the
object node, `propertyId = "PresentValue"`, `dataType` from `AttributesAsync`. (Selecting a
non-default property is an advanced path — emit the chosen `propertyId`.)
### The time-bounded-discovery wrinkle
Unlike OPC UA's synchronous `Browse` request/response, **Who-Is is a broadcast + asynchronous
I-Am collection**: there is no "the response." The session handles this by making `RootAsync`
*block for `whoIsWindowMs`* while accumulating `OnIam` callbacks into a concurrent set, then
returning the snapshot. Design points:
- Cache the discovered device table on the session after the first `RootAsync`; a re-expand
doesn't re-broadcast. Offer a cheap "rescan" that re-broadcasts and merges (devices that
came online later appear). LastUsedUtc refresh + TTL reaper eviction as usual.
- The window is a **latency floor** on the first `RootAsync` (a few seconds). That's acceptable
for an interactive picker (show a spinner "discovering devices…"), and it matches the
`ConnectBudget` pattern the Galaxy browser already uses (30s connect budget). Bound it hard.
- Foreign-device mode: the transient browse client must register with the BBMD before Who-Is,
or no I-Ams cross the router — reuse the runtime driver's registration code in the browser
(same "mirror the runtime option shape" discipline the Galaxy browser follows).
- `object-list` on a big device can be large ⇒ **segmentation** must work in the browse client
(the ela-compil client handles segmented RPM; just don't cap `maxSegmentsAccepted` too low).
---
## 5. Test-fixture strategy
BACnet has good open simulators; all are UDP servers we can containerize on the shared docker
host (`10.100.0.35`) with a `project=lmxopcua` label, driven by `lmxopcua-fix up bacnet`:
1. **`bacnet-stack` demo server** ([bacnet-stack/bacnet-stack](https://github.com/bacnet-stack/bacnet-stack))
— the reference C stack's `bacserv` demo. Configurable device-instance + object set via env;
answers Who-Is/I-Am, RP/RPM, WP, and **SubscribeCOV**. The canonical target.
2. **`bacstack-compliance-docker`** ([fh1ch/bacstack-compliance-docker](https://github.com/fh1ch/bacstack-compliance-docker))
— a ready-made **BACnet server simulator as a Docker container** (DockerHub image), built on
bacnet-stack. Lowest-friction "just run a device."
3. **`bacnet-docker`** ([mnp/bacnet-docker](https://github.com/mnp/bacnet-docker) /
[desolat/bacnet-docker](https://github.com/desolat/bacnet-docker)) — docker-compose framework
to stand up **multiple** BACnet/IP servers + clients on one host; good for a **multi-device
Who-Is** discovery test and a **BBMD/foreign-device** topology test (put a BBMD + a device on
one compose network, register the driver as a foreign device).
4. **YABE** (Yet Another BACnet Explorer, Windows GUI) — manual cross-check that our reads/COV
match a known-good client (it uses the same ela-compil library, so it's an apples-to-apples
oracle). Also a good source of protocol-behavior sanity.
5. **ela-compil `BACnet.Examples/BasicServer`** — a pure-.NET device sim we can embed directly in
the integration test process (no docker), useful for deterministic COV-notification unit/
integration tests without UDP-on-CI flakiness. Pair with `BasicAdviseCOV` as the client oracle.
**Gotcha — UDP + broadcast on docker/CI:** BACnet relies on subnet **broadcast** for Who-Is;
bridged docker networks and macOS-hosted CI don't broadcast cleanly. Two mitigations: (a) run
BACnet fixtures on the Linux docker host with `network_mode: host` or a dedicated bridge and test
from a peer on that host (the `10.100.0.35` pattern), and (b) for hermetic unit tests use the
embedded `BasicServer` + **directed (unicast) Who-Is to a known instance/address**, sidestepping
broadcast entirely. Expect an **env-gated live-integration suite** (like the historian/S7 live
gates) rather than pure in-proc for the discovery/COV/BBMD legs.
---
## 6. Effort / risk / phasing
**Overall effort: Medium-Large.** Bigger than Modbus (Modbus has no discovery, no push), roughly
on par with or slightly above the OpcUaClient driver, because of the async-wrapping of a
callback-style library + the COV lifetime/renewal machinery + BBMD.
Suggested phasing (each phase independently shippable, mirroring how OpcUaClient landed across PRs):
- **Phase 1 — Connect + Read + Discover + Browse (read-only).** `BacnetDriverOptions`,
`BacnetDriver : IDriver, ITagDiscovery, IReadable, IHostConnectivityProbe`; the async adapter
over `BacnetClient`; RP/RPM read path; Who-Is/object-list discovery; the
`Bacnet.Browser` project + AdminUI registration; typed tag editor (`TagConfigEditorMap` +
`TagConfigValidator`) or start on the raw-JSON fallback. **Foreign-device/BBMD included here**
because without it the driver can't reach most real sites. *This is the MVP and delivers real value.*
- **Phase 2 — SubscribeCOV push.** `ISubscribable` via SubscribeCOV + lifetime-renew timer +
poll fallback for non-COV devices. The headline feature over poll-only drivers.
- **Phase 3 — Write.** `IWritable` via WriteProperty with priority-array semantics + commandable
validation. Deferred deliberately (see risk below).
- **Phase 4 — History / Alarms.** `IHistoryProvider` over **Trend Log** objects
(`ReadRange`); `IAlarmSource` over intrinsic/algorithmic **event enrollment** + COV of
`status-flags`. Nice-to-have; large surface.
### Top risks
1. **BBMD / foreign-device registration + UDP broadcast reachability (HIGH).** The single most
likely thing to make it "work on my bench, fail at the customer." The server is usually not on
the controllers' L2 segment, so correct foreign-device registration (and its TTL renewal) is
load-bearing, and it's exactly what's hardest to reproduce in docker/CI. Mitigation: build BBMD
in from Phase 1, test it explicitly with the `bacnet-docker` multi-network compose, and gate a
live suite against a real site BBMD (the established env-gated live-gate pattern).
2. **COV lifetime/renewal + non-COV device fallback (HIGH).** Subscriptions silently expire if
not renewed; some devices cap concurrent COV subscriptions or don't support COV at all
(must degrade to polling without the operator noticing a gap). Getting the renewal timer,
the confirmed-vs-unconfirmed choice, and the transparent poll fallback right is subtle and
won't be caught by unit tests — needs the live/sim COV soak (mirror the continuous-historization
live-gate discipline).
Secondary risks: the ela-compil library's **callback/blocking API** needs a careful async wrapper
(TCS + timeout + cancellation) to fit the driver's `async` seams; **write priority-array**
semantics (why Phase 3 is deferred — writing the wrong priority can fight the BMS's own control
logic, a safety concern); and **segmentation** for large `object-list`/RPM results (the library
handles it, but `maxSegmentsAccepted` must not be set too low).
---
## Sources
- ela-compil/BACnet (System.IO.BACnet, MIT, net10.0): <https://github.com/ela-compil/BACnet>
- ela-compil/BACnet.Examples (WhoIs/OnIam, RPM, SubscribeCOV, BBMD, ObjectBrowse):
<https://github.com/ela-compil/BACnet.Examples>
- bacnet-stack (reference C stack + demo server): <https://github.com/bacnet-stack/bacnet-stack>
- fh1ch/bacstack-compliance-docker (BACnet device sim container): <https://github.com/fh1ch/bacstack-compliance-docker>
- mnp/bacnet-docker (multi-device compose rig): <https://github.com/mnp/bacnet-docker>
- BACnet protocol / object model / COV overviews: Chipkin (<https://docs.chipkin.com/protocols/bacnet/>),
Actility (<https://www.actility.com/what-is-the-bacnet-object-model/>),
Software Toolbox (<https://softwaretoolbox.com/resources/what-is-bacnet>)
docs/research/drivers/melsec-slmp.md
+408
View File
@@ -0,0 +1,408 @@
# Mitsubishi MELSEC SLMP / MC-Protocol driver — research
**Status:** research spike (no code). Recommendation is to build a native
**SLMP / MC-protocol** Equipment-kind driver, hand-rolling the 3E-binary frame,
patterned on the existing S7 (byte-oriented binary TCP) and Modbus (polling,
pre-declared-tag, no-browse) drivers.
**Scope note vs. `docs/v2/mitsubishi.md`:** that document catalogues reaching
MELSEC *over Modbus TCP* via bolt-on Ethernet modules (QJ71MT91, RJ71EN71 MODBUS
slave mode, FX5U built-in MODBUS, FX3U-ENET-P502…). This driver is the **native
alternative** that talks the PLC's own protocol and supersedes that lossy path.
The two can coexist — the Modbus path stays valid for sites that only enabled the
MODBUS slave — but SLMP is the strictly-better option wherever the CPU's native
Ethernet/MC endpoint is reachable.
---
## 1. Protocol summary + the gap it closes
### SLMP vs. MC 3E / 4E / 1E
**MC protocol** (MELSEC Communication protocol) is Mitsubishi's long-standing
client/server request-response protocol for reading and writing CPU device
memory over Ethernet (and serial). **SLMP** (SeamLess Message Protocol) is the
modern superset: SLMP's 3E and 4E frames are *bit-for-bit identical* to the
"QnA-compatible 3E/4E" MC-protocol frames, so an SLMP client talks to any
MC-protocol server and vice-versa. For our purposes **SLMP ≡ MC-protocol 3E/4E**;
we implement one framer and cover both product lines.
Frame families:
| Frame | Subheader (req/resp) | Use | Notes |
|---|---|---|---|
| **3E** | `5000` / `D000` | **Primary target.** Q / L / iQ-R / iQ-F / FX5 Ethernet | Stateless; no serial number. Simplest and most universal. |
| **4E** | `5400…0000` / `D400…0000` | iQ-R / iQ-F, when request/response correlation is wanted | 3E **plus** a 2-byte serial number echoed in the response — lets a client pipeline and match replies. Superset of 3E. |
| **1E** | `00`/`01`/`02`/`03` cmd bytes; `80`+cmd resp | Legacy A-series and some FX | Different, older layout. Only needed for very old CPUs. |
Each frame carries a **binary** or **ASCII** encoding. Binary is half the bytes
and what every modern deployment uses; ASCII exists mainly for text-only serial
links. We implement **3E binary** first, structure the framer so **4E binary**
is a thin superset, and treat 1E + ASCII as out-of-scope v1 (documented fallback).
### 3E binary request layout (the frame we hand-roll)
Reading `D200`, 1 word (from Mitsubishi *SLMP Reference Manual* SH-080956ENG and
the FA-Support worked example [3][4]):
```
50 00 Subheader (request 3E)
00 Network No. (0x00 = own/local network)
FF PC No. (0xFF = local/host CPU)
FF 03 Request dest module I/O (0x03FF = own CPU)
00 Request dest module station
0C 00 Request data length (little-endian; bytes that follow this field)
10 00 Monitoring timer (0x0010 = 250 ms units; 0 = wait forever)
01 04 Command (0x0401 Batch Read, little-endian on wire)
00 00 Subcommand (0x0000 = word units; 0x0001 = bit units)
C8 00 00 Head device number (200, 3 bytes little-endian)
A8 Device code (0xA8 = D, data register)
01 00 Device point count (1 word, little-endian)
```
Response: `D000` subheader, the routing echo, a data length, a **2-byte end code**
(`0000` = success; non-zero = error, e.g. `C051` device-count over range,
`4031` wrong device, `C059`/`C05C` command/subcommand error), then the payload.
Key commands:
| Command | Subcmd | Meaning | Max points |
|---|---|---|---|
| `0401` | `0000` | Batch Read, **word** units | 960 words (3E) |
| `0401` | `0001` | Batch Read, **bit** units | 7168 bits |
| `1401` | `0000` / `0001` | Batch Write, word / bit | 960 / 7168 |
| `0403` | `0000` | **Random Read** (scattered word/dword addresses) | 192 points |
| `1402` | `0000` | Random Write (scattered) | ~160 points |
| `0406` | — | Read block (multiple blocks) | — |
| `1001` / `1002` | — | Remote STOP / RUN | — |
| `0101` | — | Read CPU model name (cheap connectivity probe) | — |
### Device codes (3E binary, 1-byte code)
Bit-vs-word classification is the crux of the addressing model. Non-exhaustive;
**verify the full table against SH-080956ENG before coding** — these are the
common ones:
| Device | Code (hex) | Kind | Number base in engineering tools |
|---|---|---|---|
| X input | `9C` | **bit** | **hex** (Q/L/iQ-R), octal (FX/iQ-F) |
| Y output | `9D` | **bit** | **hex** (Q/L/iQ-R), octal (FX/iQ-F) |
| M internal relay | `90` | bit | decimal |
| L latch relay | `92` | bit | decimal |
| F annunciator | `93` | bit | decimal |
| B link relay | `A0` | **bit** | **hex** |
| SM special relay | `91` | bit | decimal |
| D data register | `A8` | **word** | decimal |
| W link register | `B4` | **word** | **hex** |
| R file register | `AF` | word | decimal |
| ZR extended file reg | `B0` | word | **hex** |
| SD special register | `A9` | word | decimal |
| TN timer current | `C2` | word | decimal |
| CN counter current | `C5` | word | decimal |
| TS/CS timer/counter contact | `C1`/`C4` | bit | decimal |
### The gap SLMP closes over the Modbus-TCP path
`docs/v2/mitsubishi.md` documents, in detail, why the Modbus path is lossy. SLMP
removes each of those failure modes:
1. **No per-site "Modbus Device Assignment" block.** SLMP addresses the CPU's
device memory directly by `<device code, number>` (e.g. `D200`, `M100`).
There is *no* 16-entry assignment table to configure in GX Works and no
"two sites with the same module expose different maps" problem — the biggest
single source of Modbus-path fragility disappears.
2. **X/Y reachable natively** as their own bit devices, not shoehorned into a
second non-zero coil bank (Modbus default maps X/Y at offset 8192+). The
hex-vs-octal number-base trap **remains** (it is a CPU convention, not a
transport artifact) — see §2/§3 — but there is no *second* Modbus-offset
translation layered on top.
3. **Full device coverage.** L, F, B, W, R, ZR, SM/SD, timers/counters are all
directly addressable. The Modbus path can only see whatever the engineer
chose to expose in the assignment block.
4. **No FC caps / sub-spec quirks.** No "QJ71MT91 doesn't support FC16", no
125-register FC03 ceiling, no odd-coil-byte truncation. SLMP batch read is
960 words in one PDU vs. Modbus' 125.
5. **Random read/write** of scattered addresses in one round-trip (`0403`/`1402`)
— impossible in Modbus without one PDU per contiguous run.
6. **Word-order (CDAB) is still a per-tag concern** (§3) — 32-bit values still
span two consecutive words low-word-first — but this is now *our* decode
choice, not something filtered through a module's fixed behavior.
### .NET library options — LICENSE analysis
| Option | License | .NET / maturity | Verdict |
|---|---|---|---|
| **HslCommunication** (`MelsecMcNet`) | **Commercial / NOT free** — "公对公签订合同", company-to-company contract + VAT invoice; source only with paid license [5][6] | net35+; very mature, widely used | **BLOCKER — do not use.** Requires a signed commercial contract. Rules it out for this repo. |
| **McProtocol** (SecondShiftEngineer) | **LGPL-3.0** [7] | netstandard2.0 (loads on net10); MC1E/3E/4E; last updated **2018** | LGPL is usable when consumed as an unmodified dynamic library, but it is a **copyleft** dependency to vet with legal, and it is **7+ years stale**. Useful as a *reference*, weak as a *dependency*. |
| **McpX** | **MIT** [8] | net7/8/9 + netstandard, cross-platform; TCP/UDP, 3E/4E binary+ASCII, batch + random + monitor + remote password; first release 2025, actively developed (v0.7.0 Jun 2026), ~53★, single maintainer | **MIT is clean.** The most license-friendly library and feature-complete, but young + one-maintainer + pre-1.0 → supply-chain/bus-factor risk for a production driver. |
| **`s-pms/melsec_mc_net`** | **MIT** [9] | **C** (not .NET) — Windows/Linux; 3E binary+ASCII, batch + typed read/write; full device-code table | Not consumable from .NET, but an **excellent MIT reference** for a hand-roll (device codes, framing, transaction serialization). |
| **libslmp / libslmp2** (Neucrede) | open-source C/C++ | C/C++ | Reference only, not .NET. |
**Recommendation: hand-roll the 3E-binary framer.** Rationale:
- The frame is small and fully specified (SH-080956ENG); the existing **S7 driver
already proves this repo hand-rolls byte-oriented binary TCP** with a clean
`IS7Plc` seam and a fake for tests. SLMP is *simpler* than S7's PDU negotiation.
- The only clean-licence library (McpX, MIT) is young/one-maintainer/pre-1.0 —
taking it as a hard dependency in a production OT server is more risk than a
~600-line framer we own and test.
- HslCommunication (the mature option) is a hard **commercial-licence blocker**.
- Keep McpX and the MIT `melsec_mc_net`/`libslmp` sources as **cross-check
references** for the framer + device-code table.
The repo already carries a **head-start**: `MelsecAddress` +`MelsecFamily` in
`ZB.MOM.WW.OtOpcUa.Driver.Modbus.Addressing` encode the hex-vs-octal X/Y family
logic (written for the Modbus path). That logic ports directly into the new
driver's addressing project.
---
## 2. Capability mapping
Same capability-interface set as Modbus/S7 (`IDriver, ITagDiscovery, IReadable,
IWritable, ISubscribable, IHostConnectivityProbe`). This is a **full read/write**
PLC driver.
| Capability | SLMP mapping |
|---|---|
| **Connect** (`IDriver.InitializeAsync`) | TCP client to CPU Ethernet port (default **502** is Modbus; SLMP default is engineer-configured, commonly `1025`/`5007` or a user-set port; **UDP optional** — SLMP supports both; start TCP-only like Modbus). Open socket, optionally issue `0101` Read-CPU-model as a connect assertion. Reconnect/keepalive/idle-disconnect knobs mirror `ModbusDriverOptions`. |
| **Read** (`IReadable`) | **Batch Read `0401`** for contiguous runs (word subcmd `0000` for D/W/R/ZR/TN/CN; bit subcmd `0001` for M/X/Y/B/L). **Random Read `0403`** to coalesce scattered addresses into one PDU. A read planner (like Modbus' block coalescing / `MaxReadGap`) groups tags by device code into batch reads, splitting at the 960-word PDU cap. |
| **Write** (`IWritable`) | **Batch Write `1401`** (word/bit) for runs; **Random Write `1402`** for scattered. Bit-write to M/Y honored; word-write to D/W/R. Write-through gated by the standard `WriteOperate` node authz + `NodeWriteRouter` like every other protocol driver (the `EquipmentTagRefResolver<TDef>` pattern). |
| **Subscribe** (`ISubscribable`) | **Poll-based**, exactly like Modbus/S7 — **SLMP has no native push/unsolicited path** for general device polling. Reuse the shared polling overlay engine (the same `ISubscribable` polling helper Modbus uses). (SLMP *does* have a "device monitor register" `0801`/monitor `0802` mechanism, but it is a stateful convenience, not a change-push; poll is the right model.) |
| **Discover** (`ITagDiscovery`) | **Offline / config-driven.** SLMP exposes **no on-wire symbol table** — the driver returns exactly the pre-declared tags from `SlmpDriverOptions.Tags`, identical to Modbus. No online enumeration. |
| **Probe** (`IHostConnectivityProbe`) | Cheap `0101` Read-CPU-model, or a 1-word `0401` read of a known device, on an interval; raise `OnHostStatusChanged` on transitions. |
| **Alarms / History** | None native. (A future scripted-alarm layer works the same as for any polling driver.) |
### Data-type mapping (SLMP words/bits → OPC UA)
| OPC UA type | SLMP encoding | Notes |
|---|---|---|
| Boolean | 1 bit device (M/X/Y/B/L) via bit-subcmd; or 1 bit of a word device | Bit-in-word needs a `bitIndex` like Modbus `BitInRegister`. |
| Int16 / UInt16 | 1 word device | Native width. |
| Int32 / UInt32 | 2 consecutive words | **word order** matters — MELSEC native is low-word-first (`CDAB` when viewed as a word pair). Per-tag `wordOrder` knob. |
| Float (Single) | 2 words | Same word-order concern. |
| Int64 / UInt64 / Double | 4 words | Same. |
| String | N words, 2 ASCII chars/word | Byte-order-within-word knob (like Modbus `StringByteOrder`); MELSEC packs low byte = first char in some setups. |
| DateTime | vendor-specific packing | v1: skip or map from a documented word layout. |
| Array | `ValueRank=1`, `arrayLength` × element-words, one batch read | Cap at 960-word PDU; auto-chunk. |
**Addressing model = device code + number + base.** A tag names a *device code*
(`D`,`M`,`X`,`Y`,`W`,`B`,`R`,`ZR`,…), a *device number*, and — critically — the
number's **base**: X/Y/B/W/ZR are **hex** on Q/L/iQ-R and X/Y are **octal** on
FX/iQ-F; D/M/L/F/R/timers/counters are **decimal** everywhere. The driver must
preserve the engineering-tool base the operator typed (the `MelsecFamily` enum
already models this). Word-vs-bit is a property of the device code and selects
the batch-read subcommand.
---
## 3. TagConfig JSON shape
Mirrors `ModbusTagDefinition` / `ModbusEquipmentTagParser` (leading-`{` marks an
equipment-tag TagConfig blob; strict enum reads reject typos → `BadNodeIdUnknown`).
Proposed per-tag fields:
| Field | Type | Meaning |
|---|---|---|
| `device` | enum string | Device code: `D`,`M`,`X`,`Y`,`W`,`B`,`R`,`ZR`,`L`,`F`,`SM`,`SD`,`TN`,`CN`,… |
| `number` | string | Device number **as the operator types it in GX Works** (kept as string to preserve hex/octal). |
| `numberBase` | enum | `Decimal` / `Hex` / `Octal` — defaulted from the driver-level `MelsecFamily`, overridable per tag. |
| `dataType` | enum | `Boolean,Int16,UInt16,Int32,UInt32,Int64,UInt64,Float,Double,String`. |
| `bitIndex` | int 0–15 | For a Boolean read from a bit of a word device (omit for true bit devices). |
| `wordOrder` | enum | `ABCD` / `CDAB` — 32/64-bit word order (MELSEC native = `CDAB`, low word first). Default `CDAB`. |
| `stringLength` | int | ASCII chars for `String` (2 per word). |
| `stringByteOrder` | enum | High-byte-first vs low-byte-first within a word. |
| `arrayLength` | int | `isArray && arrayLength>=1` → OPC UA array. |
| `writable` | bool | Defaults true; node authz is the real gate. |
**Example** — a 32-bit float production count at `D200` (hex-family Q CPU, native
CDAB word order), and a bit alarm at `M100`:
```json
{
"device": "D",
"number": "200",
"numberBase": "Decimal",
"dataType": "Float",
"wordOrder": "CDAB",
"writable": false
}
```
```json
{
"device": "M",
"number": "100",
"numberBase": "Decimal",
"dataType": "Boolean"
}
```
```json
{
"device": "X",
"number": "1A",
"numberBase": "Hex",
"dataType": "Boolean",
"_comment": "Q-series X1A = physical input 26 decimal; hex base preserved from GX Works"
}
```
Driver-level `SlmpDriverOptions` mirrors `ModbusDriverOptions`: `Host`, `Port`,
`Frame` (`ThreeE`/`FourE`), `Encoding` (`Binary`), `NetworkNo`/`PcNo`/`DestModuleIo`/
`DestStation` routing bytes (defaults `0x00/0xFF/0x03FF/0x00` = local CPU),
`MonitoringTimer`, `Family` (`Q_L_iQR`/`F_iQF`), `Timeout`, reconnect/keepalive/
idle knobs, `MaxPointsPerRead` (≤960), a read-coalescing gap budget, and the
pre-declared `Tags` list.
---
## 4. BROWSEABILITY VERDICT — **NO**
**Definitively not browseable. No `*.Browser` project is warranted.**
SLMP/MC-protocol exposes a **flat, typed device-memory space** (`D`, `M`, `X`,
`Y`, `W`, `R`, …) addressed purely by `<device code, number>`. There is **no
on-wire symbol table, no tag directory, and no metadata service** in the
protocol. The commands enumerated in the SLMP Reference Manual are memory
read/write, remote CPU control, and self-test — none returns "what tags exist."
This is structurally identical to Modbus and S7, both of which are (correctly)
non-browseable in this repo: the address space is a raw memory map, not a
discoverable namespace. The symbolic tag names live only in the GX Works project
file on the engineer's PC, never on the wire.
**Searched-for exceptions, none qualifying:**
- **CPU model / capability reads** (`0101`, self-test `0619`) return device
*types and counts*, not a symbol list — useful for a connect assertion, not
browse.
- **Device monitor register** (`0801`/`0802`) is a client-side convenience for
re-reading a *previously specified* set — the client supplies the addresses;
the CPU never volunteers them.
- **Label/tag communication** (iQ-R "device/label access via SLMP" with a name
string) exists in newer firmware but requires the client to *already know* the
global-label name and only resolves a name the engineer defined — it is a
by-name read, still **not an enumeration**. Not a browse source.
- **GX Works project (`.gx3`) / CSV label export** could seed tags *offline*, but
that is a file-import feature, not an on-wire `IDriverBrowser` session, and is
out of scope here.
Verdict: **NO browser.** Tags are authored via the pre-declared list + the typed
tag editor (§7), exactly like Modbus/S7. In `TagConfigEditorMap` the driver gets
a typed editor but **no** `IDriverBrowser`/address-picker.
---
## 5. Test-fixture strategy
Follow the repo's established **hand-rolled TCP stub** pattern (S7's `IS7Plc`
fake, FOCAS's mock, Modbus' transport seam). SLMP is a request/response binary
protocol over TCP, so a deterministic stub is straightforward and CI-friendly.
**Recommended layers:**
1. **Unit — framer round-trip tests** (no socket). Encode/decode 3E-binary
request/response byte arrays against golden vectors taken from SH-080956ENG
and the MIT `melsec_mc_net`/McpX examples. This is where the device-code table,
hex/octal number parsing, word-order (CDAB), and end-code handling get pinned.
Port the existing `MelsecAddress` address tests.
2. **In-process fake `ISlmpClient`** (mirrors `FakeFocasClient` / S7's fake) for
driver-level read/write/subscribe/discover behavior without a network.
3. **Integration — a hand-rolled SLMP server stub** (a `TcpListener` that decodes
3E-binary requests and serves a seeded device-memory dictionary), packaged as a
Docker fixture under `tests/.../Docker/` with the `project=lmxopcua` label and
an env-gated skip (like `FocasSimFixture`'s `localhost:PORT` probe). This gives
real-socket read/write round-trips, batch-read chunking, and reconnect tests
deterministically. **This is the primary integration path** — write the stub;
don't depend on vendor tooling in CI.
4. **Optional real-target gates (not in CI):**
- **GX Works3 / GX Works2 simulator (GX Simulator3)** exposes an SLMP-capable
virtual CPU but is **Windows-only, licensed, GUI-driven** — usable for a
manual bring-up gate on a Windows box, not for automated CI (same posture as
the AVEVA/mxaccessgw live gates).
- **Open-source sims:** community MC/SLMP server sims exist (e.g. Go
`moge800/gomcprotocol`, Node `plcpeople/mcprotocol` has a server mode,
`libslmp`/`libmelcli` samples). Any could back a container, but a
repo-owned .NET stub is lower-maintenance and matches house style.
- A real FX5U / iQ-R on the bench is the final acceptance gate (a `LiveIntegration`
env-gated suite, mirroring the historian live gate).
Reuse `docs/v2/mitsubishi.md`'s `Mitsubishi_<model>_<behavior>` test-naming
convention for the behavioral cases (CDAB word order, hex X20 = 32, octal X20 = 16,
960-word batch cap, end-code `C051` on over-range).
---
## 6. Effort / risk / phasing
**Overall effort: moderate** — comparable to the S7 driver. The framer is small;
the complexity budget is almost entirely in the **addressing model** (device
codes × bit/word × hex/octal/decimal base × CDAB word order), which is exactly
where MELSEC drivers go wrong. Front-load it.
**Top risks:**
1. **Addressing correctness** (highest). The hex/octal/decimal base split per
device family, plus CDAB word order for 32/64-bit values, is the #1 real-world
bug source (per `docs/v2/mitsubishi.md`). Mitigation: exhaustive framer/address
unit tests with golden vectors *before* any driver wiring; reuse `MelsecAddress`.
2. **Library/licensing** — resolved by hand-rolling (avoids the HslCommunication
commercial blocker and the McpX bus-factor risk), but it means we own the
protocol correctness. Cross-check against the MIT `melsec_mc_net` + McpX + the
SLMP Reference Manual.
3. **Frame/port/family fragmentation** — 3E vs 4E, binary vs ASCII, TCP vs UDP,
Q/L/iQ-R hex vs FX/iQ-F octal, engineer-chosen port. Mitigation: ship **3E
binary / TCP** only in v1, structure the framer so 4E is a superset, document
the rest as fallbacks (same discipline as S7).
**Phasing:**
- **Phase 0 — Addressing + framer (no I/O).** New `…Driver.Slmp.Addressing`
project (device codes, base parsing, word order — port `MelsecAddress`) and a
3E-binary encoder/decoder with golden-vector unit tests. `…Driver.Slmp.Contracts`
with `SlmpDriverOptions` + `SlmpTagDefinition` + `SlmpEquipmentTagParser`.
- **Phase 1 — Read path.** `SlmpDriver : IDriver, ITagDiscovery, IReadable,
IHostConnectivityProbe` over an `ISlmpClient` TCP seam (S7-style), with a fake
+ the Docker stub server. Batch Read `0401` + read planner/coalescing + all
scalar/array/string/word-order decoding.
- **Phase 2 — Write + Subscribe.** Add `IWritable` (Batch Write `1401`, Random
Write `1402`; write-through via the `EquipmentTagRefResolver<TDef>` +
`NodeWriteRouter` pattern) and `ISubscribable` on the shared polling overlay.
- **Phase 3 — AdminUI typed tag editor.** `SlmpTagConfigEditor` +
`SlmpTagConfigModel` (FromJson/ToJson/Validate), registered in
`TagConfigEditorMap` + `TagConfigValidator`. Driver-edit page + `IDriverProbe`
(with the `JsonStringEnumConverter` fix from the driver enum-serialization
memory). **No** `IDriverBrowser` (§4).
- **Phase 4 — Random read/write coalescing + 4E frame + live gate.** `0403`/`1402`
optimization, optional 4E, and an env-gated `LiveIntegration` suite against a
real FX5U/iQ-R or GX Simulator3.
---
## References
1. Mitsubishi Electric, *SLMP Reference Manual* (SH-080956ENG) —
https://dl.mitsubishielectric.com/dl/fa/document/manual/plc/sh080956eng/sh080956engl.pdf
2. Mitsubishi Electric, *MELSEC iQ-F FX5 User's Manual (SLMP)* (JY997D56001) —
https://dl.mitsubishielectric.com/dl/fa/document/manual/plcf/jy997d56001/jy997d56001k.pdf
3. Inductive Automation, *Understanding Mitsubishi PLCs* (3E frame, D-register
word order) —
https://support.inductiveautomation.com/hc/en-us/articles/16517576753165-Understanding-Mitsubishi-PLCs
4. FA Support Me, *PLC and PC communication via SLMP Protocol* (3E-binary worked
example) —
https://www.fasupportme.com/portal/en/kb/articles/plc-and-pc-communication-via-slmp-protocol
5. dathlin/HslCommunication (GitHub) — "Not free open source" —
https://github.com/dathlin/hslcommunication
6. HslCommunication commercial-licence page —
http://www.hslcommunication.cn/Cooperation
7. McProtocol NuGet (LGPL-3.0, MC1E/3E/4E, last updated 2018) —
https://www.nuget.org/packages/McProtocol/
8. McpX NuGet / repo (MIT, .NET 7/8/9, 3E/4E binary+ASCII, batch+random) —
https://libraries.io/nuget/McpX
9. s-pms/melsec_mc_net (GitHub, MIT, C reference impl, full device-code table) —
https://github.com/s-pms/melsec_mc_net
10. Neucrede/libslmp2 (open-source C/C++ SLMP library, reference) —
https://github.com/Neucrede/libslmp2
11. In-repo: `docs/v2/mitsubishi.md` (MELSEC-over-Modbus quirks this driver
supersedes) and `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Addressing/MelsecAddress.cs`
(hex/octal family logic to port).
docs/research/drivers/modbus-rtu.md
+422
View File
@@ -0,0 +1,422 @@
# Research: Modbus RTU (serial) support
**Status:** Research / roadmap. Not implemented.
**Date:** 2026-07-15
**Author:** research sweep
**Scope:** Add Modbus **RTU** (serial + RTU-over-TCP) to the OtOpcUa server.
---
## TL;DR
Modbus RTU is **an added transport mode on the existing `ModbusDriver`, not a new
driver.** The existing driver already splits the *protocol data unit* (PDU: function
code + data) from the *transport* (socket + MBAP framing) behind a clean
`IModbusTransport` seam, and the driver injects transports through a
`Func<ModbusDriverOptions, IModbusTransport>` factory. RTU is a second
`IModbusTransport` implementation that swaps MBAP framing for
`[address][PDU][CRC-16]` framing over a serial line (or a raw TCP socket, for
RTU-over-TCP). The register model, function codes, data-type codecs, read planner,
coalescing, deadband, write path, and OPC UA materialisation are **100% reused
unchanged**. Browseable = **NO** (flat register space, no discovery — identical to
Modbus TCP). The pragmatic primary path for a containerised Linux server is
**RTU-over-TCP to a serial→Ethernet gateway**, with direct `System.IO.Ports` serial as
a secondary path for bare-metal / device-mapped deployments. This is very likely the
**lowest-effort item on the driver roadmap**; the only real risk is serial-line
timing/behaviour on Linux and in containers.
---
## 1. Extend-vs-new-driver verdict — **EXTEND**
### Why the existing code makes this easy
The Modbus driver is already layered exactly the way you'd want in order to add a
transport. The seam is `IModbusTransport`
(`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs`):
```csharp
public interface IModbusTransport : IAsyncDisposable
{
Task ConnectAsync(CancellationToken ct);
Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct);
}
```
The interface doc comment is explicit that it takes **"a PDU (function code + data,
excluding the 7-byte MBAP header)"** and returns the **response PDU** — "the transport
owns transaction-id pairing, framing, and socket I/O." That is precisely the RTU-vs-TCP
boundary. Everything above the seam is transport-neutral:
- **`ModbusDriver.cs`** builds every PDU as a raw `byte[]` of `[functionCode, ...data]`
— FC01/02/03/04/05/06/15/16 encoders (`ReadRegisterBlockAsync`, `ReadBitBlockAsync`,
the FC05/06/15/16 write paths, the FC03→bit-swap→FC06 RMW) all call
`transport.SendAsync(unitId, pdu, ct)` and decode the returned PDU. None of them
touch MBAP, sockets, or CRC.
- **`ModbusTcpTransport.cs`** is the *only* place the 7-byte MBAP header, the
transaction-id counter (`_nextTx`), and `TcpClient`/`NetworkStream` I/O live. It
wraps each PDU as `[TxId][Proto=0][Length][UnitId] + PDU`, single-flights via a
`SemaphoreSlim _gate`, and does socket-level reconnect/retry.
- **The driver injects the transport**: `ModbusDriver`'s constructor takes
`Func<ModbusDriverOptions, IModbusTransport>? transportFactory`, defaulting to
`o => new ModbusTcpTransport(...)`. Tests already substitute in-memory fakes through
this same hook.
So the entire protocol/codec/planner/health/OPC-UA surface is transport-agnostic
today. Adding RTU means adding **one class** behind the existing seam plus the config
plumbing to select it.
### The concrete refactor
There is essentially **no refactor of existing code needed** — the split is already
done. The work is additive:
1. **New `ModbusRtuTransport : IModbusTransport`** (serial). Wraps a
`System.IO.Ports.SerialPort`. `ConnectAsync` opens the port with the configured
baud/data-bits/parity/stop-bits. `SendAsync` frames the ADU as
`[unitId][PDU][CRC-lo][CRC-hi]`, enforces the ≥3.5-character inter-frame silence
before transmit, writes, then reads the response, strips the address + validates and
strips the CRC-16, and returns the bare PDU. Reuses the same single-flight `_gate`
pattern (mandatory on RTU — see §2).
2. **New `ModbusRtuOverTcpTransport : IModbusTransport`** (RTU tunnelled over a socket).
Identical RTU framing (`[address][PDU][CRC]`, no MBAP, no TxId) but the byte stream
rides a `TcpClient`/`NetworkStream` to a serial gateway instead of a COM port. This
can share the socket-management, keepalive, idle-disconnect, and reconnect/backoff
machinery already in `ModbusTcpTransport` — the *only* difference from
`ModbusTcpTransport` is the ADU framing (CRC instead of MBAP) and the absence of a
transaction id. Consider extracting the socket lifecycle into a small shared base or
helper so both TCP variants share it; the MBAP-vs-CRC framing is the swap point.
3. **A CRC-16 helper** (`ModbusCrc.Compute(ReadOnlySpan<byte>)`) — the standard Modbus
CRC with polynomial `0xA001` (reflected `0x8005`), CRC appended low-byte-first. Put
it in `...Driver.Modbus` (or `.Addressing`).
4. **Transport-mode selection** in `ModbusDriverOptions` + the config DTO + the factory
(`ModbusDriverFactoryExtensions.CreateInstance`) — a `Transport` discriminator
(`Tcp` | `Rtu` | `RtuOverTcp`) plus the serial parameters, wiring the default
`transportFactory` closure to pick the right transport (see §3).
5. **AdminUI**: extend `ModbusDriverPage.razor` with a serial parameters panel shown
when `Transport != Tcp`. No new page — same driver page.
**Framing subtlety worth calling out.** In Modbus TCP the MBAP `Length` field tells the
transport exactly how many response bytes to read (`ModbusTcpTransport.SendOnceAsync`
reads a 7-byte header, then `Length-1` more). **RTU has no length field.** The RTU
transport must determine the response length either (a) by parsing the function code
and byte-count field (read responses carry a byte-count; FC05/06/15/16 echoes are
fixed-length; an exception response is a fixed 5 bytes with the high bit set on the FC),
or (b) by reading until an inter-character idle gap (T1.5/T3.5) elapses. Function-code-
aware length calculation is the robust choice and is simplest given the driver already
knows the FC set. This is the single genuinely new piece of logic RTU introduces.
**Verdict: extend the existing `ModbusDriver` with two new `IModbusTransport`
implementations + a transport selector.** A sibling driver would duplicate the entire
codec/planner/health/materialisation surface for zero benefit — the protocol above the
wire is identical.
---
## 2. Capability mapping — identical to TCP, four deltas
RTU is *the same Modbus application protocol* as TCP: same register model (Coils /
Discrete Inputs / Input Registers / Holding Registers), same function codes
(FC01–06, 15, 16, and the exception PDU convention), same data types, same read/write
semantics. Everything the driver does above the transport seam is unchanged.
| Capability | Modbus TCP (today) | Modbus RTU (added) | Delta? |
|---|---|---|---|
| Register model + function codes | ✅ | ✅ identical | none |
| Read (FC01/02/03/04) | ✅ | ✅ | none — same PDU |
| Write (FC05/06/15/16) | ✅ | ✅ | none — same PDU |
| Data-type codecs, byte order, arrays, strings, BCD, bit-in-register | ✅ | ✅ | none |
| Read coalescing / auto-prohibit / deadband / WriteOnChangeOnly | ✅ | ✅ | none |
| Connectivity probe (FC03@0) | ✅ | ✅ | none — goes through `SendAsync` |
| **Transport** | TCP socket | serial line / RTU-over-TCP socket | **serial vs socket** |
| **Framing** | 7-byte MBAP header + TxId; TCP guarantees integrity | `[addr][PDU][CRC-16]`; app-level CRC | **CRC-16 vs MBAP** |
| **Unit/slave id** | often 1 (one device per socket); gateway multiplexing exists | **central** — one bus, multiple drop slaves addressed by unit id | **more prominent** |
| **Timing** | TCP framing; no inter-frame constraint | **≥3.5-char inter-frame silence**, T1.5 inter-char | **timing-based framing** |
| Browse/discovery | none | none | none (see §4) |
| Historian / alarms | out of scope | out of scope | none |
**The four deltas in detail:**
1. **Transport** — a `SerialPort` (or a socket to a gateway) replaces the `TcpClient`.
The socket-reconnect / keepalive / idle-disconnect logic in `ModbusTcpTransport` is
TCP-specific and does **not** apply to a serial line (a COM port doesn't "drop" the
way a NAT'd socket does); the RTU serial transport has its own simpler
open/reopen-on-error model. RTU-over-TCP *does* reuse the socket lifecycle.
2. **Framing** — RTU wraps `[slaveAddress(1)][PDU][CRC-16-lo][CRC-16-hi]`. There is no
MBAP header and no transaction id. The CRC-16 (poly `0xA001` reflected, appended
**low byte first**) replaces TCP's transport-level integrity. The transport computes
CRC on send and validates on receive, treating a CRC mismatch as a
desync/communication error (map onto the existing `ModbusTransportDesyncException` /
`BadCommunicationError` handling).
3. **Unit-id semantics** — on RTU the unit/slave id is *the* addressing mechanism for a
multi-drop bus; a single serial line commonly hosts several slaves. The driver
already supports this: `ModbusTagDefinition.UnitId` is a per-tag override and
`ResolveUnitId` + `BuildSlaveHostName` already key per-slave resilience by
`host:port/unitN`. Multi-drop RTU "just works" with the existing per-tag UnitId
plumbing — the read planner already refuses to coalesce across UnitIds. (For RTU the
per-slave "host" key becomes `COMx/unitN` or `gatewayHost:port/unitN`.)
4. **Timing** — RTU frames are delimited by silence, not length. Requests must be
preceded by ≥3.5 character-times of idle; responses are read until the same idle gap
(or, preferably, by function-code-aware length). Character time depends on
baud/word-length: at 9600 baud, 8-N-1 (10 bits/char), 3.5 chars ≈ 3.6 ms. **Above
19200 baud the spec fixes T3.5 at 1.75 ms and T1.5 at 750 µs** rather than scaling
further. Single-flight is mandatory: RTU has no transaction id to correlate an
interleaved response, so at most one transaction may be in flight on a bus — the
existing `_gate` semaphore already provides this.
**Read + write are both fully supported**, exactly as with Modbus TCP.
Sources for framing/timing/CRC claims:
[ModbusKit RTU/ASCII/TCP comparison](https://modbuskit.com/en/blog/modbus-rtu-tcp-ascii-comprehensive-comparison),
[ModbusSimulator RTU vs TCP](https://modbussimulator.com/blog/modbus-rtu-vs-tcp-comparison-guide),
[Industrial Monitor Direct — TCP vs RTU-over-TCP](https://industrialmonitordirect.com/blogs/knowledgebase/modbus-tcp-vs-modbus-rtu-over-tcpip-protocol-differences).
---
## 3. Config JSON shape
Per-tag config is **unchanged** — the existing `ModbusTagDefinition` / `ModbusTagDto`
(region, address, data type, byte order, `UnitId` per-tag override, etc.) already
covers everything RTU needs. The additions are **driver-level transport fields** only.
Proposed additions to the driver config DTO (`ModbusDriverConfigDto`):
| Field | Type | Applies to | Notes |
|---|---|---|---|
| `Transport` | `"Tcp"` \| `"Rtu"` \| `"RtuOverTcp"` | all | Discriminator. Default `"Tcp"` (back-compat). |
| `SerialPort` | string | Rtu | COM port / device path, e.g. `"COM3"` or `"/dev/ttyUSB0"`. |
| `BaudRate` | int | Rtu | e.g. 9600, 19200, 38400, 115200. |
| `DataBits` | int | Rtu | Usually 8 (RTU). |
| `Parity` | `"None"`\|`"Even"`\|`"Odd"` | Rtu | Modbus spec default **Even**; many devices use None. |
| `StopBits` | `"One"`\|`"Two"` | Rtu | 1 with parity, 2 without, per spec. |
| `Host` / `Port` | string / int | Tcp, **RtuOverTcp** | Reused for RtuOverTcp — the serial-gateway's socket. |
| `InterFrameDelayMs` | int? | Rtu | Optional override of the computed T3.5 silence for slow/RF links. |
`Host`/`Port`/`UnitId`/`TimeoutMs`/`MaxRegistersPerRead`/... all stay. Serial fields are
ignored when `Transport=Tcp`; `Host`/`Port` are ignored when `Transport=Rtu`.
### Example A — direct serial RTU
```json
{
"Transport": "Rtu",
"SerialPort": "/dev/ttyUSB0",
"BaudRate": 19200,
"DataBits": 8,
"Parity": "Even",
"StopBits": "One",
"UnitId": 1,
"TimeoutMs": 1000,
"Tags": [
{ "Name": "Flow", "AddressString": "40001:F:ABCD", "Writable": false },
{ "Name": "Setpt", "AddressString": "40010:F", "Writable": true },
{ "Name": "Pump2Run", "Region": "Coils", "Address": 0, "DataType": "Bool",
"Writable": true, "UnitId": 2 }
]
}
```
`Pump2Run` shows a second drop slave (UnitId 2) on the same bus — no extra transport
config, just the per-tag `UnitId` override the driver already honours.
### Example B — RTU-over-TCP to a serial→Ethernet gateway
```json
{
"Transport": "RtuOverTcp",
"Host": "10.20.0.50",
"Port": 4001,
"UnitId": 1,
"TimeoutMs": 1500,
"Tags": [
{ "Name": "Temp", "AddressString": "30001:I" },
{ "Name": "Alarm", "Region": "DiscreteInputs", "Address": 5, "DataType": "Bool" }
]
}
```
Same `Host`/`Port` shape as Modbus TCP, but the wire frames are raw RTU (address +
CRC, no MBAP). This is the difference between a gateway operating in "Modbus TCP"
translation mode (use `Transport: "Tcp"`) versus "transparent/RTU passthrough" mode
(use `Transport: "RtuOverTcp"`).
**Note:** standard Modbus/TCP masters cannot parse RTU-over-TCP frames and vice-versa —
the two are wire-incompatible, so the `Transport` discriminator must match the
gateway's configured mode ([Industrial Monitor Direct](https://industrialmonitordirect.com/blogs/knowledgebase/modbus-tcp-vs-modbus-rtu-over-tcpip-protocol-differences)).
---
## 4. BROWSEABILITY VERDICT — **NO**
**Modbus RTU is not browseable — no address-space browser is warranted.** This is
identical to Modbus TCP. Modbus (any transport) exposes a **flat, untyped register
space** (coils / discrete inputs / input registers / holding registers, addressed
0–65535) with **no discovery protocol** — there is no way to enumerate which registers
exist, what they mean, or their data types. The mapping from register → engineering
meaning lives entirely in the device's vendor documentation, not on the wire.
The existing driver reflects this exactly: `ModbusDriverOptions.Tags` is documented as
"Pre-declared tag map. Modbus has no discovery protocol — the driver returns exactly
these," and `DiscoverAsync` simply materialises the authored tag list into a flat
`Modbus` folder. `RediscoverPolicy` is `Once`. RTU changes none of this. The AdminUI's
`ModbusAddressPickerBody` is an **address *builder*** (grammar helper for composing a
register string), **not** a live browser — and that stays correct for RTU too.
No browser. Authoring stays manual tag entry / address-builder assisted, same as TCP.
---
## 5. Cross-platform serial reality
The server can run on Linux (docker) as well as Windows, so serial-port availability
matters.
- **`System.IO.Ports.SerialPort` is cross-platform** on .NET (5+): it ships the
built-in implementation for **Windows and Linux**, distributed as the
`System.IO.Ports` NuGet package (current `10.0.x` for .NET 10). On Linux it binds
`/dev/tty*` devices via termios.
([NuGet System.IO.Ports](https://www.nuget.org/packages/system.io.ports/),
[MS Q&A](https://learn.microsoft.com/en-us/answers/questions/1444956/is-system-io-ports-currently-only-support-on-windo))
- **macOS is the weak platform.** Serial support on macOS/MacCatalyst is limited/flaky
(baud-rate quirks, `MacCatalyst` unsupported); developers typically fall back to
virtual serial ports for testing.
([Mark's Blog — virtual serial ports on macOS](https://mallibone.com/post/dotnet-on-macos),
[dotnet/runtime #43719](https://github.com/dotnet/runtime/issues/43719)).
This matters only for the **dev machine** (this repo's dev is macOS) — production
targets are Windows/Linux. RTU unit-testing on macOS should use fakes /
RTU-over-TCP, not a real COM port.
- **Containers add a device-mapping hurdle.** A serial device must be explicitly passed
into the container: `docker run --device=/dev/ttyUSB0` (or a compose `devices:`
entry), and USB-serial adapters can re-enumerate (`/dev/ttyUSB0` ↔ `ttyUSB1`) across
reboots/replug, so a stable `udev` symlink or `/dev/serial/by-id/...` path is
advisable. On Windows containers COM passthrough is notoriously unreliable.
([Docker forums — expose host serial port](https://forums.docker.com/t/how-to-expose-host-serial-port-to-container-correctly/81588),
[Portainer device mapping](https://oneuptime.com/blog/post/2026-03-20-map-host-devices-containers-portainer/view))
**Why RTU-over-TCP is the pragmatic primary path for this server.** OtOpcUa is deployed
as a containerised server (docker-dev rig, Linux docker host at `10.100.0.35`) that is
generally **not physically attached to an RS-485 bus.** The idiomatic industrial
topology is a **serial→Ethernet gateway** (Moxa NPort, Digi One, Lantronix, USR-TCP232,
etc.) sitting on the RS-485 multidrop and exposing it over TCP. The server then talks
**RTU-over-TCP** to the gateway — a plain socket, zero host-device mapping, no
`System.IO.Ports` dependency on the container, no udev fragility, and it reuses the
already-hardened socket lifecycle (keepalive / idle-disconnect / reconnect-backoff)
from `ModbusTcpTransport`. Direct `System.IO.Ports` serial should ship too (for
bare-metal Windows/Linux installs with a local COM port or device-mapped adapter), but
**RTU-over-TCP is the path most deployments will actually use**, and it's the lower-risk
one to build and test.
---
## 6. Test-fixture strategy
Three complementary options, in rough order of value for this repo:
1. **RTU-over-TCP against pymodbus (highest ROI, no serial hardware).** The existing
Modbus fixture already runs `pymodbus.simulator` in docker
(`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/`,
binding `:5020`). pymodbus can serve an **RTU-framed TCP** server, so an
`rtu_over_tcp` profile alongside the existing `standard`/`dl205`/`mitsubishi`/
`exception_injection` profiles exercises the real RTU framing + CRC path end-to-end
with **no serial anything** — same harness, same docker host, same
`lmxopcua-fix up modbus <profile>` workflow. This validates the CRC codec and the
function-code-aware framing, which is the only genuinely new logic.
2. **Virtual serial pair on Linux for the direct-serial transport.** `socat -d -d
pty,raw,echo=0 pty,raw,echo=0` creates a linked `/dev/pts/N` ↔ `/dev/pts/M` pair;
point a pymodbus **RTU serial** slave at one end and `ModbusRtuTransport` at the
other. (`com0com` is the Windows equivalent.) This is the only way to cover the real
`System.IO.Ports` open/read/write path without hardware, and it runs in a Linux
container or on the docker host. macOS dev can't easily do this — run it on the
Linux docker host or in CI.
3. **A dedicated RTU slave simulator** — `diagslave` (serial + TCP RTU modes) or
ModbusPal — for manual / soak testing against a virtual pair or a real USB-serial
adapter. Useful for the eventual live-gate but not for unit CI.
**Unit-level:** the CRC-16 helper gets a table-driven unit test against known Modbus CRC
vectors, and `ModbusRtuTransport`/`ModbusRtuOverTcpTransport` can be tested with an
in-memory duplex stream fake (the driver already fakes `IModbusTransport`; here we fake
one level lower, the byte stream, to assert framing + CRC + response parsing). No PLC
needed for the bulk of coverage.
**Recommended CI shape:** unit tests for CRC + RTU framing (fake stream) + an
`rtu_over_tcp` pymodbus docker profile for integration; defer real-serial (socat pair /
hardware) to an env-gated live suite like the other driver live gates.
---
## 7. Effort / risk
**Effort: LOW — likely the lowest-effort item on the driver roadmap.** Because the PDU
layer is already transport-agnostic and injected, the net-new code is small and
localised:
- `ModbusCrc` helper (~30 lines) + unit test.
- `ModbusRtuOverTcpTransport` — can largely reuse `ModbusTcpTransport`'s socket
lifecycle; the delta is CRC framing + FC-aware response length (no TxId). Extracting
the shared socket lifecycle into a base/helper is the main refactor, and it's
mechanical.
- `ModbusRtuTransport` (serial) — `SerialPort` open + the same framing + T3.5 timing.
- Config: `Transport` discriminator + serial fields on `ModbusDriverOptions`, the DTO,
and the factory closure (~1 file each).
- AdminUI: a serial-parameters panel on the existing `ModbusDriverPage.razor`, shown
when `Transport != Tcp`, + the matching config-model round-trip. **Watch the known
enum-serialization trap** (per project memory: driver pages serialize enums
numerically but factory DTOs are string-typed — add `JsonStringEnumConverter` so
`Transport`/`Parity`/`StopBits` round-trip as strings, mirroring OpcUaClient).
- **Zero** changes to codecs, planner, coalescing, health, materialisation, HistoryRead,
or the address parser.
**Risks (all manageable):**
- **RTU response framing without a length field** is the one novel piece of logic —
get the function-code-aware length calculation (and exception-PDU short-frame
detection) right, or fall back to idle-gap timeout. Cover with the fake-stream unit
tests.
- **Serial timing on Linux / in containers** — `System.IO.Ports` on Linux honours
read timeouts but fine-grained T1.5/T3.5 inter-character gating is best-effort; slow
or long RS-485/RF runs may need the `InterFrameDelayMs` override. This is the top
residual risk and the reason to lead with RTU-over-TCP.
- **macOS dev can't exercise real serial** — mitigated by making RTU-over-TCP the
primary tested path and running the socat/serial suite on the Linux docker host / CI,
not the dev Mac.
- **USB-serial device enumeration** in containers (`/dev/ttyUSB*` renumbering) — a
deployment/ops concern, addressed with `--device` + stable `by-id` paths, not a code
risk.
**Bottom line:** small, additive, low-risk. Ship RTU-over-TCP first (reuses the hardened
socket path, no host-device dependency, testable on the existing pymodbus docker
harness), then direct `System.IO.Ports` serial for bare-metal installs.
---
## Key source files (for the implementer)
- `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs` — the seam RTU
plugs into.
- `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusTcpTransport.cs` — the reference
transport; RTU-over-TCP reuses its socket lifecycle.
- `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriver.cs` — transport-agnostic PDU
builders + factory injection point (`transportFactory`).
- `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.Contracts/ModbusDriverOptions.cs` +
`ModbusEquipmentTagParser.cs` — where the transport-mode + serial options are added.
- `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriverFactoryExtensions.cs` — DTO +
transport selection.
- `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Pages/Clusters/Drivers/ModbusDriverPage.razor`
— driver config UI to extend.
- `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/` — pymodbus
fixture to add an `rtu_over_tcp` profile to.
## Sources
- [ModbusKit — RTU vs ASCII vs TCP comparison](https://modbuskit.com/en/blog/modbus-rtu-tcp-ascii-comprehensive-comparison)
- [ModbusSimulator — RTU vs TCP guide](https://modbussimulator.com/blog/modbus-rtu-vs-tcp-comparison-guide)
- [Industrial Monitor Direct — Modbus TCP vs RTU-over-TCP protocol differences](https://industrialmonitordirect.com/blogs/knowledgebase/modbus-tcp-vs-modbus-rtu-over-tcpip-protocol-differences)
- [NModbus (C# Modbus, supports serial RTU/ASCII/TCP/UDP)](https://github.com/NModbus/NModbus)
- [NuGet — System.IO.Ports](https://www.nuget.org/packages/system.io.ports/)
- [Microsoft Q&A — System.IO.Ports platform support](https://learn.microsoft.com/en-us/answers/questions/1444956/is-system-io-ports-currently-only-support-on-windo)
- [Mark's Blog — .NET virtual serial ports on macOS](https://mallibone.com/post/dotnet-on-macos)
- [dotnet/runtime #43719 — macOS SerialPort baud limitation](https://github.com/dotnet/runtime/issues/43719)
- [Docker forums — expose host serial port to container](https://forums.docker.com/t/how-to-expose-host-serial-port-to-container-correctly/81588)
- [Portainer — mapping host serial/USB devices to containers](https://oneuptime.com/blog/post/2026-03-20-map-host-devices-containers-portainer/view)
docs/research/drivers/mqtt-sparkplug.md
+292
View File
@@ -0,0 +1,292 @@
# MQTT / Sparkplug B Driver — Research & Design
**Status:** Research / design proposal (not yet implemented)
**Author:** research sweep, 2026-07-15
**Scope:** A standard **Equipment-kind** driver (same shape as Modbus/S7/AbCip/TwinCAT/FOCAS/OpcUaClient) that ingests **plain MQTT** and **Sparkplug B** data into the OtOpcUa OPC UA / UNS address space.
This is a **subscribe-first** driver: the broker pushes data, we do not poll. Because OtOpcUa is a Unified-Namespace product, Sparkplug B alignment (the de-facto MQTT UNS payload standard) is the headline capability, with plain-MQTT/JSON as the broad-compatibility fallback.
---
## 1. Protocol summary + .NET library options
### 1.1 Plain MQTT
MQTT is a lightweight pub/sub transport: a client connects to a **broker**, subscribes to **topic filters** (with `+` single-level and `#` multi-level wildcards), and receives **retained** and live messages. Payloads are opaque byte arrays — by convention JSON, a scalar string/number, or a binary blob. There is no schema, no data-type metadata, and no built-in discovery: a consumer only knows a topic exists once a message arrives on it. "Last value" is available only if the publisher set the **retain** flag (the broker keeps the last retained message per topic and replays it to new subscribers).
Relevant knobs: QoS 0/1/2, `retain`, Last-Will-and-Testament (LWT), clean-session vs. persistent-session, TLS, username/password or client-cert auth, MQTT 3.1.1 vs 5.0.
### 1.2 Sparkplug B (Eclipse Tahu / Eclipse Sparkplug, spec v3.0)
Sparkplug B is an **open specification layered on MQTT** that adds the missing pieces for industrial data: a **mandated topic namespace**, a **protobuf payload schema**, **stateful session management**, and **auto-discovery via birth certificates**. It is the dominant "UNS over MQTT" standard.
**Topic namespace:** `spBv1.0/{group_id}/{message_type}/{edge_node_id}[/{device_id}]`
- `group_id` — logical grouping (e.g. a site/area).
- `edge_node_id` — an MQTT Edge-of-Network (EoN) node.
- `device_id` — an optional physical device attached to the edge node.
**Message types:**
| Type | Meaning | Direction |
|---|---|---|
| `NBIRTH` | Edge-node **birth certificate** — enumerates every node-level metric with name, alias, datatype, and initial value; carries `bdSeq` | edge → host |
| `DBIRTH` | Device **birth certificate** — enumerates a device's metrics (name/alias/datatype/value) | edge → host |
| `NDATA` / `DDATA` | Node/device **data** — metric changes, usually referenced by **alias** only (no name) | edge → host |
| `NDEATH` / `DDEATH` | Node/device **death** — LWT-driven; marks metrics STALE/uncertain | edge → host |
| `NCMD` / `DCMD` | **Command / write** to a node or device metric (e.g. setpoint, or `Node Control/Rebirth`) | host → edge |
| `STATE` | **Primary-host** application online/offline status (`ONLINE`/`OFFLINE`), retained, LWT-backed | host → edge |
**Key semantics the driver must honour:**
- **Birth-certificate discovery.** DBIRTH/NBIRTH is the *only* place metric **names + datatypes** appear. A consumer must cache the birth to interpret later data. This is exactly what makes Sparkplug **browsable** (§4).
- **Aliases.** After birth, NDATA/DDATA typically send only a numeric `alias` + value to save bandwidth. The consumer must maintain an **alias → (name, datatype)** map per edge-node/device, rebuilt on every (re)birth.
- **Sequence numbers.** Every Sparkplug payload carries a `seq` (0–255, wraps) for gap detection; NBIRTH resets the sequence. `bdSeq` (birth/death sequence) in NBIRTH/NDEATH ties a death to its birth.
- **Rebirth.** If the host sees a gap, an unknown alias, or connects late (missed the birth), it issues an `NCMD` writing boolean `Node Control/Rebirth = true`; the edge node re-publishes NBIRTH + all DBIRTHs. This is the recovery primitive that lets a late-joining consumer recover full metadata.
- **Primary host / STATE.** A "primary host application" publishes a retained `STATE` message (with LWT set to `OFFLINE`, QoS 1, retain=true) so edge nodes know a trusted consumer is online. There **MUST be at most one** primary host client per host-id on a broker. OtOpcUa can consume as a non-primary application, or opt in as primary host (needed for guaranteed store-and-forward semantics on some edge devices).
- **Metric datatypes** (protobuf `DataType` enum): `Int8/16/32/64`, `UInt8/16/32/64`, `Float`, `Double`, `Boolean`, `String`, `DateTime`, `Text`, `UUID`, `Bytes`, `File`, `DataSet`, `Template`, plus array variants (`Int8Array`, …). Each metric = `{name, alias, timestamp, datatype, value, is_historical, is_transient, is_null}`.
Sources: [Eclipse Sparkplug spec (PDF, v2.2 — namespace/STATE rules carry into v3.0)](https://sparkplug.eclipse.org/specification/version/2.2/documents/sparkplug-specification-2.2.pdf) · [Eclipse Sparkplug normative statements](https://github.com/eclipse-sparkplug/sparkplug/blob/master/docs/normative_statements.md) · [Tahu `sparkplug_b.proto`](https://raw.githubusercontent.com/eclipse/tahu/master/sparkplug_b/sparkplug_b.proto) · [HiveMQ: Sparkplug session state](https://www.hivemq.com/blog/understanding-mqtt-topic-namespace-iiot/) · [Steve's Internet Guide: Sparkplug payloads/messages](http://www.steves-internet-guide.com/sparkplug-payloads-and-messages/) · [EMQX Sparkplug docs](https://docs.emqx.com/en/emqx/latest/data-integration/sparkplug.html)
### 1.3 .NET library options
| Library | License | Maturity | .NET 10 | Notes |
|---|---|---|---|---|
| **MQTTnet** | **MIT** | **High.** Now a **.NET Foundation** project hosted under **`dotnet/MQTTnet`**; the standard .NET MQTT client+server. Millions of NuGet downloads, active. v5.x current. | **Yes** — v5 explicitly added `dotnet10` target. | Client + broker, MQTT 3.1.1 & 5.0, TLS, WebSocket. This is the transport layer for **both** modes. |
| **SparkplugNet** | **MIT** | **Moderate.** Single-maintainer (SeppPenner), regular releases; latest **1.3.10 (2024-07-02)**. Supports Sparkplug **v3.0 / spBv1.0** (spAv1.0 obsolete). Provides `Application`, `Node`, `Device` base classes; the **Application** role receives N/DBIRTH + N/DDATA and can publish NCMD/DCMD. Lower adoption than MQTTnet. | **Targets net8.0/net9.0** today (plus "latest/LTS Core & Framework"). **No explicit net10 TFM yet** — will run on .NET 10 via net9.0 compat, but confirm at build time; may need an upstream TFM bump or a fork pin. | Wraps MQTTnet + `protobuf-net` internally. Depends on `MQTTnet >= 4.3.x` — watch for a **version clash** with a directly-referenced MQTTnet 5.x (see risk §6). |
| **MQTTnet + hand-decoded Tahu protobuf** | MIT (+ Tahu, EPL/Apache) | Build-it-yourself | Yes | Reference the Tahu `sparkplug_b.proto`, generate C# with `protobuf-net` or `Google.Protobuf`, decode payloads ourselves on top of an MQTTnet subscription. Maximum control (aliases, seq, rebirth logic exactly as we want), no third-party Sparkplug abstraction, but we own all the state-machine code. |
**Recommendation:** Use **MQTTnet (v5, MIT, .NET Foundation, net10-ready)** as the transport for both modes. For Sparkplug decoding, **start by evaluating SparkplugNet** to save the birth/alias/rebirth plumbing, but treat the **"MQTTnet + Tahu-proto hand-decode"** path as the fallback if SparkplugNet's net10 support or its internal MQTTnet-4.x pin proves awkward. The Sparkplug protobuf schema is small and stable, so hand-decoding is a bounded, well-understood effort.
Sources: [MQTTnet on NuGet (v5.2.0)](https://www.nuget.org/packages/MQTTnet/) · [dotnet/MQTTnet (GitHub)](https://github.com/dotnet/MQTTnet) · [MQTTnet — .NET Foundation](https://old.dotnetfoundation.org/projects/mqttnet) · [SparkplugNet on NuGet](https://www.nuget.org/packages/SparkplugNet) · [SparkplugNet (GitHub, SeppPenner)](https://github.com/SeppPenner/SparkplugNet)
---
## 2. Capability mapping
The driver implements the composable `IDriver` capability interfaces from `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`. Recommended set: `IDriver` + `ITagDiscovery` + `ISubscribable` + `IReadable` + `IHostConnectivityProbe` + `IRediscoverable`, and **optionally** `IWritable` (phase 2). Notably **not** a poller — unlike Modbus/S7 it does not use `PollGroupEngine`; it keeps a live broker connection and raises `OnDataChange` from the MQTT receive callback (closest existing analog: the native-push side of the OpcUaClient driver).
### 2.1 Subscribe (primary — `ISubscribable`)
The core of the driver. On `InitializeAsync`, connect to the broker (MQTTnet), set LWT/clean-session, and subscribe to the configured topic filters:
- **Plain mode:** subscribe to each authored tag's topic (or a shared wildcard) and to nothing else.
- **Sparkplug mode:** subscribe to `spBv1.0/{group_id}/#` (and `STATE/#` if primary host). Maintain per-edge-node/device **alias→metric** tables from N/DBIRTH; on N/DDATA, resolve each metric and raise `OnDataChange` with the authored tag's `FullReference`. On NDEATH/DDEATH, publish STALE/Bad-quality snapshots for that node's metrics. On a missed birth / unknown alias / seq gap, issue a **rebirth NCMD**.
`SubscribeAsync(fullReferences, publishingInterval, …)` maps requested `FullReference`s onto the live receive stream. MQTT/Sparkplug are inherently event-driven, so `publishingInterval` is advisory (used only for optional server-side coalescing/deadband, mirroring Modbus's `ShouldPublish`). The driver holds the broker connection for its whole lifetime; `OnDataChange` fires from the MQTT message handler. Emit an initial value from the retained message / last birth value on subscribe (OPC UA initial-data convention).
### 2.2 Discover (`ITagDiscovery`)
- **Sparkplug mode (rich):** DBIRTH/NBIRTH **enumerates** every metric with name + datatype → map directly onto Equipment + Tags. Natural UNS shape: `group_id` → Area/Line, `edge_node_id`/`device_id` → **Equipment**, each **metric → Tag**. Because births arrive asynchronously after connect, use `RediscoverPolicy = UntilStable` (like FOCAS) — keep discovering until the observed birth set stops growing, and implement `IRediscoverable` so a *new* DBIRTH (new device joins, or a rebirth with changed metrics) triggers an address-space rebuild.
- **Plain mode (passive):** no schema. Two options: (a) **authored-only** — the operator declares tags (topic + JSON path) up front, `DiscoverAsync` returns exactly those (like Modbus's pre-declared `Tags`, `RediscoverPolicy = Once`); or (b) **observe-and-suggest** — subscribe `#` for a bounded window, build the observed topic tree, surface it in the browser (§4) for the operator to pick. Runtime discovery stays authored-only; wildcard observation is a **browse-time** concern, not an auto-provisioning one (avoids unbounded address spaces from chatty brokers).
### 2.3 Read (`IReadable`)
MQTT has no request/response read. `ReadAsync` returns the **last-value cache**: the last message seen on the topic (plain) or the last metric value from data/birth (Sparkplug). Seed it from the broker's **retained** message at connect (plain) and from birth values (Sparkplug). If no value has been observed yet, return `GoodNoData`/uncertain rather than an error. This satisfies OPC UA reads and HistoryRead-at-time without a live round-trip.
### 2.4 Write (`IWritable`) — **verdict: defer to phase 2, not in v1**
Writes are possible but semantically heavier and lower-priority for a UNS-ingest product:
- **Sparkplug:** publish an **NCMD/DCMD** to `spBv1.0/{group}/NCMD/{node}[/{device}]` with the target metric (by name/alias) + new value. Fire-and-forget at the MQTT layer — there is **no synchronous write ack**; success is only observable when the edge node echoes the change back in the next N/DDATA. This mirrors the Galaxy gateway's fire-and-forget write (Galaxy "can never surface a write failure"), so a write returns `Good` optimistically and the value self-corrects on the next data message. Respect `WriteIdempotent` — Sparkplug commands (rebirth, pulse) are frequently non-idempotent.
- **Plain:** publish to a configured command/write topic (often distinct from the read topic) with a formatted payload.
**Recommendation:** ship v1 **read/subscribe/discover only** (a genuinely useful UNS-ingest driver on its own), add `IWritable` (NCMD/DCMD + plain publish) in phase 2. Rationale: the write path has no ack model, needs careful idempotency/rebirth handling, and most UNS-ingest deployments are read-only consumers.
### 2.5 Data-type mapping (`DriverDataType`)
Map Sparkplug metric datatypes / inferred JSON types to `DriverDataType` (`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverDataType.cs`):
| Sparkplug `DataType` | `DriverDataType` | OPC UA |
|---|---|---|
| Int8, Int16 | Int16 | Int16 |
| Int32, UInt16 | Int32 / UInt16 | Int32 / UInt16 |
| Int64, UInt32 | Int64 / UInt32 | Int64 / UInt32 |
| UInt64 | UInt64 | UInt64 |
| Float | Float32 | Float |
| Double | Double64 → **Float64** | Double |
| Boolean | Boolean | Boolean |
| String, Text, UUID | String | String |
| DateTime | DateTime | DateTime |
| Bytes, File | (String/ByteString) | ByteString — phase 2 |
| DataSet, Template | **unsupported v1** | skip / raw-JSON string |
| `*Array` variants | `IsArray=true` + element type | ValueRank=1 |
Plain-MQTT/JSON: infer from the JSON token at the configured JSON path (number→Double/Int64, bool→Boolean, string→String), or honour an explicit per-tag `dataType` override (preferred — inference is brittle). `Int8`/`UInt8` widen to `Int16`/`UInt16` (no OPC UA byte-signed distinction needed). Complex `DataSet`/`Template` metrics are out of scope for v1 (flag + skip, or expose the raw JSON as a String tag).
---
## 3. TagConfig JSON shape
Two layers, mirroring every other driver: **driver options** (broker connection, one per driver instance) parsed by the driver factory, and **per-tag `TagConfig.FullName` + config** authored on the `/uns` Equipment → Tags tab.
### 3.1 Driver options (`MqttDriverOptions`, in `.Contracts`)
```jsonc
{
"host": "10.100.0.35",
"port": 8883,
"clientId": "otopcua-uns-1",
"useTls": true,
"allowUntrustedServerCertificate": false,
"caCertificatePath": null,
"username": "otopcua",
"password": "", // supply via env, never commit
"protocolVersion": "V500", // MQTT 3.1.1 | 5.0
"cleanSession": true,
"keepAliveSeconds": 30,
"reconnectMinBackoffSeconds": 1,
"reconnectMaxBackoffSeconds": 30,
"mode": "SparkplugB", // "Plain" | "SparkplugB"
// ---- Sparkplug-only ----
"sparkplug": {
"groupId": "Plant1", // subscribe spBv1.0/Plant1/#
"hostId": "otopcua-host-1", // STATE/{hostId} identity
"actAsPrimaryHost": false,
"requestRebirthOnGap": true,
"birthObservationWindowSeconds": 15 // discovery: how long to collect births
},
// ---- Plain-only ----
"plain": {
"topicPrefix": "factory/",
"defaultQos": 1
}
}
```
### 3.2 Per-tag config — Sparkplug
`TagConfig.FullName` is the driver-side full reference the router resolves. For Sparkplug the natural key is the fully-qualified metric path; store the full descriptor in the tag config so the driver can resolve by **name or alias** and survive alias reassignment across rebirths.
```jsonc
// TagConfig for one Sparkplug metric tag
{
"groupId": "Plant1",
"edgeNodeId": "Line3EdgeNode",
"deviceId": "Filler1", // omit/null for node-level metrics
"metricName": "Temperature/degC", // stable identity across rebirths
"dataType": "Float", // from DBIRTH; explicit for safety
"isHistorized": false
}
// FullName convention: "Plant1/Line3EdgeNode/Filler1:Temperature/degC"
```
### 3.3 Per-tag config — Plain MQTT
```jsonc
// TagConfig for one plain-MQTT tag
{
"topic": "factory/line3/oven/temp",
"payloadFormat": "Json", // "Json" | "Raw" | "Scalar"
"jsonPath": "$.value", // JSONPath into the payload (Json only)
"dataType": "Double", // explicit — inference is a fallback
"qos": 1,
"retainSeed": true // seed last-value from retained msg
}
// FullName convention: "factory/line3/oven/temp#$.value"
```
Resolution mirrors Modbus's `EquipmentTagRefResolver<TDef>`: the router hands the driver a `FullReference` (the raw `TagConfig` JSON or a stable string), the driver parses+caches it once, and matches inbound messages/metrics against it.
---
## 4. Browseability verdict + browse design
### Verdict: **BROWSEABLE — both modes, but discovery is passive and time-bounded.**
Unlike an OPC UA server (synchronous request/response `Browse`) or a Galaxy repository (queryable DB), MQTT/Sparkplug discovery is **observational**: you learn what exists only by *listening* for a while. The browse session therefore **connects to the broker and observes for a bounded window**, accumulating a tree, then answers `ExpandAsync` from that accumulated snapshot. This is the central design wrinkle.
- **Sparkplug (rich, structured):** subscribe `spBv1.0/{group}/#`, optionally publish a **rebirth NCMD** to force every edge node to re-announce immediately (turns a passive wait into a near-synchronous enumeration), and collect NBIRTH/DBIRTH. Births yield a clean, typed tree: **Group → EdgeNode → Device → Metric**, with datatypes — directly pickable, no guessing.
- **Plain (best-effort):** subscribe `#` (or a configured prefix) for the observation window and build the **observed topic tree**, splitting each topic on `/`. Leaves are topics that have carried a payload; the picker shows the last payload + inferred type as `AttributesAsync`. Coverage is only as good as broker traffic during the window (a topic silent during the window is invisible) — clearly a "suggestions" browse, not an authoritative enumeration.
### `IDriverBrowser` / `IBrowseSession` design
New project **`ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser`** (parallels `.OpcUaClient.Browser` / `.Galaxy.Browser`), implementing `Commons/Browsing/IDriverBrowser` + `IBrowseSession`, registered in `EndpointRouteBuilderExtensions` alongside the others:
```csharp
// EndpointRouteBuilderExtensions.cs (~line 49)
services.AddSingleton<IDriverBrowser, OpcUaClientDriverBrowser>();
services.AddSingleton<IDriverBrowser, GalaxyDriverBrowser>();
services.AddSingleton<IDriverBrowser, MqttDriverBrowser>(); // NEW
```
**`MqttDriverBrowser.OpenAsync(configJson, ct)`** — connect an MQTTnet client from the same `MqttDriverOptions` JSON the runtime driver consumes (separate client-id suffix so it never collides with the running driver's session), subscribe to the discovery filter, and **kick off the observation window immediately**. Returns an `MqttBrowseSession` whose internal tree fills in over the window (and keeps filling as long as the session lives). Because `IBrowseSession` methods are async and the session is registered in the `BrowseSessionRegistry` (TTL-reaped, like the OPC UA one), the "wait for observation" cost is naturally amortised across the user's clicks in the picker.
**`MqttBrowseSession`:**
- `RootAsync` — Sparkplug: the set of observed **groups** (or edge nodes under the configured group). Plain: top-level topic segments. Returns `BrowseNode { Kind = Folder, HasChildrenHint = true }`. If the window hasn't yielded anything yet, return what's accumulated so far (possibly empty) and let the user re-expand — or briefly await the first births. Optionally trigger a rebirth NCMD on first `RootAsync` in Sparkplug mode to populate fast.
- `ExpandAsync(nodeId)` — walk one level down the accumulated tree (EdgeNode→Device→Metric, or next topic segment). Metrics / leaf-topics are `Kind = Leaf`.
- `AttributesAsync(nodeId)` — Sparkplug: return the metric's `AttributeInfo { Name, DriverDataType, IsArray, SecurityClass }` from the cached birth. Plain: return the leaf topic's inferred type + last-seen payload snippet as a single synthetic attribute. (Sparkplug leaves are self-describing, so unlike OPC UA the side-panel is populated.)
- `DisposeAsync` — disconnect the MQTTnet client; best-effort (registry reaper may race a disconnect), same pattern as `OpcUaClientBrowseSession`.
**Picked node → TagConfig:** on commit, the picker projects the selected `BrowseNode`/`AttributeInfo` into the §3 TagConfig JSON. Sparkplug: `NodeId` encodes `group/node/device:metric`, split into the descriptor fields. Plain: `NodeId` is the topic path → `topic` + a default `jsonPath` (`$` or operator-edited) + inferred `dataType`.
**Addressing the passive/time-bounded wrinkle explicitly:**
1. Show a live "listening… N nodes/topics discovered" affordance in the picker so the operator understands coverage is accumulating, not instantaneous.
2. In Sparkplug mode, **actively force a rebirth** to convert the passive wait into a fast, near-complete enumeration.
3. Keep the browse session alive (registry TTL) so re-expanding later reflects newly observed topics without reconnecting.
4. Always allow **manual entry** of a topic/metric (the typed tag editor, §7) as an escape hatch when a tag is silent during the window.
---
## 5. Test-fixture strategy
Follow the existing driver-fixture pattern: a `docker-compose.yml` under `tests/Drivers/.../Docker/` with the `project: lmxopcua` label, deployed to the shared Docker host `10.100.0.35` via `lmxopcua-fix sync/up` (see `CLAUDE.md` Docker Workflow).
- **Broker:** **Eclipse Mosquitto** (`eclipse-mosquitto`, tiny, MIT/EPL) as the default fixture — carries both plain MQTT and Sparkplug B unchanged. Alternative **EMQX** (`emqx/emqx`, Apache-2.0, open-source CE up to 1000 conns) when a management dashboard or built-in Sparkplug tooling is wanted. HiveMQ CE is a third option.
- **Sparkplug edge-node simulator:** options, in rough order of convenience —
- **Bevywise MQTT/IoT Simulator** — purpose-built Sparkplug B realtime data simulation.
- **Eclipse Tahu** reference implementations (Java/Python) — author a small edge-node script that publishes NBIRTH/DBIRTH then periodic NDATA/DDATA; also exercises rebirth (subscribe NCMD `Node Control/Rebirth`). Most faithful to the spec.
- A **project-owned C# simulator** using the same MQTTnet + Tahu-proto path the driver uses (kills two birds: validates our encode/decode symmetrically).
- **Plain-MQTT fixture:** a tiny publisher container (`mosquitto_pub` loop, or a Python `paho-mqtt` script) emitting JSON on a handful of topics with `retain=true` so the last-value/read path and `#`-observation browse are testable.
- **Public test brokers** (for ad-hoc manual smoke only, never CI — unauthenticated, rate-limited, no privacy): `test.mosquitto.org`, `broker.hivemq.com`, `broker.emqx.io`. Do **not** put Sparkplug production-shaped data on public brokers.
- **Live-gate pattern:** an env-gated `*.IntegrationTests` suite (à la the HistorianGateway `LiveIntegration` category) that skips cleanly when `MQTT_FIXTURE_ENDPOINT` is unset, so the suite is macOS-offline-safe.
Sources: [Cedalo: Sparkplug B on Mosquitto](https://www.cedalo.com/blog/mqtt-sparkplug-mosquitto) · [EMQ: MQTT Sparkplug step-by-step](https://www.emqx.com/en/blog/mqtt-sparkplug-in-action-a-step-by-step-tutorial) · [Bevywise Sparkplug simulation](https://www.bevywise.com/blog/sparkplug-b-mqtt-simulation/) · [Ian Craggs: getting started with MQTT + Sparkplug](https://modelbasedtesting.co.uk/2022/01/22/getting-started-with-mqtt-and-sparkplug/)
---
## 6. Effort / risk / phasing
### Recommended order: **plain MQTT first, then Sparkplug B.**
Even though Sparkplug is the headline UNS capability, plain MQTT is the smaller, dependency-light slice that stands up the whole driver skeleton (project layout, `MqttDriverOptions`, factory/probe registration, `ISubscribable` receive loop, last-value `IReadable`, browser project, typed tag editor, fixtures). Sparkplug then layers the birth/alias/rebirth state machine + protobuf decode onto a proven connection+subscribe substrate. This also front-loads the reusable plumbing and de-risks the library decision (you can validate MQTTnet v5/net10 before committing to SparkplugNet).
| Phase | Scope | Rel. effort |
|---|---|---|
| **1 — Plain MQTT** | Projects (`Driver.Mqtt`, `.Contracts`, `.Browser`) + MQTTnet connect/TLS/auth/reconnect; `ISubscribable` (topic subscribe → `OnDataChange`); `IReadable` last-value (retained seed); authored-tag `ITagDiscovery`; `IHostConnectivityProbe`; `#`-observation browser; typed tag editor + validator; Mosquitto + JSON-publisher fixtures. | **M** |
| **2 — Sparkplug B ingest** | `spBv1.0/{group}/#` subscribe; Tahu-proto decode (SparkplugNet **or** hand-rolled); N/DBIRTH → discovery (`UntilStable` + `IRediscoverable`); alias tables; seq-gap detect + **rebirth NCMD**; N/DDEATH → STALE; STATE/primary-host option; birth-driven browser + `AttributesAsync`; Sparkplug datatype map; Tahu/Bevywise simulator fixture. | **L** |
| **3 — Write-through (optional)** | `IWritable`: Sparkplug NCMD/DCMD + plain publish; optimistic-Good + self-correct on echo; `WriteIdempotent` respect; write-topic config. | **M** |
### Top risks
1. **Sparkplug state-machine correctness (aliases + rebirth + seq).** The alias→metric mapping, sequence-gap detection, late-join rebirth, and death→STALE handling are the load-bearing logic and easy to get subtly wrong (e.g. an alias reused across a rebirth with a *different* metric silently mis-routes data to the wrong tag). Mitigation: bind tags by **stable metric name**, treat alias as a per-birth cache only; rebuild the alias table on every (re)birth; test explicitly against rebirth + missed-birth + gap scenarios with a controllable simulator. This is the single biggest correctness risk.
2. **Library / dependency friction (SparkplugNet ↔ MQTTnet version + net10).** SparkplugNet pins **MQTTnet 4.3.x** and has **no explicit net10 TFM** yet; a direct MQTTnet-5 reference (needed for the plain-mode transport + net10 support) can clash with SparkplugNet's transitive 4.x, and central-package-version pinning in this repo is already known to be fragile (see the Roslyn transitive-pinning memory). Mitigation: prototype the version matrix early; be ready to drop SparkplugNet and hand-decode the Tahu proto over a single MQTTnet-5 client (the proto is small/stable), which also removes the net10 TFM concern entirely.
Secondary risks: **passive-browse coverage gaps** (silent topics invisible in the observation window — mitigated by rebirth-forcing + manual entry, §4); **unbounded address space** from wildcard auto-provisioning (mitigated by authored-only runtime discovery); **write has no ack** (accepted, modelled as optimistic-Good self-correcting like Galaxy, §2.4); **broker security** (enforce TLS + real auth, never ship public-broker defaults).
---
## Appendix — project layout (mirrors existing drivers)
```
src/Drivers/
ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ # MqttDriver : IDriver, ITagDiscovery,
# ISubscribable, IReadable,
# IHostConnectivityProbe, IRediscoverable
# (+ IWritable in phase 3)
MqttDriver.cs, MqttDriverProbe.cs, MqttDriverFactoryExtensions.cs
Sparkplug/ (SparkplugDecoder, AliasTable, BirthCache, RebirthRequester)
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ # MqttDriverOptions, MqttTagDefinition,
# MqttEquipmentTagParser
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser/ # MqttDriverBrowser, MqttBrowseSession
src/Server/.../AdminUI/
Uns/TagEditors/TagConfigEditorMap.cs # + ["Mqtt"] = MqttTagConfigEditor
Uns/TagEditors/TagConfigValidator.cs # + Mqtt validation
Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor
EndpointRouteBuilderExtensions.cs # + AddSingleton<IDriverBrowser, MqttDriverBrowser>
tests/Drivers/.../Docker/docker-compose.yml # mosquitto + sparkplug-sim + json-publisher
```
DriverType string: **`"Mqtt"`** (single canonical string across AdminUI page, probe, factory, editor map, browser — heed the Modbus `"ModbusTcp"` vs `"Modbus"` mismatch lesson; pick one and use it everywhere).
docs/research/drivers/mtconnect-agent.md
+408
View File
@@ -0,0 +1,408 @@
# MTConnect (Agent-first) Driver — Research & Design
**Status:** Research / design proposal. No code written yet.
**Author:** research pass, 2026-07-15.
**Scope:** A new standard **Equipment-kind driver** (`DriverType = "MTConnect"`) exposing an
MTConnect Agent's data under the OtOpcUa unified address space — the same shape as
Modbus / S7 / AbCip / TwinCAT / FOCAS / OpcUaClient.
---
## 0. TL;DR
- **Browseable: YES.** MTConnect `/probe` returns a fully self-describing device model
(Device → Components → DataItems) — a natural `IDriverBrowser` tree, closest in spirit to the
OpcUaClient browser.
- **Recommended library: TrakHound `MTConnect.NET` (client packages), MIT-licensed.** It is
mature (v6.9.0.2, Oct 2025, 500k+ NuGet downloads), targets **netstandard2.0** (so it loads on
.NET 10) plus net6/7/8/9, and covers MTConnect versions up to 2.5. Hand-rolling is a viable
fallback but not recommended for v1.
- **Read-only v1** (Discover + Read + Subscribe). No Write. MTConnect *Interfaces* (request/response
write-back) exist but are rare, optional, and out of scope for v1.
- **Top risks:** (1) MTConnect data is *loosely typed* — DataItems are strings with `type`/`units`
metadata, so the OPC UA data-type mapping is a heuristic, not a guarantee; (2) `CONDITION`
category has no clean scalar OPC UA analog and needs a modelling decision (fold to string vs.
native Part 9 alarm).
---
## 1. Protocol summary + .NET library options
### 1.1 The two MTConnect layers
MTConnect ([standard docs](https://docs.mtconnect.org/), [SysML model v2.5](https://model.mtconnect.org/Version2.5/Fundamentals/))
has two wire layers:
| Layer | Transport | Device model? | Role |
|---|---|---|---|
| **Agent** (primary) | HTTP REST, XML (or JSON) | **Yes** — self-describing | The queryable server. What apps talk to. |
| **Adapter / SHDR** | Raw pipe-delimited TCP (default `:7878`) | **No** | Feeds an Agent from a machine; no model, no discovery. |
We target the **Agent** as the primary source. SHDR is a possible phase-2 ingest mode (§7) but
loses auto-discovery.
### 1.2 Agent REST verbs
The Agent exposes three request types (MTConnect Standard Part 1, §5.4 / §8):
- **`/probe`** (optionally `/{device}/probe`) → an **`MTConnectDevices`** response document: the
Device Information Model — every `Device`, its nested `Components`, and each `DataItem`
definition (`id`, `type`, `category`, `subType`, `units`, `nativeUnits`, `name`).
*This is the discovery + browse surface.*
- **`/current`** (optionally `?at=<sequence>`) → an **`MTConnectStreams`** response document: a
snapshot of the latest `Observation` for every DataItem at the moment of the request. *This is the
Read surface.*
- **`/sample?from=<seq>&count=<n>&interval=<ms>`** → an **`MTConnectStreams`** document containing a
*time-ordered set* of observations since sequence `from`. *This is the Subscribe surface.*
**Streaming / long-poll.** When `interval` is supplied, the Agent holds the HTTP connection open and
pushes successive `MTConnectStreams` chunks as a **`multipart/x-mixed-replace`** boundary stream —
each chunk is one batch of new observations. Each response `Header` carries `nextSequence`; a poller
issues the next `/sample` with `from=nextSequence` to get a contiguous, gap-free stream. The Agent's
ring buffer has a finite `bufferSize`; if a consumer falls behind past the buffer, it must re-`/current`
to re-baseline. (Sources:
[MTConnect.NET README](https://github.com/TrakHound/MTConnect.NET/blob/master/README.md),
[cppagent README](https://github.com/mtconnect/cppagent/blob/master/README.md),
[Part 1 Overview](https://docs.mtconnect.org/MTC_Part_1_Overview+V1.2.pdf).)
### 1.3 The information model (probe)
`Device → Component(s) → DataItem(s)`. A `DataItem` has three **categories** (from the
[SysML model](https://model.mtconnect.org/Version2.5/Fundamentals/)):
- **SAMPLE** — continuous numeric measurement over time. Types e.g. `Position`, `Temperature`,
`SpindleSpeed`, `PathFeedrate`, `Load`, `Acceleration`. Carries `units`/`nativeUnits`.
- **EVENT** — discrete state / value change. Types e.g. `Availability` (AVAILABLE/UNAVAILABLE),
`Execution` (READY/ACTIVE/INTERRUPTED/STOPPED), `ControllerMode` (AUTOMATIC/MANUAL/...),
`Program`, `EmergencyStop`, `Block`. Values are typically controlled vocabularies (enums) or free
strings/ints.
- **CONDITION** — health/alarm state. An observation is reported as one of `Normal` / `Warning` /
`Fault` / `Unavailable`, with optional `nativeCode`, `nativeSeverity`, `qualifier`, and message
text. Subtypes: `Actuator`, `Communications`, `System`, `Temperature`, `LogicProgram`, etc.
An observation in an `MTConnectStreams` document references its definition by **`dataItemId`** (and
carries a `sequence`, `timestamp`, and value). This `dataItemId` is the stable per-tag address (§3).
### 1.4 .NET library options
**Recommended: TrakHound `MTConnect.NET` client packages (MIT).**
| Fact | Value | Source |
|---|---|---|
| Latest version | 6.9.0.2 (published 2025-10-16) | [NuGet MTConnect.NET-HTTP](https://www.nuget.org/packages/MTConnect.NET-HTTP/) |
| License | **MIT** (per latest NuGet package metadata) | [NuGet MTConnect.NET-HTTP](https://www.nuget.org/packages/MTConnect.NET-HTTP/) |
| TFMs | netstandard2.0, net6.0–net9.0, net48 (**runs on .NET 10 via netstandard2.0**) | [README](https://github.com/TrakHound/MTConnect.NET/blob/master/README.md) |
| MTConnect versions | up to 2.5; auto-strips data not valid for requested version | [README](https://github.com/TrakHound/MTConnect.NET/blob/master/README.md) |
| Maturity | 500k+ NuGet downloads; actively maintained | [README](https://github.com/TrakHound/MTConnect.NET/blob/master/README.md) |
Packages we would consume (client side only — **not** the Agent/Adapter host packages):
- **`MTConnect.NET-Common`** — model types (`IDevice`, `IComponent`, `IDataItem`, `IObservation`),
the `IMTConnectClient` / `IMTConnectEntityClient` interfaces.
- **`MTConnect.NET-HTTP`** — `MTConnectHttpClient`: probe/current/sample with polling **and**
streaming, gzip compression, XML **and** JSON. Exposes events for received documents
(`OnProbeReceived`, `OnCurrentReceived`, `OnSampleReceived`, plus a per-observation callback) —
a clean fit for pushing into `ISubscribable.OnDataChange`.
- (`MTConnect.NET-XML` / `-JSON` are pulled transitively for serialization.)
> ⚠️ **License due-diligence caveat:** older release notes and the repo copyright header have shown
> mixed "MIT" / "Apache-2.0" / "© TrakHound, All Rights Reserved" strings across versions. The
> *current* NuGet package metadata for 6.9.0.2 declares **MIT**. Before taking the dependency,
> pin a specific version and confirm that exact version's embedded `LICENSE` / package license
> expression (a legal-review checkbox, not a blocker).
**Fallback: hand-roll a minimal XML client.** The Agent protocol is just three HTTP GETs returning
XML. A hand-rolled client (`HttpClient` + `System.Xml.Linq` for probe, a `multipart/x-mixed-replace`
boundary reader for sample) is ~300–500 LoC and removes a third-party dependency + its transitive
`System.Text.Json` version constraints. **Recommendation: use the library for v1** (it handles version
negotiation, the multipart framing, buffer-overflow re-baselining, and JSON/XML dual-format — all
fiddly to reimplement correctly), and keep hand-roll as a contingency if the license review fails.
---
## 2. Capability mapping to the Equipment-kind seams
The driver implements the composable capability interfaces in
`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`. Mapping:
| OtOpcUa seam | MTConnect mechanism | Notes |
|---|---|---|
| `IDriver` (required) | Construct `MTConnectHttpClient` from config, hold `/probe` model | `InitializeAsync` does one probe; `GetHealth` reflects last successful poll + Agent `instanceId` |
| `ITagDiscovery.DiscoverAsync` | Walk the `/probe` DeviceModel, stream `Device→Component→DataItem` into `IAddressSpaceBuilder` | `RediscoverPolicy = Once` (probe is synchronous & complete). Redeploy on Agent `instanceId` change (see `IRediscoverable`, §below) |
| `IReadable.ReadAsync(fullRefs)` | `/current` snapshot, index observations by `dataItemId`, return one `DataValueSnapshot` per requested ref | Reads are idempotent (matches `IReadable` contract). Per-ref miss → `Bad`-coded snapshot, not a throw |
| `ISubscribable.SubscribeAsync` | Start the `MTConnectHttpClient` **sample stream** (`from`/`interval`); fan each received observation to `OnDataChange` keyed by `dataItemId` | One shared stream per driver instance (not per tag) — the Agent streams the whole device; filter to subscribed refs. Fire initial values from `/current` on subscribe (OPC UA convention) |
| `IWritable` | **NOT implemented (v1)** | See verdict below |
| `IRediscoverable.OnRediscoveryNeeded` | Raise when the Agent's `Header/@instanceId` or `@assetBufferSize` changes (Agent restarted / model changed) → Host rebuilds address space | Mirrors Galaxy's `DeployWatcher` pattern |
| `IHostConnectivityProbe` | Cheap periodic `/probe` HEAD or `/current` to flip Running↔Stopped | Mirrors `ModbusProbeOptions` |
| `IAlarmSource` | *(optional, phase 1.5)* map `CONDITION` Fault/Warning → Part 9 native alarm | See §2.3 |
### 2.1 Write verdict: **read-only for v1**
Justification:
1. The mainstream MTConnect surface (`/probe`, `/current`, `/sample`) is **strictly read-only** by
design — MTConnect was conceived as a read-only telemetry standard.
2. Write-back exists only via **MTConnect *Interfaces*** (a request/response handshake modelled as
its own DataItem categories — `Request`/`Response` events) which is **optional, rarely deployed**,
and semantically a state-machine handshake, not a simple "set value." Modelling it as OPC UA
`IWritable` would be misleading.
3. The Equipment-kind seams are composable — a driver implements only what its backend supports
(`IDriver` doc-comment). Omitting `IWritable` is idiomatic (Galaxy's write path is even
fire-and-forget; several drivers are read-mostly). Nodes materialize without the
`AccessLevels.CurrentWrite` bit.
Revisit Interfaces write-back in a later phase only if a concrete deployment needs it.
### 2.2 Data-type mapping (MTConnect → `DriverDataType` → OPC UA)
MTConnect DataItems are **weakly typed on the wire** (values arrive as strings). The mapping is
inferred from `category` + `type` + `units`, and stored per-tag in the TagConfig (§3) so a wrong
inference can be corrected by the author without a code change.
| MTConnect | Inferred `DriverDataType` | Rationale |
|---|---|---|
| `SAMPLE` (numeric, has `units`) | `Float64` | Continuous analog; `Double` is the safe superset |
| `SAMPLE` w/ `representation="TIME_SERIES"` | `Float64` array (ValueRank=1) | Time-series sample bursts |
| `EVENT` controlled-vocab (Execution, ControllerMode, Availability, …) | `String` | Enum-like; keep the vocab string. (Optional: OPC UA enum modelling later) |
| `EVENT` numeric (e.g. `PartCount`, `Line`) | `Int64` | Integer counters |
| `EVENT` free text (`Program`, `Block`, `Message`) | `String` | |
| `CONDITION` | `String` (v1: the state word `Normal`/`Warning`/`Fault`/`Unavailable`) **or** native alarm (§2.3) | No clean scalar analog |
| `timestamp` on every observation | → OPC UA `SourceTimestamp` | Not a separate node |
`DataValueSnapshot.StatusCode` mapping: an observation value of `UNAVAILABLE` (MTConnect's explicit
"no data" sentinel) → OPC UA `Bad`/`Uncertain` quality rather than a literal string, so downstream
sees proper quality. Empty CONDITION or missing dataItem → `Bad`.
### 2.3 CONDITION modelling (decision point)
`CONDITION` is the awkward one. Two options:
- **v1 simple:** expose each CONDITION DataItem as a `String` variable node whose value is the current
condition state (`Fault`/`Warning`/`Normal`/`Unavailable`), optionally suffixed with `nativeCode`.
Zero alarm plumbing.
- **v1.5 native:** treat a Fault/Warning CONDITION as an OtOpcUa **native alarm** by emitting a
TagConfig `alarm` object and implementing `IAlarmSource` — matching the Galaxy/Phase-B native-alarm
pattern (route on the authored dotted reference, per the alarms memory). Higher value, more work.
Recommend **v1 simple, v1.5 native** as a fast-follow.
---
## 3. TagConfig JSON shape
Consistent with the Equipment-kind model: **connection-level settings live in the driver config**
(agent base URL, auth, poll interval); **per-tag addressing lives in `TagConfig`**, and the platform
`FullName` binds the tag to a driver-side reference. For MTConnect the natural stable reference is the
**`dataItemId`** (globally unique within an Agent), so `FullName = "<dataItemId>"` (or, for
readability, the driver can also accept a `device/component/dataItemName` path and resolve it to the id
at discovery time).
### 3.1 Driver config (per driver instance) — `MTConnectDriverOptions`
```jsonc
{
"AgentUri": "https://demo.mtconnect.org", // Agent base URL (http or https)
"DeviceName": "", // optional: scope to one device (else all devices)
"PreferJson": false, // request application/json instead of XML if Agent supports it
"SampleIntervalMs": 1000, // long-poll interval passed to /sample
"SampleCount": 500, // max observations per sample chunk
"HeartbeatMs": 10000, // Agent keep-alive heartbeat
"RequestTimeoutMs": 30000,
"AllowUntrustedTls": false, // dev/on-prem self-signed agents
"Probe": { "Enabled": true, "IntervalMs": 5000 } // IHostConnectivityProbe
}
```
### 3.2 Per-tag `TagConfig` (authored on the /uns Tags tab)
```jsonc
{
"FullName": "avail", // <-- MTConnect dataItemId; the driver-side reference
"mtDevice": "OKUMA.Lathe", // optional human context (device name from probe)
"mtComponent": "Controller", // optional
"mtCategory": "EVENT", // SAMPLE | EVENT | CONDITION (from probe; drives typing)
"mtType": "AVAILABILITY", // MTConnect DataItem type
"mtSubType": null,
"dataType": "String", // resolved DriverDataType (overridable by author)
"units": null, // e.g. "CELSIUS" for a SAMPLE
"writable": false // always false for MTConnect v1
}
```
A picked SAMPLE example:
```jsonc
{
"FullName": "Xact",
"mtDevice": "OKUMA.Lathe",
"mtComponent": "Linear[X]",
"mtCategory": "SAMPLE",
"mtType": "POSITION",
"mtSubType": "ACTUAL",
"dataType": "Float64",
"units": "MILLIMETER",
"writable": false
}
```
The runtime driver resolves reads/subscriptions by `FullName` (= `dataItemId`) against the observation
index it maintains from `/current` and `/sample` — exactly the "bind by `TagConfig.FullName`" pattern
the other drivers use (`EquipmentTagRefResolver<TDef>`). Only `FullName` is load-bearing at runtime;
the `mt*`/`units` fields are author-facing metadata + type inference inputs preserved across edits.
---
## 4. Browseability verdict + browse design
### 4.1 Verdict: **YES — fully browseable.**
`/probe` returns the complete `MTConnectDevices` model in one call — a static, self-describing tree.
This is the *cleanest possible* browse source (better than Galaxy, which needs per-node expansion;
comparable to a single-shot OPC UA browse). It maps directly onto `IDriverBrowser`/`IBrowseSession`.
### 4.2 New project: `ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser`
Mirror `OpcUaClient.Browser` layout exactly:
```
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser/
MTConnectDriverBrowser.cs // IDriverBrowser: DriverType="MTConnect"; OpenAsync(configJson) -> session
MTConnectBrowseSession.cs // IBrowseSession over the fetched probe model
ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser.csproj
```
Register in AdminUI DI alongside the others
(`src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs`, ~line 49-50):
```csharp
services.AddSingleton<IDriverBrowser, MTConnectDriverBrowser>();
```
### 4.3 Session behaviour (map to `IBrowseSession`)
Unlike OPC UA (live one-level-per-call), MTConnect probe is fetched **once** in
`OpenAsync` and cached in the session; expansion is served from the in-memory tree (no further network
calls). This is simpler and faster than the OpcUaClient browser.
| Method | Returns |
|---|---|
| `OpenAsync(configJson)` | Deserialize `MTConnectDriverOptions`, `GET {AgentUri}/probe`, hold the parsed `IDevice[]` model in the session |
| `RootAsync()` | One `BrowseNode(Kind=Folder)` per **Device** (or, if `DeviceName` is scoped, the top-level **Components** of that device) |
| `ExpandAsync(nodeId)` | Children of a Device/Component: nested **Components** → `Folder` nodes; **DataItems** → `Leaf` nodes. `NodeId` encodes the path (e.g. `dev:OKUMA.Lathe`, `comp:<componentId>`, `di:<dataItemId>`) |
| `AttributesAsync(nodeId)` | For a DataItem leaf, one-or-few `AttributeInfo` rows describing it (`Name=dataItemName`, `DriverDataType=<inferred>`, `IsArray=<TIME_SERIES?>`, `SecurityClass`, `IsAlarm = (category==CONDITION)`). For Device/Component folders, empty. This drives the picker's side-panel and lets a CONDITION pre-fill a native-alarm TagConfig (like Galaxy alarm attrs) |
- `BrowseNode.NodeId` for a DataItem leaf is `di:<dataItemId>` → the picker commits it, and the
TagModal maps it to `TagConfig.FullName = <dataItemId>` plus the `mt*` metadata read from the cached
probe model.
- `BrowseNodeKind.Leaf` = DataItem (terminal, commit-on-select); `Folder` = Device/Component.
- `HasChildrenHint = true` for Devices/Components that have child components or DataItems.
### 4.4 Picked-DataItem → TagConfig
When the picker commits a `di:<dataItemId>` leaf, the AdminUI builds the TagConfig in §3.2 from the
cached probe DataItem: `FullName=id`, `mtCategory/mtType/mtSubType/units` copied verbatim, and
`dataType` set from the §2.2 inference table (author can override in the typed editor).
---
## 5. Typed tag-editor (AdminUI)
To avoid the raw-JSON fallback, add a driver-typed editor (mirrors the driver-typed tag-editor pattern):
- `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs`
(pure `FromJson`/`ToJson`/`Validate`, preserves unknown keys) — copy the Modbus template.
- `.../Components/Shared/Uns/TagEditors/MTConnectTagConfigEditor.razor` — a thin shell:
fields for `FullName` (dataItemId), read-only `mtCategory/mtType`, and a `dataType` override dropdown.
- Register `["MTConnect"] = typeof(...MTConnectTagConfigEditor)` in
`Uns/TagEditors/TagConfigEditorMap.cs`, and add validation in `TagConfigValidator`.
Because most tags are authored via the **browse picker** (which fills the model), the editor is mostly
a confirm/override surface.
---
## 6. Test-fixture strategy
Ranked by value:
1. **Public demo Agents (fastest smoke path).**
- `https://demo.mtconnect.org/` — the MTConnect Institute reference demo (probe/current/sample).
- `http://mtconnect.mazakcorp.com/` — Mazak demo agents.
- NIST **smstestbed** publishes real agent configs + `Devices.xml`
([usnistgov/smstestbed](https://github.com/usnistgov/smstestbed/blob/master/mtconnect/agent/Devices.xml)).
These give a live, real-shape model for browse + read + streaming verification with zero infra.
(Caveat: public agents are internet-dependent and not reproducible in CI.)
2. **Dockerized reference C++ Agent (reproducible integration fixture).**
The MTConnect reference Agent ([mtconnect/cppagent](https://github.com/mtconnect/cppagent)) has an
official Docker image. Add a fixture under `tests/.../Docker/docker-compose.yml` with the
`project: lmxopcua` label (per the repo's Docker workflow), seeded with a canned `Devices.xml` and
an SHDR simulator or the built-in agent adapter, exposed on the shared docker host `10.100.0.35`.
This is the integration-test analog of the Modbus/S7 sims.
3. **Canned probe/current/sample XML fixtures (unit tests).**
Capture one `MTConnectDevices` (probe) and a couple of `MTConnectStreams` (current + sample) XML
documents from a demo agent into `tests/.../MTConnect.Tests/Fixtures/`. Drive the parser,
type-inference, browse-tree, and observation-indexing logic with **no network** — the bulk of unit
coverage. This is where the tricky bits (type inference, `nextSequence` paging, `UNAVAILABLE`→Bad,
CONDITION handling, multipart chunk framing) get pinned.
Recommended: **#3 for unit CI + #2 for the env-gated integration suite + #1 for manual live smoke.**
---
## 7. Effort / risk / phasing
### Phase 1 — Agent read + subscribe + browse (the MVP)
- Projects: `Driver.MTConnect` (+ `.Contracts` for `MTConnectDriverOptions`/DTOs),
`Driver.MTConnect.Browser`, `MTConnect.Tests`.
- Implement `IDriver` + `ITagDiscovery` (probe→builder) + `IReadable` (current) +
`ISubscribable` (sample stream) + `IHostConnectivityProbe` + `IRediscoverable` (instanceId change).
- `IDriverProbe` for the AdminUI Test-Connect button (probe reachability).
- `IDriverBrowser`/`IBrowseSession` from cached probe model; register in AdminUI DI.
- Typed tag editor + `TagConfigEditorMap`/`TagConfigValidator` entries.
- Register the factory in the Host bootstrapper (mirror `ModbusDriverFactoryExtensions.Register`).
- CONDITION as `String` (simple).
- **Effort estimate:** ~1–1.5 weeks with the TrakHound library (it removes the protocol grind).
~2.5–3 weeks hand-rolled.
### Phase 1.5 (fast-follow)
- CONDITION → native OPC UA Part 9 alarms via `IAlarmSource` (Galaxy native-alarm pattern).
- `TIME_SERIES` SAMPLE arrays; EVENT controlled-vocab → OPC UA enumerations.
### Phase 2 — SHDR adapter ingest (second source mode)
- Add an optional `SourceMode: "Agent" | "Shdr"` to the driver config. In SHDR mode the driver opens
the raw pipe-delimited TCP socket (default `:7878`) and parses `<timestamp>|<key>|<value>` lines.
- **Loses auto-discovery**: SHDR has no device model, so `ITagDiscovery` cannot self-populate — tags
must be **manually authored** (like Modbus), and the browser is unavailable in SHDR mode. Type
inference falls back to author-supplied `dataType`.
- Value: connect directly to a machine adapter with no Agent deployed. Niche; do only on demand.
### Risk register
| Risk | Sev | Mitigation |
|---|---|---|
| Weak wire typing → wrong OPC UA type inference | Med | Store resolved `dataType` in TagConfig; author-overridable; unit-test the inference table against real probe XML |
| CONDITION has no scalar analog | Med | v1 = String state; v1.5 = native alarm |
| TrakHound license string inconsistent across versions | Low-Med | Pin a version; confirm its embedded MIT license in legal review before merge |
| Public demo agents unreliable for CI | Low | Docker cppagent fixture + canned XML for CI; demo agents only for manual smoke |
| Agent ring-buffer overflow if consumer stalls | Low | Library re-baselines via `/current` on sequence gap; verify + test the fall-behind path |
| `netstandard2.0`-only lib on .NET 10 | Low | netstandard2.0 loads on net10.0; validate no trim/AOT issues at build |
---
## 8. Source references
- MTConnect Standard Part 1 (Overview & Protocol): https://docs.mtconnect.org/MTC_Part_1_Overview+V1.2.pdf
- MTConnect SysML Model v2.5 (Fundamentals — Device/Component/DataItem, categories): https://model.mtconnect.org/Version2.5/Fundamentals/
- TrakHound MTConnect.NET (README, client API, TFMs, versions): https://github.com/TrakHound/MTConnect.NET/blob/master/README.md
- NuGet MTConnect.NET-HTTP (license=MIT, v6.9.0.2 2025-10-16, TFMs, deps): https://www.nuget.org/packages/MTConnect.NET-HTTP/
- MTConnect reference C++ Agent (Docker fixture): https://github.com/mtconnect/cppagent/blob/master/README.md
- NIST smstestbed agent config + Devices.xml (real fixtures): https://github.com/usnistgov/smstestbed/blob/master/mtconnect/agent/Devices.xml
- Public demo agent: https://demo.mtconnect.org/ · Mazak demo agents: http://mtconnect.mazakcorp.com/
### OtOpcUa code seams referenced
- Capability interfaces: `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/` (`IDriver`, `ITagDiscovery`, `IReadable`, `ISubscribable`, `IWritable`, `IRediscoverable`, `IHostConnectivityProbe`, `IDriverFactory`, `DriverDataType`).
- Browse seam: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/` (`IDriverBrowser`, `IBrowseSession`, `BrowseNode`, `AttributeInfo`).
- Browser template: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser/`.
- Driver template: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/` (+ `.Contracts`), factory `ModbusDriverFactoryExtensions`.
- AdminUI browser DI registration: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs:49-50`.
- Typed tag editors: `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs` + `TagConfigValidator.cs`.
- Browse design doc: `docs/plans/2026-05-28-driver-browsers-design.md`.
docs/research/drivers/omron.md
+361
View File
@@ -0,0 +1,361 @@
# Omron PLC Driver — Research & Implementation Design
**Status:** Research / design proposal (no code yet).
**Author context:** Standard **Equipment-kind driver** for the OtOpcUa server, same shape as
Modbus / S7 / AbCip / TwinCAT / FOCAS — an in-process `IDriver` implementing the composable
capability interfaces (`IReadable` / `IWritable` / `ISubscribable` / `ITagDiscovery` /
`IHostConnectivityProbe`, optionally `IDriverBrowser` in AdminUI). Points are ordinary equipment
`Tag`s bound to the driver via `TagConfig.FullName`, authored on the `/uns` Tags tab.
---
## 0. TL;DR recommendation
- **Support both transports, CIP-first.** Ship an **Omron CIP (EtherNet/IP)** path for modern
**NJ/NX/NY (Sysmac)** controllers as Phase 1, then an **Omron FINS** path for legacy
**CJ/CS/CP/CV** as Phase 2. They cover disjoint hardware generations; neither subsumes the other.
- **Library:** `libplctag` (via `libplctag.NET`, MPL-2.0) for the CIP path — **reuses almost the
entire AbCip wire layer** (`plc=omron-njnx`). For the FINS path, **hand-roll** the framing
(it is a small, well-documented binary protocol) rather than take `HslCommunication` (see §1.3).
- **Browseability:** **CIP NJ/NX is only *conditionally* browsable, and NOT via libplctag's
`@tags` walker** — that is the single most important finding. **FINS is never browsable** (flat
memory banks, no symbol table). Details in §4.
---
## 1. Transport decision
Omron exposes two entirely different Ethernet protocols depending on controller generation.
### 1.1 The two transports and what they cover
| Transport | Port / framing | Controller families | Addressing model | Browsable? |
|---|---|---|---|---|
| **EtherNet/IP CIP** | TCP/UDP **44818** (EtherNet/IP), CIP explicit messaging | **NJ / NX / NY (Sysmac)**, and CJ2/CP-family with a CIP-capable EtherNet/IP unit | **Named variables** (symbolic tags), like Rockwell Logix | **Conditionally** — see §4 |
| **FINS** | **FINS/TCP** and **FINS/UDP** on port **9600** | **CJ / CS / CP / CV**, NSJ; NJ/NX also support *FINS-over-EtherNet/IP* for back-compat | **Memory areas** (CIO/WR/HR/AR/DM/EM banks) + word/bit offset | **Never** (flat memory, no symbol table) |
Key architectural fact: **NJ/NX are CIP-native**. Their "variables" are named tags reached by CIP
explicit messaging using a symbolic-segment request path — the same mechanism ControlLogix uses,
which is why AbCip's libplctag layer already speaks it (`plc=omron-njnx`). Sources:
[Omron store KB — Using CIP to Read/Write Network Variables with NJ/NX](https://store.omron.com.au/knowledge-base/1807412-using-cip-to-read-write-network-variables-with-nj-nx),
[NJ/NX CPU Built-in EtherNet/IP Port User's Manual (W506)](https://files.omron.eu/downloads/latest/manual/en/w506_nj_nx-series_cpu_unit_built-in_ethernet_ip_port_users_manual_en.pdf).
**FINS** is the classic protocol: a compact binary header (ICF/RSV/GCT, destination & source
net/node/unit addresses, SID) followed by a 2-byte command code and its parameters. **Memory Area
Read = `0x0101`**, **Memory Area Write = `0x0102`**. FINS/TCP prepends a 16-byte "FINS" TCP header
(the ASCII magic `46 49 4E 53` = "FINS", a length, and a command/error field) and requires a small
handshake frame to obtain a node address before the first command. Sources:
[Omron W227 FINS Commands Reference Manual](https://www.myomron.com/downloads/1.Manuals/Networks/W227E12_FINS_Commands_Reference_Manual.pdf),
[Wireshark FINS dissector](https://github.com/wireshark/wireshark/blob/master/epan/dissectors/packet-omron-fins.c),
[FieldServer Omron FINS driver sheet](https://cdn.chipkin.com/assets/uploads/2016/jan/PDS_Omron_FINS.pdf).
### 1.2 FINS memory-area codes (for the TagConfig addressing model)
FINS memory-area codes are **access-width-specific** (a different code for word vs. bit access of
the same bank). Representative word-access codes:
| Bank | Word code | Bit code | Notes |
|---|---|---|---|
| CIO (Core I/O) | `0xB0` | `0x30` | word + bit addressable |
| WR (Work) | `0xB1` | `0x31` | word + bit addressable |
| HR (Holding) | `0xB2` | `0x32` | word + bit addressable |
| AR (Auxiliary) | `0xB3` | `0x33` | word + bit addressable |
| DM (Data Memory) | `0x82` | `0x02` | primary bulk data bank; word addressable |
| EM (Extended, bank 0) | `0xA0` | `0x20` | banked; code varies per EM bank (`0xA0`+bank) |
(Exact codes vary slightly by CPU family — the authoritative table is per-controller in W227 /
the CS/CJ comms manual. The driver must treat the code table as CPU-family-parameterised.) Source:
[Omron memory-area discussion](https://community.oxmaint.com/discussion-forum/omron-plc-memory-areas-understanding-ci-hr-ar-lr-dm-wr-em-extended-em-usage),
[W227 reference](https://www.myomron.com/downloads/1.Manuals/Networks/W227E12_FINS_Commands_Reference_Manual.pdf).
### 1.3 .NET library options — license analysis
| Library | Transports | License | Maturity / .NET 10 | Verdict |
|---|---|---|---|---|
| **`libplctag.NET`** (wraps native `libplctag`) | **CIP** (`plc=omron-njnx`); **also Modbus** | **native core dual-licensed MPL-2.0 / LGPL-2.1; .NET wrapper MPL-2.0** — permissive, commercial-OK, already vetted & shipped in this repo for AbCip | Mature, actively maintained; **already a proven dependency** (AbCip runs on it, net10.0) | **CIP path: adopt.** Zero new license review — same dep AbCip already carries. |
| **`HslCommunication`** | FINS **and** CIP (`OmronFinsNet`, `OmronCipNet`) | **The maintained/commercial builds are NOT free** — the original author moved HSL to a **paid commercial license**; only a frozen old community fork ([`HslCommunication-Community`](https://github.com/HslCommunication-Community/HslCommunication-Community)) remains under the last free (MIT-era) snapshot. The "MIT" claim in casual sources refers to that **stale fork**, not current releases. | Broad but the free fork is unmaintained | **Do NOT take a dependency.** License risk + the free build is abandoned. Use only as a *protocol reference* to cross-check hand-rolled framing. |
| **Hand-rolled FINS** | FINS/TCP + FINS/UDP | our own code, no third-party license | n/a — FINS framing is ~200 lines | **FINS path: hand-roll.** Framing is simple and fully documented (W227 + Wireshark dissector); mirrors how `ModbusTcpTransport` is hand-rolled in this repo. |
| `Sres.Net.EEIP` | EtherNet/IP (generic CIP) | permissive | Focused on generic CIP/assembly, not Omron symbolic variable access; weaker than libplctag for named-tag read/write | Not needed — libplctag already covers CIP. |
| `Plc.Omron.Standard` (NuGet) | FINS | permissive but tiny/low-adoption | Immature | Reference only. |
**Recommendation:** **libplctag.NET for CIP** (reuse AbCip's proven stack), **hand-rolled FINS
transport** for the memory-area path. This keeps the dependency surface identical to what the repo
already ships and dodges the HslCommunication license trap.
---
## 2. Capability mapping
The driver implements the same capability set as AbCip/Modbus. Both transports map cleanly onto the
composable interfaces in `Core.Abstractions`.
### 2.1 CIP path (NJ/NX) — reuses AbCip machinery
- **Connect / `InitializeAsync`:** libplctag Forward-Open per device. Canonical host address form
mirroring AbCip's `ab://gateway[:port]/cip-path`, e.g. `omron://gateway/path` — but for NJ/NX the
libplctag attribute string is `protocol=ab-eip&gateway=<ip>&path=<cip-path>&plc=omron-njnx&name=<tag>`.
(Community-confirmed: NJ/NX needs `plc=omron-njnx` and a CIP-bridging `path`, e.g. `18,<ip>`.)
Source: [libplctag Omron NJ/NX group thread](https://groups.google.com/g/libplctag/c/bU5MVQrPsOw).
- **Read (`IReadable`):** direct reuse of AbCip's per-tag runtime pattern — create tag handle, `ReadAsync`,
`GetStatus`, decode. Batch-read + per-host resilience (`IPerCallHostResolver`) reuse verbatim.
- **Write (`IWritable`):** full read/write. NJ/NX BOOL-in-word and array handling map onto AbCip's
existing `EncodeValue` / bit-RMW code paths.
- **Subscribe (`ISubscribable`):** **poll-based** via the shared `PollGroupEngine` (identical to
AbCip/Modbus — Omron CIP has no native subscription/unsolicited push for explicit messaging).
- **Discover (`ITagDiscovery`):** emit pre-declared tags always; controller-side browse is the
**caveat** — see §4 (libplctag's `@tags` walker does **not** work on Omron).
- **Health/Probe (`IHostConnectivityProbe`):** reuse AbCip's probe-loop pattern (cheap tag read at
interval → `HostState` transitions).
**Reuse estimate: ~70–80% of AbCip's driver body is directly reusable for the CIP path.** The
honest way to capture this is to **extract a shared CIP core** (tag runtime, poll wiring, status
mapping, host-address parsing, resilience seam) that both AbCip and OmronCip consume — rather than
copy-paste. The Omron-specific deltas are: the `plc=omron-njnx` attribute string, Omron's data-type
code set (see §2.3), string encoding differences, and the browse story.
### 2.2 FINS path (CJ/CS/CP) — new transport, Modbus-shaped
- **Connect:** open TCP to `:9600`, send the FINS/TCP node-address-request handshake, cache the
assigned client node number. (FINS/UDP variant skips the handshake.)
- **Read:** build a `0x0101` Memory Area Read frame (area code + 3-byte address + word count),
send, parse the 2-byte end-code + payload. Word-oriented; multi-word reads for 32/64-bit types.
- **Write:** `0x0102` Memory Area Write, symmetric. Bit writes use the bit-access area codes; word
writes use the word codes. RMW not required for FINS bit writes (bit codes address bits directly).
- **Subscribe:** poll-based via `PollGroupEngine` (FINS has no subscription).
- **Discover:** **pre-declared tags only** — there is no symbol table to enumerate (§4).
- **Probe:** cheap DM-word read at interval.
The FINS transport is structurally a sibling of `ModbusTcpTransport` (`IModbusTransport`): a thin
framed request/response codec behind an `IFinsTransport` seam, unit-testable against a byte-level
fake.
### 2.3 Data-type mapping
Omron CIP (NJ/NX) uses IEC 61131 variable types that align closely with the existing
`AbCipDataType` / `DriverDataType` set; FINS is raw words that the TagConfig must type explicitly.
| Omron type | CIP (NJ/NX) | FINS (word interpretation) | OPC UA / `DriverDataType` |
|---|---|---|---|
| BOOL | native BOOL | bit-area read, or bit N of a word | Boolean |
| BYTE/USINT/SINT | 1-byte | low byte of word | Byte / SByte |
| WORD/UINT/INT | 2-byte | 1 word | UInt16 / Int16 |
| DWORD/UDINT/DINT | 4-byte | 2 words (endianness matters) | UInt32 / Int32 |
| LWORD/ULINT/LINT | 8-byte | 4 words | UInt64 / Int64 |
| REAL | IEEE-754 32 | 2 words | Float |
| LREAL | IEEE-754 64 | 4 words | Double |
| STRING | CIP string | word-packed ASCII, fixed length | String |
| DATE_AND_TIME/TIME | Sysmac time types | banked words | DateTime (best-effort) |
**Word/byte-order caveat (FINS):** Omron stores multi-word values with a specific word order that
differs from naive concatenation; the FINS codec must expose a per-tag or per-device
byte/word-swap option (same class of concern the S7/Modbus drivers already handle). CIP via
libplctag handles endianness internally.
---
## 3. TagConfig JSON shape
`TagConfig.FullName` (or a JSON blob under it) carries the driver-specific addressing. Following the
AbCip equipment-tag parser convention (`AbCipEquipmentTagParser`: a leading `{` marks a TagConfig
blob; camelCase property names; strict enum reads via `TagConfigJson.TryReadEnumStrict`), the two
transports need distinct shapes discriminated by a `transport` field.
### 3.1 CIP (NJ/NX) — named-variable addressing
```json
{
"transport": "Cip",
"deviceHostAddress": "omron://10.100.0.40/1,0",
"tagName": "Conveyor.Speed",
"dataType": "Real",
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
(`tagName` is the Sysmac global-variable name; near-identical to AbCip's `tagPath`. The CIP path
reuses AbCip's `isArray`/`arrayLength`/`writable`/`dataType` semantics verbatim.)
### 3.2 FINS — memory-area addressing
```json
{
"transport": "Fins",
"deviceHostAddress": "fins://10.100.0.41:9600/0.1.0",
"memoryArea": "DM",
"address": 100,
"bit": null,
"dataType": "Int16",
"wordSwap": false,
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
- `memoryArea` ∈ `{CIO, WR, HR, AR, DM, EM}` (+ optional `emBank` for extended memory); the driver
maps the pair `(memoryArea, bit==null?word:bit)` to the correct FINS area code (§1.2).
- `address` = word offset; `bit` = 0–15 for bit access (null ⇒ word access).
- `deviceHostAddress` encodes the FINS destination `net.node.unit` triple.
- `wordSwap` handles Omron multi-word ordering for 32/64-bit types.
---
## 4. BROWSEABILITY VERDICT (transport-dependent) — the critical finding
### 4.1 FINS — **NOT browsable. Full stop.**
FINS addresses flat memory banks with no symbol/metadata table on the wire. There is nothing to
enumerate. The AdminUI must offer only manual entry (memory-area dropdown + word/bit offset). No
`IDriverBrowser` for the FINS transport.
### 4.2 CIP (NJ/NX) — **browsable in principle, but NOT via libplctag's `@tags` walker.**
This is the finding to flag loudly and to **coordinate with the AbCip-browseability audit**:
- The AbCip browse mechanism relies on libplctag's `@tags` pseudo-tag (CIP Symbol Object class
`0x6B`) to walk the controller symbol table. **libplctag's tag-listing works only on
ControlLogix / CompactLogix — it does NOT work on Omron NJ/NX**, returning `ErrorUnsupported` /
`PLCTAG_ERR_NOT_FOUND`. Confirmed upstream:
[libplctag #466 "Support for Omron NX/NJ variable listing"](https://github.com/libplctag/libplctag/issues/466),
[libplctag.NET #371 "List all tags in Omron NJ: ErrorUnsupported"](https://github.com/libplctag/libplctag.NET/issues/371).
- So the naive assumption "CIP controller-tag enumeration IS browsable, reuse the AbCip walker"
is **only half true for Omron**: the *protocol capability* exists (published variables are
reachable), but Omron NJ/NX exposes the tag list through a **different CIP object/service than
Rockwell** and libplctag hasn't implemented the Omron variant.
- **Additional gate:** on NJ/NX a variable is only reachable over the network at all if the
programmer set its **Network Publish** attribute (`Publish Only`) in **Sysmac Studio**. Tags
without Network Publish are invisible to any external client. And when *no* user variables are
published, an enumeration returns only the reserved system variables (names starting with `_`).
Source: [Omron NJ/NX EtherNet/IP User's Manual (W506)](https://files.omron.eu/downloads/latest/manual/en/w506_nj_nx-series_cpu_unit_built-in_ethernet_ip_port_users_manual_en.pdf),
[Omron store KB](https://store.omron.com.au/knowledge-base/1807412-using-cip-to-read-write-network-variables-with-nj-nx).
**Verdict:** CIP-Omron browse is achievable but requires **new protocol work, not free reuse**.
Two viable routes, in preference order:
1. **Import the Sysmac Studio tag export** (offline). NJ/NX projects export the published global
variable table (CSV/tag file). An AdminUI importer that parses this into candidate TagConfigs is
the *lowest-risk, highest-value* browse experience and needs no online CIP enumeration at all.
2. **Implement the Omron CIP tag-list service directly** (online `IBrowseSession`). Omron NJ/NX
supports reading the published-variable list via a CIP service Rockwell-differently; this means
hand-writing the request/response against the NJ/NX EtherNet/IP object model (not going through
libplctag's `@tags`). This is a research-grade effort — feasible (third-party tools like Kepware
discover Omron published tags) but non-trivial.
### 4.3 Browse-seam design (CIP path only)
If/when online CIP browse is implemented, it fits the existing seam exactly (mirroring
`OpcUaClient.Browser` / `Galaxy.Browser`, which live as separate `*.Browser` projects registered in
AdminUI DI and indexed by `DriverType`):
- New project `ZB.MOM.WW.OtOpcUa.Driver.OmronCip.Browser` implementing `IDriverBrowser`
(`DriverType = "OmronCip"`), `OpenAsync(configJson)` → `IBrowseSession`.
- `IBrowseSession`: `RootAsync` = one folder per device; `ExpandAsync` = published global variables
(and struct-member fan-out for UDT-typed variables, reusing AbCip's template-decode pattern);
`AttributesAsync` = data type / access / array shape side-panel.
- **Share a CIP-browse core with AbCip *only if* the enumeration transport is unified.** Since
Omron's tag-list service differs from Rockwell's, the honest boundary is: **share the
`BrowseNode`/`IBrowseSession` plumbing and struct-fan-out logic; do NOT expect to share the
wire-level enumerator.** Recommend a common `Cip.Browsing` helper library the AbCip browser (if
built) and OmronCip browser both consume for tree-shaping, with transport-specific enumerators
underneath. **Coordinate this boundary with the AbCip-browseability audit** so the two efforts
agree on the shared-core seam.
Note: the driver is still fully usable **without** a browser — like every driver, unmapped types
fall back to the raw-JSON TagConfig editor, and pre-declared tags always work. Browse is a
convenience layer, not a correctness dependency.
---
## 5. Test-fixture strategy
Consistent with the repo's Docker-fixture model (driver sims on the Linux host `10.100.0.35`,
controlled via `lmxopcua-fix`, compose files under `tests/.../Docker/`).
### 5.1 FINS
- **Hand-rolled byte-level fake** for unit tests (mirror the Modbus `IModbusTransport` fake) — the
primary and most valuable test surface; deterministic, no container.
- **Open-source FINS simulators** for an integration fixture:
[`hiroeorz/omron-fins-simulator`](https://github.com/hiroeorz/omron-fins-simulator) (Ruby, UDP
9600) and [`ahmadfarisfs/fins_simulator_omron`](https://github.com/ahmadfarisfs/fins_simulator_omron).
Neither ships a Docker image → wrap in a small `Dockerfile` under
`tests/Drivers/.../Omron/Docker/` and register with the `project=lmxopcua` label. Both are
UDP-oriented; FINS/TCP handshake coverage stays on the hand-rolled fake.
- Omron's own **CX-Simulator** speaks FINS but is Windows-only, license-gated (bundled in CX-One,
not separately purchasable) → **not** a CI fixture; reserve for a manual/live gate against real
hardware if available.
### 5.2 CIP (NJ/NX)
- **No credible free NJ/NX CIP simulator exists.** libplctag's own test harness targets Rockwell.
Options, in order:
- **Hand-rolled `IAbCipTagRuntime`-style fake** (Omron variant) for read/write/decode unit tests —
the same seam AbCip uses (`IAbCipTagRuntime` / factory injection) makes the driver fully
unit-testable without hardware. This is the primary CI surface.
- **Live gate against real NJ/NX hardware** (env-gated `Category=LiveIntegration`, skips cleanly
when the env var is absent — the pattern the HistorianGateway live suite already uses). This is
the only way to truly validate the `plc=omron-njnx` libplctag path, Network-Publish behavior,
and Omron string/array quirks. Flag as an **infra-gated known limitation** until hardware is on
the bench.
---
## 6. Effort / risk / phasing
**Recommended order: CIP-first, then FINS.**
Rationale: (a) CIP targets the modern/current Omron line (NJ/NX/NY) most likely in new installs;
(b) the CIP path **reuses 70–80% of the already-shipped, already-hardened AbCip stack** (libplctag,
poll engine, resilience, status mapping), so it is the fastest path to a working read/write driver;
(c) FINS, while simpler as a protocol, is a *net-new* transport codec with no reuse and targets
legacy hardware.
### Phase 1 — Omron CIP read/write (NJ/NX) · *Low–Medium effort*
- Extract a shared CIP core from AbCip (or, pragmatically, copy AbCip and specialise) →
`Driver.OmronCip` + `Driver.OmronCip.Contracts`, `DriverType="OmronCip"`, `plc=omron-njnx`.
- Capabilities: Connect/Read/Write/Subscribe(poll)/Probe; pre-declared tags via `ITagDiscovery`.
- Typed tag editor: copy `AbCipTagConfigModel`/editor, register in `TagConfigEditorMap` +
`TagConfigValidator`.
- Tests: hand-rolled runtime fake + env-gated live gate.
- **Risk:** libplctag Omron string/array edge cases (community-reported); word/UDT quirks. Medium.
### Phase 2 — Omron FINS read/write (CJ/CS/CP) · *Medium effort*
- New `IFinsTransport` codec (TCP+UDP, `0x0101`/`0x0102`, area-code table, word-swap) behind the
same `IDriver` shell; memory-area TagConfig shape (§3.2); manual-entry editor (no browse).
- Tests: byte-level fake (primary) + open-source simulator container (secondary).
- **Risk:** per-CPU-family area-code/word-order variance. Medium. No dependency/license risk (hand-rolled).
### Phase 3 — CIP online browse (NJ/NX) · *High effort / research-grade*
- Sysmac Studio tag-export importer first (cheap, high value), then optional online Omron CIP
tag-list `IBrowseSession` (`Driver.OmronCip.Browser`) — **not** libplctag `@tags`.
- **Coordinate the shared CIP-browse-core boundary with the AbCip-browseability audit.**
- **Risk:** High — Omron's tag-list service is undocumented-in-libplctag and Rockwell-different.
### Top risks (summary)
1. **CIP browse is not free reuse.** libplctag's tag-list walker does not work on Omron NJ/NX
(`ErrorUnsupported`); online browse needs net-new Omron-specific CIP work, and even then only
Network-Published variables are visible. Mitigate by shipping the offline Sysmac-export importer
first and treating online browse as a later research phase.
2. **No free NJ/NX CIP simulator** → CIP correctness for the real wire (string/array/UDT/Network-
Publish semantics) can only be proven on live hardware behind an env-gated `LiveIntegration`
suite; treat as an infra-gated known limitation until an NJ/NX is on the bench.
3. **(Secondary) FINS per-family variance** in memory-area codes and multi-word ordering — contain
with a CPU-family-parameterised code table + per-tag `wordSwap`.
---
## Sources
- [Omron store KB — Using CIP to Read/Write Network Variables with NJ/NX](https://store.omron.com.au/knowledge-base/1807412-using-cip-to-read-write-network-variables-with-nj-nx)
- [Omron NJ/NX CPU Built-in EtherNet/IP Port User's Manual (W506)](https://files.omron.eu/downloads/latest/manual/en/w506_nj_nx-series_cpu_unit_built-in_ethernet_ip_port_users_manual_en.pdf)
- [libplctag #466 — Support for Omron NX/NJ variable listing](https://github.com/libplctag/libplctag/issues/466)
- [libplctag.NET #371 — List all tags in Omron NJ: ErrorUnsupported](https://github.com/libplctag/libplctag.NET/issues/371)
- [libplctag Omron NJ/NX group thread (plc=omron-njnx, CIP path)](https://groups.google.com/g/libplctag/c/bU5MVQrPsOw)
- [Omron W227 FINS Commands Reference Manual](https://www.myomron.com/downloads/1.Manuals/Networks/W227E12_FINS_Commands_Reference_Manual.pdf)
- [Omron NX-series CPU Unit FINS Function User's Manual (W596)](https://files.omron.eu/downloads/latest/manual/en/w596_nx-series_-_cpu_unit_fins_function_users_manual_en.pdf)
- [Wireshark FINS dissector (framing reference)](https://github.com/wireshark/wireshark/blob/master/epan/dissectors/packet-omron-fins.c)
- [FieldServer Omron FINS protocol driver sheet](https://cdn.chipkin.com/assets/uploads/2016/jan/PDS_Omron_FINS.pdf)
- [Omron memory-area overview (CIO/HR/AR/DM/WR/EM)](https://community.oxmaint.com/discussion-forum/omron-plc-memory-areas-understanding-ci-hr-ar-lr-dm-wr-em-extended-em-usage)
- [HslCommunication — commercial-license fork situation](https://github.com/HslCommunication-Community/HslCommunication-Community)
- [omron-fins-simulator (open-source FINS sim)](https://github.com/hiroeorz/omron-fins-simulator)
- [fins_simulator_omron (open-source FINS sim)](https://github.com/ahmadfarisfs/fins_simulator_omron)
</content>
</invoke>
docs/research/drivers/sql-poll.md
+470
View File
@@ -0,0 +1,470 @@
# SQL Poll driver — design research
> **Status:** research / not-yet-approved. Author: driver research sweep, 2026-07-15.
> **Scope:** a new **`SqlPoll`** Equipment-kind driver that reads values from arbitrary SQL
> databases (MES/ERP staging tables, historians, custom app DBs) and surfaces them as OPC UA
> variable nodes — same shape as the existing `Modbus` / `S7` / `OpcUaClient` drivers.
> **Grounding:** `CLAUDE.md`, `IDriver.cs` + capability neighbours, `Driver.Modbus/` (polling
> template), `Commons/Browsing/` (browse seam), `docs/plans/2026-05-28-driver-browsers-design.md`.
---
## 0. TL;DR verdicts
| Question | Verdict |
|---|---|
| **Browseable?** | **YES.** SQL is self-describing via `INFORMATION_SCHEMA` (databases/schemas → tables/views → columns + types). A full `IDriverBrowser` / `IBrowseSession` is natural. |
| **Provider ship-first?** | **SQL Server** (`Microsoft.Data.SqlClient` — already in the dependency graph at 6.1.1), behind an ADO.NET `DbProviderFactory` abstraction. PostgreSQL (`Npgsql`) second, ODBC (`System.Data.Odbc`) as the universal fallback, MySQL/Oracle later. |
| **Write in v1?** | **NO — read-only v1.** Writing back to MES/ERP staging is operationally risky; ship an *optional*, opt-in, per-tag parameterized-UPSERT write mode as a later phase (design sketched in §2.4). |
| **Top risks** | (1) **SQL injection** — every identifier and value must be parameterized / allow-listed, never string-concatenated; (2) **secret handling** — connection-string credentials must come from env/secret store, never committed (mirror `ServerHistorian__ApiKey`). |
---
## 1. Design summary + provider strategy
### 1.1 Shape
`SqlPoll` is a standard Equipment-kind driver: a `SqlPollDriver : IDriver, ITagDiscovery,
IReadable, ISubscribable, IHostConnectivityProbe` living in
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.SqlPoll/`, with a `.Contracts` project for the options +
equipment-tag parser, and a `.Browser` project for the AdminUI address picker. It has **no native
push model** — subscriptions overlay a polling loop via the shared `PollGroupEngine`
(`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/PollGroupEngine.cs`), exactly as Modbus/S7/AB-CIP do.
`IWritable` and `IAlarmSource`/`IHistoryProvider` are **out of scope for v1** (writes deferred;
alarms/history aren't expressible from a poll query).
Points are ordinary equipment `Tag`s bound to the driver via `TagConfig.FullName`; reads/subscribes
resolve `FullName` to a tag definition through the shared `EquipmentTagRefResolver<TDef>`
(`Core.Abstractions/EquipmentTagRefResolver.cs`) — same two-model bridge Modbus uses (an authored
tag-table entry by name **or** an equipment tag whose reference is its raw `TagConfig` JSON,
parsed once and cached).
### 1.2 Provider abstraction
Use ADO.NET's provider-agnostic base types (`System.Data.Common`: `DbConnection`,
`DbCommand`, `DbParameter`, `DbDataReader`) obtained through a `DbProviderFactory`. In .NET
(Core/5+) there is no `machine.config`/GAC provider registry, so factories are registered in code
via `DbProviderFactories.RegisterFactory(invariantName, factory.Instance)` and each provider exposes
a static `Instance` singleton (`SqlClientFactory.Instance`, `NpgsqlFactory.Instance`,
`OdbcFactory.Instance`, `OracleClientFactory.Instance`, `MySqlConnectorFactory.Instance`)
([MS Learn: Obtain a SqlClientFactory](https://learn.microsoft.com/en-us/sql/connect/ado-net/obtain-sqlclientfactory),
[MS Learn: Obtaining a DbProviderFactory](https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/obtaining-a-dbproviderfactory)).
**Recommended approach — a small provider enum mapped to a factory**, not open-ended invariant
strings, because two provider-specific concerns leak past the ADO.NET base types and *must* be
dispatched per-provider:
1. **Identifier quoting** for the browse layer and any generated SQL —
`[SqlServer]` / `"Postgres"` / `` `MySql` `` / `"Oracle"`.
2. **Parameter marker syntax** — SQL Server & MySQL use `@p`, Npgsql uses `@p` or `:p`, Oracle
uses `:p`, ODBC uses positional `?`. A `SqlDialect` abstraction owns quoting + parameter
naming + the metadata-catalog query (see §4).
```csharp
public enum SqlProvider { SqlServer, Postgres, MySql, Odbc, Oracle }
internal interface ISqlDialect
{
DbProviderFactory Factory { get; }
string QuoteIdentifier(string ident); // [x] / "x" / `x`
string ParameterMarker(int ordinal); // @p0 / :p0 / ?
string ListColumnsSql(string? catalog); // metadata catalog query (browse)
string ListTablesSql(string? catalog);
}
```
**Ship-first recommendation: SQL Server only.** `Microsoft.Data.SqlClient` is already a repo
dependency (`Directory.Packages.props` line 53, `6.1.1`), the shared central test SQL Server is
already on the docker host (§5), and SQL Server is the dominant MES/historian staging backend in
this AVEVA/Wonderware estate (the Galaxy Repository and the HistorianGateway both sit on SQL Server).
Phase 2 adds **PostgreSQL** (`Npgsql`) and **ODBC** (`System.Data.Odbc`, the universal fallback that
reaches any DSN — DB2, Oracle via the Oracle ODBC driver, Access, etc.). MySQL/MariaDB
(`MySqlConnector`, whose `MySqlConnectorFactory` is fully async-correct, unlike Oracle's official
`MySql.Data`) and native Oracle (`Oracle.ManagedDataProvider`) are later, demand-driven additions.
`Npgsql`, `System.Data.Odbc`, and `Oracle.ManagedDataAccess.Core` are **not** yet in
`Directory.Packages.props` — each is a new `PackageVersion` entry gated behind its phase, so the
ship-first SQL-Server slice adds **zero** new NuGet dependencies.
---
## 2. Capability mapping
### 2.1 Connect — `InitializeAsync`
- Deserialize `SqlPollDriverOptions` from `DriverConfig` JSON (via the factory extensions, mirroring
`ModbusDriverFactoryExtensions.CreateInstance`). Resolve the connection string (§6 — from env/secret,
never the committed JSON). Select the `ISqlDialect` from `options.Provider`.
- **Validate** by opening one `DbConnection` and running a cheap liveness query
(`SELECT 1` — portable across SQL Server/Postgres/MySQL; Oracle needs `SELECT 1 FROM DUAL`, so the
liveness probe is a dialect method). Set `DriverHealth` = `Healthy` on success, `Faulted` on failure
(same state machine as `ModbusDriver.InitializeAsync`).
- **Pooling:** rely on ADO.NET's built-in connection pooling (keyed by exact connection-string text).
Do **not** hold one long-lived connection; open-use-dispose per poll pass so the pool manages
lifetime and a dropped server recovers transparently. Expose `Max Pool Size` etc. only via the
connection string.
- **`ReinitializeAsync`** = teardown + init (Modbus pattern). No live sockets to preserve.
### 2.2 Read / Subscribe — poll queries, **one query per table/group**
The whole efficiency story is **batching by query-group, not per-tag**. A naive "one SELECT per
tag" hammers the DB; instead the driver coalesces tags that share a source query and issues **one
round-trip per group per poll**, then slices the result back to per-tag snapshots — structurally the
same idea as `ModbusDriver.ReadCoalescedAsync` (group → single PDU → slice back).
`IReadable.ReadAsync(fullReferences)` groups the referenced tags by their **query group key**
(model-dependent, see §2.5), executes each group's query once, and maps rows/columns back to the
`DataValueSnapshot[]` in input order. `ISubscribable` delegates to `PollGroupEngine` with the same
`ReadAsync` as the reader delegate — the engine owns the loop, the 100 ms interval floor, capped-
exponential failure backoff, and change-diffing. Route poll-loop failures to `DriverHealth` via the
engine's `onError` sink + `PollBackoffCap` (Modbus/S7 precedent, 05/STAB-8/9).
**Poll interval** is driver-level (`DefaultPollInterval`, default e.g. 5 s) with an optional per-tag
or per-group override; the OPC UA subscription's requested publishing interval is clamped up to the
group's configured floor by `PollGroupEngine`.
**Per-call deadline (mandatory).** Every `DbCommand` must set `CommandTimeout` **and** run under a
linked `CancellationTokenSource` bounded by an `OperationTimeout` option. This is the SQL analogue of
the R2-01 S7 blackhole finding: a frozen DB server (network partition, lock wait, slow query) must
surface `BadTimeout`/`Degraded` and let the poll loop back off — **not** wedge the poll thread
forever. Prefer `DbCommand.ExecuteReaderAsync(ct)` with the linked token so cancellation is real, and
still set `CommandTimeout` as a server-side backstop.
### 2.3 Discover — offline / config-driven
Like Modbus, SQL has **no runtime discovery obligation**: `DiscoverAsync` materializes exactly the
authored tags (the `Tags` table in options, plus equipment tags contributed at deploy time). It does
**not** enumerate the DB schema at discovery — schema enumeration is the *browse* concern (§4), an
interactive AdminUI operation, not a deploy-time one. `RediscoverPolicy = Once`.
### 2.4 Write — verdict: **read-only v1**, optional write mode later
**v1 does not implement `IWritable`.** Rationale: MES/ERP/historian staging tables are frequently
downstream-of-record systems where an unexpected `UPDATE`/`INSERT` can corrupt a batch record,
double-post a transaction, or trip a trigger with side effects. A read-only integration is the safe,
common case and removes an entire class of risk from the first ship.
**Optional Phase-N write mode** (opt-in per driver *and* per tag): a tag may carry a
`writeCommand` describing a **parameterized** statement — never generated by string concatenation:
```jsonc
"write": {
"mode": "Update", // Update | Insert | Upsert | StoredProc
"sql": "UPDATE dbo.Setpoints SET value=@value, ts=@ts WHERE tag_name=@key",
"keyParam": "@key", "keyValue": "Line1.Speed",
"valueParam": "@value"
}
```
Writes route through `IWritable.WriteAsync` → the tag's parameterized command with the OPC UA
write value bound to `@value` (typed via the tag's declared type). Gate at three layers: (a) the OPC
UA node's `WriteOperate` role (standard `NodeWriteRouter`), (b) a driver-level `AllowWrites` master
switch (default `false`), (c) `TagConfig.writable`. Writes are **not** auto-retried unless the tag is
flagged `WriteIdempotent` (an `UPDATE ... WHERE key=` set-value is idempotent; an `INSERT` / counter
`UPDATE x=x+1` is not — mirror `DriverAttributeInfo.WriteIdempotent`). Because a failed inbound
device write reverts the node to its prior value (the write-outcome self-correction path,
master `1d797c1c`), SQL writes — unlike Galaxy's fire-and-forget — *can* surface a genuine failure,
which is the correct, honest behaviour.
### 2.5 Tag→value mapping models
Support **two** models in v1 (both cover the vast majority of MES/ERP staging shapes), plus an
escape hatch:
1. **Key-value / EAV table** *(primary — ship first).* A table of `(tagname, value, timestamp)`
rows; a tag names a `keyColumn`+`keyValue`, a `valueColumn`, and an optional `timestampColumn`.
**Group key = the table.** One `SELECT keyColumn, valueColumn, timestampColumn FROM table WHERE
keyColumn IN (@k0,@k1,…)` per poll returns every subscribed tag on that table in one round-trip;
the driver indexes rows by key and slices back. (Values `IN (@params)` — never interpolated.)
2. **Wide row** *(ship first, cheap).* One "latest status" row, many columns, each column a tag. A
tag names a `table`, a `columnName`, and a row selector (`WHERE id=@id`, or "top 1 by timestamp").
**Group key = (table, row-selector).** One `SELECT col_a, col_b, … FROM table WHERE …` per poll
maps each selected column to its tag.
3. **Arbitrary parameterized SELECT** *(escape hatch, per named query group).* Named query definitions
at the driver level (`queries: { "q1": { sql, params } }`); a tag references a query by name +
a projection (which output column / which row-key). **Group key = the named query.** This covers
joins, views, and computed projections the two structured models can't express. The SQL is authored
by an operator with `ConfigEditor` rights and stored server-side — it is *config*, not user input —
but it still binds only parameters, never interpolated tag values.
### 2.6 Type mapping (SQL → OPC UA) + timestamps
Map the column's provider metadata (`DbColumn.DataType` from `DbDataReader.GetColumnSchema()`, or the
`INFORMATION_SCHEMA` `DATA_TYPE`) to the driver-agnostic `DriverDataType`
(`Core.Abstractions/DriverDataType.cs`), which the address-space layer maps to OPC UA built-ins:
| SQL type (family) | CLR (`DbDataReader`) | `DriverDataType` |
|---|---|---|
| `bit` / `boolean` | `bool` | `Boolean` |
| `tinyint`/`smallint` | `byte`/`short` | `Int16` |
| `int`/`integer` | `int` | `Int32` |
| `bigint` | `long` | `Int64` |
| `real`/`float(24)` | `float` | `Float32` |
| `float`/`double`/`decimal`/`numeric`/`money` | `double`/`decimal`→`double` | `Float64` |
| `char`/`varchar`/`nvarchar`/`text` | `string` | `String` |
| `date`/`datetime`/`datetime2`/`timestamptz` | `DateTime`/`DateTimeOffset` | `DateTime` |
| `uniqueidentifier`/`uuid` | `Guid` | `String` |
`decimal`/`numeric` collapse to `Float64` (v1) with a documented precision-loss caveat; a tag may
override the inferred type explicitly (`"type": "Int32"`) for cases where the operator knows better
than the column metadata (e.g. a `varchar` column holding a number). `NULL` → `DataValueSnapshot`
with `Value = null` and a `Good`/`Uncertain` status per an option (`NullIsBad`, default treat as
`GoodNoData`-ish `Uncertain`).
**Source timestamp:** if a tag declares a `timestampColumn`, that column's value becomes the
`DataValueSnapshot.SourceTimestamp`; otherwise the poll-read wall-clock is used for both source and
server timestamps (the Modbus behaviour). A source timestamp is what makes SQL-poll data honest about
staleness — strongly recommend authoring one wherever the staging table has it.
---
## 3. TagConfig + driver-config JSON shape
### 3.1 Driver config (the `DriverConfig` blob)
```jsonc
{
"provider": "SqlServer",
// Connection string is resolved at Initialize from an env/secret ref — NOT stored here.
// Option A (recommended): a named ref the Host resolves from env:
"connectionStringRef": "SqlPoll_MesStaging", // => env SqlPoll__ConnectionStrings__MesStaging
"defaultPollInterval": "00:00:05",
"operationTimeout": "00:00:15", // per-query wall-clock deadline (BadTimeout)
"commandTimeout": "00:00:10", // server-side CommandTimeout backstop
"allowWrites": false, // master write kill-switch (v1: always false)
"probe": { "enabled": true, "interval": "00:00:10" },
// Optional pre-declared tag table (analogous to Modbus "Tags"); equipment tags may also
// contribute their address as raw TagConfig JSON at deploy time.
"tags": [ /* see 3.2 */ ],
// Optional named queries for the arbitrary-SELECT model.
"queries": {
"activeBatch": {
"sql": "SELECT line, speed, temp, ts FROM dbo.LineStatus WHERE active=@active",
"params": { "@active": true }
}
}
}
```
### 3.2 Per-tag TagConfig — **key-value model**
```jsonc
{
"driver": "SqlPoll",
"model": "KeyValue",
"table": "dbo.TagValues",
"keyColumn": "tag_name",
"keyValue": "Line1.Speed",
"valueColumn": "num_value",
"timestampColumn": "sample_ts",
"type": "Float64", // optional explicit override of inferred type
"writable": false
}
```
Group key for batching = `("dbo.TagValues", keyColumn, valueColumn, timestampColumn)`; all such tags
fold into one `… WHERE tag_name IN (@k0,@k1,…)` per poll.
### 3.3 Per-tag TagConfig — **wide-row model**
```jsonc
{
"driver": "SqlPoll",
"model": "WideRow",
"table": "dbo.LatestStatus",
"columnName": "oven_temp",
"rowSelector": { "whereColumn": "station_id", "whereValue": 7 },
// or "rowSelector": { "topByTimestamp": "sample_ts" } // newest row
"type": "Float64",
"writable": false
}
```
Group key = `("dbo.LatestStatus", rowSelector)`; every wide-row tag on the same row folds into one
`SELECT oven_temp, <other cols>, … FROM dbo.LatestStatus WHERE station_id=@id`.
### 3.4 Per-tag TagConfig — **named-query model**
```jsonc
{
"driver": "SqlPoll",
"model": "Query",
"query": "activeBatch",
"rowKey": { "column": "line", "value": "Line1" }, // which row this tag reads
"column": "speed", // which projected column is the value
"timestampColumn": "ts"
}
```
The equipment-tag parser (`SqlPollEquipmentTagParser`, in `.Contracts`) recognizes the blob by a
leading `{` + a `"driver":"SqlPoll"` (or a `"model"` discriminator) and maps it to a transient
`SqlPollTagDefinition` whose `Name` == the reference string — exactly the `ModbusEquipmentTagParser`
pattern, so the value the driver publishes keys the forward router correctly.
---
## 4. Browseability verdict + browse design
### 4.1 Verdict: **YES — SQL is browseable via its metadata catalog.**
Relational databases are self-describing. The picker walks the catalog three levels deep:
- **`RootAsync`** → databases / schemas (Folder nodes). SQL Server: databases (`sys.databases`) then
schemas; Postgres: databases then schemas; MySQL: schemas (== databases). For SQLite: the single
main database (Folder).
- **`ExpandAsync(schemaNodeId)`** → tables + views in that schema (Folder nodes).
- **`ExpandAsync(tableNodeId)`** → columns (Leaf nodes).
- **`AttributesAsync(columnNodeId)`** → column data type + nullability + (for key-value tables) a
hint panel. This mirrors Galaxy's two-stage attribute side-panel: pick a table (Folder), then pick
the column/key that becomes the tag.
### 4.2 Portability caveat — the metadata query is dialect-specific
`INFORMATION_SCHEMA` is the SQL-92 standard catalog and is supported by **SQL Server, PostgreSQL, and
MySQL/MariaDB** — so `SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.TABLES` and
`… COLUMN_NAME, DATA_TYPE, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME=@t` are portable
across those three. But **two important backends do not implement `INFORMATION_SCHEMA`:**
- **Oracle** uses `ALL_TABLES` / `ALL_TAB_COLUMNS` (and `USER_`/`DBA_` variants) instead
([Oracle: ALL_TAB_COLUMNS](https://docs.oracle.com/en/database/oracle/oracle-database/19/refrn/ALL_TAB_COLUMNS.html),
[dataedo: what supports INFORMATION_SCHEMA](https://dataedo.com/kb/databases/all/information_schema)).
- **SQLite** uses `sqlite_schema`/`sqlite_master` + `PRAGMA table_info(<table>)`
([SQLite: the schema table](https://sqlite.org/schematab.html)).
So the catalog query **belongs on `ISqlDialect`** (`ListTablesSql` / `ListColumnsSql`), not hardcoded.
The ship-first SQL-Server dialect uses `INFORMATION_SCHEMA`; Postgres/MySQL reuse it; the SQLite
fixture dialect (§5) uses `PRAGMA`; Oracle (later) uses `ALL_TAB_COLUMNS`. A cleaner but slower
alternative that avoids catalog SQL entirely for the *column* level is
`DbDataReader.GetColumnSchema()` on a `SELECT * … WHERE 1=0` (`SchemaOnly`) command — but it still
needs catalog SQL for the table list, so the dialect method stays.
### 4.3 Picked-column → TagConfig
On column pick, the browser emits a `BrowseNode.NodeId` that encodes
`provider|schema.table|column`, and the picker body composes the `TagConfig` blob (§3.2/3.3):
DisplayName defaults to `table.column`, the inferred `DriverDataType` from `AttributesAsync` prefills
`type`, and the operator chooses the model (key-value vs wide-row) + fills the key/row selector.
`AttributeInfo.DriverDataType` carries the SQL type; `SecurityClass` = `ViewOnly` in read-only v1.
### 4.4 New `.Browser` project layout
Follow the exact `OpcUaClient.Browser` template (`docs/plans/2026-05-28-driver-browsers-design.md`):
| Path | Purpose |
|---|---|
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.SqlPoll.Browser/SqlPollDriverBrowser.cs` | `IDriverBrowser`, `DriverType => "SqlPoll"`; opens a `DbConnection` from form JSON (transient) |
| `src/Drivers/…SqlPoll.Browser/SqlPollBrowseSession.cs` | `IBrowseSession`; `RootAsync`/`ExpandAsync`/`AttributesAsync` over the dialect's catalog queries; `SemaphoreSlim` gate (ADO.NET connection not concurrent) |
| register in `AdminUI/EndpointRouteBuilderExtensions.cs` | `services.AddSingleton<IDriverBrowser, SqlPollDriverBrowser>();` (next to the OpcUaClient/Galaxy lines, `:49-50`) |
| `AdminUI/Uns/TagEditors/TagConfigEditorMap.cs` | add `["SqlPoll"] = typeof(…SqlPollTagConfigEditor)` + `TagConfigValidator` entry (typed editor; else it falls to the raw-JSON textarea like Galaxy) |
The browser reuses the AdminUI `BrowseSessionRegistry` + reaper + 2-min idle TTL + per-call timeout
unchanged. Form JSON contains the connection secret (same trust boundary as `TestDriverConnect` and
the OpcUaClient browser) — deserialize → build connection → release, never cache the JSON.
---
## 5. Test-fixture strategy
- **SQLite in-memory / file** — the *unit-test* fixture. `Microsoft.Data.Sqlite` (already in-repo at
10.0.7) needs no container; seed a `(tag_name, num_value, sample_ts)` key-value table and a
wide-row `LatestStatus` table in-process. Covers read/subscribe mapping, type mapping, group-batch
slicing, the poll engine wiring, and the browser over `PRAGMA table_info`. Fast, offline, CI-safe on
macOS — the primary suite.
- **Central SQL Server on the docker host** (`10.100.0.35,14330`, always-on per `CLAUDE.md`) — the
*integration* fixture. A seeded `SqlPollFixture` database with the same two sample tables validates
the real `Microsoft.Data.SqlClient` path, `INFORMATION_SCHEMA` browse, `CommandTimeout`/cancellation,
and connection pooling. Env-gated (`SQLPOLL_TEST_ENDPOINT`) so it skips cleanly offline, like the
other `*.IntegrationTests`. Deploy the seed via the docker rig; label the stack `project=lmxopcua`.
- **Postgres/ODBC fixtures** land with their respective phases (a `postgres:16` container on the
docker host for Npgsql; an ODBC DSN against the same SQL Server for the `System.Data.Odbc` path).
- **Blackhole / timeout live-gate** (mirrors R2-01 S7): `docker pause` the SQL container mid-poll and
assert the read surfaces `BadTimeout` + the driver degrades + the poll loop backs off, *not* a wedged
thread. This is the single most important integration test given the frozen-peer risk class.
---
## 6. Security — secret handling
Connection-string credentials are the crown jewels and **must never be committed**. Mirror the
repo's existing pattern for `ServerHistorian:ApiKey` (supplied via env `ServerHistorian__ApiKey`,
appsettings carries only a `_comment` reminder) and the ConfigDb connection string (env-overridable):
- The `DriverConfig` JSON stores a **`connectionStringRef`** (a logical name), **not** the connection
string. The Host resolves it from configuration/env at `InitializeAsync`
(`SqlPoll__ConnectionStrings__<ref>`), so the secret lives in the process environment / secret store,
never in the config DB row or any committed file.
- If an inline `connectionString` is permitted at all (dev convenience), the AdminUI must redact it
in display/logging and it must be flagged as dev-only. **Never log the resolved connection string**
(Serilog); log only the provider + server host + database name.
- Prefer integrated/managed auth where the estate supports it (`Integrated Security=true` /
Azure AD / Kerberos) so no password transits config at all.
- The `.Browser` path receives the secret in form JSON over the authenticated Blazor circuit (same
boundary as every other driver browser + `TestDriverConnect`); it is used to build one transient
connection and then released — no `_lastConfigJson` field anywhere.
---
## 7. Effort, risk, phasing
### 7.1 Key risks
1. **SQL injection (critical).** Every value is a bound `DbParameter`; identifiers (table/column/
schema) that must appear in SQL text come **only** from `INFORMATION_SCHEMA`-validated,
dialect-`QuoteIdentifier`'d names or an allow-list derived from browse — never from free-form tag
input concatenated into a query. The named-query model stores operator-authored SQL as *config*
(ConfigEditor-gated), and even there only parameters bind runtime values. Treat any code path that
builds SQL by string concatenation of a tag field as a defect.
2. **Secret handling (critical).** Per §6 — env/secret-store only, redact in UI/logs, never commit.
3. **Frozen-peer / timeout wedge (high).** A slow/blocked DB must time out per-query and back off
(the R2-01 lesson) — `CommandTimeout` + linked-CTS `OperationTimeout` on every command; assert it
in the blackhole live-gate.
4. **Connection pooling / exhaustion (medium).** Open-use-dispose per poll; never leak a `DbConnection`
or `DbDataReader` (always `await using`); a group query that fans out to many tables must not open
an unbounded number of concurrent connections — cap group-parallelism (Modbus/S7 precedent).
5. **Provider portability (medium).** `INFORMATION_SCHEMA` covers SQL Server/Postgres/MySQL but *not*
Oracle/SQLite (§4.2); parameter markers and identifier quoting differ. The `ISqlDialect` seam
contains all of it — do not let dialect assumptions leak into the driver core.
6. **Type / precision (low-medium).** `decimal`→`Float64` precision loss; `NULL` semantics; timezone
handling on `datetime` vs `datetimeoffset`. Documented + per-tag `type` override.
### 7.2 Suggested phasing
1. **Phase 1 — SQL Server read-only, key-value + wide-row.** `.Contracts` (options + equipment-tag
parser), `SqlPollDriver` (`IDriver`/`ITagDiscovery`/`IReadable`/`ISubscribable` via `PollGroupEngine`),
`SqlServerDialect`, factory registration, `SqlPollTagConfigEditor` + validator, SQLite unit fixture,
SQL-Server integration fixture (env-gated) + blackhole live-gate. No new NuGet deps.
2. **Phase 2 — Browse.** `SqlPoll.Browser` project (`IDriverBrowser`/`IBrowseSession` over
`INFORMATION_SCHEMA`), AdminUI picker body + registry wire-up, column→TagConfig commit.
3. **Phase 3 — Named-query model + PostgreSQL + ODBC.** `PostgresDialect`/`OdbcDialect`, add `Npgsql`
+ `System.Data.Odbc` `PackageVersion`s, Postgres container fixture.
4. **Phase 4 (optional) — Write mode.** `IWritable` with per-tag parameterized UPSERT/StoredProc,
triple-gated (`WriteOperate` role + `AllowWrites` + `TagConfig.writable`), `WriteIdempotent` retry
policy, write-failure revert (already provided by the runtime).
5. **Phase 5 (demand-driven) — MySQL/MariaDB (`MySqlConnector`) + native Oracle** (`ALL_TAB_COLUMNS`
dialect).
### 7.3 Effort estimate
Phase 1 ≈ the Modbus driver minus the wire-protocol codec complexity (no byte-order/register math) but
plus the dialect seam and result-set slicing — **medium**, most of it reusable from Modbus (poll engine,
resolver, health state machine, factory shape are all shared). Phase 2 (browse) is **small** — a near-
mechanical clone of `OpcUaClient.Browser` with a different one-level expansion. The provider/dialect
work (Phase 3+) is where the long tail of portability effort lives; keeping it behind `ISqlDialect`
from day one is what makes the ship-first SQL-Server slice cheap and the rest additive.
---
## Sources
- [MS Learn — Obtain a SqlClientFactory (RegisterFactory in .NET Core, `SqlClientFactory.Instance`)](https://learn.microsoft.com/en-us/sql/connect/ado-net/obtain-sqlclientfactory)
- [MS Learn — Obtaining a DbProviderFactory (`DbProviderFactories`, provider `Instance` pattern)](https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/obtaining-a-dbproviderfactory)
- [Oracle — ALL_TAB_COLUMNS (Oracle has no INFORMATION_SCHEMA)](https://docs.oracle.com/en/database/oracle/oracle-database/19/refrn/ALL_TAB_COLUMNS.html)
- [dataedo — What is INFORMATION_SCHEMA and which databases support it](https://dataedo.com/kb/databases/all/information_schema)
- [SQLite — The Schema Table (`sqlite_schema`/`PRAGMA table_info`)](https://sqlite.org/schematab.html)
- In-repo grounding: `IDriver.cs`, `ISubscribable.cs`, `PollGroupEngine.cs`, `DriverDataType.cs`,
`Driver.Modbus/ModbusDriver.cs` + `.Contracts`, `Commons/Browsing/*`,
`Driver.OpcUaClient.Browser/*`, `AdminUI/EndpointRouteBuilderExtensions.cs`,
`AdminUI/Uns/TagEditors/TagConfigEditorMap.cs`, `docs/plans/2026-05-28-driver-browsers-design.md`,
`Directory.Packages.props` (Microsoft.Data.SqlClient 6.1.1, Microsoft.Data.Sqlite 10.0.7).