dohertj2/lmxopcua
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: driver-expansion program — 8 research reports + 7 design docs, parallel-reviewed
v2-ci / build (push) Successful in 3m14s
Details
v2-ci / unit-tests (push) Failing after 9m13s
Details

Browse Source 
Adds the driver-expansion program design (umbrella: universal Discover-backed
browser + MTConnect, MQTT/Sparkplug B, BACnet/IP, SQL poll, Omron, Modbus RTU;
MELSEC deferred) plus the per-driver research reports.

All docs went through a 7-agent parallel review against the codebase before
this commit. Highlights fixed in review:

- universal browser: FOCAS FixedTree fills post-connect -> UntilStable settle
  + FixedTree.Enabled patch; MQTT reconciled to bespoke (was contradicting the
  program doc's SupportsOnlineDiscovery=false verdict)
- modbus-rtu: SerialPort.ReadTimeout doesn't bound async BaseStream reads ->
  linked-CTS per-op deadline (R2-01 class); BCL enum reuse would leak
  System.IO.Ports into Contracts
- bacnet: DiscoveryRediscoverPolicy enum name; UDP 47808 contention; live
  suite rewritten around unicast Who-Is + BBMD (broadcast doesn't cross VMs)
- sql-poll: real tier registration via DriverFactoryRegistry.Register;
  blackhole gate must not docker-pause the shared central SQL Server
- mqtt: Sparkplug v3.0 STATE topic form; first-in-repo proto codegen noted
- omron: host hardcodes isIdempotent:false today (retry seam unshipped);
  v1 scopes UDTs to dotted-leaf access
- mtconnect: SecurityClassification.ViewOnly; factory ParseEnum<T> pattern
- program doc: both valid enum-serialization patterns; IRediscoverable is
  change-signal-gated; RTU P2 adds System.IO.Ports; label is host-side
This commit is contained in:
Joseph Doherty
2026-07-15 16:40:36 -04:00
parent 37cdbef7a5
commit 8fc147d8d4
17 changed files with 6408 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/2026-07-15-bacnet-ip-driver-design.md
+556
View File
@@ -0,0 +1,556 @@
# BACnet/IP driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Supersedes/derives from:** the research report `docs/research/drivers/bacnet-ip.md` (protocol
survey, library selection, browse verdict). This document turns that research into a concrete,
file-by-file build plan for `ZB.MOM.WW.OtOpcUa.Driver.Bacnet`.
**Reconciles with:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (BACnet browse is
delivered by the **universal discovery browser** in v1, not a bespoke `IBrowseSession`).
---
## 1. Motivation + scope
BACnet/IP (ASHRAE 135 / ISO 16484-5) is the dominant building-automation protocol — HVAC, lighting,
energy metering, access control. A BACnet driver lets the OtOpcUa server pull **facility / HVAC /
energy** points (space temperatures, setpoints, kW/kWh meters, occupancy, equipment status) into the
unified namespace alongside process data (Modbus/S7/Galaxy), so a plant's OT and facility signals live
under one OPC UA address space.
BACnet is the closest analog to the **OpcUaClient** driver in this repo: it has native **device
discovery** (Who-Is/I-Am) and native **change-of-value push** (SubscribeCOV), so it is a first-class
*browseable* + *subscribe-capable* Equipment-kind driver, not a bare poller.
**Scope (this driver is a standard Equipment-kind driver.)** BACnet points are ordinary equipment
`Tag`s bound to the BACnet driver via `TagConfig.FullName` (a JSON addressing blob — see §5), authored
through the standard `/uns` TagModal + address picker, exactly like Modbus / S7 / OpcUaClient. No alias
machinery, no bespoke namespace kind.
**v1 scope: Connect + Read + Discover (+ universal browse) + BBMD/foreign-device. COV push is P2, Write
is P3.** See the phasing in §10.
Research reference: `docs/research/drivers/bacnet-ip.md`.
---
## 2. Project layout
Two new projects, mirroring the Modbus / OpcUaClient split (transport project + a lean Contracts project
the AdminUI can reference without dragging in the UDP transport):
| Project | Contents | References |
|---|---|---|
| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet`** | `BacnetDriver` (IDriver + capability surfaces), the async adapter over the ela-compil client (`BacnetClientAdapter`), `BacnetDriverProbe`, `BacnetDriverFactoryExtensions`, data-type map | `Core.Abstractions`, `Core` (the `Core.Hosting` namespace — `DriverFactoryRegistry` — lives inside the `Core` project), `.Bacnet.Contracts`, NuGet `BACnet` |
| **`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts`** | `BacnetDriverOptions` (+ nested `ForeignDeviceOptions`/`DiscoveryOptions`/`CovOptions`/`ProbeOptions`), the `BacnetObjectType` / `BacnetPropertyId` enums the AdminUI editor binds, `BacnetTagAddress` record, `BacnetEquipmentTagParser` | `Core.Abstractions` only (zero transport deps — same discipline as `Modbus.Contracts` / `Modbus.Addressing`) |
**No `.Browser` project in v1.** BACnet browse comes *for free* from the universal discovery browser
(§4). A bespoke lazy `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser` is a documented **P-later** graduation for
large multi-device sites (§4.2), not v1 work.
**Contracts csproj** — copy `ZB.MOM.WW.OtOpcUa.Driver.Modbus.Contracts.csproj` minus its
`Modbus.Addressing` reference (BACnet has no separate addressing project): `net10.0`,
`Nullable`/`ImplicitUsings` enable, `TreatWarningsAsErrors`, `ProjectReference` to `Core.Abstractions`
only (it owns the shared `TagConfigJson` field readers used by the parser).
**NuGet.** Central package management: pin the version in `Directory.Packages.props` and reference
(version-less) from the transport project only:
```xml
<!-- Directory.Packages.props -->
<PackageVersion Include="BACnet" Version="4.0.0" /> <!-- confirm the exact current release when implementing -->
<!-- Driver.Bacnet.csproj -->
<PackageReference Include="BACnet" />
```
- **Library: `System.IO.BACnet`, ela-compil fork, NuGet package id `BACnet`, MIT license.** The engine
inside YABE; explicitly multi-targets **`net10.0`** (no TFM friction); covers everything we need —
Who-Is/I-Am, ReadProperty, ReadPropertyMultiple, WriteProperty, SubscribeCOV/SubscribeCOVProperty +
COV notifications, and BACnet/IP with BBMD + foreign-device registration.
- **The async wrinkle (§4).** The library API is **event/callback-driven and blocking-with-callbacks**,
not `async`/`Task`-first: `BacnetClient.WhoIs()` fires `OnIam` events; `ReadPropertyRequest` has a
blocking overload + a begin/end pair. The driver **must** wrap it behind a thin async adapter
(`TaskCompletionSource` + timeout + cancellation) exactly as `OpcUaClientDriver` wraps the OPC UA SDK.
Budget a dedicated `BacnetClientAdapter` layer for this.
- **One `BacnetClient` per driver instance**, not per device — a single shared
`BacnetIpUdpProtocolTransport` UDP socket multiplexes every device on the network (mirrors "single
`Session` per OpcUaClient driver"). **Port-sharing caveat:** receiving *broadcast* traffic requires
binding UDP 47808, and only one exclusive binder per host interface gets it — so **two BACnet
`DriverInstance`s deployed to the same Host node contend for the port** (shared-socket binds make
datagram delivery nondeterministic). Constrain to one BACnet driver instance per node/interface, or
give additional instances distinct `port` values (fine for sims; real devices always speak 47808).
Unicast flows (directed Who-Is, RP/RPM, COV notifications — devices reply to the request's source
address:port) work from any local port, which is what makes the probe's ephemeral-port bind (§7) safe.
---
## 3. Capability mapping (`IDriver` + capability interfaces)
Recommended surface, landing across phases:
```csharp
public sealed class BacnetDriver
: IDriver, ITagDiscovery, IReadable, IHostConnectivityProbe, // P1
ISubscribable, // P2
IWritable, // P3
IDisposable, IAsyncDisposable
```
`DriverType => "BACnet"` (see §7 for the exact casing decision).
| OtOpcUa capability | BACnet mechanism | Phase |
|---|---|---|
| `IDriver.InitializeAsync` | Bind the UDP socket on the configured interface/port; if `ForeignDevice` is configured, `Register-Foreign-Device` to the BBMD **and start the TTL-renew timer** (renew at ~TTL/2); optionally fire one warm-up `Who-Is`. Health → Healthy once the socket is bound (BACnet has no session; liveness is per-device via read/COV, surfaced through the probe). | P1 |
| `ITagDiscovery.DiscoverAsync` | Bounded `Who-Is` collect → per-device Device `object-list` (RPM, segmented) → per-object `object-name` + `units` + `object-type` → stream folders/variables into `IAddressSpaceBuilder`. `SupportsOnlineDiscovery => true`; `RediscoverPolicy => DiscoveryRediscoverPolicy.Once`. | P1 |
| `IReadable.ReadAsync` | Group the batch's references by device-instance; issue **ReadPropertyMultiple** per device for `present-value` (+ `status-flags` for quality). Fall back to per-object **ReadProperty** for devices that reported no RPM/segmentation support in their I-Am. Map result → `DataValueSnapshot`. | P1 |
| `IHostConnectivityProbe` | Per-device liveness: a cheap RP of each known device's `system-status` (or a directed Who-Is). Running↔Stopped transitions raise `OnHostStatusChanged`, scoping Bad-quality fan-out to that device's subtree. Foreign-device mode: probe also asserts "BBMD registration alive." | P1 |
| `ISubscribable.SubscribeAsync` | **SubscribeCOV** per object (native push — the headline win). `OnCOVNotification` handler fans changes to `OnDataChange`. **Lifetime-renew timer** re-subscribes before expiry. Devices that reject COV **degrade transparently to a poll loop** at `publishingInterval`, so the subscribe surface always works. | P2 |
| `IWritable.WriteAsync` | **WriteProperty** `present-value` at a configured **priority** (1–16), or `null` to relinquish. **Verdict: NOT in v1 (§3.3).** | P3 |
| `GetHealth` / `GetMemoryFootprint` / `FlushOptionalCachesAsync` | Standard. Footprint ≈ discovered-object count × ~512 B (mirror `OpcUaClientDriver.GetMemoryFootprint`). Flush drops the device/object browse cache + resets the footprint counter. | P1 |
### 3.1 `IReadable` shape (concrete)
Follow `OpcUaClientDriver.ReadAsync` structure: return one `DataValueSnapshot` per requested reference in
order; report **per-reference** failures via the snapshot's `StatusCode` (Bad-coded), throw only if the
whole client is unreachable. BACnet specifics:
1. Parse each `fullReference` with `BacnetEquipmentTagParser.TryParse` (§5); a malformed blob →
`BadNodeIdUnknown` snapshot without touching the wire.
2. Group parsed addresses by `deviceInstance`. Resolve device network address from the discovered device
table (cached from Who-Is/I-Am); a device not in the table → directed Who-Is with a short deadline,
else `BadCommunicationError`.
3. Per device: one **RPM** for all `(object, present-value + status-flags)` pairs, under the per-request
APDU deadline (§8). Non-RPM devices → sequential **RP** per object.
4. Map `BacnetValue` → `DataValueSnapshot(value, statusCode, sourceTs=null, serverTs=now)`.
`status-flags` `out-of-service`/`fault` ⇒ Uncertain/Bad; else Good (see §3.4).
### 3.2 `ISubscribable` shape (P2 — the real advantage over poll drivers)
- `SubscribeAsync` issues **SubscribeCOV** (`preferConfirmed` from config) per object, tracking a
`subscriberProcessIdentifier` per subscription. Fire the OPC UA initial-data callback with a current
read so clients get a value immediately.
- **Lifetime-renew timer.** Each subscription carries `lifetimeSeconds`; a per-driver timer re-issues
SubscribeCOV at ~`lifetime/2`. This is load-bearing — an un-renewed subscription silently expires and
the node goes stale-Good with no error. (Same class of bug as the redundancy-heartbeat and
continuous-historization renewal traps in this repo's memory — treat renewal as first-class.)
- **Transparent poll fallback.** On a SubscribeCOV reject (error-class `services`/`object`, or a device
whose I-Am advertised no COV), the driver **silently** switches that object to an internal poll loop at
`cov.fallbackPollMs` and still raises `OnDataChange`. The operator sees data either way; the
diagnostic id records which mode each object is in.
- `UnsubscribeAsync` sends SubscribeCOV with lifetime 0 (cancel) and stops any fallback poll.
### 3.3 `IWritable` — verdict: NOT v1 (P3), and why
BACnet writes target a **priority array**: a commandable output (AO/BO/MSV with a `priority-array`
property) resolves its effective value from the highest-priority non-null slot, with `relinquish-default`
as the floor. Writing `present-value` actually writes a *slot* (priority 1–16); writing the wrong slot,
or writing an absolute value where the site's BMS control logic expects to own a slot, **fights the
building's own control loops** — a genuine safety concern (e.g. commanding a damper the BMS is
sequencing). Correctly modelling commandable-vs-non-commandable objects, slot selection, and relinquish
semantics is a correctness minefield that deserves its own phase with explicit validation. So:
- **v1: read-only.** Every tag's parsed address carries `writable=false`; the driver does **not**
implement `IWritable` in P1/P2.
- **P3: `IWritable`** with an explicit per-tag `writePriority` (default 8 "Manual Operator", never 1), a
commandable-object guard (reject writes to non-commandable object types), and a `null`-relinquish path.
Mirror `OpcUaClientDriver.WriteAsync` honesty: a timeout after dispatch → `BadTimeout` (outcome
unknown), a pre-wire failure → `BadCommunicationError`.
### 3.4 Data-type mapping (`BacnetObjectType`/property → `DriverDataType`)
Driver-side map lives in `Driver.Bacnet/BacnetDataTypeMap.cs` (parallel to Galaxy's `DataTypeMap.cs`).
`DriverDataType` members available: `Boolean, Int16/32/64, UInt16/32/64, Float32, Float64, String,
DateTime, Reference`.
| BACnet present-value / property type | `DriverDataType` | Notes |
|---|---|---|
| REAL (AI/AO/AV present-value) | `Float32` | the common analog case |
| DOUBLE | `Float64` | |
| BOOLEAN | `Boolean` | |
| Enumerated Active/Inactive (BI/BO/BV) | `Boolean` | 0=inactive, 1=active |
| Enumerated multistate (MSI/MSO/MSV) | `UInt16` | present-value is a 1-based state index; `state-text` array surfaced as metadata later |
| Unsigned Integer | `UInt32` | |
| Signed Integer | `Int32` | |
| CharacterString (`object-name`, `description`) | `String` | |
| BACnetDateTime / Date / Time | `DateTime` | Schedules, Trend Log timestamps |
| ObjectIdentifier / property reference | `String` | rarely a tag target |
| Array property (element index N) | element type + `IsArray=true` | ValueRank=1, ArrayDimensions from the array length |
**Engineering Units.** The BACnet `units` enum (`degrees-celsius`, `kilowatt-hours`, `percent`, …) is
captured at discovery/browse time and surfaced as OPC UA `EngineeringUnits` (EUInformation) node metadata
via `IAddressSpaceBuilder.AddProperty`, so UNS nodes carry proper units.
**Quality from `status-flags`.** `out-of-service` or `fault` ⇒ Bad/Uncertain `StatusCode`; else Good.
Read RPM should always co-fetch `status-flags` alongside `present-value`.
---
## 4. The async wrinkle + how discovery/browse are delivered
### 4.1 The callback→async adapter
`BacnetClientAdapter` (new, in the transport project) is the single seam that turns the ela-compil
callback/blocking API into the `Task`-returning calls the driver's capability methods need. Pattern
(identical intent to how `OpcUaClientDriver` wraps the SDK):
- **Who-Is is a broadcast + asynchronous I-Am collection — there is no "the response".** The adapter
exposes `Task<IReadOnlyList<DiscoveredDevice>> WhoIsAsync(range, TimeSpan window, ct)`: subscribe
`OnIam`, `client.WhoIs(low, high)`, accumulate callbacks into a concurrent set, then `Task.Delay(window,
ct)` and return the snapshot. The `window` (`discovery.whoIsWindowMs`, default 4 s) is a **hard latency
floor** on discovery — bound it, show a spinner in the picker.
- **RP/RPM/WP/SubscribeCOV** each get a `TaskCompletionSource<T>` + `CancellationTokenSource.CancelAfter`
wrapper around the library's begin/end (or blocking) call, so per-request APDU deadlines (§8) are
enforced independently of any socket timeout — the R2-01 frozen-peer lesson (an async call must not
outlive its own wall-clock deadline just because the transport didn't time out).
- COV notifications arrive on the library's callback thread; the adapter marshals them onto the driver's
`OnDataChange` event.
### 4.2 Browse: universal in v1, bespoke-lazy P-later
**Reconciliation with `2026-07-15-universal-discovery-browser-design.md`:** BACnet browse **is** discovery
(Who-Is → object-list → properties), and `DiscoverAsync` already streams exactly the tree the picker
wants into `IAddressSpaceBuilder`. So for **small/medium sites, v1 plugs straight into the universal
`DiscoveryDriverBrowser`** — **no bespoke browser code, no `.Browser` project.** The requirements:
1. Add the opt-in flag on `ITagDiscovery` (the universal-browser design specifies this default member;
it is **not yet in `ITagDiscovery.cs`** — adding it is part of that design's P1, and BACnet consumes
it):
```csharp
bool SupportsOnlineDiscovery => false; // add to ITagDiscovery
```
`BacnetDriver` overrides it to `true`.
2. BACnet needs **no `PatchForBrowse` entry** — discovery is unconditional (unlike AbCip/TwinCAT which
are config-gated). The picker's authoring config is sufficient to open + discover.
3. The universal browser runs `InitializeAsync → DiscoverAsync → ShutdownAsync` under its open-timeout and
serves the captured tree from memory. BACnet's Who-Is window fits inside that open-timeout (default
60 s ≫ 4 s window).
The universal browser's eager, whole-tree, one-shot capture is fine for a handful of devices with modest
`object-list`s. **For large multi-device sites (dozens/hundreds of controllers, thousands of objects) the
eager one-shot is too heavy** — it Who-Ises the whole network and RPMs every device's full object-list at
open. That is the documented **P-later graduation** to a bespoke lazy browser:
> **P-later — bespoke `ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Browser`.** A hand-written `IBrowseSession`
> (registered `IDriverBrowser` with `DriverType="BACnet"`, which *overrides* the universal fallback per
> §3.1 of the universal design) with three lazy levels: `RootAsync` = bounded Who-Is → devices;
> `ExpandAsync("dev:<n>")` = that one device's `object-list` on demand; `AttributesAsync(...)` = RPM the
> object's key properties. Only build this when a real large site makes the eager capture painful.
**v1 uses the universal browser. State this clearly in the PR: BACnet ships with `SupportsOnlineDiscovery
= true` and zero browser code.**
---
## 5. TagConfig + driver-config JSON
Two layers, matching every Equipment-kind driver.
### 5.1 Driver-level config (`BacnetDriverOptions`, in `.Contracts`)
```jsonc
{
"localEndpoint": "0.0.0.0", // interface to bind; "0.0.0.0" = all
"port": 47808, // 0xBAC0 default
"localDeviceInstance": 4194302, // our own Device instance id (unique on the net)
"foreignDevice": { // null when on-segment / a BBMD is local
"bbmdAddress": "10.20.0.1",
"bbmdPort": 47808,
"ttlSeconds": 900 // registration TTL; driver auto-renews at ~TTL/2
},
"apduTimeoutMs": 3000, // per-request APDU deadline (§8)
"apduRetries": 3,
"maxSegmentsAccepted": 16, // segmentation window we advertise — do NOT set too low
"discovery": {
"whoIsWindowMs": 4000, // I-Am collect window (the time-bounded wrinkle, §4.1)
"deviceInstanceLow": 0, // optional Who-Is range filter
"deviceInstanceHigh": 4194303
},
"cov": { // P2
"preferConfirmed": true,
"lifetimeSeconds": 300, // per-subscription lifetime; renewed before expiry
"fallbackPollMs": 1000 // poll interval for devices that reject COV
},
"probe": { "enabled": true, "intervalMs": 5000 }
}
```
`BacnetDriverOptions` is a plain record/class with `[JsonStringEnumConverter]`-friendly nested option
types; the factory and probe deserialize it identically (§6). Enum knobs authored as **string names**.
### 5.2 Per-tag `TagConfig` (authored on the TagModal / emitted by the browse picker)
A BACnet tag address is fully described by **device-instance + object-type + object-instance + property-id
(+ optional array index)**. `present-value` is the default property.
```jsonc
{
"deviceInstance": 100, // target device (from I-Am)
"objectType": "AnalogInput", // BacnetObjectType enum (string name)
"objectInstance": 3, // object-instance number
"propertyId": "PresentValue", // BacnetPropertyId enum; default PresentValue
"arrayIndex": null, // non-null → element of an array property
"dataType": "Float32", // DriverDataType hint (from browse-time units/type)
"writable": false, // v1: always false; P3 enables WriteProperty
"writePriority": 8 // P3 only: BACnet command priority 1..16
}
```
### 5.3 `DriverAttributeInfo.FullName` encoding + the parser
**`FullName` is the verbatim TagConfig JSON blob** (the §5.2 object serialized to a compact string) —
exactly the Modbus convention (`ModbusEquipmentTagParser`: a leading `{` marks an equipment-tag blob).
This is what the picker commits as `TagConfig.FullName`, what discovery writes into each
`DriverAttributeInfo.FullName`, and what the driver publishes values back under, so the runtime
forward-router resolves them. There is **no separate `deviceInstance.objectType:objectInstance` string
form** — the JSON blob is the single canonical reference.
`BacnetEquipmentTagParser.TryParse(string reference, out BacnetTagAddress addr)` in `.Contracts` (copy the
Modbus parser's shape):
- Leading `{` gate; `JsonDocument.Parse`; require `deviceInstance` (number), `objectType` (enum),
`objectInstance` (number).
- **Strict enum reads** via `TagConfigJson.TryReadEnumStrict` for `objectType` / `propertyId` — a
present-but-typo'd enum **rejects** the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a
wrong object type (the R2-11 strictness lesson the Modbus parser encodes).
- `propertyId` defaults to `PresentValue` when absent; `arrayIndex` optional; `writable` defaults false
in v1.
- Produce `BacnetTagAddress(Reference: reference, DeviceInstance, ObjectType, ObjectInstance, PropertyId,
ArrayIndex, DataType)` where `Reference == reference` (the def identity, so publish keys match).
- Provide an `Inspect(reference)` warnings method (deploy-time surfacing of invalid enums / unparseable
blobs), parallel to `ModbusEquipmentTagParser.Inspect`.
---
## 6. Typed editor + validator + the enum-serialization trap
Add a driver-typed tag editor so the TagModal dispatches by `DriverType="BACnet"` instead of falling back
to the raw-JSON textarea. Copy the Modbus template pair (`Components/Shared/Uns/TagEditors/` razor shell
+ `Uns/TagEditors/*Model.cs` pure model).
1. **`BacnetTagConfigModel`** (`src/Server/.../AdminUI/Uns/TagEditors/BacnetTagConfigModel.cs`) — pure
`FromJson`/`ToJson`/`Validate`, preserving unknown keys via the `JsonObject` bag (mirror
`ModbusTagConfigModel`). Fields: `DeviceInstance:int`, `ObjectType:BacnetObjectType`,
`ObjectInstance:int`, `PropertyId:BacnetPropertyId`, `ArrayIndex:int?`, `DataType:DriverDataType`,
`Writable:bool?`, `WritePriority:int?`. `Validate()`: `deviceInstance` in 0..4194303, `objectInstance`
≥ 0, `writePriority` in 1..16 when present.
2. **`BacnetTagConfigEditor.razor`** (`Components/Shared/Uns/TagEditors/`) — thin shell binding the model;
enum dropdowns for `ObjectType` / `PropertyId` from the `.Contracts` enums.
3. **Register in both maps:**
```csharp
// TagConfigEditorMap.cs
["BACnet"] = typeof(Components.Shared.Uns.TagEditors.BacnetTagConfigEditor),
// TagConfigValidator.cs
["BACnet"] = j => BacnetTagConfigModel.FromJson(j).Validate(),
```
(Both dictionaries are `OrdinalIgnoreCase`, so casing of the key is forgiving, but keep it consistent
with `DriverType` — see §7.)
**The `JsonStringEnumConverter` enum-serialization trap (must-fix, systemic in this repo).** AdminUI
pages/models that serialize enums **numerically** while the driver factory/probe DTOs are **string-typed**
produce configs that fault the driver at runtime (documented systemic bug: driver enum-serialization).
Guard rails:
- The `BacnetTagConfigModel.ToJson` writes enums as **name strings** — use the repo's `TagConfigJson.Set`
helper (it emits enum names, as `ModbusTagConfigModel` does), never `JsonSerializer` with default enum
handling.
- `BacnetDriverFactoryExtensions.JsonOptions` **and** `BacnetDriverProbe`'s options **must** carry
`Converters = { new JsonStringEnumConverter() }` + `PropertyNameCaseInsensitive = true` +
`UnmappedMemberHandling.Skip` — identical options in both, so factory and probe parse a given config
byte-for-byte the same (copy `OpcUaClientDriverFactoryExtensions.JsonOptions`).
- Any AdminUI **driver-config** page for BACnet (the driver-level options form, when it lands) must also
register `JsonStringEnumConverter` so `ObjectType`/`PropertyId`/nested option enums round-trip as
strings.
---
## 7. Factory + registration
**`DriverType = "BACnet"`.** (Chosen casing; the editor/validator/probe/registry keys all use `"BACnet"`.
`DriverType` string comparisons in the tag-editor maps are `OrdinalIgnoreCase`, but keep every registration
site consistent to avoid the Modbus `"ModbusTcp"` vs `"Modbus"` drift that once dropped a driver to the
raw-JSON editor.)
**`BacnetDriverFactoryExtensions`** (copy `OpcUaClientDriverFactoryExtensions`):
```csharp
public static class BacnetDriverFactoryExtensions
{
public const string DriverTypeName = "BACnet";
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip,
Converters = { new JsonStringEnumConverter() },
};
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? lf = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, lf));
}
public static BacnetDriver CreateInstance(string id, string json, ILoggerFactory? lf = null) { /* deserialize BacnetDriverOptions, new BacnetDriver(...) */ }
}
```
**Wire into the two Host bootstrap sites** (`src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs`):
```csharp
// alongside the other Driver.*.Register(registry, loggerFactory) calls (~line 128-135):
Driver.Bacnet.BacnetDriverFactoryExtensions.Register(registry, loggerFactory);
// alongside the IDriverProbe TryAddEnumerable block (~line 108-114):
services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, BacnetDriverProbe>());
```
**`BacnetDriverProbe`** (`IDriverProbe`, `DriverType => "BACnet"`): deserialize `BacnetDriverOptions` with
the *same* `JsonOptions`; the "cheap connection" is a **bind + directed Who-Is** — bind the UDP socket
**on an ephemeral port** (never the configured 47808 — the runtime driver instance may already hold it
on the same node, and unicast replies come back to the probe's source port anyway; see the §2
port-sharing caveat), in foreign-device mode `Register-Foreign-Device` first, then broadcast/directed
Who-Is bounded by the probe `timeout`, return `Ok=true` with latency if ≥1 I-Am arrives, else a clear
"no BACnet devices answered / BBMD registration failed" message. Never throw — return `Ok=false` +
message (per `IDriverProbe` contract). Note the UDP-broadcast-on-CI caveat still applies to the probe
(§9).
**Probe on admin nodes.** Per the repo's split-role gotcha, `IDriverProbe` must be registered where the
AdminUI resolves `IEnumerable<IDriverProbe>` (admin-pinned), not only on driver nodes — the
`TryAddEnumerable` line above is in the shared bootstrap, which covers it, but verify on the split-role
MAIN cluster.
---
## 8. Resilience / timeout
- **Per-request APDU deadline (R2-01 frozen-peer lesson).** Every RP/RPM/WP/SubscribeCOV call runs under
`apduTimeoutMs` enforced by the adapter's `CancellationTokenSource.CancelAfter` (§4.1), **independent**
of any socket-level timeout. This is the exact class of bug R2-01 found in S7 (async read ignored socket
ReadTimeout → frozen-peer poll wedge): a BACnet device that ACKs at the BVLC layer but never returns the
APDU must surface `BadTimeout`/`BadCommunicationError` within the deadline, not wedge the read loop.
`apduRetries` bounds retry attempts for **idempotent** reads only (never writes — §3.3).
- **COV lifetime/renewal (P2).** The renew timer at ~`lifetime/2` is mandatory (§3.2). A **watchdog** on
the non-COV fallback path: if a fallback-polled object produces no successful read within N intervals,
mark it Bad and log, so a silently-dead poll doesn't masquerade as stale-Good.
- **Foreign-device TTL renewal.** In foreign-device mode a TTL-renew timer re-registers with the BBMD at
~TTL/2; a failed re-registration transitions the driver `Degraded` and the probe surfaces it. Losing the
registration silently is the top field failure mode (§2 / research §6 risk 1).
- **Segmentation.** Large `object-list` / RPM results segment across APDUs; the ela-compil client handles
windowed segment ack, but `maxSegmentsAccepted` must **not** be set too low or a big device's
object-list truncates. Default 16; document that lowering it risks truncated discovery.
- **Per-device fault scoping.** `IHostConnectivityProbe` reports per-device Running/Stopped so one dead
controller Bad-quality-fans-out only its own subtree, not the whole driver namespace (mirror Galaxy's
per-host model).
---
## 9. Test fixtures
BACnet has good open simulators; all are UDP servers we containerize on the shared docker host
(`10.100.0.35`) with a `project=lmxopcua` label, driven by `lmxopcua-fix up bacnet` (add a
`tests/.../Docker/docker-compose.yml` under a new BACnet test project, then `lmxopcua-fix sync bacnet`).
1. **`bacnet-stack` demo server (`bacserv`)** — the reference C stack's demo; configurable
device-instance + object set via env; answers Who-Is/I-Am, RP/RPM, WP, SubscribeCOV. The canonical
target. Also `fh1ch/bacstack-compliance-docker` (a ready-made DockerHub device-sim image) for the
lowest-friction single device.
2. **`mnp/bacnet-docker` (or `desolat/bacnet-docker`)** — docker-compose framework to stand up **multiple**
BACnet/IP servers + a BBMD on one host: the **multi-device Who-Is** discovery test and the
**BBMD/foreign-device topology** test (BBMD + device on one compose network; register the driver as a
foreign device across it).
3. **ela-compil `BACnet.Examples/BasicServer`** — a pure-.NET device sim embeddable **directly in the
integration-test process** (no docker): deterministic COV-notification + RP/RPM tests via **directed
(unicast) Who-Is to a known instance/address**, sidestepping broadcast entirely. Pair with
`BasicAdviseCOV` as an in-proc client oracle. This is the hermetic-unit path.
4. **YABE** (Windows GUI, same ela-compil library) — manual apples-to-apples oracle for cross-checking
reads/COV against a known-good client.
**The UDP-broadcast-on-CI gotcha.** BACnet Who-Is relies on subnet **broadcast**; bridged docker networks
and macOS-hosted CI don't broadcast cleanly. Two mitigations, exactly mirroring the existing S7/historian
live-gate discipline:
- **Hermetic unit/integration:** embedded `BasicServer` + **directed unicast Who-Is** to a known
address — no broadcast, runs on macOS/CI.
- **Broadcast / BBMD / multi-device:** an **env-gated live-integration suite** (e.g.
`BACNET_FIXTURE_ENDPOINT` present ⇒ run, absent ⇒ skip cleanly), fixture on the Linux docker host with
`network_mode: host` (or a dedicated bridge). Category `LiveIntegration` like the historian gate.
**Cross-VM reachability — be explicit about what routes:** this repo's tests execute on the dev
machine, *not* on `10.100.0.35`, and a subnet-broadcast Who-Is from an off-subnet peer **does not
route** (and bridged docker networks don't forward subnet broadcasts into containers either). So the
suite as run from the dev machine has exactly two working paths: (a) **directed unicast Who-Is** at
the fixture's host IP:port (host-mode `bacserv` answers a unicast Who-Is with a unicast I-Am to the
request's source address — verify this against the pinned `bacserv` build early, since spec-strict
stacks may broadcast the I-Am, which would never arrive); and (b) the **foreign-device/BBMD leg** —
register with a BBMD container on the docker host; BVLC `Forwarded-NPDU` relay is *unicast* and routes
across subnets, so this leg exercises real broadcast distribution cross-VM and is exactly the field
topology. A genuinely broadcast-only leg (plain Who-Is with no BBMD) can only run **on the docker host
itself** (ssh, containers sharing a compose network) — keep it a separate opt-in job. Never gate CI on
broadcast.
**Parser + map unit tests** (fully hermetic, no UDP): `BacnetEquipmentTagParser` round-trip + strictness
(typo'd `objectType` rejects), `BacnetDataTypeMap` coverage, `BacnetTagConfigModel` FromJson/ToJson
unknown-key preservation + enum-as-string emission, `TagConfigValidator["BACnet"]` bounds. Live-verify the
typed editor on docker-dev `/uns` (Razor binding bugs pass unit tests — the repo's standing rule).
---
## 10. Phasing + effort
**Overall effort: Medium-Large** — bigger than Modbus (no discovery/push there), roughly on par with or
slightly above OpcUaClient, driven by the callback→async adapter + COV lifetime machinery + BBMD.
Each phase is independently shippable (mirror how OpcUaClient landed across PRs):
- **P1 — Connect + Read + Discover (+ universal browse) + BBMD. (MVP, delivers real value.)**
`.Contracts` (`BacnetDriverOptions`, enums, `BacnetTagAddress`, `BacnetEquipmentTagParser`);
`BacnetClientAdapter` (async wrap); `BacnetDriver : IDriver, ITagDiscovery, IReadable,
IHostConnectivityProbe` with `SupportsOnlineDiscovery=true` + `RediscoverPolicy =>
DiscoveryRediscoverPolicy.Once`; RP/RPM read;
Who-Is/object-list discovery streamed to `IAddressSpaceBuilder`; **foreign-device/BBMD registration +
TTL renewal included here** (without it the driver can't reach most real sites); factory + probe +
Host registration; typed tag editor + validator. Browse works via the universal browser — **no browser
code**.
- **P2 — SubscribeCOV push.** `ISubscribable` via SubscribeCOV + lifetime-renew timer + transparent poll
fallback + fallback watchdog. The headline feature over poll-only drivers.
- **P3 — Write.** `IWritable` via WriteProperty with priority-array semantics + commandable-object
validation + relinquish. Deferred deliberately (§3.3 safety minefield).
- **P4 — History / Alarms.** `IHistoryProvider` over **Trend Log** objects (`ReadRange`); `IAlarmSource`
over intrinsic/algorithmic **event enrollment** + COV of `status-flags` → Part 9 conditions
(`DriverAttributeInfo.IsAlarm` + `MarkAsAlarmCondition`). Nice-to-have, large surface.
### Top risks (from research §6, carried forward)
1. **BBMD / foreign-device registration + UDP broadcast reachability (HIGH — the top risk).** The server
is usually not on the controllers' L2 segment, so correct foreign-device registration + TTL renewal is
load-bearing and is exactly what's hardest to reproduce in docker/CI. Mitigation: build BBMD in from
P1, test explicitly with the `bacnet-docker` multi-network compose, gate a live suite against a real
site BBMD.
2. **COV lifetime/renewal + non-COV fallback (HIGH, P2).** Subscriptions silently expire if not renewed;
some devices cap or lack COV and must degrade to polling without an operator-visible gap. Won't be
caught by unit tests — needs the live/sim COV soak (mirror the continuous-historization live-gate
discipline).
Secondary: the callback/blocking library needs a careful TCS+timeout+cancellation adapter; write
priority-array semantics (why P3 is deferred); segmentation for large object-lists (`maxSegmentsAccepted`
not too low).
---
## Appendix A — new/changed files checklist (P1)
**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Contracts/`:** `BacnetDriverOptions.cs` (+ nested
option records), `BacnetObjectType.cs`, `BacnetPropertyId.cs`, `BacnetTagAddress.cs`,
`BacnetEquipmentTagParser.cs`, `.csproj` (Core.Abstractions ref only).
**New — `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet/`:** `BacnetDriver.cs`, `BacnetClientAdapter.cs`,
`BacnetDataTypeMap.cs`, `BacnetDriverProbe.cs`, `BacnetDriverFactoryExtensions.cs`, `.csproj`
(Core.Abstractions + Core + .Contracts + NuGet `BACnet` — mirror the Modbus/OpcUaClient transport
csproj reference set).
**New — AdminUI:** `Uns/TagEditors/BacnetTagConfigModel.cs`,
`Components/Shared/Uns/TagEditors/BacnetTagConfigEditor.razor`.
**Changed:** `Directory.Packages.props` (+ `BACnet` version); the `.slnx` (add both projects);
`ITagDiscovery.cs` (+ `SupportsOnlineDiscovery` default member — shared with the universal-browser work);
`DriverFactoryBootstrap.cs` (Register + probe TryAddEnumerable); `TagConfigEditorMap.cs` +
`TagConfigValidator.cs` (`"BACnet"` entries).
**Tests:** `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Bacnet.Tests/` (parser, datatype map, adapter with fake
client), `...Driver.Bacnet.IntegrationTests/` (embedded `BasicServer` hermetic + env-gated
`LiveIntegration` for broadcast/BBMD/COV), plus AdminUI `BacnetTagConfigModel` + validator tests.
docs/plans/2026-07-15-driver-expansion-program-design.md
+211
View File
@@ -0,0 +1,211 @@
# Driver-expansion program — overall design
**Status:** program design, 2026-07-15. Umbrella for the next-driver effort: one universal
browse mechanism + six new/extended drivers (MELSEC deferred). This is the authoritative index
and the single statement of the effort's shared architecture, dependency graph, and build order.
Per-driver detail lives in the linked design docs; cross-cutting rules are stated **once here**
so the per-driver docs don't re-derive them.
## 1. Goal & scope
Extend the OtOpcUa OPC UA server's driver fleet under the existing Equipment-kind address-space
model — no new COM gateway, all in-process .NET 10 — and make more of the fleet **browsable**
from the AdminUI `/uns` address picker.
**Curated set (user, 2026-07-15):** MTConnect Agent · MQTT/Sparkplug B · MELSEC SLMP · Omron ·
BACnet/IP · SQL poll · Modbus RTU. **MELSEC is deferred** (commercial-license-forced hand-rolled
framer + the hardest addressing model on the roadmap; revisit on a Mitsubishi-shop need). The
other six are designed and scheduled below.
Alongside the new drivers, a **universal Discover-backed browser** retrofits browse to the
already-shipped fleet (AbCip, TwinCAT, FOCAS) and is the foundation every browsable new driver
builds on.
Existing fleet for context: Modbus(TCP), S7, AbCip, AbLegacy, TwinCAT, FOCAS, Galaxy,
OpcUaClient, Historian.Gateway (`src/Drivers/`).
## 2. Document map
**Program:** this file.
**Research reports** (`docs/research/drivers/`, index: [`README.md`](../research/drivers/README.md)):
- [`mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) ·
[`mqtt-sparkplug.md`](../research/drivers/mqtt-sparkplug.md) ·
[`melsec-slmp.md`](../research/drivers/melsec-slmp.md) *(deferred)* ·
[`omron.md`](../research/drivers/omron.md) ·
[`bacnet-ip.md`](../research/drivers/bacnet-ip.md) ·
[`sql-poll.md`](../research/drivers/sql-poll.md) ·
[`modbus-rtu.md`](../research/drivers/modbus-rtu.md) ·
[`00-existing-driver-browse-audit.md`](../research/drivers/00-existing-driver-browse-audit.md)
**Design docs** (`docs/plans/2026-07-15-*`):
- [`universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md) — **Wave 0**
- [`mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md)
- [`bacnet-ip-driver-design.md`](2026-07-15-bacnet-ip-driver-design.md)
- [`mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md)
- [`sql-poll-driver-design.md`](2026-07-15-sql-poll-driver-design.md)
- [`omron-driver-design.md`](2026-07-15-omron-driver-design.md)
- [`modbus-rtu-driver-design.md`](2026-07-15-modbus-rtu-driver-design.md)
Prior art the designs build on: [`2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md)
(bespoke `IBrowseSession` pattern), [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md)
(standard Equipment-kind driver), and the driver-typed tag editors design.
## 3. Shared architecture (the contract every driver in this program follows)
Every driver here is a **standard Equipment-kind driver** — an `IDriver` (lifecycle: `Initialize`
/ `Reinitialize` / `Shutdown` / `GetHealth` / footprint) composing only the capability interfaces
its backend supports, from `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`:
| Capability | Interface | This program's drivers |
|---|---|---|
| Discover | `ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder, ct)` | all (device-enumerated or authored-only) |
| Read | `IReadable` | all |
| Subscribe | `ISubscribable` | all (poll-based, except the push drivers: MQTT, BACnet COV, MTConnect's `/sample` stream) |
| Write | `IWritable` | Omron, Modbus-RTU (v1); MTConnect/MQTT/BACnet/SQL read-only in v1 |
| Probe | `IHostConnectivityProbe` | all |
| Re-discover | `IRediscoverable` | only drivers with a native backend change signal (per its doc contract) — e.g. MQTT rebirth; BACnet/SQL are discovery-capable but have no change signal and correctly omit it |
**Per-driver build pattern** (identical across the fleet — see any existing driver + the
driver-typed-editors design):
1. `ZB.MOM.WW.OtOpcUa.Driver.<Name>.Contracts` — options + tag DTOs + parser + enums (no backend
NuGet dep).
2. `ZB.MOM.WW.OtOpcUa.Driver.<Name>` — the runtime `IDriver` + capabilities.
3. `.Browser` project **only when bespoke browse is required** (see §4).
4. Factory + `DriverType` string + `Register()`, wired in the Host's `DriverFactoryBootstrap`
(factory + probe on admin nodes via `TryAddEnumerable`).
5. Typed tag editor + validator: a `<Name>TagConfigModel` (`FromJson`/`ToJson`/`Validate`) +
`TagConfigEditorMap` + `TagConfigValidator` entries.
6. Docker fixture under `tests/.../Docker/`, deployed to `10.100.0.35` via `lmxopcua-fix sync`
(the `project=lmxopcua` label is a deployment convention applied host-side — no checked-in
compose file carries a `labels:` entry today).
### 3.1 Cross-cutting rules (mandatory, stated once)
- **Enum-serialization trap** — every enum on a config/tag surface MUST serialize as **names,
never numbers**, on both sides (AdminUI page serializer **and** probe carry
`JsonStringEnumConverter`). Two shipped factory-side patterns both satisfy this — Modbus keeps
DTO fields `string?` + parses via `ParseEnum<T>` (no converter in the factory); OpcUaClient
uses enum-typed DTOs with the converter in factory **and** probe. Either is fine; a mismatch
between the authoring side and the factory side **faults the driver** at deploy. This is a
proven systemic bug — see the driver enum-serialization memory. Non-negotiable for every
driver here.
- **Per-op deadline (R2-01 frozen-peer lesson)** — every network call (read/write/discover/probe/
subscribe/agent-HTTP/SQL-query) MUST have a bounded deadline (linked-CTS / socket ReadTimeout /
`CommandTimeout`). A frozen peer must never wedge a poll. No unbounded waits anywhere.
- **`WriteIdempotent`** — writable drivers flag only tags whose replay is safe. Note the invoker
seam (`ExecuteWriteAsync(…, isIdempotent, …)`) exists but the host's write dispatch currently
hardcodes `isIdempotent: false` (the arch-review "hardcoded isIdempotent" Medium) — so **no
write auto-retries today**; drivers carry the per-tag flag now and retry activates when that
fleet plumbing lands.
- **Secrets** — connection strings / API keys / broker creds come from env/secret refs (mirror
`ServerHistorian__ApiKey`), never committed or logged.
- **Live-verify discipline** — Razor binding + deploy-inertness bugs pass unit tests and review;
every driver's picker/editor and deploy path gets a docker-dev `/run` live-verify before it's
trusted.
## 4. Browse architecture — the two-tier decision
The program's central browse decision: instead of a bespoke browser per driver, a **universal
Discover-backed browser** (`DiscoveryDriverBrowser`, Wave 0) captures any driver's
`ITagDiscovery.DiscoverAsync` output and re-presents it to the picker. Bespoke `IBrowseSession`
browsers are reserved for the cases the universal one genuinely can't serve.
**The gate:** a new default-interface member `ITagDiscovery.SupportsOnlineDiscovery => false`
(mirrors the existing `RediscoverPolicy` default member). A driver opts in when its `DiscoverAsync`
**enumerates from the device**; flat-address drivers leave it false and stay manual-entry. This
member is **added in the universal browser's P1** — **every browsable driver in this program
depends on it**, which is why the universal browser is the Wave-0 gate.
| Driver | Browse mechanism | Verdict |
|---|---|---|
| **MTConnect** | `/probe` device model via `DiscoverAsync` | **Universal** (`SupportsOnlineDiscovery=true`) |
| **BACnet/IP** | Who-Is + object-list via `DiscoverAsync` | **Universal** for small sites; bespoke-lazy `IBrowseSession` a documented P-later for large multi-device sites |
| **AbCip** *(existing)* | CIP Symbol 0x6B/0x6C in `DiscoverAsync` | **Universal** (+ `PatchForBrowse` `EnableControllerBrowse:true`) |
| **TwinCAT** *(existing)* | ADS symbol upload in `DiscoverAsync` | **Universal** (+ patch); bespoke lazy only if symbol set too big |
| **FOCAS** *(existing)* | FixedTree curated discovery | **Universal** (curated; + patch `FixedTree.Enabled:true` + an `UntilStable` settle — its tree fills post-connect, so a one-shot capture would be empty) |
| **MQTT/Sparkplug** | passive topic/birth observation window | **Bespoke** `MqttBrowseSession` — runtime discovery is authored-only, so universal would replay only authored tags |
| **SQL poll** | `INFORMATION_SCHEMA` schema walk | **Bespoke** `SqlBrowseSession` — schema walk ≠ `DiscoverAsync` |
| **Omron** | libplctag `@tags` unsupported on NJ/NX | **None live** — offline Sysmac tag-export importer (P-later); FINS never browsable |
| **Modbus RTU** | flat registers, no discovery | **None** — not browsable |
| S7, AbLegacy, Modbus(TCP) *(existing)* | flat / no on-wire symbols | **None** — documented in the audit |
Bespoke browsers **override** the universal fallback simply by being registered for their
`DriverType` (`BrowserSessionService` resolves bespoke-first). See the universal-browser design §3
for the required `BrowserSessionService` fallback change (the `ToDictionary` dup-key constraint
means the universal browser is a **separate `IUniversalDriverBrowser`**, not one of the injected
`IDriverBrowser` set).
## 5. Library & license decisions
| Driver | Library | License | Note |
|---|---|---|---|
| MTConnect | TrakHound MTConnect.NET (`-Common` + `-HTTP`) | MIT | netstd2.0 → loads on net10; pin + license-review; hand-roll fallback |
| BACnet/IP | System.IO.BACnet (ela-compil fork, `BACnet`) | MIT | multi-targets net10; YABE engine; callback API → TCS adapter |
| MQTT/Sparkplug | MQTTnet v5 + **hand-rolled Tahu protobuf** | MIT | **Not SparkplugNet** — its MQTTnet-4.x transitive pin collides with the repo's deliberately-OFF transitive pinning (Roslyn-split constraint) |
| Omron (CIP) | libplctag.NET | MPL-2.0 | same dep AbCip already ships (`plc=omron-njnx`); FINS hand-rolled |
| SQL poll | Microsoft.Data.SqlClient | — | already in-repo at 6.1.1 — zero new deps for P1 |
| Modbus RTU | extend existing Modbus | — | serial impls behind the **existing** `IModbusTransport` seam; P1 (RTU-over-TCP) zero new deps, P2 direct-serial adds `System.IO.Ports` |
| ~~MELSEC~~ | ~~hand-rolled 3E~~ | — | deferred; HslCommunication is a commercial-license blocker |
**Avoid HslCommunication** (commercial license; the "MIT" claim is a stale abandoned fork) for both
MELSEC and Omron.
## 6. Dependency graph & build order
```
Wave 0 ─ Universal browser ── adds ITagDiscovery.SupportsOnlineDiscovery ──┐
│ (DiscoveryDriverBrowser + CapturingAddressSpaceBuilder + │ every browsable
│ BrowserSessionService fallback) │ driver depends
│ │ on this member
├── immediately lights up AbCip / TwinCAT / FOCAS pickers (existing) │
▼ │
Wave 1 ─ Modbus RTU (extend, lowest effort) · SQL poll (bespoke schema browser) ◄┘
▼
Wave 2 ─ MTConnect (universal browse) · MQTT/Sparkplug (bespoke observation browser)
▼
Wave 3 ─ BACnet/IP (universal browse + COV) · Omron (CIP-first; last — no NJ/NX CIP sim)
```
| Wave | Item | Browse | Write v1 | Effort |
|---|---|---|---|---|
| 0 | Universal `DiscoveryDriverBrowser` | — | — | S–M (~3–5 d) |
| 1 | Modbus RTU | none | Yes | **S (lowest)** |
| 1 | SQL poll | bespoke schema | No | S–M |
| 2 | MTConnect Agent | universal | No | S–M |
| 2 | MQTT/Sparkplug B | bespoke observation | No | M–L |
| 3 | BACnet/IP | universal (+ bespoke-lazy P-later) | No | M–L |
| 3 | Omron | offline import | Yes | M–L |
**Sequencing rationale:** Wave 0 first because the `SupportsOnlineDiscovery` seam gates every
browsable driver and it retrofits browse to three shipped drivers for near-zero marginal cost.
Wave 1 is the low-effort/high-leverage pair (RTU extends an existing driver; SQL adds zero deps).
Wave 2 is the strategic telemetry/UNS pair. Wave 3 is the heavier/test-gated pair — Omron last
because there is **no free NJ/NX CIP simulator**, so its wire correctness is a live-hardware gate.
MELSEC is out of the sequence (deferred).
## 7. Testing & fixtures
- Fixtures live under `tests/.../Docker/`, deployed to the shared host `10.100.0.35` via
`lmxopcua-fix sync` (which owns the `project=lmxopcua` labelling host-side). Per-driver sims: `mtconnect/cppagent`, Mosquitto/EMQX +
Sparkplug simulator, `bacnet-stack`/ela-compil `BasicServer`, central SQL Server
(`10.100.0.35,14330`) + seeded table, `rtu_over_tcp` pymodbus profile + socat serial pair.
- **Env-gated live suites** for what a sim can't cover: BACnet Who-Is/BBMD UDP broadcast, Omron CIP
wire (live hardware), MQTT broker soak.
- Per driver: unit (framer/parser/type-map with golden vectors + fakes) → integration (docker
fixture) → **live `/run`** on docker-dev (picker + editor + deploy path).
## 8. Deferred / out of scope
- **MELSEC SLMP** — deferred (§1). Research complete (`melsec-slmp.md`); revisit on demand.
- **Write-back** for MTConnect (Interfaces), MQTT (NCMD/DCMD), BACnet (WriteProperty priority
array), SQL (parameterized UPSERT) — all documented as later phases in the respective designs.
- **Bespoke lazy browsers** for large BACnet sites and large ControlLogix/TwinCAT symbol sets —
graduate from universal only when eager one-shot discovery proves too heavy.
- Shared `Cip.Core` extraction (AbCip ↔ Omron CIP) — a follow-up, not a gate; keep Omron
self-contained in v1.
## 9. Status
Research: **done** (8 reports). Designs: **done** (7 design docs). Code: **not started.**
Next action: execute Wave 0 (universal browser), then Wave 1.
docs/plans/2026-07-15-modbus-rtu-driver-design.md
+380
View File
@@ -0,0 +1,380 @@
# Modbus RTU (serial + RTU-over-TCP) — implementation design
**Status:** Design / build-ready. Not implemented.
**Date:** 2026-07-15
**Scope:** Add Modbus **RTU** transport modes (direct serial + RTU-over-TCP) to the
existing `ModbusDriver`.
**Research input:** [`docs/research/drivers/modbus-rtu.md`](../research/drivers/modbus-rtu.md).
**Related:** [`docs/drivers/Modbus.md`](../drivers/Modbus.md),
[`docs/v2/modbus-addressing.md`](../v2/modbus-addressing.md),
[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
---
## 1. Motivation + extend-vs-new verdict — **EXTEND**
Add Modbus RTU as **two new `IModbusTransport` implementations behind the existing
transport seam + driver-level config plumbing to select them.** Do **not** create a
sibling driver. The Modbus application protocol above the wire (register model, function
codes FC01–06/15/16, exception-PDU convention, data-type codecs, byte order, arrays,
strings, BCD, bit-in-register, read planner + coalescing, auto-prohibit, deadband,
write path, connectivity probe, OPC-UA materialisation, HistoryRead) is **byte-for-byte
identical** across TCP and RTU — only the wire framing and the physical link differ. A
second driver would clone that entire surface for zero benefit.
### The exact seam this plugs into
The driver already splits the *PDU* (function code + data) from the *transport*
(framing + link I/O) behind one interface:
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/IModbusTransport.cs`
```csharp
public interface IModbusTransport : IAsyncDisposable
{
Task ConnectAsync(CancellationToken ct);
Task<byte[]> SendAsync(byte unitId, byte[] pdu, CancellationToken ct); // pdu = [fc, ...data], returns response PDU
}
```
- **`ModbusDriver`** builds every request as a raw PDU `byte[]` and calls
`transport.SendAsync(ResolveUnitId(tag), pdu, ct)` (e.g. `ModbusDriver.cs` lines
~1014/1035/1052/1069/1142/1154). It never touches MBAP, sockets, or CRC.
- **`ModbusTcpTransport`** is the *only* place MBAP framing, the transaction-id counter
(`_nextTx`), and `TcpClient`/`NetworkStream` I/O live — specifically `SendOnceAsync`
(`ModbusTcpTransport.cs` lines 231–296) wraps the PDU as
`[TxId(2)][Proto=0(2)][Length(2)][UnitId(1)] + PDU`, single-flights via `SemaphoreSlim
_gate`, and owns reconnect/keepalive/idle-disconnect.
- **The transport is injected**: `ModbusDriver`'s ctor takes
`Func<ModbusDriverOptions, IModbusTransport>? transportFactory`
(`ModbusDriver.cs` lines 98–114); the default closure builds `ModbusTcpTransport`.
Tests already substitute in-memory fakes through this hook.
So the work is **purely additive**: two new transport classes + a CRC-16 helper +
config/factory selection + one AdminUI panel. **Zero** changes to codecs, planner,
coalescing, health, materialisation, HistoryRead, or the address parser.
### New/changed files at a glance
| File | Change |
|---|---|
| `…/Driver.Modbus/ModbusCrc.cs` | **new** — CRC-16 (poly `0xA001`) helper |
| `…/Driver.Modbus/ModbusRtuFraming.cs` | **new** — shared ADU frame/deframe + FC-aware response sizing |
| `…/Driver.Modbus/ModbusRtuTransport.cs` | **new** — `System.IO.Ports.SerialPort` transport |
| `…/Driver.Modbus/ModbusRtuOverTcpTransport.cs` | **new** — RTU framing over a socket |
| `…/Driver.Modbus/ModbusSocketLifecycle.cs` | **new** (refactor) — socket connect/reconnect/keepalive/idle extracted from `ModbusTcpTransport`, shared by TCP + RtuOverTcp |
| `…/Driver.Modbus/ModbusTransportFactory.cs` | **new** — `Create(ModbusDriverOptions)` switch on `Transport`; used by the driver default closure **and** the probe |
| `…/Driver.Modbus.Contracts/ModbusDriverOptions.cs` | add `Transport` + serial fields (+ two enums) |
| `…/Driver.Modbus/ModbusDriverFactoryExtensions.cs` | DTO fields + wire the default closure to `ModbusTransportFactory.Create` |
| `…/Driver.Modbus/ModbusDriverProbe.cs` | build the transport via `ModbusTransportFactory` (currently hardcodes `new ModbusTcpTransport`, line 77) |
| `…/Driver.Modbus/ModbusDriver.cs` | `BuildSlaveHostName` endpoint string reflects the transport (COM/gateway vs host:port) |
| `…/AdminUI/…/Drivers/ModbusDriverPage.razor` | serial-parameter panel shown when `Transport != Tcp` |
`System.IO.Ports` (10.0.x for .NET 10) becomes a new PackageReference on
`ZB.MOM.WW.OtOpcUa.Driver.Modbus`.
---
## 2. Transport implementations
Both new transports produce the **RTU ADU**: `[slaveAddress(1)][PDU][CRC-lo][CRC-hi]` —
no MBAP header, no transaction id. CRC-16 (poly `0xA001` reflected, appended
**low byte first**) replaces TCP's transport-level integrity. The RTU framing/deframing
logic is identical between the two; factor it into `ModbusRtuFraming` so the serial and
socket transports share it and differ only in the byte-stream they read/write.
### 2a. `ModbusRtuFraming` (shared)
- `byte[] BuildAdu(byte unitId, ReadOnlySpan<byte> pdu)` → `[unit][pdu][crcLo][crcHi]`.
- `int ExpectedResponseLength(byte functionCode)` / a streaming reader that determines
response length **by parsing the function code** (RTU frames are length-less — see
§7). Read `addr(1)` + `fc(1)` first, then:
- **Exception** (`fc & 0x80`): read `excCode(1) + CRC(2)` → 5-byte ADU total; throw
`ModbusException(fc & 0x7F, excCode, …)` after CRC-validating.
- **Read responses** FC01/02/03/04: read `byteCount(1)`, then `byteCount + CRC(2)`.
- **Write echoes** FC05/06/15/16: fixed `addr + qty/value(4) + CRC(2)` → read 6 more.
- After the full ADU is in hand: validate the trailing CRC-16 (mismatch →
`ModbusTransportDesyncException(DesyncReason.…)` so it maps onto the existing
`BadCommunicationError` handling), strip the address byte + CRC, and return the bare
response PDU `[fc, ...data]` — exactly what `SendAsync`'s callers already decode.
### 2b. `ModbusRtuTransport : IModbusTransport` (direct serial)
- Wraps a `System.IO.Ports.SerialPort`. `ConnectAsync` opens the port with the
configured `PortName`/`BaudRate`/`DataBits`/`Parity`/`StopBits`; sets
`ReadTimeout`/`WriteTimeout` from `ModbusDriverOptions.Timeout`.
- **Per-op deadline (R2-01): `SerialPort.ReadTimeout` only governs the synchronous read
API — async reads over `SerialPort.BaseStream` ignore `ReadTimeout`/`WriteTimeout`
entirely** (the exact wall-clock gap class R2-01 found in the S7 driver, where async
socket reads ignored `ReadTimeout` and a frozen peer wedged the poll). Every
transaction MUST therefore run under a linked-CTS
`CancellationTokenSource.CancelAfter(Options.Timeout)` deadline, mirroring
`ModbusTcpTransport.SendOnceAsync` — timeout ⇒ close the port + surface
`ModbusTransportDesyncException(DesyncReason.Timeout)`, distinct from caller
cancellation (no teardown).
- `SendAsync`: single-flight via the same `SemaphoreSlim _gate` pattern (**mandatory** on
RTU — no TxId means at most one transaction on the bus at a time). Enforce ≥3.5-char
inter-frame idle before transmit (computed from baud/word-length, or the
`InterFrameDelayMs` override), write the ADU, then read the response using
`ModbusRtuFraming`'s FC-aware sizing, backstopped by the linked-CTS per-op deadline.
- Failure model is **simpler than TCP** — a COM port doesn't "drop" the way a NAT'd
socket does. On an I/O error or CRC desync, close + reopen the port once and retry
(reuse the single-retry shape from `ModbusTcpTransport.SendAsync`); no keepalive, no
idle-disconnect, no geometric socket backoff.
### 2c. `ModbusRtuOverTcpTransport : IModbusTransport` (RTU tunnelled over a socket)
- **Identical RTU framing** (`ModbusRtuFraming`) but the byte stream rides a
`TcpClient`/`NetworkStream` to a serial→Ethernet gateway instead of a COM port.
- **Reuses the hardened socket lifecycle** — extract the connect (IPv4-preference DNS),
`SO_KEEPALIVE`, idle-disconnect, and reconnect-with-backoff machinery from
`ModbusTcpTransport` into `ModbusSocketLifecycle` and have **both** TCP variants
compose it. The *only* delta from `ModbusTcpTransport` is: CRC framing instead of MBAP,
and no transaction id. This is the lowest-risk of the two RTU transports to build and
test (see §9).
**Modbus ASCII is out of scope** — a third, rarely-used framing (`:`-delimited hex,
LRC instead of CRC). It would be another `IModbusTransport` behind the same seam if ever
needed; not planned.
---
## 3. Cross-platform serial reality
- **`System.IO.Ports.SerialPort` is cross-platform** on .NET 10 (`System.IO.Ports`
10.0.x): Windows + Linux (`/dev/tty*` via termios). **macOS is the weak platform**
(baud quirks, `MacCatalyst` unsupported) — this repo's dev machine is macOS, so
**direct-serial cannot be exercised on the dev Mac**; use fakes / RTU-over-TCP there
and run the real-serial suite on the Linux docker host / CI (§8).
- **Containers add a device-mapping hurdle:** a serial device must be explicitly passed
in (`docker run --device=/dev/ttyUSB0` or a compose `devices:` entry), and USB-serial
adapters re-enumerate (`ttyUSB0` ↔ `ttyUSB1`) across replug — pin a stable
`/dev/serial/by-id/...` path. Windows-container COM passthrough is unreliable.
**RTU-over-TCP is the recommended primary path.** OtOpcUa deploys as a containerised
Linux server (docker-dev rig; docker host `10.100.0.35`) that is generally **not**
attached to an RS-485 bus. The idiomatic topology is a **serial→Ethernet gateway** (Moxa
NPort, Digi One, Lantronix, USR-TCP232) on the RS-485 multidrop, exposed over TCP. The
server talks **RTU-over-TCP** to it — a plain socket, **zero host-device mapping, no
`System.IO.Ports` dependency at runtime, no udev fragility**, and it reuses the already-
hardened socket lifecycle. Direct serial ships too, for bare-metal Windows/Linux installs
with a local/adapter COM port, but RTU-over-TCP is what most deployments will use.
---
## 4. Config JSON shape
**Per-tag `TagConfig` is unchanged.** `ModbusTagDefinition` / `ModbusTagDto` already carry
everything RTU needs — including the per-tag **`UnitId`** override that makes an RS-485
multi-drop bus "just work" (the read planner already refuses to coalesce across UnitIds;
`ResolveUnitId` + `BuildSlaveHostName` already key per-slave resilience by unit). The
`ModbusTagConfigEditor` in `TagConfigEditorMap` needs **no change**. Additions are
**driver-level only**.
New fields on `ModbusDriverOptions` (and the matching optional fields on
`ModbusDriverConfigDto`):
| Field | Type | Applies to | Notes |
|---|---|---|---|
| `Transport` | `ModbusTransportMode` enum (`Tcp`\|`Rtu`\|`RtuOverTcp`) | all | **Default `Tcp`** (back-compat: existing configs omit it) |
| `SerialPort` | string | Rtu | `"COM3"` / `"/dev/ttyUSB0"` / `by-id` path |
| `BaudRate` | int | Rtu | 9600 / 19200 / 38400 / 115200 |
| `DataBits` | int | Rtu | usually 8 |
| `Parity` | `Parity` enum (`None`\|`Even`\|`Odd`) | Rtu | spec default **Even**; many devices use None |
| `StopBits` | `StopBits` enum (`One`\|`Two`) | Rtu | 1 with parity, 2 without |
| `InterFrameDelayMs` | int? | Rtu | optional override of computed T3.5 for slow/RF links |
| `Host` / `Port` | reused | Tcp, **RtuOverTcp** | the gateway's socket for RtuOverTcp |
`Host`/`Port`/`UnitId`/`TimeoutMs`/`MaxRegistersPerRead`/keepalive/reconnect all stay.
Serial fields are ignored when `Transport=Tcp`; `Host`/`Port` are ignored when
`Transport=Rtu`. Keepalive/idle/reconnect apply to `Tcp` + `RtuOverTcp` only.
> Serialization note: `Parity`/`StopBits` names collide with
> `System.IO.Ports.Parity`/`StopBits` (member names match the table), but **define local
> enums in `Driver.Modbus.Contracts`** and map them to the BCL enums inside
> `ModbusRtuTransport`. Reusing the BCL enums directly would put the `System.IO.Ports`
> package on the Contracts project (and transitively on the AdminUI, which deserializes
> `ModbusDriverOptions`) — contradicting §1's plan to add the dependency to
> `Driver.Modbus` only, and the program-doc rule that Contracts carries no backend
> NuGet dep.
### Example A — direct serial RTU (multi-drop)
```json
{
"transport": "Rtu",
"serialPort": "/dev/ttyUSB0",
"baudRate": 19200,
"dataBits": 8,
"parity": "Even",
"stopBits": "One",
"unitId": 1,
"timeoutMs": 1000,
"tags": [
{ "name": "Flow", "addressString": "40001:F:ABCD", "writable": false },
{ "name": "Setpt", "addressString": "40010:F", "writable": true },
{ "name": "Pump2Run", "region": "Coils", "address": 0, "dataType": "Bool",
"writable": true, "unitId": 2 }
]
}
```
`Pump2Run` is a second drop slave (UnitId 2) on the same bus — no extra transport config,
just the per-tag `unitId` override the driver already honours.
### Example B — RTU-over-TCP to a serial→Ethernet gateway
```json
{
"transport": "RtuOverTcp",
"host": "10.20.0.50",
"port": 4001,
"unitId": 1,
"timeoutMs": 1500,
"tags": [
{ "name": "Temp", "addressString": "30001:I" },
{ "name": "Alarm", "region": "DiscreteInputs", "address": 5, "dataType": "Bool" }
]
}
```
Same `host`/`port` shape as Modbus TCP, but the wire frames are raw RTU (address + CRC,
no MBAP). The `Transport` discriminator **must match the gateway's configured mode**:
`"Tcp"` for a gateway doing Modbus/TCP translation, `"RtuOverTcp"` for
transparent/RTU-passthrough. The two wire formats are mutually unparseable.
---
## 5. The `JsonStringEnumConverter` trap — **must handle**
Per the project-wide enum-serialization bug (memory: *Driver enum-serialization bug*),
driver pages serialize enums but factory DTOs are string-typed. The new
`Transport`/`Parity`/`StopBits` enums **must round-trip as strings**, or an AdminUI-
authored RTU config faults the driver at deploy.
Good news — the plumbing is **already correct on both ends** and just needs the new fields:
- `ModbusDriverPage.razor`'s serializer (`_jsonOpts`, line 328–334) already has
`Converters = { new JsonStringEnumConverter() }` + camelCase. It serializes a
`ModbusDriverOptions` directly, so a `ModbusTransportMode`/`Parity`/`StopBits` on
`ModbusDriverOptions` emits as `"Rtu"`/`"Even"`/`"One"` automatically.
- `ModbusDriverFactoryExtensions` DTO fields are typed `string?` and parsed via the
existing `ParseEnum<T>` helper (case-insensitive) — mirror how `Family` /
`MelsecSubFamily` are already handled. **Do not** type the DTO fields as the enum.
- `ModbusDriverProbe._opts` (line 19–24) already carries `JsonStringEnumConverter`.
Add a driver-page/factory round-trip unit test asserting `"transport":"Rtu"` (string, not
`1`) — the same guard that caught S7/Modbus previously.
---
## 6. Capability mapping — identical to TCP, four deltas
Same register model, function codes, data types, read/write semantics, coalescing,
deadband, `WriteOnChangeOnly`, connectivity probe (FC03@0). **Read + write both fully
supported.** The only deltas are below the seam:
| Delta | TCP (today) | RTU (added) |
|---|---|---|
| Physical link | `TcpClient` | `SerialPort` (Rtu) / `TcpClient` (RtuOverTcp) |
| Framing | 7-byte MBAP + TxId | `[addr][PDU][CRC-16]`, no TxId |
| Integrity | TCP guarantees | app-level CRC-16 (`0xA001`) |
| Unit/slave id | often 1 | **central** — one bus, many drops by unit id (already supported per-tag) |
| Response length | MBAP `Length` field | **no length field** → FC-aware sizing (§7) |
| Timing | none | ≥3.5-char inter-frame silence; single-flight mandatory |
`ResolveHost` / `BuildSlaveHostName` (`ModbusDriver.cs` line 162,
`"{host}:{port}/unit{n}"`) should format the per-slave resilience key from the active
endpoint — `COMx/unit{n}` (Rtu) or `gatewayHost:port/unit{n}` (RtuOverTcp) — so
per-slave breakers stay distinct.
### Browseability — **NO** (reconcile w/ universal browser)
Modbus RTU is **not browsable** — a flat, untyped register space with no discovery
protocol, identical to Modbus TCP. `SupportsOnlineDiscovery=false`; the driver keeps
`RediscoverPolicy = Once` and materialises the authored tag list into a flat folder. No
browser (universal or bespoke). The AdminUI `ModbusAddressPickerBody`/`ModbusAddressBuilder`
is an address *builder* (grammar helper), not a live browser, and stays valid for RTU.
---
## 7. Resilience / timeout
- **Per-op deadline + T3.5 framing.** The linked-CTS `CancelAfter(Options.Timeout)`
per-op deadline (§2b — **not** `SerialPort.ReadTimeout`, which async reads ignore) is
the hard backstop; FC-aware sizing is the primary length signal,
with the inter-frame idle gap as the delimiter fallback. Above 19200 baud the spec
fixes T3.5≈1.75 ms / T1.5≈750 µs rather than scaling further; below it,
T3.5 ≈ 3.5 × (bits-per-char / baud) (e.g. 9600 8-N-1 ≈ 3.6 ms). `InterFrameDelayMs`
overrides for slow/long RS-485 or RF links.
- **The one genuinely new correctness risk: RTU response sizing without a length field.**
Get the FC-aware calculation + exception-PDU short-frame detection right (§2a), or the
read hangs to timeout / mis-frames. Cover exhaustively with fake-stream unit tests
(below). A CRC mismatch or truncated frame maps to
`ModbusTransportDesyncException` → the existing single reconnect-retry + status mapping.
- **RtuOverTcp reuses the socket lifecycle** — keepalive, idle-disconnect, reconnect
backoff, IPv4-preference connect — unchanged from `ModbusTcpTransport` via the extracted
`ModbusSocketLifecycle`. **Direct serial does not** use any of that (a COM port has no
NAT reaping); it uses the simpler close/reopen-once-on-error model.
- **Single-flight is mandatory on RTU** (no TxId to correlate an interleaved response) —
the existing `_gate` semaphore already provides it; keep it in both new transports.
---
## 8. Test fixtures
1. **RTU-over-TCP against pymodbus (highest ROI, no serial hardware).** Add an
`rtu_over_tcp` profile to the existing fixture
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Docker/docker-compose.yml`
(alongside `standard`/`dl205`/`mitsubishi`/`s7_1500`/`exception_injection`), running
`pymodbus.simulator` in **RTU-framed-TCP** mode. The existing profiles all share host
port `:5020` and are mutually exclusive by design; bind `rtu_over_tcp` to a fresh port
so it can co-run with `standard` (and sidestep the known shared-port stale-container
trap). Add the `project=lmxopcua` label per the program-doc fixture convention (the
checked-in compose services carry no `labels:` entries today). Drive it via
`lmxopcua-fix up modbus rtu_over_tcp` + `lmxopcua-fix sync modbus`. This exercises the
real CRC + FC-aware framing end-to-end with **no serial anything** — the only
genuinely new logic.
2. **Virtual serial pair for the direct-serial transport.** `socat -d -d pty,raw,echo=0
pty,raw,echo=0` creates a linked `/dev/pts/N` ↔ `/dev/pts/M` pair; point a pymodbus
**RTU serial** slave at one end and `ModbusRtuTransport` at the other. (`com0com` is
the Windows equivalent.) Runs on the Linux docker host / CI, **not** the dev Mac.
Defer to an **env-gated live suite** (like the other driver live gates).
3. **`diagslave`** (serial + TCP RTU) for manual/soak against a virtual pair or a real
USB-serial adapter — eventual live gate, not unit CI.
**Unit-level (no PLC):**
- `ModbusCrc` table-driven test against known Modbus CRC vectors.
- `ModbusRtuFraming` / `ModbusRtuTransport` / `ModbusRtuOverTcpTransport` against an
**in-memory duplex stream fake** (fake one level below the existing `IModbusTransport`
fakes — the byte stream) asserting ADU build, CRC append/validate, FC-aware response
parse for each FC group, and exception-PDU short-frame handling.
- Driver-page/factory round-trip test for the `JsonStringEnumConverter` guard (§5).
**Recommended CI shape:** unit (CRC + framing + config round-trip) + the `rtu_over_tcp`
pymodbus profile for integration; real-serial (socat / hardware) env-gated live.
---
## 9. Phasing + effort
**Effort: LOW — likely the lowest-effort item on the driver roadmap.** The seam is
purpose-built; net-new code is small and localised.
- **P0 — shared plumbing:** `ModbusCrc` (~30 lines) + `ModbusRtuFraming` (FC-aware sizing)
+ extract `ModbusSocketLifecycle` from `ModbusTcpTransport` (mechanical refactor;
`ModbusTcpTransport` keeps behaviour) + `ModbusTransportFactory.Create` + config fields
on `ModbusDriverOptions`/DTO + factory closure + probe wiring. Unit tests for CRC +
framing + config round-trip.
- **P1 — `ModbusRtuOverTcpTransport`** (reuses the socket lifecycle; delta = CRC framing +
FC-aware length, no TxId) + `rtu_over_tcp` pymodbus docker profile + AdminUI serial panel
wired for the RtuOverTcp subset. **Ship this first** — no host-device dependency,
testable on the existing pymodbus harness, reuses hardened socket code.
- **P2 — `ModbusRtuTransport`** (direct serial) + `System.IO.Ports` PackageReference +
socat/diagslave env-gated live suite. For bare-metal installs.
**Top risk:** serial framing/timing on Linux/in-containers (best-effort T1.5/T3.5 gating;
device enumeration; the length-less-frame response-sizing correctness). Mitigated by
leading with RTU-over-TCP (P1), exhaustive fake-stream framing tests, and the
`InterFrameDelayMs` override for slow links. Direct-serial (P2) is the residual-risk
piece and is deferred behind the env-gated live suite.
docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md
+412
View File
@@ -0,0 +1,412 @@
# MQTT / Sparkplug B driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Research input:** `docs/research/drivers/mqtt-sparkplug.md` (verdicts + protocol/library survey — this doc turns that into a build plan; it does not re-litigate the research).
**Related:** `docs/plans/2026-07-15-universal-discovery-browser-design.md` (why MQTT needs a *bespoke* browser despite the universal one — §4), `docs/plans/2026-06-12-galaxy-standard-driver-design.md` (standard Equipment-kind driver shape + fire-and-forget write precedent), `docs/plans/2026-05-28-driver-browsers-design.md` (`IDriverBrowser`/`IBrowseSession` pattern reused by the observation browser).
---
## 1. Motivation + scope
Build a standard **Equipment-kind** driver — same composable-`IDriver` shape as Modbus / S7 / AbCip / TwinCAT / FOCAS / OpcUaClient — that ingests **plain MQTT** and **Sparkplug B** into the OtOpcUa OPC UA / UNS address space.
- OtOpcUa is a **Unified-Namespace product**, and **Sparkplug B is the de-facto UNS-over-MQTT payload standard** (mandated topic namespace + protobuf schema + birth-certificate discovery + stateful sessions). Sparkplug alignment is therefore the strategic headline; **plain MQTT/JSON** is the broad-compatibility fallback for brokers that carry raw JSON topics.
- This is a **subscribe-first** driver. Unlike the polled drivers (Modbus/S7/AbCip/FOCAS with `PollGroupEngine`), it holds a **live broker connection** for its whole lifetime and raises `OnDataChange` from the MQTT receive callback. The closest existing analog is the native-push side of `OpcUaClientDriver`.
- **In scope (this design):** connect/TLS/auth/reconnect; topic + Sparkplug subscribe → `OnDataChange`; last-value `IReadable`; authored-only `ITagDiscovery`; `IRediscoverable`; `IHostConnectivityProbe`; a **bespoke observation browser**; typed AdminUI tag editor + validator; Docker fixtures + env-gated live suite.
- **Out of scope for v1:** write-through (`IWritable` — Sparkplug NCMD/DCMD + plain publish) is deferred to phase 3 (§3.4, §10); `DataSet`/`Template` Sparkplug metrics; `Bytes`/`File` metrics beyond a raw-String fallback.
Canonical **DriverType string: `"Mqtt"`** — one string used identically across the AdminUI driver page, probe, factory, editor map, validator, and browser. (Heed the Modbus `"ModbusTcp"` vs `"Modbus"` mismatch lesson: a divergent string silently drops tags to the raw-JSON editor / faults resolution. Pick `"Mqtt"` and grep to enforce it.)
---
## 2. Project layout
Three new projects under `src/Drivers/`, mirroring the OpcUaClient triad (`.Driver` + `.Contracts` + `.Browser`). All `net10.0`, `Nullable`+`ImplicitUsings` enabled, `TreatWarningsAsErrors=true` opted in per-csproj (mandatory for new projects — `Directory.Build.props` deliberately does **not** set it globally because legacy test projects would fail the build).
```
src/Drivers/
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ # options + tag DTOs + parser — zero transport deps
MqttDriverOptions.cs # driver-config DTO (broker conn + mode + sparkplug/plain sub-objects)
MqttMode.cs, MqttPayloadFormat.cs # enums (Plain|SparkplugB ; Json|Raw|Scalar)
MqttTagDefinition.cs # parsed per-tag descriptor (plain OR sparkplug variant)
MqttEquipmentTagParser.cs # TagConfig JSON → MqttTagDefinition (mirrors ModbusEquipmentTagParser)
SparkplugDataType.cs # Sparkplug metric-datatype enum + → DriverDataType map
ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ # runtime driver — references .Contracts + Core.Abstractions + Core
MqttDriver.cs # IDriver, ITagDiscovery, ISubscribable, IReadable,
# IHostConnectivityProbe, IRediscoverable (+ IWritable phase 3)
MqttDriverProbe.cs # IDriverProbe — broker CONNECT handshake
MqttDriverFactoryExtensions.cs # DriverTypeName="Mqtt"; Register(registry, loggerFactory)
MqttConnection.cs # MQTTnet client wrapper: connect/TLS/auth/reconnect/backoff
LastValueCache.cs # FullReference → last DataValueSnapshot (seeds IReadable)
Sparkplug/
SparkplugCodec.cs # Tahu-proto decode (Payload/Metric); encode NCMD (phase 3)
AliasTable.cs # per edge-node/device alias→(name,datatype); rebuilt each birth
BirthCache.cs # NBIRTH/DBIRTH metric catalog (name/alias/datatype/last value)
SequenceTracker.cs # seq (0-255 wrap) gap detection; bdSeq death→birth tie
RebirthRequester.cs # publishes NCMD Node Control/Rebirth=true
SparkplugTopic.cs # parse/format spBv1.0/{group}/{type}/{node}[/{device}]
ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser/ # bespoke observation browser — references .Contracts + Commons(.Browsing) + MQTTnet
MqttDriverBrowser.cs # IDriverBrowser (DriverType="Mqtt"); OpenAsync → MqttBrowseSession
MqttBrowseSession.cs # IBrowseSession over an accumulating observed tree
src/Server/.../AdminUI/
Uns/TagEditors/MqttTagConfigModel.cs # typed working model (FromJson/ToJson/Validate; preserves unknown keys)
Uns/TagEditors/TagConfigEditorMap.cs # + ["Mqtt"] = MqttTagConfigEditor
Uns/TagEditors/TagConfigValidator.cs # + ["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate()
Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor
EndpointRouteBuilderExtensions.cs # + AddSingleton<IDriverBrowser, MqttDriverBrowser>()
src/Server/.../Host/Drivers/DriverFactoryBootstrap.cs
# + MqttDriverFactoryExtensions.Register(registry, loggerFactory)
# + TryAddEnumerable(IDriverProbe, MqttDriverProbe)
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests/ # unit — codec/alias/seq/parser (net10, macOS-safe)
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/ # env-gated live suite (MQTT_FIXTURE_ENDPOINT)
tests/Drivers/.../Docker/docker-compose.yml # mosquitto + sparkplug-sim + json-publisher (project=lmxopcua)
```
**Why the split `.Contracts` project:** it carries `MqttDriverOptions` + `MqttTagDefinition` + `MqttEquipmentTagParser` + the enums with **zero transport (MQTTnet) dependency**, exactly like `Driver.Modbus.Contracts` depends only on `Modbus.Addressing` + `Core.Abstractions`. This lets the AdminUI typed editor (and any tooling) reference the DTO shape/enums without dragging MQTTnet into AdminUI. The runtime `.Driver` and `.Browser` both reference `.Contracts` and add MQTTnet.
### 2.1 Libraries + the pinning caveat
| Concern | Choice | Package |
|---|---|---|
| **MQTT transport (both modes)** | **MQTTnet v5** — MIT, **.NET Foundation** (`dotnet/MQTTnet`), the standard .NET MQTT client. v5 explicitly ships a `net10.0` TFM. Client + TLS + MQTT 3.1.1/5.0. | `MQTTnet` (add `PackageVersion Include="MQTTnet" Version="5.x"` to `Directory.Packages.props`) |
| **Sparkplug decode** | **Hand-rolled Tahu protobuf** (recommended) — generate C# from Tahu's `sparkplug_b.proto` with `Google.Protobuf` (`Grpc.Tools`/`protoc`), decode over a single MQTTnet-5 client. Small, stable schema; we own the alias/seq/rebirth state machine exactly as we want it. | `Google.Protobuf` + `Grpc.Tools` (build-time) |
| **Alternative Sparkplug decode** | **SparkplugNet** — MIT, provides `Application`/`Node`/`Device` base classes, saves the birth/alias plumbing. **Rejected as primary** for two repo-specific reasons (below). | `SparkplugNet` (only if the risk clears) |
**Pinning caveat (load-bearing — this is why hand-roll is recommended).** This repo uses **central package management** (`Directory.Packages.props`, `ManagePackageVersionsCentrally=true`) and deliberately **leaves `CentralPackageTransitivePinningEnabled` OFF** because turning it on breaks the Roslyn 5.0.0/4.12.0 split (see the "Transitive pinning breaks Roslyn build" memory + `Directory.Packages.props:103` comment). SparkplugNet **pins MQTTnet 4.3.x transitively and has no explicit `net10.0` TFM** yet. A direct MQTTnet-5 reference (needed for the plain-mode transport + net10) will **coexist in the graph with SparkplugNet's transitive 4.x**, and because transitive pinning is off we cannot cleanly force-align the two. Net: SparkplugNet risks a hard-to-resolve MQTTnet 4/5 diamond in exactly the pinning configuration this repo can't change.
**Decision: hand-roll Tahu-proto decode over one MQTTnet-5 client.** It removes the diamond and the net10-TFM concern entirely, and the Sparkplug payload schema (`Payload { seq, timestamp, Metric[], body }`, `Metric { name, alias, timestamp, datatype, value, is_null, ... }`) is ~90 lines of `.proto` — a bounded, well-understood effort. Phase 1 (plain MQTT) references **only MQTTnet-5**, so we validate MQTTnet-v5-on-net10 before ever touching Sparkplug. (If a future need for SparkplugNet arises, the phase-2 codec seam `SparkplugCodec` is the single swap point.)
**Vendor the proto:** commit Tahu `sparkplug_b.proto` into `Driver.Mqtt.Contracts/Protos/` (pinned to a known Tahu commit, with a provenance comment) and generate at build via `Grpc.Tools`. Don't fetch at build time.
**Existing protobuf tooling, honestly:** `Google.Protobuf` is **already centrally pinned** (`Directory.Packages.props:31`, 3.34.1) and consumed by `Driver.Galaxy` — but the Galaxy proto types ship **pre-generated** inside the `ZB.MOM.WW.MxGateway.*` NuGets; there is **no `.proto` file or `Grpc.Tools` protoc codegen anywhere in this repo today**. So `Grpc.Tools` is a new (build-time-only, `PrivateAssets=all`) package and this is the repo's first in-repo proto codegen step. If that step proves objectionable, the fallback is checking in the generated C# (the schema is small and stable) with the same provenance comment.
---
## 3. Capability mapping
`MqttDriver : IDriver, ITagDiscovery, ISubscribable, IReadable, IHostConnectivityProbe, IRediscoverable` (+ `IWritable` in phase 3). **Not** `IHistoryProvider` (historization is the server-side `isHistorized` path, unchanged). **Not** a poller — no `PollGroupEngine`.
### 3.0 Lifecycle (`IDriver`)
- `InitializeAsync(configJson, ct)` — deserialize `MqttDriverOptions` (shared `JsonSerializerOptions` with `JsonStringEnumConverter`, §6); build `MqttConnection`; connect to broker under a bounded connect deadline (§8); set LWT/clean-session; subscribe to the mode-appropriate filter (§3.1). Publish `STATE` birth if `actAsPrimaryHost` (Sparkplug).
- `ReinitializeAsync` — apply a config delta in place: if only tag membership changed, re-subscribe; if broker connection changed, tear down + reconnect. Never crash the process (marks Faulted → nodes go Bad, per driver-stability Tier A/B).
- `ShutdownAsync` — publish `STATE OFFLINE` (if primary host), disconnect the MQTTnet client, dispose.
- `GetHealth` — Connected/Reconnecting/Faulted + last-message-age; drives ServiceLevel + status dashboard.
- `GetMemoryFootprint`/`FlushOptionalCachesAsync` — footprint = last-value cache + birth caches + alias tables. The runtime driver has **no optional caches to flush**: birth/alias tables are required-for-correctness (flushing them mis-routes aliased DDATA — `IDriver.FlushOptionalCachesAsync` forbids flushing correctness state) and the last-value cache is what `IReadable` serves. Flush is a no-op; all three structures are bounded by the authored tag set + observed births. (The observation browser's accumulating tree lives in the AdminUI-side `MqttBrowseSession`, not in the runtime driver.)
### 3.1 Subscribe (`ISubscribable` — primary)
- **Plain mode:** subscribe to each authored tag's `topic` at its `qos` (dedupe identical topics; coalesce shared wildcards where the operator authored one). On message: match topic → authored tag(s), extract value at `jsonPath` (or raw/scalar), raise `OnDataChange(handle, FullReference, snapshot)`. Seed initial value from the broker's **retained** message replayed on subscribe (OPC UA initial-data convention).
- **Sparkplug mode:** subscribe `spBv1.0/{groupId}/#` (+ the host-state topic when acting as primary host — **`spBv1.0/STATE/{hostId}` with a JSON `{online, timestamp}` payload in spec v3.0**; pre-3.0 peers used top-level `STATE/{hostId}` with `ONLINE`/`OFFLINE` text — target the v3.0 form, tolerate the legacy one on receive). Maintain a **`BirthCache` + `AliasTable` per (edgeNode, device)** from NBIRTH/DBIRTH. On NDATA/DDATA: resolve each metric by **alias → (name, datatype)** via the alias table, map to the authored tag's `FullReference` **by stable metric name**, raise `OnDataChange`. On NDEATH/DDEATH: emit **STALE / Bad-quality** snapshots for that node's/device's metrics. On missed birth / unknown alias / seq gap: issue a **rebirth NCMD** (§3.6).
- `publishingInterval` is **advisory** (MQTT/Sparkplug are event-driven). Use it only for optional server-side coalescing/deadband, mirroring Modbus's `ShouldPublish`. `SubscribeAsync` returns an `ISubscriptionHandle` whose `DiagnosticId` names the mode+filter; `OnDataChange` fires from the MQTT receive handler for the whole driver lifetime.
`SubscribeAsync` maps the requested `FullReference`s onto the live receive stream: it registers them in a `FullReference → MqttTagDefinition` resolver (parsed once, cached — reuse the shared `EquipmentTagRefResolver<TDef>` from `Core.Abstractions`, exactly as Modbus does), so an inbound message/metric routes to the right node.
### 3.2 Read (`IReadable`)
MQTT has no request/response read. `ReadAsync(fullReferences)` returns the **last-value cache** (`LastValueCache`): last message seen on the topic (plain) or last metric value from data/birth (Sparkplug). Seed from the retained message at connect (plain) and from birth values (Sparkplug). If nothing has been observed yet, return a `GoodNoData`/uncertain snapshot **per reference** (never throw the batch — per `IReadable` contract, per-reference status is the snapshot's `StatusCode`). This satisfies OPC UA reads + HistoryRead-AtTime without a live round-trip.
### 3.3 Discover (`ITagDiscovery`) — authored-only
**`DiscoverAsync` replays authored tags only, both modes** — it does **not** auto-provision from wildcard/birth observation. Rationale (from research §2.2): a chatty broker or a large Sparkplug plant would produce an unbounded auto-provisioned address space. Wildcard/birth observation is a **browse-time** concern (§4), not a runtime-provisioning one.
- The operator authors tags (topic+jsonPath, or group/node/device/metric) via the typed editor / observation browser. `DiscoverAsync` streams exactly those authored tags into the `IAddressSpaceBuilder`, each as a `Variable(browseName, displayName, DriverAttributeInfo{ FullName, DriverDataType, ... })`.
- **`RediscoverPolicy`:**
- **Plain:** `Once` (authored set is static per published config — like Modbus).
- **Sparkplug:** `UntilStable` (metric *datatypes* fill in asynchronously as births arrive after connect; keep re-streaming until the authored tags' resolved datatypes stop changing — like FOCAS's FixedTree). This lets a tag authored before its first birth pick up its true datatype once the birth lands.
- **`SupportsOnlineDiscovery => false`** (the default-member from the universal-browser design). This is deliberate and **decoupled from browseability** (§4): runtime discovery is authored-only, so the universal Discover-backed browser would only replay authored tags — useless for discovering *new* topics/metrics. Leaving the flag false keeps MQTT off the universal fallback; the bespoke observation browser (§4) is what backs the picker.
### 3.4 `IRediscoverable`
Sparkplug fires `OnRediscoveryNeeded` when a **new DBIRTH** (a device joins) or a **rebirth with a changed metric set** is observed — the Core re-runs `DiscoverAsync` + diffs. `ScopeHint` = the edge-node/device folder path so the rebuild is surgical. Plain mode never fires it (no backend change signal — a config republish is the only tag-set change), so plain-only drivers effectively no-op this interface.
### 3.5 Data-type mapping
Sparkplug metric datatype (or inferred/authored plain type) → `DriverDataType` (`Boolean, Int16, Int32, Int64, UInt16, UInt32, UInt64, Float32, Float64, String, DateTime, Reference`):
| Sparkplug `DataType` | `DriverDataType` | OPC UA |
|---|---|---|
| Int8, Int16 | `Int16` (Int8 widens) | Int16 |
| Int32 | `Int32` | Int32 |
| UInt8, UInt16 | `UInt16` (UInt8 widens) | UInt16 |
| UInt32 | `UInt32` | UInt32 |
| Int64 | `Int64` | Int64 |
| UInt64 | `UInt64` | UInt64 |
| Float | `Float32` | Float |
| Double | `Float64` | Double |
| Boolean | `Boolean` | Boolean |
| String, Text, UUID | `String` | String |
| DateTime | `DateTime` | DateTime |
| Bytes, File | `String` (base64/raw fallback, v1) | String |
| DataSet, Template | **unsupported v1** — skip + warn, or expose raw-JSON as `String` | — |
| `*Array` variants | element type + `IsArray=true`, `ArrayDim` from birth | ValueRank=1 |
Live map in `SparkplugDataType.ToDriverDataType()` (`.Contracts`). **Plain mode:** honour an explicit per-tag `dataType` (preferred — inference is brittle); fall back to inferring from the JSON token at `jsonPath` (number→`Float64`/`Int64`, bool→`Boolean`, string→`String`). Int8/UInt8 widen (no OPC UA signed-byte distinction needed).
### 3.6 Sparkplug state-machine correctness (the #1 risk)
The alias→metric map, seq-gap detection, late-join rebirth, and death→STALE are load-bearing and easy to get subtly wrong. **Invariants the implementation must hold:**
1. **Bind tags by stable metric NAME, never by alias.** The authored `TagConfig` stores `metricName`; the alias is a **per-birth cache only**. An alias can be reused across a rebirth for a *different* metric — binding by alias silently mis-routes data to the wrong tag.
2. **Rebuild the alias table on every (re)birth.** NBIRTH/DBIRTH replaces the `(edgeNode,device)` `AliasTable` wholesale; never merge.
3. **Track `seq` (0–255, wraps) per edge-node.** A gap (or an unknown alias, or a data message before any birth) ⇒ request rebirth (`RebirthRequester` publishes `NCMD` writing `Node Control/Rebirth = true`), gated on `requestRebirthOnGap`. `bdSeq` ties an NDEATH to its NBIRTH.
4. **Death → STALE.** NDEATH/DDEATH (LWT-driven) emits Bad/uncertain-quality snapshots for all affected metrics; the next birth restores Good.
5. **Late join.** On connect the driver may have missed births — proactively request rebirth (or wait for the retained STATE + natural rebirth) so metadata is recovered. This is the same primitive the browser uses (§4).
Unit-test these explicitly against a controllable simulator: rebirth, missed-birth, seq-gap, alias-reuse-across-rebirth, death→stale→rebirth.
### 3.7 Write (`IWritable`) — **verdict: NOT in v1; phase 3, fire-and-forget optimistic-Good**
Writes are possible but deferred (research §2.4). When shipped in phase 3:
- **Sparkplug:** publish `NCMD`/`DCMD` to `spBv1.0/{group}/NCMD/{node}[/{device}]` with the target metric (by name+alias) + value. **No synchronous write ack** — success is only observable when the edge echoes the change in the next N/DDATA. This mirrors the **Galaxy gateway's fire-and-forget write** ("Galaxy can never surface a write failure"): return `Good` optimistically; the value self-corrects on the next data message. Respect `WriteIdempotent` — Sparkplug commands (rebirth, pulse) are frequently **non-idempotent**, so default retry off.
- **Plain:** publish to a configured command/write topic with a formatted payload.
v1 ships **read/subscribe/discover only** — a genuinely useful UNS-ingest driver. `MqttDriver` does **not** implement `IWritable` in v1 (the interface is composable; its absence means nodes materialize read-only).
---
## 4. Bespoke observation browser
### Decision — reconciling with the universal Discover-backed browser
The universal browser (`docs/plans/2026-07-15-universal-discovery-browser-design.md`) captures whatever `DiscoverAsync` streams and re-presents it, gated on `SupportsOnlineDiscovery`. **MQTT/Sparkplug is the one roadmap driver that needs a bespoke browser despite the universal one**, because:
- Runtime `DiscoverAsync` is **authored-only** (§3.3). A universal Discover-backed browser would replay only tags the operator already authored — it would **not discover new topics/metrics**, which is the entire point of a picker.
- MQTT/Sparkplug discovery is **observational**: you learn what exists only by *listening* for a bounded window. That's a fundamentally different session shape than "connect → one-shot enumerate → disconnect" — the tree **fills in over time**, and Sparkplug can be *accelerated* by forcing a rebirth NCMD.
**Considered + rejected:** feeding the Sparkplug `BirthCache` into `DiscoverAsync` device-enumeration to plug into universal. Rejected because it would make runtime discovery auto-provision from births (the unbounded-address-space failure §3.3 explicitly avoids) and still wouldn't cover plain MQTT. The birth cache stays a browse-time + subscribe-time structure, not a discovery source.
**Decision:** keep `SupportsOnlineDiscovery=false` (authored-only runtime discovery) **and** build a bespoke `IDriverBrowser` (`MqttDriverBrowser`) with an **observation-window `IBrowseSession`** (`MqttBrowseSession`). Registering the bespoke browser for `DriverType="Mqtt"` **overrides the universal fallback** for MQTT (the universal browser resolves bespoke-first — `BrowserSessionService` §3.1 of that doc).
### 4.1 `MqttDriverBrowser : IDriverBrowser`
`DriverType => "Mqtt"`. `OpenAsync(configJson, ct)`:
1. Parse the **same `MqttDriverOptions` JSON** the runtime driver consumes (shared `JsonSerializerOptions`), but connect with a **distinct client-id suffix** (`{clientId}-browse-{guid8}`) so the browse session never collides with the running driver's MQTT session / clean-session state.
2. Connect an MQTTnet client under a bounded connect budget (clamp like OpcUaClient's `PerEndpointConnectTimeout`, 5–30 s).
3. Subscribe to the discovery filter: `spBv1.0/{groupId}/#` (Sparkplug) or `#`/`{topicPrefix}#` (plain).
4. **Kick off the observation window immediately** and, in Sparkplug mode, optionally publish a **rebirth NCMD** to force every edge node to re-announce (turns the passive wait into a near-synchronous enumeration).
5. Return an `MqttBrowseSession` whose internal tree **fills in over the window and keeps filling** while the session lives. Because `IBrowseSession` methods are async and the session lands in the AdminUI `BrowseSessionRegistry` (TTL-reaped, per-call 20 s timeout), the observation cost is amortised across the user's clicks in the picker.
### 4.2 `MqttBrowseSession : IBrowseSession`
Serves from an **accumulating observed tree** (thread-safe; the MQTT receive handler mutates it, browse calls read it). Refreshes `LastUsedUtc` on each call.
- **`RootAsync`** — Sparkplug: observed **groups** (or edge nodes under the configured group). Plain: top-level topic segments. `BrowseNode { Kind=Folder, HasChildrenHint=true }`. If the window hasn't yielded yet, return what's accumulated (possibly empty) and let the user re-expand; in Sparkplug mode trigger a rebirth on first `RootAsync` to populate fast.
- **`ExpandAsync(nodeId)`** — one level down the accumulated tree. Sparkplug: **Group → EdgeNode → Device → Metric**. Plain: next topic segment (split on `/`). Metrics / leaf-topics are `Kind=Leaf`.
- **`AttributesAsync(nodeId)`** — Sparkplug: the metric's `AttributeInfo { Name, DriverDataType (from birth), IsArray, SecurityClass }` — **self-describing**, so the picker side-panel is populated (unlike the OPC UA browser which returns empty). Plain: a single synthetic attribute carrying the leaf topic's **inferred type + last-seen payload snippet**.
- **`DisposeAsync`** — disconnect the MQTTnet client, best-effort (registry reaper may race), same pattern as `OpcUaClientBrowseSession`.
### 4.3 Picked node → TagConfig
On commit the picker projects the selected `BrowseNode`/`AttributeInfo` into the §5 TagConfig JSON:
- **Sparkplug:** the `NodeId` encodes `group/node/device:metric` → split into `groupId`/`edgeNodeId`/`deviceId`/`metricName`; `dataType` from `AttributeInfo`.
- **Plain:** the `NodeId` is the topic path → `topic` + a default `jsonPath` (`$`, operator-editable) + inferred `dataType`.
### 4.4 Addressing the passive/time-bounded wrinkle explicitly
1. Picker shows a live **"listening… N nodes/topics discovered"** affordance so the operator understands coverage is accumulating, not instantaneous.
2. Sparkplug mode **actively forces a rebirth** to convert the passive wait into a fast near-complete enumeration.
3. The browse session stays alive (registry TTL) so re-expanding later reflects newly observed topics without reconnecting.
4. **Manual entry** of a topic/metric via the typed editor (§6) is always available as an escape hatch for a tag that's silent during the window.
### 4.5 Registration
```csharp
// EndpointRouteBuilderExtensions.cs — alongside the existing two (near line 49-50)
services.AddSingleton<IDriverBrowser, OpcUaClientDriverBrowser>();
services.AddSingleton<IDriverBrowser, GalaxyDriverBrowser>();
services.AddSingleton<IDriverBrowser, MqttDriverBrowser>(); // NEW — overrides universal fallback for "Mqtt"
```
---
## 5. TagConfig + driver-config JSON
Two layers, mirroring every other driver: **driver options** (broker connection, one per driver instance, parsed by the factory) and **per-tag config** (`TagConfig` JSON authored on the `/uns` Equipment → Tags tab; the driver's `FullName` is the driver-side reference the router resolves).
### 5.1 Driver options — `MqttDriverOptions` (`.Contracts`)
```jsonc
{
"host": "10.100.0.35",
"port": 8883,
"clientId": "otopcua-uns-1",
"useTls": true,
"allowUntrustedServerCertificate": false, // dev/on-prem only
"caCertificatePath": null, // PEM CA pin; null ⇒ OS trust store
"username": "otopcua",
"password": "", // supply via env, NEVER commit
"protocolVersion": "V500", // "V311" | "V500"
"cleanSession": true,
"keepAliveSeconds": 30,
"connectTimeoutSeconds": 15, // bounded connect deadline (§8)
"reconnectMinBackoffSeconds": 1,
"reconnectMaxBackoffSeconds": 30,
"mode": "SparkplugB", // "Plain" | "SparkplugB"
// ---- Sparkplug-only (null in Plain mode) ----
"sparkplug": {
"groupId": "Plant1", // subscribe spBv1.0/Plant1/#
"hostId": "otopcua-host-1", // spBv1.0/STATE/{hostId} identity (v3.0)
"actAsPrimaryHost": false, // at most ONE primary host per host-id per broker
"requestRebirthOnGap": true,
"birthObservationWindowSeconds": 15 // browse/discovery: how long to collect births
},
// ---- Plain-only (null in SparkplugB mode) ----
"plain": {
"topicPrefix": "factory/",
"defaultQos": 1
}
}
```
### 5.2 Per-tag — Sparkplug
```jsonc
{
"FullName": "Plant1/Line3EdgeNode/Filler1:Temperature/degC", // PascalCase key (composer/walker contract)
"groupId": "Plant1",
"edgeNodeId": "Line3EdgeNode",
"deviceId": "Filler1", // omit/null for node-level metrics
"metricName": "Temperature/degC", // STABLE identity across rebirths — bind by this, not alias
"dataType": "Float" // from DBIRTH; explicit for safety
}
```
`FullName` convention: `"{group}/{node}[/{device}]:{metric}"`. The driver parses `FullName` (or the descriptor fields) once, caches it, and resolves inbound metrics by **`(group,node,device,metricName)`**.
### 5.3 Per-tag — Plain MQTT
```jsonc
{
"FullName": "factory/line3/oven/temp#$.value", // topic#jsonPath — stable per-tag key
"topic": "factory/line3/oven/temp",
"payloadFormat": "Json", // "Json" | "Raw" | "Scalar"
"jsonPath": "$.value", // JSONPath into payload (Json only)
"dataType": "Double", // explicit — inference is the fallback
"qos": 1,
"retainSeed": true // seed last-value from retained msg on subscribe
}
```
`FullName` convention: `"{topic}#{jsonPath}"` (Json) or just `"{topic}"` (Raw/Scalar). `MqttEquipmentTagParser.TryParse(reference, out def)` follows `ModbusEquipmentTagParser`: a leading `{` marks a TagConfig blob; parse it, use **strict enum reads** (`TagConfigJson.TryReadEnumStrict`) so a typo'd `payloadFormat`/`dataType` rejects the tag (→ `BadNodeIdUnknown`) rather than silently defaulting to a wrong type; the def's `Name` = the reference string so published values key the forward router. Add an `Inspect(reference)` deploy-time warning pass (like Modbus) for present-but-invalid enums / unparseable blobs.
**Historization:** the server-side `isHistorized` / `historianTagname` keys are owned by the TagModal-merge seam (`TagHistorizeConfig`) and survive load→save of the typed model as preserved unknown keys — the driver does not model them (identical to `OpcUaClientTagConfigModel`).
---
## 6. Typed editor + validator
### 6.1 `MqttTagConfigModel` (`.AdminUI/Uns/TagEditors/`)
Thin typed working model over a preserved `JsonObject` key bag, per the `OpcUaClientTagConfigModel` template: `FromJson(json)` / `ToJson()` / `Validate()`, **preserving unknown keys** across a load→save (so history keys + fields this editor doesn't expose survive). It carries **both** mode shapes — a `Mode` field (Plain|SparkplugB, defaulted from which fields are present) selects which sub-shape the editor renders + validates.
- `FromJson` reads the descriptor fields for the active mode plus the always-present `FullName` (PascalCase, case-sensitive — the composer/artifact-decoder/walker read it via case-sensitive `TryGetProperty("FullName", …)`).
- `ToJson` writes `FullName` PascalCase and re-derives it from the descriptor fields (`{group}/{node}[/{device}]:{metric}` or `{topic}#{jsonPath}`) so a hand-edited descriptor stays consistent with the routed key.
- `Validate()`:
- **Sparkplug:** `groupId`, `edgeNodeId`, `metricName` required; `deviceId` optional; `dataType` must be a known Sparkplug type.
- **Plain:** `topic` required and non-wildcard (a subscription topic for a *tag* must be concrete — reject `+`/`#`); `payloadFormat=Json` ⇒ `jsonPath` required; `dataType` a known `DriverDataType`.
### 6.2 `MqttTagConfigEditor.razor`
Copy the Modbus editor template (`Components/Shared/Uns/TagEditors/`): a razor shell over the pure model, mode-switch at top, per-mode field group below, client-side validation via `Validate()`. Register:
```csharp
// TagConfigEditorMap.cs
["Mqtt"] = typeof(Components.Shared.Uns.TagEditors.MqttTagConfigEditor),
// TagConfigValidator.cs
["Mqtt"] = j => MqttTagConfigModel.FromJson(j).Validate(),
```
### 6.3 The `JsonStringEnumConverter` enum-serialization trap
**Every** JSON seam that (de)serializes `MqttDriverOptions` / tag configs — factory, probe, browser, **and the AdminUI driver-config page + probe DTO** — MUST use a `JsonSerializerOptions` that includes `new JsonStringEnumConverter()` (plus `PropertyNameCaseInsensitive=true`, `UnmappedMemberHandling=Skip`). The repo's SYSTEMIC bug (memory: "Driver enum-serialization bug"): AdminUI pages that serialize enums **numerically** while factory DTOs are **string-typed** fault the driver on any enum field. `MqttMode`, `MqttPayloadFormat`, `protocolVersion` are all enums — the AdminUI Mqtt driver page and its probe must serialize them as **strings** (mirror the OpcUaClient page). `JsonStringEnumConverter` also accepts numeric ordinals, so existing numeric configs still load. Keep one shared `JsonSerializerOptions` definition referenced by factory + probe + browser (as OpcUaClient does).
---
## 7. Factory + registration
`MqttDriverFactoryExtensions` (mirror `OpcUaClientDriverFactoryExtensions`):
```csharp
public static class MqttDriverFactoryExtensions
{
public const string DriverTypeName = "Mqtt";
private static readonly JsonSerializerOptions JsonOptions = new()
{
PropertyNameCaseInsensitive = true,
UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip,
Converters = { new JsonStringEnumConverter() },
};
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory));
}
public static MqttDriver CreateInstance(string driverInstanceId, string driverConfigJson, ILoggerFactory? lf = null)
{
var options = JsonSerializer.Deserialize<MqttDriverOptions>(driverConfigJson, JsonOptions)
?? throw new InvalidOperationException($"Mqtt driver config for '{driverInstanceId}' deserialised to null");
return new MqttDriver(options, driverInstanceId, lf?.CreateLogger<MqttDriver>());
}
}
```
Wire in the two host bootstrap sites:
```csharp
// Host/Drivers/DriverFactoryBootstrap.cs — factory registration (near line 133)
Driver.Mqtt.MqttDriverFactoryExtensions.Register(registry, loggerFactory);
// … and probe registration in AddOtOpcUaDriverProbes (near line 114)
services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, Driver.Mqtt.MqttDriverProbe>());
```
**`MqttDriverProbe : IDriverProbe`** (`DriverType => "Mqtt"`): parse the config with the **shared** `JsonSerializerOptions`, open an MQTTnet CONNECT to `host:port` under the timeout, return green + latency on CONNACK-accepted, or a targeted error (connection refused / TLS failure / auth failure / timeout). Analogous to `ModbusDriverProbe`'s TCP+FC03 handshake — CONNACK is the MQTT "device is answering" proof. **Probes are admin-pinned** (the `admin-operations` singleton dispatches by DriverType) — the `TryAddEnumerable` line must be on admin nodes (it is, via `AddOtOpcUaDriverProbes` in `Program.cs`'s `hasAdmin` block).
---
## 8. Resilience / timeout / security
- **Bounded connect.** `InitializeAsync` connects under `connectTimeoutSeconds` (default 15) via a linked CTS — **no unbounded wait** on a dead broker (heed the S7/FOCAS read-timeout lessons: an async network call that ignores its deadline wedges the whole driver).
- **Reconnect with backoff.** MQTTnet's managed reconnect (or a hand-rolled loop) with exponential backoff `reconnectMinBackoffSeconds → reconnectMaxBackoffSeconds`. On disconnect: `GetHealth` → Reconnecting; on reconnect: re-subscribe, and in Sparkplug mode **request rebirth** to recover metadata (late-join, §3.6). Nodes go Bad-quality while disconnected via the standard fan-out.
- **Subscribe deadline.** `SubscribeAsync` completes under a bounded deadline; SUBACK failure surfaces as a per-reference Bad, not a hang.
- **No poll loop** — the receive handler must be non-blocking (offload heavy decode off the MQTT callback thread; never block the MQTTnet dispatcher).
- **Broker security — enforce, never ship public-broker defaults.** Default `useTls=true`; real auth (username/password or client-cert). `allowUntrustedServerCertificate` + `caCertificatePath` mirror the ServerHistorian TLS knobs (dev/on-prem escape hatch, off by default). **Never** put Sparkplug production-shaped data on public brokers (`test.mosquitto.org` etc. — manual smoke only). Credentials come from env (`password` blank in committed JSON), same posture as `ServerHistorian__ApiKey`.
- **Primary-host guard.** At most **one** primary-host client per `hostId` per broker (Sparkplug MUST). `actAsPrimaryHost=true` on two OtOpcUa instances sharing a `hostId` (e.g. a redundancy pair) is a misconfiguration — the redundancy pair should use **distinct** host-ids (mirrors the distinct `MxAccess.ClientName` redundancy rule).
---
## 9. Test fixtures
Follow the driver-fixture pattern: `tests/Drivers/.../Docker/docker-compose.yml` with the `project: lmxopcua` label, deployed to `10.100.0.35` via `lmxopcua-fix sync/up` (`CLAUDE.md` Docker Workflow). Repo files are source-of-truth; `/opt/otopcua-mqtt/` is the mirrored deployment.
- **Broker:** **Eclipse Mosquitto** (`eclipse-mosquitto`, tiny, carries both plain + Sparkplug unchanged) as the default fixture; **EMQX** (`emqx/emqx`, Apache-2.0 CE) as the alternative when a dashboard / built-in Sparkplug tooling helps. Configure with **auth + TLS** so the fixtures exercise the real security path (not anonymous).
- **Sparkplug edge-node simulator:** prefer a **project-owned C# simulator** using the same MQTTnet + Tahu-proto path the driver uses — validates encode/decode symmetrically and gives full control of rebirth/seq-gap/death scenarios (the §3.6 test matrix). Alternatives: Eclipse Tahu reference (Java/Python) edge-node script; Bevywise MQTT/IoT Simulator (purpose-built Sparkplug B). The simulator must publish NBIRTH/DBIRTH → periodic NDATA/DDATA, honour a rebirth NCMD, and be able to inject a seq gap / alias reuse / N-D-DEATH on demand.
- **Plain-MQTT fixture:** a tiny publisher container (`mosquitto_pub` loop or a `paho-mqtt` script) emitting JSON on a handful of topics with `retain=true` so the last-value/read path **and** the `#`-observation browse are testable.
- **Env-gated live suite:** `Driver.Mqtt.IntegrationTests`, category `LiveIntegration` (à la HistorianGateway), skips cleanly when **`MQTT_FIXTURE_ENDPOINT`** is unset → macOS-offline-safe. Covers: connect/TLS/auth; plain subscribe+retained-read; Sparkplug birth→data→alias-resolve→`OnDataChange`; rebirth recovery; death→STALE; the browser observation window (birth-driven tree + rebirth-forced enumeration).
- **Live-verify (docker-dev):** author an Mqtt driver + tag via the `/uns` picker on `:9200`, deploy, confirm the node carries live broker values and the typed editor renders both modes (the usual live-`/run` discipline — Razor binding bugs pass unit tests).
Add the fixture endpoint to `CLAUDE.md`'s endpoint list once landed (e.g. `10.100.0.35:1883`/`8883`, `MQTT_FIXTURE_ENDPOINT`).
---
## 10. Phasing + effort
Order: **plain MQTT first, then Sparkplug B, then write.** Plain MQTT is the dependency-light slice that stands up the whole skeleton and **validates MQTTnet-v5-on-net10 before any Sparkplug/proto work** — front-loading the reusable plumbing and de-risking the library decision.
| Phase | Scope | Rel. effort |
|---|---|---|
| **P1 — Plain MQTT** | 3 projects; MQTTnet-5 connect/TLS/auth/reconnect+backoff (`MqttConnection`); `ISubscribable` (topic subscribe → `OnDataChange`); `IReadable` last-value (retained seed); authored-tag `ITagDiscovery` (`Once`); `IHostConnectivityProbe`; `MqttDriverProbe`; factory + host wiring; `#`-observation browser (`MqttDriverBrowser`/`MqttBrowseSession`); typed editor + validator + AdminUI driver page; Mosquitto + JSON-publisher fixtures + env-gated suite. **Validate MQTTnet-5/net10 here.** | **M** |
| **P2 — Sparkplug B ingest** | Vendored Tahu proto + `Google.Protobuf` codegen; `SparkplugCodec` decode; `spBv1.0/{group}/#` subscribe; N/DBIRTH → `BirthCache`/`AliasTable`; bind-by-name + seq-gap detect + **rebirth NCMD**; N/DDEATH → STALE; STATE/primary-host option; `ITagDiscovery` `UntilStable` + `IRediscoverable`; birth-driven browser tree + `AttributesAsync`; Sparkplug datatype map; C# edge-node simulator fixture + §3.6 test matrix. | **L** |
| **P3 — Write-through (optional)** | `IWritable`: Sparkplug NCMD/DCMD + plain publish; optimistic-Good + self-correct on echo; `WriteIdempotent` respect (default off for non-idempotent Sparkplug commands); write-topic config. | **M** |
**Top risks:** (1) **Sparkplug state-machine correctness** (aliases + rebirth + seq — §3.6; bind-by-name is the mitigation, test against a controllable simulator); (2) **library/dependency friction** — SparkplugNet's MQTTnet-4.x transitive pin ✕ this repo's fragile central pinning (transitive pinning off), mitigated by **hand-rolling the Tahu proto over one MQTTnet-5 client** (§2.1). Secondary: passive-browse coverage gaps (rebirth-forcing + manual entry), unbounded address space (authored-only runtime discovery), write-has-no-ack (accepted, optimistic-Good like Galaxy), broker security (TLS + real auth enforced).
docs/plans/2026-07-15-mtconnect-driver-design.md
+182
View File
@@ -0,0 +1,182 @@
# MTConnect (Agent-first) driver — executable implementation design
**Status:** design, build-ready. 2026-07-15.
**Research input:** [`docs/research/drivers/mtconnect-agent.md`](../research/drivers/mtconnect-agent.md) — this doc turns that research into a build spec; it does **not** re-argue the protocol findings.
**House-style references:** [`2026-06-12-galaxy-standard-driver-design.md`](2026-06-12-galaxy-standard-driver-design.md), [`2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
---
## 1. Motivation + scope
MTConnect is the dominant open read-only telemetry standard for machine tools; adding a `DriverType = "MTConnect"` Equipment-kind driver lets OtOpcUa surface a machine's self-describing device model (positions, spindle speed, execution state, availability, conditions) alongside — and complementing — the lower-level **FOCAS** driver (FOCAS reaches the Fanuc CNC directly; MTConnect reaches the vendor-neutral Agent that often front-ends that same machine). v1 is **agent-first, read-only** (Discover + Read + Subscribe, no Write) because the mainstream MTConnect surface (`/probe`, `/current`, `/sample`) is read-only by design. Full rationale, protocol details, library survey, and risk register live in the [research report](../research/drivers/mtconnect-agent.md) §1–§8; this doc assumes them.
## 2. Project layout
Two new projects, mirroring the Modbus split (contracts DTOs isolated from runtime so the AdminUI probe/editor can reference config shapes without the driver's NuGet deps). **No `.Browser` project** — see §4.
```
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/
MTConnectDriverOptions.cs // strongly-typed runtime options (like ModbusDriverOptions)
MTConnectTagDefinition.cs // one record per authored tag (FullName=dataItemId + mt* metadata)
MTConnectDataTypeInference.cs // pure category/type/units → DriverDataType table (§3.3) — shared by driver, factory, editor, browser-commit
ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj // refs Core.Abstractions only
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/
MTConnectDriver.cs // IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable
MTConnectAgentClient.cs // thin seam over MTConnectHttpClient (probe/current/sample); testable via IMTConnectAgentClient
IMTConnectAgentClient.cs // seam interface — canned-XML fake in unit tests
MTConnectObservationIndex.cs // dataItemId → latest DataValueSnapshot, updated by /current + /sample
MTConnectDriverFactoryExtensions.cs // Register(registry, loggerFactory) + CreateInstance
MTConnectDriverProbe.cs // IDriverProbe: one-shot /probe reachability
ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj // refs Core, Core.Abstractions, .Contracts + TrakHound pkgs
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/
Fixtures/*.xml // canned probe/current/sample docs (§8)
...Tests.cs
```
`.csproj` boilerplate copies Modbus verbatim: `net10.0`, `Nullable=enable`, `TreatWarningsAsErrors=true`, `GenerateDocumentationFile`, `InternalsVisibleTo` the Tests project.
### NuGet dependency
Consume **TrakHound MTConnect.NET client packages (MIT)** — pin an exact version:
```xml
<PackageReference Include="MTConnect.NET-Common" Version="6.9.0.2" />
<PackageReference Include="MTConnect.NET-HTTP" Version="6.9.0.2" />
<!-- MTConnect.NET-XML / -JSON pulled transitively for serialization -->
```
These target `netstandard2.0` (load cleanly on .NET 10) and give `MTConnectHttpClient` (probe/current/sample with polling **and** the multipart long-poll stream, gzip, XML+JSON) plus the model types (`IDevice`/`IComponent`/`IDataItem`/`IObservation`) — removing the multipart-framing / version-negotiation / buffer-overflow-rebaseline grind.
> **License-pin caveat (from research §1.4):** older TrakHound releases carried mixed MIT / Apache-2.0 / "all rights reserved" strings. Before merge, confirm the embedded `LICENSE` of the *pinned* version is MIT (a legal-review checkbox, not a blocker). **Hand-roll fallback** if review fails: `HttpClient` + `System.Xml.Linq` for probe/current, a `multipart/x-mixed-replace` boundary reader for sample (~300–500 LoC behind the same `IMTConnectAgentClient` seam — the driver above the seam is unchanged, so the fallback is a drop-in of one class).
## 3. Capability mapping (concrete seam wiring)
`MTConnectDriver` implements `IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable` — **not** `IWritable` (§3.5). One `MTConnectAgentClient` + one `MTConnectObservationIndex` per driver instance; one shared sample stream (the Agent streams the whole device model, so per-tag streams would be wasteful).
### 3.1 `IDriver`
- `InitializeAsync(json, ct)`: deserialize `MTConnectDriverOptions` (via factory), construct `MTConnectAgentClient` from `AgentUri`, run **one `/probe`** under a per-call deadline (§7). Cache the parsed `IDevice[]` model + `Header.instanceId`. Prime the `MTConnectObservationIndex` with one `/current`. Set `DriverState.Healthy` on success; `Faulted` (rethrow) on failure — the actor marks the instance Faulted, nodes go Bad, process stays up.
- `ReinitializeAsync`: tear down the sample stream, re-run Initialize. Config-only change (no address-space rebuild) unless `AgentUri`/`DeviceName` changed.
- `ShutdownAsync`: stop the sample stream, dispose the HTTP client.
- `GetHealth`: `DriverHealth(State, LastSuccessfulRead, LastError)` — `LastSuccessfulRead` = last `/current` or `/sample` chunk time.
- `GetMemoryFootprint` / `FlushOptionalCachesAsync`: footprint ≈ cached probe model + observation index; flush drops the browse/probe-model cache (re-fetchable), keeps the observation index (correctness state).
### 3.2 `ITagDiscovery` — plugs into the universal browser (§4)
```csharp
public bool SupportsOnlineDiscovery => true; // /probe enumerates from the device
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once; // probe is synchronous + complete
public async Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken ct)
```
`DiscoverAsync` fetches (or reuses the Initialize-cached) `/probe` model and **streams the full Device→Component→DataItem tree into the builder**:
- each `Device` → `builder.Folder(device.Name, device.Name)` → child builder;
- each nested `Component` → recursive `Folder(component.Name, component.DisplayName)` on the parent's child builder (component nesting becomes folder nesting — the builder-graph *is* the tree);
- each `DataItem` → `child.Variable(browseName, displayName, attr)` where
`attr = new DriverAttributeInfo(FullName: dataItem.Id, DriverDataType: MTConnectDataTypeInference.Infer(category, type, units), IsArray: representation==TIME_SERIES, ArrayDim: null, SecurityClass: SecurityClassification.ViewOnly, IsHistorized: false, IsAlarm: category==CONDITION)`.
(`ViewOnly` is the read-only-from-OPC-UA tier — `SecurityClassification` has no `ReadOnly` member.)
`browseName` = `dataItem.Name ?? dataItem.Id` (dataItem `name` is optional in MTConnect; fall back to `id`, which is guaranteed unique).
**Critical:** `DriverAttributeInfo.FullName = dataItem.Id`. This is (a) the value the universal browser commits as `TagConfig.FullName`, and (b) the key the read/subscribe paths resolve against — the two align by construction (research §3). If `DeviceName` scopes to one device, only that device's subtree is streamed.
`DeviceName` scoping and the whole-model-in-one-call nature make this a natural fit for **eager one-shot** discovery — exactly what the universal browser assumes.
### 3.3 Data-type mapping (`MTConnectDataTypeInference.Infer`)
Pure function in `.Contracts` (shared by driver, browser-commit, and the typed editor so all three agree). Weak wire typing means this is a heuristic, stored per-tag and **author-overridable** (§5):
| MTConnect | `DriverDataType` |
|---|---|
| `SAMPLE` numeric (has `units`) | `Float64` |
| `SAMPLE` `representation=TIME_SERIES` | `Float64` array (`IsArray=true`) |
| `EVENT` numeric type (`PartCount`, `Line`, …) | `Int64` |
| `EVENT` controlled-vocab (`Execution`, `ControllerMode`, `Availability`, …) | `String` |
| `EVENT` free text (`Program`, `Block`, `Message`) | `String` |
| `CONDITION` | `String` (state word; §3.6) |
**Quality mapping (`DataValueSnapshot.StatusCode`):** an observation value of `UNAVAILABLE` (MTConnect's explicit no-data sentinel) → OPC UA **`Bad`/`Uncertain`**, not the literal string. Missing dataItem / empty condition → `Bad`. `observation.timestamp` → `SourceTimestamp` (not a separate node).
### 3.4 `IReadable` — `/current`
`ReadAsync(fullReferences, ct)`: issue one `/current` under a per-call deadline, index the returned `MTConnectStreams` observations by `dataItemId` into `MTConnectObservationIndex`, then return **one `DataValueSnapshot` per requested ref in order**. A ref absent from the response → a `Bad`-coded snapshot (not a throw). Reads are idempotent (matches `IReadable` contract; `DriverCapability.Read` auto-retries). `/current` is the whole-device snapshot regardless of how many refs are requested, so batch reads cost one round-trip.
### 3.5 `ISubscribable` — `/sample` multipart long-poll
- `SubscribeAsync(fullReferences, publishingInterval, ct)`: on the **first** subscription, start the shared `MTConnectAgentClient` sample stream (`/sample?from=<nextSequence>&interval=<SampleIntervalMs>&count=<SampleCount>`). Record the subscribed ref set. Immediately fire `OnDataChange` for each subscribed ref from the primed `/current` values (OPC UA initial-data convention). Return an `ISubscriptionHandle` (monotonic id + `DiagnosticId`).
- Stream pump: each received `MTConnectStreams` chunk → for each observation whose `dataItemId` ∈ subscribed set, update the index and raise `OnDataChange(handle, dataItemId, snapshot)`. Advance `from = Header.nextSequence` for the next request (contiguous, gap-free).
- **Ring-buffer overflow:** if the Agent reports a sequence gap / `from` older than `firstSequence`, re-`/current` to re-baseline, then resume the stream from the new `nextSequence`. (TrakHound handles this internally; the hand-roll fallback must replicate it — unit-tested via a fixture with a forced gap.)
- `UnsubscribeAsync(handle, ct)`: drop that handle's refs from the subscribed set; stop the shared stream when the set empties.
- `publishingInterval` maps to the `/sample` `interval` when it is the only/first subscription; the driver polls at the finest requested interval and fans out (one stream per instance, not per tag).
### 3.6 `IWritable` — **not implemented (v1)**
Justified per research §2.1: the MTConnect Agent surface is read-only by design; write-back exists only via optional, rarely-deployed MTConnect *Interfaces* (a request/response state-machine handshake, not a "set value" — modelling it as `IWritable` would mislead). Capability interfaces are composable, so omitting `IWritable` is idiomatic (Galaxy's write path is even fire-and-forget). Nodes materialize **without** the `AccessLevels.CurrentWrite` bit automatically. Revisit Interfaces only if a concrete deployment needs it.
### 3.7 `IHostConnectivityProbe` + `IRediscoverable`
- `IHostConnectivityProbe`: expose one `HostConnectivityStatus` per Agent (HostName = `AgentUri`). A cheap periodic `/probe` (or reuse the sample-stream heartbeat) flips `Running ↔ Stopped`; raise `OnHostStatusChanged` on transition. Enable/interval from `MTConnectDriverOptions.Probe` (mirrors `ModbusProbeOptions`).
- `IRediscoverable`: watch the Agent `Header.instanceId` on every `/current`/`/sample` chunk. A change means the Agent restarted / its model changed → raise `OnRediscoveryNeeded` so `DriverHost` rebuilds the address space (mirrors Galaxy's `DeployWatcher`). This is why `RediscoverPolicy = Once` is safe: instanceId change, not polling, drives re-discovery.
### 3.8 CONDITION modelling
**v1 simple (this design):** each `CONDITION` DataItem is a `String` variable whose value is the current state word (`Normal`/`Warning`/`Fault`/`Unavailable`), optionally suffixed with `nativeCode`. Zero alarm plumbing. `IsAlarm=true` is still set on the `DriverAttributeInfo` so the browser side-panel flags it and a future upgrade needn't re-author tags.
**v1.5 (fast-follow, not in this build):** implement `IAlarmSource` + emit a TagConfig `alarm` object so Fault/Warning become native OPC UA Part 9 alarms, routing on the authored dotted `ConditionId` per the Galaxy/Phase-B native-alarm pattern.
## 4. Browse — plugs into the universal browser (reconciliation)
**The research report (§4.2) proposed a bespoke `ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser` project with a hand-written `MTConnectBrowseSession`. This design supersedes that: build NO bespoke browser.** With the [universal discovery browser](2026-07-15-universal-discovery-browser-design.md) in place (the Wave-0 gate, which lands before this Wave-2 driver), MTConnect browse is covered for free by the generic `DiscoveryDriverBrowser`, because **MTConnect browse comes from discovery** (`/probe` → the device model → the same `DiscoverAsync` stream). Concretely, the universal browser already does exactly what the proposed bespoke session would have done:
- `DiscoveryDriverBrowser.CanBrowse("MTConnect", cfg)` returns true because `TryCreate` succeeds and the instance is `ITagDiscovery { SupportsOnlineDiscovery: true }` (§3.2) → the AdminUI renders the **Browse** button.
- `OpenAsync` constructs the driver, `InitializeAsync` (the `/probe` connect), runs `DiscoverAsync` into a `CapturingAddressSpaceBuilder`, then `ShutdownAsync` — the captured tree (Device folders → Component folders → DataItem leaves) is served by `CapturedTreeBrowseSession`. Each leaf's `NodeId == DriverAttributeInfo.FullName == dataItemId`, committed directly as `TagConfig.FullName`; `IsAlarm`/`DriverDataType` flow to the side-panel and pre-fill the typed editor.
**No `PatchForBrowse` entry is needed** — MTConnect discovery is unconditional (not config-gated like AbCip/TwinCAT's `EnableControllerBrowse`); it belongs in the "no patch" set alongside OpcUaClient/BACnet. **No `IUniversalDriverBrowser`/`BrowserSessionService` change is needed** — MTConnect lights up purely by setting `SupportsOnlineDiscovery => true`. The single required action to enable browse is that one default-member override in §3.2.
The only limit (universal-browser §8) is that discovery is eager one-shot; `/probe` is already a whole-model single call, so eager is the right fit and graduating to bespoke is unlikely.
## 5. Typed tag editor + validator (AdminUI)
Avoid the raw-JSON fallback by adding a typed editor (mirrors the Modbus template exactly).
- `src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs` — pure `FromJson`/`ToJson`/`Validate`, **preserves unknown keys** via the `TagConfigJson` bag helper (copy `ModbusTagConfigModel`). Fields: `FullName` (dataItemId, required), `MtCategory`/`MtType`/`MtSubType` (enums / strings, from probe — read-only in UI), `DataType` (`DriverDataType` override dropdown), `Units`, `MtDevice`/`MtComponent` (author context). `Validate()` returns an error if `FullName` is blank.
- `.../Components/Shared/Uns/TagEditors/MTConnectTagConfigEditor.razor` — thin shell: `FullName` text, read-only `mtCategory/mtType`, a `DataType` override `<select>`. Because most tags arrive via the browse picker (which fills the model), the editor is mostly a confirm/override surface.
- Register `["MTConnect"] = typeof(Components.Shared.Uns.TagEditors.MTConnectTagConfigEditor)` in `Uns/TagEditors/TagConfigEditorMap.cs`, and `["MTConnect"] = j => MTConnectTagConfigModel.FromJson(j).Validate()` in `TagConfigValidator.cs`.
> **`JsonStringEnumConverter` enum-serialization trap (project-wide gotcha — MEMORY).** Every driver's AdminUI serialization path must round-trip enums as **name strings**, never numerics. The factory/probe DTOs are string-typed (`ParseEnum<T>`), so a numerically-serialized enum FAULTS the driver at deploy. Two concrete requirements: (1) `MTConnectTagConfigModel.ToJson` writes `MtCategory`/`DataType` via `TagConfigJson.Set(bag, "dataType", DataType)` which emits the enum **name** (the Modbus helper already does this — do not hand-roll `JsonSerializer` without `JsonStringEnumConverter`); (2) `MTConnectDriverProbe` parses with `new JsonStringEnumConverter()` in its `JsonSerializerOptions` (copy `ModbusDriverProbe._opts`); the factory keeps enum-carrying DTO fields `string?` and parses them via `ParseEnum<T>` — the Modbus factory pattern (its `JsonOptions` carries no enum converter). The `.Contracts` `MTConnectDataTypeInference` returns a `DriverDataType` enum; when it's written to JSON it must be the string.
## 6. Factory + registration
- `MTConnectDriverFactoryExtensions` (copy Modbus): `public const string DriverTypeName = "MTConnect";` `Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)` → `registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory))`. `CreateInstance` deserializes a `MTConnectDriverConfigDto` (nullable-init DTO with `Tags`, `Probe`), validates `AgentUri` present, builds `MTConnectDriverOptions`, returns `new MTConnectDriver(options, id, agentClientFactory: null, logger)`. Copy the Modbus factory `JsonOptions` (`PropertyNameCaseInsensitive`, `ReadCommentHandling=Skip`, `AllowTrailingCommas` — no enum converter: enum-carrying DTO fields stay `string?` and go through `ParseEnum<T>`, per §5's gotcha).
- **Host wiring — 3 one-line edits in `src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs`:**
1. `Register(...)` body: add `Driver.MTConnect.MTConnectDriverFactoryExtensions.Register(registry, loggerFactory);`
2. add `using MTConnectProbe = Driver.MTConnect.MTConnectDriverProbe;`
3. `AddOtOpcUaDriverProbes`: add `services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, MTConnectProbe>());`
The probe MUST be in `AddOtOpcUaDriverProbes` (not only the factory path) so it reaches **admin-only nodes** — the `admin-operations` singleton that backs Test-Connect is admin-role-pinned (the MEMORY "driver probes on admin nodes" gotcha). `TryAddEnumerable` keeps a fused admin,driver node from double-registering (a dup makes the singleton's `ToDictionary(p=>p.DriverType)` throw).
- `MTConnectDriverProbe : IDriverProbe`, `DriverType => "MTConnect"`: parse the config DTO, one-shot `GET {AgentUri}/probe` under `timeout`, `Ok=true`+latency on any valid `MTConnectDevices` response, `Ok=false`+message on TCP/HTTP/timeout failure. **Never throw** (per `IDriverProbe` contract).
- **No AdminUI `IDriverBrowser` DI line** — browse is the universal browser (§4), already registered.
## 7. Resilience / timeout (the R2-01 frozen-peer lesson)
Every agent HTTP request MUST carry a **per-call deadline** — never an unbounded wait (the R2-01 S7 finding: an async read that ignores its socket timeout wedges the poll loop against a frozen peer). Concretely:
- **`/probe` and `/current`:** wrap each call in a `CancellationTokenSource(RequestTimeoutMs)` linked to the caller's `ct`; a timeout surfaces as a `Bad`-coded snapshot (read) or a Faulted init (probe), never a hang. Set `HttpClient.Timeout` **and** a linked CTS (belt-and-suspenders — `HttpClient.Timeout` doesn't cover the response-body read of a streamed multipart).
- **`/sample` long-poll stream watchdog:** the stream is intentionally long-lived, so `HttpClient.Timeout` cannot bound it. Instead run a **watchdog**: if no chunk **and** no keep-alive heartbeat arrives within `HeartbeatMs × N` (e.g. 3× the Agent heartbeat), treat the stream as dead → cancel it, transition `HostState.Stopped`, and reconnect. The Agent's own `heartbeat` on the multipart boundary is the liveness signal; absence past the watchdog window is the frozen-peer detector.
- **Reconnect / backoff:** geometric backoff (`MinBackoffMs`→`MaxBackoffMs`) on stream drop or connect failure, mirroring `ModbusReconnectOptions`. On reconnect, re-baseline via `/current` then resume `/sample` from the fresh `nextSequence`.
- All capability calls flow through the existing Phase-6.1 `IDriverCapabilityInvoker` resilience pipeline (Read/Discover/Subscribe/Probe auto-retry); the per-call deadlines above are the driver-internal floor beneath that pipeline.
## 8. Test fixtures
1. **Canned XML unit fixtures (bulk of coverage, no network).** Capture one `MTConnectDevices` (probe) + a couple of `MTConnectStreams` (current + sample, incl. one with a forced `nextSequence` gap) from a demo agent into `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/`. Drive through the `IMTConnectAgentClient` fake to pin: `DiscoverAsync` tree shape + leaf `FullName==dataItemId`, `MTConnectDataTypeInference` table, observation indexing, `UNAVAILABLE→Bad`, CONDITION→state-word, multipart chunk framing, and ring-buffer re-baseline paging.
2. **Dockerized `mtconnect/cppagent` (reproducible integration fixture).** Add `tests/.../Docker/docker-compose.yml` with the **`project: lmxopcua`** label on every service, seeded with a canned `Devices.xml` (+ an SHDR simulator or the built-in adapter), exposed on the shared docker host `10.100.0.35`; drive it via `lmxopcua-fix up mtconnect` + `sync`. Env-gated integration suite (`*.IntegrationTests`), skips cleanly when the fixture is down — the analog of the Modbus/S7 sims.
3. **Public demo agent (manual live smoke only).** `https://demo.mtconnect.org/` (or NIST `smstestbed` `Devices.xml`) for a real-shape browse/read/stream smoke. Internet-dependent, not in CI.
Live-verify the browse picker on docker-dev per the universal-browser discipline: open the `/uns` TagModal picker for an MTConnect driver, confirm the Device→Component→DataItem tree renders and a picked leaf commits `TagConfig.FullName = <dataItemId>` (Razor binding bugs pass unit tests — always `/run` it).
## 9. Phasing + effort
- **P1 (this design) — Agent MVP, ~1–1.5 wk with TrakHound (~2.5–3 wk hand-rolled):** `.Contracts` + `Driver.MTConnect` (`IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+`IHostConnectivityProbe`+`IRediscoverable`), `MTConnectDriverProbe`, `SupportsOnlineDiscovery=true` (browse is free via universal browser — no browser code), typed editor + map/validator entries, 3-line Host registration, canned-XML unit suite + cppagent fixture. CONDITION as `String`.
- **P1.5 fast-follow:** CONDITION → native Part-9 alarms via `IAlarmSource` (Galaxy pattern); `TIME_SERIES` SAMPLE arrays; EVENT controlled-vocab → OPC UA enumerations.
- **P2 (on demand) — SHDR adapter ingest:** add `SourceMode: "Agent" | "Shdr"`; SHDR opens the raw pipe-delimited TCP socket (`:7878`). **Loses auto-discovery** (no device model → `SupportsOnlineDiscovery` must return `false` in SHDR mode → picker falls back to manual entry; tags authored like Modbus). Niche.
Detailed risk register (weak typing, CONDITION analog, TrakHound license, demo-agent CI flakiness, ring-buffer overflow, netstandard2.0-on-.NET10) is in [research §7](../research/drivers/mtconnect-agent.md#7-effort--risk--phasing) — not repeated here.
docs/plans/2026-07-15-omron-driver-design.md
+524
View File
@@ -0,0 +1,524 @@
# Omron PLC Driver — Executable Implementation Design
**Status:** Build-ready design, 2026-07-15. Turns the research report
[`docs/research/drivers/omron.md`](../research/drivers/omron.md) into a concrete implementation plan.
**Kind:** Standard **Equipment-kind driver** (same shape as Modbus / S7 / AbCip / TwinCAT / FOCAS).
Points are ordinary equipment `Tag`s bound to the driver via `TagConfig.FullName`, authored on the
`/uns` Tags tab. No alias machinery, no bespoke namespace kind.
> **Primary reuse target:** `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.AbCip/` (+ `.AbCip.Contracts/`).
> Omron NJ/NX speak CIP, so the CIP path reuses ~70–80% of AbCip's libplctag body. Read this doc
> alongside `AbCipDriver.cs`, `AbCipDriverFactoryExtensions.cs`, `AbCipEquipmentTagParser.cs`, and
> `IModbusTransport.cs` (the hand-rolled-transport template for FINS).
---
## 1. Motivation + transport decision
Omron exposes **two entirely different Ethernet protocols** depending on controller generation, and
they cover **disjoint hardware** — neither subsumes the other. We support **both, CIP-first.**
| Transport | Port / framing | Controller families | Addressing model | Online-browsable? |
|---|---|---|---|---|
| **EtherNet/IP CIP** | TCP/UDP **44818**, CIP explicit messaging | **NJ / NX / NY (Sysmac)** | **Named variables** (symbolic tags), like Rockwell Logix | **No** (libplctag `@tags` returns `ErrorUnsupported` — §4) |
| **FINS** | FINS/TCP + FINS/UDP on **9600** | **CJ / CS / CP / CV**, NSJ | **Memory areas** (CIO/WR/HR/AR/DM/EM) + word/bit offset | **No** (flat memory, no symbol table) |
**Decision — ship CIP first, FINS second:**
1. CIP targets the modern/current Omron line (NJ/NX/NY) most likely in new installs.
2. The CIP path **reuses the already-hardened AbCip stack** (libplctag, `PollGroupEngine`,
`EquipmentTagRefResolver`, resilience seam, status mapping) — fastest path to a working
read/write driver. Omron NJ/NX "variables" are CIP symbolic tags reached exactly like
ControlLogix, which is why libplctag already speaks it via `plc=omron-njnx`.
3. FINS is simpler as a protocol but is a **net-new transport codec** with zero reuse, targeting
legacy hardware.
Full protocol background + sources: [`docs/research/drivers/omron.md`](../research/drivers/omron.md) §1.
---
## 2. Project layout
Two projects, mirroring the AbCip split (`Driver` + zero-dependency `Contracts` leaf). **Both
transports live in one driver assembly** behind a single `DriverType = "Omron"`, discriminated by a
`transport` field in config — the two transports share the `IDriver` shell, poll wiring, resilience
seam, and `EquipmentTagRefResolver`, and only diverge at the wire codec.
```
src/Drivers/
ZB.MOM.WW.OtOpcUa.Driver.Omron.Contracts/ # options, enums, equipment-tag parser (NO package refs)
OmronTransport.cs # enum { Cip, Fins }
OmronDriverOptions.cs # OmronDriverOptions + OmronDeviceOptions + OmronTagDefinition + OmronProbeOptions
OmronDataType.cs # enum (Bool/SInt/Int/DInt/LInt/USInt/UInt/UDInt/ULInt/Real/LReal/String/Dt)
OmronMemoryArea.cs # enum { Cio, Wr, Hr, Ar, Dm, Em } (FINS)
OmronEquipmentTagParser.cs # TagConfig-JSON → OmronTagDefinition (mirror AbCipEquipmentTagParser)
ZB.MOM.WW.OtOpcUa.Driver.Omron/ # driver body
OmronDriver.cs # IDriver + capability interfaces; dispatches to a transport
OmronDriverFactoryExtensions.cs # Register() + ParseOptions() + config DTOs
OmronDriverProbe.cs # IDriverProbe (Test Connect) — transport-aware
OmronStatusMapper.cs # wire status → OPC UA StatusCode (shared)
OmronHostAddress.cs # omron://gw/path and fins://ip:port/net.node.unit parsers
Cip/
IOmronCipRuntime.cs # tag-runtime seam (mirror IAbCipTagRuntime) — the CI test seam
LibplctagOmronRuntime.cs # libplctag.NET impl, plc=omron-njnx
OmronCipTransport.cs # per-device tag-handle cache, read/write/probe
Fins/
IFinsTransport.cs # framed request/response seam (mirror IModbusTransport)
FinsTcpTransport.cs # hand-rolled FINS/TCP + node-address handshake
FinsUdpTransport.cs # hand-rolled FINS/UDP (no handshake)
FinsFrame.cs # header build/parse, 0x0101 read / 0x0102 write, area-code table
FinsAreaCodeTable.cs # (OmronMemoryArea, bit?) → area code, CPU-family-parameterised
```
### Library + license
| Path | Library | License | Notes |
|---|---|---|---|
| **CIP** | **`libplctag.NET`** (`<PackageReference Include="libplctag"/>`), attribute string `plc=omron-njnx` | native core dual MPL-2.0 / LGPL-2.1; .NET wrapper **MPL-2.0** | **Zero new license review** — the identical dependency AbCip already ships. |
| **FINS** | **Hand-rolled** (no third-party dep) | our own code | FINS framing is ~200 lines, fully documented (W227 + Wireshark dissector); mirrors `ModbusTcpTransport`. |
**Avoid `HslCommunication`.** The maintained builds are **commercial-paid**; the "MIT" claim in
casual sources refers to a stale, abandoned community fork
([`HslCommunication-Community`](https://github.com/HslCommunication-Community/HslCommunication-Community)),
not current releases. Use it only as a *protocol reference* to cross-check the hand-rolled FINS
framing — never as a dependency.
`Driver.Omron.csproj` mirrors `Driver.AbCip.csproj`: `net10.0`, `TreatWarningsAsErrors`,
`ProjectReference` to `Driver.Omron.Contracts`, `Core.Abstractions`, `Core`; a single
`<PackageReference Include="libplctag"/>`; `InternalsVisibleTo` the test project.
`Driver.Omron.Contracts.csproj` has **NO package references** — `ProjectReference` only to the
zero-dependency `Core.Abstractions` leaf (for the shared `TagConfigJson` readers), exactly like
`AbCip.Contracts`.
### Shared CIP core — defer
The research flags ~70–80% AbCip reuse and asks whether to extract a shared `Cip.Core`. **Decision:
keep Omron self-contained in v1** (copy AbCip's tag-runtime/poll/status patterns into
`Driver.Omron/Cip/` and specialise). Rationale: (a) a premature shared-core extraction couples two
drivers before we know Omron's real wire deltas (string encoding, array quirks, Network-Publish); (b)
Omron's tag-list enumerator is *not* shareable with Rockwell anyway (§4). **Revisit a shared
`ZB.MOM.WW.OtOpcUa.Driver.Cip.Core` extraction only if/when a third CIP driver lands or the Omron
CIP body proves byte-identical to AbCip's after the live gate** — track as a follow-up, not a v1 gate.
The deltas from AbCip are small and localized: the `plc=omron-njnx` attribute string, Omron's data-type
code set (§3), string encoding, and the browse story.
---
## 3. Capability mapping
`OmronDriver` implements the same capability set as `AbCipDriver`:
```csharp
public sealed class OmronDriver : IDriver, IReadable, IWritable, ITagDiscovery, ISubscribable,
IHostConnectivityProbe, IPerCallHostResolver, IDisposable, IAsyncDisposable
```
(No `IAlarmSource` in v1 — Omron has no analog to AbCip's ALMD projection; alarms can be authored as
scripted alarms on the equipment page like any flat-address driver.)
`InitializeAsync` reads the parsed `OmronDriverOptions`, and **per device** selects a transport
implementation by `device.Transport`:
- `Cip` → `OmronCipTransport` (libplctag tag-handle cache).
- `Fins` → `FinsTcpTransport` / `FinsUdpTransport` (framed socket codec).
Both transports expose the same internal seam the driver body calls
(`ReadAsync` / `WriteAsync` / `ProbeReadAsync`), so `OmronDriver`'s `IReadable`/`IWritable`/poll code
is transport-agnostic and routes by the resolved tag's `Transport`.
| Capability | CIP (NJ/NX) | FINS (CJ/CS/CP) |
|---|---|---|
| **`InitializeAsync`** (connect) | libplctag Forward-Open per device; attribute `protocol=ab-eip&gateway=<ip>&path=<cip-path>&plc=omron-njnx&name=<tag>` — via the .NET wrapper this is typed `Tag` properties (`Protocol.ab_eip` + `PlcType.Omron`), and AbCip's `LibplctagTagRuntime.MapPlcType` **already maps** `"omron-njnx" → PlcType.Omron` on the shipped libplctag 1.5.2 | open TCP `:9600` + FINS/TCP node-address handshake (cache assigned client node); UDP variant skips the handshake |
| **`IReadable`** | reuse AbCip per-tag runtime: create handle → `ReadAsync` → `GetStatus` → decode | build `0x0101` Memory Area Read (area code + 3-byte address + word count), parse 2-byte end-code + payload; multi-word reads for 32/64-bit |
| **`IWritable`** | full r/w; BOOL-in-word + array via AbCip's `EncodeValue`/bit-RMW paths | `0x0102` Memory Area Write; bit writes use bit-access area codes (no RMW needed — bit codes address bits directly) |
| **`ISubscribable`** | **poll-based** via shared `PollGroupEngine` (no native push) | **poll-based** via `PollGroupEngine` (FINS has no subscription) |
| **`ITagDiscovery`** | **authored-only** — emit pre-declared tags; `SupportsOnlineDiscovery=false` (§4). Sysmac tag-export importer feeds these tags offline (§4) | **authored-only**; `SupportsOnlineDiscovery=false` |
| **`IHostConnectivityProbe`** | reuse AbCip probe-loop: cheap tag read at interval → `HostState` transitions | cheap DM-word read at interval → `HostState` transitions |
The poll/subscribe wiring copies AbCip verbatim:
```csharp
_poll = new PollGroupEngine(
reader: ReadAsync,
onChange: (h, r, snap) => OnDataChange?.Invoke(this, new DataChangeEventArgs(h, r, snap)),
onError: HandlePollError,
backoffCap: TimeSpan.FromSeconds(30)); // 05/STAB-8 fleet-wide cap
```
`ResolveHost` routes each reference to its device host via `EquipmentTagRefResolver` (per-host
resilience isolation — a broken device B must not trip device A's breaker), copied from
`AbCipDriver.ResolveHost`.
### 3.1 Data-type mapping
Omron CIP (NJ/NX) uses IEC 61131 types that align with the existing `OmronDataType` (modelled on
`AbCipDataType`); FINS is raw words the TagConfig must type explicitly.
| `OmronDataType` | CIP (NJ/NX) native | FINS (word interpretation) | `DriverDataType` / OPC UA |
|---|---|---|---|
| `Bool` | BOOL | bit-area read, or bit N of a word | Boolean |
| `SInt` / `USInt` | 1-byte | low byte of a word | SByte / Byte |
| `Int` / `UInt` | 2-byte | 1 word | Int16 / UInt16 |
| `DInt` / `UDInt` | 4-byte | 2 words (word-order matters) | Int32 / UInt32 |
| `LInt` / `ULInt` | 8-byte | 4 words | Int64 / UInt64 |
| `Real` | IEEE-754 32 | 2 words | Float |
| `LReal` | IEEE-754 64 | 4 words | Double |
| `String` | CIP string | word-packed ASCII, fixed length | String |
| `Dt` | Sysmac DATE_AND_TIME/TIME | banked words | DateTime (best-effort) |
Arrays: `ValueRank=1`, `ArrayDim` from `arrayLength`, mirroring AbCip's explicit `IsArray` flag (a
1-element array is still an array — `ElementCount` alone can't carry the signal).
**UDT/structure scoping (CIP, v1):** structure *members* are addressed leaf-wise via a dotted
`tagName` (e.g. `Conveyor.Speed` — a Sysmac structure-member path resolved symbolically by
libplctag), each authored as its own typed tag. **Whole-structure reads are out of scope in v1** —
`OmronDataType` deliberately has no `Structure` marker (unlike `AbCipDataType`), because AbCip's
whole-UDT path rides on the Logix Template Object decode (`AbCipTemplateCache` /
`CipTemplateObjectDecoder`), which is Rockwell-specific and not known to apply to Omron's structure
metadata. Whether dotted member access holds for all NJ/NX structure shapes is a live-gate item
(§9).
**Word/byte-order caveat (FINS only):** Omron stores multi-word values with a specific word order
that differs from naïve concatenation. The FINS codec exposes a **per-tag `wordSwap`** option (same
class of concern S7/Modbus already handle). CIP via libplctag handles endianness internally, so no
`wordSwap` on the CIP path.
**Network Publish gate (CIP):** On NJ/NX a variable is only reachable over the network if the
programmer set its **Network Publish** attribute (`Publish Only`) in **Sysmac Studio**. Tags without
it are invisible to any external client — including our reads. Document this loudly in
`docs/drivers/Omron.md`: a CIP read of an unpublished variable fails at the wire, not a config bug.
---
## 4. Browse story — reconciliation with the universal browser
This is the single most important finding, and it must be stated honestly against the universal
discovery-browser design (`docs/plans/2026-07-15-universal-discovery-browser-design.md`).
### 4.1 Both transports set `SupportsOnlineDiscovery = false`
The universal `DiscoveryDriverBrowser` (Tier 1) works by running a driver's `DiscoverAsync` against a
`CapturingAddressSpaceBuilder` and re-presenting the captured tree. It is gated on
`ITagDiscovery.SupportsOnlineDiscovery` — a default-`false` interface member that does **not exist
yet**; the Wave-0 universal browser adds it (today `ITagDiscovery` carries only the
`RediscoverPolicy` default member). That gate exactly encodes
"does `DiscoverAsync` enumerate from the *device*, or merely replay pre-declared tags?"
**For Omron, `DiscoverAsync` can only replay pre-declared/authored tags on BOTH transports:**
- **CIP (NJ/NX):** libplctag's `@tags` walker (CIP Symbol Object class `0x6B`) — the exact mechanism
AbCip uses for controller-tag enumeration — returns **`ErrorUnsupported` / `PLCTAG_ERR_NOT_FOUND`
on Omron NJ/NX**. Omron exposes its published-variable list through a *different* CIP object/service
than Rockwell, which libplctag has not implemented (upstream
[libplctag #466](https://github.com/libplctag/libplctag/issues/466),
[libplctag.NET #371](https://github.com/libplctag/libplctag.NET/issues/371)). So online CIP
enumeration is **not available** the way it is on Rockwell.
- **FINS:** flat memory banks, no symbol/metadata table on the wire — nothing to enumerate, ever.
Therefore Omron **does not** opt into `SupportsOnlineDiscovery`. Concretely, `OmronDriver` inherits
the default:
```csharp
// OmronDriver: no override of SupportsOnlineDiscovery → stays false (authored replay).
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once;
```
Consequence per the universal-browser design §5/§7: Omron sits in the **"No — not browsable"** row
alongside Modbus/S7/MELSEC/AbLegacy. The universal browser's `CanBrowse` returns false, the AdminUI
renders **manual entry** (no Browse button), and there is **no bespoke `IBrowseSession` in v1.** This
is the honest reconciliation: the universal browser does **not** help Omron, because Omron cannot
device-enumerate.
### 4.2 The v1 browse experience — offline Sysmac tag-export importer (CIP)
The lowest-risk, highest-value browse story for CIP is **offline import**, not a live browser:
- NJ/NX projects export the published global-variable table from **Sysmac Studio** (CSV / tag file).
- Build an **AdminUI importer** that parses this export into candidate `OmronTagDefinition`s (name +
data type + array shape) and authors them as equipment tags (or fills a driver-config `Tags` list).
- This needs **no online CIP enumeration** and dodges the entire libplctag-can't-list-NJ/NX problem.
Placement: a small importer under the AdminUI Tags/driver surface (parse → preview → author), NOT an
`IDriverBrowser`. Scoped as **Phase 3** (below). FINS has no equivalent — **manual entry only**
(memory-area dropdown + word/bit offset in the typed editor).
### 4.3 Future live CIP browse (explicitly out of scope for v1)
If libplctag adds Omron NJ/NX variable listing, *or* we hand-write the Omron CIP tag-list service
(Rockwell-different, research-grade), a bespoke `Driver.Omron.Browser` implementing
`IDriverBrowser` (`DriverType="Omron"`, CIP config only) could be added later — it would **override**
the universal fallback per the two-tier resolution. It would reuse the `BrowseNode`/`IBrowseSession`
plumbing but **not** the wire enumerator (Omron's tag-list service differs from Rockwell's). Not built
in v1. Even then, only Network-Published variables would be visible.
**The driver is fully usable without any browser** — pre-declared/imported tags always work, and
unmapped types fall back to the raw-JSON TagConfig editor. Browse is convenience, not correctness.
---
## 5. TagConfig + driver-config JSON
Follows the AbCip equipment-tag convention (`AbCipEquipmentTagParser`): a leading `{` marks a
TagConfig blob; camelCase property names; strict enum reads via `TagConfigJson.TryReadEnumStrict` (a
typo'd enum **rejects** the tag → `BadNodeIdUnknown`, never a silently-wrong Good). A `transport`
discriminator distinguishes the two shapes.
### 5.1 Driver config (bound to `DriverConfig` at `DriverHost.RegisterAsync`)
```jsonc
{
"timeoutMs": 2000,
"devices": [
{ "transport": "Cip", "hostAddress": "omron://10.100.0.40/1,0", "deviceName": "NJ501-Line1" },
{ "transport": "Fins", "hostAddress": "fins://10.100.0.41:9600/0.1.0", "cpuFamily": "CJ2", "deviceName": "CJ2M-Legacy" }
],
"probe": { "enabled": true, "intervalMs": 5000, "timeoutMs": 2000, "probeReference": "Conveyor.Heartbeat" },
"tags": [ /* optional pre-declared tags; equipment tags are authored per-tag instead */ ]
}
```
- `OmronDeviceOptions`: `Transport`, `HostAddress`, optional `DeviceName`, `CpuFamily` (FINS
area-code table selector). Per-device routing keyed on `HostAddress` (via `IPerCallHostResolver`),
exactly like `AbCipDeviceOptions`.
- `OmronHostAddress.TryParse` accepts **both** forms: `omron://gateway[:port]/cip-path` (CIP) and
`fins://ip[:port]/net.node.unit` (FINS). Malformed → device fails init (never silently connects to
nothing), matching AbCip.
### 5.2 Per-tag TagConfig — CIP (NJ/NX)
```json
{
"transport": "Cip",
"deviceHostAddress": "omron://10.100.0.40/1,0",
"tagName": "Conveyor.Speed",
"dataType": "Real",
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
`tagName` is the Sysmac global-variable name (near-identical to AbCip's `tagPath`; CIP path reuses
AbCip's `isArray`/`arrayLength`/`writable`/`dataType` semantics verbatim).
### 5.3 Per-tag TagConfig — FINS
```json
{
"transport": "Fins",
"deviceHostAddress": "fins://10.100.0.41:9600/0.1.0",
"memoryArea": "DM",
"address": 100,
"bit": null,
"dataType": "Int",
"wordSwap": false,
"isArray": false,
"arrayLength": 1,
"writable": true
}
```
- `memoryArea` ∈ `{Cio, Wr, Hr, Ar, Dm, Em}` (+ optional `emBank` for extended memory). The codec
maps `(memoryArea, bit==null ? word : bit)` to the correct FINS area code via
`FinsAreaCodeTable` — a **CPU-family-parameterised** table (word vs bit codes are distinct; codes
vary slightly per CPU family, so the table is keyed on `device.CpuFamily`). Representative
word/bit codes: CIO `0xB0`/`0x30`, WR `0xB1`/`0x31`, HR `0xB2`/`0x32`, AR `0xB3`/`0x33`,
DM `0x82`/`0x02`, EM `0xA0+bank`/`0x20+bank`.
- `address` = word offset; `bit` = 0–15 for bit access (`null` ⇒ word access).
- `deviceHostAddress`'s `net.node.unit` triple is the FINS destination.
- `wordSwap` handles Omron multi-word ordering for 32/64-bit types.
### 5.4 `DriverAttributeInfo.FullName` per transport
`FullName` is the value the picker/importer commits as `TagConfig.FullName` and the driver resolves
via `EquipmentTagRefResolver`. For an **equipment tag**, `FullName` **is the raw TagConfig JSON blob**
(exactly as AbCip does — `AbCipEquipmentTagParser` sets `Name: reference`). The parser reads the
`transport` discriminator first, then the transport-specific fields. For a **pre-declared tag**,
`FullName` is the tag's `Name`. `OmronDataType.ToDriverDataType()` (extension in the driver, mirroring
`AbCipDataTypeExtensions`) maps to `DriverDataType`; `SecurityClass` = `Operate` when `writable` else
`ViewOnly`.
---
## 6. Typed editor + validator
Add **one typed editor per transport-aware model**, registered by the `"Omron"` DriverType:
- **Model:** `src/Server/.../AdminUI/Uns/TagEditors/OmronTagConfigModel.cs` — pure
`FromJson`/`ToJson`/`Validate`, preserving unknown keys via `TagConfigJson` (copy
`AbCipTagConfigModel`). It holds a `Transport` enum plus the union of CIP + FINS fields; the razor
shell shows CIP fields or FINS fields based on the selected `Transport`. `Validate()`:
CIP ⇒ `tagName` required; FINS ⇒ `memoryArea` set + `address >= 0`.
- **Editor:** `src/Server/.../AdminUI/Components/Shared/Uns/TagEditors/OmronTagConfigEditor.razor`
— thin razor shell over the model (copy `AbCipTagConfigEditor`), with a transport toggle that
swaps the field set (CIP: tagName + dataType + array; FINS: memoryArea + address + bit + dataType +
wordSwap + array).
- **Register in both maps** (one line each):
- `TagConfigEditorMap.cs`: `["Omron"] = typeof(Components.Shared.Uns.TagEditors.OmronTagConfigEditor),`
- `TagConfigValidator.cs`: `["Omron"] = j => OmronTagConfigModel.FromJson(j).Validate(),`
### The `JsonStringEnumConverter` enum-serialization trap (systemic — do NOT skip)
Per the known systemic AdminUI bug: **all driver pages serialize enums NUMERICALLY by default, but
the driver-side DTOs are string-typed** (`ParseEnum`/`TryReadEnumStrict` read enum *names*). An
AdminUI-authored config with any enum field (`transport`, `dataType`, `memoryArea`, `cpuFamily`)
serialized as an integer will **fault the driver** at parse. **Every Omron surface that serializes
config JSON must emit enum names, not numbers:** the `OmronDriverPage.razor` config serializer and
`OmronDriverProbe` attach `new JsonStringEnumConverter()` (mirror `AbCipDriverPage.razor`'s
serializer options and `AbCipDriverProbe._opts`, which both already set
`Converters = { new JsonStringEnumConverter() }`); `OmronTagConfigModel.ToJson` writes enum names
via the AdminUI `TagConfigJson.Set` helper, exactly as `AbCipTagConfigModel.ToJson` does (no
converter needed on that path). Enums serialize as their **name string**, matching
the driver's strict-name enum reads. Confirm this in the live-`/run` verify — unit tests do not catch
it (Blazor binding + numeric-enum serialization pass all unit tests).
Also add Omron to `DriverTypePicker.razor` (`_types` list) so the New-driver card grid shows it, and
add an `OmronDriverPage.razor` under `Components/Pages/Clusters/Drivers/` (copy `AbCipDriverPage.razor`)
plus the `DriverEditRouter` mapping and `DriverIdentitySection`/`EquipmentTagConfigInspector` "Omron"
entries where AbCip appears.
---
## 7. Factory + registration
`OmronDriverFactoryExtensions` mirrors `AbCipDriverFactoryExtensions`:
```csharp
public static class OmronDriverFactoryExtensions
{
public const string DriverTypeName = "Omron";
public static void Register(DriverFactoryRegistry registry)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, CreateInstance); // Tier defaults to A, matching AbCip
}
internal static OmronDriver CreateInstance(string driverInstanceId, string driverConfigJson)
=> new(ParseOptions(driverInstanceId, driverConfigJson), driverInstanceId);
internal static OmronDriverOptions ParseOptions(string driverInstanceId, string driverConfigJson) { /* … */ }
}
```
`ParseOptions` deserializes into config DTOs and maps to `OmronDriverOptions` — copy AbCip's
`ParseEnum<T>` (name-based, throws on unknown) and `PositiveTimeoutOrDefault` (clamps
`timeoutMs: 0` to the default so a misconfigured zero can't fault every op — the R2-01 sibling
hardening). `InitializeAsync` re-parses the config JSON on reinit so a changed config takes effect
(copy AbCip's re-parse-in-`InitializeAsync` pattern).
**Wiring (one line each):**
- `DriverFactoryBootstrap.Register(...)`: `Driver.Omron.OmronDriverFactoryExtensions.Register(registry);`
- `DriverFactoryBootstrap.AddOtOpcUaDriverProbes(...)`:
`services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, OmronProbe>());` (+ a
`using OmronProbe = Driver.Omron.OmronDriverProbe;` alias). Probes MUST be wired on admin nodes —
Test Connect is an admin-pinned singleton.
`OmronDriverProbe` (`IDriverProbe`, `DriverType="Omron"`) is transport-aware: for a CIP device do the
two-phase AbCip probe (bare TCP `:44818` preflight → libplctag `plc=omron-njnx` Forward-Open, treat
tag-not-found statuses as *reachable*); for a FINS device do a TCP `:9600` connect + a single
`0x0101` DM-word read (or the node-address handshake) to confirm the endpoint speaks FINS.
---
## 8. Resilience / timeout
Inherit the fleet-hardened patterns; the R2-01 frozen-peer lesson is non-negotiable for **both**
transports:
- **Per-op deadline on every read and write.** A frozen peer that accepts TCP but never answers must
not wedge a poll. CIP: libplctag `Tag.Timeout` from `options.Timeout` (`PositiveTimeoutOrDefault`
clamps a zero, as AbCip does — the libplctag setter throws on non-positive). FINS: an explicit
per-call `CancellationTokenSource.CancelAfter(options.Timeout)` around every socket
read/write, and a socket `ReceiveTimeout`, so an async FINS read cannot ignore the deadline
(the exact gap the S7 R2-01 fix closed — async reads that ignored the socket timeout).
- **Evict-on-failure parity with AbCip.** A non-zero wire status or transport exception **evicts** the
cached handle/socket so the next call re-creates it (copy `EvictRuntime`). FINS closes + reopens the
socket; CIP disposes + recreates the libplctag `Tag`.
- **Per-device connection backoff** (`ConnectionBackoff`, 05/STAB-8): inside an open backoff window a
data-path caller fails fast (no connect), reset on success — copy AbCip's `device.Backoff` usage.
- **Poll backoff cap** 30 s fleet-wide; `HandlePollError` degrades health to `Degraded` preserving
`LastSuccessfulRead`, never downgrading `Faulted`.
- **`WriteIdempotent` flagging.** Default `false` (a pulse/counter-advance/recipe-step write must not
auto-replay on a write timeout). Operators flag only tags whose semantics make replay safe
(level-set holding values, analog set-points) via the tag's `writeIdempotent` field → carried into
`DriverAttributeInfo.WriteIdempotent` (exactly as AbCip's pre-declared tags carry it). **Honest
status of the retry gate:** the invoker seam exists
(`IDriverCapabilityInvoker.ExecuteWriteAsync(hostName, isIdempotent, …)` — the non-idempotent arm
forces `RetryCount = 0`), but the host's sole write dispatch
(`DriverInstanceActor.HandleWriteAsync`) currently hardcodes `isIdempotent: false`; per-tag opt-in
is the documented-but-unshipped forward path (see the `WriteIdempotentAttribute` remarks). So Omron
carries the flag from day one, and flagged tags gain retry when that fleet-wide plumbing lands —
same position as AbCip today. FINS bit/word writes to a set-point are the typical idempotent case;
a FINS write to a command register is not.
Health surface: `GetHealth()` returns `DriverHealth`; transitions on read/write success/failure
exactly as AbCip.
---
## 9. Test fixtures
The honest gap: **there is no free NJ/NX CIP simulator.**
### CIP (NJ/NX)
- **Primary CI surface — hand-rolled `IOmronCipRuntime` fake** (the same seam pattern AbCip uses via
`IAbCipTagRuntime`/factory injection). Drives read/write/decode/array/BOOL-in-word/status-mapping
unit tests with **no hardware and no container**. This is where correctness of the driver *body* is
proven.
- **Live gate — env-gated `Category=LiveIntegration`** against real NJ/NX hardware, skipping cleanly
when the env var is absent (the pattern the HistorianGateway live suite uses). This is the **only**
way to validate the real `plc=omron-njnx` libplctag wire path, Network-Publish behavior, and Omron
string/array/UDT quirks. Env vars e.g. `OMRON_CIP_ENDPOINT` / `OMRON_CIP_TEST_TAG` /
`OMRON_CIP_WRITE_SANDBOX_TAG`. **Flag as an infra-gated known limitation** in the driver doc + this
design until an NJ/NX is on the bench. (Omron's own CX-Simulator is Windows-only + license-gated in
CX-One — not a CI fixture.)
### FINS
- **Primary — hand-rolled byte-level `IFinsTransport` fake** (mirror the Modbus `IModbusTransport`
fake): deterministic, no container, the most valuable surface. Covers `0x0101`/`0x0102` framing,
area-code table, word-swap, end-code parsing, and the FINS/TCP handshake.
- **Integration fixture — open-source FINS simulators** wrapped in a Dockerfile under
`tests/Drivers/.../Omron/Docker/` with the `project=lmxopcua` label, deployed to the Linux Docker
host `10.100.0.35` via `lmxopcua-fix sync omron` (repo compose file is the source of truth). Both
known OSS sims — [`hiroeorz/omron-fins-simulator`](https://github.com/hiroeorz/omron-fins-simulator)
and [`ahmadfarisfs/fins_simulator_omron`](https://github.com/ahmadfarisfs/fins_simulator_omron) —
are UDP-oriented and ship no image, so wrap them; FINS/TCP-handshake coverage stays on the
hand-rolled fake.
Contracts-level unit tests for `OmronEquipmentTagParser` + `OmronTagConfigModel` round-trips
(strict-enum reject, unknown-key preservation, both transport shapes) run everywhere, no fixture.
---
## 10. Phasing + effort
**Omron is scheduled LAST of the active driver set** — the no-free-CIP-sim gap means CIP correctness
can't be proven in CI, so it should land when live NJ/NX hardware is available for the gate.
| Phase | Scope | Effort | Gate |
|---|---|---|---|
| **P1 — Omron CIP r/w (NJ/NX)** | `Driver.Omron` + `.Contracts`, `DriverType="Omron"`, `OmronCipTransport` on libplctag `plc=omron-njnx`; Connect/Read/Write/Subscribe(poll)/Probe; authored + equipment tags; typed editor + validator; factory + registration | Low–Medium (70–80% AbCip reuse) | hand-rolled runtime fake (CI) + env-gated live gate (hardware) |
| **P2 — Omron FINS r/w (CJ/CS/CP)** | `IFinsTransport` codec (TCP+UDP, `0x0101`/`0x0102`, area-code table, word-swap) behind the same `IDriver` shell; memory-area TagConfig + editor fields (no browse) | Medium (net-new codec, no dep/license risk) | byte-level fake (CI) + OSS FINS sim container |
| **P3 — Sysmac tag-export importer (CIP)** | AdminUI offline importer: parse Sysmac Studio variable export → author `OmronTagDefinition`s. **Not** a live browser. | Medium | parser unit tests on real export samples |
### Top risks
1. **No free NJ/NX CIP simulator** (the headline). CIP wire correctness (string/array/UDT/
Network-Publish) is provable only on live hardware behind the env-gated `LiveIntegration` suite →
**infra-gated known limitation** until an NJ/NX is on the bench. The hand-rolled fake proves the
driver *body*, not the *wire*.
2. **CIP browse is not free reuse.** libplctag's `@tags` walker returns `ErrorUnsupported` on Omron
NJ/NX; `SupportsOnlineDiscovery=false`, the universal browser doesn't apply, and browse ships as
an offline Sysmac-export importer (P3), with online browse deferred to a later research phase.
3. **FINS per-family variance** in memory-area codes + multi-word ordering — contained by the
CPU-family-parameterised `FinsAreaCodeTable` + per-tag `wordSwap`.
4. **Enum-serialization trap** — every config JSON surface must attach `JsonStringEnumConverter`, or
AdminUI-authored Omron configs fault the driver. Live-`/run` verify catches it; unit tests don't.
---
## Sources
See [`docs/research/drivers/omron.md`](../research/drivers/omron.md) §Sources for the full list
(Omron W506/W227/W596 manuals, libplctag #466 / libplctag.NET #371, Wireshark FINS dissector,
HslCommunication license situation, OSS FINS simulators).
</content>
</invoke>
docs/plans/2026-07-15-sql-poll-driver-design.md
+685
View File
@@ -0,0 +1,685 @@
# SQL poll driver (`Sql`) — executable implementation design
> **Status:** design, build-ready. Author: driver design sweep, 2026-07-15.
> **Supersedes/derives from research:** [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
> **Grounding read:** `CLAUDE.md` (Equipment-kind driver + `TagConfig.FullName`);
> `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/` (+ `.Contracts`) as the poll-driver template;
> `src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/` capability seams
> (`IDriver`, `ITagDiscovery`, `IReadable`, `ISubscribable`, `IHostConnectivityProbe`,
> `PollGroupEngine`, `EquipmentTagRefResolver`, `DataValueSnapshot`, `DriverAttributeInfo`);
> [`docs/plans/2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md)
> (bespoke `IBrowseSession` house style);
> [`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md)
> (universal browser — see §4.0 reconciliation).
The driver type string is **`Sql`** (short, consistent with `Modbus`/`S7`/`Galaxy`). This doc
uses `Sql` throughout; the research report's working name `SqlPoll` maps 1:1.
---
## 0. TL;DR — decisions locked for the build
| Question | Decision |
|---|---|
| **Provider ship-first** | **SQL Server** via `Microsoft.Data.SqlClient` (already at `6.1.1` in `Directory.Packages.props:53`) behind an ADO.NET `DbProviderFactory` + a `SqlProvider` enum + an `ISqlDialect` seam. **Zero new NuGet deps for P1.** Postgres (`Npgsql`) + ODBC P2; MySQL/Oracle later. |
| **Browse** | **Bespoke schema-walk `IBrowseSession`** in a `Driver.Sql.Browser` project (`INFORMATION_SCHEMA`, dialect-abstracted). The **universal Discover-backed browser does NOT fit** — see §4.0. `ITagDiscovery.SupportsOnlineDiscovery` stays **`false`**. |
| **Write in v1** | **NO — read-only v1.** `IWritable` deferred to P4 as an opt-in, triple-gated, parameterized-UPSERT mode (design in §3.4). |
| **Capabilities v1** | `IDriver`, `ITagDiscovery` (authored replay only), `IReadable`, `ISubscribable` (poll via `PollGroupEngine`), `IHostConnectivityProbe`. |
| **Top 2 risks** | (1) **SQL injection** — values are always bound `DbParameter`s; identifiers only from `INFORMATION_SCHEMA`-validated + dialect-quoted names, never string-concatenated tag input. (2) **Secrets** — connection strings resolved from env/secret store via `connectionStringRef`, never committed/logged (mirror `ServerHistorian__ApiKey`). |
| **Effort** | Small–Medium. P1 ≈ Modbus minus wire-codec plus the dialect seam + result-set slicing; most infra (`PollGroupEngine`, `EquipmentTagRefResolver<TDef>`, health state machine, factory shape) is reused. |
---
## 1. Motivation + scope
Many plants expose process/production data only as **SQL rows** — MES/ERP staging tables, LIMS,
custom app databases, historian summary tables. Today OtOpcUa has no way to surface those under the
unified Equipment address space; an operator has to stand up a bespoke bridge. The `Sql` driver
closes that gap: it polls arbitrary SQL tables/views on an interval and publishes selected
columns/rows as ordinary OPC UA variable nodes — the same authoring flow (equipment page → Tags tab
→ address picker) as Modbus or S7.
**Scope v1: read-only.** Writing back into MES/ERP staging is operationally risky (a stray
`UPDATE`/`INSERT` can corrupt a batch record, double-post a transaction, or fire a trigger with side
effects). Read-only removes an entire risk class from the first ship. An opt-in write mode is
designed in §3.4 for a later phase but is **not** built in v1.
**Out of scope (v1):** `IWritable`, `IAlarmSource`, `IHistoryProvider` (alarms/history aren't
expressible from a plain poll query). SQL-poll data can still be *historized* by the server via the
standard `TagConfig.isHistorized` flag — that's the server historian path, not a driver capability.
Full research + verdicts: [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
---
## 2. Project layout
Three new projects, mirroring the Modbus split (runtime / contracts / browser), plus test projects.
All `net10.0`, `Nullable`/`ImplicitUsings` enabled, `TreatWarningsAsErrors`, matching the repo
`.csproj` conventions.
| Path | Role |
|---|---|
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts/` | Options DTOs, `SqlProvider` enum, tag-model records, `SqlEquipmentTagParser`. **Zero transport deps** (refs only `Core.Abstractions`) so AdminUI + browser can reference it without dragging `Microsoft.Data.SqlClient`. |
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql/` | `SqlDriver` (runtime capabilities), `ISqlDialect` + `SqlServerDialect`, `SqlPollReader`, factory extensions. **Owns the `Microsoft.Data.SqlClient` package ref.** |
| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser/` | `SqlDriverBrowser : IDriverBrowser`, `SqlBrowseSession : IBrowseSession` (schema walk). Refs `Commons` + `Sql.Contracts` + **`Driver.Sql`** (for `ISqlDialect`/`SqlServerDialect` — the dialect's catalog SQL *is* the browse engine, so it must be shared, not duplicated; `Microsoft.Data.SqlClient` comes transitively). This runtime-project ref is a deliberate deviation from `OpcUaClient.Browser` (which refs only its Contracts and owns the OPC UA client packages itself): the browser still opens its own transient connection. |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests/` | Unit suite (SQLite fixture, §9). |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.IntegrationTests/` | Env-gated integration suite (central SQL Server, §9). |
| `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser.Tests/` | Browser unit tests (SQLite `PRAGMA` dialect). |
Add all six to `ZB.MOM.WW.OtOpcUa.slnx`.
### 2.1 `Driver.Sql.csproj` (sketch)
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<NoWarn>$(NoWarn);CS1591</NoWarn>
<RootNamespace>ZB.MOM.WW.OtOpcUa.Driver.Sql</RootNamespace>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core.Abstractions\ZB.MOM.WW.OtOpcUa.Core.Abstractions.csproj"/>
<ProjectReference Include="..\..\Core\ZB.MOM.WW.OtOpcUa.Core\ZB.MOM.WW.OtOpcUa.Core.csproj"/>
<ProjectReference Include="..\ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts\ZB.MOM.WW.OtOpcUa.Driver.Sql.Contracts.csproj"/>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Data.SqlClient" /> <!-- version from Directory.Packages.props -->
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="ZB.MOM.WW.OtOpcUa.Driver.Sql.Tests"/>
</ItemGroup>
</Project>
```
The capability seams + `PollGroupEngine` + `EquipmentTagRefResolver` all live in
`Core.Abstractions`, but the `Core.csproj` reference is **also** needed (Modbus carries it too):
the factory extensions (§7) take `DriverFactoryRegistry`, which lives in
`Core/Hosting/DriverFactoryRegistry.cs` (`ZB.MOM.WW.OtOpcUa.Core.Hosting`). The
SQLite unit-test project pulls `Microsoft.Data.Sqlite` (already `10.0.7` in the graph) — no product
dependency on SQLite.
### 2.2 Provider strategy — `DbProviderFactory` behind `ISqlDialect`
Use ADO.NET's provider-agnostic base types (`System.Data.Common`: `DbConnection`, `DbCommand`,
`DbParameter`, `DbDataReader`) obtained through a `DbProviderFactory`. A small `SqlProvider` enum
(not open-ended invariant strings) maps to a dialect that owns the two things the base types **can't**
abstract: identifier quoting and the metadata-catalog SQL.
```csharp
// Driver.Sql.Contracts
public enum SqlProvider { SqlServer, Postgres, MySql, Odbc, Oracle }
// Driver.Sql (public — the .Browser project consumes it; only implementations touch provider packages)
public interface ISqlDialect
{
SqlProvider Provider { get; }
DbProviderFactory Factory { get; } // SqlClientFactory.Instance (P1)
string QuoteIdentifier(string ident); // [x] / "x" / `x` (rejects/escapes embedded quote chars)
string LivenessSql { get; } // "SELECT 1" (Oracle: "SELECT 1 FROM DUAL")
string ListSchemasSql { get; } // browse Root
string ListTablesSql { get; } // browse Expand(schema) — parameterized by @schema
string ListColumnsSql { get; } // browse Expand(table)/Attributes — parameterized by @schema,@table
DriverDataType MapColumnType(string sqlDataType); // INFORMATION_SCHEMA DATA_TYPE → DriverDataType
}
```
**Ship-first: `SqlServerDialect` only** (`Microsoft.Data.SqlClient`, `SqlClientFactory.Instance`,
`[bracket]` quoting, `INFORMATION_SCHEMA` catalog). P2 adds `PostgresDialect` (`Npgsql`) + a SQLite
dialect for the browser tests (`PRAGMA table_info`, `sqlite_schema`) + `OdbcDialect`. `Npgsql` /
`System.Data.Odbc` / `Oracle.ManagedDataAccess.Core` are each a **new** `PackageVersion` gated behind
their phase — the P1 SQL-Server slice adds none. In .NET there is no `machine.config` provider
registry: factories come from each provider's static `Instance` singleton
(`SqlClientFactory.Instance`, `NpgsqlFactory.Instance`, `OdbcFactory.Instance`).
---
## 3. Capability mapping
`SqlDriver : IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe`.
### 3.1 `InitializeAsync` — open + validate
1. Deserialize `SqlDriverConfigDto` from `driverConfigJson` (via `SqlDriverFactoryExtensions`,
mirroring `ModbusDriverFactoryExtensions.CreateInstance`), using a shared `JsonSerializerOptions`
with **`JsonStringEnumConverter`** (§6 — the enum-serialization trap) and
`UnmappedMemberHandling.Skip`.
2. **Resolve the connection string** from `connectionStringRef` against the process config/env
(§8.2) — never from the committed JSON.
3. Select `ISqlDialect` from `options.Provider` (P1: only `SqlServer` is constructible; others throw
a clean "provider not available in this build").
4. **Validate**: open one `DbConnection` from `dialect.Factory`, run `dialect.LivenessSql` under
`CommandTimeout` + a linked CTS (§8.3), dispose. Success → `DriverState.Healthy`; failure →
`DriverState.Faulted` with `LastError` (same state machine as `ModbusDriver.InitializeAsync`).
5. Build the `PollGroupEngine` (reader delegate = §3.2) with `minInterval = 100ms`, an `onError`
sink routing to the health surface, and a `backoffCap` (Modbus/S7 precedent, 05/STAB-8).
**Pooling:** rely on ADO.NET built-in pooling (keyed by exact connection-string text). Do **not**
hold a long-lived connection — **open-use-dispose per poll pass** (`await using var conn = …`) so the
pool manages lifetime and a dropped/restarted server recovers transparently. Expose `Max Pool Size`
etc. only through the connection string.
**`ReinitializeAsync`** = teardown + init (Modbus pattern); no live sockets to preserve.
**`GetMemoryFootprint`** ≈ the cached parsed-tag map + last-value dictionaries (small). No symbol
tables to flush; `FlushOptionalCachesAsync` is a no-op returning `Task.CompletedTask`.
### 3.2 `IReadable` / `ISubscribable` — one query per group, not per tag
The efficiency story is **batching by query-group**. A naive "one SELECT per tag" hammers the DB;
instead the driver coalesces tags that share a source query, issues **one round-trip per group per
poll**, and slices the result back to per-tag snapshots — structurally identical to
`ModbusDriver.ReadCoalescedAsync` (group → single PDU → slice back).
```csharp
public async Task<IReadOnlyList<DataValueSnapshot>> ReadAsync(
IReadOnlyList<string> fullReferences, CancellationToken ct)
{
// 1. Resolve each ref to a SqlTagDefinition via EquipmentTagRefResolver<SqlTagDefinition>.
// 2. Group by GroupKey (model-dependent, §3.6).
// 3. Per group: open connection, build ONE parameterized command, ExecuteReaderAsync(ct),
// index rows, map each tag's cell → DataValueSnapshot.
// 4. Reassemble snapshots in INPUT order (PollGroupEngine contract: N in → N out).
// Per-tag failure = a Bad-coded snapshot, NOT an exception (IReadable contract).
// Whole call throws only if the driver itself is unreachable (→ engine backoff).
}
```
`ISubscribable.SubscribeAsync` delegates to `PollGroupEngine.Subscribe(refs, publishingInterval)`
with `ReadAsync` as the reader delegate — the engine owns the loop, the 100 ms interval floor,
capped-exponential failure backoff, and change-diffing; `OnDataChange` is raised from the engine's
`onChange`. `UnsubscribeAsync` → `engine.Unsubscribe(handle)`.
**Poll interval** is driver-level (`defaultPollInterval`, default 5 s) with optional per-tag/per-group
override; the OPC UA subscription's requested publishing interval is clamped up to the group's floor
by `PollGroupEngine`.
**Per-call deadline (mandatory — R2-01 frozen-peer lesson).** Every `DbCommand` sets
`CommandTimeout` (= `commandTimeout` option, seconds) **and** runs `ExecuteReaderAsync(linkedCt)`
where the linked CTS is bounded by `operationTimeout` (wall-clock). A frozen DB (partition, lock
wait, slow query) surfaces `BadTimeout` per group and lets the poll loop back off — it must **not**
wedge the poll thread. Both are required: the linked token gives real client-side cancellation;
`CommandTimeout` is the server-side backstop.
### 3.3 `ITagDiscovery` — authored replay only (NOT schema enumeration)
`DiscoverAsync` materializes exactly the authored tags — the `tags` array in options **plus** the
equipment tags contributed at deploy time (whose reference is their raw `TagConfig` JSON). It does
**not** enumerate the DB schema: schema enumeration is a browse-time concern (§4). Each authored tag
becomes an `IAddressSpaceBuilder.Variable(browseName, displayName, DriverAttributeInfo)` with
`FullName` = the tag reference string, `DriverDataType` from the tag's declared/inferred type,
`SecurityClass = ViewOnly` (read-only v1). `RediscoverPolicy => Once`.
**`SupportsOnlineDiscovery => false`** (default member — see §4.0).
### 3.4 `IWritable` — verdict: NOT in v1
**v1 does not implement `IWritable`.** Rationale in §1. The P4 opt-in design: a tag may carry a
`write` block describing a **parameterized** statement (never string-concatenated):
```jsonc
"write": {
"mode": "Update", // Update | Insert | Upsert | StoredProc
"sql": "UPDATE dbo.Setpoints SET value=@value, ts=@ts WHERE tag_name=@key",
"keyParam": "@key", "keyValue": "Line1.Speed", "valueParam": "@value"
}
```
`WriteAsync` binds the OPC UA write value to `@value` (typed via the tag's declared type) and executes
the tag's parameterized command. **Triple-gated:** (a) the node's `WriteOperate` role (standard
`NodeWriteRouter`), (b) a driver-level `allowWrites` master switch (default `false`), (c)
`TagConfig.writable`. Not auto-retried unless the tag is `WriteIdempotent`
(`DriverAttributeInfo.WriteIdempotent`): an `UPDATE … WHERE key=` set-value is idempotent; an
`INSERT` / counter `UPDATE x=x+1` is not. Because a failed inbound device write reverts the node to
its prior value (write-outcome self-correction, master `1d797c1c`), SQL writes — unlike Galaxy's
fire-and-forget — *can* surface a genuine failure, which is the honest behaviour.
### 3.5 `IHostConnectivityProbe`
One logical host (the DB server host:port from the connection string). `GetHostStatuses()` returns a
single `HostConnectivityStatus(server, HostState.Running|Stopped|Faulted, lastChangedUtc)` derived
from the last poll/liveness outcome; `OnHostStatusChanged` fires on Running↔Stopped transitions. This
lets the Core scope Bad-quality fan-out to the driver's subtree on a DB outage (same pattern the
Galaxy driver uses for Platform/AppEngine). Optional but cheap — recommended for v1.
### 3.6 Tag→value mapping models
Three models; **(a) and (b) ship in P1**, (c) is P3.
**(a) Key-value / EAV table** *(primary).* A `(tagname, value, timestamp)` table; a tag names a
`table`, `keyColumn`+`keyValue`, `valueColumn`, optional `timestampColumn`.
**GroupKey = `(table, keyColumn, valueColumn, timestampColumn)`.** All such tags fold into one
`SELECT <keyColumn>, <valueColumn>, <timestampColumn> FROM <table> WHERE <keyColumn> IN (@k0,@k1,…)`
per poll; index rows by key, slice back. Values in the `IN` list are **bound parameters**, one per
key — never interpolated.
**(b) Wide row** *(cheap).* One "latest status" row, many columns → many tags. A tag names a `table`,
a `columnName`, and a `rowSelector` (`{whereColumn, whereValue}` or `{topByTimestamp: <col>}`).
**GroupKey = `(table, rowSelector)`.** All wide-row tags on the same row fold into one
`SELECT <col_a>, <col_b>, … FROM <table> WHERE <whereColumn>=@w` (or `ORDER BY <col> DESC` +
`TOP 1`/`LIMIT 1` via dialect); map each selected column to its tag.
**(c) Named parameterized SELECT** *(escape hatch, P3).* Named query defs at driver level
(`queries: { "q1": { sql, params } }`); a tag references a query by name + a projection (`rowKey`
selecting the row, `column` selecting the value). **GroupKey = the named query.** Covers joins/views/
computed projections. The SQL is authored by a `ConfigEditor`-privileged operator and stored
server-side — it is *config*, not runtime user input — but it still binds only parameters.
**Group execution shape (recommended):** for each distinct GroupKey the driver holds a compiled
`SqlQueryPlan { GroupKey, SqlText, ParameterBinder, RowIndexer, IReadOnlyList<SqlTagDefinition> members }`
built lazily and cached; a poll pass executes each plan's command once. Cap concurrent group
execution (per-driver semaphore, Modbus/S7 precedent) so a driver spanning many tables never opens an
unbounded number of pooled connections.
### 3.7 Type mapping (SQL → OPC UA) + timestamps
Map the column's provider metadata (`DbColumn.DataType` from `DbDataReader.GetColumnSchema()`, or the
`INFORMATION_SCHEMA` `DATA_TYPE`) to `DriverDataType` (`Core.Abstractions/DriverDataType.cs`), which
the address-space layer maps to OPC UA built-ins.
| SQL type (family) | CLR (`DbDataReader`) | `DriverDataType` |
|---|---|---|
| `bit` / `boolean` | `bool` | `Boolean` |
| `tinyint` / `smallint` | `byte`/`short` | `Int16` |
| `int` / `integer` | `int` | `Int32` |
| `bigint` | `long` | `Int64` |
| `real` / `float(24)` | `float` | `Float32` |
| `float` / `double` / `decimal` / `numeric` / `money` | `double`/`decimal`→`double` | `Float64` |
| `char` / `varchar` / `nvarchar` / `text` | `string` | `String` |
| `date` / `datetime` / `datetime2` / `timestamptz` | `DateTime`/`DateTimeOffset` | `DateTime` |
| `uniqueidentifier` / `uuid` | `Guid` | `String` |
`decimal`/`numeric` collapse to `Float64` (v1) with a documented precision-loss caveat. A tag may
**override** the inferred type explicitly (`"type": "Int32"`) — the operator sometimes knows better
than column metadata (e.g. a `varchar` column holding a number); the driver then coerces the read
cell to the declared type. `NULL` → `DataValueSnapshot { Value = null }` with `Good`/`Uncertain` per
a `nullIsBad` option (default: treat as `Uncertain`, not `Bad`).
**Source timestamp:** if a tag declares `timestampColumn`, that column's value becomes
`DataValueSnapshot.SourceTimestampUtc`; otherwise the poll-read wall-clock is used for both source and
server timestamps (the Modbus behaviour). A source timestamp is what makes SQL-poll data honest about
staleness — strongly recommend authoring one wherever the staging table has it.
---
## 4. Bespoke schema browser (`Driver.Sql.Browser`)
### 4.0 Reconciliation with the universal Discover-backed browser — **honest statement**
The [universal browser design](2026-07-15-universal-discovery-browser-design.md) proposes one generic
`DiscoveryDriverBrowser` that runs a driver's `ITagDiscovery.DiscoverAsync` against a capturing
builder and re-presents the streamed tree. **That mechanism does not fit the `Sql` driver, and here is
exactly why:**
> The universal browser only ever shows what `DiscoverAsync` streams. For `Sql`, `DiscoverAsync` is
> **authored/config-driven** (§3.3) — it replays the *already-authored* tags, it does **not** enumerate
> the database schema. A universal Discover-backed browser would therefore replay the tags the operator
> already typed in, which is useless for *discovering* new ones. Schema enumeration
> (databases/schemas → tables/views → columns) is a **browse-time** concern that lives in
> `INFORMATION_SCHEMA`, not in the runtime read path.
The universal design's own capability gate encodes this distinction (`SupportsOnlineDiscovery`:
"does `DiscoverAsync` enumerate from the backend, or merely replay pre-declared tags?"). `Sql` is in
the **replay** bucket alongside Modbus/S7. So:
- **`Sql` leaves `ITagDiscovery.SupportsOnlineDiscovery = false`** (the default member) — the universal
browser's `CanBrowse` returns false for `Sql`, and it never offers the universal fallback.
- **`Sql` ships a bespoke `IBrowseSession`** (this section) that walks the *live schema catalog*,
which is the thing an operator actually wants to browse. A registered bespoke `IDriverBrowser` for
`DriverType="Sql"` takes precedence over the universal fallback by construction
(`BrowserSessionService` resolves bespoke-first).
This is the same reason OpcUaClient/Galaxy keep bespoke browsers: the browse tree is a *different
shape* from the discovered tag set. For `Sql` it's even starker — the discovered set is a strict
subset (only what's authored), while the schema catalog is the full browsable universe.
### 4.1 The schema walk
Relational DBs are self-describing. The picker walks the catalog three levels deep, each level a
dialect-abstracted catalog query:
- **`RootAsync`** → schemas (Folder nodes). SQL Server: `SELECT DISTINCT TABLE_SCHEMA FROM INFORMATION_SCHEMA.TABLES` (optionally databases first for multi-DB servers). SQLite (test dialect): the single main DB.
- **`ExpandAsync(schemaNodeId)`** → tables + views in that schema (Folder nodes): `… TABLE_NAME, TABLE_TYPE FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA=@schema`.
- **`ExpandAsync(tableNodeId)`** → columns (Leaf nodes): `… COLUMN_NAME, DATA_TYPE, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA=@schema AND TABLE_NAME=@table`.
- **`AttributesAsync(columnNodeId)`** → column data type + nullability side-panel (mirrors Galaxy's two-stage attribute pick): `AttributeInfo(Name: column, DriverDataType: dialect.MapColumnType(dataType).ToString(), IsArray: false, SecurityClass: "ViewOnly")`.
`@schema`/`@table` are **bound parameters** in the catalog queries — even here, no interpolation.
### 4.2 Portability caveat — the catalog query is dialect-specific
`INFORMATION_SCHEMA` is the SQL-92 standard catalog, supported by **SQL Server, PostgreSQL, and
MySQL/MariaDB** — so the queries above are portable across those three. But **two backends don't
implement it:**
- **Oracle** uses `ALL_TABLES` / `ALL_TAB_COLUMNS` (`USER_`/`DBA_` variants).
- **SQLite** uses `sqlite_schema`/`sqlite_master` + `PRAGMA table_info(<table>)`.
So the catalog SQL **belongs on `ISqlDialect`** (`ListSchemasSql`/`ListTablesSql`/`ListColumnsSql` +
`MapColumnType`), not hardcoded. P1 `SqlServerDialect` uses `INFORMATION_SCHEMA`; the SQLite test
dialect uses `PRAGMA`; Postgres/MySQL reuse `INFORMATION_SCHEMA`; Oracle (later) uses
`ALL_TAB_COLUMNS`. (Alternative for the *column* level only: `DbDataReader.GetColumnSchema()` on a
`SELECT * … WHERE 1=0` `SchemaOnly` command — but the table list still needs catalog SQL, so the
dialect method stays regardless.)
### 4.3 Picked-column → `TagConfig`
On column pick the browser emits `BrowseNode.NodeId` encoding `schema.table|column`; the picker body
composes the `TagConfig` blob (§5.2/5.3): DisplayName defaults to `table.column`, the inferred
`DriverDataType` from `AttributesAsync` prefills `type`, and the operator chooses the model (key-value
vs wide-row) + fills the key/row selector. `SecurityClass = ViewOnly` in read-only v1.
### 4.4 Project + registration (mirrors `OpcUaClient.Browser`)
| Path | Purpose |
|---|---|
| `Driver.Sql.Browser/SqlDriverBrowser.cs` | `IDriverBrowser`, `DriverType => "Sql"`; `OpenAsync(configJson, ct)` deserializes options, resolves the connection string from the form JSON (same trust boundary as `TestDriverConnect` + the OpcUaClient browser — transient, never cached), opens a `DbConnection`, returns a `SqlBrowseSession`. |
| `Driver.Sql.Browser/SqlBrowseSession.cs` | `IBrowseSession`; `RootAsync`/`ExpandAsync`/`AttributesAsync` over the dialect catalog queries; a `SemaphoreSlim _gate` (one ADO.NET connection is not concurrent). Per-call work is bounded by the AdminUI's existing 20 s per-call timeout. |
| AdminUI DI (`EndpointRouteBuilderExtensions.cs`, beside `:49-50`) | `services.AddSingleton<IDriverBrowser, SqlDriverBrowser>();` |
| `AdminUI/Uns/TagEditors/TagConfigEditorMap.cs` | `["Sql"] = typeof(Components.Shared.Uns.TagEditors.SqlTagConfigEditor)` (§6). |
| A new `SqlAddressPickerBody.razor` picker body | Browse tree (reuse `DriverBrowseTree.razor`) + attribute side-panel + model/selector fields; gated by the existing `DriverOperator` policy; manual entry retained. |
The browser reuses the AdminUI `BrowseSessionRegistry` + reaper + 2-min idle TTL + per-call timeout
unchanged. **The universal browser DI line is NOT added for `Sql`** — its `CanBrowse` is false anyway,
and the bespoke registration wins.
---
## 5. TagConfig + driver-config JSON
### 5.1 Driver config (the `DriverConfig` blob)
```jsonc
{
"provider": "SqlServer",
// Connection string is resolved at Initialize from an env/secret ref — NOT stored here.
"connectionStringRef": "MesStaging", // => env Sql__ConnectionStrings__MesStaging
"defaultPollInterval": "00:00:05",
"operationTimeout": "00:00:15", // per-query wall-clock deadline (→ BadTimeout)
"commandTimeout": "00:00:10", // server-side CommandTimeout backstop (seconds granularity)
"maxConcurrentGroups": 4, // cap on concurrent group queries (pool guard)
"nullIsBad": false, // NULL cell → Uncertain (default) vs Bad
"allowWrites": false, // master write kill-switch (v1: always false)
"probe": { "enabled": true, "interval": "00:00:10" },
// Optional pre-declared tag table (analogous to Modbus "Tags"); equipment tags may also
// contribute their address as raw TagConfig JSON at deploy time.
"tags": [ /* see 5.2 / 5.3 */ ],
// Optional named queries for the arbitrary-SELECT model (P3).
"queries": {
"activeBatch": {
"sql": "SELECT line, speed, temp, ts FROM dbo.LineStatus WHERE active=@active",
"params": { "@active": true }
}
}
}
```
### 5.2 Per-tag `TagConfig` — key-value model
```jsonc
{
"driver": "Sql",
"model": "KeyValue",
"table": "dbo.TagValues",
"keyColumn": "tag_name",
"keyValue": "Line1.Speed",
"valueColumn": "num_value",
"timestampColumn": "sample_ts",
"type": "Float64", // optional explicit override of inferred type
"writable": false
}
```
GroupKey for batching = `("dbo.TagValues", "tag_name", "num_value", "sample_ts")`; all such tags fold
into one `… WHERE tag_name IN (@k0,@k1,…)` per poll.
### 5.3 Per-tag `TagConfig` — wide-row model
```jsonc
{
"driver": "Sql",
"model": "WideRow",
"table": "dbo.LatestStatus",
"columnName": "oven_temp",
"rowSelector": { "whereColumn": "station_id", "whereValue": 7 },
// or "rowSelector": { "topByTimestamp": "sample_ts" } // newest row
"type": "Float64",
"writable": false
}
```
GroupKey = `("dbo.LatestStatus", rowSelector)`; every wide-row tag on the same row folds into one
`SELECT oven_temp, <other cols>, … FROM dbo.LatestStatus WHERE station_id=@id`.
### 5.4 Per-tag `TagConfig` — named-query model (P3)
```jsonc
{
"driver": "Sql",
"model": "Query",
"query": "activeBatch",
"rowKey": { "column": "line", "value": "Line1" }, // which row this tag reads
"column": "speed", // which projected column is the value
"timestampColumn": "ts"
}
```
### 5.5 Equipment-tag parser
`SqlEquipmentTagParser.TryParse(reference, out SqlTagDefinition def)` (in `.Contracts`) mirrors
`ModbusEquipmentTagParser`: recognizes the blob by a leading `{` + a `"driver":"Sql"` (or a `"model"`
discriminator), reads the model-specific fields, uses **`TagConfigJson.TryReadEnumStrict`** for the
`model`/`type` enums (a present-but-invalid enum rejects the tag → `BadNodeIdUnknown`, per R2-11), and
maps to a transient `SqlTagDefinition` whose `Name == reference` so the value the driver publishes
keys the forward router correctly. `EquipmentTagRefResolver<SqlTagDefinition>` bridges the two
authoring models (an authored `tags`-table entry by name **or** an equipment tag whose reference is
its raw `TagConfig` JSON, parsed once and cached) — same as Modbus.
---
## 6. Typed editor + validator + the enum-serialization trap
Add `["Sql"] = typeof(…SqlTagConfigEditor)` to `TagConfigEditorMap` and a matching entry to
`TagConfigValidator`. The editor is a thin Razor shell over a pure `SqlTagConfigModel`
(`FromJson`/`ToJson`/`Validate`, preserves unknown keys) — copy the Modbus template under
`Components/Shared/Uns/TagEditors/` + `Uns/TagEditors/`. The model exposes: `Model` (KeyValue/WideRow/
Query), `Table`, `KeyColumn`/`KeyValue`/`ValueColumn`/`TimestampColumn` (key-value), `ColumnName` +
`RowSelector` (wide-row), `Query`/`RowKey`/`Column` (named), and `Type` (`DriverDataType?` override).
Without a typed editor `Sql` would fall to the generic raw-JSON textarea (like Galaxy today) — the
typed editor is worth building because the model discriminator + selector shape is error-prone by hand.
**The `JsonStringEnumConverter` enum-serialization trap (MEMORY: systemic).** Every AdminUI driver
page that serializes an enum **numerically** while the factory DTO is **string-typed** produces a
config that faults the driver at parse time (proven e2e for S7 + Modbus). **All** enum fields here —
`provider` (`SqlProvider`), `model`, `type` (`DriverDataType`) — MUST round-trip as **strings** on
**both** sides:
- The AdminUI editor/model `ToJson` and the driver-config page serializer attach
`JsonStringEnumConverter` (mirror the OpcUaClient page).
- The factory (`SqlDriverFactoryExtensions`) and `SqlEquipmentTagParser` deserialize with the same
string converter (and `TryReadEnumStrict` for tag enums).
- The `TestDriverConnect` probe uses the same options.
Add a unit test asserting `provider`/`model`/`type` serialize as `"SqlServer"`/`"KeyValue"`/`"Float64"`
(quoted strings), not `0`/`1`/`8`.
---
## 7. Factory + registration
`SqlDriverFactoryExtensions` mirrors `ModbusDriverFactoryExtensions`:
```csharp
public static class SqlDriverFactoryExtensions
{
public const string DriverTypeName = "Sql";
public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)
{
ArgumentNullException.ThrowIfNull(registry);
registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory));
}
public static SqlDriver CreateInstance(string driverInstanceId, string driverConfigJson, ILoggerFactory? loggerFactory)
{ /* deserialize DTO (string enum converter), validate connectionStringRef present, build options+dialect */ }
}
```
Wire-up (one line each):
- **Factory:** add `Driver.Sql.SqlDriverFactoryExtensions.Register(registry, loggerFactory);` to
`src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs` `Register(...)` (beside the
eight existing driver `Register` lines).
- **Tier:** declared on the factory registration itself — the optional `tier` parameter of
`DriverFactoryRegistry.Register(driverType, factory, tier)`
(`Core/Hosting/DriverFactoryRegistry.cs`), which the resilience pipeline reads via
`DriverFactoryRegistry.GetTier`. The default `DriverTier.A` (managed SDK, in-process) is correct
for `Sql` — no explicit argument needed, same as every existing driver. (Do **not** hunt for a
`DriverTypeMetadata`/`DriverTypeRegistry` registration site: that registry in `Core.Abstractions`
currently has no production population site — no existing driver registers metadata there.)
- **Probe:** if a lightweight `SqlProbe : IDriverProbe` is added for the AdminUI "Test Connect"
button, register it in `AddOtOpcUaDriverProbes` via `TryAddEnumerable` (a bare `SELECT 1` liveness
check — recommended, cheap, and admin-node registration matters per the split-role gotcha).
- **Browser:** the `AddSingleton<IDriverBrowser, SqlDriverBrowser>()` line (§4.4).
`ShouldStub()` in `DriverInstanceActor` is platform/role-driven and needs no `Sql`-specific change —
SQL Server client works cross-platform, so `Sql` runs on macOS dev too (unlike Galaxy).
---
## 8. Resilience / security / timeout
### 8.1 SQL injection (critical, risk #1)
- **Every value is a bound `DbParameter`.** The `IN (@k0,@k1,…)` key list, `WHERE col=@w` selectors,
named-query `params`, and (P4) write values are all parameters. **Zero** runtime tag input is ever
concatenated into SQL text.
- **Identifiers** (schema/table/column) that must appear as SQL text come **only** from
`INFORMATION_SCHEMA`-validated names or a browse-derived allow-list, and are emitted through
`dialect.QuoteIdentifier` (which escapes/rejects embedded quote characters). A table/column string
in a `TagConfig` is validated against the live catalog (or an allow-list) before it's ever quoted
into text; an unknown identifier rejects the tag (→ `BadNodeIdUnknown`) rather than executing.
- **Treat any code path that builds SQL by string-concatenating a tag field as a defect** — enforce
with a review checklist item + a unit test that feeds a malicious `keyValue`/`table`
(`'; DROP TABLE …`) and asserts it either binds harmlessly (value) or is rejected (identifier),
never executes.
- The named-query model stores operator-authored SQL as *config* (`ConfigEditor`-gated), and even
there only parameters bind runtime values.
### 8.2 Secrets (critical, risk #2)
Mirror `ServerHistorian:ApiKey` (env `ServerHistorian__ApiKey`, appsettings carries only a reminder
comment) and the env-overridable ConfigDb connection string:
- `DriverConfig` JSON stores a **`connectionStringRef`** (a logical name), **not** the connection
string. The driver resolves it at `InitializeAsync` by reading the environment variable directly
(`Sql__ConnectionStrings__<ref>` — the flat env-var spelling of the config key), so the secret
lives in the process environment / secret store — never in the config DB row or any committed
file. Direct env read (not `IConfiguration`) is deliberate: the factory registry materializes
drivers via a static `(id, json)` closure (Modbus pattern) with no `IConfiguration` in reach, so
this keeps the factory shape unchanged.
- If an inline `connectionString` is ever permitted (dev convenience only), the AdminUI **redacts** it
in display/logging and flags it dev-only.
- **Never log the resolved connection string.** Log only provider + server host + database name.
- Prefer integrated/managed auth where the estate supports it (`Integrated Security=true` / Azure AD /
Kerberos) so no password transits config at all.
- The `.Browser` path receives the secret in form JSON over the authenticated Blazor circuit (same
boundary as every other driver browser + `TestDriverConnect`); it builds one transient connection
and releases it — **no `_lastConfigJson` cached field anywhere.**
### 8.3 Timeout / frozen-peer (high, risk #3)
Per §3.2: `CommandTimeout` (server-side backstop) **and** `ExecuteReaderAsync(linkedCt)` bounded by
`operationTimeout` (client-side, real cancellation) on **every** command — the SQL analogue of the
R2-01 S7 blackhole finding. A frozen DB surfaces `BadTimeout` per group + degrades the driver + backs
off the poll loop; it must never wedge a poll thread. Asserted by the blackhole live-gate (§9).
### 8.4 Connection pooling / exhaustion (medium)
Open-use-dispose per poll (`await using` on both `DbConnection` and `DbDataReader` — never leak a
reader). Cap concurrent group execution via `maxConcurrentGroups` (a `SemaphoreSlim`, Modbus/S7
precedent) so a driver spanning many tables never opens an unbounded number of pooled connections.
Pool sizing is expressed only through the connection string (`Max Pool Size`).
### 8.5 Provider portability (medium)
`INFORMATION_SCHEMA` covers SQL Server/Postgres/MySQL but not Oracle/SQLite; parameter markers and
identifier quoting differ. **All of it lives behind `ISqlDialect`** — no dialect assumption leaks into
the driver core (the reader, grouping, and poll engine are provider-agnostic).
### 8.6 Type / precision (low-medium)
`decimal`→`Float64` precision loss; `NULL` semantics (`nullIsBad`); `datetime` vs `datetimeoffset`
timezone handling (normalize to UTC in `SourceTimestampUtc`). Documented + per-tag `type` override.
---
## 9. Test fixtures
- **SQLite unit fixture (primary).** `Microsoft.Data.Sqlite` (already `10.0.7` in-graph, no
container). Seed a `(tag_name, num_value, sample_ts)` key-value table and a wide-row `LatestStatus`
table in-process. Covers read/subscribe mapping, type mapping, group-batch slicing, the poll-engine
wiring, `nullIsBad`, and the browser over `PRAGMA table_info` (SQLite dialect). Fast, offline,
CI-safe on macOS. Requires a `SqliteDialect` (test-scoped is fine, or ship it as the P2 SQLite
browser dialect). Note: SQLite's dynamic typing means the dialect's `MapColumnType` maps declared
affinity, not runtime type — keep the seed columns explicitly typed.
- **Central SQL Server integration fixture.** `10.100.0.35,14330` (always-on per `CLAUDE.md`). A
seeded `SqlPollFixture` database with the same two sample tables validates the real
`Microsoft.Data.SqlClient` path, `INFORMATION_SCHEMA` browse, `CommandTimeout`/cancellation, and
pooling. **Env-gated** (`SQL_TEST_ENDPOINT` / a full `Sql__ConnectionStrings__Fixture`) so it skips
cleanly offline, like the other `*.IntegrationTests`. Deploy the seed via the docker rig; label the
stack `project=lmxopcua`.
- **Blackhole / timeout live-gate (most important integration test).** Mirror R2-01 S7: `docker pause`
a SQL Server container mid-poll and assert the read surfaces `BadTimeout`, the driver degrades, and
the poll loop backs off — **not** a wedged thread. This gate runs against a **dedicated disposable
`mssql` container** (its own compose stack under `tests/.../Docker/`, `project=lmxopcua` label) —
**never** pause the shared central SQL Server on `,14330`, which also hosts `ConfigDb` for the whole
rig. This is the single highest-value integration test given the frozen-peer risk class.
- **Injection regression test.** Feed a malicious `keyValue`/`table` and assert bind-harmless (value)
or reject (identifier), never execute (§8.1).
- **Postgres/ODBC fixtures** land with P2/P3 (a `postgres:16` container for Npgsql; an ODBC DSN
against the same SQL Server).
---
## 10. Phasing + effort
1. **P1 — SQL Server read-only, key-value + wide-row.** `.Contracts` (options DTO, `SqlProvider` enum,
tag models, `SqlEquipmentTagParser`); `SqlDriver` (`IDriver`/`ITagDiscovery`/`IReadable`/
`ISubscribable` via `PollGroupEngine` + `IHostConnectivityProbe`); `SqlServerDialect`; grouping +
result-set slicing; factory + type-metadata + probe registration; `SqlTagConfigEditor` + validator
(+ the enum-serialization test); SQLite unit fixture; env-gated SQL-Server integration fixture +
blackhole live-gate + injection test. **No new NuGet deps.**
2. **P2 — Bespoke schema browser.** `Driver.Sql.Browser` (`SqlDriverBrowser`/`SqlBrowseSession` over
`INFORMATION_SCHEMA` + a SQLite `PRAGMA` dialect for tests), `SqlAddressPickerBody.razor` + tree +
attribute side-panel, DI wire-up, column→`TagConfig` commit. Near-mechanical clone of
`OpcUaClient.Browser`. **Small.**
3. **P3 — Named-query model + PostgreSQL + ODBC.** `PostgresDialect`/`OdbcDialect`; add `Npgsql` +
`System.Data.Odbc` `PackageVersion`s; named-query grouping; Postgres container fixture.
4. **P4 (optional) — Write mode.** `IWritable` with per-tag parameterized UPSERT/StoredProc,
triple-gated (`WriteOperate` + `allowWrites` + `TagConfig.writable`), `WriteIdempotent` retry
policy (write-failure revert already provided by the runtime).
5. **P5 (demand-driven) — MySQL/MariaDB (`MySqlConnector`) + native Oracle (`ALL_TAB_COLUMNS`
dialect).**
**Effort:** P1 **Medium** (Modbus minus wire-codec, plus dialect seam + slicing; poll engine,
resolver, health machine, factory shape all reused). P2 **Small**. The provider/dialect long tail
(P3+) is where portability effort lives — keeping it behind `ISqlDialect` from day one is what makes
the P1 SQL-Server slice cheap and the rest additive.
---
## Sources / grounding
- Research report: [`docs/research/drivers/sql-poll.md`](../research/drivers/sql-poll.md).
- In-repo seams: `Core.Abstractions/{IDriver,ITagDiscovery,IReadable,ISubscribable,IHostConnectivityProbe,PollGroupEngine,EquipmentTagRefResolver,DataValueSnapshot,DriverAttributeInfo,DriverDataType,DriverHealth,TagConfigJson}.cs`.
- Templates: `Driver.Modbus/{ModbusDriver,ModbusDriverFactoryExtensions}.cs`, `Driver.Modbus.Contracts/ModbusEquipmentTagParser.cs`, `Driver.OpcUaClient.Browser/*`.
- Wire-up sites: `Host/Drivers/DriverFactoryBootstrap.cs` (`Register` + `AddOtOpcUaDriverProbes`), `AdminUI/EndpointRouteBuilderExtensions.cs:49-50` (browser DI), `AdminUI/Uns/TagEditors/{TagConfigEditorMap,TagConfigValidator}.cs`.
- Packages: `Directory.Packages.props:53-54` (`Microsoft.Data.SqlClient 6.1.1`, `Microsoft.Data.Sqlite 10.0.7`).
- Browse house style: [`docs/plans/2026-05-28-driver-browsers-design.md`](2026-05-28-driver-browsers-design.md); universal browser reconciliation: [`docs/plans/2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md).
```
docs/plans/2026-07-15-universal-discovery-browser-design.md
+296
View File
@@ -0,0 +1,296 @@
# Universal Discover-backed browser (`DiscoveryDriverBrowser`) — design
**Status:** draft, 2026-07-15. Foundational Wave-0 item of the next-driver roadmap
(`docs/research/drivers/README.md`).
## 1. Motivation
Today the AdminUI address picker is backed by one **bespoke** `IDriverBrowser` per driver
type — only **OpcUaClient** and **Galaxy** exist, each a full hand-written
`IBrowseSession` (connect + lazy level-at-a-time browse). Every other driver's picker is a
static manual-entry stub.
But the existing-driver browse audit
(`docs/research/drivers/00-existing-driver-browse-audit.md`) found that AbCip, TwinCAT, and
FOCAS **already contain complete, unit-tested symbol enumeration** — it lives in their
runtime `ITagDiscovery.DiscoverAsync`, streamed into an `IAddressSpaceBuilder`, and is
simply never surfaced to the picker. The same is true of every future discovery-capable
driver (MTConnect `/probe`, BACnet Who-Is/object-list).
The insight: **`DiscoverAsync` already produces exactly the tree the picker wants.** Instead
of writing N bespoke browsers, write **one** generic browser that captures whatever a driver
streams into the builder and re-presents it. This is the highest-leverage single piece of
browse work on the roadmap.
## 2. The seam that makes it work
`ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder builder, ct)` does not return a tree — it
**streams structured nodes into a builder** (`Core.Abstractions/IAddressSpaceBuilder.cs`):
```csharp
IAddressSpaceBuilder Folder(string browseName, string displayName); // → child builder
IVariableHandle Variable(string browseName, string displayName, DriverAttributeInfo a);
void AddProperty(string browseName, DriverDataType t, object? v);
```
Two facts close the loop:
- `Folder(...)` returns a **child builder scoped inside the folder** → the driver's hierarchy
is expressed as the builder-nesting graph. Capture the nesting and you inherit the driver's
tree (folders, UDT structure, component nesting) — this is **not** a flat dump.
- `DriverAttributeInfo.FullName` **is the driver-side full reference** for read/write
addressing — i.e. exactly the value the picker commits as `TagConfig.FullName`
(`DriverAttributeInfo.cs:13`). No per-driver commit logic needed.
So a **capturing `IAddressSpaceBuilder`** run against any driver's `DiscoverAsync` yields a
`BrowseNode` tree whose leaves already carry the commit value and the datatype metadata.
## 3. Architecture — two-tier browse
```
BrowserSessionService.OpenAsync(driverType, configJson)
│
├─ bespoke IDriverBrowser registered for driverType? ── yes ──► use it (Tier 2)
│ (OpcUaClient, Galaxy, later: large ControlLogix/BACnet)
│
└─ no bespoke browser ───► DiscoveryDriverBrowser (Tier 1, universal fallback)
gated on online-discovery capability
```
- **Tier 1 — universal `DiscoveryDriverBrowser`** (this doc): the default. Covers any driver
whose `DiscoverAsync` enumerates from the *device*.
- **Tier 2 — bespoke `IBrowseSession`** (existing pattern, `2026-05-28-driver-browsers-design.md`):
reserved for large / structured / lazy address spaces where eager one-shot discovery is too
heavy. A bespoke browser **overrides** the universal one simply by being registered for that
`DriverType`.
### 3.1 Resolution change in `BrowserSessionService` (required)
`BrowserSessionService` currently indexes browsers with
`browsers.ToDictionary(b => b.DriverType)` — **one browser per type, and a duplicate
`DriverType` key throws at construction.** Therefore the universal browser **cannot** be one
of the injected `IEnumerable<IDriverBrowser>` (it has no single `DriverType`, and registering
it under every type would collide). Instead:
- Introduce `IUniversalDriverBrowser` with
`Task<IBrowseSession> OpenAsync(string driverType, string configJson, CancellationToken ct)`
and `bool CanBrowse(string driverType, string configJson)`.
- Inject it into `BrowserSessionService` as a **separate optional dependency** (not part of the
`IDriverBrowser` set). `OpenAsync` resolves bespoke-first, falls back to the universal browser
when no bespoke browser matches **and** `CanBrowse` is true; otherwise returns the existing
"no browser registered" result (→ picker shows manual entry).
- `RootAsync`/`ExpandAsync`/`AttributesAsync`/`CloseAsync` are unchanged — the universal
browser returns an ordinary `IBrowseSession` that lands in the same `BrowseSessionRegistry`
(TTL reaper, 20 s per-call timeout) as today.
## 4. Components
### 4.1 `CapturingAddressSpaceBuilder` (the core mechanism)
An in-memory `IAddressSpaceBuilder` that records the streamed tree instead of materializing
OPC UA nodes. New home: `src/Core/ZB.MOM.WW.OtOpcUa.Commons/Browsing/` (Commons already owns
`BrowseNode`/`IBrowseSession` and is referenced by AdminUI; `IAddressSpaceBuilder`/
`DriverAttributeInfo` live in `Core.Abstractions`, which Commons can reference).
| Builder call | Captured as | Serves |
|---|---|---|
| `Folder(bn, dn)` | node `{ id = parentPath + "/" + bn, display = dn, Folder, children[] }`; returns a child capturing builder scoped to it | `BrowseNode(id, dn, Folder, HasChildrenHint=true)` |
| `Variable(bn, dn, attr)` | leaf `{ id = attr.FullName, display = dn, Leaf, attr }`; returns a stub `IVariableHandle{ FullReference = attr.FullName }` | `BrowseNode(attr.FullName, dn, Leaf, false)` + `AttributeInfo` |
| `AddProperty(...)` | attached to the current node's property bag (optional; feeds the side-panel) | `AttributesAsync` |
| `IVariableHandle.MarkAsAlarmCondition(info)` | records `IsAlarm=true` on the leaf; returns a **no-op `IAlarmConditionSink`** (browse never delivers live transitions) | `AttributeInfo.IsAlarm=true` (picker pre-fills a default `alarm` block, per existing Galaxy behaviour) |
Leaf → `AttributeInfo` mapping (drives the side-panel + commit):
`AttributeInfo(Name: bn, DriverDataType: attr.DriverDataType.ToString(), IsArray: attr.IsArray,
SecurityClass: attr.SecurityClass.ToString(), IsAlarm: attr.IsAlarm)`. Commit `NodeId` =
`attr.FullName` → `TagConfig.FullName`; `attr.ArrayDim`/`IsHistorized`/`WriteIdempotent` are
available to pre-fill the typed tag editor.
The builder enforces a **node-count cap** (config, default e.g. 50 000): once exceeded it stops
recording and marks the tree `truncated=true` so the session can surface "results truncated —
narrow the driver config or use manual entry" rather than OOM on a 100 k-node backend.
### 4.2 `DiscoveryDriverBrowser : IUniversalDriverBrowser`
`OpenAsync(driverType, configJson, ct)`:
1. `driver = _driverFactory.TryCreate(driverType, ephemeralId, PatchForBrowse(driverType, configJson))`
— `null` ⇒ throw a clean "driver type not available in this host" (see §6 wiring).
2. `if (driver is not ITagDiscovery disc || !disc.SupportsOnlineDiscovery)` ⇒ throw
"driver has no online discovery — author tags manually" (should not happen — `CanBrowse`
already gated it, but defend).
3. `await driver.InitializeAsync(patchedConfigJson, openCts.Token)` — **real device connect**,
under an **open-timeout** (config, default 60 s; separate from the 20 s per-call browse
timeout, which only covers Root/Expand/Attributes serving from memory).
4. `var capture = new CapturingAddressSpaceBuilder(nodeCap); await disc.DiscoverAsync(capture, openCts.Token);`
— eager, whole-tree, one-shot. **Except when `disc.RediscoverPolicy == UntilStable`**
(FOCAS): its `FixedTreeCache` is filled by a background loop a couple of seconds *after*
connect, so an immediate single pass captures an empty device folder (the comment above
`FocasDriver.DiscoverAsync` states this explicitly). For `UntilStable` drivers, re-run the
capture on a short interval until the node set is non-empty and stable across two passes,
bounded by the open-timeout — the same contract `DriverInstanceActor` honours at deploy.
5. `await driver.ShutdownAsync(...)` in a `finally` — the tree is fully captured; hold nothing
live. (No lazy expand ⇒ no need to keep the connection.)
6. return `new CapturedTreeBrowseSession(capture.Root)`.
`CanBrowse(driverType, configJson)` = `TryCreate` (cheap, no connect) succeeds **and** the
instance is `ITagDiscovery { SupportsOnlineDiscovery: true }`. Used by the AdminUI to decide
whether to render the **Browse** button vs. manual entry, before any connect.
### 4.3 `CapturedTreeBrowseSession : IBrowseSession`
Serves entirely from the in-memory captured tree (the shape the MTConnect research report
proposed for its cached-`/probe` session — now subsumed by this generic component): `RootAsync` = top-level nodes; `ExpandAsync(nodeId)` = the captured children of that
node; `AttributesAsync(nodeId)` = the leaf's `AttributeInfo` (+ any captured properties).
`Token`/`LastUsedUtc` per the interface; `DisposeAsync` drops the tree. No I/O after open.
## 5. The online-discovery capability gate
The distinction the gate encodes: does `DiscoverAsync` enumerate from the **device** (browsable)
or merely replay **pre-declared** tags (flat-address drivers — Modbus/S7/MELSEC/AbLegacy/
Omron-FINS, where it would show only what you already authored)?
**Add a default-interface property** on `ITagDiscovery` (mirrors the existing
`RediscoverPolicy => UntilStable` default member on the same interface — zero churn for drivers
that don't opt in):
```csharp
/// <summary>True when DiscoverAsync enumerates the tag set from the live backend
/// (browsable), rather than replaying pre-declared/authored tags. Default false.</summary>
bool SupportsOnlineDiscovery => false;
```
- **Return true (device-enumerated):** OpcUaClient, BACnet, MTConnect, and — **config-gated**
— AbCip/TwinCAT/FOCAS (their device enumeration sits behind a config flag; the browse patch
guarantees it at open time, see §5.1).
- **Leave false (authored replay):** Modbus, Modbus-RTU, S7, MELSEC, AbLegacy,
Omron (CIP + FINS) — their pickers stay manual-entry (Omron gets an offline
tag-export importer later); and **MQTT/Sparkplug + SQL poll**, whose runtime
`DiscoverAsync` is also deliberately authored-only — they get **bespoke** browsers instead
(observation-window / `INFORMATION_SCHEMA` walk — see the program doc §4), because a
universal replay of authored tags would discover nothing new.
(Galaxy/OpcUaClient keep their bespoke browsers regardless — the flag only governs the
*universal fallback*.)
### 5.1 Per-driver browse-config patch (`PatchForBrowse`) — the one wrinkle
For AbCip/TwinCAT/FOCAS, device enumeration is **config-gated**: AbCip/TwinCAT's
`DiscoverAsync` only walks the controller when `EnableControllerBrowse = true`, and FOCAS only
emits its FixedTree when `FixedTree.Enabled = true` (default `false`,
`FocasFixedTreeOptions.Enabled`) — but the picker holds the *authoring* config (often
`false`). The universal browser therefore merges a tiny, **declarative** per-type JSON
patch before constructing the driver:
```csharp
static readonly IReadOnlyDictionary<string, string> BrowsePatches = new Dictionary<string,string>(OrdinalIgnoreCase) {
["AbCip"] = """{ "EnableControllerBrowse": true }""",
["TwinCAT"] = """{ "EnableControllerBrowse": true }""",
["FOCAS"] = """{ "FixedTree": { "Enabled": true } }""",
// OpcUaClient/BACnet/MTConnect: no patch — discovery is unconditional
};
```
This is a dictionary entry, not code — it keeps the "zero per-driver *browser*" property while
being honest that browse-mode activation is one declarative fact per config-gated driver.
`SupportsOnlineDiscovery` for AbCip/TwinCAT/FOCAS returns true because the patch guarantees
browse mode at open time. (FOCAS additionally needs the §4.2 `UntilStable` settle — the patch
turns the FixedTree on; the settle waits for its cache to fill.)
## 6. Wiring & deployment
- Register in AdminUI DI (`EndpointRouteBuilderExtensions.AddAdminUI`):
`services.AddSingleton<IUniversalDriverBrowser, DiscoveryDriverBrowser>();` — **one line**,
no per-driver registration (contrast the two bespoke lines at
`EndpointRouteBuilderExtensions.cs:49-50`).
- Depends on `IDriverFactory` (Core.Abstractions). In the **fused Host** this is the real
`DriverFactoryRegistryAdapter` (all `Driver.*.Register()` ran) → every discovery-capable type
is constructible. On a **standalone AdminUI** with no driver assemblies it's
`NullDriverFactory` → `TryCreate` returns null → `CanBrowse` false → universal browse simply
unavailable (graceful; picker falls back to manual entry). No new failure mode.
- **Network/credentials posture is unchanged from the existing browsers**: OpcUaClient/Galaxy
browsers already connect to live backends *from the AdminUI process* using credentials in
`configJson`. The universal browser just extends that to more driver types — same role gating
(picker is behind the existing `ConfigEditor`/browse authorization), same
credentials-in-JSON handling, same `BrowseSessionRegistry` TTL reaping.
## 7. Where universal applies vs. bespoke
| Driver | Universal covers it? | Bespoke later? |
|---|---|---|
| **AbCip** | Yes (patch → `EnableControllerBrowse`) | Only if controller symbol set too large for eager one-shot (UDT drill-down) |
| **TwinCAT** | Yes (patch) | Only for very large symbol sets / Flat-mode caveats |
| **FOCAS** | Yes (patch → `FixedTree.Enabled` + §4.2 UntilStable settle) | Unlikely — surface is small/curated |
| **MTConnect** | Yes (`/probe`) | Optional — `/probe` is already whole-model; eager is fine |
| **BACnet** | Yes for small sites | **Yes** for large multi-device sites (lazy per-device object-list) |
| **MQTT/Sparkplug** | **No** (`SupportsOnlineDiscovery=false` — runtime discovery is authored-only by design) | **Yes** — bespoke observation-window `MqttBrowseSession` (its design doc §4) |
| **SQL poll** | **No** (authored replay) | **Yes** — bespoke `INFORMATION_SCHEMA` schema browser (its design doc) |
| **OpcUaClient** | (bespoke already) | Keep bespoke (100k-node lazy browse) |
| **Galaxy** | (bespoke already) | Keep bespoke (two-stage attribute pick) |
| Modbus/RTU, S7, MELSEC, AbLegacy, Omron | **No** (`SupportsOnlineDiscovery=false`) | N/A — not browsable (Omron: offline importer P-later) |
## 8. Limits (why bespoke browsers still exist)
1. **Eager, not lazy.** `DiscoverAsync` runs the whole enumeration at open. For huge trees
(ControlLogix 10 k+ tags, OPC UA 100 k nodes) the node-cap truncates and the UX is worse than
a bespoke per-click `ExpandAsync`. Graduate those to Tier 2.
2. **Runs the real driver in-process** (construct + connect + one-shot discover + shutdown) —
heavier than the lightweight ad-hoc bespoke session, and requires the driver assemblies in the
host (fused Host: yes).
3. **No incrementally-fetched attribute panel** beyond what `DriverAttributeInfo` +
`AddProperty` carry (fine for the picker's needs; Galaxy's richer two-stage pick stays bespoke).
## 9. Testing
- **Unit — `CapturingAddressSpaceBuilder`:** drive it with a fake `ITagDiscovery` that streams a
known Folder/Variable/AddProperty/alarm graph; assert the `BrowseNode` tree, leaf `NodeId ==
FullName`, `AttributeInfo` mapping, node-cap truncation, and the no-op alarm sink.
- **Unit — `DiscoveryDriverBrowser`:** fake `IDriverFactory` returning a fake driver; assert
Initialize→Discover→Shutdown ordering, open-timeout, `PatchForBrowse` merge, `CanBrowse` gate
(true for `SupportsOnlineDiscovery`, false otherwise / `TryCreate`→null), and the
`UntilStable` settle (a fake `UntilStable` driver whose tree only fills on the second
discovery pass must yield the full tree; a never-stable one must stop at the open-timeout).
- **Unit — `BrowserSessionService` resolution:** bespoke-first, universal-fallback, and
manual-entry-when-neither; confirm no `ToDictionary` collision.
- **Integration (fixture-gated):** point the universal browser at the existing driver fixtures on
`10.100.0.35` — AbCip (ControlLogix sim) and the OPC UA reference server — and assert a real
captured tree; reuse the driver fixtures already in `tests/`.
- **Live-verify:** on docker-dev, open the `/uns` TagModal picker for an AbCip driver and confirm
the controller tag tree renders and a picked leaf commits `TagConfig.FullName` (the usual
live-`/run` discipline — Razor binding bugs pass unit tests).
## 10. Phasing & effort
- **P1 — universal browser end-to-end:** `CapturingAddressSpaceBuilder` +
`CapturedTreeBrowseSession` + `DiscoveryDriverBrowser` + `IUniversalDriverBrowser` +
`BrowserSessionService` fallback + `ITagDiscovery.SupportsOnlineDiscovery` default member +
the AbCip/TwinCAT/FOCAS patches + the `UntilStable` settle loop (§4.2) + DI line + AdminUI
`CanBrowse`→Browse-button gate. Set `SupportsOnlineDiscovery=true` on
OpcUaClient(n/a—bespoke), AbCip, TwinCAT, FOCAS. **~3–5 days.**
- **P2 — light up new drivers for free:** as MTConnect/BACnet land, each sets
`SupportsOnlineDiscovery=true` (+ patch if config-gated) and gets a working picker with no
browser code. (MQTT/Sparkplug and SQL poll ship bespoke browsers instead — program doc §4.)
- **P3 — graduate to bespoke** only where §8.1 bites (large BACnet sites, ControlLogix UDT
drill-down): add a bespoke `IBrowseSession` that overrides the universal fallback for that type.
Net: one ~3–5 day piece replaces the previously-planned per-driver browser builds for AbCip +
TwinCAT (and pre-covers FOCAS + every future discovery driver), with bespoke lazy browsers
reserved for the genuinely-large trees.
## 11. v3 forward note (Raw/UNS two-subtree — lands after this)
This browser ships **before** v3 (`docs/plans/2026-07-15-raw-uns-two-subtree-v3-design.md`)
against the current `/uns` TagModal picker. When v3's Raw tree lands, three things about the
*consumer* change while the session/tree layer here (`CapturingAddressSpaceBuilder`,
`CapturedTreeBrowseSession`, `IUniversalDriverBrowser`, `BrowserSessionService` resolution)
carries over unchanged:
1. **Commit contract:** §2's "commits `TagConfig.FullName`" describes the pre-v3 picker. In
v3, `TagConfig` no longer carries identity (the tag's identity is its RawPath); the picked
leaf's `attr.FullName` is written into the driver-typed `TagConfig` as an ordinary
**address field**, and the commit creates **raw `Tag` rows** (name from browse name,
datatype from `DriverAttributeInfo`) under a Device/TagGroup — optionally mirroring the
captured folder nesting as `TagGroup`s.
2. **Config input:** v3 moves endpoint/connection settings from `DriverConfig` into
`DeviceConfig`. `OpenAsync(driverType, configJson, ct)`'s `configJson` becomes the
**merged Driver + Device config** composed by the caller (the `/raw` browse modal);
`PatchForBrowse` merges on top as today.
3. **Browse-button gate:** unchanged mechanism (`CanBrowse` / bespoke registration), hosted
in the `/raw` tree's "Add tags ▸ Browse device…" action instead of the equipment TagModal.