lmxopcua/docs/plans/2026-07-15-driver-expansion-program-design.md

Driver-expansion program — overall design

Status: program design, 2026-07-15. Umbrella for the next-driver effort: one universal browse mechanism + six new/extended drivers (MELSEC deferred). This is the authoritative index and the single statement of the effort's shared architecture, dependency graph, and build order. Per-driver detail lives in the linked design docs; cross-cutting rules are stated once here so the per-driver docs don't re-derive them.

1. Goal & scope

Extend the OtOpcUa OPC UA server's driver fleet under the existing Equipment-kind address-space model — no new COM gateway, all in-process .NET 10 — and make more of the fleet browsable from the AdminUI /uns address picker.

Curated set (user, 2026-07-15): MTConnect Agent · MQTT/Sparkplug B · MELSEC SLMP · Omron · BACnet/IP · SQL poll · Modbus RTU. MELSEC is deferred (commercial-license-forced hand-rolled framer + the hardest addressing model on the roadmap; revisit on a Mitsubishi-shop need). The other six are designed and scheduled below.

Alongside the new drivers, a universal Discover-backed browser retrofits browse to the already-shipped fleet (AbCip, TwinCAT, FOCAS) and is the foundation every browsable new driver builds on.

Existing fleet for context: Modbus(TCP), S7, AbCip, AbLegacy, TwinCAT, FOCAS, Galaxy, OpcUaClient, Historian.Gateway (src/Drivers/).

2. Document map

Program: this file.

Research reports (docs/research/drivers/, index: README.md):

• mtconnect-agent.md · mqtt-sparkplug.md · melsec-slmp.md (deferred) · omron.md · bacnet-ip.md · sql-poll.md · modbus-rtu.md · 00-existing-driver-browse-audit.md

Design docs (docs/plans/2026-07-15-*):

• universal-discovery-browser-design.md — Wave 0
• mtconnect-driver-design.md
• bacnet-ip-driver-design.md
• mqtt-sparkplug-driver-design.md
• sql-poll-driver-design.md
• omron-driver-design.md
• modbus-rtu-driver-design.md

Prior art the designs build on: 2026-05-28-driver-browsers-design.md (bespoke IBrowseSession pattern), 2026-06-12-galaxy-standard-driver-design.md (standard Equipment-kind driver), and the driver-typed tag editors design.

3. Shared architecture (the contract every driver in this program follows)

Every driver here is a standard Equipment-kind driver — an IDriver (lifecycle: Initialize / Reinitialize / Shutdown / GetHealth / footprint) composing only the capability interfaces its backend supports, from src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/:

Capability	Interface	This program's drivers
Discover	ITagDiscovery.DiscoverAsync(IAddressSpaceBuilder, ct)	all (device-enumerated or authored-only)
Read	IReadable	all
Subscribe	ISubscribable	all (poll-based, except the push drivers: MQTT, BACnet COV, MTConnect's /sample stream)
Write	IWritable	Omron, Modbus-RTU (v1); MTConnect/MQTT/BACnet/SQL read-only in v1
Probe	IHostConnectivityProbe	all
Re-discover	IRediscoverable	only drivers with a native backend change signal (per its doc contract) — e.g. MQTT rebirth; BACnet/SQL are discovery-capable but have no change signal and correctly omit it

Per-driver build pattern (identical across the fleet — see any existing driver + the driver-typed-editors design):

1. ZB.MOM.WW.OtOpcUa.Driver.<Name>.Contracts — options + tag DTOs + parser + enums (no backend NuGet dep).
2. ZB.MOM.WW.OtOpcUa.Driver.<Name> — the runtime IDriver + capabilities.
3. .Browser project only when bespoke browse is required (see §4).
4. Factory + DriverType string + Register(), wired in the Host's DriverFactoryBootstrap (factory + probe on admin nodes via TryAddEnumerable).
5. Typed tag editor + validator: a <Name>TagConfigModel (FromJson/ToJson/Validate) + TagConfigEditorMap + TagConfigValidator entries.
6. Docker fixture under tests/.../Docker/, deployed to 10.100.0.35 via lmxopcua-fix sync (the project=lmxopcua label is a deployment convention applied host-side — no checked-in compose file carries a labels: entry today).

3.1 Cross-cutting rules (mandatory, stated once)

• Enum-serialization trap — every enum on a config/tag surface MUST serialize as names, never numbers, on both sides (AdminUI page serializer and probe carry JsonStringEnumConverter). Two shipped factory-side patterns both satisfy this — Modbus keeps DTO fields string? + parses via ParseEnum<T> (no converter in the factory); OpcUaClient uses enum-typed DTOs with the converter in factory and probe. Either is fine; a mismatch between the authoring side and the factory side faults the driver at deploy. This is a proven systemic bug — see the driver enum-serialization memory. Non-negotiable for every driver here.
• Per-op deadline (R2-01 frozen-peer lesson) — every network call (read/write/discover/probe/ subscribe/agent-HTTP/SQL-query) MUST have a bounded deadline (linked-CTS / socket ReadTimeout / CommandTimeout). A frozen peer must never wedge a poll. No unbounded waits anywhere.
• WriteIdempotent — writable drivers flag only tags whose replay is safe. Note the invoker seam (ExecuteWriteAsync(…, isIdempotent, …)) exists but the host's write dispatch currently hardcodes isIdempotent: false (the arch-review "hardcoded isIdempotent" Medium) — so no write auto-retries today; drivers carry the per-tag flag now and retry activates when that fleet plumbing lands.
• Secrets — connection strings / API keys / broker creds come from env/secret refs (mirror ServerHistorian__ApiKey), never committed or logged.
• Ctor is connection-free — driver constructors must not open connections or touch the network; all connects happen in InitializeAsync. Every existing driver already obeys this, and the universal browser's CanBrowse throwaway-instance pattern depends on it — a ctor-connecting driver would leak connections on every capability probe.
• Live-verify discipline — Razor binding + deploy-inertness bugs pass unit tests and review; every driver's picker/editor and deploy path gets a docker-dev /run live-verify before it's trusted.

4. Browse architecture — the two-tier decision

The program's central browse decision: instead of a bespoke browser per driver, a universal Discover-backed browser (DiscoveryDriverBrowser, Wave 0) captures any driver's ITagDiscovery.DiscoverAsync output and re-presents it to the picker. Bespoke IBrowseSession browsers are reserved for the cases the universal one genuinely can't serve.

The gate: a new default-interface member ITagDiscovery.SupportsOnlineDiscovery => false (mirrors the existing RediscoverPolicy default member). A driver opts in when its DiscoverAsync enumerates from the device; flat-address drivers leave it false and stay manual-entry. This member is added in the universal browser's P1 — every browsable driver in this program depends on it, which is why the universal browser is the Wave-0 gate.

Driver	Browse mechanism	Verdict
MTConnect	/probe device model via DiscoverAsync	Universal (SupportsOnlineDiscovery=true)
BACnet/IP	Who-Is + object-list via DiscoverAsync	Universal for small sites; bespoke-lazy IBrowseSession a documented P-later for large multi-device sites
AbCip (existing)	CIP Symbol 0x6B/0x6C in DiscoverAsync	Universal (+ PatchForBrowse EnableControllerBrowse:true)
TwinCAT (existing)	ADS symbol upload in DiscoverAsync	Universal (+ patch); bespoke lazy only if symbol set too big
FOCAS (existing)	FixedTree curated discovery	Universal (curated; + patch FixedTree.Enabled:true + an UntilStable settle — its tree fills post-connect, so a one-shot capture would be empty)
MQTT/Sparkplug	passive topic/birth observation window	Bespoke MqttBrowseSession — runtime discovery is authored-only, so universal would replay only authored tags
SQL poll	INFORMATION_SCHEMA schema walk	Bespoke SqlBrowseSession — schema walk ≠ DiscoverAsync
Omron	libplctag @tags unsupported on NJ/NX	None live — offline Sysmac tag-export importer (P-later); FINS never browsable
Modbus RTU	flat registers, no discovery	None — not browsable
S7, AbLegacy, Modbus(TCP) (existing)	flat / no on-wire symbols	None — documented in the audit

Bespoke browsers override the universal fallback simply by being registered for their DriverType (BrowserSessionService resolves bespoke-first). See the universal-browser design §3 for the required BrowserSessionService fallback change (the ToDictionary dup-key constraint means the universal browser is a separate IUniversalDriverBrowser, not one of the injected IDriverBrowser set).

5. Library & license decisions

Driver	Library	License	Note
MTConnect	TrakHound MTConnect.NET (-Common + -HTTP)	MIT	netstd2.0 → loads on net10; pin + license-review; hand-roll fallback
BACnet/IP	System.IO.BACnet (ela-compil fork, BACnet)	MIT	multi-targets net10; YABE engine; callback API → TCS adapter
MQTT/Sparkplug	MQTTnet v5 + hand-rolled Tahu protobuf	MIT	Not SparkplugNet — its MQTTnet-4.x transitive pin collides with the repo's deliberately-OFF transitive pinning (Roslyn-split constraint)
Omron (CIP)	libplctag.NET	MPL-2.0	same dep AbCip already ships (plc=omron-njnx); FINS hand-rolled
SQL poll	Microsoft.Data.SqlClient	—	already in-repo at 6.1.1 — zero new deps for P1
Modbus RTU	extend existing Modbus	—	RTU-over-TCP only behind the existing IModbusTransport seam; zero new deps. Direct serial (System.IO.Ports) descoped (user, 2026-07-15) — serial buses are reached via serial→Ethernet gateways
MELSEC	hand-rolled 3E	—	deferred; HslCommunication is a commercial-license blocker

Avoid HslCommunication (commercial license; the "MIT" claim is a stale abandoned fork) for both MELSEC and Omron.

6. Dependency graph & build order

Wave 0 ─ Universal browser  ── adds ITagDiscovery.SupportsOnlineDiscovery ──┐
   │        (DiscoveryDriverBrowser + CapturingAddressSpaceBuilder +         │ every browsable
   │         BrowserSessionService fallback)                                 │ driver depends
   │                                                                         │ on this member
   ├── immediately lights up AbCip / TwinCAT / FOCAS pickers (existing)      │
   ▼                                                                         │
Wave 1 ─ Modbus RTU (extend, lowest effort)   ·   SQL poll (bespoke schema browser) ◄┘
   ▼
Wave 2 ─ MTConnect (universal browse)   ·   MQTT/Sparkplug (bespoke observation browser)
   ▼
Wave 3 ─ BACnet/IP (universal browse + COV)   ·   Omron (CIP-first; last — no NJ/NX CIP sim)

Wave	Item	Browse	Write v1	Effort
0	Universal DiscoveryDriverBrowser	—	—	S–M (~3–5 d)
1	Modbus RTU	none	Yes	S (lowest)
1	SQL poll	bespoke schema	No	S–M
2	MTConnect Agent	universal	No	S–M
2	MQTT/Sparkplug B	bespoke observation	No	M–L
3	BACnet/IP	universal (+ bespoke-lazy P-later)	No	M–L
3	Omron	offline import	Yes	M–L

Sequencing rationale: Wave 0 first because the SupportsOnlineDiscovery seam gates every browsable driver and it retrofits browse to three shipped drivers for near-zero marginal cost. Wave 1 is the low-effort/high-leverage pair (RTU extends an existing driver; SQL adds zero deps). Wave 2 is the strategic telemetry/UNS pair. Wave 3 is the heavier/test-gated pair — Omron last because there is no free NJ/NX CIP simulator, so its wire correctness is a live-hardware gate. MELSEC is out of the sequence (deferred).

7. Testing & fixtures

• Fixtures live under tests/.../Docker/, deployed to the shared host 10.100.0.35 via lmxopcua-fix sync (which owns the project=lmxopcua labelling host-side). Per-driver sims: mtconnect/cppagent, Mosquitto/EMQX + Sparkplug simulator, bacnet-stack/ela-compil BasicServer, central SQL Server (10.100.0.35,14330) + seeded table, rtu_over_tcp pymodbus profile (no serial rig — direct-serial descoped).
• Env-gated live suites for what a sim can't cover: BACnet Who-Is/BBMD UDP broadcast, Omron CIP wire (live hardware), MQTT broker soak.
• Per driver: unit (framer/parser/type-map with golden vectors + fakes) → integration (docker fixture) → live /run on docker-dev (picker + editor + deploy path).

8. Deferred / out of scope

• MELSEC SLMP — deferred (§1). Research complete (melsec-slmp.md); revisit on demand.
• Modbus direct-serial transport (ModbusRtuTransport / System.IO.Ports) — descoped (user, 2026-07-15); design record kept in the RTU doc §2b. RTU-over-TCP via a serial→Ethernet gateway is the only shipped RTU mode.
• Write-back for MTConnect (Interfaces), MQTT (NCMD/DCMD), BACnet (WriteProperty priority array), SQL (parameterized UPSERT) — all documented as later phases in the respective designs.
• Bespoke lazy browsers for large BACnet sites and large ControlLogix/TwinCAT symbol sets — graduate from universal only when eager one-shot discovery proves too heavy.
• Shared Cip.Core extraction (AbCip ↔ Omron CIP) — a follow-up, not a gate; keep Omron self-contained in v1.

9. Status

Research: done (8 reports). Designs: done (7 design docs). Code: Wave 0 landed — the universal Discover-backed browser (P1) is code-complete and unit-tested on feat/universal-discovery-browser (CapturingAddressSpaceBuilder + CapturedTreeBrowseSession

• DiscoveryDriverBrowser/IUniversalDriverBrowser + BrowserSessionService fallback + ITagDiscovery.SupportsOnlineDiscovery gate + AbCip/TwinCAT/FOCAS opt-ins & picker browse UI), with the live /run gate outstanding. Next action: complete the Wave-0 live gate, then Wave 1.