Files
lmxopcua/docs/plans/2026-07-15-mtconnect-driver-design.md
T
Joseph Doherty 8fc147d8d4
v2-ci / build (push) Successful in 3m14s
v2-ci / unit-tests (push) Failing after 9m13s
docs: driver-expansion program — 8 research reports + 7 design docs, parallel-reviewed
Adds the driver-expansion program design (umbrella: universal Discover-backed
browser + MTConnect, MQTT/Sparkplug B, BACnet/IP, SQL poll, Omron, Modbus RTU;
MELSEC deferred) plus the per-driver research reports.

All docs went through a 7-agent parallel review against the codebase before
this commit. Highlights fixed in review:

- universal browser: FOCAS FixedTree fills post-connect -> UntilStable settle
  + FixedTree.Enabled patch; MQTT reconciled to bespoke (was contradicting the
  program doc's SupportsOnlineDiscovery=false verdict)
- modbus-rtu: SerialPort.ReadTimeout doesn't bound async BaseStream reads ->
  linked-CTS per-op deadline (R2-01 class); BCL enum reuse would leak
  System.IO.Ports into Contracts
- bacnet: DiscoveryRediscoverPolicy enum name; UDP 47808 contention; live
  suite rewritten around unicast Who-Is + BBMD (broadcast doesn't cross VMs)
- sql-poll: real tier registration via DriverFactoryRegistry.Register;
  blackhole gate must not docker-pause the shared central SQL Server
- mqtt: Sparkplug v3.0 STATE topic form; first-in-repo proto codegen noted
- omron: host hardcodes isIdempotent:false today (retry seam unshipped);
  v1 scopes UDTs to dotted-leaf access
- mtconnect: SecurityClassification.ViewOnly; factory ParseEnum<T> pattern
- program doc: both valid enum-serialization patterns; IRediscoverable is
  change-signal-gated; RTU P2 adds System.IO.Ports; label is host-side
2026-07-15 16:40:36 -04:00

23 KiB
Raw Blame History

MTConnect (Agent-first) driver — executable implementation design

Status: design, build-ready. 2026-07-15. Research input: docs/research/drivers/mtconnect-agent.md — this doc turns that research into a build spec; it does not re-argue the protocol findings. House-style references: 2026-06-12-galaxy-standard-driver-design.md, 2026-07-15-universal-discovery-browser-design.md.


1. Motivation + scope

MTConnect is the dominant open read-only telemetry standard for machine tools; adding a DriverType = "MTConnect" Equipment-kind driver lets OtOpcUa surface a machine's self-describing device model (positions, spindle speed, execution state, availability, conditions) alongside — and complementing — the lower-level FOCAS driver (FOCAS reaches the Fanuc CNC directly; MTConnect reaches the vendor-neutral Agent that often front-ends that same machine). v1 is agent-first, read-only (Discover + Read + Subscribe, no Write) because the mainstream MTConnect surface (/probe, /current, /sample) is read-only by design. Full rationale, protocol details, library survey, and risk register live in the research report §1–§8; this doc assumes them.

2. Project layout

Two new projects, mirroring the Modbus split (contracts DTOs isolated from runtime so the AdminUI probe/editor can reference config shapes without the driver's NuGet deps). No .Browser project — see §4.

src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/
  MTConnectDriverOptions.cs            // strongly-typed runtime options (like ModbusDriverOptions)
  MTConnectTagDefinition.cs            // one record per authored tag (FullName=dataItemId + mt* metadata)
  MTConnectDataTypeInference.cs        // pure category/type/units → DriverDataType table (§3.3) — shared by driver, factory, editor, browser-commit
  ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj   // refs Core.Abstractions only

src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/
  MTConnectDriver.cs                   // IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable
  MTConnectAgentClient.cs              // thin seam over MTConnectHttpClient (probe/current/sample); testable via IMTConnectAgentClient
  IMTConnectAgentClient.cs             // seam interface — canned-XML fake in unit tests
  MTConnectObservationIndex.cs         // dataItemId → latest DataValueSnapshot, updated by /current + /sample
  MTConnectDriverFactoryExtensions.cs  // Register(registry, loggerFactory) + CreateInstance
  MTConnectDriverProbe.cs              // IDriverProbe: one-shot /probe reachability
  ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj   // refs Core, Core.Abstractions, .Contracts + TrakHound pkgs

tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/
  Fixtures/*.xml                       // canned probe/current/sample docs (§8)
  ...Tests.cs

.csproj boilerplate copies Modbus verbatim: net10.0, Nullable=enable, TreatWarningsAsErrors=true, GenerateDocumentationFile, InternalsVisibleTo the Tests project.

NuGet dependency

Consume TrakHound MTConnect.NET client packages (MIT) — pin an exact version:

<PackageReference Include="MTConnect.NET-Common" Version="6.9.0.2" />
<PackageReference Include="MTConnect.NET-HTTP"   Version="6.9.0.2" />
<!-- MTConnect.NET-XML / -JSON pulled transitively for serialization -->

These target netstandard2.0 (load cleanly on .NET 10) and give MTConnectHttpClient (probe/current/sample with polling and the multipart long-poll stream, gzip, XML+JSON) plus the model types (IDevice/IComponent/IDataItem/IObservation) — removing the multipart-framing / version-negotiation / buffer-overflow-rebaseline grind.

License-pin caveat (from research §1.4): older TrakHound releases carried mixed MIT / Apache-2.0 / "all rights reserved" strings. Before merge, confirm the embedded LICENSE of the pinned version is MIT (a legal-review checkbox, not a blocker). Hand-roll fallback if review fails: HttpClient + System.Xml.Linq for probe/current, a multipart/x-mixed-replace boundary reader for sample (~300500 LoC behind the same IMTConnectAgentClient seam — the driver above the seam is unchanged, so the fallback is a drop-in of one class).

3. Capability mapping (concrete seam wiring)

MTConnectDriver implements IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverablenot IWritable (§3.5). One MTConnectAgentClient + one MTConnectObservationIndex per driver instance; one shared sample stream (the Agent streams the whole device model, so per-tag streams would be wasteful).

3.1 IDriver

  • InitializeAsync(json, ct): deserialize MTConnectDriverOptions (via factory), construct MTConnectAgentClient from AgentUri, run one /probe under a per-call deadline (§7). Cache the parsed IDevice[] model + Header.instanceId. Prime the MTConnectObservationIndex with one /current. Set DriverState.Healthy on success; Faulted (rethrow) on failure — the actor marks the instance Faulted, nodes go Bad, process stays up.
  • ReinitializeAsync: tear down the sample stream, re-run Initialize. Config-only change (no address-space rebuild) unless AgentUri/DeviceName changed.
  • ShutdownAsync: stop the sample stream, dispose the HTTP client.
  • GetHealth: DriverHealth(State, LastSuccessfulRead, LastError)LastSuccessfulRead = last /current or /sample chunk time.
  • GetMemoryFootprint / FlushOptionalCachesAsync: footprint ≈ cached probe model + observation index; flush drops the browse/probe-model cache (re-fetchable), keeps the observation index (correctness state).

3.2 ITagDiscovery — plugs into the universal browser (§4)

public bool SupportsOnlineDiscovery => true;                 // /probe enumerates from the device
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once;  // probe is synchronous + complete
public async Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken ct)

DiscoverAsync fetches (or reuses the Initialize-cached) /probe model and streams the full Device→Component→DataItem tree into the builder:

  • each Devicebuilder.Folder(device.Name, device.Name) → child builder;
  • each nested Component → recursive Folder(component.Name, component.DisplayName) on the parent's child builder (component nesting becomes folder nesting — the builder-graph is the tree);
  • each DataItemchild.Variable(browseName, displayName, attr) where attr = new DriverAttributeInfo(FullName: dataItem.Id, DriverDataType: MTConnectDataTypeInference.Infer(category, type, units), IsArray: representation==TIME_SERIES, ArrayDim: null, SecurityClass: SecurityClassification.ViewOnly, IsHistorized: false, IsAlarm: category==CONDITION). (ViewOnly is the read-only-from-OPC-UA tier — SecurityClassification has no ReadOnly member.) browseName = dataItem.Name ?? dataItem.Id (dataItem name is optional in MTConnect; fall back to id, which is guaranteed unique).

Critical: DriverAttributeInfo.FullName = dataItem.Id. This is (a) the value the universal browser commits as TagConfig.FullName, and (b) the key the read/subscribe paths resolve against — the two align by construction (research §3). If DeviceName scopes to one device, only that device's subtree is streamed.

DeviceName scoping and the whole-model-in-one-call nature make this a natural fit for eager one-shot discovery — exactly what the universal browser assumes.

3.3 Data-type mapping (MTConnectDataTypeInference.Infer)

Pure function in .Contracts (shared by driver, browser-commit, and the typed editor so all three agree). Weak wire typing means this is a heuristic, stored per-tag and author-overridable (§5):

MTConnect DriverDataType
SAMPLE numeric (has units) Float64
SAMPLE representation=TIME_SERIES Float64 array (IsArray=true)
EVENT numeric type (PartCount, Line, …) Int64
EVENT controlled-vocab (Execution, ControllerMode, Availability, …) String
EVENT free text (Program, Block, Message) String
CONDITION String (state word; §3.6)

Quality mapping (DataValueSnapshot.StatusCode): an observation value of UNAVAILABLE (MTConnect's explicit no-data sentinel) → OPC UA Bad/Uncertain, not the literal string. Missing dataItem / empty condition → Bad. observation.timestampSourceTimestamp (not a separate node).

3.4 IReadable/current

ReadAsync(fullReferences, ct): issue one /current under a per-call deadline, index the returned MTConnectStreams observations by dataItemId into MTConnectObservationIndex, then return one DataValueSnapshot per requested ref in order. A ref absent from the response → a Bad-coded snapshot (not a throw). Reads are idempotent (matches IReadable contract; DriverCapability.Read auto-retries). /current is the whole-device snapshot regardless of how many refs are requested, so batch reads cost one round-trip.

3.5 ISubscribable/sample multipart long-poll

  • SubscribeAsync(fullReferences, publishingInterval, ct): on the first subscription, start the shared MTConnectAgentClient sample stream (/sample?from=<nextSequence>&interval=<SampleIntervalMs>&count=<SampleCount>). Record the subscribed ref set. Immediately fire OnDataChange for each subscribed ref from the primed /current values (OPC UA initial-data convention). Return an ISubscriptionHandle (monotonic id + DiagnosticId).
  • Stream pump: each received MTConnectStreams chunk → for each observation whose dataItemId ∈ subscribed set, update the index and raise OnDataChange(handle, dataItemId, snapshot). Advance from = Header.nextSequence for the next request (contiguous, gap-free).
  • Ring-buffer overflow: if the Agent reports a sequence gap / from older than firstSequence, re-/current to re-baseline, then resume the stream from the new nextSequence. (TrakHound handles this internally; the hand-roll fallback must replicate it — unit-tested via a fixture with a forced gap.)
  • UnsubscribeAsync(handle, ct): drop that handle's refs from the subscribed set; stop the shared stream when the set empties.
  • publishingInterval maps to the /sample interval when it is the only/first subscription; the driver polls at the finest requested interval and fans out (one stream per instance, not per tag).

3.6 IWritablenot implemented (v1)

Justified per research §2.1: the MTConnect Agent surface is read-only by design; write-back exists only via optional, rarely-deployed MTConnect Interfaces (a request/response state-machine handshake, not a "set value" — modelling it as IWritable would mislead). Capability interfaces are composable, so omitting IWritable is idiomatic (Galaxy's write path is even fire-and-forget). Nodes materialize without the AccessLevels.CurrentWrite bit automatically. Revisit Interfaces only if a concrete deployment needs it.

3.7 IHostConnectivityProbe + IRediscoverable

  • IHostConnectivityProbe: expose one HostConnectivityStatus per Agent (HostName = AgentUri). A cheap periodic /probe (or reuse the sample-stream heartbeat) flips Running ↔ Stopped; raise OnHostStatusChanged on transition. Enable/interval from MTConnectDriverOptions.Probe (mirrors ModbusProbeOptions).
  • IRediscoverable: watch the Agent Header.instanceId on every /current//sample chunk. A change means the Agent restarted / its model changed → raise OnRediscoveryNeeded so DriverHost rebuilds the address space (mirrors Galaxy's DeployWatcher). This is why RediscoverPolicy = Once is safe: instanceId change, not polling, drives re-discovery.

3.8 CONDITION modelling

v1 simple (this design): each CONDITION DataItem is a String variable whose value is the current state word (Normal/Warning/Fault/Unavailable), optionally suffixed with nativeCode. Zero alarm plumbing. IsAlarm=true is still set on the DriverAttributeInfo so the browser side-panel flags it and a future upgrade needn't re-author tags. v1.5 (fast-follow, not in this build): implement IAlarmSource + emit a TagConfig alarm object so Fault/Warning become native OPC UA Part 9 alarms, routing on the authored dotted ConditionId per the Galaxy/Phase-B native-alarm pattern.

4. Browse — plugs into the universal browser (reconciliation)

The research report (§4.2) proposed a bespoke ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Browser project with a hand-written MTConnectBrowseSession. This design supersedes that: build NO bespoke browser. With the universal discovery browser in place (the Wave-0 gate, which lands before this Wave-2 driver), MTConnect browse is covered for free by the generic DiscoveryDriverBrowser, because MTConnect browse comes from discovery (/probe → the device model → the same DiscoverAsync stream). Concretely, the universal browser already does exactly what the proposed bespoke session would have done:

  • DiscoveryDriverBrowser.CanBrowse("MTConnect", cfg) returns true because TryCreate succeeds and the instance is ITagDiscovery { SupportsOnlineDiscovery: true } (§3.2) → the AdminUI renders the Browse button.
  • OpenAsync constructs the driver, InitializeAsync (the /probe connect), runs DiscoverAsync into a CapturingAddressSpaceBuilder, then ShutdownAsync — the captured tree (Device folders → Component folders → DataItem leaves) is served by CapturedTreeBrowseSession. Each leaf's NodeId == DriverAttributeInfo.FullName == dataItemId, committed directly as TagConfig.FullName; IsAlarm/DriverDataType flow to the side-panel and pre-fill the typed editor.

No PatchForBrowse entry is needed — MTConnect discovery is unconditional (not config-gated like AbCip/TwinCAT's EnableControllerBrowse); it belongs in the "no patch" set alongside OpcUaClient/BACnet. No IUniversalDriverBrowser/BrowserSessionService change is needed — MTConnect lights up purely by setting SupportsOnlineDiscovery => true. The single required action to enable browse is that one default-member override in §3.2.

The only limit (universal-browser §8) is that discovery is eager one-shot; /probe is already a whole-model single call, so eager is the right fit and graduating to bespoke is unlikely.

5. Typed tag editor + validator (AdminUI)

Avoid the raw-JSON fallback by adding a typed editor (mirrors the Modbus template exactly).

  • src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs — pure FromJson/ToJson/Validate, preserves unknown keys via the TagConfigJson bag helper (copy ModbusTagConfigModel). Fields: FullName (dataItemId, required), MtCategory/MtType/MtSubType (enums / strings, from probe — read-only in UI), DataType (DriverDataType override dropdown), Units, MtDevice/MtComponent (author context). Validate() returns an error if FullName is blank.
  • .../Components/Shared/Uns/TagEditors/MTConnectTagConfigEditor.razor — thin shell: FullName text, read-only mtCategory/mtType, a DataType override <select>. Because most tags arrive via the browse picker (which fills the model), the editor is mostly a confirm/override surface.
  • Register ["MTConnect"] = typeof(Components.Shared.Uns.TagEditors.MTConnectTagConfigEditor) in Uns/TagEditors/TagConfigEditorMap.cs, and ["MTConnect"] = j => MTConnectTagConfigModel.FromJson(j).Validate() in TagConfigValidator.cs.

JsonStringEnumConverter enum-serialization trap (project-wide gotcha — MEMORY). Every driver's AdminUI serialization path must round-trip enums as name strings, never numerics. The factory/probe DTOs are string-typed (ParseEnum<T>), so a numerically-serialized enum FAULTS the driver at deploy. Two concrete requirements: (1) MTConnectTagConfigModel.ToJson writes MtCategory/DataType via TagConfigJson.Set(bag, "dataType", DataType) which emits the enum name (the Modbus helper already does this — do not hand-roll JsonSerializer without JsonStringEnumConverter); (2) MTConnectDriverProbe parses with new JsonStringEnumConverter() in its JsonSerializerOptions (copy ModbusDriverProbe._opts); the factory keeps enum-carrying DTO fields string? and parses them via ParseEnum<T> — the Modbus factory pattern (its JsonOptions carries no enum converter). The .Contracts MTConnectDataTypeInference returns a DriverDataType enum; when it's written to JSON it must be the string.

6. Factory + registration

  • MTConnectDriverFactoryExtensions (copy Modbus): public const string DriverTypeName = "MTConnect"; Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null)registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory)). CreateInstance deserializes a MTConnectDriverConfigDto (nullable-init DTO with Tags, Probe), validates AgentUri present, builds MTConnectDriverOptions, returns new MTConnectDriver(options, id, agentClientFactory: null, logger). Copy the Modbus factory JsonOptions (PropertyNameCaseInsensitive, ReadCommentHandling=Skip, AllowTrailingCommas — no enum converter: enum-carrying DTO fields stay string? and go through ParseEnum<T>, per §5's gotcha).
  • Host wiring — 3 one-line edits in src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs:
    1. Register(...) body: add Driver.MTConnect.MTConnectDriverFactoryExtensions.Register(registry, loggerFactory);
    2. add using MTConnectProbe = Driver.MTConnect.MTConnectDriverProbe;
    3. AddOtOpcUaDriverProbes: add services.TryAddEnumerable(ServiceDescriptor.Singleton<IDriverProbe, MTConnectProbe>()); The probe MUST be in AddOtOpcUaDriverProbes (not only the factory path) so it reaches admin-only nodes — the admin-operations singleton that backs Test-Connect is admin-role-pinned (the MEMORY "driver probes on admin nodes" gotcha). TryAddEnumerable keeps a fused admin,driver node from double-registering (a dup makes the singleton's ToDictionary(p=>p.DriverType) throw).
  • MTConnectDriverProbe : IDriverProbe, DriverType => "MTConnect": parse the config DTO, one-shot GET {AgentUri}/probe under timeout, Ok=true+latency on any valid MTConnectDevices response, Ok=false+message on TCP/HTTP/timeout failure. Never throw (per IDriverProbe contract).
  • No AdminUI IDriverBrowser DI line — browse is the universal browser (§4), already registered.

7. Resilience / timeout (the R2-01 frozen-peer lesson)

Every agent HTTP request MUST carry a per-call deadline — never an unbounded wait (the R2-01 S7 finding: an async read that ignores its socket timeout wedges the poll loop against a frozen peer). Concretely:

  • /probe and /current: wrap each call in a CancellationTokenSource(RequestTimeoutMs) linked to the caller's ct; a timeout surfaces as a Bad-coded snapshot (read) or a Faulted init (probe), never a hang. Set HttpClient.Timeout and a linked CTS (belt-and-suspenders — HttpClient.Timeout doesn't cover the response-body read of a streamed multipart).
  • /sample long-poll stream watchdog: the stream is intentionally long-lived, so HttpClient.Timeout cannot bound it. Instead run a watchdog: if no chunk and no keep-alive heartbeat arrives within HeartbeatMs × N (e.g. 3× the Agent heartbeat), treat the stream as dead → cancel it, transition HostState.Stopped, and reconnect. The Agent's own heartbeat on the multipart boundary is the liveness signal; absence past the watchdog window is the frozen-peer detector.
  • Reconnect / backoff: geometric backoff (MinBackoffMsMaxBackoffMs) on stream drop or connect failure, mirroring ModbusReconnectOptions. On reconnect, re-baseline via /current then resume /sample from the fresh nextSequence.
  • All capability calls flow through the existing Phase-6.1 IDriverCapabilityInvoker resilience pipeline (Read/Discover/Subscribe/Probe auto-retry); the per-call deadlines above are the driver-internal floor beneath that pipeline.

8. Test fixtures

  1. Canned XML unit fixtures (bulk of coverage, no network). Capture one MTConnectDevices (probe) + a couple of MTConnectStreams (current + sample, incl. one with a forced nextSequence gap) from a demo agent into tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/. Drive through the IMTConnectAgentClient fake to pin: DiscoverAsync tree shape + leaf FullName==dataItemId, MTConnectDataTypeInference table, observation indexing, UNAVAILABLE→Bad, CONDITION→state-word, multipart chunk framing, and ring-buffer re-baseline paging.
  2. Dockerized mtconnect/cppagent (reproducible integration fixture). Add tests/.../Docker/docker-compose.yml with the project: lmxopcua label on every service, seeded with a canned Devices.xml (+ an SHDR simulator or the built-in adapter), exposed on the shared docker host 10.100.0.35; drive it via lmxopcua-fix up mtconnect + sync. Env-gated integration suite (*.IntegrationTests), skips cleanly when the fixture is down — the analog of the Modbus/S7 sims.
  3. Public demo agent (manual live smoke only). https://demo.mtconnect.org/ (or NIST smstestbed Devices.xml) for a real-shape browse/read/stream smoke. Internet-dependent, not in CI. Live-verify the browse picker on docker-dev per the universal-browser discipline: open the /uns TagModal picker for an MTConnect driver, confirm the Device→Component→DataItem tree renders and a picked leaf commits TagConfig.FullName = <dataItemId> (Razor binding bugs pass unit tests — always /run it).

9. Phasing + effort

  • P1 (this design) — Agent MVP, ~11.5 wk with TrakHound (~2.53 wk hand-rolled): .Contracts + Driver.MTConnect (IDriver+ITagDiscovery+IReadable+ISubscribable+IHostConnectivityProbe+IRediscoverable), MTConnectDriverProbe, SupportsOnlineDiscovery=true (browse is free via universal browser — no browser code), typed editor + map/validator entries, 3-line Host registration, canned-XML unit suite + cppagent fixture. CONDITION as String.
  • P1.5 fast-follow: CONDITION → native Part-9 alarms via IAlarmSource (Galaxy pattern); TIME_SERIES SAMPLE arrays; EVENT controlled-vocab → OPC UA enumerations.
  • P2 (on demand) — SHDR adapter ingest: add SourceMode: "Agent" | "Shdr"; SHDR opens the raw pipe-delimited TCP socket (:7878). Loses auto-discovery (no device model → SupportsOnlineDiscovery must return false in SHDR mode → picker falls back to manual entry; tags authored like Modbus). Niche.

Detailed risk register (weak typing, CONDITION analog, TrakHound license, demo-agent CI flakiness, ring-buffer overflow, netstandard2.0-on-.NET10) is in research §7 — not repeated here.