Files

T

Joseph Doherty 203000a80a docs: MTConnect write-back recorded; stale MQTT write-through/browse-gap records reconciled

2026-08-05 06:55:14 -04:00

27 KiB

Raw Blame History

MTConnect Driver

Getting-started guide for the MTConnect Agent driver (P1 Agent MVP). This is the short path — for the full design rationale read docs/plans/2026-07-15-mtconnect-driver-design.md and the build-vs-plan record in docs/plans/2026-07-24-mtconnect-driver.md; for the fixture recipe read tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md (not duplicated here).

Live gate: see the LIVE-GATE RESULT note in the plan's Task 21 (docs/plans/2026-07-24-mtconnect-driver.md) for the docker-dev /run verification outcome (browse picker / typed editor / deploy / read / subscribe against the real Agent fixture).

What it talks to

A MTConnect Agent — the vendor-neutral telemetry endpoint that front-ends a machine tool (or fronts an adapter that itself talks to the machine over SHDR). The driver speaks plain HTTP + XML against the Agent's three standard REST paths:

/probe — the static device model (Device → Component → DataItem tree)
/current — a snapshot of every DataItem's latest observation
/sample — a multipart/x-mixed-replace long-poll stream of observation deltas, keyed by a monotonic sequence number

Those three are the whole data plane: agent-first — Discover + Read + Subscribe. There is a fourth path, and it is not part of the data plane: the Agent's optional AllowPut surface, off by default in agent.cfg, which this driver's IWritable uses (b29623cf, 2026-08-05). It sets what the Agent reports for a DataItem; nothing reaches the machine. Read Write-back before enabling it.

Built vs. planned — read this before trusting the design doc's §2

The design doc (2026-07-15-mtconnect-driver-design.md) picked the TrakHound MTConnect.NET-Common / -HTTP NuGet packages (MIT, netstandard2.0) as the primary path, with a hand-rolled fallback. The hand-rolled fallback is what shipped. Task 6/7 of the implementation plan found by reflection + live invocation that the pinned TrakHound packages ship no XML formatter (Document Formatter Not found for "xml" — the formatter lives in a separate MTConnect.NET-XML package the design didn't pin) and that MTConnectHttpClientStream exposes no injectable HttpClient handler, so it cannot be unit-tested behind the driver's IMTConnectAgentClient seam. Both package references were dropped. There is no TrakHound dependency anywhere in the shipped driver — probe/current/sample parsing is hand-rolled System.Xml.Linq plus a multipart boundary reader, entirely behind IMTConnectAgentClient. See the CORRECTION block under Task 0 of the implementation plan for the full record.

Namespace-agnostic parsing. Every parser matches XML elements on LocalName only, ignoring the document's XML namespace entirely, so MTConnect 1.3 through 2.x documents all parse without a schema-version branch. This is deliberate, not sloppy: a real Agent injects vendor-extension Components in a foreign namespace whose child DataItems elements inherit the default namespace, and namespace-strict matching would silently drop the whole vendor component (and every DataItem beneath it) rather than fail loudly. Attributes are read unqualified for the same reason — so an xsi:type attribute is never mistaken for a DataItem's own type.

Project split

Project	Target	Role
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/`	net10.0	In-process driver — `MTConnectDriver`, the hand-rolled `IMTConnectAgentClient` (probe/current/sample), the observation index, the factory
`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/`	net10.0	`MTConnectDriverOptions`, `MTConnectTagDefinition`, and the pure `MTConnectDataTypeInference` table shared by the driver, browse-commit, and the AdminUI typed editor

No .Browser project — browse comes free from the Wave-0 universal discovery browser (see Browse below).

Capability surface

MTConnectDriver : IDriver, IReadable, IWritable, ISubscribable, ITagDiscovery, IHostConnectivityProbe, IRediscoverable (src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs).

IWritable landed 2026-08-05 (b29623cf), against the Agent's AllowPut surface rather than MTConnect Interfaces. It is observation injection, not machine command, and is off at two independent gates by default — read Write-back before authoring a writable MTConnect tag. Interfaces itself — the protocol's own request/response handshake — is a different mechanism and remains deferred; see Deferred.

Capability	Path	Notes
`ITagDiscovery`	`DiscoverAsync` — streams `/probe`'s Device→Component→DataItem tree into the address-space builder	`SupportsOnlineDiscovery = true`, `RediscoverPolicy = Once`
`IReadable`	`ReadAsync` → one `/current` per call	Not the production data path — see below
`IWritable`	`WriteAsync` → `POST {agent}/{device}?{dataItemId}={value}` on the Agent's `AllowPut` surface	Off by default — needs `Write.Enabled` and the Agent's `AllowPut`; CONDITION items are refused
`ISubscribable`	`SubscribeAsync`/`OnDataChange` — the shared `/sample` long-poll pump	The production data path
`IHostConnectivityProbe`	periodic `/probe` under `Probe.*`	No consumer wires it today — see Known limitations
`IRediscoverable`	watches the Agent's `Header.instanceId`	No consumer wires it today — see Known limitations

IReadable.ReadAsync is never called by the running server. The production data plane is entirely ISubscribable — DriverInstanceActor subscribes once and lives off OnDataChange. ReadAsync exists for the Client CLI (... read -n ...) and for the unit/integration suites; it is correct and tested, just not on the hot path.

Minimum deployment

"Drivers": {
  "mtconnect-1": {
    "Type": "MTConnect",
    "Config": {
      "AgentUri": "http://10.100.0.35:5000",
      "DeviceName": null,
      "RequestTimeoutMs": 5000,
      "SampleIntervalMs": 1000,
      "SampleCount": 1000,
      "HeartbeatMs": 10000,
      "Probe":     { "Enabled": true, "IntervalMs": 5000, "TimeoutMs": 2000 },
      "Reconnect": { "MinBackoffMs": 0, "MaxBackoffMs": 30000, "BackoffMultiplier": 2.0 },
      "Write":     { "Enabled": false },
      "Tags": []
    }
  }
}

RawTags[] is not authored by hand — DriverDeviceConfigMerger injects it at deploy time from the tags authored on the /raw tree.

Config keys

Read off MTConnectDriverOptions / MTConnectDriverConfigDto (src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs, src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs):

Key	Default	Notes
`AgentUri`	— (required)	Agent base URI; the driver appends `/probe`, `/current`, `/sample`
`DeviceName`	`null`	Double duty. (1) Scopes requests to `{AgentUri}/{DeviceName}/...`; `null` = whole Agent. (2) The write path's fallback device — the device a write is addressed to when the probe model does not say which device declared the DataItem (see Write-back)
`RequestTimeoutMs`	`5000`	Per-call deadline for `/probe` and `/current`
`SampleIntervalMs`	`1000`	The Agent's `/sample?interval=` query param
`SampleCount`	`1000`	The Agent's `/sample?count=` query param
`HeartbeatMs`	`10000`	The Agent's `/sample?heartbeat=` query param — also the watchdog's liveness window
`Probe.Enabled` / `Probe.IntervalMs` / `Probe.TimeoutMs`	`true` / `5000` / `2000`	Background connectivity-probe knobs (mirrors `ModbusProbeOptions`)
`Reconnect.MinBackoffMs` / `MaxBackoffMs` / `BackoffMultiplier`	`0` / `30000` / `2.0`	Geometric backoff after a failed request or a dropped `/sample` stream
`Write.Enabled`	`false`	The driver-side half of the write-back opt-in. An absent block means disabled — see Write-back
`Tags[]`	`[]`	Pre-v3 / CLI authoring surface — one `MTConnectTagDefinition` per DataItem `id`
`RawTags[]`	`[]` (deploy-injected)	The v3 data-plane binding — see below

All timing knobs are validated strictly positive at InitializeAsync (RequirePositive, mirroring the arch-review 01/S-6 lesson: a 0 timeout does not mean "wait forever," it faults the driver). This applies to RequestTimeoutMs, HeartbeatMs, SampleIntervalMs, SampleCount, and (when the probe is enabled) Probe.Interval / Probe.Timeout.

No save-time gate on a blank AgentUri in the AdminUI. The driver form renders an inline validation notice (_form.Validate()), but DriverConfigModal.SaveAsync has no per-form validation seam to block the Save button on it — no sibling driver form has one either. A blank AgentUri saves cleanly and fails at deploy with the driver's own error message, not at authoring time.

Data plane

FullName == DataItem `id`

FullName (both MTConnectTagDefinition.FullName and every RawTags[] blob's identifier) is the MTConnect DataItem@id attribute — the value the universal browser commits and the key the driver resolves reads/subscribes against. MTConnectTagConfigModel.FromJson (the AdminUI typed editor) tries three spellings in order — fullName → dataItemId → address — and takes the first non-blank one, normalising onto fullName on save. address is accepted because that's the field name RawBrowseCommitMapper writes for a driver with no typed editor path, so a browse-committed tag stays readable even before the editor round-trips it.

Coercion-type precedence

The driver keys its data plane by RawPath (the v3 raw-tag identity), not by DataItem id directly, resolving RawPath → dataItemId internally. Each raw tag's coercion type (the DriverDataType its Agent observation is parsed into) is resolved in this order, first non-null wins:

The RawTags[] blob's driverDataType or dataType field (both spellings accepted — the typed editor writes dataType, the driver's own tags[] shape uses driverDataType).
A matching Tags[] entry naming the same DataItem id.
The Agent's own /probe declaration, via MTConnectDataTypeInference.Infer (category/type/units/representation → DriverDataType).
DriverDataType.String — the coercion that cannot fail.

Quality mapping

UNAVAILABLE — MTConnect's one explicit "I have no value" sentinel — maps to BadNoCommunication (0x80310000). This is deliberately distinct from the fleet-standard BadCommunicationError (0x80050000, used elsewhere for the driver's own transport failure to the Agent): UNAVAILABLE means the Agent is reachable and answered, it just has no device-backed value for that item. A CONDITION observation's value is taken from its element name (<Normal/> → the string "Normal", <Fault/> → "Fault"); a <Unavailable/> condition element normalises onto the same UNAVAILABLE sentinel as every other category so one comparison covers all three.

Other status codes an observation can surface:

Code	Meaning
`BadTypeMismatch` (`0x80740000`)	The Agent's text isn't a value of the tag's coerced type at all
`BadOutOfRange` (`0x803C0000`)	The Agent reported a number the coerced type can't represent
`BadNotSupported` (`0x803D0000`)	A shape this build doesn't materialize — a `TIME_SERIES` vector, or a structured `DATA_SET`/`TABLE` observation (deferred to P1.5; see Known limitations)
`BadWaitingForInitialData`	Tag is authored but the Agent hasn't reported it yet
`BadNodeIdUnknown`	DataItem id is neither authored nor ever observed

Subscribe — the `/sample` pump

One shared /sample long-poll stream per driver instance (the Agent streams the whole device model regardless of which subset is subscribed, so per-tag streams would be wasted round-trips), run under a heartbeat watchdog — HttpClient.Timeout cannot bound a long-lived stream, so a missing chunk and missing keep-alive heartbeat within HeartbeatMs × N is what detects a frozen peer.

Ring-buffer overflow (the Agent's circular observation buffer wrapped past what the driver's cursor expects) is detected two ways, both triggering a /current re-baseline before the stream resumes:

IMTConnectAgentClient.IsSequenceGap — the next chunk's firstSequence is newer than the driver's expected cursor.
An MTConnectError document reporting OUT_OF_RANGE — real Agents return this both under HTTP 200 (a normal MTConnect error document) and as a bare HTTP 400, and the client handles both.

An Agent instanceId change means the Agent restarted, and is checked before the sequence-gap check (a restart also usually trips the gap, so the two need disambiguating, not just OR-ing together). On a changed instanceId the driver clears its cached probe model and raises OnRediscoveryNeeded, which surfaces a re-browse prompt on the AdminUI /hosts page — see Known limitations for what that does and does not do.

Write-back — observation injection, off by default

⚠️ A write sets what the Agent reports, not what the machine does. The value lands in the Agent's observation buffer exactly as if an adapter had reported it — visible to every consumer of that Agent, and overwritten by the next adapter/SHDR update for that DataItem. Nothing reaches the CNC. Never document, script, or operate this as a machine command. It is genuinely useful for operator-set EVENT values, for external systems feeding DataItems the adapter does not own, and for test rigs — and nothing else.

The mechanism is the Agent's AllowPut surface (POST {agent}/{device}?{dataItemId}={value}), a standard cppagent feature that is off by default. It is not MTConnect Interfaces — see Deferred for why Interfaces was rejected as the write-back shape. Design record: 2026-08-05-mtconnect-writeback-design.md §2.2; implementation b29623cf.

Two independent gates, both off by default

The Agent — AllowPut = true in agent.cfg. Optionally scope it with AllowPutFrom = <host list> so only named hosts may write; that list is the Agent's own access control and this driver neither reads nor influences it.
The driver — "Write": { "Enabled": true } in the driver config (default false; an absent block is disabled).

Either gate alone leaves the shipped read-only behaviour unchanged: with Write.Enabled off every slot is refused BadNotWritable without an Agent being touched, and with AllowPut off the Agent refuses and the driver reports BadCommunicationError.

The OPC UA write bit is authored, not derived — read this before filing a bug

Enabling write-back does not make any existing node writable over OPC UA. The CurrentWrite access bit is composed from the authored Tag.AccessLevel alone (AddressSpaceComposer → AddressSpaceApplier, which additionally requires a non-array tag → OtOpcUaNodeManager.ComposeAccessLevel); driver capability is never consulted at compose time. An operator must set the raw tag to ReadWrite — a browse-committed tag is created Read (RawBrowseCommitMapper.MapLeaf) whatever the browse said.

Browse does stamp writable leaves SecurityClassification.Operate when Write.Enabled is on (and ViewOnly otherwise, CONDITION always ViewOnly), on the same "a node cannot advertise writability the driver would refuse" principle MQTT pinned. But that stamp is honest labelling, not the access bit: nothing persists SecurityClass — Tag has no such column — so it reaches the browse session's attribute side-panel and stops there.

What a write does, slot by slot

WriteAsync never throws and one bad slot never fails the batch (matching ReadAsync); even cancellation returns a full-length result set. The refusal order is fixed, and each step answers a different operator question:

Condition	Status	Meaning
`Write.Enabled` off, or the DataItem is CONDITION-category	`BadNotWritable` (`0x803B0000`)	Nothing was attempted; no Agent was involved
No live Agent client (pre-`InitializeAsync`, faulted initialize, post-shutdown)	`BadNotConnected` (`0x808A0000`)	This driver is not running
The reference resolves to no DataItem, or to one whose owning device cannot be named	`BadNodeIdUnknown` (`0x80340000`)	There is no Agent URL the write could be addressed to
The value has no invariant string form — `null`, or non-scalar	`BadTypeMismatch` (`0x80740000`)	Arrays are refused rather than rendered (`Convert.ToString` would emit the literal `System.Double[]`, which the Agent stores happily)
The Agent refused, or could not be reached	`BadCommunicationError` (`0x80050000`)	Includes an `MTConnectError` document returned under HTTP 200

CONDITION DataItems are excluded deliberately — they carry alarm semantics and a different level|text wire format, and are out of scope for v1.

The Agent's refusal text is logged at Warning and never stored. It stays out of DriverHealth.LastError, which is type-only by repo-wide discipline, and a refused write is not evidence the read path degraded — so a refusal leaves driver health untouched.

Device resolution needs no extra config. TagBinding.WriteTargetsByDataItemId, built at bind time from a walk of the cached probe model, remembers which device declared each DataItem, so a multi-device Agent is addressed correctly; the fallback is the configured DeviceName, and an unresolvable device is BadNodeIdUnknown rather than a guess.

Values are rendered with CultureInfo.InvariantCulture. A Boolean therefore writes the invariant string "True"/"False", not "1"/"0" — if the consuming adapter or Agent vocabulary expects something else, author the tag as a String and write the exact token.

Browse — free via the universal browser

MTConnect ships no bespoke browser project. Setting ITagDiscovery.SupportsOnlineDiscovery => true is the entire integration: the Wave-0 DiscoveryDriverBrowser sees a driver whose TryCreate succeeds and whose instance reports online discovery, renders the AdminUI Browse button, constructs the driver, runs InitializeAsync (the /probe connect)

DiscoverAsync into a CapturingAddressSpaceBuilder, then tears the throwaway instance down. Each captured leaf's NodeId is the DataItem id, committed directly as TagConfig.FullName on pick. See docs/plans/2026-07-15-universal-discovery-browser-design.md.

CONDITION modelling (v1: plain String)

Each CONDITION DataItem materializes as a String variable whose value is the current state word — Normal / Warning / Fault / UNAVAILABLE. There is no native OPC UA Part 9 alarm plumbing in this build; DriverAttributeInfo.IsAlarm = true is still stamped on a CONDITION leaf so the browse side-panel flags it and a future upgrade to native alarms doesn't need re-authoring. See Deferred.

Known limitations

These are real, not placeholders — read them before relying on the driver for anything beyond values-and-conditions.

A restarted Agent prompts an operator; it does not re-shape the address space. DriverInstanceActor now consumes OnRediscoveryNeeded (2026-07-27), so a changed instanceId raises a "re-browse" chip against this driver on /hosts carrying the reason. It is advisory: v3 authors raw tags through the /raw browse-commit flow, so until an operator re-browses the Agent and commits, any DataItem that appeared or vanished is not reflected in the served tree. A runtime graft was deliberately rejected — it would materialise nodes nobody approved and no deployment artifact records.

IHostConnectivityProbe is still unconsumed — GetHostStatuses() has no production call site and nothing writes a DriverHostStatus row. That half remains a fleet-wide gap affecting every driver that implements it (Gitea #521).
RawTagEntry.DeviceName is accepted on the wire shape but ignored for routing. One driver instance owns exactly one Agent client scoped by MTConnectDriverOptions.DeviceName (the top-level config key); the per-tag RawTagEntry.DeviceName field the v3 raw-tag identity carries is never read by the driver. Two /raw Devices authored under one MTConnect driver instance both resolve to the same one Agent connection. Real per-device routing (a client per device) would be a design change, not a bug fix.

The write path is the one place a device name is resolved per DataItem, and it does not use RawTagEntry.DeviceName either: it reads the owning device out of the cached probe model and falls back to the top-level DeviceName (see Write-back). So MTConnectDriverOptions.DeviceName does double duty — request scope for reads/browse, fallback write target for writes.
Nothing validates a raw tag's declared OPC UA DataType against the blob's coercion type. The /probe-sourced inference (MTConnectDataTypeInference) narrows how often an author picks a wrong type, but it doesn't close the gap — an authored DataType mismatched against the actual Agent observation still surfaces as BadTypeMismatch /BadOutOfRange at runtime rather than at authoring time.
sampleCount is not a legal DataItem attribute. It belongs on a TIME_SERIES observation, not the device-model declaration. A real Agent that meets sampleCount on a DataItem declaration logs The following keys were present and not expected: sampleCount and drops the entire data item from the served model. A live TIME_SERIES tag therefore always resolves to ArrayDim = null (variable-length array) in practice — the fixture's canned probe.xml/Devices.xml, which does declare a sampleCount, is unrealistic on this point and exists only to pin the parsing rule itself.
CONDITION is a plain String, not a native Part 9 alarm (see above).
TIME_SERIES arrays and structured DATA_SET/TABLE observations surface as BadNotSupported, not an approximation — see the Quality mapping table.

Deferred (not in this build)

MTConnect Interfaces. Still deferred, and for the reason design §3.6 gave: it is a rare, optional request/response state-machine handshake requiring adapter-side cooperation, not a "set value" operation, and would mislead if modelled as IWritable. Write-back itself is no longer deferred — b29623cf shipped IWritable over the Agent's AllowPut observation-injection surface instead, a different mechanism entirely. See Write-back and 2026-08-05-mtconnect-writeback-design.md §2.1.
P1.5 fast-follow (design §9): CONDITION → native OPC UA Part 9 alarms via IAlarmSource (the Galaxy native-alarm pattern); TIME_SERIES SAMPLE arrays materialized as real OPC UA arrays; EVENT controlled-vocabulary values → OPC UA enumerations.
P2 (on demand): SHDR adapter ingest. A SourceMode: "Agent" | "Shdr" switch that opens the raw pipe-delimited SHDR TCP socket directly. Loses auto-discovery (no device model without an Agent in front, so SupportsOnlineDiscovery would have to report false and tags would be authored by hand, like Modbus). Niche; not built.

Testing

Unit tests — tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ (canned-XML fixtures, no network) — 491/491 at time of writing. Covers discovery tree shape, MTConnectDataTypeInference, observation indexing, UNAVAILABLE → BadNoCommunication, CONDITION state-word mapping, multipart chunk framing, and ring-buffer re-baseline paging (both the sequence-gap and the OUT_OF_RANGE legs). The write path adds MTConnectAgentClientWriteTests (9f17d7ef — verb, URL, agent-error-in-200 detection) and MTConnectWriteTests (b29623cf — every refusal leg, the batch shape, the browse-stamping flip).
Integration tests — tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/ — env-gated on MTCONNECT_AGENT_ENDPOINT (default http://10.100.0.35:5000), skips cleanly (12 Skipped) when the fixture is unreachable, 12/12 against a real Agent. Read the fixture's own Docker/README.md for the full recipe — summary only, here:
- Image is mtconnect/agent:2.7.0.12 — not mtconnect/cppagent, which does not exist on Docker Hub (the design's original assumption).
- The stack is two services: the Agent plus a stdlib-only SHDR adapter (Docker/adapter.py, on python:3.13-alpine). An Agent with no adapter reports every observation UNAVAILABLE forever, which proves nothing.
- Deployed at /opt/otopcua-mtconnect/ on the shared docker host (10.100.0.35, project=lmxopcua label). Endpoint http://10.100.0.35:5000.
- agent.cfg must be pure ASCII — one non-ASCII byte anywhere, including a comment, makes the config parser reject the whole file with a bare Failed / Stopped at line: N.
- Bring-up: lmxopcua-fix sync mtconnect then lmxopcua-fix up mtconnect (PowerShell helper, Windows-only) — or, directly on the docker host, rsync the repo's Docker/ dir to /opt/otopcua-mtconnect/ and run docker compose up -d --wait.

27 KiB Raw Blame History Unescape Escape