From 3b92b7cc83980da1252eb2e56370f8226af7b3b7 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:43:23 -0400 Subject: [PATCH 01/34] docs(mtconnect): record TrakHound-vs-hand-rolled client decision (Task 0) --- docs/plans/2026-07-24-mtconnect-driver.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/plans/2026-07-24-mtconnect-driver.md b/docs/plans/2026-07-24-mtconnect-driver.md index 4118028b..87aa9ea6 100644 --- a/docs/plans/2026-07-24-mtconnect-driver.md +++ b/docs/plans/2026-07-24-mtconnect-driver.md @@ -48,6 +48,27 @@ git add docs/plans/2026-07-24-mtconnect-driver.md git commit -m "docs(mtconnect): record TrakHound-vs-hand-rolled client decision (Task 0)" ``` +**DECISION (verified 2026-07-24): TrakHound path.** All three checks passed against real nuget.org. +(1) `dotnet package search MTConnect.NET-Common --exact-match` and `...-HTTP --exact-match` both list +versions `3.2.0` through `6.9.0.2` on `nuget.org`, confirming `6.9.0.2` is the current top-of-list +version for both package ids (no version mismatch between the two). (2) A throwaway `net10.0` console +project in the scratchpad (outside the repo, so unaffected by this repo's `packageSourceMapping`) ran +`dotnet add package MTConnect.NET-Common -v 6.9.0.2` + `...-HTTP -v 6.9.0.2` + `dotnet restore` — both +restored cleanly with no source-mapping or compatibility errors; `project.assets.json` resolved +`MTConnect.NET-Common`, `MTConnect.NET-HTTP`, and the transitive `MTConnect.NET-TLS` (all `6.9.0.2`) +against the `net10.0` target framework, and each package's `lib/` folder contains a `netstandard2.0` +build (alongside `net6.0`–`net9.0`, `net461`–`net472`, `net47`, `net48`), so net10.0 resolves via the +in-box compatibility mapping. (3) The pinned version carries **no embedded LICENSE file** in the +`.nupkg` (`unzip -l` shows no `LICENSE*` entry in either package) — instead both nuspecs declare the +modern SPDX license-expression form `MIT` + +`https://licenses.nuget.org/MIT`, which nuget.org validates against the SPDX +list at push time and surfaces on the package page (confirmed live: the nuget.org page for +`MTConnect.NET-Common` `6.9.0.2` displays "MIT license" linked to `licenses.nuget.org/MIT`) — a +structurally stronger guarantee than a free-text embedded file for this specific version, so the +absence of a physical `LICENSE` file is not a fail. Task 1 adds +`PackageReference MTConnect.NET-Common 6.9.0.2` + `PackageReference MTConnect.NET-HTTP 6.9.0.2` to +`Driver.MTConnect.csproj`; the hand-roll fallback is not needed. + --- ## Task 1: Scaffold the two driver projects + the test project -- 2.52.0 From e36b9201134769d134cd3eaa5d7d5930b156d4b0 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:46:46 -0400 Subject: [PATCH 02/34] build(mtconnect): scaffold Contracts+Driver+Tests projects, add to slnx (Task 1) --- Directory.Packages.props | 5 +++ ZB.MOM.WW.OtOpcUa.slnx | 3 ++ ....OtOpcUa.Driver.MTConnect.Contracts.csproj | 19 ++++++++++ .../ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj | 32 +++++++++++++++++ ...M.WW.OtOpcUa.Driver.MTConnect.Tests.csproj | 35 +++++++++++++++++++ 5 files changed, 94 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj diff --git a/Directory.Packages.props b/Directory.Packages.props index baf53c23..3b622ce4 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -84,6 +84,11 @@ ships net6.0/net8.0 only. --> + + + + + + + + diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj new file mode 100644 index 00000000..9c0f0b4c --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj @@ -0,0 +1,32 @@ + + + + net10.0 + enable + enable + latest + true + true + $(NoWarn);CS1591 + ZB.MOM.WW.OtOpcUa.Driver.MTConnect + + + + + + + + + + + + + + + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj new file mode 100644 index 00000000..02432590 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj @@ -0,0 +1,35 @@ + + + + net10.0 + enable + enable + false + true + ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + + PreserveNewest + + + + -- 2.52.0 From 18742ea92b748883a92aaf88020f46add36c8a68 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:50:07 -0400 Subject: [PATCH 03/34] feat(mtconnect): driver options + tag definition records (Task 2) --- .../MTConnectDriverOptions.cs | 91 +++++++++++++++++++ .../MTConnectTagDefinition.cs | 49 ++++++++++ .../MTConnectDriverOptionsTests.cs | 80 ++++++++++++++++ 3 files changed, 220 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectTagDefinition.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs new file mode 100644 index 00000000..8b9a9031 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs @@ -0,0 +1,91 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// MTConnect Agent driver configuration. Bound from the driver's DriverConfig JSON at +/// DriverHost.RegisterAsync. The driver polls a remote MTConnect Agent's REST endpoints +/// (/probe, /current, /sample) rooted at and maps +/// each returned DataItem into an OPC UA variable per . +/// +public sealed class MTConnectDriverOptions +{ + /// + /// Gets the MTConnect Agent's base URI (e.g. http://agent:5000). Required — the + /// driver appends the standard Agent request paths (/probe, /current, + /// /sample) to this base. + /// + public required string AgentUri { get; init; } + + /// + /// Optional device-name scope. When set, requests are narrowed to + /// {AgentUri}/{DeviceName}/... so a multi-device Agent only serves one device's + /// DataItems through this driver instance. Default null = agent-wide (all devices). + /// + public string? DeviceName { get; init; } + + /// + /// Per-call HTTP deadline, in milliseconds, applied to every Agent request + /// (/probe, /current, /sample). Default 5000 — long enough for + /// a large device probe response over a LAN, short enough that a hung Agent surfaces as a + /// failed poll rather than wedging the driver. + /// + public int RequestTimeoutMs { get; init; } = 5000; + + /// + /// The interval query parameter (milliseconds) passed to the Agent's /sample + /// streaming request — the minimum time the Agent waits between successive chunks of the + /// response. Default 1000; must be non-zero so the sample pump cannot spin the + /// connection into a busy-loop. + /// + public int SampleIntervalMs { get; init; } = 1000; + + /// + /// The count query parameter passed to the Agent's /sample request — the + /// maximum number of DataItem observations returned per chunk. Default 1000, the + /// MTConnect Agent's own conventional default. + /// + public int SampleCount { get; init; } = 1000; + + /// + /// The heartbeat query parameter (milliseconds) passed to the Agent's /sample + /// streaming request — the Agent sends an empty chunk after this much idle time so the + /// client can detect a silently-stalled connection. Default 10000, matching the + /// MTConnect Agent's own conventional default; must be non-zero or a quiet connection can + /// never be distinguished from a dead one. + /// + public int HeartbeatMs { get; init; } = 10000; + + /// + /// Background connectivity-probe settings. When + /// is true the driver periodically issues a cheap /probe request and raises + /// OnHostStatusChanged on Running ↔ Stopped transitions. + /// + public MTConnectProbeOptions Probe { get; init; } = new(); + + /// + /// Reconnect backoff settings used after a failed Agent request or a dropped + /// /sample streaming connection. + /// + public MTConnectReconnectOptions Reconnect { get; init; } = new(); +} + +/// Background connectivity-probe knobs, mirroring ModbusProbeOptions. +public sealed class MTConnectProbeOptions +{ + /// Gets a value indicating whether probing is enabled. + public bool Enabled { get; init; } = true; + /// Gets the interval between probe requests. + public TimeSpan Interval { get; init; } = TimeSpan.FromSeconds(5); + /// Gets the probe request timeout. + public TimeSpan Timeout { get; init; } = TimeSpan.FromSeconds(2); +} + +/// Geometric-backoff settings for the post-failure reconnect loop, mirroring ModbusReconnectOptions. +public sealed class MTConnectReconnectOptions +{ + /// Delay before the first reconnect attempt, in milliseconds. Default 0 = immediate. + public int MinBackoffMs { get; init; } = 0; + /// Upper bound on the geometric backoff sequence, in milliseconds. + public int MaxBackoffMs { get; init; } = 30000; + /// Multiplier applied each retry. Default 2.0 doubles each step. + public double BackoffMultiplier { get; init; } = 2.0; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectTagDefinition.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectTagDefinition.cs new file mode 100644 index 00000000..c956f809 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectTagDefinition.cs @@ -0,0 +1,49 @@ +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// One MTConnect-backed OPC UA variable. Each authored tag binds a single Agent DataItem by +/// its id attribute (); the driver looks up the observation's +/// current value from the Agent's /current snapshot / /sample stream by that id. +/// +/// +/// The MTConnect DataItem's id attribute (the Agent's /probe response). This is +/// the driver's reference for read/subscribe — MTConnect is a read-only source protocol, so +/// there is no corresponding write path. +/// +/// Logical data type the DataItem's value is coerced to. +/// +/// When true, the tag is exposed as an OPC UA array (e.g. an MTConnect +/// PathPosition / vector-valued sample). Default false = scalar. +/// +/// +/// Element count when is true; ignored for scalar tags. Default +/// 0. +/// +/// +/// The DataItem's MTConnect category attribute — SAMPLE, EVENT, or +/// CONDITION. Optional metadata carried through for browse / display; not consulted by +/// the value-mapping path. Default null. +/// +/// +/// The DataItem's MTConnect type attribute (e.g. POSITION, EXECUTION). +/// Optional metadata. Default null. +/// +/// +/// The DataItem's MTConnect subType attribute (e.g. ACTUAL, COMMANDED). +/// Optional metadata. Default null. +/// +/// +/// The DataItem's MTConnect units attribute (e.g. MILLIMETER, +/// REVOLUTION/MINUTE). Optional metadata. Default null. +/// +public sealed record MTConnectTagDefinition( + string FullName, + DriverDataType DriverDataType, + bool IsArray = false, + int ArrayDim = 0, + string? MtCategory = null, + string? MtType = null, + string? MtSubType = null, + string? Units = null); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs new file mode 100644 index 00000000..0b77db18 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs @@ -0,0 +1,80 @@ +using Shouldly; +using Xunit; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 2 coverage for / : +/// defaults are sane (non-zero timers, probe on) and the required-field shape (AgentUri) binds. +/// +[Trait("Category", "Unit")] +public sealed class MTConnectDriverOptionsTests +{ + /// Default sample/heartbeat timers are non-zero and the probe is enabled by default. + [Fact] + public void Options_default_sample_and_heartbeat_are_sane() + { + var o = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }; + o.SampleIntervalMs.ShouldBeGreaterThan(0); + o.HeartbeatMs.ShouldBeGreaterThan(0); + o.Probe.Enabled.ShouldBeTrue(); + } + + /// AgentUri is the only required field; DeviceName scope defaults to unset (agent-wide). + [Fact] + public void Options_agent_uri_is_required_device_name_defaults_to_null() + { + var o = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }; + o.AgentUri.ShouldBe("http://agent:5000"); + o.DeviceName.ShouldBeNull(); + } + + /// RequestTimeoutMs and SampleCount default to sane, non-zero, non-wedging values. + [Fact] + public void Options_request_timeout_and_sample_count_are_non_zero() + { + var o = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }; + o.RequestTimeoutMs.ShouldBeGreaterThan(0); + o.SampleCount.ShouldBeGreaterThan(0); + } + + /// Probe sub-options mirror ModbusProbeOptions' shape: Enabled/Interval/Timeout, both non-zero. + [Fact] + public void Probe_defaults_have_non_zero_interval_and_timeout() + { + var probe = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }.Probe; + probe.Interval.ShouldBeGreaterThan(TimeSpan.Zero); + probe.Timeout.ShouldBeGreaterThan(TimeSpan.Zero); + } + + /// Reconnect sub-options default to a bounded, non-degenerate geometric backoff. + [Fact] + public void Reconnect_defaults_bound_backoff_growth() + { + var reconnect = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }.Reconnect; + reconnect.MaxBackoffMs.ShouldBeGreaterThan(reconnect.MinBackoffMs); + reconnect.BackoffMultiplier.ShouldBeGreaterThan(1.0); + } + + /// MTConnectTagDefinition round-trips its positional fields, including MTConnect metadata. + [Fact] + public void TagDefinition_carries_dataItemId_and_mtconnect_metadata() + { + var tag = new MTConnectTagDefinition( + FullName: "dtop_2", + DriverDataType: ZB.MOM.WW.OtOpcUa.Core.Abstractions.DriverDataType.Float64, + MtCategory: "SAMPLE", + MtType: "POSITION", + MtSubType: "ACTUAL", + Units: "MILLIMETER"); + + tag.FullName.ShouldBe("dtop_2"); + tag.DriverDataType.ShouldBe(ZB.MOM.WW.OtOpcUa.Core.Abstractions.DriverDataType.Float64); + tag.IsArray.ShouldBeFalse(); + tag.ArrayDim.ShouldBe(0); + tag.MtCategory.ShouldBe("SAMPLE"); + tag.MtType.ShouldBe("POSITION"); + tag.MtSubType.ShouldBe("ACTUAL"); + tag.Units.ShouldBe("MILLIMETER"); + } +} -- 2.52.0 From 992ffe56200f03100163557a89a283a56e424636 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:51:20 -0400 Subject: [PATCH 04/34] feat(mtconnect): IMTConnectAgentClient seam + neutral parse DTOs (Task 5) --- .../IMTConnectAgentClient.cs | 44 +++++ .../MTConnectDtos.cs | 170 ++++++++++++++++++ 2 files changed, 214 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs new file mode 100644 index 00000000..6f54ef98 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs @@ -0,0 +1,44 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// The seam between the MTConnect driver and an MTConnect Agent's HTTP surface +/// (/probe, /current, /sample). This interface is transport-neutral by +/// design — every method returns/yields the plain DTOs in MTConnectDtos.cs, never a +/// TrakHound type, an XElement, or any other wire/library artifact. That is what lets +/// every driver behaviour above this seam (Tasks 6–13) be unit-tested against a fake serving +/// canned probe/current/sample XML, with no sockets involved. +/// +/// +/// The production implementation (Task 6/7) is built on the TrakHound MTConnect.NET client +/// libraries (Task 0's decision); a fake for tests only needs to implement these three members. +/// +public interface IMTConnectAgentClient +{ + /// + /// Issues an Agent /probe request and returns the parsed device hierarchy. Called + /// once at driver initialization; the result is cached and later walked to build the OPC UA + /// browse tree, where each becomes a tag's FullName. + /// + /// Cancellation token. + Task ProbeAsync(CancellationToken ct); + + /// + /// Issues an Agent /current request and returns the latest observed value for every + /// data item. Used both to prime the observation index at startup and to re-baseline after + /// a detected sequence gap in . + /// + /// Cancellation token. + Task CurrentAsync(CancellationToken ct); + + /// + /// Opens an Agent /sample long-poll stream starting at sequence + /// and yields one per multipart chunk the Agent sends, + /// indefinitely, until is cancelled. Each yielded result's + /// is load-bearing — the caller compares + /// it against the requested to detect a ring-buffer sequence gap and + /// re-baseline via . + /// + /// The sequence number to resume streaming from. + /// Cancellation token; cancelling ends the stream. + IAsyncEnumerable SampleAsync(long from, CancellationToken ct); +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs new file mode 100644 index 00000000..76bd41c9 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs @@ -0,0 +1,170 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// Result of an Agent /probe request: the full device hierarchy the Agent exposes. +/// Produced by . This is the neutral shape both +/// the TrakHound-backed implementation and a hand-rolled XML fallback must produce — nothing +/// downstream of ever sees a TrakHound type or an +/// XElement. +/// +/// Every device the Agent's probe response declares. +public sealed record MTConnectProbeModel(IReadOnlyList Devices); + +/// +/// Common shape shared by and : +/// each may directly own data items and/or nest further components, to arbitrary depth. Backs +/// the recursive walk. +/// +public interface IMTConnectComponentContainer +{ + /// Components nested directly under this device/component (may be empty). + IReadOnlyList Components { get; } + + /// Data items declared directly on this device/component (may be empty). + IReadOnlyList DataItems { get; } +} + +/// +/// One MTConnect device from the Agent's probe response — the root of a nested +/// component/data-item tree. +/// +/// The device's id attribute. +/// The device's name attribute, when the Agent supplies one. +/// Child components declared directly under this device. +/// Data items declared directly on this device (outside any component). +public sealed record MTConnectDevice( + string Id, + string? Name, + IReadOnlyList Components, + IReadOnlyList DataItems) : IMTConnectComponentContainer; + +/// +/// One MTConnect component (e.g. Axes, Controller, Path) from the Agent's +/// probe response. Components nest to arbitrary depth — a component may itself contain further +/// child components as well as data items. +/// +/// The component's id attribute. +/// The component's name attribute, when the Agent supplies one. +/// Child components nested directly under this component. +/// Data items declared directly on this component. +public sealed record MTConnectComponent( + string Id, + string? Name, + IReadOnlyList Components, + IReadOnlyList DataItems) : IMTConnectComponentContainer; + +/// +/// One MTConnect DataItem declaration from the Agent's probe response. This is pure metadata — +/// it carries no observed value; values arrive separately via +/// / +/// and are correlated back to a data item by . +/// +/// +/// The DataItem's id attribute — globally unique within the Agent's probe response and +/// the correlation key used against . +/// +/// The DataItem's name attribute, when the Agent supplies one. +/// The DataItem's category attribute — SAMPLE, EVENT, or CONDITION. +/// The DataItem's type attribute (e.g. POSITION, EXECUTION). +/// The DataItem's subType attribute (e.g. ACTUAL, COMMANDED), when present. +/// The DataItem's units attribute (e.g. MILLIMETER, REVOLUTION/MINUTE), when present. +/// +/// The DataItem's representation attribute (e.g. TIME_SERIES, DISCRETE); +/// absent means the MTConnect default (VALUE). +/// +/// +/// The DataItem's sampleCount attribute — element count for a TIME_SERIES +/// representation. Present only on data items that carry one. +/// +public sealed record MTConnectDataItem( + string Id, + string? Name, + string Category, + string Type, + string? SubType, + string? Units, + string? Representation, + int? SampleCount); + +/// +/// Result of an Agent /current request, or of a single multipart chunk from a +/// /sample stream. Produced by and +/// . +/// +/// +/// The Agent's current instance id (its MTConnectStreams header instanceId). Changes +/// whenever the Agent restarts or its underlying device model changes; the driver watches this +/// for change to raise rediscovery rather than trusting a stale observation snapshot. +/// +/// +/// The header nextSequence — the sequence number to resume a subsequent +/// call from. +/// +/// +/// The header firstSequence — the oldest sequence number still held in the Agent's +/// circular buffer. Load-bearing for sequence-gap detection: a caller that requested +/// from a sequence older than this chunk's has fallen out of +/// the Agent's buffer and must re-baseline via . +/// +/// +/// The observations carried in this snapshot/chunk, in Agent-supplied order. Never null; +/// empty when the chunk carries no new observations (e.g. a heartbeat). +/// +public sealed record MTConnectStreamsResult( + long InstanceId, + long NextSequence, + long FirstSequence, + IReadOnlyList Observations); + +/// +/// One observed value for a single DataItem, as reported in a /current snapshot or a +/// /sample stream chunk. Deliberately dumb: is carried as the raw +/// string the Agent reported (including the literal UNAVAILABLE) with no interpretation +/// applied here — coercing UNAVAILABLE to a bad-quality status and converting the string +/// to the tag's DriverDataType are the observation index's job, not this layer's. +/// +/// +/// The reporting DataItem's id — correlates back to +/// from the probe model. +/// +/// +/// The observed value exactly as the Agent reported it, or the literal string +/// "UNAVAILABLE" when the Agent has no current value for the data item. Never +/// pre-parsed into a nullable or an enum at this layer. +/// +/// +/// The observation's Agent-reported timestamp, normalized to UTC ( +/// is always ). Becomes the OPC UA variable's SourceTimestamp. +/// +public sealed record MTConnectObservation( + string DataItemId, + string Value, + DateTime TimestampUtc); + +/// +/// Recursive helpers over the device/component tree. +/// +public static class MTConnectComponentTreeExtensions +{ + /// + /// Enumerates every data item owned by this device or component, plus every data item owned + /// by any component nested beneath it, to arbitrary depth. Order is depth-first: a node's + /// own data items first, then each child component's data items in turn. + /// + /// The device or component to walk. + public static IEnumerable AllDataItems(this IMTConnectComponentContainer container) + { + foreach (var dataItem in container.DataItems) + { + yield return dataItem; + } + + foreach (var child in container.Components) + { + foreach (var dataItem in child.AllDataItems()) + { + yield return dataItem; + } + } + } +} -- 2.52.0 From c0729ae5c0c527245a719cb9e61b22d42bea3a59 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:53:13 -0400 Subject: [PATCH 05/34] feat(mtconnect): pure data-type inference table + golden test (Task 3) --- .../MTConnectDataTypeInference.cs | 192 ++++++++++++++ .../MTConnectDataTypeInferenceTests.cs | 250 ++++++++++++++++++ 2 files changed, 442 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs new file mode 100644 index 00000000..cbf4312b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs @@ -0,0 +1,192 @@ +using System.Collections.Frozen; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// The type shape inferred for a single MTConnect DataItem: the scalar +/// plus the array shape that +/// / +/// expect. Returned by value (a readonly record struct) so the discovery loop and the +/// AdminUI editor can call the inference on a hot path without allocating. +/// +/// The scalar element type of the observation. +/// +/// true only for a SAMPLE declared representation="TIME_SERIES", whose +/// observation carries a vector of samples rather than a single value. +/// +/// +/// The probe-declared array length when is true and the device +/// model declared a positive sampleCount; null for a variable-length series +/// (many agents omit sampleCount) and always null for a scalar. +/// +public readonly record struct MTConnectInferredType(DriverDataType DataType, bool IsArray, uint? ArrayDim); + +/// +/// Translates an MTConnect DataItem's device-model metadata (category / type / units / +/// representation) into the server's . Pure and deterministic — +/// no I/O, no logging, no mutable state. +/// +/// +/// +/// This lives in .Contracts because three call sites must agree byte-for-byte, or a +/// tag's OPC UA type stops matching its authored config: ITagDiscovery.DiscoverAsync +/// (stamps on every browsed leaf), the +/// universal browser's commit path (writes the browsed leaf into TagConfig), and the +/// AdminUI typed tag editor (shows the inferred type and offers an override). +/// +/// +/// MTConnect is weakly typed on the wire — every observation is text, and the standard +/// defines hundreds of type values with more added each revision. This is therefore a +/// rule, not an enumeration, and the result is stored per tag and author-overridable. +/// The rule is asymmetric on purpose: a wrong numeric guess produces a value that +/// fails to parse and surfaces as Bad quality at runtime, whereas +/// always round-trips and the author can retype it. So every default leans to +/// String except where the standard itself guarantees a number. +/// +/// +/// Defaulting rule, per category (design §3.3): +/// +/// +/// +/// +/// SAMPLE ⇒ , including for an +/// unrecognised or missing type. The standard defines the whole SAMPLE +/// category as a continuously-varying measured quantity, so the category alone is +/// the guarantee — a missing units attribute is a gap in the device model, +/// not evidence that the observation is text. Float64 rather than an integer +/// type because a double parse accepts both 3 and 3.5. +/// +/// +/// +/// +/// EVENT ⇒ by default, with a small +/// exception list () of standard types that are +/// defined as integers. The overwhelming majority of EVENT types are controlled +/// vocabulary (Execution, ControllerMode, Availability) or free +/// text (Program, Block, Message), and the exception list is +/// deliberately short — each entry is a coercion risk, while every omission is +/// merely a string the author can retype. +/// +/// +/// +/// +/// CONDITION ⇒ always. The observation +/// is a state word (Normal / Warning / Fault / +/// Unavailable), never a number, regardless of the item's type. +/// +/// +/// +/// +/// An unknown, blank, or missing category ⇒ . +/// Probe XML in the wild is inconsistent; with no category we have no evidence the +/// observation is numeric, so we pick the type that cannot fail to parse. +/// +/// +/// +/// +/// Representation overrides the category where the two disagree about shape: +/// DATA_SET / TABLE observations are key-value maps, so they demote to +/// even on a SAMPLE; TIME_SERIES is +/// defined only for SAMPLE and is ignored (not treated as an array) elsewhere; +/// DISCRETE and VALUE are scalar and change nothing. +/// +/// +/// Matching is case-insensitive and whitespace-trimmed on every input, because agents vary +/// in how faithfully they echo the standard's spellings. +/// +/// +public static class MTConnectDataTypeInference +{ + private const string CategorySample = "SAMPLE"; + private const string CategoryEvent = "EVENT"; + + private const string RepresentationTimeSeries = "TIME_SERIES"; + private const string RepresentationDataSet = "DATA_SET"; + private const string RepresentationTable = "TABLE"; + + /// + /// The EVENT types the standard defines as integers — the exception list to the + /// "EVENT is text" default. Kept deliberately short: adding a type here is a claim that + /// every agent reports it as a parseable integer, and being wrong costs Bad quality. + /// Line is the deprecated spelling that LineNumber superseded in MTConnect 1.4; + /// both are mapped so a current-version agent is not silently mistyped. + /// + private static readonly FrozenSet NumericEventTypes = + new[] { "PartCount", "Line", "LineNumber" }.ToFrozenSet(StringComparer.OrdinalIgnoreCase); + + /// + /// Infers the OPC UA-facing type shape of an MTConnect DataItem from its device-model + /// metadata. See the type-level remarks for the full rule and its rationale. + /// + /// + /// The DataItem category — SAMPLE, EVENT, or CONDITION. + /// Case-insensitive; an unknown, blank, or null value yields + /// . + /// + /// + /// The DataItem type (e.g. Position, PartCount, Execution). + /// Case-insensitive; an unrecognised or null value falls back to the category default. + /// + /// + /// The declared engineering units, if any. Accepted for signature stability and because every + /// call site already holds it, but not load-bearing in v1: SAMPLE is numeric by + /// definition of the category, so units add no discriminating power there, and treating a + /// units-bearing EVENT as numeric would be exactly the risky coercion this rule avoids. + /// + /// + /// The DataItem representation — VALUE (default), TIME_SERIES, + /// DATA_SET, TABLE, or DISCRETE. Case-insensitive. + /// + /// + /// The probe-declared sampleCount for a TIME_SERIES item. Flows into + /// when positive; null or a non-positive + /// value yields a variable-length array (ArrayDim = null). Ignored for scalars. + /// + /// The inferred scalar type plus array shape. + public static MTConnectInferredType Infer( + string? category, + string? type = null, + string? units = null, + string? representation = null, + int? sampleCount = null) + { + _ = units; // See the parameter doc: intentionally not load-bearing in v1. + + var trimmedCategory = category.AsSpan().Trim(); + var trimmedRepresentation = representation.AsSpan().Trim(); + + // A key-value observation is never a number, whatever the category claims. Checked first so + // a DATA_SET SAMPLE cannot slip through as Float64 and go Bad on every parse. + if (Matches(trimmedRepresentation, RepresentationDataSet) || Matches(trimmedRepresentation, RepresentationTable)) + return new MTConnectInferredType(DriverDataType.String, IsArray: false, ArrayDim: null); + + if (Matches(trimmedCategory, CategorySample)) + { + // TIME_SERIES is defined for SAMPLE only; the vector is a vector of the same scalar type. + if (Matches(trimmedRepresentation, RepresentationTimeSeries)) + return new MTConnectInferredType( + DriverDataType.Float64, + IsArray: true, + ArrayDim: sampleCount is > 0 ? (uint)sampleCount.Value : null); + + return new MTConnectInferredType(DriverDataType.Float64, IsArray: false, ArrayDim: null); + } + + if (Matches(trimmedCategory, CategoryEvent)) + { + var trimmedType = type?.Trim(); + var dataType = trimmedType is { Length: > 0 } && NumericEventTypes.Contains(trimmedType) + ? DriverDataType.Int64 + : DriverDataType.String; + + return new MTConnectInferredType(dataType, IsArray: false, ArrayDim: null); + } + + // CONDITION (a state word) and every unknown / blank / missing category. + return new MTConnectInferredType(DriverDataType.String, IsArray: false, ArrayDim: null); + } + + private static bool Matches(ReadOnlySpan value, string expected) + => value.Equals(expected, StringComparison.OrdinalIgnoreCase); +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs new file mode 100644 index 00000000..1a82d954 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs @@ -0,0 +1,250 @@ +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Golden table for (design §3.3). +/// Three call sites must agree byte-for-byte on this mapping — ITagDiscovery.DiscoverAsync, +/// the universal browser's commit path, and the AdminUI typed tag editor — so the table is +/// pinned here rather than re-derived at each seam. +/// +[Trait("Category", "Unit")] +public sealed class MTConnectDataTypeInferenceTests +{ + // --------------------------------------------------------------------- + // The design §3.3 golden table. + // --------------------------------------------------------------------- + + [Theory] + // SAMPLE numeric with units → Float64. + [InlineData("SAMPLE", "Position", "MILLIMETER", null, DriverDataType.Float64)] + [InlineData("SAMPLE", "SpindleSpeed", "REVOLUTION/MINUTE", "VALUE", DriverDataType.Float64)] + // EVENT numeric type → Int64. + [InlineData("EVENT", "PartCount", null, null, DriverDataType.Int64)] + [InlineData("EVENT", "Line", null, null, DriverDataType.Int64)] + // EVENT controlled vocabulary → String. + [InlineData("EVENT", "Execution", null, null, DriverDataType.String)] + [InlineData("EVENT", "ControllerMode", null, null, DriverDataType.String)] + [InlineData("EVENT", "Availability", null, null, DriverDataType.String)] + // EVENT free text → String. + [InlineData("EVENT", "Program", null, null, DriverDataType.String)] + [InlineData("EVENT", "Block", null, null, DriverDataType.String)] + [InlineData("EVENT", "Message", null, null, DriverDataType.String)] + // CONDITION → String (the state word). + [InlineData("CONDITION", "Temperature", null, null, DriverDataType.String)] + [InlineData("CONDITION", "SystemCondition", null, null, DriverDataType.String)] + public void Infer_maps_the_v1_table(string cat, string type, string? units, string? rep, DriverDataType expected) + => MTConnectDataTypeInference.Infer(cat, type, units, rep).DataType.ShouldBe(expected); + + [Fact] + public void TimeSeries_sample_is_a_float64_array_with_declared_count() + { + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "TIME_SERIES", sampleCount: 8); + + r.DataType.ShouldBe(DriverDataType.Float64); + r.IsArray.ShouldBeTrue(); + r.ArrayDim.ShouldBe(8u); + } + + // --------------------------------------------------------------------- + // Array shape: only a SAMPLE/TIME_SERIES is an array, and ArrayDim is + // only populated when the probe declared a usable sampleCount. + // --------------------------------------------------------------------- + + [Fact] + public void TimeSeries_without_a_declared_sample_count_is_a_variable_length_array() + { + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "TIME_SERIES"); + + r.IsArray.ShouldBeTrue(); + r.ArrayDim.ShouldBeNull(); + } + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void TimeSeries_with_a_non_positive_sample_count_reports_no_dimension(int declared) + { + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "TIME_SERIES", declared); + + r.IsArray.ShouldBeTrue(); + r.ArrayDim.ShouldBeNull(); + } + + [Theory] + [InlineData("SAMPLE", "Position", "VALUE")] + [InlineData("SAMPLE", "Position", null)] + [InlineData("EVENT", "PartCount", "VALUE")] + [InlineData("CONDITION", "Temperature", null)] + public void A_scalar_item_is_never_an_array(string cat, string type, string? rep) + { + var r = MTConnectDataTypeInference.Infer(cat, type, null, rep); + + r.IsArray.ShouldBeFalse(); + r.ArrayDim.ShouldBeNull(); + } + + [Fact] + public void A_declared_sample_count_is_ignored_when_the_item_is_not_a_time_series() + { + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "VALUE", sampleCount: 8); + + r.IsArray.ShouldBeFalse(); + r.ArrayDim.ShouldBeNull(); + } + + [Theory] + [InlineData("EVENT")] + [InlineData("CONDITION")] + public void TIME_SERIES_outside_the_SAMPLE_category_is_not_treated_as_an_array(string cat) + { + // TIME_SERIES is defined only for SAMPLE. Honouring it elsewhere would hand the + // address space an array node whose observations are scalars. + var r = MTConnectDataTypeInference.Infer(cat, "Anything", null, "TIME_SERIES", sampleCount: 8); + + r.IsArray.ShouldBeFalse(); + r.ArrayDim.ShouldBeNull(); + r.DataType.ShouldBe(DriverDataType.String); + } + + // --------------------------------------------------------------------- + // The defaulting rule for unrecognised `type` values — the part of the + // table that is a *rule*, not an enumeration. MTConnect defines hundreds + // of types; the fallback per category is what actually ships. + // --------------------------------------------------------------------- + + [Theory] + [InlineData("Xacceleration")] + [InlineData("SomeVendorExtension")] + [InlineData(null)] + [InlineData("")] + public void An_unrecognised_SAMPLE_type_still_defaults_to_Float64(string? type) + => MTConnectDataTypeInference.Infer("SAMPLE", type, null, null).DataType.ShouldBe(DriverDataType.Float64); + + [Fact] + public void A_SAMPLE_without_units_is_still_numeric() + // SAMPLE is numeric by definition in the standard; a missing `units` attribute is a + // gap in the device model, not evidence that the observation is text. + => MTConnectDataTypeInference.Infer("SAMPLE", "Position", null, null).DataType + .ShouldBe(DriverDataType.Float64); + + [Theory] + [InlineData("ToolNumber")] + [InlineData("PalletId")] + [InlineData("LineLabel")] + [InlineData("SomeVendorExtension")] + [InlineData(null)] + [InlineData("")] + public void An_unrecognised_EVENT_type_defaults_to_String(string? type) + // Fail-safe direction: a wrong numeric coercion produces a Bad-quality value at runtime, + // whereas String always round-trips and the author can override. + => MTConnectDataTypeInference.Infer("EVENT", type, null, null).DataType.ShouldBe(DriverDataType.String); + + [Theory] + [InlineData("LineNumber")] + public void The_modern_spelling_of_a_known_numeric_EVENT_is_also_numeric(string type) + // `Line` was superseded by `LineNumber`; mapping only the deprecated spelling would + // silently mistype the same data item on any current-version agent. + => MTConnectDataTypeInference.Infer("EVENT", type, null, null).DataType.ShouldBe(DriverDataType.Int64); + + [Theory] + [InlineData("Anything")] + [InlineData(null)] + public void Every_CONDITION_is_a_String_regardless_of_type(string? type) + => MTConnectDataTypeInference.Infer("CONDITION", type, "CELSIUS", null).DataType + .ShouldBe(DriverDataType.String); + + // --------------------------------------------------------------------- + // Unknown / missing category — probe XML in the wild is inconsistent. + // --------------------------------------------------------------------- + + [Theory] + [InlineData(null)] + [InlineData("")] + [InlineData(" ")] + [InlineData("ASSET")] + [InlineData("nonsense")] + public void An_unknown_or_missing_category_falls_back_to_String(string? cat) + // We cannot know the observation is numeric, so we choose the type that always + // round-trips rather than one that can fail to parse. + => MTConnectDataTypeInference.Infer(cat, "Position", "MILLIMETER", null).DataType + .ShouldBe(DriverDataType.String); + + // --------------------------------------------------------------------- + // Case / whitespace tolerance. + // --------------------------------------------------------------------- + + [Theory] + [InlineData("sample", DriverDataType.Float64)] + [InlineData("Sample", DriverDataType.Float64)] + [InlineData(" SAMPLE ", DriverDataType.Float64)] + [InlineData("condition", DriverDataType.String)] + public void Category_matching_is_case_and_whitespace_insensitive(string cat, DriverDataType expected) + => MTConnectDataTypeInference.Infer(cat, "Position", "MM", null).DataType.ShouldBe(expected); + + [Theory] + [InlineData("partcount")] + [InlineData("PARTCOUNT")] + [InlineData(" PartCount ")] + public void Numeric_EVENT_type_matching_is_case_and_whitespace_insensitive(string type) + => MTConnectDataTypeInference.Infer("EVENT", type, null, null).DataType.ShouldBe(DriverDataType.Int64); + + [Theory] + [InlineData("time_series")] + [InlineData("Time_Series")] + [InlineData(" TIME_SERIES ")] + public void Representation_matching_is_case_and_whitespace_insensitive(string rep) + => MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", rep, 4).IsArray.ShouldBeTrue(); + + // --------------------------------------------------------------------- + // Non-scalar representations. A DATA_SET / TABLE observation is a + // key-value map, not a number — typing it Float64 guarantees Bad quality. + // --------------------------------------------------------------------- + + [Theory] + [InlineData("DATA_SET")] + [InlineData("TABLE")] + [InlineData("data_set")] + public void A_key_value_representation_is_a_String_even_on_a_SAMPLE(string rep) + { + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MILLIMETER", rep); + + r.DataType.ShouldBe(DriverDataType.String); + r.IsArray.ShouldBeFalse(); + } + + [Fact] + public void A_key_value_representation_also_demotes_a_numeric_EVENT() + { + var r = MTConnectDataTypeInference.Infer("EVENT", "PartCount", null, "DATA_SET"); + + r.DataType.ShouldBe(DriverDataType.String); + r.IsArray.ShouldBeFalse(); + } + + [Fact] + public void DISCRETE_is_a_scalar_representation_and_does_not_change_the_type() + { + // DISCRETE changes *when* the agent reports, not what the value is. + var r = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MILLIMETER", "DISCRETE"); + + r.DataType.ShouldBe(DriverDataType.Float64); + r.IsArray.ShouldBeFalse(); + } + + // --------------------------------------------------------------------- + // Purity — the three call sites rely on identical results for identical + // inputs with no shared state between calls. + // --------------------------------------------------------------------- + + [Fact] + public void Infer_is_deterministic() + { + var first = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "TIME_SERIES", 8); + var second = MTConnectDataTypeInference.Infer("SAMPLE", "Position", "MM", "TIME_SERIES", 8); + + second.ShouldBe(first); + } +} -- 2.52.0 From 3ca8ddf6f734040df5975700a097f771804e7544 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:53:15 -0400 Subject: [PATCH 06/34] test(mtconnect): canned probe/current/sample XML fixtures incl. forced-gap (Task 4) --- .../Fixtures/current-unavailable.xml | 40 ++++++++++++++++ .../Fixtures/current.xml | 41 +++++++++++++++++ .../Fixtures/probe.xml | 46 +++++++++++++++++++ .../Fixtures/sample-gap.xml | 39 ++++++++++++++++ .../Fixtures/sample.xml | 32 +++++++++++++ 5 files changed, 198 insertions(+) create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current-unavailable.xml create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current.xml create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/probe.xml create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample-gap.xml create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample.xml diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current-unavailable.xml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current-unavailable.xml new file mode 100644 index 00000000..0d25f8df --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current-unavailable.xml @@ -0,0 +1,40 @@ + + + +
+ + + + + UNAVAILABLE + + + + + UNAVAILABLE + UNAVAILABLE + + + UNAVAILABLE + UNAVAILABLE + UNAVAILABLE + + + + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current.xml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current.xml new file mode 100644 index 00000000..7b095386 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/current.xml @@ -0,0 +1,41 @@ + + +
+ + + + + AVAILABLE + + + + + + 123.4567 + 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 + + + + UNAVAILABLE + ACTIVE + O1234 + + + + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/probe.xml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/probe.xml new file mode 100644 index 00000000..dcf97716 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/probe.xml @@ -0,0 +1,46 @@ + + +
+ + + Test fixture device for MTConnect driver unit tests + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample-gap.xml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample-gap.xml new file mode 100644 index 00000000..bc167cc7 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample-gap.xml @@ -0,0 +1,39 @@ + + + +
+ + + + + 200.0000 + 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4.0 + + + 99 + ACTIVE + O5678 + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample.xml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample.xml new file mode 100644 index 00000000..395687b6 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/Fixtures/sample.xml @@ -0,0 +1,32 @@ + + + +
+ + + + + 124.0100 + 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 3.0 + + + 42 + ACTIVE + O1234 + + + + + -- 2.52.0 From 79e30d1ead9ead7065bb5e646c940a2a95be1f54 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:57:13 -0400 Subject: [PATCH 07/34] fix(mtconnect): infer numeric EVENT types from the probe's UPPER_SNAKE spelling (Task 3 follow-up) --- .../MTConnectDataTypeInference.cs | 88 +++++++++++++++++-- .../MTConnectDataTypeInferenceTests.cs | 61 +++++++++++++ 2 files changed, 142 insertions(+), 7 deletions(-) diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs index cbf4312b..4bddb20c 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDataTypeInference.cs @@ -92,8 +92,17 @@ public readonly record struct MTConnectInferredType(DriverDataType DataType, boo /// DISCRETE and VALUE are scalar and change nothing. /// /// -/// Matching is case-insensitive and whitespace-trimmed on every input, because agents vary -/// in how faithfully they echo the standard's spellings. +/// Matching is case-insensitive AND separator-insensitive on every input — +/// _, - and whitespace are ignored wherever they appear, so +/// PART_COUNT ≡ PartCount ≡ part-count ≡ partcount. This is not +/// mere leniency: MTConnect spells the same concept two different ways in two different +/// documents. The Devices document (/probe) writes DataItem@type in +/// UPPER_SNAKE (PART_COUNT, POSITION, PATH_FEEDRATE), while the Streams +/// documents (/current, /sample) name the observation element in PascalCase +/// (PartCount, Position). Discovery reads the probe spelling, and plain +/// case-insensitivity does not bridge the underscore — so matching only the PascalCase +/// spelling types every real agent's PART_COUNT as +/// while every unit test using the sketch's PascalCase spelling stays green. /// /// public static class MTConnectDataTypeInference @@ -111,9 +120,15 @@ public static class MTConnectDataTypeInference /// every agent reports it as a parseable integer, and being wrong costs Bad quality. /// Line is the deprecated spelling that LineNumber superseded in MTConnect 1.4; /// both are mapped so a current-version agent is not silently mistyped. + /// + /// The entries are written in the Streams (PascalCase) spelling, but the set's comparer + /// is , so the probe document's + /// PART_COUNT / LINE_NUMBER hit the same entries. Adding a spelling variant + /// here would be redundant, not additional coverage. + /// /// private static readonly FrozenSet NumericEventTypes = - new[] { "PartCount", "Line", "LineNumber" }.ToFrozenSet(StringComparer.OrdinalIgnoreCase); + new[] { "PartCount", "Line", "LineNumber" }.ToFrozenSet(SeparatorInsensitiveComparer.Instance); /// /// Infers the OPC UA-facing type shape of an MTConnect DataItem from its device-model @@ -125,8 +140,10 @@ public static class MTConnectDataTypeInference /// . /// /// - /// The DataItem type (e.g. Position, PartCount, Execution). - /// Case-insensitive; an unrecognised or null value falls back to the category default. + /// The DataItem type, in either document's spelling — the probe's + /// PART_COUNT / POSITION or the streams' PartCount / Position. + /// Matched case- and separator-insensitively (see the type-level remarks); an unrecognised + /// or null value falls back to the category default. /// /// /// The declared engineering units, if any. Accepted for signature stability and because every @@ -175,8 +192,9 @@ public static class MTConnectDataTypeInference if (Matches(trimmedCategory, CategoryEvent)) { - var trimmedType = type?.Trim(); - var dataType = trimmedType is { Length: > 0 } && NumericEventTypes.Contains(trimmedType) + // No Trim() here: the set's comparer already ignores whitespace and separators + // wherever they appear, so the raw attribute value is looked up allocation-free. + var dataType = type is { Length: > 0 } && NumericEventTypes.Contains(type) ? DriverDataType.Int64 : DriverDataType.String; @@ -189,4 +207,60 @@ public static class MTConnectDataTypeInference private static bool Matches(ReadOnlySpan value, string expected) => value.Equals(expected, StringComparison.OrdinalIgnoreCase); + + /// + /// Ordinal-ignore-case equality that additionally skips _, - and whitespace + /// wherever they occur, so the probe document's PART_COUNT and the streams + /// document's PartCount are one key. Backs . + /// + /// + /// Comparing through a comparer rather than normalising the input keeps the lookup + /// allocation-free on the discovery hot path: no string.Replace, no + /// Trim(), no temporary buffer — the raw attribute value from the parser is passed + /// straight to . + /// + private sealed class SeparatorInsensitiveComparer : IEqualityComparer + { + /// The single shared instance; the comparer is stateless. + internal static readonly SeparatorInsensitiveComparer Instance = new(); + + private SeparatorInsensitiveComparer() { } + + /// + public bool Equals(string? x, string? y) + { + if (ReferenceEquals(x, y)) return true; + if (x is null || y is null) return false; + + int i = 0, j = 0; + while (true) + { + while (i < x.Length && IsSeparator(x[i])) i++; + while (j < y.Length && IsSeparator(y[j])) j++; + + if (i == x.Length || j == y.Length) return i == x.Length && j == y.Length; + if (char.ToUpperInvariant(x[i]) != char.ToUpperInvariant(y[j])) return false; + + i++; + j++; + } + } + + /// + public int GetHashCode(string obj) + { + // Must hash exactly the characters Equals compares, or two equal spellings land in + // different buckets and the set silently misses. + var hash = new HashCode(); + foreach (var c in obj) + { + if (IsSeparator(c)) continue; + hash.Add(char.ToUpperInvariant(c)); + } + + return hash.ToHashCode(); + } + + private static bool IsSeparator(char c) => c is '_' or '-' || char.IsWhiteSpace(c); + } } diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs index 1a82d954..487778a2 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDataTypeInferenceTests.cs @@ -38,6 +38,67 @@ public sealed class MTConnectDataTypeInferenceTests public void Infer_maps_the_v1_table(string cat, string type, string? units, string? rep, DriverDataType expected) => MTConnectDataTypeInference.Infer(cat, type, units, rep).DataType.ShouldBe(expected); + // --------------------------------------------------------------------- + // The SAME golden table, spelled the way a real agent's /probe document + // spells it. MTConnect writes the same concept two ways: the Devices + // (/probe) document's `DataItem@type` is UPPER_SNAKE (PART_COUNT), while + // the Streams (/current, /sample) document's observation element name is + // PascalCase (PartCount). `DiscoverAsync` feeds the *probe* spelling, so + // an inference that only knows the PascalCase spelling is green in tests + // and wrong live. Every row below is a DataItem from Fixtures/probe.xml. + // --------------------------------------------------------------------- + + [Theory] + [InlineData("EVENT", "PART_COUNT", null, DriverDataType.Int64)] // dev1_partcount + [InlineData("SAMPLE", "POSITION", "MILLIMETER", DriverDataType.Float64)] // dev1_pos + [InlineData("EVENT", "EXECUTION", null, DriverDataType.String)] // dev1_execution + [InlineData("EVENT", "PROGRAM", null, DriverDataType.String)] // dev1_program + [InlineData("EVENT", "AVAILABILITY", null, DriverDataType.String)] // dev1_avail + [InlineData("CONDITION", "SYSTEM", null, DriverDataType.String)] // dev1_system_cond + public void Infer_maps_the_probe_documents_UPPER_SNAKE_spelling( + string cat, string type, string? units, DriverDataType expected) + => MTConnectDataTypeInference.Infer(cat, type, units, null).DataType.ShouldBe(expected); + + [Fact] + public void The_probes_TIME_SERIES_item_is_a_float64_array_of_its_declared_sample_count() + { + // dev1_vibration_ts: category=SAMPLE type=PATH_FEEDRATE representation=TIME_SERIES sampleCount=10. + var r = MTConnectDataTypeInference.Infer( + "SAMPLE", "PATH_FEEDRATE", "MILLIMETER/SECOND", "TIME_SERIES", sampleCount: 10); + + r.DataType.ShouldBe(DriverDataType.Float64); + r.IsArray.ShouldBeTrue(); + r.ArrayDim.ShouldBe(10u); + } + + [Theory] + [InlineData("PART_COUNT", "PartCount")] + [InlineData("LINE_NUMBER", "LineNumber")] + [InlineData("part_count", "partcount")] + [InlineData("PART-COUNT", "PartCount")] + [InlineData("PART_COUNT", " Part Count ")] + public void The_two_document_spellings_of_one_type_infer_identically(string probeSpelling, string streamsSpelling) + { + // The equivalence is the whole point: a tag discovered from /probe and the same tag + // named from a /current observation must land on one type, or the authored config + // stops matching the value that arrives. + var fromProbe = MTConnectDataTypeInference.Infer("EVENT", probeSpelling, null, null); + var fromStreams = MTConnectDataTypeInference.Infer("EVENT", streamsSpelling, null, null); + + fromProbe.ShouldBe(fromStreams); + fromProbe.DataType.ShouldBe(DriverDataType.Int64); + } + + [Theory] + [InlineData("PART_COUNTER")] + [InlineData("PARTS_COUNT")] + [InlineData("LINE_NUMBERS")] + [InlineData("_")] + public void Separator_insensitivity_does_not_widen_the_numeric_EVENT_exception_list(string type) + // Falsifiability control: ignoring separators must not turn a *different* type into a + // match. The exception list stays exactly PartCount / Line / LineNumber. + => MTConnectDataTypeInference.Infer("EVENT", type, null, null).DataType.ShouldBe(DriverDataType.String); + [Fact] public void TimeSeries_sample_is_a_float64_array_with_declared_count() { -- 2.52.0 From b13cf5c6e8537972a59404739d764ecbeaa660ff Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 13:57:58 -0400 Subject: [PATCH 08/34] fix(mtconnect): options carry the authored tag definitions (Task 2 follow-up) --- .../MTConnectDriverOptions.cs | 15 +++++++ .../MTConnectDriverOptionsTests.cs | 41 +++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs index 8b9a9031..0b1dce3b 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs @@ -54,6 +54,21 @@ public sealed class MTConnectDriverOptions /// public int HeartbeatMs { get; init; } = 10000; + /// + /// The authored tags this driver instance serves — one + /// per tag, keyed by the DataItem id it binds + /// (). The factory config DTO carries these and + /// builds them into the options; the driver indexes them to know each tag's target + /// when coercing an Agent observation + /// (whose wire form is always text) into a published value. + /// + /// Defaults to empty, never null: a driver instance may legitimately be + /// authored with no tags yet, and a null collection here would surface as a + /// NullReferenceException at deploy time from operator-authored config. + /// + /// + public IReadOnlyList Tags { get; init; } = []; + /// /// Background connectivity-probe settings. When /// is true the driver periodically issues a cheap /probe request and raises diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs index 0b77db18..91d1c4ca 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs @@ -56,6 +56,47 @@ public sealed class MTConnectDriverOptionsTests reconnect.BackoffMultiplier.ShouldBeGreaterThan(1.0); } + /// + /// defaults to an empty collection, never + /// null — a null collection here is an operator-authorable NullReferenceException at + /// deploy, since a driver instance authored with no tags binds this default. + /// + [Fact] + public void Options_tags_default_to_empty_and_are_never_null() + { + var o = new MTConnectDriverOptions { AgentUri = "http://agent:5000" }; + + o.Tags.ShouldNotBeNull(); + o.Tags.ShouldBeEmpty(); + } + + /// + /// A populated options object carries the authored tag definitions through unchanged — + /// the observation index (Task 8) coerces each present observation to its tag's + /// DriverDataType, so the definitions must reach the driver from the factory config. + /// + [Fact] + public void Options_round_trip_the_authored_tag_definitions() + { + var pos = new MTConnectTagDefinition( + FullName: "dev1_pos", + DriverDataType: ZB.MOM.WW.OtOpcUa.Core.Abstractions.DriverDataType.Float64, + MtCategory: "SAMPLE", + MtType: "POSITION", + Units: "MILLIMETER"); + var partCount = new MTConnectTagDefinition( + FullName: "dev1_partcount", + DriverDataType: ZB.MOM.WW.OtOpcUa.Core.Abstractions.DriverDataType.Int64, + MtCategory: "EVENT", + MtType: "PART_COUNT"); + + var o = new MTConnectDriverOptions { AgentUri = "http://agent:5000", Tags = [pos, partCount] }; + + o.Tags.Count.ShouldBe(2); + o.Tags[0].ShouldBe(pos); + o.Tags[1].ShouldBe(partCount); + } + /// MTConnectTagDefinition round-trips its positional fields, including MTConnect metadata. [Fact] public void TagDefinition_carries_dataItemId_and_mtconnect_metadata() -- 2.52.0 From 1990d2173304382499be247efc3d5bd1b5638e42 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 14:08:54 -0400 Subject: [PATCH 09/34] feat(mtconnect): agent client /probe parse to device model (Task 6) --- .../MTConnectAgentClient.cs | 142 ++++++ .../MTConnectProbeParser.cs | 254 ++++++++++ .../MTConnectProbeParseTests.cs | 474 ++++++++++++++++++ 3 files changed, 870 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs new file mode 100644 index 00000000..77c42de6 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs @@ -0,0 +1,142 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// The production — a thin wrapper +/// over an MTConnect Agent's REST surface that hands every response body to a pure parser and +/// returns only the neutral DTOs in MTConnectDtos.cs. +/// +/// +/// +/// The constructor opens no connection. It only composes the request URIs and builds +/// an (which itself dials nothing until a request is issued), so a +/// throwaway instance is safe to construct — the universal discovery browser's +/// CanBrowse pattern depends on that. +/// +/// +/// Every call is deadline-bounded. Each request runs under a +/// linked to the caller's token and cancelled after +/// , and +/// is set to the same bound. A frozen Agent surfaces as a +/// rather than wedging the caller — the lesson from this +/// repo's S7 read-leg wall-clock gap (arch-review R2-01). A non-positive timeout is rejected +/// at construction so an operator-authored 0 can never come to mean "wait forever" +/// (arch-review 01/S-6). +/// +/// +/// This class does not swallow failures into an empty model: an unreachable Agent, a non-2xx +/// status, or an unparseable body all throw. InitializeAsync (Task 9) is what catches +/// that and moves the driver to Faulted. +/// +/// +public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable +{ + private readonly HttpClient _http; + private readonly Uri _probeUri; + private readonly TimeSpan _requestTimeout; + + /// Creates a client for the Agent described by . + /// The driver options carrying the Agent URI, device scope, and per-call deadline. + /// is missing or is not an absolute HTTP(S) URI. + /// is not positive. + public MTConnectAgentClient(MTConnectDriverOptions options) + : this(options, handler: null) + { + } + + /// + /// Test seam: as , but sends through + /// the supplied handler so the request/response round trip can be exercised with no socket. + /// The caller retains ownership of . + /// + internal MTConnectAgentClient(MTConnectDriverOptions options, HttpMessageHandler? handler) + { + ArgumentNullException.ThrowIfNull(options); + + if (options.RequestTimeoutMs <= 0) + { + throw new ArgumentOutOfRangeException( + nameof(options), + options.RequestTimeoutMs, + $"{nameof(MTConnectDriverOptions.RequestTimeoutMs)} must be positive; a non-positive per-call deadline would let a frozen Agent wedge the driver indefinitely."); + } + + _requestTimeout = TimeSpan.FromMilliseconds(options.RequestTimeoutMs); + _probeUri = BuildRequestUri(options, "probe"); + + _http = handler is null ? new HttpClient() : new HttpClient(handler, disposeHandler: false); + _http.Timeout = _requestTimeout; + } + + /// + public async Task ProbeAsync(CancellationToken ct) + { + // ResponseContentRead (the default) buffers the whole body *under* this awaited, token-bound + // call, so the parse below reads from memory — no network read can outlive the deadline. + using var response = await SendAsync(_probeUri, ct).ConfigureAwait(false); + + await using var body = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false); + + return MTConnectProbeParser.Parse(body); + } + + /// + public Task CurrentAsync(CancellationToken ct) => + // Task 7 adds the /current leg (MTConnectStreamsParser); deliberately not stubbed as an + // empty result, which would read as a working call that silently returns nothing. + throw new NotImplementedException("The MTConnect /current leg lands in Task 7."); + + /// + public IAsyncEnumerable SampleAsync(long from, CancellationToken ct) => + // Task 7 adds the /sample long-poll leg (multipart chunk framing + sequence-gap detection). + throw new NotImplementedException("The MTConnect /sample leg lands in Task 7."); + + /// + public void Dispose() => _http.Dispose(); + + private async Task SendAsync(Uri uri, CancellationToken ct) + { + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + deadline.CancelAfter(_requestTimeout); + + HttpResponseMessage response; + try + { + response = await _http.GetAsync(uri, deadline.Token).ConfigureAwait(false); + } + catch (OperationCanceledException) when (!ct.IsCancellationRequested) + { + // The caller did not cancel, so this is our own deadline (or HttpClient.Timeout) firing. + // Reported as a TimeoutException because a bare "A task was canceled" tells an operator + // nothing about which Agent stopped answering. + throw new TimeoutException( + $"MTConnect Agent request to '{uri}' did not complete within {_requestTimeout.TotalMilliseconds:F0} ms."); + } + + response.EnsureSuccessStatusCode(); + + return response; + } + + private static Uri BuildRequestUri(MTConnectDriverOptions options, string request) + { + var agentUri = options.AgentUri?.Trim(); + if (string.IsNullOrEmpty(agentUri)) + { + throw new ArgumentException( + $"{nameof(MTConnectDriverOptions.AgentUri)} is required (e.g. 'http://agent:5000').", nameof(options)); + } + + if (!Uri.TryCreate(agentUri, UriKind.Absolute, out var parsed) || + (parsed.Scheme != Uri.UriSchemeHttp && parsed.Scheme != Uri.UriSchemeHttps)) + { + throw new ArgumentException( + $"{nameof(MTConnectDriverOptions.AgentUri)} '{agentUri}' is not an absolute http(s) URI.", nameof(options)); + } + + var root = parsed.GetLeftPart(UriPartial.Path).TrimEnd('/'); + var deviceName = options.DeviceName?.Trim(); + var deviceSegment = string.IsNullOrEmpty(deviceName) ? string.Empty : $"/{Uri.EscapeDataString(deviceName)}"; + + return new Uri($"{root}{deviceSegment}/{request}", UriKind.Absolute); + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs new file mode 100644 index 00000000..6250d3bf --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs @@ -0,0 +1,254 @@ +using System.Globalization; +using System.Xml; +using System.Xml.Linq; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// Turns an MTConnect Agent /probe response (an MTConnectDevices XML document) +/// into the neutral . Deliberately a pure, socket-free static +/// so the whole parse is unit-testable against canned fixtures. +/// +/// +/// +/// Why hand-rolled rather than TrakHound. The referenced TrakHound packages +/// (MTConnect.NET-Common + -HTTP 6.9.0.2) contain no +/// IResponseDocumentFormatter implementation — the XML formatter ships in the +/// separate, unreferenced MTConnect.NET-XML package — so +/// ResponseDocumentFormatter.CreateDevicesResponseDocument("xml", stream) answers +/// Success = false / "Document Formatter Not found for "xml"" (verified by +/// reflection + live invocation against this repo's own fixture, 2026-07-24). TrakHound also +/// offers no socket-free public parse entry point, which the plan requires. +/// +/// +/// Three shapes of the wire format this parser is built around. +/// (1) A device may declare data items directly on <Device><DataItems>, +/// outside any component — a walker that only recurses <Components> silently +/// drops them. +/// (2) The children of <Components> are named for the component type +/// (<Controller>, <Path>, <Axes>, …) — there is no +/// literal <Component> element — so every element child of +/// <Components> is a component. +/// (3) The document is XML-namespaced and the namespace URI carries the MTConnect version +/// (urn:mtconnect.org:MTConnectDevices:1.3 … :2.0). Elements are therefore +/// matched on and the namespace is ignored entirely — nothing +/// version-specific is hard-coded, and a vendor-extension component in its own namespace +/// (whose standard child elements still inherit the default MTConnect namespace) is not +/// silently dropped. Attributes, by contrast, are read unqualified so a prefixed attribute +/// (e.g. xsi:type) can never be mistaken for a DataItem's type. +/// +/// +/// Failure posture. Anything that is not a well-formed MTConnect device model throws +/// . It never degrades to an empty-but-successful model: +/// an empty device model that looks successful is precisely the defect class that has bitten +/// this codebase before (#485), and here it would tear the driver's browse tree down to +/// nothing while reporting healthy. +/// +/// +internal static class MTConnectProbeParser +{ + /// + /// Bound on component nesting depth. Real device models nest a handful of levels; this only + /// exists so a pathological or hostile document cannot recurse the parser into a + /// process-fatal (uncatchable) stack overflow. + /// + private const int MaxComponentDepth = 64; + + /// Parses a /probe response held as a string. + /// The raw MTConnectDevices XML document. + /// + /// The payload is empty, not well-formed XML, not an MTConnectDevices document (an + /// Agent MTConnectError document included — Agents answer a bad device path with one + /// under HTTP 200), or declares no usable device. + /// + public static MTConnectProbeModel Parse(string xml) + { + if (string.IsNullOrWhiteSpace(xml)) + { + throw new InvalidDataException("MTConnect /probe response was empty; expected an MTConnectDevices XML document."); + } + + XDocument document; + try + { + document = XDocument.Parse(xml); + } + catch (XmlException ex) + { + throw new InvalidDataException($"MTConnect /probe response is not well-formed XML: {ex.Message}", ex); + } + + return Build(document); + } + + /// Parses a /probe response read from a stream. + /// A stream positioned at the start of the XML document. + /// As for . + public static MTConnectProbeModel Parse(Stream stream) + { + ArgumentNullException.ThrowIfNull(stream); + + XDocument document; + try + { + document = XDocument.Load(stream); + } + catch (XmlException ex) + { + throw new InvalidDataException($"MTConnect /probe response is not well-formed XML: {ex.Message}", ex); + } + + return Build(document); + } + + private static MTConnectProbeModel Build(XDocument document) + { + var root = document.Root + ?? throw new InvalidDataException("MTConnect /probe response has no root element."); + + if (!IsNamed(root, "MTConnectDevices")) + { + throw new InvalidDataException(DescribeUnexpectedRoot(root)); + } + + var devicesContainer = ChildrenNamed(root, "Devices").FirstOrDefault() + ?? throw new InvalidDataException( + "MTConnect /probe response has no element; it does not describe a device model."); + + var devices = devicesContainer.Elements().Select(ReadDevice).ToList(); + if (devices.Count == 0) + { + throw new InvalidDataException( + "MTConnect /probe response declares no devices; refusing to report an empty device model as a successful probe."); + } + + return new MTConnectProbeModel(devices); + } + + private static MTConnectDevice ReadDevice(XElement element) => + new( + RequiredAttribute(element, "id", "Device"), + OptionalAttribute(element, "name"), + ReadComponents(element, depth: 1), + ReadDataItems(element)); + + private static MTConnectComponent ReadComponent(XElement element, int depth) => + new( + RequiredAttribute(element, "id", $"Component <{element.Name.LocalName}>"), + OptionalAttribute(element, "name"), + ReadComponents(element, depth + 1), + ReadDataItems(element)); + + /// + /// Every element child of this container's <Components> element, whatever it is + /// named — the element name is the component type, not the literal string "Component". + /// + private static IReadOnlyList ReadComponents(XElement container, int depth) + { + if (depth > MaxComponentDepth) + { + throw new InvalidDataException( + $"MTConnect /probe response nests components more than {MaxComponentDepth} levels deep; refusing to recurse further."); + } + + return ChildrenNamed(container, "Components") + .SelectMany(components => components.Elements()) + .Select(element => ReadComponent(element, depth)) + .ToList(); + } + + private static IReadOnlyList ReadDataItems(XElement container) => + ChildrenNamed(container, "DataItems") + .SelectMany(dataItems => ChildrenNamed(dataItems, "DataItem")) + .Select(ReadDataItem) + .ToList(); + + private static MTConnectDataItem ReadDataItem(XElement element) + { + var id = RequiredAttribute(element, "id", "DataItem"); + + return new MTConnectDataItem( + id, + OptionalAttribute(element, "name"), + RequiredAttribute(element, "category", $"DataItem '{id}'"), + RequiredAttribute(element, "type", $"DataItem '{id}'"), + OptionalAttribute(element, "subType"), + OptionalAttribute(element, "units"), + OptionalAttribute(element, "representation"), + OptionalIntAttribute(element, "sampleCount", id)); + } + + private static bool IsNamed(XElement element, string localName) => + string.Equals(element.Name.LocalName, localName, StringComparison.Ordinal); + + private static IEnumerable ChildrenNamed(XElement parent, string localName) => + parent.Elements().Where(child => IsNamed(child, localName)); + + private static string RequiredAttribute(XElement element, string name, string owner) + { + var value = OptionalAttribute(element, name); + + return value ?? throw new InvalidDataException( + $"MTConnect /probe response is malformed: {owner} has no '{name}' attribute."); + } + + /// + /// Reads an unqualified attribute, normalizing a present-but-empty value to null. + /// Only unqualified attributes are considered, so a namespace-prefixed attribute such as + /// xsi:type can never be read as a DataItem's type. + /// + private static string? OptionalAttribute(XElement element, string name) + { + var value = element.Attribute(name)?.Value; + + return string.IsNullOrWhiteSpace(value) ? null : value; + } + + private static int? OptionalIntAttribute(XElement element, string name, string dataItemId) + { + var raw = OptionalAttribute(element, name); + if (raw is null) + { + return null; + } + + if (!int.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value)) + { + throw new InvalidDataException( + $"MTConnect /probe response is malformed: DataItem '{dataItemId}' has a non-numeric '{name}' attribute ('{raw}')."); + } + + return value; + } + + /// + /// Builds the message for a root element that is not MTConnectDevices, lifting the + /// Agent's own error text when the response is an MTConnectError document (which + /// Agents return under HTTP 200 for, e.g., an unknown device name). + /// + private static string DescribeUnexpectedRoot(XElement root) + { + var message = + $"MTConnect /probe response root element is <{root.Name.LocalName}>, expected ."; + + if (!IsNamed(root, "MTConnectError")) + { + return message; + } + + var errors = root.Descendants() + .Where(e => IsNamed(e, "Error")) + .Select(e => + { + var code = OptionalAttribute(e, "errorCode"); + var text = e.Value.Trim(); + return code is null ? text : $"{code}: {text}"; + }) + .Where(text => text.Length > 0) + .ToList(); + + return errors.Count == 0 + ? message + : $"{message} The Agent reported: {string.Join("; ", errors)}"; + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs new file mode 100644 index 00000000..019758f0 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs @@ -0,0 +1,474 @@ +using System.Diagnostics; +using System.Net; +using System.Text; +using Shouldly; +using Xunit; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 6 — /probe parse into the neutral , plus the +/// leg that feeds it (URL composition + the +/// per-op deadline), all exercised with no socket. +/// +public sealed class MTConnectProbeParseTests +{ + private const string ProbeFixturePath = "Fixtures/probe.xml"; + + /// Every DataItem id the committed fixture declares — all seven, at every depth. + private static readonly string[] AllFixtureDataItemIds = + [ + "dev1_avail", // sits directly on , NOT under a component + "dev1_pos", + "dev1_vibration_ts", + "dev1_partcount", + "dev1_execution", + "dev1_program", + "dev1_system_cond" // CONDITION + ]; + + // ---------------------------------------------------------------- parse: the committed fixture + + [Fact] + public async Task Probe_parse_yields_nested_components_and_all_dataitems() + { + var xml = await File.ReadAllTextAsync(ProbeFixturePath, TestContext.Current.CancellationToken); + + var model = MTConnectProbeParser.Parse(xml); + + model.Devices.ShouldHaveSingleItem(); + var ids = model.Devices[0].AllDataItems().Select(d => d.Id).ToList(); + ids.ShouldContain("dev1_system_cond"); + model.Devices[0].Components.ShouldNotBeEmpty(); // nesting preserved + } + + [Fact] + public async Task Probe_parse_finds_every_dataitem_including_the_device_level_one() + { + var model = MTConnectProbeParser.Parse(await ReadFixtureAsync()); + + var ids = model.Devices[0].AllDataItems().Select(d => d.Id).ToList(); + + // All seven — a walker that only recurses silently drops dev1_avail. + ids.ShouldBe(AllFixtureDataItemIds, ignoreOrder: true); + model.Devices[0].DataItems.Select(d => d.Id).ShouldBe(["dev1_avail"]); + } + + [Fact] + public async Task Probe_parse_reads_the_device_identity() + { + var device = MTConnectProbeParser.Parse(await ReadFixtureAsync()).Devices[0]; + + device.Id.ShouldBe("dev1"); + device.Name.ShouldBe("VMC-3Axis"); + } + + [Fact] + public async Task Probe_parse_preserves_the_component_nesting_chain() + { + var device = MTConnectProbeParser.Parse(await ReadFixtureAsync()).Devices[0]; + + // Device -> Controller -> Path. The element names are the component TYPES (there is no + // literal element in MTConnect Devices XML). + var controller = device.Components.ShouldHaveSingleItem(); + controller.Id.ShouldBe("dev1_controller"); + controller.Name.ShouldBe("controller"); + controller.DataItems.ShouldBeEmpty(); + + var path = controller.Components.ShouldHaveSingleItem(); + path.Id.ShouldBe("dev1_path"); + path.Name.ShouldBe("path"); + path.Components.ShouldBeEmpty(); + path.DataItems.Select(d => d.Id).ShouldBe( + ["dev1_pos", "dev1_vibration_ts", "dev1_partcount", "dev1_execution", "dev1_program", "dev1_system_cond"]); + } + + [Fact] + public async Task Probe_parse_round_trips_every_dataitem_attribute() + { + var items = MTConnectProbeParser.Parse(await ReadFixtureAsync()) + .Devices[0].AllDataItems().ToDictionary(d => d.Id); + + var pos = items["dev1_pos"]; + pos.Category.ShouldBe("SAMPLE"); + pos.Type.ShouldBe("POSITION"); + pos.SubType.ShouldBe("ACTUAL"); + pos.Units.ShouldBe("MILLIMETER"); + pos.Representation.ShouldBeNull(); + pos.SampleCount.ShouldBeNull(); + + var timeSeries = items["dev1_vibration_ts"]; + timeSeries.Category.ShouldBe("SAMPLE"); + timeSeries.Type.ShouldBe("PATH_FEEDRATE"); + timeSeries.Representation.ShouldBe("TIME_SERIES"); + timeSeries.SampleCount.ShouldBe(10); + timeSeries.Units.ShouldBe("MILLIMETER/SECOND"); + + var condition = items["dev1_system_cond"]; + condition.Category.ShouldBe("CONDITION"); + condition.Type.ShouldBe("SYSTEM"); + + var availability = items["dev1_avail"]; + availability.Category.ShouldBe("EVENT"); + availability.Type.ShouldBe("AVAILABILITY"); + availability.Name.ShouldBeNull(); + availability.SubType.ShouldBeNull(); + availability.Units.ShouldBeNull(); + availability.Representation.ShouldBeNull(); + availability.SampleCount.ShouldBeNull(); + } + + [Fact] + public async Task Probe_parse_from_a_stream_matches_the_string_overload() + { + await using var stream = File.OpenRead(ProbeFixturePath); + + var fromStream = MTConnectProbeParser.Parse(stream); + + // Records compare their IReadOnlyList members by reference, so compare the flattened + // content rather than the graphs themselves. + var fromString = MTConnectProbeParser.Parse(await ReadFixtureAsync()); + fromStream.Devices.Select(d => d.Id).ShouldBe(fromString.Devices.Select(d => d.Id)); + fromStream.Devices[0].AllDataItems().ShouldBe(fromString.Devices[0].AllDataItems()); + fromStream.Devices[0].Components[0].Components[0].Id.ShouldBe("dev1_path"); + } + + // ------------------------------------------------------- parse: not shaped to the 1.3 fixture + + [Theory] + [InlineData("urn:mtconnect.org:MTConnectDevices:1.3")] + [InlineData("urn:mtconnect.org:MTConnectDevices:1.5")] + [InlineData("urn:mtconnect.org:MTConnectDevices:2.0")] + [InlineData("")] // no namespace at all + public void Probe_parse_is_agnostic_to_the_document_namespace_version(string namespaceUri) + { + var xmlns = namespaceUri.Length == 0 ? string.Empty : $" xmlns=\"{namespaceUri}\""; + var xml = $""" + + +
+ + + + + + + """; + + var model = MTConnectProbeParser.Parse(xml); + + model.Devices.ShouldHaveSingleItem().AllDataItems().Select(d => d.Id).ShouldBe(["d_avail"]); + } + + [Fact] + public void Probe_parse_recurses_components_to_arbitrary_depth() + { + // Four levels deep, and each level carries its own data item — deeper than the fixture, and + // with component element names the parser has never seen. + const string Xml = """ + + + + + + + + + + + + + + + + + + + + + + + """; + + var device = MTConnectProbeParser.Parse(Xml).Devices[0]; + + device.AllDataItems().Select(d => d.Id).ShouldBe(["l0", "l1", "l2", "l3"]); + device.Components[0].Components[0].Components[0].Id.ShouldBe("m"); + } + + [Fact] + public void Probe_parse_reads_every_device_of_a_multi_device_agent() + { + const string Xml = """ + + + + + + + + + + + """; + + var model = MTConnectProbeParser.Parse(Xml); + + model.Devices.Select(d => d.Id).ShouldBe(["agent", "d1"]); + } + + // ------------------------------------------------------------------ parse: fails loudly, never + // a silently-empty model + + [Theory] + [InlineData("")] + [InlineData(" ")] + [InlineData("not xml at all")] + [InlineData("")] // truncated mid-document + public void Probe_parse_throws_on_malformed_xml(string xml) + { + Should.Throw(() => MTConnectProbeParser.Parse(xml)); + } + + [Fact] + public void Probe_parse_throws_on_a_non_mtconnect_document() + { + var ex = Should.Throw( + () => MTConnectProbeParser.Parse("404 Not Found")); + + ex.Message.ShouldContain("MTConnectDevices"); + ex.Message.ShouldContain("html"); + } + + [Fact] + public void Probe_parse_throws_on_an_mtconnect_error_document_and_surfaces_the_agent_error() + { + // A real Agent answers an unknown device path with an MTConnectError document under HTTP 200. + const string Xml = """ + +
+ + Could not find the device 'nope' + + + """; + + var ex = Should.Throw(() => MTConnectProbeParser.Parse(Xml)); + + ex.Message.ShouldContain("NO_DEVICE"); + ex.Message.ShouldContain("Could not find the device 'nope'"); + } + + [Theory] + [InlineData("
")] // no + [InlineData("
")] // empty + public void Probe_parse_throws_when_the_document_declares_no_devices(string xml) + { + Should.Throw(() => MTConnectProbeParser.Parse(xml)); + } + + [Theory] + // DataItem missing its required id / category / type, and a garbage sampleCount. + [InlineData("")] + [InlineData("")] + [InlineData("")] + [InlineData("")] + public void Probe_parse_throws_on_a_malformed_dataitem(string dataItemXml) + { + var xml = $""" + + {dataItemXml} + + """; + + Should.Throw(() => MTConnectProbeParser.Parse(xml)); + } + + [Fact] + public void Probe_parse_throws_when_a_device_has_no_id() + { + const string Xml = """ + + + + """; + + Should.Throw(() => MTConnectProbeParser.Parse(Xml)); + } + + [Fact] + public void Probe_parse_throws_when_a_component_has_no_id() + { + const string Xml = """ + + + + """; + + Should.Throw(() => MTConnectProbeParser.Parse(Xml)); + } + + // ---------------------------------------------------------------- MTConnectAgentClient.ProbeAsync + + [Fact] + public async Task ProbeAsync_requests_the_agent_probe_path_and_returns_the_parsed_model() + { + var handler = StubHandler.RespondingWith(await ReadFixtureAsync()); + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler); + + var model = await client.ProbeAsync(TestContext.Current.CancellationToken); + + handler.LastRequestUri!.AbsoluteUri.ShouldBe("http://agent:5000/probe"); + model.Devices.ShouldHaveSingleItem().Id.ShouldBe("dev1"); + } + + [Theory] + [InlineData("http://agent:5000", null, "http://agent:5000/probe")] + [InlineData("http://agent:5000/", null, "http://agent:5000/probe")] + [InlineData("http://agent:5000", "VMC-3Axis", "http://agent:5000/VMC-3Axis/probe")] + [InlineData("http://agent:5000/", "VMC 3Axis", "http://agent:5000/VMC%203Axis/probe")] + public async Task ProbeAsync_scopes_the_request_to_the_configured_device_name( + string agentUri, string? deviceName, string expectedUri) + { + var handler = StubHandler.RespondingWith(await ReadFixtureAsync()); + using var client = new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = agentUri, DeviceName = deviceName }, handler); + + await client.ProbeAsync(TestContext.Current.CancellationToken); + + handler.LastRequestUri!.AbsoluteUri.ShouldBe(expectedUri); + } + + [Fact] + public async Task ProbeAsync_bounds_a_hung_agent_by_the_configured_request_timeout() + { + // The R2-01 frozen-peer lesson: a wedged agent must surface as a cancelled call, not a hang. + var handler = StubHandler.Hanging(); + using var client = new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000", RequestTimeoutMs = 50 }, handler); + + var started = Stopwatch.StartNew(); + var ex = await Should.ThrowAsync( + async () => await client.ProbeAsync(TestContext.Current.CancellationToken)); + + // A deadline hit must NOT be reported as a plain cancellation — the caller never cancelled. + ex.ShouldNotBeAssignableTo(); + ex.Message.ShouldContain("http://agent:5000/probe"); + started.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(10)); + } + + [Fact] + public async Task ProbeAsync_honours_the_callers_cancellation_token() + { + var handler = StubHandler.Hanging(); + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler); + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + await Should.ThrowAsync(async () => await client.ProbeAsync(cts.Token)); + } + + [Fact] + public async Task ProbeAsync_throws_rather_than_returning_an_empty_model_when_the_agent_errors() + { + var handler = StubHandler.RespondingWith("nope", HttpStatusCode.ServiceUnavailable); + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler); + + await Should.ThrowAsync( + async () => await client.ProbeAsync(TestContext.Current.CancellationToken)); + } + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void Ctor_rejects_a_non_positive_request_timeout(int requestTimeoutMs) + { + // An operator-authorable 0 must never mean "wait forever" (arch-review 01/S-6). + Should.Throw(() => new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000", RequestTimeoutMs = requestTimeoutMs })); + } + + [Theory] + [InlineData("")] + [InlineData(" ")] + [InlineData("not a uri")] + public void Ctor_rejects_an_unusable_agent_uri(string agentUri) + { + Should.Throw(() => new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = agentUri })); + } + + [Fact] + public void Ctor_opens_no_connection() + { + // The universal browser's throwaway-instance CanBrowse pattern depends on this. + var handler = StubHandler.Hanging(); + + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler); + + handler.RequestCount.ShouldBe(0); + } + + // ------------------------------------------------------------------- scope boundary: Task 7 legs + + [Fact] + public async Task CurrentAsync_is_not_implemented_until_task_7() + { + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync( + async () => await client.CurrentAsync(TestContext.Current.CancellationToken)); + ex.Message.ShouldContain("Task 7"); + } + + [Fact] + public void SampleAsync_is_not_implemented_until_task_7() + { + using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = Should.Throw(() => client.SampleAsync(1, CancellationToken.None)); + ex.Message.ShouldContain("Task 7"); + } + + private static Task ReadFixtureAsync() => + File.ReadAllTextAsync(ProbeFixturePath, TestContext.Current.CancellationToken); + + /// A canned — no socket is ever opened. + private sealed class StubHandler : HttpMessageHandler + { + private readonly string? _body; + private readonly HttpStatusCode _status; + private readonly bool _hang; + + private StubHandler(string? body, HttpStatusCode status, bool hang) + { + _body = body; + _status = status; + _hang = hang; + } + + public Uri? LastRequestUri { get; private set; } + + public int RequestCount { get; private set; } + + public static StubHandler RespondingWith(string body, HttpStatusCode status = HttpStatusCode.OK) => + new(body, status, hang: false); + + public static StubHandler Hanging() => new(null, HttpStatusCode.OK, hang: true); + + protected override async Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) + { + LastRequestUri = request.RequestUri; + RequestCount++; + + if (_hang) + { + await Task.Delay(Timeout.Infinite, cancellationToken); + } + + return new HttpResponseMessage(_status) + { + Content = new StringContent(_body ?? string.Empty, Encoding.UTF8, "text/xml") + }; + } + } +} -- 2.52.0 From d3aaa4a4110c7379c430ee779e6ef21b589bb8fc Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 14:11:40 -0400 Subject: [PATCH 10/34] =?UTF-8?q?docs(mtconnect):=20correct=20the=20Task?= =?UTF-8?q?=200=20decision=20=E2=80=94=20TrakHound=20cannot=20parse=20XML?= =?UTF-8?q?=20as=20pinned?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/plans/2026-07-24-mtconnect-driver.md | 30 +++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/plans/2026-07-24-mtconnect-driver.md b/docs/plans/2026-07-24-mtconnect-driver.md index 87aa9ea6..2a1d20db 100644 --- a/docs/plans/2026-07-24-mtconnect-driver.md +++ b/docs/plans/2026-07-24-mtconnect-driver.md @@ -69,6 +69,36 @@ absence of a physical `LICENSE` file is not a fail. Task 1 adds `PackageReference MTConnect.NET-Common 6.9.0.2` + `PackageReference MTConnect.NET-HTTP 6.9.0.2` to `Driver.MTConnect.csproj`; the hand-roll fallback is not needed. +**CORRECTION (Task 6, commit `bdfefadc`) — the decision above rested on incomplete evidence, and the +hand-roll fallback IS in use for parsing.** Task 0 verified the packages *exist, restore, and are MIT*; +it never verified they can actually parse an MTConnect document. As pinned, they cannot. Task 6 proved +this empirically before writing any parser code: reflection over both referenced assemblies +(`MTConnect.NET-Common`, 1297 types; `-HTTP`, 410 types) found **zero `IResponseDocumentFormatter` +implementations and zero public static `FromXml`/parse entry points**, and a live invocation of +TrakHound's own entry point (`ResponseDocumentFormatter.CreateDevicesResponseDocument("xml", stream)`) +against this repo's `Fixtures/probe.xml`, with both DLLs loaded, returned: + +``` +Success = False +ERROR: Document Formatter Not found for "xml" +``` + +TrakHound 6.x resolves formatters from loaded assemblies at runtime, and the XML formatter ships in a +**third package, `MTConnect.NET-XML`**, which Task 0 did not evaluate. Even with it added, TrakHound +exposes no socket-free `Parse(string)` entry point of the shape this plan mandates. So `/probe` parsing +is a hand-rolled `System.Xml.Linq` walk (~250 LoC, `MTConnectProbeParser`); the driver above the +`IMTConnectAgentClient` seam is unaffected, exactly as the decision rule anticipated. + +**The two `PackageReference`s remain and are, so far, dead weight.** Task 6 deliberately did not remove +them: the hard remaining problem is Task 7's `multipart/x-mixed-replace` chunk framing for `/sample`, +and `MTConnect.Clients.MTConnectHttpClientStream` in `-HTTP` may be usable for framing alone. **Task 7 +owns the final call:** (a) use `-HTTP` for framing only, (b) add `MTConnect.NET-XML` and reconsider +wholesale, or (c) hand-roll the boundary reader and **drop both package references** — in which case the +Task-0 rationale comment in `Directory.Packages.props` goes with them. Record the outcome here. + +**Process lesson:** "the dependency restores" is not "the dependency does the job." A library +verification checklist needs one end-to-end call against a real input, not just a restore. + --- ## Task 1: Scaffold the two driver projects + the test project -- 2.52.0 From cf43f8d0ab7899fdb4e0512b20ae5471ce2ea118 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 14:32:36 -0400 Subject: [PATCH 11/34] feat(mtconnect): current/sample parse + sequence-gap detection (Task 7) Adds the /current and /sample legs to MTConnectAgentClient: - MTConnectStreamsParser: hand-rolled System.Xml.Linq parse of an MTConnectStreams document into MTConnectStreamsResult. Observations are every element child of //, keyed by dataItemId (the element NAME is the data item's type in PascalCase, so a fixed name set would drop observations). A CONDITION's value is its element name, not its text - is empty - and normalizes onto the same UNAVAILABLE sentinel a Sample/Event carries as text, so Task 8's no-comms mapping sees one token. Timestamps are normalized to DateTimeKind.Utc explicitly. - IsSequenceGap(requestedFrom, chunk) => chunk.FirstSequence > requestedFrom, so Task 11's pump can re-baseline after a ring-buffer overflow. Equality is NOT a gap. - SampleAsync: ResponseHeadersRead + a hand-rolled multipart/x-mixed-replace frame reader (Content-length fast path, boundary-scan fallback), bounded by a heartbeat watchdog derived from HeartbeatMs so a silent Agent faults instead of wedging the pump - the S7 frozen-peer shape (arch-review R2-01). The long poll gets its own HttpClient with an infinite Timeout so the unary /probe + /current deadline stays bounded. Task 0 package decision, option (c): both TrakHound PackageReferences and their PackageVersions are dropped. MTConnectHttpClientStream takes a URL and dials its own socket (no handler/stream seam, so it cannot serve a socket-free unit suite), and it emits already-parsed documents through the formatter lookup Task 6 proved is missing - framing-only reuse was never actually on offer. The driver now depends on nothing but the BCL. Recorded in the plan's Task 0 CORRECTION block. Also folds in two Task 6 review carry-overs: the stale IMTConnectAgentClient doc comment claiming a TrakHound-backed implementation, and a probe test constructing a mixed-namespace vendor extension ( with default-namespace children) that backs the parser's ignore-the-namespace claim with a test. 187/187 green. --- Directory.Packages.props | 10 +- docs/plans/2026-07-24-mtconnect-driver.md | 47 +- .../IMTConnectAgentClient.cs | 10 +- .../MTConnectAgentClient.cs | 517 ++++++++++- .../MTConnectProbeParser.cs | 4 +- .../MTConnectStreamsParser.cs | 296 +++++++ .../ZB.MOM.WW.OtOpcUa.Driver.MTConnect.csproj | 13 +- .../MTConnectProbeParseTests.cs | 64 +- .../MTConnectStreamsParseTests.cs | 838 ++++++++++++++++++ 9 files changed, 1740 insertions(+), 59 deletions(-) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs diff --git a/Directory.Packages.props b/Directory.Packages.props index 3b622ce4..c9ad0acf 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -84,11 +84,11 @@ ships net6.0/net8.0 only. --> - - - + - - - + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs index 019758f0..1d285d5c 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectProbeParseTests.cs @@ -196,6 +196,49 @@ public sealed class MTConnectProbeParseTests device.Components[0].Components[0].Components[0].Id.ShouldBe("m"); } + [Fact] + public void Probe_parse_keeps_a_vendor_extension_component_in_its_own_namespace() + { + // The shape the parser's ignore-the-namespace rule exists for, and the reason attributes are + // read unqualified: the vendor component carries its OWN namespace prefix, while its + // standard / children inherit the DEFAULT MTConnect namespace. Matching + // components on a fully-qualified XName would silently drop and every data item + // beneath it, leaving the browse tree short with no error anywhere. + const string Xml = """ + + + + + + + + + + + + + + + + + + + + + + """; + + var device = MTConnectProbeParser.Parse(Xml).Devices[0]; + + var pump = device.Components.ShouldHaveSingleItem(); + pump.Id.ShouldBe("pump1"); + pump.Name.ShouldBe("pump"); + pump.Components.ShouldHaveSingleItem().Id.ShouldBe("imp1"); + device.AllDataItems().Select(d => d.Id).ShouldBe(["d_avail", "pump_pressure", "imp_rpm"]); + device.AllDataItems().Single(d => d.Id == "pump_pressure").Units.ShouldBe("PASCAL"); + } + [Fact] public void Probe_parse_reads_every_device_of_a_multi_device_agent() { @@ -408,27 +451,6 @@ public sealed class MTConnectProbeParseTests handler.RequestCount.ShouldBe(0); } - // ------------------------------------------------------------------- scope boundary: Task 7 legs - - [Fact] - public async Task CurrentAsync_is_not_implemented_until_task_7() - { - using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); - - var ex = await Should.ThrowAsync( - async () => await client.CurrentAsync(TestContext.Current.CancellationToken)); - ex.Message.ShouldContain("Task 7"); - } - - [Fact] - public void SampleAsync_is_not_implemented_until_task_7() - { - using var client = new MTConnectAgentClient(new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); - - var ex = Should.Throw(() => client.SampleAsync(1, CancellationToken.None)); - ex.Message.ShouldContain("Task 7"); - } - private static Task ReadFixtureAsync() => File.ReadAllTextAsync(ProbeFixturePath, TestContext.Current.CancellationToken); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs new file mode 100644 index 00000000..089aa83d --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs @@ -0,0 +1,838 @@ +using System.Diagnostics; +using System.Net; +using System.Net.Http.Headers; +using System.Text; +using Shouldly; +using Xunit; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 7 — the /current + /sample legs: +/// turning an MTConnectStreams document into , the +/// ring-buffer sequence-gap signal (), and the +/// multipart/x-mixed-replace chunk framing + heartbeat watchdog on the long-poll leg. +/// Every test runs against canned fixtures or a stubbed — no +/// socket is opened anywhere in this file. +/// +public sealed class MTConnectStreamsParseTests +{ + private const string CurrentFixture = "Fixtures/current.xml"; + private const string SampleFixture = "Fixtures/sample.xml"; + private const string GapFixture = "Fixtures/sample-gap.xml"; + private const string UnavailableFixture = "Fixtures/current-unavailable.xml"; + + /// The fixtures deliberately share one instanceId, so a gap — not a restart — is under test. + private const long FixtureInstanceId = 1655000000L; + + private static readonly string[] AllFixtureDataItemIds = + [ + "dev1_avail", + "dev1_pos", + "dev1_vibration_ts", + "dev1_partcount", + "dev1_execution", + "dev1_program", + "dev1_system_cond" + ]; + + // ------------------------------------------------------------------ header sequence parsing + + [Fact] + public void Current_parse_reads_header_sequences_and_observations() + { + var result = MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)); + + result.NextSequence.ShouldBeGreaterThan(0); + result.Observations.ShouldNotBeEmpty(); + } + + [Theory] + [InlineData(CurrentFixture, 1L, 108L)] + [InlineData(SampleFixture, 108L, 113L)] + [InlineData(GapFixture, 5000L, 5005L)] + [InlineData(UnavailableFixture, 200L, 207L)] + public void Parse_reads_first_and_next_sequence_and_the_instance_id_from_every_fixture( + string fixture, long firstSequence, long nextSequence) + { + var result = MTConnectStreamsParser.Parse(File.ReadAllText(fixture)); + + result.FirstSequence.ShouldBe(firstSequence); + result.NextSequence.ShouldBe(nextSequence); + result.InstanceId.ShouldBe(FixtureInstanceId); + } + + // ------------------------------------------------------------------------ observation values + + [Fact] + public void Current_parse_keys_every_observation_by_its_dataitem_id() + { + var result = MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)); + + result.Observations.Select(o => o.DataItemId).ShouldBe(AllFixtureDataItemIds, ignoreOrder: true); + } + + [Fact] + public void Current_parse_reads_the_good_value_the_agent_reported() + { + var observations = ParseCurrent(); + + // The observation ELEMENT is , not ; the value is its text content. + observations["dev1_pos"].Value.ShouldBe("123.4567"); + observations["dev1_execution"].Value.ShouldBe("ACTIVE"); + observations["dev1_program"].Value.ShouldBe("O1234"); + observations["dev1_avail"].Value.ShouldBe("AVAILABLE"); + } + + [Fact] + public void Current_parse_carries_the_unavailable_text_sentinel_through_verbatim() + { + // Task 8 maps UNAVAILABLE => BadNoCommunication; this layer must not pre-interpret it. + ParseCurrent()["dev1_partcount"].Value.ShouldBe("UNAVAILABLE"); + } + + [Fact] + public void Current_parse_keeps_a_time_series_vector_as_its_raw_string() + { + ParseCurrent()["dev1_vibration_ts"].Value.ShouldBe("1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0"); + } + + // ---------------------------------------------------------------------- CONDITION observations + + [Fact] + public void A_condition_observations_value_is_its_element_name_not_its_empty_text() + { + // — the element name IS the state, and the + // element is empty, so a text-content read yields "" and loses the observation entirely. + ParseCurrent()["dev1_system_cond"].Value.ShouldBe("Normal"); + } + + [Theory] + [InlineData("Normal", "Normal")] + [InlineData("Warning", "Warning")] + [InlineData("Fault", "Fault")] + public void A_condition_state_element_name_becomes_the_value(string element, string expected) + { + var result = MTConnectStreamsParser.Parse(StreamsDocument( + $"""<{element} dataItemId="c" timestamp="2026-07-24T12:00:00Z" sequence="1" type="SYSTEM"/>""")); + + result.Observations.ShouldHaveSingleItem().Value.ShouldBe(expected); + } + + [Fact] + public void A_condition_with_text_content_still_takes_its_value_from_the_element_name() + { + // Agents put the operator-facing condition message in the element text; the STATE is the name. + var result = MTConnectStreamsParser.Parse(StreamsDocument( + """Spindle overtemperature""")); + + result.Observations.ShouldHaveSingleItem().Value.ShouldBe("Fault"); + } + + [Fact] + public void An_unavailable_condition_element_is_the_unavailable_sentinel() + { + // is genuinely no-comms and must land on the exact same sentinel Task 8 + // matches for a Sample/Event's literal UNAVAILABLE text. + var observations = ParseFixture(UnavailableFixture); + + observations["dev1_system_cond"].Value.ShouldBe("UNAVAILABLE"); + } + + [Fact] + public void The_all_unavailable_fixture_is_all_unavailable() + { + var result = MTConnectStreamsParser.Parse(File.ReadAllText(UnavailableFixture)); + + result.Observations.Select(o => o.DataItemId).ShouldBe(AllFixtureDataItemIds, ignoreOrder: true); + result.Observations.ShouldAllBe(o => o.Value == "UNAVAILABLE"); + } + + // ------------------------------------------------------------------------------- timestamps + + [Fact] + public void Parse_normalizes_every_timestamp_to_utc() + { + // A wrong Kind silently shifts the OPC UA SourceTimestamp by the machine's UTC offset. + var result = MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)); + + result.Observations.ShouldAllBe(o => o.TimestampUtc.Kind == DateTimeKind.Utc); + ParseCurrent()["dev1_pos"].TimestampUtc + .ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, 200, DateTimeKind.Utc)); + } + + [Fact] + public void An_offset_timestamp_is_converted_to_utc_rather_than_taken_at_face_value() + { + var result = MTConnectStreamsParser.Parse(StreamsDocument( + """O1""")); + + var observation = result.Observations.ShouldHaveSingleItem(); + observation.TimestampUtc.Kind.ShouldBe(DateTimeKind.Utc); + observation.TimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc)); + } + + // -------------------------------------------------------------- sequence-gap detection (both ways) + + [Fact] + public void Sequence_gap_is_detected_when_firstSequence_exceeds_requested_from() + { + var chunk = MTConnectStreamsParser.Parse(File.ReadAllText(GapFixture)); + + MTConnectAgentClient.IsSequenceGap(requestedFrom: 1, chunk).ShouldBeTrue(); + } + + [Fact] + public void No_sequence_gap_is_reported_for_a_contiguous_chunk() + { + // The falsifiability leg: a helper hard-wired to true would pass the gap test alone. + var contiguous = MTConnectStreamsParser.Parse(File.ReadAllText(SampleFixture)); + + MTConnectAgentClient.IsSequenceGap(requestedFrom: 108, contiguous).ShouldBeFalse(); + } + + [Theory] + [InlineData(4999L, true)] // requested older than the buffer holds — data was lost + [InlineData(5000L, false)] // exactly the oldest retained sequence — nothing was lost + [InlineData(5001L, false)] // the agent simply had nothing older to send + public void Sequence_gap_is_strictly_firstSequence_greater_than_requested_from(long requestedFrom, bool expected) + { + var chunk = MTConnectStreamsParser.Parse(File.ReadAllText(GapFixture)); + + MTConnectAgentClient.IsSequenceGap(requestedFrom, chunk).ShouldBe(expected); + } + + // --------------------------------------------------------------- legal but degenerate documents + + [Fact] + public void A_document_with_a_valid_header_and_no_observations_parses_rather_than_throwing() + { + // /sample chunks are deltas; an idle agent legitimately sends a document with nothing in it. + const string Xml = """ + +
+ + + """; + + var result = MTConnectStreamsParser.Parse(Xml); + + result.Observations.ShouldBeEmpty(); + result.InstanceId.ShouldBe(7); + result.NextSequence.ShouldBe(10); + result.FirstSequence.ShouldBe(10); + } + + [Fact] + public void A_component_stream_may_omit_any_of_samples_events_and_condition() + { + // sample.xml carries no at all; absence is normal, not an error. + var result = MTConnectStreamsParser.Parse(File.ReadAllText(SampleFixture)); + + result.Observations.Select(o => o.DataItemId).ShouldBe( + ["dev1_pos", "dev1_vibration_ts", "dev1_partcount", "dev1_execution", "dev1_program"], ignoreOrder: true); + } + + [Fact] + public void Parse_collects_observations_across_every_device_and_component_stream() + { + const string Xml = """ + +
+ + + + AVAILABLE + + + + + 1.0 + + + + + + """; + + var result = MTConnectStreamsParser.Parse(Xml); + + result.Observations.Select(o => o.DataItemId).ShouldBe(["a_avail", "b_pos", "b_cond"]); + result.Observations[2].Value.ShouldBe("Warning"); + } + + [Theory] + [InlineData("urn:mtconnect.org:MTConnectStreams:1.3")] + [InlineData("urn:mtconnect.org:MTConnectStreams:2.0")] + [InlineData("")] + public void Parse_is_agnostic_to_the_document_namespace_version(string namespaceUri) + { + var xmlns = namespaceUri.Length == 0 ? string.Empty : $" xmlns=\"{namespaceUri}\""; + var xml = $""" + +
+ + O1 + + + """; + + MTConnectStreamsParser.Parse(xml).Observations.ShouldHaveSingleItem().DataItemId.ShouldBe("p"); + } + + [Fact] + public void Parse_from_a_stream_matches_the_string_overload() + { + using var stream = File.OpenRead(CurrentFixture); + + var fromStream = MTConnectStreamsParser.Parse(stream); + + var fromString = MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)); + fromStream.NextSequence.ShouldBe(fromString.NextSequence); + fromStream.Observations.ShouldBe(fromString.Observations); + } + + // ------------------------------------------------------------- fails loudly, never silently empty + + [Theory] + [InlineData("")] + [InlineData(" ")] + [InlineData("not xml at all")] + [InlineData("")] + public void Parse_throws_on_malformed_xml(string xml) + { + Should.Throw(() => MTConnectStreamsParser.Parse(xml)); + } + + [Fact] + public void Parse_throws_on_a_devices_document_served_where_streams_were_expected() + { + var ex = Should.Throw( + () => MTConnectStreamsParser.Parse("")); + + ex.Message.ShouldContain("MTConnectStreams"); + ex.Message.ShouldContain("MTConnectDevices"); + } + + [Fact] + public void Parse_throws_on_an_mtconnect_error_document_and_surfaces_the_agent_error() + { + // An agent answers an out-of-range `from` with an MTConnectError document under HTTP 200. + const string Xml = """ + +
+ + 'from' must be greater than 100 + + + """; + + var ex = Should.Throw(() => MTConnectStreamsParser.Parse(Xml)); + + ex.Message.ShouldContain("OUT_OF_RANGE"); + ex.Message.ShouldContain("'from' must be greater than 100"); + } + + [Theory] + [InlineData("")] // no
at all + [InlineData("""
""")] // no nextSequence + [InlineData("""
""")] // no firstSequence + [InlineData("""
""")] // no instanceId + [InlineData("""
""")] // non-numeric + [InlineData("""
""")] // non-numeric + public void Parse_throws_on_an_unusable_header(string headerXml) + { + var xml = $"{headerXml}"; + + Should.Throw(() => MTConnectStreamsParser.Parse(xml)); + } + + [Theory] + [InlineData("""1""")] // no dataItemId + [InlineData("""1""")] // no timestamp + [InlineData("""1""")] // unparseable timestamp + public void Parse_throws_on_a_malformed_observation(string observationXml) + { + Should.Throw( + () => MTConnectStreamsParser.Parse(StreamsDocument($"{observationXml}"))); + } + + [Fact] + public void Parse_reads_only_unqualified_attributes() + { + // A prefixed x:dataItemId is a vendor extension, not the observation's identity — reading it + // would invent a data item that the probe never declared. + var xml = StreamsDocument( + """1"""); + + Should.Throw(() => MTConnectStreamsParser.Parse(xml)); + } + + // ------------------------------------------------------------- MTConnectAgentClient.CurrentAsync + + [Fact] + public async Task CurrentAsync_requests_the_agent_current_path_and_returns_the_parsed_snapshot() + { + var handler = StubHandler.RespondingWith(await File.ReadAllTextAsync(CurrentFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var result = await client.CurrentAsync(Ct); + + handler.LastRequestUri!.AbsoluteUri.ShouldBe("http://agent:5000/current"); + result.NextSequence.ShouldBe(108); + result.Observations.Count.ShouldBe(7); + } + + [Fact] + public async Task CurrentAsync_scopes_the_request_to_the_configured_device_name() + { + var handler = StubHandler.RespondingWith(await File.ReadAllTextAsync(CurrentFixture, Ct)); + using var client = NewClient( + handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000/", DeviceName = "VMC 3Axis" }); + + await client.CurrentAsync(Ct); + + handler.LastRequestUri!.AbsoluteUri.ShouldBe("http://agent:5000/VMC%203Axis/current"); + } + + [Fact] + public async Task CurrentAsync_bounds_a_hung_agent_by_the_configured_request_timeout() + { + using var client = NewClient( + StubHandler.Hanging(), + new MTConnectDriverOptions { AgentUri = "http://agent:5000", RequestTimeoutMs = 50 }); + + var started = Stopwatch.StartNew(); + var ex = await Should.ThrowAsync(async () => await client.CurrentAsync(Ct)); + + ex.ShouldNotBeAssignableTo(); + ex.Message.ShouldContain("http://agent:5000/current"); + started.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(10)); + } + + [Fact] + public async Task CurrentAsync_throws_on_an_error_document_served_under_http_200() + { + var handler = StubHandler.RespondingWith( + """ + nope + """); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync(async () => await client.CurrentAsync(Ct)); + + ex.Message.ShouldContain("NO_DEVICE"); + } + + [Fact] + public async Task CurrentAsync_throws_on_a_non_success_status() + { + var handler = StubHandler.RespondingWith("nope", HttpStatusCode.ServiceUnavailable); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + await Should.ThrowAsync(async () => await client.CurrentAsync(Ct)); + } + + // -------------------------------------------------------------- MTConnectAgentClient.SampleAsync + + [Fact] + public async Task SampleAsync_composes_the_from_interval_count_and_heartbeat_query() + { + var handler = StubHandler.RespondingWithMultipart("bnd", await File.ReadAllTextAsync(SampleFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions + { + AgentUri = "http://agent:5000", + DeviceName = "VMC-3Axis", + SampleIntervalMs = 250, + SampleCount = 64, + HeartbeatMs = 7000 + }); + + await DrainAsync(client.SampleAsync(108, Ct)); + + handler.LastRequestUri!.AbsoluteUri.ShouldBe( + "http://agent:5000/VMC-3Axis/sample?from=108&interval=250&count=64&heartbeat=7000"); + } + + [Fact] + public async Task SampleAsync_yields_one_result_per_multipart_chunk() + { + var handler = StubHandler.RespondingWithMultipart( + "--------bnd", + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.Count.ShouldBe(2); + chunks[0].FirstSequence.ShouldBe(108); + chunks[0].NextSequence.ShouldBe(113); + chunks[1].FirstSequence.ShouldBe(5000); + chunks[1].NextSequence.ShouldBe(5005); + } + + [Fact] + public async Task SampleAsync_frames_chunks_from_the_boundary_when_the_agent_omits_content_length() + { + var handler = StubHandler.RespondingWithMultipart( + "bnd", + withContentLength: false, + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.Select(c => c.FirstSequence).ShouldBe([108L, 5000L]); + } + + [Fact] + public async Task SampleAsync_exposes_the_gap_signal_to_the_caller_on_the_chunk_that_carries_it() + { + // Task 11's pump: advance `from` to NextSequence while contiguous, re-baseline on a gap. + var handler = StubHandler.RespondingWithMultipart( + "bnd", + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var from = 108L; + var gaps = new List(); + await foreach (var chunk in client.SampleAsync(from, Ct)) + { + gaps.Add(MTConnectAgentClient.IsSequenceGap(from, chunk)); + from = chunk.NextSequence; + } + + gaps.ShouldBe([false, true]); + from.ShouldBe(5005); + } + + [Fact] + public async Task SampleAsync_ignores_an_empty_keepalive_part() + { + var handler = StubHandler.RespondingWithMultipart( + "bnd", + await File.ReadAllTextAsync(SampleFixture, Ct), + "", + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.Count.ShouldBe(2); + } + + [Fact] + public async Task SampleAsync_yields_a_heartbeat_document_that_carries_no_observations() + { + // The agent's real keep-alive is a well-formed but observation-free MTConnectStreams + // document — it advances the sequence, so it must reach the caller, not be swallowed. + const string Heartbeat = """ + +
+ + + """; + var handler = StubHandler.RespondingWithMultipart("bnd", Heartbeat); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(113, Ct)); + + chunks.ShouldHaveSingleItem().Observations.ShouldBeEmpty(); + MTConnectAgentClient.IsSequenceGap(113, chunks[0]).ShouldBeFalse(); + } + + [Fact(Timeout = 30_000)] + public async Task SampleAsync_faults_on_the_heartbeat_watchdog_rather_than_hanging_on_a_silent_agent() + { + // The R2-01 frozen-peer shape: the agent accepts the connection, sends one chunk, then goes + // silent forever. Without a watchdog the pump wedges with no timeout ever firing. + var prefix = StubHandler.MultipartBody("bnd", withContentLength: true, closing: false, + await File.ReadAllTextAsync(SampleFixture, Ct)); + var handler = StubHandler.RespondingWithStallingStream("bnd", prefix); + using var client = NewClient(handler, new MTConnectDriverOptions + { + AgentUri = "http://agent:5000", + RequestTimeoutMs = 100, + HeartbeatMs = 100 + }); + + var started = Stopwatch.StartNew(); + await using var chunks = client.SampleAsync(108, Ct).GetAsyncEnumerator(Ct); + + (await chunks.MoveNextAsync()).ShouldBeTrue(); + chunks.Current.FirstSequence.ShouldBe(108); + + var ex = await Should.ThrowAsync(async () => await chunks.MoveNextAsync()); + ex.ShouldNotBeAssignableTo(); + ex.Message.ShouldContain("sample"); + started.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(20)); + } + + [Fact(Timeout = 30_000)] + public async Task SampleAsync_bounds_the_response_header_phase_by_the_request_timeout() + { + // A peer that accepts the TCP connection but never answers must not wedge the pump either. + using var client = NewClient( + StubHandler.Hanging(), + new MTConnectDriverOptions { AgentUri = "http://agent:5000", RequestTimeoutMs = 50 }); + + var ex = await Should.ThrowAsync(async () => await DrainAsync(client.SampleAsync(1, Ct))); + + ex.ShouldNotBeAssignableTo(); + ex.Message.ShouldContain("/sample"); + } + + [Fact] + public async Task SampleAsync_throws_on_an_error_document_served_under_http_200() + { + // Not a multipart body at all — the agent answers a bad `from` with an error document. + var handler = StubHandler.RespondingWith( + """ + bad from + """); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync( + async () => await DrainAsync(client.SampleAsync(1, Ct))); + + ex.Message.ShouldContain("OUT_OF_RANGE"); + } + + [Fact] + public async Task SampleAsync_throws_on_a_non_success_status() + { + var handler = StubHandler.RespondingWith("nope", HttpStatusCode.BadRequest); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + await Should.ThrowAsync(async () => await DrainAsync(client.SampleAsync(1, Ct))); + } + + [Fact(Timeout = 30_000)] + public async Task SampleAsync_honours_the_callers_cancellation_token() + { + var handler = StubHandler.RespondingWithStallingStream("bnd", []); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + await Should.ThrowAsync( + async () => await DrainAsync(client.SampleAsync(1, cts.Token))); + } + + [Fact] + public async Task SampleAsync_ends_cleanly_when_the_agent_closes_the_stream() + { + var handler = StubHandler.RespondingWithMultipart("bnd", await File.ReadAllTextAsync(SampleFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.ShouldHaveSingleItem(); + } + + [Fact] + public void SampleAsync_opens_no_connection_until_it_is_enumerated() + { + var handler = StubHandler.Hanging(); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + _ = client.SampleAsync(1, CancellationToken.None); + + handler.RequestCount.ShouldBe(0); + } + + // ------------------------------------------------------------------------------------ helpers + + private static CancellationToken Ct => TestContext.Current.CancellationToken; + + private static MTConnectAgentClient NewClient(HttpMessageHandler handler, MTConnectDriverOptions options) => + new(options, handler); + + private static async Task> DrainAsync(IAsyncEnumerable source) + { + var results = new List(); + await foreach (var item in source) + { + results.Add(item); + } + + return results; + } + + private static Dictionary ParseCurrent() => ParseFixture(CurrentFixture); + + private static Dictionary ParseFixture(string path) => + MTConnectStreamsParser.Parse(File.ReadAllText(path)).Observations.ToDictionary(o => o.DataItemId); + + /// Wraps observation-container XML in a minimal but valid streams document. + private static string StreamsDocument(string componentStreamBody) => + $""" + +
+ + + {componentStreamBody} + + + + """; + + /// A canned — no socket is ever opened. + private sealed class StubHandler : HttpMessageHandler + { + private readonly Func? _responder; + private readonly bool _hang; + + private StubHandler(Func? responder, bool hang) + { + _responder = responder; + _hang = hang; + } + + public Uri? LastRequestUri { get; private set; } + + public int RequestCount { get; private set; } + + public static StubHandler RespondingWith(string body, HttpStatusCode status = HttpStatusCode.OK) => + new(() => new HttpResponseMessage(status) + { + Content = new StringContent(body, Encoding.UTF8, "text/xml") + }, hang: false); + + public static StubHandler Hanging() => new(responder: null, hang: true); + + public static StubHandler RespondingWithMultipart(string boundary, params string[] parts) => + RespondingWithMultipart(boundary, withContentLength: true, parts); + + public static StubHandler RespondingWithMultipart(string boundary, bool withContentLength, params string[] parts) + { + var body = MultipartBody(boundary, withContentLength, closing: true, parts); + + return new StubHandler(() => Multipart(boundary, new ByteArrayContent(body)), hang: false); + } + + public static StubHandler RespondingWithStallingStream(string boundary, byte[] prefix) => + new(() => Multipart(boundary, new StallingContent(prefix)), hang: false); + + /// Builds a multipart/x-mixed-replace body exactly as an Agent writes one. + public static byte[] MultipartBody(string boundary, bool withContentLength, bool closing, params string[] parts) + { + var body = new List(); + foreach (var part in parts) + { + var payload = Encoding.UTF8.GetBytes(part); + var header = new StringBuilder() + .Append("--").Append(boundary).Append("\r\n") + .Append("Content-type: text/xml\r\n"); + if (withContentLength) + { + header.Append("Content-length: ").Append(payload.Length).Append("\r\n"); + } + + header.Append("\r\n"); + body.AddRange(Encoding.ASCII.GetBytes(header.ToString())); + body.AddRange(payload); + body.AddRange("\r\n"u8.ToArray()); + } + + if (closing) + { + body.AddRange(Encoding.ASCII.GetBytes($"--{boundary}--\r\n")); + } + + return [.. body]; + } + + private static HttpResponseMessage Multipart(string boundary, HttpContent content) + { + content.Headers.ContentType = + MediaTypeHeaderValue.Parse($"multipart/x-mixed-replace; boundary={boundary}"); + + return new HttpResponseMessage(HttpStatusCode.OK) { Content = content }; + } + + protected override async Task SendAsync( + HttpRequestMessage request, CancellationToken cancellationToken) + { + LastRequestUri = request.RequestUri; + RequestCount++; + + if (_hang) + { + await Task.Delay(Timeout.Infinite, cancellationToken); + } + + return _responder!(); + } + } + + /// Serves , then never completes another read. + private sealed class StallingContent(byte[] prefix) : HttpContent + { + protected override Task CreateContentReadStreamAsync() => + Task.FromResult(new StallingStream(prefix)); + + protected override Task SerializeToStreamAsync(Stream stream, TransportContext? context) => + Task.CompletedTask; + + protected override bool TryComputeLength(out long length) + { + length = -1; + + return false; + } + } + + private sealed class StallingStream(byte[] prefix) : Stream + { + private int _position; + + public override bool CanRead => true; + + public override bool CanSeek => false; + + public override bool CanWrite => false; + + public override long Length => throw new NotSupportedException(); + + public override long Position + { + get => throw new NotSupportedException(); + set => throw new NotSupportedException(); + } + + public override async ValueTask ReadAsync(Memory buffer, CancellationToken cancellationToken = default) + { + if (_position < prefix.Length) + { + var count = Math.Min(buffer.Length, prefix.Length - _position); + prefix.AsMemory(_position, count).CopyTo(buffer); + _position += count; + + return count; + } + + await Task.Delay(Timeout.Infinite, cancellationToken); + + return 0; + } + + public override Task ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken) => + ReadAsync(buffer.AsMemory(offset, count), cancellationToken).AsTask(); + + public override void Flush() + { + } + + public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + + public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); + + public override void SetLength(long value) => throw new NotSupportedException(); + + public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + } +} -- 2.52.0 From ac0a28405558f17acd9d1ed1e6e568ab57974909 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 14:50:43 -0400 Subject: [PATCH 12/34] feat(mtconnect): observation index + UNAVAILABLE->BadNoCommunication (Task 8) --- .../MTConnectObservationIndex.cs | 406 ++++++++++ .../MTConnectObservationIndexTests.cs | 706 ++++++++++++++++++ 2 files changed, 1112 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs new file mode 100644 index 00000000..29703f0f --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs @@ -0,0 +1,406 @@ +using System.Collections.Concurrent; +using System.Collections.Frozen; +using System.Globalization; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// The driver's live dataItemId → map: the single place +/// an Agent observation's weakly-typed text becomes an OPC UA-shaped value + quality + +/// timestamp. Written by the /current baseline read and by the /sample long-poll +/// pump; read by IReadable.ReadAsync and the subscription fan-out. +/// +/// +/// +/// Quality mapping (design §3.3). MTConnect has exactly one way to say "I have no +/// value": the literal token UNAVAILABLE. It arrives as a Sample/Event's text, and — +/// after normalizes the <Unavailable/> +/// element name — as a CONDITION's value on that same exact spelling. It maps to +/// BadNoCommunication with a null value: the Agent is reachable, the machine's +/// data item is not. The match is ordinal and case-sensitive against the one spelling +/// the parser guarantees; a case-insensitive match would swallow a legitimate free-text +/// EVENT whose value happens to read "Unavailable". +/// +/// +/// Failure posture. This type sits under IReadable and must never throw across +/// that boundary on account of observation content. Every unusable observation degrades to a +/// Bad-coded snapshot that still carries the Agent's source timestamp, so an operator can +/// always see when the driver last heard about the tag even when it cannot show +/// what. The only exceptions raised are for a null +/// argument — a programming error, not data. +/// +/// +/// Status codes are the canonical Opc.Ua.StatusCodes numeric values, declared +/// once each below. Note that is 0x80740000: the +/// 0x80730000 that four sibling drivers declare under that name is really +/// BadWriteNotSupported, which would be actively misleading on a read path and +/// renders as bare "Bad" in the driver CLI's SnapshotFormatter. +/// +/// +/// Thread safety. Backed by a and +/// guaranteed per-key atomic — nothing more, deliberately. writes +/// each dataItemId exactly once per call with a single whole-snapshot assignment, so a +/// reader can never observe a torn snapshot (a Good code beside a stale value). It is +/// explicitly not a whole-batch atomic swap: OPC UA reads are per-tag and the driver +/// offers no cross-tag consistency contract, /sample chunks are deltas whose +/// interleaving is indistinguishable from two adjacent legal chunks, and a whole-index lock +/// would serialize the pump against every read for no semantic gain. +/// +/// +/// KNOWN LIMITATION — DATA_SET / TABLE observations. Their real content +/// is <Entry key=…> child elements, but the parser reads an observation's value +/// with XElement.Value, which concatenates all descendant text. Such an observation +/// therefore reaches this index as a run-together string with the keys discarded, and — +/// because the type inference correctly demotes DATA_SET/TABLE to +/// — it coerces "successfully" into a Good-coded +/// snapshot carrying noise. This index cannot detect the shape: neither +/// nor carries the +/// DataItem's representation, and no value-shape heuristic can separate a +/// concatenated entry list from a legitimate space-bearing EVENT (a Message reads +/// "Coolant level low"). The discriminating signal — the observation element having child +/// elements — exists only in , so the fix belongs there +/// (flag structured observations at parse time; this index then codes them +/// BadNotSupported alongside the TIME_SERIES deferral below). +/// +/// +internal sealed class MTConnectObservationIndex +{ + /// + /// The one token that means "the Agent has no value for this data item". Matched ordinally: + /// the parser already normalizes a CONDITION's <Unavailable/> element name onto + /// this exact spelling, so every category lands on one comparison. + /// + private const string UnavailableSentinel = "UNAVAILABLE"; + + private const uint Good = 0x00000000u; + + /// The Agent reported UNAVAILABLE, or supplied no text at all. + private const uint BadNoCommunication = 0x80310000u; + + /// The tag is authored but the Agent has not yet reported it. + private const uint BadWaitingForInitialData = 0x80320000u; + + /// The dataItemId is neither authored nor ever observed. + private const uint BadNodeIdUnknown = 0x80340000u; + + /// The Agent reported a number the authored type cannot represent. + private const uint BadOutOfRange = 0x803C0000u; + + /// The observation's shape is real data this build cannot represent (TIME_SERIES vectors). + private const uint BadNotSupported = 0x803D0000u; + + /// The Agent reported text that is not a value of the authored type at all. + private const uint BadTypeMismatch = 0x80740000u; + + /// + /// MTConnect's CONDITION state vocabulary, ranked by severity. Used only to reconcile + /// several states reported for one dataItemId within a single document — see + /// . An active Fault outranks UNAVAILABLE because a + /// fault is information and an absent value is not. Case-insensitive so a vendor's casing + /// of a state element still ranks; the parser itself emits the canonical spellings. + /// + private static readonly FrozenDictionary ConditionSeverity = + new Dictionary(StringComparer.OrdinalIgnoreCase) + { + [UnavailableSentinel] = 0, + ["Normal"] = 1, + ["Warning"] = 2, + ["Fault"] = 3, + }.ToFrozenDictionary(StringComparer.OrdinalIgnoreCase); + + private readonly ConcurrentDictionary _snapshots = new(StringComparer.Ordinal); + private readonly FrozenDictionary _tags; + private readonly TimeProvider _timeProvider; + + /// + /// Builds an index over the driver instance's authored tags. + /// + /// + /// The authored tags, keyed by (the DataItem + /// id) — normally . null or empty is + /// legal and means "no authored types": every observation is then indexed as its raw text + /// under . + /// + /// Nothing enforces FullName uniqueness at the options layer, so a duplicate is + /// operator-authorable. The last definition wins rather than throwing: refusing to + /// construct would turn an authoring slip into a driver that will not start, whereas + /// last-wins is deterministic and costs only that one tag's coercion type. + /// + /// + /// Clock behind ServerTimestampUtc; defaults to . + public MTConnectObservationIndex( + IEnumerable? tags = null, + TimeProvider? timeProvider = null) + { + _timeProvider = timeProvider ?? TimeProvider.System; + + var map = new Dictionary(StringComparer.Ordinal); + foreach (var tag in tags ?? []) + { + if (tag is null || string.IsNullOrWhiteSpace(tag.FullName)) continue; + + map[tag.FullName] = tag; + } + + _tags = map.ToFrozenDictionary(StringComparer.Ordinal); + } + + /// The number of distinct dataItemIds currently indexed. + public int Count => _snapshots.Count; + + /// + /// Folds a /current snapshot or one /sample chunk into the index. Observations + /// with no authored tag are indexed too — the Agent streams the whole device, so they are + /// normal, and dropping them would make a streaming-but-not-yet-authored data item + /// indistinguishable from one that does not exist. + /// + /// The parsed streams document. An observation-free result is a legal + /// keep-alive and leaves the index untouched. + /// is null. + public void Apply(MTConnectStreamsResult result) + { + ArgumentNullException.ThrowIfNull(result); + + if (result.Observations is not { Count: > 0 }) return; + + // Duplicates are reconciled on the RAW observations first, so each dataItemId is written + // exactly once below as a single atomic whole-snapshot assignment. Doing it the other way + // — writing every observation and merging snapshots in place — would need a read-modify-write + // per key and could expose a half-merged state to a concurrent reader. + var resolved = new Dictionary(StringComparer.Ordinal); + foreach (var observation in result.Observations) + { + if (observation is null || string.IsNullOrWhiteSpace(observation.DataItemId)) continue; + + resolved[observation.DataItemId] = + resolved.TryGetValue(observation.DataItemId, out var incumbent) + ? Reconcile(incumbent, observation) + : observation; + } + + foreach (var (dataItemId, observation) in resolved) + { + _snapshots[dataItemId] = BuildSnapshot(observation); + } + } + + /// + /// Returns the current snapshot for a DataItem. Never null and never throws: an + /// unknown or not-yet-reported id yields a Bad-coded snapshot rather than an error. + /// + /// The DataItem id the tag binds. + /// + /// The indexed snapshot; else BadWaitingForInitialData when the id is authored but + /// the Agent has not reported it yet (the standard OPC UA "subscribed, no value yet" + /// state); else BadNodeIdUnknown. The two are kept distinct because collapsing them + /// would tell an operator their correctly-authored tag does not exist. + /// + public DataValueSnapshot Get(string dataItemId) + { + if (!string.IsNullOrWhiteSpace(dataItemId) && + _snapshots.TryGetValue(dataItemId, out var snapshot)) + { + return snapshot; + } + + var status = !string.IsNullOrWhiteSpace(dataItemId) && _tags.ContainsKey(dataItemId) + ? BadWaitingForInitialData + : BadNodeIdUnknown; + + return new DataValueSnapshot(null, status, null, Now); + } + + /// + /// Drops every indexed observation. The Agent-restart re-baseline: when the streams header's + /// instanceId changes, every value the index holds describes a device model that no + /// longer exists, and serving it would be worse than serving nothing. + /// + public void Clear() => _snapshots.Clear(); + + private DateTime Now => _timeProvider.GetUtcNow().UtcDateTime; + + /// + /// Picks the winner when one dataItemId appears more than once in a single document. A + /// <Condition> container legitimately carries several simultaneously active + /// states (a Fault and a Warning at once), so for a pair of recognized condition state words + /// the most severe wins — plain last-wins would under-report an active Fault as a + /// Warning purely because of element order. Anything else is last-wins, matching the Agent's + /// ascending-sequence ordering. + /// + /// Deliberately scoped to one document: a later is the Agent's new + /// statement of the state, so a cleared fault must fall back to Normal. A worst-of that + /// spanned Applies would latch every fault forever. + /// + /// + private static MTConnectObservation Reconcile(MTConnectObservation incumbent, MTConnectObservation challenger) + { + if (ConditionSeverity.TryGetValue(incumbent.Value ?? string.Empty, out var incumbentSeverity) && + ConditionSeverity.TryGetValue(challenger.Value ?? string.Empty, out var challengerSeverity)) + { + return challengerSeverity >= incumbentSeverity ? challenger : incumbent; + } + + return challenger; + } + + private DataValueSnapshot BuildSnapshot(MTConnectObservation observation) + { + var serverTimestamp = Now; + var sourceTimestamp = DateTime.SpecifyKind(observation.TimestampUtc, DateTimeKind.Utc); + + // No text at all is the degenerate spelling of "no value": MTConnect defines UNAVAILABLE as + // the only way to report absence, so an empty element is not an empty string value. + if (string.IsNullOrWhiteSpace(observation.Value) || + string.Equals(observation.Value, UnavailableSentinel, StringComparison.Ordinal)) + { + return new DataValueSnapshot(null, BadNoCommunication, sourceTimestamp, serverTimestamp); + } + + _tags.TryGetValue(observation.DataItemId, out var tag); + + // TIME_SERIES vectors arrive as one space-separated string. P1 does not materialize OPC UA + // arrays (deferred to P1.5), and the honest answer is "real data I cannot represent" — not a + // type mismatch, and emphatically not the first element parsed out and served as Good. + // Checked AFTER the sentinel so an unavailable array tag still reports the truthful no-comms. + if (tag is { IsArray: true }) + { + return new DataValueSnapshot(null, BadNotSupported, sourceTimestamp, serverTimestamp); + } + + // An unauthored observation has no declared target type; String is the only coercion that + // cannot fail, and it carries the Agent's text with no interpretation applied. + var status = Coerce(observation.Value, tag?.DriverDataType ?? DriverDataType.String, out var value); + + return status == Good + ? new DataValueSnapshot(value, Good, sourceTimestamp, serverTimestamp) + : new DataValueSnapshot(null, status, sourceTimestamp, serverTimestamp); + } + + /// + /// Converts the Agent's text to the tag's authored type. Every parse is + /// : MTConnect is an invariant-format wire, and a + /// host running a comma-decimal culture would otherwise read 123.4567 as + /// 1234567 — a silently wrong value under Good quality, the worst possible outcome. + /// + /// on success; otherwise a Bad code and a null value. + private static uint Coerce(string raw, DriverDataType type, out object? value) + { + value = null; + var text = raw.Trim(); + + switch (type) + { + // Reference is a Galaxy-style attribute reference encoded as an OPC UA String. It is + // never a sensible authoring for MTConnect, but degrading it to text is harmless where + // rejecting it would break a tag for no protection. + case DriverDataType.String: + case DriverDataType.Reference: + value = raw; + return Good; + + case DriverDataType.Boolean: + if (bool.TryParse(text, out var flag)) { value = flag; return Good; } + if (text is "1") { value = true; return Good; } + if (text is "0") { value = false; return Good; } + return BadTypeMismatch; + + case DriverDataType.DateTime: + if (DateTime.TryParse( + text, + CultureInfo.InvariantCulture, + DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal, + out var moment)) + { + value = DateTime.SpecifyKind(moment, DateTimeKind.Utc); + return Good; + } + + return BadTypeMismatch; + + case DriverDataType.Float32: + if (float.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out var single)) + { + value = single; + return Good; + } + + return BadTypeMismatch; + + case DriverDataType.Float64: + if (double.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out var real)) + { + value = real; + return Good; + } + + return BadTypeMismatch; + + case DriverDataType.Int16: + if (short.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var i16)) + { + value = i16; + return Good; + } + + break; + + case DriverDataType.Int32: + if (int.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var i32)) + { + value = i32; + return Good; + } + + break; + + case DriverDataType.Int64: + if (long.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var i64)) + { + value = i64; + return Good; + } + + break; + + case DriverDataType.UInt16: + if (ushort.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var u16)) + { + value = u16; + return Good; + } + + break; + + case DriverDataType.UInt32: + if (uint.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var u32)) + { + value = u32; + return Good; + } + + break; + + case DriverDataType.UInt64: + if (ulong.TryParse(text, NumberStyles.Integer, CultureInfo.InvariantCulture, out var u64)) + { + value = u64; + return Good; + } + + break; + + default: + // A DriverDataType this driver has no rule for. Text always round-trips, so serving + // it beats failing a tag over an enum member added later. + value = raw; + return Good; + } + + // Integer parse failed. Distinguishing the two causes is worth one extra parse: it tells an + // operator whether to retype the tag or widen it. + return double.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out _) + ? BadOutOfRange // a number the authored type cannot represent (overflow, or a fraction) + : BadTypeMismatch; // not a number at all + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs new file mode 100644 index 00000000..205dec2a --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs @@ -0,0 +1,706 @@ +using System.Globalization; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 8 — : the thread-safe +/// dataItemId → DataValueSnapshot map that /current and the /sample pump +/// both write into, and the quality mapping that turns the Agent's weakly-typed text into an +/// OPC UA-shaped snapshot. +/// +/// Status codes are asserted as literals here on purpose. The production constants +/// are private, so a wrong numeric value in the driver cannot be masked by both +/// sides sharing one symbol — the literal is an independent statement of the OPC UA +/// numeric contract (verified against Opc.Ua.StatusCodes). +/// +/// +public sealed class MTConnectObservationIndexTests +{ + private const string CurrentFixture = "Fixtures/current.xml"; + private const string UnavailableFixture = "Fixtures/current-unavailable.xml"; + private const string SampleFixture = "Fixtures/sample.xml"; + + private const uint Good = 0x00000000u; + private const uint BadNoCommunication = 0x80310000u; + private const uint BadWaitingForInitialData = 0x80320000u; + private const uint BadNodeIdUnknown = 0x80340000u; + private const uint BadOutOfRange = 0x803C0000u; + private const uint BadNotSupported = 0x803D0000u; + private const uint BadTypeMismatch = 0x80740000u; + + /// Every dataItemId both /current fixtures report. + private static readonly string[] AllFixtureDataItemIds = + [ + "dev1_avail", + "dev1_pos", + "dev1_vibration_ts", + "dev1_partcount", + "dev1_execution", + "dev1_program", + "dev1_system_cond" + ]; + + private static MTConnectStreamsResult ParseFixture(string path) => + MTConnectStreamsParser.Parse(File.ReadAllText(path)); + + /// Wraps one synthetic observation in the minimal legal streams result. + private static MTConnectStreamsResult Synthetic(string dataItemId, string value, DateTime? timestampUtc = null) => + new( + InstanceId: 1L, + NextSequence: 2L, + FirstSequence: 1L, + Observations: [new MTConnectObservation( + dataItemId, + value, + timestampUtc ?? new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc))]); + + private static MTConnectTagDefinition Tag(string id, DriverDataType type) => new(id, type); + + // --------------------------------------------------------------- UNAVAILABLE => BadNoCommunication + + /// The plan's headline case: the Agent's UNAVAILABLE sentinel is a no-comms quality, not a value. + [Fact] + public void Unavailable_observation_maps_to_BadNoCommunication_with_null_value() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(UnavailableFixture)); + + var snap = idx.Get("dev1_partcount"); + + snap.Value.ShouldBeNull(); + snap.StatusCode.ShouldBe(0x80310000u); + } + + /// + /// Every one of the seven observations in the all-UNAVAILABLE fixture maps to no-comms — + /// including dev1_system_cond, which reaches the index as a CONDITION whose + /// <Unavailable/> element name the parser normalized onto the exact literal + /// UNAVAILABLE. A case-sensitive miss on that one spelling is the defect this covers. + /// + [Fact] + public void Every_observation_in_the_unavailable_fixture_maps_to_BadNoCommunication() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(UnavailableFixture)); + + foreach (var id in AllFixtureDataItemIds) + { + var snap = idx.Get(id); + snap.StatusCode.ShouldBe(BadNoCommunication, $"dataItemId '{id}' should be no-comms"); + snap.Value.ShouldBeNull($"dataItemId '{id}' must not carry a value when unavailable"); + } + } + + /// An UNAVAILABLE observation still carries the Agent's timestamp — the quality changed, not the clock. + [Fact] + public void Unavailable_observation_keeps_the_agent_source_timestamp() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(UnavailableFixture)); + + var snap = idx.Get("dev1_partcount"); + + snap.SourceTimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 12, 10, 0, 300, DateTimeKind.Utc)); + snap.SourceTimestampUtc!.Value.Kind.ShouldBe(DateTimeKind.Utc); + } + + /// The sentinel is matched exactly; a lowercase spelling is a real value, not a no-comms marker. + [Theory] + [InlineData("UNAVAILABLE", BadNoCommunication)] + [InlineData("Unavailable", Good)] + [InlineData("unavailable", Good)] + public void Unavailable_sentinel_is_matched_on_the_exact_literal(string value, uint expectedStatus) + { + var idx = new MTConnectObservationIndex([Tag("x", DriverDataType.String)]); + idx.Apply(Synthetic("x", value)); + + idx.Get("x").StatusCode.ShouldBe(expectedStatus); + } + + // ------------------------------------------------------------------------- present values + + /// The plan's second headline case. + [Fact] + public void Present_value_indexes_good_with_source_timestamp() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_pos"); + + snap.StatusCode.ShouldBe(0u); + snap.SourceTimestampUtc.ShouldNotBeNull(); + } + + /// SourceTimestamp is the Agent's observation timestamp, never the indexing clock. + [Fact] + public void Present_value_carries_the_observation_timestamp_not_the_index_clock() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_pos"); + + snap.SourceTimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, 200, DateTimeKind.Utc)); + snap.ServerTimestampUtc.ShouldBeGreaterThan(snap.SourceTimestampUtc!.Value); + snap.ServerTimestampUtc.Kind.ShouldBe(DateTimeKind.Utc); + } + + /// A Float64-authored SAMPLE coerces to a real double, not the raw text. + [Fact] + public void Float64_authored_tag_coerces_numeric_text_to_a_double() + { + var idx = new MTConnectObservationIndex([Tag("dev1_pos", DriverDataType.Float64)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_pos"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe(123.4567); + } + + /// An Int64-authored EVENT (PartCount) coerces to a long. sample.xml is the fixture with a present count. + [Fact] + public void Int64_authored_tag_coerces_integer_text_to_a_long() + { + var idx = new MTConnectObservationIndex([Tag("dev1_partcount", DriverDataType.Int64)]); + idx.Apply(ParseFixture(SampleFixture)); + + var snap = idx.Get("dev1_partcount"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe(42L); + } + + /// String-authored EVENT/CONDITION observations carry the Agent's text verbatim. + [Theory] + [InlineData("dev1_execution", "ACTIVE")] + [InlineData("dev1_program", "O1234")] + [InlineData("dev1_avail", "AVAILABLE")] + [InlineData("dev1_system_cond", "Normal")] + public void String_authored_tags_carry_the_raw_agent_text(string dataItemId, string expected) + { + var idx = new MTConnectObservationIndex([Tag(dataItemId, DriverDataType.String)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get(dataItemId); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe(expected); + } + + /// The remaining scalar coercions, each landing on its exact CLR type. + [Theory] + [InlineData(DriverDataType.Boolean, "true", true)] + [InlineData(DriverDataType.Boolean, "FALSE", false)] + [InlineData(DriverDataType.Boolean, "1", true)] + [InlineData(DriverDataType.Boolean, "0", false)] + [InlineData(DriverDataType.Int16, "-12", (short)-12)] + [InlineData(DriverDataType.Int32, "70000", 70000)] + [InlineData(DriverDataType.UInt16, "65535", (ushort)65535)] + [InlineData(DriverDataType.UInt32, "4000000000", 4000000000u)] + [InlineData(DriverDataType.UInt64, "18446744073709551615", 18446744073709551615ul)] + [InlineData(DriverDataType.Float32, "1.5", 1.5f)] + public void Scalar_coercions_land_on_the_authored_clr_type( + DriverDataType type, string text, object expected) + { + var idx = new MTConnectObservationIndex([Tag("x", type)]); + idx.Apply(Synthetic("x", text)); + + var snap = idx.Get("x"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBe(expected); + } + + /// A DateTime-authored tag parses ISO-8601 and lands on UTC kind. + [Fact] + public void DateTime_authored_tag_coerces_to_a_utc_datetime() + { + var idx = new MTConnectObservationIndex([Tag("x", DriverDataType.DateTime)]); + idx.Apply(Synthetic("x", "2026-07-24T12:00:00.250Z")); + + var snap = idx.Get("x"); + + snap.StatusCode.ShouldBe(Good); + var value = snap.Value.ShouldBeOfType(); + value.ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, 250, DateTimeKind.Utc)); + value.Kind.ShouldBe(DateTimeKind.Utc); + } + + /// + /// Numeric coercion is culture-invariant. On a de-DE machine a naive + /// double.TryParse("123.4567") yields 1234567 — a silently wrong value with Good + /// quality, the worst possible outcome. Run on a dedicated thread so the culture change + /// cannot leak into any parallel test. + /// + [Fact] + public void Numeric_coercion_is_culture_invariant() + { + Exception? failure = null; + + var thread = new Thread(() => + { + try + { + Thread.CurrentThread.CurrentCulture = new CultureInfo("de-DE"); + + var idx = new MTConnectObservationIndex([Tag("dev1_pos", DriverDataType.Float64)]); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Get("dev1_pos").Value.ShouldBeOfType().ShouldBe(123.4567); + } + catch (Exception ex) + { + failure = ex; + } + }); + + thread.Start(); + thread.Join(); + + failure.ShouldBeNull(); + } + + // ----------------------------------------------------------------------- coercion failures + + /// + /// A vocabulary EVENT authored as a number cannot parse. It must degrade to a Bad-coded + /// snapshot — never an exception (the index sits under IReadable), and never a + /// silently-wrong zero. + /// + [Fact] + public void Coercion_failure_yields_BadTypeMismatch_and_never_throws() + { + var idx = new MTConnectObservationIndex([Tag("dev1_execution", DriverDataType.Float64)]); + + Should.NotThrow(() => idx.Apply(ParseFixture(CurrentFixture))); + + var snap = idx.Get("dev1_execution"); + + snap.StatusCode.ShouldBe(0x80740000u); + snap.Value.ShouldBeNull(); + } + + /// Non-numeric text is a type mismatch; a number the authored type cannot hold is out of range. + [Theory] + [InlineData(DriverDataType.Float64, "ACTIVE", BadTypeMismatch)] + [InlineData(DriverDataType.Int64, "ACTIVE", BadTypeMismatch)] + [InlineData(DriverDataType.Boolean, "ACTIVE", BadTypeMismatch)] + [InlineData(DriverDataType.Int16, "99999999999", BadOutOfRange)] + [InlineData(DriverDataType.Int32, "3.5", BadOutOfRange)] + [InlineData(DriverDataType.UInt32, "-1", BadOutOfRange)] + public void Failed_coercion_distinguishes_type_mismatch_from_out_of_range( + DriverDataType type, string text, uint expectedStatus) + { + var idx = new MTConnectObservationIndex([Tag("x", type)]); + idx.Apply(Synthetic("x", text)); + + var snap = idx.Get("x"); + + snap.StatusCode.ShouldBe(expectedStatus); + snap.Value.ShouldBeNull(); + } + + /// A Bad-coded coercion failure still reports when the Agent observed it. + [Fact] + public void Failed_coercion_keeps_the_agent_source_timestamp() + { + var idx = new MTConnectObservationIndex([Tag("dev1_execution", DriverDataType.Float64)]); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Get("dev1_execution").SourceTimestampUtc + .ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, 500, DateTimeKind.Utc)); + } + + /// An observation whose text is empty is "the Agent supplied no value" — never a Good empty string. + [Theory] + [InlineData("")] + [InlineData(" ")] + public void Empty_observation_value_maps_to_BadNoCommunication(string value) + { + var idx = new MTConnectObservationIndex([Tag("x", DriverDataType.String)]); + idx.Apply(Synthetic("x", value)); + + var snap = idx.Get("x"); + + snap.StatusCode.ShouldBe(BadNoCommunication); + snap.Value.ShouldBeNull(); + } + + // -------------------------------------------------------------------------- unknown lookups + + /// A dataItemId that is neither authored nor ever observed is unknown to this driver. + [Fact] + public void Unknown_dataItemId_yields_BadNodeIdUnknown() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("no_such_data_item"); + + snap.StatusCode.ShouldBe(BadNodeIdUnknown); + snap.Value.ShouldBeNull(); + snap.SourceTimestampUtc.ShouldBeNull(); + } + + /// + /// An authored tag the Agent has not reported yet is a different condition from an unknown + /// id — it is the standard OPC UA "subscribed, no value yet" state. Collapsing the two + /// would tell an operator their correctly-authored tag does not exist. + /// + [Fact] + public void Authored_but_never_observed_tag_yields_BadWaitingForInitialData() + { + var idx = new MTConnectObservationIndex([Tag("dev1_spindle_speed", DriverDataType.Float64)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_spindle_speed"); + + snap.StatusCode.ShouldBe(BadWaitingForInitialData); + snap.Value.ShouldBeNull(); + } + + /// An empty / whitespace ref is not a lookup; it degrades rather than throwing. + [Theory] + [InlineData("")] + [InlineData(" ")] + public void Blank_dataItemId_yields_BadNodeIdUnknown(string dataItemId) + { + var idx = new MTConnectObservationIndex(); + + idx.Get(dataItemId).StatusCode.ShouldBe(BadNodeIdUnknown); + } + + // --------------------------------------------------------------------- unauthored observations + + /// + /// The Agent streams the whole device, so most observations have no authored tag. They are + /// indexed as their raw text under — the type that + /// cannot fail to coerce — rather than dropped or coded Bad. + /// + [Fact] + public void Unauthored_observation_is_indexed_as_a_good_string() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_pos"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe("123.4567"); + } + + /// Every fixture observation is indexed, authored or not — nothing is silently dropped. + [Fact] + public void Every_fixture_observation_is_indexed_even_with_no_authored_tags() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Count.ShouldBe(AllFixtureDataItemIds.Length); + foreach (var id in AllFixtureDataItemIds) + { + idx.Get(id).StatusCode.ShouldNotBe(BadNodeIdUnknown, $"'{id}' should have been indexed"); + } + } + + // -------------------------------------------------------------------------------- TIME_SERIES + + /// + /// P1 defers TIME_SERIES array materialization to P1.5. An array-authored tag therefore + /// reports BadNotSupported rather than pretending: the vector is real data this build + /// cannot represent, which is neither a type mismatch nor a comms failure. + /// + [Fact] + public void Time_series_authored_as_an_array_reports_BadNotSupported() + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("dev1_vibration_ts", DriverDataType.Float64, IsArray: true, ArrayDim: 10)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_vibration_ts"); + + snap.StatusCode.ShouldBe(BadNotSupported); + snap.Value.ShouldBeNull(); + } + + /// + /// The falsifiable half of the deferral: the vector must never be silently reduced to its + /// first element. A Good 1.1 here would be a wrong value an operator would trust. + /// + [Fact] + public void Time_series_vector_is_never_silently_parsed_as_its_first_element() + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("dev1_vibration_ts", DriverDataType.Float64, IsArray: true, ArrayDim: 10)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_vibration_ts"); + + snap.Value.ShouldNotBe(1.1); + snap.StatusCode.ShouldNotBe(Good); + } + + /// + /// No-comms outranks the unsupported shape: an UNAVAILABLE array tag is a genuine, fully + /// representable state and must report it rather than the deferral code. + /// + [Fact] + public void Unavailable_wins_over_the_unsupported_array_shape() + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("dev1_vibration_ts", DriverDataType.Float64, IsArray: true, ArrayDim: 10)]); + idx.Apply(ParseFixture(UnavailableFixture)); + + idx.Get("dev1_vibration_ts").StatusCode.ShouldBe(BadNoCommunication); + } + + /// + /// An unauthored TIME_SERIES observation carries the Agent's exact text as a string. That + /// is the raw truth with no interpretation applied — not a scalar guess. + /// + [Fact] + public void Unauthored_time_series_observation_carries_the_verbatim_vector_text() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_vibration_ts"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe("1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0"); + } + + // -------------------------------------------------------------------------------- ordering + + /// Within one Apply, the later observation for a dataItemId wins (Agent order is ascending sequence). + [Fact] + public void Last_write_wins_within_a_single_apply() + { + var idx = new MTConnectObservationIndex([Tag("x", DriverDataType.Int64)]); + + idx.Apply(new MTConnectStreamsResult(1L, 4L, 1L, [ + new MTConnectObservation("x", "1", new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc)), + new MTConnectObservation("x", "2", new DateTime(2026, 7, 24, 12, 0, 1, DateTimeKind.Utc)), + new MTConnectObservation("x", "3", new DateTime(2026, 7, 24, 12, 0, 2, DateTimeKind.Utc)), + ])); + + var snap = idx.Get("x"); + + snap.Value.ShouldBe(3L); + snap.SourceTimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 12, 0, 2, DateTimeKind.Utc)); + } + + /// + /// A <Condition> container may carry several simultaneously-active states + /// sharing one dataItemId (a Fault and a Warning at once), which the parser flattens into + /// repeats in one result. Within a single document those states are concurrent, so the most + /// severe wins — plain last-wins would under-report an active Fault as a Warning purely + /// because of element order. + /// + [Theory] + [InlineData("Fault", "Warning", "Fault")] + [InlineData("Warning", "Fault", "Fault")] + [InlineData("Normal", "Warning", "Warning")] + [InlineData("Warning", "Normal", "Warning")] + [InlineData("Fault", "Normal", "Fault")] + public void Simultaneous_condition_states_keep_the_most_severe_within_one_apply( + string first, string second, string expected) + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("c", DriverDataType.String, MtCategory: "CONDITION")]); + + idx.Apply(new MTConnectStreamsResult(1L, 3L, 1L, [ + new MTConnectObservation("c", first, new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc)), + new MTConnectObservation("c", second, new DateTime(2026, 7, 24, 12, 0, 1, DateTimeKind.Utc)), + ])); + + idx.Get("c").Value.ShouldBe(expected); + } + + /// An active fault carries more information than "no value"; it outranks UNAVAILABLE. + [Fact] + public void An_active_fault_outranks_unavailable_within_one_apply() + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("c", DriverDataType.String, MtCategory: "CONDITION")]); + + idx.Apply(new MTConnectStreamsResult(1L, 3L, 1L, [ + new MTConnectObservation("c", "UNAVAILABLE", new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc)), + new MTConnectObservation("c", "Fault", new DateTime(2026, 7, 24, 12, 0, 1, DateTimeKind.Utc)), + ])); + + var snap = idx.Get("c"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBe("Fault"); + } + + /// + /// Worst-of is scoped to ONE document. A later document is the Agent's new statement of the + /// condition's state, so a cleared fault must fall back to Normal — a worst-of that spanned + /// Applies would latch every fault forever. + /// + [Fact] + public void A_later_apply_clears_a_condition_to_its_new_state() + { + var idx = new MTConnectObservationIndex( + [new MTConnectTagDefinition("c", DriverDataType.String, MtCategory: "CONDITION")]); + + idx.Apply(Synthetic("c", "Fault")); + idx.Get("c").Value.ShouldBe("Fault"); + + idx.Apply(Synthetic("c", "Normal")); + + idx.Get("c").Value.ShouldBe("Normal"); + } + + /// Severity reconciliation is for condition words only; repeated ordinary values stay last-wins. + [Fact] + public void Repeated_non_condition_values_stay_last_wins() + { + var idx = new MTConnectObservationIndex([Tag("x", DriverDataType.String)]); + + idx.Apply(new MTConnectStreamsResult(1L, 3L, 1L, [ + new MTConnectObservation("x", "Fault", new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc)), + new MTConnectObservation("x", "O1234", new DateTime(2026, 7, 24, 12, 0, 1, DateTimeKind.Utc)), + ])); + + idx.Get("x").Value.ShouldBe("O1234"); + } + + /// A later Apply overwrites an earlier one — including Good going to no-comms. + [Fact] + public void A_later_apply_overwrites_an_earlier_one() + { + var idx = new MTConnectObservationIndex([Tag("dev1_pos", DriverDataType.Float64)]); + + idx.Apply(ParseFixture(CurrentFixture)); + idx.Get("dev1_pos").StatusCode.ShouldBe(Good); + + idx.Apply(ParseFixture(UnavailableFixture)); + idx.Get("dev1_pos").StatusCode.ShouldBe(BadNoCommunication); + } + + /// An Apply carrying no observations leaves the index untouched (an idle /sample heartbeat). + [Fact] + public void An_observation_free_apply_leaves_the_index_untouched() + { + var idx = new MTConnectObservationIndex([Tag("dev1_pos", DriverDataType.Float64)]); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Apply(new MTConnectStreamsResult(1L, 200L, 1L, [])); + + idx.Get("dev1_pos").Value.ShouldBe(123.4567); + idx.Count.ShouldBe(AllFixtureDataItemIds.Length); + } + + /// Clear drops every indexed observation — the Agent-restart re-baseline Task 11 needs. + [Fact] + public void Clear_drops_every_indexed_observation() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(ParseFixture(CurrentFixture)); + idx.Count.ShouldBe(AllFixtureDataItemIds.Length); + + idx.Clear(); + + idx.Count.ShouldBe(0); + idx.Get("dev1_pos").StatusCode.ShouldBe(BadNodeIdUnknown); + } + + // ------------------------------------------------------------------------ authored-tag map + + /// + /// Nothing enforces FullName uniqueness at the options layer, so a duplicate is operator- + /// authorable. Last definition wins rather than throwing — refusing to construct would turn + /// an authoring slip into a driver that will not start. + /// + [Fact] + public void Duplicate_authored_fullname_takes_the_last_definition() + { + var idx = new MTConnectObservationIndex([ + Tag("dev1_pos", DriverDataType.String), + Tag("dev1_pos", DriverDataType.Float64), + ]); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Get("dev1_pos").Value.ShouldBeOfType().ShouldBe(123.4567); + } + + /// The index binds to the options' Tags collection, which defaults to empty and is never null. + [Fact] + public void Index_can_be_built_from_driver_options() + { + var options = new MTConnectDriverOptions + { + AgentUri = "http://agent:5000", + Tags = [Tag("dev1_pos", DriverDataType.Float64)], + }; + + var idx = new MTConnectObservationIndex(options.Tags); + idx.Apply(ParseFixture(CurrentFixture)); + + idx.Get("dev1_pos").Value.ShouldBe(123.4567); + } + + // -------------------------------------------------------------------------- thread safety + + /// + /// /current (Task 10) and the /sample pump (Task 11) write concurrently while OPC UA reads + /// snapshot. Per-key atomicity is the contract: no throw, no torn snapshot (a Good value + /// paired with a Bad code), and every read observes one whole snapshot. + /// + [Fact] + public async Task Concurrent_applies_and_reads_stay_consistent_and_never_throw() + { + var idx = new MTConnectObservationIndex([Tag("dev1_pos", DriverDataType.Float64)]); + var current = ParseFixture(CurrentFixture); + var unavailable = ParseFixture(UnavailableFixture); + + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2)); + var writers = Enumerable.Range(0, 4).Select(i => Task.Run(() => + { + while (!cts.IsCancellationRequested) + { + idx.Apply(i % 2 == 0 ? current : unavailable); + } + })); + + var readers = Enumerable.Range(0, 4).Select(_ => Task.Run(() => + { + while (!cts.IsCancellationRequested) + { + var snap = idx.Get("dev1_pos"); + + // Exactly two snapshots are legal for this tag; anything else is a tear. + if (snap.StatusCode == 0u) + { + snap.Value.ShouldBe(123.4567); + } + else + { + snap.StatusCode.ShouldBe(BadNoCommunication); + snap.Value.ShouldBeNull(); + } + } + })); + + await Task.WhenAll(writers.Concat(readers)); + + idx.Count.ShouldBe(AllFixtureDataItemIds.Length); + } + + /// Applying a null result is a programming error, not degradable data. + [Fact] + public void Apply_rejects_a_null_result() + { + var idx = new MTConnectObservationIndex(); + + Should.Throw(() => idx.Apply(null!)); + } +} -- 2.52.0 From 5ba9f1be6f474761550f8017d88f776e29f24296 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 15:01:31 -0400 Subject: [PATCH 13/34] fix(mtconnect): make a /sample stream end loud, and correct the gap contract (Task 7 review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Code review of c46540ae approved the framing algorithm but moved the risk onto the contract around it. C1 (critical). SampleAsync returned normally on three different events: EOF, the closing --boundary--, and a response that was not multipart at all (where it yielded one parsed snapshot and finished). The seam documents the stream as yielding until ct is cancelled, so a Task 11 pump written to that contract would silently drop its subscription or hot-loop reconnecting; the non-multipart leg reported a configuration error as a healthy finished stream, which is the #485 quiet-successful-termination shape aimed at the very next task. Every non-cancellation end now throws: MTConnectStreamEndedException (ConnectionClosed / ClosingBoundary - transient) or MTConnectStreamNotSupportedException (configuration - reconnecting reproduces it forever), sharing one catchable base. The non-multipart body is still parsed first so an Agent's own MTConnectError text wins, but the document is NOT yielded. I1. IsSequenceGap moved to IMTConnectAgentClient (static) and its first parameter is now expectedFrom. The old name and doc were wrong for every chunk after the first - the correct comparand is the PREVIOUS chunk's NextSequence, and comparing against the opening "from" argument reports a gap on every chunk once the ring buffer rolls, i.e. an endless /current re-baseline storm. Both doc blocks also now record that cppagent answers from < firstSequence with an OUT_OF_RANGE MTConnectError under HTTP 200, which surfaces as InvalidDataException and NOT as a gap - so Task 11 must treat that parse failure as a re-baseline trigger too. I2. Every multipart test served the whole body from one buffer, so every framing test completed in a single ReadAsync - the split-boundary path, the split-header path and the multi-fill loop were correct by inspection only. A ChunkedStream double (N bytes per read, N = 1/3/7) now drives the fixtures through a split transport, plus a >16 KB document that forces a buffer resize. Falsifiability: removing either scan overlap, or collapsing the fill loop, fails ONLY the new chunked tests. I3. Disposal mid-enumeration surfaces as OperationCanceledException via an internal dispose-linked token, not a raw ObjectDisposedException that Task 9's re-init would inflict on an enumerating pump. I5. HeartbeatMs, SampleIntervalMs and SampleCount join RequestTimeoutMs in construction-time positive-value validation. All three reach the query string and an Agent answers heartbeat=0 with an HTTP-200 MTConnectError that names no config key. I4/M2. The no-Content-length framing fallback logs a one-shot warning (the client takes an optional ILogger); a multipart response with no boundary parameter fails fast instead of degrading into a watchdog timeout. M5/M6. Shared element/attribute reading rules extracted to MTConnectXml; the probe parser documents why it alone does not trim BOM/whitespace. The literal U+FEFF in source became a '' escape. Also, from Task 8: MTConnectObservation gained IsStructured (default false), set from element.HasElements. A DATA_SET/TABLE observation's children concatenate through element.Value to nonsense ("12" for two entries) that the index would publish as Good, and only the parser can tell that apart from a legitimate space-bearing Message. False for CONDITION observations - their value comes from the element name, so child content cannot corrupt it. 275/275 green in this suite's own files. --- docs/plans/2026-07-24-mtconnect-driver.md | 51 ++ .../IMTConnectAgentClient.cs | 76 ++- .../MTConnectAgentClient.cs | 291 +++++++--- .../MTConnectDtos.cs | 25 +- .../MTConnectProbeParser.cs | 131 ++--- .../MTConnectStreamExceptions.cs | 161 ++++++ .../MTConnectStreamsParser.cs | 108 +--- .../MTConnectXml.cs | 146 +++++ .../MTConnectStreamsParseTests.cs | 537 +++++++++++++++++- 9 files changed, 1232 insertions(+), 294 deletions(-) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamExceptions.cs create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectXml.cs diff --git a/docs/plans/2026-07-24-mtconnect-driver.md b/docs/plans/2026-07-24-mtconnect-driver.md index a3b3abef..4e6b733b 100644 --- a/docs/plans/2026-07-24-mtconnect-driver.md +++ b/docs/plans/2026-07-24-mtconnect-driver.md @@ -385,6 +385,57 @@ dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests --filter "Ful git add -A && git commit -m "feat(mtconnect): current/sample parse + sequence-gap detection (Task 7)" ``` +### Task 7 review remediation (post-`c46540ae`) — three contract changes Tasks 9/11/13 must build against + +Code review of `c46540ae` approved the framing algorithm but moved the risk onto the **contract**. Three +changes here are breaking relative to the sketch above; the rest are hardening. + +1. **`SampleAsync` never ends normally except by cancellation.** The seam documents the stream as + yielding "indefinitely, until `ct` is cancelled", but the implementation returned normally on EOF, on + the closing `--boundary--`, and — worst — on a response that was not multipart at all, where it + yielded one parsed snapshot and finished. A pump written to the documented contract has no reason to + handle normal completion, so it would silently drop its subscription or hot-loop reconnecting; the + non-multipart leg additionally reported a **configuration error as a healthy finished stream**, which + is the #485 quiet-successful-termination shape aimed straight at Task 11. Every non-cancellation end + now throws: `MTConnectStreamEndedException` (with `MTConnectStreamEndReason.ConnectionClosed` / + `ClosingBoundary` — transient, reconnect) or `MTConnectStreamNotSupportedException` (configuration — + reconnecting reproduces it forever), sharing the base `MTConnectStreamException`. The non-multipart + body is still parsed first, so an Agent's own `MTConnectError` text still wins, but the document is + **not** yielded. +2. **`IsSequenceGap` moved to `IMTConnectAgentClient` (static) and its first parameter is now + `expectedFrom`.** The sketch's `requestedFrom` name and doc were wrong for every chunk after the + first: the correct comparand is the **previous chunk's `NextSequence`**, and comparing against the + opening `from` reports a gap on every chunk once the ring buffer rolls — an endless `/current` + re-baseline storm against a healthy stream. Rehomed onto the seam so Task 11 need not reference the + concrete client. **Task 11 must also treat an `OUT_OF_RANGE` `InvalidDataException` as a re-baseline + trigger:** cppagent answers a `from` below `firstSequence` with an `MTConnectError` under HTTP 200, + so ring-buffer overflow arrives as a *parse failure*, never as a gap-bearing chunk. `IsSequenceGap` + alone does not cover it. Documented on the seam. +3. **`MTConnectObservation` gained `IsStructured`** (default `false`), set from `element.HasElements`. + A DATA_SET/TABLE observation's `` children concatenate through `element.Value` to + nonsense (`"12"` for two entries) which the index would otherwise publish as Good, and only the + parser can tell that apart from a legitimate space-bearing `Message` — no value-shape heuristic + works. Deliberately **false for CONDITION** observations: their value comes from the element *name*, + so child content cannot corrupt it. The index maps the flag to a status; the parser does not. + +Hardening in the same pass: disposal mid-enumeration now surfaces as `OperationCanceledException` +(via an internal dispose-linked token) rather than a raw `ObjectDisposedException`/`IOException` that +Task 9's re-init would otherwise inflict on an enumerating pump; `HeartbeatMs`, `SampleIntervalMs` and +`SampleCount` join `RequestTimeoutMs` in construction-time positive-value validation (all three reach +the query string, and an Agent answers `heartbeat=0` with an HTTP-200 `MTConnectError` that would name +no config key); a `multipart/*` response with no `boundary` parameter fails fast instead of degrading +into a watchdog timeout; the no-`Content-length` framing fallback logs a one-shot warning (the client +now takes an optional `ILogger`); and the shared element/attribute reading rules moved into +`MTConnectXml`, used by both parsers. + +**Coverage gap closed, and worth remembering as a pattern.** Every multipart test served the whole body +from one buffer, so **every framing test completed in a single `ReadAsync`** — the split-boundary path, +the split-header path and the multi-fill loop were correct by inspection only, on the task whose +headline risk is framing. A `ChunkedStream` double returning N bytes per read (N = 1, 3, 7) now drives +the fixtures through a split transport. Falsifiability confirms the gap was real: removing the +split-boundary overlap, removing the split-header overlap, and collapsing `EnsureAsync`'s fill loop each +fail **only** the new chunked tests — every pre-existing multipart test stays green under all three. + --- ## Task 8: `MTConnectObservationIndex` + `UNAVAILABLE → BadNoCommunication` diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs index b596827a..86b9c98d 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs @@ -16,7 +16,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// /sample long poll. (Task 0 planned to build on the TrakHound MTConnect.NET libraries; /// Tasks 6 and 7 proved they can neither parse a document nor frame the stream socket-free, and /// dropped the references — see the Task 0 CORRECTION block in the plan.) A fake for tests only -/// needs to implement these three members. +/// needs to implement these three instance members; is a static +/// protocol rule that lives here, rather than on the implementation, so a consumer written +/// against this seam never has to reference the concrete client. /// public interface IMTConnectAgentClient { @@ -38,13 +40,73 @@ public interface IMTConnectAgentClient /// /// Opens an Agent /sample long-poll stream starting at sequence - /// and yields one per multipart chunk the Agent sends, - /// indefinitely, until is cancelled. Each yielded result's - /// is load-bearing — the caller compares - /// it against the requested to detect a ring-buffer sequence gap and - /// re-baseline via . + /// and yields one per multipart chunk the Agent sends. /// + /// + /// + /// The only way this enumeration ends without throwing is being + /// cancelled. Every other way a stream can stop — the connection dropping, the Agent + /// writing its closing boundary, or the endpoint turning out not to stream at all — + /// throws an naming which. A consumer therefore + /// never has to treat "the enumeration finished" as an ambiguous signal, and cannot + /// silently lose its subscription to a data path that quietly stopped (#485). See + /// (transient — reconnect) and + /// (configuration — reconnecting will + /// reproduce it forever). + /// + /// + /// Advancing the cursor. After each chunk, the caller's next expected sequence is + /// that chunk's — not the + /// this call was opened with. Feed that running value to + /// and, on reconnect, to a fresh SampleAsync. + /// + /// /// The sequence number to resume streaming from. - /// Cancellation token; cancelling ends the stream. + /// Cancellation token; cancelling is the contracted way to end the stream. IAsyncEnumerable SampleAsync(long from, CancellationToken ct); + + /// + /// Has the Agent's ring buffer overflowed past the sequence the caller expected next? True + /// when the chunk's oldest retained sequence is strictly newer than + /// , meaning observations between the two were evicted before + /// the caller could read them and an incremental update would silently skip values. + /// + /// + /// + /// is a running cursor, not the original + /// from. It equals the from passed to only + /// for the first chunk; from the second chunk on it is the previous + /// chunk's . Comparing every chunk + /// against the original from instead reports a gap on every chunk once the + /// Agent's buffer has rolled past it — an endless /current re-baseline storm + /// against a perfectly healthy stream. + /// + /// + /// Equality is not a gap: firstSequence == expectedFrom is the ordinary + /// contiguous case where the Agent simply had nothing older to send. + /// + /// + /// This check alone does NOT cover every ring-buffer overflow. It can only judge + /// a chunk the Agent was willing to send — and cppagent answers a from that has + /// already fallen out of its buffer with an MTConnectError / OUT_OF_RANGE + /// document served under HTTP 200, which this client surfaces as an + /// out of , never as a chunk. + /// A pump that re-baselines only on IsSequenceGap will therefore fault instead of + /// recovering exactly when it has fallen furthest behind. Treat an OUT_OF_RANGE + /// parse failure as a re-baseline trigger (re-prime from , + /// resume from that snapshot's NextSequence) just like a detected gap. + /// + /// + /// The sequence the caller expected this chunk to start at. + /// The chunk the Agent answered with. + /// + /// true when observations were lost and the caller must re-baseline via + /// rather than trusting an incremental update. + /// + static bool IsSequenceGap(long expectedFrom, MTConnectStreamsResult chunk) + { + ArgumentNullException.ThrowIfNull(chunk); + + return chunk.FirstSequence > expectedFrom; + } } diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs index a30a4aa1..8ce654b1 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs @@ -1,6 +1,7 @@ using System.Globalization; using System.Runtime.CompilerServices; using System.Text; +using Microsoft.Extensions.Logging; namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; @@ -25,7 +26,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// a long poll that is supposed to outlive that bound indefinitely. Raising the one /// timeout to suit the stream would un-bound the unary deadline — precisely what arch-review /// R2-01 says must never happen — so the streaming leg gets its own client with -/// and is bounded differently instead (below). +/// and is bounded differently instead (below). In +/// production each client owns its own handler and connection pool; in tests both are built +/// over one injected handler, which the caller owns. /// /// /// Every call is deadline-bounded, but not all by the same mechanism. The unary legs @@ -41,61 +44,94 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// and the watchdog is what closes it here. /// /// -/// A non-positive RequestTimeoutMs is rejected at construction so an -/// operator-authored 0 can never come to mean "wait forever" (arch-review 01/S-6). +/// Every operator-authorable duration/count is rejected at construction if non-positive. +/// A 0 must never come to mean "wait forever" (arch-review 01/S-6), and the three +/// /sample query knobs additionally reach the wire: an Agent answers +/// heartbeat=0 with an MTConnectError under HTTP 200, which would otherwise +/// surface as a parse failure that never names the offending config key. +/// +/// +/// Disposal is safe against an in-flight enumeration. Task 9 re-initializes the +/// driver by disposing and rebuilding this client, which can happen while a pump is still +/// enumerating . Rather than letting the in-flight read fault with +/// an unclassified or that no +/// caller is written to expect, every request runs under a token linked to an internal +/// dispose token, so disposal surfaces as an ordinary +/// . /// /// /// This class does not swallow failures into an empty model: an unreachable Agent, a non-2xx -/// status, or an unparseable body all throw. InitializeAsync (Task 9) is what catches -/// that and moves the driver to Faulted. +/// status, an unparseable body, and the end of a /sample stream all throw. See +/// for why the last of those is an exception rather +/// than a normal end of enumeration. InitializeAsync (Task 9) is what catches these +/// and moves the driver to Faulted. /// /// public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable { private readonly HttpClient _http; private readonly HttpClient _streamHttp; + private readonly CancellationTokenSource _disposeCts = new(); + private readonly ILogger? _logger; private readonly Uri _probeUri; private readonly Uri _currentUri; private readonly string _sampleUriBase; private readonly string _sampleQuerySuffix; private readonly TimeSpan _requestTimeout; private readonly TimeSpan _streamIdleTimeout; + private bool _disposed; /// Creates a client for the Agent described by . /// The driver options carrying the Agent URI, device scope, and per-call deadline. + /// + /// Optional logger. Used only for conditions an operator needs to see but which are not + /// failures — today, the degraded /sample framing mode. + /// /// is missing or is not an absolute HTTP(S) URI. - /// is not positive. - public MTConnectAgentClient(MTConnectDriverOptions options) - : this(options, handler: null) + /// + /// Any of , + /// , + /// or + /// is not positive. + /// + public MTConnectAgentClient(MTConnectDriverOptions options, ILogger? logger = null) + : this(options, handler: null, logger) { } /// - /// Test seam: as , but sends through - /// the supplied handler so the request/response round trip can be exercised with no socket. - /// The caller retains ownership of . + /// Test seam: as , but + /// sends through the supplied handler so the request/response round trip can be exercised + /// with no socket. The caller retains ownership of . /// - internal MTConnectAgentClient(MTConnectDriverOptions options, HttpMessageHandler? handler) + internal MTConnectAgentClient(MTConnectDriverOptions options, HttpMessageHandler? handler, ILogger? logger = null) { ArgumentNullException.ThrowIfNull(options); - if (options.RequestTimeoutMs <= 0) - { - throw new ArgumentOutOfRangeException( - nameof(options), - options.RequestTimeoutMs, - $"{nameof(MTConnectDriverOptions.RequestTimeoutMs)} must be positive; a non-positive per-call deadline would let a frozen Agent wedge the driver indefinitely."); - } + RequirePositive( + options.RequestTimeoutMs, + nameof(MTConnectDriverOptions.RequestTimeoutMs), + "a non-positive per-call deadline would let a frozen Agent wedge the driver indefinitely"); + RequirePositive( + options.HeartbeatMs, + nameof(MTConnectDriverOptions.HeartbeatMs), + "it is sent to the Agent as 'heartbeat', which an Agent rejects with an MTConnectError under HTTP 200, and it is what bounds the /sample watchdog — a quiet connection would otherwise be indistinguishable from a dead one"); + RequirePositive( + options.SampleIntervalMs, + nameof(MTConnectDriverOptions.SampleIntervalMs), + "it is sent to the Agent as 'interval'; zero would spin the streaming connection into a busy-loop"); + RequirePositive( + options.SampleCount, + nameof(MTConnectDriverOptions.SampleCount), + "it is sent to the Agent as 'count'; a non-positive count asks the Agent for no observations"); + _logger = logger; _requestTimeout = TimeSpan.FromMilliseconds(options.RequestTimeoutMs); // Two missed heartbeats plus one request timeout of network slack. Deriving the window from // HeartbeatMs is what makes it a liveness check rather than a throughput assumption: a busy - // Agent and an idle one both emit at least one chunk per heartbeat. A non-positive - // HeartbeatMs is clamped away rather than rejected — the window then collapses to - // RequestTimeoutMs, which is still strictly positive, so the read can never be unbounded. - _streamIdleTimeout = TimeSpan.FromMilliseconds( - (2L * Math.Max(0, options.HeartbeatMs)) + options.RequestTimeoutMs); + // Agent and an idle one both emit at least one chunk per heartbeat. + _streamIdleTimeout = TimeSpan.FromMilliseconds((2L * options.HeartbeatMs) + options.RequestTimeoutMs); _probeUri = BuildRequestUri(options, "probe"); _currentUri = BuildRequestUri(options, "current"); @@ -114,29 +150,11 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable _streamHttp.Timeout = Timeout.InfiniteTimeSpan; } - /// - /// Has the Agent's ring buffer overflowed past what the caller asked for? True when the - /// chunk's oldest retained sequence is strictly newer than the sequence the caller - /// requested, which means observations between the two were evicted before the caller could - /// read them. Equality is not a gap: firstSequence == requestedFrom is the ordinary - /// contiguous case where the Agent simply had nothing older to send. - /// - /// The from sequence the caller passed to . - /// The chunk the Agent answered with. - /// - /// true when observations were lost and the caller must re-baseline via - /// rather than trusting an incremental update. - /// - public static bool IsSequenceGap(long requestedFrom, MTConnectStreamsResult chunk) - { - ArgumentNullException.ThrowIfNull(chunk); - - return chunk.FirstSequence > requestedFrom; - } - /// public async Task ProbeAsync(CancellationToken ct) { + ObjectDisposedException.ThrowIf(_disposed, this); + // ResponseContentRead (the default) buffers the whole body *under* this awaited, token-bound // call, so the parse below reads from memory — no network read can outlive the deadline. using var response = await SendAsync(_probeUri, ct).ConfigureAwait(false); @@ -149,6 +167,8 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// public async Task CurrentAsync(CancellationToken ct) { + ObjectDisposedException.ThrowIf(_disposed, this); + using var response = await SendAsync(_currentUri, ct).ConfigureAwait(false); await using var body = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false); @@ -161,34 +181,50 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable long from, [EnumeratorCancellation] CancellationToken ct) { + ObjectDisposedException.ThrowIf(_disposed, this); + var uri = BuildSampleUri(from); + // Linked to the caller's token AND to disposal, so tearing this client down mid-enumeration + // surfaces as a cancellation rather than as a raw ObjectDisposedException from the socket. + using var lifetime = CancellationTokenSource.CreateLinkedTokenSource(ct, _disposeCts.Token); + var live = lifetime.Token; + using var request = new HttpRequestMessage(HttpMethod.Get, uri); // ResponseHeadersRead, not the default ResponseContentRead: buffering a // multipart/x-mixed-replace body that is designed never to end would hang here forever. - using var response = await SendStreamRequestAsync(request, uri, ct).ConfigureAwait(false); + using var response = await SendStreamRequestAsync(request, uri, lifetime).ConfigureAwait(false); - await using var body = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false); + await using var body = await response.Content.ReadAsStreamAsync(live).ConfigureAwait(false); - var boundary = ResolveMultipartBoundary(response); + var boundary = ResolveMultipartBoundary(response, uri); if (boundary is null) { - // Not a multipart stream: the Agent answered with a single document. This is the shape - // an MTConnectError takes — Agents serve one under HTTP 200 — so the parse below is what - // surfaces the Agent's own error text rather than an empty, successful-looking stream. - yield return MTConnectStreamsParser.Parse(await ReadWholeBodyAsync(body, uri, ct).ConfigureAwait(false)); + // Not a multipart stream: the Agent answered with a single document, which means it + // ignored the streaming request entirely. Parse it first — an MTConnectError arrives in + // exactly this shape (text/xml under HTTP 200) and the Agent's own error text is far + // more useful than ours — then report the misconfiguration. The parsed snapshot is + // deliberately NOT yielded: one document followed by a healthy-looking finished stream + // is how a dead data path disguises itself as a working one (#485). + var single = await ReadWholeBodyAsync(body, uri, lifetime).ConfigureAwait(false); + _ = MTConnectStreamsParser.Parse(single); - yield break; + throw new MTConnectStreamNotSupportedException( + $"MTConnect Agent answered the /sample request '{uri}' with a single '{response.Content.Headers.ContentType?.MediaType ?? "unknown"}' document instead of a multipart/x-mixed-replace stream. It parsed as a valid MTConnectStreams document but will never stream, so no subscription is possible; check that the URI addresses an MTConnect Agent directly and is not fronted by a proxy that buffers streaming responses."); } - var reader = new MultipartStreamReader(body, boundary, _streamIdleTimeout, uri); + var reader = new MultipartStreamReader(body, boundary, _streamIdleTimeout, uri, _logger); + var delivered = 0L; while (true) { - var part = await reader.ReadNextPartAsync(ct).ConfigureAwait(false); + var part = await reader.ReadNextPartAsync(live).ConfigureAwait(false); if (part is null) { - yield break; + // The caller cancelling is the ONE contracted way out; anything else is reported. + live.ThrowIfCancellationRequested(); + + throw new MTConnectStreamEndedException(reader.EndReason, uri.AbsoluteUri, delivered); } // A frame carrying nothing but whitespace is transport filler, not a document. The @@ -200,6 +236,8 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable continue; } + delivered++; + yield return MTConnectStreamsParser.Parse(part); } } @@ -207,8 +245,30 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// public void Dispose() { + if (_disposed) + { + return; + } + + _disposed = true; + + // Cancel BEFORE disposing the clients so an in-flight read observes cancellation rather + // than a disposed handler. + _disposeCts.Cancel(); _http.Dispose(); _streamHttp.Dispose(); + _disposeCts.Dispose(); + } + + private static void RequirePositive(int value, string optionName, string because) + { + if (value <= 0) + { + throw new ArgumentOutOfRangeException( + nameof(MTConnectDriverOptions), + value, + $"{optionName} must be positive; {because}."); + } } private Uri BuildSampleUri(long from) => @@ -217,7 +277,8 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable private async Task SendAsync(Uri uri, CancellationToken ct) { - using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + using var lifetime = CancellationTokenSource.CreateLinkedTokenSource(ct, _disposeCts.Token); + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(lifetime.Token); deadline.CancelAfter(_requestTimeout); HttpResponseMessage response; @@ -225,11 +286,11 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable { response = await _http.GetAsync(uri, deadline.Token).ConfigureAwait(false); } - catch (OperationCanceledException) when (!ct.IsCancellationRequested) + catch (OperationCanceledException) when (!lifetime.IsCancellationRequested) { - // The caller did not cancel, so this is our own deadline (or HttpClient.Timeout) firing. - // Reported as a TimeoutException because a bare "A task was canceled" tells an operator - // nothing about which Agent stopped answering. + // Neither the caller nor disposal cancelled, so this is our own deadline (or + // HttpClient.Timeout) firing. Reported as a TimeoutException because a bare "A task was + // canceled" tells an operator nothing about which Agent stopped answering. throw TimedOut(uri); } @@ -243,9 +304,9 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// this phase is bounded by the unary deadline; the body is bounded by the watchdog. ///
private async Task SendStreamRequestAsync( - HttpRequestMessage request, Uri uri, CancellationToken ct) + HttpRequestMessage request, Uri uri, CancellationTokenSource lifetime) { - using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(lifetime.Token); deadline.CancelAfter(_requestTimeout); HttpResponseMessage response; @@ -255,7 +316,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable .SendAsync(request, HttpCompletionOption.ResponseHeadersRead, deadline.Token) .ConfigureAwait(false); } - catch (OperationCanceledException) when (!ct.IsCancellationRequested) + catch (OperationCanceledException) when (!lifetime.IsCancellationRequested) { throw TimedOut(uri); } @@ -266,9 +327,9 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable } /// Reads a non-multipart streaming body whole, under the unary deadline. - private async Task ReadWholeBodyAsync(Stream body, Uri uri, CancellationToken ct) + private async Task ReadWholeBodyAsync(Stream body, Uri uri, CancellationTokenSource lifetime) { - using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(lifetime.Token); deadline.CancelAfter(_requestTimeout); using var reader = new StreamReader(body, Encoding.UTF8); @@ -276,7 +337,7 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable { return await reader.ReadToEndAsync(deadline.Token).ConfigureAwait(false); } - catch (OperationCanceledException) when (!ct.IsCancellationRequested) + catch (OperationCanceledException) when (!lifetime.IsCancellationRequested) { throw TimedOut(uri); } @@ -289,7 +350,14 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// The boundary parameter of a multipart/* response, or null when the /// Agent answered with a single (non-multipart) document. /// - private static string? ResolveMultipartBoundary(HttpResponseMessage response) + /// + /// The response claims to be multipart/* but carries no boundary parameter, so + /// there is nothing to frame it by. Failed fast and named, because the alternative — falling + /// through to whole-body reading of a body that never ends — surfaces much later as a bare + /// watchdog that points an operator at the network instead of + /// at the malformed header. + /// + private static string? ResolveMultipartBoundary(HttpResponseMessage response, Uri uri) { var contentType = response.Content.Headers.ContentType; if (contentType?.MediaType is null || @@ -302,7 +370,13 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable .FirstOrDefault(p => string.Equals(p.Name, "boundary", StringComparison.OrdinalIgnoreCase))?.Value? .Trim('"'); - return string.IsNullOrEmpty(boundary) ? null : boundary; + if (string.IsNullOrEmpty(boundary)) + { + throw new MTConnectStreamNotSupportedException( + $"MTConnect Agent answered the /sample request '{uri}' with Content-Type '{contentType.MediaType}' but no 'boundary' parameter, so the multipart stream cannot be framed."); + } + + return boundary; } private static Uri BuildRequestUri(MTConnectDriverOptions options, string request) @@ -335,18 +409,22 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// /// /// Why hand-rolled. ASP.NET Core's MultipartReader is not in this - /// dependency set, and TrakHound's - /// MTConnectHttpClientStream — the one candidate the plan left open — takes a URL - /// and dials its own socket, so it can neither be fed an existing response stream nor be - /// exercised without a listener. See the Task 0 correction in the plan. + /// dependency set, and TrakHound's MTConnectHttpClientStream — the one candidate + /// the plan left open — takes a URL and dials its own socket, so it can neither be fed an + /// existing response stream nor be exercised without a listener. See the Task 0 + /// correction in the plan. /// /// /// Two framing paths, and the first one matters for latency. When the part carries /// a Content-length header — every Agent in the wild writes one — the body is read /// by length and the chunk is emitted the instant it is complete. Without it there is no /// way to know a part has ended except by seeing the next boundary, so that path - /// necessarily emits each chunk one chunk late; it exists as a correctness fallback, not - /// as the expected path. + /// necessarily emits each chunk one chunk late: with the default 10 s heartbeat a value + /// can be up to a heartbeat stale on a driver built for low-latency subscribe. It is a + /// correctness fallback, not the expected path, and it logs a one-shot warning when it + /// engages so the degradation is visible rather than merely slow. (It cannot strand the + /// final part indefinitely — a clean end of stream returns the buffered tail — though a + /// watchdog timeout on a stalled Agent does discard it.) /// /// /// Every read is watchdog-bounded. A silent peer faults the stream rather than @@ -354,7 +432,8 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable /// malfunctioning Agent cannot grow it without limit. /// /// - private sealed class MultipartStreamReader(Stream stream, string boundary, TimeSpan idleTimeout, Uri uri) + private sealed class MultipartStreamReader( + Stream stream, string boundary, TimeSpan idleTimeout, Uri uri, ILogger? logger) { private const int MaxBufferBytes = 32 * 1024 * 1024; @@ -365,10 +444,14 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable private byte[] _buffer = new byte[16 * 1024]; private int _length; private bool _finished; + private bool _warnedNoContentLength; + + /// How the stream ended. Only meaningful once a read has returned null. + public MTConnectStreamEndReason EndReason { get; private set; } = MTConnectStreamEndReason.ConnectionClosed; /// /// Reads the next part's payload, or null once the closing delimiter is seen or - /// the Agent closes the connection. + /// the Agent closes the connection — see for which. /// public async Task ReadNextPartAsync(CancellationToken ct) { @@ -380,42 +463,65 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable if (!await SkipThroughDelimiterAsync(ct).ConfigureAwait(false) || !await EnsureAsync(2, ct).ConfigureAwait(false)) { - _finished = true; - - return null; + return Finish(MTConnectStreamEndReason.ConnectionClosed); } // "--boundary--" closes the stream. if (_buffer[0] == (byte)'-' && _buffer[1] == (byte)'-') { - _finished = true; - - return null; + return Finish(MTConnectStreamEndReason.ClosingBoundary); } var headerEnd = await FindHeaderEndAsync(ct).ConfigureAwait(false); if (headerEnd < 0) { - _finished = true; - - return null; + return Finish(MTConnectStreamEndReason.ConnectionClosed); } var headers = Encoding.ASCII.GetString(_buffer, 0, headerEnd); Consume(headerEnd); - var body = ParseContentLength(headers) is { } declared - ? await ReadByLengthAsync(declared, ct).ConfigureAwait(false) - : await ReadToNextDelimiterAsync(ct).ConfigureAwait(false); + byte[]? body; + if (ParseContentLength(headers) is { } declared) + { + body = await ReadByLengthAsync(declared, ct).ConfigureAwait(false); + } + else + { + WarnOnceAboutMissingContentLength(); + body = await ReadToNextDelimiterAsync(ct).ConfigureAwait(false); + } return body is null ? null : Encoding.UTF8.GetString(body); } + private void WarnOnceAboutMissingContentLength() + { + if (_warnedNoContentLength) + { + return; + } + + _warnedNoContentLength = true; + + logger?.LogWarning( + "MTConnect /sample stream from '{AgentUri}' sends parts with no Content-length header. Falling back to boundary-scan framing, which cannot emit a chunk until the NEXT chunk begins, so every observation is delayed by up to one Agent heartbeat. Values will read stale; this is the Agent's framing, not a driver fault.", + uri); + } + + private string? Finish(MTConnectStreamEndReason reason) + { + _finished = true; + EndReason = reason; + + return null; + } + private async Task ReadByLengthAsync(int declared, CancellationToken ct) { if (!await EnsureAsync(declared, ct).ConfigureAwait(false)) { - _finished = true; + Finish(MTConnectStreamEndReason.ConnectionClosed); return null; } @@ -431,10 +537,11 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable var next = await FindDelimiterAsync(ct).ConfigureAwait(false); if (next < 0) { - // EOF with no closing delimiter: whatever is buffered is the last part. + // EOF with no closing delimiter: whatever is buffered is the last part. Returned + // rather than dropped, so a clean end of stream cannot strand a chunk. var tail = _buffer[.._length]; _length = 0; - _finished = true; + Finish(MTConnectStreamEndReason.ConnectionClosed); return tail; } @@ -477,6 +584,8 @@ public sealed class MTConnectAgentClient : IMTConnectAgentClient, IDisposable var searchedTo = 0; while (true) { + // Restart the scan far enough back that a delimiter split across two reads is still + // found — without the overlap, a boundary straddling a packet boundary is invisible. var index = IndexOf(_delimiter, Math.Max(0, searchedTo - (_delimiter.Length - 1))); if (index >= 0) { diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs index 76bd41c9..740a2d71 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDtos.cs @@ -136,10 +136,33 @@ public sealed record MTConnectStreamsResult( /// The observation's Agent-reported timestamp, normalized to UTC ( /// is always ). Becomes the OPC UA variable's SourceTimestamp. /// +/// +/// true when the Agent carried this observation's real content in child elements +/// rather than as text — an MTConnect 2.0 DATA_SET / TABLE observation, whose +/// content is a list of <Entry key="…"> elements. +/// +/// Why the flag exists, and why only the parser can set it. is +/// the element's concatenated descendant text, so a two-entry data set reads as the single +/// token "12" — keys discarded, values run together. The observation index cannot +/// detect that after the fact: neither this record nor the tag definition carries the +/// DataItem's representation, and no value-shape heuristic can work, because +/// concatenated entries are indistinguishable from a legitimate space-bearing EVENT such as +/// a Message reading "Coolant level low". The one reliable discriminator — +/// that the observation element had element children — exists only while the XML is still +/// XML. +/// +/// +/// Deliberately a neutral fact about the wire shape, not a status: mapping it to a +/// quality code is the observation index's job (it codes these +/// BadNotSupported rather than publishing concatenated noise as Good). Defaults to +/// false, the shape of every ordinary scalar observation. +/// +/// public sealed record MTConnectObservation( string DataItemId, string Value, - DateTime TimestampUtc); + DateTime TimestampUtc, + bool IsStructured = false); /// /// Recursive helpers over the device/component tree. diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs index 6f561050..6d2d7c51 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectProbeParser.cs @@ -1,4 +1,3 @@ -using System.Globalization; using System.Xml; using System.Xml.Linq; @@ -30,12 +29,18 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// literal <Component> element — so every element child of /// <Components> is a component. /// (3) The document is XML-namespaced and the namespace URI carries the MTConnect version -/// (urn:mtconnect.org:MTConnectDevices:1.3 … :2.0). Elements are therefore -/// matched on and the namespace is ignored entirely — nothing -/// version-specific is hard-coded, and a vendor-extension component in its own namespace -/// (whose standard child elements still inherit the default MTConnect namespace) is not -/// silently dropped. Attributes, by contrast, are read unqualified so a prefixed attribute -/// (e.g. xsi:type) can never be mistaken for a DataItem's type. +/// (urn:mtconnect.org:MTConnectDevices:1.3 … :2.0). Element matching and +/// attribute reading therefore go through , whose two rules +/// (match on local name, read attributes unqualified) are shared verbatim with +/// — see that type for why each rule exists. +/// +/// +/// One deliberate divergence from the streams parser: no BOM/whitespace trim here. +/// A /probe body arrives whole from HttpContent, so it begins exactly where +/// the Agent's document begins. A /sample chunk, by contrast, is lifted out of a +/// multipart frame and can carry a byte-order mark or the framing's own line break ahead of +/// the XML declaration, which XmlReader rejects — hence the trim there and not here. +/// This asymmetry is intentional, not an oversight. /// /// /// Failure posture. Anything that is not a well-formed MTConnect device model throws @@ -54,6 +59,9 @@ internal static class MTConnectProbeParser /// private const int MaxComponentDepth = 64; + /// How 's shared error messages name this document. + private const string Subject = "/probe response"; + /// Parses a /probe response held as a string. /// The raw MTConnectDevices XML document. /// @@ -106,12 +114,12 @@ internal static class MTConnectProbeParser var root = document.Root ?? throw new InvalidDataException("MTConnect /probe response has no root element."); - if (!IsNamed(root, "MTConnectDevices")) + if (!MTConnectXml.IsNamed(root, "MTConnectDevices")) { - throw new InvalidDataException(DescribeUnexpectedRoot(root)); + throw new InvalidDataException(MTConnectXml.DescribeUnexpectedRoot(root, "MTConnectDevices", Subject)); } - var devicesContainer = ChildrenNamed(root, "Devices").FirstOrDefault() + var devicesContainer = MTConnectXml.ChildrenNamed(root, "Devices").FirstOrDefault() ?? throw new InvalidDataException( "MTConnect /probe response has no element; it does not describe a device model."); @@ -127,15 +135,15 @@ internal static class MTConnectProbeParser private static MTConnectDevice ReadDevice(XElement element) => new( - RequiredAttribute(element, "id", "Device"), - OptionalAttribute(element, "name"), + MTConnectXml.RequiredAttribute(element, "id", "Device", Subject), + MTConnectXml.OptionalAttribute(element, "name"), ReadComponents(element, depth: 1), ReadDataItems(element)); private static MTConnectComponent ReadComponent(XElement element, int depth) => new( - RequiredAttribute(element, "id", $"Component <{element.Name.LocalName}>"), - OptionalAttribute(element, "name"), + MTConnectXml.RequiredAttribute(element, "id", $"Component <{element.Name.LocalName}>", Subject), + MTConnectXml.OptionalAttribute(element, "name"), ReadComponents(element, depth + 1), ReadDataItems(element)); @@ -151,104 +159,31 @@ internal static class MTConnectProbeParser $"MTConnect /probe response nests components more than {MaxComponentDepth} levels deep; refusing to recurse further."); } - return ChildrenNamed(container, "Components") + return MTConnectXml.ChildrenNamed(container, "Components") .SelectMany(components => components.Elements()) .Select(element => ReadComponent(element, depth)) .ToList(); } private static IReadOnlyList ReadDataItems(XElement container) => - ChildrenNamed(container, "DataItems") - .SelectMany(dataItems => ChildrenNamed(dataItems, "DataItem")) + MTConnectXml.ChildrenNamed(container, "DataItems") + .SelectMany(dataItems => MTConnectXml.ChildrenNamed(dataItems, "DataItem")) .Select(ReadDataItem) .ToList(); private static MTConnectDataItem ReadDataItem(XElement element) { - var id = RequiredAttribute(element, "id", "DataItem"); + var id = MTConnectXml.RequiredAttribute(element, "id", "DataItem", Subject); return new MTConnectDataItem( id, - OptionalAttribute(element, "name"), - RequiredAttribute(element, "category", $"DataItem '{id}'"), - RequiredAttribute(element, "type", $"DataItem '{id}'"), - OptionalAttribute(element, "subType"), - OptionalAttribute(element, "units"), - OptionalAttribute(element, "representation"), - OptionalIntAttribute(element, "sampleCount", id)); + MTConnectXml.OptionalAttribute(element, "name"), + MTConnectXml.RequiredAttribute(element, "category", $"DataItem '{id}'", Subject), + MTConnectXml.RequiredAttribute(element, "type", $"DataItem '{id}'", Subject), + MTConnectXml.OptionalAttribute(element, "subType"), + MTConnectXml.OptionalAttribute(element, "units"), + MTConnectXml.OptionalAttribute(element, "representation"), + MTConnectXml.OptionalIntAttribute(element, "sampleCount", $"DataItem '{id}'", Subject)); } - private static bool IsNamed(XElement element, string localName) => - string.Equals(element.Name.LocalName, localName, StringComparison.Ordinal); - - private static IEnumerable ChildrenNamed(XElement parent, string localName) => - parent.Elements().Where(child => IsNamed(child, localName)); - - private static string RequiredAttribute(XElement element, string name, string owner) - { - var value = OptionalAttribute(element, name); - - return value ?? throw new InvalidDataException( - $"MTConnect /probe response is malformed: {owner} has no '{name}' attribute."); - } - - /// - /// Reads an unqualified attribute, normalizing a present-but-empty value to null. - /// Only unqualified attributes are considered, so a namespace-prefixed attribute such as - /// xsi:type can never be read as a DataItem's type. - /// - private static string? OptionalAttribute(XElement element, string name) - { - var value = element.Attribute(name)?.Value; - - return string.IsNullOrWhiteSpace(value) ? null : value; - } - - private static int? OptionalIntAttribute(XElement element, string name, string dataItemId) - { - var raw = OptionalAttribute(element, name); - if (raw is null) - { - return null; - } - - if (!int.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value)) - { - throw new InvalidDataException( - $"MTConnect /probe response is malformed: DataItem '{dataItemId}' has a non-numeric '{name}' attribute ('{raw}')."); - } - - return value; - } - - /// - /// Builds the message for a root element that is not MTConnectDevices, lifting the - /// Agent's own error text when the response is an MTConnectError document (which - /// Agents return under HTTP 200 for, e.g., an unknown device name). - /// - private static string DescribeUnexpectedRoot(XElement root) - { - var message = - $"MTConnect /probe response root element is <{root.Name.LocalName}>, expected ."; - - if (!IsNamed(root, "MTConnectError")) - { - return message; - } - - var errors = root.Descendants() - .Where(e => IsNamed(e, "Error")) - .Select(e => - { - var code = OptionalAttribute(e, "errorCode"); - var text = e.Value.Trim(); - return code is null ? text : $"{code}: {text}"; - }) - .Where(text => text.Length > 0) - .ToList(); - - return errors.Count == 0 - ? message - : $"{message} The Agent reported: {string.Join("; ", errors)}"; - } } diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamExceptions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamExceptions.cs new file mode 100644 index 00000000..362dc16f --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamExceptions.cs @@ -0,0 +1,161 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// Base type for every way an Agent /sample long poll can stop delivering chunks for a +/// reason that is not the caller cancelling it. +/// +/// +/// +/// Why these are exceptions rather than a normal end of enumeration. +/// is contracted to yield indefinitely until +/// its token is cancelled, so a pump written to that contract has no reason to handle normal +/// completion — it would either drop the subscription silently or hot-loop reconnecting on +/// an enumerator that ends immediately. Returning normally would make "the data path +/// stopped" indistinguishable from "the data path is idle", which is the +/// quiet-successful-looking-termination shape that produced this repo's #485 defect. Every +/// non-cancellation end is therefore loud, typed, and carries the Agent URI. +/// +/// +/// Catch this base type to handle "the stream is over" uniformly; catch the two derived +/// types to distinguish a transient end (the Agent closed a connection — reconnect) +/// from a configuration error (the Agent will never stream to this request — retrying +/// is pointless until an operator changes something). +/// +/// +public abstract class MTConnectStreamException : Exception +{ + /// Creates the exception with no message. + protected MTConnectStreamException() + { + } + + /// Creates the exception with a message. + /// The operator-facing description. + protected MTConnectStreamException(string message) + : base(message) + { + } + + /// Creates the exception with a message and an inner cause. + /// The operator-facing description. + /// The underlying cause. + protected MTConnectStreamException(string message, Exception innerException) + : base(message, innerException) + { + } +} + +/// How an Agent /sample stream ended. +public enum MTConnectStreamEndReason +{ + /// + /// The transport closed mid-stream — the connection dropped, the Agent restarted, or a proxy + /// timed the connection out. Ordinary and transient: reconnect from the last + /// . + /// + ConnectionClosed = 0, + + /// + /// The Agent wrote the closing --boundary-- delimiter, ending the multipart response + /// deliberately. Usually means the request was bounded (an Agent honouring a count + /// that exhausted, or an administrative stop) rather than a fault. Also reconnectable, but + /// worth distinguishing in logs: a stream that keeps closing cleanly points at the request's + /// parameters, not at the network. + /// + ClosingBoundary = 1 +} + +/// +/// The Agent's /sample stream ended without the caller cancelling it. See +/// for why this is not a normal end of enumeration. +/// +public sealed class MTConnectStreamEndedException : MTConnectStreamException +{ + /// Creates the exception for a stream that ended for . + /// How the stream ended. + /// The /sample request URI whose stream ended. + /// How many chunks were yielded before the end. + public MTConnectStreamEndedException(MTConnectStreamEndReason reason, string agentUri, long chunksDelivered) + : base($"MTConnect /sample stream from '{agentUri}' ended after {chunksDelivered} chunk(s) without the caller cancelling it ({Describe(reason)}). The stream is contracted to run until cancelled, so this is reported rather than returned; resume from the last chunk's nextSequence.") + { + Reason = reason; + AgentUri = agentUri; + ChunksDelivered = chunksDelivered; + } + + /// Creates the exception with a message. + /// The operator-facing description. + public MTConnectStreamEndedException(string message) + : base(message) + { + AgentUri = string.Empty; + } + + /// Creates the exception with a message and an inner cause. + /// The operator-facing description. + /// The underlying cause. + public MTConnectStreamEndedException(string message, Exception innerException) + : base(message, innerException) + { + AgentUri = string.Empty; + } + + /// Creates the exception with no message. + public MTConnectStreamEndedException() + { + AgentUri = string.Empty; + } + + /// How the stream ended. + public MTConnectStreamEndReason Reason { get; } + + /// The /sample request URI whose stream ended. + public string AgentUri { get; } + + /// How many chunks were yielded before the stream ended. + public long ChunksDelivered { get; } + + private static string Describe(MTConnectStreamEndReason reason) => + reason switch + { + MTConnectStreamEndReason.ClosingBoundary => "the Agent wrote the closing multipart boundary", + _ => "the connection closed mid-stream" + }; +} + +/// +/// The Agent answered a /sample request with something that cannot be consumed as a +/// stream at all — a single non-multipart document, or a multipart/* response with no +/// boundary parameter to frame it by. +/// +/// +/// This is a configuration error, not a transient one, and is deliberately typed apart +/// from : reconnecting will produce the identical +/// response forever. The usual cause is an endpoint that is not an MTConnect Agent (a reverse +/// proxy, a load balancer's health page, or an Agent fronted by something that buffers and +/// collapses multipart/x-mixed-replace). Reported instead of yielding the one document +/// the Agent did return, because a single snapshot followed by a healthy-looking finished +/// stream is exactly how a dead data path disguises itself as a working one. +/// +public sealed class MTConnectStreamNotSupportedException : MTConnectStreamException +{ + /// Creates the exception with a message. + /// The operator-facing description. + public MTConnectStreamNotSupportedException(string message) + : base(message) + { + } + + /// Creates the exception with a message and an inner cause. + /// The operator-facing description. + /// The underlying cause. + public MTConnectStreamNotSupportedException(string message, Exception innerException) + : base(message, innerException) + { + } + + /// Creates the exception with no message. + public MTConnectStreamNotSupportedException() + { + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs index 9a9cfd21..20f0b642 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectStreamsParser.cs @@ -79,6 +79,12 @@ internal static class MTConnectStreamsParser private const string ConditionContainer = "Condition"; + /// How 's shared error messages name this document. + private const string Subject = "streams response"; + + /// How those messages name the header element. + private const string HeaderOwner = "the
"; + /// /// The three elements whose element children are observations. A ComponentStream may carry /// any subset of them, including none. @@ -105,7 +111,7 @@ internal static class MTConnectStreamsParser { // A chunk lifted out of a multipart frame can carry a byte-order mark or the framing's // trailing line break; XmlReader rejects either ahead of the XML declaration. - document = XDocument.Parse(xml.Trim('', ' ', '\t', '\r', '\n')); + document = XDocument.Parse(xml.Trim('\uFEFF', ' ', '\t', '\r', '\n')); } catch (XmlException ex) { @@ -140,19 +146,19 @@ internal static class MTConnectStreamsParser var root = document.Root ?? throw new InvalidDataException("MTConnect streams response has no root element."); - if (!IsNamed(root, "MTConnectStreams")) + if (!MTConnectXml.IsNamed(root, "MTConnectStreams")) { - throw new InvalidDataException(DescribeUnexpectedRoot(root)); + throw new InvalidDataException(MTConnectXml.DescribeUnexpectedRoot(root, "MTConnectStreams", Subject)); } - var header = ChildrenNamed(root, "Header").FirstOrDefault() + var header = MTConnectXml.ChildrenNamed(root, "Header").FirstOrDefault() ?? throw new InvalidDataException( "MTConnect streams response has no
; without it the sequence bookkeeping the sample pump runs on is unknowable."); return new MTConnectStreamsResult( - RequiredLongAttribute(header, "instanceId"), - RequiredLongAttribute(header, "nextSequence"), - RequiredLongAttribute(header, "firstSequence"), + MTConnectXml.RequiredLongAttribute(header, "instanceId", HeaderOwner, Subject), + MTConnectXml.RequiredLongAttribute(header, "nextSequence", HeaderOwner, Subject), + MTConnectXml.RequiredLongAttribute(header, "firstSequence", HeaderOwner, Subject), ReadObservations(root)); } @@ -168,7 +174,7 @@ internal static class MTConnectStreamsParser foreach (var container in root.Descendants().Where(IsObservationContainer)) { - var isCondition = IsNamed(container, ConditionContainer); + var isCondition = MTConnectXml.IsNamed(container, ConditionContainer); foreach (var element in container.Elements()) { observations.Add(ReadObservation(element, isCondition)); @@ -180,12 +186,20 @@ internal static class MTConnectStreamsParser private static MTConnectObservation ReadObservation(XElement element, bool isCondition) { - var dataItemId = RequiredAttribute(element, "dataItemId", $"Observation <{element.Name.LocalName}>"); + var dataItemId = MTConnectXml.RequiredAttribute(element, "dataItemId", $"Observation <{element.Name.LocalName}>", Subject); return new MTConnectObservation( dataItemId, isCondition ? ConditionState(element) : element.Value.Trim(), - ReadTimestampUtc(element, dataItemId)); + ReadTimestampUtc(element, dataItemId), + + // A DATA_SET/TABLE observation keeps its content in children, so the + // text read above concatenates them into nonsense ("12" for two entries). Flagged here + // because this is the only layer that can still see the distinction — see + // MTConnectObservation.IsStructured. Never true for a CONDITION: its value comes from + // the element NAME, so child content cannot corrupt it, and flagging one would throw + // away a perfectly well-determined state. + IsStructured: !isCondition && element.HasElements); } /// @@ -203,7 +217,7 @@ internal static class MTConnectStreamsParser private static DateTime ReadTimestampUtc(XElement element, string dataItemId) { - var raw = RequiredAttribute(element, "timestamp", $"Observation '{dataItemId}'"); + var raw = MTConnectXml.RequiredAttribute(element, "timestamp", $"Observation '{dataItemId}'", Subject); if (!DateTime.TryParse( raw, @@ -221,76 +235,6 @@ internal static class MTConnectStreamsParser } private static bool IsObservationContainer(XElement element) => - ObservationContainers.Any(name => IsNamed(element, name)); + ObservationContainers.Any(name => MTConnectXml.IsNamed(element, name)); - private static bool IsNamed(XElement element, string localName) => - string.Equals(element.Name.LocalName, localName, StringComparison.Ordinal); - - private static IEnumerable ChildrenNamed(XElement parent, string localName) => - parent.Elements().Where(child => IsNamed(child, localName)); - - private static string RequiredAttribute(XElement element, string name, string owner) - { - var value = OptionalAttribute(element, name); - - return value ?? throw new InvalidDataException( - $"MTConnect streams response is malformed: {owner} has no '{name}' attribute."); - } - - /// - /// Reads an unqualified attribute, normalizing a present-but-empty value to null. - /// Only unqualified attributes are considered, so a namespace-prefixed vendor attribute can - /// never be mistaken for an observation's identity or timestamp. - /// - private static string? OptionalAttribute(XElement element, string name) - { - var value = element.Attribute(name)?.Value; - - return string.IsNullOrWhiteSpace(value) ? null : value; - } - - private static long RequiredLongAttribute(XElement header, string name) - { - var raw = RequiredAttribute(header, name, "The
"); - - if (!long.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value)) - { - throw new InvalidDataException( - $"MTConnect streams response is malformed: the
has a non-numeric '{name}' attribute ('{raw}')."); - } - - return value; - } - - /// - /// Builds the message for a root element that is not MTConnectStreams, lifting the - /// Agent's own error text when the response is an MTConnectError document (which - /// Agents return under HTTP 200 for, e.g., a from outside the buffer). - /// - private static string DescribeUnexpectedRoot(XElement root) - { - var message = - $"MTConnect streams response root element is <{root.Name.LocalName}>, expected ."; - - if (!IsNamed(root, "MTConnectError")) - { - return message; - } - - var errors = root.Descendants() - .Where(e => IsNamed(e, "Error")) - .Select(e => - { - var code = OptionalAttribute(e, "errorCode"); - var text = e.Value.Trim(); - - return code is null ? text : $"{code}: {text}"; - }) - .Where(text => text.Length > 0) - .ToList(); - - return errors.Count == 0 - ? message - : $"{message} The Agent reported: {string.Join("; ", errors)}"; - } } diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectXml.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectXml.cs new file mode 100644 index 00000000..63c31262 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectXml.cs @@ -0,0 +1,146 @@ +using System.Globalization; +using System.Xml.Linq; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// The element/attribute reading rules shared by and +/// . Both documents are the same dialect of XML and must be +/// read by the same rules; keeping one copy is what stops the two parsers drifting apart on the +/// two rules below, either of which fails silently rather than loudly. +/// +/// +/// +/// Rule 1: elements are matched on , ignoring the +/// namespace. The namespace URI carries the MTConnect version +/// (urn:mtconnect.org:MTConnectDevices:1.3 … :2.0), so a fully-qualified match +/// would hard-code a version. It also silently drops vendor-extension elements declared in +/// their own namespace, whose standard children still inherit the default MTConnect +/// namespace — the browse tree simply comes back short, with no error anywhere. +/// +/// +/// Rule 2: attributes are read unqualified. A prefixed attribute +/// (xsi:type on a document root, a vendor's x:dataItemId) must never be +/// mistaken for the real type / dataItemId, which would invent a data item the +/// probe never declared. +/// +/// +internal static class MTConnectXml +{ + /// Does have the given local name, whatever its namespace? + /// The element to test. + /// The unqualified name to match. + public static bool IsNamed(XElement element, string localName) => + string.Equals(element.Name.LocalName, localName, StringComparison.Ordinal); + + /// Direct element children of with the given local name. + /// The element whose children to filter. + /// The unqualified name to match. + public static IEnumerable ChildrenNamed(XElement parent, string localName) => + parent.Elements().Where(child => IsNamed(child, localName)); + + /// + /// Reads an unqualified attribute, normalizing a present-but-empty value to null. + /// See the type-level remarks for why only unqualified attributes are considered. + /// + /// The element carrying the attribute. + /// The unqualified attribute name. + public static string? OptionalAttribute(XElement element, string name) + { + var value = element.Attribute(name)?.Value; + + return string.IsNullOrWhiteSpace(value) ? null : value; + } + + /// Reads a required unqualified attribute, or throws naming the owner and the attribute. + /// The element carrying the attribute. + /// The unqualified attribute name. + /// How to describe the element in the error message. + /// How to describe the document in the error message (e.g. "/probe response"). + /// The attribute is absent or blank. + public static string RequiredAttribute(XElement element, string name, string owner, string subject) + { + var value = OptionalAttribute(element, name); + + return value ?? throw new InvalidDataException( + $"MTConnect {subject} is malformed: {owner} has no '{name}' attribute."); + } + + /// Reads an optional unqualified attribute as an . + /// The element carrying the attribute. + /// The unqualified attribute name. + /// How to describe the element in the error message. + /// How to describe the document in the error message. + /// The attribute is present but not an integer. + public static int? OptionalIntAttribute(XElement element, string name, string owner, string subject) + { + var raw = OptionalAttribute(element, name); + if (raw is null) + { + return null; + } + + if (!int.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value)) + { + throw new InvalidDataException( + $"MTConnect {subject} is malformed: {owner} has a non-numeric '{name}' attribute ('{raw}')."); + } + + return value; + } + + /// Reads a required unqualified attribute as a . + /// The element carrying the attribute. + /// The unqualified attribute name. + /// How to describe the element in the error message. + /// How to describe the document in the error message. + /// The attribute is absent, blank, or not an integer. + public static long RequiredLongAttribute(XElement element, string name, string owner, string subject) + { + var raw = RequiredAttribute(element, name, owner, subject); + + if (!long.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value)) + { + throw new InvalidDataException( + $"MTConnect {subject} is malformed: {owner} has a non-numeric '{name}' attribute ('{raw}')."); + } + + return value; + } + + /// + /// Builds the error message for a document whose root is not what was expected, lifting the + /// Agent's own error text when the response is an MTConnectError document — which + /// Agents return under HTTP 200, so EnsureSuccessStatusCode never fires and + /// this is the only place the operator can learn what the Agent objected to. + /// + /// The document's actual root element. + /// The root element name the caller required. + /// How to describe the document in the error message. + public static string DescribeUnexpectedRoot(XElement root, string expectedRootName, string subject) + { + var message = + $"MTConnect {subject} root element is <{root.Name.LocalName}>, expected <{expectedRootName}>."; + + if (!IsNamed(root, "MTConnectError")) + { + return message; + } + + var errors = root.Descendants() + .Where(e => IsNamed(e, "Error")) + .Select(e => + { + var code = OptionalAttribute(e, "errorCode"); + var text = e.Value.Trim(); + + return code is null ? text : $"{code}: {text}"; + }) + .Where(text => text.Length > 0) + .ToList(); + + return errors.Count == 0 + ? message + : $"{message} The Agent reported: {string.Join("; ", errors)}"; + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs index 089aa83d..6df71fa6 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectStreamsParseTests.cs @@ -1,7 +1,9 @@ using System.Diagnostics; +using System.Globalization; using System.Net; using System.Net.Http.Headers; using System.Text; +using Microsoft.Extensions.Logging; using Shouldly; using Xunit; @@ -10,7 +12,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; /// /// Task 7 — the /current + /sample legs: /// turning an MTConnectStreams document into , the -/// ring-buffer sequence-gap signal (), and the +/// ring-buffer sequence-gap signal (), and the /// multipart/x-mixed-replace chunk framing + heartbeat watchdog on the long-poll leg. /// Every test runs against canned fixtures or a stubbed — no /// socket is opened anywhere in this file. @@ -179,7 +181,7 @@ public sealed class MTConnectStreamsParseTests { var chunk = MTConnectStreamsParser.Parse(File.ReadAllText(GapFixture)); - MTConnectAgentClient.IsSequenceGap(requestedFrom: 1, chunk).ShouldBeTrue(); + IMTConnectAgentClient.IsSequenceGap(expectedFrom: 1, chunk).ShouldBeTrue(); } [Fact] @@ -188,18 +190,18 @@ public sealed class MTConnectStreamsParseTests // The falsifiability leg: a helper hard-wired to true would pass the gap test alone. var contiguous = MTConnectStreamsParser.Parse(File.ReadAllText(SampleFixture)); - MTConnectAgentClient.IsSequenceGap(requestedFrom: 108, contiguous).ShouldBeFalse(); + IMTConnectAgentClient.IsSequenceGap(expectedFrom: 108, contiguous).ShouldBeFalse(); } [Theory] - [InlineData(4999L, true)] // requested older than the buffer holds — data was lost + [InlineData(4999L, true)] // expected older than the buffer holds — data was lost [InlineData(5000L, false)] // exactly the oldest retained sequence — nothing was lost [InlineData(5001L, false)] // the agent simply had nothing older to send - public void Sequence_gap_is_strictly_firstSequence_greater_than_requested_from(long requestedFrom, bool expected) + public void Sequence_gap_is_strictly_firstSequence_greater_than_expected_from(long expectedFrom, bool expected) { var chunk = MTConnectStreamsParser.Parse(File.ReadAllText(GapFixture)); - MTConnectAgentClient.IsSequenceGap(requestedFrom, chunk).ShouldBe(expected); + IMTConnectAgentClient.IsSequenceGap(expectedFrom, chunk).ShouldBe(expected); } // --------------------------------------------------------------- legal but degenerate documents @@ -499,11 +501,17 @@ public sealed class MTConnectStreamsParseTests var from = 108L; var gaps = new List(); - await foreach (var chunk in client.SampleAsync(from, Ct)) + await Should.ThrowAsync(async () => { - gaps.Add(MTConnectAgentClient.IsSequenceGap(from, chunk)); - from = chunk.NextSequence; - } + await foreach (var chunk in client.SampleAsync(from, Ct)) + { + // `from` is the RUNNING cursor, not the original argument — comparing every chunk + // against the opening `from` would report a gap on every chunk once the Agent's + // buffer rolls past it (I1). + gaps.Add(IMTConnectAgentClient.IsSequenceGap(from, chunk)); + from = chunk.NextSequence; + } + }); gaps.ShouldBe([false, true]); from.ShouldBe(5005); @@ -541,7 +549,7 @@ public sealed class MTConnectStreamsParseTests var chunks = await DrainAsync(client.SampleAsync(113, Ct)); chunks.ShouldHaveSingleItem().Observations.ShouldBeEmpty(); - MTConnectAgentClient.IsSequenceGap(113, chunks[0]).ShouldBeFalse(); + IMTConnectAgentClient.IsSequenceGap(113, chunks[0]).ShouldBeFalse(); } [Fact(Timeout = 30_000)] @@ -623,14 +631,107 @@ public sealed class MTConnectStreamsParseTests } [Fact] - public async Task SampleAsync_ends_cleanly_when_the_agent_closes_the_stream() + public async Task SampleAsync_reports_the_closing_boundary_rather_than_ending_quietly() { + // C1. The seam contracts SampleAsync to run until cancelled, so a pump has no reason to + // handle normal completion: returning here would either drop the subscription silently or + // hot-loop reconnecting on an enumerator that ends immediately (#485). var handler = StubHandler.RespondingWithMultipart("bnd", await File.ReadAllTextAsync(SampleFixture, Ct)); using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); - var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + var chunks = new List(); + var ex = await Should.ThrowAsync(async () => + { + await foreach (var chunk in client.SampleAsync(108, Ct)) + { + chunks.Add(chunk); + } + }); chunks.ShouldHaveSingleItem(); + ex.Reason.ShouldBe(MTConnectStreamEndReason.ClosingBoundary); + ex.ChunksDelivered.ShouldBe(1); + ex.AgentUri.ShouldContain("/sample?from=108"); + } + + [Fact] + public async Task SampleAsync_reports_a_connection_that_drops_mid_stream() + { + // No closing boundary — the body simply stops. Distinguished from a deliberate close so an + // operator can tell a flaky network from an Agent honouring a bounded request. + var truncated = StubHandler.MultipartBody( + "bnd", withContentLength: true, closing: false, await File.ReadAllTextAsync(SampleFixture, Ct)); + var handler = StubHandler.RespondingWithBody("bnd", truncated); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync( + async () => await EnumerateAsync(client.SampleAsync(108, Ct))); + + ex.Reason.ShouldBe(MTConnectStreamEndReason.ConnectionClosed); + ex.ChunksDelivered.ShouldBe(1); + } + + [Fact] + public async Task SampleAsync_reports_a_non_multipart_answer_as_a_configuration_error() + { + // The worst C1 shape: the Agent ignored the streaming request entirely. Yielding the one + // snapshot it did return and finishing would report a dead subscription as a healthy one. + var handler = StubHandler.RespondingWith(await File.ReadAllTextAsync(CurrentFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync( + async () => await EnumerateAsync(client.SampleAsync(108, Ct))); + + ex.Message.ShouldContain("multipart/x-mixed-replace"); + ex.Message.ShouldContain("text/xml"); + } + + [Fact] + public async Task SampleAsync_yields_nothing_at_all_from_a_non_multipart_answer() + { + // Explicitly pinned: the parsed document must NOT reach the caller. One snapshot plus a + // finished stream is precisely how a dead data path disguises itself as a working one. + var handler = StubHandler.RespondingWith(await File.ReadAllTextAsync(CurrentFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = new List(); + await Should.ThrowAsync(async () => + { + await foreach (var chunk in client.SampleAsync(108, Ct)) + { + chunks.Add(chunk); + } + }); + + chunks.ShouldBeEmpty(); + } + + [Fact] + public async Task SampleAsync_fails_fast_when_a_multipart_response_carries_no_boundary_parameter() + { + // M2. Falling through to whole-body reading would surface only as a watchdog TimeoutException + // much later, pointing an operator at the network instead of at the malformed header. + var handler = StubHandler.RespondingWithBoundarylessMultipart(); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ex = await Should.ThrowAsync( + async () => await EnumerateAsync(client.SampleAsync(1, Ct))); + + ex.Message.ShouldContain("boundary"); + } + + [Fact] + public async Task A_stream_end_and_a_stream_misconfiguration_share_one_catchable_base_type() + { + // A pump that only wants "the stream is over" should not have to name both. + var handler = StubHandler.RespondingWithMultipart("bnd", await File.ReadAllTextAsync(SampleFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var ended = await Should.ThrowAsync( + async () => await EnumerateAsync(client.SampleAsync(108, Ct))); + + ended.ShouldBeAssignableTo(); + typeof(MTConnectStreamException).IsAssignableFrom(typeof(MTConnectStreamNotSupportedException)).ShouldBeTrue(); } [Fact] @@ -644,6 +745,255 @@ public sealed class MTConnectStreamsParseTests handler.RequestCount.ShouldBe(0); } + // ------------------------------------------- I2: framing when the transport splits every read + + /// + /// Serving the whole body from one buffer completes every frame in a single + /// ReadAsync, so the split-boundary path, the split-header path and the multi-fill + /// loop are never entered — on the one behaviour whose headline risk is framing. These drive + /// the same fixtures through a transport that hands back only N bytes at a time; at N = 1 + /// every delimiter, every header line and every document is split maximally. + /// + [Theory] + [InlineData(1)] + [InlineData(3)] + [InlineData(7)] + public async Task SampleAsync_frames_correctly_when_the_transport_splits_every_boundary_and_header(int bytesPerRead) + { + var handler = StubHandler.RespondingWithChunkedMultipart( + "--------bnd", + bytesPerRead, + withContentLength: true, + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.Select(c => c.FirstSequence).ShouldBe([108L, 5000L]); + chunks.Select(c => c.NextSequence).ShouldBe([113L, 5005L]); + chunks[0].Observations.Count.ShouldBe(5); + } + + [Theory] + [InlineData(1)] + [InlineData(3)] + [InlineData(7)] + public async Task SampleAsync_boundary_scan_framing_survives_a_split_transport_too(int bytesPerRead) + { + var handler = StubHandler.RespondingWithChunkedMultipart( + "bnd", + bytesPerRead, + withContentLength: false, + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(108, Ct)); + + chunks.Select(c => c.FirstSequence).ShouldBe([108L, 5000L]); + } + + [Fact] + public async Task SampleAsync_grows_its_buffer_for_a_part_larger_than_the_initial_read_buffer() + { + // Forces Array.Resize + a Consume across the grown buffer, which a 2 KB fixture never does. + var large = LargeStreamsDocument(observationCount: 900); + Encoding.UTF8.GetByteCount(large).ShouldBeGreaterThan(16 * 1024); + + var handler = StubHandler.RespondingWithChunkedMultipart( + "bnd", bytesPerRead: 1024, withContentLength: true, large, large); + using var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + var chunks = await DrainAsync(client.SampleAsync(1, Ct)); + + chunks.Count.ShouldBe(2); + chunks[0].Observations.Count.ShouldBe(900); + chunks[1].Observations.Count.ShouldBe(900); + } + + // ------------------------------------------------------- I3: disposal during a live enumeration + + [Fact(Timeout = 30_000)] + public async Task Disposing_the_client_mid_enumeration_surfaces_as_cancellation() + { + // Task 9 re-initializes by disposing and rebuilding this client, which can happen while a + // pump is still enumerating. An unclassified ObjectDisposedException/IOException from the + // socket is something no caller is written to expect. + var prefix = StubHandler.MultipartBody( + "bnd", withContentLength: true, closing: false, await File.ReadAllTextAsync(SampleFixture, Ct)); + var handler = StubHandler.RespondingWithStallingStream("bnd", prefix); + var client = NewClient(handler, new MTConnectDriverOptions + { + AgentUri = "http://agent:5000", + RequestTimeoutMs = 5000, + HeartbeatMs = 20_000 + }); + + await using var chunks = client.SampleAsync(108, Ct).GetAsyncEnumerator(Ct); + (await chunks.MoveNextAsync()).ShouldBeTrue(); + + var pending = chunks.MoveNextAsync().AsTask(); + client.Dispose(); + + var ex = await Should.ThrowAsync(async () => await pending); + ex.ShouldNotBeOfType(); + } + + [Fact] + public async Task Using_a_disposed_client_throws_object_disposed_rather_than_dialling() + { + var handler = StubHandler.Hanging(); + var client = NewClient(handler, new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + client.Dispose(); + + await Should.ThrowAsync(async () => await client.CurrentAsync(Ct)); + await Should.ThrowAsync( + async () => await EnumerateAsync(client.SampleAsync(1, Ct))); + handler.RequestCount.ShouldBe(0); + } + + [Fact] + public void Dispose_is_idempotent() + { + var client = NewClient(StubHandler.Hanging(), new MTConnectDriverOptions { AgentUri = "http://agent:5000" }); + + client.Dispose(); + + Should.NotThrow(client.Dispose); + } + + // ------------------------------------------- I4: the degraded framing mode is operator-visible + + [Fact] + public async Task A_part_without_content_length_warns_once_that_framing_is_degraded() + { + var logger = new RecordingLogger(); + var handler = StubHandler.RespondingWithMultipart( + "bnd", + withContentLength: false, + await File.ReadAllTextAsync(SampleFixture, Ct), + await File.ReadAllTextAsync(GapFixture, Ct)); + using var client = new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler, logger); + + await DrainAsync(client.SampleAsync(108, Ct)); + + // Once, not once per part — a per-chunk warning on a long poll is its own outage. + logger.Warnings.Count.ShouldBe(1); + logger.Warnings[0].ShouldContain("Content-length"); + logger.Warnings[0].ShouldContain("http://agent:5000/sample"); + } + + [Fact] + public async Task The_normal_content_length_path_warns_about_nothing() + { + var logger = new RecordingLogger(); + var handler = StubHandler.RespondingWithMultipart("bnd", await File.ReadAllTextAsync(SampleFixture, Ct)); + using var client = new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000" }, handler, logger); + + await DrainAsync(client.SampleAsync(108, Ct)); + + logger.Warnings.ShouldBeEmpty(); + } + + // ---------------------------------------- I5: operator-authorable options that reach the wire + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void Ctor_rejects_a_non_positive_heartbeat(int heartbeatMs) + { + // heartbeat=0 goes on the query string, and an Agent answers it with an MTConnectError + // under HTTP 200 — surfacing as a parse failure that never names the offending key. + var ex = Should.Throw(() => new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000", HeartbeatMs = heartbeatMs })); + + ex.Message.ShouldContain(nameof(MTConnectDriverOptions.HeartbeatMs)); + } + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void Ctor_rejects_a_non_positive_sample_interval(int sampleIntervalMs) + { + var ex = Should.Throw(() => new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000", SampleIntervalMs = sampleIntervalMs })); + + ex.Message.ShouldContain(nameof(MTConnectDriverOptions.SampleIntervalMs)); + } + + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void Ctor_rejects_a_non_positive_sample_count(int sampleCount) + { + var ex = Should.Throw(() => new MTConnectAgentClient( + new MTConnectDriverOptions { AgentUri = "http://agent:5000", SampleCount = sampleCount })); + + ex.Message.ShouldContain(nameof(MTConnectDriverOptions.SampleCount)); + } + + // ------------------------------------------- structured (DATA_SET / TABLE) observation signal + + [Fact] + public void An_observation_whose_content_is_child_entries_is_flagged_structured() + { + // children concatenate to "12" through element.Value — keys gone, values run + // together. The index codes this BadNotSupported rather than publishing noise as Good. + var result = MTConnectStreamsParser.Parse(StreamsDocument( + """ + + 12 + + """)); + + var observation = result.Observations.ShouldHaveSingleItem(); + observation.IsStructured.ShouldBeTrue(); + observation.DataItemId.ShouldBe("ds"); + } + + [Fact] + public void A_plain_text_observation_containing_spaces_is_NOT_flagged_structured() + { + // The load-bearing case: this proves the signal is "the element had element children", not + // a whitespace heuristic. A Message reading "Coolant level low" is perfectly valid data and + // is textually indistinguishable from concatenated entries. + var result = MTConnectStreamsParser.Parse(StreamsDocument( + """Coolant level low""")); + + var observation = result.Observations.ShouldHaveSingleItem(); + observation.IsStructured.ShouldBeFalse(); + observation.Value.ShouldBe("Coolant level low"); + } + + [Fact] + public void Every_observation_in_the_committed_fixtures_is_unstructured() + { + // Includes the space-separated TIME_SERIES vector, which is text and must stay unflagged. + var current = MTConnectStreamsParser.Parse(File.ReadAllText(CurrentFixture)); + + current.Observations.ShouldAllBe(o => !o.IsStructured); + } + + [Fact] + public void A_condition_is_never_flagged_structured_even_with_child_content() + { + // A condition's value comes from the element NAME, so child content cannot corrupt it. + // Flagging one would throw away a perfectly well-determined state. + var result = MTConnectStreamsParser.Parse(StreamsDocument( + """ + + vendor extension + + """)); + + var observation = result.Observations.ShouldHaveSingleItem(); + observation.IsStructured.ShouldBeFalse(); + observation.Value.ShouldBe("Fault"); + } + // ------------------------------------------------------------------------------------ helpers private static CancellationToken Ct => TestContext.Current.CancellationToken; @@ -654,14 +1004,51 @@ public sealed class MTConnectStreamsParseTests private static async Task> DrainAsync(IAsyncEnumerable source) { var results = new List(); - await foreach (var item in source) + try { - results.Add(item); + await foreach (var item in source) + { + results.Add(item); + } + } + catch (MTConnectStreamEndedException) + { + // Expected under the C1 contract: every non-cancellation end of a /sample stream is + // loud. Tests that care about HOW it ended assert on the exception directly. } return results; } + /// Enumerates to completion, discarding chunks and letting every exception escape. + private static async Task EnumerateAsync(IAsyncEnumerable source) + { + await foreach (var _ in source) + { + // Drained for effect only. + } + } + + /// A streams document big enough to force the frame reader to grow its buffer. + private static string LargeStreamsDocument(int observationCount) + { + var samples = new StringBuilder(); + for (var i = 0; i < observationCount; i++) + { + samples.Append(CultureInfo.InvariantCulture, + $"""{i}.5"""); + } + + return $""" + +
+ + {samples} + + + """; + } + private static Dictionary ParseCurrent() => ParseFixture(CurrentFixture); private static Dictionary ParseFixture(string path) => @@ -717,6 +1104,35 @@ public sealed class MTConnectStreamsParseTests public static StubHandler RespondingWithStallingStream(string boundary, byte[] prefix) => new(() => Multipart(boundary, new StallingContent(prefix)), hang: false); + /// Serves an already-built multipart body verbatim (e.g. one with no closing delimiter). + public static StubHandler RespondingWithBody(string boundary, byte[] body) => + new(() => Multipart(boundary, new ByteArrayContent(body)), hang: false); + + /// + /// Serves a multipart body through a transport that hands back at most + /// bytes per ReadAsync, so boundaries, part + /// headers and documents are all split across reads. + /// + public static StubHandler RespondingWithChunkedMultipart( + string boundary, int bytesPerRead, bool withContentLength, params string[] parts) + { + var body = MultipartBody(boundary, withContentLength, closing: true, parts); + + return new StubHandler(() => Multipart(boundary, new ChunkedContent(body, bytesPerRead)), hang: false); + } + + /// A response that claims multipart but omits the boundary parameter. + public static StubHandler RespondingWithBoundarylessMultipart() => + new( + () => + { + var content = new ByteArrayContent([]); + content.Headers.ContentType = new MediaTypeHeaderValue("multipart/x-mixed-replace"); + + return new HttpResponseMessage(HttpStatusCode.OK) { Content = content }; + }, + hang: false); + /// Builds a multipart/x-mixed-replace body exactly as an Agent writes one. public static byte[] MultipartBody(string boundary, bool withContentLength, bool closing, params string[] parts) { @@ -769,6 +1185,97 @@ public sealed class MTConnectStreamsParseTests } } + /// Hands back at most bytesPerRead bytes per read — the split-transport double. + private sealed class ChunkedContent(byte[] body, int bytesPerRead) : HttpContent + { + protected override Task CreateContentReadStreamAsync() => + Task.FromResult(new ChunkedStream(body, bytesPerRead)); + + protected override Task SerializeToStreamAsync(Stream stream, TransportContext? context) => + stream.WriteAsync(body, 0, body.Length); + + protected override bool TryComputeLength(out long length) + { + length = body.Length; + + return true; + } + } + + private sealed class ChunkedStream(byte[] body, int bytesPerRead) : Stream + { + private int _position; + + public override bool CanRead => true; + + public override bool CanSeek => false; + + public override bool CanWrite => false; + + public override long Length => body.Length; + + public override long Position + { + get => _position; + set => throw new NotSupportedException(); + } + + public override ValueTask ReadAsync(Memory buffer, CancellationToken cancellationToken = default) + { + cancellationToken.ThrowIfCancellationRequested(); + + if (_position >= body.Length) + { + return ValueTask.FromResult(0); + } + + var count = Math.Min(Math.Min(bytesPerRead, buffer.Length), body.Length - _position); + body.AsMemory(_position, count).CopyTo(buffer); + _position += count; + + return ValueTask.FromResult(count); + } + + public override Task ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken) => + ReadAsync(buffer.AsMemory(offset, count), cancellationToken).AsTask(); + + public override void Flush() + { + } + + public override int Read(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + + public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); + + public override void SetLength(long value) => throw new NotSupportedException(); + + public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException(); + } + + /// Captures warning-level log lines so the degraded-framing notice can be asserted. + private sealed class RecordingLogger : ILogger + { + public List Warnings { get; } = []; + + public IDisposable? BeginScope(TState state) + where TState : notnull => null; + + public bool IsEnabled(LogLevel logLevel) => true; + + public void Log( + LogLevel logLevel, + EventId eventId, + TState state, + Exception? exception, + Func formatter) + { + if (logLevel == LogLevel.Warning) + { + Warnings.Add(formatter(state, exception)); + } + } + } + /// Serves , then never completes another read. private sealed class StallingContent(byte[] prefix) : HttpContent { -- 2.52.0 From 908bd7ba4ae3b5973640f24be0be6e87753ecb9a Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 15:09:42 -0400 Subject: [PATCH 14/34] fix(mtconnect): deterministic concurrency test + DATA_SET -> BadNotSupported MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The concurrency test asserted a post-Apply post-condition from inside the race window: dev1_pos is authored, so a reader beating the first Apply correctly observes BadWaitingForInitialData, which the test's two-member "legal" set omitted. Consistently red (10/10) rather than intermittent. Rewritten to assert only properties that hold at every instant — the whole (status, value, source timestamp) triple against the three legal snapshots, which is the real torn-snapshot check — plus a read counter guarding a vacuous pass and one deterministic closing Apply for an exact end state. The index is unchanged here; BadWaitingForInitialData is a designed state. Also codes structured (DATA_SET / TABLE) observations BadNotSupported now that the parser supplies MTConnectObservation.IsStructured, so a two-entry data set no longer surfaces as the Good string "12". --- .../MTConnectObservationIndex.cs | 37 +++-- .../MTConnectObservationIndexTests.cs | 149 ++++++++++++++++-- 2 files changed, 156 insertions(+), 30 deletions(-) diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs index 29703f0f..c68a870c 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectObservationIndex.cs @@ -48,20 +48,18 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// would serialize the pump against every read for no semantic gain. /// /// -/// KNOWN LIMITATION — DATA_SET / TABLE observations. Their real content -/// is <Entry key=…> child elements, but the parser reads an observation's value -/// with XElement.Value, which concatenates all descendant text. Such an observation -/// therefore reaches this index as a run-together string with the keys discarded, and — -/// because the type inference correctly demotes DATA_SET/TABLE to -/// — it coerces "successfully" into a Good-coded -/// snapshot carrying noise. This index cannot detect the shape: neither -/// nor carries the -/// DataItem's representation, and no value-shape heuristic can separate a -/// concatenated entry list from a legitimate space-bearing EVENT (a Message reads -/// "Coolant level low"). The discriminating signal — the observation element having child -/// elements — exists only in , so the fix belongs there -/// (flag structured observations at parse time; this index then codes them -/// BadNotSupported alongside the TIME_SERIES deferral below). +/// Shapes this build does not represent are coded BadNotSupported with a null +/// value rather than approximated: a TIME_SERIES vector (array materialization is +/// deferred to P1.5) and a DATA_SET / TABLE observation. The latter carries its +/// real content in <Entry key=…> child elements, which reach a text-valued +/// observation concatenated with the keys discarded — a two-entry set reads as the single +/// token "12" — and because the type inference correctly demotes those +/// representations to , coercion would otherwise +/// "succeed" into a Good-coded snapshot carrying noise. This index cannot detect that shape +/// itself (no value heuristic can separate concatenated entries from a legitimate +/// space-bearing EVENT such as a Message reading "Coolant level low"), so it relies on +/// , which +/// sets while the XML is still XML. /// /// internal sealed class MTConnectObservationIndex @@ -257,6 +255,17 @@ internal sealed class MTConnectObservationIndex return new DataValueSnapshot(null, BadNoCommunication, sourceTimestamp, serverTimestamp); } + // DATA_SET / TABLE: the Agent carried the content in child elements, so + // Value is their concatenated text with the keys discarded (a two-entry set reads "12"). + // Publishing that as a Good string would be noise an operator would trust. Checked before + // the tag lookup because it is a fact about the wire shape, true whether or not the data + // item is authored — and after the sentinel, so an unavailable data set still reports the + // truthful no-comms. + if (observation.IsStructured) + { + return new DataValueSnapshot(null, BadNotSupported, sourceTimestamp, serverTimestamp); + } + _tags.TryGetValue(observation.DataItemId, out var tag); // TIME_SERIES vectors arrive as one space-separated string. P1 does not materialize OPC UA diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs index 205dec2a..253a491f 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectObservationIndexTests.cs @@ -47,7 +47,8 @@ public sealed class MTConnectObservationIndexTests MTConnectStreamsParser.Parse(File.ReadAllText(path)); /// Wraps one synthetic observation in the minimal legal streams result. - private static MTConnectStreamsResult Synthetic(string dataItemId, string value, DateTime? timestampUtc = null) => + private static MTConnectStreamsResult Synthetic( + string dataItemId, string value, DateTime? timestampUtc = null, bool isStructured = false) => new( InstanceId: 1L, NextSequence: 2L, @@ -55,7 +56,8 @@ public sealed class MTConnectObservationIndexTests Observations: [new MTConnectObservation( dataItemId, value, - timestampUtc ?? new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc))]); + timestampUtc ?? new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc), + isStructured)]); private static MTConnectTagDefinition Tag(string id, DriverDataType type) => new(id, type); @@ -475,6 +477,92 @@ public sealed class MTConnectObservationIndexTests snap.Value.ShouldBeOfType().ShouldBe("1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0"); } + // --------------------------------------------------------------------- DATA_SET / TABLE + + /// + /// A structured (DATA_SET / TABLE) observation's real content is its <Entry> + /// children, which reach the index concatenated with the keys discarded. It is coded + /// BadNotSupported rather than published as that noise — regardless of whether the data item + /// is authored, because the shape is a fact about the wire, not about the tag. + /// + [Fact] + public void Structured_observation_reports_BadNotSupported_when_unauthored() + { + var idx = new MTConnectObservationIndex(); + idx.Apply(Synthetic("ds1", "12", isStructured: true)); + + var snap = idx.Get("ds1"); + + snap.StatusCode.ShouldBe(BadNotSupported); + snap.Value.ShouldBeNull(); + } + + /// The String authoring the type inference gives a DATA_SET must not smuggle the noise through. + [Fact] + public void Structured_observation_is_never_published_as_concatenated_text() + { + var idx = new MTConnectObservationIndex([Tag("ds1", DriverDataType.String)]); + idx.Apply(Synthetic("ds1", "12", isStructured: true)); + + var snap = idx.Get("ds1"); + + snap.Value.ShouldNotBe("12"); + snap.Value.ShouldBeNull(); + snap.StatusCode.ShouldBe(BadNotSupported); + } + + /// No-comms outranks the unsupported shape, exactly as it does for a TIME_SERIES array. + [Fact] + public void Unavailable_wins_over_the_structured_shape() + { + var idx = new MTConnectObservationIndex([Tag("ds1", DriverDataType.String)]); + idx.Apply(Synthetic("ds1", "UNAVAILABLE", isStructured: true)); + + idx.Get("ds1").StatusCode.ShouldBe(BadNoCommunication); + } + + /// + /// End-to-end through the real parser: the entry list that concatenates to the single token + /// "12" is caught by the parser's structured flag and never surfaces as a Good value. This is + /// the leg that would regress silently if the parser stopped setting the flag. + /// + [Fact] + public void A_parsed_data_set_observation_never_surfaces_as_a_good_value() + { + const string xml = """ + +
+ + + 1 + 2 + + + + """; + + var idx = new MTConnectObservationIndex([Tag("ds1", DriverDataType.String)]); + idx.Apply(MTConnectStreamsParser.Parse(xml)); + + var snap = idx.Get("ds1"); + + snap.StatusCode.ShouldBe(BadNotSupported); + snap.Value.ShouldBeNull(); + } + + /// An ordinary text observation is not structured — the flag must not be over-applied. + [Fact] + public void An_ordinary_observation_is_not_treated_as_structured() + { + var idx = new MTConnectObservationIndex([Tag("dev1_program", DriverDataType.String)]); + idx.Apply(ParseFixture(CurrentFixture)); + + var snap = idx.Get("dev1_program"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBe("O1234"); + } + // -------------------------------------------------------------------------------- ordering /// Within one Apply, the later observation for a dataItemId wins (Agent order is ascending sequence). @@ -652,8 +740,23 @@ public sealed class MTConnectObservationIndexTests /// /// /current (Task 10) and the /sample pump (Task 11) write concurrently while OPC UA reads - /// snapshot. Per-key atomicity is the contract: no throw, no torn snapshot (a Good value - /// paired with a Bad code), and every read observes one whole snapshot. + /// snapshot. Three properties, each stated so it holds at every instant rather than + /// only once the race settles: no call throws; no read observes a torn snapshot; and the + /// state after a deterministic closing Apply is exact. + /// + /// Torn-snapshot check. The whole (status, value, source timestamp) triple + /// is matched against the closed set of snapshots dev1_pos may legally present — + /// never a field in isolation. A Good code beside the unavailable document's timestamp, + /// or beside a null value, matches no member and fails. + /// + /// + /// The legal set has THREE members, not two. dev1_pos is authored, so a + /// reader that beats the first Apply correctly observes BadWaitingForInitialData — + /// the designed "authored, not yet reported" state. An earlier version of this test + /// asserted a post-Apply post-condition from inside the race window and failed whenever + /// a reader got there first: it was asserting something the index never promised, and it + /// verified nothing its name claimed. + /// /// [Fact] public async Task Concurrent_applies_and_reads_stay_consistent_and_never_throw() @@ -662,14 +765,24 @@ public sealed class MTConnectObservationIndexTests var current = ParseFixture(CurrentFixture); var unavailable = ParseFixture(UnavailableFixture); + var goodAt = new DateTime(2026, 7, 24, 12, 0, 0, 200, DateTimeKind.Utc); + (uint Status, object? Value, DateTime? Source)[] legalSnapshots = + [ + (BadWaitingForInitialData, null, null), + (Good, 123.4567, goodAt), + (BadNoCommunication, null, new DateTime(2026, 7, 24, 12, 10, 0, 100, DateTimeKind.Utc)), + ]; + + var reads = 0L; using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2)); + var writers = Enumerable.Range(0, 4).Select(i => Task.Run(() => { while (!cts.IsCancellationRequested) { idx.Apply(i % 2 == 0 ? current : unavailable); } - })); + })).ToArray(); var readers = Enumerable.Range(0, 4).Select(_ => Task.Run(() => { @@ -677,21 +790,25 @@ public sealed class MTConnectObservationIndexTests { var snap = idx.Get("dev1_pos"); - // Exactly two snapshots are legal for this tag; anything else is a tear. - if (snap.StatusCode == 0u) - { - snap.Value.ShouldBe(123.4567); - } - else - { - snap.StatusCode.ShouldBe(BadNoCommunication); - snap.Value.ShouldBeNull(); - } + legalSnapshots.ShouldContain((snap.StatusCode, snap.Value, snap.SourceTimestampUtc)); + Interlocked.Increment(ref reads); } - })); + })).ToArray(); await Task.WhenAll(writers.Concat(readers)); + // Guards a vacuous pass: a reader loop that never ran would have asserted nothing. + Interlocked.Read(ref reads).ShouldBeGreaterThan(0L); + + // The racing writers alternate documents, so the state DURING the race is unknowable by + // construction — pinning a specific one there is exactly the defect this test used to have. + // One deterministic apply, after every writer has stopped, gives an exact end state. + idx.Apply(current); + + var final = idx.Get("dev1_pos"); + final.StatusCode.ShouldBe(Good); + final.Value.ShouldBe(123.4567); + final.SourceTimestampUtc.ShouldBe(goodAt); idx.Count.ShouldBe(AllFixtureDataItemIds.Length); } -- 2.52.0 From 4fe0af36933a55d3943d723e69ad5d98528f8351 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 15:26:34 -0400 Subject: [PATCH 15/34] feat(mtconnect): driver shell IDriver lifecycle over agent-client seam (Task 9) MTConnectDriver implements IDriver only: connection-free ctor, initialize (deadline-bounded /probe + /current, caching the device model, the agent instanceId, and a primed observation index), reinitialize, shutdown, health, footprint, and cache flush. Decisions worth knowing: - The driverConfigJson argument is HONOURED, not ignored. DriverInstanceActor delivers a config change only via ReinitializeAsync(json) on the live instance, so a driver reading only its ctor options would turn every MTConnect config edit into a silent no-op that still sealed the deployment green. ParseOptions lives on the driver; Task 15's factory must delegate to it rather than deserialize a second DTO. - /probe ok but priming /current failing is Faulted, not Healthy-with-no-data: the index IS the read surface and the /sample cursor comes from /current. - ReinitializeAsync rebuilds the client whenever ANY option the client reads at construction changed - AgentUri, DeviceName, RequestTimeoutMs, HeartbeatMs, SampleIntervalMs, SampleCount - not just the first two. The timing knobs are baked into the deadlines, the watchdog window, and the /sample query string. - An unparseable config faults a cold start but leaves a RUNNING driver serving its previous valid configuration (the deployment fails instead). - IMTConnectAgentClient now extends IDisposable rather than the driver doing `as IDisposable`, which would silently no-op against a non-disposable fake and leak an HttpClient + socket handler per re-deploy. Sync, not async: every resource behind the seam is sync-disposable, and the async unwind belongs to the SampleAsync enumerator the pump owns. - Flush drops the probe/browse cache and keeps the index; every model consumer goes through GetOrFetchProbeModelAsync so a flush cannot wedge browse. Tests: CannedAgentClient (shared, deterministic - channel-backed scripted /sample with no timers, call counts, disposal tracking, failure injection) + 27 lifecycle tests. 308/308 in the project. --- .../IMTConnectAgentClient.cs | 25 +- .../MTConnectDriver.cs | 729 ++++++++++++++++++ .../CannedAgentClient.cs | 202 +++++ .../MTConnectDriverLifecycleTests.cs | 548 +++++++++++++ 4 files changed, 1503 insertions(+), 1 deletion(-) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs index 86b9c98d..1d41561e 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs @@ -9,6 +9,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// canned probe/current/sample XML, with no sockets involved. ///
/// +/// /// The production implementation, , carries no backend NuGet /// at all: it is an over the Agent's three request paths, feeding /// hand-rolled System.Xml.Linq parsers (MTConnectProbeParser / @@ -19,8 +20,30 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// needs to implement these three instance members; is a static /// protocol rule that lives here, rather than on the implementation, so a consumer written /// against this seam never has to reference the concrete client. +/// +/// +/// Why the seam is rather than +/// . The driver obtains its client through a +/// Func<MTConnectDriverOptions, IMTConnectAgentClient> and must release it on +/// ShutdownAsync and on every endpoint-changing ReinitializeAsync. Declaring +/// disposal here — instead of the driver doing as IDisposable + a null-conditional +/// call — is deliberate: that idiom silently no-ops against any future non-disposable +/// implementation, and the thing it would leak is an and its socket +/// handler, once per re-deploy, for the life of the process. +/// +/// +/// Synchronous disposal is the honest shape because every resource behind this seam is +/// synchronously disposable — two s and a +/// . An here would be a +/// synchronous method wearing a ValueTask, and it would oblige every canned-XML fake +/// to grow one too. The async unwind that a streaming client does need is already +/// covered elsewhere and is not this method's job: the enumerator +/// is itself and is owned by the pump that enumerates it, +/// while on the client converts any read still in flight +/// into an ordinary . +/// /// -public interface IMTConnectAgentClient +public interface IMTConnectAgentClient : IDisposable { /// /// Issues an Agent /probe request and returns the parsed device hierarchy. Called diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs new file mode 100644 index 00000000..abe20dd0 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -0,0 +1,729 @@ +using System.Globalization; +using System.Text.Json; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// MTConnect Agent implementation of — the lifecycle half. Everything the +/// driver serves sits behind one , so the whole class is +/// exercised against canned XML with no socket. +/// +/// +/// +/// The constructor opens nothing and builds nothing. It does not even call +/// agentClientFactory — the Wave-0 universal discovery browser constructs a throwaway +/// instance purely to ask whether the driver can browse, and a constructor that materialized +/// a client would open a connection pool per browse probe against a possibly-unreachable +/// Agent. Every connection is owned by . +/// +/// +/// The driverConfigJson argument is honoured, not decorative. The runtime hands +/// a config change to a live instance — DriverInstanceActor's ApplyDelta +/// calls with the new document and never rebuilds the object +/// — so a driver that read only its constructor options would turn every MTConnect config +/// edit into a silent no-op that still seals the deployment green. This class therefore +/// re-parses the document () whenever it carries a real body, +/// matching S7/AbCip/TwinCAT/Galaxy rather than the Modbus/FOCAS/OpcUaClient drivers that +/// ignore the argument. A blank / {} / [] document means "no config supplied" +/// and keeps the constructor options, which is what unit tests and the factory path rely on. +/// +/// +/// Failure is never dressed as success (#485). A failed initialize disposes whatever +/// client it built, drops the probe cache and the observation index, publishes +/// with the error text, and rethrows — the runtime's +/// retry/backoff loop is the recovery path. In particular a /probe that succeeds while +/// the priming /current fails is Faulted, not "Healthy with no data": the index IS the +/// read surface, so serving that state would render every tag +/// BadWaitingForInitialData indefinitely behind a green driver, and the +/// /sample cursor (Task 11) only exists because /current reported a +/// nextSequence. +/// +/// +/// Scope. This type currently implements only. +/// IReadable (Task 10), ISubscribable (Task 11), ITagDiscovery +/// (Task 12) and IHostConnectivityProbe/IRediscoverable (Task 13) are added on +/// top of the state this class already caches — the probe model, the Agent +/// instanceId, and the observation index. +/// +/// +public sealed class MTConnectDriver : IDriver +{ + /// + /// Coarse per-entry byte weights behind . The contract asks + /// for an approximate driver-attributable footprint that Core polls every 30 s to + /// decide whether to ask for a flush, so an order-of-magnitude estimate from the two things + /// that actually scale with a plant (declared data items, live observations) is the right + /// precision. Measuring real retained size would need a heap walk per poll. + /// + private const int ProbeModelBytesPerDataItem = 512; + + /// Estimated retained bytes per live dataItemId → snapshot entry. + private const int IndexBytesPerEntry = 256; + + /// Estimated retained bytes per authored . + private const int TagBytesPerEntry = 256; + + /// + /// Config-JSON reader options, mirroring the sibling driver factories. Note there is + /// deliberately no JsonStringEnumConverter: enum-carrying DTO fields stay + /// string? and go through , which is the project-wide + /// guard against the numerically-serialized-enum trap that faults a driver at deploy time. + /// + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + ReadCommentHandling = JsonCommentHandling.Skip, + AllowTrailingCommas = true, + }; + + private readonly string _driverInstanceId; + private readonly ILogger _logger; + private readonly Func _agentClientFactory; + + /// + /// Serializes the three lifecycle entry points against each other. Initialize / Reinitialize + /// / Shutdown all swap the client and the served caches; overlapping them would let a + /// shutdown dispose a client an in-flight initialize is about to publish. + /// + private readonly SemaphoreSlim _lifecycle = new(1, 1); + + private MTConnectDriverOptions _options; + private IMTConnectAgentClient? _client; + private MTConnectObservationIndex _index; + private MTConnectProbeModel? _probeModel; + private int _probeModelDataItemCount; + private long? _agentInstanceId; + private DriverHealth _health = new(DriverState.Unknown, null, null); + + /// Creates a driver instance. Opens no connection and builds no agent client. + /// + /// The typed configuration this instance starts from. A config document supplied to + /// / supersedes it. + /// + /// Stable logical id of this driver instance, from the config DB. + /// + /// Builds the Agent client from the effective options. Defaults to the production + /// ; tests inject a canned-XML fake. Called from + /// , never from this constructor. + /// + /// Optional logger; defaults to the null logger. + public MTConnectDriver( + MTConnectDriverOptions options, + string driverInstanceId, + Func? agentClientFactory = null, + ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); + + _options = options; + _driverInstanceId = driverInstanceId; + _logger = logger ?? NullLogger.Instance; + _agentClientFactory = agentClientFactory ?? (o => new MTConnectAgentClient(o, logger)); + + // Never null, so every consumer (Task 10's ReadAsync included) can ask it for a snapshot + // without a null check: an un-primed index answers BadWaitingForInitialData for an authored + // tag and BadNodeIdUnknown otherwise, which is exactly the truth before Initialize runs. + _index = new MTConnectObservationIndex(options.Tags); + } + + /// + public string DriverInstanceId => _driverInstanceId; + + /// + public string DriverType => "MTConnect"; + + /// + /// The live dataItemId → snapshot map. Consumed by IReadable (Task 10) and the + /// /sample pump (Task 11); exposed to tests as the observable proof that + /// actually primed it. + /// + internal MTConnectObservationIndex ObservationIndex => Volatile.Read(ref _index); + + /// + /// The cached /probe device model, or null when it has not been fetched or was + /// dropped by . Prefer + /// , which cannot observe the flushed hole. + /// + internal MTConnectProbeModel? CachedProbeModel => _probeModel; + + /// + /// The Agent's instanceId as of the last successful /current. Task 13 watches + /// this for change to raise rediscovery (an Agent restart invalidates every cached id). + /// + internal long? AgentInstanceId => _agentInstanceId; + + /// The options currently in force — the constructor's, or the last document applied. + internal MTConnectDriverOptions EffectiveOptions => _options; + + /// + /// + /// Builds the Agent client from the effective options, runs one deadline-bounded + /// /probe (caching the device model and its data-item count), then one + /// deadline-bounded /current to prime the observation index and record the Agent's + /// instanceId. Publishes on success; on any failure + /// disposes the client, clears the caches, publishes with + /// the error text, and rethrows. + /// + public async Task InitializeAsync(string driverConfigJson, CancellationToken cancellationToken) + { + await _lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + // A re-Initialize on a live instance must not orphan the previous client. + await TeardownCoreAsync().ConfigureAwait(false); + + MTConnectDriverOptions options; + try + { + options = ResolveOptions(driverConfigJson); + } + catch (Exception ex) + { + // A config document that will not parse is just as fatal to a cold start as an + // unreachable Agent, and it must reach GetHealth() the same way — otherwise the + // dashboard keeps showing whatever state the driver was in before. + WriteHealth(new DriverHealth(DriverState.Faulted, ReadHealth().LastSuccessfulRead, ex.Message)); + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} could not read its driver config document; the driver is Faulted.", + _driverInstanceId); + + throw; + } + + await StartCoreAsync(options, existingClient: null, cancellationToken).ConfigureAwait(false); + } + finally + { + _lifecycle.Release(); + } + } + + /// + /// + /// + /// Stops the /sample stream, then takes one of two paths depending on whether the + /// new document changes anything the reads at + /// construction: + /// + /// + /// Endpoint/transport changed (AgentUri, DeviceName, + /// RequestTimeoutMs, HeartbeatMs, SampleIntervalMs, + /// SampleCount) ⇒ full teardown and rebuild. The plan names only the first two, + /// but the other four are baked into the client's deadlines, its watchdog window, and its + /// /sample query string, so keeping the old client because the URI happened not to + /// change would silently discard the operator's edit — the same class of defect as + /// ignoring the config document altogether. + /// + /// + /// Otherwise ("config-only": authored tags, probe cadence, reconnect backoff) ⇒ + /// the live client and its connection pool are kept, and everything derived from config + /// is rebuilt: the probe model is re-fetched, the observation index is recreated against + /// the new tag types, and one /current re-primes it. Nothing config-derived + /// survives, so a retyped tag cannot keep coercing to its old type. + /// + /// + public async Task ReinitializeAsync(string driverConfigJson, CancellationToken cancellationToken) + { + await _lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + MTConnectDriverOptions incoming; + try + { + incoming = ResolveOptions(driverConfigJson); + } + catch (Exception ex) + { + // Deliberately NOT Faulted, and deliberately before any teardown: the running + // driver is still serving its previous, valid configuration. Rejecting the edit + // fails the deployment (DriverInstanceActor turns this into ApplyResult(false)); + // downing a working driver over an unreadable document would be a far larger blast + // radius than the change that was refused. + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} rejected a new driver config document and kept running its previous configuration.", + _driverInstanceId); + + throw; + } + + var existing = _client; + + await StopSampleStreamAsync().ConfigureAwait(false); + + if (existing is null || RequiresClientRebuild(_options, incoming)) + { + await TeardownCoreAsync().ConfigureAwait(false); + await StartCoreAsync(incoming, existingClient: null, cancellationToken).ConfigureAwait(false); + + return; + } + + await StartCoreAsync(incoming, existing, cancellationToken).ConfigureAwait(false); + } + finally + { + _lifecycle.Release(); + } + } + + /// + /// + /// Stops the /sample stream, disposes the Agent client, and drops the probe cache and + /// the observation index — values sourced from a torn-down connection must never keep + /// reading Good. LastSuccessfulRead is retained because it is diagnostic ("when did + /// this driver last hear from the Agent"), not served data. Idempotent. + /// + public async Task ShutdownAsync(CancellationToken cancellationToken) + { + await _lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + var lastRead = ReadHealth().LastSuccessfulRead; + + await StopSampleStreamAsync().ConfigureAwait(false); + await TeardownCoreAsync().ConfigureAwait(false); + + WriteHealth(new DriverHealth(DriverState.Unknown, lastRead, null)); + } + finally + { + _lifecycle.Release(); + } + } + + /// + public DriverHealth GetHealth() => ReadHealth(); + + /// + /// + /// Sums the three caches that scale with plant size: the /probe device model (by + /// declared data item), the live observation index (by indexed data item), and the authored + /// tag table. See for why the weights are coarse + /// constants. + /// + public long GetMemoryFootprint() => + ((long)_probeModelDataItemCount * ProbeModelBytesPerDataItem) + + ((long)ObservationIndex.Count * IndexBytesPerEntry) + + ((long)_options.Tags.Count * TagBytesPerEntry); + + /// + /// + /// Drops the /probe device model — the browse cache, which is re-derivable from the + /// Agent at any time — and keeps the observation index, which is required for correctness + /// (it is the read surface, and re-deriving it would mean a full re-prime the caller did not + /// ask for). Flushing cannot wedge the driver: every consumer of the model goes through + /// , which re-fetches and re-caches on a miss. + /// + public Task FlushOptionalCachesAsync(CancellationToken cancellationToken) + { + var dropped = _probeModelDataItemCount; + + _probeModel = null; + _probeModelDataItemCount = 0; + + if (dropped > 0) + { + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} flushed its cached /probe device model ({DataItemCount} data items); it is re-fetched on the next browse.", + _driverInstanceId, dropped); + } + + return Task.CompletedTask; + } + + /// + /// The /probe device model: the cached copy when warm, otherwise a fresh + /// deadline-bounded /probe that re-populates the cache. This is the accessor every + /// model consumer (Task 12's DiscoverAsync) must use, so that + /// can never leave browse permanently disabled. + /// + /// Cancellation token. + /// The Agent's device model. + /// The driver is not initialized. + internal async Task GetOrFetchProbeModelAsync(CancellationToken cancellationToken) + { + var cached = _probeModel; + if (cached is not null) + { + return cached; + } + + var client = _client + ?? throw new InvalidOperationException( + $"MTConnect driver '{_driverInstanceId}' has no agent client; the /probe device model cannot be fetched before InitializeAsync succeeds."); + + var model = await BoundedAsync(client.ProbeAsync, "/probe", _options.RequestTimeoutMs, cancellationToken) + .ConfigureAwait(false); + + CacheProbeModel(model); + + return model; + } + + /// + /// Builds driver options from a DriverConfig JSON document. + /// + /// + /// Lives here rather than on the factory because the driver itself needs it: the runtime + /// delivers config changes as JSON to a live instance (see the class remarks). Task 15's + /// MTConnectDriverFactoryExtensions.CreateInstance should delegate to this method + /// rather than deserializing a second DTO — two parsers for one document drift. + /// + /// Driver instance id, used only in error text. + /// The driver's DriverConfig JSON. + /// The parsed options. + /// The document is unusable or omits agentUri. + internal static MTConnectDriverOptions ParseOptions(string driverInstanceId, string driverConfigJson) + { + ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); + ArgumentException.ThrowIfNullOrWhiteSpace(driverConfigJson); + + MTConnectDriverConfigDto? dto; + try + { + dto = JsonSerializer.Deserialize(driverConfigJson, JsonOptions); + } + catch (JsonException ex) + { + throw new InvalidOperationException( + $"MTConnect driver config for '{driverInstanceId}' is not valid JSON: {ex.Message}", ex); + } + + if (dto is null) + { + throw new InvalidOperationException( + $"MTConnect driver config for '{driverInstanceId}' deserialised to null"); + } + + if (string.IsNullOrWhiteSpace(dto.AgentUri)) + { + throw new InvalidOperationException( + $"MTConnect driver config for '{driverInstanceId}' missing required AgentUri"); + } + + return new MTConnectDriverOptions + { + AgentUri = dto.AgentUri.Trim(), + DeviceName = string.IsNullOrWhiteSpace(dto.DeviceName) ? null : dto.DeviceName.Trim(), + RequestTimeoutMs = dto.RequestTimeoutMs ?? 5_000, + SampleIntervalMs = dto.SampleIntervalMs ?? 1_000, + SampleCount = dto.SampleCount ?? 1_000, + HeartbeatMs = dto.HeartbeatMs ?? 10_000, + Tags = dto.Tags is { Count: > 0 } + ? [.. dto.Tags.Select(t => BuildTag(t, driverInstanceId))] + : [], + Probe = new MTConnectProbeOptions + { + Enabled = dto.Probe?.Enabled ?? true, + Interval = TimeSpan.FromMilliseconds(dto.Probe?.IntervalMs ?? 5_000), + Timeout = TimeSpan.FromMilliseconds(dto.Probe?.TimeoutMs ?? 2_000), + }, + Reconnect = new MTConnectReconnectOptions + { + MinBackoffMs = dto.Reconnect?.MinBackoffMs ?? 0, + MaxBackoffMs = dto.Reconnect?.MaxBackoffMs ?? 30_000, + BackoffMultiplier = dto.Reconnect?.BackoffMultiplier ?? 2.0, + }, + }; + } + + // ---- lifecycle internals (all called with _lifecycle held) ---- + + /// + /// Brings the driver up against , reusing + /// when the caller determined the transport is unchanged. + /// Nothing is published until BOTH agent calls have succeeded, so a failure can never leave a + /// half-initialized instance behind. + /// + private async Task StartCoreAsync( + MTConnectDriverOptions options, IMTConnectAgentClient? existingClient, CancellationToken ct) + { + var lastRead = ReadHealth().LastSuccessfulRead; + WriteHealth(new DriverHealth(DriverState.Initializing, lastRead, null)); + + IMTConnectAgentClient? built = null; + try + { + // arch-review 01/S-6: a 0 authored here must never come to mean "wait forever". Checked + // in the driver, not only in the production client's constructor, so the guard holds for + // any client implementation behind the seam. + RequirePositive(options.RequestTimeoutMs, nameof(MTConnectDriverOptions.RequestTimeoutMs)); + RequirePositive(options.HeartbeatMs, nameof(MTConnectDriverOptions.HeartbeatMs)); + RequirePositive(options.SampleIntervalMs, nameof(MTConnectDriverOptions.SampleIntervalMs)); + RequirePositive(options.SampleCount, nameof(MTConnectDriverOptions.SampleCount)); + + var client = existingClient; + if (client is null) + { + built = _agentClientFactory(options) + ?? throw new InvalidOperationException( + $"The MTConnect agent-client factory for driver '{_driverInstanceId}' returned null."); + client = built; + } + + var probe = await BoundedAsync(client.ProbeAsync, "/probe", options.RequestTimeoutMs, ct) + .ConfigureAwait(false); + var current = await BoundedAsync(client.CurrentAsync, "/current", options.RequestTimeoutMs, ct) + .ConfigureAwait(false); + + // ---- commit point: everything below is non-throwing bookkeeping ---- + _options = options; + _client = client; + built = null; + + CacheProbeModel(probe); + _agentInstanceId = current.InstanceId; + + var index = new MTConnectObservationIndex(options.Tags); + index.Apply(current); + Volatile.Write(ref _index, index); + + WriteHealth(new DriverHealth(DriverState.Healthy, DateTime.UtcNow, null)); + + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} initialized against {AgentUri}{DeviceScope}: {DeviceCount} device(s), {ObservationCount} primed observation(s), agent instanceId {InstanceId}.", + _driverInstanceId, + options.AgentUri, + options.DeviceName is null ? string.Empty : $"/{options.DeviceName}", + probe.Devices.Count, + index.Count, + current.InstanceId); + } + catch (Exception ex) + { + // A client this attempt built is unreachable from anywhere else; one the caller handed in + // is disposed by the teardown below. Either way nothing survives a failed start. + SafeDispose(built); + await TeardownCoreAsync().ConfigureAwait(false); + + WriteHealth(new DriverHealth(DriverState.Faulted, lastRead, ex.Message)); + + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} failed to initialize against {AgentUri}; the driver is Faulted and serves no values.", + _driverInstanceId, options.AgentUri); + + throw; + } + } + + /// + /// Releases the agent client and drops every piece of served state. Never throws — it runs + /// on the failure path of , where a second exception would mask + /// the real one. + /// + private Task TeardownCoreAsync() + { + var client = _client; + _client = null; + SafeDispose(client); + + _probeModel = null; + _probeModelDataItemCount = 0; + _agentInstanceId = null; + + // A fresh index rather than Clear(): the tag table it coerces against belongs to the options + // that are being torn down. + Volatile.Write(ref _index, new MTConnectObservationIndex(_options.Tags)); + + return Task.CompletedTask; + } + + /// + /// Stops the shared /sample long-poll stream. A no-op until Task 11 introduces the + /// pump; it exists now because both and + /// must stop the stream before the client they are about + /// to dispose goes away, and that ordering is easy to lose when the pump is bolted on later. + /// + private static Task StopSampleStreamAsync() => Task.CompletedTask; + + /// The document's options when it carries a body; otherwise the ones already in force. + private MTConnectDriverOptions ResolveOptions(string driverConfigJson) => + HasConfigBody(driverConfigJson) ? ParseOptions(_driverInstanceId, driverConfigJson) : _options; + + /// + /// True when carries a real config body. The + /// bootstrapper always passes a populated document; tests and probe-only callers pass + /// "{}", "[]" or blank, which keep the constructor-supplied options. + /// + private static bool HasConfigBody(string? driverConfigJson) + { + if (string.IsNullOrWhiteSpace(driverConfigJson)) + { + return false; + } + + var trimmed = driverConfigJson.Trim(); + + return trimmed is not "{}" and not "[]"; + } + + /// + /// Whether the new options change anything reads at + /// construction, and therefore cannot be applied to a live client. See + /// for why this is wider than the plan's AgentUri/DeviceName. + /// + private static bool RequiresClientRebuild(MTConnectDriverOptions current, MTConnectDriverOptions incoming) => + !string.Equals(current.AgentUri, incoming.AgentUri, StringComparison.Ordinal) + || !string.Equals(current.DeviceName, incoming.DeviceName, StringComparison.Ordinal) + || current.RequestTimeoutMs != incoming.RequestTimeoutMs + || current.HeartbeatMs != incoming.HeartbeatMs + || current.SampleIntervalMs != incoming.SampleIntervalMs + || current.SampleCount != incoming.SampleCount; + + private void CacheProbeModel(MTConnectProbeModel model) + { + _probeModel = model; + + // Counted once here rather than re-walked on every 30 s footprint poll. + _probeModelDataItemCount = model.Devices.Sum(d => d.AllDataItems().Count()); + } + + /// + /// Runs one unary agent call under a deadline linked to the caller's token. Belt-and-braces + /// over the production client's own per-call deadline: the seam does not oblige an + /// implementation to bound itself, and an unbounded initialize would wedge the driver-host + /// actor (arch-review R2-01). + /// + private async Task BoundedAsync( + Func> call, string request, int timeoutMs, CancellationToken ct) + { + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + deadline.CancelAfter(TimeSpan.FromMilliseconds(timeoutMs)); + + try + { + return await call(deadline.Token).ConfigureAwait(false); + } + catch (OperationCanceledException) when (!ct.IsCancellationRequested) + { + // A bare "A task was canceled" names neither the Agent nor the request that stalled. + throw new TimeoutException( + $"MTConnect {request} request for driver '{_driverInstanceId}' against '{_options.AgentUri}' did not complete within {timeoutMs} ms."); + } + } + + private static void SafeDispose(IMTConnectAgentClient? client) + { + if (client is null) + { + return; + } + + try + { + client.Dispose(); + } + catch (Exception) + { + // Teardown runs on failure paths; a disposal fault must not replace the original error. + } + } + + private static void RequirePositive(int value, string optionName) + { + if (value <= 0) + { + throw new ArgumentOutOfRangeException( + nameof(MTConnectDriverOptions), + value, + $"{optionName} must be positive; a non-positive value would either un-bound a deadline or ask the Agent for nothing."); + } + } + + /// Barrier-protected read of the cross-thread health snapshot. + private DriverHealth ReadHealth() => Volatile.Read(ref _health); + + /// Barrier-protected publish of a new health snapshot. + private void WriteHealth(DriverHealth value) => Volatile.Write(ref _health, value); + + private static MTConnectTagDefinition BuildTag(MTConnectTagDto dto, string driverInstanceId) + { + if (string.IsNullOrWhiteSpace(dto.FullName)) + { + throw new InvalidOperationException( + $"MTConnect driver config for '{driverInstanceId}' has a tag with no FullName (the Agent DataItem id it binds)."); + } + + return new MTConnectTagDefinition( + dto.FullName.Trim(), + // Absent means "no declared type": String is the one coercion that cannot fail, and it + // carries the Agent's text through untouched. + dto.DriverDataType is null + ? DriverDataType.String + : ParseEnum(dto.DriverDataType, dto.FullName, driverInstanceId, nameof(MTConnectTagDefinition.DriverDataType)), + dto.IsArray ?? false, + dto.ArrayDim ?? 0, + dto.MtCategory, + dto.MtType, + dto.MtSubType, + dto.Units); + } + + /// + /// Parses an enum authored as a name. Config surfaces serialize enums as names + /// project-wide; a numerically-serialized enum is the known trap that faults a driver at + /// deploy, so this reports the offending field rather than silently taking member 0. + /// + private static T ParseEnum(string raw, string tagName, string driverInstanceId, string field) + where T : struct, Enum => + Enum.TryParse(raw, ignoreCase: true, out var parsed) && Enum.IsDefined(parsed) + ? parsed + : throw new InvalidOperationException(string.Create( + CultureInfo.InvariantCulture, + $"MTConnect driver config for '{driverInstanceId}' tag '{tagName}' has an unrecognised {field} '{raw}'. Expected one of: {string.Join(", ", Enum.GetNames())}.")); + + // ---- config DTOs ---- + + /// + /// Nullable-init mirror of the driver's DriverConfig JSON. Every property is nullable + /// so an omitted key means "use the default" rather than "reset to zero". + /// + private sealed class MTConnectDriverConfigDto + { + public string? AgentUri { get; init; } + public string? DeviceName { get; init; } + public int? RequestTimeoutMs { get; init; } + public int? SampleIntervalMs { get; init; } + public int? SampleCount { get; init; } + public int? HeartbeatMs { get; init; } + public List? Tags { get; init; } + public MTConnectProbeDto? Probe { get; init; } + public MTConnectReconnectDto? Reconnect { get; init; } + } + + /// One authored tag. DriverDataType stays a string — see . + private sealed class MTConnectTagDto + { + public string? FullName { get; init; } + public string? DriverDataType { get; init; } + public bool? IsArray { get; init; } + public int? ArrayDim { get; init; } + public string? MtCategory { get; init; } + public string? MtType { get; init; } + public string? MtSubType { get; init; } + public string? Units { get; init; } + } + + /// Background connectivity-probe knobs (Task 13 consumes them). + private sealed class MTConnectProbeDto + { + public bool? Enabled { get; init; } + public int? IntervalMs { get; init; } + public int? TimeoutMs { get; init; } + } + + /// Reconnect-backoff knobs (Task 11's pump consumes them). + private sealed class MTConnectReconnectDto + { + public int? MinBackoffMs { get; init; } + public int? MaxBackoffMs { get; init; } + public double? BackoffMultiplier { get; init; } + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs new file mode 100644 index 00000000..85acca43 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs @@ -0,0 +1,202 @@ +using System.Runtime.CompilerServices; +using System.Threading.Channels; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// The shared test double: an Agent that serves the canned +/// Fixtures/probe.xml + Fixtures/current.xml documents, counts every call, records +/// its own disposal, and lets a test drive the /sample chunk sequence by hand. +/// +/// +/// +/// Deterministic by construction — no timers, no sleeps, no polling. The +/// /sample leg reads from an unbounded that only a test +/// fills, and does not return until the driver has actually consumed +/// the chunk it wrote (each chunk carries its own completion source, signalled after the +/// enumerator's yield return resumes). A subscription test can therefore say "one +/// chunk has now been fully processed" as a fact rather than as a timing hope. +/// +/// +/// It honours the seam's stream-end contract (see +/// ): cancelling the token is the only way the +/// enumeration ends without throwing. Closing the scripted stream via +/// raises , exactly as the +/// production client does when an Agent drops the connection — so a pump written against the +/// fake cannot silently pass while mishandling the real one. +/// +/// +/// Disposal is observable () because "did the driver +/// actually release the client?" is a behaviour, not an implementation detail: a +/// ReinitializeAsync that re-points the Agent but leaks the old client keeps a live +/// connection pool per re-deploy, and nothing else in the test surface would notice. +/// +/// +internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable +{ + private readonly Channel _chunks = Channel.CreateUnbounded(); + private readonly CancellationTokenSource _disposeCts = new(); + + private int _probeCallCount; + private int _currentCallCount; + private int _sampleCallCount; + private int _disposeCount; + + private CannedAgentClient(MTConnectProbeModel probe, MTConnectStreamsResult current) + { + Probe = probe; + Current = current; + } + + /// + /// Builds a client serving the repo's canned fixtures — Fixtures/probe.xml for + /// /probe and Fixtures/current.xml for /current. + /// + /// Probe-document fixture path, relative to the test binaries. + /// Streams-document fixture path, relative to the test binaries. + public static CannedAgentClient FromFixtures( + string probeFixture = "Fixtures/probe.xml", + string currentFixture = "Fixtures/current.xml") => + new( + MTConnectProbeParser.Parse(File.ReadAllText(probeFixture)), + MTConnectStreamsParser.Parse(File.ReadAllText(currentFixture))); + + /// Parses a streams fixture into a chunk a test can script onto the sample stream. + /// Streams-document fixture path, relative to the test binaries. + public static MTConnectStreamsResult Chunk(string fixture) => + MTConnectStreamsParser.Parse(File.ReadAllText(fixture)); + + /// The document /probe answers with. Settable so a test can re-shape the model. + public MTConnectProbeModel Probe { get; set; } + + /// + /// The document /current answers with. Settable so a re-baseline (or an agent-restart + /// instanceId change) can be scripted between calls. + /// + public MTConnectStreamsResult Current { get; set; } + + /// When set, /probe throws this instead of answering. + public Exception? ProbeFailure { get; set; } + + /// When set, /current throws this instead of answering. + public Exception? CurrentFailure { get; set; } + + /// Number of /probe requests issued against this client. + public int ProbeCallCount => Volatile.Read(ref _probeCallCount); + + /// Number of /current requests issued against this client. + public int CurrentCallCount => Volatile.Read(ref _currentCallCount); + + /// Number of /sample enumerations started against this client. + public int SampleCallCount => Volatile.Read(ref _sampleCallCount); + + /// Number of calls (not clamped — a double-dispose is visible). + public int DisposeCount => Volatile.Read(ref _disposeCount); + + /// Whether the client has been disposed at least once. + public bool IsDisposed => DisposeCount > 0; + + /// The from sequence the most recent /sample enumeration was opened at. + public long? LastSampleFrom { get; private set; } + + /// + public Task ProbeAsync(CancellationToken ct) + { + ObjectDisposedException.ThrowIf(IsDisposed, this); + ct.ThrowIfCancellationRequested(); + Interlocked.Increment(ref _probeCallCount); + + return ProbeFailure is null ? Task.FromResult(Probe) : Task.FromException(ProbeFailure); + } + + /// + public Task CurrentAsync(CancellationToken ct) + { + ObjectDisposedException.ThrowIf(IsDisposed, this); + ct.ThrowIfCancellationRequested(); + Interlocked.Increment(ref _currentCallCount); + + return CurrentFailure is null + ? Task.FromResult(Current) + : Task.FromException(CurrentFailure); + } + + /// + public async IAsyncEnumerable SampleAsync( + long from, [EnumeratorCancellation] CancellationToken ct) + { + ObjectDisposedException.ThrowIf(IsDisposed, this); + Interlocked.Increment(ref _sampleCallCount); + LastSampleFrom = from; + + using var lifetime = CancellationTokenSource.CreateLinkedTokenSource(ct, _disposeCts.Token); + var delivered = 0L; + + while (true) + { + ScriptedChunk scripted; + try + { + scripted = await _chunks.Reader.ReadAsync(lifetime.Token).ConfigureAwait(false); + } + catch (ChannelClosedException) + { + // Mirrors the production client: a stream that stops for any reason other than the + // caller cancelling is an exception, never a quiet end of enumeration. + throw new MTConnectStreamEndedException( + MTConnectStreamEndReason.ConnectionClosed, "canned://agent", delivered); + } + + delivered++; + + yield return scripted.Result; + + scripted.Consumed.TrySetResult(); + } + } + + /// + /// Queues a chunk onto the scripted /sample stream and waits until the consumer has + /// processed it. This is the deterministic replacement for "wait a bit and hope the pump + /// ran". + /// + /// The chunk the Agent should send next. + /// A task completing once the enumerating consumer has moved past . + public Task PumpAsync(MTConnectStreamsResult chunk) + { + ArgumentNullException.ThrowIfNull(chunk); + + var scripted = new ScriptedChunk( + chunk, new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously)); + + if (!_chunks.Writer.TryWrite(scripted)) + { + throw new InvalidOperationException("The scripted /sample stream is already closed."); + } + + return scripted.Consumed.Task; + } + + /// + /// Closes the scripted /sample stream, which surfaces to the consumer as an + /// — the transient, reconnectable end. + /// + public void EndStream() => _chunks.Writer.TryComplete(); + + /// + public void Dispose() + { + Interlocked.Increment(ref _disposeCount); + + if (DisposeCount > 1) + { + return; + } + + _disposeCts.Cancel(); + _chunks.Writer.TryComplete(); + _disposeCts.Dispose(); + } + + private sealed record ScriptedChunk(MTConnectStreamsResult Result, TaskCompletionSource Consumed); +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs new file mode 100644 index 00000000..d08036b3 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs @@ -0,0 +1,548 @@ +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 9 — 's lifecycle over the +/// seam: construction, initialize, reinitialize, shutdown, +/// health, footprint, and cache flush. Every test runs against +/// ; no socket is opened anywhere in this file. +/// +/// +/// The load-bearing assertions here are the ones about what must NOT happen: a +/// constructor that dials, an initialize that faults but still reports Healthy, a re-point that +/// keeps talking to the old Agent, and a cache flush that wedges browse. Each of those is a +/// defect the happy-path assertions alone would pass straight over. +/// +public sealed class MTConnectDriverLifecycleTests +{ + private const string AgentUri = "http://fixture-agent:5000"; + + /// Every dataItemId the canned probe declares — the browse-cache footprint basis. + private const int FixtureDataItemCount = 7; + + private static MTConnectDriverOptions Opts( + string agentUri = AgentUri, + string? deviceName = null, + int requestTimeoutMs = 5000) => + new() + { + AgentUri = agentUri, + DeviceName = deviceName, + RequestTimeoutMs = requestTimeoutMs, + Tags = + [ + new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64), + new MTConnectTagDefinition("dev1_execution", DriverDataType.String), + ], + }; + + /// A driver wired to one canned client, plus that client, for the common case. + private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( + MTConnectDriverOptions? options = null) + { + var client = CannedAgentClient.FromFixtures(); + + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => client), client); + } + + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync() + { + var (driver, client) = NewDriver(); + await driver.InitializeAsync("{}", CancellationToken.None); + + return (driver, client); + } + + // ---- connection-free construction ---- + + [Fact] + public void Constructor_dials_nothing_and_does_not_even_build_a_client() + { + var factoryCalls = 0; + var client = CannedAgentClient.FromFixtures(); + + var driver = new MTConnectDriver(Opts(), "mt1", _ => + { + factoryCalls++; + + return client; + }); + + // The universal discovery browser constructs a throwaway instance purely to ask CanBrowse. + // If the ctor built (or worse, dialled) a client, every browse probe would open a connection + // pool against a possibly-unreachable Agent. + factoryCalls.ShouldBe(0); + client.ProbeCallCount.ShouldBe(0); + client.CurrentCallCount.ShouldBe(0); + client.SampleCallCount.ShouldBe(0); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + } + + [Fact] + public void Identity_is_the_constructor_arguments() + { + var (driver, _) = NewDriver(); + + driver.DriverInstanceId.ShouldBe("mt1"); + driver.DriverType.ShouldBe("MTConnect"); + } + + // ---- initialize ---- + + [Fact] + public async Task Initialize_primes_index_and_reports_healthy() + { + var (driver, client) = NewDriver(); + + await driver.InitializeAsync("{}", CancellationToken.None); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + client.ProbeCallCount.ShouldBe(1); + client.CurrentCallCount.ShouldBe(1); + + // The /current snapshot is what makes the driver able to answer a read at all. + driver.ObservationIndex.Count.ShouldBe(FixtureDataItemCount); + driver.ObservationIndex.Get("dev1_pos").StatusCode.ShouldBe(0u); + } + + [Fact] + public async Task Initialize_records_the_agent_instance_id_and_the_probe_model() + { + var (driver, client) = NewDriver(); + + await driver.InitializeAsync("{}", CancellationToken.None); + + driver.AgentInstanceId.ShouldBe(client.Current.InstanceId); + driver.CachedProbeModel.ShouldNotBeNull(); + driver.CachedProbeModel!.Devices.Count.ShouldBe(1); + } + + [Fact] + public async Task Initialize_stamps_last_successful_read_between_the_call_boundaries() + { + var (driver, _) = NewDriver(); + + var before = DateTime.UtcNow; + await driver.InitializeAsync("{}", CancellationToken.None); + var after = DateTime.UtcNow; + + var lastRead = driver.GetHealth().LastSuccessfulRead; + lastRead.ShouldNotBeNull(); + lastRead!.Value.ShouldBeGreaterThanOrEqualTo(before); + lastRead.Value.ShouldBeLessThanOrEqualTo(after); + } + + [Fact] + public async Task Initialize_faults_and_rethrows_when_probe_fails() + { + var (driver, client) = NewDriver(); + client.ProbeFailure = new HttpRequestException("agent unreachable"); + + var thrown = await Should.ThrowAsync( + () => driver.InitializeAsync("{}", CancellationToken.None)); + + thrown.Message.ShouldBe("agent unreachable"); + + var health = driver.GetHealth(); + health.State.ShouldBe(DriverState.Faulted); + health.LastError.ShouldNotBeNull(); + health.LastError!.ShouldContain("agent unreachable"); + + // No half-initialized object: the client it built must not survive the failure. + client.IsDisposed.ShouldBeTrue(); + } + + [Fact] + public async Task Initialize_faults_when_probe_succeeds_but_the_priming_current_fails() + { + // Deliberate posture (documented on InitializeAsync): a driver whose /probe worked but whose + // priming /current did not has an EMPTY value index and no sample cursor. Reporting Healthy + // there would render every tag BadWaitingForInitialData forever while the dashboard shows a + // green driver — the #485 "a failure that looks like success" shape. + var (driver, client) = NewDriver(); + client.CurrentFailure = new InvalidDataException("current unparseable"); + + await Should.ThrowAsync( + () => driver.InitializeAsync("{}", CancellationToken.None)); + + client.ProbeCallCount.ShouldBe(1); + driver.GetHealth().State.ShouldBe(DriverState.Faulted); + driver.GetHealth().State.ShouldNotBe(DriverState.Healthy); + driver.CachedProbeModel.ShouldBeNull(); + driver.ObservationIndex.Count.ShouldBe(0); + client.IsDisposed.ShouldBeTrue(); + } + + [Fact] + public async Task Initialize_rejects_a_non_positive_request_timeout_before_dialling() + { + // arch-review 01/S-6: an operator-authorable 0 must never come to mean "wait forever". + // Asserted through a FAKE client factory, so this proves the DRIVER validates rather than + // relying on the production client's own constructor guard. + var factoryCalls = 0; + var driver = new MTConnectDriver(Opts(requestTimeoutMs: 0), "mt1", _ => + { + factoryCalls++; + + return CannedAgentClient.FromFixtures(); + }); + + await Should.ThrowAsync( + () => driver.InitializeAsync("{}", CancellationToken.None)); + + factoryCalls.ShouldBe(0); + driver.GetHealth().State.ShouldBe(DriverState.Faulted); + } + + // ---- the driverConfigJson argument ---- + + [Fact] + public async Task Initialize_honours_the_config_json_over_the_constructor_options() + { + // The runtime delivers a config change ONLY through this argument (DriverInstanceActor + // ApplyDelta -> ReinitializeAsync(json) on the LIVE instance). Ignoring it would make every + // MTConnect config change a silent no-op that still seals the deployment green. + MTConnectDriverOptions? seen = null; + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + seen = o; + + return CannedAgentClient.FromFixtures(); + }); + + await driver.InitializeAsync( + """{"agentUri":"http://other-agent:5001","deviceName":"VMC-3Axis"}""", + CancellationToken.None); + + seen.ShouldNotBeNull(); + seen!.AgentUri.ShouldBe("http://other-agent:5001"); + seen.DeviceName.ShouldBe("VMC-3Axis"); + } + + [Fact] + public async Task Initialize_with_an_empty_config_body_keeps_the_constructor_options() + { + MTConnectDriverOptions? seen = null; + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + seen = o; + + return CannedAgentClient.FromFixtures(); + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + + seen.ShouldNotBeNull(); + seen!.AgentUri.ShouldBe(AgentUri); + seen.Tags.Count.ShouldBe(2); + } + + [Fact] + public async Task Initialize_faults_on_a_config_body_with_no_agent_uri() + { + var (driver, _) = NewDriver(); + + await Should.ThrowAsync( + () => driver.InitializeAsync("""{"deviceName":"VMC-3Axis"}""", CancellationToken.None)); + + driver.GetHealth().State.ShouldBe(DriverState.Faulted); + } + + [Fact] + public async Task Initialize_parses_authored_tags_out_of_the_config_json() + { + MTConnectDriverOptions? seen = null; + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + seen = o; + + return CannedAgentClient.FromFixtures(); + }); + + await driver.InitializeAsync( + """ + {"agentUri":"http://a:5000", + "tags":[{"fullName":"dev1_partcount","driverDataType":"Int32"}]} + """, + CancellationToken.None); + + seen.ShouldNotBeNull(); + seen!.Tags.Count.ShouldBe(1); + seen.Tags[0].FullName.ShouldBe("dev1_partcount"); + + // Enum-serialization trap (project-wide MEMORY): enum-carrying config fields are NAMES. + seen.Tags[0].DriverDataType.ShouldBe(DriverDataType.Int32); + } + + // ---- reinitialize ---- + + [Fact] + public async Task Reinitialize_with_unchanged_endpoint_config_keeps_the_same_client() + { + var built = new List(); + var driver = new MTConnectDriver(Opts(), "mt1", _ => + { + var c = CannedAgentClient.FromFixtures(); + built.Add(c); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + await driver.ReinitializeAsync( + $$"""{"agentUri":"{{AgentUri}}","tags":[{"fullName":"dev1_pos","driverDataType":"Float64"}]}""", + CancellationToken.None); + + // Config-only: the live connection pool survives, so no client churn... + built.Count.ShouldBe(1); + built[0].IsDisposed.ShouldBeFalse(); + + // ...but everything DERIVED from config is genuinely rebuilt against the new document. + built[0].ProbeCallCount.ShouldBe(2); + built[0].CurrentCallCount.ShouldBe(2); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + [Fact] + public async Task Reinitialize_with_a_changed_agent_uri_tears_down_and_rebuilds_the_client() + { + var built = new List<(MTConnectDriverOptions Options, CannedAgentClient Client)>(); + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + var c = CannedAgentClient.FromFixtures(); + built.Add((o, c)); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + await driver.ReinitializeAsync( + """{"agentUri":"http://relocated-agent:5000"}""", CancellationToken.None); + + // A re-pointed driver that kept the old client would keep reading the OLD machine while the + // deployment reports success. + built.Count.ShouldBe(2); + built[0].Client.IsDisposed.ShouldBeTrue(); + built[1].Options.AgentUri.ShouldBe("http://relocated-agent:5000"); + built[1].Client.IsDisposed.ShouldBeFalse(); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + [Fact] + public async Task Reinitialize_with_a_changed_device_scope_rebuilds_the_client() + { + var built = new List<(MTConnectDriverOptions Options, CannedAgentClient Client)>(); + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + var c = CannedAgentClient.FromFixtures(); + built.Add((o, c)); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + await driver.ReinitializeAsync( + $$"""{"agentUri":"{{AgentUri}}","deviceName":"VMC-3Axis"}""", CancellationToken.None); + + // DeviceName is baked into every request URI at client construction. + built.Count.ShouldBe(2); + built[0].Client.IsDisposed.ShouldBeTrue(); + built[1].Options.DeviceName.ShouldBe("VMC-3Axis"); + } + + [Fact] + public async Task Reinitialize_with_changed_request_knobs_rebuilds_the_client() + { + // RequestTimeoutMs / HeartbeatMs / SampleIntervalMs / SampleCount are all read by the client + // CONSTRUCTOR (deadlines, watchdog window, /sample query string). Keeping the old client + // because the URI happened not to change would silently discard the operator's edit. + var built = new List<(MTConnectDriverOptions Options, CannedAgentClient Client)>(); + var driver = new MTConnectDriver(Opts(), "mt1", o => + { + var c = CannedAgentClient.FromFixtures(); + built.Add((o, c)); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + await driver.ReinitializeAsync( + $$"""{"agentUri":"{{AgentUri}}","heartbeatMs":2500}""", CancellationToken.None); + + built.Count.ShouldBe(2); + built[1].Options.HeartbeatMs.ShouldBe(2500); + built[0].Client.IsDisposed.ShouldBeTrue(); + } + + [Fact] + public async Task Reinitialize_before_initialize_behaves_as_initialize() + { + var (driver, client) = NewDriver(); + + await driver.ReinitializeAsync("{}", CancellationToken.None); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + client.ProbeCallCount.ShouldBe(1); + } + + [Fact] + public async Task Reinitialize_that_fails_leaves_the_driver_faulted_not_healthy() + { + var built = new List(); + var driver = new MTConnectDriver(Opts(), "mt1", _ => + { + var c = CannedAgentClient.FromFixtures(); + built.Add(c); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + built[0].CurrentFailure = new HttpRequestException("agent went away"); + + // Config-only shape (only the tag list changes), so the failing leg is the re-prime against + // the client that is already live — the path a rebuild would otherwise mask. + await Should.ThrowAsync( + () => driver.ReinitializeAsync( + $$"""{"agentUri":"{{AgentUri}}","tags":[{"fullName":"dev1_pos"}]}""", CancellationToken.None)); + + built.Count.ShouldBe(1); + driver.GetHealth().State.ShouldBe(DriverState.Faulted); + driver.GetHealth().LastError.ShouldNotBeNull().ShouldContain("agent went away"); + + // A failed refresh must not leave a live client nobody owns. + built[0].IsDisposed.ShouldBeTrue(); + } + + [Fact] + public async Task Reinitialize_with_an_unreadable_config_keeps_the_running_driver_serving() + { + // Blast radius: an unparseable config edit fails the DEPLOYMENT (ApplyResult(false)); it must + // not down a driver that is happily serving its previous, valid configuration. + var (driver, client) = await InitializedDriverAsync(); + + await Should.ThrowAsync( + () => driver.ReinitializeAsync("""{"deviceName":"no-agent-uri"}""", CancellationToken.None)); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + driver.EffectiveOptions.AgentUri.ShouldBe(AgentUri); + client.IsDisposed.ShouldBeFalse(); + driver.ObservationIndex.Count.ShouldBe(FixtureDataItemCount); + } + + // ---- shutdown ---- + + [Fact] + public async Task Shutdown_disposes_the_client_and_drops_the_served_state() + { + var (driver, client) = await InitializedDriverAsync(); + var lastRead = driver.GetHealth().LastSuccessfulRead; + + await driver.ShutdownAsync(CancellationToken.None); + + client.DisposeCount.ShouldBe(1); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + + // Retained: it is diagnostic ("when did we last hear from the Agent"), not served data. + driver.GetHealth().LastSuccessfulRead.ShouldBe(lastRead); + + // Dropped: values from a torn-down connection must never keep reading Good. + driver.ObservationIndex.Count.ShouldBe(0); + driver.CachedProbeModel.ShouldBeNull(); + } + + [Fact] + public async Task Shutdown_is_idempotent() + { + var (driver, client) = await InitializedDriverAsync(); + + await driver.ShutdownAsync(CancellationToken.None); + await driver.ShutdownAsync(CancellationToken.None); + + client.DisposeCount.ShouldBe(1); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + } + + [Fact] + public async Task Shutdown_before_initialize_is_a_no_op() + { + var (driver, client) = NewDriver(); + + await driver.ShutdownAsync(CancellationToken.None); + + client.DisposeCount.ShouldBe(0); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + } + + // ---- footprint + flush ---- + + [Fact] + public async Task Memory_footprint_is_zero_before_initialize_and_positive_after() + { + var (driver, _) = NewDriver(); + + var before = driver.GetMemoryFootprint(); + await driver.InitializeAsync("{}", CancellationToken.None); + var after = driver.GetMemoryFootprint(); + + // Before init the only config-attributable state is the two authored tags. + before.ShouldBeLessThan(after); + after.ShouldBeGreaterThan(0); + } + + [Fact] + public async Task Flush_drops_the_probe_cache_and_keeps_the_observation_index() + { + var (driver, _) = await InitializedDriverAsync(); + var before = driver.GetMemoryFootprint(); + + await driver.FlushOptionalCachesAsync(CancellationToken.None); + + driver.GetMemoryFootprint().ShouldBeLessThan(before); + driver.CachedProbeModel.ShouldBeNull(); + + // The index is required for correctness (it IS the read surface) — never flushable. + driver.ObservationIndex.Count.ShouldBe(FixtureDataItemCount); + driver.ObservationIndex.Get("dev1_pos").StatusCode.ShouldBe(0u); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + [Fact] + public async Task Flush_does_not_wedge_the_driver_the_probe_model_is_refetched_on_demand() + { + // Task 12's DiscoverAsync reads the probe model. If flushing left no way back to it, a + // cache-budget breach would permanently disable browse on that driver instance. + var (driver, client) = await InitializedDriverAsync(); + await driver.FlushOptionalCachesAsync(CancellationToken.None); + + client.ProbeCallCount.ShouldBe(1); + + var model = await driver.GetOrFetchProbeModelAsync(CancellationToken.None); + + model.Devices.Count.ShouldBe(1); + client.ProbeCallCount.ShouldBe(2); + driver.CachedProbeModel.ShouldNotBeNull(); + } + + [Fact] + public async Task Probe_model_is_served_from_cache_while_it_is_warm() + { + var (driver, client) = await InitializedDriverAsync(); + + _ = await driver.GetOrFetchProbeModelAsync(CancellationToken.None); + + client.ProbeCallCount.ShouldBe(1); + } + + [Fact] + public async Task Fetching_the_probe_model_before_initialize_is_rejected_rather_than_silently_empty() + { + var (driver, _) = NewDriver(); + + await Should.ThrowAsync( + () => driver.GetOrFetchProbeModelAsync(CancellationToken.None)); + } +} -- 2.52.0 From 1bfc1722b35b911b0c567d7d338a8593a4b85ada Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 15:38:55 -0400 Subject: [PATCH 16/34] feat(mtconnect): IReadable via /current, ordered per-ref snapshots (Task 10) One deadline-bounded /current per batch, answered out of that one whole-device document: N references cost one round-trip, one snapshot each, in request order, duplicates included. DEVIATION (deliberate, from the plan's implied shared-index write): the response is indexed into a PER-CALL snapshot index, never into the driver's shared observation index. That index is written by Task 11's /sample pump and is last-write-wins with no timestamp arbitration; a read folding its /current into it would be a second writer racing the first, and /current is frequently the OLDER document (the Agent composes it while the pump's newer delta is in flight). Losing that race rolls a subscribed value backwards and leaves it there until the data item next changes. Both paths coerce through the same MTConnectObservationIndex logic, so a read and a subscription report a given observation identically -- the read path is simply a pure function of (document, authored tags). Failure posture is the inverse of InitializeAsync: nothing but genuine caller cancellation crosses the capability boundary. Unreachable Agent / blown deadline / unusable answer => every ref codes BadCommunicationError and the driver degrades (never downgrading Faulted); no client at all (pre-initialize, faulted, post-shutdown) => BadNotConnected with health untouched. The index's BadWaitingForInitialData / BadNodeIdUnknown / BadNoCommunication / BadNotSupported distinctions pass through unmodified. 21 tests pinning arity, order, duplicates, empty batch, one-round-trip, each Bad code, read-time freshness, the anti-clobber invariant, the deadline, and cancellation-propagates-but-agent-failure-does-not. CannedAgentClient gains a cancellation-observing CurrentGate (still no timers or sleeps of its own). Test csproj takes the AbCip/S7 OTOPCUA0001 NoWarn -- this suite calls the capability directly by design. 329/329 green. --- .../MTConnectDriver.cs | 186 ++++++- .../CannedAgentClient.cs | 22 +- .../MTConnectReadTests.cs | 456 ++++++++++++++++++ ...M.WW.OtOpcUa.Driver.MTConnect.Tests.csproj | 7 + 4 files changed, 658 insertions(+), 13 deletions(-) create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectReadTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index abe20dd0..71d11215 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -42,14 +42,19 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// nextSequence. /// /// -/// Scope. This type currently implements only. -/// IReadable (Task 10), ISubscribable (Task 11), ITagDiscovery -/// (Task 12) and IHostConnectivityProbe/IRediscoverable (Task 13) are added on -/// top of the state this class already caches — the probe model, the Agent -/// instanceId, and the observation index. +/// Scope. This type currently implements and +/// . ISubscribable (Task 11), ITagDiscovery (Task 12) +/// and IHostConnectivityProbe/IRediscoverable (Task 13) are added on top of +/// the state this class already caches — the probe model, the Agent instanceId, and +/// the observation index. +/// +/// +/// The read path never writes the shared observation index — see +/// . That index has exactly one writer, and Task 11's /sample +/// pump is it. /// /// -public sealed class MTConnectDriver : IDriver +public sealed class MTConnectDriver : IDriver, IReadable { /// /// Coarse per-entry byte weights behind . The contract asks @@ -66,6 +71,20 @@ public sealed class MTConnectDriver : IDriver /// Estimated retained bytes per authored . private const int TagBytesPerEntry = 256; + /// + /// The Agent could not be reached, answered unusably, or blew the per-call deadline. Distinct + /// from the index's BadNoCommunication, which means the Agent answered and said the + /// machine has no value — the two failures need different operator responses. + /// + private const uint BadCommunicationError = 0x80050000u; + + /// + /// There is no Agent client at all: the driver has not been initialized, its initialize + /// faulted, or it has been shut down. Deliberately not one of the index's codes — none of + /// them would say "this driver is not running", which is the only true statement available. + /// + private const uint BadNotConnected = 0x808A0000u; + /// /// Config-JSON reader options, mirroring the sibling driver factories. Note there is /// deliberately no JsonStringEnumConverter: enum-carrying DTO fields stay @@ -137,10 +156,16 @@ public sealed class MTConnectDriver : IDriver public string DriverType => "MTConnect"; /// - /// The live dataItemId → snapshot map. Consumed by IReadable (Task 10) and the - /// /sample pump (Task 11); exposed to tests as the observable proof that - /// actually primed it. + /// The live dataItemId → snapshot map that backs the subscription surface: + /// primed by and thereafter written only by the /sample + /// pump (Task 11). Exposed to tests as the observable proof that initialize actually primed + /// it — and, in MTConnectReadTests, that leaves it alone. /// + /// + /// Read does not consume this. indexes its own /current + /// into a per-call snapshot instead, so the pump keeps a single writer — see the failure + /// analysis in 's remarks. + /// internal MTConnectObservationIndex ObservationIndex => Volatile.Read(ref _index); /// @@ -337,6 +362,110 @@ public sealed class MTConnectDriver : IDriver return Task.CompletedTask; } + /// + /// + /// + /// One /current per batch, whatever the batch size. MTConnect's + /// /current answers for the whole device (or the whole Agent), so N references + /// cost one deadline-bounded round-trip and are answered out of that one document. There + /// is no per-reference request to make and none is made. + /// + /// + /// The response is indexed into a per-call snapshot, NOT into the driver's shared + /// observation index. That index is written by the /sample pump (Task 11) and + /// is deliberately last-write-wins with no timestamp arbitration. A read that folded its + /// /current into it would be a second writer racing the first, and the + /// /current document is frequently the older of the two — the Agent + /// composes it while the pump's newer delta is already in flight. Losing that race rolls + /// a subscribed value backwards and leaves it there until the data item next + /// changes, which for an EVENT such as Execution can be hours. Keeping the + /// read path a pure function of (document, authored tags) costs one index construction + /// per batch — negligible beside the HTTP round-trip that precedes it — and makes it + /// impossible for a read to corrupt subscription state. The reference source-of-truth is + /// unchanged either way: both paths coerce through the same + /// logic, so a read and a subscription report a + /// given observation identically. + /// + /// + /// Nothing but genuine caller cancellation crosses this boundary. An unreachable + /// Agent, a blown deadline, an unusable answer — each fails the batch's values + /// (every reference codes ) rather than the batch: an + /// exception here would also fail the references that were perfectly readable, and the + /// node-manager cannot tell a thrown read from a broken driver. This is the opposite + /// posture to , which is specified to throw so the + /// runtime's retry/backoff loop can own recovery. + /// + /// + /// The status-code distinctions the index draws — BadWaitingForInitialData for an + /// authored-but-unreported id, BadNodeIdUnknown for an unknown one, + /// BadNoCommunication for the Agent's UNAVAILABLE — are passed through + /// unmodified. They tell an operator three different things. + /// + /// + /// + /// is null — a caller bug, not device data, and the + /// one thing this method is loud about (matching 's + /// documented posture). + /// + public async Task> ReadAsync( + IReadOnlyList fullReferences, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(fullReferences); + + // Checked before anything else: an empty batch must cost the Agent nothing, and there is no + // failure it could report. + if (fullReferences.Count == 0) + { + return []; + } + + var client = _client; + var options = _options; + + if (client is null) + { + // Before InitializeAsync, after a faulted one, or after ShutdownAsync. Health is left + // exactly as it is: a read attempted against a driver that was never started (or was + // deliberately stopped) is not evidence of degradation. + return BadBatch(fullReferences.Count, BadNotConnected); + } + + try + { + var current = await BoundedAsync(client.CurrentAsync, "/current", options.RequestTimeoutMs, cancellationToken) + .ConfigureAwait(false); + + // Built from the SAME authored tags as the shared index, so the authored-vs-unknown + // distinction and every coercion behave identically — it is a snapshot, not a variant. + var snapshot = new MTConnectObservationIndex(options.Tags); + snapshot.Apply(current); + + var results = new DataValueSnapshot[fullReferences.Count]; + for (var i = 0; i < results.Length; i++) + { + // Get never throws and never returns null, including for a null/blank reference. + results[i] = snapshot.Get(fullReferences[i]); + } + + WriteHealth(new DriverHealth(DriverState.Healthy, DateTime.UtcNow, null)); + + return results; + } + catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) + { + // The caller asked us to stop. This is the ONLY exception that may propagate: a blown + // per-call deadline arrives as the TimeoutException BoundedAsync raises instead, and is + // handled below as the Agent failure it is. + throw; + } + catch (Exception ex) + { + DegradeAfterReadFailure(ex); + + return BadBatch(fullReferences.Count, BadCommunicationError); + } + } + /// /// The /probe device model: the cached copy when warm, otherwise a fresh /// deadline-bounded /probe that re-populates the cache. This is the accessor every @@ -609,6 +738,45 @@ public sealed class MTConnectDriver : IDriver } } + /// + /// One Bad-coded snapshot per requested reference — the shape every read failure degrades + /// to. Arity is preserved because the caller indexes the result positionally against the + /// references it asked for. + /// + private static DataValueSnapshot[] BadBatch(int count, uint status) + { + // No source timestamp: nothing was observed. The server timestamp is when we gave up. + var serverTimestamp = DateTime.UtcNow; + var results = new DataValueSnapshot[count]; + + for (var i = 0; i < count; i++) + { + results[i] = new DataValueSnapshot(null, status, null, serverTimestamp); + } + + return results; + } + + /// + /// Routes a read failure to the health surface (05/STAB-9's fleet-wide shape): log it and + /// degrade, preserving LastSuccessfulRead — the "when did this driver last hear from + /// the Agent" diagnostic. Never downgrades , which is a + /// stronger, config-level signal that a transient read failure must not paper over. + /// + private void DegradeAfterReadFailure(Exception ex) + { + var current = ReadHealth(); + if (current.State != DriverState.Faulted) + { + WriteHealth(new DriverHealth(DriverState.Degraded, current.LastSuccessfulRead, ex.Message)); + } + + _logger.LogWarning( + ex, + "MTConnect driver {DriverInstanceId} could not read /current from {AgentUri}; the batch is reported Bad.", + _driverInstanceId, _options.AgentUri); + } + private static void SafeDispose(IMTConnectAgentClient? client) { if (client is null) diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs index 85acca43..9953754b 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs @@ -81,6 +81,14 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable /// When set, /current throws this instead of answering. public Exception? CurrentFailure { get; set; } + /// + /// When set, /current does not answer until this source completes — an Agent that + /// accepted the request and then went quiet. The wait is cancellation-observing, so the + /// caller's own deadline is what ends it; the test never sleeps and never races a timer it + /// did not set. Leave null for the ordinary immediate answer. + /// + public TaskCompletionSource? CurrentGate { get; set; } + /// Number of /probe requests issued against this client. public int ProbeCallCount => Volatile.Read(ref _probeCallCount); @@ -110,15 +118,21 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable } /// - public Task CurrentAsync(CancellationToken ct) + public async Task CurrentAsync(CancellationToken ct) { ObjectDisposedException.ThrowIf(IsDisposed, this); ct.ThrowIfCancellationRequested(); Interlocked.Increment(ref _currentCallCount); - return CurrentFailure is null - ? Task.FromResult(Current) - : Task.FromException(CurrentFailure); + // The request was accepted; the answer is withheld until the test releases the gate or the + // caller's deadline cancels the token. + var gate = CurrentGate; + if (gate is not null) + { + await gate.Task.WaitAsync(ct).ConfigureAwait(false); + } + + return CurrentFailure is null ? Current : throw CurrentFailure; } /// diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectReadTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectReadTests.cs new file mode 100644 index 00000000..ad9288dd --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectReadTests.cs @@ -0,0 +1,456 @@ +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 10 — 's half: one /current +/// per batch, one snapshot per requested reference in request order, and a Bad code for every +/// shape of failure. Every test runs against ; no socket is +/// opened anywhere in this file. +/// +/// +/// +/// The load-bearing assertions are about arity, order, and not-throwing. The caller +/// (the runtime's dispatch layer) indexes the returned list positionally against the +/// references it asked for, so a length mismatch or a reorder is silent data corruption — +/// every value lands on the wrong tag under Good quality. And every failure this driver can +/// meet at read time (agent down, deadline blown, unparseable answer, unknown id, driver not +/// initialized) must arrive as a Bad-coded snapshot: an exception out of +/// fails the whole batch including the references that +/// were perfectly readable. +/// +/// +/// The one exception that may propagate is genuine caller cancellation, and it is +/// pinned here alongside its opposite (an agent failure under an uncancelled token) — the +/// two are one catch filter apart in the implementation, and getting the filter +/// backwards turns every dead agent into a thrown batch. +/// +/// +public sealed class MTConnectReadTests +{ + private const string AgentUri = "http://fixture-agent:5000"; + + // Canonical Opc.Ua.StatusCodes numerics, restated here so the test asserts the wire value an + // OPC UA client actually sees rather than a driver-private constant it could drift with. + private const uint Good = 0x00000000u; + private const uint BadMask = 0x80000000u; + private const uint BadCommunicationError = 0x80050000u; + private const uint BadNoCommunication = 0x80310000u; + private const uint BadWaitingForInitialData = 0x80320000u; + private const uint BadNodeIdUnknown = 0x80340000u; + private const uint BadNotSupported = 0x803D0000u; + private const uint BadNotConnected = 0x808A0000u; + + /// + /// Authored tags spanning every read outcome the fixtures can produce: a coercible Good + /// value, an UNAVAILABLE one, a TIME_SERIES vector, and one authored id no fixture + /// ever reports. + /// + private static MTConnectDriverOptions Opts(int requestTimeoutMs = 5000) => + new() + { + AgentUri = AgentUri, + RequestTimeoutMs = requestTimeoutMs, + Tags = + [ + new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64), + new MTConnectTagDefinition("dev1_execution", DriverDataType.String), + new MTConnectTagDefinition("dev1_partcount", DriverDataType.Int32), + new MTConnectTagDefinition("dev1_vibration_ts", DriverDataType.Float64, IsArray: true, ArrayDim: 10), + new MTConnectTagDefinition("dev1_never_reported", DriverDataType.Int32), + ], + }; + + private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( + MTConnectDriverOptions? options = null) + { + var client = CannedAgentClient.FromFixtures(); + + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => client), client); + } + + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( + MTConnectDriverOptions? options = null) + { + var (driver, client) = NewDriver(options); + await driver.InitializeAsync("{}", TestContext.Current.CancellationToken); + + return (driver, client); + } + + // ---- arity + order: the contract the caller indexes positionally ---- + + /// The plan's TDD case: one snapshot per ref, in order, absent ⇒ Bad rather than a throw. + [Fact] + public async Task Read_returns_one_snapshot_per_ref_in_order_absent_ref_is_bad() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["dev1_pos", "does-not-exist"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(2); + res[0].StatusCode.ShouldBe(Good); + (res[1].StatusCode & BadMask).ShouldBe(BadMask); + } + + /// + /// A mixed batch requested in an order that matches neither the document order nor any + /// dictionary enumeration order. This is the assertion that catches a driver that answers + /// with the index's own key order instead of the request's. + /// + [Fact] + public async Task Read_answers_in_request_order_not_document_order() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync( + ["dev1_program", "dev1_pos", "does-not-exist", "dev1_partcount", "dev1_execution"], + TestContext.Current.CancellationToken); + + res.Count.ShouldBe(5); + + // dev1_program is observed but NOT authored — String is the only coercion that cannot fail. + res[0].StatusCode.ShouldBe(Good); + res[0].Value.ShouldBe("O1234"); + + res[1].StatusCode.ShouldBe(Good); + res[1].Value.ShouldBe(123.4567d); + + res[2].StatusCode.ShouldBe(BadNodeIdUnknown); + + res[3].StatusCode.ShouldBe(BadNoCommunication); + + res[4].StatusCode.ShouldBe(Good); + res[4].Value.ShouldBe("ACTIVE"); + } + + /// + /// A reference repeated in one request gets a snapshot per occurrence. De-duplicating would + /// shorten the list and shift every later reference onto the wrong tag. + /// + [Fact] + public async Task Read_returns_a_snapshot_per_duplicate_occurrence() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync( + ["dev1_pos", "dev1_execution", "dev1_pos"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(3); + res[0].StatusCode.ShouldBe(Good); + res[0].Value.ShouldBe(123.4567d); + res[1].Value.ShouldBe("ACTIVE"); + res[2].StatusCode.ShouldBe(Good); + res[2].Value.ShouldBe(123.4567d); + } + + /// An empty batch is empty — and, being empty, costs the Agent nothing. + [Fact] + public async Task Read_of_an_empty_batch_is_empty_and_costs_no_round_trip() + { + var (driver, client) = await InitializedDriverAsync(); + var before = client.CurrentCallCount; + + var res = await driver.ReadAsync([], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(0); + client.CurrentCallCount.ShouldBe(before); + } + + /// + /// A batch of N references costs exactly ONE /current. The whole point of reading the + /// Agent's whole-device snapshot is that per-tag round-trips never happen. + /// + [Fact] + public async Task Read_of_many_refs_costs_exactly_one_current_call() + { + var (driver, client) = await InitializedDriverAsync(); + var before = client.CurrentCallCount; + + var res = await driver.ReadAsync( + ["dev1_pos", "dev1_execution", "dev1_partcount", "dev1_program", "does-not-exist", "dev1_pos"], + TestContext.Current.CancellationToken); + + res.Count.ShouldBe(6); + client.CurrentCallCount.ShouldBe(before + 1); + } + + // ---- status-code fidelity: the index's distinctions must survive the trip ---- + + /// A genuinely unknown id — neither authored nor ever observed. + [Fact] + public async Task Read_of_an_unknown_ref_reports_BadNodeIdUnknown() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["nope"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadNodeIdUnknown); + res[0].Value.ShouldBeNull(); + } + + /// + /// An authored id the Agent has never reported is BadWaitingForInitialData, NOT + /// BadNodeIdUnknown — collapsing the two would tell an operator their correctly + /// authored tag does not exist. + /// + [Fact] + public async Task Read_of_an_authored_but_unreported_ref_reports_BadWaitingForInitialData() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["dev1_never_reported"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadWaitingForInitialData); + } + + /// The Agent said UNAVAILABLE: reachable Agent, absent machine value. + [Fact] + public async Task Read_of_an_unavailable_ref_reports_BadNoCommunication() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["dev1_partcount"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadNoCommunication); + res[0].Value.ShouldBeNull(); + res[0].SourceTimestampUtc.ShouldNotBeNull(); + } + + /// A TIME_SERIES vector is real data this build cannot represent (P1.5 deferral). + [Fact] + public async Task Read_of_a_time_series_ref_reports_BadNotSupported() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["dev1_vibration_ts"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadNotSupported); + } + + /// A blank reference is data, not a programming error — it codes Bad, it does not throw. + [Fact] + public async Task Read_of_a_blank_ref_is_bad_not_a_throw() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["", " ", "dev1_pos"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(3); + res[0].StatusCode.ShouldBe(BadNodeIdUnknown); + res[1].StatusCode.ShouldBe(BadNodeIdUnknown); + res[2].StatusCode.ShouldBe(Good); + } + + /// The authored type is honoured: the Agent's text becomes a CLR value of that type. + [Fact] + public async Task Read_coerces_the_agent_text_to_the_authored_type() + { + var (driver, _) = await InitializedDriverAsync(); + + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(Good); + res[0].Value.ShouldBeOfType().ShouldBe(123.4567d); + res[0].SourceTimestampUtc.ShouldBe(new DateTime(2026, 7, 24, 12, 0, 0, 200, DateTimeKind.Utc)); + } + + // ---- freshness: a read is a read, not a replay of the initialize baseline ---- + + /// + /// The whole justification for spending a round-trip: an OPC UA Read must reflect the + /// device's current state, not whatever + /// happened to prime. A driver that served the primed index would pass every other test in + /// this file. + /// + [Fact] + public async Task Read_reflects_the_agent_state_at_read_time_not_the_initialize_baseline() + { + var (driver, client) = await InitializedDriverAsync(); + + // The Agent has moved on since the priming /current (sample.xml reports 124.0100). + client.Current = CannedAgentClient.Chunk("Fixtures/sample.xml"); + + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(Good); + res[0].Value.ShouldBe(124.01d); + } + + /// + /// The anti-clobber pin. ReadAsync indexes its /current into a + /// per-call snapshot and never writes the driver's shared observation index — that index has + /// exactly one writer, the Task 11 /sample pump. If a read wrote it, a /current + /// document older than the pump's newest delta would silently roll a subscribed value + /// backwards and leave it there until the next change. + /// + [Fact] + public async Task Read_does_not_write_the_shared_observation_index() + { + var (driver, client) = await InitializedDriverAsync(); + var primed = driver.ObservationIndex.Get("dev1_pos"); + + client.Current = CannedAgentClient.Chunk("Fixtures/sample.xml"); + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + // The read saw the new value... + res[0].Value.ShouldBe(124.01d); + + // ...and the shared index still holds exactly what the pump/priming left there. + var after = driver.ObservationIndex.Get("dev1_pos"); + after.Value.ShouldBe(primed.Value); + after.Value.ShouldBe(123.4567d); + } + + // ---- failure posture: Bad snapshots, never exceptions ---- + + /// + /// An unreachable Agent fails the batch's values, not the batch. Every reference codes + /// BadCommunicationError and the call returns normally. + /// + [Fact] + public async Task Read_codes_every_ref_bad_when_the_agent_call_fails() + { + var (driver, client) = await InitializedDriverAsync(); + client.CurrentFailure = new HttpRequestException("connection refused"); + + var res = await driver.ReadAsync( + ["dev1_pos", "dev1_execution", "does-not-exist"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(3); + res.ShouldAllBe(r => r.StatusCode == BadCommunicationError); + res.ShouldAllBe(r => r.Value == null); + } + + /// An answer the driver cannot make sense of is a Bad batch, not a thrown one. + [Fact] + public async Task Read_codes_every_ref_bad_when_the_agent_answer_is_unusable() + { + var (driver, client) = await InitializedDriverAsync(); + client.CurrentFailure = new System.Xml.XmlException("unexpected end of document"); + + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadCommunicationError); + } + + /// + /// A failed read degrades the driver rather than leaving it advertising Healthy, and a later + /// successful read restores it. LastSuccessfulRead survives the degradation — it is + /// the "when did this driver last hear from the Agent" diagnostic. + /// + [Fact] + public async Task Read_failure_degrades_the_driver_and_a_later_success_restores_it() + { + var (driver, client) = await InitializedDriverAsync(); + client.CurrentFailure = new HttpRequestException("connection refused"); + + await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + var degraded = driver.GetHealth(); + degraded.State.ShouldBe(DriverState.Degraded); + degraded.LastSuccessfulRead.ShouldNotBeNull(); + degraded.LastError.ShouldNotBeNullOrWhiteSpace(); + + client.CurrentFailure = null; + await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + /// + /// R2-01 — the /current runs under a bounded deadline. An Agent that accepts the + /// request and never answers surfaces as a Bad batch on the driver's own clock rather than + /// wedging the caller forever. + /// + [Fact] + public async Task Read_is_bounded_by_the_per_call_deadline() + { + var (driver, client) = await InitializedDriverAsync(Opts(requestTimeoutMs: 50)); + var gate = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + client.CurrentGate = gate; + + try + { + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(1); + res[0].StatusCode.ShouldBe(BadCommunicationError); + } + finally + { + gate.TrySetResult(); + } + } + + /// + /// The one exception that may cross the capability boundary. Note the contrast with + /// : same code path, one + /// catch filter apart. + /// + [Fact] + public async Task Read_propagates_genuine_caller_cancellation() + { + var (driver, _) = await InitializedDriverAsync(); + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + await Should.ThrowAsync( + () => driver.ReadAsync(["dev1_pos"], cts.Token)); + } + + // ---- reads outside the initialized window ---- + + /// + /// A read before InitializeAsync has no Agent to ask. It reports + /// BadNotConnected for every reference — it does not throw, it does not + /// NullReferenceException, and it does not invent a value. + /// + [Fact] + public async Task Read_before_initialize_is_bad_not_connected() + { + var (driver, client) = NewDriver(); + + var res = await driver.ReadAsync(["dev1_pos", "nope"], TestContext.Current.CancellationToken); + + res.Count.ShouldBe(2); + res.ShouldAllBe(r => r.StatusCode == BadNotConnected); + client.CurrentCallCount.ShouldBe(0); + + // An un-started driver is Unknown, not Degraded: nothing has failed yet. + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + } + + /// + /// Same posture after ShutdownAsync, and shutting down must stay reported as + /// — a read attempted against a deliberately stopped + /// driver is not a degradation. + /// + [Fact] + public async Task Read_after_shutdown_is_bad_not_connected_and_leaves_health_alone() + { + var (driver, _) = await InitializedDriverAsync(); + await driver.ShutdownAsync(TestContext.Current.CancellationToken); + + var res = await driver.ReadAsync(["dev1_pos"], TestContext.Current.CancellationToken); + + res[0].StatusCode.ShouldBe(BadNotConnected); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + } + + /// + /// A null reference list is a caller bug, not device data — the one place this method + /// is allowed to be loud, matching 's documented + /// posture ("the only exceptions raised are ArgumentNullException for a null argument"). + /// + [Fact] + public async Task Read_of_a_null_ref_list_throws_ArgumentNullException() + { + var (driver, _) = await InitializedDriverAsync(); + + await Should.ThrowAsync( + () => driver.ReadAsync(null!, TestContext.Current.CancellationToken)); + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj index 02432590..866f7a9e 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests.csproj @@ -1,5 +1,12 @@ + + + $(NoWarn);OTOPCUA0001 + + net10.0 enable -- 2.52.0 From a127954cab6236ec7cbc24d3cd086073482bd622 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 15:52:29 -0400 Subject: [PATCH 17/34] fix(mtconnect): review remediation for the driver lifecycle (Task 9) Important 1 - the probe-model triple was read outside _lifecycle while every write happened under it. Packed (Client, ProbeModel, ProbeModelDataItemCount) into one immutable AgentSession swapped by a single Volatile.Write, matching what _index already does, rather than locking the readers: both await the network, so taking _lifecycle would stall a shutdown for a whole RequestTimeoutMs and would widen the non-reentrancy hazard. GetOrFetch and Flush re-publish via CompareExchange so a late re-cache cannot undo a concurrent teardown/re-init, and a client disposed mid-fetch now surfaces as the same "not connected" InvalidOperationException browse already handles instead of a raw ObjectDisposedException. Important 2 - HasConfigBody decided emptiness by matching the literals "{}" / "[]", so "{ }", "{\n}" and every pretty-printed empty document fell through to ParseOptions, failed the required-AgentUri check, and turned a semantically empty config into a cold-start fault. Now parsed: an object/array with no elements (or a bare null) is empty; malformed text is deliberately NOT empty so ParseOptions produces the real quoted error rather than silently starting on stale options. Important 3 - documented the _lifecycle non-reentrancy hazard in the code, on both the semaphore and StopSampleStreamAsync, naming Task 11's re-baseline as the specific path that would deadlock and giving the CurrentAsync-directly pattern that avoids it. Important 4 - removed the Initialize/Reinitialize asymmetry rather than documenting it. Both now share one rule via ResolveIncomingOptions: an unreadable config document never destroys working state, and faults only a driver that had none. InitializeAsync previously tore down before parsing, so a bad document on a live instance destroyed a healthy client - the exact outcome ReinitializeAsync was written to avoid. Minors: SafeDispose traces the swallowed disposal fault at Debug (a client that cannot be released is how a handle leak starts); the RequirePositive test is a Theory over all four timing knobs, not just RequestTimeoutMs (arch-review 01/S-6); the footprint test no longer overclaims "is zero before initialize". CannedAgentClient gains a one-shot ProbeGate so a lifecycle change landing mid-request is deterministic - no timers. 346/346 (329 + 17). Six mutations verified: fetch-CAS, disposed-translation, flush retired-session guard, literal HasConfigBody, teardown-before-parse, and two dropped RequirePositive calls each fail only their own tests. --- .../MTConnectDriver.cs | 388 ++++++++++++++---- .../CannedAgentClient.cs | 33 +- .../MTConnectDriverLifecycleTests.cs | 210 +++++++++- 3 files changed, 527 insertions(+), 104 deletions(-) diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index 71d11215..8dcc9f68 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -98,6 +98,17 @@ public sealed class MTConnectDriver : IDriver, IReadable AllowTrailingCommas = true, }; + /// + /// Reader options for the emptiness probe in . Kept in step with + /// so a document the real parse would accept is never judged + /// malformed here (a stray trailing comma must not change which branch a config takes). + /// + private static readonly JsonDocumentOptions DocumentOptions = new() + { + CommentHandling = JsonCommentHandling.Skip, + AllowTrailingCommas = true, + }; + private readonly string _driverInstanceId; private readonly ILogger _logger; private readonly Func _agentClientFactory; @@ -107,16 +118,76 @@ public sealed class MTConnectDriver : IDriver, IReadable /// / Shutdown all swap the client and the served caches; overlapping them would let a /// shutdown dispose a client an in-flight initialize is about to publish. /// + /// + /// + /// ⚠️ NON-REENTRANT — never call a public lifecycle method from code that already + /// holds this. has no owner tracking, so a re-entrant + /// call does not fail: it deadlocks the driver permanently, with no exception and + /// no log line to diagnose it from. + /// + /// + /// This is aimed squarely at the /sample pump (Task 11). Its ring-buffer + /// re-baseline ("on a sequence gap, re-/current and resume") is naturally written + /// as "just call the existing re-init logic" — and + /// / both take this + /// semaphore, so from inside the pump loop that is a hang, not a re-prime. Re-baseline + /// must instead call directly on the + /// client the pump already holds (through , to keep the + /// deadline) and Apply the result to . Any future + /// shared step belongs in a private, non-locking helper that both the pump and the + /// lock-holding lifecycle methods can call — never in the public entry points. + /// + /// + /// The inverse direction is already safe: the lifecycle methods call + /// while holding this, so that method — and whatever + /// Task 11 fills it with — must never re-enter the lifecycle either. + /// + /// private readonly SemaphoreSlim _lifecycle = new(1, 1); private MTConnectDriverOptions _options; - private IMTConnectAgentClient? _client; private MTConnectObservationIndex _index; - private MTConnectProbeModel? _probeModel; - private int _probeModelDataItemCount; private long? _agentInstanceId; private DriverHealth _health = new(DriverState.Unknown, null, null); + /// + /// The live agent client plus the /probe cache derived from it, held as one + /// immutable snapshot behind a single reference and only ever replaced by a + /// of a whole new instance — the same discipline + /// uses, and for the same reason. + /// + /// + /// + /// Why one field instead of three. These three values are read outside + /// — by , by + /// (Task 12's browse), and by + /// — while every write happens under it. As three + /// separate fields they could be observed mid-update: a reader could see a dropped + /// ProbeModel beside a stale non-zero data-item count and report a footprint for + /// a cache that no longer exists, or pair one initialize's client with the previous + /// one's model. Packing them makes every observation internally consistent by + /// construction, with no lock on the read path. + /// + /// + /// Why not simply take the lock in those readers. Both readers await the + /// network, so they would hold the lifecycle lock across an HTTP round trip — a browse + /// could stall a for a full RequestTimeoutMs — and + /// every added lock site widens the non-reentrancy hazard documented on + /// . It would also make the read path inconsistent with + /// , which is deliberately lock-free. + /// + /// + /// What this does NOT fix. A reader can still capture a session moments before a + /// concurrent teardown disposes its client, and then call it — no lock-free scheme can + /// prevent that, and holding the lock across the call is the trade rejected above. + /// What it guarantees is that the failure is named: the resulting + /// is caught at the seam and reported as the same + /// "driver is not connected" error a caller already has to handle, rather than escaping + /// raw out of a browse. + /// + /// + private AgentSession? _session; + /// Creates a driver instance. Opens no connection and builds no agent client. /// /// The typed configuration this instance starts from. A config document supplied to @@ -173,7 +244,7 @@ public sealed class MTConnectDriver : IDriver, IReadable /// dropped by . Prefer /// , which cannot observe the flushed hole. /// - internal MTConnectProbeModel? CachedProbeModel => _probeModel; + internal MTConnectProbeModel? CachedProbeModel => Volatile.Read(ref _session)?.ProbeModel; /// /// The Agent's instanceId as of the last successful /current. Task 13 watches @@ -198,28 +269,15 @@ public sealed class MTConnectDriver : IDriver, IReadable await _lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false); try { + // Parse BEFORE tearing down — see ResolveIncomingOptions. Initialize is reachable on a + // live instance (DriverInstanceActor re-initialises in Connecting/Reconnecting), and + // destroying a working client before discovering the new document is unreadable would + // be the exact outcome ReinitializeAsync is written to avoid. + var options = ResolveIncomingOptions(driverConfigJson); + // A re-Initialize on a live instance must not orphan the previous client. await TeardownCoreAsync().ConfigureAwait(false); - MTConnectDriverOptions options; - try - { - options = ResolveOptions(driverConfigJson); - } - catch (Exception ex) - { - // A config document that will not parse is just as fatal to a cold start as an - // unreachable Agent, and it must reach GetHealth() the same way — otherwise the - // dashboard keeps showing whatever state the driver was in before. - WriteHealth(new DriverHealth(DriverState.Faulted, ReadHealth().LastSuccessfulRead, ex.Message)); - _logger.LogError( - ex, - "MTConnect driver {DriverInstanceId} could not read its driver config document; the driver is Faulted.", - _driverInstanceId); - - throw; - } - await StartCoreAsync(options, existingClient: null, cancellationToken).ConfigureAwait(false); } finally @@ -257,27 +315,9 @@ public sealed class MTConnectDriver : IDriver, IReadable await _lifecycle.WaitAsync(cancellationToken).ConfigureAwait(false); try { - MTConnectDriverOptions incoming; - try - { - incoming = ResolveOptions(driverConfigJson); - } - catch (Exception ex) - { - // Deliberately NOT Faulted, and deliberately before any teardown: the running - // driver is still serving its previous, valid configuration. Rejecting the edit - // fails the deployment (DriverInstanceActor turns this into ApplyResult(false)); - // downing a working driver over an unreadable document would be a far larger blast - // radius than the change that was refused. - _logger.LogError( - ex, - "MTConnect driver {DriverInstanceId} rejected a new driver config document and kept running its previous configuration.", - _driverInstanceId); + var incoming = ResolveIncomingOptions(driverConfigJson); - throw; - } - - var existing = _client; + var existing = Volatile.Read(ref _session)?.Client; await StopSampleStreamAsync().ConfigureAwait(false); @@ -333,7 +373,7 @@ public sealed class MTConnectDriver : IDriver, IReadable /// constants. /// public long GetMemoryFootprint() => - ((long)_probeModelDataItemCount * ProbeModelBytesPerDataItem) + ((long)(Volatile.Read(ref _session)?.ProbeModelDataItemCount ?? 0) * ProbeModelBytesPerDataItem) + ((long)ObservationIndex.Count * IndexBytesPerEntry) + ((long)_options.Tags.Count * TagBytesPerEntry); @@ -344,21 +384,36 @@ public sealed class MTConnectDriver : IDriver, IReadable /// (it is the read surface, and re-deriving it would mean a full re-prime the caller did not /// ask for). Flushing cannot wedge the driver: every consumer of the model goes through /// , which re-fetches and re-caches on a miss. + /// + /// The drop is a single of a whole new + /// , so the model and its data-item count can never be seen + /// out of step (which would skew ), and a flush that + /// races a concurrent teardown or re-initialize is discarded rather than resurrecting + /// the session it was reading. + /// /// public Task FlushOptionalCachesAsync(CancellationToken cancellationToken) { - var dropped = _probeModelDataItemCount; - - _probeModel = null; - _probeModelDataItemCount = 0; - - if (dropped > 0) + var session = Volatile.Read(ref _session); + if (session?.ProbeModel is null) { - _logger.LogInformation( - "MTConnect driver {DriverInstanceId} flushed its cached /probe device model ({DataItemCount} data items); it is re-fetched on the next browse.", - _driverInstanceId, dropped); + return Task.CompletedTask; } + var flushed = session with { ProbeModel = null, ProbeModelDataItemCount = 0 }; + if (!ReferenceEquals(Interlocked.CompareExchange(ref _session, flushed, session), session)) + { + // Someone swapped the session while we were building the flushed copy — their state is + // newer than ours, and publishing would undo it. Whatever they installed is either a + // teardown (no cache to flush) or a fresh initialize (a cache the operator's budget + // breach never saw), so dropping this flush is correct, not merely safe. + return Task.CompletedTask; + } + + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} flushed its cached /probe device model ({DataItemCount} data items); it is re-fetched on the next browse.", + _driverInstanceId, session.ProbeModelDataItemCount); + return Task.CompletedTask; } @@ -419,7 +474,9 @@ public sealed class MTConnectDriver : IDriver, IReadable return []; } - var client = _client; + // One read of the session reference: the client used below is the one that was live at this + // instant, never a half-swapped pairing (see the _session remarks). + var client = Volatile.Read(ref _session)?.Client; var options = _options; if (client is null) @@ -472,29 +529,54 @@ public sealed class MTConnectDriver : IDriver, IReadable /// model consumer (Task 12's DiscoverAsync) must use, so that /// can never leave browse permanently disabled. /// + /// + /// Lock-free by design — see the remarks for why this does not take + /// . It reads the session once, fetches against that + /// session's client, and re-caches only if that same session is still installed, so a + /// concurrent / can neither be + /// undone by a late re-cache nor leak a model belonging to a client that is gone. A teardown + /// that lands mid-fetch surfaces as the below rather + /// than as a raw from the disposed client. + /// /// Cancellation token. /// The Agent's device model. - /// The driver is not initialized. + /// + /// The driver is not initialized, or it was torn down while this fetch was in flight. + /// internal async Task GetOrFetchProbeModelAsync(CancellationToken cancellationToken) { - var cached = _probeModel; - if (cached is not null) + var session = Volatile.Read(ref _session) ?? throw NotConnected(); + + if (session.ProbeModel is not null) { - return cached; + return session.ProbeModel; } - var client = _client - ?? throw new InvalidOperationException( - $"MTConnect driver '{_driverInstanceId}' has no agent client; the /probe device model cannot be fetched before InitializeAsync succeeds."); + MTConnectProbeModel model; + try + { + model = await BoundedAsync(session.Client.ProbeAsync, "/probe", _options.RequestTimeoutMs, cancellationToken) + .ConfigureAwait(false); + } + catch (ObjectDisposedException ex) + { + // The lock-free window the _session remarks name: a teardown disposed this client after + // we captured it. Reported as the not-connected error the caller already handles, so + // browse has exactly one failure mode to reason about. + throw NotConnected(ex); + } - var model = await BoundedAsync(client.ProbeAsync, "/probe", _options.RequestTimeoutMs, cancellationToken) - .ConfigureAwait(false); - - CacheProbeModel(model); + // Publish only onto the session we actually fetched against. + var recached = session with { ProbeModel = model, ProbeModelDataItemCount = CountDataItems(model) }; + _ = Interlocked.CompareExchange(ref _session, recached, session); return model; } + private InvalidOperationException NotConnected(Exception? inner = null) => + new($"MTConnect driver '{_driverInstanceId}' has no live agent client; the /probe device model cannot be fetched unless InitializeAsync has succeeded and the driver has not since been shut down or re-initialized.", + inner); + /// /// Builds driver options from a DriverConfig JSON document. /// @@ -603,10 +685,11 @@ public sealed class MTConnectDriver : IDriver, IReadable // ---- commit point: everything below is non-throwing bookkeeping ---- _options = options; - _client = client; built = null; - CacheProbeModel(probe); + // One publish, so a concurrent lock-free reader sees the new client and the new probe + // cache together or neither of them. + Volatile.Write(ref _session, new AgentSession(client, probe, CountDataItems(probe))); _agentInstanceId = current.InstanceId; var index = new MTConnectObservationIndex(options.Tags); @@ -649,12 +732,13 @@ public sealed class MTConnectDriver : IDriver, IReadable /// private Task TeardownCoreAsync() { - var client = _client; - _client = null; - SafeDispose(client); + // Retire the whole session in one write, BEFORE disposing: a lock-free reader that still + // holds the old reference gets a disposed client (caught + named at each seam), never a + // session that is half-torn-down. + var session = Volatile.Read(ref _session); + Volatile.Write(ref _session, null); + SafeDispose(session?.Client); - _probeModel = null; - _probeModelDataItemCount = 0; _agentInstanceId = null; // A fresh index rather than Clear(): the tag table it coerces against belongs to the options @@ -670,17 +754,95 @@ public sealed class MTConnectDriver : IDriver, IReadable /// must stop the stream before the client they are about /// to dispose goes away, and that ordering is easy to lose when the pump is bolted on later. /// + /// + /// ⚠️ This runs while is held. Whatever Task 11 fills it with + /// — cancelling the pump's token, awaiting its task — must not call back into + /// , or + /// , directly or by awaiting a pump that is itself blocked on one + /// of them: the semaphore is non-reentrant and the result is a permanent hang with no + /// exception and no log. See the remarks for the re-baseline + /// pattern that avoids this. + /// private static Task StopSampleStreamAsync() => Task.CompletedTask; - /// The document's options when it carries a body; otherwise the ones already in force. - private MTConnectDriverOptions ResolveOptions(string driverConfigJson) => - HasConfigBody(driverConfigJson) ? ParseOptions(_driverInstanceId, driverConfigJson) : _options; + /// + /// Resolves the options a lifecycle call should apply, reporting an unreadable document + /// consistently for both entry points before any state is destroyed. + /// + /// + /// + /// One rule, deliberately shared by and + /// : a config document that cannot be read never + /// destroys working state; it faults only a driver that had none. + /// + /// + /// With a live session the driver keeps serving its previous, valid configuration and + /// health is left alone — rejecting the edit already fails the deployment + /// (DriverInstanceActor turns the throw into ApplyResult(false)), and + /// downing a working driver over an unreadable document is a far larger blast radius + /// than the change that was refused. Publishing while + /// still serving values would also be a lie in the other direction. + /// + /// + /// With no live session there is nothing to protect and every tag is already dark, so + /// the failure must reach as — + /// otherwise the dashboard keeps showing whatever state the driver was in before, which + /// is the failure-that-looks-like-success shape this codebase exists to avoid. + /// + /// + private MTConnectDriverOptions ResolveIncomingOptions(string driverConfigJson) + { + try + { + return HasConfigBody(driverConfigJson) + ? ParseOptions(_driverInstanceId, driverConfigJson) + : _options; + } + catch (Exception ex) + { + if (Volatile.Read(ref _session) is not null) + { + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} rejected a new driver config document and kept running its previous configuration.", + _driverInstanceId); + } + else + { + WriteHealth(new DriverHealth(DriverState.Faulted, ReadHealth().LastSuccessfulRead, ex.Message)); + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} could not read its driver config document and has no running configuration to fall back on; the driver is Faulted.", + _driverInstanceId); + } + + throw; + } + } /// - /// True when carries a real config body. The - /// bootstrapper always passes a populated document; tests and probe-only callers pass - /// "{}", "[]" or blank, which keep the constructor-supplied options. + /// True when carries a real config body — i.e. a JSON + /// object or array with at least one element. The bootstrapper always passes a + /// populated document; tests and probe-only callers pass a semantically empty one, which + /// keeps the constructor-supplied options. /// + /// + /// + /// Emptiness is decided by parsing, not by string comparison. Matching the + /// literals "{}" / "[]" misses every other spelling of the same document — + /// "{ }", a pretty-printed "{\n}", "{ /* nothing yet */ }" — and + /// each of those would then reach , fail the required-AgentUri + /// check, and turn a semantically empty document into a cold-start fault or a rejected + /// deployment. That is precisely the asymmetry the keep-the-constructor-options rule + /// exists to prevent. + /// + /// + /// A malformed document is emphatically NOT empty and returns true, so + /// runs and produces the real, quoted parse error. Treating + /// unreadable text as "no config supplied" would silently swallow a corrupt document and + /// start the driver on stale options — a failure wearing success's clothes. + /// + /// private static bool HasConfigBody(string? driverConfigJson) { if (string.IsNullOrWhiteSpace(driverConfigJson)) @@ -688,9 +850,26 @@ public sealed class MTConnectDriver : IDriver, IReadable return false; } - var trimmed = driverConfigJson.Trim(); + try + { + using var document = JsonDocument.Parse(driverConfigJson, DocumentOptions); - return trimmed is not "{}" and not "[]"; + return document.RootElement.ValueKind switch + { + JsonValueKind.Object => document.RootElement.EnumerateObject().Any(), + JsonValueKind.Array => document.RootElement.EnumerateArray().Any(), + + // A bare `null` document is the JSON spelling of "nothing supplied". + JsonValueKind.Null => false, + + // A scalar root is not a config object at all; let ParseOptions say so. + _ => true, + }; + } + catch (JsonException) + { + return true; + } } /// @@ -706,13 +885,12 @@ public sealed class MTConnectDriver : IDriver, IReadable || current.SampleIntervalMs != incoming.SampleIntervalMs || current.SampleCount != incoming.SampleCount; - private void CacheProbeModel(MTConnectProbeModel model) - { - _probeModel = model; - - // Counted once here rather than re-walked on every 30 s footprint poll. - _probeModelDataItemCount = model.Devices.Sum(d => d.AllDataItems().Count()); - } + /// + /// Counts every data item the device model declares. Done once, when the model is cached, + /// rather than re-walking the tree on every 30 s poll. + /// + private static int CountDataItems(MTConnectProbeModel model) => + model.Devices.Sum(d => d.AllDataItems().Count()); /// /// Runs one unary agent call under a deadline linked to the caller's token. Belt-and-braces @@ -777,7 +955,15 @@ public sealed class MTConnectDriver : IDriver, IReadable _driverInstanceId, _options.AgentUri); } - private static void SafeDispose(IMTConnectAgentClient? client) + /// + /// Disposes an agent client without letting a disposal fault escape. Teardown runs on the + /// failure path of , where a second exception would replace the + /// real one — but the swallowed exception is still traced rather than vanishing: a + /// client that cannot be released is how a socket-handle leak starts, and an empty catch is + /// the only place in this driver where a failure would leave no evidence at all. Debug + /// level, because it is diagnostic detail about an already-reported failure. + /// + private void SafeDispose(IMTConnectAgentClient? client) { if (client is null) { @@ -788,9 +974,12 @@ public sealed class MTConnectDriver : IDriver, IReadable { client.Dispose(); } - catch (Exception) + catch (Exception ex) { - // Teardown runs on failure paths; a disposal fault must not replace the original error. + _logger.LogDebug( + ex, + "MTConnect driver {DriverInstanceId} swallowed a fault while disposing its agent client during teardown; the client may not have released its connections.", + _driverInstanceId); } } @@ -847,6 +1036,25 @@ public sealed class MTConnectDriver : IDriver, IReadable CultureInfo.InvariantCulture, $"MTConnect driver config for '{driverInstanceId}' tag '{tagName}' has an unrecognised {field} '{raw}'. Expected one of: {string.Join(", ", Enum.GetNames())}.")); + /// + /// The live agent client and the /probe cache derived from it, as one immutable + /// unit. Immutable so that publishing a change is a single reference swap — see the + /// remarks for why the three values must move together. + /// + /// The agent client this session owns and will dispose on teardown. + /// + /// The cached device model, or null once has + /// dropped it. Never null at the moment a session is first published. + /// + /// + /// Data items declared by , counted once at cache time and + /// always 0 when it is null. + /// + private sealed record AgentSession( + IMTConnectAgentClient Client, + MTConnectProbeModel? ProbeModel, + int ProbeModelDataItemCount); + // ---- config DTOs ---- /// diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs index 9953754b..71fe5e52 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs @@ -81,6 +81,18 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable /// When set, /current throws this instead of answering. public Exception? CurrentFailure { get; set; } + /// + /// When set, the next /probe parks here until the test completes it, then the + /// gate clears itself so later calls answer immediately. + /// + /// + /// This is how a test holds a request "in flight" across a concurrent lifecycle change — + /// a shutdown or a re-initialize landing mid-fetch — with no timers and no races of its own. + /// The answer is captured when the request lands, not when it completes, so a test can + /// change meanwhile and still tell the two documents apart. + /// + public TaskCompletionSource? ProbeGate { get; set; } + /// /// When set, /current does not answer until this source completes — an Agent that /// accepted the request and then went quiet. The wait is cancellation-observing, so the @@ -108,13 +120,30 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable public long? LastSampleFrom { get; private set; } /// - public Task ProbeAsync(CancellationToken ct) + public async Task ProbeAsync(CancellationToken ct) { ObjectDisposedException.ThrowIf(IsDisposed, this); ct.ThrowIfCancellationRequested(); Interlocked.Increment(ref _probeCallCount); - return ProbeFailure is null ? Task.FromResult(Probe) : Task.FromException(ProbeFailure); + if (ProbeFailure is not null) + { + throw ProbeFailure; + } + + // Captured now: the Agent answers with the document it held when the request landed. + var answer = Probe; + + if (ProbeGate is { } gate) + { + ProbeGate = null; + await gate.Task.WaitAsync(ct).ConfigureAwait(false); + + // A real client whose handler was disposed mid-request fails the request; so does this. + ObjectDisposedException.ThrowIf(IsDisposed, this); + } + + return answer; } /// diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs index d08036b3..197f429e 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverLifecycleTests.cs @@ -26,12 +26,18 @@ public sealed class MTConnectDriverLifecycleTests private static MTConnectDriverOptions Opts( string agentUri = AgentUri, string? deviceName = null, - int requestTimeoutMs = 5000) => + int requestTimeoutMs = 5000, + int heartbeatMs = 10000, + int sampleIntervalMs = 1000, + int sampleCount = 1000) => new() { AgentUri = agentUri, DeviceName = deviceName, RequestTimeoutMs = requestTimeoutMs, + HeartbeatMs = heartbeatMs, + SampleIntervalMs = sampleIntervalMs, + SampleCount = sampleCount, Tags = [ new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64), @@ -176,23 +182,45 @@ public sealed class MTConnectDriverLifecycleTests client.IsDisposed.ShouldBeTrue(); } - [Fact] - public async Task Initialize_rejects_a_non_positive_request_timeout_before_dialling() + /// + /// arch-review 01/S-6: an operator-authorable 0 must never come to mean "wait + /// forever" (or "ask the Agent for nothing"). All four knobs share one + /// RequirePositive path, so all four are asserted — a guard that covered only the + /// one knob with a test is exactly how three of them would quietly lose the check. + /// + /// + /// Asserted through a FAKE client factory, so this proves the DRIVER validates rather than + /// leaning on the production client's own constructor guard. + /// + [Theory] + [InlineData("RequestTimeoutMs")] + [InlineData("HeartbeatMs")] + [InlineData("SampleIntervalMs")] + [InlineData("SampleCount")] + public async Task Initialize_rejects_a_non_positive_timing_knob_before_dialling(string knob) { - // arch-review 01/S-6: an operator-authorable 0 must never come to mean "wait forever". - // Asserted through a FAKE client factory, so this proves the DRIVER validates rather than - // relying on the production client's own constructor guard. + var options = knob switch + { + "RequestTimeoutMs" => Opts(requestTimeoutMs: 0), + "HeartbeatMs" => Opts(heartbeatMs: 0), + "SampleIntervalMs" => Opts(sampleIntervalMs: 0), + "SampleCount" => Opts(sampleCount: 0), + _ => throw new ArgumentOutOfRangeException(nameof(knob), knob, "unhandled knob"), + }; + var factoryCalls = 0; - var driver = new MTConnectDriver(Opts(requestTimeoutMs: 0), "mt1", _ => + var driver = new MTConnectDriver(options, "mt1", _ => { factoryCalls++; return CannedAgentClient.FromFixtures(); }); - await Should.ThrowAsync( + var thrown = await Should.ThrowAsync( () => driver.InitializeAsync("{}", CancellationToken.None)); + // The message must name the offending key, or an operator cannot act on it. + thrown.Message.ShouldContain(knob); factoryCalls.ShouldBe(0); driver.GetHealth().State.ShouldBe(DriverState.Faulted); } @@ -222,8 +250,24 @@ public sealed class MTConnectDriverLifecycleTests seen.DeviceName.ShouldBe("VMC-3Axis"); } - [Fact] - public async Task Initialize_with_an_empty_config_body_keeps_the_constructor_options() + /// + /// "Semantically empty" is decided by parsing, not by matching the literal two-character + /// spellings. A pretty-printer, a formatter, or a hand edit turns {} into { } + /// or {\n} without changing its meaning — and under literal matching each of those + /// reached ParseOptions, failed the required-AgentUri check, and turned an empty document + /// into a cold-start fault or a rejected deployment. + /// + [Theory] + [InlineData("{}")] + [InlineData("{ }")] + [InlineData("{\n}")] + [InlineData(" {\r\n\t} ")] + [InlineData("[]")] + [InlineData("[ ]")] + [InlineData("null")] + [InlineData("")] + [InlineData(" ")] + public async Task Initialize_with_a_semantically_empty_config_body_keeps_the_constructor_options(string json) { MTConnectDriverOptions? seen = null; var driver = new MTConnectDriver(Opts(), "mt1", o => @@ -233,11 +277,26 @@ public sealed class MTConnectDriverLifecycleTests return CannedAgentClient.FromFixtures(); }); - await driver.InitializeAsync("{}", CancellationToken.None); + await driver.InitializeAsync(json, CancellationToken.None); seen.ShouldNotBeNull(); seen!.AgentUri.ShouldBe(AgentUri); seen.Tags.Count.ShouldBe(2); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + [Fact] + public async Task Initialize_treats_a_malformed_config_body_as_a_parse_error_not_as_empty() + { + // The other half of the emptiness rule: unreadable text must NOT fall through to "no config + // supplied" and silently start the driver on stale options. + var (driver, _) = NewDriver(); + + var thrown = await Should.ThrowAsync( + () => driver.InitializeAsync("""{"agentUri": """, CancellationToken.None)); + + thrown.Message.ShouldContain("not valid JSON"); + driver.GetHealth().State.ShouldBe(DriverState.Faulted); } [Fact] @@ -251,6 +310,25 @@ public sealed class MTConnectDriverLifecycleTests driver.GetHealth().State.ShouldBe(DriverState.Faulted); } + [Fact] + public async Task Initialize_on_a_live_instance_with_an_unreadable_config_keeps_it_serving() + { + // One rule for both entry points: an unreadable document never destroys working state; it + // faults only a driver that had none. InitializeAsync IS reachable on a live instance + // (DriverInstanceActor re-initialises during Connecting/Reconnecting), so parsing must + // happen BEFORE the teardown — otherwise a bad edit kills a healthy client and the driver + // ends up in exactly the state ReinitializeAsync is written to avoid. + var (driver, client) = await InitializedDriverAsync(); + + await Should.ThrowAsync( + () => driver.InitializeAsync("""{"deviceName":"no-agent-uri"}""", CancellationToken.None)); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + driver.EffectiveOptions.AgentUri.ShouldBe(AgentUri); + client.IsDisposed.ShouldBeFalse(); + driver.ObservationIndex.Count.ShouldBe(FixtureDataItemCount); + } + [Fact] public async Task Initialize_parses_authored_tags_out_of_the_config_json() { @@ -480,7 +558,7 @@ public sealed class MTConnectDriverLifecycleTests // ---- footprint + flush ---- [Fact] - public async Task Memory_footprint_is_zero_before_initialize_and_positive_after() + public async Task Memory_footprint_grows_when_initialize_populates_the_caches() { var (driver, _) = NewDriver(); @@ -545,4 +623,112 @@ public sealed class MTConnectDriverLifecycleTests await Should.ThrowAsync( () => driver.GetOrFetchProbeModelAsync(CancellationToken.None)); } + + [Fact] + public async Task Fetching_the_probe_model_after_shutdown_is_rejected() + { + var (driver, _) = await InitializedDriverAsync(); + await driver.ShutdownAsync(CancellationToken.None); + + await Should.ThrowAsync( + () => driver.GetOrFetchProbeModelAsync(CancellationToken.None)); + } + + // ---- lock-free session snapshot: the probe fetch races the lifecycle ---- + + [Fact] + public async Task A_shutdown_landing_mid_fetch_surfaces_as_not_connected_not_as_object_disposed() + { + // Browse is deliberately lock-free (it awaits the network; holding the lifecycle lock would + // let it stall a shutdown for a whole RequestTimeoutMs), so a fetch CAN outlive the client + // it captured. What must not happen is a raw ObjectDisposedException escaping into browse — + // the caller already handles "not connected" and should have exactly one failure mode. + var (driver, client) = await InitializedDriverAsync(); + await driver.FlushOptionalCachesAsync(CancellationToken.None); + + // Held locally: the gate is one-shot, so the client has already cleared its own reference by + // the time the request is parked on it. + var gate = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + client.ProbeGate = gate; + var inFlight = driver.GetOrFetchProbeModelAsync(CancellationToken.None); + + await driver.ShutdownAsync(CancellationToken.None); + client.IsDisposed.ShouldBeTrue(); + gate.TrySetResult(); + + var thrown = await Should.ThrowAsync(() => inFlight); + thrown.InnerException.ShouldBeOfType(); + } + + [Fact] + public async Task A_fetch_that_completes_after_a_reinitialize_does_not_overwrite_the_newer_cache() + { + // The probe cache is re-published only onto the session it was fetched against. Without + // that check, a slow browse would install a device model belonging to a superseded + // configuration over the one the re-initialize just established — stale browse output that + // nothing would ever correct. + var built = new List(); + var driver = new MTConnectDriver(Opts(), "mt1", _ => + { + var c = CannedAgentClient.FromFixtures(); + built.Add(c); + + return c; + }); + + await driver.InitializeAsync("{}", CancellationToken.None); + await driver.FlushOptionalCachesAsync(CancellationToken.None); + + var client = built[0]; + var staleModel = client.Probe; + + // Park a fetch on the OLD (about to be superseded) session; it captures staleModel now. + // Held locally — the gate is one-shot and clears the client's own reference when it parks. + var gate = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + client.ProbeGate = gate; + var inFlight = driver.GetOrFetchProbeModelAsync(CancellationToken.None); + + // A config-only re-initialize: same client (so nothing is disposed), brand-new session. + var freshModel = new MTConnectProbeModel(staleModel.Devices); + client.Probe = freshModel; + await driver.ReinitializeAsync( + $$"""{"agentUri":"{{AgentUri}}","tags":[{"fullName":"dev1_pos","driverDataType":"Float64"}]}""", + CancellationToken.None); + + built.Count.ShouldBe(1); + driver.CachedProbeModel.ShouldBeSameAs(freshModel); + + // Now let the stale fetch finish. It still returns what the Agent gave it... + gate.TrySetResult(); + (await inFlight).ShouldBeSameAs(staleModel); + + // ...but it must not have published that over the newer session's cache. + driver.CachedProbeModel.ShouldBeSameAs(freshModel); + } + + [Fact] + public async Task Flush_after_shutdown_is_a_no_op_and_does_not_resurrect_the_session() + { + // Core polls the footprint and asks for a flush on its own schedule, so a flush arriving + // after the driver was torn down is ordinary, not exotic. It must observe the retired + // session and do nothing — publishing a "flushed" copy of it would reinstate a session + // holding an already-disposed client for every later caller. + // + // NOTE: this pins the retired-session guard, NOT the CompareExchange beneath it. Flush has + // no await point, so nothing can be interleaved between its read and its publish from a + // single-threaded test; the CAS is defence-in-depth against a genuinely concurrent + // lifecycle call and is deliberately claimed as such rather than as tested behaviour. The + // fetch-side CAS, which does have an await point, is covered by the test above. + var (driver, client) = await InitializedDriverAsync(); + + await driver.ShutdownAsync(CancellationToken.None); + await driver.FlushOptionalCachesAsync(CancellationToken.None); + + driver.CachedProbeModel.ShouldBeNull(); + driver.GetHealth().State.ShouldBe(DriverState.Unknown); + client.DisposeCount.ShouldBe(1); + + await Should.ThrowAsync( + () => driver.GetOrFetchProbeModelAsync(CancellationToken.None)); + } } -- 2.52.0 From 9a67ed918a00cc3993f918f20027deda820c742f Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 16:29:17 -0400 Subject: [PATCH 18/34] feat(mtconnect): ISubscribable /sample pump + ring-buffer re-baseline (Task 11) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One shared /sample long poll behind every subscription handle. Subscribe fires initial data per reference from the primed /current (Part 4 convention) and returns without waiting on the stream; the pump runs on its own task under its own token — never the caller's Subscribe token, which is disposed the moment that call returns. The pump owns every way the sequence can break: - sequence gap, measured against the RUNNING cursor (the previous chunk's nextSequence). Comparing against the opening `from` reports a gap on every chunk once the buffer rolls — a /current re-baseline storm. - OUT_OF_RANGE under HTTP 200 (InvalidDataException out of SampleAsync), which IsSequenceGap cannot see at all — without it a driver recovers from falling a little behind but not from falling a lot. - agent restart, checked BEFORE the gap because a restart resets sequences and so usually trips the gap check too; it clears the index (the held values describe a device model that no longer exists) and tells the subscribers. - every chunk advances the cursor, heartbeats included. Re-baseline calls CurrentAsync directly on the client the pump already holds: routing it through ReinitializeAsync would take the non-reentrant lifecycle semaphore from inside a pump a lifecycle method may already be awaiting, and hang the driver with no exception and no log. StopSampleStreamAsync is filled in (same call sites) and InitializeAsync now stops the stream before teardown too, so a re-Initialize on a live instance cannot leave a pump enumerating a disposed client. MTConnectStreamEnded/TimeoutException reconnect under a geometric backoff with a 100 ms growth floor (MinBackoffMs defaults to 0, and 0 x multiplier is still 0). MTConnectStreamNotSupportedException does NOT retry — it latches, reports loudly, and is cleared only by a re-initialize. Health precedence is explicit: /current and /sample fail independently, so each path owns a flag, clears only its own, and Healthy requires both clear. Neither can paper over the other's failure. 29 new tests, all deterministic — no sleeps, no wall-clock assertions. The fake grew per-generation sample streams (so a reconnect has somewhere to land), scripted /current answers, scripted sample failures, and a chunk-consumed signal that also fires when the consumer abandons the stream. 375/375 green. --- .../MTConnectDriver.cs | 967 +++++++++++++++++- .../MTConnectSampleHandle.cs | 29 + .../CannedAgentClient.cs | 165 ++- .../MTConnectSubscribeTests.cs | 827 +++++++++++++++ 4 files changed, 1943 insertions(+), 45 deletions(-) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectSampleHandle.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index 8dcc9f68..36214f5a 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -42,19 +42,19 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// nextSequence. /// /// -/// Scope. This type currently implements and -/// . ISubscribable (Task 11), ITagDiscovery (Task 12) +/// Scope. This type currently implements , +/// and . ITagDiscovery (Task 12) /// and IHostConnectivityProbe/IRediscoverable (Task 13) are added on top of /// the state this class already caches — the probe model, the Agent instanceId, and /// the observation index. /// /// /// The read path never writes the shared observation index — see -/// . That index has exactly one writer, and Task 11's /sample -/// pump is it. +/// . That index has exactly one writer: the /sample pump +/// (). /// /// -public sealed class MTConnectDriver : IDriver, IReadable +public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable { /// /// Coarse per-entry byte weights behind . The contract asks @@ -85,6 +85,19 @@ public sealed class MTConnectDriver : IDriver, IReadable /// private const uint BadNotConnected = 0x808A0000u; + /// + /// Floor the reconnect backoff grows from, in milliseconds. Load-bearing: the shipped + /// default is 0 (an immediate + /// first retry, which is what an operator wants after a one-off blip) and + /// 0 × multiplier is still 0 — so without a floor, the "geometric" backoff would be + /// an unbounded hot loop against an Agent that stays down. Mirrors + /// ModbusTcpTransport.ConnectWithBackoffAsync. + /// + private const int BackoffGrowthFloorMs = 100; + + /// Growth factor used when the authored multiplier could not grow the delay (≤ 1). + private const double DefaultBackoffMultiplier = 2.0; + /// /// Config-JSON reader options, mirroring the sibling driver factories. Note there is /// deliberately no JsonStringEnumConverter: enum-carrying DTO fields stay @@ -145,10 +158,58 @@ public sealed class MTConnectDriver : IDriver, IReadable /// private readonly SemaphoreSlim _lifecycle = new(1, 1); + /// + /// Guards the subscription registry and the pump's control block (, + /// , ). + /// + /// + /// + /// Lock order is → this, never the reverse. The lifecycle + /// methods take this lock (to start/stop the pump) while holding the semaphore; + /// therefore releases this lock before awaiting + /// , and nothing under this lock ever waits on + /// . + /// + /// + /// Nothing is awaited while it is held, and no subscriber callback is raised under + /// it. A handler is caller code: it may block, throw, or re-enter this driver, and + /// holding a lock across it would turn a slow subscriber into a stalled pump. + /// + /// + private readonly Lock _subscriptionLock = new(); + + /// Live subscriptions by id. Guarded by . + private readonly Dictionary _subscriptions = []; + private MTConnectDriverOptions _options; private MTConnectObservationIndex _index; - private long? _agentInstanceId; private DriverHealth _health = new(DriverState.Unknown, null, null); + private long _nextSubscriptionId; + + /// Cancels the shared /sample pump. Guarded by . + private CancellationTokenSource? _pumpCts; + + /// The shared /sample pump task. Guarded by . + private Task? _pumpTask; + + /// + /// The Agent answered /sample with something that cannot be streamed at all, so the + /// pump gave up. Latched — reconnecting reproduces the identical answer forever, and a + /// later is not new information. Cleared only by a lifecycle + /// start, which is the one event that means an operator changed something. Guarded by + /// . + /// + private bool _streamUnsupported; + + /// + /// Whether the /sample pump is currently failing. Kept separate from + /// because /sample and /current are different + /// Agent requests that fail independently — see . + /// + private volatile bool _streamDegraded; + + /// Whether the /current read path is currently failing. See . + private volatile bool _readDegraded; /// /// The live agent client plus the /probe cache derived from it, held as one @@ -247,10 +308,28 @@ public sealed class MTConnectDriver : IDriver, IReadable internal MTConnectProbeModel? CachedProbeModel => Volatile.Read(ref _session)?.ProbeModel; /// - /// The Agent's instanceId as of the last successful /current. Task 13 watches - /// this for change to raise rediscovery (an Agent restart invalidates every cached id). + /// The Agent's instanceId as of the last successful /current — including the + /// re-baseline the /sample pump runs when it sees the id change mid-stream. Task 13 + /// watches this for change to raise rediscovery (an Agent restart invalidates every cached + /// id). /// - internal long? AgentInstanceId => _agentInstanceId; + internal long? AgentInstanceId => Volatile.Read(ref _session)?.InstanceId; + + /// + /// The shared /sample pump's task while one is running, else null. Exposed to + /// tests as the deterministic teardown barrier: "the pump has stopped" is a task completion, + /// never a sleep. Guarded, so a test never observes a half-installed pump. + /// + internal Task? SampleStreamTask + { + get + { + lock (_subscriptionLock) + { + return _pumpTask; + } + } + } /// The options currently in force — the constructor's, or the last document applied. internal MTConnectDriverOptions EffectiveOptions => _options; @@ -275,7 +354,12 @@ public sealed class MTConnectDriver : IDriver, IReadable // be the exact outcome ReinitializeAsync is written to avoid. var options = ResolveIncomingOptions(driverConfigJson); - // A re-Initialize on a live instance must not orphan the previous client. + // A re-Initialize on a live instance must not orphan the previous client — nor the pump + // that is enumerating it. Stopping the stream FIRST is the same ordering + // ReinitializeAsync and ShutdownAsync use, and for the same reason: a pump left + // enumerating a disposed client reads the disposal as a dropped connection and + // reconnects against an object that no longer exists. + await StopSampleStreamAsync().ConfigureAwait(false); await TeardownCoreAsync().ConfigureAwait(false); await StartCoreAsync(options, existingClient: null, cancellationToken).ConfigureAwait(false); @@ -504,7 +588,8 @@ public sealed class MTConnectDriver : IDriver, IReadable results[i] = snapshot.Get(fullReferences[i]); } - WriteHealth(new DriverHealth(DriverState.Healthy, DateTime.UtcNow, null)); + _readDegraded = false; + PublishHealthy(DateTime.UtcNow); return results; } @@ -523,6 +608,648 @@ public sealed class MTConnectDriver : IDriver, IReadable } } + // ---- ISubscribable: ONE shared /sample long poll behind every handle ---- + + /// + public event EventHandler? OnDataChange; + + /// + /// + /// + /// Initial data comes from the primed /current, not from the wire. Every + /// subscribed reference gets one callback immediately (the OPC UA Part 4 convention), + /// answered out of the observation index — including the references with no value yet, + /// which report BadWaitingForInitialData. Waiting for the Agent's next chunk + /// instead would leave a fresh subscription silent for up to a whole heartbeat, + /// and permanently silent for any data item that simply is not changing. + /// + /// + /// Subscribe does not wait for the stream. The first subscription starts the + /// shared pump; the pump then opens /sample on its own task under its own + /// cancellation token. This is deliberate on both counts. The opening handshake cannot + /// be awaited inside the caller's Subscribe timeout in any useful sense — the first + /// chunk may legitimately be a heartbeat away — and the long-lived poll must NOT run + /// under the caller's token, which belongs to one Subscribe call and is disposed the + /// moment it returns. What bounds the running stream is the client's own heartbeat + /// watchdog, and what stops it is or the lifecycle. + /// + /// + /// is not honoured per subscription, and + /// cannot be: MTConnect's publish cadence is the Agent-side interval query + /// parameter of the single shared /sample request + /// (), fixed when the client is + /// built. Honouring a per-handle interval would mean one stream per subscription — the + /// Agent load this driver exists to avoid — or client-side throttling that would drop + /// transient EVENT values the Agent bothered to send. The parameter is accepted and + /// ignored, which is also what the natively-subscribing sibling drivers do. + /// + /// + /// Subscribing before is legal: the interest is recorded, + /// every reference reports "no value yet", and the initialize that follows starts the + /// pump for it. It does not throw and it does not dial an Agent that is not there yet. + /// + /// + /// + /// is null — a caller bug, and the only thing this + /// method is loud about, matching 's posture. + /// + public Task SubscribeAsync( + IReadOnlyList fullReferences, TimeSpan publishingInterval, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(fullReferences); + + // De-duplicated: a reference asked for twice in one subscription is one subscribed value, + // and would otherwise raise two identical callbacks per chunk forever. + var references = new HashSet(StringComparer.Ordinal); + foreach (var reference in fullReferences) + { + if (reference is not null) + { + references.Add(reference); + } + } + + var handle = new MTConnectSampleHandle(Interlocked.Increment(ref _nextSubscriptionId)); + + lock (_subscriptionLock) + { + _subscriptions[handle.SubscriptionId] = new SubscriptionState(handle, references); + + // No-op when a pump is already running: every handle shares the one stream. + StartSampleStreamCore(republishOnStart: false); + } + + // Raised OUTSIDE the lock — a handler is caller code (see the _subscriptionLock remarks). + var index = ObservationIndex; + foreach (var reference in references) + { + RaiseDataChange(handle, reference, index.Get(reference)); + } + + _logger.LogDebug( + "MTConnect driver {DriverInstanceId} opened subscription {SubscriptionId} over {ReferenceCount} reference(s).", + _driverInstanceId, handle.DiagnosticId, references.Count); + + return Task.FromResult(handle); + } + + /// + /// + /// Drops the handle's references and stops the shared stream when the last subscription + /// goes. An unknown handle — already unsubscribed, or issued by a different driver — is a + /// no-op rather than an error: this runs on teardown paths, including failure paths, where + /// throwing would mask whatever prompted the teardown. + /// + /// is null. + public async Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(handle); + + if (handle is not MTConnectSampleHandle ours) + { + return; + } + + bool stopStream; + lock (_subscriptionLock) + { + if (!_subscriptions.Remove(ours.SubscriptionId)) + { + return; + } + + stopStream = !HasSubscribedReferencesCore(); + } + + if (stopStream) + { + // Outside the lock: StopSampleStreamAsync awaits the pump, and the pump takes this same + // lock to read the subscription registry. Holding it here would deadlock the two. + await StopSampleStreamAsync().ConfigureAwait(false); + } + + _logger.LogDebug( + "MTConnect driver {DriverInstanceId} closed subscription {SubscriptionId}{StreamNote}.", + _driverInstanceId, ours.DiagnosticId, stopStream ? " and stopped the shared /sample stream" : string.Empty); + } + + /// + /// Starts the shared /sample pump, if there is anything to pump and anything to pump + /// from. Caller must hold . + /// + /// + /// Whether the pump should republish the observation index to every subscriber before it + /// opens the stream — true when a lifecycle start replaced the baseline the subscribers' + /// current values came from. + /// + private void StartSampleStreamCore(bool republishOnStart) + { + if (_pumpTask is not null || _streamUnsupported || !HasSubscribedReferencesCore()) + { + return; + } + + // One read of the session: the client the pump enumerates and the cursor it opens at are + // guaranteed to belong to the same initialize. + var session = Volatile.Read(ref _session); + if (session is null) + { + return; + } + + var options = _options; + var cts = new CancellationTokenSource(); + _pumpCts = cts; + _pumpTask = Task.Run( + () => RunSampleStreamAsync( + session.Client, options, session.NextSequence, session.InstanceId, republishOnStart, cts.Token), + CancellationToken.None); + } + + /// + /// The shared /sample pump: reads chunks, keeps the observation index and the + /// subscribers up to date, and owns every way the stream can stop. + /// + /// + /// + /// Never throws. It is a detached background task; an escaping exception would be + /// an unobserved fault that silently ends the driver's subscription surface while every + /// handle still looks alive. + /// + /// + /// Never calls a lifecycle method. Its re-baseline goes straight to + /// on the client it already holds — see + /// and the remarks for why + /// "just re-initialize" would be a silent permanent deadlock. + /// + /// + /// Backoff applies to failures only. A re-baseline is a protocol event, not a + /// fault, so it reopens the stream immediately and resets the attempt count; only a + /// genuine failure (dropped connection, blown heartbeat, a re-baseline that itself + /// failed) advances . + /// + /// + private async Task RunSampleStreamAsync( + IMTConnectAgentClient client, + MTConnectDriverOptions options, + long from, + long instanceId, + bool republishOnStart, + CancellationToken ct) + { + var cursor = from; + var attempt = 0; + + try + { + if (republishOnStart) + { + FanOut(SubscribedReferences(), ObservationIndex); + } + + while (!ct.IsCancellationRequested) + { + if (attempt > 0) + { + var delay = BackoffFor(attempt, options.Reconnect); + if (delay > TimeSpan.Zero) + { + try + { + await Task.Delay(delay, ct).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + return; + } + } + } + + var outcome = await ConsumeStreamAsync(client, options, cursor, instanceId, ct).ConfigureAwait(false); + cursor = outcome.Cursor; + instanceId = outcome.InstanceId; + + switch (outcome.Verdict) + { + case StreamVerdict.Cancelled: + return; + + case StreamVerdict.Unsupported: + // Configuration, not weather: reconnecting reproduces the identical answer + // forever, so retrying would be an unbounded hot loop against an endpoint + // that will never stream until an operator changes something. Latch it, say + // so loudly, and stop. A re-initialize clears the latch. + lock (_subscriptionLock) + { + _streamUnsupported = true; + } + + _streamDegraded = true; + Degrade(outcome.Error!); + + _logger.LogError( + outcome.Error, + "MTConnect driver {DriverInstanceId} cannot stream /sample from {AgentUri}: the endpoint did not answer with a framed multipart stream. Subscriptions stay registered but will receive no updates until the driver is re-initialized; check for a proxy or load balancer in front of the Agent.", + _driverInstanceId, options.AgentUri); + + return; + + case StreamVerdict.Reopen: + attempt = 0; + + break; + + default: + _streamDegraded = true; + Degrade(outcome.Error!); + attempt++; + + _logger.LogWarning( + outcome.Error, + "MTConnect driver {DriverInstanceId} lost the /sample stream from {AgentUri}; reconnecting from sequence {Cursor} (attempt {Attempt}).", + _driverInstanceId, options.AgentUri, cursor, attempt); + + break; + } + } + } + catch (Exception ex) + { + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} stopped its /sample pump on an unexpected fault; subscriptions will receive no further updates until the driver is re-initialized.", + _driverInstanceId); + + _streamDegraded = true; + Degrade(ex); + } + } + + /// + /// Enumerates one /sample connection until it ends, and classifies how it ended. + /// Re-baselines are performed here, inside the enumeration, so the index and the + /// subscribers are up to date before the stream is reopened. + /// + private async Task ConsumeStreamAsync( + IMTConnectAgentClient client, + MTConnectDriverOptions options, + long cursor, + long instanceId, + CancellationToken ct) + { + // The RUNNING cursor the gap check is made against: the previous chunk's nextSequence, equal + // to the opening `from` only for the first chunk. Comparing every chunk against the opening + // `from` instead reports a gap on every chunk once the Agent's buffer rolls past it — an + // endless /current re-baseline storm against a perfectly healthy stream. + var expected = cursor; + + try + { + await foreach (var chunk in client.SampleAsync(cursor, ct).ConfigureAwait(false)) + { + if (chunk is null) + { + continue; + } + + // CHECKED BEFORE THE GAP: an Agent restart changes instanceId *and* resets + // sequences, so a restart usually trips IsSequenceGap too — and the two need + // different handling. A gap keeps the held values (they are still this device's); + // a restart invalidates every one of them, because the device model they describe + // no longer exists. Testing the gap first would misdiagnose the restart and keep + // serving values the new Agent may never report again. + if (chunk.InstanceId != instanceId) + { + _logger.LogWarning( + "MTConnect driver {DriverInstanceId} saw {AgentUri} restart (instanceId {Previous} -> {Current}); dropping every held observation and re-baselining.", + _driverInstanceId, options.AgentUri, instanceId, chunk.InstanceId); + + // Dropped BEFORE the re-baseline, and the subscribers are told: if the /current + // then fails, holding stale values from a dead device model would be worse than + // holding none. + ObservationIndex.Clear(); + FanOut(SubscribedReferences(), ObservationIndex); + + return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + } + + if (IMTConnectAgentClient.IsSequenceGap(expected, chunk)) + { + _logger.LogWarning( + "MTConnect driver {DriverInstanceId} fell out of {AgentUri}'s buffer (expected sequence {Expected}, oldest retained {FirstSequence}); re-baselining from /current.", + _driverInstanceId, options.AgentUri, expected, chunk.FirstSequence); + + return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + } + + ApplyAndFanOut(chunk); + + // EVERY chunk advances the cursor, including an observation-free heartbeat: the + // Agent sends those precisely so a quiet connection can be told from a dead one, and + // they carry the sequence forward. Not advancing on one makes the NEXT chunk look + // like a gap. + expected = chunk.NextSequence; + + _streamDegraded = false; + PublishHealthy(DateTime.UtcNow); + } + + // The seam contracts that cancellation is the ONLY way this enumeration ends quietly, so + // a quiet end under a live token is a non-conforming client — reported as the lost + // stream it is rather than treated as "the subscription is simply idle" (#485). + return ct.IsCancellationRequested + ? new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, null) + : new StreamOutcome( + StreamVerdict.Transient, + expected, + instanceId, + new MTConnectStreamEndedException( + MTConnectStreamEndReason.ConnectionClosed, options.AgentUri, 0)); + } + catch (OperationCanceledException) when (ct.IsCancellationRequested) + { + return new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, null); + } + catch (MTConnectStreamNotSupportedException ex) + { + return new StreamOutcome(StreamVerdict.Unsupported, expected, instanceId, ex); + } + catch (InvalidDataException ex) + { + // THE OTHER HALF OF RING-BUFFER OVERFLOW, and the one IsSequenceGap cannot see: a real + // cppagent answers a `from` that has already fallen out of its buffer with an + // MTConnectError / OUT_OF_RANGE document served under HTTP 200, which the client + // surfaces here rather than as a gap-bearing chunk. Re-baselining on the gap alone would + // mean the driver recovers from falling a little behind but not from falling a lot. + _logger.LogWarning( + ex, + "MTConnect driver {DriverInstanceId} could not resume {AgentUri}'s /sample stream at sequence {Cursor} (the Agent rejected it as out of range); re-baselining from /current.", + _driverInstanceId, options.AgentUri, cursor); + + return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + } + catch (Exception ex) + { + // Transient by default: a dropped connection, a closing boundary, the heartbeat + // watchdog's TimeoutException. Reconnect under backoff. + return new StreamOutcome(StreamVerdict.Transient, expected, instanceId, ex); + } + } + + /// + /// Re-primes the observation index from a fresh /current and reports where the stream + /// should resume. The recovery path for every way the incremental sequence can be broken: + /// a ring-buffer overflow (detected as a gap, or refused outright as OUT_OF_RANGE) + /// and an Agent restart. + /// + /// + /// This calls the client directly and takes no lock. Re-baselining by calling + /// — the obvious way to write it, since that method already + /// does exactly this work — would take the non-reentrant semaphore + /// from inside a pump that a lifecycle method may already be waiting on, and hang the driver + /// permanently with no exception and no log line. + /// + private async Task RebaselineAsync( + IMTConnectAgentClient client, + MTConnectDriverOptions options, + long fallbackCursor, + long fallbackInstanceId, + CancellationToken ct) + { + try + { + var current = await BoundedAsync(client.CurrentAsync, "/current", options.RequestTimeoutMs, ct) + .ConfigureAwait(false); + + PublishAgentInstanceId(current.InstanceId); + ApplyAndFanOut(current); + + _streamDegraded = false; + PublishHealthy(DateTime.UtcNow); + + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} re-baselined from {AgentUri}'s /current ({ObservationCount} observation(s), agent instanceId {InstanceId}) and resumes /sample from sequence {NextSequence}.", + _driverInstanceId, + options.AgentUri, + current.Observations.Count, + current.InstanceId, + current.NextSequence); + + return new StreamOutcome(StreamVerdict.Reopen, current.NextSequence, current.InstanceId, null); + } + catch (OperationCanceledException) when (ct.IsCancellationRequested) + { + return new StreamOutcome(StreamVerdict.Cancelled, fallbackCursor, fallbackInstanceId, null); + } + catch (Exception ex) + { + // The Agent is unreachable or unusable right now. Treated as an ordinary transient + // failure so the reconnect backoff applies — without it, an Agent that is down would be + // re-baselined against as fast as the loop can spin. The cursor is deliberately left + // where it was: reopening there either works or lands right back here. + return new StreamOutcome(StreamVerdict.Transient, fallbackCursor, fallbackInstanceId, ex); + } + } + + /// + /// The reconnect delay before attempt , as a pure function — the + /// backoff policy is asserted directly rather than inferred from how long a test slept. + /// + /// + /// The first retry honours (0 by + /// default = immediate, which is what an operator wants after a one-off blip); every later + /// one multiplies, from a floor, up to + /// . Operator-authored nonsense cannot + /// un-bound the loop: a multiplier that cannot grow the delay falls back to + /// , and negative values clamp to zero. + /// + /// The 1-based reconnect attempt about to be made. + /// The authored backoff options. + /// How long to wait before that attempt. + internal static TimeSpan BackoffFor(int attempt, MTConnectReconnectOptions reconnect) + { + ArgumentNullException.ThrowIfNull(reconnect); + + var capMs = Math.Max(0, reconnect.MaxBackoffMs); + var minMs = Math.Max(0, reconnect.MinBackoffMs); + + if (attempt <= 1) + { + return TimeSpan.FromMilliseconds(Math.Min(minMs, capMs)); + } + + var multiplier = reconnect.BackoffMultiplier > 1.0 ? reconnect.BackoffMultiplier : DefaultBackoffMultiplier; + var delayMs = (double)Math.Max(minMs, BackoffGrowthFloorMs); + + for (var step = 2; step <= attempt; step++) + { + delayMs *= multiplier; + + if (delayMs >= capMs) + { + return TimeSpan.FromMilliseconds(capMs); + } + } + + return TimeSpan.FromMilliseconds(Math.Min(delayMs, capMs)); + } + + /// + /// Folds a /current snapshot or one /sample chunk into the shared index and + /// publishes every reference it touched to the handles that subscribe it. + /// + private void ApplyAndFanOut(MTConnectStreamsResult document) + { + var index = ObservationIndex; + index.Apply(document); + + if (document.Observations is not { Count: > 0 }) + { + return; + } + + // De-duplicated in document order: one Agent document may report a data item more than once + // (a Condition container carries several simultaneously-active states), and the index has + // already reconciled those into a single snapshot. + var touched = new List(document.Observations.Count); + var seen = new HashSet(StringComparer.Ordinal); + foreach (var observation in document.Observations) + { + if (observation is null || string.IsNullOrWhiteSpace(observation.DataItemId)) + { + continue; + } + + if (seen.Add(observation.DataItemId)) + { + touched.Add(observation.DataItemId); + } + } + + FanOut(touched, index); + } + + /// + /// Publishes the index's current snapshot of to every handle + /// that subscribes them. References nobody subscribed raise nothing: the Agent streams the + /// whole device, and republishing all of it would flood the server with values no client + /// asked for. + /// + private void FanOut(IReadOnlyList references, MTConnectObservationIndex index) + { + if (references.Count == 0) + { + return; + } + + SubscriptionState[] states; + lock (_subscriptionLock) + { + if (_subscriptions.Count == 0) + { + return; + } + + states = [.. _subscriptions.Values]; + } + + // The lock is released before any callback: handlers are caller code. + foreach (var reference in references) + { + DataValueSnapshot? snapshot = null; + + foreach (var state in states) + { + if (!state.References.Contains(reference)) + { + continue; + } + + // Read once per reference, so every handle sees the identical snapshot. + snapshot ??= index.Get(reference); + RaiseDataChange(state.Handle, reference, snapshot); + } + } + } + + /// + /// Raises one data-change callback, absorbing anything the subscriber throws. A consumer bug + /// must not tear down the pump that serves every OTHER subscriber — and the exception is + /// logged rather than swallowed silently, because a handler that throws on every value is + /// otherwise invisible. + /// + private void RaiseDataChange(MTConnectSampleHandle handle, string reference, DataValueSnapshot snapshot) + { + try + { + OnDataChange?.Invoke(this, new DataChangeEventArgs(handle, reference, snapshot)); + } + catch (Exception ex) + { + _logger.LogWarning( + ex, + "MTConnect driver {DriverInstanceId} caught a fault from a data-change subscriber for {Reference} on {SubscriptionId}; the stream continues.", + _driverInstanceId, reference, handle.DiagnosticId); + } + } + + /// Every reference any live subscription is interested in. + private List SubscribedReferences() + { + lock (_subscriptionLock) + { + var all = new HashSet(StringComparer.Ordinal); + foreach (var state in _subscriptions.Values) + { + all.UnionWith(state.References); + } + + return [.. all]; + } + } + + /// + /// Whether any live subscription actually names a reference. Caller must hold + /// . A subscription over an empty reference list is legal + /// (a caller that adds tags later) and must not open a stream by itself. + /// + private bool HasSubscribedReferencesCore() + { + foreach (var state in _subscriptions.Values) + { + if (state.References.Count > 0) + { + return true; + } + } + + return false; + } + + /// + /// Records the Agent instanceId a re-baseline observed, onto whichever session is + /// installed. A compare-and-swap for the same reason + /// uses one: if a lifecycle change lands mid-update, + /// its state is newer and this write must be dropped, not replayed over it. + /// + private void PublishAgentInstanceId(long instanceId) + { + while (true) + { + var session = Volatile.Read(ref _session); + if (session is null || session.InstanceId == instanceId) + { + return; + } + + var updated = session with { InstanceId = instanceId }; + if (ReferenceEquals(Interlocked.CompareExchange(ref _session, updated, session), session)) + { + return; + } + } + } + /// /// The /probe device model: the cached copy when warm, otherwise a fresh /// deadline-bounded /probe that re-populates the cache. This is the accessor every @@ -687,17 +1414,34 @@ public sealed class MTConnectDriver : IDriver, IReadable _options = options; built = null; - // One publish, so a concurrent lock-free reader sees the new client and the new probe - // cache together or neither of them. - Volatile.Write(ref _session, new AgentSession(client, probe, CountDataItems(probe))); - _agentInstanceId = current.InstanceId; + // One publish, so a concurrent lock-free reader sees the new client, the new probe + // cache, and the pump's opening cursor together or none of them. + Volatile.Write( + ref _session, + new AgentSession(client, probe, CountDataItems(probe), current.InstanceId, current.NextSequence)); var index = new MTConnectObservationIndex(options.Tags); index.Apply(current); Volatile.Write(ref _index, index); + // A fresh session has no failures behind it — including the /sample verdict, because a + // re-initialize is precisely the event that means an operator changed something. + _streamDegraded = false; + _readDegraded = false; + WriteHealth(new DriverHealth(DriverState.Healthy, DateTime.UtcNow, null)); + // Standing subscriptions survive a lifecycle restart; everything they hold came from the + // baseline just thrown away, so the restarted pump republishes as its first act. It is + // the PUMP that republishes rather than this method, deliberately: raising subscriber + // callbacks from here would run caller code on the thread holding the non-reentrant + // lifecycle semaphore, and a handler that touched the driver's lifecycle would deadlock. + lock (_subscriptionLock) + { + _streamUnsupported = false; + StartSampleStreamCore(republishOnStart: true); + } + _logger.LogInformation( "MTConnect driver {DriverInstanceId} initialized against {AgentUri}{DeviceScope}: {DeviceCount} device(s), {ObservationCount} primed observation(s), agent instanceId {InstanceId}.", _driverInstanceId, @@ -739,8 +1483,6 @@ public sealed class MTConnectDriver : IDriver, IReadable Volatile.Write(ref _session, null); SafeDispose(session?.Client); - _agentInstanceId = null; - // A fresh index rather than Clear(): the tag table it coerces against belongs to the options // that are being torn down. Volatile.Write(ref _index, new MTConnectObservationIndex(_options.Tags)); @@ -749,21 +1491,85 @@ public sealed class MTConnectDriver : IDriver, IReadable } /// - /// Stops the shared /sample long-poll stream. A no-op until Task 11 introduces the - /// pump; it exists now because both and - /// must stop the stream before the client they are about - /// to dispose goes away, and that ordering is easy to lose when the pump is bolted on later. + /// Stops the shared /sample long-poll stream and waits for the pump to actually be + /// gone. Idempotent, and never throws — it runs on teardown paths where a second exception + /// would mask the first. /// /// - /// ⚠️ This runs while is held. Whatever Task 11 fills it with - /// — cancelling the pump's token, awaiting its task — must not call back into - /// , or - /// , directly or by awaiting a pump that is itself blocked on one - /// of them: the semaphore is non-reentrant and the result is a permanent hang with no - /// exception and no log. See the remarks for the re-baseline - /// pattern that avoids this. + /// + /// It waits, rather than merely signalling. Every caller is about to dispose the + /// client the pump is enumerating (or install a new observation index the old pump would + /// write into). Returning while the pump is still unwinding would let it publish one more + /// value into state its owner has already retired — the exact "torn down but still + /// reporting" shape the lifecycle is written to prevent. + /// + /// + /// ⚠️ This runs while is held (from + /// , and + /// ) — and it awaits the pump. That is only safe because the + /// pump never touches : its re-baseline calls + /// directly (see + /// ) rather than re-entering a lifecycle method, and every + /// await it makes observes the pump token cancelled below. Break either property and + /// this await is a permanent hang with no exception and no log line. + /// + /// + /// The one thing outside that guarantee is a subscriber callback: an + /// handler that blocks forever blocks teardown with it. The + /// pump raises callbacks outside every lock precisely so that a slow handler only + /// delays; an infinite one is a consumer bug this driver cannot paper over. + /// /// - private static Task StopSampleStreamAsync() => Task.CompletedTask; + private async Task StopSampleStreamAsync() + { + CancellationTokenSource? cts; + Task? task; + + lock (_subscriptionLock) + { + cts = _pumpCts; + task = _pumpTask; + _pumpCts = null; + _pumpTask = null; + } + + if (cts is null) + { + return; + } + + try + { + await cts.CancelAsync().ConfigureAwait(false); + } + catch (ObjectDisposedException) + { + // Raced another stop; the pump is already going away. + } + + if (task is not null) + { + try + { + await task.ConfigureAwait(false); + } + catch (Exception ex) + { + // RunSampleStreamAsync is written not to throw, so this is belt-and-braces — but a + // teardown that rethrew would replace whatever failure prompted it. + _logger.LogDebug( + ex, + "MTConnect driver {DriverInstanceId} swallowed a fault from its /sample pump while stopping the stream.", + _driverInstanceId); + } + } + + cts.Dispose(); + + // There is no stream any more, so there is nothing for it to be degraded about. Leaving the + // flag set would pin a later successful read to Degraded forever. + _streamDegraded = false; + } /// /// Resolves the options a lifecycle call should apply, reporting an unreadable document @@ -943,11 +1749,8 @@ public sealed class MTConnectDriver : IDriver, IReadable /// private void DegradeAfterReadFailure(Exception ex) { - var current = ReadHealth(); - if (current.State != DriverState.Faulted) - { - WriteHealth(new DriverHealth(DriverState.Degraded, current.LastSuccessfulRead, ex.Message)); - } + _readDegraded = true; + Degrade(ex); _logger.LogWarning( ex, @@ -955,6 +1758,57 @@ public sealed class MTConnectDriver : IDriver, IReadable _driverInstanceId, _options.AgentUri); } + /// Publishes without downgrading a Faulted driver. + private void Degrade(Exception ex) + { + var current = ReadHealth(); + if (current.State != DriverState.Faulted) + { + WriteHealth(new DriverHealth(DriverState.Degraded, current.LastSuccessfulRead, ex.Message)); + } + } + + /// + /// Publishes — unless the other data path is currently + /// failing. + /// + /// + /// + /// Health precedence, stated deliberately. This driver has two independent Agent + /// data paths: /current (the batch read) and + /// /sample (the subscription pump). They are different HTTP requests and fail + /// separately — a proxy that collapses multipart/x-mixed-replace breaks streaming + /// while reads stay perfect; an Agent that stalls composing a whole-device snapshot + /// breaks reads while the stream keeps flowing. + /// + /// + /// The naive last-writer-wins publish would let either path paper over the other's + /// failure, and "Healthy" while half the surface returns Bad is precisely the + /// failure-wearing-success shape this codebase exists to avoid. So each path owns a + /// flag, clears only its own, and Healthy requires both clear: the driver reports + /// Degraded while anything is failing and recovers as soon as the failing path + /// itself recovers. — a config-level verdict — is + /// never downgraded by either. + /// + /// + /// When the Agent was last successfully heard from. + private void PublishHealthy(DateTime at) + { + var current = ReadHealth(); + + if (_streamDegraded || _readDegraded) + { + if (current.State != DriverState.Faulted) + { + WriteHealth(new DriverHealth(DriverState.Degraded, at, current.LastError)); + } + + return; + } + + WriteHealth(new DriverHealth(DriverState.Healthy, at, null)); + } + /// /// Disposes an agent client without letting a disposal fault escape. Teardown runs on the /// failure path of , where a second exception would replace the @@ -1050,10 +1904,53 @@ public sealed class MTConnectDriver : IDriver, IReadable /// Data items declared by , counted once at cache time and /// always 0 when it is null. /// + /// + /// The Agent's instanceId as last observed. Updated in place (by + /// ) when the pump sees the Agent restart, so the + /// session always names the Agent process the client is actually talking to. + /// + /// + /// The sequence the /sample pump should open at — the priming /current's + /// nextSequence. Lives here so that starting the pump reads the client and its cursor + /// as one consistent pair; a torn read would open the stream at another session's sequence. + /// private sealed record AgentSession( IMTConnectAgentClient Client, MTConnectProbeModel? ProbeModel, - int ProbeModelDataItemCount); + int ProbeModelDataItemCount, + long InstanceId, + long NextSequence); + + /// + /// One live subscription: its handle and the references it wants. Both are effectively + /// immutable once registered, so the pump reads without a lock — + /// it takes only to snapshot the registry itself. + /// + /// The identity handed to the caller and stamped on every event. + /// The de-duplicated DataItem ids this subscription publishes. + private sealed record SubscriptionState(MTConnectSampleHandle Handle, HashSet References); + + /// How one /sample connection ended, and therefore what the pump does next. + private enum StreamVerdict + { + /// Cancelled by teardown — stop, publish nothing, report nothing. + Cancelled = 0, + + /// A transient failure: degrade and reconnect under the backoff. + Transient = 1, + + /// Re-baselined successfully: reopen immediately at the new cursor, no backoff. + Reopen = 2, + + /// The endpoint cannot stream at all: latch, report loudly, and stop retrying. + Unsupported = 3 + } + + /// How the connection ended. + /// The sequence the next connection must open at. + /// The Agent instance the cursor belongs to. + /// The failure, when there was one. + private sealed record StreamOutcome(StreamVerdict Verdict, long Cursor, long InstanceId, Exception? Error); // ---- config DTOs ---- diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectSampleHandle.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectSampleHandle.cs new file mode 100644 index 00000000..ee0119ed --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectSampleHandle.cs @@ -0,0 +1,29 @@ +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// Driver-internal identity of one /sample subscription, matching the sibling drivers' +/// shape (Galaxy's GalaxySubscriptionHandle, the shared PollGroupEngine's +/// PollSubscriptionHandle): a monotonic per-driver id plus a diagnostic string that +/// carries it into logs. +/// +/// +/// +/// Every handle shares ONE Agent stream. Unlike a polled driver, where each +/// subscription owns a loop, MTConnect's /sample long poll is per-Agent: the driver +/// opens exactly one and fans each chunk out to whichever handles subscribe the reporting +/// DataItem. The handle is therefore purely an identity — it owns no connection, no task, +/// and no cursor — and dropping one only stops the stream when it was the last. +/// +/// +/// A for value equality on the id, so a handle that has round-tripped +/// through the caller still resolves. The id is never reused within a driver instance. +/// +/// +/// The monotonic per-driver subscription id. +internal sealed record MTConnectSampleHandle(long SubscriptionId) : ISubscriptionHandle +{ + /// + public string DiagnosticId => $"mtconnect-sub-{SubscriptionId}"; +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs index 71fe5e52..d3c2d20d 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs @@ -1,3 +1,4 @@ +using System.Collections.Concurrent; using System.Runtime.CompilerServices; using System.Threading.Channels; @@ -34,9 +35,19 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; /// internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable { - private readonly Channel _chunks = Channel.CreateUnbounded(); private readonly CancellationTokenSource _disposeCts = new(); + /// Chunks a test has scripted but not yet pumped — see . + private readonly ConcurrentQueue _script = new(); + + /// + /// The current /sample stream generation. Replaced (not merely completed) by + /// so that a pump which reconnects after a dropped stream gets a + /// live channel to read from instead of one that is permanently closed — without that, a + /// reconnect test could only ever observe the reconnect failing. + /// + private Channel _chunks = Channel.CreateUnbounded(); + private int _probeCallCount; private int _currentCallCount; private int _sampleCallCount; @@ -66,6 +77,44 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable public static MTConnectStreamsResult Chunk(string fixture) => MTConnectStreamsParser.Parse(File.ReadAllText(fixture)); + /// + /// An Agent scripted for the ring-buffer-overflow story: the driver primes from + /// Fixtures/current.xml (nextSequence 108), then the first scripted chunk is + /// Fixtures/sample-gap.xml — whose firstSequence (5000) is strictly newer than + /// the cursor, i.e. the buffer rolled past the driver — and the second is an ordinary chunk + /// that continues contiguously from the re-baseline. + /// + /// + /// The second /current answer is what makes this a story rather than a single + /// event: it advertises the buffer the gap chunk revealed (firstSequence 5000 / + /// nextSequence 5005), so a driver that re-baselines and resumes from that answer sees + /// the follow-up chunk as contiguous. A driver that re-baselined but resumed from the stale + /// cursor would report a gap on every subsequent chunk instead. + /// + public static CannedAgentClient WithGapThenResume() + { + var client = FromFixtures(); + var gap = Chunk("Fixtures/sample-gap.xml"); + + // 1st /current = the InitializeAsync prime (the fixture). 2nd = the post-gap re-baseline. + client.CurrentAnswers.Enqueue(client.Current); + client.CurrentAnswers.Enqueue( + new MTConnectStreamsResult(gap.InstanceId, gap.NextSequence, gap.FirstSequence, gap.Observations)); + + client.ScriptChunks( + gap, + new MTConnectStreamsResult( + gap.InstanceId, + gap.NextSequence + 5, + gap.NextSequence, + [ + new MTConnectObservation( + "dev1_pos", "201.5000", new DateTime(2026, 7, 24, 12, 6, 0, DateTimeKind.Utc)), + ])); + + return client; + } + /// The document /probe answers with. Settable so a test can re-shape the model. public MTConnectProbeModel Probe { get; set; } @@ -81,6 +130,31 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable /// When set, /current throws this instead of answering. public Exception? CurrentFailure { get; set; } + /// + /// Scripted /current answers, consumed in order: each call dequeues the next one and + /// promotes it to , so once the script runs out the Agent keeps + /// answering with the last thing it said rather than travelling back in time. This is how a + /// test scripts "the prime, then the post-gap re-baseline" without racing the pump for a + /// property setter. + /// + public ConcurrentQueue CurrentAnswers { get; } = new(); + + /// + /// Scripted one-shot /sample failures: each enumeration dequeues one and throws it. + /// Models a transient stream fault the Agent recovers from — e.g. the + /// a real cppagent's OUT_OF_RANGE error document + /// (served under HTTP 200) surfaces as. + /// + public ConcurrentQueue SampleFailures { get; } = new(); + + /// + /// A sticky /sample failure: every enumeration throws it, forever. Models a + /// configuration-level fault such as + /// , where reconnecting reproduces the + /// identical answer — the shape a pump must NOT retry-loop against. + /// + public Exception? SampleFailure { get; set; } + /// /// When set, the next /probe parks here until the test completes it, then the /// gate clears itself so later calls answer immediately. @@ -101,6 +175,14 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable /// public TaskCompletionSource? CurrentGate { get; set; } + /// + /// Signalled the moment a /current request lands — before + /// parks it. This is the deterministic "the caller is now inside + /// the request" barrier a test needs before landing a concurrent lifecycle change on it; + /// without it the only alternative is polling on a timer. + /// + public TaskCompletionSource? CurrentEntered { get; set; } + /// Number of /probe requests issued against this client. public int ProbeCallCount => Volatile.Read(ref _probeCallCount); @@ -152,6 +234,7 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable ObjectDisposedException.ThrowIf(IsDisposed, this); ct.ThrowIfCancellationRequested(); Interlocked.Increment(ref _currentCallCount); + CurrentEntered?.TrySetResult(); // The request was accepted; the answer is withheld until the test releases the gate or the // caller's deadline cancels the token. @@ -161,6 +244,12 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable await gate.Task.WaitAsync(ct).ConfigureAwait(false); } + // A scripted answer supersedes — and then becomes — the standing one. + if (CurrentAnswers.TryDequeue(out var scripted)) + { + Current = scripted; + } + return CurrentFailure is null ? Current : throw CurrentFailure; } @@ -172,6 +261,21 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable Interlocked.Increment(ref _sampleCallCount); LastSampleFrom = from; + // Thrown before the first chunk, exactly like a real client that fails its /sample request: + // one-shot scripts first, then the sticky "this endpoint will never stream" failure. + if (SampleFailures.TryDequeue(out var oneShot)) + { + throw oneShot; + } + + if (SampleFailure is { } sticky) + { + throw sticky; + } + + // Bound to THIS generation: EndStream swaps in a fresh channel, so a consumer that + // reconnects reads the new one rather than the closed one it just lost. + var chunks = Volatile.Read(ref _chunks); using var lifetime = CancellationTokenSource.CreateLinkedTokenSource(ct, _disposeCts.Token); var delivered = 0L; @@ -180,7 +284,7 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable ScriptedChunk scripted; try { - scripted = await _chunks.Reader.ReadAsync(lifetime.Token).ConfigureAwait(false); + scripted = await chunks.Reader.ReadAsync(lifetime.Token).ConfigureAwait(false); } catch (ChannelClosedException) { @@ -192,9 +296,18 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable delivered++; - yield return scripted.Result; - - scripted.Consumed.TrySetResult(); + try + { + yield return scripted.Result; + } + finally + { + // Signalled when the consumer is DONE with this chunk — whether it came back for the + // next one or abandoned the enumeration. The finally is load-bearing: a chunk that + // makes the pump break out (a sequence gap, an agent restart) is never resumed past, + // so signalling only after the resume would hang every re-baseline test. + scripted.Consumed.TrySetResult(); + } } } @@ -212,7 +325,7 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable var scripted = new ScriptedChunk( chunk, new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously)); - if (!_chunks.Writer.TryWrite(scripted)) + if (!Volatile.Read(ref _chunks).Writer.TryWrite(scripted)) { throw new InvalidOperationException("The scripted /sample stream is already closed."); } @@ -221,10 +334,42 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable } /// - /// Closes the scripted /sample stream, which surfaces to the consumer as an - /// — the transient, reconnectable end. + /// Queues chunks for to deliver one at a time. Nothing is sent until + /// a test asks for it — the script is a plan, not a schedule. /// - public void EndStream() => _chunks.Writer.TryComplete(); + /// The chunks the Agent should send, in order. + public void ScriptChunks(params MTConnectStreamsResult[] chunks) + { + ArgumentNullException.ThrowIfNull(chunks); + + foreach (var chunk in chunks) + { + _script.Enqueue(chunk); + } + } + + /// + /// Pumps the next scripted chunk (see ) and waits until the + /// consumer has finished with it. + /// + /// A task completing once the consumer has processed — or abandoned the stream on — that chunk. + /// The script is exhausted. + public Task PumpOnce() => + _script.TryDequeue(out var next) + ? PumpAsync(next) + : throw new InvalidOperationException( + "No scripted /sample chunk left to pump; call ScriptChunks first."); + + /// + /// Closes the current scripted /sample stream, which surfaces to its consumer as an + /// — the transient, reconnectable end — and opens + /// a fresh one so a reconnecting consumer has somewhere to land. + /// + public void EndStream() + { + var closed = Interlocked.Exchange(ref _chunks, Channel.CreateUnbounded()); + closed.Writer.TryComplete(); + } /// public void Dispose() @@ -237,7 +382,7 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable } _disposeCts.Cancel(); - _chunks.Writer.TryComplete(); + Volatile.Read(ref _chunks).Writer.TryComplete(); _disposeCts.Dispose(); } diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs new file mode 100644 index 00000000..d7b950b7 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs @@ -0,0 +1,827 @@ +using System.Collections.Concurrent; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 11 — 's half: one shared +/// /sample long-poll pump behind every handle, the ring-buffer re-baseline, and the +/// teardown that must leave nothing running. Every test runs against +/// ; no socket is opened anywhere in this file. +/// +/// +/// +/// Nothing here waits on wall-clock time. The fake completes a +/// only once the pump has finished with that +/// chunk, and exposes the pump's own task +/// () as the teardown barrier — so every +/// "the pump has now done X" statement is a fact, not a sleep. The one +/// below is a failure guard that turns a genuine hang (an +/// un-terminated retry loop, a deadlocked teardown) into a clean red test; no assertion +/// is ever made about elapsed time. +/// +/// +/// The load-bearing tests are the ones about a stream that has gone wrong. A pump +/// that only ever meets contiguous chunks is a dozen lines; the defects live in the four +/// ways it can be knocked off the sequence — the buffer rolling past the cursor +/// (IsSequenceGap), the Agent refusing an evicted from outright +/// (OUT_OF_RANGE under HTTP 200 ⇒ ), the Agent +/// restarting (a new instanceId, which ALSO looks like a gap), and the connection +/// dropping — plus the one failure it must NOT retry +/// (, where reconnecting reproduces the +/// identical answer forever). +/// +/// +public sealed class MTConnectSubscribeTests +{ + private const string AgentUri = "http://fixture-agent:5000"; + + /// The instanceId every canned fixture shares. + private const long FixtureInstanceId = 1655000000L; + + /// A DIFFERENT instanceId — the Agent came back as a new process. + private const long RestartedInstanceId = 1655999999L; + + /// The cursor Fixtures/current.xml primes the pump with. + private const long PrimedNextSequence = 108L; + + // Canonical Opc.Ua.StatusCodes numerics, restated so the test asserts the wire value a client + // actually sees rather than a driver-private constant it could drift with. + private const uint Good = 0x00000000u; + private const uint BadNoCommunication = 0x80310000u; + private const uint BadWaitingForInitialData = 0x80320000u; + private const uint BadNodeIdUnknown = 0x80340000u; + + /// + /// Failure guard for the handful of awaits that a defect could turn into a permanent hang + /// (a NotSupported retry loop, a teardown that deadlocks on the lifecycle semaphore). + /// Deliberately far longer than any of these operations could legitimately take — it exists + /// to make a hang red, not to assert a duration. + /// + private static readonly TimeSpan Watchdog = TimeSpan.FromSeconds(10); + + private static readonly DateTime ObservedAt = new(2026, 7, 24, 12, 30, 0, DateTimeKind.Utc); + + private static CancellationToken Ct => TestContext.Current.CancellationToken; + + private static MTConnectDriverOptions Opts(MTConnectReconnectOptions? reconnect = null) => + new() + { + AgentUri = AgentUri, + RequestTimeoutMs = 5000, + Reconnect = reconnect ?? new MTConnectReconnectOptions(), + Tags = + [ + new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64), + new MTConnectTagDefinition("dev1_execution", DriverDataType.String), + new MTConnectTagDefinition("dev1_partcount", DriverDataType.Int32), + new MTConnectTagDefinition("dev1_never_reported", DriverDataType.Int32), + ], + }; + + // ---- fixtures ---- + + private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( + CannedAgentClient? client = null, MTConnectDriverOptions? options = null) + { + var agent = client ?? CannedAgentClient.FromFixtures(); + + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => agent), agent); + } + + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( + CannedAgentClient? client = null, MTConnectDriverOptions? options = null) + { + var (driver, agent) = NewDriver(client, options); + await driver.InitializeAsync("{}", Ct); + + return (driver, agent); + } + + /// Attaches a recorder to the driver's data-change event and returns its log. + private static ConcurrentQueue Record(MTConnectDriver driver) + { + var seen = new ConcurrentQueue(); + driver.OnDataChange += (_, e) => seen.Enqueue(e); + + return seen; + } + + /// Builds a /sample chunk (or /current answer) for the fixture Agent. + private static MTConnectStreamsResult Chunk( + long firstSequence, long nextSequence, params (string Id, string Value)[] observations) => + ChunkFrom(FixtureInstanceId, firstSequence, nextSequence, observations); + + private static MTConnectStreamsResult ChunkFrom( + long instanceId, long firstSequence, long nextSequence, params (string Id, string Value)[] observations) => + new( + instanceId, + nextSequence, + firstSequence, + [.. observations.Select(o => new MTConnectObservation(o.Id, o.Value, ObservedAt))]); + + private static IReadOnlyList For( + ConcurrentQueue seen, string reference) => + [.. seen.Where(e => e.FullReference == reference)]; + + // ---- initial data ---- + + /// + /// The plan's first TDD case, and the OPC UA Part 4 convention: a subscription reports the + /// current value immediately, out of the index the priming /current already filled — + /// it does not make the caller wait for the Agent's next chunk, which may be a whole + /// heartbeat away. + /// + [Fact] + public async Task Subscribe_fires_initial_data_from_current() + { + var (driver, _) = await InitializedDriverAsync(); + var seen = Record(driver); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + seen.Select(e => e.FullReference).ShouldContain("dev1_pos"); + var initial = seen.Single(); + initial.Snapshot.StatusCode.ShouldBe(Good); + initial.Snapshot.Value.ShouldBe(123.4567d); + } + + /// + /// Initial data fires for EVERY subscribed reference, including the ones with no value yet. + /// A driver that fired only for the references it happens to hold a value for would leave + /// the others with no snapshot at all — indistinguishable, from the server's side, from a + /// subscription that never established. + /// + [Fact] + public async Task Subscribe_fires_initial_data_for_every_ref_including_the_valueless_ones() + { + var (driver, _) = await InitializedDriverAsync(); + var seen = Record(driver); + + await driver.SubscribeAsync( + ["dev1_pos", "dev1_partcount", "dev1_never_reported", "nope"], TimeSpan.FromMilliseconds(50), Ct); + + seen.Count.ShouldBe(4); + For(seen, "dev1_pos")[0].Snapshot.StatusCode.ShouldBe(Good); + For(seen, "dev1_partcount")[0].Snapshot.StatusCode.ShouldBe(BadNoCommunication); + For(seen, "dev1_never_reported")[0].Snapshot.StatusCode.ShouldBe(BadWaitingForInitialData); + For(seen, "nope")[0].Snapshot.StatusCode.ShouldBe(BadNodeIdUnknown); + } + + /// Every event carries the handle the caller was given — that is how it demultiplexes. + [Fact] + public async Task Subscribe_returns_a_distinct_handle_per_call_and_stamps_it_on_every_event() + { + var (driver, _) = await InitializedDriverAsync(); + var seen = Record(driver); + + var a = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + var b = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + a.ShouldNotBe(b); + a.DiagnosticId.ShouldNotBe(b.DiagnosticId); + a.DiagnosticId.ShouldNotBeNullOrWhiteSpace(); + For(seen, "dev1_pos").Select(e => e.SubscriptionHandle).ShouldBe([a, b]); + } + + /// Subscribing to nothing costs the Agent nothing — no stream is opened for it. + [Fact] + public async Task Subscribe_of_an_empty_ref_list_opens_no_stream() + { + var (driver, client) = await InitializedDriverAsync(); + + var handle = await driver.SubscribeAsync([], TimeSpan.FromMilliseconds(50), Ct); + + handle.ShouldNotBeNull(); + client.SampleCallCount.ShouldBe(0); + + // Asserted on the pump itself, not only on the call count: a pump that HAD been started + // might simply not have reached its first request yet, and would pass the count alone. + driver.SampleStreamTask.ShouldBeNull(); + + // …and it still unsubscribes cleanly, so a caller that conditionally adds refs later is symmetric. + await driver.UnsubscribeAsync(handle, Ct); + } + + // ---- the shared stream ---- + + /// + /// One Agent, one /sample long poll — no matter how many handles. MTConnect streams + /// the whole device, so a stream per subscription would multiply the Agent's connection load + /// by the number of OPC UA subscriptions for identical data. + /// + [Fact] + public async Task Two_subscriptions_share_exactly_one_sample_stream() + { + var (driver, client) = await InitializedDriverAsync(); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await driver.SubscribeAsync(["dev1_execution"], TimeSpan.FromMilliseconds(50), Ct); + + // Barrier: a chunk can only be consumed by a running enumeration, so both subscriptions have + // demonstrably landed on the same one by the time this returns. + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(1); + client.LastSampleFrom.ShouldBe(PrimedNextSequence); + } + + /// + /// A chunk updates the index for everything it carries, but only the subscribed references + /// raise a callback. The Agent streams every DataItem on the device; publishing all of them + /// would flood the server with values nothing asked for. + /// + [Fact] + public async Task Pump_raises_OnDataChange_only_for_subscribed_refs() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client.PumpAsync(CannedAgentClient.Chunk("Fixtures/sample.xml")).WaitAsync(Watchdog, Ct); + + seen.Select(e => e.FullReference).Distinct().ShouldBe(["dev1_pos"]); + For(seen, "dev1_pos")[0].Snapshot.Value.ShouldBe(124.01d); + + // The index took the whole chunk even though only one ref was published. + driver.ObservationIndex.Get("dev1_partcount").Value.ShouldBe(42); + } + + /// Each handle hears about the references IT subscribed, and only those. + [Fact] + public async Task Pump_fans_each_observation_to_every_handle_that_subscribes_it() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + var a = await driver.SubscribeAsync(["dev1_pos", "dev1_execution"], TimeSpan.FromMilliseconds(50), Ct); + var b = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client + .PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"), ("dev1_execution", "READY"))) + .WaitAsync(Watchdog, Ct); + + For(seen, "dev1_pos").Select(e => e.SubscriptionHandle).ShouldBe([a, b], ignoreOrder: true); + For(seen, "dev1_execution").Select(e => e.SubscriptionHandle).ShouldBe([a]); + } + + /// + /// The running-cursor pin. An observation-free heartbeat still advances the sequence — + /// the Agent sends one precisely so a quiet connection can be told from a dead one — and the + /// gap check must be made against the PREVIOUS chunk's nextSequence, never against the + /// from the stream was opened with. A driver that compares against the opening + /// from, or that skips an empty chunk when advancing, reports a gap on the third chunk + /// here and re-baselines against a perfectly healthy stream. + /// + [Fact] + public async Task Pump_advances_the_cursor_on_every_chunk_including_an_observation_free_one() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + // A heartbeat: no observations, and firstSequence is the buffer floor, not the cursor. + await client.PumpAsync(Chunk(1, 120)).WaitAsync(Watchdog, Ct); + + // Contiguous with the heartbeat's nextSequence — a gap ONLY if the heartbeat was ignored. + await client.PumpAsync(Chunk(120, 125, ("dev1_pos", "2.5"))).WaitAsync(Watchdog, Ct); + + client.CurrentCallCount.ShouldBe(1); // the initialize prime, and nothing else + client.SampleCallCount.ShouldBe(1); // one uninterrupted stream + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(2.5d); + } + + // ---- ring-buffer overflow ---- + + /// + /// The plan's second TDD case. The Agent's buffer rolled past the cursor + /// (firstSequence 5000 > the driver's 108), so the observations in between are + /// gone: the driver must re-baseline from /current rather than trust an incremental + /// update — and must do it once, then resume, not once per chunk forever. + /// + [Fact] + public async Task Sequence_gap_triggers_exactly_one_current_rebaseline_then_resumes() + { + var client = CannedAgentClient.WithGapThenResume(); + var (driver, _) = await InitializedDriverAsync(client); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // the gap chunk + + client.CurrentCallCount.ShouldBeGreaterThan(1); // the plan's assertion + client.CurrentCallCount.ShouldBe(2); // prime + exactly one re-baseline + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // the contiguous follow-up + + client.CurrentCallCount.ShouldBe(2); // no re-baseline storm + client.SampleCallCount.ShouldBe(2); // the stream was reopened once + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(201.5d); + } + + /// + /// …and it resumes from the re-baselined nextSequence, not from the stale + /// cursor the Agent has already evicted. Reopening at the old cursor would earn the identical + /// rejection immediately and forever. + /// + [Fact] + public async Task Sequence_gap_resumes_the_stream_from_the_rebaselined_next_sequence() + { + var client = CannedAgentClient.WithGapThenResume(); + var (driver, _) = await InitializedDriverAsync(client); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // gap + await client.PumpOnce().WaitAsync(Watchdog, Ct); // resume — only a reopened stream delivers this + + client.LastSampleFrom.ShouldBe(5005L); + } + + /// + /// The other half of the overflow story. A real cppagent answers a from that + /// has already fallen out of its buffer with an MTConnectError / OUT_OF_RANGE + /// document served under HTTP 200 — which the client surfaces as an + /// , never as a gap-bearing chunk. A pump that re-baselines + /// only on IsSequenceGap therefore fails precisely when it has fallen furthest behind. + /// + [Fact] + public async Task Out_of_range_stream_error_also_triggers_a_rebaseline() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + + client.SampleFailures.Enqueue(new InvalidDataException( + "MTConnect /sample answered an MTConnectError document (OUT_OF_RANGE) under HTTP 200")); + client.CurrentAnswers.Enqueue(Chunk(490, 500, ("dev1_pos", "50.5"))); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + // Only a pump that re-baselined and reopened the stream can consume this. + await client.PumpAsync(Chunk(500, 505, ("dev1_pos", "77.5"))).WaitAsync(Watchdog, Ct); + + client.CurrentCallCount.ShouldBe(2); + client.SampleCallCount.ShouldBe(2); + client.LastSampleFrom.ShouldBe(500L); + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(77.5d); + } + + // ---- agent restart ---- + + /// + /// The instanceId check must come BEFORE the gap check. An Agent restart changes + /// instanceId AND resets sequences, so a restart usually trips + /// IsSequenceGap too — and the two demand different handling: a gap keeps the held + /// values (they are still this device's), a restart invalidates every one of them because + /// the device model they describe no longer exists. This chunk is deliberately BOTH: new + /// instanceId and a firstSequence far past the cursor. A driver that tests the gap + /// first keeps serving dev1_execution's pre-restart value indefinitely — the new + /// Agent never reports it again, so nothing would ever overwrite it. + /// + [Fact] + public async Task Agent_restart_clears_the_index_and_is_detected_before_the_gap_path() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos", "dev1_execution"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + // The restarted Agent's /current knows nothing about dev1_execution. + client.CurrentAnswers.Enqueue(ChunkFrom(RestartedInstanceId, 40, 42, ("dev1_pos", "9.5"))); + + await client + .PumpAsync(ChunkFrom(RestartedInstanceId, 5000, 5005, ("dev1_pos", "5.5"))) + .WaitAsync(Watchdog, Ct); + + driver.AgentInstanceId.ShouldBe(RestartedInstanceId); + client.CurrentCallCount.ShouldBe(2); + + // Cleared: the pre-restart value is gone, not merely shadowed. + driver.ObservationIndex.Get("dev1_execution").StatusCode.ShouldBe(BadWaitingForInitialData); + driver.ObservationIndex.Get("dev1_pos").Value.ShouldBe(9.5d); + + // And the subscriber was TOLD its value went away, rather than being left holding a stale Good. + For(seen, "dev1_execution").Last().Snapshot.StatusCode.ShouldBe(BadWaitingForInitialData); + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(9.5d); + } + + // ---- stream ends ---- + + /// + /// A dropped connection is transient: reconnect from the cursor and keep going. It is + /// emphatically NOT a re-baseline — the sequence is still valid, and spending a + /// /current on every network blip would be a self-inflicted load. + /// + [Fact] + public async Task Stream_ended_reconnects_from_the_cursor_without_a_rebaseline() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + client.EndStream(); + + // Only a reconnected pump can consume a chunk written to the NEW stream generation. + await client.PumpAsync(Chunk(113, 118, ("dev1_pos", "2.5"))).WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(2); + client.LastSampleFrom.ShouldBe(113L); + client.CurrentCallCount.ShouldBe(1); + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(2.5d); + } + + /// + /// The hot-loop pin. means the + /// endpoint will never stream to this request — a reverse proxy, a health page, an Agent + /// fronted by something that collapses multipart/x-mixed-replace. Reconnecting + /// reproduces the identical answer forever, so the pump must give up loudly instead of + /// hammering the endpoint until an operator notices. A later subscription must not restart + /// the loop either: the answer has not changed just because someone asked again. + /// + [Fact] + public async Task Stream_not_supported_stops_the_pump_and_never_retries() + { + var (driver, client) = await InitializedDriverAsync(); + client.SampleFailure = new MTConnectStreamNotSupportedException( + "the Agent answered /sample with a single non-multipart document"); + + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + var pump = driver.SampleStreamTask.ShouldNotBeNull(); + + await pump.WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(1); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + driver.GetHealth().LastError.ShouldNotBeNullOrWhiteSpace(); + + // A full unsubscribe/resubscribe cycle must not re-open the wound either. + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + client.SampleCallCount.ShouldBe(1); + } + + /// + /// …but a re-initialize IS an operator changing something, so it clears the verdict and + /// tries again. Otherwise fixing the proxy would need a process restart. + /// + [Fact] + public async Task Reinitialize_clears_the_stream_unsupported_verdict() + { + var (driver, client) = await InitializedDriverAsync(); + client.SampleFailure = new MTConnectStreamNotSupportedException("not multipart"); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await driver.SampleStreamTask!.WaitAsync(Watchdog, Ct); + + client.SampleFailure = null; + await driver.ReinitializeAsync("{}", Ct).WaitAsync(Watchdog, Ct); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(2); + } + + // ---- backoff ---- + + /// + /// The reconnect backoff as a pure function of the attempt number: the first retry honours + /// MinBackoffMs (zero = immediate), and every later one grows geometrically to the + /// cap. The 100 ms growth floor matters more than it looks — the default + /// MinBackoffMs is 0, and 0 × multiplier is still 0, so without a floor + /// the "geometric" backoff would be an unbounded hot loop against a dead Agent. + /// + [Fact] + public void Reconnect_backoff_starts_at_min_grows_geometrically_and_caps() + { + var options = new MTConnectReconnectOptions + { + MinBackoffMs = 0, MaxBackoffMs = 1000, BackoffMultiplier = 2.0, + }; + + MTConnectDriver.BackoffFor(1, options).ShouldBe(TimeSpan.Zero); + MTConnectDriver.BackoffFor(2, options).ShouldBe(TimeSpan.FromMilliseconds(200)); + MTConnectDriver.BackoffFor(3, options).ShouldBe(TimeSpan.FromMilliseconds(400)); + MTConnectDriver.BackoffFor(4, options).ShouldBe(TimeSpan.FromMilliseconds(800)); + MTConnectDriver.BackoffFor(5, options).ShouldBe(TimeSpan.FromMilliseconds(1000)); + MTConnectDriver.BackoffFor(500, options).ShouldBe(TimeSpan.FromMilliseconds(1000)); + } + + /// + /// Operator-authored nonsense must not un-bound the loop: a multiplier of 1 (or less) would + /// never grow, and a negative delay is not a delay. Both fall back to the shipped default. + /// + [Fact] + public void Reconnect_backoff_survives_a_degenerate_multiplier() + { + var options = new MTConnectReconnectOptions + { + MinBackoffMs = -5, MaxBackoffMs = 10_000, BackoffMultiplier = 0.5, + }; + + MTConnectDriver.BackoffFor(1, options).ShouldBe(TimeSpan.Zero); + MTConnectDriver.BackoffFor(2, options).ShouldBeGreaterThan(TimeSpan.Zero); + MTConnectDriver.BackoffFor(3, options) + .ShouldBeGreaterThan(MTConnectDriver.BackoffFor(2, options)); + } + + // ---- unsubscribe ---- + + /// + /// Dropping one handle drops only its references; the shared stream keeps running for the + /// others. Tearing the stream down on the first unsubscribe would silently blind every + /// remaining subscription. + /// + [Fact] + public async Task Unsubscribe_drops_only_that_handles_refs_and_keeps_the_shared_stream() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + var a = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + var b = await driver.SubscribeAsync(["dev1_execution"], TimeSpan.FromMilliseconds(50), Ct); + var pump = driver.SampleStreamTask.ShouldNotBeNull(); + + await driver.UnsubscribeAsync(a, Ct).WaitAsync(Watchdog, Ct); + seen.Clear(); + + await client + .PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"), ("dev1_execution", "READY"))) + .WaitAsync(Watchdog, Ct); + + pump.IsCompleted.ShouldBeFalse(); + seen.Select(e => e.FullReference).Distinct().ShouldBe(["dev1_execution"]); + + await driver.UnsubscribeAsync(b, Ct).WaitAsync(Watchdog, Ct); + + pump.IsCompleted.ShouldBeTrue(); + driver.SampleStreamTask.ShouldBeNull(); + } + + /// The last unsubscribe stops the stream — and nothing is published after it. + [Fact] + public async Task Unsubscribe_of_the_last_handle_stops_the_stream_and_silences_the_driver() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + seen.Clear(); + + // The stream is gone, so this chunk has no consumer and nothing may reach the recorder. + client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).IsCompleted.ShouldBeFalse(); + + seen.ShouldBeEmpty(); + driver.SampleStreamTask.ShouldBeNull(); + } + + /// + /// Unsubscribing twice, or with a handle this driver never issued, is a no-op — not a throw. + /// Teardown paths run on failure paths, and an exception there would mask the real fault. + /// + [Fact] + public async Task Unsubscribe_is_idempotent_and_ignores_a_foreign_handle() + { + var (driver, _) = await InitializedDriverAsync(); + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + await driver.UnsubscribeAsync(new ForeignHandle(), Ct).WaitAsync(Watchdog, Ct); + } + + /// A null argument is a caller bug, and the one thing these methods are loud about. + [Fact] + public async Task Null_arguments_throw_ArgumentNullException() + { + var (driver, _) = await InitializedDriverAsync(); + + await Should.ThrowAsync( + () => driver.SubscribeAsync(null!, TimeSpan.FromMilliseconds(50), Ct)); + await Should.ThrowAsync(() => driver.UnsubscribeAsync(null!, Ct)); + } + + // ---- lifecycle ---- + + /// + /// Subscribing before the driver is initialized registers the interest and reports the + /// honest "no value yet" — it does not throw, and it does not dial an Agent that does not + /// exist yet. Initialize then picks the standing subscription up. + /// + [Fact] + public async Task Subscribe_before_initialize_registers_and_initialize_starts_the_stream() + { + var (driver, client) = NewDriver(); + var seen = Record(driver); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + client.SampleCallCount.ShouldBe(0); + driver.SampleStreamTask.ShouldBeNull(); + seen.Single().Snapshot.StatusCode.ShouldBe(BadWaitingForInitialData); + + await driver.InitializeAsync("{}", Ct); + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(1); + For(seen, "dev1_pos").Last().Snapshot.Value.ShouldBe(1.5d); + } + + /// + /// A re-initialize replaces the served state, so it must replace the pump with it: the old + /// one is stopped (it holds the old cursor, and possibly the old client) and a new one starts + /// from the fresh /current. The standing subscription survives — the caller did not + /// ask for it to be dropped — and is re-primed, because every value it holds came from a + /// baseline that has just been thrown away. + /// + [Fact] + public async Task Reinitialize_stops_the_old_pump_starts_a_new_one_and_republishes() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + var first = driver.SampleStreamTask.ShouldNotBeNull(); + + // Barrier: the first pump is demonstrably enumerating before the re-initialize lands on it. + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + client.CurrentAnswers.Enqueue(Chunk(300, 310, ("dev1_pos", "310.5"))); + seen.Clear(); + + await driver.ReinitializeAsync("{}", Ct).WaitAsync(Watchdog, Ct); + + first.IsCompleted.ShouldBeTrue(); + var second = driver.SampleStreamTask.ShouldNotBeNull(); + second.ShouldNotBeSameAs(first); + + await client.PumpAsync(Chunk(310, 315, ("dev1_pos", "315.5"))).WaitAsync(Watchdog, Ct); + + client.SampleCallCount.ShouldBe(2); + client.LastSampleFrom.ShouldBe(310L); + For(seen, "dev1_pos").Select(e => e.Snapshot.Value).ShouldBe([310.5d, 315.5d]); + } + + /// + /// Shutdown stops the pump BEFORE disposing the client. Getting that order wrong leaves the + /// pump enumerating a disposed client — which it would read as a dropped connection and try + /// to reconnect, forever, against an object that no longer exists. + /// + [Fact] + public async Task Shutdown_stops_the_pump_before_disposing_the_client_and_leaks_nothing() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + var pump = driver.SampleStreamTask.ShouldNotBeNull(); + + // Barrier: the stream is demonstrably open before the shutdown lands, so the call count + // below is a statement about reconnect churn rather than about who won a race. + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + seen.Clear(); + + await driver.ShutdownAsync(Ct).WaitAsync(Watchdog, Ct); + + pump.IsCompleted.ShouldBeTrue(); + pump.IsFaulted.ShouldBeFalse(); + driver.SampleStreamTask.ShouldBeNull(); + client.IsDisposed.ShouldBeTrue(); + + // No reconnect churn against the disposed client, and nothing published after teardown. + client.SampleCallCount.ShouldBe(1); + seen.ShouldBeEmpty(); + } + + /// + /// The deadlock pin. The lifecycle semaphore is non-reentrant, so a pump that reached + /// back into InitializeAsync/ReinitializeAsync/ShutdownAsync — the + /// natural way to write "on a gap, just re-prime" — would hang the driver permanently, with + /// no exception and no log. Here the pump is parked INSIDE its re-baseline /current + /// when a shutdown lands and takes that semaphore: the shutdown must cancel the pump and + /// complete. If the pump were waiting on the lifecycle lock instead, neither side would ever + /// move. + /// + [Fact] + public async Task Shutdown_while_the_pump_is_mid_rebaseline_does_not_deadlock() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + var entered = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + client.CurrentEntered = entered; + client.CurrentGate = release; + + // A gap forces the re-baseline; the gate holds the pump inside the /current request. + _ = client.PumpAsync(Chunk(5000, 5005, ("dev1_pos", "5.5"))); + await entered.Task.WaitAsync(Watchdog, Ct); + + try + { + await driver.ShutdownAsync(Ct).WaitAsync(Watchdog, Ct); + } + finally + { + release.TrySetResult(); + } + + driver.SampleStreamTask.ShouldBeNull(); + client.IsDisposed.ShouldBeTrue(); + } + + // ---- robustness + health ---- + + /// + /// A subscriber that throws is a bug in the consumer, not a reason to stop delivering data to + /// every other subscriber. The pump absorbs it and keeps pumping. + /// + [Fact] + public async Task A_throwing_subscriber_does_not_kill_the_pump() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = Record(driver); + driver.OnDataChange += (_, _) => throw new InvalidOperationException("subscriber blew up"); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + seen.Clear(); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + await client.PumpAsync(Chunk(113, 118, ("dev1_pos", "2.5"))).WaitAsync(Watchdog, Ct); + + driver.SampleStreamTask!.IsCompleted.ShouldBeFalse(); + For(seen, "dev1_pos").Select(e => e.Snapshot.Value).ShouldBe([1.5d, 2.5d]); + } + + /// + /// Health precedence, read side. A healthy stream must not paper over a broken read + /// path: /sample and /current are different Agent requests and can fail + /// independently, and "Healthy" while every OPC UA Read returns Bad is exactly the + /// failure-wearing-success shape this driver is written against. + /// + [Fact] + public async Task A_pumped_chunk_does_not_paper_over_a_failed_read() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + client.CurrentFailure = new HttpRequestException("connection refused"); + await driver.ReadAsync(["dev1_pos"], Ct); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + + // …and clearing the read failure does restore it, because the stream never failed. + client.CurrentFailure = null; + await driver.ReadAsync(["dev1_pos"], Ct); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + } + + /// + /// Health precedence, stream side. The mirror image: a successful read must not paper + /// over a stream that has stopped delivering. Only the path that failed may clear its own + /// degradation. + /// + [Fact] + public async Task A_successful_read_does_not_paper_over_a_broken_stream() + { + var (driver, client) = await InitializedDriverAsync(); + client.SampleFailure = new MTConnectStreamNotSupportedException("not multipart"); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await driver.SampleStreamTask!.WaitAsync(Watchdog, Ct); + + var res = await driver.ReadAsync(["dev1_pos"], Ct); + + res[0].StatusCode.ShouldBe(Good); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + } + + /// A recovered stream clears its own degradation once chunks flow again. + [Fact] + public async Task A_recovered_stream_restores_health() + { + var (driver, client) = await InitializedDriverAsync(); + client.SampleFailures.Enqueue(new TimeoutException("no chunk within the heartbeat window")); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + client.SampleCallCount.ShouldBe(2); + } + + /// A handle from some other driver — the type check must not be a cast. + private sealed class ForeignHandle : ISubscriptionHandle + { + public string DiagnosticId => "not-ours"; + } +} -- 2.52.0 From 13dd9fcd70c83ae4966d74bad5856a32f8c93fda Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 16:41:21 -0400 Subject: [PATCH 19/34] feat(mtconnect): ITagDiscovery streams tree + SupportsOnlineDiscovery opt-in (Task 12) --- .../MTConnectDriver.cs | 246 ++++++- .../CapturingBuilder.cs | 132 ++++ .../MTConnectDiscoverTests.cs | 680 ++++++++++++++++++ 3 files changed, 1053 insertions(+), 5 deletions(-) create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CapturingBuilder.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index 36214f5a..f3839fd1 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -43,10 +43,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// /// /// Scope. This type currently implements , -/// and . ITagDiscovery (Task 12) -/// and IHostConnectivityProbe/IRediscoverable (Task 13) are added on top of -/// the state this class already caches — the probe model, the Agent instanceId, and -/// the observation index. +/// , and . +/// IHostConnectivityProbe/IRediscoverable (Task 13) are added on top of the +/// state this class already caches — the probe model, the Agent instanceId, and the +/// observation index. There is deliberately no IWritable: MTConnect is a read-only +/// protocol, which is also why every discovered leaf is +/// . /// /// /// The read path never writes the shared observation index — see @@ -54,7 +56,7 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// (). /// /// -public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable +public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDiscovery { /// /// Coarse per-entry byte weights behind . The contract asks @@ -733,6 +735,240 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable _driverInstanceId, ours.DiagnosticId, stopStream ? " and stopped the shared /sample stream" : string.Empty); } + // ---- ITagDiscovery: the /probe device model, streamed as folders + variables ---- + + /// + /// + /// This one member is the whole browse opt-in. The Wave-0 universal + /// DiscoveryDriverBrowser turns any driver that answers true here into a live + /// address picker by capturing — so this driver ships no browser + /// code, no per-driver picker component, and no entry in the browser's browse-patch table + /// (its discovery is unconditional; there is no "enable browse" option to turn on first). + /// It is answered on an un-initialized instance without dialling anything, because + /// CanBrowse constructs a throwaway driver purely to read it — which is also why the + /// constructor opens nothing. + /// + public bool SupportsOnlineDiscovery => true; + + /// + /// + /// Once, not the default UntilStable: streams the + /// complete device model within the single call — /probe is one synchronous request + /// that either answers with the whole hierarchy or fails — so nothing fills in asynchronously + /// after connect and a settle loop would only re-fetch an identical answer. (The one thing + /// that does change the model is an Agent restart, and that is + /// IRediscoverable's job in Task 13, not a retry cadence's.) + /// + public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once; + + /// + /// + /// + /// Streams the Agent's /probe hierarchy verbatim: one folder per Device, + /// one per nested Component to arbitrary depth, and one variable per + /// DataItem under whichever container declares it — including the data items a + /// device declares directly, outside any component (an availability item almost + /// always lives there, and a walker that only recursed Components would silently + /// drop it). + /// + /// + /// is the DataItem@id, never the + /// browse name. The universal browser commits a browsed leaf's FullName + /// straight into TagConfig.FullName, and that is the key + /// and the /sample pump resolve an observation against + /// ( is keyed by DataItem@id). Committing + /// the name instead would produce a picker that looks right and authors tags that + /// can never report a value. The browse name — cosmetic — prefers the DataItem's + /// name and falls back to its id, which for this protocol is the common + /// case rather than an edge case. + /// + /// + /// The type shape comes from 's returned + /// record, in full — and + /// included. Recomputing the array shape + /// locally from representation/sampleCount is the tempting shortcut and it + /// is wrong: the inference also demotes DATA_SET/TABLE to + /// even on a SAMPLE, and honours + /// TIME_SERIES only on a SAMPLE. The AdminUI typed editor calls the same + /// function, so any divergence here makes the picker and the editor disagree about one + /// data item. The type attribute is passed verbatim for the same reason: + /// the probe document spells it UPPER_SNAKE (PART_COUNT) where the streams + /// document uses PascalCase, and the inference is separator-insensitive precisely to + /// bridge the two — pre-normalising it here would defeat that. + /// + /// + /// is this seam's job, deliberately not + /// the inference's: a CONDITION data item is an alarm. + /// IVariableHandle.MarkAsAlarmCondition is not called — that annotation + /// needs the driver-side refs of an alarm's sibling attributes (in-alarm flag, priority, + /// ack target), and an MTConnect condition has none: it is one text observation whose + /// value is the state word. Core's generic node manager enriches on the flag alone. + /// + /// + /// The model comes from , never from + /// . may have + /// dropped the cache under memory pressure; reading the field directly would make a + /// flushed driver browse as an Agent with no data items, permanently. + /// + /// + /// This throws, and that is the opposite of 's posture on + /// purpose. A read has a per-reference Bad status code to report a failure in-band; a + /// discovery stream has no such channel, so the only way to "fail quietly" would be to + /// emit an empty tree — indistinguishable from an Agent that genuinely declares nothing, + /// and read by an operator in the picker as a working connection to an empty machine + /// (#485: empty is not an answer). Both production callers handle the throw: the + /// universal browser surfaces it as a failed browse, and DriverInstanceActor logs + /// it and retries on its rediscover tick. + /// + /// + /// Nothing here writes the observation index. Discovery is a pure read of the + /// device model — the /sample pump remains its sole writer. + /// + /// + /// is null. + /// + /// The driver is not initialized, or it was torn down while the /probe fetch was in + /// flight (see ). + /// + public async Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(builder); + + var model = await GetOrFetchProbeModelAsync(cancellationToken).ConfigureAwait(false); + + // Read once: a concurrent re-initialize must not scope half this walk to one device and half + // to another. + var deviceScope = _options.DeviceName?.Trim(); + + var devices = 0; + var variables = 0; + + foreach (var device in model.Devices) + { + if (device is null || !InDeviceScope(device, deviceScope)) + { + continue; + } + + devices++; + var name = FolderName(device.Name, device.Id); + variables += StreamContainer(device, builder.Folder(name, name)); + } + + _logger.LogDebug( + "MTConnect driver {DriverInstanceId} streamed {DeviceCount} device(s) and {VariableCount} data item(s) from {AgentUri}'s /probe model{DeviceScope}.", + _driverInstanceId, + devices, + variables, + _options.AgentUri, + deviceScope is null or "" ? string.Empty : $" (scoped to '{deviceScope}')"); + } + + /// + /// Streams one device/component's own data items into , then recurses + /// into each nested component under a folder of its own. Returns how many variables it + /// streamed, for the diagnostic tally. + /// + /// + /// Walks the tree rather than using AllDataItems(): that helper flattens, and the + /// folder structure — which item belongs to which component — is precisely what a browse + /// picker exists to show. Recursion depth is bounded by the parser, which refuses a probe + /// document nested beyond its own limit. + /// + private static int StreamContainer(IMTConnectComponentContainer container, IAddressSpaceBuilder scope) + { + var streamed = 0; + + foreach (var dataItem in container.DataItems) + { + // An id-less data item cannot be read, subscribed, or committed as a tag — there would be + // nothing to key it by. The parser already requires the attribute; this is the guard for a + // model built any other way. + if (dataItem is null || string.IsNullOrWhiteSpace(dataItem.Id)) + { + continue; + } + + // Every field the DataItem declares, straight through — the array shape is READ OFF the + // result, never recomputed here. See the DiscoverAsync remarks. + var inferred = MTConnectDataTypeInference.Infer( + dataItem.Category, + dataItem.Type, + dataItem.Units, + dataItem.Representation, + dataItem.SampleCount); + + var browseName = string.IsNullOrWhiteSpace(dataItem.Name) ? dataItem.Id : dataItem.Name; + + scope.Variable( + browseName, + browseName, + new DriverAttributeInfo( + FullName: dataItem.Id, + DriverDataType: inferred.DataType, + IsArray: inferred.IsArray, + ArrayDim: inferred.ArrayDim, + + // MTConnect is read-only and this driver implements no IWritable, so the tier + // that is read-only from OPC UA is the only honest one. + SecurityClass: SecurityClassification.ViewOnly, + + // Historization is authored per tag, never inferred from a device model. + IsHistorized: false, + IsAlarm: IsCondition(dataItem.Category))); + + streamed++; + } + + foreach (var component in container.Components) + { + if (component is null) + { + continue; + } + + var name = FolderName(component.Name, component.Id); + streamed += StreamContainer(component, scope.Folder(name, name)); + } + + return streamed; + } + + /// + /// Whether a device is in this instance's configured scope. An unset scope is agent-wide. + /// + /// + /// + /// Matched against the device's name or its id — a device model may + /// omit the name, and its id is then what an operator would put in the config and what a + /// scoped Agent URL would carry. + /// + /// + /// Belt-and-braces, not dead code. The production client already narrows the + /// request to {AgentUri}/{DeviceName}/probe, so a conforming Agent answers with the + /// one device anyway. But an Agent (or an intermediary) that ignored the path segment and + /// answered agent-wide would otherwise publish every other machine's data items into an + /// instance scoped to one, and the operator's only evidence would be a picker full of ids + /// they never configured. Matching is exact for the same reason the URL is: a + /// differently-cased device name is not the device the Agent would have served. + /// + /// + private static bool InDeviceScope(MTConnectDevice device, string? deviceScope) => + string.IsNullOrEmpty(deviceScope) + || string.Equals(device.Name, deviceScope, StringComparison.Ordinal) + || string.Equals(device.Id, deviceScope, StringComparison.Ordinal); + + /// The folder name for a device/component: its name, or its id when it declares none. + private static string FolderName(string? name, string id) => string.IsNullOrWhiteSpace(name) ? id : name; + + /// + /// Whether a data item's category makes it an alarm. Lives here rather than in + /// because that function answers a type + /// question; this is an address-space one. + /// + private static bool IsCondition(string? category) => + category.AsSpan().Trim().Equals("CONDITION", StringComparison.OrdinalIgnoreCase); + /// /// Starts the shared /sample pump, if there is anything to pump and anything to pump /// from. Caller must hold . diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CapturingBuilder.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CapturingBuilder.cs new file mode 100644 index 00000000..eed2defd --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CapturingBuilder.cs @@ -0,0 +1,132 @@ +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// An that records the tree a driver streams into it, so +/// DiscoverAsync can be asserted on shape (which node landed under which folder) +/// and not merely on a flat list of leaves. +/// +/// +/// +/// Why this exists rather than a reference to the shipped capturing builder. Two +/// production capturing builders exist — Commons.Browsing.CapturingAddressSpaceBuilder +/// (universal browser) and Runtime.Drivers.CapturingAddressSpaceBuilder (deploy-time +/// discovery) — but this suite deliberately references only the driver and its +/// .Contracts, so neither is reachable without pulling the whole server stack into a +/// driver unit-test project. Both also flatten differently from what these tests must prove: +/// the browser's node id for a folder is a synthetic path while a leaf's id is its +/// FullName, which is exactly the conflation a "did the device-level data item land in +/// the device folder?" assertion needs to see through. +/// +/// +/// Deliberately not thread-safe. A driver streams discovery on one caller; recording +/// behind a lock would hide a driver that fanned out onto other threads. +/// +/// +internal sealed class CapturingBuilder : IAddressSpaceBuilder +{ + private readonly State _state; + private readonly string _path; + + /// Creates a root builder — the scope a driver's DiscoverAsync is handed. + public CapturingBuilder() + { + _state = new State(); + _path = string.Empty; + } + + private CapturingBuilder(State state, string path) + { + _state = state; + _path = path; + } + + /// Every folder streamed, in call order, with the slash-joined path it landed at. + public IReadOnlyList Folders => _state.Folders; + + /// Every variable streamed, in call order, with the folder path it landed under. + public IReadOnlyList Variables => _state.Variables; + + /// Every property streamed, with the scope path it was added to. + public IReadOnlyList Properties => _state.Properties; + + /// + public IAddressSpaceBuilder Folder(string browseName, string displayName) + { + var path = _path.Length == 0 ? browseName : $"{_path}/{browseName}"; + _state.Folders.Add(new CapturedFolder(path, _path, browseName, displayName)); + + return new CapturingBuilder(_state, path); + } + + /// + public IVariableHandle Variable(string browseName, string displayName, DriverAttributeInfo attributeInfo) + { + var captured = new CapturedVariable(_path, browseName, displayName, attributeInfo); + _state.Variables.Add(captured); + + return new CapturedVariableHandle(captured); + } + + /// + public void AddProperty(string browseName, DriverDataType dataType, object? value) => + _state.Properties.Add(new CapturedProperty(_path, browseName, dataType, value)); + + private sealed class State + { + public List Folders { get; } = []; + + public List Variables { get; } = []; + + public List Properties { get; } = []; + } + + private sealed class CapturedVariableHandle(CapturedVariable variable) : IVariableHandle + { + public string FullReference => variable.Attr.FullName; + + public IAlarmConditionSink MarkAsAlarmCondition(AlarmConditionInfo info) + { + variable.AlarmConditions.Add(info); + + return new NullSink(); + } + + private sealed class NullSink : IAlarmConditionSink + { + public void OnTransition(AlarmEventArgs args) + { + // Nothing to record: no test in this suite drives an alarm transition through + // discovery, and a sink that threw would fail a driver that legitimately never + // pushes one. + } + } + } +} + +/// One folder captured from a discovery stream. +/// Slash-joined path of the folder itself. +/// Slash-joined path of the scope it was added to (empty at the root). +/// The browse name the driver supplied. +/// The display name the driver supplied. +internal sealed record CapturedFolder(string Path, string ParentPath, string BrowseName, string DisplayName); + +/// One variable captured from a discovery stream. +/// Slash-joined path of the folder it landed under (empty at the root). +/// The browse name the driver supplied. +/// The display name the driver supplied. +/// The driver-side attribute metadata the driver stamped on it. +internal sealed record CapturedVariable( + string ParentPath, string BrowseName, string DisplayName, DriverAttributeInfo Attr) +{ + /// Alarm conditions the driver marked this variable with, if any. + public List AlarmConditions { get; } = []; +} + +/// One property captured from a discovery stream. +/// Slash-joined path of the node it was added to. +/// The property browse name. +/// The property data type. +/// The property value. +internal sealed record CapturedProperty(string ScopePath, string BrowseName, DriverDataType DataType, object? Value); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs new file mode 100644 index 00000000..369e366b --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs @@ -0,0 +1,680 @@ +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 12 — 's half: the +/// /probe device model streamed into an as +/// Device → Component → DataItem, and the one-member browse opt-in +/// () the Wave-0 universal browser keys off. +/// +/// +/// +/// The load-bearing assertion is FullName == DataItem@id. The universal +/// browser commits a browsed leaf's straight into +/// TagConfig.FullName, and that is the key ReadAsync / the /sample pump +/// resolve an observation against ( is keyed by +/// DataItem@id). Streaming the browse name instead would produce a picker that +/// looks perfect and commits tags that can never report a value — every one of them stuck at +/// BadWaitingForInitialData behind a Healthy driver. +/// +/// +/// The second load-bearing assertion is the type shape. Discovery must stamp exactly +/// what returns, including the array shape — the +/// AdminUI typed editor calls the same function, so any local recomputation here makes the +/// picker and the editor disagree about the same data item. PART_COUNT → Int64 is +/// called out on its own because the probe document's UPPER_SNAKE spelling is the case a +/// PascalCase-only match silently types as String. +/// +/// +public sealed class MTConnectDiscoverTests +{ + private const string AgentUri = "http://fixture-agent:5000"; + + /// Every DataItem id the canned Fixtures/probe.xml declares. + private static readonly string[] FixtureDataItemIds = + [ + "dev1_avail", + "dev1_pos", + "dev1_vibration_ts", + "dev1_partcount", + "dev1_execution", + "dev1_program", + "dev1_system_cond", + ]; + + private static MTConnectDriverOptions Opts(string? deviceName = null) => + new() + { + AgentUri = AgentUri, + DeviceName = deviceName, + + // Discovery is a pure read of the /probe device model: it must enumerate what the Agent + // declares, NOT what an operator has already authored. Left empty on purpose so a driver + // that streamed its authored tag table instead could not pass. + Tags = [], + }; + + private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( + MTConnectDriverOptions? options = null, MTConnectProbeModel? probe = null) + { + var client = CannedAgentClient.FromFixtures(); + if (probe is not null) + { + client.Probe = probe; + } + + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => client), client); + } + + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( + MTConnectDriverOptions? options = null, MTConnectProbeModel? probe = null) + { + var (driver, client) = NewDriver(options, probe); + await driver.InitializeAsync("{}", TestContext.Current.CancellationToken); + + return (driver, client); + } + + private static async Task DiscoverAsync(MTConnectDriver driver) + { + var builder = new CapturingBuilder(); + await driver.DiscoverAsync(builder, TestContext.Current.CancellationToken); + + return builder; + } + + // ---- the plan's headline test ---- + + /// + /// The whole contract in one place: the fixture's Device → Controller → Path tree reaches the + /// builder, the CONDITION data item is present under its own id, it types as + /// , and the driver declares the browse opt-in the + /// universal browser gates on. + /// + [Fact] + public async Task Discover_streams_device_component_dataitem_tree_leaf_fullname_is_dataitemid() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Select(v => v.Attr.FullName).ShouldContain("dev1_system_cond"); + cap.Variables.Single(v => v.Attr.IsAlarm).Attr.DriverDataType.ShouldBe(DriverDataType.String); + driver.SupportsOnlineDiscovery.ShouldBeTrue(); + driver.RediscoverPolicy.ShouldBe(DiscoveryRediscoverPolicy.Once); + } + + // ---- tree shape ---- + + /// + /// The folder tree mirrors the probe document exactly: one folder per Device (named by its + /// name), then one per nested Component to arbitrary depth, each under its own parent. + /// + [Fact] + public async Task Discover_materialises_a_folder_per_device_and_nested_component() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Select(f => f.Path).ShouldBe( + ["VMC-3Axis", "VMC-3Axis/controller", "VMC-3Axis/controller/path"], ignoreOrder: true); + + // Parentage, not just membership: a flat "three folders exist" assertion passes for a walker + // that streamed all three onto the root builder. + cap.Folders.Single(f => f.Path == "VMC-3Axis").ParentPath.ShouldBe(string.Empty); + cap.Folders.Single(f => f.Path == "VMC-3Axis/controller").ParentPath.ShouldBe("VMC-3Axis"); + cap.Folders.Single(f => f.Path == "VMC-3Axis/controller/path").ParentPath.ShouldBe("VMC-3Axis/controller"); + } + + /// + /// dev1_avail hangs directly off <Device><DataItems>, outside every + /// Component. A walker that only recursed <Components> would drop it silently, so + /// it is asserted to land in the device folder while the Path-owned items land in the + /// path folder. + /// + [Fact] + public async Task Discover_places_device_level_data_items_in_the_device_folder() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Single(v => v.Attr.FullName == "dev1_avail").ParentPath.ShouldBe("VMC-3Axis"); + cap.Variables + .Where(v => v.Attr.FullName != "dev1_avail") + .ShouldAllBe(v => v.ParentPath == "VMC-3Axis/controller/path"); + } + + /// + /// Every DataItem the fixture declares is streamed exactly once, and every leaf's + /// is its DataItem@id — the read/subscribe + /// key, aligned with the browser's commit by construction. + /// + [Fact] + public async Task Discover_streams_every_data_item_once_with_fullname_equal_to_its_id() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Count.ShouldBe(FixtureDataItemIds.Length); + cap.Variables.Select(v => v.Attr.FullName).ShouldBe(FixtureDataItemIds, ignoreOrder: true); + } + + /// + /// The FullName-is-the-id rule, asserted where the two can actually differ. + /// + /// + /// The fixture cannot prove this on its own — its data items declare no name, so + /// browseName and id coincide and a driver that streamed the browse name as the + /// FullName would pass every fixture-based assertion in this file. This model gives + /// every data item a name that is disjoint from its id, so the two sets cannot be confused: + /// the commit key must be the id set, and the presentation must be the name set. + /// + [Fact] + public async Task Discover_fullname_is_the_id_even_when_the_data_item_declares_a_different_name() + { + var probe = ModelWith( + new MTConnectDataItem("id_pos", "Xpos", "SAMPLE", "POSITION", null, "MILLIMETER", null, null), + new MTConnectDataItem("id_count", "PartsMade", "EVENT", "PART_COUNT", null, null, null, null), + new MTConnectDataItem("id_cond", "SystemHealth", "CONDITION", "SYSTEM", null, null, null, null), + new MTConnectDataItem( + "id_series", "Feed", "SAMPLE", "PATH_FEEDRATE", null, null, "TIME_SERIES", 10)); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Select(v => v.Attr.FullName) + .ShouldBe(["id_pos", "id_count", "id_cond", "id_series"], ignoreOrder: true); + cap.Variables.Select(v => v.BrowseName) + .ShouldBe(["Xpos", "PartsMade", "SystemHealth", "Feed"], ignoreOrder: true); + + // …and the id-keyed metadata still lands on the right leaf once the two differ. + cap.Variables.Single(v => v.Attr.FullName == "id_count").Attr.DriverDataType + .ShouldBe(DriverDataType.Int64); + cap.Variables.Single(v => v.Attr.IsAlarm).Attr.FullName.ShouldBe("id_cond"); + cap.Variables.Single(v => v.Attr.IsArray).Attr.FullName.ShouldBe("id_series"); + } + + // ---- type shape (must equal MTConnectDataTypeInference, not a local recomputation) ---- + + /// + /// The per-category defaults reach the builder unchanged: SAMPLE ⇒ Float64, controlled + /// vocabulary / free-text EVENT ⇒ String, CONDITION ⇒ String. + /// + [Theory] + [InlineData("dev1_pos", DriverDataType.Float64)] + [InlineData("dev1_avail", DriverDataType.String)] + [InlineData("dev1_execution", DriverDataType.String)] + [InlineData("dev1_program", DriverDataType.String)] + [InlineData("dev1_system_cond", DriverDataType.String)] + public async Task Discover_stamps_the_inferred_data_type(string dataItemId, DriverDataType expected) + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Single(v => v.Attr.FullName == dataItemId).Attr.DriverDataType.ShouldBe(expected); + } + + /// + /// PART_COUNT is the live-only bug this driver's inference was already fixed for once: + /// the probe document spells it UPPER_SNAKE, so a match written against the Streams + /// document's PartCount types every real agent's part counter as + /// while a PascalCase unit test stays green. Asserted at + /// the discovery seam because that is where the probe spelling is actually read. + /// + [Fact] + public async Task Discover_types_the_upper_snake_PART_COUNT_event_as_Int64() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Single(v => v.Attr.FullName == "dev1_partcount").Attr.DriverDataType + .ShouldBe(DriverDataType.Int64); + } + + /// + /// A TIME_SERIES SAMPLE is an array whose length is the probe-declared + /// sampleCount. The shape comes from 's + /// returned record — recomputing it inline at this seam is what makes discovery and the + /// AdminUI editor disagree. + /// + [Fact] + public async Task Discover_carries_the_time_series_array_shape_from_the_inference() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + var series = cap.Variables.Single(v => v.Attr.FullName == "dev1_vibration_ts").Attr; + series.DriverDataType.ShouldBe(DriverDataType.Float64); + series.IsArray.ShouldBeTrue(); + series.ArrayDim.ShouldBe(10u); + } + + /// + /// A DATA_SET SAMPLE demotes to and stays scalar — + /// a rule that lives only inside the inference. An inline + /// "IsArray = representation == TIME_SERIES, DataType = category == SAMPLE ? Float64 : String" + /// at this seam would type it Float64 and go Bad on every parse, and no fixture-only test + /// would notice. + /// + [Fact] + public async Task Discover_demotes_a_data_set_sample_to_string_per_the_inference() + { + var probe = ModelWith( + new MTConnectDataItem( + "dev9_toolset", null, "SAMPLE", "POSITION", null, "MILLIMETER", "DATA_SET", 4)); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + var attr = cap.Variables.Single(v => v.Attr.FullName == "dev9_toolset").Attr; + attr.DriverDataType.ShouldBe(DriverDataType.String); + attr.IsArray.ShouldBeFalse(); + attr.ArrayDim.ShouldBeNull(); + } + + /// + /// TIME_SERIES is defined for SAMPLE only; on an EVENT the inference ignores it. Pins + /// the other half of "use the inference's answer, do not recompute the array shape". + /// + [Fact] + public async Task Discover_ignores_time_series_on_a_non_sample_per_the_inference() + { + var probe = ModelWith( + new MTConnectDataItem("dev9_msg", null, "EVENT", "MESSAGE", null, null, "TIME_SERIES", 8)); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + var attr = cap.Variables.Single(v => v.Attr.FullName == "dev9_msg").Attr; + attr.DriverDataType.ShouldBe(DriverDataType.String); + attr.IsArray.ShouldBeFalse(); + attr.ArrayDim.ShouldBeNull(); + } + + // ---- the flags this seam owns ---- + + /// + /// is the caller's job — the inference deliberately + /// does not compute it. Exactly the CONDITION data item carries it. + /// + [Fact] + public async Task Discover_marks_only_the_condition_data_item_as_an_alarm() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Where(v => v.Attr.IsAlarm).Select(v => v.Attr.FullName).ShouldBe(["dev1_system_cond"]); + } + + /// + /// MTConnect is a read-only protocol and this driver implements no IWritable, so every + /// browsed leaf is — the tier that is read-only + /// from OPC UA. Nothing is historized by discovery either: historization is authored per tag, + /// never inferred from a device model. + /// + [Fact] + public async Task Discover_streams_every_leaf_view_only_and_not_historized() + { + var (driver, _) = await InitializedDriverAsync(); + + var cap = await DiscoverAsync(driver); + + cap.Variables.ShouldAllBe(v => v.Attr.SecurityClass == SecurityClassification.ViewOnly); + cap.Variables.ShouldAllBe(v => !v.Attr.IsHistorized); + cap.Variables.ShouldAllBe(v => v.Attr.Source == NodeSourceKind.Driver); + } + + // ---- browse names ---- + + /// + /// The browse name is the DataItem's name when it declares one and its id + /// otherwise — the fixture's data items declare none, so the fallback is the normal case for + /// this protocol, not an edge case. + /// + [Fact] + public async Task Discover_browse_name_prefers_the_data_item_name_and_falls_back_to_its_id() + { + var probe = ModelWith( + new MTConnectDataItem("dev9_named", "Xpos", "SAMPLE", "POSITION", null, "MILLIMETER", null, null), + new MTConnectDataItem("dev9_unnamed", null, "EVENT", "PROGRAM", null, null, null, null)); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + var named = cap.Variables.Single(v => v.Attr.FullName == "dev9_named"); + named.BrowseName.ShouldBe("Xpos"); + named.DisplayName.ShouldBe("Xpos"); + + var unnamed = cap.Variables.Single(v => v.Attr.FullName == "dev9_unnamed"); + unnamed.BrowseName.ShouldBe("dev9_unnamed"); + + // The browse name is presentation; the read key is not. Restated here because this is the + // one test where the two legitimately differ. + named.Attr.FullName.ShouldBe("dev9_named"); + } + + /// + /// A Device or Component with no name attribute falls back to its id rather + /// than materialising a blank folder. + /// + [Fact] + public async Task Discover_folder_names_fall_back_to_the_id_when_no_name_is_declared() + { + var probe = new MTConnectProbeModel( + [ + new MTConnectDevice( + "dev9", + null, + [ + new MTConnectComponent( + "dev9_axes", + null, + [], + [new MTConnectDataItem("dev9_x", null, "SAMPLE", "POSITION", null, null, null, null)]), + ], + []), + ]); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Select(f => f.Path).ShouldBe(["dev9", "dev9/dev9_axes"], ignoreOrder: true); + cap.Variables.Single().ParentPath.ShouldBe("dev9/dev9_axes"); + } + + // ---- DeviceName scoping ---- + + /// + /// With no DeviceName the driver is agent-wide and every device's subtree is streamed. + /// The control for the scoping test below — without it, a discovery that dropped the second + /// device for an unrelated reason would look like correct scoping. + /// + [Fact] + public async Task Discover_without_a_device_scope_streams_every_device() + { + var (driver, _) = await InitializedDriverAsync(probe: TwoDeviceModel()); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Where(f => f.ParentPath.Length == 0).Select(f => f.BrowseName) + .ShouldBe(["VMC-3Axis", "Lathe-2Axis"], ignoreOrder: true); + cap.Variables.Select(v => v.Attr.FullName).ShouldContain("dev2_avail"); + } + + /// + /// With DeviceName set, only that device's subtree is streamed. + /// + /// + /// Belt-and-braces rather than dead code: the production client already narrows the request + /// to {AgentUri}/{DeviceName}/probe, so a conforming Agent answers with one device + /// anyway. But a non-conforming Agent (or a proxy that drops the path segment) that answered + /// agent-wide would otherwise publish every other machine's data items into an instance + /// scoped to one — and the operator's only evidence would be a picker full of ids they never + /// configured. + /// + [Fact] + public async Task Discover_with_a_device_scope_streams_only_that_device_subtree() + { + var (driver, _) = await InitializedDriverAsync(Opts(deviceName: "Lathe-2Axis"), TwoDeviceModel()); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Select(f => f.Path).ShouldBe(["Lathe-2Axis"]); + cap.Variables.Select(v => v.Attr.FullName).ShouldBe(["dev2_avail"]); + } + + /// + /// A device scope naming a device the Agent does not declare streams nothing. Kept explicit + /// because the alternative reading ("scope did not match, so stream everything") would hand a + /// mis-typed device name a picker showing the whole plant. + /// + [Fact] + public async Task Discover_with_an_unmatched_device_scope_streams_nothing() + { + var (driver, _) = await InitializedDriverAsync(Opts(deviceName: "not-a-device"), TwoDeviceModel()); + + var cap = await DiscoverAsync(driver); + + cap.Folders.ShouldBeEmpty(); + cap.Variables.ShouldBeEmpty(); + } + + /// + /// A device with no name is addressable by its id — which is what a scoped + /// Agent URL would carry for it too. + /// + [Fact] + public async Task Discover_device_scope_matches_a_nameless_device_by_id() + { + var probe = new MTConnectProbeModel( + [ + new MTConnectDevice( + "dev9", null, [], [new MTConnectDataItem("dev9_a", null, "EVENT", "PROGRAM", null, null, null, null)]), + new MTConnectDevice( + "dev8", "Other", [], [new MTConnectDataItem("dev8_a", null, "EVENT", "PROGRAM", null, null, null, null)]), + ]); + + var (driver, _) = await InitializedDriverAsync(Opts(deviceName: "dev9"), probe); + + var cap = await DiscoverAsync(driver); + + cap.Variables.Select(v => v.Attr.FullName).ShouldBe(["dev9_a"]); + } + + // ---- the probe-cache contract (anti-wedge) ---- + + /// + /// Discovery is served from the cache InitializeAsync already populated — a browse + /// does not re-hit the Agent for a model it is holding. + /// + [Fact] + public async Task Discover_serves_from_the_cached_probe_model() + { + var (driver, client) = await InitializedDriverAsync(); + client.ProbeCallCount.ShouldBe(1); + + await DiscoverAsync(driver); + + client.ProbeCallCount.ShouldBe(1); + } + + /// + /// …and after has dropped that cache, + /// discovery re-fetches and still streams the full tree. This is the whole reason discovery + /// must go through GetOrFetchProbeModelAsync rather than reading the cached field: a + /// memory-budget flush must never leave browse permanently answering "this agent has no data + /// items". + /// + [Fact] + public async Task Discover_refetches_the_probe_model_after_a_cache_flush() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.FlushOptionalCachesAsync(TestContext.Current.CancellationToken); + driver.CachedProbeModel.ShouldBeNull(); + + var cap = await DiscoverAsync(driver); + + client.ProbeCallCount.ShouldBe(2); + cap.Variables.Select(v => v.Attr.FullName).ShouldBe(FixtureDataItemIds, ignoreOrder: true); + driver.CachedProbeModel.ShouldNotBeNull(); + } + + // ---- discovery never touches the subscription surface ---- + + /// + /// Discovery is a pure read of the device model: it does not write the shared observation + /// index (the /sample pump is its sole writer) and it does not issue a /current. + /// A discovery that folded a snapshot into the index would be a second writer racing the + /// pump, rolling subscribed values backwards. + /// + [Fact] + public async Task Discover_does_not_touch_the_observation_index_or_call_current() + { + var options = new MTConnectDriverOptions + { + AgentUri = AgentUri, + Tags = [new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64)], + }; + + var (driver, client) = await InitializedDriverAsync(options); + var index = driver.ObservationIndex; + var indexed = index.Count; + var currentCalls = client.CurrentCallCount; + + await DiscoverAsync(driver); + + ReferenceEquals(driver.ObservationIndex, index).ShouldBeTrue(); + driver.ObservationIndex.Count.ShouldBe(indexed); + client.CurrentCallCount.ShouldBe(currentCalls); + } + + // ---- failure posture: loud, never a silently-empty tree ---- + + /// + /// Discovery before InitializeAsync throws rather than streaming an empty tree. + /// + /// + /// This is the deliberate opposite of 's + /// never-throw posture, and the difference is what the caller can do with the answer. A + /// read has per-reference Bad status codes to report a failure in-band; a discovery + /// stream has no such channel — an empty tree is indistinguishable from "this Agent declares + /// no data items", which is the #485 shape (failure wearing success's clothes) and would read + /// to an operator in the browse picker as a working connection to an empty machine. Both + /// production callers handle the throw: the universal browser surfaces it as a failed browse, + /// and DriverInstanceActor logs it and retries. + /// + [Fact] + public async Task Discover_before_initialize_throws_and_streams_nothing() + { + var (driver, client) = NewDriver(); + var builder = new CapturingBuilder(); + + await Should.ThrowAsync( + () => driver.DiscoverAsync(builder, TestContext.Current.CancellationToken)); + + builder.Folders.ShouldBeEmpty(); + builder.Variables.ShouldBeEmpty(); + client.ProbeCallCount.ShouldBe(0); + } + + /// Same posture after ShutdownAsync — a stopped driver has no model to browse. + [Fact] + public async Task Discover_after_shutdown_throws_and_streams_nothing() + { + var (driver, _) = await InitializedDriverAsync(); + await driver.ShutdownAsync(TestContext.Current.CancellationToken); + + var builder = new CapturingBuilder(); + + await Should.ThrowAsync( + () => driver.DiscoverAsync(builder, TestContext.Current.CancellationToken)); + + builder.Variables.ShouldBeEmpty(); + } + + /// + /// An Agent that fails the /probe a discovery needs fails the discovery — it does not + /// degrade to an empty tree. Same #485 reasoning as the un-initialized case. + /// + [Fact] + public async Task Discover_propagates_an_agent_probe_failure_rather_than_streaming_an_empty_tree() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.FlushOptionalCachesAsync(TestContext.Current.CancellationToken); + client.ProbeFailure = new HttpRequestException("agent down"); + + var builder = new CapturingBuilder(); + + await Should.ThrowAsync( + () => driver.DiscoverAsync(builder, TestContext.Current.CancellationToken)); + + builder.Variables.ShouldBeEmpty(); + } + + /// Caller cancellation propagates, as it does on every other seam of this driver. + [Fact] + public async Task Discover_propagates_caller_cancellation() + { + var (driver, _) = await InitializedDriverAsync(); + await driver.FlushOptionalCachesAsync(TestContext.Current.CancellationToken); + + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + await Should.ThrowAsync( + () => driver.DiscoverAsync(new CapturingBuilder(), cts.Token)); + } + + /// A null builder is a caller bug — the one thing this method is loud about up front. + [Fact] + public async Task Discover_with_a_null_builder_throws_ArgumentNullException() + { + var (driver, _) = await InitializedDriverAsync(); + + await Should.ThrowAsync( + () => driver.DiscoverAsync(null!, TestContext.Current.CancellationToken)); + } + + // ---- the browse opt-in ---- + + /// + /// The universal browser's CanBrowse constructs a throwaway instance purely to read + /// these two members, so they must answer on an un-initialized driver without + /// dialling anything. + /// + [Fact] + public void Browse_opt_in_answers_on_an_uninitialized_instance_without_connecting() + { + var (driver, client) = NewDriver(); + + driver.ShouldBeAssignableTo(); + driver.SupportsOnlineDiscovery.ShouldBeTrue(); + driver.RediscoverPolicy.ShouldBe(DiscoveryRediscoverPolicy.Once); + client.ProbeCallCount.ShouldBe(0); + client.CurrentCallCount.ShouldBe(0); + } + + // ---- helpers ---- + + /// A one-device model whose single Path component owns . + private static MTConnectProbeModel ModelWith(params MTConnectDataItem[] dataItems) => + new( + [ + new MTConnectDevice( + "dev9", + "Probe9", + [new MTConnectComponent("dev9_path", "path", [], dataItems)], + []), + ]); + + /// + /// The canned fixture's device plus a second, independent one — the multi-device Agent the + /// DeviceName scope exists for. + /// + private static MTConnectProbeModel TwoDeviceModel() + { + var fixture = MTConnectProbeParser.Parse(File.ReadAllText("Fixtures/probe.xml")); + var second = new MTConnectDevice( + "dev2", + "Lathe-2Axis", + [], + [new MTConnectDataItem("dev2_avail", null, "EVENT", "AVAILABILITY", null, null, null, null)]); + + return new MTConnectProbeModel([.. fixture.Devices, second]); + } +} -- 2.52.0 From fa0e40796f9ec5d01b8bc5c66585d19c501cec1b Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 16:56:31 -0400 Subject: [PATCH 20/34] feat(mtconnect): host-connectivity probe + instanceId rediscover (Task 13) --- .../MTConnectDriver.cs | 474 ++++++++++++- .../MTConnectHostAndRediscoverTests.cs | 629 ++++++++++++++++++ 2 files changed, 1094 insertions(+), 9 deletions(-) create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index f3839fd1..7a129af8 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -42,21 +42,24 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// nextSequence. /// /// -/// Scope. This type currently implements , -/// , and . -/// IHostConnectivityProbe/IRediscoverable (Task 13) are added on top of the -/// state this class already caches — the probe model, the Agent instanceId, and the -/// observation index. There is deliberately no IWritable: MTConnect is a read-only -/// protocol, which is also why every discovered leaf is +/// Scope. This type implements , , +/// , , +/// and — the last two +/// built on state this class already caches (the Agent instanceId and the probe +/// model). There is deliberately no IWritable: MTConnect is a read-only protocol, +/// which is also why every discovered leaf is /// . /// /// /// The read path never writes the shared observation index — see /// . That index has exactly one writer: the /sample pump -/// (). +/// (). The same single-writer discipline covers the two +/// Task-13 fields: the connectivity probe loop is the sole writer of +/// , and it publishes nothing into +/// (see ). /// /// -public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDiscovery +public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDiscovery, IHostConnectivityProbe, IRediscoverable { /// /// Coarse per-entry byte weights behind . The contract asks @@ -128,6 +131,13 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis private readonly ILogger _logger; private readonly Func _agentClientFactory; + /// + /// How the connectivity probe loop waits between ticks. Injected so the cadence is a seam + /// rather than a wall clock: the loop is a timer, and a suite that drove it by sleeping would + /// be flaky by construction. Defaults to . + /// + private readonly Func _probeScheduler; + /// /// Serializes the three lifecycle entry points against each other. Initialize / Reinitialize /// / Shutdown all swap the client and the served caches; overlapping them would let a @@ -183,6 +193,48 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// Live subscriptions by id. Guarded by . private readonly Dictionary _subscriptions = []; + /// + /// Guards the two host-connectivity fields, which the probe loop writes and + /// reads from an arbitrary caller thread. Held for a field + /// read/write only — never across an await, and never while a status handler runs. + /// + private readonly Lock _probeLock = new(); + + /// + /// The Agent's last observed reachability. Written only by + /// , so the state and the event stream can never disagree: + /// every value this field has ever held was announced by exactly one + /// . That is why a successful + /// does not seed it — raising the event from there would run + /// caller code while the non-reentrant semaphore is held (the same + /// hazard that makes the pump, not , republish to subscribers), + /// and seeding it silently would leave a consumer that tracks the event stream disagreeing + /// with . until the first tick + /// is the honest answer. Guarded by . + /// + private HostState _hostState = HostState.Unknown; + + /// When last changed. Guarded by . + private DateTime _hostStateChangedUtc = DateTime.UtcNow; + + /// + /// Cancels the connectivity probe loop. Written only under , which + /// serializes every start and stop of the loop. + /// + private CancellationTokenSource? _probeCts; + + /// The connectivity probe loop's task, or null when none runs. See . + private Task? _probeTask; + + /// + /// The Agent instanceId the last was raised for — + /// seeded at the commit point of with the id the priming + /// /current reported. This is what makes rediscovery fire once per change + /// rather than once per chunk: every chunk after a restart carries the new id, and a + /// re-baseline that fails leaves the pump still holding the old one. + /// + private long _announcedInstanceId; + private MTConnectDriverOptions _options; private MTConnectObservationIndex _index; private DriverHealth _health = new(DriverState.Unknown, null, null); @@ -263,11 +315,17 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// , never from this constructor. /// /// Optional logger; defaults to the null logger. + /// + /// How the connectivity probe loop waits between ticks. Defaults to + /// ; tests inject a hand-driven ticker so + /// the probe's cadence is asserted deterministically instead of slept through. + /// public MTConnectDriver( MTConnectDriverOptions options, string driverInstanceId, Func? agentClientFactory = null, - ILogger? logger = null) + ILogger? logger = null, + Func? probeScheduler = null) { ArgumentNullException.ThrowIfNull(options); ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); @@ -276,6 +334,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis _driverInstanceId = driverInstanceId; _logger = logger ?? NullLogger.Instance; _agentClientFactory = agentClientFactory ?? (o => new MTConnectAgentClient(o, logger)); + _probeScheduler = probeScheduler ?? Task.Delay; // Never null, so every consumer (Task 10's ReadAsync included) can ask it for a snapshot // without a null check: an un-primed index answers BadWaitingForInitialData for an authored @@ -333,6 +392,14 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } } + /// + /// The connectivity probe loop's task while one is running, else null. Exposed to + /// tests as the deterministic teardown barrier — "the probe loop has stopped" is a task + /// completion, never a sleep — and as the proof that Probe.Enabled = false starts + /// nothing at all. + /// + internal Task? ProbeLoopTask => Volatile.Read(ref _probeTask); + /// The options currently in force — the constructor's, or the last document applied. internal MTConnectDriverOptions EffectiveOptions => _options; @@ -362,6 +429,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // enumerating a disposed client reads the disposal as a dropped connection and // reconnects against an object that no longer exists. await StopSampleStreamAsync().ConfigureAwait(false); + await StopProbeLoopAsync().ConfigureAwait(false); await TeardownCoreAsync().ConfigureAwait(false); await StartCoreAsync(options, existingClient: null, cancellationToken).ConfigureAwait(false); @@ -406,6 +474,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis var existing = Volatile.Read(ref _session)?.Client; await StopSampleStreamAsync().ConfigureAwait(false); + await StopProbeLoopAsync().ConfigureAwait(false); if (existing is null || RequiresClientRebuild(_options, incoming)) { @@ -438,6 +507,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis var lastRead = ReadHealth().LastSuccessfulRead; await StopSampleStreamAsync().ConfigureAwait(false); + await StopProbeLoopAsync().ConfigureAwait(false); await TeardownCoreAsync().ConfigureAwait(false); WriteHealth(new DriverHealth(DriverState.Unknown, lastRead, null)); @@ -969,6 +1039,351 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis private static bool IsCondition(string? category) => category.AsSpan().Trim().Equals("CONDITION", StringComparison.OrdinalIgnoreCase); + // ---- IHostConnectivityProbe: one host per Agent, on its own cheap tick ---- + + /// + /// + /// Raised from the probe loop's task, never under a lock and never under + /// . A handler is arbitrary caller code: one that blocks forever + /// blocks the loop with it (and therefore the teardown that awaits it), exactly as an + /// handler does to the pump. One that throws is absorbed. + /// + public event EventHandler? OnHostStatusChanged; + + /// + /// The single host this driver instance talks to, named by its configured + /// . Mirrors ModbusDriver.HostName's + /// host:port convention: several MTConnect drivers in one server disambiguate on the + /// Admin dashboard by endpoint rather than by driver-instance id. + /// + public string HostName => _options.AgentUri; + + /// + /// + /// One entry, always — an Agent is a single host, and the driver knows of it whether or not + /// it has been reached. Before the first probe tick (and for the whole life of an instance + /// with Probe.Enabled = false) that entry reads , which + /// is the truth: nothing has measured it. See for why a successful + /// initialize does not seed it . + /// + public IReadOnlyList GetHostStatuses() + { + lock (_probeLock) + { + return [new HostConnectivityStatus(HostName, _hostState, _hostStateChangedUtc)]; + } + } + + /// + /// Starts the periodic connectivity probe, if the operator asked for one. Caller must hold + /// . + /// + /// The client this session owns; the loop dials that one, never a later one. + /// The options this session started with. + private void StartProbeLoopCore(IMTConnectAgentClient client, MTConnectDriverOptions options) + { + if (!options.Probe.Enabled) + { + return; + } + + var cts = new CancellationTokenSource(); + _probeCts = cts; + Volatile.Write(ref _probeTask, Task.Run(() => RunProbeLoopAsync(client, options, cts.Token), CancellationToken.None)); + } + + /// + /// The connectivity probe: wait, ask the Agent whether it is answering, publish the verdict, + /// repeat. Never throws — it is a detached background task, and an escaping exception would + /// silently end host reporting while kept serving the last + /// thing it saw. + /// + /// + /// + /// It calls directly and publishes + /// nothing. Both halves of that are deliberate. + /// + /// + /// Routing it through is the obvious shortcut and + /// it is inert: that accessor answers from the cache + /// already warmed, so the loop would issue no request at all and report + /// forever no matter what the Agent was doing — a feature + /// that ticks, logs, and measures nothing. + /// + /// + /// Publishing the answer into is the opposite + /// mistake: it would put a second writer on a field the lifecycle owns under a + /// compare-and-swap, and would silently re-warm a cache that + /// — or an Agent restart, see + /// — deliberately dropped. The parsed model is + /// discarded; only the fact that a request completed is used. + /// + /// + /// Why /probe rather than the /sample heartbeat. The heartbeat is + /// free, but it only exists while something is subscribed — an instance with no + /// subscriptions would report indefinitely, and host + /// reachability would become a function of OPC UA client behaviour. A dedicated tick is + /// the one signal that is true whether or not anyone is watching. Its cost is a parse of + /// the device model per interval, which is why + /// exists. + /// + /// + /// It waits first, then probes. has just completed a + /// successful /probe one moment earlier; a tick that fired immediately would spend + /// a second identical request to learn nothing, and would do it on every re-initialize. + /// + /// + /// It never calls a lifecycle method — see the remarks. + /// awaits this task while holding that semaphore, so + /// reaching back into one would be a permanent hang with no exception and no log line. + /// + /// + private async Task RunProbeLoopAsync( + IMTConnectAgentClient client, MTConnectDriverOptions options, CancellationToken ct) + { + try + { + while (!ct.IsCancellationRequested) + { + try + { + await _probeScheduler(options.Probe.Interval, ct).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + return; + } + + if (ct.IsCancellationRequested) + { + return; + } + + bool reachable; + try + { + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(ct); + deadline.CancelAfter(options.Probe.Timeout); + + // The answer is deliberately discarded — see the remarks. + _ = await client.ProbeAsync(deadline.Token).ConfigureAwait(false); + reachable = true; + } + catch (OperationCanceledException) when (ct.IsCancellationRequested) + { + return; + } + catch (Exception ex) + { + // Unreachable, unusable, disposed, or past its deadline — from the host's point + // of view these are one fact, and the data paths report the detail. + reachable = false; + + _logger.LogDebug( + ex, + "MTConnect driver {DriverInstanceId} could not reach {AgentUri} on its connectivity probe.", + _driverInstanceId, options.AgentUri); + } + + TransitionHostTo(reachable ? HostState.Running : HostState.Stopped); + } + } + catch (Exception ex) + { + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} stopped its connectivity probe on an unexpected fault; host status for {AgentUri} will not be updated until the driver is re-initialized.", + _driverInstanceId, options.AgentUri); + } + } + + /// + /// Publishes a host state and announces it — only when it actually changed. Raising on + /// every tick would emit one event per interval per Agent forever: Core would re-fan Bad + /// quality across the host's subtree each time, and an operator watching the feed could not + /// tell a new outage from an ongoing one. + /// + private void TransitionHostTo(HostState newState) + { + HostState old; + lock (_probeLock) + { + old = _hostState; + if (old == newState) + { + return; + } + + _hostState = newState; + _hostStateChangedUtc = DateTime.UtcNow; + } + + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} host {AgentUri} went {Previous} -> {Current}.", + _driverInstanceId, HostName, old, newState); + + // Outside the lock: a handler is caller code. + try + { + OnHostStatusChanged?.Invoke(this, new HostStatusChangedEventArgs(HostName, old, newState)); + } + catch (Exception ex) + { + _logger.LogWarning( + ex, + "MTConnect driver {DriverInstanceId} caught a fault from a host-status subscriber for {AgentUri}; the probe continues.", + _driverInstanceId, HostName); + } + } + + /// + /// Stops the connectivity probe loop and waits for it to be gone. Idempotent, and never + /// throws — it runs on teardown paths where a second exception would mask the first. + /// + /// + /// ⚠️ This runs while is held and it awaits the loop, on + /// exactly the same terms as : safe only because the loop + /// never touches the lifecycle and every await it makes observes the token cancelled below. + /// The one thing outside that guarantee is an handler that + /// blocks forever — a consumer bug this driver cannot paper over. + /// + private async Task StopProbeLoopAsync() + { + var cts = _probeCts; + var task = Volatile.Read(ref _probeTask); + _probeCts = null; + Volatile.Write(ref _probeTask, null); + + if (cts is null) + { + return; + } + + try + { + await cts.CancelAsync().ConfigureAwait(false); + } + catch (ObjectDisposedException) + { + // Raced another stop; the loop is already going away. + } + + if (task is not null) + { + try + { + await task.ConfigureAwait(false); + } + catch (Exception ex) + { + // RunProbeLoopAsync is written not to throw, so this is belt-and-braces. + _logger.LogDebug( + ex, + "MTConnect driver {DriverInstanceId} swallowed a fault from its connectivity probe while stopping it.", + _driverInstanceId); + } + } + + cts.Dispose(); + } + + // ---- IRediscoverable: the Agent instanceId watch ---- + + /// + /// + /// + /// This event is what makes = + /// safe. The runtime runs exactly one + /// discovery pass per connect, so an Agent that restarts mid-run and renumbers, adds, or + /// drops data items would otherwise leave a stale address space sitting behind a + /// perfectly Healthy driver. + /// + /// + /// Raised from the /sample pump's task, outside every lock. A handler that + /// throws is absorbed and logged — a consumer bug must not tear down the stream serving + /// every subscriber. A handler that blocks blocks the pump, and one that + /// synchronously awaits a lifecycle method on this driver deadlocks it: teardown waits + /// for the pump, and the pump is waiting inside the handler. Core's consumer schedules + /// the rediscovery rather than running it inline, which is the only safe shape. + /// + /// + public event EventHandler? OnRediscoveryNeeded; + + /// + /// Handles an Agent restart observed on the wire: drop the device model it invalidated, then + /// tell Core to re-discover — once per distinct instanceId. + /// + /// + /// + /// Dropping the cached /probe model is not housekeeping — it is what stops this + /// whole capability being inert. Core's response to the event is to re-run + /// , which reads + /// ; with the cache still warm that call answers + /// from a model describing a process that no longer exists, and the "rediscovery" would + /// diff the stale tree against itself and change nothing. Dropped through the same + /// compare-and-swap uses, so a lifecycle change + /// landing mid-drop wins rather than being undone. + /// + /// + /// Once per change, not once per chunk. Every chunk after a restart carries the new + /// id, and a re-baseline that fails leaves the pump still comparing against the old one — + /// so the guard is a latch on the id that was last announced, not on the pump's cursor + /// state. A second, different restart is a second announcement; an id the Agent reuses + /// after some other id is one too, because that is genuinely a new process. + /// + /// + /// The instanceId the Agent's chunk reported. + /// The options in force, for the log line's endpoint. + private void AnnounceAgentRestart(long instanceId, MTConnectDriverOptions options) + { + if (Interlocked.Exchange(ref _announcedInstanceId, instanceId) == instanceId) + { + return; + } + + InvalidateProbeModel(); + + _logger.LogInformation( + "MTConnect driver {DriverInstanceId} dropped its cached /probe device model for {AgentUri} and raised rediscovery: agent instanceId is now {InstanceId}.", + _driverInstanceId, options.AgentUri, instanceId); + + try + { + OnRediscoveryNeeded?.Invoke(this, new RediscoveryEventArgs("MTConnect agent instanceId changed", null)); + } + catch (Exception ex) + { + _logger.LogWarning( + ex, + "MTConnect driver {DriverInstanceId} caught a fault from a rediscovery subscriber; the /sample stream continues.", + _driverInstanceId); + } + } + + /// + /// Drops the cached /probe device model so the next consumer re-fetches it. Same + /// compare-and-swap discipline as — retried, because + /// unlike a flush this drop is a correctness requirement rather than a courtesy: losing the + /// race must not leave a model from a dead Agent installed. + /// + private void InvalidateProbeModel() + { + while (true) + { + var session = Volatile.Read(ref _session); + if (session?.ProbeModel is null) + { + return; + } + + var dropped = session with { ProbeModel = null, ProbeModelDataItemCount = 0 }; + if (ReferenceEquals(Interlocked.CompareExchange(ref _session, dropped, session), session)) + { + return; + } + } + } + /// /// Starts the shared /sample pump, if there is anything to pump and anything to pump /// from. Caller must hold . @@ -1166,6 +1581,11 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis ObservationIndex.Clear(); FanOut(SubscribedReferences(), ObservationIndex); + // The driver's own state is consistent by this point, so it is safe to hand the + // restart to arbitrary caller code — which may call straight back into + // DiscoverAsync. + AnnounceAgentRestart(chunk.InstanceId, options); + return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); } @@ -1632,6 +2052,19 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis RequirePositive(options.SampleIntervalMs, nameof(MTConnectDriverOptions.SampleIntervalMs)); RequirePositive(options.SampleCount, nameof(MTConnectDriverOptions.SampleCount)); + // Same 01/S-6 guard for Task 13's two knobs, gated on the feature that reads them. A 0 + // interval turns the "periodic" probe into an unthrottled hot loop against the Agent, and + // a 0 (or negative) timeout cancels every probe before it can be answered — pinning the + // host to Stopped forever and reporting an outage that does not exist. Checked only when + // probing is ON because a disabled probe reads neither value, and faulting a deployment + // over a field the driver never touches would buy nothing; flipping Enabled back on is a + // config edit, and ReinitializeAsync runs this same validation. + if (options.Probe.Enabled) + { + RequirePositive(options.Probe.Interval, $"{nameof(MTConnectDriverOptions.Probe)}.{nameof(MTConnectProbeOptions.Interval)}"); + RequirePositive(options.Probe.Timeout, $"{nameof(MTConnectDriverOptions.Probe)}.{nameof(MTConnectProbeOptions.Timeout)}"); + } + var client = existingClient; if (client is null) { @@ -1650,6 +2083,11 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis _options = options; built = null; + // The Agent this session talks to, as of now. A restart is measured against THIS, so a + // re-initialize against a different Agent process never re-announces the change the + // operator's own action caused. + Volatile.Write(ref _announcedInstanceId, current.InstanceId); + // One publish, so a concurrent lock-free reader sees the new client, the new probe // cache, and the pump's opening cursor together or none of them. Volatile.Write( @@ -1678,6 +2116,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis StartSampleStreamCore(republishOnStart: true); } + StartProbeLoopCore(client, options); + _logger.LogInformation( "MTConnect driver {DriverInstanceId} initialized against {AgentUri}{DeviceScope}: {DeviceCount} device(s), {ObservationCount} primed observation(s), agent instanceId {InstanceId}.", _driverInstanceId, @@ -2084,6 +2524,22 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } } + /// + /// The half of the 01/S-6 guard, for the connectivity probe's two + /// knobs. A non-positive interval is an unthrottled loop; a non-positive timeout cancels + /// every request before it can be answered. + /// + private static void RequirePositive(TimeSpan value, string optionName) + { + if (value <= TimeSpan.Zero) + { + throw new ArgumentOutOfRangeException( + nameof(MTConnectDriverOptions), + value, + $"{optionName} must be positive; a non-positive value would either spin the connectivity probe against the Agent or cancel every probe before it could be answered."); + } + } + /// Barrier-protected read of the cross-thread health snapshot. private DriverHealth ReadHealth() => Volatile.Read(ref _health); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs new file mode 100644 index 00000000..14be33e9 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs @@ -0,0 +1,629 @@ +using System.Collections.Concurrent; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 13 — 's two optional capabilities: +/// (a periodic reachability tick that flips one host +/// Running ↔ Stopped) and (the Agent instanceId watch that +/// is the ONLY thing making safe). +/// +/// +/// +/// Nothing here waits on wall-clock time, and the probe is a timer. The driver takes +/// its inter-tick delay through an injected scheduler seam, and this suite supplies +/// : the loop parks in the seam, the test releases exactly one +/// tick, and the ticker's next park is the proof that the probe which followed has already +/// completed. No test sleeps, no test polls a counter, and +/// stays timer-free. +/// +/// +/// The load-bearing tests are the anti-inertness ones. Both capabilities have a +/// plausible-looking implementation that compiles, ships, and does nothing: +/// a connectivity probe routed through the driver's cached /probe accessor never +/// touches the network once the cache is warm (so it reports Running forever, whatever the +/// Agent is doing), and a rediscovery signal raised without dropping that same cache makes +/// Core re-run discovery and receive the identical stale device tree. Both are pinned here +/// by asserting the Agent was actually called. +/// +/// +public sealed class MTConnectHostAndRediscoverTests +{ + private const string AgentUri = "http://fixture-agent:5000"; + + /// The instanceId every canned fixture shares. + private const long FixtureInstanceId = 1655000000L; + + /// A DIFFERENT instanceId — the Agent came back as a new process. + private const long RestartedInstanceId = 1655999999L; + + /// A third instanceId — the Agent restarted twice. + private const long SecondRestartInstanceId = 1656111111L; + + /// The cursor Fixtures/current.xml primes the pump with. + private const long PrimedNextSequence = 108L; + + /// + /// Failure guard for the awaits a defect could turn into a permanent hang (a probe loop that + /// never parks, a teardown that deadlocks on the lifecycle semaphore). It exists to make a + /// hang red, not to assert a duration — no assertion is ever made about elapsed time. + /// + private static readonly TimeSpan Watchdog = TimeSpan.FromSeconds(10); + + private static readonly DateTime ObservedAt = new(2026, 7, 24, 12, 30, 0, DateTimeKind.Utc); + + /// The interval the probe tests author, asserted to reach the scheduler verbatim. + private static readonly TimeSpan AuthoredInterval = TimeSpan.FromSeconds(7); + + private static CancellationToken Ct => TestContext.Current.CancellationToken; + + // ---- fixtures ---- + + /// + /// Options with the background probe disabled — the default for the rediscovery half + /// of this suite, so no probe loop can add calls behind a ProbeCallCount assertion. + /// + private static MTConnectDriverOptions Opts(MTConnectProbeOptions? probe = null) => + new() + { + AgentUri = AgentUri, + RequestTimeoutMs = 5000, + Probe = probe ?? new MTConnectProbeOptions { Enabled = false }, + Tags = + [ + new MTConnectTagDefinition("dev1_pos", DriverDataType.Float64), + new MTConnectTagDefinition("dev1_execution", DriverDataType.String), + ], + }; + + private static MTConnectProbeOptions Probing(bool enabled = true) => + new() { Enabled = enabled, Interval = AuthoredInterval, Timeout = TimeSpan.FromSeconds(2) }; + + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( + CannedAgentClient? client = null, + MTConnectDriverOptions? options = null, + ManualTicker? ticker = null) + { + var agent = client ?? CannedAgentClient.FromFixtures(); + var driver = new MTConnectDriver( + options ?? Opts(), "mt1", _ => agent, probeScheduler: ticker is null ? null : ticker.WaitAsync); + + await driver.InitializeAsync("{}", Ct); + + return (driver, agent); + } + + /// + /// Brings a driver up with the probe loop wired to a manual ticker, and returns once the loop + /// is demonstrably parked in its first inter-tick wait (i.e. it has run zero probes). + /// + private static async Task<(MTConnectDriver Driver, CannedAgentClient Client, ManualTicker Ticker)> + ProbingDriverAsync(CannedAgentClient? client = null) + { + var ticker = new ManualTicker(); + var (driver, agent) = await InitializedDriverAsync(client, Opts(Probing()), ticker); + + await ticker.ParkedAsync().WaitAsync(Watchdog, Ct); + + return (driver, agent, ticker); + } + + /// + /// Releases exactly one probe tick and returns once the probe it triggered has completed — + /// proven by the loop parking again, not by a delay. + /// + private static async Task TickAsync(ManualTicker ticker) + { + ticker.Tick(); + await ticker.ParkedAsync().WaitAsync(Watchdog, Ct); + } + + private static ConcurrentQueue RecordRediscovery(MTConnectDriver driver) + { + var seen = new ConcurrentQueue(); + ((IRediscoverable)driver).OnRediscoveryNeeded += (_, e) => seen.Enqueue(e); + + return seen; + } + + private static ConcurrentQueue RecordHostStatus(MTConnectDriver driver) + { + var seen = new ConcurrentQueue(); + ((IHostConnectivityProbe)driver).OnHostStatusChanged += (_, e) => seen.Enqueue(e); + + return seen; + } + + private static MTConnectStreamsResult ChunkFrom( + long instanceId, long firstSequence, long nextSequence, params (string Id, string Value)[] observations) => + new( + instanceId, + nextSequence, + firstSequence, + [.. observations.Select(o => new MTConnectObservation(o.Id, o.Value, ObservedAt))]); + + /// A device model that is recognisably NOT the canned fixture's. + private static MTConnectProbeModel OtherModel() => + new([ + new MTConnectDevice( + "other-device", + "other-device", + [], + [new MTConnectDataItem("other_item", "other_item", "EVENT", "AVAILABILITY", null, null, null, null)]), + ]); + + // ---- IRediscoverable: the instanceId watch ---- + + /// + /// The plan's TDD case. is + /// , so the runtime runs exactly one discovery + /// pass per connect — an Agent that restarts and renumbers (or adds) data items would leave a + /// stale address space behind a Healthy driver. This event is the only thing that makes that + /// policy safe. + /// + [Fact] + public async Task InstanceId_change_raises_rediscovery() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = RecordRediscovery(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + client.CurrentAnswers.Enqueue(ChunkFrom(RestartedInstanceId, 40, 42, ("dev1_pos", "9.5"))); + + await client + .PumpAsync(ChunkFrom(RestartedInstanceId, 5000, 5005, ("dev1_pos", "5.5"))) + .WaitAsync(Watchdog, Ct); + + var raised = seen.ShouldHaveSingleItem(); + raised.Reason.ShouldBe("MTConnect agent instanceId changed"); + raised.ScopeHint.ShouldBeNull(); + } + + /// + /// Once per change, not once per chunk. The chunks that follow a restart all carry the + /// new instanceId; a driver that compared each chunk against the id it was + /// initialized with — or that simply re-raised on every chunk — would ask Core to + /// rebuild the whole address space on every heartbeat, forever. + /// + [Fact] + public async Task InstanceId_change_raises_rediscovery_once_per_change_not_once_per_chunk() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = RecordRediscovery(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + client.CurrentAnswers.Enqueue(ChunkFrom(RestartedInstanceId, 40, 42, ("dev1_pos", "9.5"))); + + await client + .PumpAsync(ChunkFrom(RestartedInstanceId, 5000, 5005, ("dev1_pos", "5.5"))) + .WaitAsync(Watchdog, Ct); + seen.Count.ShouldBe(1); + + // Two more chunks from the SAME restarted Agent, contiguous with the re-baseline. + await client.PumpAsync(ChunkFrom(RestartedInstanceId, 42, 45, ("dev1_pos", "10.5"))).WaitAsync(Watchdog, Ct); + await client.PumpAsync(ChunkFrom(RestartedInstanceId, 45, 48, ("dev1_pos", "11.5"))).WaitAsync(Watchdog, Ct); + + seen.Count.ShouldBe(1); + + // …but a SECOND restart is a second change, and must be announced. + client.CurrentAnswers.Enqueue(ChunkFrom(SecondRestartInstanceId, 10, 12, ("dev1_pos", "1.5"))); + await client + .PumpAsync(ChunkFrom(SecondRestartInstanceId, 6000, 6005, ("dev1_pos", "2.5"))) + .WaitAsync(Watchdog, Ct); + + seen.Count.ShouldBe(2); + } + + /// + /// An unchanged instanceId is the overwhelmingly common case — every ordinary chunk, + /// every heartbeat, and every ring-buffer re-baseline of a still-running Agent. None of them + /// may cost an address-space rebuild. + /// + [Fact] + public async Task Unchanged_instanceId_raises_no_rediscovery() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = RecordRediscovery(driver); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await client + .PumpAsync(ChunkFrom(FixtureInstanceId, PrimedNextSequence, 113, ("dev1_pos", "1.5"))) + .WaitAsync(Watchdog, Ct); + + // A heartbeat… + await client.PumpAsync(ChunkFrom(FixtureInstanceId, 1, 120)).WaitAsync(Watchdog, Ct); + + // …and a ring-buffer gap, which re-baselines but is NOT a restart. + client.CurrentAnswers.Enqueue(ChunkFrom(FixtureInstanceId, 5000, 5005, ("dev1_pos", "2.5"))); + await client + .PumpAsync(ChunkFrom(FixtureInstanceId, 5000, 5005, ("dev1_pos", "2.5"))) + .WaitAsync(Watchdog, Ct); + + seen.ShouldBeEmpty(); + } + + /// + /// The anti-inertness pin for . A restart invalidates the + /// cached /probe device model — it describes a process that no longer exists. Raising + /// the event while keeping that cache would have Core dutifully re-run + /// and receive the identical stale tree + /// from GetOrFetchProbeModelAsync's warm cache: a feature that fires, logs, and + /// changes nothing. + /// + [Fact] + public async Task InstanceId_change_drops_the_cached_probe_model_so_rediscovery_sees_the_new_one() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + driver.CachedProbeModel.ShouldNotBeNull(); + + // The restarted Agent declares a different device model. + client.Probe = OtherModel(); + client.CurrentAnswers.Enqueue(ChunkFrom(RestartedInstanceId, 40, 42, ("dev1_pos", "9.5"))); + + await client + .PumpAsync(ChunkFrom(RestartedInstanceId, 5000, 5005, ("dev1_pos", "5.5"))) + .WaitAsync(Watchdog, Ct); + + driver.CachedProbeModel.ShouldBeNull(); + + var probeCalls = client.ProbeCallCount; + var builder = new CapturingBuilder(); + await driver.DiscoverAsync(builder, Ct); + + // Discovery went back to the Agent, and streamed the NEW model rather than the old one. + client.ProbeCallCount.ShouldBe(probeCalls + 1); + builder.Variables.Select(v => v.Attr.FullName).ShouldBe(["other_item"]); + } + + /// + /// A rediscovery handler is arbitrary caller code. One that throws is a bug in the consumer, + /// not a reason to stop streaming data to every subscriber on this driver. + /// + [Fact] + public async Task A_throwing_rediscovery_handler_does_not_kill_the_pump() + { + var (driver, client) = await InitializedDriverAsync(); + var seen = new ConcurrentQueue(); + driver.OnDataChange += (_, e) => seen.Enqueue(e); + ((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => + throw new InvalidOperationException("rediscovery subscriber blew up"); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + client.CurrentAnswers.Enqueue(ChunkFrom(RestartedInstanceId, 40, 42, ("dev1_pos", "9.5"))); + + await client + .PumpAsync(ChunkFrom(RestartedInstanceId, 5000, 5005, ("dev1_pos", "5.5"))) + .WaitAsync(Watchdog, Ct); + + // The pump survived the throw, re-baselined, and keeps delivering. + await client.PumpAsync(ChunkFrom(RestartedInstanceId, 42, 45, ("dev1_pos", "12.5"))).WaitAsync(Watchdog, Ct); + + driver.SampleStreamTask!.IsCompleted.ShouldBeFalse(); + seen.Where(e => e.FullReference == "dev1_pos").Last().Snapshot.Value.ShouldBe(12.5d); + } + + // ---- IHostConnectivityProbe ---- + + /// + /// One host per Agent, named by the authored AgentUri — that is the identity the Admin + /// dashboard shows and the key Core scopes its Bad-quality fan-out by. Before the first tick + /// the honest answer is : the probe loop is the sole writer of + /// this field, and it has not run yet. + /// + [Fact] + public async Task Host_statuses_report_one_host_named_by_the_agent_uri() + { + var (driver, _, _) = await ProbingDriverAsync(); + + driver.HostName.ShouldBe(AgentUri); + + var status = ((IHostConnectivityProbe)driver).GetHostStatuses().ShouldHaveSingleItem(); + status.HostName.ShouldBe(AgentUri); + status.State.ShouldBe(HostState.Unknown); + } + + /// + /// The anti-inertness pin for . Every tick must + /// reach the Agent. A probe routed through the driver's cached-model accessor + /// (GetOrFetchProbeModelAsync) compiles, ships, and — with a cache warmed by + /// initialize — never issues a single request, reporting Running forever regardless of what + /// the Agent is doing. The call count is the only thing that can tell the two apart. + /// + [Fact] + public async Task Probe_loop_calls_the_agent_on_every_tick() + { + var (_, client, ticker) = await ProbingDriverAsync(); + + // Parked before the first tick: the loop waits, THEN probes, so initialize's own /probe is + // the only call so far. + client.ProbeCallCount.ShouldBe(1); + + await TickAsync(ticker); + client.ProbeCallCount.ShouldBe(2); + + await TickAsync(ticker); + client.ProbeCallCount.ShouldBe(3); + + await TickAsync(ticker); + client.ProbeCallCount.ShouldBe(4); + + // …and it waits the authored interval between them, rather than a hard-coded cadence. + ticker.LastInterval.ShouldBe(AuthoredInterval); + } + + /// + /// Running ↔ Stopped, and the event fires on the transition only. A driver that raised + /// on every tick would emit one event per interval per Agent forever; Core would re-fan Bad + /// quality across the host's subtree each time, and an operator watching the feed could not + /// tell a new outage from an ongoing one. + /// + [Fact] + public async Task Probe_flips_the_host_on_transition_only() + { + var (driver, client, ticker) = await ProbingDriverAsync(); + var seen = RecordHostStatus(driver); + + client.ProbeFailure = new HttpRequestException("agent unreachable"); + + await TickAsync(ticker); + await TickAsync(ticker); + + var down = seen.ShouldHaveSingleItem(); + down.HostName.ShouldBe(AgentUri); + down.OldState.ShouldBe(HostState.Unknown); + down.NewState.ShouldBe(HostState.Stopped); + ((IHostConnectivityProbe)driver).GetHostStatuses()[0].State.ShouldBe(HostState.Stopped); + + client.ProbeFailure = null; + + await TickAsync(ticker); + await TickAsync(ticker); + + seen.Count.ShouldBe(2); + var up = seen.Last(); + up.OldState.ShouldBe(HostState.Stopped); + up.NewState.ShouldBe(HostState.Running); + ((IHostConnectivityProbe)driver).GetHostStatuses()[0].State.ShouldBe(HostState.Running); + } + + /// + /// The probe is a liveness check, never a cache writer. Publishing its answer into + /// AgentSession.ProbeModel would put a second writer on the field the lifecycle owns + /// under a compare-and-swap, and would silently re-warm a cache that + /// FlushOptionalCachesAsync (or an Agent restart) deliberately dropped. + /// + [Fact] + public async Task Probe_ticks_never_publish_into_the_session_probe_cache() + { + var (driver, client, ticker) = await ProbingDriverAsync(); + var original = driver.CachedProbeModel.ShouldNotBeNull(); + + // The Agent now answers /probe with a different model. Nothing the connectivity probe does + // may let that model reach the driver's cache. + client.Probe = OtherModel(); + + await TickAsync(ticker); + await TickAsync(ticker); + + driver.CachedProbeModel.ShouldBeSameAs(original); + driver.GetMemoryFootprint().ShouldBeGreaterThan(0); + + // …and a flushed cache stays flushed until a real consumer asks for it. + await driver.FlushOptionalCachesAsync(Ct); + await TickAsync(ticker); + driver.CachedProbeModel.ShouldBeNull(); + } + + /// + /// Host connectivity is deliberately NOT an input to DriverHealth. The two real + /// data paths (/current reads, the /sample pump) own health and will report the + /// same outage with the failure that actually mattered. The probe's deadline + /// (Probe.Timeout, 2 s) is far tighter than RequestTimeoutMs and its request is + /// the largest document the Agent serves, so letting it degrade the driver would flap + /// a perfectly working instance on a slow-but-alive Agent. + /// + [Fact] + public async Task An_unreachable_host_does_not_degrade_driver_health() + { + var (driver, client, ticker) = await ProbingDriverAsync(); + client.ProbeFailure = new HttpRequestException("agent unreachable"); + + await TickAsync(ticker); + + ((IHostConnectivityProbe)driver).GetHostStatuses()[0].State.ShouldBe(HostState.Stopped); + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + + // …and the converse: a reachable host does not clear a genuinely failing read path either. + client.ProbeFailure = null; + client.CurrentFailure = new HttpRequestException("connection refused"); + await driver.ReadAsync(["dev1_pos"], Ct); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + + await TickAsync(ticker); + + ((IHostConnectivityProbe)driver).GetHostStatuses()[0].State.ShouldBe(HostState.Running); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + } + + /// + /// Probe.Enabled = false means no loop, no scheduler wait, and not one extra request. + /// An operator who turned probing off must not still be paying for it. + /// + [Fact] + public async Task Probe_disabled_starts_no_loop_and_makes_no_calls() + { + var ticker = new ManualTicker(); + var (driver, client) = await InitializedDriverAsync(options: Opts(Probing(enabled: false)), ticker: ticker); + + driver.ProbeLoopTask.ShouldBeNull(); + ticker.Waits.ShouldBe(0); + client.ProbeCallCount.ShouldBe(1); // the initialize prime, and nothing else + + // The capability is still implemented — it just has nothing to report. + ((IHostConnectivityProbe)driver).GetHostStatuses() + .ShouldHaveSingleItem().State.ShouldBe(HostState.Unknown); + } + + /// + /// Teardown stops the timer and waits for it. A loop left running would keep dialling an + /// Agent through a client its owner has already disposed — the "torn down but still + /// reporting" shape the rest of this driver's lifecycle is written to prevent. + /// + [Fact] + public async Task Shutdown_stops_the_probe_loop_and_makes_no_further_calls() + { + var (driver, client, ticker) = await ProbingDriverAsync(); + await TickAsync(ticker); + var loop = driver.ProbeLoopTask.ShouldNotBeNull(); + + await driver.ShutdownAsync(Ct).WaitAsync(Watchdog, Ct); + + loop.IsCompleted.ShouldBeTrue(); + loop.IsFaulted.ShouldBeFalse(); + driver.ProbeLoopTask.ShouldBeNull(); + client.IsDisposed.ShouldBeTrue(); + + // The loop is provably gone (ShutdownAsync awaited it), so releasing another tick can reach + // nobody — no probe against a disposed client, and no further waits on the scheduler. + var calls = client.ProbeCallCount; + var waits = ticker.Waits; + ticker.Tick(); + + client.ProbeCallCount.ShouldBe(calls); + ticker.Waits.ShouldBe(waits); + } + + /// + /// A re-initialize replaces the client, so it must replace the loop that dials it — the old + /// one holds a reference to a client the re-initialize may be about to dispose. + /// + [Fact] + public async Task Reinitialize_replaces_the_probe_loop() + { + var (driver, _, ticker) = await ProbingDriverAsync(); + var first = driver.ProbeLoopTask.ShouldNotBeNull(); + + await driver.ReinitializeAsync("{}", Ct).WaitAsync(Watchdog, Ct); + + first.IsCompleted.ShouldBeTrue(); + var second = driver.ProbeLoopTask.ShouldNotBeNull(); + second.ShouldNotBeSameAs(first); + + // The replacement loop is live: it parks, ticks, and probes like the first. + await ticker.ParkedAsync().WaitAsync(Watchdog, Ct); + await TickAsync(ticker); + } + + // ---- operator-authorable zeroes (arch-review 01/S-6) ---- + + /// + /// arch-review 01/S-6, applied to the two knobs Task 13 is the first consumer of. A + /// 0 interval turns the "periodic" probe into an unthrottled hot loop against the + /// Agent; a 0 timeout cancels every probe before it can be answered, pinning the host + /// to Stopped forever and reporting an outage that does not exist. Both are refused with the + /// same RequirePositive posture as the four timing knobs Task 9 guarded. + /// + [Theory] + [InlineData(0, 2000)] + [InlineData(-1, 2000)] + [InlineData(5000, 0)] + [InlineData(5000, -1)] + public async Task A_non_positive_probe_interval_or_timeout_is_refused(int intervalMs, int timeoutMs) + { + var options = Opts(new MTConnectProbeOptions + { + Enabled = true, + Interval = TimeSpan.FromMilliseconds(intervalMs), + Timeout = TimeSpan.FromMilliseconds(timeoutMs), + }); + + var driver = new MTConnectDriver(options, "mt1", _ => CannedAgentClient.FromFixtures()); + + await Should.ThrowAsync(() => driver.InitializeAsync("{}", Ct)); + driver.GetHealth().State.ShouldBe(DriverState.Faulted); + } + + /// + /// …but only when probing is on. Faulting a deployment over a field the driver never reads + /// would be a new way for a deploy to fail with nothing gained; flipping + /// Enabled back on is a config edit that re-runs the same validation. + /// + [Fact] + public async Task A_non_positive_probe_interval_is_ignored_when_probing_is_disabled() + { + var options = Opts(new MTConnectProbeOptions + { + Enabled = false, Interval = TimeSpan.Zero, Timeout = TimeSpan.Zero, + }); + + var (driver, _) = await InitializedDriverAsync(options: options); + + driver.GetHealth().State.ShouldBe(DriverState.Healthy); + driver.ProbeLoopTask.ShouldBeNull(); + } + + /// + /// A hand-driven replacement for : the + /// probe loop parks here, and a test releases exactly one tick at a time. The park signal is + /// what makes the suite deterministic — when the loop is parked again, the probe that + /// followed the previous release has demonstrably finished. + /// + /// + /// installs the next park signal before releasing the + /// current wait, so a loop that races ahead and parks again immediately cannot lose the + /// signal a test is about to await. + /// + private sealed class ManualTicker + { + private readonly Lock _gate = new(); + private TaskCompletionSource _parked = new(TaskCreationOptions.RunContinuationsAsynchronously); + private TaskCompletionSource _release = new(TaskCreationOptions.RunContinuationsAsynchronously); + private int _waits; + + /// How many times the loop has parked in this scheduler. + public int Waits => Volatile.Read(ref _waits); + + /// The interval the loop most recently asked to wait for. + public TimeSpan LastInterval { get; private set; } + + /// The scheduler seam handed to the driver. + public async Task WaitAsync(TimeSpan interval, CancellationToken ct) + { + TaskCompletionSource release; + lock (_gate) + { + LastInterval = interval; + Interlocked.Increment(ref _waits); + release = _release; + _parked.TrySetResult(); + } + + await release.Task.WaitAsync(ct).ConfigureAwait(false); + } + + /// Completes once the loop is parked in the current wait. + public Task ParkedAsync() + { + lock (_gate) + { + return _parked.Task; + } + } + + /// Releases the current wait so exactly one probe runs. + public void Tick() + { + lock (_gate) + { + var release = _release; + _release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + _parked = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + release.TrySetResult(); + } + } + } +} -- 2.52.0 From dfbcb0e7f0519426a38499965d80825fa90146d1 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:16:13 -0400 Subject: [PATCH 21/34] fix(mtconnect): close the silent-refusal + teardown-wedge defects in the pump (Task 11 review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit C1 (critical): a latched "this endpoint cannot stream" verdict could end with a GREEN driver holding a permanently silent subscription. DriverInstanceActor handles a tag-set change as Unsubscribe-then-Subscribe with no re-initialize; the unsubscribe cleared the stream-degraded flag, the resubscribe was refused by the latch without a word, and one successful read then reported Healthy. StartSampleStreamCore now re-asserts the degradation and logs a Warning naming the remedy, and teardown no longer clears the flag while the latch is set. NOT promoted to Faulted: DriverHealthReport maps any Faulted driver to a /readyz 503, so that would de-ready the whole node over one misconfigured Agent whose reads are perfectly healthy. Faulted stays for the case that earns it. I1: teardown waits are bounded (5 s) and honour the caller's token. They were unbounded and ignored it, so DriverInstanceActor's 5 s budget was inert and PostStop — which blocks a dispatcher thread on ShutdownAsync while holding the lifecycle semaphore — could be wedged forever by one blocking subscriber. The cancellation source is disposed only when the loop is provably gone. Applied to StopProbeLoopAsync too: Task 13 reproduced the identical shape, and closing the class in one method while leaving the other as the pattern to copy is worse than not closing it. I2: the reconnect ladder was monotonic for the process lifetime, so ~9 unrelated drops pinned the driver at MaxBackoffMs forever. A stream that delivered before dropping now starts a fresh ladder (at attempt 1, not 0, so MinBackoffMs is still honoured). I3: MaxBackoffMs <= 0 clamped every delay to zero — an operator-authorable reconnect spin. Treated as unset, like a multiplier that cannot grow. I4: the session published instanceId without NextSequence, so after a restart it held the new agent's id beside the dead one's cursor, and a gap/OUT_OF_RANGE re-baseline (id unchanged) never published the cursor at all. Both move in one CAS, and the pump publishes its cursor as it exits. I5: OnDataChange was a multicast Invoke — the first throwing handler aborted the rest of the list, starving every later subscriber. Now walked by hand with the catch inside the loop. The test that claimed to cover this registered the recorder BEFORE the thrower, so it passed regardless; swapped, it was red. Faults are latched to one Warning per stream generation (Debug thereafter) so a consistently-throwing consumer cannot flood the log. Also: the pump restarts after an unexpected fault (IsCompleted, not null); a device-scope that matches nothing is a Warning, not a Debug tally; a device or component with neither name nor id is skipped rather than emitting a blank path segment; DiscoverAsync's remark no longer claims DriverInstanceActor retries (it does not for a Once driver, and injection is dormant in v3); and two flake seeds in the fake are gone (Dispose re-read its own counter; _disposeCts was disposed under a racing SampleAsync). 439/439. Every changed behaviour falsified by mutation. --- .../MTConnectDriver.cs | 442 +++++++++++++++--- .../CannedAgentClient.cs | 69 ++- .../MTConnectDiscoverTests.cs | 104 ++++- .../MTConnectHostAndRediscoverTests.cs | 47 ++ .../MTConnectSubscribeTests.cs | 277 ++++++++++- .../RecordingDriverLogger.cs | 65 +++ 6 files changed, 926 insertions(+), 78 deletions(-) create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/RecordingDriverLogger.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index 7a129af8..6b6c4f77 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -103,6 +103,31 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// Growth factor used when the authored multiplier could not grow the delay (≤ 1). private const double DefaultBackoffMultiplier = 2.0; + /// + /// Backoff cap used when is authored + /// non-positive. Treated as "unset" rather than as "no cap" for the same reason a + /// multiplier ≤ 1 falls back to : a zero cap clamps + /// EVERY delay to zero, which turns the reconnect loop into a spin against a refused + /// connection — one Warning per iteration, as fast as the socket can fail. Mirrors the + /// shipped default. + /// + private const int DefaultMaxBackoffMs = 30_000; + + /// + /// How long teardown waits for a background loop (the /sample pump, the connectivity + /// probe) to unwind before abandoning the wait and carrying on. + /// + /// + /// An unbounded wait here is not merely slow — it wedges an actor-system thread. + /// DriverInstanceActor.PostStop calls with + /// GetAwaiter().GetResult(), i.e. a synchronous block on an Akka dispatcher thread, + /// while this driver holds . So a subscriber callback that never + /// returns would take the dispatcher thread AND every subsequent lifecycle call on this + /// driver with it, permanently. Five seconds matches the budget + /// DriverInstanceActor already puts around . + /// + private static readonly TimeSpan TeardownWait = TimeSpan.FromSeconds(5); + /// /// Config-JSON reader options, mirroring the sibling driver factories. Note there is /// deliberately no JsonStringEnumConverter: enum-carrying DTO fields stay @@ -265,6 +290,21 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// Whether the /current read path is currently failing. See . private volatile bool _readDegraded; + /// + /// Consecutive /sample reconnect attempts that have not yet been vindicated by + /// a delivered chunk — the input to . Reset by any stream that + /// actually delivered something, so an agent that drops a connection once an hour is never + /// treated as one that has been failing all day. + /// + private volatile int _reconnectAttempts; + + /// + /// Latch behind the once-per-stream-generation subscriber-fault Warning. A consumer that + /// throws on every value would otherwise emit one Warning per observation per handle + /// — thousands a second on a busy Agent, which buries the very log it is trying to raise. + /// + private int _subscriberFaultLogged; + /// /// The live agent client plus the /probe cache derived from it, held as one /// immutable snapshot behind a single reference and only ever replaced by a @@ -376,6 +416,21 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// internal long? AgentInstanceId => Volatile.Read(ref _session)?.InstanceId; + /// + /// The sequence the next /sample stream will open at — the priming /current's + /// nextSequence, advanced by every re-baseline and by the pump as it stops. Paired + /// with in one session record, so the two are never observed + /// out of step. + /// + internal long? AgentNextSequence => Volatile.Read(ref _session)?.NextSequence; + + /// + /// Consecutive /sample reconnect attempts not yet vindicated by a delivered chunk — + /// the input to , exposed so the reset policy can be asserted + /// without measuring how long anything slept. + /// + internal int ReconnectAttempts => _reconnectAttempts; + /// /// The shared /sample pump's task while one is running, else null. Exposed to /// tests as the deterministic teardown barrier: "the pump has stopped" is a task completion, @@ -428,8 +483,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // ReinitializeAsync and ShutdownAsync use, and for the same reason: a pump left // enumerating a disposed client reads the disposal as a dropped connection and // reconnects against an object that no longer exists. - await StopSampleStreamAsync().ConfigureAwait(false); - await StopProbeLoopAsync().ConfigureAwait(false); + await StopSampleStreamAsync(cancellationToken).ConfigureAwait(false); + await StopProbeLoopAsync(cancellationToken).ConfigureAwait(false); await TeardownCoreAsync().ConfigureAwait(false); await StartCoreAsync(options, existingClient: null, cancellationToken).ConfigureAwait(false); @@ -473,8 +528,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis var existing = Volatile.Read(ref _session)?.Client; - await StopSampleStreamAsync().ConfigureAwait(false); - await StopProbeLoopAsync().ConfigureAwait(false); + await StopSampleStreamAsync(cancellationToken).ConfigureAwait(false); + await StopProbeLoopAsync(cancellationToken).ConfigureAwait(false); if (existing is null || RequiresClientRebuild(_options, incoming)) { @@ -506,8 +561,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis { var lastRead = ReadHealth().LastSuccessfulRead; - await StopSampleStreamAsync().ConfigureAwait(false); - await StopProbeLoopAsync().ConfigureAwait(false); + await StopSampleStreamAsync(cancellationToken).ConfigureAwait(false); + await StopProbeLoopAsync(cancellationToken).ConfigureAwait(false); await TeardownCoreAsync().ConfigureAwait(false); WriteHealth(new DriverHealth(DriverState.Unknown, lastRead, null)); @@ -797,7 +852,9 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis { // Outside the lock: StopSampleStreamAsync awaits the pump, and the pump takes this same // lock to read the subscription registry. Holding it here would deadlock the two. - await StopSampleStreamAsync().ConfigureAwait(false); + // The caller's token bounds the wait: DriverInstanceActor puts a 5 s budget around this + // call, and until now that budget was inert. + await StopSampleStreamAsync(cancellationToken).ConfigureAwait(false); } _logger.LogDebug( @@ -886,9 +943,22 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// discovery stream has no such channel, so the only way to "fail quietly" would be to /// emit an empty tree — indistinguishable from an Agent that genuinely declares nothing, /// and read by an operator in the picker as a working connection to an empty machine - /// (#485: empty is not an answer). Both production callers handle the throw: the - /// universal browser surfaces it as a failed browse, and DriverInstanceActor logs - /// it and retries on its rediscover tick. + /// (#485: empty is not an answer). + /// + /// + /// What the callers actually do with the throw — worth stating precisely, because + /// the reassuring version ("it is logged and retried") is not true here. The live + /// consumer is the /raw browse-commit path + /// (DiscoveryDriverBrowser/BrowserSessionService), which surfaces it to the + /// operator as a failed browse — that is the case this posture is chosen for. + /// DriverInstanceActor.HandleRediscoverAsync catches it, logs a Warning, and + /// publishes an EMPTY node set; it does not reschedule, because retrying is a + /// behaviour and this driver is + /// . And that publish currently reaches + /// nothing: DriverHostActor.HandleDiscoveredNodes short-circuits unconditionally + /// (discovered-node injection is dormant in v3 — raw tags are authored through + /// browse-commit instead), so at that seam the throw-versus-empty choice has no runtime + /// effect today. It is made for the browse path, and for whatever re-enables injection. /// /// /// Nothing here writes the observation index. Discovery is a pure read of the @@ -920,11 +990,37 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis continue; } - devices++; var name = FolderName(device.Name, device.Id); + if (string.IsNullOrWhiteSpace(name)) + { + // Neither a name nor an id: there is no browse path that could identify this device, + // and folding it in would emit a blank tree segment (and a blank NodeId component + // downstream). Its data items are unreachable either way. + _logger.LogWarning( + "MTConnect driver {DriverInstanceId} skipped a device in {AgentUri}'s /probe model that declares neither a name nor an id; it cannot be given a browse path.", + _driverInstanceId, _options.AgentUri); + + continue; + } + + devices++; variables += StreamContainer(device, builder.Folder(name, name)); } + // A configured scope that matches NO device is an authoring error — almost always a typo or + // a device renamed in the Agent — and its only symptom is a browse that succeeds and shows + // nothing. Reported at Warning, because at Debug the operator's evidence is an empty picker + // and no explanation. An agent-wide browse that finds nothing is a different statement (the + // Agent declares no devices) and stays a Debug tally. + if (devices == 0 && !string.IsNullOrEmpty(deviceScope)) + { + _logger.LogWarning( + "MTConnect driver {DriverInstanceId} is scoped to device '{DeviceScope}', but {AgentUri}'s /probe model declares no device with that name or id ({DeclaredCount} declared). The browse is empty because the scope matched nothing, not because the Agent has nothing.", + _driverInstanceId, deviceScope, _options.AgentUri, model.Devices.Count); + + return; + } + _logger.LogDebug( "MTConnect driver {DriverInstanceId} streamed {DeviceCount} device(s) and {VariableCount} data item(s) from {AgentUri}'s /probe model{DeviceScope}.", _driverInstanceId, @@ -997,7 +1093,18 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis continue; } + // Guarded on the RESOLVED folder name, not on the id alone: a component with a blank id + // but a real name browses perfectly well (the id is only the fallback), while one with + // neither would open a folder named "" — a blank segment in every browse path beneath + // it. The data-item guard above is stricter for a different reason: THAT id is the + // correlation key an observation is resolved by, so a blank one is unreadable whatever + // it is called. var name = FolderName(component.Name, component.Id); + if (string.IsNullOrWhiteSpace(name)) + { + continue; + } + streamed += StreamContainer(component, scope.Folder(name, name)); } @@ -1244,10 +1351,12 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// ⚠️ This runs while is held and it awaits the loop, on /// exactly the same terms as : safe only because the loop /// never touches the lifecycle and every await it makes observes the token cancelled below. - /// The one thing outside that guarantee is an handler that - /// blocks forever — a consumer bug this driver cannot paper over. + /// The wait is bounded by and the caller's token for the same + /// reason: an handler that never returns would otherwise + /// wedge the Akka dispatcher thread DriverInstanceActor.PostStop blocks on. /// - private async Task StopProbeLoopAsync() + /// The caller's deadline for the teardown wait. + private async Task StopProbeLoopAsync(CancellationToken cancellationToken = default) { var cts = _probeCts; var task = Volatile.Read(ref _probeTask); @@ -1268,11 +1377,21 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // Raced another stop; the loop is already going away. } + var stopped = true; if (task is not null) { try { - await task.ConfigureAwait(false); + await task.WaitAsync(TeardownWait, cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) when (ex is TimeoutException or OperationCanceledException) + { + stopped = false; + + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} gave up waiting for its connectivity probe to stop after {TeardownWait}; the usual cause is an OnHostStatusChanged subscriber that blocks. Teardown continues.", + _driverInstanceId, TeardownWait); } catch (Exception ex) { @@ -1284,7 +1403,11 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } } - cts.Dispose(); + if (stopped) + { + // Only once the loop is provably gone — see StopSampleStreamAsync for why. + cts.Dispose(); + } } // ---- IRediscoverable: the Agent instanceId watch ---- @@ -1395,7 +1518,33 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// private void StartSampleStreamCore(bool republishOnStart) { - if (_pumpTask is not null || _streamUnsupported || !HasSubscribedReferencesCore()) + // `IsCompleted: false`, not `is not null`: a pump that has already exited is not a running + // stream, and conflating the two means a pump killed by an unexpected fault could never be + // revived by a resubscribe. The genuinely-unrepeatable case is the latch below, not this. + if (_pumpTask is { IsCompleted: false }) + { + return; + } + + if (_streamUnsupported) + { + // Refusing IS correct — reconnecting reproduces the identical answer forever — but it + // must never be silent. DriverInstanceActor handles a tag-set change as + // Unsubscribe-then-Subscribe with NO re-initialize, and the unsubscribe stops a stream + // that is not running; without this, the resubscribe returns a handle, the caller is + // told SubscriptionEstablished, and the next successful read reports a perfectly green + // driver over a subscription that can never deliver a value (#485). + _streamDegraded = true; + Degrade($"The MTConnect Agent at '{_options.AgentUri}' does not serve a framed /sample stream; subscriptions cannot deliver until the driver is re-initialized."); + + _logger.LogWarning( + "MTConnect driver {DriverInstanceId} refused to start a /sample stream for {AgentUri}: the endpoint was already found not to stream. The subscription is registered but will receive NO updates until the driver is re-initialized.", + _driverInstanceId, _options.AgentUri); + + return; + } + + if (!HasSubscribedReferencesCore()) { return; } @@ -1408,6 +1557,15 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis return; } + // A previous pump ran to completion; its cancellation source is ours to release before the + // field is overwritten, or every revived stream leaks one. + _pumpCts?.Dispose(); + _pumpTask = null; + + // A new stream generation gets a fresh subscriber-fault Warning budget (see + // _subscriberFaultLogged). + Interlocked.Exchange(ref _subscriberFaultLogged, 0); + var options = _options; var cts = new CancellationTokenSource(); _pumpCts = cts; @@ -1450,6 +1608,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis { var cursor = from; var attempt = 0; + _reconnectAttempts = 0; try { @@ -1507,18 +1666,26 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis case StreamVerdict.Reopen: attempt = 0; + _reconnectAttempts = 0; break; default: _streamDegraded = true; Degrade(outcome.Error!); - attempt++; + + // A stream that DELIVERED before it dropped proves the endpoint works, so + // its failure starts a fresh backoff ladder. Without this the counter is + // monotonic for the life of the process: a healthy agent that drops a + // connection once an hour would be pinned at MaxBackoffMs within a day, and + // "the first retry is immediate" would be true exactly once ever. + attempt = outcome.DeliveredAny ? 1 : attempt + 1; + _reconnectAttempts = attempt; _logger.LogWarning( outcome.Error, - "MTConnect driver {DriverInstanceId} lost the /sample stream from {AgentUri}; reconnecting from sequence {Cursor} (attempt {Attempt}).", - _driverInstanceId, options.AgentUri, cursor, attempt); + "MTConnect driver {DriverInstanceId} lost the /sample stream from {AgentUri}; reconnecting from sequence {Cursor} (attempt {Attempt}, delivered={Delivered}).", + _driverInstanceId, options.AgentUri, cursor, attempt, outcome.DeliveredAny); break; } @@ -1534,6 +1701,16 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis _streamDegraded = true; Degrade(ex); } + finally + { + // The session's cursor is the opening `from` of the NEXT pump, and until now it only + // ever held the priming /current's. A last-unsubscribe + resubscribe (no re-initialize) + // would therefore reopen at a sequence the Agent has long since evicted: it self-heals + // via OUT_OF_RANGE, but only after a wasted round trip, and a still-buffered stale + // cursor replays historical observations to subscribers first. One publish per pump + // lifetime, dropped if a lifecycle change has already installed a newer session. + PublishAgentCursor(instanceId, cursor); + } } /// @@ -1554,6 +1731,11 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // endless /current re-baseline storm against a perfectly healthy stream. var expected = cursor; + // Whether THIS connection ever handed us a chunk. A stream that delivered and then dropped + // is a working endpoint having a bad moment; one that never delivered may be an endpoint + // that cannot work at all. They must not share a backoff ladder — see RunSampleStreamAsync. + var delivered = false; + try { await foreach (var chunk in client.SampleAsync(cursor, ct).ConfigureAwait(false)) @@ -1586,7 +1768,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // DiscoverAsync. AnnounceAgentRestart(chunk.InstanceId, options); - return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + return await RebaselineAsync(client, options, expected, instanceId, delivered, ct) + .ConfigureAwait(false); } if (IMTConnectAgentClient.IsSequenceGap(expected, chunk)) @@ -1595,10 +1778,12 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis "MTConnect driver {DriverInstanceId} fell out of {AgentUri}'s buffer (expected sequence {Expected}, oldest retained {FirstSequence}); re-baselining from /current.", _driverInstanceId, options.AgentUri, expected, chunk.FirstSequence); - return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + return await RebaselineAsync(client, options, expected, instanceId, delivered, ct) + .ConfigureAwait(false); } ApplyAndFanOut(chunk); + delivered = true; // EVERY chunk advances the cursor, including an observation-free heartbeat: the // Agent sends those precisely so a quiet connection can be told from a dead one, and @@ -1614,21 +1799,22 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // a quiet end under a live token is a non-conforming client — reported as the lost // stream it is rather than treated as "the subscription is simply idle" (#485). return ct.IsCancellationRequested - ? new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, null) + ? new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, delivered, null) : new StreamOutcome( StreamVerdict.Transient, expected, instanceId, + delivered, new MTConnectStreamEndedException( MTConnectStreamEndReason.ConnectionClosed, options.AgentUri, 0)); } catch (OperationCanceledException) when (ct.IsCancellationRequested) { - return new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, null); + return new StreamOutcome(StreamVerdict.Cancelled, expected, instanceId, delivered, null); } catch (MTConnectStreamNotSupportedException ex) { - return new StreamOutcome(StreamVerdict.Unsupported, expected, instanceId, ex); + return new StreamOutcome(StreamVerdict.Unsupported, expected, instanceId, delivered, ex); } catch (InvalidDataException ex) { @@ -1642,13 +1828,14 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis "MTConnect driver {DriverInstanceId} could not resume {AgentUri}'s /sample stream at sequence {Cursor} (the Agent rejected it as out of range); re-baselining from /current.", _driverInstanceId, options.AgentUri, cursor); - return await RebaselineAsync(client, options, expected, instanceId, ct).ConfigureAwait(false); + return await RebaselineAsync(client, options, expected, instanceId, delivered, ct) + .ConfigureAwait(false); } catch (Exception ex) { // Transient by default: a dropped connection, a closing boundary, the heartbeat // watchdog's TimeoutException. Reconnect under backoff. - return new StreamOutcome(StreamVerdict.Transient, expected, instanceId, ex); + return new StreamOutcome(StreamVerdict.Transient, expected, instanceId, delivered, ex); } } @@ -1670,6 +1857,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis MTConnectDriverOptions options, long fallbackCursor, long fallbackInstanceId, + bool deliveredAny, CancellationToken ct) { try @@ -1677,7 +1865,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis var current = await BoundedAsync(client.CurrentAsync, "/current", options.RequestTimeoutMs, ct) .ConfigureAwait(false); - PublishAgentInstanceId(current.InstanceId); + PublishAgentCursor(current.InstanceId, current.NextSequence); ApplyAndFanOut(current); _streamDegraded = false; @@ -1691,11 +1879,13 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis current.InstanceId, current.NextSequence); - return new StreamOutcome(StreamVerdict.Reopen, current.NextSequence, current.InstanceId, null); + return new StreamOutcome( + StreamVerdict.Reopen, current.NextSequence, current.InstanceId, deliveredAny, null); } catch (OperationCanceledException) when (ct.IsCancellationRequested) { - return new StreamOutcome(StreamVerdict.Cancelled, fallbackCursor, fallbackInstanceId, null); + return new StreamOutcome( + StreamVerdict.Cancelled, fallbackCursor, fallbackInstanceId, deliveredAny, null); } catch (Exception ex) { @@ -1703,7 +1893,8 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // failure so the reconnect backoff applies — without it, an Agent that is down would be // re-baselined against as fast as the loop can spin. The cursor is deliberately left // where it was: reopening there either works or lands right back here. - return new StreamOutcome(StreamVerdict.Transient, fallbackCursor, fallbackInstanceId, ex); + return new StreamOutcome( + StreamVerdict.Transient, fallbackCursor, fallbackInstanceId, deliveredAny, ex); } } @@ -1717,7 +1908,9 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// one multiplies, from a floor, up to /// . Operator-authored nonsense cannot /// un-bound the loop: a multiplier that cannot grow the delay falls back to - /// , and negative values clamp to zero. + /// , a non-positive cap falls back to + /// (clamping to it would make EVERY delay zero — a spin, + /// not a backoff), and a negative minimum clamps to zero. /// /// The 1-based reconnect attempt about to be made. /// The authored backoff options. @@ -1726,7 +1919,9 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis { ArgumentNullException.ThrowIfNull(reconnect); - var capMs = Math.Max(0, reconnect.MaxBackoffMs); + // A non-positive cap is "unset", NOT "no cap". Math.Max(0, ...) would clamp every delay to + // zero and turn the reconnect loop into a spin against a refused connection. + var capMs = reconnect.MaxBackoffMs > 0 ? reconnect.MaxBackoffMs : DefaultMaxBackoffMs; var minMs = Math.Max(0, reconnect.MinBackoffMs); if (attempt <= 1) @@ -1829,24 +2024,73 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } /// - /// Raises one data-change callback, absorbing anything the subscriber throws. A consumer bug - /// must not tear down the pump that serves every OTHER subscriber — and the exception is - /// logged rather than swallowed silently, because a handler that throws on every value is - /// otherwise invisible. + /// Raises one data-change callback to every subscriber, absorbing anything each one + /// throws. A consumer bug must not tear down the pump that serves the others — nor rob them + /// of the value. /// + /// + /// + /// The invocation list is walked by hand, with the try/catch INSIDE the loop. A + /// plain OnDataChange?.Invoke(...) is a multicast call: the first handler that + /// throws aborts the rest of the list, so one broken consumer silently starves every + /// other subscriber of that observation — and a single try/catch around the whole + /// invoke absorbs the exception while leaving the starvation in place, which reads as + /// "isolated" in a log and is not. + /// + /// + /// One is shared across the list deliberately: it is + /// an immutable record, and allocating per handler on the hot path would buy nothing. + /// + /// private void RaiseDataChange(MTConnectSampleHandle handle, string reference, DataValueSnapshot snapshot) { - try + var handlers = OnDataChange; + if (handlers is null) { - OnDataChange?.Invoke(this, new DataChangeEventArgs(handle, reference, snapshot)); + return; } - catch (Exception ex) + + var args = new DataChangeEventArgs(handle, reference, snapshot); + + foreach (var registration in handlers.GetInvocationList()) + { + try + { + ((EventHandler)registration).Invoke(this, args); + } + catch (Exception ex) + { + LogSubscriberFault(ex, reference, handle); + } + } + } + + /// + /// Reports a faulting data-change subscriber once per stream generation at Warning, + /// and at Debug thereafter. + /// + /// + /// A consumer that throws on every value would otherwise emit one Warning per observation + /// per handle — on a busy Agent, thousands a second, which buries the incident it is + /// reporting along with everything else in the log. Nothing is lost: the later faults are + /// still emitted, at Debug, and the latch is cleared whenever the pump (re)starts. + /// + private void LogSubscriberFault(Exception ex, string reference, MTConnectSampleHandle handle) + { + if (Interlocked.Exchange(ref _subscriberFaultLogged, 1) == 0) { _logger.LogWarning( ex, - "MTConnect driver {DriverInstanceId} caught a fault from a data-change subscriber for {Reference} on {SubscriptionId}; the stream continues.", + "MTConnect driver {DriverInstanceId} caught a fault from a data-change subscriber for {Reference} on {SubscriptionId}; the stream continues and every other subscriber still received the value. Further subscriber faults are logged at Debug until the stream restarts.", _driverInstanceId, reference, handle.DiagnosticId); + + return; } + + _logger.LogDebug( + ex, + "MTConnect driver {DriverInstanceId} caught a further fault from a data-change subscriber for {Reference} on {SubscriptionId}.", + _driverInstanceId, reference, handle.DiagnosticId); } /// Every reference any live subscription is interested in. @@ -1883,22 +2127,44 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } /// - /// Records the Agent instanceId a re-baseline observed, onto whichever session is - /// installed. A compare-and-swap for the same reason - /// uses one: if a lifecycle change lands mid-update, - /// its state is newer and this write must be dropped, not replayed over it. + /// Records where the Agent now is — its instanceId and the sequence a stream + /// should resume from — onto whichever session is installed. /// - private void PublishAgentInstanceId(long instanceId) + /// + /// + /// Both fields move together, in one compare-and-swap. They are one fact about + /// one Agent process, and the session's own contract is that a reader gets the client + /// and its cursor as a consistent pair. Publishing the id alone (as this did) left the + /// session holding a NEW instanceId beside the cursor of the process that had just + /// died — the exact tear the packing exists to prevent. + /// + /// + /// There is no "the id did not change, so skip" early return, because the two + /// re-baselines that do NOT change the id — a sequence gap and an OUT_OF_RANGE + /// rejection — are precisely the ones whose whole purpose is to move the cursor. + /// Suppressing the write there would leave the next pump reopening at the stale + /// sequence that had just been rejected. + /// + /// + /// Compare-and-swap for the same reason uses one: + /// a lifecycle change landing mid-update has newer state, so this write is dropped + /// rather than replayed over it. + /// + /// + /// The Agent instanceId last observed. + /// The sequence a /sample stream should resume from. + private void PublishAgentCursor(long instanceId, long nextSequence) { while (true) { var session = Volatile.Read(ref _session); - if (session is null || session.InstanceId == instanceId) + if (session is null || + (session.InstanceId == instanceId && session.NextSequence == nextSequence)) { return; } - var updated = session with { InstanceId = instanceId }; + var updated = session with { InstanceId = instanceId, NextSequence = nextSequence }; if (ReferenceEquals(Interlocked.CompareExchange(ref _session, updated, session), session)) { return; @@ -2190,13 +2456,19 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// this await is a permanent hang with no exception and no log line. /// /// - /// The one thing outside that guarantee is a subscriber callback: an - /// handler that blocks forever blocks teardown with it. The - /// pump raises callbacks outside every lock precisely so that a slow handler only - /// delays; an infinite one is a consumer bug this driver cannot paper over. + /// The wait is bounded (, and the caller's token) — the + /// one thing outside the guarantee above is a subscriber callback, and an + /// handler that never returns would otherwise block teardown + /// forever. That is not merely slow: DriverInstanceActor.PostStop blocks an Akka + /// dispatcher thread on while this driver holds + /// , so an unbounded wait wedges an actor-system thread and + /// every later lifecycle call with it. Abandoning the wait is safe because the pump's + /// registry slots are cleared BEFORE it: a pump left running cannot be resurrected into + /// , and it is already cancelled. /// /// - private async Task StopSampleStreamAsync() + /// The caller's deadline for the teardown wait. + private async Task StopSampleStreamAsync(CancellationToken cancellationToken = default) { CancellationTokenSource? cts; Task? task; @@ -2223,11 +2495,21 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis // Raced another stop; the pump is already going away. } + var stopped = true; if (task is not null) { try { - await task.ConfigureAwait(false); + await task.WaitAsync(TeardownWait, cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) when (ex is TimeoutException or OperationCanceledException) + { + stopped = false; + + _logger.LogError( + ex, + "MTConnect driver {DriverInstanceId} gave up waiting for its /sample pump to stop after {TeardownWait}. The pump is cancelled and can publish nothing further, but something inside it is not returning — the usual cause is an OnDataChange subscriber that blocks. Teardown continues.", + _driverInstanceId, TeardownWait); } catch (Exception ex) { @@ -2240,11 +2522,25 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } } - cts.Dispose(); + if (stopped) + { + // Disposed ONLY once the pump is provably gone: an abandoned pump still holds this + // source's token, and disposing underneath it turns its next linked-token creation into + // an ObjectDisposedException. One leaked CTS beats a mystery exception. + cts.Dispose(); + } - // There is no stream any more, so there is nothing for it to be degraded about. Leaving the - // flag set would pin a later successful read to Degraded forever. - _streamDegraded = false; + // Nothing to be degraded about once the stream is gone — UNLESS the endpoint is latched as + // unable to stream at all, in which case there is no prospect of one coming back, and + // clearing this would let the next successful read report a green driver over a subscription + // that can never deliver (#485). See StartSampleStreamCore. + lock (_subscriptionLock) + { + if (!_streamUnsupported) + { + _streamDegraded = false; + } + } } /// @@ -2435,12 +2731,28 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis } /// Publishes without downgrading a Faulted driver. - private void Degrade(Exception ex) + private void Degrade(Exception ex) => Degrade(ex.Message); + + /// + /// Publishes with an explanatory message, without + /// downgrading a Faulted driver. + /// + /// + /// Deliberately not , even for the "an operator must fix + /// this" cases such as an endpoint that cannot stream. In this codebase Faulted is not + /// merely a stronger word: DriverHealthReport maps any Faulted driver to a + /// /readyz 503, taking the whole server node out of readiness. A driver whose + /// reads are perfectly healthy and whose subscriptions are dead is degraded, not unready — + /// de-readying the node over one misconfigured Agent would be a far larger blast radius than + /// the fault it reports. Faulted stays reserved for the case that earns it: an initialize + /// that failed, after which the driver serves nothing at all. + /// + private void Degrade(string message) { var current = ReadHealth(); if (current.State != DriverState.Faulted) { - WriteHealth(new DriverHealth(DriverState.Degraded, current.LastSuccessfulRead, ex.Message)); + WriteHealth(new DriverHealth(DriverState.Degraded, current.LastSuccessfulRead, message)); } } @@ -2598,7 +2910,7 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// /// /// The Agent's instanceId as last observed. Updated in place (by - /// ) when the pump sees the Agent restart, so the + /// ) when the pump sees the Agent restart, so the /// session always names the Agent process the client is actually talking to. /// /// @@ -2641,8 +2953,14 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis /// How the connection ended. /// The sequence the next connection must open at. /// The Agent instance the cursor belongs to. + /// + /// Whether this connection yielded at least one chunk before it ended. Distinguishes a + /// working endpoint having a bad moment from one that has never worked, which is what keeps + /// the reconnect backoff from ratcheting monotonically over a process lifetime. + /// /// The failure, when there was one. - private sealed record StreamOutcome(StreamVerdict Verdict, long Cursor, long InstanceId, Exception? Error); + private sealed record StreamOutcome( + StreamVerdict Verdict, long Cursor, long InstanceId, bool DeliveredAny, Exception? Error); // ---- config DTOs ---- diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs index d3c2d20d..47617451 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs @@ -37,6 +37,9 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable { private readonly CancellationTokenSource _disposeCts = new(); + private readonly Lock _sampleWaitersLock = new(); + private readonly List<(int Threshold, TaskCompletionSource Waiter)> _sampleWaiters = []; + /// Chunks a test has scripted but not yet pumped — see . private readonly ConcurrentQueue _script = new(); @@ -201,6 +204,47 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable /// The from sequence the most recent /sample enumeration was opened at. public long? LastSampleFrom { get; private set; } + /// + /// A task completing once has reached + /// — the deterministic "the pump has now opened its Nth stream" + /// barrier, for the reconnect paths where no chunk is ever delivered and + /// therefore cannot be the barrier. + /// + /// The enumeration count to wait for. + public Task WaitForSampleCallsAsync(int count) + { + lock (_sampleWaitersLock) + { + if (Volatile.Read(ref _sampleCallCount) >= count) + { + return Task.CompletedTask; + } + + var waiter = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + _sampleWaiters.Add((count, waiter)); + + return waiter.Task; + } + } + + /// Releases any waiter the new call count satisfies. + private void ReleaseSampleWaiters(int reached) + { + lock (_sampleWaitersLock) + { + for (var i = _sampleWaiters.Count - 1; i >= 0; i--) + { + if (_sampleWaiters[i].Threshold > reached) + { + continue; + } + + _sampleWaiters[i].Waiter.TrySetResult(); + _sampleWaiters.RemoveAt(i); + } + } + } + /// public async Task ProbeAsync(CancellationToken ct) { @@ -258,7 +302,7 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable long from, [EnumeratorCancellation] CancellationToken ct) { ObjectDisposedException.ThrowIf(IsDisposed, this); - Interlocked.Increment(ref _sampleCallCount); + ReleaseSampleWaiters(Interlocked.Increment(ref _sampleCallCount)); LastSampleFrom = from; // Thrown before the first chunk, exactly like a real client that fails its /sample request: @@ -372,18 +416,33 @@ internal sealed class CannedAgentClient : IMTConnectAgentClient, IDisposable } /// + /// + /// + /// The first-disposer test uses the value + /// returned, not a re-read of the counter. Re-reading is a race: two concurrent + /// disposes can both increment and then both read 2, so BOTH take the early return and + /// the stream is never closed — a subscription test would then hang waiting for a + /// teardown that silently did nothing. + /// + /// + /// is cancelled but deliberately NOT disposed. A + /// /sample enumeration that is starting concurrently reaches + /// CreateLinkedTokenSource(ct, _disposeCts.Token), and reading .Token on a + /// disposed source throws — a flake that would + /// surface as a random unrelated failure in whichever test happened to lose the race. + /// Leaking one cancelled source per fake is free; a cancelled source needs no disposal + /// to release anything a test cares about. + /// + /// public void Dispose() { - Interlocked.Increment(ref _disposeCount); - - if (DisposeCount > 1) + if (Interlocked.Increment(ref _disposeCount) > 1) { return; } _disposeCts.Cancel(); Volatile.Read(ref _chunks).Writer.TryComplete(); - _disposeCts.Dispose(); } private sealed record ScriptedChunk(MTConnectStreamsResult Result, TaskCompletionSource Consumed); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs index 369e366b..6e02e552 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDiscoverTests.cs @@ -58,7 +58,9 @@ public sealed class MTConnectDiscoverTests }; private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( - MTConnectDriverOptions? options = null, MTConnectProbeModel? probe = null) + MTConnectDriverOptions? options = null, + MTConnectProbeModel? probe = null, + RecordingDriverLogger? logger = null) { var client = CannedAgentClient.FromFixtures(); if (probe is not null) @@ -66,13 +68,15 @@ public sealed class MTConnectDiscoverTests client.Probe = probe; } - return (new MTConnectDriver(options ?? Opts(), "mt1", _ => client), client); + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => client, logger), client); } private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( - MTConnectDriverOptions? options = null, MTConnectProbeModel? probe = null) + MTConnectDriverOptions? options = null, + MTConnectProbeModel? probe = null, + RecordingDriverLogger? logger = null) { - var (driver, client) = NewDriver(options, probe); + var (driver, client) = NewDriver(options, probe, logger); await driver.InitializeAsync("{}", TestContext.Current.CancellationToken); return (driver, client); @@ -476,6 +480,93 @@ public sealed class MTConnectDiscoverTests cap.Variables.Select(v => v.Attr.FullName).ShouldBe(["dev9_a"]); } + /// + /// A configured DeviceName that matches nothing is an authoring error whose only + /// symptom is an empty picker. It must be reported at Warning — at Debug the operator + /// sees a browse that "worked" and a machine that apparently declares nothing, with no + /// evidence pointing at their typo. + /// + [Fact] + public async Task Discover_warns_when_the_configured_device_scope_matches_nothing() + { + var logger = new RecordingDriverLogger(); + var (driver, _) = await InitializedDriverAsync(Opts(deviceName: "no-such-device"), logger: logger); + + var cap = await DiscoverAsync(driver); + + cap.Variables.ShouldBeEmpty(); + logger.WarningsSnapshot() + .ShouldContain(w => w.Contains("no-such-device", StringComparison.Ordinal)); + } + + /// + /// …and an agent-wide browse that finds nothing is NOT that warning: it is a different + /// statement (the Agent declares no devices) and must not cry scope-typo. + /// + [Fact] + public async Task Discover_does_not_warn_about_scope_when_no_scope_is_configured() + { + var logger = new RecordingDriverLogger(); + var (driver, _) = await InitializedDriverAsync(probe: new MTConnectProbeModel([]), logger: logger); + + var cap = await DiscoverAsync(driver); + + cap.Variables.ShouldBeEmpty(); + logger.WarningsSnapshot().ShouldBeEmpty(); + } + + /// + /// A component declaring neither a name nor an id has no browse path at all, + /// and folding it in would open a folder called "" — a blank segment in the path of + /// everything beneath it. It is skipped; its siblings are unaffected. (A component with a + /// blank id but a real name is fine — the id is only the fallback.) + /// + [Fact] + public async Task Discover_skips_a_component_with_neither_a_name_nor_an_id() + { + var item = new MTConnectDataItem("dev9_x", null, "SAMPLE", "POSITION", null, null, null, null); + var probe = new MTConnectProbeModel( + [ + new MTConnectDevice( + "dev9", + "VMC", + [ + new MTConnectComponent(" ", " ", [], [item]), + new MTConnectComponent(" ", "Axes", [], [item]), + new MTConnectComponent("dev9_ctrl", null, [], [item]), + ], + []), + ]); + + var (driver, _) = await InitializedDriverAsync(probe: probe); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Select(f => f.Path).ShouldBe(["VMC", "VMC/Axes", "VMC/dev9_ctrl"], ignoreOrder: true); + cap.Folders.ShouldAllBe(f => !f.Path.Contains("//", StringComparison.Ordinal)); + } + + /// The same rule one level up: a device with no browse path is skipped, and said so. + [Fact] + public async Task Discover_skips_a_device_with_neither_a_name_nor_an_id() + { + var item = new MTConnectDataItem("dev9_x", null, "SAMPLE", "POSITION", null, null, null, null); + var logger = new RecordingDriverLogger(); + var probe = new MTConnectProbeModel( + [ + new MTConnectDevice(" ", " ", [], [item]), + new MTConnectDevice("dev9", "VMC", [], [item]), + ]); + + var (driver, _) = await InitializedDriverAsync(probe: probe, logger: logger); + + var cap = await DiscoverAsync(driver); + + cap.Folders.Select(f => f.Path).ShouldBe(["VMC"]); + logger.WarningsSnapshot() + .ShouldContain(w => w.Contains("neither a name nor an id", StringComparison.Ordinal)); + } + // ---- the probe-cache contract (anti-wedge) ---- /// @@ -556,7 +647,10 @@ public sealed class MTConnectDiscoverTests /// no data items", which is the #485 shape (failure wearing success's clothes) and would read /// to an operator in the browse picker as a working connection to an empty machine. Both /// production callers handle the throw: the universal browser surfaces it as a failed browse, - /// and DriverInstanceActor logs it and retries. + /// and DriverInstanceActor logs it, publishes an empty node set, and does NOT retry + /// (retrying is an UntilStable behaviour; this driver is Once) — which today + /// reaches nothing anyway, since DriverHostActor's discovered-node injection is + /// dormant in v3. The browse path is the consumer this posture is chosen for. /// [Fact] public async Task Discover_before_initialize_throws_and_streams_nothing() diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs index 14be33e9..0b9e3e1b 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectHostAndRediscoverTests.cs @@ -121,6 +121,53 @@ public sealed class MTConnectHostAndRediscoverTests await ticker.ParkedAsync().WaitAsync(Watchdog, Ct); } + /// + /// Teardown symmetry with the /sample pump (remediation I1). The probe loop is + /// the second long-lived background task in this driver, and it is stopped the same way and + /// from the same place: while the lifecycle semaphore is held, on a path + /// DriverInstanceActor.PostStop blocks an Akka dispatcher thread on. So its wait is + /// bounded and honours the caller's token too — a blocking + /// handler must not be able to wedge + /// shutdown, any more than a blocking data-change handler can. + /// + /// + /// Its blast radius is genuinely smaller than the pump's (each iteration is bounded by the + /// validated-positive Probe.Timeout, where the pump's is an unbounded long poll), but + /// leaving the two divergent would make the un-bounded one the pattern a future reader + /// copies — the defect class would be closed in one place and re-opened in the other, in the + /// same file. + /// + [Fact] + public async Task Shutdown_honours_the_callers_deadline_when_a_host_status_subscriber_blocks() + { + var (driver, _, ticker) = await ProbingDriverAsync(); + var blocking = new ManualResetEventSlim(false); + var entered = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + + ((IHostConnectivityProbe)driver).OnHostStatusChanged += (_, _) => + { + entered.TrySetResult(); + blocking.Wait(); + }; + + try + { + // The first successful probe transitions Unknown -> Running and raises the event, which + // now blocks inside the loop. + ticker.Tick(); + await entered.Task.WaitAsync(Watchdog, Ct); + + using var deadline = new CancellationTokenSource(TimeSpan.FromMilliseconds(100)); + await driver.ShutdownAsync(deadline.Token).WaitAsync(Watchdog, Ct); + + driver.ProbeLoopTask.ShouldBeNull(); + } + finally + { + blocking.Set(); + } + } + private static ConcurrentQueue RecordRediscovery(MTConnectDriver driver) { var seen = new ConcurrentQueue(); diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs index d7b950b7..b12c32ff 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectSubscribeTests.cs @@ -84,17 +84,21 @@ public sealed class MTConnectSubscribeTests // ---- fixtures ---- private static (MTConnectDriver Driver, CannedAgentClient Client) NewDriver( - CannedAgentClient? client = null, MTConnectDriverOptions? options = null) + CannedAgentClient? client = null, + MTConnectDriverOptions? options = null, + RecordingDriverLogger? logger = null) { var agent = client ?? CannedAgentClient.FromFixtures(); - return (new MTConnectDriver(options ?? Opts(), "mt1", _ => agent), agent); + return (new MTConnectDriver(options ?? Opts(), "mt1", _ => agent, logger), agent); } private static async Task<(MTConnectDriver Driver, CannedAgentClient Client)> InitializedDriverAsync( - CannedAgentClient? client = null, MTConnectDriverOptions? options = null) + CannedAgentClient? client = null, + MTConnectDriverOptions? options = null, + RecordingDriverLogger? logger = null) { - var (driver, agent) = NewDriver(client, options); + var (driver, agent) = NewDriver(client, options, logger); await driver.InitializeAsync("{}", Ct); return (driver, agent); @@ -464,11 +468,24 @@ public sealed class MTConnectSubscribeTests driver.GetHealth().State.ShouldBe(DriverState.Degraded); driver.GetHealth().LastError.ShouldNotBeNullOrWhiteSpace(); - // A full unsubscribe/resubscribe cycle must not re-open the wound either. + // A full unsubscribe/resubscribe cycle must not re-open the wound either. This is exactly + // what DriverInstanceActor does for a tag-set change: UnsubscribeAsync then SubscribeAsync, + // with NO re-initialize in between. await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); client.SampleCallCount.ShouldBe(1); + + // ...and the driver must not read back GREEN afterwards. The unsubscribe stopped a stream + // that was never running, the resubscribe was silently refused by the latch, and one + // successful read was then enough to clear the last degradation flag — leaving a Healthy + // driver holding a subscription that can never deliver a value. That is #485 in the + // subscription plane: an established handle, an Established reply, and permanent silence. + var res = await driver.ReadAsync(["dev1_pos"], Ct); + + res[0].StatusCode.ShouldBe(Good); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + driver.GetHealth().LastError.ShouldNotBeNullOrWhiteSpace(); } /// @@ -747,8 +764,12 @@ public sealed class MTConnectSubscribeTests public async Task A_throwing_subscriber_does_not_kill_the_pump() { var (driver, client) = await InitializedDriverAsync(); - var seen = Record(driver); + // Registration ORDER is the whole point: the thrower goes FIRST. A multicast invoke aborts + // the rest of the invocation list at the first exception, so with the recorder registered + // first it would run before the throw and its assertions would hold however badly the + // driver isolates subscribers — the test would pass without testing anything. driver.OnDataChange += (_, _) => throw new InvalidOperationException("subscriber blew up"); + var seen = Record(driver); await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); seen.Clear(); @@ -819,6 +840,250 @@ public sealed class MTConnectSubscribeTests client.SampleCallCount.ShouldBe(2); } + /// + /// C1, the other half. Refusing to start the stream is correct, but it must not be + /// silent: the operator's only other evidence is a subscription that never delivers. The + /// refusal re-asserts the degradation AND says so, naming the remedy. + /// + [Fact] + public async Task A_refused_stream_start_re_asserts_degraded_and_says_so() + { + var logger = new RecordingDriverLogger(); + var (driver, client) = await InitializedDriverAsync(logger: logger); + client.SampleFailure = new MTConnectStreamNotSupportedException("not multipart"); + + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await driver.SampleStreamTask!.WaitAsync(Watchdog, Ct); + + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + logger.WarningsSnapshot() + .ShouldContain(w => w.Contains("refused to start a /sample stream", StringComparison.Ordinal)); + } + + /// + /// C1, the window the refusal path cannot cover. Dropping the last subscription stops + /// a stream, and teardown clears the stream degradation with it — but NOT while the endpoint + /// is latched as unable to stream at all. That latch is a standing fact about the Agent, + /// discovered at runtime and true whether or not anyone is subscribed right now; letting a + /// successful read clear it here would make the driver flicker green between subscriptions + /// and report the impairment only while someone happens to be listening. + /// + [Fact] + public async Task A_latched_endpoint_stays_degraded_even_with_no_subscriptions_left() + { + var (driver, client) = await InitializedDriverAsync(); + client.SampleFailure = new MTConnectStreamNotSupportedException("not multipart"); + + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + await driver.SampleStreamTask!.WaitAsync(Watchdog, Ct); + + // No subscriptions left at all — so nothing re-asserts the degradation on a refused start. + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + + var res = await driver.ReadAsync(["dev1_pos"], Ct); + + res[0].StatusCode.ShouldBe(Good); + driver.GetHealth().State.ShouldBe(DriverState.Degraded); + } + + /// + /// I1 — the caller's teardown deadline must be real. DriverInstanceActor wraps + /// this call in a 5 s and blocks an Akka dispatcher + /// thread on the shutdown path, so a wait that ignored the token would let one blocking + /// OnDataChange handler wedge an actor-system thread — while holding the lifecycle + /// semaphore, taking every later lifecycle call on this driver with it. + /// + [Fact] + public async Task Unsubscribe_honours_the_callers_deadline_when_a_subscriber_blocks() + { + var (driver, client) = await InitializedDriverAsync(); + var blocking = new ManualResetEventSlim(false); + var entered = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + + // Blocks only on the PUMPED value — the initial-data callback fires on the subscribe + // thread, and blocking there would prove nothing about teardown. + driver.OnDataChange += (_, e) => + { + if (e.Snapshot.Value is not 1.5d) + { + return; + } + + entered.TrySetResult(); + blocking.Wait(); + }; + + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + try + { + _ = client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))); + await entered.Task.WaitAsync(Watchdog, Ct); + + // The pump is now stuck inside caller code. The unsubscribe must still return. + using var deadline = new CancellationTokenSource(TimeSpan.FromMilliseconds(100)); + await driver.UnsubscribeAsync(handle, deadline.Token).WaitAsync(Watchdog, Ct); + + driver.SampleStreamTask.ShouldBeNull(); + } + finally + { + blocking.Set(); + } + } + + /// + /// I2 — the backoff ladder is per-outage, not per-process. A stream that delivered + /// before it dropped proves the endpoint works, so its failure starts a fresh ladder. Without + /// this the counter only ever climbs: an agent that drops one connection an hour would be + /// pinned at MaxBackoffMs within a day, having never once failed twice in a row. + /// + [Fact] + public async Task Reconnect_attempts_reset_when_the_stream_delivered_before_dropping() + { + var (driver, client) = await InitializedDriverAsync(); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + client.EndStream(); + await client.PumpAsync(Chunk(113, 118, ("dev1_pos", "2.5"))).WaitAsync(Watchdog, Ct); + client.EndStream(); + await client.PumpAsync(Chunk(118, 123, ("dev1_pos", "3.5"))).WaitAsync(Watchdog, Ct); + + // Two unrelated drops, each preceded by a delivered chunk: each starts a fresh ladder, so + // the count is 1 (this outage's first retry, which still honours MinBackoffMs) — never 2. + driver.ReconnectAttempts.ShouldBe(1); + client.SampleCallCount.ShouldBe(3); + } + + /// + /// …and the counter still accumulates when nothing is delivered, which is the case the + /// backoff exists for. Asserted as monotonic growth rather than an exact value, because the + /// pump may have advanced again between the barrier and the read. + /// + [Fact] + public async Task Reconnect_attempts_accumulate_while_the_stream_delivers_nothing() + { + // A 1 ms cap keeps the failing loop from actually sleeping; nothing here asserts a duration. + var options = Opts(new MTConnectReconnectOptions { MinBackoffMs = 0, MaxBackoffMs = 1 }); + var (driver, client) = await InitializedDriverAsync(options: options); + client.SampleFailure = new TimeoutException("no chunk within the heartbeat window"); + + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + try + { + await client.WaitForSampleCallsAsync(3).WaitAsync(Watchdog, Ct); + var early = driver.ReconnectAttempts; + + await client.WaitForSampleCallsAsync(6).WaitAsync(Watchdog, Ct); + + early.ShouldBeGreaterThanOrEqualTo(2); + driver.ReconnectAttempts.ShouldBeGreaterThan(early); + } + finally + { + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + } + } + + /// + /// I3 — a non-positive cap is an operator-authorable hot loop. Clamping to it (the + /// obvious Math.Max(0, …)) makes EVERY delay zero, so the pump reconnect-spins as fast + /// as a refused connection returns, one Warning per iteration. Treated as "unset" instead, + /// exactly like a multiplier that cannot grow. + /// + [Theory] + [InlineData(0)] + [InlineData(-1)] + public void Reconnect_backoff_treats_a_non_positive_cap_as_unset(int maxBackoffMs) + { + var options = new MTConnectReconnectOptions + { + MinBackoffMs = 0, MaxBackoffMs = maxBackoffMs, BackoffMultiplier = 2.0, + }; + + // The first retry is still immediate — that is MinBackoffMs, and it is honoured. + MTConnectDriver.BackoffFor(1, options).ShouldBe(TimeSpan.Zero); + + // Every later one must actually back off. + MTConnectDriver.BackoffFor(2, options).ShouldBeGreaterThan(TimeSpan.Zero); + MTConnectDriver.BackoffFor(9, options).ShouldBeGreaterThan(MTConnectDriver.BackoffFor(2, options)); + } + + /// + /// I4 — the session's instanceId and cursor are one fact and must move together. + /// Publishing the id alone left the session holding the NEW agent's id beside the OLD + /// agent's cursor, which is precisely the tear the record exists to prevent — and on a gap + /// or OUT_OF_RANGE re-baseline (where the id does not change at all) the cursor was + /// not published even once. + /// + [Fact] + public async Task Rebaseline_publishes_the_cursor_beside_the_instance_id() + { + var client = CannedAgentClient.WithGapThenResume(); + var (driver, _) = await InitializedDriverAsync(client); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + driver.AgentNextSequence.ShouldBe(PrimedNextSequence); + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // the gap chunk + + driver.AgentNextSequence.ShouldBe(5005L); + driver.AgentInstanceId.ShouldBe(FixtureInstanceId); + } + + /// + /// …and the consequence that makes it matter: the NEXT pump opens at the re-baselined + /// cursor. A last-unsubscribe followed by a resubscribe is an ordinary tag-set change + /// (DriverInstanceActor does exactly that, with no re-initialize), and a stale cursor + /// there means reopening at a sequence the Agent has already evicted. + /// + [Fact] + public async Task A_restarted_pump_resumes_from_the_rebaselined_cursor() + { + var client = CannedAgentClient.WithGapThenResume(); + var (driver, _) = await InitializedDriverAsync(client); + var handle = await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // gap -> re-baseline to 5005 + + await driver.UnsubscribeAsync(handle, Ct).WaitAsync(Watchdog, Ct); + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + + await client.PumpOnce().WaitAsync(Watchdog, Ct); // the contiguous follow-up + + client.LastSampleFrom.ShouldBe(5005L); + } + + /// + /// I5 — one broken subscriber must not starve the others. A multicast + /// Invoke aborts the invocation list at the first exception, so a single try/catch + /// around it absorbs the exception while silently robbing every later subscriber of the + /// value — which reads as "isolated" in the log and is not. The thrower sits BETWEEN the two + /// recorders on purpose. + /// + [Fact] + public async Task Every_subscriber_receives_a_value_even_when_one_in_the_middle_throws() + { + var (driver, client) = await InitializedDriverAsync(); + var first = Record(driver); + driver.OnDataChange += (_, _) => throw new InvalidOperationException("subscriber blew up"); + var last = Record(driver); + + await driver.SubscribeAsync(["dev1_pos"], TimeSpan.FromMilliseconds(50), Ct); + first.Clear(); + last.Clear(); + + await client.PumpAsync(Chunk(PrimedNextSequence, 113, ("dev1_pos", "1.5"))).WaitAsync(Watchdog, Ct); + + For(first, "dev1_pos").Select(e => e.Snapshot.Value).ShouldBe([1.5d]); + For(last, "dev1_pos").Select(e => e.Snapshot.Value).ShouldBe([1.5d]); + } + /// A handle from some other driver — the type check must not be a cast. private sealed class ForeignHandle : ISubscriptionHandle { diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/RecordingDriverLogger.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/RecordingDriverLogger.cs new file mode 100644 index 00000000..a2193020 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/RecordingDriverLogger.cs @@ -0,0 +1,65 @@ +using Microsoft.Extensions.Logging; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Captures what the driver logs, so that "and it says so" can be asserted rather than assumed. +/// +/// +/// Used where the log line is the behaviour: a browse scope that matches nothing, and a +/// subscription the driver silently refuses to serve. Both are cases whose only other symptom is +/// an operator staring at an empty panel, so a fix that emits nothing is not a fix. +/// +internal sealed class RecordingDriverLogger : ILogger +{ + /// Formatted lines, in order. + public List Warnings { get; } = []; + + /// Formatted lines, in order. + public List Errors { get; } = []; + + /// + public IDisposable? BeginScope(TState state) + where TState : notnull => null; + + /// + public bool IsEnabled(LogLevel logLevel) => true; + + /// + public void Log( + LogLevel logLevel, + EventId eventId, + TState state, + Exception? exception, + Func formatter) + { + ArgumentNullException.ThrowIfNull(formatter); + + var sink = logLevel switch + { + LogLevel.Warning => Warnings, + LogLevel.Error => Errors, + _ => null, + }; + + // Locked: the /sample pump logs from its own task while the test thread reads. + if (sink is null) + { + return; + } + + lock (sink) + { + sink.Add(formatter(state, exception)); + } + } + + /// A snapshot copy of , safe to enumerate while the pump runs. + public IReadOnlyList WarningsSnapshot() + { + lock (Warnings) + { + return [.. Warnings]; + } + } +} -- 2.52.0 From 23cd47d54b4f01b0e2499406589dada958d3d259 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:25:27 -0400 Subject: [PATCH 22/34] feat(mtconnect): IDriverProbe one-shot /probe reachability (Task 14) --- .../MTConnectDriverProbe.cs | 200 +++++++++++++++ .../MTConnectDriverProbeTests.cs | 235 ++++++++++++++++++ 2 files changed, 435 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs new file mode 100644 index 00000000..a6afb152 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs @@ -0,0 +1,200 @@ +using System.Diagnostics; +using System.Net.Http; +using System.Text.Json; +using System.Text.Json.Serialization; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// One-shot MTConnect Agent reachability probe for the AdminUI's Test Connect button. Issues a +/// single GET {AgentUri}/probe under the supplied deadline and reports success/failure +/// with latency — it does not construct or initialize an instance. +/// +/// +/// +/// Deliberately does NOT delegate to . That +/// method builds the full driver options document, including every authored +/// MTConnectTagDefinition (BuildTag can throw on a malformed tag entry that has +/// nothing to do with reachability). A probe only needs agentUri, so this type parses +/// its own minimal DTO — a config that is invalid for the driver (bad tag, bad data type) can +/// still be reachability-tested, matching the AdminUI's "does the endpoint answer" intent for +/// Test Connect. It also avoids a compile-time dependency on the factory DTO shape +/// (MTConnectDriverFactoryExtensions), which is authored separately (Task 15). +/// +/// +/// Deliberately DOES reuse and +/// for the actual round trip. Hand-rolling the HTTP +/// call would have to re-derive the exact behaviour the client already provides: per-call +/// bounding, a linked-CTS deadline, and — most importantly — +/// correct handling of an Agent's MTConnectError document served under HTTP 200 (so +/// EnsureSuccessStatusCode never fires). already +/// lifts the Agent's own error text into the thrown exception's message; re-parsing by hand +/// here would either duplicate that logic or under-validate (accepting any 200 response as +/// healthy, which is precisely the "empty-but-successful" defect class this codebase has hit +/// before, #485). The one-off cost of a full probe-document parse is negligible next to that +/// risk. +/// +/// +/// Never throws (the contract): every failure — unparseable +/// JSON, missing/malformed agentUri, DNS/TCP failure, a non-2xx response, a 200 body +/// that is not a valid MTConnectDevices document, and timeout — becomes +/// Ok = false with a message. Genuine caller cancellation is likewise converted rather +/// than left to propagate, matching ModbusDriverProbe's posture (its linked-CTS catch +/// does not distinguish the caller's token from its own deadline). +/// +/// +/// Non-positive timeout (arch-review 01/S-6). A 0 or negative +/// timeout does NOT mean "wait forever" — it is replaced with +/// (5s, matching 's own default) so an +/// operator-authorable field can never brick the probe UI. +/// +/// +/// Secrets discipline. AgentUri may carry userinfo credentials +/// (http://user:pass@host/). Every message this type constructs from the URI uses a +/// redacted display form () built ONLY from +/// /// +/// — never and never the raw config string — so a password can +/// never reach the AdminUI via the returned . +/// +/// +public sealed class MTConnectDriverProbe : IDriverProbe +{ + /// Replaces a non-positive caller timeout — see the type remarks. + private static readonly TimeSpan FallbackTimeout = TimeSpan.FromSeconds(5); + + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip, + Converters = { new JsonStringEnumConverter() }, + }; + + /// Test seam: routes the reused through a stub handler so response-shape cases need no socket. + private readonly HttpMessageHandler? _handler; + + /// Creates the probe used in production (real sockets). + public MTConnectDriverProbe() + : this(handler: null) + { + } + + /// Test seam constructor — see . + internal MTConnectDriverProbe(HttpMessageHandler? handler) + { + _handler = handler; + } + + /// + public string DriverType => "MTConnect"; + + /// Minimal probe-only config shape. Deliberately independent of the factory DTO — see the type remarks. + private sealed class MTConnectProbeConfigDto + { + public string? AgentUri { get; set; } + } + + /// + public async Task ProbeAsync(string configJson, TimeSpan timeout, CancellationToken ct) + { + MTConnectProbeConfigDto? dto; + try + { + dto = JsonSerializer.Deserialize(configJson, JsonOptions); + } + catch (Exception ex) + { + return new(false, $"Config JSON is invalid: {ex.Message}", null); + } + + if (dto is null) + { + return new(false, "Config JSON deserialized to null.", null); + } + + var agentUri = dto.AgentUri?.Trim(); + if (string.IsNullOrWhiteSpace(agentUri)) + { + return new(false, "Config has no agentUri to probe.", null); + } + + if (!Uri.TryCreate(agentUri, UriKind.Absolute, out var parsed) || + (parsed.Scheme != Uri.UriSchemeHttp && parsed.Scheme != Uri.UriSchemeHttps)) + { + return new(false, "Config's agentUri is not an absolute http(s) URI.", null); + } + + var safeUri = RedactedDisplay(parsed); + + // arch-review 01/S-6: a non-positive timeout must never mean "wait forever". + var effectiveTimeout = timeout > TimeSpan.Zero ? timeout : FallbackTimeout; + var requestTimeoutMs = Math.Max(1, (int)Math.Ceiling(effectiveTimeout.TotalMilliseconds)); + + var sw = Stopwatch.StartNew(); + using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct); + cts.CancelAfter(effectiveTimeout); + + var options = new MTConnectDriverOptions + { + AgentUri = agentUri, + RequestTimeoutMs = requestTimeoutMs, + }; + + try + { + using var client = new MTConnectAgentClient(options, _handler); + _ = await client.ProbeAsync(cts.Token).ConfigureAwait(false); + sw.Stop(); + + return new(true, $"MTConnect /probe OK ({safeUri})", sw.Elapsed); + } + catch (OperationCanceledException) + { + // Covers both the caller's token and our own CancelAfter deadline firing — see the type + // remarks on why cancellation is never allowed to propagate. + return new(false, $"Probe timed out after {effectiveTimeout.TotalSeconds:F0}s.", null); + } + catch (TimeoutException) + { + // MTConnectAgentClient's own request-deadline signal. + return new(false, $"Probe timed out after {effectiveTimeout.TotalSeconds:F0}s.", null); + } + catch (HttpRequestException ex) + { + // Connection failure or non-2xx status. HttpRequestException messages describe the + // socket endpoint or status code, never the request URI/userinfo, so ex.Message is safe. + return new(false, $"Request to {safeUri} failed: {ex.Message}", null); + } + catch (InvalidDataException ex) + { + // MTConnectProbeParser: not well-formed XML, not an MTConnectDevices document (including + // an MTConnectError-under-200, whose own errorCode/text ex.Message already carries), or + // a document with no devices. Never includes the request URI. + return new(false, $"Agent at {safeUri} did not return a valid MTConnect device model: {ex.Message}", null); + } + catch (ArgumentException) + { + // Defensive: MTConnectAgentClient's own AgentUri validation, reached only if this + // method's own Uri.TryCreate above passed but the client's stricter check disagreed. + // Never forward the exception's message verbatim — it can echo the raw (possibly + // credentialed) config string. + return new(false, "Config's agentUri was rejected by the MTConnect client as invalid.", null); + } + catch (Exception ex) + { + return new(false, ex.Message, null); + } + } + + /// + /// Builds a display form of for use in returned messages, composed + /// ONLY from scheme/host/port/path — never — so a credentialed + /// AgentUri can never leak a password into the AdminUI. + /// + private static string RedactedDisplay(Uri uri) + { + var portSegment = uri.IsDefaultPort ? string.Empty : $":{uri.Port}"; + + return $"{uri.Scheme}://{uri.Host}{portSegment}{uri.AbsolutePath}"; + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs new file mode 100644 index 00000000..8e25ba29 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs @@ -0,0 +1,235 @@ +using System.Net; +using System.Net.Http; +using Shouldly; +using Xunit; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 14 — : the AdminUI Test Connect reachability probe. +/// Response-shape cases (non-MTConnect body, MTConnectError-under-200, a valid device model) are +/// exercised through a stubbed via the internal test-seam +/// constructor, so no socket is needed; the unreachable-agent and non-positive-timeout cases use +/// a real (deliberately unroutable/refused) TCP target, mirroring the plan's canned example. +/// +[Trait("Category", "Unit")] +public sealed class MTConnectDriverProbeTests +{ + private static readonly TimeSpan QuickTimeout = TimeSpan.FromSeconds(3); + + // ── stub handler ───────────────────────────────────────────────────────── + + /// Answers every request with a canned response (or throws) via a delegate, with no socket. + private sealed class StubHandler(Func> respond) + : HttpMessageHandler + { + protected override Task SendAsync(HttpRequestMessage request, CancellationToken ct) => + respond(request, ct); + } + + private static HttpResponseMessage XmlResponse(string xml) => new(HttpStatusCode.OK) + { + Content = new StringContent(xml, System.Text.Encoding.UTF8, "application/xml"), + }; + + private static MTConnectDriverProbe ProbeWithHandler( + Func> respond) => + new(new StubHandler(respond)); + + private const string ValidProbeXml = + """ + + +
+ + + + + + + + + """; + + private const string MTConnectErrorXml = + """ + + +
+ + Could not find the device named 'nope'. + + + """; + + private const string NotMTConnectXml = + """ + + + """; + + // ── from the plan (verbatim) ───────────────────────────────────────────── + + [Fact] + public async Task Probe_on_unreachable_agent_returns_not_ok_and_does_not_throw() + { + var probe = new MTConnectDriverProbe(); + var r = await probe.ProbeAsync("{\"agentUri\":\"http://127.0.0.1:1/\"}", TimeSpan.FromMilliseconds(300), default); + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + } + + [Fact] + public async Task Probe_on_blank_config_returns_not_ok() + => (await new MTConnectDriverProbe().ProbeAsync("{}", TimeSpan.FromSeconds(1), default)).Ok.ShouldBeFalse(); + + // ── unparseable JSON ────────────────────────────────────────────────────── + + [Fact] + public async Task Unparseable_json_returns_not_ok_and_does_not_throw() + { + var probe = new MTConnectDriverProbe(); + + var r = await probe.ProbeAsync("not valid json {{", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldContain("invalid", Case.Insensitive); + } + + // ── malformed / relative URI ───────────────────────────────────────────── + + [Theory] + [InlineData("not a uri")] + [InlineData("/relative/path")] + [InlineData("ftp://host/probe")] + public async Task Malformed_or_non_http_agentUri_returns_not_ok(string agentUri) + { + var probe = new MTConnectDriverProbe(); + + var r = await probe.ProbeAsync($"{{\"agentUri\":\"{agentUri}\"}}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + } + + // ── credential redaction ───────────────────────────────────────────────── + + [Fact] + public async Task AgentUri_credentials_never_appear_in_the_message() + { + var probe = new MTConnectDriverProbe(); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://sneaky-user:sneaky-pass@127.0.0.1:1/\"}", + TimeSpan.FromMilliseconds(300), + CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldNotContain("sneaky-pass"); + r.Message.ShouldNotContain("sneaky-user:sneaky-pass"); + } + + // ── 200 body that is not MTConnect at all ──────────────────────────────── + + [Fact] + public async Task NonMTConnect_200_body_returns_not_ok() + { + var probe = ProbeWithHandler((_, _) => Task.FromResult(XmlResponse(NotMTConnectXml))); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Latency.ShouldBeNull(); + } + + // ── MTConnectError under HTTP 200 ──────────────────────────────────────── + + [Fact] + public async Task MTConnectError_under_200_returns_not_ok_with_agents_own_message() + { + var probe = ProbeWithHandler((_, _) => Task.FromResult(XmlResponse(MTConnectErrorXml))); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldContain("NO_DEVICE"); + r.Message.ShouldContain("Could not find the device named 'nope'."); + } + + // ── valid MTConnectDevices body ─────────────────────────────────────────── + + [Fact] + public async Task Valid_probe_document_returns_ok_true_with_latency() + { + var probe = ProbeWithHandler((_, _) => Task.FromResult(XmlResponse(ValidProbeXml))); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeTrue(); + r.Message.ShouldNotBeNull(); + r.Latency.ShouldNotBeNull(); + r.Latency!.Value.ShouldBeGreaterThanOrEqualTo(TimeSpan.Zero); + r.Latency!.Value.ShouldBeLessThan(QuickTimeout); + } + + // ── timeout bounded (hard hang-risk guard) ─────────────────────────────── + + [Fact(Timeout = 5000)] + public async Task Frozen_peer_times_out_within_the_requested_deadline_not_hang() + { + var probe = ProbeWithHandler(async (_, ct) => + { + await Task.Delay(Timeout.Infinite, ct); // never answers until cancelled + throw new UnreachableException(); + }); + + var sw = System.Diagnostics.Stopwatch.StartNew(); + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", TimeSpan.FromMilliseconds(300), CancellationToken.None); + sw.Stop(); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldContain("timed out", Case.Insensitive); + sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(4)); + } + + // ── non-positive timeout does not mean "wait forever" ──────────────────── + + [Fact(Timeout = 8000)] + public async Task Non_positive_timeout_falls_back_to_a_bounded_default_not_infinite() + { + var probe = ProbeWithHandler(async (_, ct) => + { + await Task.Delay(Timeout.Infinite, ct); // never answers until cancelled + throw new UnreachableException(); + }); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", TimeSpan.Zero, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + } + + [Fact] + public async Task Negative_timeout_on_unreachable_agent_still_returns_promptly() + { + var probe = new MTConnectDriverProbe(); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://127.0.0.1:1/\"}", TimeSpan.FromSeconds(-1), CancellationToken.None); + + r.Ok.ShouldBeFalse(); + } + + /// Local stand-in so the stub handler's "never returns" branch has a throw statement to satisfy flow analysis. + private sealed class UnreachableException : Exception; +} -- 2.52.0 From ba89d6d0ffaec0aace51d67ab93fb8ca9c29bb5c Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:26:15 -0400 Subject: [PATCH 23/34] feat(mtconnect): driver factory delegating to the single config parser (Task 15) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The factory carries NO config DTO of its own: CreateInstance delegates to MTConnectDriver.ParseOptions, the single authority for the config document. A second DTO would drift from the parse the runtime's ReinitializeAsync path uses, and the drift would only surface at deploy time. Construction is connection-free (agentClientFactory: null) because the Wave-0 universal browser builds a throwaway instance per browse probe, and the returned instance is never narrowed — its five capability interfaces ARE the runtime's dispatch surface. Pinned by tests asserting all five plus the deliberate absence of IWritable. DriverTypeName is the literal "MTConnect"; Task 16 adds DriverTypeNames.MTConnect and the host registration that must equal it. --- .../MTConnectDriverFactoryExtensions.cs | 113 ++++++ .../MTConnectFactoryTests.cs | 352 ++++++++++++++++++ 2 files changed, 465 insertions(+) create mode 100644 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs create mode 100644 tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectFactoryTests.cs diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs new file mode 100644 index 00000000..34797fa7 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs @@ -0,0 +1,113 @@ +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Core.Hosting; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; + +/// +/// Static factory registration helper for . The Host registers it +/// once at startup; the bootstrapper then materialises MTConnect DriverInstance rows from +/// the deployed configuration into live driver instances. Mirrors +/// ModbusDriverFactoryExtensions / FocasDriverFactoryExtensions. +/// +/// +/// +/// There is exactly one parser for the config document. This factory does not carry a +/// config DTO of its own — it delegates to , which +/// the driver itself must own because the runtime delivers a config change to a live +/// instance (DriverInstanceActor assigns its driver once and calls +/// ReinitializeAsync(newJson) thereafter). A second DTO here would drift from that +/// one silently, and the drift would first show up at deploy time as an operator's edit +/// being applied differently by the two paths — or not at all. +/// +/// +/// Construction opens nothing. The Wave-0 universal discovery browser builds a +/// throwaway instance through this factory purely to read +/// ITagDiscovery.SupportsOnlineDiscovery, and never initializes it — so a factory +/// that dialled the Agent (or that pre-built an ) would +/// open a connection per AdminUI browse render against a possibly-unreachable Agent. The +/// agent-client factory is therefore left null: builds the +/// production client inside InitializeAsync. +/// +/// +/// The instance is returned as its concrete type. The runtime resolves every optional +/// capability (, , +/// , , +/// ) by pattern-matching the object +/// hands back, so nothing here may narrow it — a +/// narrowed return is how a capability ends up implemented but never dispatched. There is +/// deliberately no IWritable: the MTConnect Agent surface is read-only. +/// +/// +public static class MTConnectDriverFactoryExtensions +{ + /// + /// The DriverInstance.DriverType value this factory answers to. + /// + /// + /// Kept identical to — the registry key and the + /// instance's self-reported type must agree or the runtime would look the driver up under a + /// name it never registered. Task 16 adds the matching DriverTypeNames.MTConnect + /// constant (the repo's convention is that dispatch maps key off those constants) and wires + /// the Host registration + guard test; this literal is what that constant must equal. + /// + public const string DriverTypeName = "MTConnect"; + + /// + /// Register the MTConnect factory with the driver registry. The optional + /// is captured at registration time and used to construct + /// an per driver instance — without it the driver runs + /// with the null logger (tests and standalone callers stay unchanged). + /// + /// The driver factory registry to register with. + /// Optional logger factory for creating loggers per driver instance. + public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null) + { + ArgumentNullException.ThrowIfNull(registry); + registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory)); + } + + /// Public for the Server-side bootstrapper + test consumers. + /// The unique identifier for the driver instance. + /// The driver's DriverConfig JSON. + /// The constructed instance. + public static MTConnectDriver CreateInstance(string driverInstanceId, string driverConfigJson) + => CreateInstance(driverInstanceId, driverConfigJson, loggerFactory: null); + + /// Logger-aware overload — used by 's closure when wired through DI. + /// The unique identifier for the driver instance. + /// The driver's DriverConfig JSON. + /// Optional logger factory for creating loggers per driver instance. + /// The constructed instance. + /// + /// or is null or blank. + /// + /// + /// The config document is unparseable, deserialises to null, omits the required + /// agentUri, or carries an unauthorable enum/tag value. Throwing is correct at this + /// seam: a deployment carrying such a row must fail rather than register a driver pointed at + /// nothing, and the discovery browser's CanBrowse catches factory exceptions so a + /// half-authored config merely disables the address picker. + /// + public static MTConnectDriver CreateInstance( + string driverInstanceId, string driverConfigJson, ILoggerFactory? loggerFactory) + { + ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); + ArgumentException.ThrowIfNullOrWhiteSpace(driverConfigJson); + + // The single config authority — see the class remarks for why this is not a second DTO. + // Note the asymmetry with the driver's own lifecycle path: an empty ("{}" / "[]") document + // there means "no config supplied, keep the constructor options", but this factory HAS no + // constructor options to keep, so an empty document is simply a config with no agentUri and + // must fault. + var options = MTConnectDriver.ParseOptions(driverInstanceId, driverConfigJson); + + // Named arguments deliberately: the ctor's trailing parameters are all optional, so a + // future insertion could silently re-bind a positional argument to the wrong one. + return new MTConnectDriver( + options, + driverInstanceId, + agentClientFactory: null, + logger: loggerFactory?.CreateLogger()); + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectFactoryTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectFactoryTests.cs new file mode 100644 index 00000000..49dea03c --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectFactoryTests.cs @@ -0,0 +1,352 @@ +using System.Diagnostics; +using System.Net; +using System.Net.Sockets; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Core.Hosting; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests; + +/// +/// Task 15 — : the seam that turns a +/// DriverInstance row's DriverConfig JSON into a live . +/// +/// Three properties carry the weight here. (1) One parser. The factory delegates to +/// MTConnectDriver.ParseOptions rather than deserializing a second DTO, so the config +/// document has a single authority and the runtime's ReinitializeAsync path cannot +/// drift from the bootstrap path. (2) Connection-free construction. The Wave-0 +/// universal browser builds a throwaway instance purely to read +/// SupportsOnlineDiscovery, so a factory that dialled would open a socket per AdminUI +/// browse render against a possibly-unreachable Agent. (3) No dropped capability. The +/// registry hands the runtime an ; every capability is resolved by +/// pattern-matching the concrete instance, so the interface set on the object the factory +/// returns IS the dispatch surface. +/// +/// +public sealed class MTConnectFactoryTests +{ + private const uint Good = 0x00000000u; + private const uint BadWaitingForInitialData = 0x80320000u; + private const uint BadNodeIdUnknown = 0x80340000u; + + // ----------------------------------------------------------------- construction + + /// The plan's headline case: a one-key document is a complete MTConnect config. + [Fact] + public void Factory_builds_a_driver_from_minimal_config() + { + var d = MTConnectDriverFactoryExtensions.CreateInstance("mt1", "{\"agentUri\":\"http://a:5000\"}"); + + d.DriverType.ShouldBe("MTConnect"); + d.DriverInstanceId.ShouldBe("mt1"); + } + + /// + /// The factory's own type name and the instance's must be + /// the same string, or a registered factory would materialise a driver the runtime looks up + /// under a different key. + /// + [Fact] + public void DriverTypeName_matches_the_instances_DriverType() + { + var d = MTConnectDriverFactoryExtensions.CreateInstance("mt1", "{\"agentUri\":\"http://a:5000\"}"); + + MTConnectDriverFactoryExtensions.DriverTypeName.ShouldBe("MTConnect"); + d.DriverType.ShouldBe(MTConnectDriverFactoryExtensions.DriverTypeName); + } + + // ----------------------------------------------------------------- rejection + + /// + /// No agentUri means there is no Agent to talk to. Throwing is correct: the browser's + /// CanBrowse catches factory exceptions (a half-authored config just disables the + /// picker), and a deployment carrying such a row must fail rather than register a driver + /// pointed at nothing. + /// + [Fact] + public void Factory_rejects_config_without_agentUri() + => Should.Throw(() => MTConnectDriverFactoryExtensions.CreateInstance("mt1", "{}")); + + /// + /// A whitespace-only agentUri is the same absence spelled differently — it must not + /// slip past the required check and become a driver dialling an empty URI. + /// + [Fact] + public void Factory_rejects_a_blank_agentUri() + => Should.Throw( + () => MTConnectDriverFactoryExtensions.CreateInstance("mt1", "{\"agentUri\":\" \"}")); + + /// + /// Unparseable JSON faults loudly rather than silently yielding a defaulted driver — the + /// "empty bytes are not an empty configuration" rule (#485) applied at the factory seam. + /// + [Fact] + public void Factory_rejects_unparseable_json_rather_than_defaulting() + { + var ex = Should.Throw( + () => MTConnectDriverFactoryExtensions.CreateInstance("mt1", "{ this is not json")); + + ex.Message.ShouldContain("mt1"); + } + + /// A JSON null document is not a config either. + [Fact] + public void Factory_rejects_a_null_json_document() + => Should.Throw(() => MTConnectDriverFactoryExtensions.CreateInstance("mt1", "null")); + + /// An empty config string has nothing to build from; the id likewise. + [Theory] + [InlineData("mt1", "")] + [InlineData("mt1", " ")] + [InlineData("", "{\"agentUri\":\"http://a:5000\"}")] + public void Factory_rejects_empty_arguments(string id, string json) + => Should.Throw(() => MTConnectDriverFactoryExtensions.CreateInstance(id, json)); + + // ----------------------------------------------------------------- connection-free + + /// + /// The factory opens no socket. Asserted directly: a real listener is bound, the + /// driver is built against its address, and nothing ever arrives on the accept queue. A + /// regression here would make every AdminUI browse render dial the Agent. + /// + [Fact] + public async Task CreateInstance_opens_no_socket_to_the_configured_agent() + { + var listener = new TcpListener(IPAddress.Loopback, 0); + listener.Start(); + try + { + var port = ((IPEndPoint)listener.LocalEndpoint).Port; + + var sw = Stopwatch.StartNew(); + var driver = MTConnectDriverFactoryExtensions.CreateInstance( + "mt1", $"{{\"agentUri\":\"http://127.0.0.1:{port}\"}}"); + sw.Stop(); + + driver.ShouldNotBeNull(); + + // Generous grace for a connect that a background thread might land late. + await Task.Delay(250, TestContext.Current.CancellationToken); + + listener.Pending().ShouldBeFalse( + "MTConnectDriverFactoryExtensions.CreateInstance dialled the Agent — the browser " + + "builds a throwaway instance per browse probe, so construction must open nothing."); + sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(2)); + } + finally + { + listener.Stop(); + listener.Dispose(); + } + } + + /// + /// The same property against an address that can never answer (TEST-NET-1, RFC 5737): + /// construction returns promptly instead of blocking on a connect that will only end in a + /// timeout. + /// + [Fact] + public void CreateInstance_returns_immediately_for_a_black_hole_agent() + { + var sw = Stopwatch.StartNew(); + var driver = MTConnectDriverFactoryExtensions.CreateInstance( + "mt1", "{\"agentUri\":\"http://192.0.2.1:5000\"}"); + sw.Stop(); + + driver.DriverInstanceId.ShouldBe("mt1"); + sw.Elapsed.ShouldBeLessThan(TimeSpan.FromSeconds(2)); + } + + // ----------------------------------------------------------------- registry wiring + + /// + /// puts the type in the registry + /// under the same key the bootstrapper looks up, and the registered delegate — not just the + /// static helper — builds a working instance. + /// + [Fact] + public void Register_adds_the_type_and_the_registered_delegate_builds_a_driver() + { + var registry = new DriverFactoryRegistry(); + + MTConnectDriverFactoryExtensions.Register(registry); + + registry.RegisteredTypes.ShouldContain("MTConnect"); + + var factory = registry.TryGet("MTConnect"); + factory.ShouldNotBeNull(); + + var driver = factory("mt-from-registry", "{\"agentUri\":\"http://a:5000\"}"); + + driver.DriverInstanceId.ShouldBe("mt-from-registry"); + driver.DriverType.ShouldBe("MTConnect"); + } + + /// + /// The anti-dormancy pin. The runtime resolves every optional capability by + /// pattern-matching the instance the registry hands back, so a capability the driver + /// implements but the factory's return path hides would be implemented-yet-never-dispatched + /// — a failure shape this repo has shipped twice. Asserted on the + /// the registry delegate returns, which is exactly what the bootstrapper holds. + /// + [Fact] + public void Registered_delegate_returns_an_instance_carrying_every_capability() + { + var registry = new DriverFactoryRegistry(); + MTConnectDriverFactoryExtensions.Register(registry); + + IDriver driver = registry.TryGet("MTConnect")!("mt1", "{\"agentUri\":\"http://a:5000\"}"); + + driver.ShouldBeOfType(); + (driver is IReadable).ShouldBeTrue("MTConnect must dispatch reads"); + (driver is ISubscribable).ShouldBeTrue("MTConnect must dispatch subscriptions"); + (driver is ITagDiscovery).ShouldBeTrue("MTConnect must dispatch browse/discovery"); + (driver is IHostConnectivityProbe).ShouldBeTrue("MTConnect must dispatch host connectivity"); + (driver is IRediscoverable).ShouldBeTrue("MTConnect must dispatch rediscovery"); + } + + /// + /// The other half of the capability pin: MTConnect's Agent surface is read-only, so the + /// instance must NOT be . If it ever became writable, the write gate + /// would start offering an operation the protocol cannot honour. + /// + [Fact] + public void Registered_delegate_returns_an_instance_that_is_not_writable() + { + var registry = new DriverFactoryRegistry(); + MTConnectDriverFactoryExtensions.Register(registry); + + IDriver driver = registry.TryGet("MTConnect")!("mt1", "{\"agentUri\":\"http://a:5000\"}"); + + (driver is IWritable).ShouldBeFalse("the MTConnect Agent surface is read-only"); + } + + /// Registering into a null registry is a wiring bug, not a silent no-op. + [Fact] + public void Register_rejects_a_null_registry() + => Should.Throw(() => MTConnectDriverFactoryExtensions.Register(null!)); + + // ----------------------------------------------------------------- config round-trip + + /// + /// An authored tags array reaches options.Tags. Observed through the + /// observation index the constructor builds from those options: an authored id answers + /// BadWaitingForInitialData ("subscribed, no value yet"), an unauthored one answers + /// BadNodeIdUnknown. Status codes are literals so a wrong constant cannot be masked + /// by both sides sharing one symbol. + /// + [Fact] + public void Authored_tags_round_trip_into_the_driver_options() + { + const string json = """ + { + "agentUri": "http://a:5000", + "tags": [ + { "fullName": "dev1_pos", "driverDataType": "Float64" }, + { "fullName": "dev1_execution", "driverDataType": "String" } + ] + } + """; + + var driver = MTConnectDriverFactoryExtensions.CreateInstance("mt1", json); + + driver.ObservationIndex.Get("dev1_pos").StatusCode.ShouldBe(BadWaitingForInitialData); + driver.ObservationIndex.Get("dev1_execution").StatusCode.ShouldBe(BadWaitingForInitialData); + driver.ObservationIndex.Get("dev1_never_authored").StatusCode.ShouldBe(BadNodeIdUnknown); + } + + /// + /// An enum authored as a name parses to that member — proved by behaviour, not by + /// reading the option back: a Float64 tag coerces the Agent's text into a real + /// , whereas the enum's member 0 (Boolean) or a defaulted + /// String would not. + /// + [Fact] + public void Enum_fields_authored_as_names_parse_to_that_member() + { + var driver = MTConnectDriverFactoryExtensions.CreateInstance( + "mt1", + """ + { "agentUri": "http://a:5000", + "tags": [ { "fullName": "dev1_pos", "driverDataType": "Float64" } ] } + """); + + driver.ObservationIndex.Apply(new MTConnectStreamsResult( + InstanceId: 1L, + NextSequence: 2L, + FirstSequence: 1L, + Observations: [new MTConnectObservation( + "dev1_pos", "12.5", new DateTime(2026, 7, 24, 12, 0, 0, DateTimeKind.Utc), false)])); + + var snap = driver.ObservationIndex.Get("dev1_pos"); + + snap.StatusCode.ShouldBe(Good); + snap.Value.ShouldBeOfType().ShouldBe(12.5d); + } + + /// + /// The enum-serialization trap. The factory deliberately carries no + /// JsonStringEnumConverter: a numerically-serialized enum — the shape an AdminUI page + /// that serialized the enum by value would emit — must fault at deploy rather than silently + /// landing on member 0 (Boolean), which would mis-coerce every value of that tag. + /// + [Fact] + public void Numerically_serialized_enum_faults_rather_than_landing_on_member_zero() + => Should.Throw(() => MTConnectDriverFactoryExtensions.CreateInstance( + "mt1", + """ + { "agentUri": "http://a:5000", + "tags": [ { "fullName": "dev1_pos", "driverDataType": 8 } ] } + """)); + + /// An enum name that names no member is an authoring error, reported with the field. + [Fact] + public void Unknown_enum_name_faults_naming_the_offending_tag() + { + var ex = Should.Throw(() => MTConnectDriverFactoryExtensions.CreateInstance( + "mt1", + """ + { "agentUri": "http://a:5000", + "tags": [ { "fullName": "dev1_pos", "driverDataType": "Flot64" } ] } + """)); + + ex.Message.ShouldContain("dev1_pos"); + } + + /// + /// Scalar knobs reach the options too — the factory must not quietly drop a section it does + /// not itself read. Proved through the one seam a non-initialized driver exposes: an + /// authored reconnect block changes the backoff the driver would apply. + /// + [Fact] + public void Reconnect_and_probe_sections_round_trip_into_the_driver_options() + { + const string json = """ + { + "agentUri": "http://a:5000", + "deviceName": "dev1", + "requestTimeoutMs": 1234, + "reconnect": { "minBackoffMs": 500, "maxBackoffMs": 4000, "backoffMultiplier": 3.0 }, + "probe": { "enabled": false, "intervalMs": 7000, "timeoutMs": 900 } + } + """; + + var options = MTConnectDriver.ParseOptions("mt1", json); + + options.AgentUri.ShouldBe("http://a:5000"); + options.DeviceName.ShouldBe("dev1"); + options.RequestTimeoutMs.ShouldBe(1234); + options.Reconnect.MinBackoffMs.ShouldBe(500); + options.Reconnect.MaxBackoffMs.ShouldBe(4000); + options.Reconnect.BackoffMultiplier.ShouldBe(3.0); + options.Probe.Enabled.ShouldBeFalse(); + options.Probe.Interval.ShouldBe(TimeSpan.FromMilliseconds(7000)); + options.Probe.Timeout.ShouldBe(TimeSpan.FromMilliseconds(900)); + + // …and the factory is built on exactly that parse, not a second DTO: the same document + // through the factory must produce an instance, and the same rejection rules (asserted + // above) hold for both entry points. + MTConnectDriverFactoryExtensions.CreateInstance("mt1", json).DriverInstanceId.ShouldBe("mt1"); + } +} -- 2.52.0 From a107a1a8b965ea02d674ce9d5fce0cfc25e78104 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:43:27 -0400 Subject: [PATCH 24/34] feat(mtconnect): host registration + DriverTypeNames const + guard parity (Task 16) Register the MTConnect factory + probe with the Host, add the DriverTypeNames.MTConnect constant, and keep DriverTypeNamesGuardTests green. - DriverTypeNames: new MTConnect const + appended to All. - DriverFactoryBootstrap: MTConnectDriverFactoryExtensions.Register in Register(...) at the default Tier A (managed HttpClient/XML, in-process; Tier C is the only tier arming process-recycle, wrong here), and TryAddEnumerable(IDriverProbe -> MTConnectDriverProbe) in AddOtOpcUaDriverProbes so the probe reaches admin-only nodes (the admin-pinned Test-Connect singleton) without double-registering on a fused admin,driver node. - Host.csproj: ProjectReference to Driver.MTConnect (required to compile the Register call). - Core.Abstractions.Tests.csproj: ProjectReference to Driver.MTConnect so the guard's bin scan discovers the factory. This and the constant MUST land together -- verified RED (2/4 facts) with the constant alone. - DriverProbeRegistrationTests: MTConnect added to AdminUiDriverTypeKeys; its idempotency fact asserts distinct-probe-count and goes RED without this (and RED if TryAddEnumerable is downgraded to AddSingleton). Reconciles all four "MTConnect" literals onto the one constant: the factory's DriverTypeName, MTConnectDriver.DriverType, and MTConnectDriverProbe.DriverType now all alias DriverTypeNames.MTConnect (no circular reference -- Core.Abstractions is a leaf both driver projects already reference). This is the repo's TwinCat/Focas anti-drift convention. --- .../DriverTypeNames.cs | 4 ++++ .../MTConnectDriver.cs | 7 ++++++- .../MTConnectDriverFactoryExtensions.cs | 15 +++++++++------ .../MTConnectDriverProbe.cs | 7 ++++++- .../Drivers/DriverFactoryBootstrap.cs | 9 +++++++++ .../ZB.MOM.WW.OtOpcUa.Host.csproj | 1 + ....MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj | 1 + .../DriverProbeRegistrationTests.cs | 1 + 8 files changed, 37 insertions(+), 8 deletions(-) diff --git a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs index 04c01047..4ae116f1 100644 --- a/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs +++ b/src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs @@ -58,6 +58,9 @@ public static class DriverTypeNames /// MQTT / Sparkplug B broker-subscription driver. public const string Mqtt = "Mqtt"; + /// MTConnect Agent driver — read-only HTTP/XML over an Agent's probe/current/sample surface. + public const string MTConnect = "MTConnect"; + /// /// Every driver-type string declared above, for callers that need to enumerate /// the full set (e.g. validation of an authored DriverType). @@ -75,5 +78,6 @@ public static class DriverTypeNames Calculation, Sql, Mqtt, + MTConnect, ]; } diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs index 6b6c4f77..babf65b2 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs @@ -386,7 +386,12 @@ public sealed class MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDis public string DriverInstanceId => _driverInstanceId; /// - public string DriverType => "MTConnect"; + /// + /// Sourced from the constant rather than a literal — the + /// repo-wide fix for the historical TwinCat/Focas drift, where a driver's own + /// spelling diverged from the constant every dispatch map keys off. + /// + public string DriverType => DriverTypeNames.MTConnect; /// /// The live dataItemId → snapshot map that backs the subscription surface: diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs index 34797fa7..40faa2bd 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs @@ -45,13 +45,16 @@ public static class MTConnectDriverFactoryExtensions /// The DriverInstance.DriverType value this factory answers to. /// /// - /// Kept identical to — the registry key and the - /// instance's self-reported type must agree or the runtime would look the driver up under a - /// name it never registered. Task 16 adds the matching DriverTypeNames.MTConnect - /// constant (the repo's convention is that dispatch maps key off those constants) and wires - /// the Host registration + guard test; this literal is what that constant must equal. + /// Aliased to , not a literal. The registry key, + /// the instance's self-reported , the probe's + /// DriverType, and the constant every dispatch map keys off are therefore the same + /// symbol — the repo-wide fix for the historical TwinCat/Focas drift, where a + /// driver's own spelling silently diverged from the constant and dispatch maps missed it. + /// Retained as a named const (rather than deleted in favour of the constant) so the existing + /// factory callers and the sibling *DriverFactoryExtensions.DriverTypeName convention + /// keep working; it is now an alias with no independent value. /// - public const string DriverTypeName = "MTConnect"; + public const string DriverTypeName = DriverTypeNames.MTConnect; /// /// Register the MTConnect factory with the driver registry. The optional diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs index a6afb152..7768ec7b 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs @@ -86,7 +86,12 @@ public sealed class MTConnectDriverProbe : IDriverProbe } /// - public string DriverType => "MTConnect"; + /// + /// Sourced from the constant, not a literal — the probe is + /// looked up by AdminOperationsActor under this key, so a drift from the constant + /// silently disables the Test Connect button rather than failing anything at build time. + /// + public string DriverType => DriverTypeNames.MTConnect; /// Minimal probe-only config shape. Deliberately independent of the factory DTO — see the type remarks. private sealed class MTConnectProbeConfigDto diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs index d6fc6d1a..3b743031 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs @@ -20,6 +20,7 @@ using GalaxyProbe = Driver.Galaxy.GalaxyDriverProbe; using CalculationProbe = Driver.Calculation.CalculationDriverProbe; using SqlProbe = Driver.Sql.SqlDriverProbe; using MqttProbe = Driver.Mqtt.MqttDriverProbe; +using MTConnectProbe = Driver.MTConnect.MTConnectDriverProbe; /// /// Wires every cross-platform driver assembly's Register(registry, loggerFactory) @@ -126,6 +127,7 @@ public static class DriverFactoryBootstrap services.TryAddEnumerable(ServiceDescriptor.Singleton()); services.TryAddEnumerable(ServiceDescriptor.Singleton()); services.TryAddEnumerable(ServiceDescriptor.Singleton()); + services.TryAddEnumerable(ServiceDescriptor.Singleton()); return services; } @@ -149,6 +151,13 @@ public static class DriverFactoryBootstrap Driver.Galaxy.GalaxyDriverFactoryExtensions.Register(registry, secretResolver, loggerFactory); Driver.Modbus.ModbusDriverFactoryExtensions.Register(registry, loggerFactory); Driver.Mqtt.MqttDriverFactoryExtensions.Register(registry, loggerFactory); + // Tier A (the Register default): fully managed — HttpClient + System.Xml.Linq, no native SDK, + // no COM. Tier C is the only tier that arms process-level recycle (MemoryRecycle hard-breach / + // ScheduledRecycleScheduler), which would be wrong here: the driver is in-process and killing + // it kills every OPC UA session. The long-lived /sample stream does not argue for a slower + // tier either — SubscribeAsync returns synchronously after starting the pump on a background + // task, so the Tier A 5s Subscribe budget never covers the stream's lifetime. + Driver.MTConnect.MTConnectDriverFactoryExtensions.Register(registry, loggerFactory); Driver.OpcUaClient.OpcUaClientDriverFactoryExtensions.Register(registry, loggerFactory, secretResolver); Driver.S7.S7DriverFactoryExtensions.Register(registry); Driver.Sql.SqlDriverFactoryExtensions.Register(registry, loggerFactory); diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj b/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj index bccc5685..1819d602 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj @@ -75,6 +75,7 @@ + diff --git a/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj b/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj index c9be0506..c8ed6243 100644 --- a/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj +++ b/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj @@ -38,6 +38,7 @@ + diff --git a/tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/DriverProbeRegistrationTests.cs b/tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/DriverProbeRegistrationTests.cs index c2e74697..61d4a148 100644 --- a/tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/DriverProbeRegistrationTests.cs +++ b/tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests/DriverProbeRegistrationTests.cs @@ -32,6 +32,7 @@ public sealed class DriverProbeRegistrationTests "OpcUaClient", "GalaxyMxGateway", "Calculation", // v3 Batch 2 pseudo-driver; CalculationProbe registered in DriverFactoryBootstrap + "MTConnect", // MTConnectProbe — read-only Agent driver; probe issues a bare GET /probe ]; [Fact] -- 2.52.0 From c232a3ce674a509c2e24ec31982136cf9343c544 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:49:23 -0400 Subject: [PATCH 25/34] fix(mtconnect): cap CancelAfter overflow + stop leaking ex.Message in probe fallback (review follow-up) Two review findings on Task 14's MTConnectDriverProbe, both red-first: - CancelAfter(effectiveTimeout) sat before the try block with only a low-end floor on a non-positive timeout, never a high-end cap. A pathological timeout (e.g. TimeSpan.FromDays(1000)) threw an uncaught ArgumentOutOfRangeException straight out of CancelAfter's ~49.7-day legal range, violating the probe's own "Never throws" contract. Not reachable through today's only caller (AdminOperationsActor clamps to [1,60]s and wraps the call), but a landmine for any future caller without that clamp. Fixed by clamping effectiveTimeout to a new MaxTimeout (10 minutes). - The final `catch (Exception ex) { return new(false, ex.Message, null); }` forwarded ex.Message verbatim for any exception type not enumerated above it, breaking the redaction discipline every other catch was audited for (AgentUri may carry userinfo credentials). Not exploitable by any currently-reachable exception type, but an open door for a future one. Fixed to build a message from the already-redacted safeUri + the exception's type name instead of its message. Also: a genuine non-2xx stub-handler test (prior HttpRequestException coverage was connection-refused only), a credentialed-agentUri + successful-probe redaction test, and a one-line doc remark on the accepted reachable-vs-deployable gap. --- .../MTConnectDriverProbe.cs | 36 +++++++- .../MTConnectDriverProbeTests.cs | 84 +++++++++++++++++++ 2 files changed, 118 insertions(+), 2 deletions(-) diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs index 7768ec7b..85658b9a 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverProbe.cs @@ -23,6 +23,14 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect; /// (MTConnectDriverFactoryExtensions), which is authored separately (Task 15). /// /// +/// Accepted gap: reachable ≠ deployable. Because only agentUri is read, a +/// config can show green on Test Connect and still fail to deploy — e.g. a non-positive +/// requestTimeoutMs or a malformed tag entry the factory's full parse would reject. +/// This is the direct consequence of the independent-parsing decision above, judged +/// acceptable because Test Connect's job is "is the endpoint there", not "will this exact +/// config deploy". +/// +/// /// Deliberately DOES reuse and /// for the actual round trip. Hand-rolling the HTTP /// call would have to re-derive the exact behaviour the client already provides: per-call @@ -63,6 +71,18 @@ public sealed class MTConnectDriverProbe : IDriverProbe /// Replaces a non-positive caller timeout — see the type remarks. private static readonly TimeSpan FallbackTimeout = TimeSpan.FromSeconds(5); + /// + /// Caps an excessive caller timeout. CancellationTokenSource.CancelAfter(TimeSpan)'s + /// legal range tops out near ~49.7 days ( ms minus one); an + /// uncapped pathological value (e.g. a caller passing TimeSpan.FromDays(1000)) throws + /// straight out of + /// CancelAfter, which would violate the "Never throws" contract just as surely as a + /// non-positive timeout meaning "wait forever" would (arch-review 01/S-6 — this is the same + /// defect class, the high end rather than the low end). 10 minutes is generous for a + /// reachability probe and leaves an enormous margin under the legal ceiling. + /// + private static readonly TimeSpan MaxTimeout = TimeSpan.FromMinutes(10); + private static readonly JsonSerializerOptions JsonOptions = new() { PropertyNameCaseInsensitive = true, @@ -131,8 +151,15 @@ public sealed class MTConnectDriverProbe : IDriverProbe var safeUri = RedactedDisplay(parsed); - // arch-review 01/S-6: a non-positive timeout must never mean "wait forever". + // arch-review 01/S-6: a non-positive timeout must never mean "wait forever" (low end), and a + // pathologically large one must never overrun CancelAfter's legal range (high end) — see + // MaxTimeout. var effectiveTimeout = timeout > TimeSpan.Zero ? timeout : FallbackTimeout; + if (effectiveTimeout > MaxTimeout) + { + effectiveTimeout = MaxTimeout; + } + var requestTimeoutMs = Math.Max(1, (int)Math.Ceiling(effectiveTimeout.TotalMilliseconds)); var sw = Stopwatch.StartNew(); @@ -187,7 +214,12 @@ public sealed class MTConnectDriverProbe : IDriverProbe } catch (Exception ex) { - return new(false, ex.Message, null); + // Last-resort catch for an exception type none of the above enumerate. Deliberately does + // NOT forward ex.Message: unlike the typed catches above (each individually audited for + // whether its message can carry the raw AgentUri/userinfo), an unenumerated type has no + // such audit, so ex.Message is an open door for a credential leak. safeUri + the + // exception's TYPE name is enough for an operator to act on without that risk. + return new(false, $"Probe against {safeUri} failed unexpectedly ({ex.GetType().Name}).", null); } } diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs index 8e25ba29..03ce5781 100644 --- a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverProbeTests.cs @@ -232,4 +232,88 @@ public sealed class MTConnectDriverProbeTests /// Local stand-in so the stub handler's "never returns" branch has a throw statement to satisfy flow analysis. private sealed class UnreachableException : Exception; + + // ── review follow-up: pathological (out-of-CancelAfter-range) timeout ──── + + /// + /// Reproduces the reviewer's finding: CancelAfter's legal range tops out near ~49.7 + /// days, and the caller-supplied timeout reached it uncapped (only floored on the low + /// end). A caller with no clamp of its own — unlike today's only caller, + /// AdminOperationsActor, which clamps to [1,60]s — must still get a + /// , never an escaping + /// : that is the whole point of the "Never throws" + /// contract. + /// + [Fact] + public async Task Pathologically_large_timeout_does_not_throw() + { + var probe = new MTConnectDriverProbe(); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://127.0.0.1:1/\"}", TimeSpan.FromDays(1000), CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + } + + // ── review follow-up: unenumerated exception type must not leak ex.Message ── + + /// Message text crafted to look like it embedded a credentialed URI, exactly what an unaudited ex.Message could leak. + private sealed class LeakyException() + : Exception("dial failed for http://leaky-user:leaky-pass@internal-host/probe (boom)"); + + /// + /// Reproduces the reviewer's finding: the final catch (Exception ex) forwarded + /// ex.Message verbatim for any type not enumerated above it, breaking the redaction + /// discipline every other catch was audited for. No currently-reachable exception type + /// exploits this, but nothing stopped a future one from doing so either — this test closes + /// that open door by asserting the final catch never echoes exception text. + /// + [Fact] + public async Task Unenumerated_exception_type_does_not_leak_ex_Message_via_final_catch() + { + var probe = ProbeWithHandler((_, _) => Task.FromException(new LeakyException())); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldNotContain("leaky-pass"); + r.Message.ShouldNotContain("leaky-user:leaky-pass"); + } + + // ── minor: genuine non-2xx status (not just connection-refused) ────────── + + [Fact] + public async Task NonSuccess_status_code_returns_not_ok() + { + var probe = ProbeWithHandler((_, _) => Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound) + { + Content = new StringContent(string.Empty), + })); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeFalse(); + r.Message.ShouldNotBeNull(); + r.Latency.ShouldBeNull(); + } + + // ── minor: credentialed agentUri + a SUCCESSFUL response must still redact ── + + [Fact] + public async Task Credentialed_agentUri_is_redacted_even_on_a_successful_probe() + { + var probe = ProbeWithHandler((_, _) => Task.FromResult(XmlResponse(ValidProbeXml))); + + var r = await probe.ProbeAsync( + "{\"agentUri\":\"http://stub-user:stub-pass@stub-agent/\"}", QuickTimeout, CancellationToken.None); + + r.Ok.ShouldBeTrue(); + r.Message.ShouldNotBeNull(); + r.Message.ShouldNotContain("stub-pass"); + r.Message.ShouldNotContain("stub-user:stub-pass"); + } } -- 2.52.0 From b0046d6959c9752cbcd1014360c91838514506f6 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 17:56:22 -0400 Subject: [PATCH 26/34] feat(mtconnect): typed AdminUI tag-config model (Task 17) MTConnectTagConfigModel: pure FromJson/ToJson/Validate for the MTConnect tag TagConfig JSON blob, mirroring ModbusTagConfigModel/OpcUaClientTagConfigModel. Fields: fullName (DataItem@id, required), dataType (DriverDataType override, written as an enum name string), mtCategory/mtType/mtSubType/units (probe metadata), mtDevice/mtComponent (author context). Preserves unknown JSON keys across load->save (isHistorized/historianTagname and anything else). Guards against the enum-serialization trap (numeric dataType would fault the driver at deploy) both on write (always TagConfigJson.Set via enum name) and on read (Validate() rejects a dataType that was stored as a bare digit string, since Enum.TryParse silently accepts numeric text). AdminUI.csproj gains a ProjectReference to Driver.MTConnect.Contracts, matching the existing POCO-only driver-Contracts pattern used by the other six typed tag editors. Scope: pure model only (Task 17). The razor editor shell + TagConfigEditorMap/ TagConfigValidator registration is Task 18. --- .../Uns/TagEditors/MTConnectTagConfigModel.cs | 130 +++++++++++++ .../ZB.MOM.WW.OtOpcUa.AdminUI.csproj | 1 + .../Uns/MTConnectTagConfigModelTests.cs | 179 ++++++++++++++++++ 3 files changed, 310 insertions(+) create mode 100644 src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs create mode 100644 tests/Server/ZB.MOM.WW.OtOpcUa.AdminUI.Tests/Uns/MTConnectTagConfigModelTests.cs diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs new file mode 100644 index 00000000..20bb263a --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MTConnectTagConfigModel.cs @@ -0,0 +1,130 @@ +using System.Text.Json.Nodes; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors; + +/// Typed working model for an MTConnect tag's TagConfig JSON. The tag binds a single Agent +/// DataItem by its id attribute ( — MTConnect is read-only, so +/// there is no address encoding beyond the id); the remaining fields are probe-sourced display +/// metadata plus an author-settable type override. Preserves unrecognised JSON keys across a +/// load→save. +/// +/// +/// /// come +/// from the Agent's /probe response (see MTConnectTagDefinition in the driver's +/// Contracts project) and are shown read-only by Task 18's editor — this model still round-trips +/// them so a browse-commit that stamped them survives a manual edit of . +/// / are pure author context (which physical +/// device/component this DataItem belongs to); nothing downstream consumes them. +/// +/// +/// is an override of the driver's own probe-based inference +/// (MTConnectDataTypeInference.Infer, which the browse-commit path already applied when it +/// wrote and this field). It is deliberately NOT recomputed here — this +/// editor only re-serialises whatever the operator picks or the browse commit stamped, never a +/// locally-reinvented inference that could disagree with the one true rule. +/// +/// +public sealed class MTConnectTagConfigModel +{ + /// The MTConnect DataItem's id attribute — the driver's read/subscribe key. Required. + public string FullName { get; set; } = ""; + + /// Logical data type override for the DataItem's value. Defaults to + /// — MTConnect is weakly typed on the wire, so an unset override should never coerce to a numeric type + /// that can fail to parse (mirrors MTConnectDataTypeInference's own String-leaning default). + public DriverDataType DataType { get; set; } = DriverDataType.String; + + /// The DataItem's probe-sourced category (SAMPLE/EVENT/CONDITION). Read-only display metadata. + public string? MtCategory { get; set; } + + /// The DataItem's probe-sourced type (e.g. POSITION, EXECUTION). Read-only display metadata. + public string? MtType { get; set; } + + /// The DataItem's probe-sourced subType (e.g. ACTUAL, COMMANDED). Read-only display metadata. + public string? MtSubType { get; set; } + + /// The DataItem's probe-sourced units (e.g. MILLIMETER). Read-only display metadata. + public string? Units { get; set; } + + /// Author-entered note identifying the owning MTConnect device. Pure authoring context; not + /// consumed by the driver or the runtime. + public string? MtDevice { get; set; } + + /// Author-entered note identifying the owning MTConnect component. Pure authoring context; not + /// consumed by the driver or the runtime. + public string? MtComponent { get; set; } + + // The as-loaded raw "dataType" JSON value, kept only to detect the ParseEnum numeric-text trap in + // Validate() below — Enum.TryParse (which TagConfigJson.GetEnum uses) silently accepts a quoted + // number ("8" -> Float64) exactly as readily as a name, so a config that was ever hand-authored or + // migrated with a numeric dataType would otherwise load into a *valid-looking* enum value with no + // signal that it isn't a name. ToJson() always re-writes DataType as a name, so this can only ever + // be non-null on the FIRST load of an already-poisoned config, never after a save through this model. + private string? _rawDataType; + + private JsonObject _bag = new(); + + /// Loads a model from a TagConfig JSON string, defaulting any absent field and retaining + /// every original key (so fields this editor doesn't expose survive a load→save). + /// The raw TagConfig JSON string, or null for a new/empty config. + /// The populated . + public static MTConnectTagConfigModel FromJson(string? json) + { + var o = TagConfigJson.ParseOrNew(json); + return new MTConnectTagConfigModel + { + FullName = TagConfigJson.GetString(o, "fullName") ?? "", + DataType = TagConfigJson.GetEnum(o, "dataType", DriverDataType.String), + MtCategory = TagConfigJson.GetString(o, "mtCategory"), + MtType = TagConfigJson.GetString(o, "mtType"), + MtSubType = TagConfigJson.GetString(o, "mtSubType"), + Units = TagConfigJson.GetString(o, "units"), + MtDevice = TagConfigJson.GetString(o, "mtDevice"), + MtComponent = TagConfigJson.GetString(o, "mtComponent"), + _rawDataType = TagConfigJson.GetString(o, "dataType"), + _bag = o, + }; + } + + /// Serialises this model back to a TagConfig JSON string over the preserved key bag. + /// dataType is always written as its enum NAME (never a bare number — see the + /// enum-serialization trap in the class remarks); the optional metadata keys are omitted when + /// blank rather than persisted as empty strings. + /// The serialised TagConfig JSON string. + public string ToJson() + { + TagConfigJson.Set(_bag, "fullName", FullName.Trim()); + TagConfigJson.Set(_bag, "dataType", DataType); + TagConfigJson.Set(_bag, "mtCategory", Blank(MtCategory)); + TagConfigJson.Set(_bag, "mtType", Blank(MtType)); + TagConfigJson.Set(_bag, "mtSubType", Blank(MtSubType)); + TagConfigJson.Set(_bag, "units", Blank(Units)); + TagConfigJson.Set(_bag, "mtDevice", Blank(MtDevice)); + TagConfigJson.Set(_bag, "mtComponent", Blank(MtComponent)); + return TagConfigJson.Serialize(_bag); + } + + /// Validation hook; returns an error message or null when the model is valid. + /// An error message describing the validation failure, or null when the model is valid. + public string? Validate() + { + if (string.IsNullOrWhiteSpace(FullName)) + { + return "A DataItem id (fullName) is required."; + } + + // See the _rawDataType remarks: a quoted number parses as a "valid" enum today but is never + // something this editor itself would have written, so surface it instead of silently accepting it. + if (_rawDataType is { Length: > 0 } raw && raw.All(char.IsAsciiDigit)) + { + return $"dataType \"{raw}\" is a numeric value, not a named type — re-select a data type."; + } + + return null; + } + + // Normalises a blank/whitespace-only string to null so TagConfigJson.Set omits the key rather than + // persisting an empty string. + private static string? Blank(string? value) => string.IsNullOrWhiteSpace(value) ? null : value; +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj index 01e0e8b3..a81f6b25 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj @@ -37,6 +37,7 @@ + + +
+ + + + MTConnect integration-test fixture + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md new file mode 100644 index 00000000..d9fa1e8c --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md @@ -0,0 +1,105 @@ +# MTConnect integration-test fixture — the official C++ Agent + an SHDR data source + +The MTConnect C++ Agent (`mtconnect/agent`, pinned) serving a canned device model, fed live +data by a standard-library SHDR adapter. No image build step: both services run stock images +with this folder's files bind-mounted. + +> **The published image is `mtconnect/agent`, not `mtconnect/cppagent`.** `cppagent` is the +> name of the source project on GitHub; there is no Docker Hub repository under that name. + +| File | Purpose | +|---|---| +| [`docker-compose.yml`](docker-compose.yml) | Two services: `agent` (published on :5000) and `adapter` (internal, :7878) | +| [`agent.cfg`](agent.cfg) | Agent configuration, bind-mounted at `/mtconnect/config/agent.cfg` (the image's CMD path) | +| [`Devices.xml`](Devices.xml) | The canned device model the Agent serves from `/probe` | +| [`adapter.py`](adapter.py) | SHDR feeder — the data source. Pure stdlib, runs on a stock `python:*-alpine` | + +## Why there is an adapter service + +The `mtconnect/agent` image ships **only** the agent binary plus its schemas and styles — there +is no bundled simulator. An Agent with no adapter answers `/probe` correctly and then reports +**every observation `UNAVAILABLE` forever**, which cannot prove that reads return real values or +that the `/sample` long poll delivers anything. `adapter.py` is what makes the fixture live. + +The Agent dials **out** to the adapter (see the `Adapters` block in `agent.cfg`); the adapter is +not published to the host. + +## Run + +From the shared Docker host (stack dir `/opt/otopcua-mtconnect`): + +```bash +docker compose up -d --wait +docker compose logs -f agent +docker compose down +``` + +From a dev box via the helper (see CLAUDE.md "Docker Workflow"): + +```powershell +lmxopcua-fix sync mtconnect # push this folder to /opt/otopcua-mtconnect/ +lmxopcua-fix up mtconnect # single-service-shaped stack, no profile argument +lmxopcua-fix logs mtconnect +lmxopcua-fix down mtconnect +``` + +### Running it on a Mac + +macOS **squats port 5000** — AirPlay Receiver (ControlCenter) binds `*:5000` and wins the race, +so `docker port` reports a healthy publish while every request answers `403 Forbidden` with +`Server: AirTunes`. Use the host-port override: + +```bash +MTCONNECT_AGENT_HOST_PORT=5555 docker compose up -d --wait +MTCONNECT_AGENT_ENDPOINT=http://127.0.0.1:5555 dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests +``` + +## Endpoint + +- Default: `http://10.100.0.35:5000` (the shared Docker host; 5000 is the Agent's own default port). +- Override with `MTCONNECT_AGENT_ENDPOINT` to point at a real Agent on a machine tool. +- `MTCONNECT_AGENT_HOST_PORT` changes only the **published host port** of the fixture container. + +`MTConnectAgentFixture` issues one `GET {endpoint}/probe` at collection init and records a +`SkipReason` when it fails, so the suite skips cleanly on a box with no fixture running. + +## The seeded device model + +Every named DataItem deliberately has `name != id` — the inverse of the driver's hand-authored +unit fixtures, and the only arrangement under which confusing the browse name with the +observation correlation key is visible. + +| DataItem `id` | `name` | Category | Type | Repr. | Inferred type | +|---|---|---|---|---|---| +| `fixture_avail` | `Favail` | EVENT | AVAILABILITY | | String | +| `fixture_x_pos` | `Xact` | SAMPLE | POSITION (ACTUAL, MILLIMETER) | | Float64 — **moves** | +| `fixture_x_load` | `Xload` | SAMPLE | LOAD (PERCENT) | | Float64 — **moves** | +| `fixture_x_travel` | `Xtravel` | CONDITION | POSITION | | String + IsAlarm | +| `fixture_c_speed` | `Cspeed` | SAMPLE | ROTARY_VELOCITY (REVOLUTION/MINUTE) | | Float64 — **moves** | +| `fixture_c_temp_series` | `Ctemps` | SAMPLE | TEMPERATURE (CELSIUS) | TIME_SERIES | Float64 **array**, ArrayDim `null` | +| `fixture_mode` | `Cmode` | EVENT | CONTROLLER_MODE | | String | +| `fixture_execution` | `Pexec` | EVENT | EXECUTION | | String | +| `fixture_partcount` | `Pcount` | EVENT | **PART_COUNT** | | **Int64** — the regression case | +| `fixture_linenumber` | `Pline` | EVENT | LINE_NUMBER | | Int64 | +| `fixture_program` | `Pprogram` | EVENT | PROGRAM | | String | +| `fixture_block` | `Pblock` | EVENT | BLOCK | | String — **never fed ⇒ UNAVAILABLE** | +| `fixture_varset` | `Pvars` | EVENT | VARIABLE | DATA_SET | String (structured ⇒ BadNotSupported) | +| `fixture_logic` | `Plogic` | CONDITION | LOGIC_PROGRAM | | String + IsAlarm | +| `fixture_asset_changed` / `fixture_asset_removed` | — | EVENT | ASSET_CHANGED / ASSET_REMOVED | | String | + +The Agent additionally injects **its own `` self-model device** (connection status, +observation update rate, adapter URI). That is normal for every MTConnect 2.x Agent and the test +suite excludes it from value-plane assertions — its update-rate samples tick whether or not any +adapter is attached, so including them would let "the stream delivered a changed value" pass +against an Agent with no data source at all. + +## Two things a real Agent taught us (both cost a fixture restart to find) + +1. **`agent.cfg` must be pure ASCII.** A single non-ASCII byte — even inside a comment — makes + the config parser reject the whole file with a bare `Failed / Stopped at line: N` and exit. +2. **`sampleCount` is not a DataItem attribute.** It belongs to a TIME_SERIES *observation*. An + Agent that meets it on a declaration logs + `The following keys were present and not expected: sampleCount` followed by + `DataItems: Invalid element 'DataItem'` and **drops the entire data item** from the device + model. `MTConnectDataTypeInference` therefore only ever sees `sampleCount = null` from a real + Agent, so a live TIME_SERIES tag is always a variable-length array. diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/adapter.py b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/adapter.py new file mode 100644 index 00000000..06019cc3 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/adapter.py @@ -0,0 +1,229 @@ +#!/usr/bin/env python3 +"""SHDR adapter feeding the OtOpcUa MTConnect integration-test fixture live data. + +The mtconnect/agent image ships *only* the agent binary plus its schemas and styles -- +there is no bundled simulator. Without an adapter the Agent answers /probe correctly but +reports every observation UNAVAILABLE forever, which cannot prove the two things this +fixture exists for: that ReadAsync returns real coerced values, and that the driver's +/sample long poll delivers OnDataChange from a genuinely moving stream. + +So this is the data source. It speaks SHDR (the Agent's plain-text adapter protocol) over +TCP: the Agent dials IN as the client (see agent.cfg's Adapters block), and every line we +write is `||[||...]`, where matches a DataItem's +`name` attribute in Devices.xml (falling back to its `id` for the unnamed ones). + +Deliberate behaviours -- each one is asserted on by the integration suite: + + * Xact / Xload / Cspeed move on EVERY tick. "The value changed between two reads" is the + only assertion that distinguishes a live stream from a cached /current snapshot. + * Pcount (PART_COUNT) increments on a slower cadence, so it is both a *changing* value + and an integer -- the live half of the UPPER_SNAKE PART_COUNT -> Int64 inference. + * Pblock is NEVER written. The Agent therefore reports it UNAVAILABLE for the life of + the fixture, which is what exercises UNAVAILABLE -> BadNoCommunication. + * Conditions are re-asserted periodically rather than once at connect, so a driver that + attaches mid-run still sees them. + +Pure standard library on purpose: the container is a stock `python:*-alpine` with this +file bind-mounted, so the fixture needs no image build step at all (unlike the Modbus / +S7 fixtures, whose simulators do). +""" + +from __future__ import annotations + +import argparse +import datetime +import math +import selectors +import socket +import sys +import threading +import time + +# How often the periodic feed writes a batch of observations. Comfortably faster than the +# driver's default SampleIntervalMs (1000) so a subscription sees several distinct chunks +# inside a short test timeout. +TICK_SECONDS = 0.25 + +# The Agent's PING/PONG watchdog: it sends `* PING` and expects `* PONG ` back. The +# number is how long the Agent should wait before declaring us dead. +PONG_TIMEOUT_MS = 10000 + +EXECUTION_STATES = ("ACTIVE", "READY", "INTERRUPTED", "ACTIVE", "STOPPED") +CONTROLLER_MODES = ("AUTOMATIC", "MANUAL", "AUTOMATIC", "SEMI_AUTOMATIC") + + +def timestamp() -> str: + """UTC in the ISO-8601 form the Agent expects (milliseconds, trailing Z).""" + now = datetime.datetime.now(datetime.timezone.utc) + + return now.strftime("%Y-%m-%dT%H:%M:%S.") + f"{now.microsecond // 1000:03d}Z" + + +class Feed: + """One connected Agent. Owns the socket for the life of that connection.""" + + def __init__(self, conn: socket.socket, peer: str) -> None: + self._conn = conn + self._peer = peer + self._started = time.monotonic() + self._ticks = 0 + + # ---- wire ---- + + def send(self, line: str) -> None: + self._conn.sendall((line + "\n").encode("utf-8")) + + def send_observations(self, pairs: list[tuple[str, str]]) -> None: + """One SHDR line carrying several key/value pairs, all sharing one timestamp.""" + if not pairs: + return + + body = "|".join(f"{key}|{value}" for key, value in pairs) + self.send(f"{timestamp()}|{body}") + + # ---- content ---- + + def send_initial_state(self) -> None: + """Everything that is not periodic, sent once as soon as the Agent attaches.""" + self.send_observations( + [ + ("Favail", "AVAILABLE"), + ("Pprogram", "OTOPCUA-FIXTURE.NC"), + # DATA_SET representation: the Agent renders these as children, + # which is precisely the wire shape the driver demotes to String / BadNotSupported. + ("Pvars", "toolNumber=7 offsetX=1.25 offsetZ=-0.5"), + ] + ) + self.send_conditions() + + def send_conditions(self) -> None: + """SHDR condition form: |||||.""" + stamp = timestamp() + self.send(f"{stamp}|Xtravel|normal||||") + self.send(f"{stamp}|Plogic|normal||||") + + def tick(self) -> None: + """One periodic batch. Called every TICK_SECONDS.""" + self._ticks += 1 + elapsed = time.monotonic() - self._started + + # Continuously-moving SAMPLEs. Rounded to 4 dp so the value is a clean double on + # the wire and still differs between any two consecutive ticks. + position = round(120.0 + 40.0 * math.sin(elapsed / 2.0), 4) + load = round(45.0 + 15.0 * math.sin(elapsed / 3.7), 4) + speed = round(1500.0 + 250.0 * math.sin(elapsed / 5.0), 4) + + pairs: list[tuple[str, str]] = [ + ("Xact", f"{position}"), + ("Xload", f"{load}"), + ("Cspeed", f"{speed}"), + ] + + # PART_COUNT: an integer EVENT that also moves. Slower than the samples so it reads + # like a real counter rather than a signal. + pairs.append(("Pcount", str(100 + int(elapsed // 5)))) + pairs.append(("Pline", str(10 + (self._ticks % 90)))) + + self.send_observations(pairs) + + # TIME_SERIES form: ||| ... + series = " ".join( + f"{round(21.5 + 0.5 * math.sin(elapsed + i / 4.0), 3)}" for i in range(8) + ) + self.send(f"{timestamp()}|Ctemps|8|100|{series}") + + # Controlled-vocabulary EVENTs, on their own slow cadences. + if self._ticks % 28 == 1: + self.send_observations( + [("Pexec", EXECUTION_STATES[(self._ticks // 28) % len(EXECUTION_STATES)])] + ) + if self._ticks % 44 == 1: + self.send_observations( + [("Cmode", CONTROLLER_MODES[(self._ticks // 44) % len(CONTROLLER_MODES)])] + ) + + # Re-assert conditions occasionally so a late-attaching driver still sees them. + if self._ticks % 60 == 0: + self.send_conditions() + + # NOTE: Pblock is never written, on purpose. See the module docstring. + + +def handle(conn: socket.socket, peer: str) -> None: + feed = Feed(conn, peer) + print(f"[adapter] agent connected from {peer}", flush=True) + + selector = selectors.DefaultSelector() + selector.register(conn, selectors.EVENT_READ) + + pending = b"" + next_tick = time.monotonic() + + try: + feed.send_initial_state() + + while True: + now = time.monotonic() + if now >= next_tick: + feed.tick() + # Absolute schedule, not `now + TICK`: drift-free, and a slow tick cannot + # compound into an ever-widening gap the Agent reads as a stall. + next_tick += TICK_SECONDS + if next_tick < now: + next_tick = now + TICK_SECONDS + + for _ in selector.select(timeout=max(0.0, next_tick - time.monotonic())): + chunk = conn.recv(4096) + if not chunk: + print(f"[adapter] agent {peer} closed the connection", flush=True) + + return + + pending += chunk + while b"\n" in pending: + line, pending = pending.split(b"\n", 1) + text = line.decode("utf-8", errors="replace").strip() + # The Agent's heartbeat. Answering is mandatory: an Agent that gets no + # PONG tears the connection down and every observation goes UNAVAILABLE. + if text.startswith("* PING"): + feed.send(f"* PONG {PONG_TIMEOUT_MS}") + except (BrokenPipeError, ConnectionResetError, OSError) as ex: + print(f"[adapter] connection to {peer} ended: {ex}", flush=True) + finally: + selector.close() + try: + conn.close() + except OSError: + pass + + +def main() -> int: + parser = argparse.ArgumentParser(description="SHDR adapter for the OtOpcUa MTConnect fixture.") + parser.add_argument("--host", default="0.0.0.0", help="Bind address (default: all interfaces).") + parser.add_argument("--port", type=int, default=7878, help="SHDR listen port (default: 7878).") + args = parser.parse_args() + + listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + listener.bind((args.host, args.port)) + listener.listen(8) + print(f"[adapter] listening on {args.host}:{args.port}", flush=True) + + try: + while True: + conn, addr = listener.accept() + conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) + # One thread per Agent connection. The Agent reconnects after any restart, and + # a serialized accept loop would leave the new connection unserved while the + # dead one's socket was still being reaped. + threading.Thread( + target=handle, args=(conn, f"{addr[0]}:{addr[1]}"), daemon=True + ).start() + except KeyboardInterrupt: + return 0 + finally: + listener.close() + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg new file mode 100644 index 00000000..bbc22692 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg @@ -0,0 +1,50 @@ +# MTConnect C++ Agent configuration for the OtOpcUa integration-test fixture. +# +# ASCII ONLY. The Agent's config parser rejects the whole file on a non-ASCII byte -- +# even inside a comment -- with a bare "Failed / Stopped at line: N" and exits. An em +# dash in a comment is enough to make the fixture never start. +# +# The image's CMD is `/usr/bin/mtcagent run /mtconnect/config/agent.cfg`, so this file is +# bind-mounted at exactly that path (see docker-compose.yml). Paths below are ABSOLUTE on +# purpose: a relative `Devices` is resolved against the Agent's working directory +# (/home/agent), not against this file, and would silently start an Agent with no device +# model. + +Devices = /mtconnect/config/Devices.xml +Port = 5000 +ServiceName = OtOpcUaMTConnectFixture + +# 2^14 = 16384 observations retained. Generous on purpose: the driver's /sample pump +# re-baselines through /current when its cursor falls out of the Agent's ring buffer, and +# a small buffer would make the fixture exercise that recovery path on every run instead +# of the steady-state streaming the suite is here to prove. +BufferSize = 14 +CheckpointFrequency = 1000 +MaxAssets = 128 + +# MTConnect is a read-only source for this driver; the Agent must not accept writes. +AllowPut = false + +SchemaVersion = 2.0 +MonitorConfigFiles = false +Pretty = true + +# The block name must equal the Device name attribute in Devices.xml (OtFixtureCnc) -- +# that is how the Agent binds an adapter connection to a device. "adapter" is the compose +# service name of the SHDR feeder; the Agent dials OUT to it. +Adapters { + OtFixtureCnc { + Host = adapter + Port = 7878 + ReconnectInterval = 2000 + } +} + +# Log to stdout rather than a file: /mtconnect/log is an image-declared VOLUME that is +# created root-owned while the Agent runs as uid 1000, so file logging fails to open. +# stdout also puts adapter-connect / device-model errors straight into `docker logs`, +# which is how you diagnose a fixture that comes up but reports everything UNAVAILABLE. +logger_config { + logging_level = info + output = cout +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/docker-compose.yml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/docker-compose.yml new file mode 100644 index 00000000..547861dc --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/docker-compose.yml @@ -0,0 +1,79 @@ +# MTConnect integration-test fixture — the official MTConnect C++ Agent, fed live data by +# a standard-library SHDR adapter. +# +# TWO services, and both are required: +# +# agent mtconnect/agent (the cppagent image). Serves /probe, /current and /sample on +# :5000. NOTE the repository name — the source project is `mtconnect/cppagent` +# but the published image is `mtconnect/agent`; `mtconnect/cppagent` does not +# exist on Docker Hub. +# adapter The data source. The agent image ships only the agent binary plus schemas and +# styles — there is NO bundled simulator — so without this every observation is +# UNAVAILABLE forever and the suite could prove nothing about reads or streaming. +# The Agent dials OUT to it (agent.cfg's Adapters block); it is not published to +# the host. +# +# Why pinned: the `latest` tag moves and a fixture that silently changes device-model +# behaviour turns a driver regression into an unexplained red. Bump deliberately. +# +# Usage (from the docker host, stack dir /opt/otopcua-mtconnect): +# docker compose up -d --wait +# docker compose down +# +# Or from a dev box, via the helper (see CLAUDE.md "Docker Workflow"): +# lmxopcua-fix sync mtconnect +# lmxopcua-fix up mtconnect # single-service-shaped stack, no profile argument +services: + agent: + image: mtconnect/agent:2.7.0.12 + container_name: otopcua-mtconnect-agent + restart: "no" + labels: + project: lmxopcua + depends_on: + adapter: + condition: service_started + ports: + # Host port is overridable because macOS squats :5000 — AirPlay Receiver + # (ControlCenter) binds *:5000 and WINS the race, so `docker port` reports a healthy + # publish while every request answers "403 Forbidden / Server: AirTunes". On the + # Linux docker host the default is correct and needs no override. To run the fixture + # on a Mac: + # MTCONNECT_AGENT_HOST_PORT=5555 docker compose up -d --wait + # MTCONNECT_AGENT_ENDPOINT=http://127.0.0.1:5555 dotnet test ... + - "${MTCONNECT_AGENT_HOST_PORT:-5000}:5000" + volumes: + # Individual FILE binds, not a directory bind: /mtconnect/config is a VOLUME declared + # by the image, and mounting this whole folder over it would also drop docker-compose.yml + # and adapter.py into the Agent's config directory. + - ./agent.cfg:/mtconnect/config/agent.cfg:ro + - ./Devices.xml:/mtconnect/config/Devices.xml:ro + healthcheck: + # The image is alpine-based, so busybox wget is present (there is no curl). A 200 from + # /probe is the real readiness signal — the port accepts connections before the device + # model is parsed. + test: ["CMD-SHELL", "wget -q -O /dev/null http://127.0.0.1:5000/probe || exit 1"] + interval: 5s + timeout: 3s + retries: 12 + start_period: 5s + + adapter: + image: python:3.13-alpine + container_name: otopcua-mtconnect-adapter + restart: "no" + labels: + project: lmxopcua + volumes: + - ./adapter.py:/fixtures/adapter.py:ro + # Stock image + a bind-mounted script: this fixture needs no `build:` step at all, unlike + # the Modbus / S7 / AB fixtures whose simulators are built from a Dockerfile. + command: ["python", "-u", "/fixtures/adapter.py", "--host", "0.0.0.0", "--port", "7878"] + expose: + - "7878" + healthcheck: + test: ["CMD-SHELL", "python -c \"import socket; socket.create_connection(('127.0.0.1', 7878), timeout=2).close()\" || exit 1"] + interval: 5s + timeout: 3s + retries: 6 + start_period: 3s diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentFixture.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentFixture.cs new file mode 100644 index 00000000..dad972f8 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentFixture.cs @@ -0,0 +1,190 @@ +using System.Net.Http; +using System.Xml.Linq; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests; + +/// +/// Reachability probe for a live MTConnect Agent (the mtconnect/agent container in +/// Docker/docker-compose.yml, or a real Agent on a machine tool). Reads +/// MTCONNECT_AGENT_ENDPOINT (default http://10.100.0.35:5000 — the shared Docker +/// host) and issues ONE GET {endpoint}/probe at fixture construction. Each test checks +/// and calls Assert.Skip when the Agent was unreachable, so a +/// dev box with no fixture running still passes dotnet test cleanly — the same pattern +/// as ModbusSimulatorFixture / S7SimulatorFixture. +/// +/// +/// +/// The probe is an HTTP round-trip, not a TCP connect — unlike the Modbus and S7 +/// fixtures, whose protocols have nothing cheaper. An Agent's port accepts connections +/// before its device model is parsed, and a mis-authored Devices.xml produces an +/// Agent that answers TCP and then fails every request. A TCP-only probe would let the +/// whole suite run and fail confusingly rather than skip with an actionable message. +/// +/// +/// The probe response is kept () and is the source of +/// truth every assertion in this suite is written against. Nothing here may hard-code a +/// DataItem id: the seeded Devices.xml is expected to change, and a real Agent on a +/// real machine is a completely different model. Tests therefore say "for every DataItem +/// the Agent declares with category=EVENT and type PART_COUNT, the discovered variable must +/// be Int64" — a statement that survives any device model, including one with zero such +/// items (which self-skips rather than passing vacuously). +/// +/// +/// A collection fixture, so the probe runs once per session rather than once per +/// test: against a firewalled endpoint each attempt costs the full timeout. +/// +/// +public sealed class MTConnectAgentFixture : IAsyncDisposable +{ + /// The shared Docker host (see CLAUDE.md "Docker Workflow"); port 5000 is the Agent's own default. + private const string DefaultEndpoint = "http://10.100.0.35:5000"; + + private const string EndpointEnvVar = "MTCONNECT_AGENT_ENDPOINT"; + + /// + /// Bound on the one-shot reachability probe. Deliberately generous relative to a TCP probe: + /// a cold Agent parses its device model on the first request, and a large real-world model + /// can take a second or two to serialize. + /// + private static readonly TimeSpan ProbeTimeout = TimeSpan.FromSeconds(10); + + private static readonly string HowToStart = + $"Start the fixture (docker compose -f Docker/docker-compose.yml up -d --wait) or point " + + $"{EndpointEnvVar} at a live Agent, then re-run."; + + /// Gets the Agent's base URI, exactly as an authored agentUri would carry it. + public string AgentUri { get; } + + /// Gets the skip reason when the Agent is unreachable or unusable; otherwise null. + public string? SkipReason { get; } + + /// + /// Gets the Agent's own /probe response, parsed as XML — null exactly when + /// is set. Tests cross-reference discovery output against this so + /// their assertions describe the Agent's declared model rather than a hard-coded fixture. + /// + public XDocument? ProbeDocument { get; } + + /// Initializes a new instance of the class. + public MTConnectAgentFixture() + { + AgentUri = (Environment.GetEnvironmentVariable(EndpointEnvVar) ?? DefaultEndpoint).Trim().TrimEnd('/'); + + if (!Uri.TryCreate(AgentUri, UriKind.Absolute, out var parsed) || + (parsed.Scheme != Uri.UriSchemeHttp && parsed.Scheme != Uri.UriSchemeHttps)) + { + SkipReason = $"{EndpointEnvVar} ('{AgentUri}') is not an absolute http(s) URI. {HowToStart}"; + + return; + } + + try + { + using var http = new HttpClient { Timeout = ProbeTimeout }; + using var response = http.GetAsync($"{AgentUri}/probe").GetAwaiter().GetResult(); + + if (!response.IsSuccessStatusCode) + { + SkipReason = + $"MTConnect Agent at {AgentUri} answered /probe with HTTP {(int)response.StatusCode} " + + $"({response.ReasonPhrase}). {HowToStart}"; + + return; + } + + var body = response.Content.ReadAsStringAsync().GetAwaiter().GetResult(); + var document = XDocument.Parse(body); + + // An MTConnectError document (or anything else) served under a 2xx must skip, not run. + // The suite's whole value is asserting against a real device model; running it against + // a document that is not one produces confusing red rather than an honest skip. + if (document.Root is null || document.Root.Name.LocalName != "MTConnectDevices") + { + SkipReason = + $"MTConnect Agent at {AgentUri} answered /probe with a <{document.Root?.Name.LocalName ?? "?"}> " + + $"document, not . {HowToStart}"; + + return; + } + + ProbeDocument = document; + } + catch (Exception ex) + { + SkipReason = $"MTConnect Agent at {AgentUri} is unreachable: {ex.GetType().Name}: {ex.Message}. {HowToStart}"; + } + } + + /// + /// Gets every DataItem the Agent declares, flattened out of + /// with the attributes the type inference consumes. + /// + /// + /// Read straight off the XML with LINQ-to-XML rather than through the driver's own + /// MTConnectProbeParser: that parser is what several of these tests are checking, and + /// an assertion that ran the code under test to build its own expectation could only ever + /// prove the code agrees with itself. Matching is on local names because an MTConnect + /// document is namespaced and the namespace URI carries the schema version. + /// + public IReadOnlyList DeclaredDataItems => + ProbeDocument is null + ? [] + : [.. ProbeDocument.Descendants() + .Where(e => e.Name.LocalName == "DataItem") + .Select(e => new DeclaredDataItem( + Id: (string?)e.Attribute("id") ?? string.Empty, + Name: (string?)e.Attribute("name"), + Category: (string?)e.Attribute("category") ?? string.Empty, + Type: (string?)e.Attribute("type") ?? string.Empty, + Units: (string?)e.Attribute("units"), + Representation: (string?)e.Attribute("representation"), + InComposition: e.Ancestors().Any(a => a.Name.LocalName == "Compositions"), + InAgentSelfModel: e.Ancestors().Any(a => a.Name.LocalName == "Agent"))) + .Where(d => d.Id.Length > 0)]; + + /// + public ValueTask DisposeAsync() => ValueTask.CompletedTask; +} + +/// +/// One DataItem as the live Agent declares it in its /probe response — the +/// expectation side of every discovery assertion in this suite. +/// +/// The id attribute: the observation correlation key, and what discovery must stamp as FullName. +/// The name attribute when present: cosmetic, and what discovery must use as the browse name. +/// The category attribute — SAMPLE, EVENT, or CONDITION. +/// The type attribute in the probe document's UPPER_SNAKE spelling (PART_COUNT, not PartCount). +/// The units attribute when present. +/// The representation attribute when present (TIME_SERIES, DATA_SET, …). +/// +/// true when this DataItem is declared under a <Compositions> element rather +/// than under a component's own <DataItems>. The driver's discovery walk recurses +/// <Components> only, so a composition-owned item is legitimately absent from the +/// browse tree; assertions about full coverage of the device model must exclude these or they +/// become a false red on any Agent whose model uses compositions. +/// +/// +/// true when this DataItem belongs to the Agent's own <Agent> self-model +/// (connection status, observation update rate, adapter URI, …) rather than to a +/// <Device>. Every MTConnect 2.x Agent publishes one, and its update-rate SAMPLEs +/// move continuously whether or not any adapter is attached — so a "the stream delivered a +/// changed value" assertion that did not exclude these would pass against an Agent with no data +/// source at all, which is exactly the vacuous green this suite must not have. +/// +public sealed record DeclaredDataItem( + string Id, + string? Name, + string Category, + string Type, + string? Units, + string? Representation, + bool InComposition, + bool InAgentSelfModel); + +/// Collection definition so the reachability probe runs once per test session. +[Xunit.CollectionDefinition(Name)] +public sealed class MTConnectAgentCollection : Xunit.ICollectionFixture +{ + /// The collection name every test class in this suite joins. + public const string Name = "MTConnectAgent"; +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentIntegrationTests.cs b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentIntegrationTests.cs new file mode 100644 index 00000000..2e08a622 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentIntegrationTests.cs @@ -0,0 +1,697 @@ +using System.Collections.Concurrent; +using System.Globalization; +using System.Net.Http; +using System.Net.Sockets; +using System.Text.Json; +using System.Xml.Linq; +using Shouldly; +using Xunit; +using ZB.MOM.WW.OtOpcUa.Commons.Types; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests; + +/// +/// The MTConnect driver against a real MTConnect Agent (the mtconnect/agent +/// fixture in Docker/, or a live Agent on a machine tool via +/// MTCONNECT_AGENT_ENDPOINT). Everything else in the driver's test surface runs on canned +/// XML; this suite is the only thing that touches a socket. +/// +/// +/// +/// What only a live Agent can prove, and therefore what every test here is aimed at: +/// +/// +/// +/// +/// UPPER_SNAKE type spellings. A /probe document writes +/// DataItem@type as PART_COUNT; a streams document names the same +/// concept PartCount. An earlier revision of the inference matched only the +/// PascalCase spelling, typed every real Agent's part counter as +/// , and passed every unit test — because the +/// hand-authored fixtures used the streams spelling in the probe document too. +/// +/// +/// +/// +/// name differs from id. On a real Agent most DataItems carry +/// both, and they differ. The canned fixtures use ids that double as names, which is +/// the one arrangement under which confusing the two is invisible. +/// +/// +/// +/// +/// A real multipart/x-mixed-replace stream. The canned subscribe tests +/// drive a fake pump that hands the driver pre-parsed chunks; real boundaries, real +/// chunked transfer-encoding, and real heartbeats are a different thing entirely. +/// +/// +/// +/// +/// Assertions are structural, never by literal id. Every expectation is computed from +/// the Agent's OWN /probe response (), +/// so the suite survives an edit to the seeded Devices.xml and can be pointed at a +/// real machine tool. A test whose subject the Agent does not declare (no CONDITION, no +/// PART_COUNT) Assert.Skips rather than passing vacuously. +/// +/// +/// Two things this suite deliberately does NOT cover, with reasons. +/// (a) MTConnectError under HTTP 200. The driver handles it and canned tests +/// pin it, but a modern Agent cannot be made to produce it: mtconnect/agent 2.7 +/// answers an unknown device with 404 and an out-of-range sequence with 400, +/// each carrying a well-formed MTConnectError body (verified live). The under-200 +/// shape belongs to older/third-party Agents and stays canned-only. +/// (b) Writes. MTConnect is a read-only source protocol and the driver implements no +/// IWritable; the fixture Agent runs AllowPut = false to match. +/// +/// +[Collection(MTConnectAgentCollection.Name)] +[Trait("Category", "Integration")] +[Trait("Driver", "MTConnect")] +public sealed class MTConnectAgentIntegrationTests(MTConnectAgentFixture agent) +{ + /// + /// Hard per-test ceiling, in milliseconds. Every test also bounds its own awaits, but a + /// live-integration suite must never be able to wedge a build against a half-up fixture — + /// an Agent that completes its TCP handshake and then never answers would otherwise hang + /// inside the driver's own retry loop. + /// + private const int TestTimeoutMs = 90_000; + + /// Canonical Opc.Ua.StatusCodes numerics, restated so the assertion names the wire value a client sees. + private const uint Good = 0x00000000u; + + private const uint BadNoCommunication = 0x80310000u; + + /// How long a subscription test waits for the live /sample stream to move a value. + private static readonly TimeSpan StreamWait = TimeSpan.FromSeconds(45); + + private static CancellationToken Ct => TestContext.Current.CancellationToken; + + // ---- reachability ---- + + /// + /// The AdminUI Test Connect path against a live Agent: + /// must go green and report a latency. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Probe_reports_the_live_agent_reachable() + { + SkipIfDown(); + + var result = await new MTConnectDriverProbe() + .ProbeAsync(ProbeConfig(agent.AgentUri), TimeSpan.FromSeconds(15), Ct); + + result.Ok.ShouldBeTrue($"Test Connect must go green against the live Agent. Message: {result.Message}"); + result.Message.ShouldNotBeNull(); + result.Message.ShouldContain("/probe"); + result.Latency.ShouldNotBeNull(); + } + + /// + /// Falsifiability control for : the same + /// probe against a port nothing is listening on must go RED. Without this, a probe that + /// returned Ok = true unconditionally would satisfy the green test forever. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Probe_reports_a_dead_endpoint_as_unreachable() + { + SkipIfDown(); + + var result = await new MTConnectDriverProbe() + .ProbeAsync(ProbeConfig($"http://127.0.0.1:{ClosedLoopbackPort()}"), TimeSpan.FromSeconds(5), Ct); + + result.Ok.ShouldBeFalse("a probe against a closed port must fail, or the green probe proves nothing"); + result.Latency.ShouldBeNull(); + } + + // ---- discovery: the /probe device model ---- + + /// + /// DiscoverAsync streams a non-empty, genuinely nested tree, and every leaf in it is + /// a DataItem the Agent actually declares — no invented references, and nothing dropped + /// except the composition-owned items the walk is documented not to visit. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_streams_the_agents_declared_device_model() + { + SkipIfDown(); + + var capture = await DiscoverAsync(); + + capture.Variables.ShouldNotBeEmpty("a live Agent declares data items; an empty browse tree is the #485 shape"); + capture.Folders.ShouldNotBeEmpty(); + capture.Folders.ShouldContain( + f => f.ParentPath.Length > 0, + "the walk must recurse , not just emit one folder per device"); + + var declared = agent.DeclaredDataItems; + var declaredIds = declared.Select(d => d.Id).ToHashSet(StringComparer.Ordinal); + var discoveredIds = capture.Variables.Select(v => v.Attr.FullName).ToHashSet(StringComparer.Ordinal); + + discoveredIds.Except(declaredIds).ShouldBeEmpty("discovery must not invent references the Agent never declared"); + + var expected = declared.Where(d => !d.InComposition).Select(d => d.Id).ToHashSet(StringComparer.Ordinal); + expected.Except(discoveredIds).ShouldBeEmpty("every component-owned DataItem the Agent declares must be browsable"); + } + + /// + /// The regression this whole fixture exists for. An EVENT whose probe-document + /// type is the UPPER_SNAKE PART_COUNT / LINE_NUMBER must infer + /// . Matching only the streams document's PascalCase + /// PartCount types it — green in every unit test, + /// wrong on every real Agent. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_types_an_upper_snake_numeric_event_as_Int64() + { + SkipIfDown(); + + var numericEvents = agent.DeclaredDataItems + .Where(d => Is(d.Category, "EVENT")) + .Where(d => Is(d.Type, "PART_COUNT") || Is(d.Type, "LINE_NUMBER") || Is(d.Type, "LINE")) + .ToList(); + + if (numericEvents.Count == 0) + { + Assert.Skip( + $"The Agent at {agent.AgentUri} declares no PART_COUNT / LINE_NUMBER EVENT, so the " + + "UPPER_SNAKE numeric-event inference has nothing to assert against here."); + } + + var capture = await DiscoverAsync(); + + foreach (var item in numericEvents) + { + var variable = Leaf(capture, item.Id); + variable.Attr.DriverDataType.ShouldBe( + DriverDataType.Int64, + $"DataItem '{item.Id}' (category={item.Category}, type={item.Type}) is an integer EVENT; " + + "typing it String is the UPPER_SNAKE-vs-PascalCase defect this suite exists to catch"); + variable.Attr.IsArray.ShouldBeFalse(); + } + } + + /// + /// A SAMPLE is a measured quantity by definition of the category, so it must infer + /// — scalar for an ordinary sample, an array for a + /// TIME_SERIES. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_types_samples_as_Float64_and_time_series_as_a_float_array() + { + SkipIfDown(); + + var samples = agent.DeclaredDataItems + .Where(d => Is(d.Category, "SAMPLE") && !d.InComposition) + .Where(d => !Is(d.Representation, "DATA_SET") && !Is(d.Representation, "TABLE")) + .ToList(); + + if (samples.Count == 0) + { + Assert.Skip($"The Agent at {agent.AgentUri} declares no scalar/time-series SAMPLE data items."); + } + + var capture = await DiscoverAsync(); + + foreach (var item in samples) + { + var variable = Leaf(capture, item.Id); + variable.Attr.DriverDataType.ShouldBe( + DriverDataType.Float64, + $"DataItem '{item.Id}' is category=SAMPLE (type={item.Type}, units={item.Units ?? ""})"); + + if (Is(item.Representation, "TIME_SERIES")) + { + variable.Attr.IsArray.ShouldBeTrue($"'{item.Id}' declares representation=TIME_SERIES"); + + // Not a bug and not an oversight: `sampleCount` is an attribute of a TIME_SERIES + // *observation*, not of a DataItem declaration. A real Agent REJECTS the whole + // DataItem if the declaration carries one ("The following keys were present and not + // expected: sampleCount" -> "Invalid element 'DataItem'"), so the inference can only + // ever see null here and a live TIME_SERIES tag is always variable-length. Asserted + // rather than ignored so a future change that starts fabricating a dimension is loud. + variable.Attr.ArrayDim.ShouldBeNull( + $"'{item.Id}': a real Agent never carries sampleCount on a DataItem declaration"); + } + else + { + variable.Attr.IsArray.ShouldBeFalse($"'{item.Id}' declares no TIME_SERIES representation"); + } + } + } + + /// + /// A CONDITION is a state word, never a number, whatever its type says — and it + /// is the one category discovery flags as an alarm. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_types_a_condition_as_a_string_alarm() + { + SkipIfDown(); + + var conditions = agent.DeclaredDataItems + .Where(d => Is(d.Category, "CONDITION") && !d.InComposition) + .ToList(); + + if (conditions.Count == 0) + { + Assert.Skip($"The Agent at {agent.AgentUri} declares no CONDITION data items."); + } + + var capture = await DiscoverAsync(); + + foreach (var item in conditions) + { + var variable = Leaf(capture, item.Id); + variable.Attr.DriverDataType.ShouldBe(DriverDataType.String, $"CONDITION '{item.Id}' reports a state word"); + variable.Attr.IsAlarm.ShouldBeTrue($"CONDITION '{item.Id}' is an alarm-bearing data item"); + variable.Attr.SecurityClass.ShouldBe(SecurityClassification.ViewOnly); + } + } + + /// + /// The FullName-vs-browse-name split, on the shape that makes it visible. The + /// driver-side reference must be the DataItem's id (the observation correlation key) + /// while the browse name is its name. Committing the name instead produces a picker + /// that looks perfect and authors tags that can never report a value — and it is invisible on + /// the canned fixtures, whose ids double as names. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_binds_FullName_to_the_DataItem_id_when_the_name_differs() + { + SkipIfDown(); + + var renamed = agent.DeclaredDataItems + .Where(d => !d.InComposition) + .Where(d => !string.IsNullOrEmpty(d.Name) && !string.Equals(d.Name, d.Id, StringComparison.Ordinal)) + .ToList(); + + if (renamed.Count == 0) + { + Assert.Skip( + $"Every DataItem the Agent at {agent.AgentUri} declares has name == id (or no name), so " + + "the id-vs-name split cannot be observed against this device model."); + } + + var capture = await DiscoverAsync(); + + foreach (var item in renamed) + { + var variable = Leaf(capture, item.Id); + variable.Attr.FullName.ShouldBe(item.Id, "the driver-side reference must be the DataItem id"); + variable.BrowseName.ShouldBe(item.Name, "the browse name must prefer the DataItem name"); + } + } + + /// + /// A device scope that matches nothing must fail loudly. It must never degrade to an + /// empty-but-successful browse — indistinguishable from a working connection to a machine + /// that declares nothing (#485). + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Discovery_scoped_to_an_unknown_device_fails_rather_than_browsing_empty() + { + SkipIfDown(); + + var driver = new MTConnectDriver( + LiveOptions(deviceName: $"NoSuchDevice-{Guid.NewGuid():N}"), "mtconnect-live-unknown-device"); + var capture = new DiscoveryCapture(); + + try + { + // Either leg may be the one that fails — the Agent answers /probe and /current for an + // unknown device with 404 + an MTConnectError body — so the assertion is that ONE of + // them did, and that nothing was streamed. + var failed = false; + try + { + await driver.InitializeAsync(ConfigJson(LiveOptions(deviceName: $"NoSuchDevice-{Guid.NewGuid():N}")), Ct); + await driver.DiscoverAsync(capture, Ct); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + failed = true; + } + + failed.ShouldBeTrue("an unknown device scope must surface as a failure, never as an empty browse"); + capture.Variables.ShouldBeEmpty(); + } + finally + { + await driver.ShutdownAsync(CancellationToken.None); + } + } + + // ---- reads: the /current snapshot ---- + + /// + /// ReadAsync returns live, coerced values off the Agent's /current snapshot for + /// the numeric samples it is currently reporting. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Read_returns_live_values_for_the_agents_numeric_samples() + { + SkipIfDown(); + + var ids = await NumericSampleIdsAsync(); + if (ids.Count == 0) + { + Assert.Skip( + $"The Agent at {agent.AgentUri} is currently reporting no numeric SAMPLE value " + + "(every sample is UNAVAILABLE). Is the fixture's adapter service running?"); + } + + await using var live = await LiveDriverAsync(ids.Select(id => Tag(id, DriverDataType.Float64))); + + var results = await live.Driver.ReadAsync([.. ids], Ct); + + results.Count.ShouldBe(ids.Count); + + var good = results.Where(r => r.StatusCode == Good).ToList(); + good.ShouldNotBeEmpty("at least one numeric SAMPLE the Agent reports a value for must read Good"); + good.ShouldAllBe(r => r.Value is double); + good.ShouldAllBe(r => r.SourceTimestampUtc != null && r.SourceTimestampUtc.Value.Kind == DateTimeKind.Utc); + } + + /// + /// The Agent's literal UNAVAILABLE must map to BadNoCommunication — not to a + /// Good empty string, and not to the "no value yet" code, which means something different to + /// an operator. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Read_maps_an_unavailable_observation_to_BadNoCommunication() + { + SkipIfDown(); + + var unavailable = (await CurrentAsync()) + .Descendants() + .Where(e => (string?)e.Attribute("dataItemId") is { Length: > 0 }) + .Where(e => !e.HasElements && string.Equals(e.Value.Trim(), "UNAVAILABLE", StringComparison.Ordinal)) + .Select(e => (string)e.Attribute("dataItemId")!) + .Distinct(StringComparer.Ordinal) + .ToList(); + + if (unavailable.Count == 0) + { + Assert.Skip( + $"The Agent at {agent.AgentUri} is currently reporting no UNAVAILABLE observation, so the " + + "UNAVAILABLE -> BadNoCommunication mapping has nothing live to assert against."); + } + + await using var live = await LiveDriverAsync(unavailable.Select(id => Tag(id, DriverDataType.String))); + + var results = await live.Driver.ReadAsync([.. unavailable], Ct); + + results.Count.ShouldBe(unavailable.Count); + results.ShouldAllBe(r => r.StatusCode == BadNoCommunication); + results.ShouldAllBe(r => r.Value == null); + } + + // ---- subscriptions: the /sample long poll ---- + + /// + /// The live streaming proof. A real multipart/x-mixed-replace body, with real + /// boundaries and real heartbeats, must drive OnDataChange — and must drive it with a + /// value that has actually MOVED since the primed /current baseline. Asserting merely + /// that a callback arrived would be satisfied by the priming callback alone, which is + /// answered out of the snapshot and proves nothing about the stream. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Subscribe_delivers_a_changed_value_from_the_live_sample_stream() + { + SkipIfDown(); + + var ids = await NumericSampleIdsAsync(); + if (ids.Count == 0) + { + Assert.Skip( + $"The Agent at {agent.AgentUri} is currently reporting no numeric SAMPLE value, so no " + + "observation can be expected to move. Is the fixture's adapter service running?"); + } + + await using var live = await LiveDriverAsync(ids.Select(id => Tag(id, DriverDataType.Float64))); + + var seen = new ConcurrentQueue(); + var moved = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var baseline = new ConcurrentDictionary(StringComparer.Ordinal); + + live.Driver.OnDataChange += (_, e) => + { + seen.Enqueue(e); + if (e.Snapshot.StatusCode != Good || e.Snapshot.Value is not double value) + { + return; + } + + // First Good value per reference is the baseline (the priming callback); a later one + // that differs is the stream doing its job. + if (!baseline.TryAdd(e.FullReference, value) && + baseline.TryGetValue(e.FullReference, out var first) && + Math.Abs(first - value) > 1e-9) + { + moved.TrySetResult(); + } + }; + + var handle = await live.Driver.SubscribeAsync([.. ids], TimeSpan.FromMilliseconds(200), Ct); + try + { + using var deadline = CancellationTokenSource.CreateLinkedTokenSource(Ct); + deadline.CancelAfter(StreamWait); + + var finished = await Task.WhenAny(moved.Task, Task.Delay(Timeout.Infinite, deadline.Token)); + + (finished == moved.Task).ShouldBeTrue( + $"the live /sample stream must deliver a CHANGED value within {StreamWait.TotalSeconds:F0}s. " + + $"Callbacks seen: {seen.Count}; references with a Good baseline: {baseline.Count}."); + + seen.Count.ShouldBeGreaterThan(ids.Count, "more callbacks than the one priming callback per reference"); + } + finally + { + await live.Driver.UnsubscribeAsync(handle, CancellationToken.None); + } + } + + // ---- the v3 production shape: RawPath identity ---- + + /// + /// The shape a deployed driver actually serves: the artifact injects RawTags + /// (RawPath + a TagConfig blob naming the DataItem id), and the driver must read and + /// publish by RawPath. A driver that answered under its own DataItem id would be + /// dropped silently by DriverHostActor's RawPath-keyed routing table, with every unit + /// test still green. + /// + [Fact(Timeout = TestTimeoutMs)] + public async Task Read_and_subscribe_key_on_RawPath_when_the_artifact_supplies_RawTags() + { + SkipIfDown(); + + var ids = (await NumericSampleIdsAsync()).Take(4).ToList(); + if (ids.Count == 0) + { + Assert.Skip($"The Agent at {agent.AgentUri} is currently reporting no numeric SAMPLE value."); + } + + var rawPaths = ids.ToDictionary(id => id, id => $"Plant/MTConnect/live/{id}", StringComparer.Ordinal); + + // The real merge DeploymentArtifact.TryReadSpec runs, not a hand-written JSON literal: this + // proves the driver binds the shape the deploy artifact actually produces. + var merged = DriverDeviceConfigMerger.Merge( + ConfigJson(LiveOptions()), + [new DriverDeviceConfigMerger.DeviceRow("live", "{}")], + [.. ids.Select(id => new RawTagEntry( + rawPaths[id], + $$"""{"fullName":"{{id}}","driverDataType":"Float64"}""", + WriteIdempotent: false, + DeviceName: "live"))]); + + var driver = new MTConnectDriver(LiveOptions(), "mtconnect-live-rawtags"); + try + { + await driver.InitializeAsync(merged, Ct); + + var byRawPath = await driver.ReadAsync([.. rawPaths.Values], Ct); + byRawPath.Count.ShouldBe(ids.Count); + byRawPath.ShouldContain(r => r.StatusCode == Good && r.Value is double, "reads must resolve by RawPath"); + + var seen = new ConcurrentQueue(); + driver.OnDataChange += (_, e) => seen.Enqueue(e); + + var handle = await driver.SubscribeAsync([.. rawPaths.Values], TimeSpan.FromMilliseconds(200), Ct); + try + { + seen.ShouldNotBeEmpty("subscribe primes every reference from the /current snapshot"); + seen.Select(e => e.FullReference) + .ShouldAllBe(reference => rawPaths.Values.Contains(reference)); + } + finally + { + await driver.UnsubscribeAsync(handle, CancellationToken.None); + } + } + finally + { + await driver.ShutdownAsync(CancellationToken.None); + } + } + + // ---- helpers ---- + + private void SkipIfDown() + { + if (agent.SkipReason is not null) + { + Assert.Skip(agent.SkipReason); + } + } + + private static bool Is(string? value, string expected) => + string.Equals(value?.Trim(), expected, StringComparison.OrdinalIgnoreCase); + + /// The minimal Test Connect config: the probe reads nothing but agentUri. + private static string ProbeConfig(string agentUri) => + JsonSerializer.Serialize(new { agentUri }); + + /// + /// A loopback port with nothing listening on it: bound to :0 to let the OS pick a free one, + /// then released. Racy in principle, effectively certain in practice, and far more reliable + /// than guessing an unused constant. + /// + private static int ClosedLoopbackPort() + { + var listener = new TcpListener(System.Net.IPAddress.Loopback, 0); + listener.Start(); + var port = ((System.Net.IPEndPoint)listener.LocalEndpoint).Port; + listener.Stop(); + + return port; + } + + private MTConnectDriverOptions LiveOptions(string? deviceName = null) => + new() + { + AgentUri = agent.AgentUri, + DeviceName = deviceName, + RequestTimeoutMs = 15_000, + + // Faster than the 1000 ms default so a subscription sees several distinct chunks inside + // StreamWait; the heartbeat stays well under it so a silent Agent is caught, not waited on. + SampleIntervalMs = 200, + SampleCount = 500, + HeartbeatMs = 5_000, + + // The background connectivity probe is off: it dials the Agent on its own cadence and + // would interleave request logs with the ones a failing test needs to be read from. + Probe = new MTConnectProbeOptions { Enabled = false }, + }; + + private static string ConfigJson(MTConnectDriverOptions options, IEnumerable? tags = null) => + JsonSerializer.Serialize(new + { + agentUri = options.AgentUri, + deviceName = options.DeviceName, + requestTimeoutMs = options.RequestTimeoutMs, + sampleIntervalMs = options.SampleIntervalMs, + sampleCount = options.SampleCount, + heartbeatMs = options.HeartbeatMs, + probe = new { enabled = false }, + tags = (tags ?? []).Select(t => new + { + fullName = t.FullName, + driverDataType = t.DriverDataType.ToString(), + }), + }); + + private static MTConnectTagDefinition Tag(string dataItemId, DriverDataType type) => new(dataItemId, type); + + /// Initializes a driver against the live Agent with the supplied authored tags. + private async Task LiveDriverAsync(IEnumerable tags) + { + var options = LiveOptions(); + var driver = new MTConnectDriver(options, "mtconnect-live"); + await driver.InitializeAsync(ConfigJson(options, tags), Ct); + + return new LiveDriver(driver); + } + + /// Initializes a driver and captures one full DiscoverAsync pass. + private async Task DiscoverAsync() + { + await using var live = await LiveDriverAsync([]); + + var capture = new DiscoveryCapture(); + await live.Driver.DiscoverAsync(capture, Ct); + + return capture; + } + + /// The single captured variable for a DataItem id, asserting it was streamed exactly once. + private static CapturedVariable Leaf(DiscoveryCapture capture, string dataItemId) + { + var matches = capture.Variables.Where(v => v.Attr.FullName == dataItemId).ToList(); + matches.Count.ShouldBe(1, $"DataItem '{dataItemId}' must be streamed exactly once by discovery"); + + return matches[0]; + } + + /// The Agent's /current snapshot, fetched straight over HTTP. + /// + /// Read outside the driver on purpose: several assertions need to know what the Agent is + /// reporting right now in order to build their expectation, and computing that with + /// the parser under test would only prove the code agrees with itself. + /// + private async Task CurrentAsync() + { + using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(15) }; + var body = await http.GetStringAsync($"{agent.AgentUri}/current", Ct); + + return XDocument.Parse(body); + } + + /// + /// The DataItem ids the Agent is currently reporting a parseable number for: the live subset + /// a read or subscription can make a non-vacuous statement about. + /// + /// + /// The Agent's own self-model is excluded, and that exclusion is load-bearing. Every + /// MTConnect 2.x Agent publishes an <Agent> device whose + /// OBSERVATION_UPDATE_RATE / ASSET_UPDATE_RATE SAMPLEs tick continuously + /// regardless of whether any adapter is attached. Including them made + /// pass with the + /// fixture's data source deliberately stopped (verified) — it was proving the Agent's own + /// heartbeat, not the device stream. Restricted to <Device>-owned samples, an + /// Agent with no live source yields an empty set and the test skips with an actionable + /// message instead. + /// + private async Task> NumericSampleIdsAsync() + { + var declaredSamples = agent.DeclaredDataItems + .Where(d => Is(d.Category, "SAMPLE") && !d.InAgentSelfModel && !d.InComposition) + .Where(d => !Is(d.Representation, "TIME_SERIES") && !Is(d.Representation, "DATA_SET") && !Is(d.Representation, "TABLE")) + .Select(d => d.Id) + .ToHashSet(StringComparer.Ordinal); + + return [.. (await CurrentAsync()) + .Descendants() + .Where(e => !e.HasElements) + .Select(e => ((string?)e.Attribute("dataItemId"), e.Value.Trim())) + .Where(pair => pair.Item1 is { Length: > 0 } && declaredSamples.Contains(pair.Item1)) + .Where(pair => double.TryParse(pair.Item2, NumberStyles.Float, CultureInfo.InvariantCulture, out _)) + .Select(pair => pair.Item1!) + .Distinct(StringComparer.Ordinal)]; + } + + /// + /// Scope guard: is not disposable (its teardown is + /// ShutdownAsync), and a test that failed mid-way would otherwise leave a live + /// /sample long poll running against the fixture for the rest of the session. + /// + private sealed class LiveDriver(MTConnectDriver driver) : IAsyncDisposable + { + public MTConnectDriver Driver { get; } = driver; + + public async ValueTask DisposeAsync() => await Driver.ShutdownAsync(CancellationToken.None); + } +} diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests.csproj b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests.csproj new file mode 100644 index 00000000..0499fd05 --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests.csproj @@ -0,0 +1,41 @@ + + + + + $(NoWarn);OTOPCUA0001 + + + + net10.0 + enable + enable + false + true + ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + + + + + -- 2.52.0 From 71ccccef7cfae4d1b483bc8427f6e78c31172372 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Fri, 24 Jul 2026 20:29:51 -0400 Subject: [PATCH 31/34] docs(mtconnect): P1 driver guide + tracking-doc update, defer write-back (Task 22) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - docs/drivers/MTConnect.md: getting-started guide for the P1 Agent MVP — config keys read off MTConnectDriverOptions/ConfigDto, capability surface, RawPath->dataItemId coercion precedence, UNAVAILABLE->BadNoCommunication + the rest of the quality-code table, CONDITION-as-String, browse-via-the- universal-browser, the fixture recipe (links the existing IntegrationTests Docker README instead of duplicating it), and a Known limitations section covering the two pre-existing fleet-wide gaps this build surfaced (IRediscoverable/IHostConnectivityProbe have no server consumer; no driver form blocks Save on validation). Records write-back (MTConnect Interfaces), P1.5 (native alarms, TIME_SERIES arrays, EVENT->enum), and P2 (SHDR ingest) as deferred per design doc SS3.6/SS9. Documents where the build diverged from the design: TrakHound MTConnect.NET was dropped entirely (no XML formatter in the pinned packages, no injectable HTTP handler) in favor of a hand-rolled System.Xml.Linq client, namespace-agnostic on LocalName. - docs/drivers/README.md: adds MTConnect to the ground-truth driver table, the per-driver doc list, and the fixture coverage-map list. - docs/plans/2026-07-24-driver-expansion-tracking.md: marks the MTConnect Wave-2 row Done (22/23 tasks; Task 21 live gate tracked separately in the plan file, not touched here), corrects mtconnect/cppagent -> mtconnect/agent in the fixture note, and links the new driver guide. Deferred-cleanup decision (Task 1 review follow-up): removed GenerateDocumentationFile/NoWarn CS1591 from Driver.MTConnect.Contracts's csproj to match all eight sibling .Contracts projects, none of which carry it. Verified both the Contracts project alone and the full solution still build clean (0 warnings, 0 errors) with the flags gone — the existing XML doc comments don't depend on GenerateDocumentationFile to compile. --- docs/drivers/MTConnect.md | 344 ++++++++++++++++++ docs/drivers/README.md | 6 + .../2026-07-24-driver-expansion-tracking.md | 47 ++- ....OtOpcUa.Driver.MTConnect.Contracts.csproj | 2 - 4 files changed, 384 insertions(+), 15 deletions(-) create mode 100644 docs/drivers/MTConnect.md diff --git a/docs/drivers/MTConnect.md b/docs/drivers/MTConnect.md new file mode 100644 index 00000000..f1f15662 --- /dev/null +++ b/docs/drivers/MTConnect.md @@ -0,0 +1,344 @@ +# MTConnect Driver + +Getting-started guide for the MTConnect Agent driver (P1 Agent MVP). This is +the short path — for the full design rationale read +[`docs/plans/2026-07-15-mtconnect-driver-design.md`](../plans/2026-07-15-mtconnect-driver-design.md) +and the build-vs-plan record in +[`docs/plans/2026-07-24-mtconnect-driver.md`](../plans/2026-07-24-mtconnect-driver.md); for the +fixture recipe read +[`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md`](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md) +(not duplicated here). + +> **Live gate:** see the LIVE-GATE RESULT note in the plan's Task 21 +> (`docs/plans/2026-07-24-mtconnect-driver.md`) for the docker-dev `/run` verification outcome +> (browse picker / typed editor / deploy / read / subscribe against the real Agent fixture). + +## What it talks to + +A **MTConnect Agent** — the vendor-neutral read-only telemetry endpoint that +front-ends a machine tool (or fronts an adapter that itself talks to the +machine over SHDR). The driver speaks plain HTTP + XML against the Agent's +three standard REST paths: + +- `/probe` — the static device model (Device → Component → DataItem tree) +- `/current` — a snapshot of every DataItem's latest observation +- `/sample` — a `multipart/x-mixed-replace` long-poll stream of observation + deltas, keyed by a monotonic sequence number + +v1 is **agent-first and read-only** — Discover + Read + Subscribe, no Write — +because the mainstream Agent surface has no "set value" operation. + +## Built vs. planned — read this before trusting the design doc's §2 + +The design doc ([`2026-07-15-mtconnect-driver-design.md`](../plans/2026-07-15-mtconnect-driver-design.md)) +picked the **TrakHound `MTConnect.NET-Common` / `-HTTP`** NuGet packages (MIT, +`netstandard2.0`) as the primary path, with a hand-rolled fallback. **The +hand-rolled fallback is what shipped.** Task 6/7 of the implementation plan +found by reflection + live invocation that the pinned TrakHound packages +ship **no XML formatter** (`Document Formatter Not found for "xml"` — the +formatter lives in a separate `MTConnect.NET-XML` package the design didn't +pin) and that `MTConnectHttpClientStream` exposes no injectable `HttpClient` +handler, so it cannot be unit-tested behind the driver's `IMTConnectAgentClient` +seam. Both package references were dropped. There is **no TrakHound +dependency anywhere in the shipped driver** — probe/current/sample parsing is +hand-rolled `System.Xml.Linq` plus a multipart boundary reader, entirely +behind `IMTConnectAgentClient`. See the CORRECTION block under Task 0 of the +implementation plan for the full record. + +**Namespace-agnostic parsing.** Every parser matches XML elements on +`LocalName` only, ignoring the document's XML namespace entirely, so +MTConnect 1.3 through 2.x documents all parse without a schema-version +branch. This is deliberate, not sloppy: a real Agent injects vendor-extension +`Component`s in a foreign namespace whose child `DataItems` elements inherit +the *default* namespace, and namespace-strict matching would silently drop +the whole vendor component (and every DataItem beneath it) rather than fail +loudly. Attributes are read **unqualified** for the same reason — so an +`xsi:type` attribute is never mistaken for a DataItem's own `type`. + +## Project split + +| Project | Target | Role | +|---------|--------|------| +| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/` | net10.0 | In-process driver — `MTConnectDriver`, the hand-rolled `IMTConnectAgentClient` (probe/current/sample), the observation index, the factory | +| `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/` | net10.0 | `MTConnectDriverOptions`, `MTConnectTagDefinition`, and the pure `MTConnectDataTypeInference` table shared by the driver, browse-commit, and the AdminUI typed editor | + +No `.Browser` project — browse comes free from the Wave-0 universal +discovery browser (see [Browse](#browse--free-via-the-universal-browser) +below). + +## Capability surface + +`MTConnectDriver : IDriver, IReadable, ISubscribable, ITagDiscovery, IHostConnectivityProbe, IRediscoverable` +(`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs`). + +Deliberately **not** `IWritable` — the Agent surface is read-only by design. +Write-back exists only via optional, rarely-deployed MTConnect *Interfaces* +(a request/response handshake, not a "set value" operation), and is out of +scope for this build — see [Deferred](#deferred-not-in-this-build). + +| Capability | Path | Notes | +|------------|------|-------| +| `ITagDiscovery` | `DiscoverAsync` — streams `/probe`'s Device→Component→DataItem tree into the address-space builder | `SupportsOnlineDiscovery = true`, `RediscoverPolicy = Once` | +| `IReadable` | `ReadAsync` → one `/current` per call | **Not the production data path** — see below | +| `ISubscribable` | `SubscribeAsync`/`OnDataChange` — the shared `/sample` long-poll pump | **The production data path** | +| `IHostConnectivityProbe` | periodic `/probe` under `Probe.*` | No consumer wires it today — see [Known limitations](#known-limitations) | +| `IRediscoverable` | watches the Agent's `Header.instanceId` | No consumer wires it today — see [Known limitations](#known-limitations) | + +**`IReadable.ReadAsync` is never called by the running server.** The +production data plane is entirely `ISubscribable` — `DriverInstanceActor` +subscribes once and lives off `OnDataChange`. `ReadAsync` exists for the +Client CLI (`... read -n ...`) and for the unit/integration suites; it is +correct and tested, just not on the hot path. + +## Minimum deployment + +```jsonc +"Drivers": { + "mtconnect-1": { + "Type": "MTConnect", + "Config": { + "AgentUri": "http://10.100.0.35:5000", + "DeviceName": null, + "RequestTimeoutMs": 5000, + "SampleIntervalMs": 1000, + "SampleCount": 1000, + "HeartbeatMs": 10000, + "Probe": { "Enabled": true, "IntervalMs": 5000, "TimeoutMs": 2000 }, + "Reconnect": { "MinBackoffMs": 0, "MaxBackoffMs": 30000, "BackoffMultiplier": 2.0 }, + "Tags": [] + } + } +} +``` + +`RawTags[]` is not authored by hand — `DriverDeviceConfigMerger` injects it +at deploy time from the tags authored on the `/raw` tree. + +### Config keys + +Read off `MTConnectDriverOptions` / `MTConnectDriverConfigDto` +(`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs`, +`src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs`): + +| Key | Default | Notes | +|---|---|---| +| `AgentUri` | — (required) | Agent base URI; the driver appends `/probe`, `/current`, `/sample` | +| `DeviceName` | `null` | Scopes requests to `{AgentUri}/{DeviceName}/...`; `null` = whole Agent | +| `RequestTimeoutMs` | `5000` | Per-call deadline for `/probe` and `/current` | +| `SampleIntervalMs` | `1000` | The Agent's `/sample?interval=` query param | +| `SampleCount` | `1000` | The Agent's `/sample?count=` query param | +| `HeartbeatMs` | `10000` | The Agent's `/sample?heartbeat=` query param — also the watchdog's liveness window | +| `Probe.Enabled` / `Probe.IntervalMs` / `Probe.TimeoutMs` | `true` / `5000` / `2000` | Background connectivity-probe knobs (mirrors `ModbusProbeOptions`) | +| `Reconnect.MinBackoffMs` / `MaxBackoffMs` / `BackoffMultiplier` | `0` / `30000` / `2.0` | Geometric backoff after a failed request or a dropped `/sample` stream | +| `Tags[]` | `[]` | Pre-v3 / CLI authoring surface — one `MTConnectTagDefinition` per DataItem `id` | +| `RawTags[]` | `[]` (deploy-injected) | The v3 data-plane binding — see below | + +**All timing knobs are validated strictly positive** at `InitializeAsync` +(`RequirePositive`, mirroring the arch-review 01/S-6 lesson: a `0` timeout +does not mean "wait forever," it faults the driver). This applies to +`RequestTimeoutMs`, `HeartbeatMs`, `SampleIntervalMs`, `SampleCount`, and +(when the probe is enabled) `Probe.Interval` / `Probe.Timeout`. + +**No save-time gate on a blank `AgentUri` in the AdminUI.** The driver form +renders an inline validation notice (`_form.Validate()`), but +`DriverConfigModal.SaveAsync` has no per-form validation seam to block the +Save button on it — no sibling driver form has one either. A blank +`AgentUri` saves cleanly and fails at deploy with the driver's own error +message, not at authoring time. + +## Data plane + +### FullName == DataItem `id` + +`FullName` (both `MTConnectTagDefinition.FullName` and every `RawTags[]` +blob's identifier) is the MTConnect `DataItem@id` attribute — the value the +universal browser commits and the key the driver resolves reads/subscribes +against. `MTConnectTagConfigModel.FromJson` (the AdminUI typed editor) tries +three spellings in order — `fullName` → `dataItemId` → `address` — and takes +the first non-blank one, normalising onto `fullName` on save. `address` is +accepted because that's the field name `RawBrowseCommitMapper` writes for a +driver with no typed editor path, so a browse-committed tag stays readable +even before the editor round-trips it. + +### Coercion-type precedence + +The driver keys its data plane by **RawPath** (the v3 raw-tag identity), not +by DataItem id directly, resolving RawPath → dataItemId internally. Each raw +tag's coercion type (the `DriverDataType` its Agent observation is parsed +into) is resolved in this order, first non-null wins: + +1. The `RawTags[]` blob's `driverDataType` or `dataType` field (both + spellings accepted — the typed editor writes `dataType`, the driver's own + `tags[]` shape uses `driverDataType`). +2. A matching `Tags[]` entry naming the same DataItem id. +3. The Agent's own `/probe` declaration, via `MTConnectDataTypeInference.Infer` + (category/type/units/representation → `DriverDataType`). +4. `DriverDataType.String` — the coercion that cannot fail. + +### Quality mapping + +`UNAVAILABLE` — MTConnect's one explicit "I have no value" sentinel — maps to +**`BadNoCommunication`** (`0x80310000`). This is deliberately distinct from +the fleet-standard `BadCommunicationError` (`0x80050000`, used elsewhere for +the driver's *own* transport failure to the Agent): `UNAVAILABLE` means the +Agent is reachable and answered, it just has no device-backed value for that +item. A `CONDITION` observation's value is taken from its **element name** +(`` → the string `"Normal"`, `` → `"Fault"`); a `` +condition element normalises onto the same `UNAVAILABLE` sentinel as every +other category so one comparison covers all three. + +Other status codes an observation can surface: + +| Code | Meaning | +|---|---| +| `BadTypeMismatch` (`0x80740000`) | The Agent's text isn't a value of the tag's coerced type at all | +| `BadOutOfRange` (`0x803C0000`) | The Agent reported a number the coerced type can't represent | +| `BadNotSupported` (`0x803D0000`) | A shape this build doesn't materialize — a `TIME_SERIES` vector, or a structured `DATA_SET`/`TABLE` observation (deferred to P1.5; see [Known limitations](#known-limitations)) | +| `BadWaitingForInitialData` | Tag is authored but the Agent hasn't reported it yet | +| `BadNodeIdUnknown` | DataItem id is neither authored nor ever observed | + +### Subscribe — the `/sample` pump + +One shared `/sample` long-poll stream per driver instance (the Agent streams +the whole device model regardless of which subset is subscribed, so +per-tag streams would be wasted round-trips), run under a heartbeat +watchdog — `HttpClient.Timeout` cannot bound a long-lived stream, so a +missing chunk **and** missing keep-alive heartbeat within +`HeartbeatMs × N` is what detects a frozen peer. + +Ring-buffer overflow (the Agent's circular observation buffer wrapped past +what the driver's cursor expects) is detected **two ways**, both triggering a +`/current` re-baseline before the stream resumes: + +1. `IMTConnectAgentClient.IsSequenceGap` — the next chunk's `firstSequence` + is newer than the driver's expected cursor. +2. An `MTConnectError` document reporting `OUT_OF_RANGE` — real Agents return + this both under HTTP 200 (a normal MTConnect error document) and as a bare + HTTP 400, and the client handles both. + +**An Agent `instanceId` change means the Agent restarted**, and is checked +**before** the sequence-gap check (a restart also usually trips the gap, so +the two need disambiguating, not just OR-ing together). On a changed +`instanceId` the driver clears its cached probe model and raises +`OnRediscoveryNeeded` — see [Known limitations](#known-limitations) for why +that signal currently has no consumer. + +## Browse — free via the universal browser + +MTConnect ships **no bespoke browser project**. Setting +`ITagDiscovery.SupportsOnlineDiscovery => true` is the entire integration: +the Wave-0 `DiscoveryDriverBrowser` sees a driver whose `TryCreate` succeeds +and whose instance reports online discovery, renders the AdminUI **Browse** +button, constructs the driver, runs `InitializeAsync` (the `/probe` connect) ++ `DiscoverAsync` into a `CapturingAddressSpaceBuilder`, then tears the +throwaway instance down. Each captured leaf's NodeId is the DataItem `id`, +committed directly as `TagConfig.FullName` on pick. See +[`docs/plans/2026-07-15-universal-discovery-browser-design.md`](../plans/2026-07-15-universal-discovery-browser-design.md). + +## CONDITION modelling (v1: plain String) + +Each `CONDITION` DataItem materializes as a `String` variable whose value is +the current state word — `Normal` / `Warning` / `Fault` / `UNAVAILABLE`. +There is no native OPC UA Part 9 alarm plumbing in this build; +`DriverAttributeInfo.IsAlarm = true` is still stamped on a CONDITION leaf so +the browse side-panel flags it and a future upgrade to native alarms doesn't +need re-authoring. See [Deferred](#deferred-not-in-this-build). + +## Known limitations + +These are real, not placeholders — read them before relying on the driver +for anything beyond values-and-conditions. + +1. **`IRediscoverable` and `IHostConnectivityProbe` have no consumer in the + server.** `DriverInstanceActor` wires only `ISubscribable.OnDataChange` and + `IAlarmSource.OnAlarmEvent`; nothing in the server subscribes to + `OnRediscoveryNeeded` or `OnHostStatusChanged` except `GalaxyDriver` + wiring its own internal `DeployWatcher` sub-component. So a restarted + Agent (a changed `instanceId`) leaves a stale address space behind an + otherwise-Healthy driver. **This is a pre-existing fleet-wide gap + affecting every driver that implements either interface** (eight drivers + besides Galaxy), not something specific to MTConnect — this build simply + surfaced it again. +2. **`RawTagEntry.DeviceName` is accepted on the wire shape but ignored for + routing.** One driver instance owns exactly one Agent client scoped by + `MTConnectDriverOptions.DeviceName` (the top-level config key); the + per-tag `RawTagEntry.DeviceName` field the v3 raw-tag identity carries is + never read by the driver. Two `/raw` Devices authored under one MTConnect + driver instance both resolve to the same one Agent connection. Real + per-device routing (a client per device) would be a design change, not a + bug fix. +3. **Nothing validates a raw tag's declared OPC UA `DataType` against the + blob's coercion type.** The `/probe`-sourced inference + (`MTConnectDataTypeInference`) narrows how often an author picks a wrong + type, but it doesn't close the gap — an authored `DataType` mismatched + against the actual Agent observation still surfaces as `BadTypeMismatch` + /`BadOutOfRange` at runtime rather than at authoring time. +4. **`sampleCount` is not a legal `DataItem` attribute.** It belongs on a + `TIME_SERIES` *observation*, not the device-model declaration. A real + Agent that meets `sampleCount` on a `DataItem` declaration logs `The + following keys were present and not expected: sampleCount` and **drops + the entire data item** from the served model. A live TIME_SERIES tag + therefore always resolves to `ArrayDim = null` (variable-length array) in + practice — the fixture's canned `probe.xml`/`Devices.xml`, which does + declare a `sampleCount`, is unrealistic on this point and exists only to + pin the parsing rule itself. +5. **CONDITION is a plain `String`, not a native Part 9 alarm** (see above). +6. **`TIME_SERIES` arrays and structured `DATA_SET`/`TABLE` observations + surface as `BadNotSupported`**, not an approximation — see the Quality + mapping table. + +## Deferred (not in this build) + +- **Write-back (MTConnect Interfaces).** Design + [§3.6](../plans/2026-07-15-mtconnect-driver-design.md#36-iwritable--not-implemented-v1): + the mainstream Agent surface is read-only by design; Interfaces is a rare, + optional request/response handshake that would mislead if modelled as + `IWritable`. Revisit only on a concrete deployment need. +- **P1.5 fast-follow** (design + [§9](../plans/2026-07-15-mtconnect-driver-design.md#9-phasing--effort)): + CONDITION → native OPC UA Part 9 alarms via `IAlarmSource` (the Galaxy + native-alarm pattern); `TIME_SERIES` SAMPLE arrays materialized as real + OPC UA arrays; EVENT controlled-vocabulary values → OPC UA enumerations. +- **P2 (on demand): SHDR adapter ingest.** A `SourceMode: "Agent" | "Shdr"` + switch that opens the raw pipe-delimited SHDR TCP socket directly. Loses + auto-discovery (no device model without an Agent in front, so + `SupportsOnlineDiscovery` would have to report `false` and tags would be + authored by hand, like Modbus). Niche; not built. + +## Testing + +- **Unit tests** — `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/` + (canned-XML fixtures, no network) — 491/491 at time of writing. Covers + discovery tree shape, `MTConnectDataTypeInference`, observation indexing, + `UNAVAILABLE` → `BadNoCommunication`, CONDITION state-word mapping, + multipart chunk framing, and ring-buffer re-baseline paging (both the + sequence-gap and the `OUT_OF_RANGE` legs). +- **Integration tests** — + `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/` — env-gated + on `MTCONNECT_AGENT_ENDPOINT` (default `http://10.100.0.35:5000`), skips + cleanly (12 Skipped) when the fixture is unreachable, 12/12 against a real + Agent. Read the fixture's own + [`Docker/README.md`](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md) + for the full recipe — summary only, here: + - Image is **`mtconnect/agent:2.7.0.12`** — not `mtconnect/cppagent`, which + does not exist on Docker Hub (the design's original assumption). + - The stack is **two services**: the Agent plus a stdlib-only SHDR adapter + (`Docker/adapter.py`, on `python:3.13-alpine`). An Agent with no adapter + reports every observation `UNAVAILABLE` forever, which proves nothing. + - Deployed at `/opt/otopcua-mtconnect/` on the shared docker host + (`10.100.0.35`, `project=lmxopcua` label). Endpoint + `http://10.100.0.35:5000`. + - `agent.cfg` must be **pure ASCII** — one non-ASCII byte anywhere, + including a comment, makes the config parser reject the whole file with + a bare `Failed / Stopped at line: N`. + - Bring-up: `lmxopcua-fix sync mtconnect` then `lmxopcua-fix up mtconnect` + (PowerShell helper, Windows-only) — or, directly on the docker host, + `rsync` the repo's `Docker/` dir to `/opt/otopcua-mtconnect/` and run + `docker compose up -d --wait`. + +## Further reading + +- [`docs/plans/2026-07-15-mtconnect-driver-design.md`](../plans/2026-07-15-mtconnect-driver-design.md) — full design: capability wiring, browse reconciliation, typed-editor spec, resilience/timeout rules, phasing +- [`docs/plans/2026-07-24-mtconnect-driver.md`](../plans/2026-07-24-mtconnect-driver.md) — the executable implementation plan and the task-by-task build record, including where it diverged from the design (TrakHound → hand-rolled) +- [`docs/plans/2026-07-24-driver-expansion-tracking.md`](../plans/2026-07-24-driver-expansion-tracking.md) — Wave-2 program tracking +- [Docker fixture README](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md) — the authoritative fixture recipe (seeded device model, endpoint, Mac AirPlay port-5000 gotcha) diff --git a/docs/drivers/README.md b/docs/drivers/README.md index 98c86067..7eb23242 100644 --- a/docs/drivers/README.md +++ b/docs/drivers/README.md @@ -30,6 +30,7 @@ Driver type metadata is registered at startup in `DriverTypeRegistry` (`src/Core | [FOCAS](FOCAS.md) | `Driver.FOCAS` | A | Pure-managed `FocasWireClient` — FOCAS/2 Ethernet binary protocol on TCP:8193, inlined into the driver assembly | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IAlarmSource | `IWritable` is implemented but read-only by design — `WriteAsync` returns `BadNotWritable` for every point. CNC-shaped data model (axes, spindle, PMC, macros, alarms) not a flat tag map. Previously Tier-C (Host + P/Invoke + shim DLL); retired in the 2026-04-24 migration when the managed wire client landed | | [OPC UA Client](OpcUaClient.md) | `Driver.OpcUaClient` | B | OPCFoundation `Opc.Ua.Client` | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe | Gateway/aggregation driver — the only driver implementing driver-side `IHistoryProvider` (forwards HistoryRead to the upstream server). Opens a single `Session` against a remote OPC UA server and re-exposes its address space. Owns its own `ApplicationConfiguration` (distinct from `Client.Shared`) because it's always-on with keep-alive + `TransferSubscriptions` across SDK reconnect, not an interactive CLI | | [MQTT](Mqtt.md) | `Driver.Mqtt` (+ `.Browser`, `.Contracts`) | A | [MQTTnet 5](https://github.com/dotnet/MQTTnet) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable | **Push, not poll, and read-only** — the broker delivers PUBLISH frames the driver forwards straight to `ISubscribable.OnDataChange`; `IReadable` serves the last retained/received value from cache, and there is **no `IWritable`** (nothing publishes back). Subscriptions are indexed **by topic filter, not by topic**, so wildcard tags survive a reconnect. A rejected CONNACK throws `MqttConnectRejectedException`: unrecoverable codes → `Faulted` + supervisor stop, transient → retry. **Two modes:** `Plain` binds a tag to a concrete topic + JSON path; `SparkplugB` decodes vendored Eclipse-Tahu protobuf under one `spBv1.0/{groupId}/#` subscription and binds by the `group/edgeNode/device?/metric` tuple (birth/alias/seq-gap state machine, death→STALE, scoped rebirth NCMD). `IRediscoverable` fires on a DBIRTH but is **inert platform-side** — see [#507](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/507) | +| [MTConnect](MTConnect.md) | `Driver.MTConnect` (+ `.Contracts`) | — | Hand-rolled `System.Xml.Linq` HTTP/XML client against a vendor-neutral MTConnect Agent (`/probe`, `/current`, `/sample`) — no TrakHound dependency (dropped; see the driver doc) | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IRediscoverable | Read-only by design (no `IWritable`). Production data plane is entirely `ISubscribable`/`OnDataChange` — `IReadable` is CLI/test-only. Browse is free via the Wave-0 universal discovery browser, no bespoke browser project. `IRediscoverable`/`IHostConnectivityProbe` have no server-side consumer yet — a pre-existing fleet-wide gap, not MTConnect-specific | | [Historian.Gateway](../Historian.md) | `Driver.Historian.Gateway` | — | `ZB.MOM.WW.HistorianGateway.Client` gRPC (`historian_gateway.v1`) | IHistorianDataSource (server-side read backend) + alarm `SendEvent` writer + `WriteLiveValues` recorder + `IHistorianProvisioning` | Not a tag driver — the sole historian backend. Registers `GatewayHistorianDataSource : IHistorianDataSource` for HistoryRead and serves alarm-write + continuous historization through the gateway. No `IDriver`/`ITagDiscovery` surface. (The retired Wonderware sidecar backend it replaced is documented at [Historian.Wonderware.md](Historian.Wonderware.md).) | ## Per-driver documentation @@ -50,6 +51,10 @@ Driver type metadata is registered at startup in `DriverTypeRegistry` (`src/Core - [OpcUaClient.md](OpcUaClient.md) — OPC UA Client (gateway/aggregation) driver: remote-server session, driver-side HistoryRead forwarding, reconnect behaviour - [Mqtt.md](Mqtt.md) — MQTT broker-subscribe driver: broker/TLS/auth config, topic + JSON-path tag binding, retained-value seeding, `#`-observation browser, CONNACK-rejection semantics, **Sparkplug B** (birth/alias/seq/rebirth state machine, birth-driven browse picker, death→STALE), and the **Known gaps** table +- **MTConnect** has a short getting-started doc because the hand-rolled-vs-TrakHound divergence, the + data-plane precedence rules, and a non-trivial known-limitations list need explaining up front: + - [MTConnect.md](MTConnect.md) — Agent config, `FullName`==DataItem `id`, `UNAVAILABLE`→`BadNoCommunication`, CONDITION-as-`String`, browse-via-universal-browser, fixture recipe, deferred write-back + - **Historian.Gateway** (server-side historian backend, not a tag driver) is documented in the main guide: - [../Historian.md](../Historian.md) — HistorianGateway backend: read-path registration, HistoryRead dispatch, alarm store-and-forward (`SendEvent`), continuous historization (`WriteLiveValues`), `EnsureTags` provisioning, config keys, deployment prerequisites. (The retired Wonderware sidecar backend it replaced: [Historian.Wonderware.md](Historian.Wonderware.md).) @@ -68,6 +73,7 @@ Each driver has a dedicated fixture doc that lays out what the integration / uni - [OPC UA Client](OpcUaClient-Test-Fixture.md) — Dockerized `opc-plc` integration suite (task #215): real Secure Channel + Session, read + subscribe verified end-to-end; write not yet exercised in the integration suite; exhaustive capability matrix (reconnect, failover, cert-auth, history, alarms) via unit suite with mocked `Session` - [MQTT](Mqtt-Test-Fixture.md) — Dockerized `eclipse-mosquitto` with **TLS + real auth on both listeners (no anonymous fallback)** + a JSON-publisher sidecar; env-gated live suite (**15 tests**) — 7 plain (TLS/auth connect, CA pinning both ways, subscribe→value, retained seeding, wrong-password rejection) + 8 Sparkplug against a **project-owned C# edge-node simulator** on the `sparkplug` compose profile (birth→alias bind, DDATA, NDEATH→STALE, rebirth NCMD round-trip). Births are never retained, so `docker restart otopcua-sparkplug-sim` is how you force a fresh one - [Galaxy](../v1/drivers/Galaxy-Test-Fixture.md) — richest harness: gateway E2E + ZB SQL live-smoke + MXAccess opt-in (v1 archive) +- [MTConnect](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md) — Dockerized `mtconnect/agent` + stdlib SHDR adapter (two services, the Agent alone reports everything `UNAVAILABLE`); 12/12 against the real Agent, canned XML unit fixtures cover the bulk (491/491) ## Related cross-driver docs diff --git a/docs/plans/2026-07-24-driver-expansion-tracking.md b/docs/plans/2026-07-24-driver-expansion-tracking.md index fe4bfce6..f4c9fe03 100644 --- a/docs/plans/2026-07-24-driver-expansion-tracking.md +++ b/docs/plans/2026-07-24-driver-expansion-tracking.md @@ -32,7 +32,7 @@ it reaches 📝. | **0** | Universal Discover-backed browser | [design](2026-07-15-universal-discovery-browser-design.md) | [plan](2026-07-15-universal-discovery-browser-implementation.md) · [tasks](2026-07-15-universal-discovery-browser-implementation.md.tasks.json) | 🟡 **Live gate open** | S–M | none (retrofits shipped drivers) | | **1** | Modbus RTU (extend Modbus) | [design](2026-07-15-modbus-rtu-driver-design.md) | [plan](2026-07-24-modbus-rtu-driver.md) · [tasks](2026-07-24-modbus-rtu-driver.md.tasks.json) | 🟢 **Built — live gate PASSED** (11 tasks, branch `feat/modbus-rtu-driver`, pending merge) | **S (lowest)** | `pymodbus` `rtu_over_tcp` — CI-simulatable, no new infra | | **1** | SQL poll | [design](2026-07-15-sql-poll-driver-design.md) | [plan](2026-07-24-sql-poll-driver.md) · [tasks](2026-07-24-sql-poll-driver.md.tasks.json) | 📝 **Plan ready** (22 tasks) | S–M | **existing central SQL Server** `10.100.0.35,14330` + SQLite unit | -| **2** | MTConnect Agent | [design](2026-07-15-mtconnect-driver-design.md) | [plan](2026-07-24-mtconnect-driver.md) · [tasks](2026-07-24-mtconnect-driver.md.tasks.json) | 📝 **Plan ready** (23 tasks) | S–M | Dockerized `mtconnect/cppagent` — CI-simulatable | +| **2** | MTConnect Agent | [design](2026-07-15-mtconnect-driver-design.md) | [plan](2026-07-24-mtconnect-driver.md) · [tasks](2026-07-24-mtconnect-driver.md.tasks.json) | ✅ **COMPLETE** — P1 Agent MVP, all 23 tasks + the 5-leg live gate PASSED (branch `feat/mtconnect-driver`, PR #506) | S–M | Dockerized `mtconnect/agent:2.7.0.12` (not `cppagent`) — CI-simulatable, live at `10.100.0.35:5000` | | **2** | MQTT / Sparkplug B | [design](2026-07-15-mqtt-sparkplug-driver-design.md) | [plan](2026-07-24-mqtt-sparkplug-driver.md) · [tasks](2026-07-24-mqtt-sparkplug-driver.md.tasks.json) | ✅ **COMPLETE** — P1 + P2 shipped, both live-gated (Tasks 0–26) | M–L | Mosquitto TLS+auth **+ C# Sparkplug edge-node simulator**, live at `10.100.0.35:8883` | > Waves 3+ (BACnet/IP, Omron) and the deferred MELSEC are tracked in the program design doc §6/§8, @@ -134,15 +134,36 @@ SQL poll reuses the always-on central SQL Server. This is the recommended next b Both CI-simulatable on the shared docker host, no hardware. -### MTConnect Agent — 📝 Plan ready +### MTConnect Agent — ✅ Done (P1 Agent MVP) - **Design:** [`2026-07-15-mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md) -- **Implementation plan:** [`2026-07-24-mtconnect-driver.md`](2026-07-24-mtconnect-driver.md) · [`.tasks.json`](2026-07-24-mtconnect-driver.md.tasks.json) — **23 tasks**. Task 0 is the TrakHound-vs-hand-rolled client decision; browse-picker live-verify is gated on Wave-0 #468. -- **Scope:** P1 Agent MVP (`IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+probe+rediscover), - browse **free via the Wave-0 universal browser** (`SupportsOnlineDiscovery=true`, no browser code), - typed editor, `UNAVAILABLE→BadNoCommunication` mapping, ring-buffer re-baseline paging. -- **Fixture:** canned XML unit fixtures (bulk of coverage) + a dockerized `mtconnect/cppagent` - integration fixture (env-gated). Depends on Wave 0's browse seam being live-verified (#468). -- **Effort:** S–M (≈1–1.5 wk with TrakHound, ≈2.5–3 wk hand-rolled). +- **Implementation plan:** [`2026-07-24-mtconnect-driver.md`](2026-07-24-mtconnect-driver.md) · [`.tasks.json`](2026-07-24-mtconnect-driver.md.tasks.json) — **23 tasks; 22 built, Task 21 (live `/run` gate) tracked separately in the plan file.** +- **Driver guide:** [`docs/drivers/MTConnect.md`](../drivers/MTConnect.md) — config keys, capability + surface, data-plane precedence rules, quality-code mapping, and (most load-bearing) the **Known + limitations** section. +- **Scope shipped:** `IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+`IHostConnectivityProbe`+`IRediscoverable` + (deliberately no `IWritable` — read-only Agent surface), browse **free via the Wave-0 universal + browser** (`SupportsOnlineDiscovery=true`, no browser project), typed AdminUI tag editor, + `UNAVAILABLE→BadNoCommunication` mapping, CONDITION-as-`String`, ring-buffer re-baseline paging + (both the sequence-gap and `OUT_OF_RANGE` legs). +- **Diverged from the design:** Task 0/6/7 dropped the planned TrakHound `MTConnect.NET-Common`/`-HTTP` + dependency entirely — the pinned packages ship no XML formatter and the HTTP stream client has no + injectable handler to test behind the driver's seam. Parsing is hand-rolled `System.Xml.Linq`, + matching on `LocalName` only (namespace-agnostic, so 1.3–2.x all parse). See the driver guide's + "Built vs. planned" section. +- **Test results:** MTConnect unit suite 491/491; integration suite 12/12 against a real Agent + (`mtconnect/agent:2.7.0.12`, **not** `mtconnect/cppagent` — that Docker Hub repo doesn't exist); + skips cleanly (12 Skipped) offline. AdminUI 749/749; full solution 0 build errors. +- **Deferred (documented, not built):** write-back via MTConnect Interfaces (design §3.6); P1.5 + CONDITION→native Part-9 alarms, `TIME_SERIES` array materialization, EVENT→enum vocab (design §9); + P2 SHDR adapter ingest (design §9). Also two pre-existing fleet-wide gaps this build surfaced + rather than caused: `IRediscoverable`/`IHostConnectivityProbe` have no server-side consumer (eight + drivers besides Galaxy), and no driver form blocks Save on a required-field validation error. +- **Fixture:** canned XML unit fixtures (bulk of coverage) + a dockerized `mtconnect/agent` + + stdlib-SHDR-adapter integration fixture (env-gated), deployed at `/opt/otopcua-mtconnect/` on the + shared docker host. See the fixture's own + [`Docker/README.md`](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md). +- **Effort:** S–M, hand-rolled (the TrakHound estimate in the design doc did not apply once that + path was ruled out). ### MQTT / Sparkplug B — ✅ **COMPLETE** (P1 + P2, both live-gated) - **Design:** [`2026-07-15-mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md) @@ -219,12 +240,12 @@ Both CI-simulatable on the shared docker host, no hardware. ## Next actions -1. **Close the Wave-0 live gate** (Gitea #468) — unblocks browse verification for MTConnect/BACnet. +1. **Close the Wave-0 live gate** (Gitea #468) — unblocks browse verification for BACnet (MTConnect + shipped its own Task-21 live `/run` gate independently — see the plan file). 2. **Build Wave 1** — Modbus RTU first (lowest effort, no new infra), then SQL poll. Both plans are 📝 ready; run the command from the [Executing a plan](#executing-a-plan-git-worktree--subagent-per-task) table (subagent-driven-development in a git worktree). -3. **Build Wave 2** — MTConnect's browse leg wants #468 closed first. **MQTT / Sparkplug B is - COMPLETE** (P1 + P2, both live-gated); its only remaining work is optional — P3 write-through - (`IWritable`: NCMD/DCMD + plain publish) and the two browse-UI gaps recorded above. +3. **Wave 2 is done.** Both deliverables shipped and live-gated: **MQTT / Sparkplug B** (P1 + P2) and **MTConnect Agent** (P1 Agent MVP, PR #506). Remaining work on both is optional follow-on — MQTT P3 write-through (`IWritable`: NCMD/DCMD + plain publish) plus its two browse-UI + gaps, and MTConnect write-back (deliberately deferred; the Agent surface is read-only). Update this file's Summary table and per-wave status whenever a deliverable changes state. diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj index f886c435..18943b79 100644 --- a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts.csproj @@ -5,8 +5,6 @@ enable enable true - true - $(NoWarn);CS1591