feat(mtconnect): current/sample parse + sequence-gap detection (Task 7)
Adds the /current and /sample legs to MTConnectAgentClient: - MTConnectStreamsParser: hand-rolled System.Xml.Linq parse of an MTConnectStreams document into MTConnectStreamsResult. Observations are every element child of <Samples>/<Events>/<Condition>, keyed by dataItemId (the element NAME is the data item's type in PascalCase, so a fixed name set would drop observations). A CONDITION's value is its element name, not its text - <Normal/> is empty - and <Unavailable/> normalizes onto the same UNAVAILABLE sentinel a Sample/Event carries as text, so Task 8's no-comms mapping sees one token. Timestamps are normalized to DateTimeKind.Utc explicitly. - IsSequenceGap(requestedFrom, chunk) => chunk.FirstSequence > requestedFrom, so Task 11's pump can re-baseline after a ring-buffer overflow. Equality is NOT a gap. - SampleAsync: ResponseHeadersRead + a hand-rolled multipart/x-mixed-replace frame reader (Content-length fast path, boundary-scan fallback), bounded by a heartbeat watchdog derived from HeartbeatMs so a silent Agent faults instead of wedging the pump - the S7 frozen-peer shape (arch-review R2-01). The long poll gets its own HttpClient with an infinite Timeout so the unary /probe + /current deadline stays bounded. Task 0 package decision, option (c): both TrakHound PackageReferences and their PackageVersions are dropped. MTConnectHttpClientStream takes a URL and dials its own socket (no handler/stream seam, so it cannot serve a socket-free unit suite), and it emits already-parsed documents through the formatter lookup Task 6 proved is missing - framing-only reuse was never actually on offer. The driver now depends on nothing but the BCL. Recorded in the plan's Task 0 CORRECTION block. Also folds in two Task 6 review carry-overs: the stale IMTConnectAgentClient doc comment claiming a TrakHound-backed implementation, and a probe test constructing a mixed-namespace vendor extension (<x:Pump> with default-namespace <DataItems> children) that backs the parser's ignore-the-namespace claim with a test. 187/187 green.
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,296 @@
|
||||
using System.Globalization;
|
||||
using System.Xml;
|
||||
using System.Xml.Linq;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect;
|
||||
|
||||
/// <summary>
|
||||
/// Turns an MTConnect Agent <c>MTConnectStreams</c> document — a <c>/current</c> snapshot or one
|
||||
/// chunk of a <c>/sample</c> multipart stream — into the neutral
|
||||
/// <see cref="MTConnectStreamsResult"/>. Deliberately a pure, socket-free static so the whole
|
||||
/// parse is unit-testable against canned fixtures; chunk framing lives in
|
||||
/// <see cref="MTConnectAgentClient"/> and hands this parser one chunk's XML at a time.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why hand-rolled rather than TrakHound.</b> Same finding as
|
||||
/// <see cref="MTConnectProbeParser"/>: the packages pinned by Task 0 ship no XML document
|
||||
/// formatter, and TrakHound's streaming client offers no socket-free entry point. Both
|
||||
/// package references were dropped in Task 7; the driver depends only on the BCL.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>The Streams document is shaped nothing like the Devices document</b>, and three
|
||||
/// differences drive this parser's design.
|
||||
/// </para>
|
||||
/// <list type="number">
|
||||
/// <item>
|
||||
/// <description>
|
||||
/// <b>The observation element's name is the data item's type in PascalCase</b>
|
||||
/// (<c>Position</c>, <c>PartCount</c>, <c>Execution</c>) — not the probe's
|
||||
/// UPPER_SNAKE <c>POSITION</c> / <c>PART_COUNT</c>. The standard defines hundreds of
|
||||
/// types and vendors add more, so matching a fixed set of element names would drop
|
||||
/// observations silently. <b>Every</b> element child of <c><Samples></c>,
|
||||
/// <c><Events></c> and <c><Condition></c> is therefore an observation,
|
||||
/// keyed by its <c>dataItemId</c> attribute — the only identity the probe model and
|
||||
/// the streams document actually share.
|
||||
/// </description>
|
||||
/// </item>
|
||||
/// <item>
|
||||
/// <description>
|
||||
/// <b>A CONDITION observation's value is its ELEMENT NAME, not its text.</b>
|
||||
/// <c><Normal/></c> / <c><Warning/></c> / <c><Fault/></c> /
|
||||
/// <c><Unavailable/></c> carry the state in the name and are typically empty;
|
||||
/// where text is present it is the operator-facing message, not the state. Reading
|
||||
/// the text would report every condition as an empty string.
|
||||
/// <c><Unavailable/></c> is normalized to the literal
|
||||
/// <see cref="UnavailableSentinel"/> so a no-comms condition lands on exactly the
|
||||
/// same sentinel as a Sample/Event whose text is <c>UNAVAILABLE</c> — the
|
||||
/// observation index maps that one token to bad quality.
|
||||
/// </description>
|
||||
/// </item>
|
||||
/// <item>
|
||||
/// <description>
|
||||
/// <b>Timestamps are normalized to UTC explicitly.</b> MTConnect timestamps are
|
||||
/// ISO-8601 Zulu, but a bare <c>DateTime.Parse</c> yields <c>Local</c> or
|
||||
/// <c>Unspecified</c> depending on machine settings — and this value becomes the
|
||||
/// OPC UA SourceTimestamp, so a wrong <see cref="DateTimeKind"/> shifts every
|
||||
/// timestamp by the host's UTC offset with nothing to show for it.
|
||||
/// </description>
|
||||
/// </item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// <b>Failure posture.</b> A malformed document, a non-<c>MTConnectStreams</c> root (an Agent
|
||||
/// <c>MTConnectError</c> served under HTTP 200 included), an unusable header, or an
|
||||
/// observation missing its <c>dataItemId</c>/<c>timestamp</c> all throw
|
||||
/// <see cref="InvalidDataException"/>. A document with a valid header and <i>zero</i>
|
||||
/// observations, by contrast, is entirely legal — <c>/sample</c> chunks are deltas and an
|
||||
/// idle Agent's keep-alive is an observation-free document — so it parses to an empty
|
||||
/// observation list rather than throwing.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal static class MTConnectStreamsParser
|
||||
{
|
||||
/// <summary>
|
||||
/// The single token that means "the Agent has no value for this data item". Sample/Event
|
||||
/// observations carry it as text; a CONDITION carries it as the element name
|
||||
/// <c><Unavailable/></c> and is normalized onto this spelling here.
|
||||
/// </summary>
|
||||
private const string UnavailableSentinel = "UNAVAILABLE";
|
||||
|
||||
private const string ConditionContainer = "Condition";
|
||||
|
||||
/// <summary>
|
||||
/// The three elements whose element children are observations. A ComponentStream may carry
|
||||
/// any subset of them, including none.
|
||||
/// </summary>
|
||||
private static readonly string[] ObservationContainers = ["Samples", "Events", ConditionContainer];
|
||||
|
||||
/// <summary>Parses a <c>/current</c> response, or one <c>/sample</c> chunk, held as a string.</summary>
|
||||
/// <param name="xml">The raw <c>MTConnectStreams</c> XML document.</param>
|
||||
/// <exception cref="InvalidDataException">
|
||||
/// The payload is empty, not well-formed XML, not an <c>MTConnectStreams</c> document (an
|
||||
/// Agent <c>MTConnectError</c> included — Agents answer a bad request with one under HTTP
|
||||
/// 200), carries no usable <c><Header></c>, or carries a malformed observation.
|
||||
/// </exception>
|
||||
public static MTConnectStreamsResult Parse(string xml)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(xml))
|
||||
{
|
||||
throw new InvalidDataException(
|
||||
"MTConnect streams response was empty; expected an MTConnectStreams XML document.");
|
||||
}
|
||||
|
||||
XDocument document;
|
||||
try
|
||||
{
|
||||
// A chunk lifted out of a multipart frame can carry a byte-order mark or the framing's
|
||||
// trailing line break; XmlReader rejects either ahead of the XML declaration.
|
||||
document = XDocument.Parse(xml.Trim('', ' ', '\t', '\r', '\n'));
|
||||
}
|
||||
catch (XmlException ex)
|
||||
{
|
||||
throw new InvalidDataException($"MTConnect streams response is not well-formed XML: {ex.Message}", ex);
|
||||
}
|
||||
|
||||
return Build(document);
|
||||
}
|
||||
|
||||
/// <summary>Parses a <c>/current</c> response read from a stream.</summary>
|
||||
/// <param name="stream">A stream positioned at the start of the XML document.</param>
|
||||
/// <exception cref="InvalidDataException">As for <see cref="Parse(string)"/>.</exception>
|
||||
public static MTConnectStreamsResult Parse(Stream stream)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(stream);
|
||||
|
||||
XDocument document;
|
||||
try
|
||||
{
|
||||
document = XDocument.Load(stream);
|
||||
}
|
||||
catch (XmlException ex)
|
||||
{
|
||||
throw new InvalidDataException($"MTConnect streams response is not well-formed XML: {ex.Message}", ex);
|
||||
}
|
||||
|
||||
return Build(document);
|
||||
}
|
||||
|
||||
private static MTConnectStreamsResult Build(XDocument document)
|
||||
{
|
||||
var root = document.Root
|
||||
?? throw new InvalidDataException("MTConnect streams response has no root element.");
|
||||
|
||||
if (!IsNamed(root, "MTConnectStreams"))
|
||||
{
|
||||
throw new InvalidDataException(DescribeUnexpectedRoot(root));
|
||||
}
|
||||
|
||||
var header = ChildrenNamed(root, "Header").FirstOrDefault()
|
||||
?? throw new InvalidDataException(
|
||||
"MTConnect streams response has no <Header>; without it the sequence bookkeeping the sample pump runs on is unknowable.");
|
||||
|
||||
return new MTConnectStreamsResult(
|
||||
RequiredLongAttribute(header, "instanceId"),
|
||||
RequiredLongAttribute(header, "nextSequence"),
|
||||
RequiredLongAttribute(header, "firstSequence"),
|
||||
ReadObservations(root));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Collects every observation in the document, in Agent-supplied order. Containers are found
|
||||
/// by descending the whole <c><Streams></c> subtree rather than by walking a fixed
|
||||
/// DeviceStream → ComponentStream chain, so a vendor's extra nesting level cannot silently
|
||||
/// hide a device's observations.
|
||||
/// </summary>
|
||||
private static IReadOnlyList<MTConnectObservation> ReadObservations(XElement root)
|
||||
{
|
||||
var observations = new List<MTConnectObservation>();
|
||||
|
||||
foreach (var container in root.Descendants().Where(IsObservationContainer))
|
||||
{
|
||||
var isCondition = IsNamed(container, ConditionContainer);
|
||||
foreach (var element in container.Elements())
|
||||
{
|
||||
observations.Add(ReadObservation(element, isCondition));
|
||||
}
|
||||
}
|
||||
|
||||
return observations;
|
||||
}
|
||||
|
||||
private static MTConnectObservation ReadObservation(XElement element, bool isCondition)
|
||||
{
|
||||
var dataItemId = RequiredAttribute(element, "dataItemId", $"Observation <{element.Name.LocalName}>");
|
||||
|
||||
return new MTConnectObservation(
|
||||
dataItemId,
|
||||
isCondition ? ConditionState(element) : element.Value.Trim(),
|
||||
ReadTimestampUtc(element, dataItemId));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A condition's state is its element name. <c>Unavailable</c> is normalized to the
|
||||
/// <see cref="UnavailableSentinel"/> spelling so downstream quality mapping has exactly one
|
||||
/// token to recognize; every other state is reported as-named (<c>Normal</c>, <c>Warning</c>,
|
||||
/// <c>Fault</c>, and any vendor state).
|
||||
/// </summary>
|
||||
private static string ConditionState(XElement element)
|
||||
{
|
||||
var state = element.Name.LocalName;
|
||||
|
||||
return string.Equals(state, "Unavailable", StringComparison.OrdinalIgnoreCase) ? UnavailableSentinel : state;
|
||||
}
|
||||
|
||||
private static DateTime ReadTimestampUtc(XElement element, string dataItemId)
|
||||
{
|
||||
var raw = RequiredAttribute(element, "timestamp", $"Observation '{dataItemId}'");
|
||||
|
||||
if (!DateTime.TryParse(
|
||||
raw,
|
||||
CultureInfo.InvariantCulture,
|
||||
DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal,
|
||||
out var parsed))
|
||||
{
|
||||
throw new InvalidDataException(
|
||||
$"MTConnect streams response is malformed: observation '{dataItemId}' has an unparseable 'timestamp' ('{raw}').");
|
||||
}
|
||||
|
||||
// AdjustToUniversal already yields Utc; stated explicitly so a future styles change cannot
|
||||
// quietly hand the OPC UA layer a Local or Unspecified SourceTimestamp.
|
||||
return DateTime.SpecifyKind(parsed, DateTimeKind.Utc);
|
||||
}
|
||||
|
||||
private static bool IsObservationContainer(XElement element) =>
|
||||
ObservationContainers.Any(name => IsNamed(element, name));
|
||||
|
||||
private static bool IsNamed(XElement element, string localName) =>
|
||||
string.Equals(element.Name.LocalName, localName, StringComparison.Ordinal);
|
||||
|
||||
private static IEnumerable<XElement> ChildrenNamed(XElement parent, string localName) =>
|
||||
parent.Elements().Where(child => IsNamed(child, localName));
|
||||
|
||||
private static string RequiredAttribute(XElement element, string name, string owner)
|
||||
{
|
||||
var value = OptionalAttribute(element, name);
|
||||
|
||||
return value ?? throw new InvalidDataException(
|
||||
$"MTConnect streams response is malformed: {owner} has no '{name}' attribute.");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Reads an unqualified attribute, normalizing a present-but-empty value to <c>null</c>.
|
||||
/// Only unqualified attributes are considered, so a namespace-prefixed vendor attribute can
|
||||
/// never be mistaken for an observation's identity or timestamp.
|
||||
/// </summary>
|
||||
private static string? OptionalAttribute(XElement element, string name)
|
||||
{
|
||||
var value = element.Attribute(name)?.Value;
|
||||
|
||||
return string.IsNullOrWhiteSpace(value) ? null : value;
|
||||
}
|
||||
|
||||
private static long RequiredLongAttribute(XElement header, string name)
|
||||
{
|
||||
var raw = RequiredAttribute(header, name, "The <Header>");
|
||||
|
||||
if (!long.TryParse(raw, NumberStyles.Integer, CultureInfo.InvariantCulture, out var value))
|
||||
{
|
||||
throw new InvalidDataException(
|
||||
$"MTConnect streams response is malformed: the <Header> has a non-numeric '{name}' attribute ('{raw}').");
|
||||
}
|
||||
|
||||
return value;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Builds the message for a root element that is not <c>MTConnectStreams</c>, lifting the
|
||||
/// Agent's own error text when the response is an <c>MTConnectError</c> document (which
|
||||
/// Agents return under HTTP 200 for, e.g., a <c>from</c> outside the buffer).
|
||||
/// </summary>
|
||||
private static string DescribeUnexpectedRoot(XElement root)
|
||||
{
|
||||
var message =
|
||||
$"MTConnect streams response root element is <{root.Name.LocalName}>, expected <MTConnectStreams>.";
|
||||
|
||||
if (!IsNamed(root, "MTConnectError"))
|
||||
{
|
||||
return message;
|
||||
}
|
||||
|
||||
var errors = root.Descendants()
|
||||
.Where(e => IsNamed(e, "Error"))
|
||||
.Select(e =>
|
||||
{
|
||||
var code = OptionalAttribute(e, "errorCode");
|
||||
var text = e.Value.Trim();
|
||||
|
||||
return code is null ? text : $"{code}: {text}";
|
||||
})
|
||||
.Where(text => text.Length > 0)
|
||||
.ToList();
|
||||
|
||||
return errors.Count == 0
|
||||
? message
|
||||
: $"{message} The Agent reported: {string.Join("; ", errors)}";
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user