Files
lmxopcua/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests/SparkplugCodecTests.cs
T
Joseph Doherty c74ccc55b5 feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):

  Template  → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
  DataSet   → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
  PropertySet → node metadata, never tags — it describes a metric rather than being one

Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.

**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.

**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).

**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.

The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.

Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.

Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.

Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 08:14:04 -04:00

440 lines
18 KiB
C#

using Google.Protobuf;
using Org.Eclipse.Tahu.Protobuf;
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug;
namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests;
/// <summary>
/// Covers <see cref="SparkplugCodec"/>: the golden NBIRTH/NDATA vectors decode into the driver-side
/// projection with explicit presence preserved, and no input — malformed, truncated, empty, or
/// simply not Sparkplug — makes the decoder throw.
/// </summary>
public sealed class SparkplugCodecTests
{
/// <summary>
/// The headline case from the plan: a golden NBIRTH yields its sequence number and its metric
/// catalog, with name, alias, datatype and value all surviving.
/// </summary>
[Fact]
public void Decode_GoldenNbirth_ExtractsMetrics()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
payload.IsValid.ShouldBeTrue();
payload.Seq.ShouldBe(0UL);
payload.Metrics.ShouldContain(m =>
m.Name == "Temperature"
&& m.Alias == 5UL
&& m.DataType == DataType.Float
&& Equals(m.Value, 21.5f));
}
/// <summary>
/// <b>Explicit presence, case 1.</b> A Sparkplug NBIRTH is REQUIRED to carry <c>seq = 0</c>, so a
/// decoder that reads absence off a zero value reports "no sequence" for every birth ever
/// published — and the sequence tracker then has no birth to anchor on.
/// </summary>
[Fact]
public void Decode_GoldenNbirth_SeqZero_IsPresentNotAbsent()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
payload.Seq.HasValue.ShouldBeTrue();
payload.Seq!.Value.ShouldBe(0UL);
}
/// <summary>
/// <b>Explicit presence, case 2.</b> A payload that genuinely omits <c>seq</c> must report
/// absence, not zero — the mirror image of the test above, and the pair is what makes the
/// distinction falsifiable rather than incidental.
/// </summary>
[Fact]
public void Decode_PayloadWithoutSeq_ReportsAbsentSeq()
{
var wire = new Payload { Timestamp = 1UL }.ToByteArray();
SparkplugCodec.Decode(wire).Seq.ShouldBeNull();
}
/// <summary>
/// <b>Explicit presence, case 3.</b> Every real Sparkplug DATA metric omits <c>name</c> and
/// <c>datatype</c> and carries only its alias. Reporting an absent name as the empty string
/// would make it indistinguishable from a (malformed) metric genuinely named "".
/// </summary>
[Fact]
public void Decode_GoldenNdata_MetricsCarryAliasOnly_NameAndDataTypeAbsent()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NdataFile));
payload.IsValid.ShouldBeTrue();
payload.Seq.ShouldBe(1UL);
payload.Metrics.Count.ShouldBe(3);
foreach (var metric in payload.Metrics)
{
metric.Name.ShouldBeNull();
metric.DataType.ShouldBeNull();
metric.Alias.HasValue.ShouldBeTrue();
}
payload.Metrics.ShouldContain(m => m.Alias == SparkplugGoldenPayloads.TemperatureAlias
&& Equals(m.Value, 22.25f));
}
/// <summary>
/// <b>Explicit presence, case 4.</b> "The metric's value is null" (<c>is_null</c>) is a different
/// fact from "the metric carried no value field", and both differ from "the value is zero".
/// </summary>
[Fact]
public void Decode_GoldenNbirth_ExplicitNullMetric_IsNullKind()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
var missing = payload.Metrics.Single(m => m.Name == "Missing");
missing.ValueKind.ShouldBe(SparkplugValueKind.Null);
missing.Value.ShouldBeNull();
missing.DataType.ShouldBe(DataType.Float);
}
/// <summary>
/// <b>Explicit presence, case 5 — the one that makes the other four falsifiable.</b> A publisher
/// may legitimately set a metric's name to the empty string, its datatype to <c>Unknown</c> (0)
/// and its timestamp to 0 (the Unix epoch). Those are present-and-default, and the golden
/// vectors alone cannot tell a presence check from a default check because nothing in them sets
/// a field to its own default. This pins the distinction directly: set-to-default must decode as
/// PRESENT, unset must decode as ABSENT, on every field that carries presence.
/// </summary>
[Fact]
public void Decode_FieldsSetToTheirOwnDefault_ArePresent_WhileUnsetFieldsAreAbsent()
{
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric
{
Name = string.Empty,
Alias = 0UL,
Datatype = (uint)DataType.Unknown,
Timestamp = 0UL,
IntValue = 0U,
});
payload.Metrics.Add(new Payload.Types.Metric { StringValue = "unset-everything-else" });
var decoded = SparkplugCodec.Decode(payload.ToByteArray());
var present = decoded.Metrics[0];
present.Name.ShouldBe(string.Empty);
present.Alias.ShouldBe(0UL);
present.DataType.ShouldBe(DataType.Unknown);
present.TimestampMs.ShouldBe(0UL);
var absent = decoded.Metrics[1];
absent.Name.ShouldBeNull();
absent.Alias.ShouldBeNull();
absent.DataType.ShouldBeNull();
absent.TimestampMs.ShouldBeNull();
}
/// <summary>A metric carrying neither a value nor <c>is_null</c> decodes as absent, not null.</summary>
[Fact]
public void Decode_MetricWithNoValueAndNoIsNull_IsAbsentKind()
{
var payload = new Payload { Seq = 4UL };
payload.Metrics.Add(new Payload.Types.Metric { Name = "Bare" });
var decoded = SparkplugCodec.Decode(payload.ToByteArray());
decoded.Metrics.Single().ValueKind.ShouldBe(SparkplugValueKind.Absent);
decoded.Metrics.Single().Value.ShouldBeNull();
}
/// <summary>
/// Every scalar arm of the metric value <c>oneof</c> projects to a CLR value of the wire's own
/// type — nothing is coerced here; coercion against the authored tag type happens downstream.
/// </summary>
[Fact]
public void Decode_GoldenNbirth_ProjectsEveryScalarValueArm()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
var byName = payload.Metrics.Where(m => m.Name is not null).ToDictionary(m => m.Name!, m => m);
byName["bdSeq"].Value.ShouldBe(7UL); // long_value → ulong
byName["Node Control/Rebirth"].Value.ShouldBe(false); // boolean_value → bool
byName["Temperature"].Value.ShouldBe(21.5f); // float_value → float
byName["Pressure"].Value.ShouldBe(101.325d); // double_value → double
byName["Running"].Value.ShouldBe(true);
byName["Serial"].Value.ShouldBe("EDGE-A-001"); // string_value → string
byName["Count"].Value.ShouldBe(unchecked((uint)-42)); // int_value → uint, RAW
}
/// <summary>A <c>bytes</c> metric projects to a byte array rather than a protobuf <c>ByteString</c>.</summary>
[Fact]
public void Decode_BytesMetric_ProjectsToByteArray()
{
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric
{
Name = "Blob",
Datatype = (uint)DataType.Bytes,
BytesValue = ByteString.CopyFrom(1, 2, 3),
});
var decoded = SparkplugCodec.Decode(payload.ToByteArray());
decoded.Metrics.Single().Value.ShouldBeOfType<byte[]>().ShouldBe(new byte[] { 1, 2, 3 });
}
/// <summary>
/// A <c>DataSet</c> metric decodes to <see cref="SparkplugValueKind.Structured"/> with its cells
/// flattened (Gitea #515) — never an exception on the MQTT dispatcher thread, and never a
/// silently-dropped metric that looks identical to one that was never published.
/// </summary>
[Fact]
public void Decode_GoldenNbirth_DataSetMetric_FlattensToCells()
{
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
var config = payload.Metrics.Single(m => m.Name == "Config");
config.ValueKind.ShouldBe(SparkplugValueKind.Structured);
// The PARENT carries no value — its members do. A parent that reported a value would be a
// guess about which cell "the" value is.
config.Value.ShouldBeNull();
config.DataType.ShouldBe(DataType.DataSet);
var cell = config.MemberList.Single();
cell.Path.ShouldBe("col0/0");
cell.DataType.ShouldBe(DataType.Int32);
cell.Value.ShouldBe(1U);
}
/// <summary>A <c>Template</c> metric flattens its members to sibling paths.</summary>
[Fact]
public void Decode_TemplateMetric_FlattensMembersToSiblingPaths()
{
var template = new Payload.Types.Template { Version = "1", IsDefinition = false };
template.Metrics.Add(new Payload.Types.Metric
{
Name = "Speed", Datatype = (uint)DataType.Double, DoubleValue = 12.5,
});
template.Metrics.Add(new Payload.Types.Metric
{
Name = "Running", Datatype = (uint)DataType.Boolean, BooleanValue = true,
});
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric
{
Name = "Motor",
Datatype = (uint)DataType.Template,
TemplateValue = template,
});
var metric = SparkplugCodec.Decode(payload.ToByteArray()).Metrics.Single();
metric.ValueKind.ShouldBe(SparkplugValueKind.Structured);
metric.MemberList.Select(m => m.Path).ShouldBe(["Speed", "Running"]);
metric.MemberList[0].Value.ShouldBe(12.5);
metric.MemberList[1].Value.ShouldBe(true);
}
/// <summary>
/// A template <b>definition</b> is refused rather than flattened, and refusal is still the
/// visible <see cref="SparkplugValueKind.Unsupported"/> kind.
/// </summary>
[Fact]
public void Decode_TemplateDefinition_StaysUnsupported()
{
// A definition is the UDT's SHAPE, published so consumers can understand instances — its member
// metrics carry no values. Flattening it would materialise a parallel set of tags that can
// never read, indistinguishable in the picker from the real instance's.
var definition = new Payload.Types.Template { Version = "1", IsDefinition = true };
definition.Metrics.Add(new Payload.Types.Metric { Name = "Speed", Datatype = (uint)DataType.Double });
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric
{
Name = "MotorType", Datatype = (uint)DataType.Template, TemplateValue = definition,
});
var metric = SparkplugCodec.Decode(payload.ToByteArray()).Metrics.Single();
metric.ValueKind.ShouldBe(SparkplugValueKind.Unsupported);
metric.MemberList.ShouldBeEmpty();
}
/// <summary>Payload and per-metric timestamps survive as raw Sparkplug epoch milliseconds.</summary>
[Fact]
public void Decode_GoldenVectors_PreserveTimestamps()
{
var nbirth = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
nbirth.TimestampMs.ShouldBe(SparkplugGoldenPayloads.NbirthTimestampMs);
nbirth.Metrics.Single(m => m.Name == "Temperature").TimestampMs
.ShouldBe(SparkplugGoldenPayloads.NbirthTimestampMs);
// A metric that carried no timestamp of its own reports absence, not the payload's.
nbirth.Metrics.Single(m => m.Name == "Pressure").TimestampMs.ShouldBeNull();
}
/// <summary>
/// Sparkplug carries signed integers up to 32 bits as two's complement in the UNSIGNED
/// <c>int_value</c> field. The codec hands over the raw wire value; this is the helper that
/// turns it back into the number the edge node meant.
/// </summary>
[Theory]
[InlineData(DataType.Int8, -5)]
[InlineData(DataType.Int16, -1234)]
[InlineData(DataType.Int32, -42)]
public void ReinterpretSigned_RecoversNegativeIntegersFromTheUnsignedWireField(DataType datatype, int expected)
{
var raw = unchecked((uint)expected);
var result = SparkplugCodec.ReinterpretSigned(raw, datatype);
Convert.ToInt64(result).ShouldBe(expected);
}
/// <summary>The 64-bit arm reinterprets <c>long_value</c>; the golden NBIRTH's Count is the 32-bit arm.</summary>
[Fact]
public void ReinterpretSigned_HandlesInt64_AndTheGoldenCountMetric()
{
SparkplugCodec.ReinterpretSigned(unchecked((ulong)-9_000_000_000L), DataType.Int64).ShouldBe(-9_000_000_000L);
var payload = SparkplugCodec.Decode(SparkplugGoldenPayloads.Read(SparkplugGoldenPayloads.NbirthFile));
var count = payload.Metrics.Single(m => m.Name == "Count");
SparkplugCodec.ReinterpretSigned(count.Value, count.DataType!.Value)
.ShouldBe(SparkplugGoldenPayloads.CountBirthValue);
}
/// <summary>Unsigned and non-integer datatypes pass through untouched.</summary>
[Fact]
public void ReinterpretSigned_LeavesUnsignedAndNonIntegerValuesAlone()
{
SparkplugCodec.ReinterpretSigned(7U, DataType.Uint32).ShouldBe(7U);
SparkplugCodec.ReinterpretSigned(21.5f, DataType.Float).ShouldBe(21.5f);
SparkplugCodec.ReinterpretSigned("text", DataType.String).ShouldBe("text");
SparkplugCodec.ReinterpretSigned(null, DataType.Int32).ShouldBeNull();
}
/// <summary>
/// A datatype index this build's enum does not define (a future Sparkplug revision, or a
/// misbehaving publisher) is preserved verbatim rather than dropped or coerced to Unknown —
/// the mapping layer decides what to do with it, and it cannot decide about data it never saw.
/// </summary>
[Fact]
public void Decode_UnrecognisedDataTypeIndex_IsPreservedVerbatim()
{
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric { Name = "Future", Datatype = 250U, IntValue = 1U });
var decoded = SparkplugCodec.Decode(payload.ToByteArray());
((int)decoded.Metrics.Single().DataType!.Value).ShouldBe(250);
}
/// <summary>
/// <b>Never throws, case 1.</b> Zero-length input is not "a payload with no metrics" — it is no
/// answer at all, and is reported as invalid so a truncated-to-nothing body cannot pass as a
/// well-formed message with an empty metric list.
/// </summary>
[Fact]
public void TryDecode_EmptyInput_IsInvalid_AndDoesNotThrow()
{
SparkplugCodec.TryDecode([], out var payload).ShouldBeFalse();
payload.IsValid.ShouldBeFalse();
payload.Metrics.ShouldBeEmpty();
payload.Seq.ShouldBeNull();
}
/// <summary>
/// <b>Never throws, case 2.</b> Garbage bytes: a decoder that let an exception escape here would
/// take down the MQTT dispatcher thread — every subscription on the connection, not one tag.
/// </summary>
[Fact]
public void TryDecode_GarbageBytes_NeverThrows()
{
var random = new Random(20260724);
for (var i = 0; i < 2000; i++)
{
var bytes = new byte[random.Next(1, 64)];
random.NextBytes(bytes);
// The verdict is irrelevant — some random byte strings ARE valid protobuf. Not throwing is
// the whole assertion.
Should.NotThrow(() => SparkplugCodec.TryDecode(bytes, out _));
}
}
/// <summary>
/// <b>Never throws, case 3.</b> Every truncation of a real payload — the shape a dropped
/// connection or a size-capped broker actually produces.
/// </summary>
[Fact]
public void TryDecode_EveryTruncationOfAGoldenVector_NeverThrows()
{
foreach (var file in new[] { SparkplugGoldenPayloads.NbirthFile, SparkplugGoldenPayloads.NdataFile })
{
var full = SparkplugGoldenPayloads.Read(file);
for (var length = 0; length <= full.Length; length++)
{
var prefix = full.AsSpan(0, length).ToArray();
Should.NotThrow(() => SparkplugCodec.TryDecode(prefix, out _));
}
}
}
/// <summary>
/// <b>Never throws, case 4.</b> A valid protobuf message that is not a Sparkplug payload. Protobuf
/// is permissive, so this may well parse (as unknown fields) — what matters is that it neither
/// throws nor invents metrics.
/// </summary>
[Fact]
public void TryDecode_ValidProtobufThatIsNotASparkplugPayload_NeverThrows()
{
// A DataSet message, serialized standalone — legal protobuf, wrong schema.
var alien = new Payload.Types.DataSet { NumOfColumns = 3UL };
alien.Columns.Add("a");
Should.NotThrow(() => SparkplugCodec.TryDecode(alien.ToByteArray(), out _));
}
/// <summary>
/// <b>Never throws, case 5.</b> A deeply nested Template — protobuf's recursion limit rejects it,
/// and the rejection must arrive as a verdict rather than as an exception.
/// </summary>
[Fact]
public void TryDecode_PathologicallyNestedTemplate_NeverThrows()
{
var template = new Payload.Types.Template { Version = "1" };
for (var depth = 0; depth < 200; depth++)
{
var outer = new Payload.Types.Template { Version = "1" };
outer.Metrics.Add(new Payload.Types.Metric { Name = "n", TemplateValue = template });
template = outer;
}
var payload = new Payload();
payload.Metrics.Add(new Payload.Types.Metric { Name = "Deep", TemplateValue = template });
Should.NotThrow(() => SparkplugCodec.TryDecode(payload.ToByteArray(), out _));
}
/// <summary>
/// <see cref="SparkplugCodec.Decode"/> is the convenience shape the plan's call sites use: it
/// never returns <see langword="null"/> and never throws, reporting failure through
/// <see cref="SparkplugPayload.IsValid"/>.
/// </summary>
[Fact]
public void Decode_OnUndecodableInput_ReturnsTheInvalidPayload_RatherThanThrowing()
{
var payload = SparkplugCodec.Decode([]);
payload.ShouldNotBeNull();
payload.IsValid.ShouldBeFalse();
payload.Metrics.ShouldBeEmpty();
}
}