dohertj2/lmxopcua
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)

Browse Source 
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):

  Template  → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
  DataSet   → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
  PropertySet → node metadata, never tags — it describes a metric rather than being one

Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.

**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.

**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).

**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.

The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.

Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.

Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.

Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-31 08:14:04 -04:00
parent 7f2108dcf2
commit c74ccc55b5
15 changed files with 1900 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/AliasTable.cs
+30
View File
@@ -128,6 +128,36 @@ public sealed class AliasTable
aliasCollisions++;
(collidedAliases ??= []).Add(alias);
}
// Gitea #515: a Template/DataSet metric also installs one binding per flattened member,
// named `{parent}/{memberPath}`. The parent's own binding above is kept — a DATA metric
// carries only the parent's ALIAS, so resolving that alias to the parent's name is the only
// way the member names can be composed at all. But the parent is not itself servable (its
// Sparkplug datatype maps to no DriverDataType), so it is the member bindings that browse
// enumerates and that authored tags bind to.
//
// Members carry NO alias, deliberately: aliases are per-metric on the wire, so inventing
// one here would create a number the edge node never assigned, which a later birth could
// legitimately hand to a different metric.
foreach (var member in metric.MemberList)
{
if (string.IsNullOrWhiteSpace(member.Path))
{
rejectedUnnamed++;
continue;
}
var memberBinding = new SparkplugMetricBinding(
$"{binding.Name}{SparkplugFlattener.MemberSeparator}{member.Path}",
Alias: null,
member.DataType ?? TahuDataType.Unknown);
if (!byName.TryAdd(memberBinding.Name, memberBinding))
{
duplicateNames++;
byName[memberBinding.Name] = memberBinding;
}
}
}
if (collidedAliases is not null)
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugIngestor.cs
+98 -3
View File
@@ -1177,6 +1177,15 @@ public sealed class SparkplugIngestor
SparkplugMetric metric,
DateTime? sourceUtc)
{
// Gitea #515: a Template/DataSet has no value of its own — its flattened members do. Expand
// here, at the single funnel both the DATA and the birth-value paths already pass through, so
// neither can end up supporting structured metrics while the other silently does not.
if (metric.ValueKind == SparkplugValueKind.Structured)
{
PublishStructuredMembers(scope, binding, metric, sourceUtc);
return;
}
// Bound by NAME. The alias got us here; it is never the key.
var key = new SparkplugMetricKey(scope.GroupId, scope.EdgeNodeId, scope.DeviceId, binding.Name);
if (!_authored.ByMetric.TryGetValue(key, out var defs))
@@ -1287,6 +1296,62 @@ public sealed class SparkplugIngestor
: BadTypeMismatch(def, binding, target, stamp, serverUtc);
}
/// <summary>
/// Publishes each flattened member of a structured (<c>Template</c> / <c>DataSet</c>) metric as
/// the sibling metric <c>{parent}/{memberPath}</c> (Gitea #515).
/// </summary>
/// <remarks>
/// <para>
/// <b>Each member is re-entered through <see cref="PublishMetric"/> as a scalar</b>, rather
/// than duplicating the coercion, array-gate and authored-tag fan-out here. A second copy of
/// that logic would be free to drift — and the drift would be silent, since both paths
/// publish values that look plausible.
/// </para>
/// <para>
/// <b>The member's datatype falls back to the BIRTH's.</b> A DDATA template usually
/// re-declares its members' datatypes, but it is not obliged to; the binding installed at
/// birth is the authority when it does not. Guessing from the CLR type of the wire value
/// would type a signed Int32 of -42 as a UInt32 of 4294967254 — the exact
/// plausible-and-wrong failure <c>ReinterpretSigned</c> exists to prevent.
/// </para>
/// </remarks>
private void PublishStructuredMembers(
SparkplugScope scope,
SparkplugMetricBinding parent,
SparkplugMetric metric,
DateTime? sourceUtc)
{
var table = Births.Find(scope);
foreach (var member in metric.MemberList)
{
if (string.IsNullOrWhiteSpace(member.Path))
{
continue;
}
var name = $"{parent.Name}{SparkplugFlattener.MemberSeparator}{member.Path}";
// Wire datatype first, birth binding second. Unknown last — which the typed layer refuses
// once, loudly, rather than coercing.
var dataType = member.DataType
?? table?.ResolveByName(name)?.DataType
?? Org.Eclipse.Tahu.Protobuf.DataType.Unknown;
PublishMetric(
scope,
new SparkplugMetricBinding(name, Alias: null, dataType),
new SparkplugMetric(
Name: name,
Alias: null,
DataType: dataType,
TimestampMs: metric.TimestampMs,
ValueKind: member.ValueKind,
Value: member.Value),
sourceUtc);
}
}
/// <summary>
/// The metric decoded, but does not fit the tag's effective type. Warned once per tag; the value
/// itself is deliberately never logged — a payload value is plant data, not diagnostics.
@@ -1370,9 +1435,7 @@ public sealed class SparkplugIngestor
try
{
subscribers(this, new SparkplugBirthObservedEventArgs(
scope,
[.. payload.Metrics.Where(m => m.Name is not null).Select(m => m.Name!)]));
subscribers(this, new SparkplugBirthObservedEventArgs(scope, [.. DiscoverableNames(payload)]));
}
catch (Exception ex)
{
@@ -1380,6 +1443,38 @@ public sealed class SparkplugIngestor
}
}
/// <summary>
/// The metric names a birth contributes to the <i>discoverable address space</i> — which is
/// what the rediscovery change gate must compare, not the raw wire names.
/// </summary>
/// <remarks>
/// Gitea #515: a structured metric contributes its flattened MEMBERS and not itself. Comparing
/// raw wire names instead would make the gate blind to the change that matters most for a UDT:
/// a template gaining or losing a member leaves the parent metric's name identical, so the
/// signature would not move and the operator would never be told the tree changed.
/// </remarks>
private static IEnumerable<string> DiscoverableNames(SparkplugPayload payload)
{
foreach (var metric in payload.Metrics)
{
if (metric.Name is not { } name)
{
continue;
}
if (metric.ValueKind != SparkplugValueKind.Structured)
{
yield return name;
continue;
}
foreach (var member in metric.MemberList)
{
yield return $"{name}{SparkplugFlattener.MemberSeparator}{member.Path}";
}
}
}
// ---- sequence + rebirth ----
private SequenceTracker TrackerFor(SparkplugNodeKey node) =>