feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)

`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):

  Template  → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
  DataSet   → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
  PropertySet → node metadata, never tags — it describes a metric rather than being one

Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.

**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.

**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).

**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.

The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.

Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.

Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.

Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-31 08:14:04 -04:00
parent 7f2108dcf2
commit c74ccc55b5
15 changed files with 1900 additions and 53 deletions
@@ -585,20 +585,77 @@ internal sealed class MqttBrowseSession : IRebirthCapableBrowseSession
continue;
}
var metricPath = ownerPath + MetricSeparator + name;
if (metricPath.Length > MaxTopicChars)
// Gitea #515: a Template/DataSet metric contributes its flattened MEMBERS as leaves, never
// itself. The parent has no value and no servable datatype, so a leaf for it would offer
// the operator an address that can only ever commit a tag which never reads.
if (metric.ValueKind == SparkplugValueKind.Structured)
{
Volatile.Write(ref _oversized, 1);
foreach (var member in metric.MemberList)
{
var memberName = $"{name}{SparkplugFlattener.MemberSeparator}{member.Path}";
if (!IsUsableLabel(memberName))
{
Volatile.Write(ref _oversized, 1);
continue;
}
if (!AddMetricLeaf(owner, ownerPath, ownerScope, memberName, member.DataType, parsed.Type, alias: null))
{
return; // node cap tripped
}
}
continue;
}
var node = GetOrCreate(owner.Children, metricPath, name, BrowseNodeKind.Leaf, ownerScope);
if (node is null) return; // node cap tripped
node.RecordMetric(new SparkplugMetricInfo(name, metric.DataType, parsed.Type, metric.Alias));
if (!AddMetricLeaf(owner, ownerPath, ownerScope, name, metric.DataType, parsed.Type, metric.Alias))
{
return; // node cap tripped
}
}
}
/// <summary>
/// Places one metric leaf under <paramref name="owner"/>.
/// </summary>
/// <param name="owner">The owning edge-node / device folder.</param>
/// <param name="ownerPath">That folder's path.</param>
/// <param name="ownerScope">The scope stamped on the leaf.</param>
/// <param name="metricName">
/// The metric's Sparkplug name — for a flattened structured member, the composed
/// <c>{parent}/{memberPath}</c>. It travels intact: it is the binding key, and only the
/// <i>display</i> name is later coerced to a legal RawPath segment (Gitea #509).
/// </param>
/// <param name="dataType">The declared datatype, or null.</param>
/// <param name="messageType">The birth message type this leaf was seen in.</param>
/// <param name="alias">The metric's alias, or null — flattened members carry none of their own.</param>
/// <returns><see langword="false"/> when the node cap tripped and the caller must stop.</returns>
private bool AddMetricLeaf(
TopicNode owner,
string ownerPath,
SparkplugScope ownerScope,
string metricName,
Org.Eclipse.Tahu.Protobuf.DataType? dataType,
SparkplugMessageType messageType,
ulong? alias)
{
var metricPath = ownerPath + MetricSeparator + metricName;
if (metricPath.Length > MaxTopicChars)
{
Volatile.Write(ref _oversized, 1);
return true;
}
var node = GetOrCreate(owner.Children, metricPath, metricName, BrowseNodeKind.Leaf, ownerScope);
if (node is null)
{
return false;
}
node.RecordMetric(new SparkplugMetricInfo(metricName, dataType, messageType, alias));
return true;
}
/// <summary>
/// Finds or creates one tree node under <paramref name="level"/>, honouring the node cap.
/// </summary>
@@ -698,6 +755,40 @@ internal sealed class MqttBrowseSession : IRebirthCapableBrowseSession
ObserveBirth(groupId, edgeNodeId, device, metric);
}
/// <summary>
/// Test seam feeding a birth whose metric is a structured <c>Template</c> (Gitea #515).
/// </summary>
/// <param name="groupId">The Sparkplug group id.</param>
/// <param name="edgeNodeId">The Sparkplug edge-node id.</param>
/// <param name="device">The device id, or <see langword="null"/> for an NBIRTH.</param>
/// <param name="metricName">The template metric's own name.</param>
/// <param name="members">The template's members, as <c>(name, datatype)</c>.</param>
internal void ObserveTemplateBirthForTest(
string groupId,
string edgeNodeId,
string? device,
string metricName,
params (string Name, SparkplugDataType DataType)[] members)
{
var template = new Payload.Types.Template { Version = "1", IsDefinition = false };
foreach (var (name, dataType) in members)
{
var member = new Payload.Types.Metric { Name = name, Datatype = (uint)dataType };
// A value is required: a member carrying only a name and a datatype is indistinguishable
// from a template DEFINITION's member, which the flattener deliberately refuses.
member.DoubleValue = 0;
template.Metrics.Add(member);
}
ObserveBirth(groupId, edgeNodeId, device, new Payload.Types.Metric
{
Name = metricName,
Datatype = (uint)SparkplugDataType.Template,
TemplateValue = template,
});
}
/// <summary>Test seam feeding a birth certificate that declares no metrics at all.</summary>
/// <param name="groupId">The Sparkplug group id.</param>
/// <param name="edgeNodeId">The Sparkplug edge-node id.</param>