feat(mqtt): Sparkplug UntilStable discovery + rediscover-on-DBIRTH

RediscoverPolicy is now mode-dependent: UntilStable in Sparkplug B (a tag's
dataType is optional -- the birth certificate declares it, so the discovered
tree fills in asynchronously), Once in Plain (nothing about the authored set
arrives on the wire).

DiscoverAsync resolves each Sparkplug tag's datatype per pass from the live
BirthCache, with the ingest path's own precedence: authored dataType wins,
else the birth's, else the record default. An unsupported Sparkplug type
(DataSet/Template/PropertySet/Unknown) falls back rather than blanking the
tag. The authored tag SET is never conditional on a birth -- an authored tag
is part of the declared configuration, not something the plant grants by
publishing.

SparkplugIngestor.BirthObserved is wired to RaiseRediscoveryNeeded behind a
two-part change gate, which is the anti-storm mechanism rather than an
optimisation: fire only for a scope an authored tag binds, and only when the
birth's metric-name SET (ordered-distinct, ordinal) differs from the last one
seen for that scope. Edge nodes re-birth freely -- on their own timer, on
every reconnect via the late-join rebirth fan-out, and once per rebirth NCMD
the gap policy sends -- and firing on each of those would make a healthy plant
a permanent address-space-rebuild loop. No extra debounce: what survives the
gate is a real edit to the served tree, and delaying it would need its own
trailing-edge flush to avoid dropping the last change.

The signature map is NOT cleared on death/reconnect/ingest rebuild (the
ingestor drops its whole birth cache on reconnect, but "the driver forgot" is
not "the address space changed"); it is pruned when a redeploy stops
authoring a scope.

ScopeHint is the "Mqtt" discovery folder, deliberately not the Sparkplug scope
path the plan named: the discovered tree is flat, so an edge-node/device path
would name a subtree that does not exist and a consumer scoping a rebuild on
it would rebuild nothing. The scope is carried in Reason instead.

Falsifiability: always-fire reddens the identical-rebirth + reordered-rebirth
pins; always-authored-datatype reddens the birth-fill pin; dropping the
authored-scope filter reddens the unauthored-device pin. 524/524 MQTT unit
tests green (was 513).

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-24 22:28:08 -04:00
parent 8538fb798b
commit 6917d8805d
2 changed files with 561 additions and 19 deletions
@@ -1,3 +1,5 @@
using System.Collections.Concurrent;
using System.Collections.Frozen;
using System.Text.Json; using System.Text.Json;
using Microsoft.Extensions.Logging; using Microsoft.Extensions.Logging;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions; using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
@@ -18,9 +20,10 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt;
/// <see cref="DiscoverAsync"/> therefore replays exactly the raw tags the deploy artifact /// <see cref="DiscoverAsync"/> therefore replays exactly the raw tags the deploy artifact
/// authored (<see cref="MqttDriverOptions.RawTags"/>) and nothing else, and /// authored (<see cref="MqttDriverOptions.RawTags"/>) and nothing else, and
/// <see cref="ITagDiscovery.SupportsOnlineDiscovery"/> stays <see langword="false"/> so the /// <see cref="ITagDiscovery.SupportsOnlineDiscovery"/> stays <see langword="false"/> so the
/// universal browser never treats broker traffic as a tag catalogue. The policy is /// universal browser never treats broker traffic as a tag catalogue. In
/// <see cref="DiscoveryRediscoverPolicy.Once"/>: the authored set is fully known /// <see cref="MqttMode.Plain"/> the policy is <see cref="DiscoveryRediscoverPolicy.Once"/>:
/// synchronously, so re-running discovery on a timer can only produce the same tree. /// the authored set — and every authored tag's datatype — is fully known synchronously, so
/// re-running discovery on a timer can only produce the same tree.
/// </para> /// </para>
/// <para> /// <para>
/// <b>Two ingest paths, one driver.</b> <see cref="MqttMode.Plain"/> routes through /// <b>Two ingest paths, one driver.</b> <see cref="MqttMode.Plain"/> routes through
@@ -32,12 +35,18 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt;
/// driver-owned <see cref="LastValueCache"/>, so <c>IReadable</c> reads one store either way. /// driver-owned <see cref="LastValueCache"/>, so <c>IReadable</c> reads one store either way.
/// </para> /// </para>
/// <para> /// <para>
/// <b><see cref="OnRediscoveryNeeded"/> still never fires.</b> Sparkplug B is the mode that /// <b><see cref="OnRediscoveryNeeded"/> fires in Sparkplug B only, and only on a real change.</b>
/// eventually raises it — a DBIRTH can introduce metrics the previous birth did not carry — /// A Sparkplug tag's <c>dataType</c> is optional — the birth certificate declares it — so the
/// but the policy flip to <see cref="DiscoveryRediscoverPolicy.UntilStable"/> and the wiring /// discovered tree fills in asynchronously and the policy is
/// from <see cref="SparkplugIngestor.BirthObserved"/> to /// <see cref="DiscoveryRediscoverPolicy.UntilStable"/>. <see cref="SparkplugIngestor.BirthObserved"/>
/// <see cref="RaiseRediscoveryNeeded"/> are Task 22. The seams exist on both sides; nothing /// is wired to <see cref="RaiseRediscoveryNeeded"/> through
/// connects them yet. /// <see cref="OnBirthObserved"/>, which fires <b>only</b> when the birth's metric-name SET
/// differs from the last one seen for that scope (and only for scopes an authored tag actually
/// binds). That gate is the anti-storm mechanism, not an optimisation: rediscovery triggers an
/// address-space rebuild, edge nodes re-birth freely — on their own timer, on every reconnect
/// via the late-join rebirth, and once per NCMD the ingestor's gap policy sends — and firing on
/// each of those would turn a healthy plant into a permanent rebuild loop. Plain mode never
/// raises it at all; its authored set changes only by redeploy.
/// </para> /// </para>
/// <para> /// <para>
/// <b>Composition order is load-bearing.</b> <see cref="MqttSubscriptionManager.AttachTo"/> /// <b>Composition order is load-bearing.</b> <see cref="MqttSubscriptionManager.AttachTo"/>
@@ -65,6 +74,13 @@ public sealed class MqttDriver
/// </summary> /// </summary>
private const long ApproxBytesPerAuthoredTag = 512; private const long ApproxBytesPerAuthoredTag = 512;
/// <summary>
/// The single folder <see cref="DiscoverAsync"/> streams every variable under — and therefore
/// the only subtree path a rediscovery <c>ScopeHint</c> can name truthfully. See
/// <see cref="OnBirthObserved"/>.
/// </summary>
private const string DiscoveryFolderName = "Mqtt";
/// <summary>How often the host-connectivity poll samples <see cref="MqttConnection.State"/>.</summary> /// <summary>How often the host-connectivity poll samples <see cref="MqttConnection.State"/>.</summary>
private static readonly TimeSpan HostProbeInterval = TimeSpan.FromSeconds(1); private static readonly TimeSpan HostProbeInterval = TimeSpan.FromSeconds(1);
@@ -80,6 +96,23 @@ public sealed class MqttDriver
private readonly object _hostLock = new(); private readonly object _hostLock = new();
/// <summary>
/// Per authored Sparkplug scope, the signature of the metric-name set its most recent birth
/// declared — the rediscovery change gate's memory. Written on MQTTnet's dispatcher thread and
/// read nowhere else, but concurrent because a reconnect's birth flood and the dispatcher are
/// not guaranteed to be the same thread forever.
/// </summary>
/// <remarks>
/// <b>Deliberately NOT cleared on a death, a reconnect or an ingest rebuild.</b> The ingestor
/// drops its whole <see cref="SparkplugIngestor.Births"/> cache on every reconnect (an alias may
/// have been rebound while the driver was away) — but "the driver forgot" is not "the address
/// space changed". Clearing this alongside it would make every reconnect's late-join rebirth
/// flood look like a tree full of brand-new scopes and fire one rediscovery per authored scope
/// on every single flap, which is the storm this map exists to prevent. It survives so that an
/// identical rebirth stays silent.
/// </remarks>
private readonly ConcurrentDictionary<SparkplugScope, string> _birthSignatures = new();
/// <summary>Guards <see cref="InitializeAsync"/> / <see cref="ReinitializeAsync"/> / <see cref="ShutdownAsync"/> against each other.</summary> /// <summary>Guards <see cref="InitializeAsync"/> / <see cref="ReinitializeAsync"/> / <see cref="ShutdownAsync"/> against each other.</summary>
private readonly SemaphoreSlim _lifecycleGate = new(1, 1); private readonly SemaphoreSlim _lifecycleGate = new(1, 1);
@@ -109,6 +142,23 @@ public sealed class MqttDriver
/// </summary> /// </summary>
private IReadOnlyList<string> _authoredRawPaths = []; private IReadOnlyList<string> _authoredRawPaths = [];
/// <summary>
/// The Sparkplug scopes at least one authored tag binds — empty outside
/// <see cref="MqttMode.SparkplugB"/>. Rebuilt wholesale by <see cref="RegisterAuthoredTags"/>
/// alongside the authored table it is derived from.
/// </summary>
/// <remarks>
/// The rediscovery gate's first question. The ingestor applies a birth for <i>any</i> device
/// under an authored edge node (it has to — a DBIRTH is a sequenced member of that node's
/// stream), so on a large plant it observes births for devices this deployment never reads. Such
/// a birth cannot change <see cref="DiscoverAsync"/>'s output by construction, because discovery
/// replays the authored set — so firing rediscovery for it would rebuild the address space in
/// response to traffic the configuration has no opinion about. Gating on this set also bounds
/// <see cref="_birthSignatures"/> by <i>configuration</i> rather than by whatever the plant
/// happens to publish — the same rule the ingestor's own authored-edge-node filter follows.
/// </remarks>
private FrozenSet<SparkplugScope> _authoredScopes = FrozenSet<SparkplugScope>.Empty;
private MqttConnection? _connection; private MqttConnection? _connection;
private CancellationTokenSource? _hostProbeCts; private CancellationTokenSource? _hostProbeCts;
@@ -357,11 +407,29 @@ public sealed class MqttDriver
/// <inheritdoc /> /// <inheritdoc />
/// <remarks> /// <remarks>
/// Plain mode discovers synchronously from the authored set, so one pass is all there is. /// <para>
/// Sparkplug B (P2) fills its metric set in asynchronously from birth certificates and /// Plain mode discovers synchronously from the authored set — the RawPaths <i>and</i> their
/// switches this to <see cref="DiscoveryRediscoverPolicy.UntilStable"/>. /// datatypes are both in the deploy artifact — so one pass is all there is.
/// </para>
/// <para>
/// Sparkplug B is <see cref="DiscoveryRediscoverPolicy.UntilStable"/> because half of what
/// it discovers arrives on the wire: <c>dataType</c> is optional in the Sparkplug tag shape
/// (the birth certificate declares it), so a tag authored with nothing but its
/// <c>(group, node, device, metric)</c> tuple can only be streamed with the record's default
/// type until its NBIRTH/DBIRTH lands. The host's <c>UntilStable</c> contract — re-run
/// discovery until the captured set is non-empty and its signature repeats — is exactly the
/// retry that lets those passes catch a birth that had not yet arrived at connect.
/// </para>
/// <para>
/// <b>The retry is a floor, not the mechanism.</b> The host's stability signature is built
/// from FullReferences alone, so a pass whose only change is a filled-in datatype still
/// looks "stable" and the loop stops. That is why the same fill-in also fires
/// <see cref="OnRediscoveryNeeded"/> (see <see cref="OnBirthObserved"/>): a birth arriving
/// after the loop settled is the case the timer cannot cover.
/// </para>
/// </remarks> /// </remarks>
public DiscoveryRediscoverPolicy RediscoverPolicy => DiscoveryRediscoverPolicy.Once; public DiscoveryRediscoverPolicy RediscoverPolicy =>
IsSparkplug ? DiscoveryRediscoverPolicy.UntilStable : DiscoveryRediscoverPolicy.Once;
/// <inheritdoc /> /// <inheritdoc />
/// <remarks> /// <remarks>
@@ -375,10 +443,21 @@ public sealed class MqttDriver
/// Streams the authored tag set — and only that — into <paramref name="builder"/>. /// Streams the authored tag set — and only that — into <paramref name="builder"/>.
/// </summary> /// </summary>
/// <remarks> /// <remarks>
/// The enumeration source is the driver's own list of registered RawPaths, never anything /// <para>
/// observed on the wire, so no amount of broker traffic can add a node. A RawPath whose /// The enumeration source is the driver's own list of registered RawPaths, never anything
/// TagConfig failed to map is absent from the manager and is skipped here too — it surfaces /// observed on the wire, so no amount of broker traffic can add a node. A RawPath whose
/// as <c>BadNodeIdUnknown</c> on read rather than as a wrongly-typed variable. /// TagConfig failed to map is absent from the manager and is skipped here too — it surfaces
/// as <c>BadNodeIdUnknown</c> on read rather than as a wrongly-typed variable.
/// </para>
/// <para>
/// <b>Sparkplug B: the datatype is resolved per pass, the tag set never is.</b> An authored
/// tag is streamed on <i>every</i> pass, birth or no birth — it is part of the declared
/// configuration, not something the plant grants by publishing, and hiding it until a birth
/// arrives would make the node's existence depend on wire traffic (and, on a deployment
/// whose other tags do carry an authored type, would hide it behind a "non-empty and stable"
/// verdict the host reaches without it). What a birth changes is the <i>type</i>: see
/// <see cref="ResolveDiscoveredDataType"/>.
/// </para>
/// </remarks> /// </remarks>
/// <param name="builder">The address space builder to stream discovered nodes into.</param> /// <param name="builder">The address space builder to stream discovered nodes into.</param>
/// <param name="cancellationToken">Cancellation for the discovery operation.</param> /// <param name="cancellationToken">Cancellation for the discovery operation.</param>
@@ -388,7 +467,7 @@ public sealed class MqttDriver
ArgumentNullException.ThrowIfNull(builder); ArgumentNullException.ThrowIfNull(builder);
cancellationToken.ThrowIfCancellationRequested(); cancellationToken.ThrowIfCancellationRequested();
var folder = builder.Folder("Mqtt", "Mqtt"); var folder = builder.Folder(DiscoveryFolderName, DiscoveryFolderName);
foreach (var rawPath in _authoredRawPaths) foreach (var rawPath in _authoredRawPaths)
{ {
if (!TryResolve(rawPath, out var def)) if (!TryResolve(rawPath, out var def))
@@ -398,7 +477,7 @@ public sealed class MqttDriver
folder.Variable(def.Name, def.Name, new DriverAttributeInfo( folder.Variable(def.Name, def.Name, new DriverAttributeInfo(
FullName: def.Name, FullName: def.Name,
DriverDataType: def.DataType, DriverDataType: ResolveDiscoveredDataType(def),
IsArray: false, IsArray: false,
ArrayDim: null, ArrayDim: null,
// MQTT ingest is one-way in P1 — this driver implements no IWritable leg, so every // MQTT ingest is one-way in P1 — this driver implements no IWritable leg, so every
@@ -412,6 +491,43 @@ public sealed class MqttDriver
return Task.CompletedTask; return Task.CompletedTask;
} }
/// <summary>
/// The datatype <see cref="DiscoverAsync"/> reports for one authored tag: the authored type when
/// the operator declared one, otherwise the type the metric's live birth certificate declared,
/// otherwise the record's default.
/// </summary>
/// <remarks>
/// <para>
/// <b>The precedence is the ingest path's, restated.</b> <c>SparkplugIngestor.PublishMetric</c>
/// coerces every value with <c>DataTypeAuthored ? authored : birth</c>; a discovery surface
/// that disagreed with it would advertise a type no value ever arrives as. The two must be
/// changed together.
/// </para>
/// <para>
/// An unsupported Sparkplug type (DataSet / Template / PropertySet / Unknown) maps to
/// <see langword="null"/> and falls back rather than blanking the tag — the ingest path
/// already refuses those metrics once, loudly, and a discovery pass is not the place to
/// relitigate it.
/// </para>
/// </remarks>
/// <param name="def">The authored tag definition.</param>
/// <returns>The effective data type for this pass.</returns>
private DriverDataType ResolveDiscoveredDataType(MqttTagDefinition def)
{
if (def.DataTypeAuthored || _sparkplug is not { } sparkplug)
{
return def.DataType;
}
if (def.GroupId is not { } groupId || def.EdgeNodeId is not { } edgeNodeId || def.MetricName is not { } metric)
{
return def.DataType;
}
var table = sparkplug.Births.Find(new SparkplugScope(groupId, edgeNodeId, def.DeviceId));
return table?.ResolveByName(metric)?.DataType.ToDriverDataType() ?? def.DataType;
}
// ---- ISubscribable (straight through to the manager) ---- // ---- ISubscribable (straight through to the manager) ----
/// <inheritdoc /> /// <inheritdoc />
@@ -495,6 +611,85 @@ public sealed class MqttDriver
internal void RaiseRediscoveryNeeded(string reason, string? scopeHint = null) internal void RaiseRediscoveryNeeded(string reason, string? scopeHint = null)
=> OnRediscoveryNeeded?.Invoke(this, new RediscoveryEventArgs(reason, scopeHint)); => OnRediscoveryNeeded?.Invoke(this, new RediscoveryEventArgs(reason, scopeHint));
/// <summary>
/// The rediscovery decision: a birth changed what <see cref="DiscoverAsync"/> can produce only
/// when it is for an <b>authored</b> scope <b>and</b> its metric-name set differs from the last
/// one seen for that scope. Anything else is dropped.
/// </summary>
/// <remarks>
/// <para>
/// <b>Both conditions are load-bearing, and both are anti-storm.</b> Rediscovery triggers an
/// address-space rebuild — expensive and disruptive on a running server. An edge node
/// re-births freely: on its own schedule, on every reconnect (the ingestor's late-join
/// rebirth fan-out), and once per rebirth NCMD its gap policy sends. Firing on each of those
/// is a permanent rebuild loop against a plant that is behaving perfectly. Firing on
/// <i>changed metric set only</i> is the primary defence; the authored-scope filter is the
/// second, and removes the whole class of churn sourced from devices this deployment does
/// not read (see <see cref="_authoredScopes"/>).
/// </para>
/// <para>
/// <b>The signature is order-insensitive</b> — ordered-distinct, ordinal. Nothing obliges an
/// edge node to keep its birth's metric ordering stable, and a reordered birth describes the
/// same address space; comparing the raw sequence would leak the storm back in.
/// </para>
/// <para>
/// <b>No additional debounce or coalescing.</b> One was considered and rejected: the change
/// gate already collapses the unbounded sources (rebirth timers, reconnect floods, NCMD
/// answers) to zero, and what survives it is a genuine metric-set change — of which a given
/// scope has a handful in its lifetime, each one a real edit to the served tree. A timer on
/// top would only delay a rebuild that must happen, and would need its own trailing-edge
/// flush to avoid dropping the last change entirely. If a consumer ever needs coalescing
/// across many scopes birthing at once (a whole-plant restart), that belongs in the consumer,
/// which alone knows what a rebuild costs it — today there is no such consumer at all.
/// </para>
/// <para>
/// <b>Runs on MQTTnet's dispatcher thread</b> (via the ingestor's own contained raise), so it
/// does no I/O and never throws — <see cref="SparkplugIngestor"/> catches a throwing
/// subscriber, but relying on that would still have cost the rest of the birth's fan-out.
/// </para>
/// </remarks>
private void OnBirthObserved(object? sender, SparkplugBirthObservedEventArgs e)
{
if (!_authoredScopes.Contains(e.Scope))
{
// A birth for a scope no authored tag binds cannot change the authored-only discovery tree.
return;
}
var signature = BirthSignature(e.MetricNames);
if (_birthSignatures.TryGetValue(e.Scope, out var previous)
&& string.Equals(previous, signature, StringComparison.Ordinal))
{
return; // Same metric set as last time — nothing to rediscover.
}
_birthSignatures[e.Scope] = signature;
var reason = previous is null
? $"Sparkplug birth observed for '{e.Scope}' ({e.MetricNames.Count} metric(s))"
: $"Sparkplug rebirth changed the metric set for '{e.Scope}' ({e.MetricNames.Count} metric(s))";
_logger?.LogInformation(
"MQTT driver '{DriverId}': {Reason}; requesting rediscovery.",
_driverInstanceId,
reason);
// ScopeHint names the folder DiscoverAsync actually streams under — deliberately NOT the
// Sparkplug scope path. The discovered tree is flat (one 'Mqtt' folder of RawPath-named
// variables); it has no edge-node/device folders, so a hint of 'Plant1/EdgeA/Filler1' would
// name a subtree that does not exist and a consumer scoping a rebuild on it would rebuild
// nothing. The scope belongs in the Reason, which is the documented diagnostic field.
RaiseRediscoveryNeeded(reason, DiscoveryFolderName);
}
/// <summary>An order-insensitive, ordinal signature of a birth's metric-name set.</summary>
/// <param name="metricNames">The names the birth declared.</param>
/// <returns>The comparable signature.</returns>
private static string BirthSignature(IReadOnlyList<string> metricNames)
=> string.Join(
'\u0001', // A separator no Sparkplug metric name can contain, so two sets cannot alias.
metricNames.Distinct(StringComparer.Ordinal).Order(StringComparer.Ordinal));
// ---- disposal ---- // ---- disposal ----
/// <inheritdoc /> /// <inheritdoc />
@@ -764,6 +959,12 @@ public sealed class MqttDriver
// The published reference is the RawPath the ingestor already stamped on the args; the driver // The published reference is the RawPath the ingestor already stamped on the args; the driver
// only re-raises with itself as sender. // only re-raises with itself as sender.
ingestor.OnDataChange += (_, args) => OnDataChange?.Invoke(this, args); ingestor.OnDataChange += (_, args) => OnDataChange?.Invoke(this, args);
// The ingestor detects a birth; the driver decides whether it changed the address space. Wired
// here rather than in InitializeAsync so a driver rebuilt for a config delta re-arms it — an
// ingest rebuild that silently dropped this subscription would leave a Sparkplug driver whose
// datatypes fill in and whose rediscovery never fires again, with nothing to show for it.
ingestor.BirthObserved += OnBirthObserved;
return ingestor; return ingestor;
} }
@@ -779,6 +980,20 @@ public sealed class MqttDriver
// Enumerate in authoring order; the manager owns which of these actually mapped, and // Enumerate in authoring order; the manager owns which of these actually mapped, and
// DiscoverAsync skips the rest. // DiscoverAsync skips the rest.
_authoredRawPaths = [.. options.RawTags.Select(t => t.RawPath)]; _authoredRawPaths = [.. options.RawTags.Select(t => t.RawPath)];
_authoredScopes = BuildAuthoredScopes();
// Drop change-gate memory for scopes this deploy stopped authoring — the same housekeeping
// SparkplugIngestor.Register does for its trackers, and for the same reason: without it a scope
// nobody reads keeps a slot across every future redeploy. Re-authoring such a scope later means
// its next birth reads as new and fires once, which is correct — a newly authored scope IS a
// change to the discovered tree.
foreach (var scope in _birthSignatures.Keys)
{
if (!_authoredScopes.Contains(scope))
{
_birthSignatures.TryRemove(scope, out _);
}
}
if (mapped != options.RawTags.Count) if (mapped != options.RawTags.Count)
{ {
@@ -791,6 +1006,33 @@ public sealed class MqttDriver
} }
} }
/// <summary>
/// The Sparkplug scopes the currently registered authored tags bind, derived from the same
/// definitions <see cref="DiscoverAsync"/> enumerates — empty outside
/// <see cref="MqttMode.SparkplugB"/>, and empty for any tag whose TagConfig did not map.
/// </summary>
/// <returns>The authored scope set.</returns>
private FrozenSet<SparkplugScope> BuildAuthoredScopes()
{
if (_sparkplug is null)
{
return FrozenSet<SparkplugScope>.Empty;
}
var scopes = new HashSet<SparkplugScope>();
foreach (var rawPath in _authoredRawPaths)
{
if (TryResolve(rawPath, out var def)
&& def.GroupId is { } groupId
&& def.EdgeNodeId is { } edgeNodeId)
{
scopes.Add(new SparkplugScope(groupId, edgeNodeId, def.DeviceId));
}
}
return scopes.ToFrozenSet();
}
/// <summary> /// <summary>
/// Whether two option sets describe the same broker session. Note the explicit /// Whether two option sets describe the same broker session. Note the explicit
/// <see cref="object.Equals(object?)"/>: the identity projections are <c>object</c>-typed /// <see cref="object.Equals(object?)"/>: the identity projections are <c>object</c>-typed
@@ -1,7 +1,11 @@
using System.Text; using System.Text;
using Google.Protobuf;
using Org.Eclipse.Tahu.Protobuf;
using Shouldly; using Shouldly;
using Xunit; using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions; using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug;
using TahuDataType = Org.Eclipse.Tahu.Protobuf.DataType;
namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests; namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Tests;
@@ -364,6 +368,302 @@ public sealed class MqttDriverDiscoveryTests
statuses[0].State.ShouldBe(HostState.Unknown); statuses[0].State.ShouldBe(HostState.Unknown);
} }
// =================================================================================
// Task 22 — Sparkplug discovery policy, birth-filled datatypes, rediscovery trigger
// =================================================================================
/// <summary>
/// Sparkplug's discovered shape is not fully known at connect: a tag authored without a
/// <c>dataType</c> takes its type from the birth certificate, which arrives asynchronously.
/// That is precisely the <see cref="DiscoveryRediscoverPolicy.UntilStable"/> contract — the
/// host re-runs discovery until the captured set settles.
/// </summary>
[Fact]
public void SparkplugMode_RediscoverPolicy_IsUntilStable()
=> SparkplugDriver().RediscoverPolicy.ShouldBe(DiscoveryRediscoverPolicy.UntilStable);
/// <summary>
/// The mode gate, asserted from the other side: plain MQTT's authored set is fully known
/// synchronously, so it must stay <see cref="DiscoveryRediscoverPolicy.Once"/>. A blanket flip
/// to <c>UntilStable</c> would make every plain deployment re-run discovery on a timer for a
/// tree that can never change.
/// </summary>
[Fact]
public void PlainMode_RediscoverPolicy_StaysOnce()
=> PlainDriver(Tag("Plant/Mqtt/dev1/Temp", "f/t")).RediscoverPolicy.ShouldBe(DiscoveryRediscoverPolicy.Once);
/// <summary>
/// A DBIRTH for an authored device the driver has never held a birth for introduces the
/// metrics' datatypes, so the discovered tree genuinely changed: rediscovery fires.
/// </summary>
[Fact]
public void NewDbirth_FiresOnRediscoveryNeeded()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
var fired = 0;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => Interlocked.Increment(ref fired);
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
fired.ShouldBeGreaterThan(0);
}
/// <summary>
/// <b>The primary anti-storm pin.</b> An edge node that re-births on a timer (or a flapping
/// connection driving the late-join rebirth) republishes the same metric set over and over. If
/// every birth fired rediscovery, each one would trigger an address-space rebuild — expensive
/// and disruptive on a running server, forever, for a tree that never changed.
/// </summary>
[Fact]
public void IdenticalRebirth_DoesNotFireRediscovery()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
// Subscribe only AFTER the first birth so the count below is purely the rebirths'.
var fired = 0;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => Interlocked.Increment(ref fired);
for (var i = 0; i < 5; i++)
{
FeedNodeBirth(driver, seq: 0, bdSeq: (ulong)(i + 2));
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
}
fired.ShouldBe(0);
}
/// <summary>
/// The other half of the gate: a rebirth that actually changes the metric set (a metric added,
/// removed or renamed) DOES change what discovery can resolve, and must fire.
/// </summary>
[Fact]
public void RebirthWithChangedMetricSet_FiresRediscovery()
{
var driver = SparkplugDriver(
SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")),
SpTag(SpPressPath, SpBlob(SpDevice, "Pressure")));
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
var fired = 0;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => Interlocked.Increment(ref fired);
FeedDeviceBirth(driver, seq: 2, ("Temperature", 5UL, TahuDataType.Float), ("Pressure", 6UL, TahuDataType.Float));
fired.ShouldBe(1);
}
/// <summary>
/// Metric ORDER is not the metric SET. A birth that lists the same metrics in a different order
/// describes the same address space; firing on it would reintroduce the rebuild storm through
/// the back door, because nothing obliges an edge node to keep its birth order stable.
/// </summary>
[Fact]
public void RebirthReorderingTheSameMetrics_DoesNotFireRediscovery()
{
var driver = SparkplugDriver(
SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")),
SpTag(SpPressPath, SpBlob(SpDevice, "Pressure")));
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float), ("Pressure", 6UL, TahuDataType.Float));
var fired = 0;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => Interlocked.Increment(ref fired);
FeedDeviceBirth(driver, seq: 2, ("Pressure", 6UL, TahuDataType.Float), ("Temperature", 5UL, TahuDataType.Float));
fired.ShouldBe(0);
}
/// <summary>
/// A birth for a device NO authored tag names cannot change <see cref="MqttDriver.DiscoverAsync"/>'s
/// output — discovery replays the authored set. A large plant DBIRTHs devices this deployment
/// never reads on their own schedules; firing for those is a rebuild storm sourced from traffic
/// the configuration has no opinion about.
/// </summary>
[Fact]
public void BirthForAnUnauthoredDevice_DoesNotFireRediscovery()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
FeedNodeBirth(driver);
var fired = 0;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, _) => Interlocked.Increment(ref fired);
// Same (authored) edge node, a device nothing binds — the ingestor still applies the birth.
FeedDeviceBirth(driver, "Filler99", seq: 1, ("Temperature", 5UL, TahuDataType.Float));
fired.ShouldBe(0);
}
/// <summary>
/// The rediscovery event names the subtree this driver's discovery actually emits (the
/// <c>Mqtt</c> folder) so a consumer scoping a rebuild on it scopes to exactly the tree that
/// changed, and carries the Sparkplug scope in the diagnostic reason.
/// </summary>
[Fact]
public void RediscoveryEvent_ScopeHintIsTheDiscoveryFolder_ReasonNamesTheScope()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
RediscoveryEventArgs? args = null;
((IRediscoverable)driver).OnRediscoveryNeeded += (_, e) => args = e;
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
args.ShouldNotBeNull();
args.ScopeHint.ShouldBe("Mqtt");
args.Reason.ShouldContain($"{SpGroup}/{SpNode}/{SpDevice}");
}
/// <summary>
/// <b>The datatype fill-in.</b> <c>dataType</c> is optional in the Sparkplug tag shape — the
/// birth certificate declares it. Before any birth the tag can only report the record default;
/// once the DBIRTH lands, discovery must re-stream it with the type the birth declared.
/// </summary>
[Fact]
public async Task DiscoverAsync_Sparkplug_UnauthoredDataType_FillsInFromTheBirth()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
var before = new CapturingAddressSpaceBuilder();
await driver.DiscoverAsync(before, TestContext.Current.CancellationToken);
// The node exists from the first pass — an authored tag is part of the declared configuration,
// not something the plant grants by publishing.
before.Variables.Select(v => v.Info.FullName).ShouldBe([SpTempPath]);
before.Variables[0].Info.DriverDataType.ShouldBe(DriverDataType.String); // the record default
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
var after = new CapturingAddressSpaceBuilder();
await driver.DiscoverAsync(after, TestContext.Current.CancellationToken);
after.Variables.Select(v => v.Info.FullName).ShouldBe([SpTempPath]);
after.Variables[0].Info.DriverDataType.ShouldBe(DriverDataType.Float32);
}
/// <summary>
/// The authored <c>dataType</c> wins over the birth's — the same precedence the ingest path
/// applies when it coerces a value (<c>DataTypeAuthored ? authored : birth</c>). A discovery
/// surface that disagreed with the publish surface would report a type no value ever arrives as.
/// </summary>
[Fact]
public async Task DiscoverAsync_Sparkplug_AuthoredDataType_WinsOverTheBirth()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature", "Int32")));
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.Float));
var b = new CapturingAddressSpaceBuilder();
await driver.DiscoverAsync(b, TestContext.Current.CancellationToken);
b.Variables[0].Info.DriverDataType.ShouldBe(DriverDataType.Int32);
}
/// <summary>
/// A birth declaring a Sparkplug type this driver cannot map (DataSet / Template / PropertySet
/// / Unknown) must not blank the tag's type — <c>ToDriverDataType()</c> returns null for those,
/// and discovery falls back to the authored/default type rather than to nothing.
/// </summary>
[Fact]
public async Task DiscoverAsync_Sparkplug_UnsupportedBirthType_FallsBackToTheDefault()
{
var driver = SparkplugDriver(SpTag(SpTempPath, SpBlob(SpDevice, "Temperature")));
FeedNodeBirth(driver);
FeedDeviceBirth(driver, seq: 1, ("Temperature", 5UL, TahuDataType.DataSet));
var b = new CapturingAddressSpaceBuilder();
await driver.DiscoverAsync(b, TestContext.Current.CancellationToken);
b.Variables.Count.ShouldBe(1);
b.Variables[0].Info.DriverDataType.ShouldBe(DriverDataType.String);
}
// ---- Sparkplug fixtures ----
private const string SpGroup = "Plant1";
private const string SpNode = "EdgeA";
private const string SpDevice = "Filler1";
private const string SpTempPath = "Plant/Mqtt/spb/Filler1Temp";
private const string SpPressPath = "Plant/Mqtt/spb/Filler1Press";
/// <summary>A Sparkplug tag blob. <c>dataType</c> is omitted unless supplied — it is optional.</summary>
private static string SpBlob(string? device, string metric, string? dataType = null)
{
var deviceKey = device is null ? "" : $"""{'"'}deviceId{'"'}:"{device}",""";
var typeKey = dataType is null ? "" : $""","dataType":"{dataType}" """;
return $$"""{"groupId":"{{SpGroup}}","edgeNodeId":"{{SpNode}}",{{deviceKey}}"metricName":"{{metric}}"{{typeKey}}}""";
}
private static RawTagEntry SpTag(string rawPath, string blob) => new(rawPath, blob, WriteIdempotent: false);
private static MqttDriver SparkplugDriver(params RawTagEntry[] tags)
=> new(
new MqttDriverOptions
{
Mode = MqttMode.SparkplugB,
Sparkplug = new MqttSparkplugOptions { GroupId = SpGroup },
RawTags = tags,
},
"d",
null);
/// <summary>
/// Feeds a real NBIRTH through the real codec + ingest state machine — no broker, and no
/// test-only shortcut on the driver. <c>Dispatch</c> is the method MQTTnet's dispatcher thread
/// reaches, so this exercises the production path end to end.
/// </summary>
private static void FeedNodeBirth(MqttDriver driver, ulong seq = 0, ulong bdSeq = 1)
{
var payload = new Payload { Seq = seq, Timestamp = 1721822400000UL };
payload.Metrics.Add(new Payload.Types.Metric
{
Name = "bdSeq",
Datatype = (uint)TahuDataType.Uint64,
LongValue = bdSeq,
});
payload.Metrics.Add(SpBirthMetric("Uptime", 7UL, TahuDataType.Int64));
driver.Sparkplug!.Dispatch(
new SparkplugTopic(SparkplugMessageType.NBIRTH, SpGroup, SpNode, null, null),
SparkplugCodec.Decode(payload.ToByteArray()));
}
private static void FeedDeviceBirth(
MqttDriver driver,
ulong seq,
params (string Name, ulong Alias, TahuDataType Type)[] metrics)
=> FeedDeviceBirth(driver, SpDevice, seq, metrics);
private static void FeedDeviceBirth(
MqttDriver driver,
string device,
ulong seq,
params (string Name, ulong Alias, TahuDataType Type)[] metrics)
{
var payload = new Payload { Seq = seq, Timestamp = 1721822400000UL };
foreach (var (name, alias, type) in metrics)
{
payload.Metrics.Add(SpBirthMetric(name, alias, type));
}
driver.Sparkplug!.Dispatch(
new SparkplugTopic(SparkplugMessageType.DBIRTH, SpGroup, SpNode, device, null),
SparkplugCodec.Decode(payload.ToByteArray()));
}
/// <summary>
/// A birth metric declaring name/alias/datatype and <b>no value</b> — a birth's job here is to
/// declare the catalog, and these tests assert on the catalog, never on a published value.
/// </summary>
private static Payload.Types.Metric SpBirthMetric(string name, ulong alias, TahuDataType type)
=> new() { Name = name, Alias = alias, Datatype = (uint)type };
// ---- test doubles ---- // ---- test doubles ----
/// <summary> /// <summary>