diff --git a/CLAUDE.md b/CLAUDE.md index d274a661..8946cba0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -168,13 +168,29 @@ central SQL Server. > recipes (S7 blackhole, GLAuth outage, HistorianGateway LiveIntegration), the integration-test > harness, the docker-dev rig, and the deploy setup. Start there; the sections below + `docs/v2/dev-environment.md` carry the detail. -> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. The driver fixture compose files under `tests/.../Docker/docker-compose.yml` now carry a `project: lmxopcua` label on every service. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28). +> **Migrated 2026-04-28**: Docker config + host moved off this dev VM (DESKTOP-6JL3KKO) onto the shared Linux Docker host (`DOCKER`, 10.100.0.35) so the dev VM could shed WSL2/Hyper-V and have its GPU re-attached via ESXi passthrough. Docker Desktop is no longer installed here. All checked-in `appsettings.json` defaults, fixture-class default endpoints, and `e2e-config.sample.json` were rewritten to target `10.100.0.35`. See `docs/v2/dev-environment.md` for the full rewrite (header dated 2026-04-28). + +> ⚠️ **The `project: lmxopcua` label is aspirational, not universal.** This note used to claim every fixture compose file carries it. As of 2026-07-24 only the **MQTT** fixture actually does — `grep -rn "labels:" tests --include=*.yml` matches `Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml` and nothing else. So `docker ps --filter label=project=lmxopcua` returns the MQTT fixture alone, not the fleet. Add the label when you touch an older fixture; until then enumerate by container name. Docker workloads run on a shared Linux host at **`10.100.0.35`** — not on this VM. Stacks live at `/opt/otopcua-/` on the host and carry the `project=lmxopcua` label so they're discoverable via `docker ps --filter label=project=lmxopcua`. -**`docker -H ssh://...` does NOT work from this VM.** Windows OpenSSH ↔ docker.exe stdio bridging hangs (`docker system dial-stdio` runs server-side but no API data flows). Use the helper below — it SSHes into the docker host and runs `docker compose` server-side. +**`docker -H ssh://...` does NOT work from the Windows VM.** Windows OpenSSH ↔ docker.exe stdio bridging hangs (`docker system dial-stdio` runs server-side but no API data flows). Use the helper below — it SSHes into the docker host and runs `docker compose` server-side. -**Use `lmxopcua-fix.ps1` (in `~/bin`) to control fixtures from this VM:** +> **On macOS there is no `lmxopcua-fix` — drive the host over SSH directly.** The helper is a Windows-VM +> script; `~/bin` is empty on the Mac and the commands below will not resolve. Use passwordless SSH +> (and `rsync` in place of `sync`), which is what `infra/README.md` §1 documents as the Mac path: +> +> ```bash +> ssh dohertj2@10.100.0.35 'docker ps --format "{{.Names}}\t{{.Status}}"' +> ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mqtt && docker compose up -d' +> rsync -av tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/ \ +> dohertj2@10.100.0.35:/opt/otopcua-mqtt/ # the `sync` step, by hand +> ``` +> +> A local container runtime *does* exist on the Mac (OrbStack), so the `docker-dev/` rig itself runs +> locally there — it is only the shared **fixtures** that live on `10.100.0.35`. + +**Use `lmxopcua-fix.ps1` (in `~/bin`) to control fixtures from the Windows VM:** ```powershell lmxopcua-fix ls # list all lmxopcua-tagged containers on the host @@ -195,6 +211,16 @@ lmxopcua-fix sync modbus # rsync this repo's tests/.../Docker/ - AB CIP: `10.100.0.35:44818` (`AB_SERVER_ENDPOINT`) - S7: `10.100.0.35:1102` (`S7_SIM_ENDPOINT`) - OPC UA reference (opc-plc): `opc.tcp://10.100.0.35:50000` (`OPCUA_SIM_ENDPOINT`) +- MQTT (Mosquitto, **auth on both listeners — no anonymous fallback**): `10.100.0.35:8883` TLS+auth + (`MQTT_FIXTURE_ENDPOINT`) · `10.100.0.35:1883` plaintext-but-authenticated (`MQTT_FIXTURE_PLAIN_ENDPOINT`). + Also needs `MQTT_FIXTURE_USERNAME` / `MQTT_FIXTURE_PASSWORD`, and `MQTT_FIXTURE_CA_CERT` to pin the + fixture CA (`scp dohertj2@10.100.0.35:/opt/otopcua-mqtt/secrets/ca.crt`). A co-running publisher loops + JSON under `otopcua/fixture/…`. +- MQTT **Sparkplug B**: the same broker, plus the `sparkplug` compose profile's `otopcua-sparkplug-sim` + (project-owned C# edge-node simulator) — group **`OtOpcUaSim`**, edge nodes **`EdgeA`**/**`EdgeB`**, + `EdgeA` device **`Filler1`**, ~2 s cadence. It subscribes to `spBv1.0/OtOpcUaSim/NCMD/{node}` and + re-births on request. **Births are never retained**, so `docker restart otopcua-sparkplug-sim` is how + you force a fresh one while a browse window is open. Override any endpoint via the env var to point at a real PLC. The local OtOpcUa server runs on this VM at `opc.tcp://localhost:4840` — **that's not on the docker host**. diff --git a/Directory.Packages.props b/Directory.Packages.props index 87785628..baf53c23 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -75,6 +75,15 @@ + + + + + + + + + + + + + + + diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttDriverOptions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttDriverOptions.cs new file mode 100644 index 00000000..8acb9148 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttDriverOptions.cs @@ -0,0 +1,225 @@ +using System.ComponentModel.DataAnnotations; +using System.Text; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// MQTT / Sparkplug B driver configuration. Bound from DriverConfig JSON at +/// driver-host registration time. Models the settings documented in +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.1. +/// +/// +/// A record (not a plain class) so a future secret-resolution seam can produce a +/// credential-resolved copy with a with expression — mirrors +/// OpcUaClientDriverOptions. selects the ingest shape; +/// and are nullable sub-objects — only the one +/// matching is populated. +/// +public sealed record MqttDriverOptions +{ + /// Broker hostname or IP address. + public string Host { get; init; } = "localhost"; + + /// Broker TCP port. + [Range(1, 65535)] + public int Port { get; init; } = 8883; + + /// + /// MQTT client identifier sent at CONNECT. Leave unset to let the driver generate one + /// (a stable per-instance id is recommended so broker-side ACLs / session state persist + /// across reconnects). + /// + public string? ClientId { get; init; } + + /// + /// When true, connect over TLS. Default true — this driver never ships an + /// anonymous/plaintext-by-default posture; a deployment must opt into false for an + /// on-prem/dev broker with no TLS listener. + /// + public bool UseTls { get; init; } = true; + + /// + /// When true, accept any self-signed / untrusted broker certificate. Dev/on-prem + /// escape hatch only — mirrors the ServerHistorian TLS knobs. Must stay + /// false in production so MITM attacks against the broker connection fail closed. + /// + public bool AllowUntrustedServerCertificate { get; init; } = false; + + /// + /// PEM CA file pinning the broker's TLS chain. null/empty uses the OS trust + /// store. + /// + public string? CaCertificatePath { get; init; } + + /// Username for broker authentication. null connects without a username. + public string? Username { get; init; } + + /// + /// Password for broker authentication. Blank in committed JSON — supply via env, never + /// commit or log. + /// + public string? Password { get; init; } = ""; + + /// MQTT protocol version to negotiate at CONNECT. + public MqttProtocolVersion ProtocolVersion { get; init; } = MqttProtocolVersion.V500; + + /// Whether to request a clean session (v3.1.1) / clean start (v5.0) at CONNECT. + public bool CleanSession { get; init; } = true; + + /// Keep-alive interval sent at CONNECT. + [Range(1, int.MaxValue)] + public int KeepAliveSeconds { get; init; } = 30; + + /// Bounded connect deadline (see design §8) — the driver never hangs past this. + [Range(1, int.MaxValue)] + public int ConnectTimeoutSeconds { get; init; } = 15; + + /// Initial reconnect backoff after a connection drop. + [Range(1, int.MaxValue)] + public int ReconnectMinBackoffSeconds { get; init; } = 1; + + /// Cap on the exponential reconnect backoff. + [Range(1, int.MaxValue)] + public int ReconnectMaxBackoffSeconds { get; init; } = 30; + + /// + /// Ceiling on an inbound message body, in bytes. A larger message is refused before any + /// decode or parse and degrades its own tags to BadDecodingError. Default 1 MiB. + /// + /// + /// Decode and parse both run synchronously on MQTTnet's shared dispatcher thread, so their + /// cost is paid by every subscription, not just the offending topic — without a bound, + /// one publisher shipping a multi-megabyte body stalls the whole driver's delivery. "Unbounded" + /// is deliberately not offered; a non-positive value falls back to the driver's own 1 MiB + /// default. The literal is duplicated from MqttSubscriptionManager.DefaultMaxPayloadBytes + /// because this .Contracts assembly is referenced by the driver assembly and + /// cannot reference it back; the manager's constructor is the single enforcement point. + /// + [Range(1, int.MaxValue)] + public int MaxPayloadBytes { get; init; } = 1024 * 1024; + + /// Selects the ingest shape — Plain topics or Sparkplug B. + public MqttMode Mode { get; init; } = MqttMode.Plain; + + /// + /// The cluster's authored raw MQTT tags, as delivered by the deploy artifact: each carries the + /// tag's RawPath (its v3 identity and driver wire reference) plus its TagConfig + /// blob. maps each through at + /// initialize and re-registers the set wholesale on every reinitialize, so a redeploy + /// that drops a tag stops feeding it. + /// + /// + /// This is also the only source of the driver's discoverable node set — plain MQTT has + /// no browsable address space, and a tag that is not authored here is never materialized no + /// matter how much traffic its topic carries. Mirrors ModbusDriverOptions.RawTags / + /// FocasDriverOptions.RawTags. + /// + public IReadOnlyList RawTags { get; init; } = []; + + /// + /// Sparkplug-only settings. Populated when is + /// ; null in Plain mode. + /// + public MqttSparkplugOptions? Sparkplug { get; init; } + + /// + /// Plain-mode-only settings. Populated when is + /// ; null in Sparkplug B mode. + /// + public MqttPlainOptions? Plain { get; init; } + + /// + /// Record-generated ToString() member printer, overridden so + /// never renders in plaintext — this DTO routinely lands in logs / exception messages / + /// debugger watches, and the plan's cross-cutting rule is explicit: never commit or log + /// creds. A set password prints as ***; unset (null/empty) renders + /// distinguishably so the redaction can't be mistaken for a real secret. + /// + private bool PrintMembers(StringBuilder builder) + { + var passwordDisplay = Password switch + { + null => "", + "" => "", + _ => "***", + }; + + builder.Append("Host = ").Append(Host); + builder.Append(", Port = ").Append(Port); + builder.Append(", ClientId = ").Append(ClientId); + builder.Append(", UseTls = ").Append(UseTls); + builder.Append(", AllowUntrustedServerCertificate = ").Append(AllowUntrustedServerCertificate); + builder.Append(", CaCertificatePath = ").Append(CaCertificatePath); + builder.Append(", Username = ").Append(Username); + builder.Append(", Password = ").Append(passwordDisplay); + builder.Append(", ProtocolVersion = ").Append(ProtocolVersion); + builder.Append(", CleanSession = ").Append(CleanSession); + builder.Append(", KeepAliveSeconds = ").Append(KeepAliveSeconds); + builder.Append(", ConnectTimeoutSeconds = ").Append(ConnectTimeoutSeconds); + builder.Append(", ReconnectMinBackoffSeconds = ").Append(ReconnectMinBackoffSeconds); + builder.Append(", ReconnectMaxBackoffSeconds = ").Append(ReconnectMaxBackoffSeconds); + builder.Append(", MaxPayloadBytes = ").Append(MaxPayloadBytes); + builder.Append(", Mode = ").Append(Mode); + builder.Append(", Sparkplug = ").Append(Sparkplug); + builder.Append(", Plain = ").Append(Plain); + // Count only: a deployment routinely authors thousands of raw tags and each carries a full + // TagConfig blob, so printing the list would turn any log of this DTO into a config dump. + builder.Append(", RawTags = ").Append(RawTags.Count).Append(" tag(s)"); + return true; + } +} + +/// +/// Sparkplug B settings for an instance in +/// mode. See +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.1. +/// +public sealed record MqttSparkplugOptions +{ + /// Sparkplug group id — the driver subscribes spBv1.0/{GroupId}/#. + public string GroupId { get; init; } = ""; + + /// + /// Sparkplug Host Application identity — published as spBv1.0/STATE/{HostId} + /// (Sparkplug v3.0). + /// + public string HostId { get; init; } = ""; + + /// + /// When true, this driver instance acts as the Sparkplug Primary Host + /// Application. At most one primary host may exist per host-id per broker. + /// + public bool ActAsPrimaryHost { get; init; } = false; + + /// + /// When true, a detected sequence-number gap triggers a Sparkplug rebirth + /// request (NCMD) rather than silently continuing with stale metric state. + /// + public bool RequestRebirthOnGap { get; init; } = true; + + /// + /// How long, in seconds, browse/discovery collects NBIRTH/DBIRTH traffic before + /// considering the observed metric set stable. + /// + [Range(1, int.MaxValue)] + public int BirthObservationWindowSeconds { get; init; } = 15; +} + +/// +/// Plain-MQTT-mode settings for an instance in +/// mode. See +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.1. +/// +public sealed record MqttPlainOptions +{ + /// + /// Optional prefix prepended when the driver needs to compose a topic (e.g. discovery + /// scoping); per-tag topics are authored explicitly and are unaffected. + /// + public string TopicPrefix { get; init; } = ""; + + /// Default QoS used when a tag's own qos is unset. + [Range(0, 2)] + public int DefaultQos { get; init; } = 1; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttJson.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttJson.cs new file mode 100644 index 00000000..944f653d --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttJson.cs @@ -0,0 +1,53 @@ +using System.Text.Json; +using System.Text.Json.Serialization; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// The single instance every MQTT driver-config seam +/// parses through — the runtime factory +/// (MqttDriverFactoryExtensions), the Test-connect probe (MqttDriverProbe), the +/// driver's own ReinitializeAsync re-parse, and the address-picker browser +/// (MqttDriverBrowser). +/// +/// +/// +/// Why one instance, and why here. Divergent per-seam options are this repo's +/// documented systemic enum bug: an AdminUI-authored config with an enum-valued field is +/// accepted by the seam that carries a and faults the +/// one that does not, so "Test connect" goes green and the deployed driver dies. Two other +/// drivers already carry a copy per seam. Rather than repeat that, this driver keeps exactly +/// one instance. +/// +/// +/// It lives in .Contracts — the assembly that owns and +/// the three enums the converter exists for — rather than in the factory or the probe, +/// because .Contracts is the only assembly all four consumers already reference. +/// In particular the browser lives in its own assembly and reaches the runtime .Driver +/// project only through a deliberate, documented layering exception that is scheduled to be +/// removed (see the ProjectReference comment in the browser's csproj); anchoring the +/// options in .Driver would resurrect the duplicate the day that reference goes away. +/// +/// +/// Not a general-purpose JSON policy. UnmappedMemberHandling.Skip means an +/// unknown key is ignored rather than rejected — deliberate, so a config blob authored +/// against a newer driver still binds — and PropertyNameCaseInsensitive accepts both +/// the camelCase the AdminUI emits and the PascalCase a hand-edited blob may carry. A +/// becomes read-only on first use, so this instance is +/// safe to share across threads and must never be mutated after startup. +/// +/// +public static class MqttJson +{ + /// + /// The shared options. Enum-valued knobs (, + /// , ) round-trip by + /// name; numeric ordinals still bind, so an older blob is not broken by this. + /// + public static readonly JsonSerializerOptions Options = new() + { + PropertyNameCaseInsensitive = true, + UnmappedMemberHandling = JsonUnmappedMemberHandling.Skip, + Converters = { new JsonStringEnumConverter() }, + }; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttMode.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttMode.cs new file mode 100644 index 00000000..86f907c7 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttMode.cs @@ -0,0 +1,14 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// Selects the ingest shape an instance uses. See +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.1. +/// +public enum MqttMode +{ + /// Plain MQTT — the driver subscribes to authored topics directly. + Plain, + + /// Sparkplug B — the driver decodes Tahu-encoded NBIRTH/DBIRTH/NDATA/DDATA payloads. + SparkplugB, +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttPayloadFormat.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttPayloadFormat.cs new file mode 100644 index 00000000..c8ff9fa0 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttPayloadFormat.cs @@ -0,0 +1,17 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// How a Plain-mode MQTT tag's payload is decoded. See +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.3. +/// +public enum MqttPayloadFormat +{ + /// The payload is a JSON document; a JSONPath selects the value (jsonPath). + Json, + + /// The payload bytes are used as-is (e.g. binary / opaque). + Raw, + + /// The payload is a bare scalar string (e.g. "23.5"), parsed by dataType. + Scalar, +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttProtocolVersion.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttProtocolVersion.cs new file mode 100644 index 00000000..8a674e34 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttProtocolVersion.cs @@ -0,0 +1,14 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// MQTT wire protocol version negotiated at CONNECT. See +/// docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.1. +/// +public enum MqttProtocolVersion +{ + /// MQTT 3.1.1. + V311, + + /// MQTT 5.0. + V500, +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagConfigKeys.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagConfigKeys.cs new file mode 100644 index 00000000..4a11784b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagConfigKeys.cs @@ -0,0 +1,40 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// The TagConfig JSON key names that make up an MQTT tag's address, in the two shapes +/// the driver binds by: a single (Plain) or the +/// /// +/// tuple (Sparkplug B). +/// +/// +/// +/// These exist because the address is produced in one place and consumed in another, and the +/// two used to agree only by coincidence: the AdminUI browse-commit mapper writes the blob, +/// reads it, and a key-name drift between them is not a +/// compile error — it is a tag that deploys clean, resolves to nothing, and reports +/// BadNodeIdUnknown at runtime with no authoring-time signal at all. Single-sourcing +/// the names makes that drift impossible rather than merely unlikely. +/// +/// +/// Address keys only. The behavioural keys (payloadFormat, jsonPath, +/// dataType, qos, retainSeed) are deliberately not here — nothing produces +/// them from a browse, so they carry no cross-component drift risk. +/// +/// +public static class MqttTagConfigKeys +{ + /// The concrete MQTT topic a Plain-mode tag subscribes to. + public const string Topic = "topic"; + + /// The Sparkplug group id — the first segment of the tag's binding tuple. + public const string GroupId = "groupId"; + + /// The Sparkplug edge-node id. + public const string EdgeNodeId = "edgeNodeId"; + + /// The Sparkplug device id; absent for a metric published by the edge node itself. + public const string DeviceId = "deviceId"; + + /// The Sparkplug metric's stable name — the tag's binding key across rebirths. + public const string MetricName = "metricName"; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinition.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinition.cs new file mode 100644 index 00000000..15c3c3eb --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinition.cs @@ -0,0 +1,85 @@ +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// The driver's internal per-tag descriptor — the parsed form of an authored raw tag's +/// TagConfig JSON, produced by and +/// looked up through the shared exactly as Modbus does. +/// See docs/plans/2026-07-15-mqtt-sparkplug-driver-design.md §5.2 / §5.3. +/// +/// The record carries both ingest shapes: the Plain-MQTT fields +/// ( … ), populated by +/// , and the Sparkplug B descriptor fields +/// (, , , +/// ), populated by +/// (Task 21). A Sparkplug tag is +/// resolved by the stable (group, node, device, metricName) tuple, never by the +/// per-birth metric alias — an alias may be reused across a rebirth for a different metric. +/// +/// +/// Which factory runs is chosen by the driver's , never sniffed from +/// the blob. A blob carrying both shapes' keys is legal (the AdminUI editor preserves +/// unknown keys through a load→save, so a tag retyped from Plain to Sparkplug keeps its old +/// topic), and a heuristic would make the same authored tag mean two different things +/// depending on which key happened to survive. +/// +/// +/// +/// The definition's identity: the tag's RawPath — the cluster-scoped slash path that is the +/// v3 driver wire reference. This is the key resolves +/// on, the key OnDataChange must publish under, and the key DriverHostActor fans out +/// to the raw + UNS NodeIds. It is emphatically not the authored TagConfig blob: the +/// pre-v3 blob-as-reference shape is retired, and publishing under a blob key would silently miss +/// the RawPath-keyed fan-out while every unit test still passed. +/// +/// The concrete MQTT topic this tag subscribes to (Plain mode). +/// How the received payload is decoded (Plain mode). +/// +/// JSONPath selecting the value inside a payload. Defaults +/// to the document root ($) when the blob omits it; meaningless for +/// / . +/// +/// +/// The tag's declared value type. Explicit authoring is strongly preferred — payload-shape +/// inference is the brittle fallback (design §4). +/// +/// +/// The per-tag subscription QoS (0–2), or to inherit the driver-level +/// . +/// +/// +/// Whether the broker's retained message for seeds this tag's initial value +/// on subscribe (the OPC UA initial-data convention). Defaults to . +/// +/// +/// Whether came from an authored dataType key or is merely this +/// record's default. Load-bearing in Sparkplug mode only, where dataType is optional: +/// a Sparkplug metric's type is declared by its own birth certificate, so an unauthored tag must +/// take the type the NBIRTH/DBIRTH declared rather than silently coercing every value to the +/// default. Both factories set it from the key's presence so it +/// means the same thing in either mode; nothing on the Plain ingest path reads it. +/// +/// The Sparkplug group id; for a Plain-mode tag. +/// The Sparkplug edge-node id; for a Plain-mode tag. +/// +/// The Sparkplug device id, or for a metric published by the edge node +/// itself (and for every Plain-mode tag). +/// +/// +/// The Sparkplug metric name — the tag's stable identity across rebirths, and the key the ingest +/// state machine binds by. for a Plain-mode tag. +/// +public sealed record MqttTagDefinition( + string Name, + string Topic, + MqttPayloadFormat PayloadFormat, + string JsonPath, + DriverDataType DataType, + int? Qos, + bool RetainSeed, + bool DataTypeAuthored = true, + string? GroupId = null, + string? EdgeNodeId = null, + string? DeviceId = null, + string? MetricName = null); diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinitionFactory.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinitionFactory.cs new file mode 100644 index 00000000..ecdcf121 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/MqttTagDefinitionFactory.cs @@ -0,0 +1,290 @@ +using System.Text.Json; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// v3 pure mapper: turns an authored raw tag's TagConfig JSON (the shape produced by the +/// AdminUI MqttTagConfigModel) into an . Under v3 a tag's +/// identity is its RawPath (a cluster-scoped slash path), not the address blob — so the +/// produced definition's is the RawPath the driver was handed, +/// which is exactly the wire reference the driver's RawPath → def resolver keys on. The driver +/// builds that table by mapping each the deploy artifact delivers through +/// . +/// +/// Two entry points carry deliberately different strictness contracts. +/// is the runtime path: it never throws, and anything +/// malformed returns so the driver surfaces BadNodeIdUnknown +/// rather than a misleading Good off a wrong-typed default. is the +/// deploy-time path: it returns human-readable warnings so a bad tag config surfaces at +/// deploy instead of silently going dark at runtime. +/// +/// +/// Two runtime entry points, one per ingest shape — (Plain) +/// and (Sparkplug B). The caller picks by the driver's +/// ; neither sniffs the blob. That is deliberate: the AdminUI editor +/// preserves unknown keys through a load→save, so a tag retyped from Plain to Sparkplug keeps +/// its old topic and a presence heuristic would make one authored blob mean two +/// different things depending on which key happened to survive. The driver's mode is +/// the single authority. +/// +/// +/// No ToTagConfig inverse — a deliberate YAGNI call. The six sibling factories +/// each carry one solely to serve their Driver.<X>.Cli project, which synthesises +/// blobs from operator flags; the MQTT plan defines no +/// Mqtt.Cli. The AdminUI typed editor does not need one either — the +/// <Driver>TagConfigModel template serialises through its own preserved +/// JsonObject key bag and references no driver factory (verified: no +/// TagDefinitionFactory reference exists anywhere under the AdminUI project). Add the +/// inverse when a real caller appears, not before. +/// +/// +public static class MqttTagDefinitionFactory +{ + /// The JSONPath applied when a Json-format blob omits jsonPath: the document root. + private const string RootJsonPath = "$"; + + /// The MQTT wildcard characters — illegal in a tag's (concrete) subscription topic. + private static readonly char[] TopicWildcards = ['+', '#']; + + /// + /// Maps an authored TagConfig object to a typed definition keyed by . + /// The input is always an authored TagConfig JSON object (there is no name-vs-blob heuristic in v3). + /// Enum fields (payloadFormat / dataType) and qos are read STRICTLY — a + /// present-but-invalid (typo'd) value rejects the whole tag (returns ⇒ the + /// driver surfaces BadNodeIdUnknown) rather than silently defaulting to a wrong-typed Good or + /// a downgraded delivery guarantee. Never throws. + /// + /// A wildcard topic (+ / #) is deliberately NOT rejected here: it is + /// ambiguous rather than unparseable, and is the operator-visible surface + /// for it. Rejecting it at runtime would turn an authoring mistake into a silent + /// BadNodeIdUnknown with no stated cause. + /// + /// + /// The authored equipment-tag TagConfig JSON. + /// The tag's RawPath — becomes the definition's identity (Name). + /// The mapped definition when this returns . + /// when is a valid MQTT tag-config object. + public static bool FromTagConfig(string tagConfig, string rawPath, out MqttTagDefinition def) + { + def = null!; + if (string.IsNullOrWhiteSpace(tagConfig) || tagConfig[0] != '{') return false; + try + { + using var doc = JsonDocument.Parse(tagConfig); + var root = doc.RootElement; + if (root.ValueKind != JsonValueKind.Object) return false; + + // topic is the whole point of a Plain-mode tag — without one there is nothing to subscribe + // to, so an absent/blank value is a hard reject rather than a defaulted empty subscription. + var topic = ReadString(root, MqttTagConfigKeys.Topic); + if (string.IsNullOrWhiteSpace(topic)) return false; + + // Strict enum reads: a present-but-invalid (typo'd) value rejects the whole tag + // (→ BadNodeIdUnknown) instead of silently defaulting to a wrong-typed Good. + if (!TagConfigJson.TryReadEnumStrict(root, "payloadFormat", MqttPayloadFormat.Json, out var payloadFormat)) + return false; + var dataTypeAuthored = root.TryGetProperty("dataType", out _); + if (!TagConfigJson.TryReadEnumStrict(root, "dataType", DriverDataType.String, out var dataType)) + return false; + + // qos is read with the same strictness, for the same reason: silently absorbing a malformed + // "qos":"high" / "qos":1.5 / "qos":5 into the driver-level default would hand the operator a + // WEAKER delivery guarantee than the one they authored, with nothing to show for it. + if (!TryReadQosStrict(root, out var qos)) return false; + + var jsonPath = ReadString(root, "jsonPath"); + if (string.IsNullOrEmpty(jsonPath)) jsonPath = RootJsonPath; + + def = new MqttTagDefinition( + Name: rawPath, + Topic: topic, + PayloadFormat: payloadFormat, + JsonPath: jsonPath, + DataType: dataType, + Qos: qos, + RetainSeed: ReadBoolOrDefault(root, "retainSeed", defaultValue: true), + DataTypeAuthored: dataTypeAuthored); + return true; + } + catch (JsonException) { return false; } + catch (FormatException) { return false; } + catch (InvalidOperationException) { return false; } + } + + /// + /// Maps an authored TagConfig object to a Sparkplug B definition keyed by + /// . The tag's binding identity is the stable + /// (groupId, edgeNodeId, deviceId?, metricName) tuple; deviceId is optional (absent + /// means the metric is published by the edge node itself), and there is no topic — + /// a Sparkplug driver subscribes one group-wide filter and routes by the decoded topic + birth + /// certificate, never by a per-tag topic. Never throws. + /// + /// + /// + /// dataType is OPTIONAL here, unlike in Plain mode. A Sparkplug metric declares + /// its own datatype in its birth certificate, so an unauthored tag legitimately takes the + /// type the birth declares — that is the whole point of Task 22's UntilStable + /// discovery. It is still read strictly when present (a typo'd value rejects the tag, + /// exactly as in Plain mode) and the outcome is recorded on + /// so the ingest path can tell "the operator + /// declared String" from "nobody declared anything and the record defaulted". + /// + /// + /// Plain-shape keys are read but not required. topic/payloadFormat/ + /// jsonPath are meaningless to the Sparkplug ingest path and are deliberately NOT + /// validated — a blob retyped in the editor keeps them, and rejecting the tag for a stale + /// leftover key would take a correctly-authored Sparkplug tag dark with no stated cause. + /// + /// + /// The authored equipment-tag TagConfig JSON. + /// The tag's RawPath — becomes the definition's identity (Name). + /// The mapped definition when this returns . + /// when the blob carries a usable Sparkplug descriptor. + public static bool FromSparkplugTagConfig(string tagConfig, string rawPath, out MqttTagDefinition def) + { + def = null!; + if (string.IsNullOrWhiteSpace(tagConfig) || tagConfig[0] != '{') return false; + try + { + using var doc = JsonDocument.Parse(tagConfig); + var root = doc.RootElement; + if (root.ValueKind != JsonValueKind.Object) return false; + + // The binding tuple. Without all three of these there is nothing a birth certificate could + // ever bind the tag to, so an absent/blank value is a hard reject (→ BadNodeIdUnknown) + // rather than a definition that can never resolve and reports nothing about why. + var groupId = ReadString(root, MqttTagConfigKeys.GroupId); + var edgeNodeId = ReadString(root, MqttTagConfigKeys.EdgeNodeId); + var metricName = ReadString(root, MqttTagConfigKeys.MetricName); + if (string.IsNullOrWhiteSpace(groupId) + || string.IsNullOrWhiteSpace(edgeNodeId) + || string.IsNullOrWhiteSpace(metricName)) + { + return false; + } + + // Optional: a node-level metric has no device segment. Blank normalises to absent so + // "deviceId":"" and an omitted key describe the same scope rather than two. + var deviceId = ReadString(root, MqttTagConfigKeys.DeviceId); + if (string.IsNullOrWhiteSpace(deviceId)) deviceId = null; + + // Strict, but only when present — see the remarks. + var dataTypeAuthored = root.TryGetProperty("dataType", out _); + if (!TagConfigJson.TryReadEnumStrict(root, "dataType", DriverDataType.String, out var dataType)) + return false; + + if (!TryReadQosStrict(root, out var qos)) return false; + + def = new MqttTagDefinition( + Name: rawPath, + // Plain-shape fields carried through verbatim; the Sparkplug ingest path reads none of + // them, and a retyped blob's leftovers must not change what the tag means. + Topic: ReadString(root, MqttTagConfigKeys.Topic), + PayloadFormat: MqttPayloadFormat.Json, + JsonPath: RootJsonPath, + DataType: dataType, + Qos: qos, + RetainSeed: ReadBoolOrDefault(root, "retainSeed", defaultValue: true), + DataTypeAuthored: dataTypeAuthored, + GroupId: groupId, + EdgeNodeId: edgeNodeId, + DeviceId: deviceId, + MetricName: metricName); + return true; + } + catch (JsonException) { return false; } + catch (FormatException) { return false; } + catch (InvalidOperationException) { return false; } + } + + /// + /// Deploy-time inspection of an equipment-tag TagConfig blob. Returns human-readable + /// warnings for a structurally unparseable blob (which the runtime turns into a silent + /// BadNodeIdUnknown), for present-but-invalid payloadFormat / dataType / + /// qos values, and for a wildcard tag topic (a tag bound to + / # + /// would receive values from many topics — ambiguous, and almost never what the operator meant). + /// Empty when the blob is clean or is not an equipment-tag TagConfig object. Never throws. + /// + /// The equipment tag's TagConfig JSON. + /// The warnings; empty when clean. + public static IReadOnlyList Inspect(string reference) + { + var warnings = new List(); + if (string.IsNullOrWhiteSpace(reference) || reference[0] != '{') return warnings; + try + { + using var doc = JsonDocument.Parse(reference); + var root = doc.RootElement; + if (root.ValueKind != JsonValueKind.Object) + { + warnings.Add("Mqtt TagConfig root is not a JSON object — the tag will not resolve (BadNodeIdUnknown)."); + return warnings; + } + foreach (var w in new[] + { + TagConfigJson.DescribeInvalidEnum(root, "payloadFormat"), + TagConfigJson.DescribeInvalidEnum(root, "dataType"), + DescribeInvalidQos(root), + }) + { + if (w is not null) warnings.Add(w); + } + var topic = ReadString(root, "topic"); + if (topic.IndexOfAny(TopicWildcards) >= 0) + { + warnings.Add( + $"value '{topic}' for 'topic' contains an MQTT wildcard (+ or #); a tag's topic must be " + + "concrete, or the tag will be fed by every matching topic."); + } + } + catch (JsonException) + { + warnings.Add("Mqtt TagConfig is not valid JSON — the tag will not resolve (BadNodeIdUnknown)."); + } + return warnings; + } + + /// + /// Strict qos read, mirroring 's + /// absent / valid / present-but-invalid split: absent ⇒ (the driver-level + /// wins); a JSON integer in 0–2 ⇒ that value; anything + /// else present (non-number, non-integer, or out of range) ⇒ the read FAILS. + /// + /// The TagConfig root object. + /// The parsed QoS, or when the field is absent. + /// only when qos is present but not a legal MQTT QoS. + private static bool TryReadQosStrict(JsonElement o, out int? qos) + { + qos = null; + if (!o.TryGetProperty("qos", out var e)) return true; // absent + if (e.ValueKind != JsonValueKind.Number || !e.TryGetInt32(out var v)) return false; + if (v is < 0 or > 2) return false; + qos = v; + return true; + } + + /// + /// A human-readable warning for a present-but-invalid qos field, or + /// when it is absent or valid — the qos counterpart of + /// . + /// + /// The TagConfig root object. + /// The warning text, or . + private static string? DescribeInvalidQos(JsonElement o) + { + if (!o.TryGetProperty("qos", out var e)) return null; + if (e.ValueKind == JsonValueKind.Number && e.TryGetInt32(out var v) && v is >= 0 and <= 2) return null; + return $"value '{e.GetRawText()}' for 'qos' is not a valid MQTT QoS; valid: 0, 1, 2"; + } + + private static string ReadString(JsonElement o, string name) + => o.TryGetProperty(name, out var e) && e.ValueKind == JsonValueKind.String + ? e.GetString() ?? "" + : ""; + + private static bool ReadBoolOrDefault(JsonElement o, string name, bool defaultValue) + => o.TryGetProperty(name, out var e) && e.ValueKind is JsonValueKind.True or JsonValueKind.False + ? e.GetBoolean() + : defaultValue; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/Protos/sparkplug_b.proto b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/Protos/sparkplug_b.proto new file mode 100644 index 00000000..9bbdc73a --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/Protos/sparkplug_b.proto @@ -0,0 +1,261 @@ +// ===================================================================================================== +// VENDORED THIRD-PARTY FILE — DO NOT EDIT. Re-vendor from upstream instead (recipe below). +// +// Upstream project Eclipse Tahu — the Sparkplug B reference implementation +// Upstream repo https://github.com/eclipse-tahu/tahu +// Upstream path sparkplug_b/sparkplug_b.proto +// Pinned commit 5736e404889d4b95910613040a99ba79589ffb13 (master, 2023-11-06) +// Permalink https://raw.githubusercontent.com/eclipse-tahu/tahu/5736e404889d4b95910613040a99ba79589ffb13/sparkplug_b/sparkplug_b.proto +// git blob SHA-1 bf72ab5f09a333afabcb40fd45362ffbb0c8c5bd (matches the tree entry at that commit) +// content SHA-256 4432c5c483b7fb9732d0594c98a2e97dca5e517e39c5374a8b918d837f0b4a19 (8330 bytes) +// last modified 46f25e79f34234e6145d11108660dfd9133ae50d (2022-05-16, template_ref comment fix) +// License Eclipse Public License 2.0 (EPL-2.0) — https://www.eclipse.org/legal/epl-2.0/ +// Copyright (c) Cirrus Link Solutions and others. The upstream copyright + SPDX +// header is preserved verbatim as the first lines of the copied body below. +// +// Everything from line 38 down ("// * Copyright (c) 2015, 2018 Cirrus Link Solutions…") is a +// BYTE-FOR-BYTE copy of the upstream file. Only these header lines were added. To re-verify: +// +// curl -sSL -o /tmp/upstream.proto +// tail -n +38 src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/Protos/sparkplug_b.proto > /tmp/vendored.proto +// diff /tmp/upstream.proto /tmp/vendored.proto && shasum -a 256 /tmp/vendored.proto +// +// WHY VENDORED rather than a NuGet package: SparkplugNet is the only .NET Sparkplug library and it is +// stale (1.3.10, 2024-07-02), transitively pins MQTTnet 4.3.6.1152 against this repo's MQTTnet 5.2.0, +// and ships net6.0/net8.0 only — no net10.0 TFM. So the schema is vendored and decoded directly with +// Google.Protobuf, over the same single MQTTnet-5 client the plain-MQTT path already uses. +// +// WHY THE proto2 FILE and not the sibling sparkplug_b_c_sharp.proto that upstream also ships: this is +// the NORMATIVE Sparkplug B schema (the C# sibling is a lossy proto3 restatement that drops explicit +// presence and rewrites the `extensions` ranges as `google.protobuf.Any`). protoc from Grpc.Tools +// generates valid C# from proto2, and the generated namespace is identical (Org.Eclipse.Tahu.Protobuf, +// PascalCased from `package org.eclipse.tahu.protobuf` — there is no `option csharp_namespace`). Keeping +// proto2 buys explicit presence — Has{Name,Alias,Seq,IsNull,Datatype} — which the decoder needs to tell +// "field absent" from "field present and zero" (an NBIRTH legitimately carries seq = 0, and a DATA +// metric legitimately omits `name` and carries only `alias`). +// ===================================================================================================== + +// * Copyright (c) 2015, 2018 Cirrus Link Solutions and others +// * +// * This program and the accompanying materials are made available under the +// * terms of the Eclipse Public License 2.0 which is available at +// * http://www.eclipse.org/legal/epl-2.0. +// * +// * SPDX-License-Identifier: EPL-2.0 +// * +// * Contributors: +// * Cirrus Link Solutions - initial implementation + +// +// To compile: +// cd client_libraries/java +// protoc --proto_path=../../ --java_out=src/main/java ../../sparkplug_b.proto +// + +syntax = "proto2"; + +package org.eclipse.tahu.protobuf; + +option java_package = "org.eclipse.tahu.protobuf"; +option java_outer_classname = "SparkplugBProto"; + +enum DataType { + // Indexes of Data Types + + // Unknown placeholder for future expansion. + Unknown = 0; + + // Basic Types + Int8 = 1; + Int16 = 2; + Int32 = 3; + Int64 = 4; + UInt8 = 5; + UInt16 = 6; + UInt32 = 7; + UInt64 = 8; + Float = 9; + Double = 10; + Boolean = 11; + String = 12; + DateTime = 13; + Text = 14; + + // Additional Metric Types + UUID = 15; + DataSet = 16; + Bytes = 17; + File = 18; + Template = 19; + + // Additional PropertyValue Types + PropertySet = 20; + PropertySetList = 21; + + // Array Types + Int8Array = 22; + Int16Array = 23; + Int32Array = 24; + Int64Array = 25; + UInt8Array = 26; + UInt16Array = 27; + UInt32Array = 28; + UInt64Array = 29; + FloatArray = 30; + DoubleArray = 31; + BooleanArray = 32; + StringArray = 33; + DateTimeArray = 34; +} + +message Payload { + + message Template { + + message Parameter { + optional string name = 1; + optional uint32 type = 2; + + oneof value { + uint32 int_value = 3; + uint64 long_value = 4; + float float_value = 5; + double double_value = 6; + bool boolean_value = 7; + string string_value = 8; + ParameterValueExtension extension_value = 9; + } + + message ParameterValueExtension { + extensions 1 to max; + } + } + + optional string version = 1; // The version of the Template to prevent mismatches + repeated Metric metrics = 2; // Each metric includes a name, datatype, and optionally a value + repeated Parameter parameters = 3; + optional string template_ref = 4; // MUST be a reference to a template definition if this is an instance (i.e. the name of the template definition) - MUST be omitted for template definitions + optional bool is_definition = 5; + extensions 6 to max; + } + + message DataSet { + + message DataSetValue { + + oneof value { + uint32 int_value = 1; + uint64 long_value = 2; + float float_value = 3; + double double_value = 4; + bool boolean_value = 5; + string string_value = 6; + DataSetValueExtension extension_value = 7; + } + + message DataSetValueExtension { + extensions 1 to max; + } + } + + message Row { + repeated DataSetValue elements = 1; + extensions 2 to max; // For third party extensions + } + + optional uint64 num_of_columns = 1; + repeated string columns = 2; + repeated uint32 types = 3; + repeated Row rows = 4; + extensions 5 to max; // For third party extensions + } + + message PropertyValue { + + optional uint32 type = 1; + optional bool is_null = 2; + + oneof value { + uint32 int_value = 3; + uint64 long_value = 4; + float float_value = 5; + double double_value = 6; + bool boolean_value = 7; + string string_value = 8; + PropertySet propertyset_value = 9; + PropertySetList propertysets_value = 10; // List of Property Values + PropertyValueExtension extension_value = 11; + } + + message PropertyValueExtension { + extensions 1 to max; + } + } + + message PropertySet { + repeated string keys = 1; // Names of the properties + repeated PropertyValue values = 2; + extensions 3 to max; + } + + message PropertySetList { + repeated PropertySet propertyset = 1; + extensions 2 to max; + } + + message MetaData { + // Bytes specific metadata + optional bool is_multi_part = 1; + + // General metadata + optional string content_type = 2; // Content/Media type + optional uint64 size = 3; // File size, String size, Multi-part size, etc + optional uint64 seq = 4; // Sequence number for multi-part messages + + // File metadata + optional string file_name = 5; // File name + optional string file_type = 6; // File type (i.e. xml, json, txt, cpp, etc) + optional string md5 = 7; // md5 of data + + // Catchalls and future expansion + optional string description = 8; // Could be anything such as json or xml of custom properties + extensions 9 to max; + } + + message Metric { + + optional string name = 1; // Metric name - should only be included on birth + optional uint64 alias = 2; // Metric alias - tied to name on birth and included in all later DATA messages + optional uint64 timestamp = 3; // Timestamp associated with data acquisition time + optional uint32 datatype = 4; // DataType of the metric/tag value + optional bool is_historical = 5; // If this is historical data and should not update real time tag + optional bool is_transient = 6; // Tells consuming clients such as MQTT Engine to not store this as a tag + optional bool is_null = 7; // If this is null - explicitly say so rather than using -1, false, etc for some datatypes. + optional MetaData metadata = 8; // Metadata for the payload + optional PropertySet properties = 9; + + oneof value { + uint32 int_value = 10; + uint64 long_value = 11; + float float_value = 12; + double double_value = 13; + bool boolean_value = 14; + string string_value = 15; + bytes bytes_value = 16; // Bytes, File + DataSet dataset_value = 17; + Template template_value = 18; + MetricValueExtension extension_value = 19; + } + + message MetricValueExtension { + extensions 1 to max; + } + } + + optional uint64 timestamp = 1; // Timestamp at message sending time + repeated Metric metrics = 2; // Repeated forever - no limit in Google Protobufs + optional uint64 seq = 3; // Sequence number + optional string uuid = 4; // UUID to track message type in terms of schema definitions + optional bytes body = 5; // To optionally bypass the whole definition above + extensions 6 to max; // For third party extensions +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/SparkplugDataType.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/SparkplugDataType.cs new file mode 100644 index 00000000..20bc98b9 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/SparkplugDataType.cs @@ -0,0 +1,139 @@ +// `SparkplugDataType` is an ALIAS for the vendored proto's generated `Org.Eclipse.Tahu.Protobuf.DataType` +// enum, not a second, hand-maintained enum. See the remarks on `SparkplugDataTypeExtensions` below for +// the reasoning; the short version is that a duplicate enum is a drift hazard this repo has a documented +// systemic bug class around (see CLAUDE.md "Driver enum-serialization bug"), and `Payload.Types.Metric`'s +// wire-level `Datatype` field is a raw `uint32` anyway — nothing structurally forces a second CLR enum to +// exist, so the lowest-risk shape is for `SparkplugDataType` to be the exact same type as the generated +// one, not a value-compatible lookalike that some cast has to bridge. +global using SparkplugDataType = Org.Eclipse.Tahu.Protobuf.DataType; + +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// Maps a Sparkplug B metric (the vendored Eclipse Tahu proto's +/// generated Org.Eclipse.Tahu.Protobuf.DataType enum — see the global using alias +/// above) to a driver-agnostic , per design doc §3.5. +/// +/// +/// +/// Why an alias, not a duplicate enum. The obvious "clean seam" shape is a fresh +/// Driver.Mqtt.Contracts-owned enum with its own ToDriverDataType() — but that +/// creates a second definition of the same 35-member vocabulary that has to be hand-kept in +/// sync with whatever Eclipse Tahu's sparkplug_b.proto defines, which is exactly the +/// enum-drift shape this repo already has a name for (CLAUDE.md's "Driver enum-serialization +/// bug (AdminUI authoring)" — a systemic mismatch between two enums meant to describe the same +/// thing). It also does not match how the decoder actually produces values: Sparkplug's +/// Metric.datatype wire field is a raw uint32 (see sparkplug_b.proto line +/// 230 — optional uint32 datatype = 4), not the DataType enum type itself, so +/// SparkplugCodec (Task 16) already casts the decoded value straight to the generated +/// Org.Eclipse.Tahu.Protobuf.DataType (locally aliased there as TahuDataType). +/// A second, hand-duplicated enum would force every downstream consumer (Tasks 18/19/20/21) to +/// cast between two value-compatible-but-nominally-different enums to call this extension +/// method — extra surface for exactly zero benefit, since both "enums" would need to enumerate +/// the identical 35 members in the identical order to stay castable. +/// +/// +/// Making a global using alias for the generated type +/// sidesteps all of that: there is no second definition to drift, by construction — the alias +/// and the generated enum are the exact same CLR type. below is +/// written as an extension on purely for call-site readability; +/// it is equally callable as someDecodedDatatype.ToDriverDataType() against a plain +/// Org.Eclipse.Tahu.Protobuf.DataType value with no cast, which is exactly the shape +/// SparkplugCodec's decoded metrics hand back. +/// +/// +/// Drift guard. Because there is only one enum, SparkplugDataTypeTests' +/// completeness test (ToDriverDataType_HandlesEveryGeneratedDataTypeMember_...) can +/// enumerate Enum.GetValues<SparkplugDataType>() — which, being the alias, is +/// literally the live generated member set — and assert every member is either mapped or on +/// the explicit unsupported list below. If Eclipse Tahu's proto ever gains a member, that test +/// picks it up automatically and fails until this map makes an explicit decision about it, +/// rather than the new member silently falling through a duplicate-enum's stale default. +/// +/// +/// Per design §3.5: Int8/UInt8 widen to / +/// (no OPC UA signed-byte distinction is needed and +/// has no 8-bit members at all); Float/Double map to +/// / — note there is +/// no DriverDataType.Double member, only Float64; Text/UUID +/// (generated as Uuid — protoc mangles the wire spelling, see +/// SparkplugProtoCodegenTests.DataTypeEnum_CSharpNamesAreProtocMangled_NotTheProtoSpelling) +/// /Bytes/File all fall back to (v1 base64/raw +/// fallback for Bytes/File, per design); every *Array variant maps to its scalar element +/// type ( carries the "and it's an array" bit separately, since +/// itself has no array concept — that lives at the OPC UA +/// ValueRank/ArrayDimensions layer the address-space builder owns). DataSet/ +/// Template/PropertySet/PropertySetList/Unknown are unsupported in +/// v1 and map to — deliberately, not a guessed +/// , so a caller has to make an explicit skip-or-warn +/// decision (unlike Galaxy's DataTypeMap, which silently defaults unknown codes to +/// String for legacy wire-compatibility reasons that do not apply here). +/// +/// +public static class SparkplugDataTypeExtensions +{ + /// + /// Maps a Sparkplug metric datatype to the equivalent , or + /// if the type is unsupported in v1 (DataSet, Template, + /// PropertySet, PropertySetList, Unknown). Callers must treat + /// as an explicit "skip and warn", never coerce it to a guessed type. + /// + public static DriverDataType? ToDriverDataType(this SparkplugDataType dataType) => dataType switch + { + SparkplugDataType.Int8 => DriverDataType.Int16, + SparkplugDataType.Int16 => DriverDataType.Int16, + SparkplugDataType.Int32 => DriverDataType.Int32, + SparkplugDataType.Int64 => DriverDataType.Int64, + SparkplugDataType.Uint8 => DriverDataType.UInt16, + SparkplugDataType.Uint16 => DriverDataType.UInt16, + SparkplugDataType.Uint32 => DriverDataType.UInt32, + SparkplugDataType.Uint64 => DriverDataType.UInt64, + SparkplugDataType.Float => DriverDataType.Float32, + SparkplugDataType.Double => DriverDataType.Float64, + SparkplugDataType.Boolean => DriverDataType.Boolean, + SparkplugDataType.String => DriverDataType.String, + SparkplugDataType.DateTime => DriverDataType.DateTime, + SparkplugDataType.Text => DriverDataType.String, + SparkplugDataType.Uuid => DriverDataType.String, + SparkplugDataType.Bytes => DriverDataType.String, + SparkplugDataType.File => DriverDataType.String, + + SparkplugDataType.Int8Array => DriverDataType.Int16, + SparkplugDataType.Int16Array => DriverDataType.Int16, + SparkplugDataType.Int32Array => DriverDataType.Int32, + SparkplugDataType.Int64Array => DriverDataType.Int64, + SparkplugDataType.Uint8Array => DriverDataType.UInt16, + SparkplugDataType.Uint16Array => DriverDataType.UInt16, + SparkplugDataType.Uint32Array => DriverDataType.UInt32, + SparkplugDataType.Uint64Array => DriverDataType.UInt64, + SparkplugDataType.FloatArray => DriverDataType.Float32, + SparkplugDataType.DoubleArray => DriverDataType.Float64, + SparkplugDataType.BooleanArray => DriverDataType.Boolean, + SparkplugDataType.StringArray => DriverDataType.String, + SparkplugDataType.DateTimeArray => DriverDataType.DateTime, + + // Unsupported v1 (design §3.5): DataSet/Template are deferred scope; PropertySet/ + // PropertySetList are PropertyValue-only metadata types that never legitimately appear as a + // Metric's own datatype; Unknown is the proto's explicit "placeholder for future expansion" + // (index 0). All five fall through here deliberately rather than being listed with a fake + // mapping. + _ => null, + }; + + /// + /// Whether is one of the 13 *Array variants. Combine with + /// (which already returns the *element* type for an array + /// variant) to build an OPC UA ValueRank=1 array node per design §3.5. + /// + public static bool IsSparkplugArray(this SparkplugDataType dataType) => dataType switch + { + SparkplugDataType.Int8Array or SparkplugDataType.Int16Array or SparkplugDataType.Int32Array + or SparkplugDataType.Int64Array or SparkplugDataType.Uint8Array or SparkplugDataType.Uint16Array + or SparkplugDataType.Uint32Array or SparkplugDataType.Uint64Array or SparkplugDataType.FloatArray + or SparkplugDataType.DoubleArray or SparkplugDataType.BooleanArray or SparkplugDataType.StringArray + or SparkplugDataType.DateTimeArray => true, + _ => false, + }; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts.csproj b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts.csproj new file mode 100644 index 00000000..dae6d0b6 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Contracts.csproj @@ -0,0 +1,36 @@ + + + net10.0 + enable + enable + true + + + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/LastValueCache.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/LastValueCache.cs new file mode 100644 index 00000000..f759707f --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/LastValueCache.cs @@ -0,0 +1,63 @@ +using System.Collections.Concurrent; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// Thread-safe last-observed-value store backing for MQTT/Sparkplug B. +/// MQTT is a subscribe-first protocol — the driver holds one live broker connection and values +/// arrive pushed, not polled. But the OPC UA server still issues polled IReadable.ReadAsync +/// batches, so this cache is the bridge: the subscription path (message handler) calls +/// with the newest observed value per reference, and the read path serves +/// the most recent one via . +/// +/// +/// Keyed by RawPath — the v3 driver-reference identity (see +/// EquipmentTagRefResolver<TDef> and , +/// which is the RawPath) — never a topic/JSON-path-derived key. The parameter is +/// therefore named rawPath throughout rather than "topic" or "key" to keep that identity +/// fact visible at every call site. +/// +public sealed class LastValueCache +{ + // OPC UA BadWaitingForInitialData (0x80320000) — the repo-wide convention for "no value has + // been observed yet" (mirrored by CalculationDriver before its first evaluation, + // VirtualTagEngine for a freshly-materialised node, and OtOpcUaNodeManager / + // AddressSpaceApplier for a just-deployed variable). MQTT is subscribe-first, so an unseen + // RawPath is in exactly that state until its first publish arrives. Deliberately NOT + // GoodNoData — that code is reserved (see NullHistorianDataSource, OtOpcUaNodeManager + // HistoryRead paths) for "the historian window held no samples", a different question from + // "has this live reference ever been observed". + private const uint BadWaitingForInitialData = 0x80320000u; + + private readonly ConcurrentDictionary _values = new(); + + /// + /// Record the newest observed value for . Called from the MQTT + /// subscription/message-handling path. A null or empty + /// is a no-op — never throws. + /// + /// The RawPath identifying the tag. + /// The newest observed value/quality/timestamps for that tag. + public void Update(string rawPath, DataValueSnapshot snapshot) + { + if (string.IsNullOrEmpty(rawPath)) return; + _values[rawPath] = snapshot; + } + + /// + /// Read the last observed value for . Never throws: a + /// null/empty or a RawPath never observed both return a + /// snapshot rather than an exception, so a batch read + /// covering many references degrades per-reference instead of failing the whole call. + /// + /// The RawPath identifying the tag. + /// The last observed snapshot, or a BadWaitingForInitialData snapshot if none has been observed. + public DataValueSnapshot Read(string rawPath) + { + if (!string.IsNullOrEmpty(rawPath) && _values.TryGetValue(rawPath, out var snapshot)) + return snapshot; + + return new DataValueSnapshot(null, BadWaitingForInitialData, null, DateTime.UtcNow); + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttConnection.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttConnection.cs new file mode 100644 index 00000000..1733fefc --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttConnection.cs @@ -0,0 +1,1561 @@ +using System.Buffers; +using System.Net.Security; +using System.Security.Cryptography; +using System.Security.Cryptography.X509Certificates; +using Microsoft.Extensions.Logging; +using MQTTnet; +using MQTTnet.Protocol; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// Receives one inbound MQTT application message. Invoked on MQTTnet's own dispatcher +/// thread: an implementation must not block, must not do I/O and must not throw, or it stalls +/// the client's pump for every other subscription. +/// +/// The concrete topic the message arrived on (never a filter). +/// +/// The message body. Only valid for the duration of the call — the underlying buffer belongs to +/// MQTTnet, so anything an implementation wants to keep must be copied out. +/// +/// +/// The message's MQTT retain flag as delivered to this client, i.e. true for +/// the broker's stored last-known value replayed at subscribe time, and false for an +/// ordinary live publish. This is the retained seed signal. +/// +public delegate void MqttMessageObserver(string topic, ReadOnlySpan payload, bool retained); + +/// One topic filter to SUBSCRIBE, as the subscription manager wants it established. +/// The topic filter (concrete, or carrying MQTT wildcards). +/// Requested QoS, 0–2. +/// +/// Whether the broker should replay its retained message for this filter at subscribe time. +/// Honoured natively on MQTT 5.0 (retain handling); on 3.1.1 the broker always replays and the +/// manager drops the seed client-side instead. +/// +public sealed record MqttTopicSubscription(string Topic, int Qos, bool SeedRetained); + +/// The broker's SUBACK verdict for one requested filter. +/// The filter this outcome answers. +/// Whether the broker granted the subscription. +/// The broker's reason code / string, for logs and diagnostics. +public sealed record MqttSubscribeOutcome(string Topic, bool Granted, string Reason); + +/// +/// The one seam through which the subscription manager establishes MQTT subscriptions. +/// is the production implementation; the interface exists so the +/// manager's SUBACK handling (per-filter grant / rejection, total failure) is exercisable without +/// a broker. +/// +public interface IMqttSubscribeTransport +{ + /// + /// Issues one SUBSCRIBE carrying every filter and returns the broker's per-filter verdict. + /// Implementations must be bounded — a broker that accepts SUBSCRIBE and never SUBACKs must + /// fail at a deadline, not hang. + /// + /// The filters to establish; never empty. + /// Caller cancellation, linked with the implementation's deadline. + /// One outcome per requested filter. + Task> SubscribeAsync( + IReadOnlyList filters, + CancellationToken cancellationToken); +} + +/// Lifecycle state of an . +public enum MqttConnectionState +{ + /// Constructed, or the first connect has not yet succeeded. No supervisor is running. + Disconnected = 0, + + /// An MQTT session is established. + Connected = 1, + + /// + /// The session dropped and the supervisor is retrying under backoff. A broker that is merely + /// down stays here indefinitely — being unreachable is never a fault. + /// + Reconnecting = 2, + + /// + /// Unrecoverable without a driver restart: retrying cannot help, so the supervisor has + /// stopped. Two causes — unusable configuration (a failure raised while assembling the client + /// options, before any I/O), or the supervisor itself dying on an unexpected exception. A + /// broker that is merely unreachable never lands here. + /// + Faulted = 3, + + /// has run. Terminal. + Disposed = 4, +} + +/// +/// The broker completed the handshake and refused the CONNECT, returning a non-success +/// CONNACK reason code. +/// +/// +/// +/// This exists because MQTTnet 5 does not throw on an unsuccessful CONNACK — it returns +/// the outcome in and leaves the client +/// disconnected, and v5 removed v4's ThrowOnNonSuccessfulConnectResponse option, so +/// inspecting the result is the only way to see it. A caller that ignores the result gets a +/// connect that "succeeded" against a broker which rejected it: a driver configured with the +/// wrong broker password would report Healthy and never receive a single value. That is +/// exactly what the Task-13 live fixture caught. +/// +/// +/// distinguishes a rejection no amount of retrying can fix +/// (credentials, identity, protocol, an invalid Last-Will) from a genuinely transient refusal +/// (broker unavailable/busy, quota, rate limit). See +/// for the full mapping. +/// +/// +public sealed class MqttConnectRejectedException : Exception +{ + internal MqttConnectRejectedException(MqttClientConnectResultCode resultCode, bool isUnrecoverable, string message) + : base(message) + { + ResultCode = resultCode; + IsUnrecoverable = isUnrecoverable; + } + + /// Whether retrying against this broker with this configuration can never succeed. + public bool IsUnrecoverable { get; } + + /// The CONNACK reason code the broker returned. + public MqttClientConnectResultCode ResultCode { get; } +} + +/// +/// Owns the driver's single live MQTTnet-5 client: assembles the broker connection options +/// from (endpoint, protocol version, session, credentials, +/// TLS + CA pin), connects under a bounded deadline, and keeps the session alive with a +/// hand-rolled reconnect supervisor. +/// +/// +/// +/// Connection-free construction. The constructor stores configuration only — it +/// opens no socket, creates no client and starts no supervisor. The universal-browser +/// CanBrowse pattern constructs a throwaway instance purely to ask what driver type +/// it is, so a constructor that dialled a broker would stall the AdminUI. +/// +/// +/// Bounded connect. links the caller's token with a +/// deadline, and the same value is +/// fed to MQTTnet's own . A broker that accepts the +/// TCP connection and then never answers CONNACK — the frozen-peer shape that wedged the +/// S7 read path (arch-review R2-01) — fails at the deadline instead of hanging forever. +/// Every wait in this type is bounded the same way, teardown included. +/// +/// +/// Hand-rolled reconnect. MQTTnet v5 dropped v4's ManagedMqttClient, so there +/// is no library-managed reconnect: this type supplies it. The supervisor starts on the +/// first successful connect (a failed first attempt is the caller's to retry — the +/// driver-host resilience layer re-runs InitializeAsync — and must not leave a +/// background task hammering an endpoint the caller has given up on). MQTTnet's +/// DisconnectedAsync callback only signals a semaphore and returns, so the library's +/// own dispatcher thread is never blocked by a backoff sleep. +/// +/// +/// is load-bearing. MQTT subscriptions do not survive a +/// clean session, and a reconnect that completes without re-subscribing produces a +/// healthy-looking connection that receives nothing, forever, with no error, no exception +/// and no bad status. So the callback fires on every successful reconnect — including +/// persistent-session reconnects, where re-subscribing is merely redundant. Re-subscribing +/// an already-subscribed topic is harmless; missing one is silent death. If the callback +/// fails, the freshly-established session is torn down and retried rather than left +/// connected-but-deaf. It is invoked under a cancellation token and a +/// ceiling: a subscriber that hangs on +/// a broker which accepts SUBSCRIBE and never SUBACKs — the frozen-peer shape again — must +/// not be able to park the supervisor or outlive . +/// +/// +/// Connect is idempotent. MQTTnet throws (and raises no DisconnectedAsync) if +/// asked to connect a client that is already connected, so both connect paths check first and +/// return a no-op. The two paths genuinely race: the supervisor is told to expect the driver +/// host to re-run InitializeAsync, so a caller can restore the session while the +/// supervisor sits in backoff, and the supervisor can restore it a moment before a caller +/// asks. Without the check, the first case leaves the supervisor failing forever against a +/// perfectly healthy connection (inflating attempt, so the next genuine outage +/// waits instead of min) and the +/// second throws an undocumented out of +/// on a working session. When the supervisor finds the session +/// already restored it stops retrying without firing — the +/// caller that reconnected owns its own subscribe, per the +/// contract. +/// +/// +/// State transitions. is terminal; nothing +/// moves off it. +/// +/// +/// +/// +/// set by on success (or on finding the session +/// already up), and by the supervisor only after +/// has completed — so on the reconnect path +/// Connected means "connected and re-subscribed". +/// +/// +/// +/// +/// +/// set by the DisconnectedAsync callback when a live session drops, and +/// held by the supervisor for the whole retry + re-subscribe sequence. +/// +/// +/// +/// +/// +/// set when the client options cannot be assembled, or when the supervisor exits +/// on an unexpected exception. Both mean no further recovery without a restart. +/// +/// +/// +/// +/// set first thing in . +/// +/// +/// and can legitimately disagree in both +/// directions and are not interchangeable: Reconnecting with +/// IsConnected == true is the window where the socket is up but the re-subscribe has +/// not finished, and Connected with IsConnected == false is the instant between +/// the socket dropping and MQTTnet raising its callback. is the health +/// surface; is the transport fact. +/// +/// +/// Credentials never leak. Nothing here logs or formats +/// ; the options record itself redacts it in +/// ToString(). +/// +/// +/// Lifecycle is serialised. Connect (caller-initiated or supervisor-initiated) and +/// dispose run under a single gate, and a connect that completes re-checks the disposed flag +/// while still holding that gate. This closes the connection leak the Task-3 review +/// traced: dispose could observe a still-null client, dispose nothing and return, after +/// which the in-flight connect assigned a client and connected successfully — leaving a live +/// socket no later dispose could ever reach. Now the losing side of that race disposes the +/// client it created and reports . +/// +/// +/// The publish leg is deliberately narrow. exists to satisfy +/// — whose entire production caller is +/// — not to make this a general MQTT publisher. This +/// driver has no IWritable leg in v1, so a Sparkplug rebirth NCMD is the only +/// outbound application message it ever sends. Sparkplug rebirth-on-reconnect itself is decided +/// by hanging off ; this type +/// only carries the bytes. +/// +/// +public sealed class MqttConnection : IAsyncDisposable, IMqttSubscribeTransport, Sparkplug.IMqttPublishTransport +{ + private readonly string _driverId; + + /// + /// Serialises connect (both callers of it) against dispose. Never disposed: a late MQTTnet + /// callback can still reach this instance after teardown, and a disposed + /// would throw inside the library's own dispatcher. + /// + private readonly SemaphoreSlim _lifecycleGate = new(1, 1); + + /// + /// Cancelled by ; aborts the supervisor, any in-flight connect and + /// any in-flight callback. Never disposed — deliberately, like the + /// two semaphores: a connect racing dispose reads + /// after its own disposed check, and disposing the source would turn that benign loser into an + /// naming the wrong type. It holds no timer and no + /// surviving registrations (every linked source here is using-scoped), so leaving it + /// undisposed costs nothing. + /// + private readonly CancellationTokenSource _lifetimeCts = new(); + + private readonly ILogger? _logger; + private readonly MqttDriverOptions _options; + + /// + /// Wakes the supervisor. Unbounded max count: spurious releases (a failed connect attempt + /// also raises DisconnectedAsync) are drained harmlessly by the still-connected guard, + /// whereas a bounded semaphore would throw on the extra release. Never disposed, for the same + /// reason as . + /// + private readonly SemaphoreSlim _reconnectWake = new(0); + + private IMqttClient? _client; + private int _disposed; + private long _lastMessageTicksUtc; + private Task? _reconnectWorker; + private int _state = (int)MqttConnectionState.Disconnected; + + /// Stores configuration only — no network, no client instantiation, no supervisor. + /// Broker connection settings. + /// Driver instance id, used only for log/diagnostic correlation. + /// Optional logger; never receives credentials. + public MqttConnection(MqttDriverOptions options, string driverId, ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(driverId); + + _options = options; + _driverId = driverId; + _logger = logger; + } + + /// + /// Raised after every successful reconnect (never after the initial + /// , which the caller already sequences its own subscribe behind, + /// and never when the supervisor merely finds the session already restored by such a caller). + /// Subscribers must re-establish their MQTT subscriptions here — see the type remarks for + /// why skipping it is silent death. Every subscriber is invoked even if an earlier one + /// throws; a throwing subscriber causes the session to be torn down and retried. + /// + /// + /// The supplied token is cancelled by , and the whole fan-out is + /// additionally capped at — a subscriber + /// that ignores its token cannot park the supervisor or make it outlive teardown. Subscribers + /// should flow the token into their own network calls rather than relying on that ceiling, + /// which abandons rather than stops the offending work. + /// + public event Func? Reconnected; + + /// + /// Raised for every inbound application message, on MQTTnet's own dispatcher thread. This + /// connection does no routing, parsing or typing of its own — that is the subscription + /// manager's job; here the message is only stamped onto and + /// handed on. + /// + /// + /// Handlers must not block, do I/O or throw. A throwing handler is caught and logged here + /// rather than being allowed to escape into the library's pump — one bad tag must not stop + /// delivery for every other subscriber. + /// + public event MqttMessageObserver? MessageReceived; + + /// Whether the underlying client currently holds an established MQTT session. + public bool IsConnected => _client?.IsConnected ?? false; + + /// + /// UTC timestamp of the most recent inbound application message, or null if none has + /// arrived on this instance. Connection health only — nothing here routes or parses the + /// message; that is the subscription manager's job (Task 6). + /// + /// + /// The plan calls this member LastMessageAgeUtc, which conflates a timestamp with an + /// age; it is split here into the instant () and the elapsed span + /// (). + /// + public DateTime? LastMessageUtc + { + get + { + var ticks = Interlocked.Read(ref _lastMessageTicksUtc); + return ticks == 0 ? null : new DateTime(ticks, DateTimeKind.Utc); + } + } + + /// + /// How long ago the most recent inbound application message arrived, or null if none + /// has. A connection that is with a large age is + /// the "reconnected but never re-subscribed" shape. + /// + public TimeSpan? LastMessageAge => LastMessageUtc is { } at ? DateTime.UtcNow - at : null; + + /// Current lifecycle state. See . + public MqttConnectionState State => (MqttConnectionState)Volatile.Read(ref _state); + + /// + /// Test seam: awaited inside the gated connect immediately after the broker session is + /// established and before the disposed re-check, so a test can reproduce the exact + /// connect/dispose interleaving that used to leak a live connection. Always null in + /// production. + /// + internal Func? AfterConnectHookForTests { get; set; } + + private bool Disposed => Volatile.Read(ref _disposed) != 0; + + /// + public async ValueTask DisposeAsync() + { + if (Interlocked.Exchange(ref _disposed, 1) != 0) + { + return; + } + + SetState(MqttConnectionState.Disposed); + + // Stop the supervisor and abort any in-flight connect FIRST, so the gate below is not held + // for a full connect deadline by work that is already pointless. + try + { + await _lifetimeCts.CancelAsync().ConfigureAwait(false); + } + catch (ObjectDisposedException) + { + // Nothing to cancel. + } + + var worker = Volatile.Read(ref _reconnectWorker); + if (worker is not null) + { + try + { + await worker.WaitAsync(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)).ConfigureAwait(false); + } + catch (Exception ex) + { + _logger?.LogDebug(ex, "MQTT driver '{DriverId}': reconnect supervisor did not stop cleanly.", _driverId); + } + } + + // Bounded even here: a wedged connect must not turn dispose into a hang. If the gate cannot + // be taken we still claim the client — double-disposing an MqttClient is harmless, leaking a + // live one is not. + var gated = await _lifecycleGate + .WaitAsync(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)) + .ConfigureAwait(false); + if (!gated) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': lifecycle gate still held at dispose; tearing the client down anyway.", + _driverId); + } + + try + { + var client = Interlocked.Exchange(ref _client, null); + if (client is not null) + { + await CloseAsync(client).ConfigureAwait(false); + } + } + finally + { + if (gated) + { + _lifecycleGate.Release(); + } + } + } + + /// + /// The backoff a supervisor waits before reconnect attempt : + /// minSeconds × 2^attempt, saturated at . Pure — no + /// clock, no state, no jitter (deliberately: the schedule is asserted by exact equality, and + /// with one connection per driver instance there is no thundering herd to spread out). + /// + /// + /// Computed in rather than by shifting, because min << attempt + /// wraps — 1 << 32 is 1 — which would silently turn a long outage back + /// into a once-a-second hammering of a dead broker. A maxSeconds below + /// is misconfiguration; the floor wins, because the floor + /// is what stops the loop going hot. + /// + /// Zero-based attempt index; negatives are treated as the first attempt. + /// Delay before the first attempt, and the floor for every later one. + /// Cap on the exponential growth. + public static TimeSpan NextBackoff(int attempt, int minSeconds, int maxSeconds) + { + var min = Math.Max(1d, minSeconds); + var max = Math.Max(min, maxSeconds); + + if (attempt <= 0) + { + return TimeSpan.FromSeconds(min); + } + + // Math.Pow saturates to +Infinity instead of wrapping, so a huge attempt count clamps to max. + var seconds = min * Math.Pow(2d, attempt); + return TimeSpan.FromSeconds(seconds >= max || double.IsNaN(seconds) ? max : seconds); + } + + /// + /// Connects to the broker, failing at + /// rather than waiting indefinitely on an unresponsive peer. On success the reconnect + /// supervisor takes over keeping the session alive. + /// + /// + /// + /// May be retried on the same instance after a failed attempt — the underlying + /// is created once and reused across attempts. Safe to call + /// concurrently with : the two are serialised, and a connect + /// that loses the race disposes whatever it built and throws + /// . + /// + /// + /// Idempotent. Calling it on a session that is already established — including one + /// the reconnect supervisor restored a moment earlier — is a no-op that returns normally, + /// not the MQTTnet would raise for a + /// connect-while-connected. The caller still owns re-establishing its own subscriptions + /// after this returns; is not fired for this path. + /// + /// + /// Caller cancellation; linked with the connect deadline. + /// The connect deadline elapsed. + /// was cancelled. + /// + /// The connection was disposed, either before the attempt started or while it was in flight. + /// + /// + /// The broker refused the CONNECT (non-success CONNACK — bad credentials, unsupported protocol + /// version, server unavailable …). MQTTnet reports this as a result, not an exception, + /// so this method inspects the CONNACK and raises it: a caller that treats "did not throw" as + /// "connected" would otherwise report a healthy session the broker never granted. + /// + public async Task ConnectAsync(CancellationToken cancellationToken) + { + ObjectDisposedException.ThrowIf(Disposed, this); + + await ConnectCoreAsync(supervisorAttempt: false, cancellationToken).ConfigureAwait(false); + } + + /// + /// Invokes every subscriber. Walks the invocation list explicitly: + /// awaiting a multicast delegate directly yields only the last subscriber's task and + /// abandons the earlier ones, so a single throwing subscriber would silently skip every + /// subscriber behind it — and a skipped re-subscribe is a topic that goes dark. Failures are + /// collected and rethrown after every subscriber has had its turn. + /// + /// Passed to every subscriber; cancelled by . + internal async Task FireReconnectedAsync(CancellationToken cancellationToken) + { + var subscribers = Reconnected; + if (subscribers is null) + { + return; + } + + List? failures = null; + foreach (var subscriber in subscribers.GetInvocationList().Cast>()) + { + try + { + await subscriber(cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) + { + (failures ??= []).Add(ex); + } + } + + if (failures is not null) + { + throw new AggregateException( + $"MQTT driver '{_driverId}': {failures.Count} re-subscribe callback(s) failed after reconnect.", + failures); + } + } + + /// + /// The single gated connect both entry points funnel through. + /// + /// + /// true when the reconnect supervisor is calling. It suppresses publishing + /// on success, because on the reconnect path + /// "connected" is only true once has re-subscribed — publishing it + /// here would advertise a healthy connection during the window in which it has no + /// subscriptions at all. It also suppresses starting the supervisor (it is the supervisor). + /// + /// Caller cancellation; linked with the connect deadline. + /// + /// true if this call established the session; false if it found one already + /// established and did nothing. The supervisor uses false to stop retrying without + /// firing — whoever established that session owns its subscribe. + /// + private async Task ConnectCoreAsync(bool supervisorAttempt, CancellationToken cancellationToken) + { + // The options are rebuilt per attempt so a rotated CA file is picked up on the next connect + // rather than being pinned for the life of the process. This is also the only step that can + // fail unrecoverably — it is pure config assembly, so retrying it can never help. + MqttClientOptions clientOptions; + try + { + clientOptions = BuildClientOptions(_options, clientIdSuffix: null, _logger); + } + catch (Exception ex) + { + SetState(MqttConnectionState.Faulted); + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': broker configuration is unusable; no amount of retrying will help.", + _driverId); + throw; + } + + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)); + using var linked = CancellationTokenSource.CreateLinkedTokenSource( + cancellationToken, + deadline.Token, + _lifetimeCts.Token); + + _logger?.LogDebug( + "MQTT driver '{DriverId}': connecting to {Host}:{Port} (tls={UseTls}, timeout={TimeoutSeconds}s).", + _driverId, + _options.Host, + _options.Port, + _options.UseTls, + _options.ConnectTimeoutSeconds); + + try + { + await _lifecycleGate.WaitAsync(linked.Token).ConfigureAwait(false); + } + catch (Exception ex) + { + throw Classify(ex, cancellationToken, deadline); + } + + try + { + // Dispose won the race outright: it has already flagged the instance and disposed + // whatever existed. Creating a client now would be creating one nobody can reach. + ObjectDisposedException.ThrowIf(Disposed, this); + + var client = _client; + if (client is null) + { + client = new MqttClientFactory().CreateMqttClient(); + AttachHandlers(client); + _client = client; + } + + // Idempotence, and the whole point of the gate: MQTTnet throws + // "not allowed to connect with a server after the connection is established" for a + // connect-while-connected AND raises no DisconnectedAsync for it, so neither the retry + // loop nor Classify would ever recover. Both entry points race for real — see the type + // remarks — so whichever arrives second must find the session and leave it alone. + if (client.IsConnected) + { + SetState(MqttConnectionState.Connected); + StartSupervisorIfCallerConnect(supervisorAttempt); + _logger?.LogDebug( + "MQTT driver '{DriverId}': connect to {Host}:{Port} skipped — the session is already established.", + _driverId, + _options.Host, + _options.Port); + return false; + } + + var connack = await client.ConnectAsync(clientOptions, linked.Token).ConfigureAwait(false); + + // MQTTnet 5 does NOT throw for a broker that refused the CONNECT — it hands the reason + // back here and leaves the client disconnected. Skipping this check is how a wrong broker + // password produced a Healthy driver that never received a value (Task-13 live gate). + if (connack is not null && connack.ResultCode != MqttClientConnectResultCode.Success) + { + throw BuildRejection(connack, supervisorAttempt); + } + + if (AfterConnectHookForTests is { } hook) + { + await hook().ConfigureAwait(false); + } + + // The race the Task-3 review traced: dispose flagged the instance while this connect was + // in flight, then blocked on the very gate this call holds — so it disposed nothing and + // the session below would have been unreachable forever. Hand it back here instead. + if (Disposed) + { + Interlocked.CompareExchange(ref _client, null, client); + await CloseAsync(client).ConfigureAwait(false); + throw new ObjectDisposedException( + nameof(MqttConnection), + $"MQTT driver '{_driverId}': the connection was disposed while a connect to " + + $"{_options.Host}:{_options.Port} was in flight; the established session was torn down."); + } + + // On the supervisor path the session is up but has no subscriptions yet, so State stays + // Reconnecting until FireReconnectedAsync succeeds — see the state table in the remarks. + if (!supervisorAttempt) + { + SetState(MqttConnectionState.Connected); + } + + StartSupervisorIfCallerConnect(supervisorAttempt); + return true; + } + catch (ObjectDisposedException) + { + throw; + } + // A refused CONNECT is already fully classified — it must not be re-wrapped as a timeout or a + // cancellation just because a token happened to fire while the CONNACK was being read. + catch (MqttConnectRejectedException) + { + throw; + } + catch (Exception ex) + { + throw Classify(ex, cancellationToken, deadline); + } + finally + { + _lifecycleGate.Release(); + } + } + + /// + /// Turns a refused CONNACK into the thrown contract, setting + /// first when the refusal is unrecoverable — which is what stops the supervisor + /// ( returns on Faulted) from retrying every + /// backoff period against a broker that will never accept these credentials. + /// + private MqttConnectRejectedException BuildRejection(MqttClientConnectResult connack, bool supervisorAttempt) + { + var unrecoverable = IsUnrecoverableConnackRejection(connack.ResultCode); + var message = DescribeConnackRejection($"{_options.Host}:{_options.Port}", connack.ResultCode); + + if (unrecoverable) + { + SetState(MqttConnectionState.Faulted); + _logger?.LogError( + "MQTT driver '{DriverId}': {Message} Retrying cannot help; fix the configuration and redeploy.", + _driverId, + message); + } + else + { + // Retryable: leave the state alone — Disconnected before a first connect, Reconnecting + // under the supervisor — so the backoff loop keeps trying at its bounded rate. + _logger?.LogWarning( + "MQTT driver '{DriverId}': {Message} Treating as transient{Suffix}.", + _driverId, + message, + supervisorAttempt ? " and continuing to retry" : string.Empty); + } + + return new MqttConnectRejectedException(connack.ResultCode, unrecoverable, message); + } + + /// + /// Whether a refused CONNACK can never be fixed by retrying, and so must fault the connection + /// rather than leave it looping. + /// + /// + /// + /// Unrecoverable covers credentials/identity (BadUserNameOrPassword, + /// NotAuthorized, ClientIdentifierNotValid, Banned, + /// BadAuthenticationMethod), protocol mismatch (UnsupportedProtocolVersion, + /// MalformedPacket, ProtocolError, PacketTooLarge) and a CONNECT whose + /// Last-Will the broker will not accept (TopicNameInvalid, + /// PayloadFormatInvalid, RetainNotSupported, QoSNotSupported). Every + /// one is a property of the configuration we keep re-sending. + /// + /// + /// ServerMoved (0x9D) is unrecoverable and UseAnotherServer (0x9C) is not, + /// because MQTT 5 defines the first as "permanently use another server" and the second as + /// "temporarily use another server" — the spec, not a guess. + /// + /// + /// Everything else — including UnspecifiedError, ImplementationSpecificError + /// and any code a future broker invents — is treated as transient. That default is + /// deliberate: wrongly faulting a transient refusal takes a recoverable driver down until + /// someone redeploys, whereas wrongly retrying a permanent one costs a bounded, visibly + /// Reconnecting, backoff-paced attempt that names the reason code in the log. + /// + /// + internal static bool IsUnrecoverableConnackRejection(MqttClientConnectResultCode code) => code switch + { + MqttClientConnectResultCode.MalformedPacket + or MqttClientConnectResultCode.ProtocolError + or MqttClientConnectResultCode.UnsupportedProtocolVersion + or MqttClientConnectResultCode.ClientIdentifierNotValid + or MqttClientConnectResultCode.BadUserNameOrPassword + or MqttClientConnectResultCode.NotAuthorized + or MqttClientConnectResultCode.Banned + or MqttClientConnectResultCode.BadAuthenticationMethod + or MqttClientConnectResultCode.TopicNameInvalid + or MqttClientConnectResultCode.PacketTooLarge + or MqttClientConnectResultCode.PayloadFormatInvalid + or MqttClientConnectResultCode.RetainNotSupported + or MqttClientConnectResultCode.QoSNotSupported + or MqttClientConnectResultCode.ServerMoved => true, + _ => false, + }; + + /// + /// Renders a refused CONNACK as a targeted, credential-free operator message. Shared with + /// MqttDriverProbe so the AdminUI "Test connect" button and the running driver describe + /// the same rejection in the same words. + /// + internal static string DescribeConnackRejection(string target, MqttClientConnectResultCode code) => code switch + { + MqttClientConnectResultCode.NotAuthorized or MqttClientConnectResultCode.BadUserNameOrPassword + => $"Broker at {target} rejected the credentials ({code}).", + MqttClientConnectResultCode.UnsupportedProtocolVersion + => $"Broker at {target} does not support the configured protocol version ({code}).", + MqttClientConnectResultCode.Banned + => $"Broker at {target} has banned this client ({code}).", + _ => $"Broker at {target} refused CONNECT: {code}.", + }; + + /// + /// Starts the supervisor on the first successful caller connect. Always called while + /// holding the lifecycle gate, so the ??= needs no further synchronisation. + /// + private void StartSupervisorIfCallerConnect(bool supervisorAttempt) + { + if (!supervisorAttempt) + { + _reconnectWorker ??= Task.Run(() => ReconnectSupervisorAsync(_lifetimeCts.Token)); + } + } + + /// + /// Maps a bounded-operation failure onto this type's documented contract. MQTTnet wraps a + /// cancelled connect in MqttConnectingFailedException rather than letting the + /// surface, so the legs are told apart by which + /// token fired, not by exception type. Precedence: teardown, then caller intent, then the + /// deadline. + /// + /// The failure to classify. + /// The caller's token — cancelled means caller intent. + /// The operation's own deadline source. + /// The operation name for the message ("connect", "subscribe"). + private Exception Classify( + Exception ex, + CancellationToken cancellationToken, + CancellationTokenSource deadline, + string operation = "connect") + { + if (Disposed) + { + return new ObjectDisposedException( + nameof(MqttConnection), + new InvalidOperationException( + $"MQTT driver '{_driverId}': {operation} to {_options.Host}:{_options.Port} was aborted because the " + + "connection was disposed while the attempt was in flight.", + ex)); + } + + if (cancellationToken.IsCancellationRequested) + { + return new OperationCanceledException( + $"MQTT driver '{_driverId}': {operation} to {_options.Host}:{_options.Port} was cancelled.", + ex, + cancellationToken); + } + + if (deadline.IsCancellationRequested) + { + return new TimeoutException( + $"MQTT driver '{_driverId}': {operation} to {_options.Host}:{_options.Port} did not complete within " + + $"{_options.ConnectTimeoutSeconds}s.", + ex); + } + + return ex; + } + + private void AttachHandlers(IMqttClient client) + { + client.DisconnectedAsync += OnDisconnectedAsync; + client.ApplicationMessageReceivedAsync += OnApplicationMessageReceivedAsync; + } + + /// + /// Runs on MQTTnet's own dispatcher, so it does exactly two non-blocking things and returns. + /// Sleeping the backoff here would stall the client's internal pump. + /// + private Task OnDisconnectedAsync(MqttClientDisconnectedEventArgs args) + { + if (Disposed || State is MqttConnectionState.Faulted or MqttConnectionState.Disposed) + { + return Task.CompletedTask; + } + + if (State == MqttConnectionState.Connected) + { + SetState(MqttConnectionState.Reconnecting); + } + + // Released unconditionally rather than only when the client had been connected: a missed + // wake is a permanently dark connection, whereas a spurious one costs a loop iteration. + _reconnectWake.Release(); + return Task.CompletedTask; + } + + private Task OnApplicationMessageReceivedAsync(MqttApplicationMessageReceivedEventArgs args) + { + Interlocked.Exchange(ref _lastMessageTicksUtc, DateTime.UtcNow.Ticks); + + var observers = MessageReceived; + if (observers is null) + { + return Task.CompletedTask; + } + + var message = args.ApplicationMessage; + var payload = message.Payload; + + // Same idiom the browse session uses: the common single-segment case is served straight off + // the library's buffer; a fragmented sequence is flattened once. Either way the span is only + // valid for this call, which is exactly what MqttMessageObserver documents. + ReadOnlySpan body = payload.IsSingleSegment ? payload.FirstSpan : payload.ToArray(); + + try + { + observers(message.Topic, body, message.Retain); + } + catch (Exception ex) + { + // An exception escaping here surfaces inside MQTTnet's own pump. Contain it: a broken + // observer must degrade its own tags, never stop delivery for every other subscription. + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': an inbound-message observer threw; the message was dropped.", + _driverId); + } + + return Task.CompletedTask; + } + + /// + /// Issues one SUBSCRIBE carrying every requested filter, bounded by + /// so a broker that accepts SUBSCRIBE and + /// never answers SUBACK — the frozen-peer shape again — fails at the deadline instead of + /// parking the caller (and, on the reconnect path, the supervisor) forever. + /// + /// + /// The deadline is this method's own linked source, deliberately not + /// : that knob already governs every MQTTnet operation + /// and repurposing it would couple the subscribe budget to the connect budget in a way neither + /// side could change independently. + /// + /// A rejected filter is not an exception: the returned outcomes carry the broker's + /// per-filter verdict so the caller can degrade exactly the affected references. Only a + /// failure of the SUBSCRIBE itself (transport error, deadline, teardown) throws. + /// + /// + /// The filters to establish. + /// Caller cancellation; linked with the subscribe deadline. + /// One outcome per requested filter, in request order. + /// The subscribe deadline elapsed. + /// was cancelled. + /// The connection was disposed. + /// There is no established session to subscribe on. + public async Task> SubscribeAsync( + IReadOnlyList filters, + CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(filters); + ObjectDisposedException.ThrowIf(Disposed, this); + + if (filters.Count == 0) + { + return []; + } + + var client = _client + ?? throw new InvalidOperationException( + $"MQTT driver '{_driverId}': cannot subscribe before a session to {_options.Host}:{_options.Port} " + + "has been established."); + + var builder = new MqttClientSubscribeOptionsBuilder(); + foreach (var filter in filters) + { + builder = builder.WithTopicFilter(f => f + .WithTopic(filter.Topic) + .WithQualityOfServiceLevel(MapQos(filter.Qos)) + // Only meaningful on MQTT 5.0. On 3.1.1 the broker always replays its retained + // message, so the manager also drops unwanted seeds client-side off the retain flag. + .WithRetainHandling(filter.SeedRetained + ? MqttRetainHandling.SendAtSubscribe + : MqttRetainHandling.DoNotSendOnSubscribe)); + } + + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)); + using var linked = CancellationTokenSource.CreateLinkedTokenSource( + cancellationToken, + deadline.Token, + _lifetimeCts.Token); + + MqttClientSubscribeResult result; + try + { + result = await client.SubscribeAsync(builder.Build(), linked.Token).ConfigureAwait(false); + } + catch (Exception ex) + { + throw Classify(ex, cancellationToken, deadline, operation: "subscribe"); + } + + // Pair each requested filter with its SUBACK item positionally: MQTT guarantees SUBACK reason + // codes arrive in the order of the SUBSCRIBE's filters. A short/absent SUBACK (a + // specification-violating broker) is reported as ungranted rather than silently assumed good. + var items = result.Items as IList ?? [.. result.Items]; + var outcomes = new List(filters.Count); + for (var i = 0; i < filters.Count; i++) + { + if (i >= items.Count) + { + outcomes.Add(new MqttSubscribeOutcome(filters[i].Topic, Granted: false, "NoSubAckReasonCode")); + continue; + } + + var code = items[i].ResultCode; + var granted = code is MqttClientSubscribeResultCode.GrantedQoS0 + or MqttClientSubscribeResultCode.GrantedQoS1 + or MqttClientSubscribeResultCode.GrantedQoS2; + outcomes.Add(new MqttSubscribeOutcome(filters[i].Topic, granted, code.ToString())); + } + + return outcomes; + } + + /// + /// Publishes one application message, bounded by + /// so a broker that accepts PUBLISH and + /// never completes it cannot park the caller — the same frozen-peer rule every other wait here + /// follows. + /// + /// + /// + /// A refused PUBLISH throws. MQTTnet reports a broker rejection as a + /// result (as it does for CONNACK — see ), + /// so a caller that treated "did not throw" as "published" would silently drop a rebirth + /// NCMD against a broker whose ACL denies the NCMD topic and then wait forever for the + /// birth it never asked for. + /// + /// + /// Publishing on a down session is refused up front rather than left to MQTTnet's + /// own exception, so the message names the driver and the broker. A rebirth request lost to + /// a reconnect is not a fault: the reconnect path re-requests one itself. + /// + /// + /// The concrete topic to publish on. + /// The message body. + /// Requested QoS, 0–2; out-of-range values clamp. + /// The MQTT retain flag. + /// Caller cancellation; linked with the publish deadline. + /// A task that completes when the broker has accepted the message. + /// The publish deadline elapsed. + /// was cancelled. + /// The connection was disposed. + /// There is no established session, or the broker refused the message. + public async Task PublishAsync( + string topic, + byte[] payload, + int qos, + bool retain, + CancellationToken cancellationToken) + { + ArgumentException.ThrowIfNullOrEmpty(topic); + ArgumentNullException.ThrowIfNull(payload); + ObjectDisposedException.ThrowIf(Disposed, this); + + var client = _client; + if (client is null || !client.IsConnected) + { + throw new InvalidOperationException( + $"MQTT driver '{_driverId}': cannot publish '{topic}' — there is no established session to " + + $"{_options.Host}:{_options.Port}."); + } + + var message = new MqttApplicationMessageBuilder() + .WithTopic(topic) + .WithPayload(payload) + .WithQualityOfServiceLevel(MapQos(qos)) + .WithRetainFlag(retain) + .Build(); + + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)); + using var linked = CancellationTokenSource.CreateLinkedTokenSource( + cancellationToken, + deadline.Token, + _lifetimeCts.Token); + + MqttClientPublishResult result; + try + { + result = await client.PublishAsync(message, linked.Token).ConfigureAwait(false); + } + catch (Exception ex) + { + throw Classify(ex, cancellationToken, deadline, operation: "publish"); + } + + if (result is not null && !result.IsSuccess) + { + throw new InvalidOperationException( + $"MQTT driver '{_driverId}': broker at {_options.Host}:{_options.Port} refused PUBLISH " + + $"'{topic}': {result.ReasonCode}."); + } + } + + /// Maps a configured QoS integer onto MQTTnet's enum, clamping an out-of-range value. + private static MqttQualityOfServiceLevel MapQos(int qos) => qos switch + { + <= 0 => MqttQualityOfServiceLevel.AtMostOnce, + 1 => MqttQualityOfServiceLevel.AtLeastOnce, + _ => MqttQualityOfServiceLevel.ExactlyOnce, + }; + + /// + /// The reconnect loop MQTTnet v5 no longer provides. Started on the first successful connect + /// and stopped by cancelling the lifetime token; it owns every + /// reconnect attempt so the library's dispatcher never waits on one. + /// + private async Task ReconnectSupervisorAsync(CancellationToken cancellationToken) + { + try + { + while (!cancellationToken.IsCancellationRequested) + { + await _reconnectWake.WaitAsync(cancellationToken).ConfigureAwait(false); + + if (Disposed || State == MqttConnectionState.Faulted) + { + return; + } + + if (IsConnected) + { + continue; // Spurious wake (e.g. a failed attempt's own disconnect callback). + } + + SetState(MqttConnectionState.Reconnecting); + + for (var attempt = 0; !cancellationToken.IsCancellationRequested && !Disposed; attempt++) + { + // A caller's ConnectAsync may have restored the session while we slept. Bail + // before burning the backoff, not just before burning an attempt. + if (IsConnected) + { + AdoptSessionRestoredByCaller(); + break; + } + + await Task + .Delay( + NextBackoff( + attempt, + _options.ReconnectMinBackoffSeconds, + _options.ReconnectMaxBackoffSeconds), + cancellationToken) + .ConfigureAwait(false); + + bool established; + try + { + established = await ConnectCoreAsync(supervisorAttempt: true, cancellationToken) + .ConfigureAwait(false); + } + catch (ObjectDisposedException) when (Disposed) + { + return; + } + catch (Exception ex) + { + if (State == MqttConnectionState.Faulted) + { + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': reconnect abandoned — the broker configuration is unusable.", + _driverId); + return; + } + + _logger?.LogDebug( + ex, + "MQTT driver '{DriverId}': reconnect attempt {Attempt} to {Host}:{Port} failed.", + _driverId, + attempt + 1, + _options.Host, + _options.Port); + continue; + } + + if (!established) + { + // Someone else restored the session between the backoff and the connect. + // Firing Reconnected here would re-subscribe on top of a caller that is about + // to do exactly that itself, and — worse — leaving the loop running would + // keep failing against a healthy connection forever. + AdoptSessionRestoredByCaller(); + break; + } + + if (!await TryReSubscribeAsync(cancellationToken).ConfigureAwait(false)) + { + continue; + } + + // Only now is "Connected" true in the sense this type sells it: connected AND + // subscribed. + SetState(MqttConnectionState.Connected); + _logger?.LogInformation( + "MQTT driver '{DriverId}': reconnected to {Host}:{Port} after {Attempts} attempt(s); " + + "subscriptions re-established.", + _driverId, + _options.Host, + _options.Port, + attempt + 1); + break; + } + } + } + catch (OperationCanceledException) + { + // Dispose cancelled the lifetime token — the expected way this loop ends. + } + catch (Exception ex) + { + // Reaching here means the connection is permanently dark, which is exactly the failure + // this supervisor exists to prevent — it must never be swallowed quietly, and State must + // not keep reporting Reconnecting, which every consumer reads as "recovering". + SetState(MqttConnectionState.Faulted); + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': reconnect supervisor stopped unexpectedly; the broker session will not " + + "recover without a driver restart.", + _driverId); + } + } + + /// + /// The session came back without this supervisor's help — a caller's + /// won the race while we were in backoff. Publish the truth and stand down; deliberately does + /// not fire , because that caller owns its own re-subscribe. + /// + private void AdoptSessionRestoredByCaller() + { + SetState(MqttConnectionState.Connected); + _logger?.LogDebug( + "MQTT driver '{DriverId}': reconnect stood down — the session to {Host}:{Port} was restored elsewhere.", + _driverId, + _options.Host, + _options.Port); + } + + /// + /// Fires — ALWAYS, on every reconnect, including persistent + /// sessions where it is merely redundant, because subscriptions do not survive a clean + /// session and a reconnect that skips this leaves a healthy-looking, permanently deaf client. + /// + /// + /// false if the re-subscribe failed or overran its deadline, having torn the session + /// down so the next attempt starts from a clean CONNECT — serving a connection that receives + /// nothing is the one outcome this type must never produce. + /// + private async Task TryReSubscribeAsync(CancellationToken cancellationToken) + { + // Bounded like every other wait here. A subscriber that flows the token cancels promptly; one + // that ignores it is abandoned at the ceiling rather than being allowed to park the + // supervisor past DisposeAsync. + var fanOut = FireReconnectedAsync(cancellationToken); + + // The ceiling abandons the task rather than stopping it, so its eventual failure must still + // be observed or it resurfaces as an unobserved TaskException on the finalizer thread. + _ = fanOut.ContinueWith( + static abandoned => _ = abandoned.Exception, + CancellationToken.None, + TaskContinuationOptions.OnlyOnFaulted | TaskContinuationOptions.ExecuteSynchronously, + TaskScheduler.Default); + + try + { + await fanOut + .WaitAsync(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds), cancellationToken) + .ConfigureAwait(false); + return true; + } + catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested) + { + throw; // Teardown — let the supervisor's own cancellation handling end the loop. + } + catch (Exception ex) + { + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': re-subscribe after reconnect failed or overran {TimeoutSeconds}s; tearing " + + "the session down and retrying rather than serving a connection that receives nothing.", + _driverId, + _options.ConnectTimeoutSeconds); + SetState(MqttConnectionState.Reconnecting); + await ForceDisconnectAsync().ConfigureAwait(false); + return false; + } + } + + /// + /// Drops a session that is connected but unusable, so the next supervisor pass starts from a + /// clean CONNECT. Best-effort and bounded — failure here just means the next attempt sees a + /// client that is already down. + /// + private async Task ForceDisconnectAsync() + { + var client = _client; + if (client is null) + { + return; + } + + try + { + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)); + await client.DisconnectAsync(new MqttClientDisconnectOptions(), deadline.Token).ConfigureAwait(false); + } + catch (Exception ex) + { + _logger?.LogDebug(ex, "MQTT driver '{DriverId}': forced disconnect failed.", _driverId); + } + } + + private async Task CloseAsync(IMqttClient client) + { + try + { + if (client.IsConnected) + { + // Bounded even on teardown — a wedged broker must not stall driver shutdown. + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)); + await client.DisconnectAsync(new MqttClientDisconnectOptions(), deadline.Token).ConfigureAwait(false); + } + } + catch (Exception ex) + { + _logger?.LogDebug(ex, "MQTT driver '{DriverId}': disconnect during dispose failed; disposing anyway.", _driverId); + } + finally + { + client.Dispose(); + } + } + + /// + /// is terminal — nothing may move off it. A + /// check-then-write would let a thread preempted between the two steps resurrect a disposed + /// connection's state, so the transition is a compare-and-swap: the terminal check and the + /// write are the same atomic operation. + /// + private void SetState(MqttConnectionState state) + { + var desired = (int)state; + while (true) + { + var current = Volatile.Read(ref _state); + if (current == (int)MqttConnectionState.Disposed || current == desired) + { + return; + } + + if (Interlocked.CompareExchange(ref _state, desired, current) == current) + { + return; + } + } + } + + /// + /// Assembles the MQTTnet client options from . Pure — it performs + /// no I/O and touches no network, so it is safe to call on a config-validation path. A + /// configured is read lazily by the + /// returned certificate-validation callback at handshake time, not here. + /// + /// Broker connection settings. + /// + /// Appended to , so a browse/probe session can share + /// the configured identity without colliding with the driver's own client id. When both are + /// empty MQTTnet generates a random client id. + /// + /// Optional logger for the deferred CA-pin load; never receives credentials. + public static MqttClientOptions BuildClientOptions( + MqttDriverOptions options, + string? clientIdSuffix, + ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(options); + + var builder = new MqttClientOptionsBuilder() + .WithTcpServer(options.Host, options.Port) + .WithProtocolVersion(MapProtocolVersion(options.ProtocolVersion)) + .WithCleanSession(options.CleanSession) + .WithKeepAlivePeriod(TimeSpan.FromSeconds(options.KeepAliveSeconds)) + .WithTimeout(TimeSpan.FromSeconds(options.ConnectTimeoutSeconds)); + + var clientId = string.Concat(options.ClientId ?? string.Empty, clientIdSuffix ?? string.Empty); + if (!string.IsNullOrWhiteSpace(clientId)) + { + builder = builder.WithClientId(clientId); + } + + // An empty username means "connect anonymously"; MQTTnet would otherwise send an empty + // CONNECT username, which some brokers treat as a failed auth rather than as anonymous. + if (!string.IsNullOrEmpty(options.Username)) + { + builder = builder.WithCredentials(options.Username, options.Password ?? string.Empty); + } + + builder = builder.WithTlsOptions(tls => ConfigureTls(tls, options, logger)); + + return builder.Build(); + } + + private static void ConfigureTls(MqttClientTlsOptionsBuilder tls, MqttDriverOptions options, ILogger? logger) + { + if (!options.UseTls) + { + tls.UseTls(false); + return; + } + + tls.UseTls(true) + // Explicit SNI / hostname-verification target. Left unset, name validation depends on + // how MQTTnet infers the host from the endpoint — pin it to the configured host. + .WithTargetHost(options.Host); + + if (options.AllowUntrustedServerCertificate) + { + // Dev / on-prem escape hatch, off unless explicitly enabled — mirrors the + // ServerHistorian TLS knobs. This is the ONLY branch that accepts a bad certificate. + tls.WithAllowUntrustedCertificates(true) + .WithIgnoreCertificateChainErrors(true) + .WithCertificateValidationHandler(static _ => true); + return; + } + + if (string.IsNullOrWhiteSpace(options.CaCertificatePath)) + { + // No pin: MQTTnet's default handler validates against the OS trust store. + return; + } + + tls.WithCertificateValidationHandler(CreatePinnedCaValidator(options.CaCertificatePath, logger)); + } + + /// + /// Builds a certificate-validation callback that pins the broker chain to the PEM CA at + /// . The file is loaded lazily on first validation + /// (i.e. during the TLS handshake) so that this — and therefore + /// — stays free of I/O. Any failure to load or to chain + /// up to the pinned CA rejects the certificate: fail closed. + /// + private static Func CreatePinnedCaValidator( + string caCertificatePath, + ILogger? logger) + { + var trustedRoots = new Lazy( + () => LoadPemCa(caCertificatePath, logger), + LazyThreadSafetyMode.ExecutionAndPublication); + + return args => ValidateAgainstPinnedCa(args, trustedRoots.Value, caCertificatePath, logger); + } + + private static X509Certificate2Collection? LoadPemCa(string caCertificatePath, ILogger? logger) + { + try + { + var collection = new X509Certificate2Collection(); + collection.ImportFromPemFile(caCertificatePath); + if (collection.Count != 0) + { + return collection; + } + + logger?.LogError("MQTT CA pin '{CaCertificatePath}' contains no certificates; broker TLS will be rejected.", caCertificatePath); + } + catch (Exception ex) + { + logger?.LogError(ex, "MQTT CA pin '{CaCertificatePath}' could not be loaded; broker TLS will be rejected.", caCertificatePath); + } + + return null; + } + + private static bool ValidateAgainstPinnedCa( + MqttClientCertificateValidationEventArgs args, + X509Certificate2Collection? trustedRoots, + string caCertificatePath, + ILogger? logger) + { + if (trustedRoots is null || trustedRoots.Count == 0) + { + // Unreadable pin ⇒ no trust anchor ⇒ refuse. Never silently degrade to the OS store. + return false; + } + + // The pin replaces the trust anchor only. Anything else the platform flagged — a hostname + // mismatch, an absent certificate — remains fatal. + if ((args.SslPolicyErrors & ~SslPolicyErrors.RemoteCertificateChainErrors) != SslPolicyErrors.None) + { + logger?.LogError("MQTT broker certificate rejected: {SslPolicyErrors}.", args.SslPolicyErrors); + return false; + } + + if (args.Certificate is null) + { + return false; + } + + var presented = args.Certificate as X509Certificate2; + X509Certificate2? owned = null; + if (presented is null) + { + owned = X509CertificateLoader.LoadCertificate(args.Certificate.Export(X509ContentType.Cert)); + presented = owned; + } + + try + { + using var chain = new X509Chain(); + chain.ChainPolicy.TrustMode = X509ChainTrustMode.CustomRootTrust; + chain.ChainPolicy.CustomTrustStore.AddRange(trustedRoots); + + // Revocation is deliberately not checked: the pin exists precisely for self-issued + // dev / on-prem CAs, which typically publish no CRL or OCSP responder, so checking + // would fail every such chain. The pin itself is the revocation story — remove the CA. + chain.ChainPolicy.RevocationMode = X509RevocationMode.NoCheck; + + // Seed the intermediates the broker presented during the handshake. A leaf signed by + // an intermediate that chains up to the pinned root is a common broker topology, and + // that intermediate arrives on the wire rather than being installed locally — without + // it in the candidate pool a legitimate trust path simply cannot be built. + // CustomRootTrust still means only `trustedRoots` may terminate the chain, so a rogue + // self-signed cert smuggled in here cannot become an anchor. + // Both sources are read: platforms differ in whether the peer-supplied intermediates + // land in the incoming chain's elements, its ExtraStore, or both. + if (args.Chain is not null) + { + foreach (var element in args.Chain.ChainElements) + { + chain.ChainPolicy.ExtraStore.Add(element.Certificate); + } + + chain.ChainPolicy.ExtraStore.AddRange(args.Chain.ChainPolicy.ExtraStore); + } + + if (chain.Build(presented)) + { + return true; + } + + logger?.LogError( + "MQTT broker certificate does not chain to the pinned CA '{CaCertificatePath}': {ChainStatus}.", + caCertificatePath, + string.Join(", ", chain.ChainStatus.Select(s => s.Status))); + return false; + } + catch (CryptographicException ex) + { + // X509Chain.Build throws on a malformed certificate or an unusable policy. An exception + // escaping a TLS validation callback surfaces as an opaque handshake crash, so classify + // it here and refuse: an unverifiable certificate is a rejected certificate. + logger?.LogError( + ex, + "MQTT broker certificate could not be validated against the pinned CA '{CaCertificatePath}'; rejecting.", + caCertificatePath); + return false; + } + finally + { + owned?.Dispose(); + } + } + + private static MQTTnet.Formatter.MqttProtocolVersion MapProtocolVersion(MqttProtocolVersion version) + => version switch + { + MqttProtocolVersion.V311 => MQTTnet.Formatter.MqttProtocolVersion.V311, + MqttProtocolVersion.V500 => MQTTnet.Formatter.MqttProtocolVersion.V500, + _ => throw new ArgumentOutOfRangeException(nameof(version), version, "Unsupported MQTT protocol version."), + }; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriver.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriver.cs new file mode 100644 index 00000000..a6814efb --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriver.cs @@ -0,0 +1,1196 @@ +using System.Collections.Concurrent; +using System.Collections.Frozen; +using System.Text.Json; +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// The MQTT / Sparkplug B driver shell: composes (broker session + +/// reconnect supervisor), (authored table, topic routing, +/// value extraction) and the manager's (the push→poll bridge) into +/// the driver capability set the Core consumes. +/// +/// +/// +/// Authored-only discovery. Plain MQTT has no browsable address space — a broker will +/// happily carry thousands of topics this deployment has nothing to do with. +/// therefore replays exactly the raw tags the deploy artifact +/// authored () and nothing else, and +/// stays so the +/// universal browser never treats broker traffic as a tag catalogue. In +/// the policy is : +/// the authored set — and every authored tag's datatype — is fully known synchronously, so +/// re-running discovery on a timer can only produce the same tree. +/// +/// +/// Two ingest paths, one driver. routes through +/// (authored topics); +/// routes through (one group-wide filter, birth certificates, +/// alias resolution, seq gaps, rebirth NCMDs). Exactly one is live, selected by +/// and rebuilt whenever +/// changes anything either captures. They share the +/// driver-owned , so IReadable reads one store either way. +/// +/// +/// fires in Sparkplug B only, and only on a real change. +/// A Sparkplug tag's dataType is optional — the birth certificate declares it — so the +/// discovered tree fills in asynchronously and the policy is +/// . +/// is wired to through +/// , which fires only when the birth's metric-name SET +/// differs from the last one seen for that scope (and only for scopes an authored tag actually +/// binds). That gate is the anti-storm mechanism, not an optimisation: rediscovery triggers an +/// address-space rebuild, edge nodes re-birth freely — on their own timer, on every reconnect +/// via the late-join rebirth, and once per NCMD the ingestor's gap policy sends — and firing on +/// each of those would turn a healthy plant into a permanent rebuild loop. Plain mode never +/// raises it at all; its authored set changes only by redeploy. +/// +/// +/// Composition order is load-bearing. +/// must run before any subscribe and before the connect completes — it is what wires +/// message delivery and, critically, the reconnect re-subscribe. The connection's +/// Reconnected handler is passed through unwrapped: the manager throws when it can +/// re-establish nothing, and that throw is precisely what tears the session down and retries. +/// Swallowing it would leave a driver that reports Connected and receives nothing. +/// +/// +/// Constructor is connection-free. It maps the authored TagConfig blobs (pure, no I/O) +/// so discovery and reads work off a configured-but-not-yet-connected driver; every network +/// operation happens in . +/// +/// +public sealed class MqttDriver + : IDriver, ITagDiscovery, ISubscribable, IReadable, IHostConnectivityProbe, IRediscoverable, IAsyncDisposable, IDisposable +{ + /// + /// Rough per-tag driver-attributable cost: the parsed (five + /// strings — RawPath, topic, JSONPath and the record header) plus its cached + /// and the two dictionary entries that index it. An estimate + /// by construction — GetMemoryFootprint exists to drive a cache-budget decision, not + /// to be exact. + /// + private const long ApproxBytesPerAuthoredTag = 512; + + /// + /// The single folder streams every variable under — and therefore + /// the only subtree path a rediscovery ScopeHint can name truthfully. See + /// . + /// + private const string DiscoveryFolderName = "Mqtt"; + + /// How often the host-connectivity poll samples . + private static readonly TimeSpan HostProbeInterval = TimeSpan.FromSeconds(1); + + private readonly string _driverInstanceId; + private readonly ILogger? _logger; + + /// + /// Owned by the driver, not by the manager, so a manager rebuilt for changed ingest settings + /// inherits the observed values instead of silently regressing every node to + /// BadWaitingForInitialData. + /// + private readonly LastValueCache _values = new(); + + private readonly object _hostLock = new(); + + /// + /// Per authored Sparkplug scope, the signature of the metric-name set its most recent birth + /// declared — the rediscovery change gate's memory. Written on MQTTnet's dispatcher thread and + /// read nowhere else, but concurrent because a reconnect's birth flood and the dispatcher are + /// not guaranteed to be the same thread forever. + /// + /// + /// Deliberately NOT cleared on a death, a reconnect or an ingest rebuild. The ingestor + /// drops its whole cache on every reconnect (an alias may + /// have been rebound while the driver was away) — but "the driver forgot" is not "the address + /// space changed". Clearing this alongside it would make every reconnect's late-join rebirth + /// flood look like a tree full of brand-new scopes and fire one rediscovery per authored scope + /// on every single flap, which is the storm this map exists to prevent. It survives so that an + /// identical rebirth stays silent. + /// + private readonly ConcurrentDictionary _birthSignatures = new(); + + /// Guards / / against each other. + private readonly SemaphoreSlim _lifecycleGate = new(1, 1); + + // _options / _subscriptions / _authoredRawPaths are written only under _lifecycleGate (or in the + // ctor) and read freely elsewhere, WITHOUT the Volatile discipline _health / _hostState get. + // That is deliberate, not an oversight: all three are reference/interface fields, so a reader + // sees either the whole old object or the whole new one — never a torn one. The looser fields + // are the ones a reader may legitimately observe one publish stale (a read served from the + // previous authored table is a correct read of a value that was correct a microsecond ago); + // _health and _hostState feed ServiceLevel and the status dashboard, where a stale per-core copy + // is an operator-visible lie about whether the driver is up. + private MqttDriverOptions _options; + private MqttSubscriptionManager _subscriptions; + + /// + /// The Sparkplug B ingest path, or in . + /// Exactly one of this and is fed authored tags and attached to the + /// connection; the other exists but is never registered against, so it can neither route a + /// message nor answer a resolve. + /// + private SparkplugIngestor? _sparkplug; + + /// + /// RawPaths of the currently registered authored tags, in authoring order — the discovery + /// enumeration set. The manager resolves a RawPath to its definition but does not enumerate, + /// and enumerating from here is what makes "authored only" structural rather than incidental. + /// + private IReadOnlyList _authoredRawPaths = []; + + /// + /// The Sparkplug scopes at least one authored tag binds — empty outside + /// . Rebuilt wholesale by + /// alongside the authored table it is derived from. + /// + /// + /// The rediscovery gate's first question. The ingestor applies a birth for any device + /// under an authored edge node (it has to — a DBIRTH is a sequenced member of that node's + /// stream), so on a large plant it observes births for devices this deployment never reads. Such + /// a birth cannot change 's output by construction, because discovery + /// replays the authored set — so firing rediscovery for it would rebuild the address space in + /// response to traffic the configuration has no opinion about. Gating on this set also bounds + /// by configuration rather than by whatever the plant + /// happens to publish — the same rule the ingestor's own authored-edge-node filter follows. + /// + private FrozenSet _authoredScopes = FrozenSet.Empty; + + private MqttConnection? _connection; + private CancellationTokenSource? _hostProbeCts; + + private DriverHealth _health = new(DriverState.Unknown, null, null); + private HostState _hostState = HostState.Unknown; + private DateTime _hostStateChangedUtc = DateTime.UtcNow; + private int _disposed; + + /// Initializes a new driver. Stores + maps configuration only — no network, no client. + /// + /// Driver configuration, including the authored . + /// Task 9's factory deserializes it from the DriverConfig JSON; a later + /// may replace it from a fresh config blob. + /// + /// Stable logical id of this driver instance. + /// Optional logger; never receives credentials or payload bodies. + public MqttDriver(MqttDriverOptions options, string driverInstanceId, ILogger? logger = null) + { + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); + + _options = options; + _driverInstanceId = driverInstanceId; + _logger = logger; + _subscriptions = CreateSubscriptionManager(options); + _sparkplug = CreateSparkplugIngestor(options); + RegisterAuthoredTags(options); + } + + /// + public event EventHandler? OnDataChange; + + /// + public event EventHandler? OnHostStatusChanged; + + /// + public event EventHandler? OnRediscoveryNeeded; + + /// + public string DriverInstanceId => _driverInstanceId; + + /// + public string DriverType => DriverTypeNames.Mqtt; + + /// + /// The ingest path this driver composes. Internal: the P2 Sparkplug handler feeds the same + /// OnDataChange + sinks through it, and the shell tests + /// drive to simulate broker traffic + /// without a broker. + /// + internal MqttSubscriptionManager Subscriptions => _subscriptions; + + /// + /// The Sparkplug B ingest path, or outside + /// . Internal: the shell tests drive + /// to simulate an edge node without a broker, and + /// Tasks 22/23 read its . + /// + internal SparkplugIngestor? Sparkplug => _sparkplug; + + /// Whether this driver instance ingests Sparkplug B rather than plain topics. + private bool IsSparkplug => _options.Mode == MqttMode.SparkplugB; + + /// + /// Test seam: awaited inside while the lifecycle gate is + /// held, immediately before the broker connect. Lets a test park a lifecycle operation + /// mid-flight and prove serializes behind it rather than racing + /// past a half-built session. Mirrors 's own + /// AfterConnectHookForTests, which exists for the same class of race. Always + /// in production. + /// + internal Func? BeforeConnectHookForTests { get; set; } + + // ---- IDriver: lifecycle ---- + + /// + /// Adopts (falling back to the constructor's options when + /// it is blank or unusable), registers the authored tags, and opens the broker session. + /// + /// + /// Ordering is the contract: register → attach → connect. Attaching before the connect + /// completes means the reconnect re-subscribe is wired for the very first drop, and the + /// manager's SUBSCRIBE transport is live before the OPC UA server's first subscribe arrives. + /// + /// The driver configuration as JSON. + /// Cancellation for the connect. + /// A task that represents the asynchronous operation. + public async Task InitializeAsync(string driverConfigJson, CancellationToken cancellationToken) + { + await _lifecycleGate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + await InitializeCoreAsync(driverConfigJson, cancellationToken).ConfigureAwait(false); + } + finally + { + _lifecycleGate.Release(); + } + } + + /// + /// Applies a config delta in place. A tag-only delta re-registers the authored table without + /// touching the broker session; a delta that changes how the driver connects or ingests + /// rebuilds the session. + /// + /// + /// A bad delta never faults the driver. Unparseable or unusable config is logged and + /// discarded, leaving the running configuration exactly as it was — a redeploy carrying one + /// malformed driver blob must not take a healthy driver's whole address space Bad. A genuine + /// connect failure against a changed endpoint is a different thing and does fault, + /// exactly as would. + /// + /// The driver configuration as JSON. + /// Cancellation for any reconnect the delta forces. + /// A task that represents the asynchronous operation. + public async Task ReinitializeAsync(string driverConfigJson, CancellationToken cancellationToken) + { + MqttDriverOptions? next; + try + { + next = ParseOptions(driverConfigJson); + } + catch (Exception ex) + { + // Defence in depth: ParseOptions already swallows the JSON failure modes. + _logger?.LogWarning( + ex, + "MQTT driver '{DriverId}': reinitialize config could not be read; keeping the running configuration.", + _driverInstanceId); + return; + } + + if (next is null) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': reinitialize config was blank or unusable; keeping the running configuration.", + _driverInstanceId); + return; + } + + await _lifecycleGate.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + if (!SameSession(next, _options)) + { + // Broker endpoint / credentials / ingest shape changed — the live session cannot be + // reused. Full rebuild; a failure here is a real fault, not a bad delta. + _logger?.LogInformation( + "MQTT driver '{DriverId}': reinitialize changes the broker session; rebuilding.", + _driverInstanceId); + await TeardownAsync().ConfigureAwait(false); + await InitializeCoreAsync(driverConfigJson, cancellationToken).ConfigureAwait(false); + return; + } + + _options = next; + RegisterAuthoredTags(next); + _logger?.LogInformation( + "MQTT driver '{DriverId}': reinitialize applied in place; {TagCount} authored tag(s).", + _driverInstanceId, + _authoredRawPaths.Count); + } + finally + { + _lifecycleGate.Release(); + } + } + + /// + public async Task ShutdownAsync(CancellationToken cancellationToken) + { + var lastMessage = ReadHealth().LastSuccessfulRead; + if (await _lifecycleGate.WaitAsync(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds), cancellationToken) + .ConfigureAwait(false)) + { + try + { + await TeardownAsync().ConfigureAwait(false); + } + finally + { + _lifecycleGate.Release(); + } + } + else + { + // A wedged initialize must not turn shutdown into a hang; tear the session down anyway. + _logger?.LogWarning( + "MQTT driver '{DriverId}': lifecycle gate still held at shutdown; tearing the session down anyway.", + _driverInstanceId); + await TeardownAsync().ConfigureAwait(false); + } + + WriteHealth(new DriverHealth(DriverState.Unknown, lastMessage, null)); + TransitionHost(HostState.Unknown); + } + + /// + /// Current health. Derived live from the connection's own state where one exists, so a + /// background reconnect is visible without the driver having to mirror every transition. + /// LastSuccessfulRead carries the last inbound message instant — MQTT never polls, so + /// "last message age" is the only evidence the session is actually delivering. + /// + /// The driver's current health snapshot. + public DriverHealth GetHealth() + { + var stored = ReadHealth(); + var connection = _connection; + if (connection is null) + { + return stored; + } + + var lastMessage = connection.LastMessageUtc ?? stored.LastSuccessfulRead; + return connection.State switch + { + MqttConnectionState.Connected => new DriverHealth(DriverState.Healthy, lastMessage, stored.LastError), + MqttConnectionState.Reconnecting => new DriverHealth(DriverState.Reconnecting, lastMessage, stored.LastError), + MqttConnectionState.Faulted => new DriverHealth(DriverState.Faulted, lastMessage, stored.LastError), + // Disconnected before the first successful connect, or Disposed after teardown: the + // stored snapshot (Initializing / Faulted / Unknown) is the more informative answer. + _ => stored with { LastSuccessfulRead = lastMessage }, + }; + } + + /// + /// Approximate driver-attributable footprint: the authored definition table and the + /// last-value cache that indexes it. Both are sized by the authored tag count, which is the + /// only thing a deployment can grow. + /// + /// The approximate memory footprint in bytes. + public long GetMemoryFootprint() => _authoredRawPaths.Count * ApproxBytesPerAuthoredTag; + + /// + /// No-op, deliberately. This driver holds no optional cache: the authored table is the + /// address space itself, and the last-value cache is — MQTT is + /// subscribe-first, so a dropped value is not re-fetchable, and flushing it would turn every + /// node Bad until its topic next published, which for a slow-changing tag can be never. + /// Birth/alias state (Sparkplug, P2) is correctness state for the same reason. + /// + /// Unused; present for the shape. + /// A completed task. + public Task FlushOptionalCachesAsync(CancellationToken cancellationToken) => Task.CompletedTask; + + // ---- ITagDiscovery ---- + + /// + /// + /// + /// Plain mode discovers synchronously from the authored set — the RawPaths and their + /// datatypes are both in the deploy artifact — so one pass is all there is. + /// + /// + /// Sparkplug B is because half of what + /// it discovers arrives on the wire: dataType is optional in the Sparkplug tag shape + /// (the birth certificate declares it), so a tag authored with nothing but its + /// (group, node, device, metric) tuple can only be streamed with the record's default + /// type until its NBIRTH/DBIRTH lands. The host's UntilStable contract — re-run + /// discovery until the captured set is non-empty and its signature repeats — is exactly the + /// retry that lets those passes catch a birth that had not yet arrived at connect. + /// + /// + /// The retry is a floor, not the mechanism. The host's stability signature is built + /// from FullReferences alone, so a pass whose only change is a filled-in datatype still + /// looks "stable" and the loop stops. That is why the same fill-in also fires + /// (see ): a birth arriving + /// after the loop settled is the case the timer cannot cover. + /// + /// + public DiscoveryRediscoverPolicy RediscoverPolicy => + IsSparkplug ? DiscoveryRediscoverPolicy.UntilStable : DiscoveryRediscoverPolicy.Once; + + /// + /// + /// Always : replays authored tags, it does + /// not enumerate a backend. A broker's live topic set is not a tag catalogue — the + /// Driver.Mqtt.Browser project is the surface for observing traffic during authoring. + /// + public bool SupportsOnlineDiscovery => false; + + /// + /// Streams the authored tag set — and only that — into . + /// + /// + /// + /// The enumeration source is the driver's own list of registered RawPaths, never anything + /// observed on the wire, so no amount of broker traffic can add a node. A RawPath whose + /// TagConfig failed to map is absent from the manager and is skipped here too — it surfaces + /// as BadNodeIdUnknown on read rather than as a wrongly-typed variable. + /// + /// + /// Sparkplug B: the datatype is resolved per pass, the tag set never is. An authored + /// tag is streamed on every pass, birth or no birth — it is part of the declared + /// configuration, not something the plant grants by publishing, and hiding it until a birth + /// arrives would make the node's existence depend on wire traffic (and, on a deployment + /// whose other tags do carry an authored type, would hide it behind a "non-empty and stable" + /// verdict the host reaches without it). What a birth changes is the type: see + /// . + /// + /// + /// The address space builder to stream discovered nodes into. + /// Cancellation for the discovery operation. + /// A task that represents the asynchronous operation. + public Task DiscoverAsync(IAddressSpaceBuilder builder, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(builder); + cancellationToken.ThrowIfCancellationRequested(); + + var folder = builder.Folder(DiscoveryFolderName, DiscoveryFolderName); + foreach (var rawPath in _authoredRawPaths) + { + if (!TryResolve(rawPath, out var def)) + { + continue; + } + + folder.Variable(def.Name, def.Name, new DriverAttributeInfo( + FullName: def.Name, + DriverDataType: ResolveDiscoveredDataType(def), + IsArray: false, + ArrayDim: null, + // MQTT ingest is one-way in P1 — this driver implements no IWritable leg, so every + // node is ViewOnly rather than an Operate node whose writes would silently vanish. + SecurityClass: SecurityClassification.ViewOnly, + IsHistorized: false, + IsAlarm: false, + WriteIdempotent: false)); + } + + return Task.CompletedTask; + } + + /// + /// The datatype reports for one authored tag: the authored type when + /// the operator declared one, otherwise the type the metric's live birth certificate declared, + /// otherwise the record's default. + /// + /// + /// + /// The precedence is the ingest path's, restated. SparkplugIngestor.PublishMetric + /// coerces every value with DataTypeAuthored ? authored : birth; a discovery surface + /// that disagreed with it would advertise a type no value ever arrives as. The two must be + /// changed together. + /// + /// + /// An unsupported Sparkplug type (DataSet / Template / PropertySet / Unknown) maps to + /// and falls back rather than blanking the tag — the ingest path + /// already refuses those metrics once, loudly, and a discovery pass is not the place to + /// relitigate it. + /// + /// + /// The authored tag definition. + /// The effective data type for this pass. + private DriverDataType ResolveDiscoveredDataType(MqttTagDefinition def) + { + if (def.DataTypeAuthored || _sparkplug is not { } sparkplug) + { + return def.DataType; + } + + if (def.GroupId is not { } groupId || def.EdgeNodeId is not { } edgeNodeId || def.MetricName is not { } metric) + { + return def.DataType; + } + + var table = sparkplug.Births.Find(new SparkplugScope(groupId, edgeNodeId, def.DeviceId)); + return table?.ResolveByName(metric)?.DataType.ToDriverDataType() ?? def.DataType; + } + + // ---- ISubscribable (straight through to the manager) ---- + + /// + public Task SubscribeAsync( + IReadOnlyList fullReferences, + TimeSpan publishingInterval, + CancellationToken cancellationToken) + => _sparkplug is { } sparkplug + ? sparkplug.SubscribeAsync(fullReferences, publishingInterval, cancellationToken) + : _subscriptions.SubscribeAsync(fullReferences, publishingInterval, cancellationToken); + + /// + public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken) + => _sparkplug is { } sparkplug + ? sparkplug.UnsubscribeAsync(handle, cancellationToken) + : _subscriptions.UnsubscribeAsync(handle, cancellationToken); + + /// Resolves a RawPath through whichever ingest path this driver's mode selected. + /// The driver wire reference. + /// The definition when this returns . + /// when the reference is an authored tag of the live ingest path. + private bool TryResolve(string rawPath, out MqttTagDefinition def) + => _sparkplug is { } sparkplug + ? sparkplug.TryResolve(rawPath, out def) + : _subscriptions.TryResolve(rawPath, out def); + + // ---- IReadable ---- + + /// + /// Serves the batch from the last-value cache. MQTT is push-based: there is nothing to poll, + /// so a read is always "what was most recently published". + /// + /// + /// Never throws. A reference that has not been observed — including one that is not an + /// authored tag at all — returns BadWaitingForInitialData in its own slot. Failing the + /// whole call would let one stale reference blank every other node in the same OPC UA read. + /// + /// The RawPath references to read. + /// Unused; the read touches no network. + /// One snapshot per requested reference, in request order. + public Task> ReadAsync( + IReadOnlyList fullReferences, + CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(fullReferences); + + var results = new DataValueSnapshot[fullReferences.Count]; + for (var i = 0; i < fullReferences.Count; i++) + { + // The driver-owned cache, not the live ingest path's: both are handed this same instance, + // and reading it directly means a mid-flight ingest rebuild can never blank a read. + results[i] = _values.Read(fullReferences[i]); + } + + return Task.FromResult>(results); + } + + // ---- IHostConnectivityProbe ---- + + /// The single broker this driver instance talks to, as host:port. + public string HostName => $"{_options.Host}:{_options.Port}"; + + /// + public IReadOnlyList GetHostStatuses() + { + lock (_hostLock) + { + return [new HostConnectivityStatus(HostName, _hostState, _hostStateChangedUtc)]; + } + } + + // ---- IRediscoverable ---- + + /// + /// Raises . Nothing in calls it + /// — the authored set only changes by redeploy. It exists for the Sparkplug B path, where a + /// DBIRTH can introduce metrics the previous birth certificate did not carry. + /// + /// Driver-supplied reason for the diagnostic log. + /// Optional subtree hint; means the whole tree. + internal void RaiseRediscoveryNeeded(string reason, string? scopeHint = null) + => OnRediscoveryNeeded?.Invoke(this, new RediscoveryEventArgs(reason, scopeHint)); + + /// + /// The rediscovery decision: a birth changed what can produce only + /// when it is for an authored scope and its metric-name set differs from the last + /// one seen for that scope. Anything else is dropped. + /// + /// + /// + /// Both conditions are load-bearing, and both are anti-storm. Rediscovery triggers an + /// address-space rebuild — expensive and disruptive on a running server. An edge node + /// re-births freely: on its own schedule, on every reconnect (the ingestor's late-join + /// rebirth fan-out), and once per rebirth NCMD its gap policy sends. Firing on each of those + /// is a permanent rebuild loop against a plant that is behaving perfectly. Firing on + /// changed metric set only is the primary defence; the authored-scope filter is the + /// second, and removes the whole class of churn sourced from devices this deployment does + /// not read (see ). + /// + /// + /// The signature is order-insensitive — ordered-distinct, ordinal. Nothing obliges an + /// edge node to keep its birth's metric ordering stable, and a reordered birth describes the + /// same address space; comparing the raw sequence would leak the storm back in. + /// + /// + /// No additional debounce or coalescing. One was considered and rejected: the change + /// gate already collapses the unbounded sources (rebirth timers, reconnect floods, NCMD + /// answers) to zero, and what survives it is a genuine metric-set change — of which a given + /// scope has a handful in its lifetime, each one a real edit to the served tree. A timer on + /// top would only delay a rebuild that must happen, and would need its own trailing-edge + /// flush to avoid dropping the last change entirely. If a consumer ever needs coalescing + /// across many scopes birthing at once (a whole-plant restart), that belongs in the consumer, + /// which alone knows what a rebuild costs it — today there is no such consumer at all. + /// + /// + /// Runs on MQTTnet's dispatcher thread (via the ingestor's own contained raise), so it + /// does no I/O and never throws — catches a throwing + /// subscriber, but relying on that would still have cost the rest of the birth's fan-out. + /// + /// + private void OnBirthObserved(object? sender, SparkplugBirthObservedEventArgs e) + { + if (!_authoredScopes.Contains(e.Scope)) + { + // A birth for a scope no authored tag binds cannot change the authored-only discovery tree. + return; + } + + var signature = BirthSignature(e.MetricNames); + if (_birthSignatures.TryGetValue(e.Scope, out var previous) + && string.Equals(previous, signature, StringComparison.Ordinal)) + { + return; // Same metric set as last time — nothing to rediscover. + } + + _birthSignatures[e.Scope] = signature; + + var reason = previous is null + ? $"Sparkplug birth observed for '{e.Scope}' ({e.MetricNames.Count} metric(s))" + : $"Sparkplug rebirth changed the metric set for '{e.Scope}' ({e.MetricNames.Count} metric(s))"; + + _logger?.LogInformation( + "MQTT driver '{DriverId}': {Reason}; requesting rediscovery.", + _driverInstanceId, + reason); + + // ScopeHint names the folder DiscoverAsync actually streams under — deliberately NOT the + // Sparkplug scope path. The discovered tree is flat (one 'Mqtt' folder of RawPath-named + // variables); it has no edge-node/device folders, so a hint of 'Plant1/EdgeA/Filler1' would + // name a subtree that does not exist and a consumer scoping a rebuild on it would rebuild + // nothing. The scope belongs in the Reason, which is the documented diagnostic field. + RaiseRediscoveryNeeded(reason, DiscoveryFolderName); + } + + /// An order-insensitive, ordinal signature of a birth's metric-name set. + /// The names the birth declared. + /// The comparable signature. + private static string BirthSignature(IReadOnlyList metricNames) + => string.Join( + '\u0001', // A separator no Sparkplug metric name can contain, so two sets cannot alias. + metricNames.Distinct(StringComparer.Ordinal).Order(StringComparer.Ordinal)); + + // ---- disposal ---- + + /// + public void Dispose() => DisposeAsync().AsTask().GetAwaiter().GetResult(); + + /// + /// Performs the same teardown as , under the same lifecycle + /// gate, so a caller that uses await using without an explicit shutdown does not + /// leak a live broker session. + /// + /// + /// + /// The gate wait is the whole point, not ceremony. An ungated dispose that ran + /// while 's rebuild branch held the gate would observe + /// _connection == null — the rebuild has torn the old session down but not yet + /// assigned the new one — do nothing, and set _disposed. The rebuild would then + /// complete, assign a live, connected connection and start a host-probe loop that + /// no later dispose can ever reach, because the disposed guard short-circuits them all. + /// That is the orphaned-connection class 's own remarks + /// describe having already fixed once. Waiting for the gate makes this dispose tear down + /// whatever the in-flight operation ends up assigning. + /// + /// + /// _disposed is set before the wait, so an in-flight + /// sees it at its post-connect re-check and refuses to + /// publish the connection at all — which is what closes the residual window on the + /// bounded-timeout fallback path below. + /// + /// + /// Separately (and for an unrelated reason): the gate object itself is deliberately never + /// Dispose()d — same call as 's semaphores. A caller + /// that disposes before its last would otherwise get an + /// naming , which says + /// nothing useful; the gate holds no timer and no surviving registration, so leaving it + /// undisposed costs nothing. + /// + /// + /// A task that represents the asynchronous dispose. + public async ValueTask DisposeAsync() + { + if (Interlocked.Exchange(ref _disposed, 1) != 0) + { + return; + } + + if (await _lifecycleGate.WaitAsync(TimeSpan.FromSeconds(_options.ConnectTimeoutSeconds)) + .ConfigureAwait(false)) + { + try + { + await TeardownAsync().ConfigureAwait(false); + } + finally + { + _lifecycleGate.Release(); + } + + return; + } + + // Bounded even here: a wedged initialize must not turn dispose into a hang. The post-connect + // disposed re-check in InitializeCoreAsync is what stops that operation from publishing a + // connection behind this teardown's back. + _logger?.LogWarning( + "MQTT driver '{DriverId}': lifecycle gate still held at dispose; tearing the session down anyway.", + _driverInstanceId); + await TeardownAsync().ConfigureAwait(false); + } + + // ---- internals ---- + + /// The gate-held body of . + private async Task InitializeCoreAsync(string driverConfigJson, CancellationToken cancellationToken) + { + WriteHealth(new DriverHealth(DriverState.Initializing, ReadHealth().LastSuccessfulRead, null)); + try + { + // A blank / unusable blob keeps the constructor's options rather than degrading the + // driver to defaults pointing at localhost. + var parsed = ParseOptions(driverConfigJson); + if (parsed is not null) + { + AdoptOptions(parsed); + } + + RegisterAuthoredTags(_options); + + var connection = new MqttConnection(_options, _driverInstanceId, _logger); + + // MUST precede the connect: this wires message delivery AND the reconnect re-subscribe. + // The handler is passed through unwrapped on purpose — its throw-on-total-failure is what + // tears a deaf session down and retries it. + if (_sparkplug is { } attaching) + { + attaching.AttachTo(connection); + } + else + { + _subscriptions.AttachTo(connection); + } + + if (BeforeConnectHookForTests is { } hook) + { + await hook(cancellationToken).ConfigureAwait(false); + } + + await connection.ConnectAsync(cancellationToken).ConfigureAwait(false); + + // Re-check disposal AFTER the connect, before publishing the connection. DisposeAsync + // normally waits for this gate, but its wait is bounded — on the wedged-gate fallback + // path it can run concurrently with this method, and assigning a live connection behind + // a completed teardown is exactly the orphaned-session bug. Losing the race means + // disposing what we just built, never leaking it. + if (Volatile.Read(ref _disposed) != 0) + { + await connection.DisposeAsync().ConfigureAwait(false); + throw new ObjectDisposedException(nameof(MqttDriver)); + } + + // Sparkplug subscribes ONCE, here, rather than on the OPC UA server's first subscribe: the + // driver must see birth certificates whether or not anything is subscribed yet, because a + // birth is where every alias and datatype comes from. This also issues the late-join + // rebirth (§3.6 #5) — the initial connect does not fire Reconnected. + // + // It runs BEFORE `_connection` is published, and disposes the session it was given if it + // fails. Publishing first would leave a live, supervised connection behind a throw: the + // driver-host resilience layer re-runs this method, the retry overwrites `_connection`, and + // the first session becomes an orphan no later dispose can reach — the exact leak + // MqttConnection's own remarks describe having already fixed once. + if (_sparkplug is { } sparkplug) + { + try + { + await sparkplug.EstablishAsync(cancellationToken).ConfigureAwait(false); + } + catch + { + await connection.DisposeAsync().ConfigureAwait(false); + throw; + } + } + + _connection = connection; + + WriteHealth(new DriverHealth(DriverState.Healthy, connection.LastMessageUtc, null)); + TransitionHost(HostState.Running); + StartHostProbe(); + } + catch (Exception ex) + { + WriteHealth(new DriverHealth(DriverState.Faulted, ReadHealth().LastSuccessfulRead, ex.Message)); + TransitionHost(HostState.Faulted); + throw; + } + } + + /// + /// Deserializes a driver-config blob. Returns — never throws — when the + /// blob is blank or unusable, so both the initialize fallback and the reinitialize + /// keep-running-config rule read off one answer. + /// + private MqttDriverOptions? ParseOptions(string driverConfigJson) + { + if (string.IsNullOrWhiteSpace(driverConfigJson)) + { + return null; + } + + try + { + // MqttJson.Options — the ONE shared instance (factory / probe / this / browser), + // deliberately: enums must round-trip by NAME. A second, divergent + // JsonSerializerOptions is this repo's documented systemic enum bug, where an + // AdminUI-authored config with a string enum field faults the driver. + return JsonSerializer.Deserialize(driverConfigJson, MqttJson.Options); + } + catch (JsonException ex) + { + _logger?.LogWarning( + ex, + "MQTT driver '{DriverId}': driver config JSON is invalid.", + _driverInstanceId); + return null; + } + catch (NotSupportedException ex) + { + _logger?.LogWarning( + ex, + "MQTT driver '{DriverId}': driver config JSON could not be bound to the options shape.", + _driverInstanceId); + return null; + } + } + + /// + /// Adopts new options, rebuilding the subscription manager only when a setting it captures at + /// construction actually changed. The rebuilt manager inherits the driver-owned + /// , so adopting new options never blanks observed values. + /// + private void AdoptOptions(MqttDriverOptions next) + { + if (!SameIngest(next, _options)) + { + _subscriptions = CreateSubscriptionManager(next); + _sparkplug = CreateSparkplugIngestor(next); + } + + _options = next; + } + + /// Builds a manager over the driver-owned cache and bridges its notifications to . + private MqttSubscriptionManager CreateSubscriptionManager(MqttDriverOptions options) + { + var manager = new MqttSubscriptionManager( + options, + _driverInstanceId, + transport: null, + logger: _logger, + cache: _values, + maxPayloadBytes: options.MaxPayloadBytes); + + // The published reference is the RawPath the manager already stamped on the args; the driver + // only re-raises with itself as sender. + manager.OnDataChange += (_, args) => OnDataChange?.Invoke(this, args); + return manager; + } + + /// + /// Builds the Sparkplug B ingest path for , or + /// outside . Shares the driver-owned + /// for the same reason the plain manager does: a rebuild for + /// changed settings must not blank every observed value. + /// + private SparkplugIngestor? CreateSparkplugIngestor(MqttDriverOptions options) + { + if (options.Mode != MqttMode.SparkplugB) + { + return null; + } + + if (options.Sparkplug is { ActAsPrimaryHost: true }) + { + // Loud, because the flag reads as if it does something. Being a Sparkplug Primary Host is a + // three-part contract — a retained ONLINE after CONNECT, a matching retained OFFLINE + // registered as the MQTT Last Will AT CONNECT TIME, and an explicit OFFLINE on clean + // shutdown — and edge nodes stop publishing while their host is offline. Shipping the + // ONLINE half without the Will is strictly worse than shipping neither: a crashed node + // would leave a retained ONLINE asserting forever that a dead host is alive, which is the + // exact state the mechanism exists to prevent. The Will belongs in + // MqttConnection.BuildClientOptions; until it is there this flag stays inert and said so. + _logger?.LogWarning( + "MQTT driver '{DriverId}': Sparkplug actAsPrimaryHost is set but primary-host STATE " + + "publishing is not implemented — this driver observes STATE and never publishes one. " + + "Edge nodes gated on this host id will not see it come online.", + _driverInstanceId); + } + + var ingestor = new SparkplugIngestor( + options, + _driverInstanceId, + subscribeTransport: null, + publishTransport: null, + logger: _logger, + cache: _values, + maxPayloadBytes: options.MaxPayloadBytes); + + // The published reference is the RawPath the ingestor already stamped on the args; the driver + // only re-raises with itself as sender. + ingestor.OnDataChange += (_, args) => OnDataChange?.Invoke(this, args); + + // The ingestor detects a birth; the driver decides whether it changed the address space. Wired + // here rather than in InitializeAsync so a driver rebuilt for a config delta re-arms it — an + // ingest rebuild that silently dropped this subscription would leave a Sparkplug driver whose + // datatypes fill in and whose rediscovery never fires again, with nothing to show for it. + ingestor.BirthObserved += OnBirthObserved; + return ingestor; + } + + /// Registers the authored raw tags wholesale and refreshes the discovery enumeration set. + private void RegisterAuthoredTags(MqttDriverOptions options) + { + // Wholesale into the LIVE path only. Feeding Sparkplug tags to the plain parser would log a + // "did not map" warning for every one of them (they carry no topic) and vice versa. + var mapped = _sparkplug is { } sparkplug + ? sparkplug.Register(options.RawTags) + : _subscriptions.Register(options.RawTags); + + // Enumerate in authoring order; the manager owns which of these actually mapped, and + // DiscoverAsync skips the rest. + _authoredRawPaths = [.. options.RawTags.Select(t => t.RawPath)]; + _authoredScopes = BuildAuthoredScopes(); + + // Drop change-gate memory for scopes this deploy stopped authoring — the same housekeeping + // SparkplugIngestor.Register does for its trackers, and for the same reason: without it a scope + // nobody reads keeps a slot across every future redeploy. Re-authoring such a scope later means + // its next birth reads as new and fires once, which is correct — a newly authored scope IS a + // change to the discovered tree. + foreach (var scope in _birthSignatures.Keys) + { + if (!_authoredScopes.Contains(scope)) + { + _birthSignatures.TryRemove(scope, out _); + } + } + + if (mapped != options.RawTags.Count) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': {Mapped} of {Authored} authored tag(s) mapped to a definition; " + + "the rest will report BadNodeIdUnknown.", + _driverInstanceId, + mapped, + options.RawTags.Count); + } + } + + /// + /// The Sparkplug scopes the currently registered authored tags bind, derived from the same + /// definitions enumerates — empty outside + /// , and empty for any tag whose TagConfig did not map. + /// + /// The authored scope set. + private FrozenSet BuildAuthoredScopes() + { + if (_sparkplug is null) + { + return FrozenSet.Empty; + } + + var scopes = new HashSet(); + foreach (var rawPath in _authoredRawPaths) + { + if (TryResolve(rawPath, out var def) + && def.GroupId is { } groupId + && def.EdgeNodeId is { } edgeNodeId) + { + scopes.Add(new SparkplugScope(groupId, edgeNodeId, def.DeviceId)); + } + } + + return scopes.ToFrozenSet(); + } + + /// + /// Whether two option sets describe the same broker session. Note the explicit + /// : the identity projections are object-typed + /// anonymous instances, so != would compare references and report "changed" + /// every single time — which would tear down and rebuild a perfectly good session on every + /// redeploy, and (via ) swap in a subscription manager that is not + /// attached to the live connection. + /// + private static bool SameSession(MqttDriverOptions a, MqttDriverOptions b) + => SessionIdentity(a).Equals(SessionIdentity(b)); + + /// + /// Whether two option sets produce an identical ingest path — + /// in Plain mode, in Sparkplug B. + /// + /// + /// ⚠️ Every setting an ingest path captures at construction MUST be named by + /// . A setting that is captured but unnamed makes a delta + /// changing it compare "same ingest", get applied in place, and silently never reach the + /// component that needed rebuilding — the driver would keep subscribing to the OLD Sparkplug + /// group id and report perfect health. Task 21 closed the Sparkplug half of that gap by naming + /// the whole record (it has value equality, so a member + /// added there is covered automatically — which is the point of naming the record rather than + /// enumerating its fields). + /// + private static bool SameIngest(MqttDriverOptions a, MqttDriverOptions b) + => IngestIdentity(a).Equals(IngestIdentity(b)); + + /// + /// The settings whose change invalidates the live broker session. Projected onto an anonymous + /// type (structural Equals) rather than comparing the whole options record, whose + /// RawTags list would make every comparison unequal by reference. + /// + private static object SessionIdentity(MqttDriverOptions o) => new + { + o.Host, + o.Port, + o.ClientId, + o.UseTls, + o.AllowUntrustedServerCertificate, + o.CaCertificatePath, + o.Username, + o.Password, + o.ProtocolVersion, + o.CleanSession, + o.KeepAliveSeconds, + o.ConnectTimeoutSeconds, + o.ReconnectMinBackoffSeconds, + o.ReconnectMaxBackoffSeconds, + Ingest = IngestIdentity(o), + }; + + /// The settings the live ingest path captures at construction. + /// + /// is named as a whole record rather than field-by-field: it + /// has structural value equality, so a member added to it in a later task is covered here with + /// no edit — and the failure mode of forgetting one is silent (see ). + /// + private static object IngestIdentity(MqttDriverOptions o) => new + { + o.Mode, + DefaultQos = o.Plain?.DefaultQos, + o.MaxPayloadBytes, + o.Sparkplug, + }; + + /// Shared teardown for , and the session rebuild. + private async Task TeardownAsync() + { + var probe = Interlocked.Exchange(ref _hostProbeCts, null); + if (probe is not null) + { + await probe.CancelAsync().ConfigureAwait(false); + probe.Dispose(); + } + + var connection = Interlocked.Exchange(ref _connection, null); + if (connection is not null) + { + await connection.DisposeAsync().ConfigureAwait(false); + } + } + + /// + /// Starts the host-connectivity poll. exposes state but no + /// state-changed event, and an whose event never fires is + /// a capability that looks wired and is inert — so the transition signal is sampled here. + /// + private void StartHostProbe() + { + var cts = new CancellationTokenSource(); + var previous = Interlocked.Exchange(ref _hostProbeCts, cts); + if (previous is not null) + { + // Cancel before disposing: disposing a live source leaves its loop running against a + // token that will never be cancelled, and a second probe would fight the first. + previous.Cancel(); + previous.Dispose(); + } + + _ = Task.Run(() => HostProbeLoopAsync(cts.Token), cts.Token); + } + + private async Task HostProbeLoopAsync(CancellationToken ct) + { + try + { + while (!ct.IsCancellationRequested) + { + await Task.Delay(HostProbeInterval, ct).ConfigureAwait(false); + + var connection = _connection; + if (connection is null) + { + continue; + } + + TransitionHost(connection.State switch + { + MqttConnectionState.Connected => HostState.Running, + MqttConnectionState.Reconnecting or MqttConnectionState.Disconnected => HostState.Stopped, + MqttConnectionState.Faulted => HostState.Faulted, + _ => HostState.Unknown, + }); + } + } + catch (OperationCanceledException) + { + // Shutdown. + } + catch (Exception ex) + { + _logger?.LogWarning(ex, "MQTT driver '{DriverId}': host connectivity probe stopped.", _driverInstanceId); + } + } + + /// Publishes a host state transition once; repeats of the current state are ignored. + private void TransitionHost(HostState next) + { + HostState previous; + lock (_hostLock) + { + if (_hostState == next) + { + return; + } + + previous = _hostState; + _hostState = next; + _hostStateChangedUtc = DateTime.UtcNow; + } + + OnHostStatusChanged?.Invoke(this, new HostStatusChangedEventArgs(HostName, previous, next)); + } + + /// Barrier-protected read of the multi-threaded health field. + private DriverHealth ReadHealth() => Volatile.Read(ref _health); + + /// Barrier-protected publish of a new health snapshot. + private void WriteHealth(DriverHealth value) => Volatile.Write(ref _health, value); +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverFactoryExtensions.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverFactoryExtensions.cs new file mode 100644 index 00000000..dc6d9bea --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverFactoryExtensions.cs @@ -0,0 +1,84 @@ +using System.Text.Json; +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Core.Hosting; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// Registers the MQTT / Sparkplug B driver with the . The +/// Host's DriverFactoryBootstrap calls once at startup; the +/// driver-instance bootstrapper then materialises DriverInstance rows of type +/// into live instances. +/// +/// +/// +/// Direct deserialization, no intermediate DTO. The DriverConfig blob binds +/// straight onto — the same shape +/// , MqttDriverBrowser and +/// already bind, and the shape the options record +/// was designed for (every knob carries its own default, and rawTags is a plain +/// list needing no translation). Mirrors +/// OpcUaClientDriverFactoryExtensions. ModbusDriverFactoryExtensions' separate +/// DTO exists to service a legacy nullable-everything blob plus string→enum parsing this +/// driver does with a converter instead; copying it here would add a fourth parse +/// shape for one config, which is precisely the divergence +/// exists to prevent. +/// +/// +/// Connection-free. parses and constructs only — the +/// constructor touches no network, and the registry contract +/// forbids the factory from calling InitializeAsync itself (the driver host owns the +/// retry semantics). +/// +/// +public static class MqttDriverFactoryExtensions +{ + /// + /// Driver type name — matches DriverInstance.DriverType values. Sourced from + /// so the registration key can never drift from the + /// constant the dispatch maps, the probe and the browser reference. + /// + public const string DriverTypeName = DriverTypeNames.Mqtt; + + /// + /// Register the MQTT factory with the driver registry. The optional + /// is captured at registration time and used to construct an + /// per driver instance — without it the driver runs with no + /// logger (standalone/test callers stay unchanged). + /// + /// The driver factory registry to register with. + /// Optional logger factory used to create per-instance loggers. + public static void Register(DriverFactoryRegistry registry, ILoggerFactory? loggerFactory = null) + { + ArgumentNullException.ThrowIfNull(registry); + registry.Register(DriverTypeName, (id, json) => CreateInstance(id, json, loggerFactory)); + } + + /// Public for the Server-side bootstrapper + test consumers. + /// Stable logical id of the driver instance. + /// The DriverConfig JSON blob for the instance. + /// Optional logger factory for the per-instance logger. + /// A configured, not-yet-connected . + /// + /// or is blank. + /// + /// The config blob deserialised to null. + /// The config blob is not valid JSON for the options shape. + public static MqttDriver CreateInstance( + string driverInstanceId, + string driverConfigJson, + ILoggerFactory? loggerFactory = null) + { + ArgumentException.ThrowIfNullOrWhiteSpace(driverInstanceId); + ArgumentException.ThrowIfNullOrWhiteSpace(driverConfigJson); + + // MqttJson.Options — the one shared instance (see its remarks). A local copy here is the + // repo's documented systemic enum bug in the making. + var options = JsonSerializer.Deserialize(driverConfigJson, MqttJson.Options) + ?? throw new InvalidOperationException( + $"MQTT driver config for '{driverInstanceId}' deserialised to null"); + + return new MqttDriver(options, driverInstanceId, loggerFactory?.CreateLogger()); + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverProbe.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverProbe.cs new file mode 100644 index 00000000..d14886a4 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttDriverProbe.cs @@ -0,0 +1,226 @@ +using System.Diagnostics; +using System.Net.Sockets; +using System.Security.Authentication; +using System.Text.Json; +using Microsoft.Extensions.Logging.Abstractions; +using MQTTnet; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// CONNECT-handshake probe for the -shaped driver config. +/// Opens a bounded MQTT CONNECT against the configured broker; a CONNACK-accepted response +/// (MqttClientConnectResultCode.Success) is the "device is answering" proof (green + +/// latency). Refused/TLS/auth/timeout failures each surface a targeted message. Mirrors +/// ModbusDriverProbe's shape. +/// +/// +/// +/// Reuses for the connect (TLS, +/// CA-pin, credentials, protocol-version mapping) rather than hand-rolling a second +/// connect path — a duplicate would be a security divergence. A distinct client-id suffix +/// (see ) keeps a probe from colliding with the driver's +/// own session: an MQTT broker disconnects an existing client when a new one CONNECTs +/// with the same client id, so a probe that reused the driver's id would knock the +/// running driver offline every time an operator clicked "Test connect". +/// +/// +/// A broker-rejected CONNACK is not a thrown exception. Live-probed against +/// MQTTnet 5.2.0.1603: IMqttClient.ConnectAsync returns a +/// MqttClientConnectResult whose ResultCode carries the broker's CONNACK +/// reason (e.g. NotAuthorized, BadUserNameOrPassword) — it does +/// not throw for a rejected CONNACK. Only transport-level failures throw, and +/// their shape varies by where the failure occurs: a closed/refused port throws +/// MqttCommunicationException wrapping a ; an +/// untrusted broker certificate throws MqttCommunicationException wrapping an +/// ; a broker that accepts the TCP handshake and +/// then never answers CONNACK throws either MqttConnectingFailedException +/// (wrapping "Connection closed") or a bare +/// ("MQTT connect canceled"), inconsistently, depending on whether MQTTnet's own +/// internal Timeout or the deadline token wins the race. Consequently this probe — +/// like — never switches on exception type to +/// detect a timeout; it checks the deadline token itself, which is authoritative +/// regardless of which exception shape MQTTnet happened to throw. +/// +/// +public sealed class MqttDriverProbe : IDriverProbe +{ + /// + /// Marks the transient probe identity in the broker's client-id/session logs — mirrors the + /// browser's own -browse-{guid8} suffix so the two transient sessions are + /// distinguishable in broker-side logs, and so a probe can never collide with (and knock + /// offline) the driver's own live session. + /// + internal const string ProbeClientIdPrefix = "-probe-"; + + /// + public string DriverType => DriverTypeNames.Mqtt; + + /// + public async Task ProbeAsync(string configJson, TimeSpan timeout, CancellationToken ct) + { + MqttDriverOptions? options; + try + { + // MqttJson.Options — the one shared instance across factory / probe / driver / browser + // (see its remarks): enums must round-trip by NAME or an AdminUI-authored config that + // Test-connect accepts would fault the deployed driver. + options = JsonSerializer.Deserialize(configJson, MqttJson.Options); + } + catch (Exception ex) + { + return new DriverProbeResult(false, $"Config JSON is invalid: {ex.Message}", null); + } + + if (options is null) + { + return new DriverProbeResult(false, "Config JSON deserialized to null.", null); + } + + if (string.IsNullOrWhiteSpace(options.Host) || options.Port is <= 0 or > 65535) + { + return new DriverProbeResult(false, "Config has no host/port to probe.", null); + } + + // Honour the config's own connectTimeoutSeconds (factory parity, mirrors + // ModbusDriverProbe) — the caller's timeout is the fallback when the config omits it. + var effectiveSeconds = options.ConnectTimeoutSeconds > 0 + ? options.ConnectTimeoutSeconds + : (int)Math.Ceiling(timeout.TotalSeconds); + if (effectiveSeconds <= 0) + { + effectiveSeconds = 1; + } + + // Rebuild with the resolved timeout so BuildClientOptions' own WithTimeout(...) and this + // method's deadline agree — both must expire at the same instant for the deadline check + // below to be a reliable signal regardless of which of the two actually threw. + var probeOptions = options with { ConnectTimeoutSeconds = effectiveSeconds }; + + MqttClientOptions clientOptions; + try + { + clientOptions = MqttConnection.BuildClientOptions( + probeOptions, + ProbeClientIdPrefix + Guid.NewGuid().ToString("N")[..8], + NullLogger.Instance); + } + catch (Exception ex) + { + // Never let a configured Password reach the message — BuildClientOptions itself never + // logs it, and no exception path through it can format credential values. + return new DriverProbeResult(false, $"Config could not be used to build a connection: {ex.Message}", null); + } + + var target = $"{probeOptions.Host}:{probeOptions.Port}"; + var sw = Stopwatch.StartNew(); + + using var deadline = new CancellationTokenSource(TimeSpan.FromSeconds(effectiveSeconds)); + using var linked = CancellationTokenSource.CreateLinkedTokenSource(ct, deadline.Token); + + using var client = new MqttClientFactory().CreateMqttClient(); + + MqttClientConnectResult result; + try + { + result = await client.ConnectAsync(clientOptions, linked.Token).ConfigureAwait(false); + } + catch (Exception ex) + { + return new DriverProbeResult(false, Classify(target, ex, ct, deadline, effectiveSeconds), null); + } + + try + { + if (result.ResultCode != MqttClientConnectResultCode.Success) + { + return new DriverProbeResult(false, ClassifyRejectedConnack(target, result.ResultCode), null); + } + + sw.Stop(); + return new DriverProbeResult(true, "MQTT CONNECT OK", sw.Elapsed); + } + finally + { + await DisconnectBestEffortAsync(client, effectiveSeconds).ConfigureAwait(false); + } + } + + /// + /// Maps a broker-rejected CONNACK ( not + /// Success — never an exception, see the type remarks) to a targeted, credential-free + /// message. + /// + /// + /// Delegates to so the "Test connect" + /// button and the running driver report an identical rejection in identical words — they now + /// both classify CONNACK, and disagreeing about it was the exact confusion the Task-13 live + /// gate surfaced. + /// + private static string ClassifyRejectedConnack(string target, MqttClientConnectResultCode code) + => MqttConnection.DescribeConnackRejection(target, code); + + /// + /// Classifies a thrown connect failure. Precedence mirrors + /// : caller cancellation, then the deadline, then the + /// exception's own shape — the deadline check is authoritative regardless of which exception + /// type MQTTnet happened to throw (see the type remarks for why exception type alone is not + /// reliable here). + /// + private static string Classify( + string target, + Exception ex, + CancellationToken callerToken, + CancellationTokenSource deadline, + int effectiveSeconds) + { + if (callerToken.IsCancellationRequested) + { + return "Probe was cancelled."; + } + + if (deadline.IsCancellationRequested) + { + return $"Probe timed out after {effectiveSeconds}s."; + } + + // Walk to the root cause — MQTTnet wraps transport/TLS failures one or two levels deep. + var root = ex; + while (root.InnerException is not null) + { + root = root.InnerException; + } + + return root switch + { + SocketException se => $"Connect to {target} failed: {se.SocketErrorCode}", + AuthenticationException => $"TLS handshake with {target} failed: {root.Message}", + _ => $"Connect to {target} failed: {root.Message}", + }; + } + + /// + /// Best-effort teardown of a session this probe established. Failure here is not the + /// probe's business — the CONNECT outcome already decided Ok/failure — and it must never be + /// allowed to hang past the connect budget. + /// + private static async Task DisconnectBestEffortAsync(IMqttClient client, int timeoutSeconds) + { + if (!client.IsConnected) + { + return; + } + + try + { + using var disconnectDeadline = new CancellationTokenSource(TimeSpan.FromSeconds(timeoutSeconds)); + await client.DisconnectAsync(new MqttClientDisconnectOptions(), disconnectDeadline.Token) + .ConfigureAwait(false); + } + catch + { + // Best-effort only. + } + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttSubscriptionManager.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttSubscriptionManager.cs new file mode 100644 index 00000000..8438f456 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/MqttSubscriptionManager.cs @@ -0,0 +1,1327 @@ +using System.Collections.Concurrent; +using System.Collections.Frozen; +using System.Globalization; +using System.Text; +using System.Text.Json; +using Microsoft.Extensions.Logging; +using MQTTnet; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +/// +/// The plain-MQTT ingest path: holds the driver's authored RawPath → +/// table, establishes one broker subscription per distinct authored +/// topic, and turns each inbound message into notifications plus a +/// update. +/// +/// +/// +/// The published reference is the RawPath. Every this +/// type raises carries — the tag's RawPath — as its +/// , never the topic and never the authored +/// TagConfig blob. DriverHostActor's dual-namespace fan-out is keyed by RawPath; +/// publishing under any other key produces a driver that passes every unit test and delivers +/// nothing at all in production. +/// +/// +/// A message fans out to EVERY matching tag. Several tags routinely share one topic +/// (one JSON document, one JSONPath per signal), so the topic index maps a topic to a +/// list of definitions and delivery walks all of them. Stopping at the first match is +/// the silent-starvation bug this shape exists to prevent, and it is pinned by a test. +/// +/// +/// Nothing here may throw on MQTTnet's dispatcher thread. +/// runs on it: extraction failures degrade the offending +/// tag to a Bad status code, a throwing downstream subscriber +/// is caught (and the tags behind it in the fan-out are still delivered), and no path does I/O +/// or takes a lock. +/// +/// +/// Unauthored topics are ignored. A wildcard-authored tag, or simply a chatty broker, +/// delivers messages this driver never asked for. They are dropped: MQTT ingest never +/// auto-provisions a tag the operator did not author. +/// +/// +/// Wildcards are matched for real. An authored topic carrying + / # is +/// accepted by the parser (warned at deploy, see MqttTagDefinitionFactory.Inspect), so +/// matching uses MQTTnet's own rather than string +/// equality — the same comparer a broker applies. Concrete topics stay a single dictionary +/// lookup; the linear wildcard scan runs only when a wildcard tag actually exists. +/// +/// +/// Sparkplug seam. P1 ingests plain topics only. Sparkplug B (Task 21) decodes a +/// protobuf payload and resolves metrics by the stable +/// (group, node, device, metricName) tuple rather than by topic; it belongs in its own +/// handler alongside this one, feeding the same + +/// sinks. Nothing Sparkplug-shaped is built here. +/// +/// +public sealed class MqttSubscriptionManager +{ + /// + /// Default — 1 MiB. Generous for MQTT telemetry (which is + /// kilobytes) while still bounding the dispatcher-thread work any one publisher can impose. + /// + public const int DefaultMaxPayloadBytes = 1024 * 1024; + + // OPC UA status codes (values verified against Opc.Ua.StatusCodes, not recalled). + private const uint StatusGood = 0x00000000u; + + /// The payload could not be decoded, or the JSONPath selected nothing. + private const uint StatusBadDecodingError = 0x80070000u; + + /// The payload decoded but does not fit the tag's declared dataType. + private const uint StatusBadTypeMismatch = 0x80740000u; + + /// The subscribed reference is not an authored MQTT tag. + private const uint StatusBadNodeIdUnknown = 0x80340000u; + + /// The broker refused, or never answered, the SUBSCRIBE for this reference's topic. + private const uint StatusBadCommunicationError = 0x80050000u; + + private readonly int _defaultQos; + private readonly string _driverId; + private readonly ILogger? _logger; + private readonly MqttMode _mode; + + /// RawPath → the handle of the subscription currently covering it. + private readonly ConcurrentDictionary _handleByRawPath = new(StringComparer.Ordinal); + + /// Handle → the references it was created for, so unsubscribe can undo exactly its own. + private readonly ConcurrentDictionary _refsByHandle = new(); + + /// Topics already SUBSCRIBEd on the live session — the dedupe set and the re-subscribe set. + private readonly ConcurrentDictionary _subscribedTopics = new(StringComparer.Ordinal); + + /// RawPaths whose extraction failure has already been logged loudly once. + private readonly ConcurrentDictionary _warnedRefs = new(StringComparer.Ordinal); + + private readonly EquipmentTagRefResolver _resolver; + + private long _handleSeq; + + /// The authored table + its topic index, swapped atomically by . + private volatile AuthoredTable _authored = AuthoredTable.Empty; + + private IMqttSubscribeTransport? _transport; + + /// Initializes a new instance of the class. + /// Driver options — supplies the ingest mode and the default QoS. + /// Driver instance id, used only for log correlation. + /// + /// The SUBSCRIBE seam. records subscriptions without dialling a broker + /// (the shape tests use, and the shape a not-yet-connected driver is in); + /// supplies the live connection. + /// + /// Optional logger; never receives credentials or payload bodies. + /// + /// The last-value store backing IReadable. Defaults to a fresh one owned by this + /// manager and exposed as . + /// + /// + /// Ceiling on an inbound message body; see . Non-positive values + /// fall back to — "unbounded" is not offered, because the + /// bound exists to protect a shared dispatcher thread. + /// + public MqttSubscriptionManager( + MqttDriverOptions options, + string driverId, + IMqttSubscribeTransport? transport = null, + ILogger? logger = null, + LastValueCache? cache = null, + int maxPayloadBytes = DefaultMaxPayloadBytes) + { + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(driverId); + + MaxPayloadBytes = maxPayloadBytes > 0 ? maxPayloadBytes : DefaultMaxPayloadBytes; + _mode = options.Mode; + _defaultQos = options.Plain?.DefaultQos ?? 1; + _driverId = driverId; + _transport = transport; + _logger = logger; + Values = cache ?? new LastValueCache(); + _resolver = new EquipmentTagRefResolver( + rawPath => _authored.ByRawPath.TryGetValue(rawPath, out var def) ? def : null); + } + + /// + public event EventHandler? OnDataChange; + + /// + /// The last observed value per RawPath — the bridge from MQTT's push model to the OPC UA + /// server's polled IReadable.ReadAsync batches. + /// + public LastValueCache Values { get; } + + /// + /// Ceiling on an inbound message body. A larger message is refused before any decode or parse + /// and degrades its own tags to BadDecodingError. + /// + /// + /// Decode and parse both run synchronously on MQTTnet's shared dispatcher thread, so their cost + /// is paid by every subscription, not just the offending topic. Without a bound, one + /// publisher shipping a multi-megabyte body — or a hostile one shipping a huge nested document + /// — stalls the whole driver's delivery for as long as the parse takes. Sized for telemetry + /// payloads, which are kilobytes; a deployment that legitimately needs more should raise this + /// deliberately rather than run unbounded. + /// + /// Follow-up: this is a constructor knob, not yet an operator-facing one. Promoting + /// it to an MqttDriverOptions key (and pairing it with MQTTnet's + /// WithMaximumPacketSize, so an oversized body is refused at the protocol layer + /// rather than after the client has already buffered it) needs an edit to the + /// .Contracts project that this task was scoped out of. + /// + /// + public int MaxPayloadBytes { get; } + + /// + /// Wires this manager onto a live connection: inbound messages route to + /// and every reconnect re-establishes the subscriptions through + /// . The connection also becomes this manager's SUBSCRIBE + /// transport. + /// + /// + /// The manager does not own the connection's lifetime and never detaches — the driver disposes + /// both together. Call once per connection, and before the first + /// : a subscribe issued with no transport attached records its + /// intent (and warns) but sends nothing to a broker, and would only be established if the + /// connection later dropped and recovered. + /// + /// The connection to observe. + public void AttachTo(MqttConnection connection) + { + ArgumentNullException.ThrowIfNull(connection); + + _transport = connection; + connection.MessageReceived += HandleMessage; + connection.Reconnected += OnReconnectedAsync; + } + + /// + /// Replaces the authored tag table with the deploy's raw tags, rebuilding the topic index. + /// Wholesale, not additive: a redeploy that drops a tag must stop feeding it. + /// + /// + /// A whose TagConfig does not map is skipped with a warning, + /// never thrown — one bad tag must not fail the whole driver's initialise. Such a reference + /// then resolves to nothing and surfaces BadNodeIdUnknown, which is the stated contract + /// of MqttTagDefinitionFactory.FromTagConfig. + /// + /// The authored raw tags delivered by the deploy artifact. + /// How many entries mapped to a usable definition. + public int Register(IEnumerable rawTags) + { + ArgumentNullException.ThrowIfNull(rawTags); + + var byRawPath = new Dictionary(StringComparer.Ordinal); + foreach (var entry in rawTags) + { + if (MqttTagDefinitionFactory.FromTagConfig(entry.TagConfig, entry.RawPath, out var def)) + { + byRawPath[entry.RawPath] = def; + } + else + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': tag config did not map to a definition; skipping. RawPath={RawPath}", + _driverId, + entry.RawPath); + } + } + + var table = AuthoredTable.Build(byRawPath); + _authored = table; + _warnedRefs.Clear(); + + // Drop state belonging to tags this deploy removed. Without this, a topic whose last tag was + // deleted stays in _subscribedTopics and is dutifully re-SUBSCRIBEd on every future reconnect, + // forever — and its RawPaths keep a handle mapping that can never fire again. + foreach (var topic in _subscribedTopics.Keys) + { + if (!table.ByFilter.ContainsKey(topic)) + { + _subscribedTopics.TryRemove(topic, out _); + } + } + + foreach (var rawPath in _handleByRawPath.Keys) + { + if (!table.ByRawPath.ContainsKey(rawPath)) + { + _handleByRawPath.TryRemove(rawPath, out _); + } + } + + return byRawPath.Count; + } + + /// Resolves a RawPath to its authored definition through the shared v3 resolver. + /// The driver wire reference. + /// The definition, when this returns . + /// when the reference is an authored MQTT tag. + public bool TryResolve(string rawPath, out MqttTagDefinition def) => _resolver.TryResolve(rawPath, out def!); + + /// + /// Subscribes a batch of RawPath references: records them against a new handle and issues one + /// SUBSCRIBE for every distinct authored topic not already established. + /// + /// + /// + /// This is the first subscribe. deliberately + /// does not fire after the initial ConnectAsync, so the manager must issue the + /// opening SUBSCRIBE itself; only re-establishes it later. + /// + /// + /// Never throws, never hangs. A reference that is not an authored tag is recorded as + /// BadNodeIdUnknown; a SUBACK rejection, transport failure or subscribe deadline + /// degrades exactly the affected references to BadCommunicationError. Either way a + /// handle is returned — the OPC UA server's subscribe call must not fail because one topic + /// is ACL-denied. Recovery for a transient failure comes from the reconnect path; + /// a persistent broker-side refusal is visible as a Bad status on the affected nodes. + /// + /// + /// The RawPath references to subscribe. + /// + /// Ignored. MQTT is push-based — the broker's publish rate is the notification rate, and this + /// driver neither polls nor throttles. + /// + /// Cancellation for the SUBSCRIBE round-trip. + /// The handle every resulting is attributed to. + public async Task SubscribeAsync( + IReadOnlyList fullReferences, + TimeSpan publishingInterval, + CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(fullReferences); + + var defs = new List(fullReferences.Count); + var now = DateTime.UtcNow; + foreach (var reference in fullReferences) + { + if (_resolver.TryResolve(reference, out var def)) + { + defs.Add(def); + continue; + } + + // Not an authored MQTT tag. Say so, rather than leaving it in "waiting for initial data" + // forever — a reference that will never be fed is a different fault from one that has + // simply not been fed yet. + _logger?.LogWarning( + "MQTT driver '{DriverId}': subscribe reference '{RawPath}' is not an authored MQTT tag.", + _driverId, + reference); + Values.Update(reference, new DataValueSnapshot(null, StatusBadNodeIdUnknown, null, now)); + } + + var handle = new MqttSubscriptionHandle(BuildDiagnosticId(defs)); + _refsByHandle[handle] = [.. defs.Select(d => d.Name)]; + foreach (var def in defs) + { + _handleByRawPath[def.Name] = handle; + } + + // One SUBSCRIBE per distinct topic, and only for topics not already live: re-SUBSCRIBEing an + // established filter is legal but pointless, and it re-triggers the retained seed. + var pending = defs.Where(d => !_subscribedTopics.ContainsKey(d.Topic)).ToList(); + await EstablishAsync(BuildFilters(pending, _defaultQos), cancellationToken).ConfigureAwait(false); + + return handle; + } + + /// + /// Stops attributing notifications to . References another live + /// subscription also covers keep publishing. + /// + /// + /// The broker-side subscription is deliberately left in place: MQTT UNSUBSCRIBE is per-filter, + /// several handles can share one filter, and an idle filter costs only the traffic a topic + /// already produces. The session teardown drops them all. + /// + /// The handle returned by . + /// Unused; present for the ISubscribable shape. + /// A completed task. + public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken) + { + if (handle is not null && _refsByHandle.TryRemove(handle, out var refs)) + { + foreach (var rawPath in refs) + { + // Only clear the mapping if it is still OURS — a later overlapping subscription may + // have taken the reference over, and it must keep publishing. + _handleByRawPath.TryRemove(new KeyValuePair(rawPath, handle)); + } + } + + return Task.CompletedTask; + } + + /// + /// Re-establishes every live subscription after a reconnect. Wired to + /// by . + /// + /// + /// + /// Idempotent — re-SUBSCRIBEing an already-subscribed filter is expected on a + /// persistent session and is harmless; missing one is a healthy-looking client that + /// receives nothing forever. + /// + /// + /// Total failure THROWS, partial rejection does not. This is a deliberate split. + /// 's contract is that a throwing Reconnected subscriber + /// tears the fresh session down and retries under backoff — the correct response to "no + /// subscription was re-established", because the alternative is serving a connected, + /// permanently deaf driver. But if the broker granted some filters and refused others + /// (an ACL denying one topic), tearing the session down would retry forever and take + /// every tag dark to punish one; those references degrade to Bad instead and the + /// rest keep flowing. That is the plan's "SUBACK failure → per-ref Bad, not hang", scoped + /// to the case where it does not cost the whole driver. + /// + /// + /// The connection's lifetime token; cancelled by its dispose. + /// A task that completes when the re-subscribe has been answered. + /// + /// The subscribe failed outright, or every filter was refused — the caller is expected to tear + /// the session down and retry. + /// + public async Task OnReconnectedAsync(CancellationToken cancellationToken) + { + var authored = _authored; + var subscribed = _subscribedTopics.Keys.ToList(); + + // ByFilter, NOT ByExactTopic: these keys are subscribed FILTER strings, so a wildcard-authored + // tag's key is its pattern ("f/+/temp"). Reading the concrete-only index here misses every + // wildcard tag, drops it out of the re-subscribe set, and leaves it dark forever behind a + // connection that reports healthy — see the AuthoredTable remarks. + var live = subscribed + .Select(topic => authored.ByFilter.TryGetValue(topic, out var defs) ? defs : []) + .SelectMany(defs => defs) + .ToList(); + + var filters = BuildFilters(live, _defaultQos); + if (filters.Count == 0) + { + // Silence here is how the wildcard defect hid: no exception, no log, no SUBSCRIBE. If any + // topic was live, resolving zero filters for it is an anomaly and must be loud. + if (subscribed.Count > 0) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': reconnect resolved NO filters for {TopicCount} live topic(s) " + + "({Topics}); those subscriptions are not being re-established.", + _driverId, + subscribed.Count, + string.Join(", ", subscribed)); + } + else + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': reconnect had no live subscriptions to re-establish.", + _driverId); + } + + return; + } + + var granted = await EstablishAsync(filters, cancellationToken).ConfigureAwait(false); + if (granted == 0) + { + throw new InvalidOperationException( + $"MQTT driver '{_driverId}': no subscription could be re-established after reconnect " + + $"({filters.Count} filter(s) requested); tearing the session down rather than serving a " + + "connection that receives nothing."); + } + } + + /// + /// Routes one inbound message to every authored tag whose topic matches, extracts the value, + /// updates , and raises for the tags that are + /// currently subscribed. + /// + /// + /// + /// Runs on MQTTnet's dispatcher thread: no I/O, no locks, and no path throws. A tag + /// whose payload is malformed degrades to a Bad status code and the fan-out continues to + /// the next tag. + /// + /// + /// Bounded work. A body larger than is refused + /// before any decode or parse: the whole point of the dispatcher discipline is that + /// one topic cannot stall delivery for every other subscription, and an unbounded + /// GetString / JsonDocument.Parse over a multi-megabyte body does exactly + /// that. The oversized message degrades its own tags to BadDecodingError. + /// + /// + /// Parsed once. The documented shape is several tags sharing one JSON document with + /// a JSONPath each, so the document is parsed at most once per message and shared across + /// the whole fan-out rather than re-parsed per tag. + /// + /// + /// The concrete topic the message arrived on. + /// The message body; valid only for the duration of this call. + /// The message's retain flag — true is the broker's subscribe-time seed. + public void HandleMessage(string topic, ReadOnlySpan payload, bool retained) + { + if (string.IsNullOrEmpty(topic)) + { + return; + } + + var authored = _authored; + var now = DateTime.UtcNow; + var oversized = payload.Length > MaxPayloadBytes; + + // Zero-alloc until a Json-format tag actually matches; disposed in the finally below. + var json = default(PayloadJson); + try + { + // Concrete topics — the overwhelmingly common case — cost one lookup, so a chatty broker + // publishing thousands of topics this driver does not care about stays cheap. + if (authored.ByExactTopic.TryGetValue(topic, out var exact)) + { + foreach (var def in exact) + { + Deliver(def, payload, retained, now, oversized, ref json); + } + } + + // A published topic name may not itself contain a wildcard, so a wildcard-authored + // definition can never also be an exact-index hit: no tag is delivered twice. + foreach (var def in authored.Wildcards) + { + if (MqttTopicFilterComparer.Compare(topic, def.Topic) == MqttTopicFilterCompareResult.IsMatch) + { + Deliver(def, payload, retained, now, oversized, ref json); + } + } + } + finally + { + json.Dispose(); + } + } + + /// + /// Collapses a set of definitions into the filters to SUBSCRIBE: one per distinct topic, + /// carrying the strongest QoS any tag on that topic asked for (so no tag silently gets a + /// weaker delivery guarantee than it was authored with), and seeding from the retained message + /// if any tag on the topic wants it (tags that do not are filtered on the retain flag at + /// delivery). + /// + /// The definitions to collapse. + /// The driver-level QoS applied to tags that authored none. + /// One filter per distinct topic. + internal static IReadOnlyList BuildFilters( + IEnumerable defs, + int defaultQos) + { + var byTopic = new Dictionary(StringComparer.Ordinal); + foreach (var def in defs) + { + var qos = def.Qos ?? defaultQos; + byTopic[def.Topic] = byTopic.TryGetValue(def.Topic, out var existing) + ? existing with + { + Qos = Math.Max(existing.Qos, qos), + SeedRetained = existing.SeedRetained || def.RetainSeed, + } + : new MqttTopicSubscription(def.Topic, qos, def.RetainSeed); + } + + return [.. byTopic.Values]; + } + + /// + /// Issues the SUBSCRIBE and applies its verdict: granted topics join the live set, refused ones + /// degrade their references to BadCommunicationError. + /// + /// The filters to establish; an empty list is a no-op. + /// Cancellation for the round-trip. + /// How many filters the broker granted. + private async Task EstablishAsync( + IReadOnlyList filters, + CancellationToken cancellationToken) + { + if (filters.Count == 0) + { + return 0; + } + + var transport = _transport; + if (transport is null) + { + // No transport: the intent is recorded so the reconnect path establishes it, but NOTHING + // reaches a broker now. Loud, because outside tests this means the composition order is + // wrong — AttachTo must run before the OPC UA server's first subscribe, or these topics + // stay unestablished until the connection happens to drop and recover. + foreach (var filter in filters) + { + _subscribedTopics[filter.Topic] = 0; + } + + _logger?.LogWarning( + "MQTT driver '{DriverId}': {FilterCount} subscription(s) recorded with no connection attached; " + + "nothing was sent to a broker. Call AttachTo before subscribing.", + _driverId, + filters.Count); + + return filters.Count; + } + + IReadOnlyList outcomes; + try + { + outcomes = await transport.SubscribeAsync(filters, cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) + { + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': SUBSCRIBE for {FilterCount} filter(s) failed; the affected tags degrade to Bad.", + _driverId, + filters.Count); + foreach (var filter in filters) + { + DegradeTopic(filter.Topic, ex.GetType().Name); + } + + return 0; + } + + var granted = 0; + foreach (var outcome in outcomes) + { + if (outcome.Granted) + { + _subscribedTopics[outcome.Topic] = 0; + granted++; + } + else + { + _logger?.LogError( + "MQTT driver '{DriverId}': broker refused SUBSCRIBE '{Topic}' ({Reason}); its tags degrade to Bad.", + _driverId, + outcome.Topic, + outcome.Reason); + DegradeTopic(outcome.Topic, outcome.Reason); + } + } + + return granted; + } + + /// Marks every authored reference on as unreachable. + /// + /// is a subscribed filter string (it came from a requested + /// filter or a SUBACK outcome), so the lookup must go through ByFilter — the + /// concrete-only index misses every wildcard-authored tag and this method would then return + /// having silently degraded nothing, leaving the tag at BadWaitingForInitialData (or + /// worse, its last Good value) after the broker refused its subscription. + /// + /// The topic filter whose subscription could not be established. + /// The failure reason, for logs. + private void DegradeTopic(string topic, string reason) + { + _subscribedTopics.TryRemove(topic, out _); + + if (!_authored.ByFilter.TryGetValue(topic, out var defs)) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': cannot degrade tags for unestablished filter '{Topic}' ({Reason}) — " + + "no authored tag resolves to it.", + _driverId, + topic, + reason); + return; + } + + var now = DateTime.UtcNow; + var snapshot = new DataValueSnapshot(null, StatusBadCommunicationError, null, now); + foreach (var def in defs) + { + Values.Update(def.Name, snapshot); + Raise(def.Name, snapshot); + } + + _logger?.LogDebug( + "MQTT driver '{DriverId}': degraded {Count} tag(s) on '{Topic}' ({Reason}).", + _driverId, + defs.Length, + topic, + reason); + } + + /// + /// Delivers one message to one tag: honours the retained-seed opt-out, extracts the value, + /// caches it and raises the notification. Never throws. + /// + /// The authored tag this message matched. + /// The message body. + /// The message's retain flag. + /// One clock read shared by the whole fan-out, so siblings agree. + /// Whether the body exceeded . + /// The fan-out's shared JSON parse, created on first use. + private void Deliver( + MqttTagDefinition def, + ReadOnlySpan payload, + bool retained, + DateTime nowUtc, + bool oversized, + ref PayloadJson json) + { + // MQTT 3.1.1 has no retain-handling option, so the broker always replays its retained message + // on subscribe; a tag that opted out of seeding is filtered here rather than at the broker. + if (retained && !def.RetainSeed) + { + return; + } + + DataValueSnapshot snapshot; + try + { + // The cap is enforced before any decode/parse, so an oversized body costs one comparison + // per matched tag rather than a full pass over the payload. + snapshot = oversized + ? new DataValueSnapshot(null, StatusBadDecodingError, null, nowUtc) + : Extract(def, payload, nowUtc, ref json); + } + catch (Exception ex) + { + // Defence in depth: Extract is written not to throw, but this runs on the library's + // dispatcher and an escape here would stall delivery for every other subscription. + snapshot = new DataValueSnapshot(null, StatusBadDecodingError, null, nowUtc); + _logger?.LogDebug(ex, "MQTT driver '{DriverId}': extraction threw for '{RawPath}'.", _driverId, def.Name); + } + + if (snapshot.StatusCode != StatusGood && _warnedRefs.TryAdd(def.Name, 0)) + { + // Loud once per reference per deploy, quiet thereafter: a mis-authored jsonPath on a 10 Hz + // topic must be diagnosable without drowning the log. + _logger?.LogWarning( + "MQTT driver '{DriverId}': tag '{RawPath}' (topic '{Topic}', {Format}, jsonPath '{JsonPath}', " + + "{DataType}) could not be extracted from its payload; status 0x{Status:X8}.", + _driverId, + def.Name, + def.Topic, + def.PayloadFormat, + def.JsonPath, + def.DataType, + snapshot.StatusCode); + } + + Values.Update(def.Name, snapshot); + Raise(def.Name, snapshot); + } + + /// + /// Raises for a reference, if a live subscription covers it. + /// A throwing subscriber is contained so it cannot starve the rest of the fan-out — or, worse, + /// escape onto MQTTnet's dispatcher thread. + /// + /// The RawPath — the reference the notification is published under. + /// The value/quality/timestamps to publish. + private void Raise(string rawPath, DataValueSnapshot snapshot) + { + if (!_handleByRawPath.TryGetValue(rawPath, out var handle)) + { + // Authored but not subscribed: the value is cached (so a read answers) and nothing is + // published — there is no subscription to attribute it to. + return; + } + + try + { + OnDataChange?.Invoke(this, new DataChangeEventArgs(handle, rawPath, snapshot)); + } + catch (Exception ex) + { + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': a data-change subscriber threw for '{RawPath}'.", + _driverId, + rawPath); + } + } + + /// + /// Extracts the tag's value from a payload according to its . + /// Never throws: every failure becomes a Bad status code on the returned snapshot. + /// + /// The authored tag definition. + /// The message body. + /// Timestamp for the snapshot — MQTT carries no source timestamp of its own. + /// The fan-out's shared JSON parse, created on first use by a Json-format tag. + /// The extracted snapshot. + internal static DataValueSnapshot Extract( + MqttTagDefinition def, + ReadOnlySpan payload, + DateTime nowUtc, + ref PayloadJson json) + { + switch (def.PayloadFormat) + { + case MqttPayloadFormat.Json: + return ExtractJson(def, payload, nowUtc, ref json); + + case MqttPayloadFormat.Raw: + // "Bytes as-is": the payload's UTF-8 text, published verbatim. dataType is deliberately + // NOT applied — Scalar is the format that coerces. Authoring Raw with a numeric + // dataType is a mis-authoring, and one the deploy-time inspection should grow a + // warning for. + return TryDecodeUtf8(payload, out var raw) + ? new DataValueSnapshot(raw, StatusGood, nowUtc, nowUtc) + : Bad(StatusBadDecodingError, nowUtc); + + case MqttPayloadFormat.Scalar: + if (!TryDecodeUtf8(payload, out var text)) + { + return Bad(StatusBadDecodingError, nowUtc); + } + + return TryCoerceText(text, def.DataType, out var scalar) + ? new DataValueSnapshot(scalar, StatusGood, nowUtc, nowUtc) + : Bad(StatusBadTypeMismatch, nowUtc); + + default: + return Bad(StatusBadDecodingError, nowUtc); + } + } + + private static DataValueSnapshot Bad(uint status, DateTime nowUtc) => new(null, status, null, nowUtc); + + /// + /// Selects the tag's JSONPath out of the fan-out's shared parse and coerces it to the + /// tag's data type. The document is parsed at most once per message however many tags share the + /// topic — the documented "one JSON document, one JSONPath per signal" shape would otherwise + /// re-parse it once per tag, on the dispatcher thread. + /// + /// The authored tag definition. + /// The message body. + /// Timestamp for the snapshot. + /// The shared parse; parses on first call and caches the outcome. + /// The extracted snapshot. + private static DataValueSnapshot ExtractJson( + MqttTagDefinition def, + ReadOnlySpan payload, + DateTime nowUtc, + ref PayloadJson json) + { + if (!json.TryGetRoot(payload, out var root)) + { + return Bad(StatusBadDecodingError, nowUtc); + } + + if (!TryEvaluateJsonPath(root, def.JsonPath, out var selected)) + { + return Bad(StatusBadDecodingError, nowUtc); + } + + return TryCoerce(selected, def.DataType, out var value) + ? new DataValueSnapshot(value, StatusGood, nowUtc, nowUtc) + : Bad(StatusBadTypeMismatch, nowUtc); + } + + /// + /// One message's JSON parse, shared across the whole fan-out. A mutable struct passed by + /// ref so the common case — a message no Json-format tag matched — allocates nothing at + /// all, while a message several Json tags share is parsed exactly once. + /// + internal struct PayloadJson : IDisposable + { + private JsonDocument? _doc; + private bool _attempted; + private bool _parsed; + + /// + /// The parsed document root, parsing on first call. A malformed payload is remembered as a + /// failure so the fan-out's remaining tags do not each retry the same doomed parse. + /// + /// The message body; only read on the first call. + /// The document root when this returns . + /// when the payload is valid JSON. + public bool TryGetRoot(ReadOnlySpan payload, out JsonElement root) + { + if (!_attempted) + { + _attempted = true; + try + { + var reader = new Utf8JsonReader(payload); + _parsed = JsonDocument.TryParseValue(ref reader, out _doc); + } + catch (JsonException) + { + _parsed = false; + } + } + + root = _parsed ? _doc!.RootElement : default; + return _parsed; + } + + /// + public void Dispose() + { + _doc?.Dispose(); + _doc = null; + } + } + + /// + /// Evaluates the supported subset of JSONPath against a document: the root ($), + /// dotted member access ($.a.b), bracketed member access ($['a'] / $["a"]) + /// and non-negative array indexing ($.a[0]). + /// + /// + /// Filters, slices, wildcards and recursive descent ($..x) are deliberately not + /// supported: this repo pins no JSONPath package, adding one for an expression form that + /// selects a set — while a tag needs exactly one scalar — is the wrong trade. An + /// unsupported expression is reported as a miss, so the tag surfaces + /// BadDecodingError plus the one-shot warning rather than a wrong value. + /// + /// The parsed document root. + /// The authored JSONPath. + /// The selected element when this returns . + /// when the path is supported and selects an element. + internal static bool TryEvaluateJsonPath(JsonElement root, string path, out JsonElement value) + { + value = root; + if (string.IsNullOrEmpty(path) || path[0] != '$') + { + return false; + } + + var i = 1; + while (i < path.Length) + { + if (path[i] == '.') + { + i++; + if (i >= path.Length || path[i] == '.') + { + return false; // trailing '.', or recursive descent '..' + } + + var start = i; + while (i < path.Length && path[i] != '.' && path[i] != '[') + { + i++; + } + + var name = path[start..i]; + if (name.Length == 0 || name == "*") + { + return false; + } + + if (value.ValueKind != JsonValueKind.Object || !value.TryGetProperty(name, out var member)) + { + return false; + } + + value = member; + } + else if (path[i] == '[') + { + var close = path.IndexOf(']', i); + if (close < 0) + { + return false; + } + + var inner = path[(i + 1)..close].Trim(); + i = close + 1; + + if (inner.Length >= 2 + && ((inner[0] == '\'' && inner[^1] == '\'') || (inner[0] == '"' && inner[^1] == '"'))) + { + var name = inner[1..^1]; + if (value.ValueKind != JsonValueKind.Object || !value.TryGetProperty(name, out var member)) + { + return false; + } + + value = member; + } + else if (int.TryParse(inner, NumberStyles.Integer, CultureInfo.InvariantCulture, out var index) + && index >= 0) + { + if (value.ValueKind != JsonValueKind.Array || index >= value.GetArrayLength()) + { + return false; + } + + value = value[index]; + } + else + { + return false; // slice, filter or wildcard — outside the supported subset + } + } + else + { + return false; + } + } + + return true; + } + + /// Coerces a selected JSON element to the tag's declared data type. + /// The selected element. + /// The tag's declared type. + /// The coerced value when this returns . + /// when the element fits the declared type. + internal static bool TryCoerce(JsonElement element, DriverDataType type, out object? value) + { + value = null; + + if (element.ValueKind is JsonValueKind.Null or JsonValueKind.Undefined) + { + return false; + } + + if (type is DriverDataType.String or DriverDataType.Reference) + { + // A String tag pointed at an object/array gets that subtree's JSON text — the authored + // intent ("give me this fragment") rather than a refusal. + value = element.ValueKind == JsonValueKind.String ? element.GetString() : element.GetRawText(); + return true; + } + + // A JSON string holding a number ({"v":"21.5"}) is common in the wild; parse it as text rather + // than refusing a payload the operator plainly meant. + if (element.ValueKind == JsonValueKind.String) + { + return TryCoerceText(element.GetString() ?? string.Empty, type, out value); + } + + switch (type) + { + case DriverDataType.Boolean: + if (element.ValueKind == JsonValueKind.True) { value = true; return true; } + if (element.ValueKind == JsonValueKind.False) { value = false; return true; } + if (element.ValueKind == JsonValueKind.Number && element.TryGetDouble(out var b)) { value = b != 0d; return true; } + return false; + + case DriverDataType.Int16: + case DriverDataType.Int32: + case DriverDataType.Int64: + case DriverDataType.UInt16: + case DriverDataType.UInt32: + case DriverDataType.UInt64: + if (element.ValueKind != JsonValueKind.Number) + { + return false; + } + + // Fast path: the JSON number is already written as an integer. + if (TryExactInteger(element, type, out value)) + { + return true; + } + + // Fallback: an integral-valued REAL. JS/Python edge gateways routinely serialize an + // integer as 5.0 (or 5E2), and System.Text.Json's TryGetInt32/64 are syntactic — they + // refuse it. Refusing here would mean an integer tag fed by such a gateway silently + // never receives data, so an exactly-integral real is accepted; a genuinely fractional + // value (5.5) is still refused rather than rounded. + return TryIntegralReal(element.GetRawText(), type, out value); + + case DriverDataType.Float32: + if (element.ValueKind == JsonValueKind.Number && element.TryGetSingle(out var f32)) { value = f32; return true; } + return false; + + case DriverDataType.Float64: + if (element.ValueKind == JsonValueKind.Number && element.TryGetDouble(out var f64)) { value = f64; return true; } + return false; + + case DriverDataType.DateTime: + // Only ISO-8601 text is accepted; a bare number is ambiguous (epoch seconds? ms?) and + // guessing would silently place samples decades apart. + return false; + + default: + return false; + } + } + + /// Parses a textual payload (or JSON string) into the tag's declared data type. + /// The decoded text. + /// The tag's declared type. + /// The parsed value when this returns . + /// when the text parses as the declared type. + internal static bool TryCoerceText(string text, DriverDataType type, out object? value) + { + value = null; + var inv = CultureInfo.InvariantCulture; + var trimmed = text.Trim(); + + switch (type) + { + case DriverDataType.String: + case DriverDataType.Reference: + // Untrimmed: for a String tag the payload IS the value, whitespace included. + value = text; + return true; + + case DriverDataType.Boolean: + if (bool.TryParse(trimmed, out var b)) { value = b; return true; } + if (long.TryParse(trimmed, NumberStyles.Integer, inv, out var bi)) { value = bi != 0L; return true; } + return false; + + case DriverDataType.Int16: + case DriverDataType.Int32: + case DriverDataType.Int64: + case DriverDataType.UInt16: + case DriverDataType.UInt32: + case DriverDataType.UInt64: + // Same two-step as the JSON path: exact integer text first, then an integral-valued + // real ("5.0"), which a Scalar-format edge gateway emits just as readily. + if (TryExactIntegerText(trimmed, type, out value)) + { + return true; + } + + return TryIntegralReal(trimmed, type, out value); + + case DriverDataType.Float32: + if (float.TryParse(trimmed, NumberStyles.Float, inv, out var f32)) { value = f32; return true; } + return false; + + case DriverDataType.Float64: + if (double.TryParse(trimmed, NumberStyles.Float, inv, out var f64)) { value = f64; return true; } + return false; + + case DriverDataType.DateTime: + if (DateTime.TryParse( + trimmed, + inv, + DateTimeStyles.RoundtripKind | DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal, + out var dt)) + { + value = dt; + return true; + } + + return false; + + default: + return false; + } + } + + /// Reads a JSON number that is written as an exact integer of the requested type. + /// The JSON number. + /// The target integer type. + /// The boxed value when this returns . + /// when the number is an exact integer in range. + private static bool TryExactInteger(JsonElement element, DriverDataType type, out object? value) + { + value = type switch + { + DriverDataType.Int16 => element.TryGetInt16(out var v) ? v : null, + DriverDataType.Int32 => element.TryGetInt32(out var v) ? v : null, + DriverDataType.Int64 => element.TryGetInt64(out var v) ? v : null, + DriverDataType.UInt16 => element.TryGetUInt16(out var v) ? v : null, + DriverDataType.UInt32 => element.TryGetUInt32(out var v) ? v : null, + DriverDataType.UInt64 => element.TryGetUInt64(out var v) ? v : (object?)null, + _ => null, + }; + + return value is not null; + } + + /// Parses text written as an exact integer of the requested type. + /// The trimmed text. + /// The target integer type. + /// The boxed value when this returns . + /// when the text is an exact integer in range. + private static bool TryExactIntegerText(string text, DriverDataType type, out object? value) + { + var inv = CultureInfo.InvariantCulture; + value = type switch + { + DriverDataType.Int16 => short.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : null, + DriverDataType.Int32 => int.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : null, + DriverDataType.Int64 => long.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : null, + DriverDataType.UInt16 => ushort.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : null, + DriverDataType.UInt32 => uint.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : null, + DriverDataType.UInt64 => ulong.TryParse(text, NumberStyles.Integer, inv, out var v) ? v : (object?)null, + _ => null, + }; + + return value is not null; + } + + /// + /// Accepts a real that is exactly integral (5.0, 5E2) as an integer of the + /// requested type. A fractional value is refused, never rounded — silently turning 5.5 into 5 + /// or 6 would be a wrong value, which is worse than no value. + /// + /// + /// Parsed as rather than so the integrality test and + /// the conversion are exact across the whole Int64/UInt64 range, where double loses precision + /// above 2^53. Out-of-range values overflow the conversion and are refused. + /// + /// The number's text (JSON raw text, or a Scalar payload). + /// The target integer type. + /// The boxed value when this returns . + /// when the text is an integral real in range. + private static bool TryIntegralReal(string text, DriverDataType type, out object? value) + { + value = null; + + if (!decimal.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out var d) + || d != decimal.Truncate(d)) + { + return false; + } + + try + { + value = type switch + { + DriverDataType.Int16 => (short)d, + DriverDataType.Int32 => (int)d, + DriverDataType.Int64 => (long)d, + DriverDataType.UInt16 => (ushort)d, + DriverDataType.UInt32 => (uint)d, + DriverDataType.UInt64 => (ulong)d, + _ => null, + }; + } + catch (OverflowException) + { + // decimal → integer conversions throw rather than wrap; out of range is a refusal. + value = null; + } + + return value is not null; + } + + /// + /// Decodes a payload as UTF-8, refusing invalid byte sequences rather than rendering a genuinely + /// binary body as a field of replacement characters (the same choice the browse session makes). + /// + /// The message body. + /// The decoded text when this returns . + /// when the payload is valid UTF-8. + private static bool TryDecodeUtf8(ReadOnlySpan payload, out string text) + { + try + { + text = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false, throwOnInvalidBytes: true) + .GetString(payload); + return true; + } + catch (DecoderFallbackException) + { + text = string.Empty; + return false; + } + } + + /// + /// Builds the handle's diagnostic id: the ingest mode plus the topics it covers, bounded so a + /// thousand-tag subscription does not produce a thousand-topic string in every log line. + /// + /// The definitions the subscription covers. + /// The diagnostic id. + private string BuildDiagnosticId(IReadOnlyCollection defs) + { + const int MaxNamedTopics = 5; + + var id = Interlocked.Increment(ref _handleSeq); + var topics = defs.Select(d => d.Topic).Distinct(StringComparer.Ordinal).ToList(); + var named = string.Join(',', topics.Take(MaxNamedTopics)); + var overflow = topics.Count > MaxNamedTopics ? $",+{topics.Count - MaxNamedTopics}" : string.Empty; + + return $"mqtt:{_mode}:{id}:[{named}{overflow}]"; + } + + /// Opaque subscription identity; the diagnostic id names the mode and its topics. + /// The human-readable identity surfaced for diagnostics. + private sealed record MqttSubscriptionHandle(string DiagnosticId) : ISubscriptionHandle; + + /// + /// The authored tag table and its derived indexes, published as one immutable snapshot so the + /// dispatcher thread reads a consistent view while a redeploy rebuilds it. + /// + /// + /// Two topic indexes, and the difference is load-bearing. Delivery matches an + /// incoming published topic, which never carries a wildcard, so it uses + /// (concrete only) plus a comparer scan of . + /// Every path that instead reconstructs state from a subscribed filter string — + /// reconnect re-subscribe and per-topic degradation — must use , which is + /// keyed by the tag's authored topic including wildcard patterns, because that is what + /// lands in _subscribedTopics and in a SUBACK outcome. Looking a wildcard filter up in + /// the concrete index always misses, and a miss on those paths is silent: the tag simply stops + /// updating behind a healthy-looking connection, keeping its last Good value forever. + /// + /// RawPath → definition; the v3 resolver's backing table. + /// + /// Authored topic filter string → every definition bound to it, wildcard patterns + /// included. Keyed the same way _subscribedTopics and SUBACK outcomes are. + /// + /// Concrete topic → its definitions; the delivery fast path. + /// Definitions whose authored topic carries an MQTT wildcard. + private sealed record AuthoredTable( + FrozenDictionary ByRawPath, + FrozenDictionary ByFilter, + FrozenDictionary ByExactTopic, + MqttTagDefinition[] Wildcards) + { + public static readonly AuthoredTable Empty = + Build(new Dictionary(StringComparer.Ordinal)); + + /// True when a topic string carries an MQTT wildcard and is therefore a filter, not a name. + /// The authored topic. + /// when it contains + or #. + public static bool IsWildcard(string topic) => + topic.Contains(MqttTopicFilterComparer.SingleLevelWildcard, StringComparison.Ordinal) + || topic.Contains(MqttTopicFilterComparer.MultiLevelWildcard, StringComparison.Ordinal); + + /// Builds the snapshot: every def into ByFilter, concrete/wildcard split for delivery. + /// The mapped definitions, keyed by RawPath. + /// The immutable snapshot. + public static AuthoredTable Build(IReadOnlyDictionary byRawPath) + { + var byFilter = new Dictionary>(StringComparer.Ordinal); + var concrete = new Dictionary>(StringComparer.Ordinal); + var wildcards = new List(); + + foreach (var def in byRawPath.Values) + { + // EVERY def lands here, wildcard or not — this is the index the reconnect and degrade + // paths read, and omitting wildcards from it is exactly the C1 silent-death defect. + if (!byFilter.TryGetValue(def.Topic, out var all)) + { + byFilter[def.Topic] = all = []; + } + + all.Add(def); + + if (IsWildcard(def.Topic)) + { + wildcards.Add(def); + continue; + } + + if (!concrete.TryGetValue(def.Topic, out var list)) + { + concrete[def.Topic] = list = []; + } + + list.Add(def); + } + + return new AuthoredTable( + byRawPath.ToFrozenDictionary(StringComparer.Ordinal), + byFilter.ToFrozenDictionary(kv => kv.Key, kv => kv.Value.ToArray(), StringComparer.Ordinal), + concrete.ToFrozenDictionary(kv => kv.Key, kv => kv.Value.ToArray(), StringComparer.Ordinal), + [.. wildcards]); + } + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/AliasTable.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/AliasTable.cs new file mode 100644 index 00000000..45d66b0b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/AliasTable.cs @@ -0,0 +1,263 @@ +using System.Collections.Frozen; +using System.Diagnostics.CodeAnalysis; +using TahuDataType = Org.Eclipse.Tahu.Protobuf.DataType; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// One (edgeNode, device) scope's metric catalog, as declared by its most recent +/// NBIRTH/DBIRTH: the alias→metric cache and the stable name→metric index, rebuilt wholesale +/// on every birth. Design doc §3.6 invariants #1 and #2. +/// +/// +/// +/// The alias is a per-birth cache; the NAME is the binding key. Sparkplug's alias +/// mechanism exists to shrink DATA payloads — a BIRTH declares name ↔ alias and every +/// subsequent DATA metric carries only the alias — and the alias is scoped to that one +/// birth. After a rebirth an edge node is entirely free to point alias 5 at a different +/// metric. So an authored tag binds by ; the alias is +/// only ever the lookup that turns an incoming DATA metric back into a name. Binding a tag to +/// an alias instead means a rebirth silently starts routing Pressure into a Temperature tag — +/// good quality, plausible values, completely wrong, and invisible unless something +/// specifically reuses an alias across a rebirth. +/// +/// +/// REPLACES; it never merges or upserts. The whole +/// snapshot — both indexes — is built fresh and published in one assignment. An alias the new +/// birth did not declare is gone, including when the new birth declares no metrics at all. This +/// is the single behaviour the type exists to guarantee: a merge (or an "optimisation" that +/// skips an empty birth) reintroduces exactly the stale-alias mis-route above. +/// +/// +/// Both indexes are replaced together, from the same birth. They are deliberately held +/// in one type rather than split across an "alias table" and a separate "birth cache": they are +/// two views of one birth, and any seam between them is somewhere the two can be replaced +/// independently and skew — an alias resolving through the new birth into a name catalog still +/// holding the old one. +/// +/// +/// Thread-safety: an immutable snapshot swapped atomically, the same discipline +/// MqttSubscriptionManager.AuthoredTable uses and for the same reason — this is read on +/// MQTTnet's shared dispatcher thread as messages arrive, and a reader must never see a +/// half-rebuilt map. No lock is taken on any path. +/// +/// +/// Last values do not live here. The running last-observed value per tag is +/// 's, keyed by RawPath — the identity every publish in this driver +/// is made under. A second copy keyed by metric name would be a duplicate source of truth with +/// a different key, and it would grow with every metric the plant births whether or not any tag +/// was ever authored for it. A birth's own values flow straight through the ingest path to that +/// cache; nothing is retained here. +/// +/// +public sealed class AliasTable +{ + private volatile Snapshot _snapshot = Snapshot.Empty; + + /// How many distinct metrics the most recent birth declared. + public int Count => _snapshot.Metrics.Length; + + /// + /// Every metric the most recent birth declared. The consumer's STALE fan-out on a death, and + /// the discovery path's "which metrics does this scope have" question, both read this. + /// + public IReadOnlyList Metrics => _snapshot.Metrics; + + /// + /// Replaces this scope's whole catalog with — the metrics of one + /// NBIRTH/DBIRTH payload, exactly as projected them. + /// + /// The birth payload's metrics. + /// + /// What was installed, and what was refused — the caller's only window onto a malformed birth, + /// since this type takes no logger. + /// + /// + /// + /// Only an unusable NAME is a rejection. A metric with a blank/absent name cannot be + /// bound to an authored tag by any route, and storing it alias-only would create a binding + /// that can never resolve (or, worse, one that matches a tag whose authored metric name + /// happens to be blank), so it is dropped and counted. + /// + /// + /// A missing datatype is NOT a rejection — it becomes + /// . Dropping such a metric would turn every DATA message + /// for its alias into an unknown-alias event, which the ingest state machine answers with a + /// rebirth request; the edge node would answer with the same malformed birth, and the pair + /// would loop. Kept as Unknown, the typed layer refuses the value once + /// (ToDriverDataType() returns null for it) and nothing storms. + /// + /// + /// An alias claimed by two metrics in one birth resolves to neither. Any choice + /// between them is a coin flip between two different signals, which is the mis-route this + /// type exists to prevent; the alias is dropped (the consumer sees an unknown alias and can + /// request a rebirth) while both metrics stay bindable by name. A duplicate name, by + /// contrast, is last-wins: the name is the binding key, so evicting it would take the + /// authored tag dark entirely. + /// + /// + public BirthApplyResult RebuildFromBirth(IEnumerable birthMetrics) + { + ArgumentNullException.ThrowIfNull(birthMetrics); + + var byName = new Dictionary(StringComparer.Ordinal); + var byAlias = new Dictionary(); + HashSet? collidedAliases = null; + var rejectedUnnamed = 0; + var duplicateNames = 0; + var aliasCollisions = 0; + + foreach (var metric in birthMetrics) + { + if (string.IsNullOrWhiteSpace(metric.Name)) + { + rejectedUnnamed++; + continue; + } + + var binding = new SparkplugMetricBinding(metric.Name, metric.Alias, metric.DataType ?? TahuDataType.Unknown); + + if (!byName.TryAdd(binding.Name, binding)) + { + duplicateNames++; + byName[binding.Name] = binding; + } + + if (binding.Alias is { } alias && !byAlias.TryAdd(alias, binding)) + { + aliasCollisions++; + (collidedAliases ??= []).Add(alias); + } + } + + if (collidedAliases is not null) + { + foreach (var alias in collidedAliases) + { + byAlias.Remove(alias); + } + } + + // A duplicate name is last-wins in `byName`, so the alias index can still hold the losing + // binding — drop those too rather than let an alias resolve to a name that resolves elsewhere. + if (duplicateNames > 0) + { + foreach (var alias in byAlias.Where(kv => !ReferenceEquals(byName[kv.Value.Name], kv.Value)) + .Select(kv => kv.Key).ToList()) + { + byAlias.Remove(alias); + } + } + + // THE replace. One assignment of a fully-built snapshot — no merge, no upsert, and no + // short-circuit for an empty birth (an empty birth legitimately clears the scope). + _snapshot = new Snapshot( + byAlias.ToFrozenDictionary(), + byName.ToFrozenDictionary(StringComparer.Ordinal), + [.. byName.Values]); + + return new BirthApplyResult(byName.Count, rejectedUnnamed, aliasCollisions, duplicateNames); + } + + /// + /// Resolves a DATA metric's alias to the metric the most recent birth bound it to, or + /// when this birth declared no such alias. + /// + /// The alias carried by an inbound DATA metric. + /// The binding, or — which the consumer treats as an unknown alias. + public SparkplugMetricBinding? Resolve(ulong alias) => + _snapshot.ByAlias.TryGetValue(alias, out var binding) ? binding : null; + + /// The try-shape. + /// The alias carried by an inbound DATA metric. + /// The binding when this returns . + /// when the most recent birth declared this alias. + public bool TryResolve(ulong alias, [NotNullWhen(true)] out SparkplugMetricBinding? binding) + { + binding = Resolve(alias); + return binding is not null; + } + + /// + /// Resolves by the stable metric name — the binding key. Used for a DATA metric that carried its + /// name rather than an alias (aliases are optional in Sparkplug), and to answer "does this scope + /// publish the metric this tag was authored against". + /// + /// The stable metric name. Matched ordinally; Sparkplug names are case-sensitive. + /// The binding, or . + public SparkplugMetricBinding? ResolveByName(string name) => + !string.IsNullOrEmpty(name) && _snapshot.ByName.TryGetValue(name, out var binding) ? binding : null; + + /// The try-shape. + /// The stable metric name. + /// The binding when this returns . + /// when the most recent birth declared this metric. + public bool TryResolveByName(string name, [NotNullWhen(true)] out SparkplugMetricBinding? binding) + { + binding = ResolveByName(name); + return binding is not null; + } + + /// + /// One birth's catalog, immutable and published in a single assignment so the dispatcher thread + /// always reads one whole birth's worth of state. + /// + /// Alias → binding, for the birth that declared it. Collided aliases are absent. + /// Stable metric name → binding — the index authored tags bind through. + /// Every declared metric, for the STALE fan-out and discovery. + private sealed record Snapshot( + FrozenDictionary ByAlias, + FrozenDictionary ByName, + SparkplugMetricBinding[] Metrics) + { + public static readonly Snapshot Empty = new( + FrozenDictionary.Empty, + FrozenDictionary.Empty, + []); + } +} + +/// One metric as its birth declared it: the stable name, the per-birth alias, the datatype. +/// +/// The stable metric name — the binding key. An authored tag's metricName matches +/// against this, never against . +/// +/// +/// The alias this birth assigned, or when the birth declared none (legal — +/// such a publisher sends names in DATA too). Valid only until the next birth for this scope. +/// +/// +/// The datatype the birth declared, or when it declared none. +/// This is the only place a DATA metric's datatype can come from — DATA carries none. +/// +public sealed record SparkplugMetricBinding(string Name, ulong? Alias, TahuDataType DataType) +{ + /// + /// Applies to a raw DATA value using + /// this birth's datatype. + /// + /// The raw from a DATA metric. + /// The signed value for a signed-integer datatype; otherwise. + /// + /// Exposed here because this is the one object that holds the datatype at the moment a DATA + /// value is being resolved, and forgetting the call is silent: Sparkplug carries Int8/16/32/64 + /// as two's complement in an unsigned proto field, so a metric whose value is -42 + /// publishes as 4294967254 — plausible, Good quality, and invisible until someone reads a gauge. + /// + public object? Reinterpret(object? rawValue) => SparkplugCodec.ReinterpretSigned(rawValue, DataType); +} + +/// What one installed, and what it refused. +/// Distinct named metrics installed. +/// Metrics dropped for carrying no usable name. +/// Aliases claimed by more than one metric, and therefore left unresolvable. +/// Metrics whose name a later metric in the same birth overwrote. +public readonly record struct BirthApplyResult( + int Accepted, + int RejectedUnnamed, + int AliasCollisions, + int DuplicateNames) +{ + /// Whether the birth was malformed in any way worth logging. + public bool HasAnomaly => RejectedUnnamed > 0 || AliasCollisions > 0 || DuplicateNames > 0; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/BirthCache.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/BirthCache.cs new file mode 100644 index 00000000..126238e3 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/BirthCache.cs @@ -0,0 +1,234 @@ +using System.Collections.Concurrent; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// The driver's live Sparkplug birth state: one per +/// , with the spec's node→device invalidation rules applied on birth +/// and on death. Design doc §3.6 invariants #2 and #4. +/// +/// +/// +/// Scoping. A scope is the full (group, edgeNode, device?) triple — never the +/// device id alone and never node/device without the group, or one plant's +/// Filler1 would answer for another's. A node-scoped birth (NBIRTH) owns the metrics the +/// edge node publishes itself; each DBIRTH owns its device's. They are separate scopes because +/// Sparkplug gives them separate alias spaces: alias 5 on EdgeA and alias 5 on +/// EdgeA/Filler1 are unrelated metrics. +/// +/// +/// An NBIRTH invalidates every device under that node. Per the Sparkplug spec an NBIRTH +/// means the edge node (re)started and must re-publish a DBIRTH for each of its devices, so +/// every previously published DBIRTH is void. Keeping a device's alias table across its node's +/// rebirth is the same stale-alias mis-route as +/// guards against, one scope up — and it is easier to +/// miss, because nothing about the DBIRTH itself changed. The invalidated devices are returned +/// () rather than merely dropped, so the +/// consumer can fan STALE out over their metrics without a lookup against a table that has +/// already been evicted. +/// +/// +/// A death evicts, it does not blank. After an NDEATH/DDEATH the scope is absent, +/// which is what makes a DATA message arriving before the next birth a data-before-birth event +/// (⇒ request a rebirth, §3.6 invariant #3) rather than a silent unknown-alias drop against a +/// table that still looks alive. +/// +/// +/// Thread-safety. A of scopes, each +/// holding an whose own state is an atomically swapped immutable +/// snapshot. A rebirth rebuilds the existing table in place rather than replacing the +/// table object, so a consumer holding a reference to a scope's table always observes the newest +/// birth instead of quietly reading a detached older one. No lock is taken on any path — this is +/// read (and written) on MQTTnet's shared dispatcher thread. +/// +/// +/// Not a value store. See the remarks on : running values belong +/// to , keyed by RawPath. +/// +/// +public sealed class BirthCache +{ + private readonly ConcurrentDictionary _scopes = new(); + + /// How many scopes currently hold a live birth. + public int Count => _scopes.Count; + + /// A snapshot of the scopes currently holding a live birth. + public IReadOnlyList Scopes => [.. _scopes.Keys]; + + /// + /// Applies an NBIRTH: rebuilds the edge node's own catalog and invalidates every device + /// under it, per the Sparkplug spec. + /// + /// The Sparkplug group id. + /// The Sparkplug edge-node id. + /// The NBIRTH payload's metrics. + /// The node's apply result plus the devices this birth invalidated, with their metrics. + public NodeBirthOutcome ApplyNodeBirth(string groupId, string edgeNodeId, IEnumerable birthMetrics) + { + ArgumentException.ThrowIfNullOrEmpty(groupId); + ArgumentException.ThrowIfNullOrEmpty(edgeNodeId); + ArgumentNullException.ThrowIfNull(birthMetrics); + + // Devices first: a device table must never outlive the node birth that voided it, whatever the + // node's own rebuild does. + var invalidated = Evict(scope => scope.DeviceId is not null && scope.IsUnder(groupId, edgeNodeId)); + + var result = TableFor(new SparkplugScope(groupId, edgeNodeId, null)).RebuildFromBirth(birthMetrics); + return new NodeBirthOutcome(result, invalidated); + } + + /// Applies a DBIRTH: rebuilds exactly that device's catalog, wholesale. + /// The Sparkplug group id. + /// The Sparkplug edge-node id. + /// The Sparkplug device id. + /// The DBIRTH payload's metrics. + /// What the birth installed, and what it refused. + public BirthApplyResult ApplyDeviceBirth( + string groupId, + string edgeNodeId, + string deviceId, + IEnumerable birthMetrics) + { + ArgumentException.ThrowIfNullOrEmpty(groupId); + ArgumentException.ThrowIfNullOrEmpty(edgeNodeId); + ArgumentException.ThrowIfNullOrEmpty(deviceId); + ArgumentNullException.ThrowIfNull(birthMetrics); + + return TableFor(new SparkplugScope(groupId, edgeNodeId, deviceId)).RebuildFromBirth(birthMetrics); + } + + /// + /// The scope's catalog, or when no live birth has been seen for it — + /// which is the consumer's data-before-birth signal. + /// + /// The scope to look up. + /// The catalog, or . + public AliasTable? Find(SparkplugScope scope) => _scopes.TryGetValue(scope, out var table) ? table : null; + + /// + /// Applies an NDEATH: forgets the edge node and every device under it, returning what was + /// forgotten so the consumer can emit STALE for those metrics. + /// + /// The Sparkplug group id. + /// The Sparkplug edge-node id. + /// The evicted scopes and their metrics; empty when nothing was live. + public IReadOnlyList ForgetNode(string groupId, string edgeNodeId) + { + ArgumentException.ThrowIfNullOrEmpty(groupId); + ArgumentException.ThrowIfNullOrEmpty(edgeNodeId); + + return Evict(scope => scope.IsUnder(groupId, edgeNodeId)); + } + + /// + /// Applies a DDEATH: forgets exactly one device, returning its metrics for the STALE fan-out. + /// The node's own catalog is untouched. + /// + /// The Sparkplug group id. + /// The Sparkplug edge-node id. + /// The Sparkplug device id. + /// The evicted scope and its metrics when this returns . + /// when the device had a live birth to forget. + public bool TryForgetDevice( + string groupId, + string edgeNodeId, + string deviceId, + out SparkplugScopeMetrics removed) + { + ArgumentException.ThrowIfNullOrEmpty(groupId); + ArgumentException.ThrowIfNullOrEmpty(edgeNodeId); + ArgumentException.ThrowIfNullOrEmpty(deviceId); + + var scope = new SparkplugScope(groupId, edgeNodeId, deviceId); + if (_scopes.TryRemove(scope, out var table)) + { + removed = new SparkplugScopeMetrics(scope, table.Metrics); + return true; + } + + removed = default; + return false; + } + + /// Forgets every scope — the disconnect/reinitialise reset. + public void Clear() => _scopes.Clear(); + + /// + /// The scope's catalog, created empty on first use. Rebuilds happen in place on the + /// returned instance, so a consumer holding it observes the newest birth rather than a detached + /// older one. + /// + /// The scope. + /// The scope's catalog. + private AliasTable TableFor(SparkplugScope scope) => _scopes.GetOrAdd(scope, static _ => new AliasTable()); + + /// Removes every scope matching , capturing their metrics first. + /// Which scopes to evict. + /// The evicted scopes and the metrics they held. + private List Evict(Func predicate) + { + List? evicted = null; + + // ConcurrentDictionary enumeration is weakly consistent, which is exactly right here: this runs + // on the dispatcher thread and only needs to see the scopes that existed when the birth/death + // arrived. + foreach (var scope in _scopes.Keys) + { + if (!predicate(scope) || !_scopes.TryRemove(scope, out var table)) + { + continue; + } + + (evicted ??= []).Add(new SparkplugScopeMetrics(scope, table.Metrics)); + } + + return evicted ?? []; + } +} + +/// +/// A Sparkplug metric-namespace scope: an edge node, or one device under it. The identity every +/// birth, death and alias resolution is keyed by. +/// +/// The Sparkplug group id. +/// The Sparkplug edge-node id. +/// The Sparkplug device id, or for the node's own scope. +public readonly record struct SparkplugScope(string GroupId, string EdgeNodeId, string? DeviceId) +{ + /// Whether this is a device scope (a DBIRTH's) rather than an edge node's own. + public bool IsDevice => DeviceId is not null; + + /// The owning edge node's scope — this instance itself when it is already node-scoped. + public SparkplugScope NodeScope => DeviceId is null ? this : new SparkplugScope(GroupId, EdgeNodeId, null); + + /// Whether this scope belongs to the given edge node (the node's own scope included). + /// The Sparkplug group id. + /// The Sparkplug edge-node id. + /// when both segments match ordinally. + public bool IsUnder(string groupId, string edgeNodeId) => + string.Equals(GroupId, groupId, StringComparison.Ordinal) + && string.Equals(EdgeNodeId, edgeNodeId, StringComparison.Ordinal); + + /// + public override string ToString() => + DeviceId is null ? $"{GroupId}/{EdgeNodeId}" : $"{GroupId}/{EdgeNodeId}/{DeviceId}"; +} + +/// +/// A scope's metrics, captured at the moment it was evicted — the STALE fan-out's input, taken +/// before the eviction so it cannot race a re-birth of the same scope. +/// +/// The scope that was evicted. +/// The metrics its last birth declared. +public readonly record struct SparkplugScopeMetrics(SparkplugScope Scope, IReadOnlyList Metrics); + +/// The result of an NBIRTH: the node's own apply, plus the devices it invalidated. +/// What the node's catalog rebuild installed and refused. +/// +/// The device scopes this NBIRTH voided, with the metrics they held — the STALE fan-out's input +/// until each device re-DBIRTHs. +/// +public readonly record struct NodeBirthOutcome( + BirthApplyResult Result, + IReadOnlyList InvalidatedDevices); diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/RebirthRequester.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/RebirthRequester.cs new file mode 100644 index 00000000..8f4451cb --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/RebirthRequester.cs @@ -0,0 +1,146 @@ +using Google.Protobuf; +using Org.Eclipse.Tahu.Protobuf; +using TahuDataType = Org.Eclipse.Tahu.Protobuf.DataType; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// The one seam through which a Sparkplug NCMD is published — is the +/// production implementation once it grows a publish leg (Task 21); tests substitute a fake so the +/// encode/topic/QoS/retain contract is exercisable without a broker. Deliberately narrower than +/// "publish anything": a Sparkplug driver's only outbound traffic is NCMD/DCMD, so this is not a +/// general MQTT client abstraction in waiting. +/// +public interface IMqttPublishTransport +{ + /// + /// Publishes one application message. Implementations must be bounded — a broker that accepts + /// PUBLISH and never completes it (QoS > 0 awaiting PUBACK) must fail at a deadline, not + /// hang; supplies one, but an implementation used + /// directly by a future caller must not rely on that. + /// + /// The concrete topic to publish on. + /// The message body. + /// Requested QoS, 0–2. + /// The MQTT retain flag. + /// Caller cancellation. + Task PublishAsync(string topic, byte[] payload, int qos, bool retain, CancellationToken cancellationToken); +} + +/// +/// Encodes and publishes a Sparkplug B rebirth request — an NCMD carrying the well-known +/// Node Control/Rebirth boolean metric, addressed to one edge node's command topic. See +/// Sparkplug B v3.0 spec §6/§7 and design doc §3.6. +/// +/// +/// +/// is pure. It does no I/O and touches no transport, so it is +/// trivially unit-testable and safe to call from anywhere (the ingest state machine on a +/// detected gap, the browser on an operator click) without pulling in a live connection. The +/// bounded, transport-facing publish is the separate overload. +/// +/// +/// No seq. Sparkplug's sequence-number stream (§6.4.1) belongs to the messages an +/// edge node itself publishes (NBIRTH/NDATA/DBIRTH/DDATA/NDEATH/DDEATH) — it is how a host +/// detects a gap in that node's outbound stream. An NCMD flows the other direction, host +/// → node, and is not a member of the node's sequence at all: stamping one on here would not be +/// "wrong per spec" so much as meaningless, and a strict edge-node implementation that validated +/// it against its own counter would have grounds to reject the command outright. +/// +/// +/// QoS 0, retain — always. NCMD is ordinary Sparkplug command +/// traffic, published at the same QoS 0 the spec uses for NDATA/DDATA/NBIRTH/DBIRTH (only +/// NDEATH — the broker-issued Will — and STATE use QoS 1). is the one +/// that matters operationally: a retained NCMD would be replayed by the broker to the edge node +/// on every future subscribe — including the node's own reconnect — driving it into a +/// permanent rebirth loop from a single stale command. See the carry-forward landmine on +/// MqttDriverBrowser/NDEATH-as-Will for the sibling failure shape this type must not +/// introduce on the publish side. +/// +/// +/// Datatype and both timestamps are set. Node Control/Rebirth is Boolean; +/// setting explicitly (rather than leaving it for the +/// node to infer) and stamping both the metric and payload timestamp matches what a +/// well-behaved command payload carries per spec §6.4.13 and what the Task-25 simulator (and any +/// real Cirrus-Link-derived edge node) expects to validate against. An edge node that treats a +/// missing/ambiguous field as malformed and silently drops the command would otherwise leave the +/// driver retrying forever with no visible error — the whole point of a rebirth request is to +/// recover from exactly that kind of silent desync, so the request itself must not risk being +/// the next cause of one. +/// +/// +public static class RebirthRequester +{ + /// The well-known Sparkplug B node-control metric name that triggers a rebirth. + public const string RebirthMetricName = "Node Control/Rebirth"; + + /// + /// Default bounded deadline for when the caller does not supply one. + /// Deliberately this type's own value rather than MqttDriverOptions.ConnectTimeoutSeconds + /// — that knob already governs every MQTTnet connect/subscribe operation, and repurposing it + /// would couple the rebirth-publish budget to the connect budget in a way neither side could + /// tune independently (the same reasoning MqttConnection.SubscribeAsync documents for its + /// own deadline). + /// + public static readonly TimeSpan DefaultRequestTimeout = TimeSpan.FromSeconds(5); + + /// + /// Builds the wire bytes and target topic for a rebirth-request NCMD, without publishing it. + /// + /// The Sparkplug group id. + /// The Sparkplug edge-node id to address the command to. + /// The NCMD topic (spBv1.0/{groupId}/NCMD/{edgeNodeId}) and its encoded payload. + /// / is null or empty. + public static (string Topic, byte[] Bytes) Build(string groupId, string edgeNodeId) + { + // Validates both ids and does the topic assembly — see the "don't hand-concatenate" remark on + // SparkplugTopic.Format itself. + var topic = SparkplugTopic.Format(groupId, SparkplugMessageType.NCMD, edgeNodeId); + + var timestampMs = (ulong)DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(); + + var payload = new Payload { Timestamp = timestampMs }; + payload.Metrics.Add(new Payload.Types.Metric + { + Name = RebirthMetricName, + Datatype = (uint)TahuDataType.Boolean, + Timestamp = timestampMs, + BooleanValue = true, + }); + + return (topic, payload.ToByteArray()); + } + + /// + /// Builds and publishes a rebirth-request NCMD under a bounded deadline. QoS 0, retain + /// — see the type remarks for why both are fixed rather than + /// configurable. + /// + /// The publish seam — the live connection, or a test fake. + /// The Sparkplug group id. + /// The Sparkplug edge-node id to address the command to. + /// Caller cancellation; linked with the publish deadline. + /// + /// The publish deadline. Defaults to when omitted or + /// non-positive. + /// + /// / is null or empty. + /// is null. + public static async Task RequestAsync( + IMqttPublishTransport transport, + string groupId, + string edgeNodeId, + CancellationToken cancellationToken, + TimeSpan? timeout = null) + { + ArgumentNullException.ThrowIfNull(transport); + + var (topic, bytes) = Build(groupId, edgeNodeId); + + var deadlineSpan = timeout is { } t && t > TimeSpan.Zero ? t : DefaultRequestTimeout; + using var deadline = new CancellationTokenSource(deadlineSpan); + using var linked = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken, deadline.Token); + + await transport.PublishAsync(topic, bytes, qos: 0, retain: false, linked.Token).ConfigureAwait(false); + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SequenceTracker.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SequenceTracker.cs new file mode 100644 index 00000000..d614f05b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SequenceTracker.cs @@ -0,0 +1,386 @@ +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// Per-edge-node Sparkplug B stream-continuity state: the wrapping seq counter that says +/// whether the driver has missed a message, and the bdSeq session token that ties an +/// NDEATH to the NBIRTH it belongs to. Design doc §3.6 invariant #3. +/// +/// +/// +/// One instance per edge node — this object is the edge node's stream state. It +/// holds no key and no map. Sparkplug numbers messages per edge node (an edge node's devices +/// share its counter, so a DDATA advances the same sequence an NDATA does), and the ingest +/// state machine already keeps per-scope state for the alias table and birth cache; giving +/// this type a second internal dictionary would mean two maps whose lifetimes must be kept in +/// step by hand, and a tracker that outlived a removed edge node would go on answering +/// questions about a stream nobody is reading. +/// +/// +/// This type detects; it does not react. It never publishes, never requests a rebirth +/// and never emits a quality change — the ingest state machine reads its verdicts and decides. +/// The split matters because "was a message lost" is a fact about the wire, while "request a +/// rebirth now" is a policy gated on requestRebirthOnGap. +/// +/// +/// Thread-safe under a plain monitor. Sparkplug messages arrive on MQTTnet's shared +/// dispatcher thread while the reconnect path can call from another, and +/// every operation here is a read-modify-write of two coupled fields — the immutable-snapshot +/// discipline MqttSubscriptionManager uses fits a set that is swapped wholesale, not a +/// counter that is advanced. The critical sections are a handful of instructions each and +/// nothing inside them can block, so the simple lock this repo uses elsewhere +/// (MqttDriver._hostLock) is the right shape. +/// +/// +/// Nothing here throws, for any input. Same reasoning as +/// : this sits behind an unauthenticated firehose on the +/// dispatcher thread, so a spec-violating publisher gets a verdict, never an exception that +/// would stall delivery for every subscription on the connection. +/// +/// +public sealed class SequenceTracker +{ + /// + /// The largest legal Sparkplug seq. The counter is a byte on the wire in spirit but a + /// uint64 in the schema, so the range check lives here — see . + /// + public const ulong MaxSeq = 255; + + private readonly object _gate = new(); + + /// The last credible sequence number, meaningful only when . + private byte _lastSeq; + + /// Whether any credible sequence number has been observed since the last reset. + private bool _hasLastSeq; + + /// Whether the current baseline was established by a birth rather than adopted mid-stream. + private bool _birthSynchronized; + + /// The current session's bdSeq, or null when no birth has supplied one. + private ulong? _birthBdSeq; + + /// + /// Whether the current sequence baseline came from a birth. also after + /// a first message that was not a birth — the driver joined mid-stream and holds no + /// alias table, which is a different situation from a gap even though both call for a rebirth. + /// + public bool IsBirthSynchronized + { + get { lock (_gate) return _birthSynchronized; } + } + + /// + /// The last credible sequence number observed, or when none has been — + /// before the first message, or after . + /// + public byte? LastSeq + { + get { lock (_gate) return _hasLastSeq ? _lastSeq : null; } + } + + /// + /// The bdSeq of the most recent birth, or when no birth has + /// supplied one. This is the token matches against. + /// + public ulong? BirthBdSeq + { + get { lock (_gate) return _birthBdSeq; } + } + + /// + /// Records an NBIRTH: restarts the sequence at the birth's seq and adopts its + /// bdSeq as the current session token. + /// + /// + /// The birth payload's seq — , which is + /// when the message carried none. + /// + /// + /// The birth's session token, read from its bdSeq metric with + /// ; when the birth carried none. + /// + /// + /// when the birth established a usable sequence baseline; + /// when its seq was absent or out of range. + /// + /// + /// + /// An NBIRTH is never a gap. It restarts the sequence by definition, so it must not + /// be run through — doing so would flag a gap on the very message + /// that just resynchronized everything, and answer a birth with a request for another one. + /// + /// + /// A DBIRTH must NOT be offered here — it belongs to . Only the + /// NBIRTH restarts the sequence; a DBIRTH carries the next number in the edge node's + /// ongoing stream, and only the NBIRTH (with the NDEATH) carries bdSeq at all. + /// Routing a DBIRTH here would do two wrong things at once: silently rebase the sequence + /// mid-stream, and — because the DBIRTH has no bdSeq to pass — wipe the node's + /// session token to null. A stale Last Will arriving afterwards would then find + /// nothing to be compared against, fall through the fail-toward-stale rule in + /// , and kill the live node this type exists to + /// protect. The method is named for the node deliberately, so that call site reads wrong. + /// + /// + /// An in-range seq other than the spec-required 0 is adopted, not refused. + /// Refusing it would leave the tracker unsynchronized against a publisher whose stream is + /// otherwise perfectly followable, and every message that followed would demand a fresh + /// rebirth — the storm this type exists to prevent, arrived at by being strict. + /// + /// + /// The bdSeq is recorded even when the seq was unusable. Session + /// identity and position-in-stream are independent facts; discarding the tie because the + /// sequence number was garbled would leave the next NDEATH unmatchable, which is the one + /// thing that lets a stale will kill a live node. + /// + /// + public bool AcceptNodeBirth(ulong? seq, ulong? bdSeq = null) + { + lock (_gate) + { + _birthBdSeq = bdSeq; + + if (seq is not { } value || value > MaxSeq) + { + _hasLastSeq = false; + _birthSynchronized = false; + return false; + } + + _lastSeq = (byte)value; + _hasLastSeq = true; + _birthSynchronized = true; + return true; + } + } + + /// + /// Offers the next sequenced message's seq — DBIRTH, NDATA, DDATA or DDEATH — + /// and reports whether the stream is still contiguous. + /// + /// + /// The payload's seq — , which is + /// when the message carried none. + /// + /// + /// when this seq is the one that follows the last; + /// when a message was missed, when the value is out of range, or when + /// it is absent. A is the caller's cue to request a rebirth. + /// + /// + /// + /// The wrap is the whole point: next-expected is (last + 1) & 0xFF, so + /// 255 → 0 is the sequence continuing, not a gap. Getting the boundary wrong fails + /// loudly in one direction and silently in the other. Read the wrap as a gap and every + /// edge node is asked to rebirth once per 256 messages, so a busy node spends its life + /// republishing births instead of publishing data. Miss a real gap and the driver serves + /// values it knows are behind the device, at Good quality, indefinitely. + /// + /// + /// A gap is reported once, then the tracker resynchronizes onto the observed value. + /// Holding the baseline at the pre-gap number would make every following message report a + /// gap too, so a single lost message would become a permanent rebirth storm. + /// + /// + /// An out-of-range seq is refused, never masked into range. The codec hands + /// this over as a precisely so a spec-violating publisher's value is + /// not silently truncated into a plausible-looking one, and masking would finish the job + /// it declined to do: with the baseline at 255, a seq of 256 masks to 0 — exactly + /// the value the wrap expects next — and the bogus message would be accepted as + /// contiguous. Refused values do not become the baseline either, so the next genuine + /// message is still measured against the last credible one. + /// + /// + /// The first message is not a gap — there is nothing to measure it against. It is + /// adopted as the baseline, and stays + /// to record that the driver joined mid-stream. + /// + /// + /// NDEATH must not be offered here. An NDEATH is the broker-published Last Will and + /// carries no seq at all; feeding it in would report a gap on a message that never + /// had a place in the sequence. It is tied to its birth by + /// instead. A DDEATH is published by the edge node + /// and is sequenced, so it does belong here. + /// + /// + public bool Accept(ulong? seq) + { + lock (_gate) + { + // Refused without becoming the baseline: see the out-of-range remarks above. + if (seq is not { } value || value > MaxSeq) + { + return false; + } + + var current = (byte)value; + + if (!_hasLastSeq) + { + _lastSeq = current; + _hasLastSeq = true; + return true; + } + + var expected = (byte)((_lastSeq + 1) & 0xFF); + + // Resynchronize even on a gap — one lost message must not become a permanent storm. + _lastSeq = current; + + return current == expected; + } + } + + /// + /// Reports whether an NDEATH belongs to the session currently believed alive — the + /// bdSeq tie of §3.6 invariant #3. + /// + /// + /// The death payload's bdSeq, read with ; + /// when it carried none. + /// + /// + /// when the death applies and the node's metrics should go stale; + /// when it is a previous session's will and must be ignored. + /// + /// + /// + /// What this prevents. An edge node's connection drops, it reconnects and publishes + /// a fresh NBIRTH — and only then does the broker notice the old connection died + /// and deliver its retained Last Will. The NDEATH arrives after the NBIRTH, carrying the + /// previous session's bdSeq. Without this comparison that stale will marks a live, + /// freshly-born node dead and drives every one of its tags to Bad quality, with no further + /// message coming to correct it until the node happens to rebirth again. + /// + /// + /// Unknowns fail toward stale, deliberately. A death arriving before any birth was + /// seen, a death carrying no bdSeq, or a birth that carried none all resolve to + /// . The two error directions are not symmetric: acting on a death + /// wrongly marks tags stale until the next birth restores them (§3.6 invariant #4, a + /// self-correcting error an operator can see), while ignoring a real death leaves a dead + /// node's last values flowing at Good quality forever. Only a positive mismatch — + /// two known, different tokens — is evidence enough to discard a death. + /// + /// + /// Pure. It answers a question and changes nothing, so the caller can ask it while + /// logging or deciding. A death that is acted on needs no reset here: the node is + /// offline, and the next birth calls which restarts the sequence + /// anyway. + /// + /// + public bool IsDeathForCurrentSession(ulong? deathBdSeq) + { + lock (_gate) + { + if (_birthBdSeq is not { } birth || deathBdSeq is not { } death) + { + return true; + } + + return birth == death; + } + } + + /// + /// Returns the tracker to its virgin state: no sequence baseline, not birth-synchronized, no + /// session token. + /// + /// + /// The reconnect seam (§3.6 invariant #5 — late join). After a dropped connection the driver + /// has missed an unknown number of messages, so the baseline it was holding is no longer + /// evidence of anything: carrying it across would let the stream look contiguous purely + /// because the edge node happened to publish the right number of messages while the driver was + /// away. + /// + public void Reset() + { + lock (_gate) + { + _lastSeq = 0; + _hasLastSeq = false; + _birthSynchronized = false; + _birthBdSeq = null; + } + } + + /// + /// Reads the bdSeq session token out of a decoded NBIRTH or NDEATH payload. + /// + /// The decoded payload; and invalid are both handled. + /// The token on success; 0 otherwise. + /// when the payload carried a usable bdSeq metric. + /// + /// + /// bdSeq is not a top-level payload field — unlike seq it rides as an + /// ordinary metric named bdSeq, which is why reading it is a helper rather than a + /// property. The name is matched exactly (ordinal): the spec fixes it, so a case variant + /// is far more likely to be a different metric than a typo of this one. + /// + /// + /// Signed values are reinterpreted before they are judged. Sparkplug carries + /// Int64 as two's complement in an unsigned field, so a negative bdSeq + /// reaches the projection as an enormous . Read raw it would become a + /// plausible-looking session token minted out of nonsense; run through + /// it is visibly negative and refused. + /// + /// + /// The value is treated as an opaque token, not as a 0–255 counter. Its only job + /// here is equality against the birth's, and a range check could never make a match more + /// or less correct — it could only discard the one piece of evidence tying a death to its + /// birth. This is the opposite call from seq, whose range is load-bearing precisely + /// because its arithmetic wraps. + /// + /// + public static bool TryReadBdSeq(SparkplugPayload? payload, out ulong bdSeq) + { + bdSeq = 0; + + if (payload is not { IsValid: true }) + { + return false; + } + + foreach (var metric in payload.Metrics) + { + if (!string.Equals(metric.Name, "bdSeq", StringComparison.Ordinal)) + { + continue; + } + + if (metric.ValueKind != SparkplugValueKind.Scalar) + { + return false; + } + + var value = metric.DataType is { } datatype + ? SparkplugCodec.ReinterpretSigned(metric.Value, datatype) + : metric.Value; + + switch (value) + { + case ulong u: + bdSeq = u; + return true; + case uint u: + bdSeq = u; + return true; + case long l when l >= 0: + bdSeq = (ulong)l; + return true; + case int i when i >= 0: + bdSeq = (ulong)i; + return true; + case short s when s >= 0: + bdSeq = (ulong)s; + return true; + case sbyte sb when sb >= 0: + bdSeq = (ulong)sb; + return true; + default: + // Negative, or a wire type no integer token can come from. + return false; + } + } + + return false; + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugCodec.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugCodec.cs new file mode 100644 index 00000000..b428ee19 --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugCodec.cs @@ -0,0 +1,313 @@ +using Google.Protobuf; +using Org.Eclipse.Tahu.Protobuf; +using TahuDataType = Org.Eclipse.Tahu.Protobuf.DataType; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// Decodes Sparkplug-B wire bytes into — the driver-side projection +/// the ingest state machine consumes. Decode only; the NCMD encode path lives in +/// RebirthRequester. +/// +/// +/// +/// Nothing here throws, for any input. This runs on MQTTnet's shared dispatcher thread, +/// behind an unauthenticated firehose the plant's edge nodes publish into. An escaping +/// exception would not degrade one tag — it would stall or kill delivery for every +/// subscription on the connection. Garbage bytes, a truncated body, a zero-length payload, a +/// valid protobuf of some other schema and a pathologically nested Template all resolve to a +/// verdict: returns and +/// returns . +/// +/// +/// Zero-length input is invalid, not empty. Protobuf would happily parse zero bytes as +/// "a Payload with every field defaulted", but in Sparkplug an empty MQTT body is never a +/// legitimate message — treating it as a well-formed payload carrying no metrics is how a +/// truncated-to-nothing body gets mistaken for a real one. It is reported as invalid. +/// +/// +/// Explicit presence is preserved, everywhere. The vendored schema is proto2 precisely +/// so absence and zero stay distinguishable, and every projection below reads +/// Has{Seq,Name,Alias,Datatype,Timestamp,IsNull} rather than testing a value against its +/// default. Two cases make this load-bearing rather than pedantic: a Sparkplug NBIRTH is +/// REQUIRED to carry seq = 0, and every DATA metric after a birth carries an alias with +/// no name and no datatype. A zero-check decoder reports "no sequence" for every +/// birth and "" for every DATA metric name. +/// +/// +/// Values are projected raw — nothing is coerced or reinterpreted here. A metric's value +/// arrives as the CLR type of the wire field that carried it (see +/// ); coercion against the authored tag's declared +/// DriverDataType is the consumer's job, and follows this driver's standing rule that a +/// value which does not fit is refused rather than silently converted. The one wire-level +/// translation this type does offer is , because Sparkplug's +/// two's-complement-in-an-unsigned-field encoding of signed integers is a property of the +/// wire, not of the tag. +/// +/// +/// DataSet and Template metrics are out of scope for v1. They decode to +/// with a null value — deliberately visible, so a +/// consumer can warn about a metric it cannot serve instead of silently treating it as one +/// that was never published. +/// +/// +public static class SparkplugCodec +{ + /// Decodes Sparkplug-B wire bytes, reporting success rather than throwing. + /// The MQTT message body. + /// + /// The decoded payload on success; otherwise. Never + /// . + /// + /// + /// when the bytes parsed as a Sparkplug-B Payload. Protobuf is a + /// permissive, self-describing-only-by-convention format, so this means "parsed" rather than + /// "was genuinely produced by a Sparkplug edge node" — unknown fields are preserved by + /// Google.Protobuf and simply ignored here. + /// + public static bool TryDecode(ReadOnlySpan wire, out SparkplugPayload payload) + { + payload = SparkplugPayload.Invalid; + + if (wire.IsEmpty) + { + return false; + } + + try + { + var proto = Payload.Parser.ParseFrom(wire); + + var metrics = proto.Metrics.Count == 0 + ? [] + : new SparkplugMetric[proto.Metrics.Count]; + + for (var i = 0; i < proto.Metrics.Count; i++) + { + metrics[i] = ProjectMetric(proto.Metrics[i]); + } + + payload = new SparkplugPayload( + IsValid: true, + Seq: proto.HasSeq ? proto.Seq : null, + TimestampMs: proto.HasTimestamp ? proto.Timestamp : null, + Metrics: metrics); + + return true; + } + catch (Exception) + { + // Deliberately broad, and deliberately spanning the projection as well as the parse. + // InvalidProtocolBufferException covers malformed, truncated and over-nested input, but + // this is the dispatcher-thread boundary and the projection walks an attacker-shaped + // object graph: the contract is a verdict for EVERY input, so a guard that stopped at + // the parse would be a contract with a hole in it. Narrowing the catch would trade the + // guarantee for a taxonomy nobody downstream can act on. + payload = SparkplugPayload.Invalid; + return false; + } + } + + /// + /// Decodes Sparkplug-B wire bytes, returning when they + /// cannot be decoded. + /// + /// The MQTT message body. + /// The decoded payload — never . + /// + /// The convenience shape. Check : an undecodable + /// body and a genuinely metric-less one both present as an empty + /// , and only the flag tells them apart. Prefer + /// where the verdict drives control flow. + /// + public static SparkplugPayload Decode(ReadOnlySpan wire) => + TryDecode(wire, out var payload) ? payload : SparkplugPayload.Invalid; + + /// + /// Reinterprets a raw metric value as the signed integer Sparkplug encoded it as, when its + /// datatype is a signed integer type; returns the value unchanged for every other datatype. + /// + /// + /// A — a for the 8/16/32-bit arms, a + /// for the 64-bit arm. + /// + /// The metric's Sparkplug datatype, from its birth or its own field. + /// + /// / / / for the + /// signed integer datatypes; untouched otherwise (including when it is + /// or not the wire type the datatype implies). + /// + /// + /// Sparkplug carries Int8/Int16/Int32 in the unsigned + /// int_value field and Int64 in the unsigned long_value field, as two's + /// complement. Handing the raw field to a consumer publishes 4294967254 for a tag whose + /// value is -42 — a wrong value that looks entirely plausible, arrives with Good + /// quality, and is invisible until someone reads a gauge. This is the single call that undoes + /// it, and it is separate from the decode itself because a DATA metric carries no datatype of + /// its own: only the consumer, holding the alias table built from the birth, knows which + /// datatype applies. + /// + public static object? ReinterpretSigned(object? value, TahuDataType datatype) => datatype switch + { + TahuDataType.Int8 when value is uint raw => unchecked((sbyte)raw), + TahuDataType.Int16 when value is uint raw => unchecked((short)raw), + TahuDataType.Int32 when value is uint raw => unchecked((int)raw), + TahuDataType.Int64 when value is ulong raw => unchecked((long)raw), + _ => value, + }; + + /// Projects one generated metric onto the driver-side shape, presence intact. + /// The generated metric. + /// The projection. + private static SparkplugMetric ProjectMetric(Payload.Types.Metric metric) + { + var (kind, value) = ProjectValue(metric); + + return new SparkplugMetric( + Name: metric.HasName ? metric.Name : null, + Alias: metric.HasAlias ? metric.Alias : null, + + // Cast, never filter: an index this build's enum does not define is a future Sparkplug + // revision or a misbehaving publisher, and dropping it would deny the mapping layer the + // only evidence it has. + DataType: metric.HasDatatype ? (TahuDataType)metric.Datatype : null, + TimestampMs: metric.HasTimestamp ? metric.Timestamp : null, + ValueKind: kind, + Value: value); + } + + /// Resolves a metric's value oneof to a kind plus a raw CLR value. + /// The generated metric. + /// The value kind and the projected value. + /// + /// is_null wins over the oneof: the Sparkplug spec's whole reason for the field is + /// that some datatypes have no spare sentinel, so a publisher that sets it means "null" even if + /// a value field is also populated. + /// + private static (SparkplugValueKind Kind, object? Value) ProjectValue(Payload.Types.Metric metric) + { + if (metric.HasIsNull && metric.IsNull) + { + return (SparkplugValueKind.Null, null); + } + + return metric.ValueCase switch + { + Payload.Types.Metric.ValueOneofCase.IntValue => (SparkplugValueKind.Scalar, metric.IntValue), + Payload.Types.Metric.ValueOneofCase.LongValue => (SparkplugValueKind.Scalar, metric.LongValue), + Payload.Types.Metric.ValueOneofCase.FloatValue => (SparkplugValueKind.Scalar, metric.FloatValue), + Payload.Types.Metric.ValueOneofCase.DoubleValue => (SparkplugValueKind.Scalar, metric.DoubleValue), + Payload.Types.Metric.ValueOneofCase.BooleanValue => (SparkplugValueKind.Scalar, metric.BooleanValue), + Payload.Types.Metric.ValueOneofCase.StringValue => (SparkplugValueKind.Scalar, metric.StringValue), + + // Copied out of the ByteString: the projection must outlive the parsed message. + Payload.Types.Metric.ValueOneofCase.BytesValue => + (SparkplugValueKind.Scalar, metric.BytesValue.ToByteArray()), + + // v1 scope boundary — decoded as a visible refusal, not as an exception or a silent drop. + Payload.Types.Metric.ValueOneofCase.DatasetValue => (SparkplugValueKind.Unsupported, null), + Payload.Types.Metric.ValueOneofCase.TemplateValue => (SparkplugValueKind.Unsupported, null), + Payload.Types.Metric.ValueOneofCase.ExtensionValue => (SparkplugValueKind.Unsupported, null), + + _ => (SparkplugValueKind.Absent, null), + }; + } +} + +/// +/// A decoded Sparkplug-B payload: the driver-side projection of the generated Payload, +/// carrying only what the ingest state machine consumes. +/// +/// +/// for a payload that could not be decoded. An invalid payload also has an +/// empty list, so this flag is the only thing distinguishing an +/// undecodable body from a legitimately metric-less one. +/// +/// +/// The payload sequence number, or when the message carried none. Kept as +/// the wire's rather than narrowed to a byte: the Sparkplug range is 0–255, but +/// a publisher that violates it is reporting a fact the consumer should be able to see and reject, +/// not one this layer should silently truncate into a plausible-looking sequence number. +/// +/// The payload timestamp in Sparkplug epoch milliseconds, or null when absent. +/// The payload's metrics, in wire order. Never . +public sealed record SparkplugPayload( + bool IsValid, + ulong? Seq, + ulong? TimestampMs, + IReadOnlyList Metrics) +{ + /// The shared "could not decode this" result. + public static readonly SparkplugPayload Invalid = new(false, null, null, []); +} + +/// One decoded Sparkplug metric. +/// +/// The metric's stable name, or when the message omitted it — which every +/// real DATA metric after a birth does, carrying only . Distinguishing +/// this from an empty string is the reason the vendored schema is proto2. +/// +/// The per-birth alias, or when the metric carried none. +/// +/// The metric's declared datatype, or when absent (again, the normal case +/// for a DATA metric — its datatype comes from the birth). An index this build's enum does not +/// define is preserved as an undefined enum value rather than dropped. +/// +/// +/// The metric's own acquisition timestamp in Sparkplug epoch milliseconds, or +/// when it carried none — in which case the payload's timestamp applies. +/// +/// +/// What means. Check this before reading the value: a null value is +/// ambiguous between "explicitly null", "absent" and "a kind v1 does not support". +/// +/// +/// The raw wire value, boxed: (int_value), +/// (long_value), , , , +/// or [] — and for every +/// other than . +/// +/// Signed integers arrive as their unsigned two's-complement wire value — a +/// DataType.Int32 metric holding -42 is a of 4294967254 here. Run it +/// through with the metric's datatype (from the +/// birth, for a DATA metric) before publishing it. +/// +/// +/// Boxing is deliberate: the consumer coerces against the authored tag's declared +/// DriverDataType and hands the result to a DataValueSnapshot, which boxes +/// anyway, so a discriminated-union shape would buy an unboxing hop and cost every consumer a +/// switch over a dozen arms. +/// +/// +public readonly record struct SparkplugMetric( + string? Name, + ulong? Alias, + TahuDataType? DataType, + ulong? TimestampMs, + SparkplugValueKind ValueKind, + object? Value); + +/// What a decoded metric's represents. +/// +/// Three of the four members have a null and mean entirely +/// different things, which is exactly why the distinction is carried explicitly rather than left +/// for a consumer to infer from a null. +/// +public enum SparkplugValueKind +{ + /// The metric's value oneof carried nothing at all. + Absent = 0, + + /// The metric set is_null: it exists, and its value is explicitly null. + Null, + + /// holds the raw wire value. + Scalar, + + /// + /// A DataSet, Template or extension value — decoded and reported, but not + /// supported in v1. The consumer should warn and skip the metric rather than treat it as + /// missing. + /// + Unsupported, +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugIngestor.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugIngestor.cs new file mode 100644 index 00000000..cb14d71f --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugIngestor.cs @@ -0,0 +1,2100 @@ +using System.Collections.Concurrent; +using System.Collections.Frozen; +using System.Globalization; +using System.Text; +using System.Text.Json; +using Microsoft.Extensions.Logging; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// The Sparkplug B ingest state machine — design doc §3.6, the driver's correctness core. It owns +/// the authored (group, node, device?, metric) → RawPath index, the live +/// , one per edge node, and the rebirth +/// policy; it turns every decoded NBIRTH/DBIRTH/NDATA/DDATA/NDEATH/DDEATH/STATE into +/// notifications plus updates, or into a +/// bounded, debounced rebirth NCMD. +/// +/// +/// +/// The published reference is the RawPath. Every this +/// type raises carries — the tag's RawPath — never a +/// composed Sparkplug reference such as Plant1/EdgeA/Filler1:Temperature and never the +/// topic. DriverHostActor's dual-namespace fan-out is keyed by RawPath, so publishing +/// under a Sparkplug-shaped key produces a driver that satisfies a naive unit test and delivers +/// nothing at all in production. The Sparkplug tuple is the lookup; the RawPath is the +/// identity. +/// +/// +/// Bind by metric NAME, resolve by alias (§3.6 #1). A DATA metric carries only an alias; +/// the alias is resolved through the scope's — rebuilt wholesale by +/// every birth — back to the stable metric name, and that name is what the authored index +/// is keyed by. Binding a tag to an alias silently mis-routes Pressure into a Temperature tag +/// the first time an edge node reuses an alias across a rebirth: good quality, plausible values, +/// completely wrong. +/// +/// +/// Every signed integer goes through . +/// Sparkplug carries Int8/16/32/64 as two's complement in an unsigned proto field, so a +/// metric whose value is -42 arrives as 4294967254. Skipping the call publishes a plausible, +/// Good-quality, completely wrong number. +/// +/// +/// An invalid payload never mutates state. An undecodable body advances no sequence +/// tracker, clears no alias table and evicts no birth — including for an NDEATH. Garbage bytes +/// are not evidence of anything, and a hostile or merely broken publisher on a group-wide +/// # subscription must not be able to blank the address space with them. +/// +/// +/// NDEATH is not part of the sequence. It is the broker-published Last Will and carries +/// no seq; it is tied to its birth by +/// instead. Feeding it to reports a gap on every death and +/// answers a node that has just died with a rebirth request it can never serve. +/// +/// +/// Rebirth requests are debounced per edge node. A single lost message on a busy node +/// produces one unknown-alias or data-before-birth event per metric per message until the birth +/// lands; without a floor between requests that is the NCMD storm the design warns about. See +/// . +/// +/// +/// Unauthored edge nodes are ignored outright. The driver subscribes +/// spBv1.0/{group}/#, so a large plant delivers traffic for every node in the group. Only +/// messages whose (group, edgeNode) appears in the authored index are processed at all — +/// which is what bounds the sequence-tracker and birth-cache maps by configuration rather +/// than by whatever the plant happens to publish. Device-level filtering happens later, at the +/// metric-resolution step, because an unauthored device's DDATA still advances its edge node's +/// sequence and dropping it early would manufacture a gap. +/// +/// +/// Nothing here throws on MQTTnet's dispatcher thread. runs on +/// it: every failure degrades a tag or drops a message, a throwing +/// subscriber is contained, and the rebirth publish is dispatched off-thread. +/// +/// +/// Primary-host STATE publishing is deliberately NOT implemented — see +/// and the remarks on +/// . This type observes STATE; it never publishes one. +/// +/// +public sealed class SparkplugIngestor +{ + /// + /// Minimum interval between two rebirth NCMDs aimed at the same edge node, when the caller does + /// not supply one. Sized against the observation that a rebirth is a full metadata republish an + /// edge node needs a moment to assemble: repeatedly commanding a node that is already + /// republishing is the storm, not the cure. + /// + public static readonly TimeSpan DefaultRebirthDebounce = TimeSpan.FromSeconds(10); + + // OPC UA status codes (values verified against Opc.Ua.StatusCodes, not recalled). + private const uint StatusGood = 0x00000000u; + + /// + /// The source has announced it is offline (NDEATH/DDEATH), or a birth voided the scope that was + /// feeding the tag. BadCommunicationError rather than the arguably more literal + /// BadNoCommunication (0x80310000) because it is what this repo's drivers actually + /// produce for a link that has stopped delivering — Modbus, S7, TwinCAT, FOCAS, + /// OpcUaClient and this driver's own plain-mode path all emit it, while no driver emits + /// BadNoCommunication at all (it appears once, in a CLI formatter's label map). Following the + /// precedent keeps one code meaning "not communicating" across the whole fleet. + /// + private const uint StatusBadCommunicationError = 0x80050000u; + + /// The metric's value does not fit the tag's effective data type — including an explicit Sparkplug null. + private const uint StatusBadTypeMismatch = 0x80740000u; + + /// The subscribed reference is not an authored Sparkplug tag. + private const uint StatusBadNodeIdUnknown = 0x80340000u; + + /// + /// Hard ceiling on . Every key fed to is derived + /// from authored configuration, so the set is bounded by construction — this is the belt to that + /// braces. A key set that grows off wire input is a memory leak on the dispatcher path, + /// and getting one of the derivations subtly wrong later must degrade to "stops warning" rather + /// than to unbounded growth. + /// + private const int MaxWarnKeys = 1024; + + /// + /// How many consecutive metadata rebirths one edge node may be asked for before the + /// driver gives up and says so. Bounds the rebirth loop the per-node debounce cannot break: the + /// debounce caps the RATE, not the LIFETIME, so a node whose birth this driver can never use — + /// one over , or one that simply never comes — would otherwise be + /// commanded every debounce period for the life of the process, answering each time with the + /// same unusable payload. + /// + private const int MaxConsecutiveMetadataRebirths = 3; + + private readonly string _driverId; + private readonly ILogger? _logger; + private readonly TimeSpan _rebirthDebounce; + private readonly Func _utcNow; + + /// RawPath → the handle of the subscription currently covering it. + private readonly ConcurrentDictionary _handleByRawPath = new(StringComparer.Ordinal); + + /// Handle → the references it was created for, so unsubscribe can undo exactly its own. + private readonly ConcurrentDictionary _refsByHandle = new(); + + /// One stream-continuity tracker per authored edge node; bounded by the authored set. + private readonly ConcurrentDictionary _trackers = new(); + + /// Last rebirth-NCMD instant per edge node, in UTC ticks — the debounce floor. + private readonly ConcurrentDictionary _lastRebirthTicks = new(); + + /// + /// Edge nodes whose NBIRTH has been seen in the current session, whether or not its + /// seq was readable. Distinct from , + /// which additionally requires a usable seq — see . + /// + private readonly ConcurrentDictionary _nodeBirthSeen = new(); + + /// + /// Consecutive metadata rebirths requested from an edge node since its last applied + /// birth. Capped by — see + /// . + /// + private readonly ConcurrentDictionary _metadataRebirths = new(); + + /// Keys whose anomaly has already been logged loudly once, cleared by . + private readonly ConcurrentDictionary _warned = new(StringComparer.Ordinal); + + /// In-flight rebirth publishes, so teardown and tests can drain rather than race them. + private readonly List _pendingRebirths = []; + + private readonly EquipmentTagRefResolver _resolver; + + private long _handleSeq; + + /// The authored table + its Sparkplug indexes, swapped atomically by . + private volatile AuthoredTable _authored = AuthoredTable.Empty; + + private readonly MqttDriverOptions _options; + private volatile SparkplugHostState? _hostState; + private IMqttSubscribeTransport? _subscribeTransport; + private IMqttPublishTransport? _publishTransport; + + /// Initializes a new instance of the class. + /// + /// Driver configuration. supplies the group id, the + /// host id and the rebirth policy; a section is treated as the defaults. + /// + /// Driver instance id, used only for log correlation. + /// + /// The SUBSCRIBE seam. records the group filter without dialling a broker + /// (the shape tests use); supplies the live connection. + /// + /// + /// The NCMD publish seam. disables rebirth requests, which are then + /// logged and skipped rather than throwing on the dispatcher thread. + /// + /// Optional logger; never receives credentials or payload bodies. + /// The last-value store backing IReadable; defaults to a fresh one. + /// + /// Ceiling on an inbound message body. Non-positive falls back to + /// — "unbounded" is not offered, + /// because the bound protects a shared dispatcher thread. + /// + /// + /// Minimum interval between rebirth NCMDs aimed at one edge node. Defaults to + /// ; disables the floor (tests). + /// + /// Clock seam for the debounce; defaults to . + public SparkplugIngestor( + MqttDriverOptions options, + string driverId, + IMqttSubscribeTransport? subscribeTransport = null, + IMqttPublishTransport? publishTransport = null, + ILogger? logger = null, + LastValueCache? cache = null, + int maxPayloadBytes = MqttSubscriptionManager.DefaultMaxPayloadBytes, + TimeSpan? rebirthDebounce = null, + Func? utcNow = null) + { + ArgumentNullException.ThrowIfNull(options); + ArgumentException.ThrowIfNullOrWhiteSpace(driverId); + + _options = options; + _driverId = driverId; + _subscribeTransport = subscribeTransport; + _publishTransport = publishTransport; + _logger = logger; + Values = cache ?? new LastValueCache(); + MaxPayloadBytes = maxPayloadBytes > 0 ? maxPayloadBytes : MqttSubscriptionManager.DefaultMaxPayloadBytes; + _rebirthDebounce = rebirthDebounce is { } d && d >= TimeSpan.Zero ? d : DefaultRebirthDebounce; + _utcNow = utcNow ?? (() => DateTime.UtcNow); + _resolver = new EquipmentTagRefResolver( + rawPath => _authored.ByRawPath.TryGetValue(rawPath, out var def) ? def : null); + } + + /// + public event EventHandler? OnDataChange; + + /// + /// Raised after a birth certificate has been applied — the seam Task 22 wires to + /// IRediscoverable.OnRediscoveryNeeded so a DBIRTH introducing metrics the previous birth + /// did not carry rebuilds the address space. Deliberately not wired here: this type detects, the + /// driver decides. + /// + public event EventHandler? BirthObserved; + + /// Raised when a primary-host STATE message is observed. Informational; gates nothing. + public event EventHandler? HostStateObserved; + + /// The last observed value per RawPath — the push→poll bridge IReadable serves from. + public LastValueCache Values { get; } + + /// The live birth state: one per observed scope. Read by Tasks 22/23. + public BirthCache Births { get; } = new(); + + /// Ceiling on an inbound message body; a larger message is dropped before any decode. + public int MaxPayloadBytes { get; } + + /// + /// The last observed primary-host STATE, or when none has been seen. + /// + public SparkplugHostState? HostState => _hostState; + + /// The group-wide filter this ingestor subscribes, or when no group is configured. + public string? GroupFilter + { + get + { + var group = _options.Sparkplug?.GroupId; + return string.IsNullOrWhiteSpace(group) ? null : $"spBv1.0/{group}/#"; + } + } + + /// + /// The primary-host STATE filter this ingestor subscribes, or when no host + /// id is configured. Deliberately narrowed to the configured host id rather than + /// spBv1.0/STATE/#: this driver has no use for a stranger's host state, and a wildcard + /// would grow an unbounded observation map off traffic nobody asked for. + /// + public string? StateFilter + { + get + { + var hostId = _options.Sparkplug?.HostId; + return string.IsNullOrWhiteSpace(hostId) ? null : SparkplugTopic.FormatState(hostId); + } + } + + /// + /// The pre-3.0 primary-host STATE filter (STATE/{hostId}, no namespace prefix), or + /// when no host id is configured. Subscribed alongside + /// so the legacy form this driver already parses can actually arrive. + /// + public string? LegacyStateFilter + { + get + { + var hostId = _options.Sparkplug?.HostId; + return string.IsNullOrWhiteSpace(hostId) ? null : $"STATE/{hostId}"; + } + } + + /// + /// Wires this ingestor onto a live connection: inbound messages route to + /// , the connection becomes both transports, and every reconnect runs + /// (re-subscribe + birth-state reset + late-join rebirth). + /// + /// The connection to observe. + public void AttachTo(MqttConnection connection) + { + ArgumentNullException.ThrowIfNull(connection); + + // Binding to a session means the previous one is gone. Reset FIRST, before delivery is wired: + // with a persistent session a broker may push queued messages the moment CONNACK lands, ahead + // of our own SUBSCRIBE, and those must not resolve against the old session's aliases. + ResetSessionState("attach to a new broker session"); + + _subscribeTransport = connection; + _publishTransport = connection; + connection.MessageReceived += HandleMessage; + connection.Reconnected += OnReconnectedAsync; + } + + /// + /// Replaces the authored tag table with the deploy's raw tags, rebuilding the Sparkplug indexes. + /// Wholesale, not additive: a redeploy that drops a tag must stop feeding it. + /// + /// + /// A redeploy that ADDS an edge node does not rebirth it here. Until that deploy the node + /// was unauthored, so its traffic was filtered out and the driver holds no birth for it. It + /// recovers on the node's next DATA message, which lands as data-before-birth and asks for one — + /// self-correcting, and deliberately not a fresh NCMD fan-out on every redeploy, which on a large + /// plant would make a routine config change a group-wide rebirth storm. + /// + /// The authored raw tags delivered by the deploy artifact. + /// How many entries mapped to a usable Sparkplug definition. + public int Register(IEnumerable rawTags) + { + ArgumentNullException.ThrowIfNull(rawTags); + + var byRawPath = new Dictionary(StringComparer.Ordinal); + foreach (var entry in rawTags) + { + if (MqttTagDefinitionFactory.FromSparkplugTagConfig(entry.TagConfig, entry.RawPath, out var def)) + { + byRawPath[entry.RawPath] = def; + } + else + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': tag config did not map to a Sparkplug definition " + + "(groupId/edgeNodeId/metricName are all required); skipping. RawPath={RawPath}", + _driverId, + entry.RawPath); + } + } + + var table = AuthoredTable.Build(byRawPath); + _authored = table; + _warned.Clear(); + + // Drop state belonging to edge nodes this deploy removed. Without this a tracker (and its + // rebirth debounce slot) for a node nobody reads survives every future redeploy. + foreach (var key in _trackers.Keys) + { + if (!table.EdgeNodes.Contains(key)) + { + _trackers.TryRemove(key, out _); + _lastRebirthTicks.TryRemove(key, out _); + _nodeBirthSeen.TryRemove(key, out _); + _metadataRebirths.TryRemove(key, out _); + } + } + + // …and their BIRTHS, which is the half that actually mis-routes. While an edge node is + // unauthored its traffic is dropped at Dispatch, so its cached alias table stops refreshing — + // it FREEZES rather than going stale-and-obvious. Deploy A authors Plant1/EdgeA/Filler1, + // deploy B drops it, deploy C re-adds it a week later having missed every rebirth in between: + // Births.Find still answers with the week-old table and the next DDATA publishes the wrong + // metric's value under the right RawPath at Good quality. It is also why Births would otherwise + // grow monotonically across redeploys. + // + // The predicate is edge-node departure ONLY, deliberately narrower than "absent from ByScope". + // A device with no authored tags but whose EDGE NODE is still authored keeps receiving traffic + // (the Dispatch filter is node-level, and OnDeviceBirth runs ahead of the authored-scope + // filter), so its birth is live, not frozen — evicting it would throw away correct state and + // cost a needless rebirth round-trip the moment a later deploy authored a tag on it. + foreach (var scope in Births.Scopes) + { + var owner = new SparkplugNodeKey(scope.GroupId, scope.EdgeNodeId); + if (!table.EdgeNodes.Contains(owner)) + { + // Forgets the node scope and every device under it in one call; idempotent, so a node + // contributing several scopes to this loop costs one eviction and some no-ops. + Births.ForgetNode(owner.GroupId, owner.EdgeNodeId); + } + } + + foreach (var rawPath in _handleByRawPath.Keys) + { + if (!table.ByRawPath.ContainsKey(rawPath)) + { + _handleByRawPath.TryRemove(rawPath, out _); + } + } + + return byRawPath.Count; + } + + /// Resolves a RawPath to its authored definition through the shared v3 resolver. + /// The driver wire reference. + /// The definition, when this returns . + /// when the reference is an authored Sparkplug tag. + public bool TryResolve(string rawPath, out MqttTagDefinition def) => _resolver.TryResolve(rawPath, out def!); + + /// The authored RawPaths, in no particular order — the discovery enumeration source. + public IReadOnlyCollection AuthoredRawPaths => _authored.ByRawPath.Keys; + + /// + /// Establishes the group-wide (and, when configured, STATE) subscription and requests a + /// late-join rebirth from every authored edge node. + /// + /// + /// + /// Unlike plain mode, the subscribe is NOT driven by . A + /// Sparkplug driver must see birth certificates whether or not the OPC UA server has + /// subscribed anything — the birth is where datatypes and aliases come from, and Task 22's + /// discovery depends on having seen one. So the driver establishes the filter once, at + /// connect, and only records which references may be published. + /// + /// + /// Total failure throws. There is exactly one filter and it covers every tag, so a + /// broker that refuses it leaves the whole driver deaf — the one outcome that must be a + /// visible fault (the driver-host resilience layer re-runs initialize under backoff) rather + /// than a healthy-looking driver receiving nothing. This is the same rule + /// applies, stated once. + /// + /// + /// Cancellation for the SUBSCRIBE round-trip. + /// A task that completes when the subscribe has been answered. + /// No group id is configured, or every filter was refused. + public async Task EstablishAsync(CancellationToken cancellationToken) + { + // The SAME reset OnReconnectedAsync performs, and for the same reason — this ingestor OUTLIVES + // a session rebuild. `MqttDriver.AdoptOptions` only replaces it when `!SameIngest`, but + // `SessionIdentity` is a strict SUPERSET of `IngestIdentity`: changing Host, Port, ClientId, + // credentials, TLS or a timeout alone yields `SameSession == false` AND `SameIngest == true`, + // so `ReinitializeAsync` tears the session down and re-establishes on this very instance. + // `TeardownAsync` touches no ingest state. The driver-host resilience layer re-running + // `InitializeAsync` after a fault is the second path onto this line. + // + // Without the reset, an edge node that restarted during the changeover and rebound alias 5 from + // Temperature to Pressure would have its first DDATA on the new session resolved against the + // OLD table — publishing Pressure's value under the Temperature RawPath at Good quality. The + // late-join rebirth below does not close that: it is dispatched off-thread (DATA can arrive + // first), passes the debounce, and is a no-op under `requestRebirthOnGap: false` or against a + // node that ignores the NCMD. + ResetSessionState("establish a broker session"); + + await SubscribeGroupAsync(cancellationToken).ConfigureAwait(false); + RequestLateJoinRebirths("initial connect"); + } + + /// + /// Records a batch of RawPath references against a new handle. Issues no SUBSCRIBE — the + /// group-wide filter is established once by ; see its remarks. + /// + /// The RawPath references to subscribe. + /// Ignored — Sparkplug is push-based. + /// Unused; this call touches no network. + /// The handle every resulting is attributed to. + public Task SubscribeAsync( + IReadOnlyList fullReferences, + TimeSpan publishingInterval, + CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(fullReferences); + + var defs = new List(fullReferences.Count); + var now = _utcNow(); + foreach (var reference in fullReferences) + { + if (_resolver.TryResolve(reference, out var def)) + { + defs.Add(def); + continue; + } + + // Not an authored Sparkplug tag. Say so rather than leaving it in "waiting for initial + // data" forever — a reference that will never be fed is a different fault from one that + // has simply not been fed yet. + _logger?.LogWarning( + "MQTT driver '{DriverId}': subscribe reference '{RawPath}' is not an authored Sparkplug tag.", + _driverId, + reference); + Values.Update(reference, new DataValueSnapshot(null, StatusBadNodeIdUnknown, null, now)); + } + + var handle = new SparkplugSubscriptionHandle( + $"sparkplug:{Interlocked.Increment(ref _handleSeq)}:[{defs.Count} ref(s)]"); + _refsByHandle[handle] = [.. defs.Select(d => d.Name)]; + foreach (var def in defs) + { + _handleByRawPath[def.Name] = handle; + } + + return Task.FromResult(handle); + } + + /// + /// Stops attributing notifications to . References another live + /// subscription also covers keep publishing; the broker-side group filter is untouched. + /// + /// The handle returned by . + /// Unused; present for the ISubscribable shape. + /// A completed task. + public Task UnsubscribeAsync(ISubscriptionHandle handle, CancellationToken cancellationToken) + { + if (handle is not null && _refsByHandle.TryRemove(handle, out var refs)) + { + foreach (var rawPath in refs) + { + // Only clear the mapping if it is still OURS — a later overlapping subscription may + // have taken the reference over, and it must keep publishing. + _handleByRawPath.TryRemove(new KeyValuePair(rawPath, handle)); + } + } + + return Task.CompletedTask; + } + + /// + /// Recovers the ingest state after a reconnect: drops every birth, resets every sequence + /// tracker, re-establishes the group filter and requests a late-join rebirth from every authored + /// edge node (§3.6 invariant #5). + /// + /// + /// + /// Clearing the birth cache is not housekeeping — it is the correctness step. Across + /// an outage of unknown length an edge node may have restarted and rebound its aliases; a + /// DDATA resolved against the pre-outage table would route a value to whichever tag used to + /// own that alias. Same for the sequence baseline: carried across, a stream could look + /// contiguous purely because the node published the right number of messages while the + /// driver was away. + /// + /// + /// Values are deliberately not staled here. Reconnect is a driver-side event, not a + /// statement about the plant; the last observed values remain the best available answer + /// until the rebirth lands, and the connection's own health surface already reports + /// Reconnecting. Only a death — the source saying it is offline — stales a tag. + /// + /// + /// The connection's lifetime token; cancelled by its dispose. + /// A task that completes when the re-subscribe has been answered. + /// Every filter was refused — the caller tears the session down. + public async Task OnReconnectedAsync(CancellationToken cancellationToken) + { + ResetSessionState("reconnect"); + + await SubscribeGroupAsync(cancellationToken).ConfigureAwait(false); + RequestLateJoinRebirths("reconnect"); + } + + /// + /// Drops every scrap of per-session ingest state: births, sequence baselines, the birth-seen + /// set and the late-join counters. Called from every seam at which the session underneath + /// this ingestor changes — , and + /// . + /// + /// + /// + /// This is the correctness step, not housekeeping — and the reason it lives in one + /// method called from three places rather than inline in one. Every piece of state cleared + /// here is a claim about a session that no longer exists: an alias table binds names to + /// numbers the previous edge-node session chose, and a sequence baseline is only + /// evidence relative to the stream it was taken from. Carried across, an alias resolves to + /// whichever metric used to own it (good quality, plausible value, wrong tag) and a fresh + /// stream can look contiguous purely by coincidence. + /// + /// + /// Unconditionally safe. On a freshly-constructed ingestor every collection is + /// already empty, so calling it three times during one connect costs three no-ops. + /// + /// + /// Values are deliberately not staled here. A session change is a driver-side event, + /// not a statement about the plant; last observed values remain the best available answer + /// until a birth lands, and the connection's own health surface already reports the outage. + /// Only a death — the source saying it is offline — stales a tag. + /// + /// + /// Why the reset is happening, for the diagnostic log. + private void ResetSessionState(string reason) + { + var births = Births.Count; + Births.Clear(); + _trackers.Clear(); + _nodeBirthSeen.Clear(); + _metadataRebirths.Clear(); + + if (births > 0) + { + _logger?.LogInformation( + "MQTT driver '{DriverId}': dropped {Count} Sparkplug birth scope(s) and every sequence " + + "baseline — {Reason}. Aliases will be rebuilt from the next birth certificate.", + _driverId, + births, + reason); + } + } + + /// + /// Routes one inbound MQTT message: parses the topic, decodes the Sparkplug payload and hands it + /// to . Runs on MQTTnet's dispatcher thread and never throws. + /// + /// The concrete topic the message arrived on. + /// The message body; valid only for the duration of this call. + /// + /// The message's retain flag. Not filtered on: a retained NBIRTH/STATE is exactly the + /// late-join metadata this driver wants, and Sparkplug DATA is never published retained. + /// + public void HandleMessage(string topic, ReadOnlySpan payload, bool retained) + { + try + { + if (!SparkplugTopic.TryParse(topic, out var parsed)) + { + // A group-wide '#' subscription legitimately delivers topics this driver has no opinion + // about. Debug, not warn — an operator must not have to read one line per stray message. + _logger?.LogDebug( + "MQTT driver '{DriverId}': '{Topic}' is not a Sparkplug topic; ignored.", + _driverId, + topic); + return; + } + + if (parsed.Type == SparkplugMessageType.STATE) + { + // A STATE body is a handful of bytes; an oversized one is a broken publisher, not a + // dispatcher-thread hazard, and it is logged at Debug rather than through WarnOnce + // because its key would be a host id this driver did not choose. + if (payload.Length > MaxPayloadBytes) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': STATE on '{Topic}' exceeded the {Max}-byte ceiling; dropped.", + _driverId, + topic, + MaxPayloadBytes); + return; + } + + HandleState(parsed, payload); + return; + } + + // The authored-node filter runs HERE as well as inside Dispatch, and the order matters for + // two reasons. It keeps the oversize warning's key bounded — keyed on the raw topic it grew + // once per distinct topic off a group-wide '#' subscription, i.e. off wire input a plant (or + // a hostile publisher) controls. And it drops an unauthored node's body before the protobuf + // parse rather than after, which on a busy group is most of the decode work this driver + // would otherwise do for traffic it discards. + if (parsed.GroupId is not { } groupId || parsed.EdgeNodeId is not { } edgeNodeId) + { + return; + } + + var node = new SparkplugNodeKey(groupId, edgeNodeId); + if (!_authored.EdgeNodes.Contains(node)) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': {Type} for unauthored edge node '{Node}'; ignored.", + _driverId, + parsed.Type, + node); + return; + } + + if (payload.Length > MaxPayloadBytes) + { + // Refused BEFORE any decode: the whole point of the bound is that one publisher cannot + // impose an unbounded protobuf parse on the shared dispatcher thread. Nothing mutates. + // + // An oversized NBIRTH is the worst case and is called out by name: the driver cannot + // learn this node's aliases at all, so every subsequent DATA message is unroutable. The + // late-join cap in CheckSequence is what stops that becoming a permanent NCMD loop. + WarnOnce( + $"oversize:{node}", + "MQTT driver '{DriverId}': {Type} from '{Node}' carried {Bytes} bytes, over the " + + "{Max}-byte ceiling; dropped. Raise maxPayloadBytes if this node's birth is " + + "legitimately this large.", + _driverId, + parsed.Type, + node, + payload.Length, + MaxPayloadBytes); + return; + } + + Dispatch(parsed, SparkplugCodec.Decode(payload)); + } + catch (Exception ex) + { + // Defence in depth: every path below is written not to throw, but an escape here surfaces + // inside MQTTnet's own pump and stalls delivery for every subscription on the connection. + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': Sparkplug ingest threw for '{Topic}'; the message was dropped.", + _driverId, + topic); + } + } + + /// + /// Routes one decoded Sparkplug message — the state machine proper, and the seam the + /// §3.6 matrix is driven through without a broker. + /// + /// The parsed topic. + /// The decoded payload. Checked for validity here, not by the caller. + public void Dispatch(SparkplugTopic topic, SparkplugPayload payload) + { + ArgumentNullException.ThrowIfNull(topic); + ArgumentNullException.ThrowIfNull(payload); + + if (topic.Type == SparkplugMessageType.STATE) + { + return; // STATE carries no Sparkplug payload; HandleMessage routes it separately. + } + + // NCMD/DCMD flow host → node. Our own rebirth requests come back on the '#' subscription; + // treating them as ingest would be, at best, noise. + if (topic.Type is SparkplugMessageType.NCMD or SparkplugMessageType.DCMD) + { + return; + } + + if (topic.GroupId is not { } groupId || topic.EdgeNodeId is not { } edgeNodeId) + { + return; // TryParse guarantees both for a non-STATE topic; belt and braces. + } + + var node = new SparkplugNodeKey(groupId, edgeNodeId); + if (!_authored.EdgeNodes.Contains(node)) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': {Type} for unauthored edge node '{Node}'; ignored.", + _driverId, + topic.Type, + node); + return; + } + + if (!payload.IsValid) + { + // NOT a rebirth trigger and NOT a state mutation — see the type remarks. An undecodable + // body is not evidence that anything about the node's stream has changed. + WarnOnce( + $"invalid:{node}", + "MQTT driver '{DriverId}': {Type} from '{Node}' could not be decoded as a Sparkplug payload; dropped.", + _driverId, + topic.Type, + node); + return; + } + + switch (topic.Type) + { + case SparkplugMessageType.NBIRTH: + OnNodeBirth(node, payload); + break; + + case SparkplugMessageType.DBIRTH when topic.DeviceId is { } dbirthDevice: + OnDeviceBirth(node, dbirthDevice, payload); + break; + + case SparkplugMessageType.NDATA: + OnData(node, new SparkplugScope(groupId, edgeNodeId, null), payload); + break; + + case SparkplugMessageType.DDATA when topic.DeviceId is { } ddataDevice: + OnData(node, new SparkplugScope(groupId, edgeNodeId, ddataDevice), payload); + break; + + case SparkplugMessageType.NDEATH: + OnNodeDeath(node, payload); + break; + + case SparkplugMessageType.DDEATH when topic.DeviceId is { } ddeathDevice: + OnDeviceDeath(node, ddeathDevice, payload); + break; + + default: + break; + } + } + + /// + /// Awaits every rebirth publish this ingestor has dispatched and not yet observed. Test seam and + /// teardown helper — the publishes themselves are fire-and-forget by design, because a rebirth + /// NCMD must never be able to park MQTTnet's dispatcher thread. + /// + /// A task that completes when no dispatched rebirth publish is outstanding. + internal async Task DrainPendingRebirthsAsync() + { + Task[] pending; + lock (_pendingRebirths) + { + pending = [.. _pendingRebirths]; + } + + foreach (var task in pending) + { + try + { + await task.ConfigureAwait(false); + } + catch (Exception) + { + // Already logged where it was dispatched; a failed rebirth is not a caller's problem. + } + } + } + + // ---- births ---- + + /// + /// Applies an NBIRTH: restarts the node's sequence at the birth's seq, adopts its + /// bdSeq session token, rebuilds the node's catalog, stales every device the birth + /// invalidated, and publishes the birth's own values. + /// + private void OnNodeBirth(SparkplugNodeKey node, SparkplugPayload payload) + { + // AcceptNodeBirth, never Accept: an NBIRTH restarts the sequence by definition, so running it + // through the gap detector would flag a gap on the very message that resynchronized everything + // and answer a birth with a demand for another one. + // Recorded BEFORE the seq is judged, and independently of the verdict: "a birth arrived" and + // "the birth established a sequence baseline" are different facts, and conflating them is the + // rebirth loop CheckSequence documents. + _nodeBirthSeen[node] = 0; + _metadataRebirths.TryRemove(node, out _); + + var bdSeq = SequenceTracker.TryReadBdSeq(payload, out var token) ? token : (ulong?)null; + if (!TrackerFor(node).AcceptNodeBirth(payload.Seq, bdSeq)) + { + // Deliberately NOT a rebirth trigger: the metadata just arrived, and commanding another + // birth from a node whose seq we cannot read would loop against the same malformed payload. + WarnOnce( + $"birthseq:{node}", + "MQTT driver '{DriverId}': NBIRTH from '{Node}' carried no usable seq ({Seq}); " + + "gap detection is disabled for this node until its next birth.", + _driverId, + node, + payload.Seq); + } + + var outcome = Births.ApplyNodeBirth(node.GroupId, node.EdgeNodeId, payload.Metrics); + LogBirthAnomaly(node.ToString(), outcome.Result); + + var sourceUtc = ToUtc(payload.TimestampMs); + + // An NBIRTH voids every prior DBIRTH under the node (Sparkplug spec): those devices MUST + // re-DBIRTH, and until they do their aliases are meaningless. Staling them is what stops the + // driver serving values resolved against a table it has just thrown away. + foreach (var invalidated in outcome.InvalidatedDevices) + { + FanStale(invalidated.Scope, sourceUtc, "its edge node re-birthed; awaiting a fresh DBIRTH"); + } + + PublishBirthValues(new SparkplugScope(node.GroupId, node.EdgeNodeId, null), payload, sourceUtc); + RaiseBirthObserved(new SparkplugScope(node.GroupId, node.EdgeNodeId, null), payload); + } + + /// Applies a DBIRTH: rebuilds exactly that device's catalog and publishes its values. + private void OnDeviceBirth(SparkplugNodeKey node, string deviceId, SparkplugPayload payload) + { + // A DBIRTH is a sequenced member of the edge node's stream — only the NBIRTH restarts it. + CheckSequence(node, payload, "DBIRTH"); + + // Metadata arrived, so the give-up counter re-arms: a device that re-births after a bad patch + // must not stay permanently un-askable because of requests made before it recovered. + _metadataRebirths.TryRemove(node, out _); + + var result = Births.ApplyDeviceBirth(node.GroupId, node.EdgeNodeId, deviceId, payload.Metrics); + var scope = new SparkplugScope(node.GroupId, node.EdgeNodeId, deviceId); + LogBirthAnomaly(scope.ToString(), result); + + var sourceUtc = ToUtc(payload.TimestampMs); + PublishBirthValues(scope, payload, sourceUtc); + RaiseBirthObserved(scope, payload); + } + + // ---- data ---- + + /// + /// Applies an NDATA/DDATA: checks stream continuity, resolves every metric through the scope's + /// birth-built alias table, and publishes under the authored tag's RawPath. + /// + private void OnData(SparkplugNodeKey node, SparkplugScope scope, SparkplugPayload payload) + { + CheckSequence(node, payload, scope.IsDevice ? "DDATA" : "NDATA"); + + // Scope-level filtering happens HERE, after the sequence check, never before it: an unauthored + // device's DDATA is still a member of its edge node's stream, and skipping it earlier would + // manufacture a gap on the very next authored message. Past this point there is nothing this + // scope could publish — so there is also nothing worth demanding a rebirth for. Without the + // guard, a device nobody authored that publishes DDATA without ever DBIRTHing (spec-violating, + // but they exist) would drive a permanent NCMD at the debounce cadence, forever. + if (!_authored.ByScope.ContainsKey(scope)) + { + return; + } + + var table = Births.Find(scope); + if (table is null) + { + // Data before any birth (§3.6 #3): we hold no alias table for this scope, so every metric + // in this payload is unroutable. One rebirth request for the whole message, not one per + // metric — the debounce would collapse them anyway, but not asking is cheaper than asking + // and being suppressed. + WarnOnce( + $"prebirth:{scope}", + "MQTT driver '{DriverId}': data for '{Scope}' arrived before any birth certificate; " + + "requesting a rebirth.", + _driverId, + scope); + RequestMetadataRebirth(node, "data before birth"); + return; + } + + var sourceUtc = ToUtc(payload.TimestampMs); + var unknownAlias = false; + + foreach (var metric in payload.Metrics) + { + var binding = ResolveBinding(table, metric); + if (binding is null) + { + // The birth this table was built from never declared this alias/name — our metadata is + // behind the node's. Same remedy as data-before-birth, different log. + unknownAlias = true; + continue; + } + + PublishMetric(scope, binding, metric, sourceUtc); + } + + if (unknownAlias) + { + WarnOnce( + $"alias:{scope}", + "MQTT driver '{DriverId}': data for '{Scope}' carried a metric the current birth does not " + + "declare; requesting a rebirth.", + _driverId, + scope); + RequestMetadataRebirth(node, "unknown alias"); + } + } + + /// + /// Resolves a DATA metric to its birth binding: by alias when it carries one (the normal + /// case — a DATA metric after a birth carries an alias and neither name nor datatype), otherwise + /// by name, since aliases are optional in Sparkplug. + /// + private static SparkplugMetricBinding? ResolveBinding(AliasTable table, SparkplugMetric metric) + { + // NAME FIRST, deliberately. A DATA metric that carries a name has stated its identity in the + // one vocabulary that survives a rebirth; the alias is a per-birth compression detail. When the + // two disagree — a publisher that recycled an alias mid-stream, or a birth this driver applied + // out of order — preferring the alias would resolve to whichever metric currently occupies that + // slot, which is precisely the mis-route this type's binding rule exists to prevent. The common + // case costs nothing: a real post-birth DATA metric carries no name at all, so this is a null + // check before the alias lookup. + if (metric.Name is { } name && table.ResolveByName(name) is { } byName) + { + return byName; + } + + return metric.Alias is { } alias ? table.Resolve(alias) : null; + } + + // ---- deaths ---- + + /// + /// Applies an NDEATH: verifies it belongs to the session currently believed alive, then forgets + /// the node and every device under it and stales their authored tags. + /// + private void OnNodeDeath(SparkplugNodeKey node, SparkplugPayload payload) + { + // NOT SequenceTracker.Accept. An NDEATH is the broker-published Last Will: it carries no seq at + // all, so Accept would report a gap on every death and answer a node that has just died with a + // rebirth request. The bdSeq tie is the whole mechanism instead. + var tracker = TrackerFor(node); + var deathBdSeq = SequenceTracker.TryReadBdSeq(payload, out var token) ? token : (ulong?)null; + if (!tracker.IsDeathForCurrentSession(deathBdSeq)) + { + // A previous session's will, delivered after the node already reconnected and re-birthed. + // Acting on it would mark a live node dead with nothing coming to correct it. + _logger?.LogInformation( + "MQTT driver '{DriverId}': ignoring a stale NDEATH for '{Node}' (bdSeq {DeathBdSeq} ≠ current " + + "{BirthBdSeq}); the node re-birthed before its old will was delivered.", + _driverId, + node, + deathBdSeq, + tracker.BirthBdSeq); + return; + } + + Births.ForgetNode(node.GroupId, node.EdgeNodeId); + + var sourceUtc = ToUtc(payload.TimestampMs); + foreach (var scope in _authored.ScopesUnder(node)) + { + FanStale(scope, sourceUtc, "its edge node published NDEATH"); + } + } + + /// Applies a DDEATH: forgets exactly that device and stales its authored tags. + private void OnDeviceDeath(SparkplugNodeKey node, string deviceId, SparkplugPayload payload) + { + // A DDEATH IS published by the edge node and IS sequenced — unlike an NDEATH. + CheckSequence(node, payload, "DDEATH"); + + Births.TryForgetDevice(node.GroupId, node.EdgeNodeId, deviceId, out _); + + FanStale( + new SparkplugScope(node.GroupId, node.EdgeNodeId, deviceId), + ToUtc(payload.TimestampMs), + "the device published DDEATH"); + } + + // ---- STATE ---- + + /// + /// Records an observed primary-host STATE message. + /// + /// + /// + /// Receive only. This driver never publishes STATE, and + /// is therefore inert — deliberately, and + /// loudly. Being a Sparkplug Primary Host is not "publish an ONLINE message": it is a + /// three-part contract — a retained ONLINE STATE published after CONNECT, a matching + /// retained OFFLINE registered as the client's MQTT Last Will at CONNECT time, + /// and an explicit OFFLINE on clean shutdown. Edge nodes subscribe to it and stop publishing + /// while their host is offline. Shipping the ONLINE half without the Will is strictly worse + /// than shipping neither: an OtOpcUa node that crashes would leave a retained ONLINE asserting + /// forever that a dead host is alive, which is precisely the state the mechanism exists to + /// prevent. The Will has to be set inside MqttConnection.BuildClientOptions — a + /// different file, a different task — so this task implements the observation half and + /// warns when the flag is set. See the design doc §7/§8. + /// + /// + /// Both payload forms are accepted on receive: the v3.0 JSON {"online":bool,...} and + /// the pre-3.0 bare ONLINE/OFFLINE text. + /// + /// + private void HandleState(SparkplugTopic topic, ReadOnlySpan payload) + { + if (topic.HostId is not { } hostId) + { + return; + } + + if (!TryReadStateOnline(payload, out var online)) + { + WarnOnce( + $"state:{hostId}", + "MQTT driver '{DriverId}': STATE for host '{HostId}' carried an unrecognised payload; ignored.", + _driverId, + hostId); + return; + } + + var previous = _hostState; + var state = new SparkplugHostState(hostId, online, _utcNow()); + _hostState = state; + + if (previous is null || previous.Online != online || !string.Equals(previous.HostId, hostId, StringComparison.Ordinal)) + { + _logger?.LogInformation( + "MQTT driver '{DriverId}': Sparkplug host '{HostId}' is {State}.", + _driverId, + hostId, + online ? "ONLINE" : "OFFLINE"); + } + + try + { + HostStateObserved?.Invoke(this, new SparkplugHostStateEventArgs(state)); + } + catch (Exception ex) + { + // Same containment as every other raise here — this runs on MQTTnet's dispatcher thread. + _logger?.LogError(ex, "MQTT driver '{DriverId}': a host-state subscriber threw.", _driverId); + } + } + + /// Reads the online flag out of a STATE body — v3.0 JSON first, then the legacy text form. + private static bool TryReadStateOnline(ReadOnlySpan payload, out bool online) + { + online = false; + if (payload.IsEmpty) + { + return false; + } + + try + { + var reader = new Utf8JsonReader(payload); + if (JsonDocument.TryParseValue(ref reader, out var doc)) + { + using (doc) + { + if (doc.RootElement.ValueKind == JsonValueKind.Object + && doc.RootElement.TryGetProperty("online", out var flag) + && flag.ValueKind is JsonValueKind.True or JsonValueKind.False) + { + online = flag.GetBoolean(); + return true; + } + } + } + } + catch (JsonException) + { + // Fall through to the legacy text form. + } + + Span text = stackalloc char[16]; + if (payload.Length > text.Length || !Encoding.UTF8.TryGetChars(payload, text, out var written)) + { + return false; + } + + var trimmed = text[..written].Trim(); + if (trimmed.Equals("ONLINE", StringComparison.OrdinalIgnoreCase)) + { + online = true; + return true; + } + + if (trimmed.Equals("OFFLINE", StringComparison.OrdinalIgnoreCase)) + { + online = false; + return true; + } + + return false; + } + + // ---- publish ---- + + /// + /// Publishes every named metric a birth certificate carried. This is what restores Good quality + /// after a death: a birth is a full snapshot, not a delta. + /// + private void PublishBirthValues(SparkplugScope scope, SparkplugPayload payload, DateTime? sourceUtc) + { + var table = Births.Find(scope); + if (table is null) + { + return; + } + + foreach (var metric in payload.Metrics) + { + if (metric.Name is not { } name || table.ResolveByName(name) is not { } binding) + { + continue; + } + + PublishMetric(scope, binding, metric, sourceUtc); + } + } + + /// + /// Publishes one resolved metric to every authored tag bound to + /// (scope, binding.Name) — under the tag's RawPath. + /// + private void PublishMetric( + SparkplugScope scope, + SparkplugMetricBinding binding, + SparkplugMetric metric, + DateTime? sourceUtc) + { + // Bound by NAME. The alias got us here; it is never the key. + var key = new SparkplugMetricKey(scope.GroupId, scope.EdgeNodeId, scope.DeviceId, binding.Name); + if (!_authored.ByMetric.TryGetValue(key, out var defs)) + { + return; // A published metric nobody authored a tag for. Not an error — that is most of them. + } + + // Arrays slip past the unsupported-type gate below, because ToDriverDataType maps every + // *Array variant to its ELEMENT type and is therefore never null for one. Design §3.5 defers + // array support, and the failure is not benign: the value rides in bytes_value as a byte[] at + // ValueKind.Scalar, so a numeric target throws its way to BadTypeMismatch (safe, wrong reason) + // while a String/Reference target reaches Convert.ToBase64String and publishes a packed binary + // buffer as a plausible string at GOOD quality. Gate on the array bit itself. + if (binding.DataType.IsSparkplugArray()) + { + WarnOnce( + $"array-type:{key}", + "MQTT driver '{DriverId}': metric '{Metric}' on '{Scope}' is a Sparkplug array " + + "({DataType}); array metrics are not supported in v1 and its tag(s) are not being fed.", + _driverId, + binding.Name, + scope, + binding.DataType); + return; + } + + var birthType = binding.DataType.ToDriverDataType(); + if (birthType is null) + { + // DataSet / Template / PropertySet / Unknown — warn-and-skip per the datatype map's + // contract, and explicitly NOT a rebirth trigger: the node would answer with the same + // unsupported metric and the pair would loop. + WarnOnce( + $"unsupported-type:{key}", + "MQTT driver '{DriverId}': metric '{Metric}' on '{Scope}' has unsupported Sparkplug type " + + "{DataType}; its tag(s) are not being fed.", + _driverId, + binding.Name, + scope, + binding.DataType); + return; + } + + var serverUtc = _utcNow(); + + foreach (var def in defs) + { + // The authored dataType wins when the operator declared one; otherwise the birth's own + // declaration does. That is what lets a tag be authored with nothing but the metric tuple. + var target = def.DataTypeAuthored ? def.DataType : birthType.Value; + if (BuildSnapshot(def, binding, metric, target, sourceUtc, serverUtc) is not { } snapshot) + { + continue; + } + + Values.Update(def.Name, snapshot); + Raise(def.Name, snapshot); + } + } + + /// + /// Turns one metric's wire value into a snapshot for one authored tag, or + /// when the metric carried no value at all — in which case the tag keeps whatever it last had + /// rather than being re-published unchanged or given an invented quality. + /// + private DataValueSnapshot? BuildSnapshot( + MqttTagDefinition def, + SparkplugMetricBinding binding, + SparkplugMetric metric, + DriverDataType target, + DateTime? sourceUtc, + DateTime serverUtc) + { + // Metric timestamp wins over the payload's — it is the acquisition instant at the source. + var stamp = ToUtc(metric.TimestampMs) ?? sourceUtc; + + switch (metric.ValueKind) + { + case SparkplugValueKind.Scalar: + break; + + case SparkplugValueKind.Null: + // An explicit is_null. The tag has a declared scalar type and null is not a value of + // it — the same verdict the plain-mode path reaches for a JSON null. + return new DataValueSnapshot(null, StatusBadTypeMismatch, stamp, serverUtc); + + default: + // Absent (no value oneof at all) or Unsupported (DataSet/Template). Neither is a value; + // leave the tag holding whatever it last had rather than inventing a quality for it. + WarnOnce( + $"valuekind:{def.Name}", + "MQTT driver '{DriverId}': metric '{Metric}' for '{RawPath}' carried {ValueKind} rather " + + "than a value; skipped.", + _driverId, + binding.Name, + def.Name, + metric.ValueKind); + return null; + } + + // THE call. Sparkplug carries signed integers as two's complement in an unsigned field, and a + // DATA metric carries no datatype of its own — only the birth binding knows which applies. + // Skipping this publishes 4294967254 for a value of -42: plausible, Good, and wrong. + var raw = binding.Reinterpret(metric.Value); + + return SparkplugValueCoercion.TryCoerce(raw, target, out var value) + ? new DataValueSnapshot(value, StatusGood, stamp, serverUtc) + : BadTypeMismatch(def, binding, target, stamp, serverUtc); + } + + /// + /// The metric decoded, but does not fit the tag's effective type. Warned once per tag; the value + /// itself is deliberately never logged — a payload value is plant data, not diagnostics. + /// + private DataValueSnapshot BadTypeMismatch( + MqttTagDefinition def, + SparkplugMetricBinding binding, + DriverDataType target, + DateTime? stamp, + DateTime serverUtc) + { + WarnOnce( + $"coerce:{def.Name}", + "MQTT driver '{DriverId}': metric '{Metric}' ({Sparkplug}) does not fit tag '{RawPath}'s " + + "{Target}; status BadTypeMismatch.", + _driverId, + binding.Name, + binding.DataType, + def.Name, + target); + + return new DataValueSnapshot(null, StatusBadTypeMismatch, stamp, serverUtc); + } + + /// Marks every authored tag in as no longer communicating. + private void FanStale(SparkplugScope scope, DateTime? sourceUtc, string reason) + { + if (!_authored.ByScope.TryGetValue(scope, out var defs)) + { + return; + } + + var snapshot = new DataValueSnapshot(null, StatusBadCommunicationError, sourceUtc, _utcNow()); + foreach (var def in defs) + { + Values.Update(def.Name, snapshot); + Raise(def.Name, snapshot); + } + + _logger?.LogInformation( + "MQTT driver '{DriverId}': staled {Count} tag(s) on '{Scope}' — {Reason}.", + _driverId, + defs.Length, + scope, + reason); + } + + /// + /// Raises for a RawPath, if a live subscription covers it. A throwing + /// subscriber is contained so it cannot starve the rest of the fan-out or escape onto MQTTnet's + /// dispatcher thread. + /// + private void Raise(string rawPath, DataValueSnapshot snapshot) + { + if (!_handleByRawPath.TryGetValue(rawPath, out var handle)) + { + return; // Authored but not subscribed: cached so a read answers, nothing to attribute to. + } + + try + { + OnDataChange?.Invoke(this, new DataChangeEventArgs(handle, rawPath, snapshot)); + } + catch (Exception ex) + { + _logger?.LogError( + ex, + "MQTT driver '{DriverId}': a data-change subscriber threw for '{RawPath}'.", + _driverId, + rawPath); + } + } + + private void RaiseBirthObserved(SparkplugScope scope, SparkplugPayload payload) + { + var subscribers = BirthObserved; + if (subscribers is null) + { + return; + } + + try + { + subscribers(this, new SparkplugBirthObservedEventArgs( + scope, + [.. payload.Metrics.Where(m => m.Name is not null).Select(m => m.Name!)])); + } + catch (Exception ex) + { + _logger?.LogError(ex, "MQTT driver '{DriverId}': a birth-observed subscriber threw.", _driverId); + } + } + + // ---- sequence + rebirth ---- + + private SequenceTracker TrackerFor(SparkplugNodeKey node) => + _trackers.GetOrAdd(node, static _ => new SequenceTracker()); + + /// + /// Offers a sequenced message's seq and requests a rebirth when the stream is not + /// contiguous — or when it is contiguous only because the driver joined mid-stream and has never + /// seen this node's NBIRTH. + /// + /// + /// The second case is the one that is easy to miss: returns + /// for the first message after a reset (there is nothing to measure it + /// against), so a driver that only checked its return value would sit happily behind a stream + /// whose birth certificate — and therefore whose entire alias table — it never received. + /// + private void CheckSequence(SparkplugNodeKey node, SparkplugPayload payload, string kind) + { + var tracker = TrackerFor(node); + if (!tracker.Accept(payload.Seq)) + { + WarnOnce( + $"gap:{node}", + "MQTT driver '{DriverId}': {Kind} from '{Node}' broke the sequence (seq {Seq}, last {Last}); " + + "requesting a rebirth.", + _driverId, + kind, + node, + payload.Seq, + tracker.LastSeq); + RequestRebirth(node, "sequence gap"); + return; + } + + if (tracker.IsBirthSynchronized) + { + return; + } + + // "Birth-synchronized" in SequenceTracker's sense means a birth established a usable SEQUENCE + // BASELINE — it is false both when no NBIRTH has arrived (a genuine late join, which a rebirth + // fixes) and when one arrived carrying a seq this driver could not read (which a rebirth does + // NOT fix: the node answers with the same unusable birth). Reacting to the two identically is + // what turns the second into a permanent NCMD loop at the debounce cadence, defeating the guard + // OnNodeBirth deliberately applies one method over. Whether a birth ARRIVED is this type's own + // fact to keep, so it keeps it. + if (_nodeBirthSeen.ContainsKey(node)) + { + return; + } + + WarnOnce( + $"latejoin:{node}", + "MQTT driver '{DriverId}': {Kind} from '{Node}' arrived with no NBIRTH seen since connect; " + + "requesting a rebirth.", + _driverId, + kind, + node); + + // The residual loop the birth-seen flag cannot see: an NBIRTH that never reaches Dispatch at + // all, because it exceeded MaxPayloadBytes or because the node never sends one. Nothing about + // asking again changes that, so it goes through the capped path. + RequestMetadataRebirth(node, "late join (no NBIRTH observed)"); + } + + /// + /// Requests a rebirth for a missing-metadata reason — late join, data before birth, or an + /// alias the current birth does not declare — under a per-node cap on consecutive attempts. + /// + /// + /// + /// Why these three and not a sequence gap. A gap is self-limiting: the tracker + /// resynchronizes onto the observed value, so the very next message is contiguous and asks + /// for nothing. The three reasons routed here are not — every one of them means "this + /// driver holds no usable catalog for this node", and if the node cannot supply one (its + /// birth is over the payload ceiling, or it ignores NCMD entirely) then every + /// subsequent message re-raises the same condition. The debounce caps the rate at one + /// request per period; only this caps the lifetime. + /// + /// + /// Consecutive, not cumulative. Any applied birth clears the counter, so a node that + /// recovers is not permanently penalised for having once been unreachable. + /// + /// + /// The edge node to command. + /// Why, for the log. + private void RequestMetadataRebirth(SparkplugNodeKey node, string reason) + { + var attempts = _metadataRebirths.AddOrUpdate(node, 1, static (_, n) => n + 1); + if (attempts > MaxConsecutiveMetadataRebirths) + { + WarnOnce( + $"metadata-exhausted:{node}", + "MQTT driver '{DriverId}': '{Node}' has not delivered a usable birth certificate after " + + "{Attempts} rebirth request(s); giving up until one arrives on its own. Its tags will " + + "not update. Check the node's birth payload size against maxPayloadBytes, its seq " + + "field, and that it honours NCMD Node Control/Rebirth.", + _driverId, + node, + MaxConsecutiveMetadataRebirths); + return; + } + + RequestRebirth(node, reason); + } + + /// + /// Requests a rebirth from every authored edge node — §3.6 invariant #5. Called on connect and + /// on every reconnect. + /// + /// + /// Bounded by configuration, not by the plant. Sparkplug has no group-wide rebirth + /// command — NCMD addresses exactly one edge node — so this is inherently one publish per node. + /// The set is the authored edge nodes rather than every node in the group, which is what + /// keeps a 200-node group from producing 200 NCMDs for the two nodes this deployment reads. Each + /// one still passes through the per-node debounce, so a connection that flaps cannot multiply + /// them. + /// + private void RequestLateJoinRebirths(string reason) + { + foreach (var node in _authored.EdgeNodes) + { + RequestRebirth(node, reason); + } + } + + /// + /// Publishes a rebirth NCMD to one edge node, subject to the + /// policy and the per-node debounce. + /// Never blocks the caller and never throws. + /// + private void RequestRebirth(SparkplugNodeKey node, string reason) + { + if (_options.Sparkplug is { RequestRebirthOnGap: false }) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': would request a rebirth from '{Node}' ({Reason}) but " + + "requestRebirthOnGap is off.", + _driverId, + node, + reason); + return; + } + + if (_publishTransport is not { } transport) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': cannot request a rebirth from '{Node}' ({Reason}) — no publish " + + "transport is attached.", + _driverId, + node, + reason); + return; + } + + if (!TryAcquireRebirthSlot(node)) + { + _logger?.LogDebug( + "MQTT driver '{DriverId}': rebirth request to '{Node}' ({Reason}) suppressed; one was sent " + + "less than {Debounce} ago.", + _driverId, + node, + reason, + _rebirthDebounce); + return; + } + + _logger?.LogInformation( + "MQTT driver '{DriverId}': requesting a Sparkplug rebirth from '{Node}' — {Reason}.", + _driverId, + node, + reason); + + // Off the dispatcher thread: RebirthRequester.RequestAsync serializes a payload and hands it to + // MQTTnet's publish path, and neither belongs on the thread that is delivering every other + // subscription's messages. It is bounded by its own 5s deadline. + TrackRebirth(Task.Run(async () => + { + try + { + await RebirthRequester + .RequestAsync(transport, node.GroupId, node.EdgeNodeId, CancellationToken.None) + .ConfigureAwait(false); + } + catch (Exception ex) + { + _logger?.LogWarning( + ex, + "MQTT driver '{DriverId}': the rebirth NCMD to '{Node}' failed; the next gap will retry.", + _driverId, + node); + } + })); + } + + /// + /// Claims this node's rebirth slot, or reports that one was claimed too recently. Lock-free: this + /// runs on MQTTnet's dispatcher thread. + /// + private bool TryAcquireRebirthSlot(SparkplugNodeKey node) + { + if (_rebirthDebounce <= TimeSpan.Zero) + { + return true; + } + + var now = _utcNow().Ticks; + while (true) + { + if (!_lastRebirthTicks.TryGetValue(node, out var last)) + { + if (_lastRebirthTicks.TryAdd(node, now)) + { + return true; + } + + continue; + } + + if (now - last < _rebirthDebounce.Ticks) + { + return false; + } + + if (_lastRebirthTicks.TryUpdate(node, now, last)) + { + return true; + } + } + } + + private void TrackRebirth(Task task) + { + lock (_pendingRebirths) + { + _pendingRebirths.RemoveAll(static t => t.IsCompleted); + _pendingRebirths.Add(task); + } + } + + // ---- subscribe ---- + + /// Issues the group-wide (and STATE) SUBSCRIBE, throwing when nothing could be established. + private async Task SubscribeGroupAsync(CancellationToken cancellationToken) + { + if (GroupFilter is not { } groupFilter) + { + throw new InvalidOperationException( + $"MQTT driver '{_driverId}': Sparkplug mode requires a groupId — there is no filter to " + + "subscribe, so the driver would receive nothing."); + } + + // QoS 1 for both. Sparkplug publishes NBIRTH/DBIRTH/DATA at QoS 0 and NDEATH/STATE at QoS 1; + // the SUBSCRIBE's QoS is the ceiling the broker may deliver at, so asking for 1 keeps the + // death and state messages at their intended guarantee and costs nothing for the rest. + var filters = new List(2) + { + // SeedRetained: a retained NBIRTH/STATE replayed at subscribe time IS the late-join + // metadata this driver wants — the opposite of the plain-mode tag opt-out. + new(groupFilter, Qos: 1, SeedRetained: true), + }; + + if (StateFilter is { } stateFilter) + { + filters.Add(new MqttTopicSubscription(stateFilter, Qos: 1, SeedRetained: true)); + + // The pre-3.0 form, which HandleState already parses. Subscribing only the v3.0 topic made + // that tolerance unreachable in production — the legacy branch could be exercised by a test + // calling HandleMessage directly and by nothing else. Design §3.1 is explicit: emit v3.0, + // tolerate legacy on receive, and tolerating requires asking for it. + filters.Add(new MqttTopicSubscription( + LegacyStateFilter!, + Qos: 1, + SeedRetained: true)); + } + + var transport = _subscribeTransport; + if (transport is null) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': Sparkplug subscription recorded with no connection attached; " + + "nothing was sent to a broker. Call AttachTo before establishing.", + _driverId); + return; + } + + var outcomes = await transport.SubscribeAsync(filters, cancellationToken).ConfigureAwait(false); + + // The group filter is the one that matters: without it the driver is deaf. A refused STATE + // filter costs an observability signal, not the data plane, so it warns rather than throws. + var groupGranted = outcomes.Any(o => o.Granted && string.Equals(o.Topic, groupFilter, StringComparison.Ordinal)); + if (!groupGranted) + { + throw new InvalidOperationException( + $"MQTT driver '{_driverId}': the broker refused '{groupFilter}'; tearing the session down " + + "rather than serving a Sparkplug driver that receives nothing."); + } + + foreach (var refused in outcomes.Where(o => !o.Granted)) + { + _logger?.LogWarning( + "MQTT driver '{DriverId}': broker refused SUBSCRIBE '{Topic}' ({Reason}).", + _driverId, + refused.Topic, + refused.Reason); + } + } + + // ---- helpers ---- + + private void LogBirthAnomaly(string scope, BirthApplyResult result) + { + if (!result.HasAnomaly) + { + return; + } + + // The only window onto a malformed birth — AliasTable takes no logger by design. + _logger?.LogWarning( + "MQTT driver '{DriverId}': birth for '{Scope}' was malformed — {Accepted} accepted, " + + "{Unnamed} unnamed, {AliasCollisions} alias collision(s), {DuplicateNames} duplicate name(s).", + _driverId, + scope, + result.Accepted, + result.RejectedUnnamed, + result.AliasCollisions, + result.DuplicateNames); + } + + private static DateTime? ToUtc(ulong? epochMs) => + epochMs is { } ms && ms <= (ulong)DateTimeOffset.MaxValue.ToUnixTimeMilliseconds() + ? DateTimeOffset.FromUnixTimeMilliseconds((long)ms).UtcDateTime + : null; + + /// + /// Logs a warning the first time is seen and stays quiet thereafter. A + /// 10 Hz metric with a mis-authored datatype must be diagnosable without drowning the log; the + /// key set is bounded because every path that calls this has already filtered to authored nodes, + /// scopes or RawPaths, and it is cleared on every . + /// + private void WarnOnce(string key, string message, params object?[] args) + { + // The ceiling is unreachable by design — see MaxWarnKeys. It exists so that a derivation that + // turns out NOT to be bounded (as the oversize path's raw-topic key was) degrades to silence + // instead of to unbounded growth on the dispatcher path. + if (_warned.Count < MaxWarnKeys && _warned.TryAdd(key, 0)) + { +#pragma warning disable CA2254 // Template is a compile-time constant at every call site. + _logger?.LogWarning(message, args); +#pragma warning restore CA2254 + } + } + + /// Opaque subscription identity for the Sparkplug ingest path. + private sealed record SparkplugSubscriptionHandle(string DiagnosticId) : ISubscriptionHandle; + + /// + /// The authored tag table and its Sparkplug indexes, published as one immutable snapshot so the + /// dispatcher thread reads a consistent view while a redeploy rebuilds it. + /// + /// RawPath → definition; the v3 resolver's backing table. + /// + /// (group, node, device?, metricName) → every definition bound to it. A list, + /// because two raw tags may legitimately project the same Sparkplug metric. + /// + /// Scope → its definitions; the death/invalidation STALE fan-out's input. + /// + /// The distinct (group, edgeNode) pairs any authored tag names — the ingest filter, the + /// late-join rebirth set, and what bounds the per-node tracker map. + /// + private sealed record AuthoredTable( + FrozenDictionary ByRawPath, + FrozenDictionary ByMetric, + FrozenDictionary ByScope, + FrozenSet EdgeNodes) + { + public static readonly AuthoredTable Empty = + Build(new Dictionary(StringComparer.Ordinal)); + + /// Every authored scope belonging to one edge node — the NDEATH fan-out set. + /// The edge node. + /// The scopes, node-own and device alike. + public IEnumerable ScopesUnder(SparkplugNodeKey node) => + ByScope.Keys.Where(s => s.IsUnder(node.GroupId, node.EdgeNodeId)); + + /// Builds the snapshot from the mapped definitions. + /// The mapped definitions, keyed by RawPath. + /// The immutable snapshot. + public static AuthoredTable Build(IReadOnlyDictionary byRawPath) + { + var byMetric = new Dictionary>(); + var byScope = new Dictionary>(); + var nodes = new HashSet(); + + foreach (var def in byRawPath.Values) + { + // The factory guarantees all three; a definition that reached here without them could + // never be routed, so it is skipped rather than indexed under a nonsense key. + if (def.GroupId is not { } group || def.EdgeNodeId is not { } node || def.MetricName is not { } metric) + { + continue; + } + + var scope = new SparkplugScope(group, node, def.DeviceId); + nodes.Add(new SparkplugNodeKey(group, node)); + + if (!byMetric.TryGetValue(new SparkplugMetricKey(group, node, def.DeviceId, metric), out var metricList)) + { + byMetric[new SparkplugMetricKey(group, node, def.DeviceId, metric)] = metricList = []; + } + + metricList.Add(def); + + if (!byScope.TryGetValue(scope, out var scopeList)) + { + byScope[scope] = scopeList = []; + } + + scopeList.Add(def); + } + + return new AuthoredTable( + byRawPath.ToFrozenDictionary(StringComparer.Ordinal), + byMetric.ToFrozenDictionary(kv => kv.Key, kv => kv.Value.ToArray()), + byScope.ToFrozenDictionary(kv => kv.Key, kv => kv.Value.ToArray()), + nodes.ToFrozenSet()); + } + } +} + +/// +/// One edge node's identity — the unit Sparkplug sequences messages by, and the unit a rebirth NCMD +/// is addressed to. +/// +/// The Sparkplug group id. +/// The Sparkplug edge-node id. +public readonly record struct SparkplugNodeKey(string GroupId, string EdgeNodeId) +{ + /// + public override string ToString() => $"{GroupId}/{EdgeNodeId}"; +} + +/// +/// An authored tag's Sparkplug binding key: the stable tuple a tag is resolved by, across every +/// rebirth. The metric name is part of it; the per-birth alias never is. +/// +/// The Sparkplug group id. +/// The Sparkplug edge-node id. +/// The Sparkplug device id, or for a node-level metric. +/// The stable metric name. +public readonly record struct SparkplugMetricKey( + string GroupId, + string EdgeNodeId, + string? DeviceId, + string MetricName) +{ + /// + public override string ToString() => + DeviceId is null + ? $"{GroupId}/{EdgeNodeId}:{MetricName}" + : $"{GroupId}/{EdgeNodeId}/{DeviceId}:{MetricName}"; +} + +/// An observed Sparkplug primary-host STATE. +/// The host id the STATE topic named. +/// Whether the host declared itself online. +/// When this driver observed it. +public sealed record SparkplugHostState(string HostId, bool Online, DateTime ObservedUtc); + +/// Event payload for . +/// The observed state. +public sealed class SparkplugHostStateEventArgs(SparkplugHostState state) : EventArgs +{ + /// The observed state. + public SparkplugHostState State { get; } = state; +} + +/// +/// Event payload for — the Task-22 rediscovery seam. +/// +/// The scope whose birth certificate was applied. +/// The named metrics the birth declared, in wire order. +public sealed class SparkplugBirthObservedEventArgs(SparkplugScope scope, IReadOnlyList metricNames) + : EventArgs +{ + /// The scope whose birth certificate was applied. + public SparkplugScope Scope { get; } = scope; + + /// The named metrics the birth declared. + public IReadOnlyList MetricNames { get; } = metricNames; +} + +/// +/// Coerces a Sparkplug wire value — after +/// has undone the two's-complement encoding — onto +/// an authored tag's . +/// +/// +/// +/// A value that does not fit is refused, never silently converted. Same rule the +/// plain-MQTT path follows: a wrong value delivered at Good quality is worse than no value. +/// The one deliberate latitude is an exactly integral real (5.0) for an integer +/// tag — edge gateways emit those routinely — while a genuinely fractional one is refused +/// rather than rounded. +/// +/// +/// Sparkplug DateTime is epoch milliseconds, carried in the unsigned 64-bit +/// field, not an ISO string. Reading it as a number and calling it a timestamp is the whole +/// conversion. +/// +/// +internal static class SparkplugValueCoercion +{ + /// Coerces onto . + /// The reinterpreted wire value. + /// The tag's effective data type. + /// The coerced value when this returns . + /// when the value fits the target type. + public static bool TryCoerce(object? raw, DriverDataType target, out object? value) + { + value = null; + if (raw is null) + { + return false; + } + + try + { + switch (target) + { + case DriverDataType.String: + case DriverDataType.Reference: + // Bytes/File metrics map to String per the datatype map; base64 is the v1 fallback + // the design names, and it round-trips where a lossy UTF-8 decode would not. + value = raw switch + { + string s => s, + byte[] bytes => Convert.ToBase64String(bytes), + bool b => b ? "true" : "false", + _ => Convert.ToString(raw, CultureInfo.InvariantCulture), + }; + return value is not null; + + case DriverDataType.Boolean: + return TryCoerceBoolean(raw, out value); + + case DriverDataType.Int16: + case DriverDataType.Int32: + case DriverDataType.Int64: + case DriverDataType.UInt16: + case DriverDataType.UInt32: + case DriverDataType.UInt64: + return TryCoerceInteger(raw, target, out value); + + case DriverDataType.Float32: + if (raw is string f32Text) + { + return float.TryParse(f32Text, NumberStyles.Float, CultureInfo.InvariantCulture, out var pf) + && Assign(pf, out value); + } + + var asSingle = Convert.ToSingle(raw, CultureInfo.InvariantCulture); + + // Convert.ToSingle SATURATES rather than throwing: a Double of 1e300 onto a Float32 + // tag becomes +Infinity, which would publish at GOOD quality as a number no gauge + // can render and no comparison behaves sensibly against. A source that was itself + // infinite is passed through — that is the publisher's own value, not our overflow. + if (!float.IsFinite(asSingle) && IsFiniteSource(raw)) + { + return false; + } + + value = asSingle; + return true; + + case DriverDataType.Float64: + if (raw is string f64Text) + { + return double.TryParse(f64Text, NumberStyles.Float, CultureInfo.InvariantCulture, out var pd) + && Assign(pd, out value); + } + + var asDouble = Convert.ToDouble(raw, CultureInfo.InvariantCulture); + if (!double.IsFinite(asDouble) && IsFiniteSource(raw)) + { + return false; + } + + value = asDouble; + return true; + + case DriverDataType.DateTime: + return TryCoerceDateTime(raw, out value); + + default: + return false; + } + } + catch (Exception ex) when (ex is OverflowException or FormatException or InvalidCastException) + { + // Out of range, unparseable, or a wire type that cannot become the target at all. All three + // are "does not fit", which is a Bad status on one tag — never an exception on the + // dispatcher thread. + value = null; + return false; + } + } + + private static bool Assign(T parsed, out object? value) + { + value = parsed; + return true; + } + + /// + /// Whether the wire value was itself a finite number — i.e. whether a non-finite result is + /// our narrowing overflow rather than the publisher's own Infinity/NaN. + /// + private static bool IsFiniteSource(object raw) => raw switch + { + float f => float.IsFinite(f), + double d => double.IsFinite(d), + _ => true, + }; + + private static bool TryCoerceBoolean(object raw, out object? value) + { + switch (raw) + { + case bool b: + value = b; + return true; + case string s when bool.TryParse(s.Trim(), out var parsed): + value = parsed; + return true; + case string s: + value = null; + return long.TryParse(s.Trim(), NumberStyles.Integer, CultureInfo.InvariantCulture, out var n) + && Assign(n != 0L, out value); + case byte[]: + value = null; + return false; + default: + // Any numeric wire type: nonzero is true, matching the plain-mode path's rule. + value = Convert.ToDouble(raw, CultureInfo.InvariantCulture) != 0d; + return true; + } + } + + private static bool TryCoerceInteger(object raw, DriverDataType target, out object? value) + { + value = null; + + // Reals (and textual reals) must be exactly integral: rounding 5.5 to 5 or 6 is a wrong value, + // which is worse than no value. Parsed as decimal so the test and the conversion stay exact + // across the whole Int64/UInt64 range, where double loses precision above 2^53. + if (raw is float or double or decimal or string) + { + var text = raw as string ?? Convert.ToString(raw, CultureInfo.InvariantCulture); + if (text is null + || !decimal.TryParse(text, NumberStyles.Float, CultureInfo.InvariantCulture, out var dec) + || dec != decimal.Truncate(dec)) + { + return false; + } + + raw = dec; + } + + value = target switch + { + DriverDataType.Int16 => Convert.ToInt16(raw, CultureInfo.InvariantCulture), + DriverDataType.Int32 => Convert.ToInt32(raw, CultureInfo.InvariantCulture), + DriverDataType.Int64 => Convert.ToInt64(raw, CultureInfo.InvariantCulture), + DriverDataType.UInt16 => Convert.ToUInt16(raw, CultureInfo.InvariantCulture), + DriverDataType.UInt32 => Convert.ToUInt32(raw, CultureInfo.InvariantCulture), + DriverDataType.UInt64 => Convert.ToUInt64(raw, CultureInfo.InvariantCulture), + _ => null, + }; + + return value is not null; + } + + private static bool TryCoerceDateTime(object raw, out object? value) + { + value = null; + + switch (raw) + { + case DateTime dt: + value = dt; + return true; + + case string s: + return DateTime.TryParse( + s, + CultureInfo.InvariantCulture, + DateTimeStyles.RoundtripKind | DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal, + out var parsed) + && Assign(parsed, out value); + + case bool: + case byte[]: + return false; + + default: + // Sparkplug DateTime rides as epoch milliseconds in the unsigned 64-bit value field. + var ms = Convert.ToInt64(raw, CultureInfo.InvariantCulture); + if (ms < DateTimeOffset.MinValue.ToUnixTimeMilliseconds() + || ms > DateTimeOffset.MaxValue.ToUnixTimeMilliseconds()) + { + return false; + } + + value = DateTimeOffset.FromUnixTimeMilliseconds(ms).UtcDateTime; + return true; + } + } +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugTopic.cs b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugTopic.cs new file mode 100644 index 00000000..6f65be5b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/Sparkplug/SparkplugTopic.cs @@ -0,0 +1,291 @@ +using System.Diagnostics.CodeAnalysis; + +namespace ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Sparkplug; + +/// +/// The Sparkplug B topic-namespace element that identifies a message's purpose — the third +/// segment of spBv1.0/{group}/{type}/{node}[/{device}], or the second segment of the +/// differently-shaped spBv1.0/STATE/{hostId}. See design doc §3.1/§3.6 and Sparkplug B +/// v3.0 spec §6 (Topic Namespace Elements). +/// +public enum SparkplugMessageType +{ + /// Node birth certificate — (re)publishes an edge node's full metric/alias set. + NBIRTH, + + /// Device birth certificate — (re)publishes a device's full metric/alias set. + DBIRTH, + + /// Node data — incremental metric updates owned by the edge node itself. + NDATA, + + /// Device data — incremental metric updates for a device under the edge node. + DDATA, + + /// Node death certificate (the edge node's MQTT Will) — the node has gone offline. + NDEATH, + + /// Device death certificate — the device has gone offline (published by its edge node). + DDEATH, + + /// Node command — a write/rebirth request addressed to the edge node. + NCMD, + + /// Device command — a write request addressed to a device under the edge node. + DCMD, + + /// + /// Primary-host online/offline state. Shaped differently from every other message type — see + /// the remarks on and . + /// + STATE, +} + +/// +/// A parsed Sparkplug B MQTT topic. See design doc §3.1/§3.6 and Sparkplug B v3.0 spec §6. +/// +/// The message's purpose. +/// +/// The Sparkplug group id. for , +/// always populated otherwise. +/// +/// +/// The Sparkplug edge-node id. for , +/// always populated otherwise. +/// +/// +/// The Sparkplug device id, present only for the device-scoped message types +/// (// +/// /); +/// for node-scoped messages and for . +/// +/// +/// The primary-host id, populated only for ; carries the +/// third topic segment of spBv1.0/STATE/{hostId} (or the second segment of the legacy +/// pre-3.0 STATE/{hostId} form). for every other message type. +/// +/// +/// +/// Never throws on arbitrary input. is the primitive everything +/// else is built on — it is fed every topic string the broker delivers under a +/// spBv1.0/{groupId}/# subscription, none of it validated ahead of time, so it returns +/// for anything malformed rather than throwing. is +/// a throwing convenience wrapper for call sites (tests, hand-built topics) that already know +/// the string is well-formed. +/// +/// +/// STATE does not fit the {group}/{type}/{node} mould — handled honestly, not +/// force-fit. The Sparkplug v3.0 spec shapes the primary-host state topic as +/// spBv1.0/STATE/{hostId}: the message-type segment sits where a group id would +/// otherwise be, and there is no edge-node or device segment at all. Pre-3.0 peers additionally +/// published a bare STATE/{hostId} with no spBv1.0 namespace prefix. This parser +/// targets the v3.0 form and tolerates the legacy one on receive (design §3.1); +/// and only ever produce the v3.0 form. A parsed STATE topic carries +/// its host id in and leaves // +/// — every other message type is the mirror image +/// (/ populated, null). +/// +/// +/// Group/edge-node/device/host segments are treated as opaque identifiers: validated only for +/// non-emptiness and the absence of the MQTT wildcard characters +/# (a broker +/// never delivers a PUBLISH on a topic containing either, so a topic string that does is +/// malformed, not a legitimate id worth preserving). No further character-set restriction is +/// applied — Sparkplug does not constrain id charsets beyond "not a topic-level separator". +/// +/// +public sealed record SparkplugTopic( + SparkplugMessageType Type, + string? GroupId, + string? EdgeNodeId, + string? DeviceId, + string? HostId) +{ + private const string Namespace = "spBv1.0"; + + /// + /// Attempts to parse as a Sparkplug B topic. Never throws — returns + /// (and a ) for + /// anything that is not a well-formed Sparkplug topic, including /empty + /// input, wrong namespace, an unrecognised message-type segment, a device-scoped message + /// missing its device segment (or vice versa), or a segment containing an MQTT wildcard. + /// + public static bool TryParse(string? topic, [NotNullWhen(true)] out SparkplugTopic? result) + { + result = null; + if (string.IsNullOrEmpty(topic)) + { + return false; + } + + var segments = topic.Split('/'); + + // Legacy pre-3.0 STATE form: `STATE/{hostId}`, no `spBv1.0` namespace prefix. + if (segments.Length == 2 && segments[0] == "STATE") + { + return TryBuildState(segments[1], out result); + } + + if (segments.Length < 2 || segments[0] != Namespace) + { + return false; + } + + // v3.0 STATE form: `spBv1.0/STATE/{hostId}`. + if (segments[1] == "STATE") + { + return segments.Length == 3 && TryBuildState(segments[2], out result); + } + + // Every other message type: `spBv1.0/{group}/{type}/{node}[/{device}]`. + if (segments.Length is not (4 or 5)) + { + return false; + } + + if (!TryParseMessageType(segments[2], out var type)) + { + return false; + } + + var groupId = segments[1]; + var edgeNodeId = segments[3]; + if (!IsValidSegment(groupId) || !IsValidSegment(edgeNodeId)) + { + return false; + } + + var expectsDevice = IsDeviceScoped(type); + var hasDeviceSegment = segments.Length == 5; + if (expectsDevice != hasDeviceSegment) + { + return false; + } + + string? deviceId = null; + if (hasDeviceSegment) + { + deviceId = segments[4]; + if (!IsValidSegment(deviceId)) + { + return false; + } + } + + result = new SparkplugTopic(type, groupId, edgeNodeId, deviceId, null); + return true; + } + + /// + /// Parses , throwing if it is not a + /// well-formed Sparkplug B topic. See for the non-throwing form. + /// + public static SparkplugTopic Parse(string? topic) + { + if (TryParse(topic, out var result)) + { + return result; + } + + throw new FormatException($"'{topic}' is not a valid Sparkplug B topic."); + } + + /// + /// Builds a Sparkplug B topic string for a non-STATE message — e.g. + /// Format("Plant1", SparkplugMessageType.NCMD, "EdgeA") for the Task 20 write path, so + /// it does not have to hand-concatenate segments itself. + /// + /// + /// / is null/empty, or + /// is (use + /// instead — STATE has no group/edge-node/device segments). + /// + public static string Format(string groupId, SparkplugMessageType type, string edgeNodeId, string? deviceId = null) + { + ArgumentException.ThrowIfNullOrEmpty(groupId); + ArgumentException.ThrowIfNullOrEmpty(edgeNodeId); + if (type == SparkplugMessageType.STATE) + { + throw new ArgumentException("Use FormatState to build a STATE topic.", nameof(type)); + } + + return deviceId is null + ? $"{Namespace}/{groupId}/{type}/{edgeNodeId}" + : $"{Namespace}/{groupId}/{type}/{edgeNodeId}/{deviceId}"; + } + + /// Builds the v3.0 STATE topic string spBv1.0/STATE/{hostId}. + public static string FormatState(string hostId) + { + ArgumentException.ThrowIfNullOrEmpty(hostId); + return $"{Namespace}/STATE/{hostId}"; + } + + /// + /// Formats this instance back to its topic string (always the v3.0 STATE form for STATE + /// topics, even if this instance was parsed from the legacy no-prefix form). + /// + /// + /// A required field for this instance's is — i.e. + /// this instance was not built by / and violates the + /// invariants documented on the type. + /// + public string ToTopicString() => Type == SparkplugMessageType.STATE + ? FormatState(HostId ?? throw new InvalidOperationException("STATE topic is missing HostId.")) + : Format( + GroupId ?? throw new InvalidOperationException("Non-STATE topic is missing GroupId."), + Type, + EdgeNodeId ?? throw new InvalidOperationException("Non-STATE topic is missing EdgeNodeId."), + DeviceId); + + private static bool TryBuildState(string hostId, out SparkplugTopic? result) + { + result = null; + if (!IsValidSegment(hostId)) + { + return false; + } + + result = new SparkplugTopic(SparkplugMessageType.STATE, null, null, null, hostId); + return true; + } + + private static bool TryParseMessageType(string segment, out SparkplugMessageType type) + { + switch (segment) + { + case nameof(SparkplugMessageType.NBIRTH): + type = SparkplugMessageType.NBIRTH; + return true; + case nameof(SparkplugMessageType.DBIRTH): + type = SparkplugMessageType.DBIRTH; + return true; + case nameof(SparkplugMessageType.NDATA): + type = SparkplugMessageType.NDATA; + return true; + case nameof(SparkplugMessageType.DDATA): + type = SparkplugMessageType.DDATA; + return true; + case nameof(SparkplugMessageType.NDEATH): + type = SparkplugMessageType.NDEATH; + return true; + case nameof(SparkplugMessageType.DDEATH): + type = SparkplugMessageType.DDEATH; + return true; + case nameof(SparkplugMessageType.NCMD): + type = SparkplugMessageType.NCMD; + return true; + case nameof(SparkplugMessageType.DCMD): + type = SparkplugMessageType.DCMD; + return true; + default: + type = default; + return false; + } + } + + private static bool IsDeviceScoped(SparkplugMessageType type) => type is + SparkplugMessageType.DBIRTH or SparkplugMessageType.DDATA or SparkplugMessageType.DDEATH or SparkplugMessageType.DCMD; + + private static bool IsValidSegment(string segment) => + segment.Length > 0 && segment.IndexOf('+') < 0 && segment.IndexOf('#') < 0; +} diff --git a/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.csproj b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.csproj new file mode 100644 index 00000000..55ffef0b --- /dev/null +++ b/src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.csproj @@ -0,0 +1,29 @@ + + + + net10.0 + enable + enable + latest + true + true + $(NoWarn);CS1591 + ZB.MOM.WW.OtOpcUa.Driver.Mqtt + ZB.MOM.WW.OtOpcUa.Driver.Mqtt + + + + + + + + + + + + + + + + + diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/BrowserSessionService.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/BrowserSessionService.cs index ffb829db..4af233fc 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/BrowserSessionService.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/BrowserSessionService.cs @@ -1,5 +1,8 @@ +using Microsoft.AspNetCore.Authorization; +using Microsoft.AspNetCore.Components.Authorization; using Microsoft.Extensions.Logging; using ZB.MOM.WW.OtOpcUa.Commons.Browsing; +using ZB.MOM.WW.OtOpcUa.Security.Auth; namespace ZB.MOM.WW.OtOpcUa.AdminUI.Browsing; @@ -10,11 +13,19 @@ namespace ZB.MOM.WW.OtOpcUa.AdminUI.Browsing; /// expand/attributes call in a 20-second linked CTS so a stuck driver cannot /// stall the UI indefinitely. /// +/// The bespoke per-driver browsers. +/// The open-session registry. +/// The logger. +/// The Discover-backed fallback browser. +/// Policy evaluator for . +/// Supplies the calling circuit's user. public sealed class BrowserSessionService( IEnumerable browsers, BrowseSessionRegistry registry, ILogger logger, - IUniversalDriverBrowser universalBrowser) : IBrowserSessionService + IUniversalDriverBrowser universalBrowser, + IAuthorizationService authorizationService, + AuthenticationStateProvider authenticationStateProvider) : IBrowserSessionService { /// Upper bound on a single root/expand/attributes call. public static readonly TimeSpan PerCallTimeout = TimeSpan.FromSeconds(20); @@ -75,6 +86,64 @@ public sealed class BrowserSessionService( public Task> AttributesAsync(Guid token, string nodeId, CancellationToken ct) => InvokeAsync>(token, ct, (s, c) => s.AttributesAsync(nodeId, c)); + /// + /// + /// + /// The authorization check is the point of routing this through the service at all. + /// Every other member here is read-only; this one causes the browse session to publish onto + /// a live plant broker, so it is gated on the same DriverOperator policy the picker + /// bodies evaluate before rendering their Browse affordance. A render-time check alone is + /// not a control — it decides what a page draws, not what a circuit may invoke — so the + /// check is made here, where the action actually happens, and it fails closed. + /// + /// + /// Deliberately not wrapped in -style swallowing: unlike + /// a failed expand, a failed or refused rebirth is something the operator must see, so the + /// exception propagates to the caller for display. + /// + /// + public async Task RequestRebirthAsync(Guid token, string scope, CancellationToken ct) + { + if (!registry.TryGet(token, out var session)) + throw new BrowseSessionNotFoundException(token); + + if (session is not IRebirthCapableBrowseSession rebirthable) + { + throw new NotSupportedException( + $"Browse session {token} ({session.GetType().Name}) has no re-announce action."); + } + + var state = await authenticationStateProvider.GetAuthenticationStateAsync().ConfigureAwait(false); + var authorized = await authorizationService + .AuthorizeAsync(state.User, resource: null, AdminUiPolicies.DriverOperator) + .ConfigureAwait(false); + + if (!authorized.Succeeded) + { + logger.LogWarning( + "Rebirth request REFUSED for scope '{Scope}' on browse session {Token}: caller does not " + + "satisfy the {Policy} policy.", scope, token, AdminUiPolicies.DriverOperator); + throw new UnauthorizedAccessException( + $"Requesting a rebirth requires the {AdminUiPolicies.DriverOperator} policy."); + } + + using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct); + cts.CancelAfter(PerCallTimeout); + + var published = await rebirthable.RequestRebirthAsync(scope, cts.Token).ConfigureAwait(false); + + logger.LogInformation( + "Rebirth requested by {User} for scope '{Scope}' on browse session {Token}: {Published} " + + "message(s) published.", + state.User.Identity?.Name ?? "(anonymous)", scope, token, published); + + return published; + } + + /// + public bool CanRequestRebirth(Guid token) => + registry.TryGet(token, out var session) && session is IRebirthCapableBrowseSession { RebirthAvailable: true }; + /// public async Task CloseAsync(Guid token) { diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/IBrowserSessionService.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/IBrowserSessionService.cs index 9e25fbb5..0a70f888 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/IBrowserSessionService.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Browsing/IBrowserSessionService.cs @@ -50,6 +50,42 @@ public interface IBrowserSessionService /// The attributes of the node. Task> AttributesAsync(Guid token, string nodeId, CancellationToken ct); + /// + /// Asks the remote peer(s) addressed by to re-announce themselves, on + /// a session whose driver offers that action (today: MQTT in Sparkplug B mode, where it is a + /// Sparkplug rebirth-request NCMD). + /// + /// Registry handle for the open browse session. + /// The driver-specific target — for Sparkplug, a browse node id from the + /// session's own tree, or a bare {group}/{edgeNode}. + /// Cancellation token for the operation. + /// The number of request messages published. + /// + /// The only browse operation that writes to the device network, and therefore the only + /// one carrying an authorization check: the caller must satisfy the same + /// DriverOperator policy that gates the picker's Browse affordance. Everything else on + /// this facade is read-only. + /// + /// The token is unknown (or was reaped). + /// The caller is not a DriverOperator. + /// This session's driver has no re-announce action. + Task RequestRebirthAsync(Guid token, string scope, CancellationToken ct); + + /// + /// True when the open session behind can actually re-announce — i.e. + /// is worth offering. Cheap (no I/O); never throws; false for + /// an unknown/reaped token. + /// + /// Registry handle for the open browse session. + /// True when the picker should draw a Request-rebirth affordance. + /// + /// A capability probe, not an authorization check — the DriverOperator gate lives on + /// and fails closed there. The picker gates on both: this + /// decides whether the action exists at all (a plain-MQTT window has none), the policy decides + /// whether this operator may fire it. + /// + bool CanRequestRebirth(Guid token); + /// Removes the session from the registry and disposes it. No-op for unknown tokens. /// Registry handle for the browse session to close. /// A task that represents the asynchronous operation. diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceForm.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceForm.razor new file mode 100644 index 00000000..c44627be --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceForm.razor @@ -0,0 +1,61 @@ +@* MQTT device form — informational only. An MQTT driver instance holds exactly one broker session, + configured on the DRIVER (MqttDriverForm), so there is no per-device connection endpoint to author; + devices are pure organisational grouping in the RawPath. The DeviceConfig is round-tripped verbatim, + and Test-connect uses the driver's broker settings via the merged config. Same shape as + GalaxyDeviceForm. + + The warning below fires only for a pre-form deployment that authored the connection on the device: + DriverDeviceConfigMerger merges a SOLE device's keys up over the driver's, so those keys keep + winning over what the driver form shows. *@ +@using ZB.MOM.WW.OtOpcUa.AdminUI.Components.Shared.Drivers.DeviceForms + +
+
Connection
+
+ MQTT connects to a single broker configured on the driver — there is no per-device + endpoint to author here. Edit host / port / TLS / credentials on the driver's config. Devices under an + MQTT driver exist purely to group tags in the RawPath. +
+
+ +@if (_model.LegacyConnectionKeys.Count > 0) +{ +
+
Legacy connection keys on this device
+
+

+ This device's config still carries broker connection keys from before the driver form existed: + @string.Join(", ", _model.LegacyConnectionKeys). +

+

+ While this driver has exactly one device these keys are merged up and + override the driver form's values — so the driver form may not show what is + actually in effect. They are preserved (nothing is dropped), but the durable fix is to author the + connection on the driver and clear these keys: a second device on this driver stops the merge-up + entirely and the connection would vanish. +

+
+
+} + +@code { + /// The device's DeviceConfig JSON (round-tripped verbatim — MQTT has no per-device endpoint). + [Parameter] public string DeviceConfigJson { get; set; } = "{}"; + /// Fired when the (preserved) DeviceConfig changes — no-op UI, kept for the modal contract. + [Parameter] public EventCallback DeviceConfigJsonChanged { get; set; } + + private MqttDeviceModel _model = MqttDeviceModel.FromJson(null); + private string? _lastParsed; + + protected override void OnParametersSet() + { + if (!string.Equals(_lastParsed, DeviceConfigJson, StringComparison.Ordinal)) + { + _model = MqttDeviceModel.FromJson(DeviceConfigJson); + _lastParsed = DeviceConfigJson; + } + } + + /// Serialises the (preserved) DeviceConfig JSON. + public string GetConfigJson() => _model.ToJson(); +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceModel.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceModel.cs new file mode 100644 index 00000000..c49d0656 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceForms/MqttDeviceModel.cs @@ -0,0 +1,73 @@ +using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors; + +namespace ZB.MOM.WW.OtOpcUa.AdminUI.Components.Shared.Drivers.DeviceForms; + +/// +/// Device model for the MQTT / Sparkplug B driver. An MQTT driver instance holds exactly one +/// broker session, configured on the driver (MqttDriverForm) — so, like Galaxy, there +/// is no per-device connection endpoint to author and this model round-trips the DeviceConfig +/// JSON verbatim. Devices under an MQTT driver are pure organisational grouping in the RawPath. +/// +/// +/// +/// Why the connection is not authored here. DriverDeviceConfigMerger merges a device's +/// keys up to the top level only when the driver has exactly one device. A broker connection +/// authored on the device would therefore disappear from the merged blob the moment an operator adds +/// a second device — a silent, deploy-time-invisible outage. Driver-level authoring is correct for +/// 0, 1 or N devices. +/// +/// +/// exists because that merge-up still happens. A +/// pre-form deployment (hand-authored or SQL-seeded, as the P1 live gate had to do) may carry +/// Host/Port/… on a sole device's DeviceConfig, and those keys keep winning over +/// the driver form's values. They are preserved (never silently dropped) and reported so the device +/// form can tell the operator why the driver form's host is not the one in effect. +/// +/// +public sealed class MqttDeviceModel +{ + /// + /// MqttDriverOptions connection-surface key names that a legacy DeviceConfig may + /// carry and that would merge up over the driver form's values. + /// + private static readonly string[] ConnectionKeyNames = + [ + "Host", "Port", "ClientId", "UseTls", "AllowUntrustedServerCertificate", + "CaCertificatePath", "Username", "Password", "ProtocolVersion", "CleanSession", + "KeepAliveSeconds", "ConnectTimeoutSeconds", "ReconnectMinBackoffSeconds", + "ReconnectMaxBackoffSeconds", + ]; + + private System.Text.Json.Nodes.JsonObject _bag = new(); + + /// + /// The connection keys this DeviceConfig actually carries, in the casing they were + /// authored with — empty for the normal case. Non-empty means a sole-device merge-up will + /// override the driver form for those keys. + /// + public IReadOnlyList LegacyConnectionKeys { get; private set; } = []; + + /// Loads a model from a DeviceConfig JSON string, retaining every original key. + /// The raw DeviceConfig JSON string, or null for a new/empty device. + /// The populated . + public static MqttDeviceModel FromJson(string? json) + { + var bag = TagConfigJson.ParseOrNew(json); + return new MqttDeviceModel + { + _bag = bag, + LegacyConnectionKeys = bag + .Select(p => p.Key) + .Where(k => ConnectionKeyNames.Contains(k, StringComparer.OrdinalIgnoreCase)) + .ToList(), + }; + } + + /// Serialises the (preserved) DeviceConfig back to a JSON string. + /// The serialised DeviceConfig JSON string. + public string ToJson() => TagConfigJson.Serialize(_bag); + + /// Validation hook; MQTT device rows carry no endpoint, so always valid. + /// null — no per-device endpoint to validate. + public string? Validate() => null; +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceModal.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceModal.razor index a13dfce4..8d8d851e 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceModal.razor +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DeviceModal.razor @@ -65,6 +65,9 @@ case DriverTypeNames.Galaxy: break; + case DriverTypeNames.Mqtt: + + break; default:
No typed device form for driver type @_driverType.
break; diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverBrowseTree.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverBrowseTree.razor index 5437eff1..3b5124ad 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverBrowseTree.razor +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverBrowseTree.razor @@ -34,6 +34,12 @@ /// Fired when the user clicks a leaf (or any node — caller decides what to do with it). [Parameter] public EventCallback OnNodeSelected { get; set; } + /// The same click as , plus the node's captured ancestor-folder + /// path (root→parent) — which is how a caller learns a node's DEPTH, and therefore what kind of thing + /// it is in a driver whose tree levels mean different things (MQTT/Sparkplug: group / edge node / + /// device). Both callbacks fire; wiring only the first is the existing behaviour. + [Parameter] public EventCallback OnNodeSelectedWithPath { get; set; } + /// When true, leaves render a selection checkbox and clicking a leaf toggles it (WP6 "Browse /// device…" multi-select) instead of single-selecting. Folders still expand/navigate as usual. Default /// false preserves the single-select address-picker behavior. @@ -93,10 +99,12 @@ finally { item.Loading = false; StateHasChanged(); } } - private async Task SelectAsync(TreeItem item) + private async Task SelectAsync(TreeItem item, IReadOnlyList ancestorPath) { _selectedNodeIdLocal = item.Node.NodeId; await OnNodeSelected.InvokeAsync(item.Node); + await OnNodeSelectedWithPath.InvokeAsync( + new ZB.MOM.WW.OtOpcUa.AdminUI.Browsing.BrowseLeafSelection(item.Node, ancestorPath)); } private async Task ToggleLeafAsync(TreeItem item, IReadOnlyList ancestorPath) @@ -133,17 +141,27 @@ { } + @* BUTTONS, NOT . This is a Blazor Web App: blazor.web.js installs a + document-level click interceptor for enhanced navigation, and it resolves a bare "#" + against rather than the current URL — so clicking a node label + navigated the whole AdminUI to "/", tore down the circuit, and destroyed whatever modal + the tree was hosted in (losing the browse session AND the tag selection with it). + @onclick:preventDefault does not help: it suppresses the BROWSER's default action, not + Blazor's own interceptor. A } else { - @item.Node.DisplayName + } @if (item.Node.Kind == BrowseNodeKind.Leaf) { diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverConfigModal.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverConfigModal.razor index 16402154..7990462a 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverConfigModal.razor +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverConfigModal.razor @@ -60,14 +60,30 @@ case DriverTypeNames.Sql: break; + case DriverTypeNames.Mqtt: + + break; default:
No typed config form for driver type @_driverType.
break; } -

- The connection endpoint + Test-connect live on the driver's device — open the device modal to author host/port and verify connectivity. -

+ @* Galaxy + MQTT hold ONE connection per driver instance, so they author it here and + their device forms are informational. Every other driver splits the endpoint onto + the device (the v3 endpoint→DeviceConfig split). *@ + @if (_driverType is DriverTypeNames.Galaxy or DriverTypeNames.Mqtt) + { +

+ This driver holds a single connection, authored above. Test-connect lives on its + device — open the device modal to verify connectivity. +

+ } + else + { +

+ The connection endpoint + Test-connect live on the driver's device — open the device modal to author host/port and verify connectivity. +

+ } @if (_saveError is not null) { diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverForm.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverForm.razor new file mode 100644 index 00000000..cb18f6bc --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverForm.razor @@ -0,0 +1,331 @@ +@* Embeddable MQTT / Sparkplug B driver config form body. Hosted by DriverConfigModal. + + UNLIKE Modbus / S7 / OPC UA Client, the connection is authored HERE, not on the device: an MQTT + driver instance holds exactly ONE broker session, so MqttDeviceForm is informational (the Galaxy + precedent). DriverDeviceConfigMerger only merges a device's keys up when the driver has exactly one + device, so a device-authored broker connection would silently vanish the moment a second device + exists. All parsing/serialisation goes through the ONE shared MqttJson.Options instance, via the + pure MqttDriverFormModel — enums must round-trip by NAME (this repo's systemic driver-enum bug). + + The Sparkplug sub-object (groupId / hostId / actAsPrimaryHost / requestRebirthOnGap / + birthObservationWindowSeconds) is authored by the Mode == SparkplugB branch below. It is MERGED over + whatever the inbound blob had rather than replacing it, and in Plain mode it is not touched at all — + see MqttDriverFormModel.ToJson. Until the P2 live gate this branch was a "not implemented yet" + placeholder, which left the group id — the driver's entire subscription filter — unauthorable from + the UI after Sparkplug ingest had shipped. *@ +@using ZB.MOM.WW.OtOpcUa.AdminUI.Components.Shared.Drivers +@using ZB.MOM.WW.OtOpcUa.Driver.Mqtt + +@* Broker connection *@ +
+
Broker connection
+
+
+
+ + +
Broker hostname or IP. The connection lives on the driver — MQTT has one broker session per driver.
+
+
+ + +
8883 for TLS, 1883 for plaintext.
+
+
+ + +
Blank lets the client generate a unique id per connection. That is the correct setting.
+
+ @if (!string.IsNullOrWhiteSpace(_form.ClientId)) + { +
+ +
+ } +
+
+
+ +@* Transport security *@ +
+
Transport security
+
+
+
+
+ + +
+
Default on. Must match the broker's listener.
+
+
+
+ + +
+
Accepts any self-signed broker cert.
+
+
+ + +
PEM file pinning the broker's chain. Blank uses the OS trust store.
+
+ @if (_form.UseTls && _form.AllowUntrustedServerCertificate) + { +
+ +
+ } + @if (!_form.UseTls) + { +
+ +
+ } +
+
+
+ +@* Authentication *@ +
+
Authentication
+
+
+
+ + +
Blank connects without credentials.
+
+
+ + +
Stored in the driver config and carried in the deployment artifact. Never written to a log.
+
+
+
+
+ +@* Session *@ +
+
Session
+
+
+
+ + + @foreach (var v in Enum.GetValues()) + { + + } + +
Default V500 (MQTT 5.0).
+
+
+
+ + +
+
+
+ + +
Default 30 s.
+
+
+ + +
Default 15 s. Must be at least 1 — also bounds Test connect.
+
+
+ + +
Default 1 s.
+
+
+ + +
Default 30 s. Caps the exponential backoff.
+
+
+
+
+ +@* Ingest *@ +
+
Ingest
+
+
+
+ + + @foreach (var v in Enum.GetValues()) + { + + } + +
Plain = topic-bound tags. Sparkplug B = birth-described metrics under one group.
+
+
+ + +
Default 1048576 (1 MiB). A larger message is refused before decode.
+
+ + @if (_form.Mode == MqttMode.Plain) + { +
+ + +
Scopes browse/discovery. Per-tag topics are authored explicitly and unaffected.
+
+
+ + @* A plain select, not InputSelect: InputSelect natively parses string + enum only, + and QoS is an int. Same control shape MqttTagConfigEditor uses for per-tag QoS. *@ + +
Applied when a tag's own QoS is unset.
+
+ } + else + { + @* Sparkplug B. The group id is the ONE mandatory field: it is the driver's entire + subscription filter (spBv1.0/{GroupId}/#), so a blank one leaves the driver connected, + Healthy, and ingesting nothing. Validate() surfaces that inline; like every other knob + on this form it is ADVISORY — DriverConfigModal saves regardless — which is why the + model clamps on serialize instead of relying on the operator reading the notice. *@ +
+ + +
+ Subscribes spBv1.0/@(string.IsNullOrWhiteSpace(_form.SparkplugGroupId) ? "{GroupId}" : _form.SparkplugGroupId)/#. + No /, +, or #. +
+
+
+ + +
Default 15 s. How long browse/discovery collects births before calling the metric set stable.
+
+
+ + +
Sparkplug Host Application identity. Receive-only in this version.
+
+
+
+ + +
+
A detected seq gap asks the edge node to re-announce (NCMD), rather than running on stale metric state.
+
+ + +
+
+ + @if (_form.SparkplugActAsPrimaryHost) + { + @* Shown rather than the flag being hidden: an operator who needs a primary host must + learn it is absent HERE, not from a plant that never sees a STATE message. The + driver logs the same warning at startup so the two cannot drift. *@ +
+ +
+ } + +
+ +
+ } +
+
+
+ +@if (_validationError is not null) +{ +
@_validationError
+} + + + +@code { + /// The driver-level DriverConfig JSON (broker connection + ingest mode). + [Parameter] public string DriverConfigJson { get; set; } = "{}"; + /// Fired whenever a field changes — enables @bind-DriverConfigJson. + [Parameter] public EventCallback DriverConfigJsonChanged { get; set; } + /// The per-instance resilience-pipeline overrides JSON, or null. + [Parameter] public string? ResilienceConfig { get; set; } + /// Fired when the resilience overrides change. + [Parameter] public EventCallback ResilienceConfigChanged { get; set; } + + private MqttDriverFormModel _form = MqttDriverFormModel.FromJson(null); + private string? _lastParsed; + private string? _validationError; + + protected override void OnParametersSet() + { + // Re-parse only when the inbound value actually changed, so an unrelated parent re-render + // cannot clobber an in-progress edit. + if (!string.Equals(_lastParsed, DriverConfigJson, StringComparison.Ordinal)) + { + _form = MqttDriverFormModel.FromJson(DriverConfigJson); + _validationError = _form.Validate(); + _lastParsed = DriverConfigJson; + } + } + + /// Serialises the current config to DriverConfig JSON (PascalCase keys, enums as names). + public string GetConfigJson() => _form.ToJson(); + + // TryParse so a bad/empty change value can never throw into the Blazor circuit — it falls back to + // the driver's own default QoS rather than poisoning the config. + private Task OnDefaultQosChangedAsync(object? value) + { + _form.DefaultQos = int.TryParse(value?.ToString(), out var qos) ? qos : 1; + return EmitAsync(); + } + + private async Task EmitAsync() + { + _validationError = _form.Validate(); + var json = GetConfigJson(); + _lastParsed = json; + await DriverConfigJsonChanged.InvokeAsync(json); + } + + private async Task OnResilienceChanged(string? r) + { + ResilienceConfig = r; + await ResilienceConfigChanged.InvokeAsync(r); + } +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverFormModel.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverFormModel.cs new file mode 100644 index 00000000..a003bc63 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/Forms/MqttDriverFormModel.cs @@ -0,0 +1,430 @@ +using System.Text.Json; +using System.Text.Json.Nodes; +using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors; +using ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +namespace ZB.MOM.WW.OtOpcUa.AdminUI.Components.Shared.Drivers.Forms; + +/// +/// Typed working model behind MqttDriverForm — the MQTT driver's whole broker-connection and +/// ingest surface, as authored in the /raw driver-config modal. Kept as a plain class (not +/// nested in the razor) so every rule here is unit-testable; the razor is a thin shell over it. +/// +/// +/// +/// The connection lives on the DRIVER, not the device. MQTT holds exactly one broker session +/// per driver instance, so MqttDeviceForm is informational (the Galaxy precedent) and this +/// form owns host/port/TLS/credentials. Authoring them on the device would work only while the +/// driver has exactly ONE device — DriverDeviceConfigMerger merges a sole device's keys up to +/// the top level and stops doing so the moment a second device exists, at which point the broker +/// connection would silently vanish from the merged blob. +/// +/// +/// One JSON instance, always . Every MQTT config seam — the +/// runtime factory, the Test-connect probe, the driver's re-parse, the address-picker browser and +/// now this form — parses and serialises through that single shared instance. Divergent per-seam +/// options are this repo's documented systemic driver-enum bug: a form that wrote +/// "mode": 1 would be accepted by a seam carrying a JsonStringEnumConverter and fault +/// the one that does not, so "Test connect" goes green and the deployed driver dies. Consequently +/// the emitted keys are PascalCase (the shared instance carries no camelCase naming policy) +/// and enum values are names. +/// +/// +/// Preserving what this form does not author. The inbound blob's keys are retained in a bag +/// and the typed fields are written over it, so (a) the Sparkplug sub-object — P2, Task 21+ — +/// survives a load→save untouched, and (b) any key a newer driver adds is not silently dropped by an +/// older AdminUI. RawTags is the one key deliberately REMOVED: the deploy artifact injects it +/// via DriverDeviceConfigMerger, and a form-authored copy would be dead weight. +/// +/// +public sealed class MqttDriverFormModel +{ + /// + /// The property name the deploy artifact owns; never authored + /// here and stripped from the emitted blob. + /// + private const string RawTagsKey = nameof(MqttDriverOptions.RawTags); + + /// + /// The property name for the Sparkplug sub-object. Authored here + /// in mode and otherwise preserved verbatim from the inbound + /// blob — see for why the mode decides which of the two happens. + /// + private const string SparkplugKey = nameof(MqttDriverOptions.Sparkplug); + + /// + /// Characters that cannot appear in a Sparkplug group id. It is a literal MQTT topic segment + /// inside the driver's one subscription filter spBv1.0/{GroupId}/#, so a / would + /// silently widen the filter and a +/# is not even legal mid-segment. Mirrors + /// MqttTagConfigModel's identical rule on the tag side, so the driver and the tags bound + /// to it cannot disagree about what a group id may be. + /// + private static readonly char[] SparkplugGroupIdIllegalChars = ['/', '+', '#']; + + // --- Broker connection ---------------------------------------------------------------------- + + /// Broker hostname or IP address. + public string Host { get; set; } = "localhost"; + + /// Broker TCP port (8883 TLS / 1883 plaintext by convention). + public int Port { get; set; } = 8883; + + /// + /// MQTT client identifier sent at CONNECT. Blank is the correct default — see + /// . + /// + public string ClientId { get; set; } = ""; + + // --- Transport security --------------------------------------------------------------------- + + /// Connect over TLS. Defaults true; this driver ships no plaintext-by-default posture. + public bool UseTls { get; set; } = true; + + /// Accept any self-signed / untrusted broker certificate. Dev / on-prem escape hatch only. + public bool AllowUntrustedServerCertificate { get; set; } + + /// PEM CA file pinning the broker's TLS chain; blank uses the OS trust store. + public string CaCertificatePath { get; set; } = ""; + + // --- Authentication ------------------------------------------------------------------------- + + /// Username for broker authentication; blank connects without one. + public string Username { get; set; } = ""; + + /// + /// Password for broker authentication. Stored in the driver's config blob and carried in the + /// deployment artifact — the same plaintext-round-trip posture the OPC UA Client driver form + /// uses. Never logged: redacts it in its own member printer. + /// + public string Password { get; set; } = ""; + + // --- Session -------------------------------------------------------------------------------- + + /// MQTT protocol version negotiated at CONNECT. + public MqttProtocolVersion ProtocolVersion { get; set; } = MqttProtocolVersion.V500; + + /// Request a clean session (v3.1.1) / clean start (v5.0) at CONNECT. + public bool CleanSession { get; set; } = true; + + /// Keep-alive interval, in seconds, sent at CONNECT. + public int KeepAliveSeconds { get; set; } = 30; + + /// Bounded connect deadline, in seconds — the driver never hangs past this. + public int ConnectTimeoutSeconds { get; set; } = 15; + + /// Initial reconnect backoff, in seconds, after a connection drop. + public int ReconnectMinBackoffSeconds { get; set; } = 1; + + /// Cap on the exponential reconnect backoff, in seconds. + public int ReconnectMaxBackoffSeconds { get; set; } = 30; + + // --- Ingest --------------------------------------------------------------------------------- + + /// Selects the ingest shape — Plain topics or Sparkplug B (the latter is P2). + public MqttMode Mode { get; set; } = MqttMode.Plain; + + /// Ceiling on an inbound message body, in bytes; a larger message is refused before decode. + public int MaxPayloadBytes { get; set; } = 1024 * 1024; + + /// Plain mode: optional prefix used when the driver composes a topic (e.g. browse scoping). + public string TopicPrefix { get; set; } = ""; + + /// Plain mode: default subscription QoS applied when a tag's own qos is unset. + public int DefaultQos { get; set; } = 1; + + // --- Sparkplug B ---------------------------------------------------------------------------- + + /// + /// Sparkplug mode: the group id. Required — it is the driver's entire subscription scope + /// (spBv1.0/{GroupId}/#), so a blank one subscribes to nothing and the driver ingests + /// nothing while still reporting a healthy broker connection. + /// + public string SparkplugGroupId { get; set; } = ""; + + /// + /// Sparkplug mode: the Host Application identity. Optional, and receive-only in this + /// version — see . + /// + public string SparkplugHostId { get; set; } = ""; + + /// + /// Sparkplug mode: claim the Primary Host Application role. Not implemented — STATE + /// publishing is out of scope, and the driver logs a warning rather than being silently inert + /// when this is set. Surfaced (rather than hidden) so an operator who needs it learns that from + /// the form instead of from a quiet plant. + /// + public bool SparkplugActAsPrimaryHost { get; set; } + + /// Sparkplug mode: answer a detected sequence-number gap with a rebirth request (NCMD). + public bool SparkplugRequestRebirthOnGap { get; set; } = true; + + /// Sparkplug mode: how long browse/discovery collects births before calling the set stable. + public int SparkplugBirthObservationWindowSeconds { get; set; } = 15; + + /// + /// The inbound blob's keys, retained so any key a newer driver adds survives a load→save. + /// + private JsonObject _bag = new(); + + /// + /// The operator-facing consequence of pinning , surfaced by the form + /// whenever it is non-blank. Observed live during the P1 gate: MQTT requires client ids to be + /// unique per broker, and both nodes of a redundant pair run the driver — so a fixed id makes + /// them evict each other in a few-second reconnect loop while both still report Healthy. + /// + public const string ClientIdPairWarning = + "Both nodes of a redundant pair run this driver. MQTT client ids must be unique per broker, so a " + + "fixed client id makes the two nodes evict each other forever — they reconnect every few seconds " + + "while both still report Healthy. Leave this blank unless exactly one node will ever connect."; + + /// + /// Loads a model from a DriverConfig JSON string. Typed values are bound through + /// (the same instance the runtime factory uses, so what this form + /// shows is what the driver would see); the raw key bag is retained alongside for preservation. + /// Never throws — a blank/malformed blob degrades to the driver's own defaults. + /// + /// The raw DriverConfig JSON string, or null for a new driver. + /// The populated . + public static MqttDriverFormModel FromJson(string? json) + { + var bag = TagConfigJson.ParseOrNew(json); + var o = TryDeserialize(json) ?? new MqttDriverOptions(); + + return new MqttDriverFormModel + { + Host = o.Host, + Port = o.Port, + ClientId = o.ClientId ?? "", + UseTls = o.UseTls, + AllowUntrustedServerCertificate = o.AllowUntrustedServerCertificate, + CaCertificatePath = o.CaCertificatePath ?? "", + Username = o.Username ?? "", + Password = o.Password ?? "", + ProtocolVersion = o.ProtocolVersion, + CleanSession = o.CleanSession, + KeepAliveSeconds = o.KeepAliveSeconds, + ConnectTimeoutSeconds = o.ConnectTimeoutSeconds, + ReconnectMinBackoffSeconds = o.ReconnectMinBackoffSeconds, + ReconnectMaxBackoffSeconds = o.ReconnectMaxBackoffSeconds, + Mode = o.Mode, + MaxPayloadBytes = o.MaxPayloadBytes, + TopicPrefix = o.Plain?.TopicPrefix ?? "", + DefaultQos = o.Plain?.DefaultQos ?? new MqttPlainOptions().DefaultQos, + SparkplugGroupId = o.Sparkplug?.GroupId ?? "", + SparkplugHostId = o.Sparkplug?.HostId ?? "", + SparkplugActAsPrimaryHost = o.Sparkplug?.ActAsPrimaryHost ?? false, + // Defaulted from the record, not from a literal, so the form shows the driver's own default + // for a blob that has no Sparkplug sub-object at all. + SparkplugRequestRebirthOnGap = + o.Sparkplug?.RequestRebirthOnGap ?? new MqttSparkplugOptions().RequestRebirthOnGap, + SparkplugBirthObservationWindowSeconds = + o.Sparkplug?.BirthObservationWindowSeconds + ?? new MqttSparkplugOptions().BirthObservationWindowSeconds, + _bag = bag, + }; + } + + /// + /// Builds the typed options this form authors. Every numeric knob is clamped into the + /// range declares, so an operator who ignores + /// and saves anyway still cannot persist a driver-bricking blob — a + /// connectTimeoutSeconds: 0 is exactly the operator-authorable brick this repo has hit + /// before. stays empty (the deploy artifact owns it) and + /// is emitted only in + /// mode; both are finished by . + /// + /// The clamped, driver-legal options record. + public MqttDriverOptions ToOptions() => new() + { + Host = Host.Trim(), + Port = Math.Clamp(Port, 1, 65535), + ClientId = Blank(ClientId), + UseTls = UseTls, + AllowUntrustedServerCertificate = AllowUntrustedServerCertificate, + CaCertificatePath = Blank(CaCertificatePath), + Username = Blank(Username), + Password = Password, + ProtocolVersion = ProtocolVersion, + CleanSession = CleanSession, + KeepAliveSeconds = Math.Max(1, KeepAliveSeconds), + ConnectTimeoutSeconds = Math.Max(1, ConnectTimeoutSeconds), + ReconnectMinBackoffSeconds = Math.Max(1, ReconnectMinBackoffSeconds), + ReconnectMaxBackoffSeconds = Math.Max(1, ReconnectMaxBackoffSeconds), + Mode = Mode, + MaxPayloadBytes = Math.Max(1, MaxPayloadBytes), + Plain = new MqttPlainOptions + { + TopicPrefix = TopicPrefix.Trim(), + DefaultQos = Math.Clamp(DefaultQos, 0, 2), + }, + // Null in Plain mode so ToJson leaves whatever the inbound blob had; see its remarks. + Sparkplug = Mode == MqttMode.SparkplugB + ? new MqttSparkplugOptions + { + GroupId = SparkplugGroupId.Trim(), + HostId = SparkplugHostId.Trim(), + ActAsPrimaryHost = SparkplugActAsPrimaryHost, + RequestRebirthOnGap = SparkplugRequestRebirthOnGap, + BirthObservationWindowSeconds = Math.Max(1, SparkplugBirthObservationWindowSeconds), + } + : null, + }; + + /// + /// Serialises the authored fields over the preserved key bag and returns the DriverConfig + /// JSON. Keys are PascalCase and enums are names, because the serialisation runs through + /// — see the type remarks. RawTags is removed (the deploy + /// artifact owns it). + /// + /// + /// Sparkplug is merged, never replaced, and only in Sparkplug mode. In + /// the sub-object is left exactly as the inbound blob had it, so an + /// operator who flips a Sparkplug driver to Plain to look at it and flips back does not lose the + /// group id. In the five fields this form owns are written + /// over the existing sub-object rather than replacing it, so a key a newer driver adds + /// inside Sparkplug survives an older AdminUI — the same preserve-what-you-do-not-author + /// discipline the top-level bag applies. + /// + /// The serialised DriverConfig JSON string. + public string ToJson() + { + var typed = JsonSerializer.SerializeToNode(ToOptions(), MqttJson.Options)!.AsObject(); + + // Never let this form's always-empty placeholder overwrite what it does not own. + typed.Remove(RawTagsKey); + + // Pulled out of the flat copy loop below: unlike every other key, this one merges. + var authoredSparkplug = TakeIgnoringCase(typed, SparkplugKey) as JsonObject; + + foreach (var (key, value) in typed.ToList()) + { + // Case-insensitive replace: MqttJson.Options binds a hand-edited camelCase blob happily, so + // the bag may hold "host" while we emit "Host" — writing both would leave two keys fighting. + RemoveIgnoringCase(_bag, key); + _bag[key] = value?.DeepClone(); + } + + if (authoredSparkplug is not null) + { + // Merge over whatever was already there, under the key name the bag already uses so a + // camelCase hand-edited blob does not end up with both "sparkplug" and "Sparkplug". + var existingKey = FindIgnoringCase(_bag, SparkplugKey) ?? SparkplugKey; + if (_bag[existingKey] is not JsonObject target) + { + target = new JsonObject(); + _bag[existingKey] = target; + } + + foreach (var (key, value) in authoredSparkplug.ToList()) + { + RemoveIgnoringCase(target, key); + target[key] = value?.DeepClone(); + } + } + + RemoveIgnoringCase(_bag, RawTagsKey); + return TagConfigJson.Serialize(_bag); + } + + /// + /// Client-side validation surfaced inline by the form. Returns the first error, or null + /// when the model is valid. Every bound mirrors a [Range] on + /// (or, for QoS, on ), so the + /// authoring surface accepts exactly what the driver accepts. + /// + /// An error message describing the validation failure, or null when valid. + public string? Validate() + { + if (string.IsNullOrWhiteSpace(Host)) { return "A broker host is required."; } + if (Port is < 1 or > 65535) { return "Port must be between 1 and 65535."; } + if (KeepAliveSeconds < 1) { return "Keep-alive must be at least 1 second."; } + if (ConnectTimeoutSeconds < 1) + { + return "Connect timeout must be at least 1 second — 0 would make every connect attempt " + + "expire instantly and the driver would never come up."; + } + if (ReconnectMinBackoffSeconds < 1) { return "Minimum reconnect backoff must be at least 1 second."; } + if (ReconnectMaxBackoffSeconds < 1) { return "Maximum reconnect backoff must be at least 1 second."; } + if (ReconnectMaxBackoffSeconds < ReconnectMinBackoffSeconds) + { + return "Maximum reconnect backoff must not be below the minimum."; + } + if (MaxPayloadBytes < 1) { return "Max payload bytes must be at least 1."; } + if (DefaultQos is < 0 or > 2) { return "Default QoS must be 0, 1 or 2."; } + + if (Mode == MqttMode.SparkplugB) + { + var groupId = SparkplugGroupId.Trim(); + if (groupId.Length == 0) + { + return "A Sparkplug group ID is required — it is the driver's whole subscription scope " + + "(spBv1.0/{GroupId}/#), so a blank one ingests nothing."; + } + + if (groupId.IndexOfAny(SparkplugGroupIdIllegalChars) >= 0) + { + return $"Sparkplug group ID '{groupId}' contains a character ('/', '+', or '#') that " + + "cannot appear in an MQTT topic segment."; + } + + if (SparkplugBirthObservationWindowSeconds < 1) + { + return "Birth observation window must be at least 1 second."; + } + } + + return null; + } + + /// Blank ⇒ null so the key is omitted and the driver's own default applies. + /// The raw form value. + /// The trimmed value, or null when blank. + private static string? Blank(string? value) + => string.IsNullOrWhiteSpace(value) ? null : value.Trim(); + + /// Removes every key matching case-insensitively. + /// The object to mutate. + /// The key name to remove. + private static void RemoveIgnoringCase(JsonObject o, string name) + { + foreach (var key in o.Select(p => p.Key) + .Where(k => string.Equals(k, name, StringComparison.OrdinalIgnoreCase)) + .ToList()) + { + o.Remove(key); + } + } + + /// The actual key in matching case-insensitively. + /// The object to search. + /// The key name to look for. + /// The matching key as it is spelled in , or null. + private static string? FindIgnoringCase(JsonObject o, string name) + => o.Select(p => p.Key) + .FirstOrDefault(k => string.Equals(k, name, StringComparison.OrdinalIgnoreCase)); + + /// Removes and returns the value of a case-insensitively matched key. + /// The object to mutate. + /// The key name to take. + /// The detached value, or null when the key was absent (or its value was null). + private static JsonNode? TakeIgnoringCase(JsonObject o, string name) + { + if (FindIgnoringCase(o, name) is not { } key) { return null; } + + var value = o[key]; + o.Remove(key); + + // Detached before return: a node still parented to `typed` cannot be re-parented into the bag. + return value?.DeepClone(); + } + + /// Binds the blob through the shared options; null on blank/malformed input. + /// The raw DriverConfig JSON. + /// The bound options, or null. + private static MqttDriverOptions? TryDeserialize(string? json) + { + if (string.IsNullOrWhiteSpace(json)) { return null; } + try { return JsonSerializer.Deserialize(json, MqttJson.Options); } + catch (JsonException) { return null; } + } +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawBrowseModal.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawBrowseModal.razor index a3ba3e70..144a112f 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawBrowseModal.razor +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawBrowseModal.razor @@ -11,6 +11,8 @@ SupportsOnlineDiscovery (AbCip/TwinCAT/FOCAS); otherwise browse is unavailable and the modal grays out with a tooltip. *@ @implements IAsyncDisposable +@using Microsoft.AspNetCore.Authorization +@using Microsoft.AspNetCore.Components.Authorization @using ZB.MOM.WW.OtOpcUa.AdminUI.Browsing @using ZB.MOM.WW.OtOpcUa.AdminUI.Components.Shared.Drivers @using ZB.MOM.WW.OtOpcUa.AdminUI.Uns @@ -18,6 +20,8 @@ @inject IBrowserSessionService BrowserService @inject IRawTreeService Svc @inject RawTagCsvExportReader ExportReader +@inject AuthenticationStateProvider AuthState +@inject IAuthorizationService AuthorizationService @if (Visible) { @@ -66,7 +70,22 @@ else {
- Browser open +
+ Browser open + @* An observation window ACCUMULATES — the session keeps recording every + message after the tree was first rendered — but the tree itself loads + exactly once, when it is created. Without this the operator sees the + t=0 snapshot forever: on MQTT the first render is usually empty (a + topic that has not published yet is invisible) and on Sparkplug it is + almost always empty, because births are never retained. Re-keying the + tree re-runs its root load against the SAME session, so nothing + reconnects and nothing already observed is lost. *@ + +
@@ -76,8 +95,90 @@
- + + + @if (_canRebirth && _canOperate) + { + @* Sparkplug only: births are never retained, so a quiet-but-healthy plant + shows an EMPTY tree until one lands. This is the only way to fill it on + demand — and the only browse action that publishes to the live broker, + hence the explicit confirm step. *@ +
+
+
+ Request rebirth + + — asks edge nodes to republish their metric set, so this tree fills + without waiting for a natural birth. + +
+ @if (!_rebirthConfirming) + { + + } +
+ + @if (_rebirthTarget is null) + { +
+ Click a group, edge node, device or metric in the tree above to choose the scope. +
+ } + else + { +
+ Scope: @_rebirthTarget.Kind + @_rebirthTarget.Scope +
+
@_rebirthTarget.Detail
+ } + + @if (_rebirthConfirming && _rebirthTarget is not null) + { +
+
+ This publishes to the live broker: a Sparkplug + rebirth-request command addressed to @_rebirthTarget.Kind + (@_rebirthTarget.Scope). + @if (_rebirthTarget.IsGroup) + { + + Every edge node observed in this group is addressed; a group with + more than @MqttGroupRebirthNodeCap observed edge nodes is refused + outright, and nothing is published. + + } +
+
+ + +
+
+ } + + @if (_rebirthOutcome is not null) + { +
@_rebirthOutcome
+ } + @if (_rebirthError is not null) + { +
@_rebirthError
+ } +
+ }
@_selected.Count tag@(_selected.Count == 1 ? "" : "s") selected. @@ -116,6 +217,11 @@ /// type (the OPC UA Client tree). Driver-typed browsers (AbCip/TwinCAT/FOCAS) report the real type. private const string DefaultDataType = "Double"; + /// Mirrors MqttBrowseSession.MaxGroupRebirthNodes for the confirm-step wording. The + /// session owns the rule and refuses whole past it; this is the operator-facing number, and a drift + /// only ever makes the warning less precise, never the refusal wrong. + private const int MqttGroupRebirthNodeCap = 32; + /// Whether the modal is shown. Two-way (@bind-Visible) so the modal can self-close. [Parameter] public bool Visible { get; set; } @@ -154,9 +260,32 @@ private bool _busy; private List _commitErrors = new(); + /// + /// Bumped to force a fresh DriverBrowseTree against the same session — see the Refresh + /// button's remarks. Seeded from the session so re-opening a modal always starts a new generation. + /// + private int _treeGeneration; + + // Request-rebirth affordance (MQTT/Sparkplug only). _canOperate is defence in depth — the real gate + // is server-side in BrowserSessionService.RequestRebirthAsync, which fails closed. + private bool _canOperate; + private bool _canRebirth; + private RebirthTarget? _rebirthTarget; + private bool _rebirthConfirming; + private bool _rebirthBusy; + private string? _rebirthOutcome; + private string? _rebirthError; + private readonly Dictionary _selected = new(StringComparer.Ordinal); private readonly HashSet _selectedIds = new(StringComparer.Ordinal); + protected override async Task OnInitializedAsync() + { + var auth = await AuthState.GetAuthenticationStateAsync(); + var result = await AuthorizationService.AuthorizeAsync(auth.User, null, AdminUiPolicies.DriverOperator); + _canOperate = result.Succeeded; + } + protected override async Task OnParametersSetAsync() { if (!Visible) @@ -185,6 +314,8 @@ _groupPrefix = null; _canBrowse = false; _disabledReason = null; + ResetRebirthState(); + _canRebirth = false; _resolving = true; StateHasChanged(); @@ -238,7 +369,13 @@ try { var result = await BrowserService.OpenAsync(_driverType, _mergedConfig, default); - if (result.Ok) { _token = result.Token; } + if (result.Ok) + { + _token = result.Token; + // Capability, not authorization: a plain-MQTT window (and every non-MQTT browser) has no + // re-announce action at all, so the affordance must not be drawn for it. + _canRebirth = BrowserService.CanRequestRebirth(_token); + } else { _openError = result.Message; } } finally @@ -263,6 +400,7 @@ // the display name + the default data type. string name = sel.Leaf.DisplayName; string? driverDataType = null; + IReadOnlyDictionary? addressFields = null; try { var attrs = await BrowserService.AttributesAsync(_token, nodeId, default); @@ -270,15 +408,164 @@ { name = string.IsNullOrWhiteSpace(attrs[0].Name) ? sel.Leaf.DisplayName : attrs[0].Name; driverDataType = attrs[0].DriverDataType; + // The STRUCTURED address, for a driver whose binding is a tuple (MQTT/Sparkplug). Never + // reconstructed from the node id: a metric name may contain '/', so the id cannot be + // split back into (group, edgeNode, device?, metric) — see RawBrowseCommitMapper. + addressFields = attrs[0].AddressFields; } } catch { // Attribute lookup is best-effort — a leaf is still selectable with display-name + default type. + // For a tuple-addressed driver that also means "no address", which CommitAsync refuses in words. } _selectedIds.Add(nodeId); - _selected[nodeId] = new SelectedLeaf(nodeId, name, driverDataType, sel.FolderPath); + _selected[nodeId] = new SelectedLeaf(nodeId, name, driverDataType, sel.FolderPath, addressFields); + + // A metric is also a legal rebirth scope (it resolves up to its owning edge node), so a toggle + // moves the scope the same way a folder click does. + SetRebirthTarget(sel.Leaf, sel.FolderPath); + } + + /// Tracks the tree click that defines the Request-rebirth scope (folders and metrics alike). + private void OnScopeNodeSelected(BrowseLeafSelection sel) => SetRebirthTarget(sel.Leaf, sel.FolderPath); + + /// + /// Re-derives the rebirth scope from the clicked node, and — because the scope changed — drops any + /// pending confirmation. A confirm step that survived a selection change would publish to a target + /// the operator is no longer looking at. + /// + private void SetRebirthTarget(BrowseNode node, IReadOnlyList ancestorPath) + { + if (!_canRebirth) { return; } + + var target = DescribeRebirthTarget(node, ancestorPath); + if (_rebirthTarget?.Scope == target.Scope) { return; } + + _rebirthTarget = target; + _rebirthConfirming = false; + _rebirthOutcome = null; + _rebirthError = null; + } + + /// + /// Names what a clicked node means as a rebirth scope. The DEPTH comes from the captured ancestor + /// path, not from splitting the node id — the id is the session's own key and the AdminUI must not + /// reverse-engineer its shape. + /// + /// The clicked browse node; its NodeId is passed to the session verbatim. + /// The node's ancestor display names, root→parent. + /// The described target. + private static RebirthTarget DescribeRebirthTarget(BrowseNode node, IReadOnlyList ancestorPath) + { + var depth = ancestorPath.Count; + var name = node.DisplayName; + + // Sparkplug has no device- or metric-scoped rebirth: an NCMD addresses an EDGE NODE, so a + // deeper selection resolves upward. Say so, rather than letting the operator assume the + // narrower scope they clicked. + if (node.Kind == BrowseNodeKind.Leaf && depth >= 2) + { + return new RebirthTarget( + node.NodeId, + $"edge node {ancestorPath[1]}", + $"Selected the metric '{name}'. Sparkplug addresses rebirth at the edge node, so " + + $"{ancestorPath[1]} republishes its whole metric set.", + IsGroup: false); + } + + return depth switch + { + 0 => new RebirthTarget( + node.NodeId, + $"group {name}", + $"Every edge node observed in group '{name}' is asked to republish.", + IsGroup: true), + 1 => new RebirthTarget( + node.NodeId, + $"edge node {name}", + $"Edge node '{name}' in group '{ancestorPath[0]}' republishes its whole metric set.", + IsGroup: false), + 2 => new RebirthTarget( + node.NodeId, + $"edge node {ancestorPath[1]}", + $"Selected the device '{name}'. Sparkplug addresses rebirth at the edge node, so " + + $"{ancestorPath[1]} republishes its whole metric set.", + IsGroup: false), + _ => new RebirthTarget(node.NodeId, "the selected node", "", IsGroup: false), + }; + } + + /// Arms the confirm step. Publishing is deliberately two clicks — it reaches a live plant broker. + private void BeginRebirth() + { + _rebirthOutcome = null; + _rebirthError = null; + _rebirthConfirming = true; + } + + /// Disarms the confirm step without publishing anything. + private void CancelRebirth() => _rebirthConfirming = false; + + /// + /// The one browse action that publishes. Exceptions are SHOWN, not swallowed: a refusal (an + /// over-wide group scope, a missing DriverOperator grant, an unresolvable scope) is exactly what + /// the operator needs to read, and nothing was published in any of those cases. + /// + private async Task ConfirmRebirthAsync() + { + if (_rebirthTarget is null) { return; } + _rebirthBusy = true; + _rebirthOutcome = null; + _rebirthError = null; + StateHasChanged(); + try + { + var published = await BrowserService.RequestRebirthAsync(_token, _rebirthTarget.Scope, default); + _rebirthOutcome = + $"Rebirth requested for {_rebirthTarget.Kind} — {published} command" + + (published == 1 ? "" : "s") + + " published. Re-expand the tree in a few seconds to see the republished metrics."; + _rebirthConfirming = false; + } + catch (Exception ex) + { + _rebirthError = ex.Message; + } + finally + { + _rebirthBusy = false; + StateHasChanged(); + } + } + + /// + /// Re-reads the root of the open session's observed tree. Only the tree is rebuilt: the session, + /// its broker connection and everything it has recorded are untouched, so this is strictly a + /// re-render and never re-observes from scratch. + /// + /// + /// The tag selection is deliberately kept — _selectedIds is keyed by browse node id + /// and a refresh does not renumber anything, so an operator who ticked leaves, waited for more + /// of the plant to appear, and refreshed does not silently lose the ticks. The rebirth scope IS + /// cleared, because the node it pointed at may no longer be in the rebuilt tree and a stale + /// armed scope is exactly the thing that panel's two-click confirm exists to prevent. + /// + private void RefreshTree() + { + _treeGeneration++; + ResetRebirthState(); + } + + /// Clears every rebirth-panel field (modal re-open, browser close). + private void ResetRebirthState() + { + _rebirthTarget = null; + _rebirthConfirming = false; + _rebirthBusy = false; + _rebirthOutcome = null; + _rebirthError = null; } private async Task CommitAsync() @@ -288,10 +575,21 @@ StateHasChanged(); try { + // Refuse a selection the driver could not bind, IN WORDS — for a tuple-addressed driver + // (MQTT/Sparkplug) a leaf whose attribute lookup returned nothing carries no address, and + // committing it would produce a tag that deploys clean and then reports BadNodeIdUnknown + // forever with nothing to point at. + _commitErrors = _selected.Values + .Select(s => RawBrowseCommitMapper.DescribeUncommittableLeaf(_driverType, s.Name, s.AddressFields)) + .Where(e => e is not null) + .Select(e => e!) + .ToList(); + if (_commitErrors.Count > 0) { return; } + var rows = _selected.Values .Select(s => RawBrowseCommitMapper.MapLeaf( _driverType, s.NodeId, s.Name, s.DriverDataType, DefaultDataType, - _groupPrefix, s.FolderPath, _createGroups)) + _groupPrefix, s.FolderPath, _createGroups, s.AddressFields)) .ToList(); var outcome = await Svc.ImportTagsAsync(_effectiveDeviceId, rows); @@ -314,6 +612,8 @@ { var t = _token; _token = Guid.Empty; + _canRebirth = false; + ResetRebirthState(); Visible = false; _open = false; await VisibleChanged.InvokeAsync(false); @@ -335,5 +635,19 @@ return ValueTask.CompletedTask; } - private sealed record SelectedLeaf(string NodeId, string Name, string? DriverDataType, IReadOnlyList FolderPath); + /// The structured address the browse session stated for this leaf, for a + /// driver whose binding is a tuple rather than the NodeId itself; null for every other driver. + private sealed record SelectedLeaf( + string NodeId, + string Name, + string? DriverDataType, + IReadOnlyList FolderPath, + IReadOnlyDictionary? AddressFields = null); + + /// A described Request-rebirth scope: what gets published, and to what. + /// The browse node id handed to the session verbatim. + /// What the scope resolves to, in operator words (e.g. "edge node EdgeA"). + /// The one-line consequence, spelled out before the confirm step. + /// True for a whole-group fan-out — the bounded, all-or-nothing case. + private sealed record RebirthTarget(string Scope, string Kind, string Detail, bool IsGroup); } diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawDriverTypeDialog.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawDriverTypeDialog.razor index 24b37fa2..3ddc867f 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawDriverTypeDialog.razor +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Raw/RawDriverTypeDialog.razor @@ -67,6 +67,7 @@ ("TwinCAT", DriverTypeNames.TwinCAT), ("FOCAS", DriverTypeNames.FOCAS), ("OpcUaClient", DriverTypeNames.OpcUaClient), + ("MQTT", DriverTypeNames.Mqtt), ("Galaxy", DriverTypeNames.Galaxy), ("Sql", DriverTypeNames.Sql), ("Calculation", "Calculation"), diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor new file mode 100644 index 00000000..456c1626 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Uns/TagEditors/MqttTagConfigEditor.razor @@ -0,0 +1,212 @@ +@* Typed TagConfig editor for the Mqtt driver. Same (ConfigJson/ConfigJsonChanged/DriverType/ + GetDriverConfigJson) parameter shape every typed editor takes; the last two are accepted for + dispatch uniformity and unused here (MQTT has no address-builder picker — topics are discovered + through the /raw browse tree, not composed from parts). + + The shape switch is real (it drives which field group renders and what Validate() applies) but it + is a UI-only choice derived from the blob, never serialised — the driver takes Plain vs SparkplugB + from its DRIVER config, and the tag blob has no 'mode' key. A SparkplugB tag survives a save→reopen + because its descriptor keys (groupId/edgeNodeId/deviceId/metricName) re-infer the mode on load — + there is nothing else that needs to persist. *@ +@using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors +@using ZB.MOM.WW.OtOpcUa.Core.Abstractions +@using ZB.MOM.WW.OtOpcUa.Driver.Mqtt + +
+
+ + +
+ + @if (_m.Mode == MqttMode.Plain) + { +
+ + +
Concrete topic — MQTT wildcards (+ / #) are not valid for a tag.
+
+
+ + +
+ @if (_m.PayloadFormat == MqttPayloadFormat.Json) + { +
+ + +
Use $ for the whole document.
+
+ } +
+ + @* The driver's own DriverDataType members — authoring anything outside this set produces a + blob the driver's strict parser rejects at deploy (there is no 'Double'; it is Float64). *@ + +
+
+ + +
+
+ + +
+ } + else + { +
+ + +
The driver subscribes spBv1.0/@(string.IsNullOrEmpty(_m.GroupId) ? "{GroupId}" : _m.GroupId)/#. No /, +, or #.
+
+
+ + +
The publishing edge node's id.
+
+
+ + +
Leave blank for a metric published by the edge node itself.
+
+
+ + +
The metric's stable name from the birth certificate — unlike the ids above, this MAY contain / (e.g. Properties/Hardware Make).
+
+
+ + @* Same DriverDataType set as Plain's field — the factory reads a Sparkplug 'dataType' key as + a DriverDataType override, not the raw wire SparkplugDataType. Absent ⇒ take whatever type + the birth certificate declares (the driver's UntilStable discovery). *@ + +
+
+ + +
+
+ + +
+ } + + @if (_validationError is not null) + { +
@_validationError
+ } +
+ +@code { + /// The tag's TagConfig JSON, owned by the host modal. + [Parameter] public string? ConfigJson { get; set; } + /// Raised with the re-serialised TagConfig JSON after every field edit. + [Parameter] public EventCallback ConfigJsonChanged { get; set; } + /// DriverType of the owning driver — accepted for dispatch uniformity; unused here. + [Parameter] public string DriverType { get; set; } = ""; + /// Live accessor for the owning driver's DriverConfig JSON — accepted for dispatch uniformity; unused here. + [Parameter] public Func GetDriverConfigJson { get; set; } = () => "{}"; + + private MqttTagConfigModel _m = new(); + private string? _lastConfigJson; + private string? _validationError; + + // Re-parse only when the incoming JSON actually changes, so an unrelated parent re-render + // (Blazor Server live-status pushes do this) can't reset the user's in-progress edits. + protected override void OnParametersSet() + { + if (ConfigJson == _lastConfigJson) { return; } + _lastConfigJson = ConfigJson; + _m = MqttTagConfigModel.FromJson(ConfigJson); + _validationError = _m.Validate(); + } + + // TryParse so a bad/empty change value can never throw into the Blazor circuit — it falls back. + private static TEnum ParseEnum(object? v, TEnum fallback) where TEnum : struct, Enum + => Enum.TryParse(v?.ToString(), out var r) ? r : fallback; + + // "" (the "(from birth certificate)" sentinel option) ⇒ null ⇒ the key is omitted. + private static TEnum? ParseNullableEnum(object? v) where TEnum : struct, Enum + => Enum.TryParse(v?.ToString(), out var r) ? r : null; + + // "" ⇒ null ⇒ the key is omitted and the driver's own default applies. + private static int? ParseNullableInt(object? v) + => int.TryParse(v?.ToString(), out var i) ? i : null; + + private static bool? ParseNullableBool(object? v) + => bool.TryParse(v?.ToString(), out var b) ? b : null; + + private async Task Update(Action apply) + { + apply(); + _validationError = _m.Validate(); + var json = _m.ToJson(); + + // Keep the OnParametersSet guard in step with what we just emitted, so the host echoing the new + // JSON straight back as a parameter cannot re-parse and clobber an in-progress edit. + // + // WHY THIS DIVERGES FROM THE MODBUS TEMPLATE (which does not do this): the general landmine is a + // DERIVED, NON-PERSISTED UI FIELD INFERRED FROM BLOB CONTENT. Mode is exactly that — it is + // re-inferred by FromJson from the Sparkplug keys and never serialised, so a plain re-parse of + // our own output would silently reset the operator's shape selection to Plain on the very next + // render. Modbus has no such field, which is why it needs no guard. Task 24's Sparkplug field + // group will be tempted to add more of them; each one needs this guard to hold. + // + // TRADE (does not manifest today): the editor will not pick up a change the HOST makes to the + // JSON while it is open. That is safe only because RawTagModal/RawManualTagEntryModal mutate + // _form.TagConfig exclusively through this callback. A host-side feature that rewrites the blob + // out-of-band — a "reformat JSON" action, a bulk address re-pick — would reintroduce staleness + // here and must re-key the guard (e.g. compare against a host-supplied revision, not the text). + _lastConfigJson = json; + await ConfigJsonChanged.InvokeAsync(json); + } +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs index f19cbcf7..1759ac25 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs @@ -10,6 +10,7 @@ using ZB.MOM.WW.OtOpcUa.AdminUI.Uns; using ZB.MOM.WW.OtOpcUa.Commons.Browsing; using ZB.MOM.WW.OtOpcUa.Core.Abstractions; using ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Browser; +using ZB.MOM.WW.OtOpcUa.Driver.Mqtt.Browser; using ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Browser; using ZB.MOM.WW.OtOpcUa.Driver.Sql.Browser; using ZB.MOM.WW.Secrets.Ui; @@ -80,6 +81,7 @@ public static class EndpointRouteBuilderExtensions // for Sql: SqlDriver.CanBrowse is false, and this bespoke IDriverBrowser wins by construction // (BrowserSessionService indexes bespoke browsers by DriverType before falling back to universal). services.AddSingleton(); + services.AddSingleton(); // Universal Discover-backed fallback browser (Wave 0). IDriverFactory is bound by the // fused Host's DriverFactoryBootstrap; a standalone AdminUI has none → NullDriverFactory // → CanBrowse false → pickers gracefully stay manual-entry (design §6). diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/RawBrowseCommitMapper.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/RawBrowseCommitMapper.cs index 7ee70dc3..47ef905d 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/RawBrowseCommitMapper.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/RawBrowseCommitMapper.cs @@ -3,6 +3,7 @@ using ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors; using ZB.MOM.WW.OtOpcUa.Commons.Types; using ZB.MOM.WW.OtOpcUa.Configuration.Enums; using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Driver.Mqtt; namespace ZB.MOM.WW.OtOpcUa.AdminUI.Uns; @@ -33,6 +34,9 @@ public static class RawBrowseCommitMapper /// The captured browse-folder nesting above the leaf (root→parent display /// names); mirrored onto nested TagGroups only when is true. /// When true, mirror as nested TagGroups. + /// The leaf's structured address (), + /// for a driver whose binding is a tuple rather than a single reference string; null for every driver + /// whose address is the itself. /// The assembled import row. public static RawTagImportRow MapLeaf( string driverType, @@ -42,10 +46,11 @@ public static class RawBrowseCommitMapper string defaultDataType, string? groupPrefix, IReadOnlyList? folderPath, - bool createGroups) + bool createGroups, + IReadOnlyDictionary? addressFields = null) { var dataType = MapDataType(driverDataType) ?? defaultDataType; - var tagConfig = BuildTagConfig(driverType, fullName); + var tagConfig = BuildTagConfig(driverType, fullName, addressFields); var mirrored = createGroups && folderPath is { Count: > 0 } ? string.Join(RawPaths.Separator, folderPath) : null; @@ -96,13 +101,25 @@ public static class RawBrowseCommitMapper /// address field to and leaving every other field at its model default. Reuses /// the <Driver>TagConfigModel for driver types that have a typed editor; the model-less Galaxy /// driver gets the canonical camelCase attributeRef key directly. + /// + /// MQTT is the one driver whose address is not a single string — a Sparkplug tag binds by a + /// (group, edgeNode, device?, metric) tuple — so it is built from the browse session's stated + /// instead. See . + /// /// /// The owning device's driver type. /// The leaf's driver-side full reference to write into the address field. + /// The leaf's structured address, when the driver binds by a tuple — + /// see . Ignored by every single-reference driver. /// The serialised driver-typed TagConfig JSON. - public static string BuildTagConfig(string driverType, string fullName) + public static string BuildTagConfig( + string driverType, + string fullName, + IReadOnlyDictionary? addressFields = null) { var address = fullName ?? ""; + if (Is(driverType, DriverTypeNames.Mqtt)) + return BuildMqttTagConfig(address, addressFields); if (Is(driverType, DriverTypeNames.OpcUaClient)) return new OpcUaClientTagConfigModel { NodeId = address }.ToJson(); if (Is(driverType, DriverTypeNames.AbCip)) @@ -123,6 +140,103 @@ public static class RawBrowseCommitMapper return WriteSingleKey("address", address); } + /// + /// Builds the TagConfig for a browse-committed MQTT leaf, whose address is a + /// descriptor rather than a single reference string: topic in Plain mode, and the + /// groupId/edgeNodeId/deviceId?/metricName tuple in Sparkplug B mode. + /// + /// The leaf's browse node id — the Plain-mode fallback address only. + /// The structured address the browse session stated. + /// The serialised TagConfig JSON. + /// + /// + /// The address is read from , never parsed out of + /// . A Sparkplug browse node id is + /// {group}/{node}[/{device}]::{metric}, and a metric name legitimately contains / + /// (Node Control/Rebirth) — so splitting the id cannot recover the tuple in general, and a + /// mapper that guessed would silently bind the wrong metric. The session that decoded the birth + /// already holds the decomposition and hands it over; this method only picks the keys it knows. + /// + /// + /// Plain vs Sparkplug is likewise stated, not inferred — the driver type reaching + /// here is just Mqtt, and the mode lives on the driver config, not on the tag. The + /// producing session knows its own mode and says so by which keys it emits: a + /// metricName ⇒ Sparkplug, a topic ⇒ Plain. (The blob deliberately carries no + /// mode key: MqttTagDefinitionFactory takes the shape from the driver's mode and + /// would ignore one, and MqttTagConfigModel re-infers it from these same keys.) + /// + /// + /// Keys are whitelisted, not splatted: these values came off a broker, and everything + /// beyond the address (payload format, JSONPath, QoS, dataType) stays at the driver's own + /// defaults for the operator to author afterwards. A leaf with no stated address at all is + /// rejected before this is reached — see . + /// + /// + private static string BuildMqttTagConfig(string fullName, IReadOnlyDictionary? addressFields) + { + if (TryGetField(addressFields, MqttTagConfigKeys.MetricName) is { } metricName) + { + var o = new JsonObject + { + [MqttTagConfigKeys.GroupId] = TryGetField(addressFields, MqttTagConfigKeys.GroupId) ?? "", + [MqttTagConfigKeys.EdgeNodeId] = TryGetField(addressFields, MqttTagConfigKeys.EdgeNodeId) ?? "", + [MqttTagConfigKeys.MetricName] = metricName, + }; + + // Absent, not blank, for a node-level metric — the two describe the same scope to the + // factory, but only the absent form says "this metric has no device" to a reader. + if (TryGetField(addressFields, MqttTagConfigKeys.DeviceId) is { } deviceId) + o[MqttTagConfigKeys.DeviceId] = deviceId; + + return o.ToJsonString(); + } + + // Plain: the session states the topic; the node id is the same value and is the fallback for a + // session that stated nothing (a shape DescribeUncommittableLeaf refuses upstream). + return WriteSingleKey( + MqttTagConfigKeys.Topic, + TryGetField(addressFields, MqttTagConfigKeys.Topic) ?? fullName); + } + + /// + /// Describes why a selected browse leaf cannot be committed as a working tag, or null when it + /// can. The AdminUI calls this before mapping so an unbindable selection fails at commit, in + /// words. + /// + /// The owning device's driver type. + /// The leaf's browse name, for the message. + /// The structured address the browse session stated, if any. + /// The operator-facing reason, or null when the leaf is committable. + /// + /// Only MQTT can fail this today, and only in one shape: a leaf whose attribute lookup returned + /// nothing, so no address was stated. Committing it anyway is the defect this whole seam exists + /// to close — a Sparkplug tuple cannot be reconstructed from the node id, so the row would deploy + /// clean and then report BadNodeIdUnknown forever with nothing to point at. Every + /// single-reference driver is unaffected: its address IS the node id. + /// + public static string? DescribeUncommittableLeaf( + string driverType, + string browseName, + IReadOnlyDictionary? addressFields) + { + if (!Is(driverType, DriverTypeNames.Mqtt)) return null; + if (TryGetField(addressFields, MqttTagConfigKeys.MetricName) is not null) return null; + if (TryGetField(addressFields, MqttTagConfigKeys.Topic) is not null) return null; + + return $"'{browseName}' could not be committed: the browse session reported no MQTT address for it " + + "(its attribute lookup returned nothing). Re-open the browser and re-select it, or author " + + "the tag manually."; + } + + /// Reads one non-blank address field, or null when it is absent or blank. + /// The stated address fields, or null. + /// The field to read. + /// The trimmed value, or null. + private static string? TryGetField(IReadOnlyDictionary? fields, string key) + => fields is not null && fields.TryGetValue(key, out var v) && !string.IsNullOrWhiteSpace(v) + ? v.Trim() + : null; + /// /// Combines a target-group path prefix with an optional mirrored sub-path, matching the WP5 CSV import /// combine semantics: blank collapses to null, and a present prefix + suffix join with the RawPath diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MqttTagConfigModel.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MqttTagConfigModel.cs new file mode 100644 index 00000000..68587153 --- /dev/null +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/MqttTagConfigModel.cs @@ -0,0 +1,396 @@ +using System.Text.Json.Nodes; +using ZB.MOM.WW.OtOpcUa.Core.Abstractions; +using ZB.MOM.WW.OtOpcUa.Driver.Mqtt; + +namespace ZB.MOM.WW.OtOpcUa.AdminUI.Uns.TagEditors; + +/// +/// Typed working model for an MQTT tag's TagConfig JSON — the driver-specific binding fields +/// (name / access level / writability live on the Tag entity). Preserves unrecognised JSON keys +/// across a load→save. +/// +/// +/// +/// The authoritative consumer of the produced blob is +/// MqttTagDefinitionFactory.FromTagConfig (Driver.Mqtt.Contracts); every key name and +/// strictness rule here mirrors that factory rather than inventing an editor-side schema. +/// +/// +/// There is deliberately no FullName (or any other identity) key. Under the v3 +/// identity contract a tag is identified by its RawPath — the factory keys the produced +/// definition's Name off the RawPath it is handed, and the TagConfig is a pure address blob. +/// Writing a composed identity key here would be dead weight nothing reads. +/// +/// +/// is a UI-only sub-shape selector, inferred from the blob (the presence +/// of any Sparkplug descriptor key) and never serialised — the driver takes its +/// Plain/SparkplugB mode from the driver config, not from a tag, so +/// persisting a per-tag mode key would add a field the contract does not define. +/// Consequently a SparkplugB tag survives a save→reopen purely because the descriptor keys it +/// writes (/// +/// ) re-infer the mode on the next load — there is nothing else to persist. +/// +/// +public sealed class MqttTagConfigModel +{ + private const string TopicKey = "topic"; + private const string PayloadFormatKey = "payloadFormat"; + private const string JsonPathKey = "jsonPath"; + private const string DataTypeKey = "dataType"; + private const string QosKey = "qos"; + private const string RetainSeedKey = "retainSeed"; + private const string GroupIdKey = "groupId"; + private const string EdgeNodeIdKey = "edgeNodeId"; + private const string DeviceIdKey = "deviceId"; + private const string MetricNameKey = "metricName"; + + /// + /// The JSONPath the driver applies when the blob omits jsonPath — the document root. + /// Seeded into an unauthored tag's field so the "no extraction needed" case is one click away + /// rather than something the operator has to know. + /// + private const string RootJsonPath = "$"; + + /// The Sparkplug B descriptor keys — any non-blank one marks a tag SparkplugB (see ). + private static readonly string[] SparkplugKeys = [GroupIdKey, EdgeNodeIdKey, DeviceIdKey, MetricNameKey]; + + /// The MQTT wildcard characters; a tag's subscription topic must be concrete. + private static readonly char[] TopicWildcards = ['+', '#']; + + /// + /// Characters illegal in a Sparkplug group/edge-node/device id. / is rejected because + /// these ids are literal MQTT topic segments — a decoded incoming id can never contain one + /// (the broker has already split the topic into segments before the driver sees it), so an + /// authored id containing / could never match a real birth: a permanently dead binding, + /// not an ambiguous one. +/# are rejected for the same "no legitimate + /// interpretation" reason the Plain topic wildcard check uses — here doubly so, since + /// 's only consumer builds the literal subscription + /// filter spBv1.0/{GroupId}/#, and embedding a wildcard character mid-segment is not + /// even legal there. Not enforced by MqttTagDefinitionFactory.FromSparkplugTagConfig + /// — this is an editor-side rule stricter than the runtime parser, the same shape as the Plain + /// topic-wildcard rule below. + /// + private static readonly char[] SparkplugSegmentIllegalChars = ['/', '+', '#']; + + /// + /// Which ingest shape this tag is authored under — inferred from the blob, never serialised. + /// + public MqttMode Mode { get; set; } = MqttMode.Plain; + + /// The concrete MQTT topic the tag subscribes to (Plain mode). Required; no wildcards. + public string Topic { get; set; } = ""; + + /// How the received payload is decoded (Plain mode). + public MqttPayloadFormat PayloadFormat { get; set; } = MqttPayloadFormat.Json; + + /// + /// JSONPath selecting the value inside a payload. Blank is + /// legal and NOT a validation failure — the key is omitted and the driver applies the document + /// root ($), which is the real "publisher puts a bare JSON scalar on the topic" case. + /// An unauthored tag is seeded with $ by so that common case + /// is visible and one click away; see . + /// + public string JsonPath { get; set; } = ""; + + /// The tag's declared value type (Plain mode; always written). The driver's set. + public DriverDataType DataType { get; set; } = DriverDataType.String; + + /// + /// Per-tag subscription QoS (0–2), or null to omit the key and inherit the driver-level + /// default. An absent key stays absent through a load→save. Shared by both modes — + /// MqttTagDefinitionFactory reads qos identically for Plain and Sparkplug. + /// + public int? Qos { get; set; } + + /// + /// Whether the broker's retained message seeds the tag's initial value, or null to omit + /// the key and inherit the driver's default (true). An absent key stays absent. Shared by + /// both modes — MqttTagDefinitionFactory reads retainSeed identically for Plain and + /// Sparkplug. + /// + public bool? RetainSeed { get; set; } + + /// The Sparkplug group id (Sparkplug mode). Required; becomes part of the subscription filter. + public string GroupId { get; set; } = ""; + + /// The Sparkplug edge-node id (Sparkplug mode). Required. + public string EdgeNodeId { get; set; } = ""; + + /// + /// The Sparkplug device id (Sparkplug mode), or blank for a metric published by the edge node + /// itself rather than a device beneath it. Optional. + /// + public string DeviceId { get; set; } = ""; + + /// + /// The Sparkplug metric's stable name (Sparkplug mode). Required. Unlike + /// // this is NOT a topic + /// segment — it is read from the birth/data payload — so it is deliberately unrestricted and MAY + /// contain / (the canonical Sparkplug examples do: Node Control/Rebirth, + /// Properties/Hardware Make). + /// + public string MetricName { get; set; } = ""; + + /// + /// Sparkplug-mode data-type OVERRIDE (Sparkplug mode only), or null to omit the key and + /// let the tag take whatever type the metric's birth certificate declares — the whole point of + /// the driver's UntilStable discovery. Distinct from , which is the + /// always-written Plain-mode field; both read/write the same dataType JSON key + /// (MqttTagDefinitionFactory.FromSparkplugTagConfig reads it as a + /// override, strictly, when present — the same enum Plain mode uses, not the raw wire + /// SparkplugDataType). + /// + public DriverDataType? MetricDataType { get; set; } + + private JsonObject _bag = new(); + + /// + /// Loads a model from a TagConfig JSON string, defaulting any absent field and retaining every + /// original key (so fields this editor doesn't expose — history intent, array/alarm objects, and + /// the Sparkplug descriptor keys — survive a load→save). + /// + /// + /// An unauthored Plain tag — no topic AND no jsonPath, i.e. a brand-new tag + /// rather than an existing one the operator deliberately left path-less — has its + /// seeded to . The condition is deliberately + /// narrow: an existing tag that carries a topic and no jsonPath keeps the key ABSENT + /// through a load→save, so this can never rewrite already-deployed blobs. + /// + /// The raw TagConfig JSON string, or null for a new/empty config. + /// The populated . + public static MqttTagConfigModel FromJson(string? json) + { + var o = TagConfigJson.ParseOrNew(json); + var topic = TagConfigJson.GetString(o, TopicKey) ?? ""; + var jsonPath = TagConfigJson.GetString(o, JsonPathKey) ?? ""; + var mode = InferMode(o); + if (mode == MqttMode.Plain && topic.Length == 0 && jsonPath.Length == 0) { jsonPath = RootJsonPath; } + + return new MqttTagConfigModel + { + Mode = mode, + Topic = topic, + PayloadFormat = TagConfigJson.GetEnum(o, PayloadFormatKey, MqttPayloadFormat.Json), + JsonPath = jsonPath, + DataType = TagConfigJson.GetEnum(o, DataTypeKey, DriverDataType.String), + Qos = GetIntNullable(o, QosKey), + RetainSeed = TagConfigJson.GetBoolNullable(o, RetainSeedKey), + GroupId = TagConfigJson.GetString(o, GroupIdKey) ?? "", + EdgeNodeId = TagConfigJson.GetString(o, EdgeNodeIdKey) ?? "", + DeviceId = TagConfigJson.GetString(o, DeviceIdKey) ?? "", + MetricName = TagConfigJson.GetString(o, MetricNameKey) ?? "", + // Same "dataType" key as Plain's DataType above, read as an optional override — see the + // MetricDataType doc comment for why there are two typed fields over one JSON key. + MetricDataType = GetEnumNullable(o, DataTypeKey), + _bag = o, + }; + } + + /// + /// Serialises this model back to a TagConfig JSON string over the preserved key bag. Enums are + /// written as their names (the driver reads them strictly by name; an ordinal would be + /// rejected outright), and blank/null optionals are written as an absent key so the driver's own + /// defaults apply. + /// + /// + /// dataType is the one key both modes write, from two different typed fields + /// ( for Plain — always present; for + /// Sparkplug — omitted unless the operator set an override): which field wins is decided by + /// at write time. Every other key is unconditional — mode-inapplicable keys + /// (e.g. topic while Sparkplug, or groupId while Plain) are written/cleared exactly + /// as their typed field says, which naturally preserves a retyped tag's other-mode leftovers + /// untouched until the operator edits that field too. + /// + /// The serialised TagConfig JSON string. + public string ToJson() + { + // Blank ⇒ omit, like every other optional here: toggling the shape dropdown on an untouched tag + // must not leave a stray "topic":"" behind. + TagConfigJson.Set(_bag, TopicKey, string.IsNullOrWhiteSpace(Topic) ? null : Topic.Trim()); + TagConfigJson.Set(_bag, PayloadFormatKey, PayloadFormat); + TagConfigJson.Set(_bag, JsonPathKey, string.IsNullOrWhiteSpace(JsonPath) ? null : JsonPath.Trim()); + TagConfigJson.Set(_bag, QosKey, Qos); + TagConfigJson.Set(_bag, RetainSeedKey, RetainSeed); + + // Mode-dependent: Plain always writes DataType; Sparkplug writes MetricDataType (null ⇒ omitted, + // "take the birth's declared type"). + TagConfigJson.Set(_bag, DataTypeKey, Mode == MqttMode.SparkplugB ? MetricDataType : DataType); + + TagConfigJson.Set(_bag, GroupIdKey, string.IsNullOrWhiteSpace(GroupId) ? null : GroupId.Trim()); + TagConfigJson.Set(_bag, EdgeNodeIdKey, string.IsNullOrWhiteSpace(EdgeNodeId) ? null : EdgeNodeId.Trim()); + TagConfigJson.Set(_bag, DeviceIdKey, string.IsNullOrWhiteSpace(DeviceId) ? null : DeviceId.Trim()); + TagConfigJson.Set(_bag, MetricNameKey, string.IsNullOrWhiteSpace(MetricName) ? null : MetricName.Trim()); + return TagConfigJson.Serialize(_bag); + } + + /// + /// Client-side validation run by before a tag is saved. + /// Returns the first error, or null when the config is valid. + /// + /// + /// + /// Exactly ONE rule is deliberately stricter than the runtime parser: a wildcard topic, + /// which the runtime accepts and Inspect only warns about at deploy. It earns the + /// strictness because a wildcard has no legitimate single-Tag interpretation — one Tag holds one + /// value, and +/# would feed it from many source topics. Everything else — + /// required topic, strict enums, QoS 0–2 — matches MqttTagDefinitionFactory exactly. + /// + /// + /// A blank jsonPath under a Json payload is explicitly accepted, matching + /// the runtime's document-root default. Rejecting it would make this editor refuse a config the + /// driver handles happily — inverting the "authoring surface accepts ⇔ publish accepts" + /// principle — and, because this validator also gates the CSV-import review grid + /// (RawManualTagEntryModal), would block whole import batches over a sane default. The + /// operator is guided by the seeded $ from instead of a blocker. + /// + /// + /// The strict enum/QoS checks read the ORIGINAL key bag, not the defaulted typed fields, so a + /// blob that never went through this editor (e.g. a CSV import) cannot pass validation while the + /// driver would reject it. + /// + /// + /// Sparkplug rules mirror MqttTagDefinitionFactory.FromSparkplugTagConfig: + /// groupId/edgeNodeId/metricName required (blank rejected exactly as the + /// factory hard-rejects them), deviceId optional, dataType and qos read + /// strictly but ONLY when present (matching TryReadEnumStrict's absent/valid/invalid + /// split — a Sparkplug tag legitimately omits dataType and takes whatever the birth + /// certificate declares). Plain-only fields (topic, payloadFormat, jsonPath) + /// are NOT checked in this mode, matching the factory's "Plain-shape keys are read but not + /// required" stance for a retyped blob's leftovers. + /// + /// + /// ONE Sparkplug rule is likewise stricter than the factory: groupId/edgeNodeId/ + /// deviceId reject /, +, and # — see + /// for why (a decoded incoming id can never contain + /// one, so an authored id that does is a permanently dead binding, and +/# would + /// corrupt the driver's own subscription filter). metricName carries NO such restriction — + /// it is not a topic segment, and Sparkplug's own canonical metric names use / (e.g. + /// Node Control/Rebirth); rejecting it would block legitimate authoring. + /// + /// + /// An error message describing the validation failure, or null when valid. + public string? Validate() + { + if (Mode == MqttMode.SparkplugB) { return ValidateSparkplug(); } + + if (DescribeInvalidEnum(_bag, PayloadFormatKey) is { } pfError) { return pfError; } + if (DescribeInvalidEnum(_bag, DataTypeKey) is { } dtError) { return dtError; } + if (DescribeInvalidQos(_bag) is { } qosError) { return qosError; } + + var topic = Topic.Trim(); + if (string.IsNullOrEmpty(topic)) { return "A topic is required."; } + if (topic.IndexOfAny(TopicWildcards) >= 0) + { + return $"Topic '{topic}' contains an MQTT wildcard (+ or #); a tag's topic must be concrete, " + + "or the tag would be fed by every matching topic."; + } + + // NB no jsonPath rule — see the remarks above. A blank path is the driver's document-root + // default, not an authoring error. + return null; + } + + /// Sparkplug-mode half of — see its remarks for the full rationale. + /// An error message describing the validation failure, or null when valid. + private string? ValidateSparkplug() + { + // Same DriverDataType enum + same strict absent/valid/invalid split as Plain's DataType check — + // dataType is one JSON key read by both FromTagConfig and FromSparkplugTagConfig identically. + if (DescribeInvalidEnum(_bag, DataTypeKey) is { } dtError) { return dtError; } + if (DescribeInvalidQos(_bag) is { } qosError) { return qosError; } + + var groupId = GroupId.Trim(); + var edgeNodeId = EdgeNodeId.Trim(); + var deviceId = DeviceId.Trim(); + var metricName = MetricName.Trim(); + + if (string.IsNullOrEmpty(groupId)) { return "A Sparkplug group ID is required."; } + if (string.IsNullOrEmpty(edgeNodeId)) { return "A Sparkplug edge node ID is required."; } + if (string.IsNullOrEmpty(metricName)) { return "A Sparkplug metric name is required."; } + + if (DescribeInvalidSparkplugSegment("group ID", groupId) is { } gErr) { return gErr; } + if (DescribeInvalidSparkplugSegment("edge node ID", edgeNodeId) is { } eErr) { return eErr; } + // deviceId is optional — only validated when the operator actually supplied one. + if (deviceId.Length > 0 && DescribeInvalidSparkplugSegment("device ID", deviceId) is { } dErr) { return dErr; } + + return null; + } + + /// + /// Describes an that is illegal as a Sparkplug group/edge-node/device + /// id segment (see ), or null when clean. + /// + /// The human-readable field name for the error message. + /// The trimmed, non-blank field value to check. + /// The error text, or null. + private static string? DescribeInvalidSparkplugSegment(string fieldLabel, string value) + => value.IndexOfAny(SparkplugSegmentIllegalChars) >= 0 + ? $"Sparkplug {fieldLabel} '{value}' contains a character ('/', '+', or '#') that cannot appear " + + "in an MQTT topic segment; a decoded incoming id can never contain one, so this could never bind." + : null; + + /// + /// Infers the editor's sub-shape from the blob: any non-blank Sparkplug descriptor key marks a + /// Sparkplug tag, otherwise Plain. + /// + /// The parsed TagConfig key bag. + /// The inferred mode. + private static MqttMode InferMode(JsonObject o) + => SparkplugKeys.Any(k => !string.IsNullOrWhiteSpace(TagConfigJson.GetString(o, k))) + ? MqttMode.SparkplugB + : MqttMode.Plain; + + /// Reads an int value, or null when absent/null/not an integer (so an absent key stays absent). + /// The JSON object to read from. + /// The property name to read. + /// The int value, or null. + private static int? GetIntNullable(JsonObject o, string name) + => o.TryGetPropertyValue(name, out var n) && n is JsonValue v && v.TryGetValue(out var i) ? i : null; + + /// + /// Reads an enum by its serialised name, or null when absent/unparseable — the nullable + /// counterpart of , used for + /// so "the key is absent" (take the birth's declared type) is distinguishable from any real member. + /// + /// The enum type to parse. + /// The JSON object to read from. + /// The property name to read. + /// The parsed enum value, or null. + private static TEnum? GetEnumNullable(JsonObject o, string name) where TEnum : struct, Enum + => TagConfigJson.GetString(o, name) is { } s && Enum.TryParse(s, ignoreCase: true, out var v) ? v : null; + + /// + /// Describes a present-but-invalid enum field, or null when it is absent or valid. + /// Mirrors TagConfigJson.TryReadEnum's absent/valid/invalid split exactly — including its + /// treatment of a present-but-non-string value as ABSENT — so the editor never blocks a blob the + /// driver would happily default. + /// + /// The enum type expected. + /// The TagConfig key bag. + /// The property name to describe. + /// The error text, or null. + private static string? DescribeInvalidEnum(JsonObject o, string name) where TEnum : struct, Enum + { + if (!o.TryGetPropertyValue(name, out var n) || n is not JsonValue v || !v.TryGetValue(out var raw)) + { + return null; + } + return Enum.TryParse(raw, ignoreCase: true, out _) + ? null + : $"'{raw}' is not a valid {name}; valid: {string.Join(", ", Enum.GetNames())}."; + } + + /// + /// Describes a present-but-invalid qos field, or null when it is absent or a legal + /// MQTT QoS. Matches the factory's strict read: absent ⇒ fine; a JSON integer 0–2 ⇒ fine; + /// anything else present (non-number, non-integer, out of range) ⇒ rejected. + /// + /// The TagConfig key bag. + /// The error text, or null. + private static string? DescribeInvalidQos(JsonObject o) + { + if (!o.TryGetPropertyValue(QosKey, out var n) || n is null) { return null; } + if (n is JsonValue v && v.TryGetValue(out var i) && i is >= 0 and <= 2) { return null; } + return $"'{n.ToJsonString()}' is not a valid QoS; valid: 0, 1, 2."; + } +} diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs index e10c9a50..98a81ba2 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs @@ -26,6 +26,7 @@ public static class TagConfigEditorMap // constant is deliberately absent until Task 11 wires the factory (DriverTypeNamesGuardTests // asserts bidirectional parity). Task 11 repoints all Sql keys onto DriverTypeNames.Sql. [SqlDriver.DriverTypeName] = typeof(Components.Shared.Uns.TagEditors.SqlTagConfigEditor), + [DriverTypeNames.Mqtt] = typeof(Components.Shared.Uns.TagEditors.MqttTagConfigEditor), }; /// Returns the editor component type for a driver type, or null if none is registered. diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigValidator.cs b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigValidator.cs index 50ee89af..bd56b19d 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigValidator.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigValidator.cs @@ -26,6 +26,7 @@ public static class TagConfigValidator [DriverTypeNames.Calculation] = j => CalculationTagConfigModel.FromJson(j).Validate(), // Keyed off SqlDriver.DriverTypeName (= "Sql") — see the note in TagConfigEditorMap. [SqlDriver.DriverTypeName] = j => SqlTagConfigModel.FromJson(j).Validate(), + [DriverTypeNames.Mqtt] = j => MqttTagConfigModel.FromJson(j).Validate(), }; /// diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj index 8e483207..01e0e8b3 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj +++ b/src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj @@ -43,6 +43,7 @@ Driver.Sql + .Contracts + Microsoft.Data.SqlClient transitively — reviewed/accepted, see the Browser project's own csproj comment (AdminUI already carries SqlClient for ConfigDb). --> + diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs b/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs index 2687366e..d6fc6d1a 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs @@ -19,6 +19,7 @@ using OpcUaProbe = Driver.OpcUaClient.OpcUaClientDriverProbe; using GalaxyProbe = Driver.Galaxy.GalaxyDriverProbe; using CalculationProbe = Driver.Calculation.CalculationDriverProbe; using SqlProbe = Driver.Sql.SqlDriverProbe; +using MqttProbe = Driver.Mqtt.MqttDriverProbe; /// /// Wires every cross-platform driver assembly's Register(registry, loggerFactory) @@ -124,6 +125,7 @@ public static class DriverFactoryBootstrap services.TryAddEnumerable(ServiceDescriptor.Singleton()); services.TryAddEnumerable(ServiceDescriptor.Singleton()); services.TryAddEnumerable(ServiceDescriptor.Singleton()); + services.TryAddEnumerable(ServiceDescriptor.Singleton()); return services; } @@ -146,6 +148,7 @@ public static class DriverFactoryBootstrap Driver.FOCAS.FocasDriverFactoryExtensions.Register(registry); Driver.Galaxy.GalaxyDriverFactoryExtensions.Register(registry, secretResolver, loggerFactory); Driver.Modbus.ModbusDriverFactoryExtensions.Register(registry, loggerFactory); + Driver.Mqtt.MqttDriverFactoryExtensions.Register(registry, loggerFactory); Driver.OpcUaClient.OpcUaClientDriverFactoryExtensions.Register(registry, loggerFactory, secretResolver); Driver.S7.S7DriverFactoryExtensions.Register(registry); Driver.Sql.SqlDriverFactoryExtensions.Register(registry, loggerFactory); diff --git a/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj b/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj index b3fc5bad..bccc5685 100644 --- a/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj +++ b/src/Server/ZB.MOM.WW.OtOpcUa.Host/ZB.MOM.WW.OtOpcUa.Host.csproj @@ -74,6 +74,7 @@ + diff --git a/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj b/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj index 182518eb..c9be0506 100644 --- a/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj +++ b/tests/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests/ZB.MOM.WW.OtOpcUa.Core.Abstractions.Tests.csproj @@ -23,11 +23,12 @@ - + diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/.gitignore b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/.gitignore new file mode 100644 index 00000000..7e91479f --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/.gitignore @@ -0,0 +1,19 @@ +# Fixture credential + TLS material produced by gen-fixture-material.sh. +# +# NEVER commit anything in here: it holds the broker password hash, the fixture CA's +# PRIVATE key and the server's private key. The generator script is the artifact in +# git; its output is regenerated per machine. +secrets/ + +# Build output of publish-simulator.sh — the `sparkplug-sim` service's bind-mounted app +# directory. Regenerated per machine from the SparkplugSimulator project, exactly like +# secrets/ is regenerated by gen-fixture-material.sh; the SCRIPT is the artifact in git. +simulator/app/ + +# Belt-and-braces: catch key/cert material written directly into this directory by a +# hand-run openssl command that forgot the secrets/ prefix. +*.key +*.crt +*.csr +*.srl +passwd diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml new file mode 100644 index 00000000..2c42a8fb --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml @@ -0,0 +1,125 @@ +# MQTT integration-test fixture — Eclipse Mosquitto (auth + TLS) + a JSON publisher +# + an optional Sparkplug B edge-node simulator. +# +# PREREQUISITE — run this first, in this directory: +# +# MQTT_FIXTURE_PASSWORD='' ./gen-fixture-material.sh +# +# It writes ./secrets/ (password file + CA + server certificate/key), which is +# gitignored and which both services below mount. Without it the broker will not +# start: there is no anonymous fallback, by design. +# +# Bring up (on the shared Docker host — see infra/README.md §1): +# +# ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mqtt \ +# && MQTT_FIXTURE_USERNAME=otopcua MQTT_FIXTURE_PASSWORD= docker compose up -d' +# +# Endpoints: 10.100.0.35:8883 (TLS + auth) · 10.100.0.35:1883 (plaintext + auth, smoke) +# +# `mosquitto` + `publisher` carry no profile: they are always wanted together (a broker +# with nothing publishing into it tests nothing) and they bind different ports so nothing +# contends. `sparkplug-sim` DOES carry one (`--profile sparkplug`), because it is a +# manual-driving aid rather than a test dependency — see its own banner. +services: + mosquitto: + image: eclipse-mosquitto:2.0.22 + container_name: otopcua-mosquitto + restart: unless-stopped + # Every lmxopcua fixture container carries this so the shared Docker host stays + # discoverable with `docker ps --filter label=project=lmxopcua`. + labels: + project: lmxopcua + ports: + - "1883:1883" + - "8883:8883" + volumes: + - ./mosquitto.conf:/mosquitto/config/mosquitto.conf:ro + - ./secrets:/mosquitto/secrets:ro + healthcheck: + # Authenticated round-trip against the broker's own $SYS tree ($$ escapes the + # compose variable syntax). Proves the password file loaded and the listener + # accepts a real session — a bare port check would go green on a broker that + # rejects every CONNECT. + test: + - CMD-SHELL + - >- + mosquitto_sub -h 127.0.0.1 -p 1883 + -u "$$MQTT_FIXTURE_USERNAME" -P "$$MQTT_FIXTURE_PASSWORD" + -t '$$SYS/broker/uptime' -C 1 -W 3 + interval: 5s + timeout: 8s + retries: 12 + start_period: 5s + environment: + # Consumed only by the healthcheck above; the broker itself reads the hashed + # password file. Supplied by the operator's environment at `docker compose up` — + # never defaulted here, so a missing value fails loudly instead of silently + # provisioning a known-password broker. + MQTT_FIXTURE_USERNAME: ${MQTT_FIXTURE_USERNAME:?set MQTT_FIXTURE_USERNAME (must match gen-fixture-material.sh)} + MQTT_FIXTURE_PASSWORD: ${MQTT_FIXTURE_PASSWORD:?set MQTT_FIXTURE_PASSWORD (must match gen-fixture-material.sh)} + + publisher: + image: eclipse-mosquitto:2.0.22 + container_name: otopcua-mqtt-publisher + restart: unless-stopped + labels: + project: lmxopcua + depends_on: + mosquitto: + condition: service_healthy + volumes: + - ./publisher/publish.sh:/publish.sh:ro + environment: + BROKER_HOST: mosquitto + BROKER_PORT: "1883" + TOPIC_PREFIX: ${MQTT_FIXTURE_TOPIC_PREFIX:-otopcua/fixture} + PUBLISH_INTERVAL: ${MQTT_FIXTURE_PUBLISH_INTERVAL:-2} + MQTT_FIXTURE_USERNAME: ${MQTT_FIXTURE_USERNAME:?set MQTT_FIXTURE_USERNAME} + MQTT_FIXTURE_PASSWORD: ${MQTT_FIXTURE_PASSWORD:?set MQTT_FIXTURE_PASSWORD} + entrypoint: ["/bin/sh", "/publish.sh"] + + # --------------------------------------------------------------------------- + # Sparkplug B edge-node simulator — the SAME `SparkplugEdgeNode` engine the live + # tests drive, run standalone (see SparkplugSimulator/Program.cs). + # + # WHAT IT IS FOR, AND WHAT IT IS NOT FOR. It exists so an operator (and the Task-26 + # AdminUI `/run` verification) has a live Sparkplug plant to point the address picker + # at: NBIRTH/DBIRTH then N/DDATA every couple of seconds, honouring rebirth NCMDs. + # `SparkplugLiveTests` does NOT use it and must not — the §3.6 matrix needs an edge + # node it can command mid-test ("now reuse that alias", "now skip a seq", "now die"), + # which a container cannot be. Hence the profile: the default `docker compose up` + # leaves it out, and nothing in the test suite depends on it. + # + # PREREQUISITE: `./publish-simulator.sh` — the app directory below is build output, + # gitignored and rsynced like the rest of this folder. Without it the container has + # nothing to run. + # --------------------------------------------------------------------------- + sparkplug-sim: + image: mcr.microsoft.com/dotnet/runtime:10.0 + container_name: otopcua-sparkplug-sim + profiles: ["sparkplug"] + restart: unless-stopped + labels: + project: lmxopcua + depends_on: + mosquitto: + condition: service_healthy + volumes: + # Framework-dependent publish output: portable IL, so it runs under the stock + # runtime image whatever this file was rsynced from. + - ./simulator/app:/app:ro + # The fixture CA, so the simulator PINS the broker exactly as the driver does + # rather than trusting anything. The cert's SAN list includes DNS:mosquitto (see + # gen-fixture-material.sh), which is the host name used below. + - ./secrets/ca.crt:/secrets/ca.crt:ro + environment: + MQTT_SIM_HOST: mosquitto + MQTT_SIM_PORT: "8883" + MQTT_SIM_USE_TLS: "true" + MQTT_SIM_CA_CERT: /secrets/ca.crt + MQTT_SIM_GROUP: ${MQTT_SIM_GROUP:-OtOpcUaSim} + MQTT_SIM_NODES: ${MQTT_SIM_NODES:-EdgeA,EdgeB} + MQTT_SIM_INTERVAL_SECONDS: ${MQTT_SIM_INTERVAL_SECONDS:-2} + MQTT_FIXTURE_USERNAME: ${MQTT_FIXTURE_USERNAME:?set MQTT_FIXTURE_USERNAME} + MQTT_FIXTURE_PASSWORD: ${MQTT_FIXTURE_PASSWORD:?set MQTT_FIXTURE_PASSWORD} + command: ["dotnet", "/app/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.SparkplugSimulator.dll"] diff --git a/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/gen-fixture-material.sh b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/gen-fixture-material.sh new file mode 100755 index 00000000..3b72538f --- /dev/null +++ b/tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/gen-fixture-material.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env sh +# --------------------------------------------------------------------------- +# Generates the MQTT fixture's password file + TLS material into ./secrets/. +# +# NOTHING this script writes is committed: ./secrets/ is excluded by the sibling +# .gitignore. The SCRIPT is the artifact in git, the OUTPUT never is. Re-run it on any +# machine that needs the fixture (or on the Docker host); the material is throwaway by +# construction — a self-issued CA, a 2048-bit server key, and whatever password the +# operator supplies in the environment. +# +# Usage: +# MQTT_FIXTURE_PASSWORD='' ./gen-fixture-material.sh +# +# Environment: +# MQTT_FIXTURE_PASSWORD (required) broker password. No default — a committed or +# defaulted password is exactly what the plan's +# "secrets from env, never committed" rule forbids. +# MQTT_FIXTURE_USERNAME broker username (default: otopcua) +# MQTT_FIXTURE_SAN_IPS comma-separated IP SANs for the server certificate +# (default: 10.100.0.35,127.0.0.1) +# MQTT_FIXTURE_SAN_DNS comma-separated DNS SANs +# (default: localhost,mosquitto) +# MQTT_FIXTURE_CERT_CN server certificate CN (default: first IP SAN) +# MQTT_FIXTURE_CERT_DAYS certificate lifetime in days (default: 825) +# MQTT_FIXTURE_SECRETS_DIR output directory (default: