dohertj2/lmxopcua
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(mqtt): write-through (IWritable) — Plain publish + Sparkplug NCMD/DCMD (#508)

Browse Source 
MqttDriver implemented no IWritable, so every MQTT node materialized read-only
regardless of what the operator authored. It now publishes.

  Plain      -> the tag's writeTopic; UTF-8 text (Raw/Scalar) or a bare JSON
                literal (Json)
  Sparkplug  -> DCMD for a device metric, NCMD for an edge-node metric, carrying
                the metric BY NAME with an explicit Datatype and no seq

Both at QoS 0, retain:false. Retain is deliberately not configurable — a retained
command is replayed by the broker on every future subscribe, including the
device's own reconnect, so one stale write would be re-applied forever.

writeTopic is a new, separate TagConfig key, and its ABSENCE is meaningful: the
write is refused BadNotWritable and the node is discovered ViewOnly. The issue's
"do not skip" warning drove the design — the one outcome worse than read-only
nodes is a node that looks writable, returns Good and publishes nothing. Reusing
the read topic would be worse still: it would echo back and read as a confirmed
device value. Browse-time writability is computed by the SAME encoder the write
path uses, so a node cannot advertise writability it would then refuse.

Good means published, not applied — MQTT has no application-level write ack, so
this is Galaxy's optimistic-Good posture and inherits its property: a failed
DEVICE write can never be surfaced. A failed PUBLISH is reported Bad; the
optimism covers what the device did with the command, never whether it was sent.
WriteIdempotent stays default-off.

The encoder is pure and lives in .Contracts, so both wire shapes are testable
without a broker. Unrepresentable values and types with no Sparkplug encoding are
refused rather than coerced — a Reference must not silently become a String on
the wire, and "not-a-number" must not become 0 on a live plant.

LIVE GATE PASSED against the real Mosquitto fixture on 10.100.0.35: 21/21
integration tests, including 6 new ones that observe the published message with
an INDEPENDENT broker subscriber — a fake asserting "publish was called" could
not tell the feature working from the feature publishing to the wrong topic.

The gate earned its keep: the NCMD test initially failed because NCMD is a SHARED
channel — the driver publishes its own rebirth request (Node Control/Rebirth) on
the very topic an edge-node metric write uses, and it arrived first. Correct
driver behaviour, wrong test; the watcher now matches on metric name. Documented,
because anything consuming that topic needs to know. DCMD has no contention.

Offline: 24 encoder + 9 driver tests. The four new status constants are NAMED so
StatusCodeParityTests can see them (#497/#519) — it went 258 -> 262 and validated
each against the pinned SDK.

Also corrects the Mqtt.md "Known gaps" table: prunes the six rows closed in this
round, and fixes the #507 row, which still described rediscovery as inert after
that issue was closed as superseded and its injection path deleted.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
Joseph Doherty
2026-07-31 06:33:23 -04:00
parent 7f53bd96ec
commit 72be19225d
11 changed files with 1441 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/drivers/Mqtt.md
+98 -17
View File
@@ -24,20 +24,24 @@ for the design.
|---|---|
| `Driver.Mqtt.Contracts` | Config records, enums, `MqttTagDefinition` + the pure `MqttTagDefinitionFactory`, and the one shared `MqttJson.Options`. **Deliberately transport-free** — no MQTTnet reference |
| `Driver.Mqtt` | Runtime: `MqttDriver`, `MqttConnection` (connect + reconnect supervisor), `MqttSubscriptionManager`, `LastValueCache`, factory registration, `MqttDriverProbe` |
| `Driver.Mqtt.Transport` | `MqttClientOptionsFactory` — TLS posture, PEM CA-pin chain validator, credentials, protocol mapping. Connection **policy** only; nothing here connects |
| `Driver.Mqtt.Browser` | `MqttDriverBrowser` + `MqttBrowseSession` — the AdminUI address picker's `#`-observation session |
> `.Browser` references `.Driver` (not just `.Contracts`) — a **documented, deliberate exception**
> so browse can reuse `MqttConnection.BuildClientOptions` (TLS / CA-pin / credentials / protocol
> mapping). Its cost is pulling `Core` into the AdminUI graph. The csproj carries a boxed warning;
> do not copy the pattern into a new driver's browser. The clean fix — extract the pure
> options builder down into `.Contracts` — is tracked as
> [#512](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/512).
> **`.Browser` depends on the shared leaves only** — `.Contracts` + `.Transport`, never the runtime
> `.Driver`. It used to reference `.Driver` under a documented exception; #512 removed it via two
> extractions (the client-options builder to `.Transport`, and the Sparkplug protocol layer —
> `SparkplugTopic`, `SparkplugCodec`, `RebirthRequester` — to `.Contracts`, beside the Tahu wire
> schema they encode). `MqttBrowserLayeringTests` guards the graph in both directions.
>
> There must stay **exactly one copy** of the TLS/CA-pin logic. A second copy would not break the
> build — it would silently diverge, browse quietly not honouring a pinned CA while the driver does.
## Capability Surface
```csharp
public sealed class MqttDriver
: IDriver, ITagDiscovery, ISubscribable, IReadable, IHostConnectivityProbe, IRediscoverable, IAsyncDisposable, IDisposable
: IDriver, ITagDiscovery, ISubscribable, IReadable, IWritable, IHostConnectivityProbe, IRediscoverable,
IAsyncDisposable, IDisposable
```
| Capability | Entry point | Notes |
@@ -46,12 +50,15 @@ public sealed class MqttDriver
| `ITagDiscovery` | `DiscoverAsync` | Replays **only the authored `rawTags`** into an `Mqtt` folder. `SupportsOnlineDiscovery => false`. `RediscoverPolicy` is `Once` in Plain and **`UntilStable` in Sparkplug** (half of what a Sparkplug tag needs — its datatype — is only knowable from a birth) |
| `ISubscribable` | `SubscribeAsync` / `OnDataChange` | One SUBSCRIBE per distinct authored topic. **`publishingInterval` is ignored** — MQTT is push-based; this driver neither polls nor throttles |
| `IReadable` | `ReadAsync` | Serves the last received/retained value from `LastValueCache`; never throws. A never-seen tag reads `BadWaitingForInitialData` |
| `IWritable` | `WriteAsync` | Publishes to the tag's **`writeTopic`** (Plain) or as an **NCMD/DCMD** command (Sparkplug). One status per request; a tag with no write destination is refused `BadNotWritable`. See [Write-through](#write-through) |
| `IHostConnectivityProbe` | `GetHostStatuses` | 1-second poll of `MqttConnection.State`; `HostName` is `"{host}:{port}"` |
| `IRediscoverable` | `OnRediscoveryNeeded` | Nothing raises it in Plain mode. **Sparkplug raises it** when a birth for an *authored* scope carries a metric-name set different from the last one seen for that scope — both conditions are anti-storm, see [Rediscovery](#rediscovery-sparkplug) |
**Not implemented:** `IWritable`, `IPerCallHostResolver`, `IAlarmSource`, `IHistoryProvider`.
Materialized nodes are `SecurityClassification.ViewOnly`, `IsAlarm: false`, `IsHistorized: false` —
*"MQTT ingest is one-way … ViewOnly rather than an Operate node whose writes would silently vanish."*
**Not implemented:** `IPerCallHostResolver`, `IAlarmSource`, `IHistoryProvider`.
Materialized nodes are `IsAlarm: false`, `IsHistorized: false`. A node is
`SecurityClassification.Operate` when its tag declares a write destination and
`ViewOnly` when it does not — the browse-time answer comes from the same encoder the write path uses,
so a node can never advertise writability it would then refuse.
Driver-type string: **`Mqtt`** (`DriverTypeNames.Mqtt`).
@@ -274,6 +281,81 @@ nothing forever, so `OnReconnectedAsync` **throws** when zero filters are grante
down rather than swallowing it. A SUBACK failure degrades just the affected RawPaths to
`BadCommunicationError`.
## Write-through
`IWritable` landed with [#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508). Before it,
every MQTT node materialized read-only regardless of what the operator authored.
| Mode | Destination | Payload |
|---|---|---|
| **Plain** | the tag's `writeTopic` | the value as UTF-8 text (`Raw`/`Scalar`) or a bare JSON literal (`Json`) |
| **Sparkplug B** | `spBv1.0/{group}/DCMD/{node}/{device}` for a device metric, `spBv1.0/{group}/NCMD/{node}` for an edge-node metric | a Sparkplug `Payload` carrying the metric **by name**, with an explicit `Datatype` and no `seq` |
Both publish at **QoS 0, `retain: false`**. Retain is the one that matters and is deliberately not
configurable: a retained command would be replayed by the broker to the device on *every* future
subscribe — including the device's own reconnect — so one stale write would be re-applied forever.
### `writeTopic` is separate from `topic`, and its absence is meaningful
A tag commonly reads `…/state` and writes `…/set`, so the destination is its own authored field. A
Plain tag with **no** `writeTopic` is refused `BadNotWritable` — it is not published anywhere, and the
node is discovered `ViewOnly`.
> This is the whole point of the design. The one outcome worse than read-only nodes is a node that
> *looks* writable, accepts a write, returns `Good`, and publishes nothing. Publishing back onto the
> tag's own read topic would be worse still: it would echo straight back and read as a confirmed
> device value.
```jsonc
{
"topic": "factory/oven/temp/state",
"writeTopic": "factory/oven/temp/set", // omit for a read-only tag
"payloadFormat": "Scalar",
"dataType": "Float64"
}
```
Sparkplug needs no extra key — a command is addressed by the binding tuple the tag already carries.
### `Good` means published, not applied
MQTT has no application-level write acknowledgement, so a successful PUBLISH is the strongest truth
available at this seam — the same optimistic-Good posture Galaxy's fire-and-forget write takes. Device
truth arrives afterwards as the echoed value on the read topic and corrects the node through ordinary
ingest (write-outcome self-correction, [#5](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/5)).
**Inherited consequence:** this driver can never surface a failed *device* write, exactly as Galaxy
cannot. A publish that itself fails **is** reported `Bad` — the optimism covers what the device did
with the command, never whether it was sent.
### Status codes
| Situation | Status |
|---|---|
| Published | `Good` |
| Reference is not an authored tag | `BadNodeIdUnknown` |
| No write destination, or a data type with no command encoding | `BadNotWritable` |
| Value not representable in the tag's declared type | `BadTypeMismatch` |
| Publish failed, or no live broker connection | `BadCommunicationError` |
One status per request, in request order; one bad request never fails the rest of the batch.
### `WriteIdempotent` stays default-off
MQTT gives no write ack and a Sparkplug command is not necessarily idempotent (a pulse, a recipe step,
a counter increment), so the resilience pipeline must not replay one. Operators opt in per tag.
### Two things to know
- **`jsonPath` is not applied on write.** It is a read-side extraction from the *read* topic's
document; the write topic is a different topic whose payload shape nothing in the tag config
describes. A `Json` write emits a bare JSON literal. Reconstructing a wrapped document from an
extraction path would be guesswork.
- **NCMD is a shared channel.** The driver publishes its own rebirth requests
(`Node Control/Rebirth`) on `spBv1.0/{group}/NCMD/{node}` — the same topic an edge-node metric write
uses. Anything consuming that topic must match on the metric name. The live gate caught exactly this:
a rebirth NCMD arrived before the write command. DCMD has no such contention.
## Browse
`MqttDriverBrowser` opens a **read-only observation session**: it subscribes at QoS 0 to `#` (or
@@ -384,14 +466,13 @@ identical words. See [TestConnectProbes.md](TestConnectProbes.md).
Every row is tracked. None is a silent failure — each either refuses in words or logs.
**Closed 2026-07-31:** write-through ([#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508), live-gated), the slashed-metric browse-commit
([#509](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/509)), the stale rebirth affordance ([#510](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/510)), the `.Browser` layering exception
([#512](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/512)), the stray Sparkplug `payloadFormat` ([#513](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/513)) and rebirth on a
not-yet-birthed plant ([#514](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/514)).
| Gap | Issue | Effect | Notes |
|---|:---:|---|---|
| **Write-through (`IWritable`)** | [#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508) | Every MQTT node is read-only | Deferred to P3 — NCMD/DCMD + plain publish, optimistic-Good + self-correct on echo |
| **Rediscovery is inert** | [#507](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/507) | A DBIRTH introducing a metric does not change the OPC UA tree; redeploy to pick it up | v3 **platform** gap, not MQTT-specific — nothing subscribes to `OnRediscoveryNeeded` and `DriverHostActor.HandleDiscoveredNodes` hard-returns. Also affects Galaxy / TwinCAT / AbCip. Driver-side decision is log-observable |
| **Rebirth needs a non-empty tree** | [#514](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/514) | `Request rebirth…` cannot be armed on a plant that has not birthed since the window opened | See [Refresh](#refresh). Session side already supports a bare `{group}/{edgeNode}` scope |
| **`_canRebirth` is captured at browse-open** | [#510](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/510) | A session reaped for idleness (2 min) still draws the button, and using it errors | The error is shown, not swallowed; reopen the browser |
| **Rediscovery is advisory, not a rebuild** | [#507](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/507) (closed, superseded) | A DBIRTH introducing a metric does not change the served OPC UA tree; an operator must re-browse the device and commit | **No longer "inert".** `DriverInstanceActor` now subscribes to `OnRediscoveryNeeded` and the raise rides the driver-health snapshot to the AdminUI `/hosts` page as a **re-browse chip**. It deliberately does not inject nodes: v3 authors raw tags through the `/raw` browse-commit flow, and grafting nodes at runtime would materialise ones nobody approved and no deployment artifact records. #507's injection path was deleted rather than revived |
| **Primary-host STATE publishing** | [#511](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/511) | `ActAsPrimaryHost` does nothing | Logs a startup warning rather than being silently inert — see the `Sparkplug` sub-object table |
| **A metric whose name contains `/` cannot be browse-committed** | [#509](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/509) | The commit is refused **whole**, in words (`Row N: Name must not contain '/'`) | The derived **tag Name** is a RawPath segment and may not contain `/`, while `metricName` legitimately may. This blocks the spec-mandatory `Node Control/Rebirth`. **Workaround: Manual entry** — author the tag with a `/`-free name and type the slashed `metricName` into the Sparkplug editor, which accepts it. Nothing is silently mis-bound |
| **`DataSet` / `Template` / `PropertySet` metrics** | [#515](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/515) | Skipped with a warning | v1-unsupported; never coerced into a guessed type |
| **The tag editor writes `payloadFormat` into a Sparkplug blob** | [#513](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/513) | Cosmetic only | `payloadFormat` is Plain-only; the factory routes on the Sparkplug keys and ignores it. Noise in the blob, not a mis-binding |
| **`.Browser` references `.Driver`, not just `.Contracts`** | [#512](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/512) | A documented layering exception | Taken to reuse `MqttConnection.BuildClientOptions` rather than duplicate TLS/CA-pin logic — see the note at the top of this doc |