lmxopcua/docs/drivers/Mqtt.md

# MQTT Driver

In-process MQTT **broker-subscribe** driver. It holds one MQTTnet 5 client against one broker,
subscribes to the topics its authored tags name, and forwards each PUBLISH straight to
`ISubscribable.OnDataChange`. Ingest is **push, not poll**; writes go back out as ordinary publishes
(see [Write-through](#write-through)).

**Two ingest shapes, selected by the driver's `Mode`:** `Plain` binds a tag to a concrete MQTT topic;
`SparkplugB` binds it to a `group / edge node / device? / metric` tuple and decodes Eclipse-Tahu
protobuf under one `spBv1.0/{GroupId}/#` subscription. Both run over the same single client.

See [Mqtt-Test-Fixture.md](Mqtt-Test-Fixture.md) for the harness and
[`../plans/2026-07-15-mqtt-sparkplug-driver-design.md`](../plans/2026-07-15-mqtt-sparkplug-driver-design.md)
for the design.

> **Status.** P1 (plain MQTT), P2 (Sparkplug B ingest), write-through
> ([#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508)) and the Sparkplug primary-host
> claim ([#511](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/511)) are all shipped and
> live-gated against the Mosquitto + Sparkplug-simulator fixture. See
> [Known gaps](#known-gaps) — every entry there carries a tracking issue.

## Project Layout

| Project | Holds |
|---|---|
| `Driver.Mqtt.Contracts` | Config records, enums, `MqttTagDefinition` + the pure `MqttTagDefinitionFactory`, and the one shared `MqttJson.Options`. **Deliberately transport-free** — no MQTTnet reference |
| `Driver.Mqtt` | Runtime: `MqttDriver`, `MqttConnection` (connect + reconnect supervisor), `MqttSubscriptionManager`, `LastValueCache`, `SparkplugIngestor`, `SparkplugPrimaryHost`, factory registration, `MqttDriverProbe` |
| `Driver.Mqtt.Transport` | `MqttClientOptionsFactory` — TLS posture, PEM CA-pin chain validator, credentials, protocol mapping. Connection **policy** only; nothing here connects |
| `Driver.Mqtt.Browser` | `MqttDriverBrowser` + `MqttBrowseSession` — the AdminUI address picker's `#`-observation session |

> **`.Browser` depends on the shared leaves only** — `.Contracts` + `.Transport`, never the runtime
> `.Driver`. It used to reference `.Driver` under a documented exception; #512 removed it via two
> extractions (the client-options builder to `.Transport`, and the Sparkplug protocol layer —
> `SparkplugTopic`, `SparkplugCodec`, `RebirthRequester` — to `.Contracts`, beside the Tahu wire
> schema they encode). `MqttBrowserLayeringTests` guards the graph in both directions.
>
> There must stay **exactly one copy** of the TLS/CA-pin logic. A second copy would not break the
> build — it would silently diverge, browse quietly not honouring a pinned CA while the driver does.

## Capability Surface

```csharp
public sealed class MqttDriver
    : IDriver, ITagDiscovery, ISubscribable, IReadable, IWritable, IHostConnectivityProbe, IRediscoverable,
      IAsyncDisposable, IDisposable
```

| Capability | Entry point | Notes |
|---|---|---|
| `IDriver` | `InitializeAsync` / `ReinitializeAsync` / `ShutdownAsync` / `GetHealth` | Order **register → attach → connect** is contractual. `ReinitializeAsync` applies a tag-only delta in place, and rebuilds the session when the broker identity changed |
| `ITagDiscovery` | `DiscoverAsync` | Replays **only the authored `rawTags`** into an `Mqtt` folder. `SupportsOnlineDiscovery => false`. `RediscoverPolicy` is `Once` in Plain and **`UntilStable` in Sparkplug** (half of what a Sparkplug tag needs — its datatype — is only knowable from a birth) |
| `ISubscribable` | `SubscribeAsync` / `OnDataChange` | One SUBSCRIBE per distinct authored topic. **`publishingInterval` is ignored** — MQTT is push-based; this driver neither polls nor throttles |
| `IReadable` | `ReadAsync` | Serves the last received/retained value from `LastValueCache`; never throws. A never-seen tag reads `BadWaitingForInitialData` |
| `IWritable` | `WriteAsync` | Publishes to the tag's **`writeTopic`** (Plain) or as an **NCMD/DCMD** command (Sparkplug). One status per request; a tag with no write destination is refused `BadNotWritable`. See [Write-through](#write-through) |
| `IHostConnectivityProbe` | `GetHostStatuses` | 1-second poll of `MqttConnection.State`; `HostName` is `"{host}:{port}"` |
| `IRediscoverable` | `OnRediscoveryNeeded` | Nothing raises it in Plain mode. **Sparkplug raises it** when a birth for an *authored* scope carries a metric-name set different from the last one seen for that scope — both conditions are anti-storm, see [Rediscovery](#rediscovery-sparkplug) |

**Not implemented:** `IPerCallHostResolver`, `IAlarmSource`.
Materialized nodes are `IsAlarm: false`, `IsHistorized: false`. A node is
`SecurityClassification.Operate` when its tag declares a write destination and
`ViewOnly` when it does not — the browse-time answer comes from the same encoder the write path uses,
so a node can never advertise writability it would then refuse.

Driver-type string: **`Mqtt`** (`DriverTypeNames.Mqtt`).

## Configuration

Bound through `MqttJson.Options` — **case-insensitive**, enums by **name**, unknown members skipped.

**The whole connection is authored on the DRIVER** (`DriverConfig`), via `MqttDriverForm` in the `/raw`
driver-config modal. Unlike Modbus / S7 / OPC UA Client, MQTT does **not** use the v3
endpoint→`DeviceConfig` split: an MQTT driver instance holds exactly one broker session, so
`MqttDeviceForm` is informational (the same shape as `GalaxyDeviceForm`) and devices under an MQTT
driver are pure organisational grouping in the RawPath.

> **Why not the device.** `DriverDeviceConfigMerger` merges a device's keys up to the top level *only
> when the driver has exactly one device*. A device-authored broker connection therefore disappears
> from the merged blob the moment a second device is added — a silent outage with nothing to see at
> deploy time. Driver-level authoring is correct for 0, 1 or N devices. A pre-form deployment that put
> the connection on a sole device still works (merge-up wins over the driver's keys) and
> `MqttDeviceForm` flags those keys so the mismatch is visible; move them to the driver.

`DriverDeviceConfigMerger` still merges driver + device and injects `rawTags`.

```jsonc
// DriverConfig — authored by MqttDriverForm. PascalCase keys, enums as names.
{
  "Host": "10.100.0.35",
  "Port": 8883,
  "UseTls": true,
  "AllowUntrustedServerCertificate": false,
  "CaCertificatePath": null,      // pins the chain; null = OS trust store
  "Username": "otopcua",
  "Password": "",                 // never commit
  "ProtocolVersion": "V500",
  "CleanSession": true,
  "KeepAliveSeconds": 30,
  "ConnectTimeoutSeconds": 15,
  "ReconnectMinBackoffSeconds": 1,
  "ReconnectMaxBackoffSeconds": 30,
  "MaxPayloadBytes": 1048576,
  "Mode": "Plain",
  "Plain": { "TopicPrefix": "", "DefaultQos": 1 }
}

// DeviceConfig — empty; MQTT has no per-device endpoint.
{}
```

### `Sparkplug` sub-object (`Mode: "SparkplugB"`)

```jsonc
{
  "Mode": "SparkplugB",
  "Sparkplug": {
    "GroupId": "OtOpcUaSim",            // REQUIRED — the driver's whole subscription scope
    "HostId": "",                       // Host Application identity — spBv1.0/STATE/{HostId}
    "ActAsPrimaryHost": false,          // claim the primary host id (Primary-gated; see below)
    "RequestRebirthOnGap": true,        // a seq gap asks the edge node to re-announce (NCMD)
    "BirthObservationWindowSeconds": 15 // how long discovery collects births before calling it stable
  }
}
```

| Key | Default | Notes |
|---|---|---|
| `GroupId` | `""` | **Required.** The driver subscribes exactly `spBv1.0/{GroupId}/#`. Blank ⇒ it connects, reports `Healthy`, and ingests **nothing**. It is a literal topic segment, so `/`, `+`, `#` are refused by the form |
| `HostId` | `""` | Sparkplug Host Application identity — the `spBv1.0/STATE/{HostId}` topic. Optional for observation; **required** to claim. Set without `ActAsPrimaryHost`, the driver only observes |
| `ActAsPrimaryHost` | `false` | Claim the Sparkplug Primary Host Application identity. **Primary-gated** — only the node holding the redundancy Primary role claims it. See [Primary host](#primary-host-actasprimaryhost) |
| `RequestRebirthOnGap` | `true` | On a detected sequence-number gap, publish a rebirth-request NCMD instead of running on stale metric state |
| `BirthObservationWindowSeconds` | `15` | Discovery's birth-collection window. Must be ≥ 1 |

`MqttDriverForm` authors all five in Sparkplug mode. The sub-object is **merged, never replaced** (a
key a newer driver adds inside it survives an older AdminUI), and in `Plain` mode it is not touched at
all — so flipping a driver to Plain to look at it and back does not lose the group id. The form never
writes `RawTags`; the deploy artifact owns it.

> **This was a live-gate finding.** Through the P2 implementation `MqttDriverForm` still carried its P1
> placeholder — switching Mode to `SparkplugB` rendered a "not available yet" notice and **no group-id
> field**, so once Sparkplug ingest shipped there was no way to author a Sparkplug driver from the
> AdminUI at all. Pinned by `MqttDriverFormModelTests`' Sparkplug cases.

Every key has a default, so nothing is strictly required by the binder. Notable ones:
`port` **8883**, `useTls` **true**, `protocolVersion` **V500**, `cleanSession` **true**,
`keepAliveSeconds` **30**, `connectTimeoutSeconds` **15**,
`reconnectMinBackoffSeconds` **1** / `reconnectMaxBackoffSeconds` **30**,
`maxPayloadBytes` **1 MiB**, `allowUntrustedServerCertificate` **false**.

> ⚠️ **Leave `clientId` unset on a redundant pair.** Both nodes of a pair run the driver. MQTT
> requires client ids to be unique per broker, so a *fixed* `clientId` makes the two nodes evict each
> other forever — the broker logs `Client <id> already connected, closing old connection` and both
> nodes reconnect every few seconds while still reporting `Healthy`. Unset (the default) lets MQTTnet
> generate a unique id per connection, which is correct. This was observed live during the P1 gate.
> (The probe and browser already self-uniquify with `-probe-` / `-browse-` suffixes.) `MqttDriverForm`
> defaults the field blank and raises this warning inline the moment a value is typed.

The form validates every knob against the driver's own `[Range]` bounds (port 1–65535, keep-alive /
connect-timeout / backoffs ≥ 1 s, max-backoff ≥ min-backoff, `maxPayloadBytes` ≥ 1, QoS 0–2) and
additionally **clamps** on serialize, so an operator who ignores the inline error still cannot persist
a `connectTimeoutSeconds: 0` driver-brick.

## Tag Configuration

A tag binds in **one of two shapes**, and which one is decided by *which keys are present*, not by a
`mode` key on the tag. `MqttTagDefinitionFactory` reads the Sparkplug tuple when any Sparkplug
descriptor key is set, and the topic otherwise; the AdminUI's "Tag shape" selector infers the same way
on reopen. The key names are single-sourced in `MqttTagConfigKeys` so the browse-commit mapper and the
factory cannot drift.

### Plain shape

| Key | Type | Default | Notes |
|---|---|---|---|
| `topic` | string | — | **Required.** Absent/blank rejects the tag ⇒ `BadNodeIdUnknown` |
| `payloadFormat` | `Json` \| `Raw` \| `Scalar` | `Json` | **Strict** — a typo'd value rejects the tag |
| `jsonPath` | string | `"$"` | Absent *or empty* ⇒ document root. `Json` only |
| `dataType` | `DriverDataType` | `String` | **Strict.** `Float64`, not `Double` — see below |
| `qos` | int 0–2 | driver's `defaultQos` | **Strict** — a malformed value rejects rather than silently weakening delivery |
| `retainSeed` | bool | `true` | Seed this tag from the broker's retained message on subscribe |

There is **no `mode` key** — the AdminUI's "Tag shape" selector is UI-only, inferred from the blob and
never serialized.

**`dataType` uses `DriverDataType`, which has `Float64` and has no `Double`.** Authoring `"Double"` is
a strict-enum rejection, not a fallback.

Payload handling: `Json` selects via JSONPath then coerces (miss ⇒ `BadDecodingError`, coerce fail ⇒
`BadTypeMismatch`); `Raw` returns the payload's UTF-8 text verbatim and **ignores `dataType`**;
`Scalar` decodes then coerces. The JSONPath subset is `$`, `$.a.b`, `$['a']`, `$.a[0]` — **no
filters, slices, wildcards or recursive descent**.

### Wildcards

The **runtime accepts** a `+`/`#` topic (it is ambiguous, not unparseable) and the deploy-time
`Inspect` pass **warns**. The **AdminUI editor rejects** it outright — the one place the editor is
deliberately stricter than the driver, because one Tag holds one value and a wildcard would feed it
from many source topics. Wildcard tags are indexed **by filter**, not by concrete topic, so they
survive a reconnect; a message fans out to *every* matching tag, and unauthored topics are dropped.

### Sparkplug shape

| Key | Type | Default | Notes |
|---|---|---|---|
| `groupId` | string | — | **Required.** Must match the driver's `Sparkplug.GroupId` to ever receive a message |
| `edgeNodeId` | string | — | **Required.** |
| `deviceId` | string | *absent* | **Omit** for a metric published by the edge node itself (NBIRTH/NDATA). Absent is meaningfully different from `""` |
| `metricName` | string | — | **Required.** The tag's stable binding key across rebirths. **May contain `/`** — `Node Control/Rebirth` and `Properties/Serial Number` are canonical Sparkplug names, and the whole name is one value, never split. Also how a flattened structured member is addressed: `Motor/Speed`, `Recipe/Target/0` (see [Structured metrics](#structured-metrics-template--dataset--propertyset)) |
| `dataType` | `DriverDataType` | *from the birth* | **Optional override.** Absent ⇒ take whatever type the birth certificate declared. The AdminUI's "(from birth certificate)" option removes the key entirely |
| `isHistorized` / `historianTagname` | — | — | Server-side historization, unchanged from Plain |

`payloadFormat`, `jsonPath`, `qos` and `retainSeed` are **Plain-only** and meaningless here: a
Sparkplug body is protobuf, and the subscription is one driver-wide `spBv1.0/{GroupId}/#`, not a
per-tag filter.

`groupId` / `edgeNodeId` / `deviceId` are literal MQTT topic segments, so the AdminUI editor refuses
`/`, `+` and `#` in them — a decoded incoming id can never contain one, so such a tag could never
bind. `metricName` is deliberately **exempt**: it is not a topic segment.

```jsonc
// A device metric.
{"groupId":"OtOpcUaSim","edgeNodeId":"EdgeA","deviceId":"Filler1","metricName":"FillCount"}

// A node-level metric — note deviceId is ABSENT, not blank.
{"groupId":"OtOpcUaSim","edgeNodeId":"EdgeA","metricName":"Temperature"}

// A flattened Template member, and a flattened DataSet cell. `Motor` and `Recipe` are what the wire
// carries; nothing on the wire is ever called by these names.
{"groupId":"OtOpcUaSim","edgeNodeId":"EdgeA","deviceId":"Filler1","metricName":"Motor/Stator/Temp"}
{"groupId":"OtOpcUaSim","edgeNodeId":"EdgeA","deviceId":"Filler1","metricName":"Recipe/Target/1"}
```

## Sparkplug B Ingest

One subscription (`spBv1.0/{GroupId}/#`) feeds a per-edge-node state machine
(`SparkplugIngestor` + `SparkplugCodec` / `AliasTable` / `BirthCache` / `SequenceTracker`):

- **Births are the only self-describing message.** NBIRTH/DBIRTH name every metric and assign its
  alias; NDATA/DDATA carry an **alias and no name at all**. So the alias table is rebuilt *per birth*
  and tags bind **by name**, never by alias — aliases are per-birth and may be reused for a different
  metric after a rebirth.
- **NBIRTH is node-scoped, DBIRTH is device-scoped**, and routing a DBIRTH through the node path
  wipes `bdSeq`. They are handled separately on purpose.
- **Sequence tracking**: `seq` is a wrapping 0–255 counter. A detected gap raises a rebirth request
  when `RequestRebirthOnGap` is on, rather than silently continuing on stale metric state.
- **Death → STALE.** An NDEATH whose `bdSeq` matches the live session (the bdSeq tie is what stops a
  stale death from killing a fresh session) marks every metric under that edge node `Bad` /
  `BadNoCommunication`. A DDEATH does the same for one device. The next birth restores them.
- **Structured metrics flatten to sibling tags** — `Template` and `DataSet`; see
  [Structured metrics](#structured-metrics-template--dataset--propertyset).
- **Unsupported datatypes are skipped with a warning, never guessed**: `PropertySetList`, `Unknown`,
  extension values, and any `Template`/`DataSet` that cannot be flattened honestly. `Int8`→`Int16` and
  `UInt8`→`UInt16` widen; `*Array` variants map to the element type with the array bit set.
- **Ingest state is reset at every session/authoring seam** — a reconnect or a `ReinitializeAsync`
  must not carry a previous session's alias table into a new one.

### Rediscovery (Sparkplug)

`OnRediscoveryNeeded` fires only when a birth is **for an authored scope** *and* its metric-name set
**differs** from the last one seen for that scope (order-insensitive, ordinal). Both conditions are
anti-storm, not optimisations: rediscovery triggers an address-space rebuild, and an edge node
re-births freely — on its own schedule, on every reconnect, and once per rebirth NCMD.

> ⚠️ **The signal is an operator prompt, not a rebuild.** `DriverInstanceActor` subscribes and the
> raise rides the driver-health snapshot to the AdminUI `/hosts` page as a **re-browse chip** — but the
> served address space still comes from the deploy artifact, so **a DBIRTH introducing a new metric does
> not change the OPC UA tree**. Re-browse the device and commit. That is deliberate: v3 authors raw tags
> through the `/raw` browse-commit flow, and grafting nodes at runtime would materialise ones nobody
> approved and no deployment artifact records.

## Structured metrics (`Template` / `DataSet` / `PropertySet`)

Sparkplug metrics are not all scalars. `SparkplugFlattener`
([#515](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/515)) turns the structured ones into
addressable tags:

| Metric kind | Mapping | Example |
|---|---|---|
| `Template` (UDT instance) | members become **sibling metrics**, `/`-joined, recursively | `Motor` → `Motor/Speed`, `Motor/Stator/Temp` |
| `DataSet` | **one metric per cell**, `{column}/{rowIndex}` | `Recipe` → `Recipe/Step/0`, `Recipe/Target/1` |
| `PropertySet` (a metric's `properties`) | node **metadata**, never tags | `engUnit`, `Range/Low` |

Flattened members are authored exactly like any other Sparkplug metric — the composed name goes in
`metricName`. The browse picker offers them directly (the structured **parent** is deliberately *not*
offered: it has no value and no servable datatype, so a tag committed against it could never read).

**Why flattening rather than OPC UA Structures.** Owner decision, 2026-07-31. Structures are truer to
Sparkplug's model but much harder to consume — flattened scalars are readable by any OPC UA client
with no type dictionary, and the result mirrors how the `/raw` tree already works.

### Things worth knowing before you touch this

- **The row index is always present, including on row 0.** `Recipe/Target/0`, never `Recipe/Target`.
  Emitting a bare column name for a single-row table and an indexed one otherwise would make a tag's
  name depend on how many rows the plant published — so a table growing from one row to two would
  rename every tag already authored against it.
- **A DATA metric carries the parent's alias and no name**, so member names cannot be composed until
  the alias table has resolved the parent. Flattening at decode time alone is not enough; the
  expansion happens in `SparkplugIngestor.PublishStructuredMembers`, after resolution.
- **Member datatype falls back to the birth's.** A DDATA template may omit its members' datatypes.
  Guessing from the CLR type of the wire value would read a signed `Int32` of −42 out of its unsigned
  field as 4294967254 — plausible, `Good`, and wrong.
- **Skip-and-warn survives, and refusal is whole.** A template **definition** (`is_definition`), an
  unnamed member, an extension value, a column/type count mismatch, a short row, or a payload past
  `MaxDepth` (5) / `MaxMembers` (512) refuses the **entire** metric rather than publishing part of it.
  A silently half-flattened UDT looks exactly like a complete one in the picker.
- **Member paths contain `/`**, which `RawBrowseCommitMapper.ToRawSegment`
  ([#509](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/509)) already handles: the slashed name
  travels intact as `metricName` while the derived tag **Name** is coerced to a legal RawPath segment,
  with a collision guard for two members that would sanitise onto one name.
- **The rediscovery signature tracks members, not the parent.** A UDT gaining or losing a member leaves
  the parent metric's name identical, so a gate built on raw wire names would never move.

`PropertySet` is projected as metadata rather than as sibling tags because it *describes* a metric
(units, ranges, quality hints) — surfacing it as tags would invent plant signals out of documentation.
`PropertySetList` and property extension values are dropped rather than stringified.

## Primary host (`actAsPrimaryHost`)

A Sparkplug **Primary Host Application** is the thing edge nodes watch to decide whether anyone is
listening — many stop publishing while their host is offline. Claiming it is a three-part contract,
and this driver ships all three or none of it
([#511](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/511)):

1. a **retained** `{"online":true,"timestamp":…}` on `spBv1.0/STATE/{hostId}`, QoS 1, published after CONNACK;
2. a matching **retained `OFFLINE` armed as the MQTT Last Will**, so a crash still reports the truth;
3. an explicit `OFFLINE` on a clean stand-down, so edge nodes do not wait out a keep-alive timeout.

A claim the broker keeps refusing backs off (STAB-27): 2 s, 4 s, 8 s … capped at 30 s between attempts, one
Warning per attempt, reset by a successful claim or a stand-down. The role is still sampled every second.

Set `Sparkplug.HostId` and `Sparkplug.ActAsPrimaryHost: true`. A blank host id is **refused** with a
warning — the driver does not publish to `spBv1.0/STATE/`, which is a legal topic reaching nobody.

### The claim is a whole connection, not a message

A Will can only be registered in the **CONNECT packet**. It cannot be added to a live session, changed,
or withdrawn. So the claim gets its **own** broker connection (client id `{clientId}-spbhost`, forced
clean-session) whose lifetime *is* the claim: opened on gaining Primary, closed on losing it. Every leg
then follows from the connection's lifecycle rather than from bookkeeping that could drift.

The ingest session is untouched by a role change — no dropped subscriptions, no lost birth cache, and
no data gap at the Secondary→Primary transition, which is the moment a plant most needs values flowing.

### Primary-gated — and why the obvious alternatives are wrong

Both nodes of a redundant pair subscribe independently (MQTT fan-out is per-connection, and the Primary
gate applies downstream of ingest), so an ungated claim means both claim the same host id. The sharp
failure is not the two fighting over the retained `STATE`. It is that the **Secondary's** death fires
*its* Will, publishing `OFFLINE` while the Primary is perfectly healthy — every edge node in the group
stops publishing, and redundancy has caused the outage the mechanism exists to prevent.

Registering the Will only on whichever node is Primary *at connect time* fails the other way: a node
that later gains Primary would publish `ONLINE` with no Will behind it, and a crash would leave a
retained `ONLINE` asserting forever that a dead host is alive.

Hence: bind the claim to a connection, and rebuild the connection on role change.

| Node state | STATE connection | Retained `STATE` |
|---|---|---|
| Primary | open, Will armed | `ONLINE` |
| Secondary / Detached | **never opened** | — (whatever the Primary published) |
| Role unknown, driver peer present | not opened (fail closed) | — |
| Role unknown, no driver peer (single node) | open | `ONLINE` |

The verdict comes from `PrimaryGatePolicy.ShouldServiceAsPrimary` via `IDriverPrimaryRoleSource`
(`Core.Abstractions`), published by `DriverHostActor` at host start, on every cluster membership change and on
every redundancy snapshot, and **sampled on a 1-second timer** rather than evented — a process-singleton
raising events at per-instance drivers outlives every driver it holds a reference to.

> ⚠️ **"Driver peer present" is this node's own cluster view.** Before the node has joined, it sees no
> members, which is the single-node row: a node cold-starting into a pair whose peer is already Primary
> can open the claim until its join completes (seconds; up to the bootstrap guard's 25 s partner probe),
> so both nodes may briefly hold it (review R6 MINOR-2).

> **Accepted trade: a brief edge-visible `OFFLINE` during failover**, while the old Primary stands down
> and the new one connects and republishes. That is a genuine state transition — for that window there
> really is no host — not a glitch worth hiding.

### Operational notes

- **At most one primary host per host id per broker.** Two separate OtOpcUa deployments pointing at one
  broker with the same `hostId` will fight, and nothing in either can detect the other. The gate covers
  a redundant *pair*, not two independent deployments.
- **A reconnect republishes the birth.** A broker restart drops retained messages; without the
  republish the host would be connected and believed offline by everyone, silently.
- **A failed claim never faults the driver.** A broker whose ACL denies `spBv1.0/STATE/#` costs a
  warning and a plant-visible missing host, not every tag's value. Ingest is the data plane; the claim
  is an assertion to other applications.
- **The driver still observes its own `STATE`.** The ingestor subscribes the configured host id
  regardless, so a claiming node sees its own retained `ONLINE` come back. Nothing gates on it, so the
  echo costs one log line.

## Connection + Failure Semantics

`MqttConnectionState`: `Disconnected → Connected → Reconnecting → Faulted → Disposed`.
A broker that is merely down stays **`Reconnecting` indefinitely** — unreachable is never a fault.
Backoff is exponential between the two `reconnectBackoff` bounds.

**MQTTnet 5 does not throw on a rejected CONNACK** — it returns the code and leaves the client
disconnected. Left unhandled, a wrong password produced a `Healthy` driver that never received a
value. `MqttConnection` therefore raises `MqttConnectRejectedException`:

- **Unrecoverable ⇒ `Faulted`, supervisor stops** (retrying cannot help): `MalformedPacket`,
  `ProtocolError`, `UnsupportedProtocolVersion`, `ClientIdentifierNotValid`, `BadUserNameOrPassword`,
  `NotAuthorized`, `Banned`, `BadAuthenticationMethod`, `TopicNameInvalid`, `PacketTooLarge`,
  `PayloadFormatInvalid`, `RetainNotSupported`, `QoSNotSupported`, `ServerMoved`.
- **Everything else is transient** and keeps retrying — including `UnspecifiedError` and any future
  code. (`ServerMoved` is permanent, `UseAnotherServer` is temporary — per the MQTT 5 spec.)

A reconnect that completes without re-subscribing would be a healthy-looking connection that receives
nothing forever, so `OnReconnectedAsync` **throws** when zero filters are granted, tearing the session
down rather than swallowing it. A SUBACK failure degrades just the affected RawPaths to
`BadCommunicationError`.

There are **two** kinds of reconnect and only one of them fires that callback. A drop the connection's
own supervisor recovers keeps the same `MqttConnection`, so `Reconnected` → `OnReconnectedAsync`
re-establishes every live filter. A **session rebuild** — `ShutdownAsync` + `InitializeAsync` (the #527
comms-loss watchdog's retry path, and a `ReinitializeAsync` that changes the endpoint or credentials) — builds
a *new* connection and re-attaches the *same* `MqttSubscriptionManager`, and the re-subscribe then comes
from the host (`DriverInstanceActor.ResubscribeDesired`) rather than from the callback. `AttachTo`
therefore **forgets the filters the previous session had established**: the dedupe set is a claim about
one session, and carrying it across made the host's re-subscribe resolve an empty pending set, send no
SUBSCRIBE at all, and leave the driver Connected/Healthy and permanently deaf. Sparkplug's
`SparkplugIngestor.AttachTo` resets its own session state for the same reason.

## Quality on connection loss and re-subscribe

The quality contract this implements is stated once, in
[`docs/DriverLifecycle.md` §"The quality contract"](../DriverLifecycle.md#the-quality-contract).
Two edge behaviours make MQTT — a push-model driver — honour it. A poll driver gets both free from
`PollGroupEngine` (a fresh subscription's first poll runs with `forceRaise: true`); MQTT had neither,
which is archreview 05/STAB-18.

- **Connection loss stales, keeping the value.** On the `Connected → Reconnecting` edge
  `MqttConnection.ConnectionLost` fires, and both ingest paths mark every **already-observed**
  reference `BadNoCommunication` with its **last value retained** (server-side; a client reads the Bad status with a null value) and publish the change. A
  never-observed reference (still `BadWaitingForInitialData`) and an already-Bad one are left alone.
  The event does **not** fire for a failed *initial* connect or during dispose — neither is an outage.
- **Every (re)subscribe pushes the cached snapshots.** `SubscribeAsync` republishes each observed
  reference's cached snapshot onto the new handle before any broker traffic. This is what refills a
  node an address-space rebuild reseeded `BadWaitingForInitialData` on a quiet topic — a rebuild
  triggers no retained replay and, in Sparkplug, no rebirth.

**Why the stale is what makes recovery work.** The plain path suppresses a message whose value *and*
status equal the driver-owned cache, and that cache survives reconnects and redeploys. So the retained
message a broker replays on reconnect — routinely the identical payload — was suppressed, and every
node the host had latched Bad stayed Bad until the plant value happened to change. Staling the status
first makes the identical replay a change. (A regression test for this must replay the **identical**
payload; a different value defeats the suppression on its own and proves nothing.)

**Two "not communicating" codes, deliberately.** `BadNoCommunication` (0x80310000) means *the link is
down* — the connection-loss stale, and what the host's comms-loss fan-out writes onto the value nodes.
`BadCommunicationError` (0x80050000) keeps its narrower Sparkplug meaning: the **source announced its
own death** (NDEATH/DDEATH), or a birth voided the scope feeding the tag. A refused SUBSCRIBE filter
also degrades to `BadCommunicationError`.

**Operator-visible change:** an MQTT tag now goes Bad when the broker session drops, rather than
holding its last value at Good until the host's 90 s comms-loss watchdog notices — which, since
MQTTnet usually reconnects well inside that window, in practice meant never.

## Write-through

`IWritable` landed with [#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508). Before it,
every MQTT node materialized read-only regardless of what the operator authored.

| Mode | Destination | Payload |
|---|---|---|
| **Plain** | the tag's `writeTopic` | the value as UTF-8 text (`Raw`/`Scalar`) or a bare JSON literal (`Json`) |
| **Sparkplug B** | `spBv1.0/{group}/DCMD/{node}/{device}` for a device metric, `spBv1.0/{group}/NCMD/{node}` for an edge-node metric | a Sparkplug `Payload` carrying the metric **by name**, with an explicit `Datatype` and no `seq` |

Both publish with `retain: false`. A **Plain** write goes at **QoS 1** (STAB-26): the publish completes only
once the broker PUBACKs it, so `Good` means "the broker accepted it", and a broker refusal or a connection
that died under the send is `Bad`. **Sparkplug commands stay at QoS 0** — a redelivered NCMD/DCMD would be
re-applied. A Sparkplug metric the tag did not give a `dataType` is commanded with the type its live birth
certificate declared (STAB-25), the same authored-beats-birth precedence reads use — using the birth's **own**
Sparkplug datatype, not the lossy driver-type mapping, so Int8/UInt8/Text/Uuid/Bytes/File metrics are commanded
with their real type; a birth type the encoder cannot represent (any `*Array`, DataSet, Template, …) is refused
`BadNotSupported` rather than sent as another type. Retain is the one that
matters and is deliberately not configurable: a retained command would be replayed by the broker to the device on *every* future
subscribe — including the device's own reconnect — so one stale write would be re-applied forever.

**Plain QoS 1 is at-least-once.** Know the consequences before pointing a Plain `writeTopic` at a non-idempotent
device topic:

- A lost PUBACK can make a device subscribed at QoS ≥ 1 with a persistent session receive the command **twice**
  (at QoS 0 delivery was downgraded and a duplicate could not happen).
- A lost PUBACK, or the `ConnectTimeoutSeconds` (15 s) deadline, reports `Bad` and the node is **reverted**, even
  though the broker may already have delivered the write. `Good` is "the broker accepted it", `Bad` is "not
  confirmed", never "not delivered".
- Each write holds `DriverInstanceActor`'s mailbox until the PUBACK arrives — up to 15 s per write, one after
  another; comparable to a Modbus round trip.
- A broker advertising MQTT 5 Maximum QoS 0 will disconnect on every Plain write.

There is no per-tag QoS option; make the receiving topic idempotent (absolute setpoints, not increments).

### `writeTopic` is separate from `topic`, and its absence is meaningful

A tag commonly reads `…/state` and writes `…/set`, so the destination is its own authored field. A
Plain tag with **no** `writeTopic` is refused `BadNotWritable` — it is not published anywhere, and the
node is discovered `ViewOnly`.

> This is the whole point of the design. The one outcome worse than read-only nodes is a node that
> *looks* writable, accepts a write, returns `Good`, and publishes nothing. Publishing back onto the
> tag's own read topic would be worse still: it would echo straight back and read as a confirmed
> device value.

```jsonc
{
  "topic":      "factory/oven/temp/state",
  "writeTopic": "factory/oven/temp/set",     // omit for a read-only tag
  "payloadFormat": "Scalar",
  "dataType":   "Float64"
}
```

Sparkplug needs no extra key — a command is addressed by the binding tuple the tag already carries.

### `Good` means published, not applied

MQTT has no application-level write acknowledgement, so a successful PUBLISH is the strongest truth
available at this seam — the same optimistic-Good posture Galaxy's fire-and-forget write takes. Device
truth arrives afterwards as the echoed value on the read topic and corrects the node through ordinary
ingest (write-outcome self-correction, [#5](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/5)).

**Inherited consequence:** this driver can never surface a failed *device* write, exactly as Galaxy
cannot. A publish that itself fails **is** reported `Bad` — the optimism covers what the device did
with the command, never whether it was sent.

### Status codes

| Situation | Status |
|---|---|
| Published | `Good` |
| Reference is not an authored tag | `BadNodeIdUnknown` |
| No write destination, or a data type with no command encoding | `BadNotWritable` |
| Value not representable in the tag's declared type | `BadTypeMismatch` |
| Publish failed, or no live broker connection | `BadCommunicationError` |

One status per request, in request order; one bad request never fails the rest of the batch.

### `WriteIdempotent` stays default-off

MQTT gives no write ack and a Sparkplug command is not necessarily idempotent (a pulse, a recipe step,
a counter increment), so the resilience pipeline must not replay one. Operators opt in per tag.

### Two things to know

- **`jsonPath` is not applied on write.** It is a read-side extraction from the *read* topic's
  document; the write topic is a different topic whose payload shape nothing in the tag config
  describes. A `Json` write emits a bare JSON literal. Reconstructing a wrapped document from an
  extraction path would be guesswork.
- **NCMD is a shared channel.** The driver publishes its own rebirth requests
  (`Node Control/Rebirth`) on `spBv1.0/{group}/NCMD/{node}` — the same topic an edge-node metric write
  uses. Anything consuming that topic must match on the metric name. The live gate caught exactly this:
  a rebirth NCMD arrived before the write command. DCMD has no such contention.

## Browse

`MqttDriverBrowser` opens a **read-only observation session**: it subscribes at QoS 0 to `#` (or
`{topicPrefix}/#`) under a `-browse-`-suffixed client id and builds a `/`-segment tree from whatever
publishes. **Nothing is ever published.** It is *inherently incomplete* — a topic that stays silent
during the window is invisible — so manual entry remains the escape hatch.

Bounds: 50 000 nodes (then `⚠ Observation truncated`), 1024-char topics / 256-char segments (then
`⚠ Some topics were too long`), 512-byte payload inspection. Sessions are reaped after 2 minutes idle.

**Sparkplug B browse serves a different tree from the same passive window**: observed NBIRTH/DBIRTH
certificates are decoded into `Group → EdgeNode → [Device] → Metric` (a birth is the only Sparkplug
message that names its metrics). Same node cap, same bounds, same "publishes nothing" guarantee.

The bespoke browser wins over the universal `DiscoveryDriverBrowser`, which would decline MQTT anyway
(`SupportsOnlineDiscovery` is `false`).

### Browse-commit writes the address the factory reads

A committed leaf's address is a **descriptor**, not a single reference string, so
`RawBrowseCommitMapper` reads it from the `AttributeInfo.AddressFields` the session states — Plain
emits `topic`; Sparkplug emits `groupId` / `edgeNodeId` / `deviceId?` / `metricName`, the names
`MqttTagConfigKeys` single-sources for both the mapper and `MqttTagDefinitionFactory`.

**It is never parsed out of the browse node id.** A Sparkplug node id is
`{group}/{node}[/{device}]::{metric}` and a metric name legitimately contains `/`
(`Node Control/Rebirth`), so splitting the id cannot recover the tuple. Which keys the session emits
is also how the mapper learns the shape — the driver *type* reaching it is just `Mqtt`, and the mode
lives on the driver config. A leaf whose attribute lookup returned nothing carries no address and is
refused **at commit, in words**, rather than committed as a tag that deploys clean and then reports
`BadNodeIdUnknown` forever.

### Request rebirth (Sparkplug only)

Births are never retained, so a healthy but quiet plant shows an **empty** picker tree until one
lands. The `Request rebirth` affordance in the browse modal is the way to fill it on demand: it
publishes a Sparkplug rebirth-request NCMD (`Node Control/Rebirth`) to the selected scope — the **one
and only** browse action that publishes anything.

- **Scope** is the last node clicked in the tree. A device or metric resolves **up to its owning edge
  node** (Sparkplug has no device-scoped rebirth); a group fans out to every observed edge node
  beneath it and is refused **whole** past 32, publishing nothing.
- **An empty tree can still arm a rebirth** ([#514](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/514),
  fixed `7f53bd96`): when no scope is selected the panel offers a `group/edgeNode` input, so a plant
  whose edge nodes all birthed before the window opened — the exact case the affordance exists for —
  is reachable without a tree click. Deliberately an **edge node**, never a group: a typed group
  broadcast is a self-inflicted broker storm on a large plant.
- **Two clicks, never one** — `Request rebirth…` arms a confirm panel naming the resolved scope and
  its consequence; changing the selection disarms it.
- **Gated on `DriverOperator`**, enforced server-side in `BrowserSessionService.RequestRebirthAsync`
  (fail-closed) before the session is touched; the UI gate is defence in depth. A refusal — policy,
  over-wide group, unresolvable scope — is **shown**, not swallowed.
- Offered only for a Sparkplug session (`IRebirthCapableBrowseSession.RebirthAvailable`); a Plain
  MQTT window publishes nothing, ever.

### Refresh

An observation window **accumulates**, but the tree renders **once**, when it is created — and
`OpenAsync` returns as soon as the SUBSCRIBE is granted, without waiting for the birth-observation
window. So the first render is normally empty on Plain (a topic that has not published yet is
invisible) and *almost always* empty on Sparkplug (births are never retained). **Refresh** re-reads
the root against the same session: nothing reconnects, nothing already observed is lost, and the tag
selection is kept. Only the armed rebirth scope is cleared, because the node it named may no longer be
in the rebuilt tree.

> **Also fixed by this gate, and not MQTT-specific:** every node label in the shared
> `DriverBrowseTree` was an `<a href="#">`. `blazor.web.js`'s enhanced-navigation click interceptor
> resolves a bare `#` against `<base href="/">`, so clicking a node **label** navigated the whole
> AdminUI to `/` and destroyed the hosting modal (`@onclick:preventDefault` suppresses the browser's
> default action, not Blazor's interceptor). The labels are now `<button type="button">`. This
> affected **every** driver's picker since 2026-05-28; it surfaced here because choosing a rebirth
> scope is the first flow that requires clicking a label rather than the ▶ toggle.

## Test Connect

`MqttDriverProbe` performs **one CONNECT** (no SUBSCRIBE, no publish) under a `-probe-` client id and
disconnects. Success message: `"MQTT CONNECT OK"`. A rejected CONNACK is described by the same
`DescribeConnackRejection` the running driver uses, so the button and the driver report a rejection in
identical words. See [TestConnectProbes.md](TestConnectProbes.md).

## Testing

- **Unit** — `tests/Drivers/…Driver.Mqtt.Tests` (**581 tests**): tag-config mapping (both shapes),
  JSONPath subset, strict-enum rejection, subscription indexing (incl. the wildcard-by-filter case),
  reconnect/rebuild branches, CONNACK classification, and the Sparkplug half — golden-payload decode,
  topic parse/format, datatype map, alias/birth-cache rebuild, sequence + bdSeq-death ties, the
  rediscovery change gate, and the browse session's "publishes nothing" assertion.
- **Live (env-gated)** — `tests/Drivers/…Driver.Mqtt.IntegrationTests` (**29 tests**) against the
  Mosquitto broker + the C# Sparkplug edge-node simulator; skips cleanly when `MQTT_FIXTURE_ENDPOINT`
  is unset. See [Mqtt-Test-Fixture.md](Mqtt-Test-Fixture.md) and `infra/README.md` §3.
  <br/>The primary-host legs (`MqttPrimaryHostLiveTests`) include the one the ordinary lifecycle cannot
  reach: `AbandonClaimForTests` drops the socket with **no** `DISCONNECT`, because a clean disconnect
  makes the broker *discard* the Will by protocol rule — so a test driven through `StopAsync` would
  observe the explicit stand-down `OFFLINE` and pass against an implementation that never armed a Will
  at all.
- **AdminUI** — `MqttDriverFormModelTests` / `MqttTagConfigModelTests` cover both editors'
  round-trip, mode inference and validation. AdminUI has no bUnit, so the razor shells themselves are
  only ever proven by a live `/run` gate.

## Operational Notes

- **No alarms, no driver-side history.** Historization is a server-side concern (`isHistorized`).
- **Both pair nodes subscribe independently** — MQTT fan-out is per-connection, and the Primary gate
  applies downstream, not to ingest. This is also why `clientId` must stay unset (above).
- **Unauthored traffic is ignored.** A busy broker is normal; the driver never auto-provisions a tag.

## Known gaps

Every row is tracked. None is a silent failure — each either refuses in words or logs.

**Closed 2026-07-31:** write-through ([#508](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/508), live-gated), primary-host STATE publishing
([#511](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/511), live-gated), structured
`Template`/`DataSet`/`PropertySet` metrics ([#515](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/515), live-gated), the slashed-metric browse-commit
([#509](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/509)), the stale rebirth affordance ([#510](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/510)), the `.Browser` layering exception
([#512](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/512)), the stray Sparkplug `payloadFormat` ([#513](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/513)) and rebirth on a
not-yet-birthed plant ([#514](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/514)).

| Gap | Issue | Effect | Notes |
|---|:---:|---|---|
| **Rediscovery is advisory, not a rebuild** | [#507](https://gitea.dohertylan.com/dohertj2/lmxopcua/issues/507) (closed, superseded) | A DBIRTH introducing a metric does not change the served OPC UA tree; an operator must re-browse the device and commit | **No longer "inert".** `DriverInstanceActor` now subscribes to `OnRediscoveryNeeded` and the raise rides the driver-health snapshot to the AdminUI `/hosts` page as a **re-browse chip**. It deliberately does not inject nodes: v3 authors raw tags through the `/raw` browse-commit flow, and grafting nodes at runtime would materialise ones nobody approved and no deployment artifact records. #507's injection path was deleted rather than revived |