feat(mqtt): read Sparkplug B array metrics as 1-D OPC UA arrays (lmxopcua#754)
v2-ci / unit-tests (push) Has been cancelled

R06:M-5 made Sparkplug *Array leaves unbindable because the ingestor refused
them. The ingestor now decodes the packed bytes_value of all 13 array types
(new SparkplugArrayCodec: LE fixed widths, int64-ms DateTime, uint32-count +
MSB-first BooleanArray, NUL-terminated UTF-8 StringArray; golden vectors are
the spec's examples, reproduced byte-for-byte by SparkplugNet 1.3.10) and
publishes a typed 1-D array through the scalar coercion, all-or-nothing.

Shape uses the fleet's one array model: the tag's platform isArray/arrayLength
keys (ValueRank=1, ArrayDimensions=[arrayLength] or [0] = variable length).
The driver reads them with TagConfigIntent's exact semantics and never rejects
on them, so composer, driver, authoring model and deploy inspector agree
(MqttArrayShapeParityTests). Shape mismatch (array metric on a scalar tag, or
the reverse) is BadTypeMismatch - a String tag no longer gets the packed
buffer as base64; a malformed body or a length differing from a declared
arrayLength is BadDecodingError (MTConnect #627 verdict). Template/DataSet
members of an array type take the same path. Arrays stay read-only (encoder
refuses them; MQTT is not in DriverArrayWriteSupport).

Browser: array leaves are bindable again and state isArray in AddressFields;
the commit mapper writes "isArray": true (no arrayLength). Discovery reports
IsArray/ArrayDim. Docs: Mqtt.md "Array metrics", design table, drivers rule.

Tests: codec (golden + malformed), ingest through the real codec + Dispatch
(the project's MiniBroker never forwards a PUBLISH, so there is no in-process
broker harness), browse, discovery, commit mapper, parity corpus. Mutating the
fix back out fails 18 MQTT + 9 AdminUI tests.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:58:38 -05:00
parent 919009c5ef
commit 7f29a75edb
19 changed files with 1112 additions and 53 deletions
+40 -6
View File
@@ -210,6 +210,7 @@ survive a reconnect; a message fans out to *every* matching tag, and unauthored
| `metricName` | string | — | **Required.** The tag's stable binding key across rebirths. **May contain `/`** — `Node Control/Rebirth` and `Properties/Serial Number` are canonical Sparkplug names, and the whole name is one value, never split. Also how a flattened structured member is addressed: `Motor/Speed`, `Recipe/Target/0` (see [Structured metrics](#structured-metrics-template--dataset--propertyset)) |
| `dataType` | `DriverDataType` | *from the birth* | **Optional override.** Absent ⇒ take whatever type the birth certificate declared. The AdminUI's "(from birth certificate)" option removes the key entirely |
| `isHistorized` / `historianTagname` | — | — | Server-side historization, unchanged from Plain |
| `isArray` / `arrayLength` | bool / uint | *absent* (scalar) | **Required `true` for a `*Array` metric** — see [Array metrics](#array-metrics-754). `arrayLength` is optional (absent ⇒ variable length) |
`payloadFormat`, `jsonPath`, `qos` and `retainSeed` are **Plain-only** and meaningless here: a
Sparkplug body is protobuf, and the subscription is one driver-wide `spBv1.0/{GroupId}/#`, not a
@@ -252,13 +253,45 @@ One subscription (`spBv1.0/{GroupId}/#`) feeds a per-edge-node state machine
[Structured metrics](#structured-metrics-template--dataset--propertyset).
- **Unsupported datatypes are skipped with a warning, never guessed**: `PropertySetList`, `Unknown`,
extension values, and any `Template`/`DataSet` that cannot be flattened honestly. `Int8`→`Int16` and
`UInt8`→`UInt16` widen; `*Array` variants map to the element type with the array bit set — but
**array metrics are not read in v1** (warned once, never fed; → #754), so the browse picker offers them, and any
metric whose type is unsupported, **without an address**: the browse-commit refuses them rather than
committing a tag that waits for initial data forever (R06:M-5).
`UInt8`→`UInt16` widen; `*Array` variants map to the element type with the array bit set and are read as
**1-D OPC UA arrays** (see [Array metrics](#array-metrics-754)). The browse picker offers a metric whose type
is unsupported **without an address**: the browse-commit refuses it rather than committing a tag that waits
for initial data forever (R06:M-5).
- **Ingest state is reset at every session/authoring seam** — a reconnect or a `ReinitializeAsync`
must not carry a previous session's alias table into a new one.
### Array metrics (#754)
The 13 Sparkplug `*Array` datatypes (`Int8Array`…`UInt64Array`, `FloatArray`, `DoubleArray`, `BooleanArray`,
`StringArray`, `DateTimeArray`) are decoded from the metric's packed `bytes_value` into a **1-D array** of the
tag's element type (`SparkplugArrayCodec`), using the fleet's one array model: the tag carries the platform
`"isArray": true` key, so the node is `ValueRank = 1` with the element `DataType`, and `ArrayDimensions` is
`[arrayLength]` when one is declared or `[0]` (variable length) when not — a Sparkplug array's length can change
from one DATA message to the next, so the browse-commit declares none.
- **Wire format** (Sparkplug B 3.0): little-endian, no header — 1/2/4/8 bytes per element; `DateTimeArray` is
int64 epoch ms; `BooleanArray` is a uint32 LE count then the bits packed **most-significant-bit first**;
`StringArray` is NUL-terminated UTF-8. The tests pin the spec's own examples, which SparkplugNet reproduces.
- **Element type:** the authored `dataType` when present, else the birth's element type (`Int8`/`UInt8` widen to
`Int16`/`UInt16` as for scalars). Every element goes through the scalar coercion, **all-or-nothing**.
- **Shape must match.** An array metric on a tag without `isArray`, or a scalar metric on an `isArray` tag, is
`BadTypeMismatch` (warned once, naming the fix) — never the packed buffer, which a String tag used to receive
as base64 at Good.
- **Refusals:** a malformed body (not a whole number of elements, a Boolean count the body cannot hold,
an unterminated string, invalid UTF-8) or a vector whose length differs from a declared `arrayLength` is
`BadDecodingError` (MTConnect's TIME_SERIES verdict, #627); an element that does not fit the effective type,
an explicit null, or a non-bytes value is `BadTypeMismatch`.
- **Read-only.** Array writes are not encoded (`MqttWriteEncoder` refuses `*Array` births) and MQTT is not in
`DriverArrayWriteSupport`, so the node is materialised read-only even on a `ReadWrite` tag.
- **The array keys are read exactly as the composer reads them** (`isArray` only as the literal `true`;
`arrayLength` only as a uint number) and are never a rejection cause, so authoring, deploy gate, node shape
and driver agree (`MqttArrayShapeParityTests`). Template/DataSet members of an array type decode the same way.
```jsonc
// A variable-length array metric (what a browse-commit writes).
{"groupId":"OtOpcUaSim","edgeNodeId":"EdgeA","deviceId":"Filler1","metricName":"Profile","isArray":true}
```
### Rediscovery (Sparkplug)
`OnRediscoveryNeeded` fires only when a birth is **for an authored scope** *and* its metric-name set
@@ -595,8 +628,9 @@ The bespoke browser wins over the universal `DiscoveryDriverBrowser`, which woul
A committed leaf's address is a **descriptor**, not a single reference string, so
`RawBrowseCommitMapper` reads it from the `AttributeInfo.AddressFields` the session states — Plain
emits `topic`; Sparkplug emits `groupId` / `edgeNodeId` / `deviceId?` / `metricName`, the names
`MqttTagConfigKeys` single-sources for both the mapper and `MqttTagDefinitionFactory`.
emits `topic`; Sparkplug emits `groupId` / `edgeNodeId` / `deviceId?` / `metricName` — plus `isArray` for a
`*Array` metric, written as the JSON literal `true` (#754) — the names `MqttTagConfigKeys` single-sources for
both the mapper and `MqttTagDefinitionFactory`.
**It is never parsed out of the browse node id.** A Sparkplug node id is
`{group}/{node}[/{device}]::{metric}` and a metric name legitimately contains `/`
@@ -148,7 +148,7 @@ Sparkplug metric datatype (or inferred/authored plain type) → `DriverDataType`
| DateTime | `DateTime` | DateTime |
| Bytes, File | `String` (base64/raw fallback, v1) | String |
| DataSet, Template | **unsupported v1** — skip + warn, or expose raw-JSON as `String` | — |
| `*Array` variants | element type + `IsArray=true`, `ArrayDim` from birth | ValueRank=1 |
| `*Array` variants | element type + `IsArray=true`; ArrayDimensions from the tag's `arrayLength` (absent ⇒ variable length, `[0]`) — implemented #754, see `docs/drivers/Mqtt.md` §"Array metrics" | ValueRank=1 |
Live map in `SparkplugDataType.ToDriverDataType()` (`.Contracts`). **Plain mode:** honour an explicit per-tag `dataType` (preferred — inference is brittle); fall back to inferring from the JSON token at `jsonPath` (number→`Float64`/`Int64`, bool→`Boolean`, string→`String`). Int8/UInt8 widen (no OPC UA signed-byte distinction needed).