Joseph Doherty
d5bd4226ee
Task 26 — the P2 milestone gate. Ran the live /run verification on an isolated
docker-dev rig (project otopcua-mqtt, ports 9210/4850-4851/14350, image built
from this branch) against the real Mosquitto TLS+auth broker and the C#
Sparkplug edge-node simulator on 10.100.0.35, and recorded the result.
The gate found three defects. Two are the same defect class the P1 gate found
twice — a hand-maintained AdminUI surface left behind by a driver-side feature —
and the third is a pre-existing cross-driver bug that only a Sparkplug flow
could surface.
1. CRITICAL, FIXED — MqttDriverForm still shipped its P1 Sparkplug PLACEHOLDER.
Switching Mode to SparkplugB rendered a "not available yet" notice and NO
Group ID field, so with Sparkplug ingest fully shipped there was still no
way to author a Sparkplug driver from the AdminUI at all. Sparkplug.GroupId
is the driver's entire subscription filter (spBv1.0/{GroupId}/#): blank ⇒
connected, Healthy, ingesting nothing. Now authors all five Sparkplug keys,
MERGES over the existing sub-object rather than replacing it (so a key a
newer driver adds inside it survives an older AdminUI), leaves it untouched
in Plain mode, and validates the group id as the topic segment it is.
Pinned by 8 MqttDriverFormModelTests cases, verified falsifiable — 8 RED
with the fix stubbed out.
2. PRE-EXISTING + CROSS-DRIVER, FIXED — every node label in the shared
DriverBrowseTree was an <a href="#">. In a Blazor Web App, blazor.web.js's
enhanced-navigation click interceptor resolves a bare "#" against
<base href="/">, so clicking ANY browse-tree node label navigated the whole
AdminUI to "/", tore down the circuit, and destroyed the hosting modal —
losing the browse session AND the tag selection. @onclick:preventDefault
does not help: it suppresses the browser's default action, not Blazor's own
interceptor. Labels are now <button type="button">. Dates to the component's
introduction (2026-05-28) and affects EVERY driver's picker; it surfaced
only now because choosing a rebirth scope is the first flow that requires
clicking a label rather than the ▶ toggle (always a button, always fine).
3. FIXED (mitigation) — the browse tree rendered exactly once, at open, and
never again, while OpenAsync returns as soon as the SUBSCRIBE is granted.
On Sparkplug that means it is almost always empty forever, because births
are never retained. Added a Refresh button that re-reads the root against
the SAME session (nothing reconnects, nothing observed is lost, the tag
selection is kept, only the armed rebirth scope is cleared).
Live gate result (all against the real broker + simulator, group OtOpcUaSim):
- driver authored end-to-end through the fixed MqttDriverForm; blob is
Mode:SparkplugB + Sparkplug.GroupId:OtOpcUaSim with enums as names
- browse: BROWSER OPEN chip, Sparkplug-only rebirth panel, tree renders
Group → EdgeNode → [Device] → Metric with the device folder Filler1 and
node-level metric leaves SIDE BY SIDE, and "Node Control/Rebirth" as ONE
leaf (the metric-name-is-not-a-topic-segment rule holding)
- rebirth: group scope names the group + the 32-node cap, Cancel published
NOTHING (simulator log unchanged); edge-node scope → "1 command published"
and the simulator logged "rebirth NCMD #1 received; republishing metadata";
group scope → "2 commands published", both edge nodes re-birthed; a device
and a metric both resolve UP to "edge node EdgeA"; the panel does NOT render
for a Plain MQTT device
- browse-commit wrote bindable tuples with no topic/address key —
{"groupId","edgeNodeId","metricName","deviceId"} for the device metric and
deviceId ABSENT (not blank) for the node-level one, datatypes inherited from
the birth (Int64 / Float)
- deployed (Sealed), and both nodes served live changing values through OPC UA
at the 2 s cadence — Good 0x00000000, no BadNodeIdUnknown
- death→STALE: stopping the simulator drove both nodes to 0x80000000 with an
empty value ~2 s later, and restarting it recovered them to Good on the new
birth (FillCount back to its birth-declared 1000, then counting)
- rediscovery fired exactly TWICE (once per authored scope: OtOpcUaSim/EdgeA
7 metrics, OtOpcUaSim/EdgeA/Filler1 3 metrics) and was then gated across
many subsequent births/rebirths — the anti-storm change gate working, and
EdgeB correctly filtered out as an unauthored scope
- tag editor: opens in SparkplugB (mode inference on reopen) with all four
descriptor fields populated; blank group → "A Sparkplug group ID is
required."; "Plant/1" rejected; "Node Control/Rebirth" ACCEPTED and saved
with the slash whole; dataType override persists and its key is ABSENT again
after selecting "(from birth certificate)"
Two things are NOT verified, and are recorded rather than claimed:
- Rediscovery is INERT in v3 and this is a platform gap, not an MQTT one:
nothing subscribes to IRediscoverable.OnRediscoveryNeeded and
DriverHostActor.HandleDiscoveredNodes hard-returns. A DBIRTH introducing a
metric does NOT change the OPC UA tree; redeploy. Verified from the driver's
log line only, exactly as far as it can be verified.
- A metric name containing "/" cannot be BROWSE-COMMITTED: the derived raw tag
Name is a RawPath segment and may not contain "/", so the spec-mandatory
"Node Control/Rebirth" is refused ("Row 3: Name must not contain '/'"). The
refusal is loud and all-or-nothing, never a silently mis-bound tag, and
Manual entry is the working path. Fixing it needs a name-sanitisation policy
with a collision answer, so it was recorded rather than guessed at.
Also left open + documented: an empty browse tree cannot arm a rebirth (the
scope comes from a tree click, and the session side already accepts a bare
{group}/{edgeNode} — only a UI path is missing); _canRebirth is captured at
browse-open; the tag editor injects the Plain-only payloadFormat key into a
Sparkplug blob (cosmetic — the factory routes on the tuple).
Suites: offline 1528 passed / 0 failed (581 Driver.Mqtt + 809 AdminUI + 138
Core.Abstractions); live 15/15 against the broker + simulator.
Docs: docs/drivers/Mqtt.md brought fully up to date for P2 (Sparkplug config
sub-object, both tag shapes, the ingest state machine, rediscovery, browse +
rebirth + Refresh, a Known-gaps table); Mqtt-Test-Fixture.md gains the simulator
and drops its "Sparkplug has no fixture" claims; infra/README.md §3 gains the
simulator row; CLAUDE.md gains the simulator endpoint facts; the tracking doc
marks MQTT/Sparkplug COMPLETE. docker-dev/docker-compose.mqtt.yml is the
isolated-rig overlay the gate ran on.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
134 lines
7.4 KiB
Markdown
134 lines
7.4 KiB
Markdown
# MQTT test fixture
|
|
|
|
Coverage map + gap inventory for the MQTT driver's harness.
|
|
|
|
**TL;DR:** the unit suite (581 tests) carries the mapping, indexing, Sparkplug state-machine and
|
|
failure-classification logic; the live suite (15 tests) proves the parts only a real broker can prove
|
|
— TLS, real authentication, CA pinning in both directions, retained-message seeding, that a rejected
|
|
CONNACK is actually surfaced, and the Sparkplug birth/alias/rebirth/death path against a real
|
|
edge-node simulator.
|
|
|
|
## What the fixture is
|
|
|
|
`eclipse-mosquitto:2.0.22` plus a second container of the same image running `publisher/publish.sh`
|
|
(the image ships `mosquitto_pub`, so no bespoke image is needed), plus — behind the `sparkplug`
|
|
compose profile — **`otopcua-sparkplug-sim`**, a project-owned C# Sparkplug-B edge-node simulator
|
|
(`tests/Drivers/….Driver.Mqtt.IntegrationTests/SparkplugSimulator/`). Compose lives at
|
|
`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/`; the deployed stack is
|
|
`/opt/otopcua-mqtt` on the shared Docker host.
|
|
|
|
**Two authenticated listeners, no anonymous fallback** (`allow_anonymous false`) — deliberate: an
|
|
anonymous broker would let a broken auth config pass every test.
|
|
|
|
| Listener | Endpoint | Env var |
|
|
|---|---|---|
|
|
| TLS + auth | `10.100.0.35:8883` | `MQTT_FIXTURE_ENDPOINT` |
|
|
| plaintext + auth | `10.100.0.35:1883` | `MQTT_FIXTURE_PLAIN_ENDPOINT` |
|
|
|
|
`MqttFixture` gates the suite on `MQTT_FIXTURE_ENDPOINT` / `MQTT_FIXTURE_USERNAME` /
|
|
`MQTT_FIXTURE_PASSWORD` (+ `MQTT_FIXTURE_CA_CERT`, `MQTT_FIXTURE_TOPIC_PREFIX`) and **skips cleanly**
|
|
when they are absent, so the suite is safe to run offline.
|
|
|
|
**Unlike every other fixture it needs generated material first** — `./gen-fixture-material.sh` writes
|
|
the gitignored `secrets/` (password file, CA, server cert/key). The broker will not start without it.
|
|
Bring-up and the CA-fetch recipe are in [`infra/README.md`](../../infra/README.md) §3.
|
|
|
|
### Published topics
|
|
|
|
| Topic | Shape | Retain | Why it exists |
|
|
|---|---|---|---|
|
|
| `otopcua/fixture/retained/seed` | `{"value":42.5,…}` | ✅ **published exactly once** | The only way to prove retained *seeding*. On a republished topic a retained seed is indistinguishable from a fast live publish |
|
|
| `otopcua/fixture/line1/temperature` | `{"value":…,"unit":"C","seq":n}` | ✅ | JSON + JSONPath extraction |
|
|
| `otopcua/fixture/line1/counter` | `{"value":n}` | ✅ | Monotonic, for ordering |
|
|
| `otopcua/fixture/line1/state` | `RUNNING` (bare text) | ✅ | Non-JSON `Raw`/`Scalar` payloads |
|
|
| `otopcua/fixture/noise/chatter` | JSON | ❌ | **Never authored as a tag** — a broker carrying unauthored traffic is the normal case and the driver must stay silent about it |
|
|
|
|
## What it actually covers
|
|
|
|
The 7 live tests: TLS+auth connect; connect pinned to the fixture CA; connect pinned to a *foreign*
|
|
CA is **rejected** (the falsifiability control — a pinning test that only ever passes proves nothing);
|
|
wrong password is rejected **and surfaced** (the MQTTnet-5 silent-`Healthy` defect); plain
|
|
subscribe→value under a RawPath; retained message seeds a value on subscribe; and an unauthored topic
|
|
stays silent while an authored tag keeps flowing.
|
|
|
|
### The Sparkplug simulator
|
|
|
|
Group **`OtOpcUaSim`**; edge nodes **`EdgeA`** and **`EdgeB`**; `EdgeA` additionally publishes device
|
|
**`Filler1`**. Node metrics `Temperature` (Float), `Pressure` (Double), `Count` (Int32), `Running`
|
|
(Boolean), `Serial` (String) plus the mandatory `bdSeq` and **`Node Control/Rebirth`** — the last one
|
|
deliberately carries a `/`, so it exercises the "a metric name is not a topic segment" rule. Device
|
|
metrics: `Temperature` (Float), `FillCount` (Int64), `Jammed` (Boolean). Cadence ~2 s.
|
|
|
|
It **subscribes to `spBv1.0/OtOpcUaSim/NCMD/{node}`** and re-births on a rebirth request, so it
|
|
exercises the real NCMD round trip rather than simulating it. Births are not retained (per spec), so
|
|
`docker restart otopcua-sparkplug-sim` is the way to force a fresh birth on demand:
|
|
|
|
```bash
|
|
ssh dohertj2@10.100.0.35 'cd /opt/otopcua-mqtt && docker compose --profile sparkplug up -d'
|
|
ssh dohertj2@10.100.0.35 'docker restart otopcua-sparkplug-sim' # force a birth
|
|
ssh dohertj2@10.100.0.35 'docker logs -f otopcua-sparkplug-sim'
|
|
```
|
|
|
|
Live-gated end-to-end (2026-07-24, P1 milestone): deployed on the docker-dev rig against this fixture,
|
|
both tags resolved and served changing values through OPC UA, and the AdminUI `#`-observation picker
|
|
rendered the real topic tree. Re-gated for P2 (2026-07-25) against the Sparkplug simulator — see
|
|
[Mqtt.md](Mqtt.md).
|
|
|
|
## What it does NOT cover
|
|
|
|
### 1. Sparkplug primary-host STATE, and write-through
|
|
The simulator never observes a Host Application `STATE`, because the driver never publishes one
|
|
(`ActAsPrimaryHost` is unimplemented). Nor is there any DCMD/NCMD **command** coverage: rebirth is the
|
|
only NCMD the driver ever sends, and `IWritable` is deferred.
|
|
|
|
Sparkplug **seq-gap injection** is unit-proven only — the simulator always publishes a well-formed
|
|
monotonic `seq`, so no live test drives the gap → rebirth path end-to-end.
|
|
|
|
### 2. Broker-side failure injection
|
|
No test kills the broker mid-session, partitions the network, or forces a SUBACK failure against a
|
|
real broker. Reconnect, backoff and per-topic degradation are **unit-proven only**.
|
|
|
|
### 3. Wildcard tags against a live broker
|
|
The by-filter indexing fix that keeps wildcard tags alive across a reconnect is covered by unit tests;
|
|
no live test authors a `+`/`#` tag and bounces the connection.
|
|
|
|
### 4. QoS 2 delivery semantics
|
|
The fixture publishes at QoS 0/1. QoS 2's exactly-once handshake is never exercised end-to-end.
|
|
|
|
### 5. Scale
|
|
One publisher, five topics, a ~2 s cadence. Nothing probes the 50 000-node browse cap, the 1 MiB
|
|
payload cap, or thousands of authored tags.
|
|
|
|
### 6. Redundant-pair client-id collision
|
|
Found by hand during the P1 gate (a fixed `clientId` makes pair nodes evict each other), **not** by a
|
|
test. No automated coverage asserts that two concurrent drivers coexist.
|
|
|
|
## When to trust the MQTT fixture, when to reach for a real broker
|
|
|
|
| Question | Fixture answers it? |
|
|
|---|---|
|
|
| Does TLS + auth work, and does bad auth fail loudly? | ✅ yes |
|
|
| Is a retained message seeded on subscribe? | ✅ yes |
|
|
| Does JSONPath extraction + coercion behave? | ✅ yes (unit + live) |
|
|
| Does the driver ignore unauthored traffic? | ✅ yes |
|
|
| Does it survive a broker restart / flapping link? | ❌ unit-only |
|
|
| Does it hold up at plant tag counts? | ❌ no |
|
|
| Does a Sparkplug birth → alias → DATA → death cycle work? | ✅ yes (live, vs. the simulator) |
|
|
| Does an NCMD rebirth request actually get answered? | ✅ yes (the simulator subscribes + re-births) |
|
|
| Sparkplug seq-gap → rebirth | ❌ unit-only (the sim never skips a seq) |
|
|
| Sparkplug primary-host STATE / DCMD writes | ❌ not implemented |
|
|
|
|
## Follow-up candidates
|
|
|
|
1. Broker-restart / SUBACK-failure live tests (closes gap 2).
|
|
2. A live wildcard-across-reconnect test (closes gap 3).
|
|
3. A concurrent-two-driver test pinning the client-id hazard (closes gap 6).
|
|
4. A seq-gap injection switch on the simulator, closing the last unit-only Sparkplug leg.
|
|
|
|
## Key fixture / config files
|
|
|
|
- `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Mqtt.IntegrationTests/Docker/docker-compose.yml`
|
|
- `tests/Drivers/…/Docker/mosquitto.conf` · `Docker/gen-fixture-material.sh` · `Docker/publisher/publish.sh`
|
|
- `tests/Drivers/…/MqttFixture.cs` · `PlainMqttLiveTests.cs`
|
|
- [`infra/README.md`](../../infra/README.md) §3 — bring-up, endpoints, CA fetch
|