Joseph Doherty
07a4a5ff5e
Task 14 — the P1 milestone gate. Ran the live /run verification on a docker-dev
rig against the real Mosquitto TLS+auth fixture, and recorded the result.
The gate did its job: it found MQTT was unauthorable in production.
RawDriverTypeDialog's driver-type list is a hand-maintained array that nobody
added MQTT to, so the /raw "New driver" picker never offered it — with every
other layer (contracts, driver, browser, factory, host registration, typed tag
editor) complete and 266 green unit tests. Nothing tied that array to
DriverTypeNames and AdminUI has no bUnit, so no offline test could see it.
Fixed, plus RawDriverTypeDialogParityTests: a reflection parity guard that
fails for ANY DriverTypeNames entry missing from the picker. Verified
falsifiable — RED with "…cannot be authored from the /raw New-driver picker,
so they are unreachable in production: Mqtt" before the one-line fix.
A second gap is left OPEN and documented, not fixed (no task ever covered it):
MqttDriverForm / MqttDeviceForm do not exist, and DriverConfigModal/DeviceModal
have no raw-JSON fallback — so an operator can create an MQTT driver but cannot
author its broker endpoint or credentials. P1 is not operator-complete until
those two razor forms land. The gate authored config via SQL to get past it.
Live gate result (isolated 2-node MAIN stack, image built from this branch,
fixture at 10.100.0.35:8883, AllowUntrustedServerCertificate rather than
pinning the CA into the rig):
- typed MQTT tag editor renders and round-trips; Json shows the JSON-path
field, Raw/Scalar hide it; wildcard topic a/+/c blocked at save; data-type
list offers Float64 and no Double; qos/retainSeed absent on "(driver
default)" and qos:2 written as a number; isHistorized/historianTagname
survive a topic-only edit; SparkplugB renders the P2 placeholder and
switching back keeps Plain values; jsonPath is guidance not a gate in all
three shapes ($ seeded only for a brand-new blank tag, blank stays absent,
clearing saves fine)
- deployed, and both editor-written blobs resolved and served changing live
values through OPC UA (22.8 / 1629, StatusCode Good) — no BadNodeIdUnknown
- the bespoke #-observation browser drove a real broker session and rendered
the fixture's actual topic tree (line1/{counter,state,temperature},
noise/chatter, retained/seed); a Modbus device's picker still correctly
reports "Browsing unavailable"
Also found live and documented (not a code change): both nodes of a redundant
pair run the driver, so a FIXED clientId makes them evict each other forever —
the broker logs "already connected, closing old connection" and both nodes
reconnect every ~2s while still reporting Healthy. Unset (the default) is
correct; confirmed 0 reconnects/60s on both nodes after removing it.
Suites: offline 1134 passed / 0 failed (266 Driver.Mqtt + 730 AdminUI incl. the
3 new guards + 138 Core.Abstractions); live 7/7 against the fixture.
Docs: new docs/drivers/Mqtt.md + Mqtt-Test-Fixture.md (the per-driver + fixture
convention every other driver follows), MQTT rows in docs/drivers/README.md,
TestConnectProbes.md, root README.md, and infra/README.md §3.
CLAUDE.md drift corrected while adding the MQTT endpoints:
- the "every fixture carries project: lmxopcua" claim was false — only the
MQTT fixture does; the filter returns one stack, not the fleet
- lmxopcua-fix.ps1 is Windows-VM-only; on macOS drive the host over ssh/rsync
(and the docker-dev rig itself runs locally under OrbStack)
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
187 lines
14 KiB
Markdown
187 lines
14 KiB
Markdown
# Driver-expansion — Waves 0–2 tracking
|
||
|
||
> **Living status tracker** for the driver-expansion program (Waves 0, 1, 2). One row per
|
||
> deliverable, each pointing at its **design doc**, its **implementation plan** (once written),
|
||
> and its current **status**. The authoritative *architecture* index is the program design doc;
|
||
> this file is the authoritative *progress* index.
|
||
>
|
||
> **Program design (architecture / shared contract / build order):**
|
||
> [`2026-07-15-driver-expansion-program-design.md`](2026-07-15-driver-expansion-program-design.md)
|
||
>
|
||
> Last updated: 2026-07-24.
|
||
|
||
## Legend
|
||
|
||
| Status | Meaning |
|
||
|---|---|
|
||
| ✅ **Done** | Merged to master, tests green. |
|
||
| 🟡 **Live gate open** | Code merged; a live `/run` or hardware-gated verification still outstanding. |
|
||
| 📝 **Plan ready** | Executable implementation plan (`*-implementation.md` + `.tasks.json`) written; not yet built. |
|
||
| 📐 **Design only** | Design doc exists; **no** implementation plan yet — writing-plans is the next step. |
|
||
| ⛔ **Not started** | No design, no plan. |
|
||
|
||
**Doc types** (per the writing-plans skill): a *design* states architecture, decisions, and risks;
|
||
an *implementation plan* is the bite-sized, TDD, file-path-level task list the subagent-driven
|
||
executor runs off, with a co-located `.tasks.json` for resume. A deliverable is only buildable once
|
||
it reaches 📝.
|
||
|
||
## Summary
|
||
|
||
| Wave | Deliverable | Design | Impl. plan | Status | Effort | Fixture / hardware |
|
||
|---|---|---|---|---|---|---|
|
||
| **0** | Universal Discover-backed browser | [design](2026-07-15-universal-discovery-browser-design.md) | [plan](2026-07-15-universal-discovery-browser-implementation.md) · [tasks](2026-07-15-universal-discovery-browser-implementation.md.tasks.json) | 🟡 **Live gate open** | S–M | none (retrofits shipped drivers) |
|
||
| **1** | Modbus RTU (extend Modbus) | [design](2026-07-15-modbus-rtu-driver-design.md) | [plan](2026-07-24-modbus-rtu-driver.md) · [tasks](2026-07-24-modbus-rtu-driver.md.tasks.json) | 📝 **Plan ready** (11 tasks) | **S (lowest)** | `pymodbus` `rtu_over_tcp` — CI-simulatable, no new infra |
|
||
| **1** | SQL poll | [design](2026-07-15-sql-poll-driver-design.md) | [plan](2026-07-24-sql-poll-driver.md) · [tasks](2026-07-24-sql-poll-driver.md.tasks.json) | 📝 **Plan ready** (22 tasks) | S–M | **existing central SQL Server** `10.100.0.35,14330` + SQLite unit |
|
||
| **2** | MTConnect Agent | [design](2026-07-15-mtconnect-driver-design.md) | [plan](2026-07-24-mtconnect-driver.md) · [tasks](2026-07-24-mtconnect-driver.md.tasks.json) | 📝 **Plan ready** (23 tasks) | S–M | Dockerized `mtconnect/cppagent` — CI-simulatable |
|
||
| **2** | MQTT / Sparkplug B | [design](2026-07-15-mqtt-sparkplug-driver-design.md) | [plan](2026-07-24-mqtt-sparkplug-driver.md) · [tasks](2026-07-24-mqtt-sparkplug-driver.md.tasks.json) | ✅ **P1 done** (Tasks 0–14, live-gated) · 📝 P2 Sparkplug plan ready (Tasks 15–26) | M–L | Mosquitto TLS+auth fixture **live** at `10.100.0.35:8883` |
|
||
|
||
> Waves 3+ (BACnet/IP, Omron) and the deferred MELSEC are tracked in the program design doc §6/§8,
|
||
> not here. Omron CIP is the program's sole real-hardware wire gate; BACnet's broadcast/BBMD leg is
|
||
> env-gated live. Both are out of this file's scope until they're pulled forward.
|
||
|
||
---
|
||
|
||
## Executing a plan (git worktree + subagent-per-task)
|
||
|
||
Each 📝 plan is executed with the **subagent-driven-development** skill: it sets up an isolated
|
||
**git worktree** first (via `using-git-worktrees`), then dispatches a **fresh subagent per task**,
|
||
running the classification-driven review chain (`trivial` = implement only … `high-risk` =
|
||
spec-review → code-review → integration review) between tasks. The `.tasks.json` next to each plan
|
||
tracks progress and lets a later session resume.
|
||
|
||
**Paste one of these into Claude Code to build a driver:**
|
||
|
||
| Deliverable | Command |
|
||
|---|---|
|
||
| Modbus RTU (Wave 1) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-modbus-rtu-driver.md in a new git worktree` |
|
||
| SQL poll (Wave 1) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-sql-poll-driver.md in a new git worktree` |
|
||
| MTConnect (Wave 2) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-mtconnect-driver.md in a new git worktree` |
|
||
| MQTT/Sparkplug (Wave 2) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-mqtt-sparkplug-driver.md in a new git worktree` |
|
||
|
||
- **Slash-command form** (equivalent): `/superpowers-extended-cc:subagent-driven-development <plan-path>`.
|
||
- **Parallel-session / resume form** (batch execution with checkpoints, no fresh-subagent-per-task):
|
||
`/superpowers-extended-cc:executing-plans <plan-path>` — reads the same `.tasks.json` and continues
|
||
from the first pending task.
|
||
- **Recommended order:** Modbus RTU → SQL poll → MTConnect → MQTT/Sparkplug (lowest effort first;
|
||
see each wave below for the gating notes). Run one plan per worktree; the four plans are
|
||
independent, so separate worktrees may run concurrently.
|
||
|
||
---
|
||
|
||
## Wave 0 — Universal Discover-backed browser 🟡
|
||
|
||
**What it is.** One generic `DiscoveryDriverBrowser` (+ `CapturingAddressSpaceBuilder`,
|
||
`CapturedTreeBrowseSession`, `BrowserSessionService` fallback, and the
|
||
`ITagDiscovery.SupportsOnlineDiscovery` gate) that turns any driver's `ITagDiscovery.DiscoverAsync`
|
||
into an AdminUI browse tree. It is the **Wave-0 gate**: every browsable new driver depends on this
|
||
seam, and it retrofits browse to the already-shipped AbCip / TwinCAT / FOCAS drivers for near-zero
|
||
marginal cost.
|
||
|
||
- **Design:** [`2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md)
|
||
- **Implementation plan:** [`2026-07-15-universal-discovery-browser-implementation.md`](2026-07-15-universal-discovery-browser-implementation.md) · [`.tasks.json`](2026-07-15-universal-discovery-browser-implementation.md.tasks.json)
|
||
- **Status: 🟡 code-complete + merged, live gate open.**
|
||
- Merged to master — `056887d6` (*Merge feat/universal-discovery-browser — Wave-0 universal Discover-backed browser*), 2026-07-15.
|
||
- Implementation plan: **19 / 19 tasks completed.**
|
||
- Lit up AbCip / TwinCAT / FOCAS pickers with zero per-driver browse code.
|
||
- **Outstanding:** the full tree-render live `/run` gate is **fixture-blocked** — tracked as **Gitea #468**. Complete this before leaning on browse in Wave 2/3.
|
||
|
||
---
|
||
|
||
## Wave 1 — low-effort / high-leverage pair 📝
|
||
|
||
Both are **fully CI-simulatable with zero new hardware**; Modbus RTU needs no new infra at all, and
|
||
SQL poll reuses the always-on central SQL Server. This is the recommended next build.
|
||
|
||
### Modbus RTU — 📝 Plan ready
|
||
- **Design:** [`2026-07-15-modbus-rtu-driver-design.md`](2026-07-15-modbus-rtu-driver-design.md)
|
||
- **Implementation plan:** [`2026-07-24-modbus-rtu-driver.md`](2026-07-24-modbus-rtu-driver.md) · [`.tasks.json`](2026-07-24-modbus-rtu-driver.md.tasks.json) — **11 tasks** (1 trivial / 5 small / 2 standard / 3 high-risk).
|
||
- **Scope:** extend the existing Modbus driver with a `Transport` selector (RTU CRC-16 + FC-aware
|
||
length, no TxId) + a `rtu_over_tcp` `pymodbus` docker profile + the AdminUI selector.
|
||
**Direct-serial transport is descoped** (user, 2026-07-15) — RTU-over-TCP via a serial→Ethernet
|
||
gateway is the only shipped mode, so there is no serial hardware gate.
|
||
- **Fixture:** add an `rtu_over_tcp` profile to the existing Modbus integration fixture. First P1
|
||
step is confirming the pymodbus simulator exposes the RTU framer on a TCP server.
|
||
- **Effort:** **S — the lowest on the roadmap.** Recommended first.
|
||
|
||
### SQL poll — 📝 Plan ready
|
||
- **Design:** [`2026-07-15-sql-poll-driver-design.md`](2026-07-15-sql-poll-driver-design.md)
|
||
- **Implementation plan:** [`2026-07-24-sql-poll-driver.md`](2026-07-24-sql-poll-driver.md) · [`.tasks.json`](2026-07-24-sql-poll-driver.md.tasks.json) — **22 tasks**. `ISqlDialect` seam in from day one; only the SQL Server dialect built in v1 (Postgres/ODBC deferred).
|
||
- **Scope:** a bespoke schema-browser driver polling a SQL table into equipment tags (SQL Server P1;
|
||
Postgres/ODBC land in P2/P3 behind `ISqlDialect`).
|
||
- **Fixture:** SQLite unit fixture (primary) + an **env-gated integration fixture against the
|
||
existing central SQL Server** (`10.100.0.35,14330`) with a seeded `SqlPollFixture` DB. The
|
||
blackhole/timeout live-gate pauses a **dedicated** `mssql` container — **never** the shared
|
||
central SQL Server (it hosts `ConfigDb`).
|
||
- **Effort:** S–M.
|
||
|
||
---
|
||
|
||
## Wave 2 — strategic telemetry / UNS pair 📝
|
||
|
||
Both CI-simulatable on the shared docker host, no hardware.
|
||
|
||
### MTConnect Agent — 📝 Plan ready
|
||
- **Design:** [`2026-07-15-mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md)
|
||
- **Implementation plan:** [`2026-07-24-mtconnect-driver.md`](2026-07-24-mtconnect-driver.md) · [`.tasks.json`](2026-07-24-mtconnect-driver.md.tasks.json) — **23 tasks**. Task 0 is the TrakHound-vs-hand-rolled client decision; browse-picker live-verify is gated on Wave-0 #468.
|
||
- **Scope:** P1 Agent MVP (`IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+probe+rediscover),
|
||
browse **free via the Wave-0 universal browser** (`SupportsOnlineDiscovery=true`, no browser code),
|
||
typed editor, `UNAVAILABLE→BadNoCommunication` mapping, ring-buffer re-baseline paging.
|
||
- **Fixture:** canned XML unit fixtures (bulk of coverage) + a dockerized `mtconnect/cppagent`
|
||
integration fixture (env-gated). Depends on Wave 0's browse seam being live-verified (#468).
|
||
- **Effort:** S–M (≈1–1.5 wk with TrakHound, ≈2.5–3 wk hand-rolled).
|
||
|
||
### MQTT / Sparkplug B — ✅ P1 done (live-gated) · 📝 P2 plan ready
|
||
- **Design:** [`2026-07-15-mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md)
|
||
- **Implementation plan:** [`2026-07-24-mqtt-sparkplug-driver.md`](2026-07-24-mqtt-sparkplug-driver.md) · [`.tasks.json`](2026-07-24-mqtt-sparkplug-driver.md.tasks.json) — **27 tasks** (P1 plain MQTT = Tasks 0–14, a complete shippable milestone; P2 Sparkplug B = Tasks 15–26). Task 0 is the mandatory MQTTnet-5/net10 + central-pinning validation spike.
|
||
- **Scope:** P1 plain MQTT (MQTTnet-5 connect/TLS/auth + hand-rolled reconnect, subscribe→
|
||
`OnDataChange`, retained last-value read, `#`-observation browser); P2 Sparkplug B ingest
|
||
(vendored Tahu proto, birth/alias/seq-gap/rebirth state machine, death→STALE).
|
||
- **Fixture:** Mosquitto TLS+auth broker + JSON publisher sidecar, **live** at `10.100.0.35:8883`
|
||
(`:1883` plaintext-but-authenticated); stack `/opt/otopcua-mqtt`, compose in
|
||
`tests/Drivers/….Driver.Mqtt.IntegrationTests/Docker/`. Env-gated live suite
|
||
(`MQTT_FIXTURE_ENDPOINT`, see `infra/README.md` §3). The **C# Sparkplug edge-node simulator** is P2
|
||
work and does not exist yet.
|
||
- **P1 status (2026-07-24):** Tasks 0–14 complete. `.Contracts`/`.Driver`/`.Browser` triad + factory,
|
||
`DriverTypeNames.Mqtt`, host factory/probe registration, AdminUI typed tag editor + bespoke
|
||
`#`-observation browser. **Offline 1131 passed / 0 failed** (266 driver · 727 AdminUI · 138
|
||
Core.Abstractions); **live 7/7** against the fixture.
|
||
- **P1 live gate found two AdminUI authoring gaps** (the gate's whole point — both were invisible to
|
||
266 green unit tests, because AdminUI has no bUnit and nothing tied these hand-maintained lists to
|
||
`DriverTypeNames`):
|
||
1. **FIXED —** `RawDriverTypeDialog`'s option array never got an MQTT row, so **no operator could
|
||
create an MQTT driver at all**. Fixed, plus a reflection parity guard
|
||
(`RawDriverTypeDialogParityTests`, 3 tests) that fails for *any* future `DriverTypeNames` entry
|
||
missing from the picker — so the next driver fails a test instead of a live gate.
|
||
2. **OPEN — `MqttDriverForm` / `MqttDeviceForm` do not exist.** `DriverConfigModal` and
|
||
`DeviceModal` dispatch on `DriverType` and fall through to *"No typed config form for driver
|
||
type Mqtt"* — and **neither modal has a raw-JSON fallback**, unlike the `/uns` TagModal. So an
|
||
operator can create an MQTT driver but cannot author its broker host/port/TLS/credentials
|
||
through the AdminUI at all. The P1 gate authored `DriverConfig`/`DeviceConfig` by direct SQL to
|
||
get past this. **P1 is not operator-complete until these two razor forms land** (a
|
||
`MqttDriverForm` + `MqttDeviceForm` pair modelled on the OpcUaClient ones, plus their
|
||
serialization tests). This is a plan omission, not a regression: no task ever covered the
|
||
driver/device forms — Task 6's "typed editor" was the *tag* editor.
|
||
- **Redundant-pair hazard worth knowing (documented, not a code change):** both nodes of a pair run
|
||
the driver, so a **fixed `clientId` makes them evict each other forever** — the broker logs
|
||
`already connected, closing old connection` and both nodes reconnect every ~2 s while still
|
||
reporting `Healthy`. Leaving `clientId` unset (the default) is correct. Observed live; captured in
|
||
`docs/drivers/Mqtt.md`. A future `MqttDriverForm` should not offer a free-text client id without
|
||
this warning.
|
||
- **Effort:** M–L. **Top risks (P2):** Sparkplug state-machine correctness; MQTTnet-5/net10 + the repo's
|
||
fragile central pinning (mitigated by hand-rolling Tahu over one MQTTnet-5 client) — the pinning risk
|
||
is **retired**: P1 shipped on MQTTnet 5 with the repo's central pinning intact.
|
||
|
||
---
|
||
|
||
## Next actions
|
||
|
||
1. **Close the Wave-0 live gate** (Gitea #468) — unblocks browse verification for MTConnect/BACnet.
|
||
2. **Build Wave 1** — Modbus RTU first (lowest effort, no new infra), then SQL poll. Both plans are
|
||
📝 ready; run the command from the [Executing a plan](#executing-a-plan-git-worktree--subagent-per-task)
|
||
table (subagent-driven-development in a git worktree).
|
||
3. **Build Wave 2** — MTConnect's browse leg wants #468 closed first. **MQTT P1 is done and
|
||
live-gated**; the remaining MQTT work is **P2 Sparkplug B** (Tasks 15–26), whose first step is the
|
||
vendored Tahu proto + the project-owned C# edge-node simulator the P1 fixture does not include.
|
||
|
||
Update this file's Summary table and per-wave status whenever a deliverable changes state.
|