Files
lmxopcua/docs/plans/2026-07-24-driver-expansion-tracking.md
T
Joseph Doherty 71ccccef7c docs(mtconnect): P1 driver guide + tracking-doc update, defer write-back (Task 22)
- docs/drivers/MTConnect.md: getting-started guide for the P1 Agent MVP —
  config keys read off MTConnectDriverOptions/ConfigDto, capability surface,
  RawPath->dataItemId coercion precedence, UNAVAILABLE->BadNoCommunication +
  the rest of the quality-code table, CONDITION-as-String, browse-via-the-
  universal-browser, the fixture recipe (links the existing IntegrationTests
  Docker README instead of duplicating it), and a Known limitations section
  covering the two pre-existing fleet-wide gaps this build surfaced
  (IRediscoverable/IHostConnectivityProbe have no server consumer; no driver
  form blocks Save on validation). Records write-back (MTConnect Interfaces),
  P1.5 (native alarms, TIME_SERIES arrays, EVENT->enum), and P2 (SHDR ingest)
  as deferred per design doc SS3.6/SS9. Documents where the build diverged
  from the design: TrakHound MTConnect.NET was dropped entirely (no XML
  formatter in the pinned packages, no injectable HTTP handler) in favor of
  a hand-rolled System.Xml.Linq client, namespace-agnostic on LocalName.
- docs/drivers/README.md: adds MTConnect to the ground-truth driver table,
  the per-driver doc list, and the fixture coverage-map list.
- docs/plans/2026-07-24-driver-expansion-tracking.md: marks the MTConnect
  Wave-2 row Done (22/23 tasks; Task 21 live gate tracked separately in the
  plan file, not touched here), corrects mtconnect/cppagent -> mtconnect/agent
  in the fixture note, and links the new driver guide.

Deferred-cleanup decision (Task 1 review follow-up): removed
GenerateDocumentationFile/NoWarn CS1591 from Driver.MTConnect.Contracts's
csproj to match all eight sibling .Contracts projects, none of which carry
it. Verified both the Contracts project alone and the full solution still
build clean (0 warnings, 0 errors) with the flags gone — the existing XML
doc comments don't depend on GenerateDocumentationFile to compile.
2026-07-27 13:59:26 -04:00

252 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Driver-expansion — Waves 02 tracking
> **Living status tracker** for the driver-expansion program (Waves 0, 1, 2). One row per
> deliverable, each pointing at its **design doc**, its **implementation plan** (once written),
> and its current **status**. The authoritative *architecture* index is the program design doc;
> this file is the authoritative *progress* index.
>
> **Program design (architecture / shared contract / build order):**
> [`2026-07-15-driver-expansion-program-design.md`](2026-07-15-driver-expansion-program-design.md)
>
> Last updated: 2026-07-24.
## Legend
| Status | Meaning |
|---|---|
| ✅ **Done** | Merged to master, tests green. |
| 🟡 **Live gate open** | Code merged; a live `/run` or hardware-gated verification still outstanding. |
| 📝 **Plan ready** | Executable implementation plan (`*-implementation.md` + `.tasks.json`) written; not yet built. |
| 📐 **Design only** | Design doc exists; **no** implementation plan yet — writing-plans is the next step. |
| ⛔ **Not started** | No design, no plan. |
**Doc types** (per the writing-plans skill): a *design* states architecture, decisions, and risks;
an *implementation plan* is the bite-sized, TDD, file-path-level task list the subagent-driven
executor runs off, with a co-located `.tasks.json` for resume. A deliverable is only buildable once
it reaches 📝.
## Summary
| Wave | Deliverable | Design | Impl. plan | Status | Effort | Fixture / hardware |
|---|---|---|---|---|---|---|
| **0** | Universal Discover-backed browser | [design](2026-07-15-universal-discovery-browser-design.md) | [plan](2026-07-15-universal-discovery-browser-implementation.md) · [tasks](2026-07-15-universal-discovery-browser-implementation.md.tasks.json) | 🟡 **Live gate open** | SM | none (retrofits shipped drivers) |
| **1** | Modbus RTU (extend Modbus) | [design](2026-07-15-modbus-rtu-driver-design.md) | [plan](2026-07-24-modbus-rtu-driver.md) · [tasks](2026-07-24-modbus-rtu-driver.md.tasks.json) | 🟢 **Built — live gate PASSED** (11 tasks, branch `feat/modbus-rtu-driver`, pending merge) | **S (lowest)** | `pymodbus` `rtu_over_tcp` — CI-simulatable, no new infra |
| **1** | SQL poll | [design](2026-07-15-sql-poll-driver-design.md) | [plan](2026-07-24-sql-poll-driver.md) · [tasks](2026-07-24-sql-poll-driver.md.tasks.json) | 📝 **Plan ready** (22 tasks) | SM | **existing central SQL Server** `10.100.0.35,14330` + SQLite unit |
| **2** | MTConnect Agent | [design](2026-07-15-mtconnect-driver-design.md) | [plan](2026-07-24-mtconnect-driver.md) · [tasks](2026-07-24-mtconnect-driver.md.tasks.json) | ✅ **COMPLETE** — P1 Agent MVP, all 23 tasks + the 5-leg live gate PASSED (branch `feat/mtconnect-driver`, PR #506) | SM | Dockerized `mtconnect/agent:2.7.0.12` (not `cppagent`) — CI-simulatable, live at `10.100.0.35:5000` |
| **2** | MQTT / Sparkplug B | [design](2026-07-15-mqtt-sparkplug-driver-design.md) | [plan](2026-07-24-mqtt-sparkplug-driver.md) · [tasks](2026-07-24-mqtt-sparkplug-driver.md.tasks.json) | ✅ **COMPLETE** — P1 + P2 shipped, both live-gated (Tasks 026) | ML | Mosquitto TLS+auth **+ C# Sparkplug edge-node simulator**, live at `10.100.0.35:8883` |
> Waves 3+ (BACnet/IP, Omron) and the deferred MELSEC are tracked in the program design doc §6/§8,
> not here. Omron CIP is the program's sole real-hardware wire gate; BACnet's broadcast/BBMD leg is
> env-gated live. Both are out of this file's scope until they're pulled forward.
---
## Executing a plan (git worktree + subagent-per-task)
Each 📝 plan is executed with the **subagent-driven-development** skill: it sets up an isolated
**git worktree** first (via `using-git-worktrees`), then dispatches a **fresh subagent per task**,
running the classification-driven review chain (`trivial` = implement only … `high-risk` =
spec-review → code-review → integration review) between tasks. The `.tasks.json` next to each plan
tracks progress and lets a later session resume.
**Paste one of these into Claude Code to build a driver:**
| Deliverable | Command |
|---|---|
| Modbus RTU (Wave 1) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-modbus-rtu-driver.md in a new git worktree` |
| SQL poll (Wave 1) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-sql-poll-driver.md in a new git worktree` |
| MTConnect (Wave 2) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-mtconnect-driver.md in a new git worktree` |
| MQTT/Sparkplug (Wave 2) | `Use superpowers-extended-cc:subagent-driven-development to execute docs/plans/2026-07-24-mqtt-sparkplug-driver.md in a new git worktree` |
- **Slash-command form** (equivalent): `/superpowers-extended-cc:subagent-driven-development <plan-path>`.
- **Parallel-session / resume form** (batch execution with checkpoints, no fresh-subagent-per-task):
`/superpowers-extended-cc:executing-plans <plan-path>` — reads the same `.tasks.json` and continues
from the first pending task.
- **Recommended order:** Modbus RTU → SQL poll → MTConnect → MQTT/Sparkplug (lowest effort first;
see each wave below for the gating notes). Run one plan per worktree; the four plans are
independent, so separate worktrees may run concurrently.
---
## Wave 0 — Universal Discover-backed browser 🟡
**What it is.** One generic `DiscoveryDriverBrowser` (+ `CapturingAddressSpaceBuilder`,
`CapturedTreeBrowseSession`, `BrowserSessionService` fallback, and the
`ITagDiscovery.SupportsOnlineDiscovery` gate) that turns any driver's `ITagDiscovery.DiscoverAsync`
into an AdminUI browse tree. It is the **Wave-0 gate**: every browsable new driver depends on this
seam, and it retrofits browse to the already-shipped AbCip / TwinCAT / FOCAS drivers for near-zero
marginal cost.
- **Design:** [`2026-07-15-universal-discovery-browser-design.md`](2026-07-15-universal-discovery-browser-design.md)
- **Implementation plan:** [`2026-07-15-universal-discovery-browser-implementation.md`](2026-07-15-universal-discovery-browser-implementation.md) · [`.tasks.json`](2026-07-15-universal-discovery-browser-implementation.md.tasks.json)
- **Status: 🟡 code-complete + merged, live gate open.**
- Merged to master — `056887d6` (*Merge feat/universal-discovery-browser — Wave-0 universal Discover-backed browser*), 2026-07-15.
- Implementation plan: **19 / 19 tasks completed.**
- Lit up AbCip / TwinCAT / FOCAS pickers with zero per-driver browse code.
- **Outstanding:** the full tree-render live `/run` gate is **fixture-blocked** — tracked as **Gitea #468**. Complete this before leaning on browse in Wave 2/3.
---
## Wave 1 — low-effort / high-leverage pair 📝
Both are **fully CI-simulatable with zero new hardware**; Modbus RTU needs no new infra at all, and
SQL poll reuses the always-on central SQL Server. This is the recommended next build.
### Modbus RTU — 🟢 Built, live gate PASSED (branch `feat/modbus-rtu-driver`, pending merge)
- **Design:** [`2026-07-15-modbus-rtu-driver-design.md`](2026-07-15-modbus-rtu-driver-design.md)
- **Implementation plan:** [`2026-07-24-modbus-rtu-driver.md`](2026-07-24-modbus-rtu-driver.md) · [`.tasks.json`](2026-07-24-modbus-rtu-driver.md.tasks.json) — **11 tasks** (1 trivial / 5 small / 2 standard / 3 high-risk), all complete via subagent-driven-development.
- **Scope:** extend the existing Modbus driver with a `Transport` selector (RTU CRC-16 + FC-aware
length, no TxId) + a `rtu_over_tcp` `pymodbus` docker profile + the AdminUI selector.
**Direct-serial transport is descoped** (user, 2026-07-15) — RTU-over-TCP via a serial→Ethernet
gateway is the only shipped mode, so there is no serial hardware gate.
- **Fixture:** added the `rtu_over_tcp` profile (`framer=rtu`, host `:5021`, co-runs with `standard`)
to the Modbus integration fixture. **Unknown confirmed:** pymodbus 3.13's simulator DOES honour
`framer=rtu` on a TCP server (wire-probed a pure RTU FC03 response, no MBAP) — so the simple profile
path was taken; **no stdlib fallback server was needed.**
- **Verification (all green):** unit suite **344/344**; full solution build 0 errors; the 2 RTU
integration tests pass **live** against the real `framer=rtu` fixture (read HR5→5, write+readback
HR200←4242). Per-task review chain (spec + code reviewers) + a final whole-branch review, all
approved-to-merge.
- **Live `/run` gate — PASSED (2026-07-24, docker-dev rebuilt from this branch):**
- AdminUI `/raw` Modbus **driver** config modal renders the new **Transport** selector (with the
operator hint) — set to `RtuOverTcp`, saved.
- **Device Test Connect → `OK · 23 MS`** — the probe built a `ModbusRtuOverTcpTransport` via the
factory and spoke **FC03 over real RTU framing** to `10.100.0.35:5021`.
- Authored raw tags HR5 (r/o) + HR200 (r/w) → **deployment `06ec1632` Sealed** (all 6 nodes acked).
- Client.CLI on `opc.tcp://localhost:4840` (`multi-role`/`password`): **read** `ns=2;s=rtu-gate/Device1/HR5`
**5, Good**; **write** `ns=2;s=rtu-gate/Device1/HR200` = 4242 → readback **4242, Good**.
- **Effort:** **S — the lowest on the roadmap.** Delivered first, as recommended.
### SQL poll — 📝 Plan ready
- **Design:** [`2026-07-15-sql-poll-driver-design.md`](2026-07-15-sql-poll-driver-design.md)
- **Implementation plan:** [`2026-07-24-sql-poll-driver.md`](2026-07-24-sql-poll-driver.md) · [`.tasks.json`](2026-07-24-sql-poll-driver.md.tasks.json) — **22 tasks**. `ISqlDialect` seam in from day one; only the SQL Server dialect built in v1 (Postgres/ODBC deferred).
- **Scope:** a bespoke schema-browser driver polling a SQL table into equipment tags (SQL Server P1;
Postgres/ODBC land in P2/P3 behind `ISqlDialect`).
- **Fixture:** SQLite unit fixture (primary) + an **env-gated integration fixture against the
existing central SQL Server** (`10.100.0.35,14330`) with a seeded `SqlPollFixture` DB. The
blackhole/timeout live-gate pauses a **dedicated** `mssql` container — **never** the shared
central SQL Server (it hosts `ConfigDb`).
- **Effort:** SM.
---
## Wave 2 — strategic telemetry / UNS pair 📝
Both CI-simulatable on the shared docker host, no hardware.
### MTConnect Agent — ✅ Done (P1 Agent MVP)
- **Design:** [`2026-07-15-mtconnect-driver-design.md`](2026-07-15-mtconnect-driver-design.md)
- **Implementation plan:** [`2026-07-24-mtconnect-driver.md`](2026-07-24-mtconnect-driver.md) · [`.tasks.json`](2026-07-24-mtconnect-driver.md.tasks.json) — **23 tasks; 22 built, Task 21 (live `/run` gate) tracked separately in the plan file.**
- **Driver guide:** [`docs/drivers/MTConnect.md`](../drivers/MTConnect.md) — config keys, capability
surface, data-plane precedence rules, quality-code mapping, and (most load-bearing) the **Known
limitations** section.
- **Scope shipped:** `IDriver`+`ITagDiscovery`+`IReadable`+`ISubscribable`+`IHostConnectivityProbe`+`IRediscoverable`
(deliberately no `IWritable` — read-only Agent surface), browse **free via the Wave-0 universal
browser** (`SupportsOnlineDiscovery=true`, no browser project), typed AdminUI tag editor,
`UNAVAILABLE→BadNoCommunication` mapping, CONDITION-as-`String`, ring-buffer re-baseline paging
(both the sequence-gap and `OUT_OF_RANGE` legs).
- **Diverged from the design:** Task 0/6/7 dropped the planned TrakHound `MTConnect.NET-Common`/`-HTTP`
dependency entirely — the pinned packages ship no XML formatter and the HTTP stream client has no
injectable handler to test behind the driver's seam. Parsing is hand-rolled `System.Xml.Linq`,
matching on `LocalName` only (namespace-agnostic, so 1.32.x all parse). See the driver guide's
"Built vs. planned" section.
- **Test results:** MTConnect unit suite 491/491; integration suite 12/12 against a real Agent
(`mtconnect/agent:2.7.0.12`, **not** `mtconnect/cppagent` — that Docker Hub repo doesn't exist);
skips cleanly (12 Skipped) offline. AdminUI 749/749; full solution 0 build errors.
- **Deferred (documented, not built):** write-back via MTConnect Interfaces (design §3.6); P1.5
CONDITION→native Part-9 alarms, `TIME_SERIES` array materialization, EVENT→enum vocab (design §9);
P2 SHDR adapter ingest (design §9). Also two pre-existing fleet-wide gaps this build surfaced
rather than caused: `IRediscoverable`/`IHostConnectivityProbe` have no server-side consumer (eight
drivers besides Galaxy), and no driver form blocks Save on a required-field validation error.
- **Fixture:** canned XML unit fixtures (bulk of coverage) + a dockerized `mtconnect/agent` +
stdlib-SHDR-adapter integration fixture (env-gated), deployed at `/opt/otopcua-mtconnect/` on the
shared docker host. See the fixture's own
[`Docker/README.md`](../../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/README.md).
- **Effort:** SM, hand-rolled (the TrakHound estimate in the design doc did not apply once that
path was ruled out).
### MQTT / Sparkplug B — ✅ **COMPLETE** (P1 + P2, both live-gated)
- **Design:** [`2026-07-15-mqtt-sparkplug-driver-design.md`](2026-07-15-mqtt-sparkplug-driver-design.md)
- **Implementation plan:** [`2026-07-24-mqtt-sparkplug-driver.md`](2026-07-24-mqtt-sparkplug-driver.md) · [`.tasks.json`](2026-07-24-mqtt-sparkplug-driver.md.tasks.json) — **27 tasks, all done** (P1 plain MQTT = Tasks 014; P2 Sparkplug B = Tasks 1526).
- **Scope delivered:** P1 plain MQTT (MQTTnet-5 connect/TLS/auth + hand-rolled reconnect, subscribe→
`OnDataChange`, retained last-value read, `#`-observation browser); P2 Sparkplug B ingest
(vendored Tahu proto + `Grpc.Tools` codegen, birth/alias/seq-gap/rebirth state machine,
death→STALE, birth-driven browse tree, scoped Request-rebirth NCMD, typed tag editor).
**Write-through (`IWritable`) is deferred to P3** — every MQTT node materializes read-only.
- **Fixture:** Mosquitto TLS+auth broker + JSON publisher sidecar, **live** at `10.100.0.35:8883`
(`:1883` plaintext-but-authenticated); stack `/opt/otopcua-mqtt`, compose in
`tests/Drivers/…​.Driver.Mqtt.IntegrationTests/Docker/`. Env-gated live suite
(`MQTT_FIXTURE_ENDPOINT`, see `infra/README.md` §3). The **project-owned C# Sparkplug edge-node
simulator** (`--profile sparkplug`, group `OtOpcUaSim`, nodes `EdgeA`/`EdgeB`, `EdgeA` device
`Filler1`) shipped with P2 and answers rebirth NCMDs.
- **P2 status (2026-07-25):** Tasks 1526 complete. **Offline 1528 passed / 0 failed**
(581 driver · 809 AdminUI · 138 Core.Abstractions); **live 15/15** against the broker + simulator.
- **P1 live gate found two AdminUI authoring gaps** (the gate's whole point — both invisible to green
unit tests, because AdminUI has no bUnit and nothing tied these hand-maintained lists to
`DriverTypeNames`):
1. **FIXED —** `RawDriverTypeDialog`'s option array never got an MQTT row, so **no operator could
create an MQTT driver at all**. Fixed, plus a reflection parity guard
(`RawDriverTypeDialogParityTests`) that fails for *any* future `DriverTypeNames` entry
missing from the picker.
2. **FIXED —** `MqttDriverForm` / `MqttDeviceForm` did not exist, so the broker connection was
unauthorable from the AdminUI. Both shipped after the P1 gate.
- **P2 live gate found a third one, of the same family — FIXED:** `MqttDriverForm` still carried its
P1 Sparkplug **placeholder**. Switching Mode to `SparkplugB` rendered a *"not available yet"* notice
and **no Group ID field** — so once Sparkplug ingest shipped there was still **no way to author a
Sparkplug driver from the AdminUI**, and `Sparkplug.GroupId` is the driver's entire subscription
filter (`spBv1.0/{GroupId}/#`): blank ⇒ connected, `Healthy`, ingesting nothing. Fixed (all five
Sparkplug keys, merge-not-replace on save, group-id segment validation) + 8 falsifiable
`MqttDriverFormModelTests` cases. **Three gates, three instances of the same defect class: a
hand-maintained AdminUI surface left behind by a driver-side feature.**
- **P2 live gate — two open UI gaps, documented not fixed** (see `docs/drivers/Mqtt.md` §Known gaps):
- **A completely empty browse tree cannot arm a rebirth.** The scope comes from a tree click, so on
a plant that has not birthed since the window opened `Request rebirth…` stays disabled — the very
case the affordance exists for. `MqttBrowseSession.ResolveRebirthTargets` already accepts a bare
`{group}/{edgeNode}` ("the prime rebirth target"); only a UI path to enter one is missing.
Mitigation shipped: a **Refresh** button on the browse tree (it re-reads the same session, which
previously rendered exactly once at open and never again).
- **P2 live gate also found a PRE-EXISTING, cross-driver AdminUI defect — FIXED:** every node label in
the shared `DriverBrowseTree` was an `<a href="#">`. In a Blazor Web App, `blazor.web.js`'s
enhanced-navigation click interceptor resolves a bare `#` against `<base href="/">`, so **clicking
any browse-tree node label navigated the whole AdminUI to `/`**, tore down the circuit and destroyed
the hosting modal — losing the browse session and the tag selection. `@onclick:preventDefault` does
not help: it suppresses the browser's default action, not Blazor's own interceptor. The labels are
now `<button type="button">`. It dates to the component's introduction (2026-05-28) and affects
**every** driver's picker; it only surfaced now because Sparkplug's Request-rebirth is the first
feature that requires clicking a *label* rather than the ▶ toggle (always a `<button>`, always fine).
- **A metric name containing `/` cannot be browse-committed.** The derived raw **tag Name** is a
RawPath segment and may not contain `/`, while `metricName` legitimately may — so the
spec-mandatory `Node Control/Rebirth` is refused at commit (`Row N: Name must not contain '/'`).
The refusal is loud and all-or-nothing, never a silently mis-bound tag, and **Manual entry** is
the working path (the Sparkplug tag editor accepts a slashed metric name). Fixing it needs a
name-sanitisation policy with a collision answer, so it was recorded rather than guessed at.
- `_canRebirth` is captured at browse-open, so a reaped session still draws the button.
- The tag editor injects the Plain-only `payloadFormat` key into a Sparkplug blob — cosmetic; the
factory routes on the Sparkplug tuple and ignores it.
- **Rediscovery is inert in v3 (platform gap, not MQTT's).** The Sparkplug driver raises
`OnRediscoveryNeeded` on a changed birth metric-set, but **nothing subscribes to it** and
`DriverHostActor.HandleDiscoveredNodes` hard-returns. A DBIRTH introducing a metric does **not**
change the OPC UA tree — redeploy. Driver-side behaviour is log-observable only.
- **Redundant-pair hazard (documented, not a code change):** both nodes of a pair run the driver, so a
**fixed `clientId` makes them evict each other forever** — the broker logs
`already connected, closing old connection` and both nodes reconnect every ~2 s while still
reporting `Healthy`. Unset (the default) is correct. `MqttDriverForm` warns inline the moment a
value is typed.
- **Effort:** ML, delivered. The MQTTnet-5/net10 + central-pinning risk is **retired** (Task 0's
spike held through both phases); Sparkplug state-machine correctness was carried by golden-payload
vectors + the live simulator.
---
## Next actions
1. **Close the Wave-0 live gate** (Gitea #468) — unblocks browse verification for BACnet (MTConnect
shipped its own Task-21 live `/run` gate independently — see the plan file).
2. **Build Wave 1** — Modbus RTU first (lowest effort, no new infra), then SQL poll. Both plans are
📝 ready; run the command from the [Executing a plan](#executing-a-plan-git-worktree--subagent-per-task)
table (subagent-driven-development in a git worktree).
3. **Wave 2 is done.** Both deliverables shipped and live-gated: **MQTT / Sparkplug B** (P1 + P2) and **MTConnect Agent** (P1 Agent MVP, PR #506). Remaining work on both is optional follow-on — MQTT P3 write-through (`IWritable`: NCMD/DCMD + plain publish) plus its two browse-UI
gaps, and MTConnect write-back (deliberately deferred; the Agent surface is read-only).
Update this file's Summary table and per-wave status whenever a deliverable changes state.