docs(plan): MTConnect write-back design + implementation plan; MQTT P3 recorded as already shipped

docs/plans/2026-08-05-mtconnect-writeback-design.md

@@ -0,0 +1,151 @@
+# MTConnect write-back + MQTT write-through record reconciliation — design
+
+**Status:** DRAFT 2026-08-05.
+
+## 1. Scope correction: the MQTT half of this request is already shipped
+
+The request ("MQTT P3 write-through plus its two documented browse-UI gaps, MTConnect write-back")
+came from `docs/plans/2026-07-24-driver-expansion-tracking.md` §Next actions (lines 288–289) — which
+is **stale**. On master today:
+
+| Item | Status | Evidence |
+|---|---|---|
+| MQTT P3 write-through (`IWritable`: plain `writeTopic` publish + Sparkplug NCMD/DCMD) | **Shipped, live-gated** — #508, `72be1922`, 2026-07-31 | `MqttDriver.cs:65-67` implements `IWritable`; `Mqtt.md:16-20` status banner |
+| Browse-UI gap 1 (empty tree cannot arm a rebirth) | **Fixed** — #514, `7f53bd96` | `RawBrowseModal.razor:127-157` manual `{group}/{edgeNode}` scope input |
+| Browse-UI gap 2 (slashed metric name refused at browse-commit) | **Fixed** — #509, `7f53bd96` | `RawBrowseCommitMapper.ToRawSegment` (`:107`), collision refusal (`:117`) |
+
+So the only MQTT work left is **documentation reconciliation** — three stale records that now
+contradict the code and `Mqtt.md`'s own closure line (`:597-602`):
+
+1. `docs/drivers/Mqtt.md:550-556` — still prints the #514 rebirth gap as an open "Known gap".
+2. `docs/drivers/Mqtt.md:586-587` — "Read-only. No writes reach the broker; every node is ViewOnly"
+   contradicts §Write-through (`:407-480`) in the same file.
+3. `docs/plans/2026-07-24-driver-expansion-tracking.md:288-289` §Next actions still lists MQTT P3 +
+   the two gaps as remaining work; the P2 section (`:239-263`) still describes the gaps as
+   "documented not fixed".
+4. `deferment.md:132-133` — "`IWritable` is absent from MTConnect, MQTT, Sql and Calculation" was
+   audited pre-#508 and is wrong about MQTT (and, after this branch, about MTConnect).
+
+## 2. MTConnect write-back — the real work
+
+### 2.1 What was deferred, and what mechanism to build on
+
+The driver shipped deliberately read-only (`MTConnectDriver.cs:63` — no `IWritable`;
+`docs/plans/2026-07-15-mtconnect-driver-design.md` §3.6). The deferral named **MTConnect
+Interfaces** (a request/response state-machine handshake between two agents) as the only write-back
+shape and judged it misleading to model as `IWritable` — "revisit only on a concrete deployment
+need". This request is that need, but Interfaces is still the wrong mechanism: it is rarely
+deployed, requires adapter-side cooperation, and is a state machine rather than a value write.
+
+The mechanism this design uses instead is the **Agent's HTTP PUT/POST surface** — a standard
+cppagent feature, off by default, enabled by `AllowPut = true` (optionally scoped by
+`AllowPutFrom = <host list>`) in `agent.cfg`. With it enabled, `PUT`/`POST
+{agent}/{deviceName}?{dataItemId}={value}` sets the named observation in the agent's buffer exactly
+as if an adapter had reported it. The repo's own integration fixture already carries the key —
+currently `AllowPut = false` with a "must not accept writes" comment
+(`tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg:25-26`).
+
+### 2.2 Honest semantics — this is observation injection, not machine command
+
+A PUT write updates **what the agent reports**, not the machine: the value lands in the agent's
+observation buffer, is visible to every agent consumer, and the next adapter/SHDR update for that
+data item overwrites it. That is genuinely useful (operator-set EVENT values, external systems
+feeding data items the adapter does not own, test rigs) but it must never be documented as
+commanding the CNC. The docs section this branch writes states this prominently, and the feature is
+**opt-in at two independent layers**: the agent (`AllowPut`) and the driver config
+(`write.enabled`, default `false`). Either alone leaves today's read-only behaviour untouched.
+
+### 2.3 Driver design
+
+**Config.** `MTConnectDriverConfigDto` (+ typed `MTConnectDriverOptions`) gains a `Write` block:
+
+```json
+"write": { "enabled": true }
+```
+
+Default absent/`false` — nothing changes for existing deployments. No per-tag key: writability is
+derived (below), matching how the driver derives everything else from the probe model.
+
+**Capability.** `MTConnectDriver` adds `IWritable`. The factory already returns the concrete type
+(`MTConnectDriverFactoryExtensions.cs:33-39` documents why), so capability dispatch picks it up
+with no registry change.
+
+**`WriteAsync` shape** (mirrors `MqttDriver.cs:670-745` — never throws, batch shape preserved,
+per-slot codes):
+
+- `write.enabled == false` → every slot `BadNotWritable` (0x803B0000).
+- No live client (pre-Initialize / post-Shutdown) → `BadNotConnected`, matching `ReadAsync`'s
+  convention (`MTConnectDriver.cs:628-695`).
+- RawPath does not resolve via `TagBinding.ToDataItemId` → `BadNodeIdUnknown`.
+- Data item is CONDITION-category, or the value is null / not representable as an invariant string
+  → `BadNotWritable` / `BadTypeMismatch` respectively. (Condition PUTs use a different level|text
+  wire format and carry alarm semantics — excluded from v1.)
+- HTTP call: `POST {agentBase}/{deviceName}?{dataItemId}={value}` via the agent-client seam. The
+  owning `deviceName` comes from a `dataItemId → deviceName` map built at bind time from the cached
+  probe model (falling back to `options.DeviceName`); a multi-device agent is addressed correctly
+  without config.
+- Agent refusal (non-2xx, or an `MTConnectError` document in a 200 body) → `BadCommunicationError`
+  with the agent's own error text logged (never in `LastError` — type-only discipline).
+- `OperationCanceledException` fills remaining slots and returns; any other exception → log warning
+  + `BadCommunicationError`.
+- Values format with `CultureInfo.InvariantCulture`; booleans as the raw invariant string.
+
+**Seam extension.** `IMTConnectAgentClient` gains
+`Task<MTConnectWriteOutcome> WriteObservationAsync(string deviceName, string dataItemId, string value, CancellationToken ct)`
+(outcome = accepted / agent-error-with-text / transport-error). `MTConnectAgentClient` implements
+it over `_http` (the unary client with the bounded timeout) using `HttpMethod.Post`; the stub
+`HttpMessageHandler` ctor seam (`MTConnectAgentClient.cs:107`) covers response-shape unit tests
+with no socket. The unit-test `CannedAgentClient` records write calls and returns scripted
+outcomes.
+
+**Discovery stamping.** `DiscoverAsync` currently stamps every leaf
+`SecurityClassification.ViewOnly` (`MTConnectDriver.cs:1040-1042`) with a comment saying that is
+the only honest tier for a driver with no `IWritable`. That derivation becomes conditional:
+`CanWrite(dataItem)` (= `write.enabled` && category is not CONDITION) → `Operate`, else `ViewOnly`
+— the same "a node cannot advertise writability the driver would refuse" principle MQTT pinned
+(`MqttDriver.cs:504`, `CanWrite` at `:625-627`). The implementer must verify how materialization
+derives the `CurrentWrite` access bit (capability presence + SecurityClass) so a stamped-Operate
+node is actually writable over OPC UA, and a ViewOnly node is not.
+
+**Doc comments to update in the same change:** the class remarks
+(`MTConnectDriver.cs:46-53` "deliberately no IWritable"), the factory remark
+(`MTConnectDriverFactoryExtensions.cs:39`), and `MTConnectTagDefinition.cs:10-14` ("no
+corresponding write path").
+
+### 2.4 Out of scope
+
+- **MTConnect Interfaces** — still deferred, same rationale as §3.6; this design does not touch it.
+- Asset writes (`POST /asset`), per-tag write opt-outs, write retry (`isIdempotent` stays `false`
+  at `DriverInstanceActor.cs:776`), and PUT `time=` timestamps — YAGNI.
+- Any AdminUI surface — the driver form gains nothing; `write.enabled` is authored in the existing
+  driver-config JSON. (No chrome gate needed for this branch.)
+
+### 2.5 Testing
+
+1. **Unit (offline, falsifiable).** New `MTConnectWriteTests`: batch contract + every refusal leg
+   via `CannedAgentClient` (disabled-by-default, unknown RawPath, condition category, null value,
+   no client, cancellation fill, agent-error outcome). Encoder-free — the write is a URL, so the
+   URL/verb/device-resolution assertions live in `MTConnectAgentClient` stub-handler tests
+   (MTConnectError-in-200 detection included). Discovery stamping test: same probe fixture browsed
+   with write on/off flips leaf SecurityClass, CONDITION stays ViewOnly.
+2. **Integration (env-gated, docker fixture).** Flip the fixture to `AllowPut = true` (update the
+   comment to say writes are now in scope) and add a write leg: write via the driver → read
+   `/current` back → the observation carries the written value; a second leg proves an
+   `AllowPut = false`-style refusal maps to `BadCommunicationError` is *not* needed (we cannot run
+   two agent configs in one fixture) — instead a bad-dataItemId write asserts the agent-error
+   mapping. Remove the "(b) Writes" not-covered remark
+   (`MTConnectAgentIntegrationTests.cs:64-65`).
+3. **Live gate (no chrome).** Against the docker-dev rig + the shared fixture agent: author an
+   MTConnect driver with `write.enabled`, deploy, `Client.CLI write` to a writable node, confirm
+   via `/current` and via OPC UA read-back. Runnable headless by the executing session.
+
+## 3. Bookkeeping (both halves)
+
+- `docs/drivers/MTConnect.md`: capability surface (`:71-77`), Deferred (`:295-301`), new
+  §Write-back section with the §2.2 semantics warning.
+- `docs/plans/2026-07-15-mtconnect-driver-design.md` §3.6: superseded note pointing here.
+- `docs/plans/2026-07-24-driver-expansion-tracking.md`: MQTT P2 gap bullets + §Next actions
+  rewritten to record #508/#509/#514 closure and this branch's MTConnect write-back.
+- `docs/drivers/Mqtt.md`: delete stale blocks `:550-556` and `:586-587`.
+- `deferment.md`: correct `:132-133` (post-branch: `IWritable` absent from Sql and Calculation
+  only) and the MTConnect row (`:126`).

docs/plans/2026-08-05-mtconnect-writeback.md

@@ -0,0 +1,183 @@
+# MTConnect write-back Implementation Plan
+
+> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development
+> (opus implementers per user instruction).
+
+**Goal:** Implement `IWritable` on `MTConnectDriver` via the Agent's HTTP PUT/POST surface
+(opt-in at agent `AllowPut` AND driver `write.enabled`), and reconcile the stale MQTT
+write-through/browse-gap records.
+
+**Architecture:** See `docs/plans/2026-08-05-mtconnect-writeback-design.md` (in this branch).
+Write = observation injection into the agent buffer, never machine command. Batch-safe
+`WriteAsync` mirroring `MqttDriver.cs:670-745`; new seam method on `IMTConnectAgentClient`;
+discovery stamps `Operate` only for leaves the driver would accept.
+
+**Tech Stack:** .NET 10, xUnit v3 + Shouldly, existing `CannedAgentClient` / stub
+`HttpMessageHandler` seams. No new packages.
+
+**Worktree:** `/Users/dohertj2/Desktop/OtOpcUa/.claude/worktrees/mqtt-mtconnect-write`
+(branch `feature/mqtt-mtconnect-write`). All paths below relative to it.
+
+---
+
+### Task 0: Write options block (config + parse)
+
+**Classification:** small
+**Estimated implement time:** ~4 min
+**Parallelizable with:** Task 1
+
+**Files:**
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectDriverOptions.cs`
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs` (config DTO region
+  ~:3257-3345 + `ParseOptions` ~:2222)
+- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectDriverOptionsTests.cs`,
+  `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectFactoryTests.cs`
+
+TDD: failing test first — `{"write":{"enabled":true}}` parses to `Options.Write.Enabled == true`;
+absent block defaults to `false`; malformed value behaves like the sibling `Probe`/`Reconnect`
+blocks (match their existing tolerance/failure semantics exactly — read them first). Add
+`MTConnectWriteOptions` (sealed record, `Enabled`, default false) on `MTConnectDriverOptions`
+following the `Probe`/`Reconnect` pattern, a nullable `WriteDto` on the config DTO, wire through
+`ParseOptions`. Commit `feat(mtconnect): parse the write.enabled config block`.
+
+### Task 1: Agent-client write seam
+
+**Classification:** standard
+**Estimated implement time:** ~5 min
+**Parallelizable with:** Task 0
+
+**Files:**
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/IMTConnectAgentClient.cs`
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectAgentClient.cs`
+- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectAgentClientWriteTests.cs` (new)
+
+Add to the seam:
+
+```csharp
+Task<MTConnectWriteOutcome> WriteObservationAsync(
+    string deviceName, string dataItemId, string value, CancellationToken cancellationToken);
+```
+
+with `MTConnectWriteOutcome` a small readonly record: `Accepted` bool + `AgentError` string?
+(agent-produced MTConnectError text) — transport failures THROW (HttpRequestException etc.), the
+driver maps them; agent refusals return `Accepted=false` with the text. Implementation in
+`MTConnectAgentClient` over `_http` (the bounded-timeout unary client): `HttpMethod.Post` to
+`{base}/{deviceName}?{Uri.EscapeDataString(dataItemId)}={Uri.EscapeDataString(value)}` built with
+the same left-part logic as `BuildRequestUri` (:382-403). A 2xx whose body parses as an
+`MTConnectError` document is a refusal (reuse the LocalName-only XML helpers in
+`MTConnectXml.cs`); non-2xx is a refusal carrying the status; empty 2xx is accepted.
+
+TDD via the internal `HttpMessageHandler` ctor seam (`MTConnectAgentClient.cs:107`), the same
+pattern `MTConnectDriverProbeTests.cs:22-28` uses: assert verb, exact URI (device path + escaped
+query), accepted on empty 200, refusal on MTConnectError-in-200 (agent text surfaced), refusal on
+404, throw on handler-thrown transport error. Commit
+`feat(mtconnect): agent-client observation write (HTTP POST, AllowPut surface)`.
+
+### Task 2: `MTConnectDriver` implements `IWritable` + discovery stamping
+
+**Classification:** high-risk
+**Estimated implement time:** ~5 min
+**Parallelizable with:** none
+
+**Files:**
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriver.cs` (class decl :63,
+  remarks :46-53, WriteAsync new, `TagBinding` device-name map ~:3170-3213, `DiscoverAsync`
+  stamping :1030-1046)
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect/MTConnectDriverFactoryExtensions.cs`
+  (remark :33-39)
+- Modify: `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Contracts/MTConnectTagDefinition.cs`
+  (remark :10-14)
+- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/CannedAgentClient.cs`
+  (record writes, scripted outcomes)
+- Test: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests/MTConnectWriteTests.cs` (new),
+  `MTConnectDiscoverTests.cs` (stamping cases)
+
+Per design §2.3. Status-code constants: follow MQTT's named-const pattern
+(`MqttDriver.cs:76-89`) — `StatusBadNodeIdUnknown 0x80340000`, `StatusBadNotWritable 0x803B0000`,
+`StatusBadTypeMismatch 0x80740000`, `StatusBadCommunicationError 0x80050000`, plus
+`BadNotConnected` matching `ReadAsync`'s existing constant. Refusal order per slot: disabled →
+BadNotWritable; no client → BadNotConnected; unresolved RawPath → BadNodeIdUnknown; CONDITION
+category → BadNotWritable; null/unrepresentable value → BadTypeMismatch; then the seam call —
+`Accepted=false` or thrown transport error → BadCommunicationError (log agent text at Warning,
+never into `LastError`). `OperationCanceledException` fills remaining slots with
+BadCommunicationError and returns. Never throw. Success = `WriteResult(0u)`.
+
+Device-name resolution: extend `TagBinding` (or a parallel FrozenDictionary built in
+`BuildBinding` ~:2887-2980) with `dataItemId → owning deviceName` from the probe model; fall back
+to `options.DeviceName`; unresolvable device → BadNodeIdUnknown. Values format via
+`Convert.ToString(value, CultureInfo.InvariantCulture)`.
+
+Discovery: `CanWrite(dataItem)` = `options.Write.Enabled && !IsCondition(category)` →
+`SecurityClassification.Operate` else `ViewOnly`; update the :1040-1042 comment. **Verify** how
+materialization derives `CurrentWrite` (search for how MQTT's Operate-stamped nodes get the write
+bit — SecurityClass + capability) and state the finding in the commit message; if a server-side
+gate keys on the driver capability alone, note that existing ViewOnly-committed raw tags keep
+their authored SecurityClass.
+
+TDD: every refusal leg observed red first where feasible; the batch-shape test (one bad slot
+doesn't fail the batch) mirrors `MqttDriverWriteTests.cs:74`. The publish itself is proven by
+Task 3 live legs, not by asserting a fake was called beyond the seam-call recording (same
+philosophy as `MqttDriverWriteTests.cs:11-15`). Commit
+`feat(mtconnect): IWritable via agent observation writes (#468-era deferral closed)`.
+
+### Task 3: Integration fixture + write legs
+
+**Classification:** standard
+**Estimated implement time:** ~5 min
+**Parallelizable with:** Task 4
+
+**Files:**
+- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg`
+  (`AllowPut = true` + comment rewrite)
+- Modify: `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/MTConnectAgentIntegrationTests.cs`
+  (remove the "(b) Writes" not-covered remark :64-65; add write legs)
+
+Legs (env-gated by the existing `MTCONNECT_AGENT_ENDPOINT` fixture, `Assert.Skip` clean):
+1. Driver with `write.enabled` writes a distinct value to a writable EVENT data item (pick one
+   from `Devices.xml` the SHDR adapter does not continuously overwrite — read `adapter.py` to
+   choose; if every item is pumped, add a static EVENT item to `Devices.xml`) → poll `/current`
+   until the observation carries the value (bounded wait, `StreamWait` pattern).
+2. A write naming a nonexistent dataItemId → `BadCommunicationError` (agent refusal mapping).
+3. `write.enabled` absent → same write returns `BadNotWritable` without touching the agent.
+Commit `test(mtconnect): live write legs against the AllowPut fixture`.
+
+### Task 4: Docs reconciliation (MTConnect + stale MQTT records)
+
+**Classification:** small
+**Estimated implement time:** ~5 min
+**Parallelizable with:** Task 3
+
+**Files:**
+- Modify: `docs/drivers/MTConnect.md` (capability :71-77, Deferred :295-301, new §Write-back)
+- Modify: `docs/plans/2026-07-15-mtconnect-driver-design.md` (§3.6 superseded note)
+- Modify: `docs/drivers/Mqtt.md` (delete stale :550-556 and :586-587)
+- Modify: `docs/plans/2026-07-24-driver-expansion-tracking.md` (P2 gap bullets :239-263 → closed
+  #509/#510/#513/#514; §Next actions :288-289 rewritten)
+- Modify: `deferment.md` (:126 MTConnect row, :132-133 four-driver IWritable line → Sql +
+  Calculation only)
+
+Per design §1 + §3. The Write-back section MUST carry the §2.2 semantics warning (observation
+injection, adapter overwrites, two-layer opt-in, `AllowPutFrom` pointer). Commit
+`docs: MTConnect write-back recorded; stale MQTT write-through/browse-gap records reconciled`.
+
+### Task 5: Full-suite verification
+
+**Classification:** small
+**Estimated implement time:** ~4 min
+**Parallelizable with:** none
+
+`dotnet build ZB.MOM.WW.OtOpcUa.slnx` (0 errors) + full MTConnect unit suite + AdminUI suite +
+`dotnet test --filter "FullyQualifiedName~Mqtt"` (docs-only but cheap insurance). Integration
+suite compiles; live legs skip offline. Fix fallout, commit if changes.
+
+### Task 6: Live gate (main session, headless — NO chrome)
+
+**Classification:** standard
+**Estimated implement time:** ~10 min (main session, after merge)
+
+Main session, not a subagent: bring up the MTConnect fixture (`ssh dohertj2@10.100.0.35` compose
+in `/opt/otopcua-mtconnect/` — `lmxopcua-fix sync` equivalent via rsync first, since agent.cfg
+changed), run the Task 3 legs live (`dotnet test --filter` the new write tests with
+`MTCONNECT_AGENT_ENDPOINT` set), then the docker-dev rig leg: author + deploy an MTConnect driver
+with `write.enabled`, `Client.CLI write` a value, read it back via Client.CLI and `/current`.
+Record results in the design doc §Outcome.

docs/plans/2026-08-05-mtconnect-writeback.md.tasks.json

@@ -0,0 +1,13 @@
+{
+  "planPath": "docs/plans/2026-08-05-mtconnect-writeback.md",
+  "tasks": [
+    { "id": 0, "subject": "Task 0: Write options block (config + parse)", "status": "pending" },
+    { "id": 1, "subject": "Task 1: Agent-client write seam", "status": "pending" },
+    { "id": 2, "subject": "Task 2: MTConnectDriver IWritable + discovery stamping", "status": "pending", "blockedBy": [0, 1] },
+    { "id": 3, "subject": "Task 3: Integration fixture + write legs", "status": "pending", "blockedBy": [2] },
+    { "id": 4, "subject": "Task 4: Docs reconciliation (MTConnect + stale MQTT records)", "status": "pending", "blockedBy": [2] },
+    { "id": 5, "subject": "Task 5: Full-suite verification", "status": "pending", "blockedBy": [3, 4] },
+    { "id": 6, "subject": "Task 6: Live gate (main session, headless)", "status": "pending", "blockedBy": [5] }
+  ],
+  "lastUpdated": "2026-08-05T00:00:00Z"
+}