docs(plans): native .NET EtherNet/IP+CIP+PCCC transport to replace libplctag under AbCip/AbLegacy — design

docs/plans/2026-10-07-native-ab-transport-design.md

@@ -0,0 +1,265 @@
+# Native .NET transport for AbCip + AbLegacy (replace libplctag) — design
+
+**Date:** 2026-10-07 · **Branch:** `feat/native-ab-transport` · **Status:** design, awaiting review
+
+## 1. Goal
+
+Replace the `libplctag` NuGet (managed wrapper + native `plctag` binary, pinned `1.5.2`) under the
+AbCip and AbLegacy drivers with a fully managed EtherNet/IP + CIP + PCCC client owned by this repo.
+libplctag is kept, behind a switch, as a fallback until the native path is proven.
+
+### Why (all four apply)
+
+1. **Native-binary pain** — RID-specific packaging, container/ARM images, an unmanaged dependency in
+   an otherwise all-.NET 10 server.
+2. **Correctness and control** — wrapper defects we cannot fix: `ElementSize` returns `null` after a
+   read (`68cf3ae8`), there is no `ConnectionSize` property (the value is plumbed and dropped),
+   status codes are opaque, and libplctag never checks the CIP type the controller returns (`e3d6b76e`
+   could only add a width check).
+3. **Performance** — one native tag handle per `(device, tag)`; no control over request packing or
+   scheduling.
+4. **Policy** — no native dependencies; libplctag's LGPL/MPL licensing.
+
+### Non-goals
+
+- No change to the driver surface above the transport seams: the address space, tag-config JSON,
+  the poll engine, the RMW serialization, the backoff, and the decoders all stay.
+- No new PLC families. No CIP Class-1 (implicit I/O) connections. No Omron (`omron-njnx` stays an
+  unmapped libplctag value only while the fallback exists).
+- Whole-UDT **writes** stay unsupported, as they are today.
+
+## 2. Decisions (all taken at the recommended option)
+
+| # | Question | Decision |
+|---|---|---|
+| D1 | Build or buy | **Build** an in-repo managed stack. Existing managed CIP libraries fail the licensing/control goals. |
+| D2 | Cutover | **Side-by-side behind a per-driver `Transport` switch** (`Native` \| `Libplctag`), the same dark-switch discipline as `MeshTransport` / `ConfigSource`. |
+| D3 | Proof standard | **Flip per evidence tier** (§8). Native becomes default for AbCip once reads/browse/templates pass the real presses read-only; AbCip writes and all of AbLegacy flip on simulator proof, recorded as a known gap. libplctag stays as the fallback until a bench controller exists. |
+| D4 | Where the stack lives | A new shared project, **`ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport`**, referenced by both drivers (precedent: `Mqtt.Transport`, `OpcUaClient.Session`). Pure protocol: no OtOpcUa, no OPC UA, no logging-framework dependency beyond `Microsoft.Extensions.Logging.Abstractions`. |
+| D5 | Connection model | **One CIP connection per device** (`host:port` + CIP path), shared by every tag runtime on that device. libplctag already shares internally; the native stack makes it explicit. |
+| D6 | Request packing | **Transparent coalescing inside the connection** into Multiple Service Packet (0x0A) requests, within the negotiated connection size, when `AllowPacking`. The `IAbCipTagRuntime` interface does not change. |
+| D7 | Status neutrality | A **driver-owned `AbStatus` enum + `AbTransportException`** replace libplctag's `Status` / `LibPlcTagException` in the mappers, probes and `AbCipDriver`. The libplctag adapter translates into them. |
+| D8 | Decoder reuse | The native enumerator and template reader **emit the same byte layout libplctag's `@tags` / `@udt/{id}` pseudo-tags return**, so `CipSymbolObjectDecoder` / `CipTemplateObjectDecoder` and their real-capture tests are shared unchanged, and the two transports can be diffed byte for byte. |
+| D9 | Primary correctness evidence | **Differential testing**: the same reads through both transports against the same target (`ab_server`, then the real presses read-only), compared on raw buffer and decoded value. |
+
+## 3. Architecture
+
+```
+AbCipDriver / AbLegacyDriver          (unchanged except factory selection + AbStatus)
+   │  IAbCipTagFactory / IAbCipTagEnumeratorFactory / IAbCipTemplateReaderFactory / IAbLegacyTagFactory
+   ├── Libplctag*  adapters           (existing; now translate Status → AbStatus)   ← fallback
+   └── Native*     adapters           (new, in each driver project)
+          │
+          ▼
+   ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport   (new, pure managed)
+     Eip/      EncapsulationClient — TCP framing, RegisterSession/UnRegisterSession,
+               ListIdentity, SendRRData (UCMM), SendUnitData (connected)
+     Cip/      CipPath (port segments from "1,0", ANSI symbolic 0x91, member/element
+               segments 0x28/0x29/0x2A), CipService, CipGeneralStatus + extended status
+     Cip/      ConnectionManager — Forward Open (0x54) / Large Forward Open (0x5B) /
+               Forward Close (0x4E), sequence counts, connection-size negotiation,
+               UCMM-only mode (Micro800)
+     Cip/      RequestScheduler — per-connection queue, MSP coalescing, in-flight bound,
+               per-request timeout + cancellation
+     Logix/    Read Tag (0x4C) / Read Tag Fragmented (0x52) / Write Tag (0x4D) /
+               Write Tag Fragmented (0x53); Logix type codes incl. struct handle (0xA0)
+     Logix/    SymbolLister (Get Instance Attribute List 0x55 on class 0x6B, paged)
+     Logix/    TemplateReader (Get Attribute List 0x03 + Read 0x4C on class 0x6C, chunked)
+     Pccc/     Execute PCCC (0x4B, class 0x67) with requestor id; SLC/MicroLogix protected
+               typed logical read/write (0x0F/0xA2, 0x0F/0xAA); PLC-5 typed read/write
+               (0x0F/0x68, 0x0F/0x67); PCCC STS/EXT STS
+     DeviceSessionPool — one session+connection per device key, ref-counted
+```
+
+### 3.1 Units and their contracts
+
+- **`EncapsulationClient`**: owns one TCP socket. It sends and receives 24-byte-header encapsulation
+  frames and correlates replies by sender context. It knows nothing about CIP. Testable against canned
+  byte streams.
+- **`CipPath`**: a pure encoder. It turns `"1,0"` (and `"1,0,2,10.0.0.5"` style routes) into port
+  segments, and turns a Logix tag name such as `Program:P.Arr[3].Member.5` into symbolic, element and
+  member segments. `AbCipTagPath` already parses names; `CipPath` encodes them. Golden-byte unit tests.
+- **`ConnectionManager`**: opens and closes the connected session for a device.
+  - **Connection size.** It honours `ConnectionSize` (the field libplctag drops today). Above 511
+    bytes it uses Large Forward Open. If the target rejects a large open (status 0x01, extended 0x0109
+    or 0x0111), it falls back to a regular Forward Open at the size the target advertises.
+  - **Micro800** (`SupportsConnectedMessaging = false`) runs UCMM-only: no Forward Open.
+- **`RequestScheduler`**: one queue per connection.
+  - **Coalescing.** Requests that arrive within one scheduling turn are packed into an MSP until the
+    next service would overflow the connection size. A reply is fanned back to each waiter with its own
+    per-service status.
+  - **Oversize requests.** A request too large for one frame goes through the fragmented service loop.
+  - **Bounds.** At most one in-flight connected request per connection by default (`MaxInFlight`,
+    default 1), the conservative choice for a production press.
+- **`DeviceSessionPool`**: keyed by `(host, port, cipPath, familyAttribute)`. A runtime acquires a
+  lease on first `InitializeAsync` and releases it on `Dispose`. The last release sends Forward Close
+  and UnRegisterSession.
+
+### 3.2 Native adapters (in the driver projects)
+
+- **`NativeAbCipTagRuntime : IAbCipTagRuntime`**
+  - **Reads.** `InitializeAsync` acquires the device lease and resolves the encoded path once.
+    `ReadAsync` issues Read Tag with `ElementCount`. The reply's **CIP type code + struct handle** is
+    kept with the buffer.
+  - **Decoding.** `Decode*` reads the managed buffer, little-endian. Strings use the Logix
+    `LEN(u32) + DATA[n]` layout with the stride taken from the reply. BOOL arrays are 32-bit packed.
+  - **Size and status.** `GetSize()` is the real reply length, so the stride fallback in
+    `ResolveArrayStride` stops being necessary. `GetStatus()` returns `AbStatus`.
+  - **Writes.** `WriteAsync` issues Write Tag with the type taken from the last read, or from the
+    authored type for atomics. A struct-typed write (STRING) needs the struct handle, so a write with
+    no prior read performs one read first.
+- **`NativeAbCipTagEnumerator`** reads controller-scope and `Program:X` listings through
+  `SymbolLister`, and emits the concatenated entry bytes in the `@tags` layout.
+- **`NativeAbCipTemplateReader`** emits the 14-byte header that libplctag synthesizes, then the raw
+  template bytes (D8).
+- **`NativeAbLegacyTagRuntime : IAbLegacyTagRuntime`**
+  - **Addressing.** Encodes `AbLegacyAddress` (file type, file number, element, sub-element) into PCCC
+    typed-logical addressing, or PLC-5 system addressing for `plc5`.
+  - **Arrays.** Multi-element reads use the element count.
+  - **Bits.** Bit-in-word writes stay at the driver level (`WriteBitInWordAsync`), as today.
+
+### 3.3 The switch
+
+- `AbCipDriverOptions.Transport` and `AbLegacyDriverOptions.Transport` take values
+  `Native | Libplctag`, are serialized as **strings** (`JsonStringEnumConverter`, per the
+  driver-enum-serialization trap), and are absent ⇒ the current default.
+- `Ab*DriverFactoryExtensions.CreateInstance` picks the factory set from the option. Test-injected
+  factories still win.
+- The **probe** (`Ab*DriverProbe`, i.e. Test Connect) honours the same option. A probe that tests a
+  different transport than the driver runs would be a lie.
+- **AdminUI**: a `Transport` dropdown in the AbCip and AbLegacy driver-config modals under the `/raw`
+  tree. It needs live `/run` verification on docker-dev, since there is no bUnit.
+- Default sequence: `Libplctag` at merge, flipped per §8, and the fallback stays selectable.
+
+## 4. Data flow (a poll read)
+
+1. The driver poll calls `runtime.ReadAsync(ct)` on N runtimes for one device. The concurrency
+   window is unchanged.
+2. Each runtime enqueues a Read Tag request on the device's `RequestScheduler`.
+3. The scheduler packs as many as fit into one MSP, sends it via SendUnitData on the shared
+   connection, awaits the reply, splits it by service, and completes each waiter with
+   `(status, typeCode, structHandle, payload)`.
+4. The runtime stores the payload. The driver calls `Decode*` exactly as it does today.
+
+The result: N round-trips through N libplctag handles become ⌈N / fit⌉ round-trips through one
+connection. That is the performance win, with no driver change.
+
+## 5. Error handling and status
+
+- **`AbStatus`** (driver-owned) carries the values the mappers and probes use today: `Ok`,
+  `Pending`, `Timeout`, `NotFound`, `NotAllowed`, `OutOfBounds`, `TooLarge`, `TooSmall`,
+  `Unsupported`, `BadConnection`, `BadReply`, `Partial`, `Abort`, plus a new `TypeMismatch`.
+- **CIP general status mapping (native):**
+  - `0x00` ⇒ `Ok`
+  - `0x04` / `0x05` (path segment / destination unknown) ⇒ `NotFound`
+  - `0x06` ⇒ `Partial` (continue fragmenting)
+  - `0x08` ⇒ `Unsupported`
+  - `0x0F` ⇒ `NotAllowed`
+  - `0x13` ⇒ `TooSmall`
+  - `0x15` ⇒ `TooLarge`
+  - `0xFF` ⇒ by extended status (`0x2105` ⇒ `OutOfBounds`, `0x2107` ⇒ `TypeMismatch`)
+  - socket, encapsulation or timeout faults ⇒ `BadConnection` / `Timeout`
+- **PCCC mapping.** STS/EXT STS map the same way: address-does-not-exist ⇒ `NotFound`, illegal
+  command/format ⇒ `Unsupported`, and so on.
+- **`AbTransportException(AbStatus, string detail)`** is thrown by both transports.
+  `AbCipDriver.cs:1188`'s "the controller answered, so don't throttle the device" check
+  (`IsReachableStatus`) keys on it. **That preserves 05/STAB-8 semantics.**
+- **Type check, native only.** The reply's CIP type code is compared with the authored type.
+  - A mismatch ⇒ `BadTypeMismatch`. This now also catches **same-width** mistakes (DINT authored,
+    REAL returned) that `e3d6b76e` documents as undetectable.
+  - ⚠️ **This is client-visible.** Tags authored wrong from the legacy Kepware export that read Good
+    garbage under libplctag will read Bad under native. That is the fix working, but it must be
+    in the release note and in `docs/drivers/AbCip.md`.
+- **Press safety.**
+  - **Reconnect.** A dropped connection is re-opened through the driver's existing `device.Backoff`,
+    never in a tight loop.
+  - **One open at a time.** `ConnectionManager` enforces at most one Forward Open in flight per
+    device, plus a minimum interval between opens (default 1 s).
+  - **Pinned by a test.** A regression test asserts a bounded Forward Open rate against a refusing
+    target. ignitionoee recorded **~15 Forward Opens/s** at a press from a misconfigured client.
+  - **Close.** Forward Close on dispose. The connection timeout multiplier and RPI are set so a dead
+    client's connection expires on the PLC within seconds, not minutes.
+
+## 6. Packaging
+
+- New project `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport/` and its test project
+  `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport.Tests/`, both added to
+  `ZB.MOM.WW.OtOpcUa.slnx`.
+- While the fallback exists, both drivers keep the `libplctag` package. Libplctag code is confined
+  to the `Libplctag*.cs` adapters. After phase 2 no other file has `using libplctag`, and a
+  source-scan guard test enforces that.
+- On deletion (phase 6) the package, its `Directory.Packages.props` pin, the adapters and the
+  `Libplctag*` create-param members (`LibplctagGateway`; `LibplctagPlcAttribute` is renamed
+  `PlcAttribute`) go.
+
+## 7. Testing
+
+1. **Golden-frame unit tests** (`Ab.Transport.Tests`).
+   - **Encode** every frame type: RegisterSession, Forward Open / Large / Close, MSP, Read / Write
+     (Fragmented), 0x55 listing, template attribute reads, Execute PCCC.
+   - **Decode** every reply type, including error replies.
+   - **Fixtures** are byte captures taken once from libplctag talking to `ab_server` (pcap, stored as
+     hex), plus the **existing real-press captures** in `CipSymbolObjectWireCaptureTests` and
+     `CipTemplateObjectWireCaptureTests`.
+   - ⚠️ A capture from libplctag ⇄ `ab_server` proves wire parity with libplctag, not correctness. The
+     real-press captures are the independent reference (the #468 lesson: verify a fixture with the
+     reference client).
+2. **Scheduler and connection tests.** These run against an in-process fake encapsulation endpoint.
+   They cover:
+   - MSP packing boundaries at connection size −1, exact and +1
+   - per-service error fan-out
+   - cancellation mid-MSP
+   - the Forward Open rate bound
+   - large-open fallback
+   - lease ref-counting and Forward Close on last release
+3. **Integration suites run on both transports.** The existing `AbCip.IntegrationTests` and
+   `AbLegacy.IntegrationTests` against `ab_server` (controllogix / compactlogix / micro800 / guardlogix;
+   slc500 / micrologix / plc5) are parameterized over `Transport`. A native run must be green on every
+   test the libplctag run is green on.
+4. **Differential harness.** A `LiveIntegration`-category test, env-gated like the HistorianGateway
+   live suite and skipped when the env vars are absent. It reads the same tag set through both
+   transports and asserts equal raw buffers and equal decoded values:
+   - against `ab_server`;
+   - against a real press, **read-only**. It runs browse (`@tags` + every program listing), every
+     template, and a sample of scalar, array, STRING and UDT-member reads.
+5. **AdminUI `/run`.** Author `Transport: Native` on a docker-dev AbCip driver pointed at the
+   `ab_server` fixture, deploy, read, then flip it back.
+
+## 8. Phased delivery and gates
+
+| Phase | Delivers | Gate |
+|---|---|---|
+| **P0 Seam** | `AbStatus` + `AbTransportException`; mappers, probes, `AbCipDriver` and `AbLegacyDriver` decoupled from libplctag types; the 9 unit-test files that build `LibPlcTagException` / `Status` fakes (`AbCipDriverWriteTests`, `AbCipDriverProbeTests`, `AbCipDriverReadTests`, `AbCipStatusMapperTests`, `AbCipDriverCodeReviewRegressionTests`, `AbCipUnknownTagReadTests`, `AbLegacyEvictOnFailureTests`, `AbLegacyReadWriteTests`, `AbLegacyHostAndStatusTests`) moved onto `AbStatus` / `AbTransportException`, with the libplctag translation itself tested in the adapters; the `Transport` option (string enum) + factory selection, including a `--transport` flag on `otopcua-abcip-cli` / `otopcua-ablegacy-cli` (whose help text names libplctag); the source-scan guard | Existing suites green; behaviour identical (default `Libplctag`) |
+| **P1 Transport core** | Encapsulation, `CipPath`, `ConnectionManager`, `RequestScheduler` (MSP + fragmentation), `DeviceSessionPool` | Golden-frame + scheduler tests |
+| **P2 AbCip native** | `NativeAbCipTagRuntime`, enumerator, template reader; CIP type check | AbCip integration suite green on both transports vs `ab_server` |
+| **P3 AbLegacy native** | PCCC layer + `NativeAbLegacyTagRuntime` | AbLegacy integration suite green on both transports vs `ab_server` (slc500 / micrologix / plc5) |
+| **P4 Switch surface** | AdminUI dropdown, probe honours the switch, docs (`AbCip.md`, `AbLegacy.md`, fixture docs) | docker-dev `/run` |
+| **P5 Real-hardware gate + flip** | Differential run against the CR presses read-only (VPN from the Mac); a perf comparison (round-trips + p50/p95 poll latency for one press's tag set, native vs libplctag); then **default ⇒ `Native` for both drivers** | AbCip: diff clean on browse / templates / reads, no worse latency, Forward Open count during the run = 1. AbLegacy: simulator proof only, **recorded as a known gap** |
+| **P6 Delete libplctag** *(conditional)* | Remove the package, adapters and `Libplctag*` members | **Blocked until a bench controller** proves writes (CompactLogix or Micro800) and PCCC (SLC / MicroLogix). Until then libplctag stays selectable |
+
+### Real-hardware constraints (from `~/Desktop/ignitionoee`)
+
+- **Reachable targets:** 20 × CompactLogix **5069-L320ERS2 rev 34.11** presses at CR
+  (Z-35162..Z-35181; Z-35162 = `10.209.70.160:44818`), reachable from the Mac over VPN. This is
+  the same rig `cc876bdf` verified against.
+- **They are production machines and strictly read-only.** P5 performs **no writes**, opens a
+  single connection per press, and touches one press at a time.
+- **No legacy PCCC hardware is reachable.** The CR presses were confirmed non-MicroLogix by
+  ListIdentity. The Delmia inventory lists PLC-5/04s at Warsaw with no addresses, and as far as I
+  know a PLC-5/04 has no built-in Ethernet port. There is no ControlLogix and no Micro800 either. So
+  routed paths beyond `1,0` and Micro800 UCMM-only mode are simulator-proven only.
+
+## 9. Known gaps and risks
+
+- **AbCip writes are simulator-proven only.** Mitigation: Write Tag shares framing with the
+  read-proven path. The risk sits in the encoding details (STRING struct handle, type codes), which
+  have golden tests and run against `ab_server`. libplctag stays selectable.
+- **AbLegacy is simulator-proven only**, against `ab_server`'s PCCC emulation, which shares
+  libplctag's view of the protocol. This is the weakest evidence in the plan, stated plainly.
+- **`ab_server` patches** (`0001` symbol listing, `0002` CIP 0x04 for not-found) mean the simulator is
+  already partly ours. A native/libplctag disagreement against it must be checked against the real
+  press before blaming either side.
+- **The same-width type check changes client-visible quality** (§5).
+- **Behaviour libplctag hides that we now own:** connection keep-alive, sequence-count wrap, the
+  session-registration lifetime, and the TCP reconnect after an idle teardown by a NAT or firewall.
+  Each one gets a scheduler/connection test.