lmxopcua/docs/plans/2026-10-07-native-ab-transport-design.md

# Native .NET transport for AbCip + AbLegacy (replace libplctag) — design

**Date:** 2026-10-07 · **Branch:** `feat/native-ab-transport` · **Status:** design, awaiting review

## 1. Goal

Replace the `libplctag` NuGet (managed wrapper + native `plctag` binary, pinned `1.5.2`) under the
AbCip and AbLegacy drivers with a fully managed EtherNet/IP + CIP + PCCC client owned by this repo.
libplctag is kept, behind a switch, as a fallback until the native path is proven.

### Why (all four apply)

1. **Native-binary pain** — RID-specific packaging, container/ARM images, an unmanaged dependency in
   an otherwise all-.NET 10 server.
2. **Correctness and control** — wrapper defects we cannot fix: `ElementSize` returns `null` after a
   read (`68cf3ae8`), there is no `ConnectionSize` property (the value is plumbed and dropped),
   status codes are opaque, and libplctag never checks the CIP type the controller returns (`e3d6b76e`
   could only add a width check).
3. **Performance** — one native tag handle per `(device, tag)`; no control over request packing or
   scheduling.
4. **Policy** — no native dependencies; libplctag's LGPL/MPL licensing.

### Non-goals

- No change to the driver surface above the transport seams: the address space, tag-config JSON,
  the poll engine, the RMW serialization, the backoff, and the decoders all stay.
- No new PLC families. No CIP Class-1 (implicit I/O) connections. No Omron (`omron-njnx` stays an
  unmapped libplctag value only while the fallback exists).
- Whole-UDT **writes** stay unsupported, as they are today.

## 2. Decisions (all taken at the recommended option)

| # | Question | Decision |
|---|---|---|
| D1 | Build or buy | **Build** an in-repo managed stack. Existing managed CIP libraries fail the licensing/control goals. |
| D2 | Cutover | **Side-by-side behind a per-driver `Transport` switch** (`Native` \| `Libplctag`), the same dark-switch discipline as `MeshTransport` / `ConfigSource`. |
| D3 | Proof standard | **Flip per evidence tier** (§8). Native becomes default for AbCip once reads/browse/templates pass the real presses read-only; AbCip writes and all of AbLegacy flip on simulator proof, recorded as a known gap. libplctag stays as the fallback until a bench controller exists. |
| D4 | Where the stack lives | A new shared project, **`ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport`**, referenced by both drivers (precedent: `Mqtt.Transport`, `OpcUaClient.Session`). Pure protocol: no OtOpcUa, no OPC UA, no logging-framework dependency beyond `Microsoft.Extensions.Logging.Abstractions`. |
| D5 | Connection model | **One CIP connection per device** (`host:port` + CIP path), shared by every tag runtime on that device. libplctag already shares internally; the native stack makes it explicit. |
| D6 | Request packing | **Transparent coalescing inside the connection** into Multiple Service Packet (0x0A) requests, within the negotiated connection size, when `AllowPacking`. The `IAbCipTagRuntime` interface does not change. |
| D7 | Status neutrality | A **driver-owned `AbStatus` enum + `AbTransportException`** replace libplctag's `Status` / `LibPlcTagException` in the mappers, probes and `AbCipDriver`. The libplctag adapter translates into them. |
| D8 | Decoder reuse | The native enumerator and template reader **emit the same byte layout libplctag's `@tags` / `@udt/{id}` pseudo-tags return**, so `CipSymbolObjectDecoder` / `CipTemplateObjectDecoder` and their real-capture tests are shared unchanged, and the two transports can be diffed byte for byte. |
| D9 | Primary correctness evidence | **Differential testing**: the same reads through both transports against the same target (`ab_server`, then the real presses read-only), compared on raw buffer and decoded value. |

## 3. Architecture

```
AbCipDriver / AbLegacyDriver          (unchanged except factory selection + AbStatus)
   │  IAbCipTagFactory / IAbCipTagEnumeratorFactory / IAbCipTemplateReaderFactory / IAbLegacyTagFactory
   ├── Libplctag*  adapters           (existing; now translate Status → AbStatus)   ← fallback
   └── Native*     adapters           (new, in each driver project)
          │
          ▼
   ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport   (new, pure managed)
     Eip/      EncapsulationClient — TCP framing, RegisterSession/UnRegisterSession,
               ListIdentity, SendRRData (UCMM), SendUnitData (connected)
     Cip/      CipPath (port segments from "1,0", ANSI symbolic 0x91, member/element
               segments 0x28/0x29/0x2A), CipService, CipGeneralStatus + extended status
     Cip/      ConnectionManager — Forward Open (0x54) / Large Forward Open (0x5B) /
               Forward Close (0x4E), sequence counts, connection-size negotiation,
               UCMM-only mode (Micro800)
     Cip/      RequestScheduler — per-connection queue, MSP coalescing, in-flight bound,
               per-request timeout + cancellation
     Logix/    Read Tag (0x4C) / Read Tag Fragmented (0x52) / Write Tag (0x4D) /
               Write Tag Fragmented (0x53); Logix type codes incl. struct handle (0xA0)
     Logix/    SymbolLister (Get Instance Attribute List 0x55 on class 0x6B, paged)
     Logix/    TemplateReader (Get Attribute List 0x03 + Read 0x4C on class 0x6C, chunked)
     Pccc/     Execute PCCC (0x4B, class 0x67) with requestor id; SLC/MicroLogix protected
               typed logical read/write (0x0F/0xA2, 0x0F/0xAA); PLC-5 typed read/write
               (0x0F/0x68, 0x0F/0x67); PCCC STS/EXT STS
     DeviceSessionPool — one session+connection per device key, ref-counted
```

### 3.1 Units and their contracts

- **`EncapsulationClient`**: owns one TCP socket. It sends and receives 24-byte-header encapsulation
  frames and correlates replies by sender context. It knows nothing about CIP. Testable against canned
  byte streams.
- **`CipPath`**: a pure encoder. It turns `"1,0"` (and `"1,0,2,10.0.0.5"` style routes) into port
  segments, and turns a Logix tag name such as `Program:P.Arr[3].Member.5` into symbolic, element and
  member segments. `AbCipTagPath` already parses names; `CipPath` encodes them. Golden-byte unit tests.
- **`ConnectionManager`**: opens and closes the connected session for a device.
  - **Connection size.** It honours `ConnectionSize` (the field libplctag drops today). Above 511
    bytes it uses Large Forward Open. If the target rejects a large open (status 0x01, extended 0x0109
    or 0x0111), it falls back to a regular Forward Open at the size the target advertises.
  - **Micro800** (`SupportsConnectedMessaging = false`) runs UCMM-only: no Forward Open.
- **`RequestScheduler`**: one queue per connection.
  - **Coalescing.** Requests that arrive within one scheduling turn are packed into an MSP until the
    next service would overflow the connection size. A reply is fanned back to each waiter with its own
    per-service status.
  - **Oversize requests.** A request too large for one frame goes through the fragmented service loop.
  - **Bounds.** At most one in-flight connected request per connection by default (`MaxInFlight`,
    default 1), the conservative choice for a production press.
- **`DeviceSessionPool`**: keyed by `(host, port, cipPath, familyAttribute)`. A runtime acquires a
  lease on first `InitializeAsync` and releases it on `Dispose`. The last release sends Forward Close
  and UnRegisterSession.

### 3.2 Native adapters (in the driver projects)

- **`NativeAbCipTagRuntime : IAbCipTagRuntime`**
  - **Reads.** `InitializeAsync` acquires the device lease and resolves the encoded path once.
    `ReadAsync` issues Read Tag with `ElementCount`. The reply's **CIP type code + struct handle** is
    kept with the buffer.
  - **Decoding.** `Decode*` reads the managed buffer, little-endian. Strings use the Logix
    `LEN(u32) + DATA[n]` layout with the stride taken from the reply. BOOL arrays are 32-bit packed.
  - **Size and status.** `GetSize()` is the real reply length, so the stride fallback in
    `ResolveArrayStride` stops being necessary. `GetStatus()` returns `AbStatus`.
  - **Writes.** `WriteAsync` issues Write Tag with the type taken from the last read, or from the
    authored type for atomics. A struct-typed write (STRING) needs the struct handle, so a write with
    no prior read performs one read first.
- **`NativeAbCipTagEnumerator`** reads controller-scope and `Program:X` listings through
  `SymbolLister`, and emits the concatenated entry bytes in the `@tags` layout.
- **`NativeAbCipTemplateReader`** emits the 14-byte header that libplctag synthesizes, then the raw
  template bytes (D8).
- **`NativeAbLegacyTagRuntime : IAbLegacyTagRuntime`**
  - **Addressing.** Encodes `AbLegacyAddress` (file type, file number, element, sub-element) into PCCC
    typed-logical addressing, or PLC-5 system addressing for `plc5`.
  - **Arrays.** Multi-element reads use the element count.
  - **Bits.** Bit-in-word writes stay at the driver level (`WriteBitInWordAsync`), as today.

### 3.3 The switch

- `AbCipDriverOptions.Transport` and `AbLegacyDriverOptions.Transport` take values
  `Native | Libplctag`, are serialized as **strings** (`JsonStringEnumConverter`, per the
  driver-enum-serialization trap), and are absent ⇒ the current default.
- `Ab*DriverFactoryExtensions.CreateInstance` picks the factory set from the option. Test-injected
  factories still win.
- The **probe** (`Ab*DriverProbe`, i.e. Test Connect) honours the same option. A probe that tests a
  different transport than the driver runs would be a lie.
- **AdminUI**: a `Transport` dropdown in the AbCip and AbLegacy driver-config modals under the `/raw`
  tree. It needs live `/run` verification on docker-dev, since there is no bUnit.
- Default sequence: `Libplctag` at merge, flipped per §8, and the fallback stays selectable.

## 4. Data flow (a poll read)

1. The driver poll calls `runtime.ReadAsync(ct)` on N runtimes for one device. The concurrency
   window is unchanged.
2. Each runtime enqueues a Read Tag request on the device's `RequestScheduler`.
3. The scheduler packs as many as fit into one MSP, sends it via SendUnitData on the shared
   connection, awaits the reply, splits it by service, and completes each waiter with
   `(status, typeCode, structHandle, payload)`.
4. The runtime stores the payload. The driver calls `Decode*` exactly as it does today.

The result: N round-trips through N libplctag handles become ⌈N / fit⌉ round-trips through one
connection. That is the performance win, with no driver change.

## 5. Error handling and status

- **`AbStatus`** (driver-owned) carries the values the mappers and probes use today: `Ok`,
  `Pending`, `Timeout`, `NotFound`, `NotAllowed`, `OutOfBounds`, `TooLarge`, `TooSmall`,
  `Unsupported`, `BadConnection`, `BadReply`, `Partial`, `Abort`, plus a new `TypeMismatch`.
- **CIP general status mapping (native):**
  - `0x00` ⇒ `Ok`
  - `0x04` / `0x05` (path segment / destination unknown) ⇒ `NotFound`
  - `0x06` ⇒ `Partial` (continue fragmenting)
  - `0x08` ⇒ `Unsupported`
  - `0x0F` ⇒ `NotAllowed`
  - `0x13` ⇒ `TooSmall`
  - `0x15` ⇒ `TooLarge`
  - `0xFF` ⇒ by extended status (`0x2105` ⇒ `OutOfBounds`, `0x2107` ⇒ `TypeMismatch`)
  - socket, encapsulation or timeout faults ⇒ `BadConnection` / `Timeout`
- **PCCC mapping.** STS/EXT STS map the same way: address-does-not-exist ⇒ `NotFound`, illegal
  command/format ⇒ `Unsupported`, and so on.
- **`AbTransportException(AbStatus, string detail)`** is thrown by both transports.
  `AbCipDriver.cs:1188`'s "the controller answered, so don't throttle the device" check
  (`IsReachableStatus`) keys on it. **That preserves 05/STAB-8 semantics.**
- **Type check, native only.** The reply's CIP type code is compared with the authored type.
  - A mismatch ⇒ `BadTypeMismatch`. This now also catches **same-width** mistakes (DINT authored,
    REAL returned) that `e3d6b76e` documents as undetectable.
  - ⚠️ **This is client-visible.** Tags authored wrong from the legacy Kepware export that read Good
    garbage under libplctag will read Bad under native. That is the fix working, but it must be
    in the release note and in `docs/drivers/AbCip.md`.
- **Press safety.**
  - **Reconnect.** A dropped connection is re-opened through the driver's existing `device.Backoff`,
    never in a tight loop.
  - **One open at a time.** `ConnectionManager` enforces at most one Forward Open in flight per
    device, plus a minimum interval between opens (default 1 s).
  - **Pinned by a test.** A regression test asserts a bounded Forward Open rate against a refusing
    target. ignitionoee recorded **~15 Forward Opens/s** at a press from a misconfigured client.
  - **Close.** Forward Close on dispose. The connection timeout multiplier and RPI are set so a dead
    client's connection expires on the PLC within seconds, not minutes.

## 6. Packaging

- New project `src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport/` and its test project
  `tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Ab.Transport.Tests/`, both added to
  `ZB.MOM.WW.OtOpcUa.slnx`.
- While the fallback exists, both drivers keep the `libplctag` package. Libplctag code is confined
  to the `Libplctag*.cs` adapters. After phase 2 no other file has `using libplctag`, and a
  source-scan guard test enforces that.
- On deletion (phase 6) the package, its `Directory.Packages.props` pin, the adapters and the
  `Libplctag*` create-param members (`LibplctagGateway`; `LibplctagPlcAttribute` is renamed
  `PlcAttribute`) go.

## 7. Testing

1. **Golden-frame unit tests** (`Ab.Transport.Tests`).
   - **Encode** every frame type: RegisterSession, Forward Open / Large / Close, MSP, Read / Write
     (Fragmented), 0x55 listing, template attribute reads, Execute PCCC.
   - **Decode** every reply type, including error replies.
   - **Fixtures** are byte captures taken once from libplctag talking to `ab_server` (pcap, stored as
     hex), plus the **existing real-press captures** in `CipSymbolObjectWireCaptureTests` and
     `CipTemplateObjectWireCaptureTests`.
   - ⚠️ A capture from libplctag ⇄ `ab_server` proves wire parity with libplctag, not correctness. The
     real-press captures are the independent reference (the #468 lesson: verify a fixture with the
     reference client).
2. **Scheduler and connection tests.** These run against an in-process fake encapsulation endpoint.
   They cover:
   - MSP packing boundaries at connection size −1, exact and +1
   - per-service error fan-out
   - cancellation mid-MSP
   - the Forward Open rate bound
   - large-open fallback
   - lease ref-counting and Forward Close on last release
3. **Integration suites run on both transports.** The existing `AbCip.IntegrationTests` and
   `AbLegacy.IntegrationTests` against `ab_server` (controllogix / compactlogix / micro800 / guardlogix;
   slc500 / micrologix / plc5) are parameterized over `Transport`. A native run must be green on every
   test the libplctag run is green on.
4. **Differential harness.** A `LiveIntegration`-category test, env-gated like the HistorianGateway
   live suite and skipped when the env vars are absent. It reads the same tag set through both
   transports and asserts equal raw buffers and equal decoded values:
   - against `ab_server`;
   - against a real press, **read-only**. It runs browse (`@tags` + every program listing), every
     template, and a sample of scalar, array, STRING and UDT-member reads.
5. **AdminUI `/run`.** Author `Transport: Native` on a docker-dev AbCip driver pointed at the
   `ab_server` fixture, deploy, read, then flip it back.

## 8. Phased delivery and gates

| Phase | Delivers | Gate |
|---|---|---|
| **P0 Seam** | `AbStatus` + `AbTransportException`; mappers, probes, `AbCipDriver` and `AbLegacyDriver` decoupled from libplctag types; the 9 unit-test files that build `LibPlcTagException` / `Status` fakes (`AbCipDriverWriteTests`, `AbCipDriverProbeTests`, `AbCipDriverReadTests`, `AbCipStatusMapperTests`, `AbCipDriverCodeReviewRegressionTests`, `AbCipUnknownTagReadTests`, `AbLegacyEvictOnFailureTests`, `AbLegacyReadWriteTests`, `AbLegacyHostAndStatusTests`) moved onto `AbStatus` / `AbTransportException`, with the libplctag translation itself tested in the adapters; the `Transport` option (string enum) + factory selection, including a `--transport` flag on `otopcua-abcip-cli` / `otopcua-ablegacy-cli` (whose help text names libplctag); the source-scan guard | Existing suites green; behaviour identical (default `Libplctag`) |
| **P1 Transport core** | Encapsulation, `CipPath`, `ConnectionManager`, `RequestScheduler` (MSP + fragmentation), `DeviceSessionPool` | Golden-frame + scheduler tests |
| **P2 AbCip native** | `NativeAbCipTagRuntime`, enumerator, template reader; CIP type check | AbCip integration suite green on both transports vs `ab_server` |
| **P3 AbLegacy native** | PCCC layer + `NativeAbLegacyTagRuntime` | AbLegacy integration suite green on both transports vs `ab_server` (slc500 / micrologix / plc5) |
| **P4 Switch surface** | AdminUI dropdown, probe honours the switch, docs (`AbCip.md`, `AbLegacy.md`, fixture docs) | docker-dev `/run` |
| **P5 Real-hardware gate + flip** | Differential run against the CR presses read-only (VPN from the Mac); a perf comparison (round-trips + p50/p95 poll latency for one press's tag set, native vs libplctag); then **default ⇒ `Native` for both drivers** | AbCip: diff clean on browse / templates / reads, no worse latency, Forward Open count during the run = 1. AbLegacy: simulator proof only, **recorded as a known gap** |
| **P6 Delete libplctag** *(conditional)* | Remove the package, adapters and `Libplctag*` members | **Blocked until a bench controller** proves writes (CompactLogix or Micro800) and PCCC (SLC / MicroLogix). Until then libplctag stays selectable |

### Real-hardware constraints (from `~/Desktop/ignitionoee`)

- **Reachable targets:** 20 × CompactLogix **5069-L320ERS2 rev 34.11** presses at CR
  (Z-35162..Z-35181; Z-35162 = `10.209.70.160:44818`), reachable from the Mac over VPN. This is
  the same rig `cc876bdf` verified against.
- **They are production machines and strictly read-only.** P5 performs **no writes**, opens a
  single connection per press, and touches one press at a time.
- **No legacy PCCC hardware is reachable.** The CR presses were confirmed non-MicroLogix by
  ListIdentity. The Delmia inventory lists PLC-5/04s at Warsaw with no addresses, and as far as I
  know a PLC-5/04 has no built-in Ethernet port. There is no ControlLogix and no Micro800 either. So
  routed paths beyond `1,0` and Micro800 UCMM-only mode are simulator-proven only.

## 9. Known gaps and risks

- **AbCip writes are simulator-proven only.** Mitigation: Write Tag shares framing with the
  read-proven path. The risk sits in the encoding details (STRING struct handle, type codes), which
  have golden tests and run against `ab_server`. libplctag stays selectable.
- **AbLegacy is simulator-proven only**, against `ab_server`'s PCCC emulation, which shares
  libplctag's view of the protocol. This is the weakest evidence in the plan, stated plainly.
- **`ab_server` patches** (`0001` symbol listing, `0002` CIP 0x04 for not-found) mean the simulator is
  already partly ours. A native/libplctag disagreement against it must be checked against the real
  press before blaming either side.
- **The same-width type check changes client-visible quality** (§5).
- **Behaviour libplctag hides that we now own:** connection keep-alive, sequence-count wrap, the
  session-registration lifetime, and the TCP reconnect after an idle teardown by a NAT or firewall.
  Each one gets a scheduler/connection test.