lmxopcua/docs/drivers/TwinCAT-Test-Fixture.md

# TwinCAT test fixture

Coverage map + gap inventory for the Beckhoff TwinCAT ADS driver.

**TL;DR:** Integration-test scaffolding lives at
`tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/` (task #221).
`TwinCATXarFixture` probes TCP 48898 on an operator-supplied VM; three
smoke tests (read / write / native notification) run end-to-end through
the real ADS stack when the VM is reachable, skip cleanly otherwise.
**Remaining operational work**: stand up a TwinCAT 3 XAR runtime in a
Hyper-V VM, author the `.tsproj` project documented at
`TwinCatProject/README.md`, rotate the 7-day trial license (or buy a
paid runtime). Unit tests via `FakeTwinCATClient` still carry the
exhaustive contract coverage.

TwinCAT is the only driver outside Galaxy that uses **native
notifications** (no polling) for `ISubscribable`, and the fake exposes a
fire-event harness so notification routing is contract-tested rigorously
at the unit layer.

## What the fixture is

**Integration layer** (task #221, scaffolded):
`tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/` —
`TwinCATXarFixture` TCP-probes ADS port 48898 on the host specified by
`TWINCAT_TARGET_HOST` + requires `TWINCAT_TARGET_NETID` (AmsNetId of the
VM). No fixture-owned lifecycle — XAR can't run in Docker because it
bypasses the Windows kernel scheduler, so the VM stays
operator-managed. `TwinCatProject/README.md` documents the required
`.tsproj` project state; the file itself lands once the XAR VM is up +
the project is authored. Three smoke tests:
`Driver_reads_seeded_DINT_through_real_ADS`,
`Driver_write_then_read_round_trip_on_scratch_REAL`, and
`Driver_subscribe_receives_native_ADS_notifications_on_counter_changes`
— all skip cleanly via `[TwinCATFact]` when the runtime isn't
reachable.

**Unit layer**: `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/` is
still the primary coverage. `FakeTwinCATClient` also fakes the
`AddDeviceNotification` flow so tests can trigger callbacks without a
running runtime.

## What it actually covers

### Integration (XAR VM, task #221 — code scaffolded, needs VM + project)

- `TwinCAT3SmokeTests.Driver_reads_seeded_DINT_through_real_ADS` — real AMS
  handshake + ADS read of `GVL_Fixture.nCounter` (seeded at 1234, MAIN
  increments each cycle)
- `TwinCAT3SmokeTests.Driver_write_then_read_round_trip_on_scratch_REAL` —
  real ADS write + read on `GVL_Fixture.rSetpoint`
- `TwinCAT3SmokeTests.Driver_subscribe_receives_native_ADS_notifications_on_counter_changes`
  — real `AddDeviceNotification` against the cycle-incrementing counter;
  observes `OnDataChange` firing within 3 s of subscribe

All three gated on `TWINCAT_TARGET_HOST` + `TWINCAT_TARGET_NETID` env
vars; skip cleanly via `[TwinCATFact]` when the VM isn't reachable or
vars are unset.

PR 4.1 / #315 adds `TwinCATUdtBrowseTests.Driver_browses_UDT_tree_and_flattens_to_atomic_leaves`
which exercises `TwinCATDriver.DiscoverAsync` end-to-end against the
`GVL_Plant` UDT fixture. Asserts the discovery surface emits one OPC UA
variable per atomic leaf and folds `aAlarmRecords[1..2000]` into a
single `IsArrayRoot` placeholder when the element count exceeds the
default 1024-element cap (UDT per-member coverage; see
`TwinCatProject/README.md §Complex hierarchy` for the supporting DUTs).

### Unit

- `TwinCATAmsAddressTests` — `ads://<netId>:<port>` parsing + routing
- `TwinCATCapabilityTests` — data-type mapping (primitives + declared UDTs),
  read-only classification
- `TwinCATReadWriteTests` — read + write through the fake, status mapping
- `TwinCATSymbolPathTests` — symbol-path routing for nested struct members
- `TwinCATSymbolBrowserTests` — `ITagDiscovery.DiscoverAsync` via
  `ReadSymbolsAsync` (#188) + system-symbol filtering
- `TwinCATTypeWalkerTests` — PR 4.1 / #315 nested-UDT decomposition:
  atomic / single-level struct / nested struct / array-of-atomic
  (in / over `MaxArrayExpansion`) / array-of-struct / alias chain /
  pointer skip / self-referencing struct depth-cap / per-leaf
  `MaxArrayExpansion` honored / ReadOnly propagation. Stub `IDataType`
  / `IStructType` / `IArrayType` / `IMember` / `IDimensionCollection`
  trees built in-test so the walker is exercised without
  `Beckhoff.TwinCAT.Ads`-internal ctors.
- `TwinCATNativeNotificationTests` — `AddDeviceNotification` (#189)
  registration, callback-delivery-to-`OnDataChange` wiring, unregister on
  unsubscribe
- `TwinCATDriverTests` — `IDriver` lifecycle

Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
`IPerCallHostResolver`.

## What it does NOT cover

### 1. AMS / ADS wire traffic

No real AMS router frame is exchanged. Beckhoff's `TwinCAT.Ads` NuGet (their
own .NET SDK, not libplctag-style OSS) has no in-process fake; tests stub
the `ITwinCATClient` abstraction above it.

### 2. Multi-route AMS

ADS supports chained routes (`<localNetId> → <routerNetId> → <targetNetId>`)
for PLCs behind an EC master / IPC gateway. Parse coverage exists; wire-path
coverage doesn't.

### 3. Notification reliability under jitter

`AddDeviceNotification` delivers at the runtime's cycle boundary; under high
CPU load or network jitter real notifications can coalesce. The fake fires
one callback per test invocation — real callback-coalescing behavior is
untested.

PR 3.1 (#313) makes the per-tag `MaxDelay` configurable via
`TwinCATTagDefinition.MaxDelayMs` — the runtime can buffer changes for up to
that many milliseconds before dispatch, deliberately coalescing bursty
high-frequency signals so the OPC UA queue downstream doesn't flood. Default
`null` / `0` preserves the pre-PR-3.1 "fire ASAP" behaviour.
`TwinCATMaxDelayTests.Driver_coalesces_notifications_at_max_delay` exercises
the wire-side coalescer end-to-end against `GVL_Fixture.nCounter`; the unit
suite (`TwinCATNativeNotificationTests`) covers the plumbing contract via
the `FakeTwinCATClient.FakeNotification.MaxDelayMs` capture.

### 4. TC2 vs TC3 variant handling

TwinCAT 2 (ADS v1) and TwinCAT 3 (ADS v2) have subtly different
`GetSymbolInfoByName` semantics + symbol-table layouts. Driver targets TC3;
TC2 compatibility is not exercised.

### 5. Cycle-time alignment for `ISubscribable`

Native ADS notifications fire on the PLC cycle boundary. The fake test
harness assumes notifications fire on a timer the test controls;
cycle-aligned firing under real PLC control is not verified.

### 6. Alarms / history

Driver doesn't implement `IAlarmSource` or `IHistoryProvider` — not in
scope for this driver family. TwinCAT 3's TcEventLogger could theoretically
back an `IAlarmSource`, but shipping that is a separate feature.

## When to trust TwinCAT tests, when to reach for a rig

| Question | Unit tests | Real TwinCAT runtime |
| --- | --- | --- |
| "Does the AMS address parser accept X?" | yes | - |
| "Does notification → `OnDataChange` wire correctly?" | yes (contract) | yes |
| "Does symbol browsing filter TwinCAT internals?" | yes | yes |
| "Does a real ADS read return correct bytes?" | no | yes (required) |
| "Do notifications coalesce under load?" | no | yes (required) |
| "Does a TC2 PLC work the same as TC3?" | no | yes (required) |

## Performance

PR 2.1 (Sum-read / Sum-write, IndexGroup `0xF080..0xF084`) replaced the per-tag
`ReadValueAsync` loop in `TwinCATDriver.ReadAsync` / `WriteAsync` with a
bucketed bulk dispatch — N tags addressed against the same device flow through a
single ADS sum-command round-trip via `SumInstancePathAnyTypeRead` (read) and
`SumWriteBySymbolPath` (write). Whole-array tags + bit-extracted BOOL tags
remain on the per-tag fallback path because the sum surface only marshals
scalars and bit-RMW writes need the per-parent serialisation lock.

**Baseline → Sum-command delta** (dev box, 1000 × DINT, XAR VM over LAN):

| Path | Round-trips | Wall-clock |
| --- | --- | --- |
| Per-tag loop (pre-PR 2.1) | 1000 | ~5–8 s |
| Sum-command bulk (PR 2.1) | 1 | ~250–600 ms |
| Ratio | — | ≥ 10× typical, ≥ 5× CI floor |

The perf-tier test
`TwinCATSumCommandPerfTests.Driver_sum_read_1000_tags_beats_loop_baseline_by_5x`
asserts the ratio with a conservative 5× lower bound that survives noisy CI /
VM scheduling. It is gated behind both `TWINCAT_TARGET_NETID` (XAR reachable)
and `TWINCAT_PERF=1` (operator opt-in) — perf runs aren't part of the default
integration pass because they hit the wire heavily.

The required fixture state (1000-DINT GVL + churn POU) is documented in
`TwinCatProject/README.md §Performance scenarios`; XAE-form sources land at
`TwinCatProject/PLC/GVLs/GVL_Perf.TcGVL` + `TwinCatProject/PLC/POUs/FB_PerfChurn.TcPOU`.

### Handle caching (PR 2.2)

Per-tag reads / writes route through an in-process ADS variable-handle cache.
The first read of a symbol resolves a handle via `CreateVariableHandleAsync`;
subsequent reads / writes of the same symbol issue against the cached handle.
On the wire this trades a multi-byte symbolic path (`GVL_Perf.aTags[742]` =
20+ bytes) for a 4-byte handle, and the device server skips name resolution
on every subsequent op. Cache lifetime is process-scoped; entries are evicted
on `AdsErrorCode.DeviceSymbolVersionInvalid` (with one retry against a fresh
handle), wiped on reconnect (handles are per-AMS-session), and deleted
best-effort on driver disposal.

`TwinCATHandleCachePerfTests.Driver_handle_cache_avoids_repeat_symbol_resolution`
asserts the contract on real XAR by reading 50 symbols twice and verifying
the second pass issues zero new `CreateVariableHandleAsync` calls. It runs
under the standard `[TwinCATFact]` gate (XAR reachable; no `TWINCAT_PERF`
opt-in needed because 50 symbols is cheap).

**Self-invalidation (PR 2.3)**: handle cache is now self-invalidating on
TwinCAT online changes. `AdsTwinCATClient` registers an
`AdsSymbolVersionChanged` event listener (Beckhoff's high-level wrapper
around the SymbolVersion ADS notification, IndexGroup `0xF008`) on connect;
when the PLC's symbol-version counter increments — full re-init after a
download / activate-config — the listener fires and wipes the handle cache
proactively. Three-layered defence in depth: (1) proactive listener
preempts the next read entirely on full re-inits, (2) the
`DeviceSymbolVersionInvalid` evict-and-retry path from PR 2.2 catches the
narrower "symbol survives but its descriptor moved" race, and (3)
operators can still call `ITwinCATClient.FlushOptionalCachesAsync` manually
for the truly-paranoid case. The bulk Sum-read / Sum-write path remains
on symbolic paths in PR 2.2 (the bulk path's per-call symbol resolution
is already amortised across N tags; the perf delta vs. handle-batched
bulk is marginal — tracked as a follow-up for the Phase-2 perf sweep).

## Diagnostics

PR 3.2 (#314) augments the probe loop. On every successful tick (post `ReadStateAsync`)
the driver also reads four well-known system symbols off the AMS target and stashes
them on `DeviceState.LastDiagnostics` as a `TwinCATDeviceDiagnostics` record. The same
snapshot is folded into `DriverHealth.Diagnostics` so the cross-driver
`driver-diagnostics` RPC (added for Modbus, task #154) renders TwinCAT cycle-time /
jitter / online-change counters next to its peers without a per-driver special-case.

| Symbol | Type | Diagnostic key | Notes |
| --- | --- | --- | --- |
| `TwinCAT_SystemInfoVarList._AppInfo.AppName` | `STRING(80)` | (record only) | Running PLC project name, e.g. `"Plc1"` |
| `TwinCAT_SystemInfoVarList._AppInfo.OnlineChangeCnt` | `UDINT` | `TwinCAT.OnlineChangeCnt` | Increments on every accepted online change; informational |
| `TwinCAT_SystemInfoVarList._TaskInfo[1].CycleTime` | `UDINT` (100 ns ticks) | `TwinCAT.CycleTimeMs` | Configured task period after `÷10000` ms conversion |
| `TwinCAT_SystemInfoVarList._TaskInfo[1].LastExecTime` | `UDINT` (100 ns ticks) | `TwinCAT.LastExecTimeMs` | Wall-clock duration of the last task tick |
| (computed) | `double` | `TwinCAT.JitterMs` | `LastExecTimeMs - CycleTimeMs`; positive = overrun |
| (computed) | `long` | `TwinCAT.OnlineChangeIncrements` | Cumulative deltas observed since the driver started; only emitted once non-zero |

Each individual read is wrapped in best-effort try/catch. A runtime that doesn't
expose `_TaskInfo[1]` (older TwinCAT 2 builds, some soft-PLC implementations) still
produces a partial snapshot; the missing fields fall back to the previous tick's value
or the type default for the first probe tick. Wholesale failure of all four reads
leaves the previous snapshot in place and the next tick retries.

Single-device deployments produce flat keys (`TwinCAT.CycleTimeMs`); multi-device
deployments prefix with the AMS host address (`TwinCAT.<hostAddress>.CycleTimeMs`)
so the readout is unambiguous when one driver instance owns multiple AMS targets.

Wire-level coverage lives in
`TwinCATDiagnosticsIntegrationTests.Probe_loop_surfaces_cycle_time_and_online_change_count`
(asserts `CycleTimeMs > 0` + `OnlineChangeCnt >= 0` within one probe interval against a
reachable XAR runtime). Unit-level coverage of the dictionary shape, the per-symbol
try/catch, and the multi-device prefixing lives in `TwinCATDeviceDiagnosticsTests` —
the `FakeTwinCATClient.SetSystemSymbolValue` helper drives the surface deterministically.

## Follow-up candidates

1. **XAR VM live-population** — scaffolding is in place (this PR); the
   remaining work is operational: stand up the Hyper-V VM, install XAR,
   author the `.tsproj` per `TwinCatProject/README.md`, configure the
   bilateral ADS route, set `TWINCAT_TARGET_HOST` + `TWINCAT_TARGET_NETID`
   on the dev box. Then the three smoke tests transition skip → pass.
   Tracked as #221.
2. **License-rotation automation** — XAR's 7-day trial expires on
   schedule. Either automate `TcActivate.exe /reactivate` via a
   scheduled task on the VM (not officially supported; reportedly works
   for some TC3 builds), or buy a paid runtime license (~$1k one-time
   per runtime per CPU) to kill the rotation. The doc at
   `TwinCatProject/README.md` §License rotation walks through both.
3. **Lab rig** — cheapest IPC (CX7000 / CX9020) on a dedicated network;
   the only route that covers TC2 + real EtherCAT I/O timing + cycle
   jitter under CPU load.

## Key fixture / config files

- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCATXarFixture.cs`
  — TCP probe + skip-attributes + env-var parsing
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCAT3SmokeTests.cs`
  — three wire-level smoke tests
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCatProject/README.md`
  — project spec + VM setup + license-rotation notes
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/FakeTwinCATClient.cs` —
  in-process fake with the notification-fire harness used by
  `TwinCATNativeNotificationTests`
- `src/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT/TwinCATDriver.cs` — ctor takes
  `ITwinCATClientFactory`