218 lines
10 KiB
Markdown
218 lines
10 KiB
Markdown
# TwinCAT test fixture
|
||
|
||
Coverage map + gap inventory for the Beckhoff TwinCAT ADS driver.
|
||
|
||
**TL;DR:** Integration-test scaffolding lives at
|
||
`tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/` (task #221).
|
||
`TwinCATXarFixture` probes TCP 48898 on an operator-supplied VM; three
|
||
smoke tests (read / write / native notification) run end-to-end through
|
||
the real ADS stack when the VM is reachable, skip cleanly otherwise.
|
||
**Remaining operational work**: stand up a TwinCAT 3 XAR runtime in a
|
||
Hyper-V VM, author the `.tsproj` project documented at
|
||
`TwinCatProject/README.md`, rotate the 7-day trial license (or buy a
|
||
paid runtime). Unit tests via `FakeTwinCATClient` still carry the
|
||
exhaustive contract coverage.
|
||
|
||
TwinCAT is the only driver outside Galaxy that uses **native
|
||
notifications** (no polling) for `ISubscribable`, and the fake exposes a
|
||
fire-event harness so notification routing is contract-tested rigorously
|
||
at the unit layer.
|
||
|
||
## What the fixture is
|
||
|
||
**Integration layer** (task #221, scaffolded):
|
||
`tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/` —
|
||
`TwinCATXarFixture` TCP-probes ADS port 48898 on the host specified by
|
||
`TWINCAT_TARGET_HOST` + requires `TWINCAT_TARGET_NETID` (AmsNetId of the
|
||
VM). No fixture-owned lifecycle — XAR can't run in Docker because it
|
||
bypasses the Windows kernel scheduler, so the VM stays
|
||
operator-managed. `TwinCatProject/README.md` documents the required
|
||
`.tsproj` project state; the file itself lands once the XAR VM is up +
|
||
the project is authored. Three smoke tests:
|
||
`Driver_reads_seeded_DINT_through_real_ADS`,
|
||
`Driver_write_then_read_round_trip_on_scratch_REAL`, and
|
||
`Driver_subscribe_receives_native_ADS_notifications_on_counter_changes`
|
||
— all skip cleanly via `[TwinCATFact]` when the runtime isn't
|
||
reachable.
|
||
|
||
**Unit layer**: `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/` is
|
||
still the primary coverage. `FakeTwinCATClient` also fakes the
|
||
`AddDeviceNotification` flow so tests can trigger callbacks without a
|
||
running runtime.
|
||
|
||
## What it actually covers
|
||
|
||
### Integration (XAR VM, task #221 — code scaffolded, needs VM + project)
|
||
|
||
- `TwinCAT3SmokeTests.Driver_reads_seeded_DINT_through_real_ADS` — real AMS
|
||
handshake + ADS read of `GVL_Fixture.nCounter` (seeded at 1234, MAIN
|
||
increments each cycle)
|
||
- `TwinCAT3SmokeTests.Driver_write_then_read_round_trip_on_scratch_REAL` —
|
||
real ADS write + read on `GVL_Fixture.rSetpoint`
|
||
- `TwinCAT3SmokeTests.Driver_subscribe_receives_native_ADS_notifications_on_counter_changes`
|
||
— real `AddDeviceNotification` against the cycle-incrementing counter;
|
||
observes `OnDataChange` firing within 3 s of subscribe
|
||
|
||
All three gated on `TWINCAT_TARGET_HOST` + `TWINCAT_TARGET_NETID` env
|
||
vars; skip cleanly via `[TwinCATFact]` when the VM isn't reachable or
|
||
vars are unset.
|
||
|
||
### Unit
|
||
|
||
- `TwinCATAmsAddressTests` — `ads://<netId>:<port>` parsing + routing
|
||
- `TwinCATCapabilityTests` — data-type mapping (primitives + declared UDTs),
|
||
read-only classification
|
||
- `TwinCATReadWriteTests` — read + write through the fake, status mapping
|
||
- `TwinCATSymbolPathTests` — symbol-path routing for nested struct members
|
||
- `TwinCATSymbolBrowserTests` — `ITagDiscovery.DiscoverAsync` via
|
||
`ReadSymbolsAsync` (#188) + system-symbol filtering
|
||
- `TwinCATNativeNotificationTests` — `AddDeviceNotification` (#189)
|
||
registration, callback-delivery-to-`OnDataChange` wiring, unregister on
|
||
unsubscribe
|
||
- `TwinCATDriverTests` — `IDriver` lifecycle
|
||
|
||
Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
|
||
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
|
||
`IPerCallHostResolver`.
|
||
|
||
## What it does NOT cover
|
||
|
||
### 1. AMS / ADS wire traffic
|
||
|
||
No real AMS router frame is exchanged. Beckhoff's `TwinCAT.Ads` NuGet (their
|
||
own .NET SDK, not libplctag-style OSS) has no in-process fake; tests stub
|
||
the `ITwinCATClient` abstraction above it.
|
||
|
||
### 2. Multi-route AMS
|
||
|
||
ADS supports chained routes (`<localNetId> → <routerNetId> → <targetNetId>`)
|
||
for PLCs behind an EC master / IPC gateway. Parse coverage exists; wire-path
|
||
coverage doesn't.
|
||
|
||
### 3. Notification reliability under jitter
|
||
|
||
`AddDeviceNotification` delivers at the runtime's cycle boundary; under high
|
||
CPU load or network jitter real notifications can coalesce. The fake fires
|
||
one callback per test invocation — real callback-coalescing behavior is
|
||
untested.
|
||
|
||
### 4. TC2 vs TC3 variant handling
|
||
|
||
TwinCAT 2 (ADS v1) and TwinCAT 3 (ADS v2) have subtly different
|
||
`GetSymbolInfoByName` semantics + symbol-table layouts. Driver targets TC3;
|
||
TC2 compatibility is not exercised.
|
||
|
||
### 5. Cycle-time alignment for `ISubscribable`
|
||
|
||
Native ADS notifications fire on the PLC cycle boundary. The fake test
|
||
harness assumes notifications fire on a timer the test controls;
|
||
cycle-aligned firing under real PLC control is not verified.
|
||
|
||
### 6. Alarms / history
|
||
|
||
Driver doesn't implement `IAlarmSource` or `IHistoryProvider` — not in
|
||
scope for this driver family. TwinCAT 3's TcEventLogger could theoretically
|
||
back an `IAlarmSource`, but shipping that is a separate feature.
|
||
|
||
## When to trust TwinCAT tests, when to reach for a rig
|
||
|
||
| Question | Unit tests | Real TwinCAT runtime |
|
||
| --- | --- | --- |
|
||
| "Does the AMS address parser accept X?" | yes | - |
|
||
| "Does notification → `OnDataChange` wire correctly?" | yes (contract) | yes |
|
||
| "Does symbol browsing filter TwinCAT internals?" | yes | yes |
|
||
| "Does a real ADS read return correct bytes?" | no | yes (required) |
|
||
| "Do notifications coalesce under load?" | no | yes (required) |
|
||
| "Does a TC2 PLC work the same as TC3?" | no | yes (required) |
|
||
|
||
## Performance
|
||
|
||
PR 2.1 (Sum-read / Sum-write, IndexGroup `0xF080..0xF084`) replaced the per-tag
|
||
`ReadValueAsync` loop in `TwinCATDriver.ReadAsync` / `WriteAsync` with a
|
||
bucketed bulk dispatch — N tags addressed against the same device flow through a
|
||
single ADS sum-command round-trip via `SumInstancePathAnyTypeRead` (read) and
|
||
`SumWriteBySymbolPath` (write). Whole-array tags + bit-extracted BOOL tags
|
||
remain on the per-tag fallback path because the sum surface only marshals
|
||
scalars and bit-RMW writes need the per-parent serialisation lock.
|
||
|
||
**Baseline → Sum-command delta** (dev box, 1000 × DINT, XAR VM over LAN):
|
||
|
||
| Path | Round-trips | Wall-clock |
|
||
| --- | --- | --- |
|
||
| Per-tag loop (pre-PR 2.1) | 1000 | ~5–8 s |
|
||
| Sum-command bulk (PR 2.1) | 1 | ~250–600 ms |
|
||
| Ratio | — | ≥ 10× typical, ≥ 5× CI floor |
|
||
|
||
The perf-tier test
|
||
`TwinCATSumCommandPerfTests.Driver_sum_read_1000_tags_beats_loop_baseline_by_5x`
|
||
asserts the ratio with a conservative 5× lower bound that survives noisy CI /
|
||
VM scheduling. It is gated behind both `TWINCAT_TARGET_NETID` (XAR reachable)
|
||
and `TWINCAT_PERF=1` (operator opt-in) — perf runs aren't part of the default
|
||
integration pass because they hit the wire heavily.
|
||
|
||
The required fixture state (1000-DINT GVL + churn POU) is documented in
|
||
`TwinCatProject/README.md §Performance scenarios`; XAE-form sources land at
|
||
`TwinCatProject/PLC/GVLs/GVL_Perf.TcGVL` + `TwinCatProject/PLC/POUs/FB_PerfChurn.TcPOU`.
|
||
|
||
### Handle caching (PR 2.2)
|
||
|
||
Per-tag reads / writes route through an in-process ADS variable-handle cache.
|
||
The first read of a symbol resolves a handle via `CreateVariableHandleAsync`;
|
||
subsequent reads / writes of the same symbol issue against the cached handle.
|
||
On the wire this trades a multi-byte symbolic path (`GVL_Perf.aTags[742]` =
|
||
20+ bytes) for a 4-byte handle, and the device server skips name resolution
|
||
on every subsequent op. Cache lifetime is process-scoped; entries are evicted
|
||
on `AdsErrorCode.DeviceSymbolVersionInvalid` (with one retry against a fresh
|
||
handle), wiped on reconnect (handles are per-AMS-session), and deleted
|
||
best-effort on driver disposal.
|
||
|
||
`TwinCATHandleCachePerfTests.Driver_handle_cache_avoids_repeat_symbol_resolution`
|
||
asserts the contract on real XAR by reading 50 symbols twice and verifying
|
||
the second pass issues zero new `CreateVariableHandleAsync` calls. It runs
|
||
under the standard `[TwinCATFact]` gate (XAR reachable; no `TWINCAT_PERF`
|
||
opt-in needed because 50 symbols is cheap).
|
||
|
||
**Staleness caveat**: handles can go stale after a TwinCAT online change
|
||
(POU edit + activate). Until PR 2.3 ships the proactive Symbol-Version
|
||
invalidation listener, the safety net is twofold: (1) the
|
||
`DeviceSymbolVersionInvalid` evict-and-retry path catches cases where the
|
||
descriptor moves but the symbol survives, and (2) operators can call
|
||
`ITwinCATClient.FlushOptionalCachesAsync` manually after a known online
|
||
change to wipe the cache without forcing a full reconnect. The bulk
|
||
Sum-read / Sum-write path remains on symbolic paths in PR 2.2 (the bulk
|
||
path's per-call symbol resolution is already amortised across N tags;
|
||
the perf delta vs. handle-batched bulk is marginal — tracked as a
|
||
follow-up for the Phase-2 perf sweep).
|
||
|
||
## Follow-up candidates
|
||
|
||
1. **XAR VM live-population** — scaffolding is in place (this PR); the
|
||
remaining work is operational: stand up the Hyper-V VM, install XAR,
|
||
author the `.tsproj` per `TwinCatProject/README.md`, configure the
|
||
bilateral ADS route, set `TWINCAT_TARGET_HOST` + `TWINCAT_TARGET_NETID`
|
||
on the dev box. Then the three smoke tests transition skip → pass.
|
||
Tracked as #221.
|
||
2. **License-rotation automation** — XAR's 7-day trial expires on
|
||
schedule. Either automate `TcActivate.exe /reactivate` via a
|
||
scheduled task on the VM (not officially supported; reportedly works
|
||
for some TC3 builds), or buy a paid runtime license (~$1k one-time
|
||
per runtime per CPU) to kill the rotation. The doc at
|
||
`TwinCatProject/README.md` §License rotation walks through both.
|
||
3. **Lab rig** — cheapest IPC (CX7000 / CX9020) on a dedicated network;
|
||
the only route that covers TC2 + real EtherCAT I/O timing + cycle
|
||
jitter under CPU load.
|
||
|
||
## Key fixture / config files
|
||
|
||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCATXarFixture.cs`
|
||
— TCP probe + skip-attributes + env-var parsing
|
||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCAT3SmokeTests.cs`
|
||
— three wire-level smoke tests
|
||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.IntegrationTests/TwinCatProject/README.md`
|
||
— project spec + VM setup + license-rotation notes
|
||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/FakeTwinCATClient.cs` —
|
||
in-process fake with the notification-fire harness used by
|
||
`TwinCATNativeNotificationTests`
|
||
- `src/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT/TwinCATDriver.cs` — ctor takes
|
||
`ITwinCATClientFactory`
|