S7 integration fixture — python-snap7 server closes the wire-level coverage gap (#216) + per-driver fixture coverage docs for every driver in the fleet. Closes #216. Two shipments in one PR because the docs landed as I surveyed each driver's fixture + the S7 work is the first wire-level-gap closer pulled from that survey.
S7 integration — AbCip/Modbus already have real-simulator integration suites; S7 had zero wire-level coverage despite being a Tier-A driver (all unit tests mocked IS7Client). Picked python-snap7's `snap7.server.Server` over raw Snap7 C library because `pip install` beats per-OS binary-pin maintenance, the package ships a Python __main__ shim that mirrors our existing pymodbus serve.ps1 + *.json pattern structurally, and the python-snap7 project is actively maintained. New project `tests/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests/` with four moving parts: (a) `Snap7ServerFixture` — collection-scoped TCP probe on `localhost:1102` that sets `SkipReason` when the simulator's not running, matching the `ModbusSimulatorFixture` shape one directory over (same S7_SIM_ENDPOINT env var override convention for pointing at a real S7 CPU on port 102); (b) `PythonSnap7/` — `serve.ps1` wrapper + `server.py` shim + `s7_1500.json` seed profile + `README.md` documenting install / run / known limitations; (c) `S7_1500/S7_1500Profile.cs` — driver-side `S7DriverOptions` whose tag addresses map 1:1 to the JSON profile's seed offsets (DB1.DBW0 u16, DB1.DBW10 i16, DB1.DBD20 i32, DB1.DBD30 f32, DB1.DBX50.3 bool, DB1.DBW100 scratch); (d) `S7_1500SmokeTests` — three tests proving typed reads + write-then-read round-trip work through real S7netplus + real ISO-on-TCP + real snap7 server. Picked port 1102 default instead of S7-standard 102 because 102 is privileged on Linux + triggers Windows Firewall prompt; S7netplus 0.20 has a 5-arg `Plc(CpuType, host, port, rack, slot)` ctor that lets the driver honour `S7DriverOptions.Port`, but the existing driver code called the 4-arg overload + silently hardcoded 102. One-line driver fix (S7Driver.cs:87) threads `_options.Port` through — the S7 unit suite (58/58) still passes unchanged because every unit test uses a fake IS7Client that never sees the real ctor. Server seed-type matrix in `server.py` covers u8 / i8 / u16 / i16 / u32 / i32 / f32 / bool-with-bit / ascii (S7 STRING with max_len header). register_area takes the SrvArea enum value, not the string name — a 15-minute debug after the first test run caught that; documented inline. Per-driver test-fixture coverage docs — eight new files in `docs/drivers/` laying out what each driver's harness actually benchmarks vs. what's trusted from field deployments. Pattern mirrors the AbServer-Test-Fixture.md doc that shipped earlier in this arc: TL;DR → What the fixture is → What it actually covers → What it does NOT cover → When-to-trust table → Follow-up candidates → Key files. Ugly truth the survey made visible: Galaxy + Modbus + (now) S7 + AB CIP have real wire-level coverage; AB Legacy / TwinCAT / FOCAS / OpcUaClient are still contract-only because their libraries ship no fake + no open-source simulator exists (AB Legacy PCCC), no public simulator exists (FOCAS), the vendor SDK has no in-process fake (TwinCAT/ADS.NET), or the test wiring just hasn't happened yet (OpcUaClient could trivially loopback against this repo's own server — flagged as #215). Each doc names the specific follow-up route: Snap7 server for S7 (done), TwinCAT 3 developer-runtime auto-restart for TwinCAT, Tier-C out-of-process Host for FOCAS, lab rigs for AB Legacy + hardware-gated bits of the others. `docs/drivers/README.md` gains a coverage-map section linking all eight. Tracking tasks #215-#222 filed for each PR-able follow-up. Build clean (driver + integration project + docs); S7.Tests 58/58 (unchanged); S7.IntegrationTests 3/3 (new, verified end-to-end against a live python-snap7 server: `driver_reads_seeded_u16_through_real_S7comm`, `driver_reads_seeded_typed_batch`, `driver_write_then_read_round_trip_on_scratch_word`). Next fixture follow-up is #215 (OpcUaClient loopback against own server) — highest ROI of the remaining set, zero external deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
97
docs/drivers/AbLegacy-Test-Fixture.md
Normal file
97
docs/drivers/AbLegacy-Test-Fixture.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# AB Legacy test fixture
|
||||
|
||||
Coverage map + gap inventory for the AB Legacy (PCCC) driver — SLC 500 /
|
||||
MicroLogix / PLC-5 / LogixPccc-mode.
|
||||
|
||||
**TL;DR: there is no integration fixture.** Everything runs through a
|
||||
`FakeAbLegacyTag` injected via `IAbLegacyTagFactory`. libplctag powers the
|
||||
real wire path but ships no in-process fake, and `ab_server` has no PCCC
|
||||
emulation either — so PCCC behavior against real hardware is trusted from
|
||||
field deployments, not from CI.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
Nothing at the integration layer.
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.AbLegacy.Tests/` is unit-only, all tests
|
||||
tagged `[Trait("Category", "Unit")]`. The driver accepts
|
||||
`IAbLegacyTagFactory` via ctor DI; every test supplies a `FakeAbLegacyTag`.
|
||||
|
||||
## What it actually covers (unit only)
|
||||
|
||||
- `AbLegacyAddressTests` — PCCC address parsing for SLC / MicroLogix / PLC-5
|
||||
/ LogixPccc-mode (`N7:0`, `F8:12`, `B3:0/5`, etc.)
|
||||
- `AbLegacyCapabilityTests` — data type mapping, read-only enforcement
|
||||
- `AbLegacyReadWriteTests` — read + write happy + error paths against the fake
|
||||
- `AbLegacyBitRmwTests` — bit-within-DINT read-modify-write serialization via
|
||||
per-parent `SemaphoreSlim` (mirrors the AB CIP + FOCAS PMC-bit pattern from #181)
|
||||
- `AbLegacyHostAndStatusTests` — probe + host-status transitions driven by
|
||||
fake-returned statuses
|
||||
- `AbLegacyDriverTests` — `IDriver` lifecycle
|
||||
|
||||
Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
|
||||
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
|
||||
`IPerCallHostResolver`.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. Wire-level PCCC
|
||||
|
||||
No PCCC frame is sent by the test suite. libplctag's PCCC subset (DF1,
|
||||
ControlNet-over-EtherNet, PLC-5 native EtherNet) is untested here;
|
||||
driver-side correctness depends on libplctag being correct.
|
||||
|
||||
### 2. Family-specific behavior
|
||||
|
||||
- SLC 500 timeout + retry thresholds (SLC's comm module has known slow-response
|
||||
edges) — unit fakes don't simulate timing.
|
||||
- MicroLogix 1100 / 1400 max-connection-count limits — not stressed.
|
||||
- PLC-5 native EtherNet connection setup (PCCC-encapsulated-in-CIP vs raw
|
||||
CSPv4) — routing covered at parse level only.
|
||||
|
||||
### 3. Multi-device routing
|
||||
|
||||
`IPerCallHostResolver` contract is verified; real PCCC wire routing across
|
||||
multiple gateways is not.
|
||||
|
||||
### 4. Alarms / history
|
||||
|
||||
PCCC has no alarm object + no history object. Driver doesn't implement
|
||||
`IAlarmSource` or `IHistoryProvider` — no test coverage is the correct shape.
|
||||
|
||||
### 5. File-type coverage
|
||||
|
||||
PCCC has many file types (N, F, B, T, C, R, S, ST, A) — the parser tests
|
||||
cover the common ones but uncommon ones (`R` counters, `S` status files,
|
||||
`A` ASCII strings) have thin coverage.
|
||||
|
||||
## When to trust AB Legacy tests, when to reach for a rig
|
||||
|
||||
| Question | Unit tests | Real PLC |
|
||||
| --- | --- | --- |
|
||||
| "Does `N7:0/5` parse correctly?" | yes | - |
|
||||
| "Does bit-in-word RMW serialize concurrent writers?" | yes | yes |
|
||||
| "Does the driver lifecycle hang / crash?" | yes | yes |
|
||||
| "Does a real read against an SLC 500 return correct bytes?" | no | yes (required) |
|
||||
| "Does MicroLogix 1100 respect its connection-count cap?" | no | yes (required) |
|
||||
| "Do PLC-5 ST-files round-trip correctly?" | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. **Nothing open-source** — libplctag's test suite runs against real
|
||||
hardware; there is no public PCCC simulator comparable to `pymodbus` or
|
||||
`ab_server`.
|
||||
2. **Lab rig** — cheapest path is a used SLC 5/05 or MicroLogix 1100 on a
|
||||
dedicated network; the parts are end-of-life but still available. PLC-5
|
||||
and LogixPccc-mode behavior require those specific controllers.
|
||||
3. **libplctag upstream test harness** — the project's own `tests/` folder
|
||||
has PCCC cases we could try to adapt, but they assume specific hardware.
|
||||
|
||||
AB Legacy is inherently a trust-the-library driver until someone stands up
|
||||
a rig.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.AbLegacy.Tests/FakeAbLegacyTag.cs` —
|
||||
in-process fake + factory
|
||||
- `src/ZB.MOM.WW.OtOpcUa.Driver.AbLegacy/AbLegacyDriver.cs` — scope remarks
|
||||
at the top of the file
|
||||
147
docs/drivers/AbServer-Test-Fixture.md
Normal file
147
docs/drivers/AbServer-Test-Fixture.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# ab_server test fixture
|
||||
|
||||
Coverage map + gap inventory for the AB CIP integration fixture backed by
|
||||
libplctag's `ab_server` simulator.
|
||||
|
||||
**TL;DR:** `ab_server` is a connectivity + atomic-read smoke harness for the AB
|
||||
CIP driver. It does **not** benchmark UDTs, alarms, or any family-specific
|
||||
quirk. UDT / alarm / quirk behavior is verified only by unit tests with
|
||||
`FakeAbCipTagRuntime`.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
- **Binary**: `ab_server` / `ab_server.exe` from libplctag
|
||||
([libplctag/libplctag](https://github.com/libplctag/libplctag) +
|
||||
[kyle-github/ab_server](https://github.com/kyle-github/ab_server), MIT).
|
||||
Resolved off `PATH` by `AbServerFixture.LocateBinary`; tests skip via
|
||||
`[AbServerFact]` / `[AbServerTheory]` when missing.
|
||||
- **Lifecycle**: `AbServerFixture` (`tests/ZB.MOM.WW.OtOpcUa.Driver.AbCip.IntegrationTests/AbServerFixture.cs`)
|
||||
starts the simulator with a profile-specific `--plc` arg + `--tag` seeds,
|
||||
waits ~500 ms, kills on `DisposeAsync`.
|
||||
- **Profiles**: `KnownProfiles.{ControlLogix, CompactLogix, Micro800, GuardLogix}`
|
||||
in `AbServerProfile.cs`. Each composes a CLI arg list + seed-tag set; their
|
||||
own `Notes` fields document the quirks called out below.
|
||||
- **Tests**: one smoke, `AbCipReadSmokeTests.Driver_reads_seeded_DInt_from_ab_server`,
|
||||
parametrized over all four profiles.
|
||||
|
||||
## What it actually covers
|
||||
|
||||
- Read path: driver → libplctag → CIP-over-EtherNet/IP → simulator → back.
|
||||
- Atomic Logix types per seed: `DINT`, `REAL`, `BOOL`, `SINT`, `STRING`.
|
||||
- One `DINT[16]` array tag (ControlLogix profile only).
|
||||
- `--plc controllogix` and `--plc compactlogix` mode dispatch.
|
||||
- The skip-on-missing-binary behavior (`AbServerFactAttribute`) so a fresh
|
||||
clone without the simulator stays green.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
Each gap below is either stated explicitly in the profile's `Notes` field or
|
||||
inferable from the seed-tag set + smoke-test surface.
|
||||
|
||||
### 1. UDTs / CIP Template Object (class 0x6C)
|
||||
|
||||
ControlLogix profile `Notes`: *"ab_server lacks full UDT emulation."*
|
||||
|
||||
Unverified against `ab_server`:
|
||||
|
||||
- PR 6 structured read/write (`AbCipStructureMember` fan-out)
|
||||
- #179 Template Object shape reader (`CipTemplateObjectDecoder` + `FetchUdtShapeAsync`)
|
||||
- #194 whole-UDT read optimization (`AbCipUdtReadPlanner` +
|
||||
`AbCipUdtMemberLayout` + the `ReadGroupAsync` path in `AbCipDriver`)
|
||||
|
||||
Unit coverage: `AbCipFetchUdtShapeTests`, `CipTemplateObjectDecoderTests`,
|
||||
`AbCipUdtMemberLayoutTests`, `AbCipUdtReadPlannerTests`,
|
||||
`AbCipDriverWholeUdtReadTests` — all with golden Template-Object byte buffers
|
||||
+ offset-keyed `FakeAbCipTag` values.
|
||||
|
||||
### 2. ALMD / ALMA alarm projection (#177)
|
||||
|
||||
Depends on the ALMD UDT shape, which `ab_server` cannot emulate. The
|
||||
`OnAlarmEvent` raise/clear path + ack-write semantics are not exercised
|
||||
end-to-end.
|
||||
|
||||
Unit coverage: `AbCipAlarmProjectionTests` — fakes feed `InFaulted` /
|
||||
`Severity` via `ValuesByOffset` + assert the emitted `AlarmEventArgs`.
|
||||
|
||||
### 3. Micro800 unconnected-only path
|
||||
|
||||
Micro800 profile `Notes`: *"ab_server has no --plc micro800 — falls back to
|
||||
controllogix emulation."*
|
||||
|
||||
The empty routing path + unconnected-session requirement (PR 11) is unit-tested
|
||||
but never challenged at the CIP wire level. Real Micro800 (2080-series) on a
|
||||
lab rig would be the authoritative benchmark.
|
||||
|
||||
### 4. GuardLogix safety subsystem
|
||||
|
||||
GuardLogix profile `Notes`: *"ab_server doesn't emulate the safety
|
||||
subsystem."*
|
||||
|
||||
Only the `_S`-suffix naming classifier (PR 12, `SecurityClassification.ViewOnly`
|
||||
forced on safety tags) runs. Actual safety-partition write rejection — what
|
||||
happens when a non-safety write lands on a real `1756-L8xS` — is not exercised.
|
||||
|
||||
### 5. CompactLogix narrow ConnectionSize cap
|
||||
|
||||
CompactLogix profile `Notes`: *"ab_server lacks the narrower limit itself."*
|
||||
|
||||
Driver-side `AbCipPlcFamilyProfile` caps `ConnectionSize` at the CompactLogix
|
||||
value per PR 10, but `ab_server` accepts whatever the client asks for — the
|
||||
cap's correctness is trusted from its unit test, never stressed against a
|
||||
simulator that rejects oversized requests.
|
||||
|
||||
### 6. BOOL-within-DINT read-modify-write (#181)
|
||||
|
||||
The `AbCipDriver.WriteBitInDIntAsync` RMW path + its per-parent `SemaphoreSlim`
|
||||
serialization is unit-tested only (`AbCipBoolInDIntRmwTests`). `ab_server`
|
||||
seeds a plain `TestBOOL` tag; the `.N` bit-within-DINT syntax that triggers
|
||||
the RMW path is not exercised end-to-end.
|
||||
|
||||
### 7. Capability surfaces beyond read
|
||||
|
||||
No smoke test for:
|
||||
|
||||
- `IWritable.WriteAsync`
|
||||
- `ITagDiscovery.DiscoverAsync` (`@tags` walker)
|
||||
- `ISubscribable.SubscribeAsync` (poll-group engine)
|
||||
- `IHostConnectivityProbe` state transitions under wire failure
|
||||
- `IPerCallHostResolver` multi-device routing
|
||||
|
||||
The driver implements all of these + they have unit coverage, but the only
|
||||
end-to-end path `ab_server` validates today is atomic `ReadAsync`.
|
||||
|
||||
## When to trust ab_server, when to reach for a rig
|
||||
|
||||
| Question | ab_server | Unit tests | Lab rig |
|
||||
| --- | --- | --- | --- |
|
||||
| "Does the driver talk CIP at all?" | yes | - | - |
|
||||
| "Is my atomic read path wired correctly?" | yes | yes | yes |
|
||||
| "Does whole-UDT grouping work?" | no | yes | yes (required) |
|
||||
| "Do ALMD alarms raise + clear?" | no | yes | yes (required) |
|
||||
| "Is Micro800 unconnected-only enforced wire-side?" | no (emulated as CLX) | partial | yes (required) |
|
||||
| "Does GuardLogix reject non-safety writes on safety tags?" | no | no | yes (required) |
|
||||
| "Does CompactLogix refuse oversized ConnectionSize?" | no | partial | yes (required) |
|
||||
| "Does BOOL-in-DINT RMW race against concurrent writers?" | no | yes | yes (stress) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
If integration-level UDT / alarm / quirk proof becomes a shipping gate, the
|
||||
options are roughly:
|
||||
|
||||
1. **Extend `ab_server`** upstream — the project accepts PRs + already carries
|
||||
a CIP framing layer that UDT emulation could plug into.
|
||||
2. **Swap in a richer simulator** — e.g.
|
||||
[OpenPLC](https://www.openplcproject.com/) or pycomm3's test harness, if
|
||||
either exposes UDTs over EtherNet/IP in a way libplctag can consume.
|
||||
3. **Stand up a lab rig** — physical `1756-L7x` / `5069-L3x` / `2080-LC30` /
|
||||
`1756-L8xS` controllers running Rockwell Studio 5000 projects; wire into
|
||||
CI via a self-hosted runner. The only path that covers safety + narrow
|
||||
ConnectionSize + real ALMD execution.
|
||||
|
||||
See also:
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.AbCip.IntegrationTests/AbServerFixture.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.AbCip.IntegrationTests/AbServerProfile.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.AbCip.IntegrationTests/AbCipReadSmokeTests.cs`
|
||||
- `docs/v2/test-data-sources.md` §2 — the broader test-data-source picking
|
||||
rationale this fixture slots into
|
||||
95
docs/drivers/FOCAS-Test-Fixture.md
Normal file
95
docs/drivers/FOCAS-Test-Fixture.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# FOCAS test fixture
|
||||
|
||||
Coverage map + gap inventory for the FANUC FOCAS2 CNC driver.
|
||||
|
||||
**TL;DR: there is no integration fixture.** Every test uses a
|
||||
`FakeFocasClient` injected via `IFocasClientFactory`. Fanuc's FOCAS library
|
||||
(`Fwlib32.dll`) is closed-source proprietary with no public simulator;
|
||||
CNC-side behavior is trusted from field deployments.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
Nothing at the integration layer.
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/` is unit-only. The driver ships
|
||||
as Tier C (process-isolated) per `docs/v2/driver-stability.md` because the
|
||||
FANUC DLL has known crash modes; tests can't replicate those in-process.
|
||||
|
||||
## What it actually covers (unit only)
|
||||
|
||||
- `FocasCapabilityTests` — data-type mapping (PMC bit / word / float,
|
||||
macro variable types, parameter types)
|
||||
- `FocasReadWriteTests` — read + write against the fake, FOCAS native status
|
||||
→ OPC UA StatusCode mapping
|
||||
- `FocasScaffoldingTests` — `IDriver` lifecycle + multi-device routing
|
||||
- `FocasPmcBitRmwTests` — PMC bit read-modify-write synchronization (per-byte
|
||||
`SemaphoreSlim`, mirrors the AB / Modbus pattern from #181)
|
||||
- `FwlibNativeHelperTests` — `Focas32.dll` → `Fwlib32.dll` bridge validation
|
||||
+ P/Invoke signature validation
|
||||
|
||||
Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
|
||||
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
|
||||
`IPerCallHostResolver`.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. FOCAS wire traffic
|
||||
|
||||
No FOCAS TCP frame is sent. `Fwlib32.dll`'s TCP-to-FANUC-gateway exchange is
|
||||
closed-source; the driver trusts the P/Invoke layer per #193. Real CNC
|
||||
correctness is trusted from field deployments.
|
||||
|
||||
### 2. Alarm / parameter-change callbacks
|
||||
|
||||
FOCAS has no push model — the driver polls via the shared `PollGroupEngine`.
|
||||
There are no CNC-initiated callbacks to test; the absence is by design.
|
||||
|
||||
### 3. Macro / ladder variable types
|
||||
|
||||
FANUC has CNC-specific extensions (macro variables `#100-#999`, system
|
||||
variables `#1000-#5000`, PMC timers / counters / keep-relays) whose
|
||||
per-address semantics differ across 0i-F / 30i / 31i / 32i Series. Driver
|
||||
covers the common address shapes; per-model quirks are not stressed.
|
||||
|
||||
### 4. Model-specific behavior
|
||||
|
||||
- Alarm retention across power cycles (model-specific CNC behavior)
|
||||
- Parameter range enforcement (CNC rejects out-of-range writes)
|
||||
- MTB (machine tool builder) custom screens that expose non-standard data
|
||||
|
||||
### 5. Tier-C process isolation behavior
|
||||
|
||||
Per driver-stability.md, FOCAS should run process-isolated because
|
||||
`Fwlib32.dll` has documented crash modes. The test suite runs in-process +
|
||||
only exercises the happy path + mapped error codes — a native access
|
||||
violation from the DLL would take the test host down. The process-isolation
|
||||
path (similar to Galaxy's out-of-process Host) has been scoped but not
|
||||
implemented.
|
||||
|
||||
## When to trust FOCAS tests, when to reach for a rig
|
||||
|
||||
| Question | Unit tests | Real CNC |
|
||||
| --- | --- | --- |
|
||||
| "Does PMC address `R100.3` route to the right bit?" | yes | yes |
|
||||
| "Does the FANUC status → OPC UA StatusCode map cover every documented code?" | yes (contract) | yes |
|
||||
| "Does a real read against a 30i Series return correct bytes?" | no | yes (required) |
|
||||
| "Does `Fwlib32.dll` crash on concurrent reads?" | no | yes (stress) |
|
||||
| "Do macro variables round-trip across power cycles?" | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. **Nothing public** — Fanuc's FOCAS Developer Kit ships an emulator DLL
|
||||
but it's under NDA + tied to licensed dev-kit installations; can't
|
||||
redistribute for CI.
|
||||
2. **Lab rig** — used FANUC 0i-F simulator controller (or a retired machine
|
||||
tool) on a dedicated network; only path that covers real CNC behavior.
|
||||
3. **Process isolation first** — before trusting FOCAS in production at
|
||||
scale, shipping the Tier-C out-of-process Host architecture (similar to
|
||||
Galaxy) is higher value than a CI simulator.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/FakeFocasClient.cs` —
|
||||
in-process fake implementing `IFocasClient`
|
||||
- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs` — ctor takes
|
||||
`IFocasClientFactory`
|
||||
- `docs/v2/driver-stability.md` — Tier C scope + process-isolation rationale
|
||||
164
docs/drivers/Galaxy-Test-Fixture.md
Normal file
164
docs/drivers/Galaxy-Test-Fixture.md
Normal file
@@ -0,0 +1,164 @@
|
||||
# Galaxy test fixture
|
||||
|
||||
Coverage map + gap inventory for the Galaxy driver — out-of-process Host
|
||||
(net48 x86 MXAccess COM) + Proxy (net10) + Shared protocol.
|
||||
|
||||
**TL;DR: Galaxy has the richest test harness in the fleet** — real Host
|
||||
subprocess spawn, real ZB SQL queries, IPC parity checks against the v1
|
||||
LmxProxy reference, + live-smoke tests when MXAccess runtime is actually
|
||||
installed. Gaps are live-plant + failover-shaped: the E2E suite covers the
|
||||
representative ~50-tag deployment but not large-site discovery stress, real
|
||||
Rockwell/Siemens PLC enumeration through MXAccess, or ZB SQL Always-On
|
||||
replica failover.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
Multi-project test topology:
|
||||
|
||||
- **E2E parity** —
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.E2E/ParityFixture.cs` spawns the
|
||||
production `OtOpcUa.Driver.Galaxy.Host.exe` as a subprocess, opens the
|
||||
named-pipe IPC, connects `GalaxyProxyDriver` + runs hierarchy / stability
|
||||
parity tests against both.
|
||||
- **Host.Tests** —
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host.Tests/` — direct Host process
|
||||
testing (18+ test classes covering alarm discovery, AVEVA prerequisite
|
||||
checks, IPC dispatcher, alarm tracker, probe manager, historian
|
||||
cluster/quality/wiring, history read, OPC UA attribute mapping,
|
||||
subscription lifecycle, reconnect, multi-host proxy, ADS address routing,
|
||||
expression evaluation) + `GalaxyRepositoryLiveSmokeTests` that hit real
|
||||
ZB SQL.
|
||||
- **Proxy.Tests** — `GalaxyProxyDriver` client contract tests.
|
||||
- **Shared.Tests** — shared protocol + address model.
|
||||
- **TestSupport** — test helpers reused across the above.
|
||||
|
||||
## How tests skip
|
||||
|
||||
- **E2E parity**: `ParityFixture.SkipIfUnavailable()` runs at class init and
|
||||
checks Windows-only, non-admin user, ZB SQL reachable on
|
||||
`localhost:1433`, Host EXE built in the expected `bin/` folder. Any miss
|
||||
→ tests skip.
|
||||
- **Live-smoke** (`GalaxyRepositoryLiveSmokeTests`): `Assert.Skip` when ZB
|
||||
unreachable. A `per project_galaxy_host_installed` memory on this repo's
|
||||
dev box notes the MXAccess runtime is installed + pipe ACL denies Admins,
|
||||
so live tests must run from a non-elevated shell.
|
||||
- **Unit** tests (Shared, Proxy contract, most Host.Tests) have no skip —
|
||||
they run anywhere.
|
||||
|
||||
## What it actually covers
|
||||
|
||||
### E2E parity suite
|
||||
|
||||
- `HierarchyParityTests` — Host address-space hierarchy vs v1 LmxProxy
|
||||
reference (same ZB, same Galaxy, same shape)
|
||||
- `StabilityFindingsRegressionTests` — probe subscription failure
|
||||
handling + host-status mutation guard from the v1 stability findings
|
||||
backlog
|
||||
|
||||
### Host.Tests (representative)
|
||||
|
||||
- Alarm discovery → subsystem setup
|
||||
- AVEVA prerequisite checks (runtime installed, platform deployed, etc.)
|
||||
- IPC dispatcher — request/response routing over the named pipe
|
||||
- Alarm tracker state machine
|
||||
- Probe manager — per-runtime probe subscription + reconnect
|
||||
- Historian cluster / quality / wiring — Aveva Historian integration
|
||||
- OPC UA attribute mapping
|
||||
- Subscription lifecycle + reconnect
|
||||
- Multi-host proxy routing
|
||||
- ADS address routing + expression evaluation (Galaxy's legacy expression
|
||||
language)
|
||||
|
||||
### Live-smoke
|
||||
|
||||
- `GalaxyRepositoryLiveSmokeTests` — real SQL against ZB database, verifies
|
||||
the ZB schema + `LocalPlatform` scope filter + change-detection query
|
||||
shape match production.
|
||||
|
||||
### Capability surfaces hit
|
||||
|
||||
All of them: `IDriver`, `IReadable`, `IWritable`, `ITagDiscovery`,
|
||||
`ISubscribable`, `IHostConnectivityProbe`, `IPerCallHostResolver`,
|
||||
`IAlarmSource`, `IHistoryProvider`. Galaxy is the only driver where every
|
||||
interface sees both contract + real-integration coverage.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. MXAccess COM by default
|
||||
|
||||
The E2E parity suite backs subscriptions via the DB-only path; MXAccess COM
|
||||
integration opts in via a separate live-smoke. So "does the MXAccess STA
|
||||
pump correctly handle real Wonderware runtime events" is exercised only
|
||||
when the operator runs live smoke on a machine with MXAccess installed.
|
||||
|
||||
### 2. Real Rockwell / Siemens PLC enumeration
|
||||
|
||||
Galaxy runtime talks to PLCs through MXAccess (Device Integration Objects).
|
||||
The CI parity suite uses a representative ~50-tag deployment; large sites
|
||||
(1000+ tag hierarchies, multi-Galaxy replication, deeply-nested templates)
|
||||
are not stressed.
|
||||
|
||||
### 3. ZB SQL Always-On failover
|
||||
|
||||
Live-smoke hits a single SQL instance. Real production ZB often runs on
|
||||
Always-On availability groups; replica failover behavior is not tested.
|
||||
|
||||
### 4. Galaxy replication / backup-restore
|
||||
|
||||
Galaxy supports backup + partial replication across platforms — these
|
||||
rewrite the ZB schema in ways that change the contained_name vs tag_name
|
||||
mapping. Not exercised.
|
||||
|
||||
### 5. Historian failover
|
||||
|
||||
Aveva Historian can be clustered. `historian cluster / quality` tests
|
||||
verify the cluster-config query; they don't exercise actual failover
|
||||
(primary dies → secondary takes over mid-HistoryRead).
|
||||
|
||||
### 6. AVEVA runtime version matrix
|
||||
|
||||
MXAccess COM contract varies subtly across System Platform 2017 / 2020 /
|
||||
2023. The live-smoke runs against whatever version is installed on the dev
|
||||
box; CI has no AVEVA installed at all (licensing + footprint).
|
||||
|
||||
## When to trust the Galaxy suite, when to reach for a live plant
|
||||
|
||||
| Question | E2E parity | Live-smoke | Real plant |
|
||||
| --- | --- | --- | --- |
|
||||
| "Does Host spawn + IPC round-trip work?" | yes | yes | yes |
|
||||
| "Does the ZB schema query match production shape?" | partial | yes | yes |
|
||||
| "Does MXAccess COM handle runtime reconnect correctly?" | no | yes | yes |
|
||||
| "Does the driver scale to 1000+ tags on one Galaxy?" | no | partial | yes (required) |
|
||||
| "Does historian failover mid-read return a clean error?" | no | no | yes (required) |
|
||||
| "Does System Platform 2023's MXAccess differ from 2020?" | no | partial | yes (required) |
|
||||
| "Does ZB Always-On replica failover preserve generation?" | no | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. **System Platform 2023 live-smoke matrix** — set up a second dev box
|
||||
running SP2023; run the same live-smoke against both to catch COM-contract
|
||||
drift early.
|
||||
2. **Synthetic large-site fixture** — script a ZB populator that creates a
|
||||
1000-Equipment / 20000-tag hierarchy, run the parity suite against it.
|
||||
Catches O(N) → O(N²) discovery regressions.
|
||||
3. **Historian failover scripted test** — with a two-node AVEVA Historian
|
||||
cluster, tear down primary mid-HistoryRead + verify the driver's failover
|
||||
behavior + error surface.
|
||||
4. **ZB Always-On CI** — SQL Server 2022 on Linux supports Always-On;
|
||||
could stand up a two-replica group for replica-failover coverage.
|
||||
|
||||
This is already the best-tested driver; the remaining work is site-scale
|
||||
+ production-topology coverage, not capability coverage.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.E2E/ParityFixture.cs` — E2E fixture
|
||||
that spawns Host + connects Proxy
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host.Tests/GalaxyRepositoryLiveSmokeTests.cs`
|
||||
— live ZB smoke with `Assert.Skip` gate
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.TestSupport/` — shared helpers
|
||||
- `docs/drivers/Galaxy.md` — COM bridge + STA pump + IPC architecture
|
||||
- `docs/drivers/Galaxy-Repository.md` — ZB SQL reader + `LocalPlatform`
|
||||
scope filter + change detection
|
||||
- `docs/v2/aveva-system-platform-io-research.md` — MXAccess + Wonderware
|
||||
background
|
||||
113
docs/drivers/Modbus-Test-Fixture.md
Normal file
113
docs/drivers/Modbus-Test-Fixture.md
Normal file
@@ -0,0 +1,113 @@
|
||||
# Modbus test fixture
|
||||
|
||||
Coverage map + gap inventory for the Modbus TCP driver's integration-test
|
||||
harness backed by `pymodbus` simulator profiles per PLC family.
|
||||
|
||||
**TL;DR:** Modbus is the best-covered driver — a real `pymodbus` server on
|
||||
localhost with per-family seed-register profiles, plus a skip-gate when the
|
||||
simulator port isn't reachable. Covers DL205 / Mitsubishi MELSEC / Siemens
|
||||
S7-1500 family quirks end-to-end. Gaps are mostly error-path + alarm/history
|
||||
shaped (neither is a Modbus-side concept).
|
||||
|
||||
## What the fixture is
|
||||
|
||||
- **Simulator**: `pymodbus` (Python, BSD) driven from PowerShell + per-family
|
||||
JSON profiles under
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Pymodbus/`.
|
||||
- **Lifecycle**: `ModbusSimulatorFixture` (collection-scoped) TCP-probes
|
||||
`localhost:5020` on first use. `MODBUS_SIM_ENDPOINT` env var overrides the
|
||||
endpoint so the same suite can target a real PLC.
|
||||
- **Profiles**: `DL205Profile`, `MitsubishiProfile`, `S7_1500Profile` —
|
||||
each composes device-specific register-format + quirk-seed JSON for pymodbus.
|
||||
- **Tests skip** via `Assert.Skip(sim.SkipReason)` when the probe fails; no
|
||||
custom FactAttribute needed because `ModbusSimulatorCollection` carries the
|
||||
skip reason.
|
||||
|
||||
## What it actually covers
|
||||
|
||||
### DL205 (Automation Direct)
|
||||
|
||||
- `DL205SmokeTests` — FC16 write → FC03 read round-trip on holding register
|
||||
- `DL205CoilMappingTests` — Y-output / C-relay / X-input address mapping
|
||||
(octal → Modbus offset)
|
||||
- `DL205ExceptionCodeTests` — Modbus exception → OPC UA StatusCode mapping
|
||||
- `DL205FloatCdabQuirkTests` — CDAB word-swap float encoding
|
||||
- `DL205StringQuirkTests` — packed-string V-memory layout
|
||||
- `DL205VMemoryQuirkTests` — V-memory octal addressing
|
||||
- `DL205XInputTests` — X-register read-only enforcement
|
||||
|
||||
### Mitsubishi MELSEC
|
||||
|
||||
- `MitsubishiSmokeTests` — read + write round-trip
|
||||
- `MitsubishiQuirkTests` — word-order, device-code mapping (D/M/X/Y ranges)
|
||||
|
||||
### Siemens S7-1500 (Modbus gateway flavor)
|
||||
|
||||
- `S7_1500SmokeTests` — read + write round-trip
|
||||
- `S7_ByteOrderTests` — ABCD/DCBA/BADC/CDAB byte-order matrix
|
||||
|
||||
### Capability surfaces hit
|
||||
|
||||
- `IReadable` + `IWritable` — full round-trip
|
||||
- `ISubscribable` — via the shared `PollGroupEngine` (polled subscription)
|
||||
- `IHostConnectivityProbe` — TCP-reach transitions
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. No `ITagDiscovery`
|
||||
|
||||
Modbus has no symbol table — the driver requires a static tag map from
|
||||
`DriverConfig`. There is no discovery path to test + none in the fixture.
|
||||
|
||||
### 2. Error-path fuzzing
|
||||
|
||||
`pymodbus` serves the seeded values happily; the fixture can't easily inject
|
||||
exception responses (code 0x01–0x0B) or malformed PDUs. The
|
||||
`AbCipStatusMapper`-equivalent for exception codes is unit-tested via
|
||||
`DL205ExceptionCodeTests` but the simulator itself never refuses a read.
|
||||
|
||||
### 3. Variant-specific quirks beyond the three profiles
|
||||
|
||||
- FX5U / QJ71MT91 Mitsubishi variants — profile scaffolds exist, no tests yet
|
||||
- Non-S7-1500 Siemens (S7-1200 / ET200SP) — byte-order covered but
|
||||
connection-pool + fragmentation quirks untested
|
||||
- DL205-family cousins (DL06, DL260) — no dedicated profile
|
||||
|
||||
### 4. Subscription stress
|
||||
|
||||
`PollGroupEngine` is unit-tested standalone but the simulator doesn't exercise
|
||||
it under multi-register packing stress (FC03 with 125-register batches,
|
||||
boundary splits, etc.).
|
||||
|
||||
### 5. Alarms / history
|
||||
|
||||
Not a Modbus concept. Driver doesn't implement `IAlarmSource` or
|
||||
`IHistoryProvider`; no test coverage is the correct shape.
|
||||
|
||||
## When to trust the Modbus fixture, when to reach for a rig
|
||||
|
||||
| Question | Fixture | Unit tests | Real PLC |
|
||||
| --- | --- | --- | --- |
|
||||
| "Does FC03/FC06/FC16 work end-to-end?" | yes | - | yes |
|
||||
| "Does DL205 octal addressing map correctly?" | yes | yes | yes |
|
||||
| "Does float CDAB word-swap round-trip?" | yes | yes | yes |
|
||||
| "Does the driver handle exception responses?" | no | yes | yes (required) |
|
||||
| "Does packing 125 regs into one FC03 work?" | no | no | yes (required) |
|
||||
| "Does FX5U behave like Q-series?" | no | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. Add `MODBUS_SIM_ENDPOINT` override documentation to
|
||||
`docs/v2/test-data-sources.md` so operators can point the suite at a lab rig.
|
||||
2. Extend `pymodbus` profiles to inject exception responses — a JSON flag per
|
||||
register saying "next read returns exception 0x04."
|
||||
3. Add an FX5U profile once a lab rig is available; the scaffolding is in place.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/ModbusSimulatorFixture.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/DL205/DL205Profile.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Mitsubishi/MitsubishiProfile.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/S7/S7_1500Profile.cs`
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Modbus.IntegrationTests/Pymodbus/` — simulator
|
||||
driver script + per-family JSON profiles
|
||||
139
docs/drivers/OpcUaClient-Test-Fixture.md
Normal file
139
docs/drivers/OpcUaClient-Test-Fixture.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# OPC UA Client test fixture
|
||||
|
||||
Coverage map + gap inventory for the OPC UA Client (gateway / aggregation)
|
||||
driver.
|
||||
|
||||
**TL;DR: there is no integration fixture.** Tests mock the OPC UA SDK's
|
||||
`Session` + `Subscription` types directly; there is no upstream OPC UA
|
||||
server standup in CI. The irony is not lost — this repo *is* an OPC UA
|
||||
server, and the integration fixtures for `OpcUaApplicationHost`
|
||||
(`tests/ZB.MOM.WW.OtOpcUa.Server.Tests/OpcUaServerIntegrationTests.cs` +
|
||||
`OpcUaEquipmentWalkerIntegrationTests.cs`) stand up the server-side stack
|
||||
end-to-end. The client-side driver could in principle wire against one of
|
||||
those, but doesn't today.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
Nothing at the integration layer.
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Tests/` is unit-only. Tests
|
||||
inject fakes through the driver's construction path; the
|
||||
OPCFoundation.NetStandard `Session` surface is wrapped behind an interface
|
||||
the tests mock.
|
||||
|
||||
## What it actually covers (unit only)
|
||||
|
||||
The surface is broad because `OpcUaClientDriver` is the richest-capability
|
||||
driver in the fleet (it's a gateway for another OPC UA server, so it
|
||||
mirrors the full capability matrix):
|
||||
|
||||
- `OpcUaClientDriverScaffoldTests` — `IDriver` lifecycle
|
||||
- `OpcUaClientReadWriteTests` — read + write lifecycle
|
||||
- `OpcUaClientSubscribeAndProbeTests` — monitored-item subscription + probe
|
||||
state transitions
|
||||
- `OpcUaClientDiscoveryTests` — `GetEndpoints` + endpoint selection
|
||||
- `OpcUaClientAttributeMappingTests` — OPC UA node attribute → driver value
|
||||
mapping
|
||||
- `OpcUaClientSecurityPolicyTests` — `SignAndEncrypt` / `Sign` / `None`
|
||||
policy negotiation contract
|
||||
- `OpcUaClientCertAuthTests` — cert store paths, revocation-list config
|
||||
- `OpcUaClientReconnectTests` — SDK reconnect hook + `TransferSubscriptions`
|
||||
across the disconnect boundary
|
||||
- `OpcUaClientFailoverTests` — primary → secondary session fallback per
|
||||
driver config
|
||||
- `OpcUaClientAlarmTests` — A&E severity bucket (1–1000 → Low / Medium /
|
||||
High / Critical), subscribe / unsubscribe / ack contract
|
||||
- `OpcUaClientHistoryTests` — historical data read + interpolation contract
|
||||
|
||||
Capability surfaces whose contract is verified: `IDriver`, `ITagDiscovery`,
|
||||
`IReadable`, `IWritable`, `ISubscribable`, `IHostConnectivityProbe`,
|
||||
`IAlarmSource`, `IHistoryProvider`.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. Real stack exchange
|
||||
|
||||
No UA Secure Channel is ever opened. Every test mocks `Session.ReadAsync`,
|
||||
`Session.CreateSubscription`, `Session.AddItem`, etc. — the SDK itself is
|
||||
trusted. Certificate validation, signing, nonce handling, chunk assembly,
|
||||
keep-alive cadence — all SDK-internal and untested here.
|
||||
|
||||
### 2. Subscription transfer across reconnect
|
||||
|
||||
Contract test: "after a simulated reconnect, `TransferSubscriptions` is
|
||||
called with the right handles." Real behavior: SDK re-publishes against the
|
||||
new channel and some events can be lost depending on publish-queue state.
|
||||
The lossy window is not characterized.
|
||||
|
||||
### 3. Large-scale subscription stress
|
||||
|
||||
100+ monitored items with heterogeneous publish intervals under a single
|
||||
session — the shape that breaks publish-queue-size tuning in the wild — is
|
||||
not exercised.
|
||||
|
||||
### 4. Real historian mappings
|
||||
|
||||
`IHistoryProvider.ReadRawAsync` + `ReadProcessedAsync` +
|
||||
`ReadAtTimeAsync` + `ReadEventsAsync` are contract-mocked. Against a real
|
||||
historian (AVEVA Historian, Prosys historian, Kepware LocalHistorian) each
|
||||
has specific interpolation + bad-quality-handling quirks the contract test
|
||||
doesn't see.
|
||||
|
||||
### 5. Real A&E events
|
||||
|
||||
Alarm subscription is mocked via filtered monitored items; the actual
|
||||
`EventFilter` select-clause behavior against a server that exposes typed
|
||||
ConditionType events (non-base `BaseEventType`) is not verified.
|
||||
|
||||
### 6. Authentication variants
|
||||
|
||||
- Anonymous, UserName/Password, X509 cert tokens — each is contract-tested
|
||||
but not exchanged against a server that actually enforces each.
|
||||
- LDAP-backed `UserName` (matching this repo's server-side
|
||||
`LdapUserAuthenticator`) requires a live LDAP round-trip; not tested.
|
||||
|
||||
## When to trust OpcUaClient tests, when to reach for a server
|
||||
|
||||
| Question | Unit tests | Real upstream server |
|
||||
| --- | --- | --- |
|
||||
| "Does severity 750 bucket as High?" | yes | yes |
|
||||
| "Does the driver call `TransferSubscriptions` after reconnect?" | yes | yes |
|
||||
| "Does a real OPC UA read/write round-trip work?" | no | yes (required) |
|
||||
| "Does event-filter-based alarm subscription return ConditionType events?" | no | yes (required) |
|
||||
| "Does history read from AVEVA Historian return correct aggregates?" | no | yes (required) |
|
||||
| "Does the SDK's publish queue lose notifications under load?" | no | yes (stress) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
The easiest win here is to **wire the client driver tests against this
|
||||
repo's own server**. The integration test project
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Server.Tests/OpcUaServerIntegrationTests.cs`
|
||||
already stands up a real OPC UA server on a non-default port with a seeded
|
||||
FakeDriver. An `OpcUaClientLiveLoopbackTests` that connects the client
|
||||
driver to that server would give:
|
||||
|
||||
- Real Secure Channel negotiation
|
||||
- Real Session / Subscription / MonitoredItem exchange
|
||||
- Real read/write round-trip
|
||||
- Real certificate validation (the integration test already sets up PKI)
|
||||
|
||||
It wouldn't cover upstream-server-specific quirks (AVEVA Historian, Kepware,
|
||||
Prosys), but it would cover 80% of the SDK surface the driver sits on top
|
||||
of.
|
||||
|
||||
Beyond that:
|
||||
|
||||
1. **Prosys OPC UA Simulation Server** — free, Windows-available, scriptable.
|
||||
2. **UaExpert Server-Side Simulator** — Unified Automation's sample server;
|
||||
good coverage of typed ConditionType events.
|
||||
3. **Dedicated historian integration lab** — only path for
|
||||
historian-specific coverage.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Tests/` — unit tests with
|
||||
mocked `Session`
|
||||
- `src/ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient/OpcUaClientDriver.cs` — ctor +
|
||||
session-factory seam tests mock through
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Server.Tests/OpcUaServerIntegrationTests.cs` —
|
||||
the server-side integration harness a future loopback client test could
|
||||
piggyback on
|
||||
@@ -37,6 +37,19 @@ Driver type metadata is registered at startup in `DriverTypeRegistry` (`src/ZB.M
|
||||
|
||||
- **All other drivers** share a single per-driver specification in [docs/v2/driver-specs.md](../v2/driver-specs.md) — addressing, data-type maps, connection settings, and quirks live there. That file is the authoritative per-driver reference; this index points at it rather than duplicating.
|
||||
|
||||
## Test-fixture coverage maps
|
||||
|
||||
Each driver has a dedicated fixture doc that lays out what the integration / unit harness actually covers vs. what's trusted from field deployments. Read the relevant one before claiming "green suite = production-ready" for a driver.
|
||||
|
||||
- [AB CIP](AbServer-Test-Fixture.md) — `ab_server` simulator, atomic-read smoke only; UDT / ALMD / family quirks are unit-only
|
||||
- [Modbus](Modbus-Test-Fixture.md) — `pymodbus` + per-family profiles; best-covered driver, gaps are error-path-shaped
|
||||
- [Siemens S7](S7-Test-Fixture.md) — no integration fixture, unit-only via fake `IS7Client`
|
||||
- [AB Legacy](AbLegacy-Test-Fixture.md) — no integration fixture, unit-only via `FakeAbLegacyTag` (libplctag PCCC)
|
||||
- [TwinCAT](TwinCAT-Test-Fixture.md) — no integration fixture, unit-only via `FakeTwinCATClient` with native-notification harness
|
||||
- [FOCAS](FOCAS-Test-Fixture.md) — no integration fixture, unit-only via `FakeFocasClient`; Tier C out-of-process isolation scoped but not shipped
|
||||
- [OPC UA Client](OpcUaClient-Test-Fixture.md) — no integration fixture, unit-only via mocked `Session`; loopback against this repo's own server is the obvious next step
|
||||
- [Galaxy](Galaxy-Test-Fixture.md) — richest harness: E2E Host subprocess + ZB SQL live-smoke + MXAccess opt-in
|
||||
|
||||
## Related cross-driver docs
|
||||
|
||||
- [HistoricalDataAccess.md](../HistoricalDataAccess.md) — `IHistoryProvider` dispatch, aggregate mapping, continuation points. The Galaxy driver's Aveva Historian implementation is the first; OPC UA Client forwards to the upstream server; other drivers do not implement the interface and return `BadHistoryOperationUnsupported`.
|
||||
|
||||
119
docs/drivers/S7-Test-Fixture.md
Normal file
119
docs/drivers/S7-Test-Fixture.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Siemens S7 test fixture
|
||||
|
||||
Coverage map + gap inventory for the S7 driver.
|
||||
|
||||
**TL;DR:** S7 now has a wire-level integration fixture backed by
|
||||
[python-snap7](https://github.com/gijzelaerr/python-snap7)'s `Server` class
|
||||
(task #216). Atomic reads (u16 / i16 / i32 / f32 / bool-with-bit) + DB
|
||||
write-then-read round-trip are exercised end-to-end through S7netplus +
|
||||
real ISO-on-TCP on `localhost:1102`. Unit tests still carry everything
|
||||
else (address parsing, error-branch handling, probe-loop contract). Gaps
|
||||
remaining are variant-quirk-shaped: Optimized-DB symbolic access, PG/OP
|
||||
session types, PUT/GET-disabled enforcement — all need real hardware.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
**Integration layer** (task #216):
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.S7.IntegrationTests/` stands up a
|
||||
python-snap7 `Server` via `PythonSnap7/serve.ps1 -Profile s7_1500` on
|
||||
`localhost:1102`. `Snap7ServerFixture` probes the port at collection init
|
||||
+ skips with a clear message when unreachable (matches the pymodbus
|
||||
pattern). `server.py` reads a JSON profile + seeds DB/MB bytes at declared
|
||||
offsets; seeds are typed (`u16` / `i16` / `i32` / `f32` / `bool` / `ascii`
|
||||
for S7 STRING).
|
||||
|
||||
**Unit layer**: `tests/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/` covers
|
||||
everything the wire-level suite doesn't — address parsing, error
|
||||
branches, probe-loop contract. All tests tagged
|
||||
`[Trait("Category", "Unit")]`.
|
||||
|
||||
The driver ctor change that made this possible:
|
||||
`Plc(CpuType, host, port, rack, slot)` — S7netplus 0.20's 5-arg overload
|
||||
— wires `S7DriverOptions.Port` through so the simulator can bind 1102
|
||||
(non-privileged) instead of 102 (root / Firewall-prompt territory).
|
||||
|
||||
## What it actually covers
|
||||
|
||||
### Integration (python-snap7, task #216)
|
||||
|
||||
- `S7_1500SmokeTests.Driver_reads_seeded_u16_through_real_S7comm` — DB1.DBW0
|
||||
read via real S7netplus over TCP + simulator; proves handshake + read path
|
||||
- `S7_1500SmokeTests.Driver_reads_seeded_typed_batch` — i16, i32, f32,
|
||||
bool-with-bit in one batch call; proves typed decode per S7DataType
|
||||
- `S7_1500SmokeTests.Driver_write_then_read_round_trip_on_scratch_word` —
|
||||
`DB1.DBW100` write → read-back; proves write path + buffer visibility
|
||||
|
||||
### Unit
|
||||
|
||||
- `S7AddressParserTests` — S7 address syntax (`DB1.DBD0`, `M10.3`, `IW4`, etc.)
|
||||
- `S7DriverScaffoldTests` — `IDriver` lifecycle (init / reinit / shutdown / health)
|
||||
- `S7DriverReadWriteTests` — error paths (uninitialized read/write, bad
|
||||
addresses, transport exceptions)
|
||||
- `S7DiscoveryAndSubscribeTests` — `ITagDiscovery.DiscoverAsync` + polled
|
||||
`ISubscribable` contract with the shared `PollGroupEngine`
|
||||
|
||||
Capability surfaces whose contract is verified: `IDriver`, `ITagDiscovery`,
|
||||
`IReadable`, `IWritable`, `ISubscribable`, `IHostConnectivityProbe`.
|
||||
Wire-level surfaces verified: `IReadable`, `IWritable`.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. Wire-level anything
|
||||
|
||||
No ISO-on-TCP frame is ever sent during the test suite. S7netplus is the only
|
||||
wire-path abstraction and it has no in-process fake mode; the shipping choice
|
||||
was to contract-test via `IS7Client` rather than patch into S7netplus
|
||||
internals.
|
||||
|
||||
### 2. Read/write happy path
|
||||
|
||||
Every `S7DriverReadWriteTests` case exercises error branches. A successful
|
||||
read returning real PLC data is not tested end-to-end — the return value is
|
||||
whatever the fake says it is.
|
||||
|
||||
### 3. Mailbox serialization under concurrent reads
|
||||
|
||||
The driver's `SemaphoreSlim` serializes S7netplus calls because the S7 CPU's
|
||||
comm mailbox is scanned at most once per cycle. Contention behavior under
|
||||
real PLC latency is not exercised.
|
||||
|
||||
### 4. Variant quirks
|
||||
|
||||
S7-1200 vs S7-1500 vs S7-300/400 connection semantics (PG vs OP vs S7-Basic)
|
||||
not differentiated at test time.
|
||||
|
||||
### 5. Data types beyond the scalars
|
||||
|
||||
UDT fan-out, `STRING` with length-prefix quirks, `DTL` / `DATE_AND_TIME`,
|
||||
arrays of structs — not covered.
|
||||
|
||||
## When to trust the S7 tests, when to reach for a rig
|
||||
|
||||
| Question | Unit tests | Real PLC |
|
||||
| --- | --- | --- |
|
||||
| "Does the address parser accept X syntax?" | yes | - |
|
||||
| "Does the driver lifecycle hang / crash?" | yes | yes |
|
||||
| "Does a real read against an S7-1500 return correct bytes?" | no | yes (required) |
|
||||
| "Does mailbox serialization actually prevent PG timeouts?" | no | yes (required) |
|
||||
| "Does a UDT fan-out produce usable member variables?" | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. **Snap7 server** — [Snap7](https://snap7.sourceforge.net/) ships a
|
||||
C-library-based S7 server that could run in-CI on Linux. A pinned build +
|
||||
a fixture shape similar to `ab_server` would give S7 parity with Modbus /
|
||||
AB CIP coverage.
|
||||
2. **Plcsim Advanced** — Siemens' paid emulator. Licensed per-seat; fits a
|
||||
lab rig but not CI.
|
||||
3. **Real S7 lab rig** — cheapest physical PLC (CPU 1212C) on a dedicated
|
||||
network port, wired via self-hosted runner.
|
||||
|
||||
Without any of these, S7 driver correctness against real hardware is trusted
|
||||
from field deployments, not from the test suite.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.S7.Tests/` — unit tests only, no harness
|
||||
- `src/ZB.MOM.WW.OtOpcUa.Driver.S7/S7Driver.cs` — ctor takes
|
||||
`IS7ClientFactory` which tests fake; docstring lines 8-20 note the deferred
|
||||
integration fixture
|
||||
111
docs/drivers/TwinCAT-Test-Fixture.md
Normal file
111
docs/drivers/TwinCAT-Test-Fixture.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# TwinCAT test fixture
|
||||
|
||||
Coverage map + gap inventory for the Beckhoff TwinCAT ADS driver.
|
||||
|
||||
**TL;DR: there is no integration fixture.** Every test uses a
|
||||
`FakeTwinCATClient` injected via `ITwinCATClientFactory`. Beckhoff's ADS
|
||||
library has no open-source simulator; ADS traffic against real TwinCAT
|
||||
runtimes is trusted from field deployments.
|
||||
|
||||
The silver lining: TwinCAT is the only driver outside Galaxy that uses
|
||||
**native notifications** (no polling) for `ISubscribable`, and the fake
|
||||
exposes a fire-event harness so notification routing is contract-tested
|
||||
rigorously — just not on the wire.
|
||||
|
||||
## What the fixture is
|
||||
|
||||
Nothing at the integration layer.
|
||||
`tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/` is unit-only.
|
||||
`FakeTwinCATClient` also fakes the `AddDeviceNotification` flow so tests can
|
||||
trigger callbacks without a running runtime.
|
||||
|
||||
## What it actually covers (unit only)
|
||||
|
||||
- `TwinCATAmsAddressTests` — `ads://<netId>:<port>` parsing + routing
|
||||
- `TwinCATCapabilityTests` — data-type mapping (primitives + declared UDTs),
|
||||
read-only classification
|
||||
- `TwinCATReadWriteTests` — read + write through the fake, status mapping
|
||||
- `TwinCATSymbolPathTests` — symbol-path routing for nested struct members
|
||||
- `TwinCATSymbolBrowserTests` — `ITagDiscovery.DiscoverAsync` via
|
||||
`ReadSymbolsAsync` (#188) + system-symbol filtering
|
||||
- `TwinCATNativeNotificationTests` — `AddDeviceNotification` (#189)
|
||||
registration, callback-delivery-to-`OnDataChange` wiring, unregister on
|
||||
unsubscribe
|
||||
- `TwinCATDriverTests` — `IDriver` lifecycle
|
||||
|
||||
Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
|
||||
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
|
||||
`IPerCallHostResolver`.
|
||||
|
||||
## What it does NOT cover
|
||||
|
||||
### 1. AMS / ADS wire traffic
|
||||
|
||||
No real AMS router frame is exchanged. Beckhoff's `TwinCAT.Ads` NuGet (their
|
||||
own .NET SDK, not libplctag-style OSS) has no in-process fake; tests stub
|
||||
the `ITwinCATClient` abstraction above it.
|
||||
|
||||
### 2. Multi-route AMS
|
||||
|
||||
ADS supports chained routes (`<localNetId> → <routerNetId> → <targetNetId>`)
|
||||
for PLCs behind an EC master / IPC gateway. Parse coverage exists; wire-path
|
||||
coverage doesn't.
|
||||
|
||||
### 3. Notification reliability under jitter
|
||||
|
||||
`AddDeviceNotification` delivers at the runtime's cycle boundary; under high
|
||||
CPU load or network jitter real notifications can coalesce. The fake fires
|
||||
one callback per test invocation — real callback-coalescing behavior is
|
||||
untested.
|
||||
|
||||
### 4. TC2 vs TC3 variant handling
|
||||
|
||||
TwinCAT 2 (ADS v1) and TwinCAT 3 (ADS v2) have subtly different
|
||||
`GetSymbolInfoByName` semantics + symbol-table layouts. Driver targets TC3;
|
||||
TC2 compatibility is not exercised.
|
||||
|
||||
### 5. Cycle-time alignment for `ISubscribable`
|
||||
|
||||
Native ADS notifications fire on the PLC cycle boundary. The fake test
|
||||
harness assumes notifications fire on a timer the test controls;
|
||||
cycle-aligned firing under real PLC control is not verified.
|
||||
|
||||
### 6. Alarms / history
|
||||
|
||||
Driver doesn't implement `IAlarmSource` or `IHistoryProvider` — not in
|
||||
scope for this driver family. TwinCAT 3's TcEventLogger could theoretically
|
||||
back an `IAlarmSource`, but shipping that is a separate feature.
|
||||
|
||||
## When to trust TwinCAT tests, when to reach for a rig
|
||||
|
||||
| Question | Unit tests | Real TwinCAT runtime |
|
||||
| --- | --- | --- |
|
||||
| "Does the AMS address parser accept X?" | yes | - |
|
||||
| "Does notification → `OnDataChange` wire correctly?" | yes (contract) | yes |
|
||||
| "Does symbol browsing filter TwinCAT internals?" | yes | yes |
|
||||
| "Does a real ADS read return correct bytes?" | no | yes (required) |
|
||||
| "Do notifications coalesce under load?" | no | yes (required) |
|
||||
| "Does a TC2 PLC work the same as TC3?" | no | yes (required) |
|
||||
|
||||
## Follow-up candidates
|
||||
|
||||
1. **TwinCAT 3 runtime on CI** — Beckhoff ships a free developer runtime
|
||||
(7-day trial, restartable). Could run on a Windows CI runner with a
|
||||
helper that auto-restarts the runtime every 6 days. Works but operational
|
||||
overhead.
|
||||
2. **AdsSimulator** — Beckhoff has a closed-source "ADS simulator" library
|
||||
used internally; not publicly available.
|
||||
3. **Lab rig** — cheapest IPC (CX7000 / CX9020) on a dedicated network; the
|
||||
only route that covers TC2 + real notification behavior + EtherCAT I/O
|
||||
effects.
|
||||
|
||||
Without a rig, TwinCAT correctness is trusted from the fake matching
|
||||
reality, which has held across field deployments so far.
|
||||
|
||||
## Key fixture / config files
|
||||
|
||||
- `tests/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT.Tests/FakeTwinCATClient.cs` —
|
||||
in-process fake with the notification-fire harness used by
|
||||
`TwinCATNativeNotificationTests`
|
||||
- `src/ZB.MOM.WW.OtOpcUa.Driver.TwinCAT/TwinCATDriver.cs` — ctor takes
|
||||
`ITwinCATClientFactory`
|
||||
Reference in New Issue
Block a user