FOCAS — retire Tier-C split, inline managed wire client, make read-only

Migration closes the FOCAS Tier-C architecture. OtOpcUa previously had
`Driver.FOCAS.Host` (NSSM-wrapped Windows service loading Fwlib64.dll via
P/Invoke) + `Driver.FOCAS.Shared` (MessagePack IPC contracts) + a C shim
DLL stand-in for unit tests. All of it is deleted; the driver is now a
single in-process managed assembly talking the FOCAS/2 Ethernet binary
protocol directly on TCP:8193.

Architecture

- Pure-managed `FocasWireClient` inlined at `src/.../Driver.FOCAS/Wire/`
  (owner-imported — see Wire/FocasWireClient.cs for the full surface).
  Opens two TCP sockets, runs the initiate handshake, serialises requests
  on socket 2 through a semaphore, closes cleanly with PDU + socket
  teardown. Both sync `IDisposable` and async `IAsyncDisposable`.
- `WireFocasClient` (same folder) adapts the wire client to OtOpcUa's
  `IFocasClient` surface — fixed-tree reads, PARAM/MACRO/PMC addresses,
  alarms. Writes return `BadNotWritable` by design — OtOpcUa is read-only
  against FOCAS.
- `FocasDriverFactoryExtensions` now accepts `"Backend": "wire"` (default)
  and `"Backend": "unimplemented"`. Legacy `ipc` and `fwlib` backends are
  rejected at startup with a diagnostic pointing at the migration doc.

Deletions

- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Host/` — whole project + Ipc/,
  Backend/, Stability/, Program.cs.
- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Shared/` — Contracts/, FrameReader,
  FrameWriter, whole project.
- `tests/...Driver.FOCAS.Host.Tests/` + `.Shared.Tests/` — whole projects.
- `src/.../Driver.FOCAS/FwlibNative.cs` + `FwlibFocasClient.cs` — 21
  P/Invokes + 7 `Pack=1` marshalling structs + the Fwlib-backed
  `IFocasClient` implementation.
- `src/.../Driver.FOCAS/Ipc/` + `Supervisor/` — IPC client wrapper +
  Host-process supervisor (backoff, circuit breaker, heartbeat, post-
  mortem reader, process launcher).
- `scripts/install/Install-FocasHost.ps1` — NSSM service installer.
- `tests/.../Driver.FOCAS.Tests/{IpcFocasClientTests, IpcLoopback,
  FwlibNativeHelperTests, PostMortemReaderCompatibilityTests,
  SupervisorTests, FocasDriverFactoryExtensionsTests}.cs` — tests that
  exercised the retired surfaces.
- `tests/.../Driver.FOCAS.IntegrationTests/Shim/` — the zig-built C shim
  DLL that masqueraded as Fwlib64.dll.

Solution changes

- `ZB.MOM.WW.OtOpcUa.slnx` drops the 4 retired project refs.
- `src/.../Driver.FOCAS.csproj` drops the Shared ProjectReference, adds
  `Microsoft.Extensions.Logging.Abstractions` for the optional `ILogger`
  hook in `FocasWireClient`.
- `src/.../Driver.FOCAS.Cli.csproj` drops the six `<Content Include>`
  entries that copied `vendor/fanuc/*.dll` into the CLI bin. CLI now uses
  `WireFocasClient` directly.
- `FocasDriver` default factory flips to `Wire.WireFocasClientFactory`.

Integration tests

- New `tests/.../Driver.FOCAS.IntegrationTests/` project covering fixed-
  tree reads (identity, axes, dynamic, program, operation mode, timers,
  spindle load + max RPM, servo meters), user-authored PARAM / MACRO /
  PMC reads, `DiscoverAsync` emission, `SubscribeAsync` + `OnDataChange`,
  `IAlarmSource` raise/clear transitions, and `ProbeAsync` /
  `OnHostStatusChanged`. 9 e2e tests against the focas-mock fixture
  (Docker container with the vendored Python mock's native FOCAS/2
  Ethernet responder).
- `scripts/integration/run-focas.ps1` orchestrates compose up → tests →
  compose down. Dropped the shim-build stage + DLL-copy step + the split
  testhost workaround (the latter only existed because of native-DLL
  lifecycle bugs the shim tripped).
- Docker compose collapses from 11 per-series services to one `focas-sim`
  service. Tests seed per-series state via `mock_load_profile` at test
  start.
- Vendored focas-mock snapshot refreshed to pick up upstream's native
  FOCAS/2 Ethernet responder (was 660 lines, now 1018) — the
  pre-refresh snapshot only spoke the JSON admin protocol.

Tests

- 145/145 unit tests in `Driver.FOCAS.Tests` pass (was 208 pre-deletion;
  63 removed tests exercised the retired IPC/shim/supervisor/Fwlib
  surfaces).
- 9/9 integration tests pass against the refreshed mock.
- `FocasScaffoldingTests.Unimplemented_factory_throws_on_Create…` updated
  to assert the new diagnostic message pointing at
  `docs/drivers/FOCAS.md` rather than the now-gone `Fwlib64.dll`.

Docs

- `docs/drivers/FOCAS.md` rewritten for the managed wire topology —
  deployment collapses to one `"Backend": "wire"` config block, no
  separate service, no DLL deployment, no pipe ACL.
- `docs/drivers/FOCAS-Test-Fixture.md` updated — single TCP probe skip
  gate instead of TCP + shim probe; fewer moving parts.
- `docs/drivers/README.md` row for FOCAS reflects the Tier-A managed
  topology (previously listed Tier-C + `Fwlib64.dll` P/Invoke).
- `docs/Driver.FOCAS.Cli.md` drops the Tier-C architecture-note section.
- `docs/v2/implementation/focas-isolation-plan.md` marked historical —
  the plan it documents was executed then superseded by the wire client.
- `docs/v2/v2-release-readiness.md` re-audited 2026-04-24. Phase 5
  driver complement closed. FOCAS change-log entry added.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-04-24 14:10:59 -04:00
parent 404b54add0
commit 4b0664bd55
105 changed files with 19530 additions and 4873 deletions

View File

@@ -5,27 +5,22 @@ protocol. Uses the **same** `FocasDriver` the OtOpcUa server does — PMC R/G/F
file registers, axis bits, parameters, and macro variables — all through
`FocasAddressParser` syntax.
Sixth of the driver test-client CLIs, added alongside the Tier-C isolation
work tracked in task #220.
Sixth of the driver test-client CLIs.
## Architecture note
FOCAS is a Tier-C driver: `Fwlib32.dll` is a proprietary 32-bit Fanuc library
with a documented habit of crashing its hosting process on network errors.
The target runtime deployment splits the driver into an in-process
`FocasProxyDriver` (.NET 10 x64) and an out-of-process `Driver.FOCAS.Host`
(.NET 4.8 x86 Windows service) that owns the DLL — see
[v2/implementation/focas-isolation-plan.md](v2/implementation/focas-isolation-plan.md)
and
[v2/implementation/phase-6-1-resilience-and-observability.md](v2/implementation/phase-6-1-resilience-and-observability.md)
for topology + supervisor / respawn / back-pressure design.
FOCAS is an in-process driver. The pure-managed `WireFocasClient`
speaks the FOCAS2 binary protocol directly over TCP:8193, removing the
Tier-C process-isolation split that the historical P/Invoke + out-of-
process Host arrangement required. The CLI loads `FocasDriver` with
`WireFocasClientFactory` and talks to the CNC without any native
components.
The CLI skips the proxy and loads `FocasDriver` directly (via
`FwlibFocasClientFactory`, which P/Invokes `Fwlib32.dll` in the CLI's own
process). There is **no public simulator** for FOCAS; a meaningful probe
requires a real CNC + a licensed `Fwlib32.dll` on `PATH` (or next to the
executable). On a dev box without the DLL, every wire call surfaces as
`BadCommunicationError` — still useful as a "CLI wire-up is correct" signal.
A dev-friendly mock is available — start
`tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Docker/docker-compose.yml`
and point `--cnc-host` at `localhost` for end-to-end CLI exercises
without a real CNC. See
[drivers/FOCAS-Test-Fixture.md](drivers/FOCAS-Test-Fixture.md).
## Build + run
@@ -152,7 +147,8 @@ fails.
**"Why did this macro flip?"** → `subscribe` to the macro, let the
operator reproduce the cycle, watch the HH:mm:ss.fff timeline.
**"Is the Fwlib32 DLL wired up?"** → `probe` against any host. A
`DllNotFoundException` surfacing as `BadCommunicationError` with a
matching `Last error` line means the driver is loading but the DLL is
missing; anything else means a transport-layer problem.
**"Can I reach the CNC on TCP:8193?"** → `probe` against any host. A
`BadCommunicationError` means the wire client couldn't open a socket
(firewall / wrong host / FOCAS Ethernet option unlicensed on the CNC).
`BadDeviceFailure` after a successful connect means the CNC is rejecting
the session setup — check the CNC's FOCAS option and password settings.

View File

@@ -2,132 +2,149 @@
Coverage map + gap inventory for the FANUC FOCAS2 CNC driver.
**TL;DR: there is no integration fixture.** Every test uses a
`FakeFocasClient` injected via `IFocasClientFactory`. Fanuc's FOCAS library
(`Fwlib32.dll`) is closed-source proprietary with no public simulator;
CNC-side behavior is trusted from field deployments.
**Status:** as of 2026-04-24, OtOpcUa speaks FOCAS2 directly over TCP
via the pure-managed [`Focas.Wire`](https://github.com/Ladder99/focas-mock/tree/main/dotnet/Focas.Wire)
client. Integration tests run the managed driver end-to-end against the
vendored `focas-mock` Python server (at
[`tests/.../Docker/focas-mock/`](../../tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Docker/focas-mock/VENDORED.md))
whose native FOCAS Ethernet responder is verified PDU-by-PDU against the
real `fwlibe64.dll`.
## What the fixture is
No shim DLL, no P/Invoke, no licensed binary — any dev box or CI runner
with Docker can run the full fixture end-to-end.
Nothing at the integration layer.
`tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/` is unit-only. The driver ships
as Tier C (process-isolated) per `docs/v2/driver-stability.md` because the
FANUC DLL has known crash modes; tests can't replicate those in-process.
Hardware validation against a real CNC is still useful to catch
series-specific firmware quirks (see [§ Hardware-only gaps](#hardware-only-gaps))
but the mock's wire responder covers every FOCAS call OtOpcUa issues.
## What it actually covers (unit only)
## What the fixture covers
- `FocasCapabilityTests` — data-type mapping (PMC bit / word / float,
macro variable types, parameter types)
- `FocasCapabilityMatrixTests` — per-CNC-series range validation (macro
/ parameter / PMC letter + number) across 16i / 0i-D / 0i-F /
30i / PowerMotion. See [`docs/v2/focas-version-matrix.md`](../v2/focas-version-matrix.md)
for the authoritative matrix. 46 theory cases lock every documented
range boundary — widening a range without updating the doc fails a
test.
- `FocasReadWriteTests` — read + write against the fake, FOCAS native status
→ OPC UA StatusCode mapping
### Unit layer (no container required)
`tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/` uses `FakeFocasClient`
injected via `IFocasClientFactory`:
- `FocasCapabilityTests` — data-type mapping (PMC bit / byte / word /
long / float / double, macro variable types, parameter types)
- `FocasCapabilityMatrixTests` — per-CNC-series range validation across
16i / 0i-D / 0i-F / 30i / Power Motion, 46 theory cases locking every
documented range boundary. See
[`docs/v2/focas-version-matrix.md`](../v2/focas-version-matrix.md).
- `FocasReadWriteTests` — read / write contract against the fake, FOCAS
native status → OPC UA `StatusCode` mapping
- `FocasScaffoldingTests``IDriver` lifecycle + multi-device routing
- `FocasPmcBitRmwTests` — PMC bit read-modify-write synchronization (per-byte
`SemaphoreSlim`, mirrors the AB / Modbus pattern from #181)
- `FwlibNativeHelperTests``Focas32.dll``Fwlib32.dll` bridge validation
+ P/Invoke signature validation
- `FocasPmcBitRmwTests` — PMC bit read-modify-write synchronisation
- `FocasAlarmProjectionTests` — raise / clear diffing, severity mapping
- `FocasHandleRecycleTests` — proactive session recycle cadence
Capability surfaces whose contract is verified: `IDriver`, `IReadable`,
`IWritable`, `ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
`IPerCallHostResolver`.
`ITagDiscovery`, `ISubscribable`, `IHostConnectivityProbe`,
`IPerCallHostResolver`, `IAlarmSource`. `IWritable` intentionally
returns `BadNotWritable` — OtOpcUa is read-only against FOCAS.
Pre-flight validation runs in `FocasDriver.InitializeAsync` — configs
referencing out-of-range addresses fail at load time with a diagnostic
message naming the CNC series + documented limit. This closes the
cheap half of the hardware-free stability gap; Tier-C process
isolation (task #220) closes the expensive half — see
[`docs/v2/implementation/focas-isolation-plan.md`](../v2/implementation/focas-isolation-plan.md).
message naming the CNC series + documented limit.
## What it does NOT cover
### Integration layer (mock only, no CNC, no shim)
### 1. FOCAS wire traffic
`tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/` drives the
managed `FocasDriver` end-to-end. A single gate:
No FOCAS TCP frame is sent. `Fwlib32.dll`'s TCP-to-FANUC-gateway exchange is
closed-source; the driver trusts the P/Invoke layer per #193. Real CNC
correctness is trusted from field deployments.
**Docker compose up** — tests skip when the TCP probe to
`localhost:8193` fails with a pointer to the compose command.
### 2. Alarm / parameter-change callbacks
When the mock is up, `WireFocasClient` dials it over TCP exactly like a
real CNC, and the mock's native FOCAS Ethernet responder replies with
binary PDUs against the documented command IDs. Covered assertions:
FOCAS has no push model — the driver polls via the shared `PollGroupEngine`.
There are no CNC-initiated callbacks to test; the absence is by design.
- Session open / close (`cnc_allclibhndl3` + `cnc_freelibhndl`)
- Parameter read-back after `mock_patch` seed → `cnc_rdparam`
- Macro read-back after seed → `cnc_rdmacro` (scaled-decimal
translation verified)
- PMC range read after seed → `pmc_rdpmcrng`
- `IAlarmSource` raise + clear transitions after `mock_patch`
alarm-list changes → `cnc_rdalmmsg2`
- Fixed-tree bootstrap: identity / axes / spindle / program / timers /
servo meters populate via `cnc_sysinfo`, `cnc_rdaxisname`,
`cnc_rdspdlname`, `cnc_rddynamic2`, `cnc_exeprgname2`,
`cnc_rdblkcount`, `cnc_rdopmode`, `cnc_rdsvmeter`, `cnc_rdspload`,
`cnc_rdspmaxrpm`, `cnc_rdtimer`
- Per-series profile selection via `mock_load_profile` — tests can
pin one profile and assert series-gated capability suppression
### 3. Macro / ladder variable types
### E2E script (CLI)
FANUC has CNC-specific extensions (macro variables `#100-#999`, system
variables `#1000-#5000`, PMC timers / counters / keep-relays) whose
per-address semantics differ across 0i-F / 30i / 31i / 32i Series. Driver
covers the common address shapes; per-model quirks are not stressed.
`scripts/e2e/test-focas.ps1` drives the Client.CLI against a running
OtOpcUa server. Accepts:
### 4. Model-specific behavior
- `-CncHost` / `-CncPort` for real hardware
- `-ProfileName <compose-profile>` for the Docker mock
- `-Series <csv>` for per-series matrix mode
- `-HandleLeakCycles <N>` for handle-leak stress
- Alarm retention across power cycles (model-specific CNC behavior)
- Parameter range enforcement (CNC rejects out-of-range writes)
- MTB (machine tool builder) custom screens that expose non-standard data
## Hardware-only gaps
### 5. Tier-C process isolation — architecture shipped, Fwlib32 integration hardware-gated
The mock has parity with the real `fwlibe64.dll` for the calls OtOpcUa
issues, but a real CNC can still surface things a reference
implementation can't:
The Tier-C architecture is now in place as of PRs #169#173 (FOCAS
PR AE, task #220):
1. **Series-specific firmware quirks** — alarm retention across power
cycles, parameter range enforcement by the CNC (not the driver),
MTB custom screens, series-specific option bits. Each series has
documented behaviours that only a bench CNC exercises.
2. **Wire-level stress** — burst reads, concurrent device writes,
network-partition recovery under load. The mock handles these
correctly but production behaviour is the source of truth.
3. **Transient operational states** — alarm floods, emergency-stop
transitions, power-on resync. These are easy to stub but hard to
cover comprehensively in the mock.
- `Driver.FOCAS.Shared` carries MessagePack IPC contracts
- `Driver.FOCAS.Host` (.NET 4.8 x86 Windows service via NSSM) accepts
a connection on a strictly-ACL'd named pipe + dispatches frames to
an `IFocasBackend`
- `Driver.FOCAS.Ipc.IpcFocasClient` implements the `IFocasClient` DI
seam by forwarding over IPC — swap the DI registration and the
driver runs Tier-C with zero other changes
- `Driver.FOCAS.Supervisor.FocasHostSupervisor` owns the spawn +
heartbeat + respawn + 3-in-5min crash-loop breaker + sticky alert
- `Driver.FOCAS.Host.Stability.PostMortemMmf`
`Driver.FOCAS.Supervisor.PostMortemReader` — ring-buffer of the
last ~1000 IPC operations survives a Host crash
Track the close-out under task #54 (live-CNC smoke). When the rig
lands, the hardware path runs alongside the mock path; the mock
stays as the CI quality gate.
The one remaining gap is the production `FwlibHostedBackend`: an
`IFocasBackend` implementation that wraps the licensed
`Fwlib32.dll` P/Invoke. That's hardware-gated on task #222 — we
need a CNC on the bench (or the licensed FANUC developer kit DLL
with a test harness) to validate it. Until then, the Host ships
`FakeFocasBackend` + `UnconfiguredFocasBackend`. Setting
`OTOPCUA_FOCAS_BACKEND=fake` lets operators smoke-test the whole
Tier-C pipeline end-to-end without any CNC.
## When to trust each layer
## When to trust FOCAS tests, when to reach for a rig
| Question | Unit | Integration (mock) | Real CNC |
| --- | :---: | :---: | :---: |
| "Does PMC address `R100.3` route to the right bit?" | ✅ | ✅ | ✅ |
| "Does the Fanuc status → OPC UA StatusCode map cover every documented code?" | ✅ (contract) | ✅ | ✅ |
| "Does `FocasDriver.ReadAsync` correctly decode a seeded parameter?" | no | ✅ | ✅ |
| "Does `IAlarmSource` fire raise + clear events?" | ✅ (Fake) | ✅ (wire) | ✅ |
| "Does a real read against a 30i Series return correct bytes?" | no | ✅ (via profile) | ✅ (required) |
| "Do series-specific firmware quirks behave as documented?" | no | no | ✅ (required) |
| "Does the driver survive real network partitions?" | no | partial (socket kill) | ✅ (required) |
| Question | Unit tests | Real CNC |
| --- | --- | --- |
| "Does PMC address `R100.3` route to the right bit?" | yes | yes |
| "Does the FANUC status → OPC UA StatusCode map cover every documented code?" | yes (contract) | yes |
| "Does a real read against a 30i Series return correct bytes?" | no | yes (required) |
| "Does `Fwlib32.dll` crash on concurrent reads?" | no | yes (stress) |
| "Do macro variables round-trip across power cycles?" | no | yes (required) |
## Running the integration fixture
## Follow-up candidates
```powershell
# 1) Start the mock on a chosen profile.
docker compose -f tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Docker/docker-compose.yml up -d
1. **Nothing public** — Fanuc's FOCAS Developer Kit ships an emulator DLL
but it's under NDA + tied to licensed dev-kit installations; can't
redistribute for CI.
2. **Lab rig** — used FANUC 0i-F simulator controller (or a retired machine
tool) on a dedicated network; only path that covers real CNC behavior.
3. **Process isolation first** — before trusting FOCAS in production at
scale, shipping the Tier-C out-of-process Host architecture (similar to
Galaxy) is higher value than a CI simulator.
# 2) Run the tests. No shim build, no DLL copy — the driver dials the mock directly.
dotnet test tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/
```
Or use `scripts/integration/run-focas.ps1` which wraps compose up / test
/ compose down and accepts `-Profile <name>` to pin a per-series run.
## Key fixture / config files
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Docker/focas-mock/`
— vendored `focas-mock` Python source + Dockerfile
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Docker/docker-compose.yml`
— per-series compose profiles
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/FocasSimFixture.cs`
— collection fixture + mock admin API client
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Series/FixedTreePopulatesTests.cs`
— fixed-tree end-to-end tests
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/Series/WireBackendTests.cs`
— pure-wire-backend end-to-end tests
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/FakeFocasClient.cs`
in-process fake implementing `IFocasClient`
- `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/FocasCapabilityMatrixTests.cs`
— parameterized theories locking the per-series matrix
- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs` — ctor takes
`IFocasClientFactory`
in-process unit fake
- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/Wire/WireFocasClient.cs` — the
managed wire client backing production deployments
- `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasCapabilityMatrix.cs`
per-CNC-series range validator (the matrix the doc describes)
per-series range validator
- `docs/v2/focas-version-matrix.md` — authoritative range reference
- `docs/v2/implementation/focas-isolation-plan.md` — Tier-C isolation
plan (task #220)
- `docs/v2/driver-stability.md` — Tier C scope + process-isolation rationale

238
docs/drivers/FOCAS.md Normal file
View File

@@ -0,0 +1,238 @@
# FOCAS Driver
Getting-started guide for the FANUC FOCAS2 driver. This is the short path — for
the exhaustive per-node mapping read [`docs/v2/driver-specs.md §7`](../v2/driver-specs.md),
for deployment details read [`docs/v2/focas-deployment.md`](../v2/focas-deployment.md),
for the test-harness map read [FOCAS-Test-Fixture.md](FOCAS-Test-Fixture.md).
## What it talks to
FANUC CNCs (0i-D / 0i-F / 0i-MF / 0i-TF / 16i / 30i / 31i / 32i / Power Motion i)
over the proprietary FOCAS2 protocol on TCP port 8193. The wire is spoken
directly by the pure-managed [`Focas.Wire`](https://github.com/Ladder99/focas-mock)
client — no Fwlib64.dll, no P/Invoke, no out-of-process isolation needed.
OtOpcUa is **read-only** against FOCAS; all reads go over the native wire
protocol using the documented command IDs. Writes return
`BadNotWritable` by design.
## Project split
| Project | Target | Role |
|---------|--------|------|
| `src/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/` | net10.0 | In-process driver — hosts `WireFocasClient` which speaks FOCAS2 over TCP directly |
Previous `Driver.FOCAS.Host` / `Driver.FOCAS.Shared` Tier-C split has been
retired — the managed wire client removes the native-crash blast radius
that justified the out-of-process service.
## Minimum deployment
Register the driver instance in the main server's `appsettings.json`. No
separate service, no DLL deployment, no shared-secret handshake:
```jsonc
"Drivers": {
"focas-cnc-1": {
"Type": "FOCAS",
"Config": {
"Backend": "wire",
"Devices": [
{ "HostAddress": "focas://10.20.30.40:8193", "Series": "ThirtyOne_i" }
],
"Tags": [
{ "Name": "Mode", "DeviceHostAddress": "focas://10.20.30.40:8193",
"Address": "PARAM:3402", "DataType": "Int32", "Writable": false },
{ "Name": "SpndLoad", "DeviceHostAddress": "focas://10.20.30.40:8193",
"Address": "MACRO:500", "DataType": "Float64", "Writable": false }
]
}
}
}
```
The main server opens two TCP sockets per configured device and speaks the
FOCAS2 binary protocol directly. No local privileged components, no
platform bitness constraint — the driver runs on every host OtOpcUa runs
on.
## Address forms
| Form | Example | Meaning |
|------|---------|---------|
| `X0.0` / `R100` / `R100.3` | PMC bit or byte | Letter + number; optional `.bit` for bit access |
| `PARAM:1815` / `PARAM:1815/0` | CNC parameter | Number + optional axis index |
| `MACRO:500` | Custom macro variable | System / user macro variable number |
Addresses are validated against the per-device `Series` at `InitializeAsync`
a config referencing a number outside the documented range for that series
fails at load time with an error message naming the limit. See
[`docs/v2/focas-version-matrix.md`](../v2/focas-version-matrix.md) for the
authoritative range table.
## Backend selection
The driver picks its client from `Config.Backend`:
| Value | Client | Use it for |
|-------|--------|------------|
| `wire` (default) | `WireFocasClient` | Production — pure-managed FOCAS2 over TCP |
| `unimplemented` / `none` / `stub` | `UnimplementedFocasClientFactory` | Scaffolding a DriverInstance row before the CNC endpoint is reachable |
Previous backends (`fwlib`, `fwlib32`, `ipc`) have been retired along
with `Driver.FOCAS.Host` and the Fwlib P/Invoke path. Configs that still
reference them will throw at startup with a message pointing here.
## Capability surface
| Capability | Wire path | Notes |
|------------|-----------|-------|
| `IReadable` | `ReadAsync``cnc_rdpmcrng` / `cnc_rdparam` / `cnc_rdmacro` | One TCP request/response per read; `Focas.Wire` serializes requests on socket 2 internally |
| `IWritable` | returns `BadNotWritable` | OtOpcUa is read-only against FOCAS by design — no `cnc_wrparam` / `pmc_wrpmcrng` / `cnc_wrmacro` path is implemented |
| `ITagDiscovery` | `DiscoverAsync` | Emits `FOCAS/{device}/{tag}` folders per configured device |
| `ISubscribable` | polled via shared `PollGroupEngine` | FOCAS has no push model — subscriptions turn into per-tag polling groups |
| `IHostConnectivityProbe` | periodic `cnc_rdcncstat` | Probe cadence is `Probe.Interval`; transitions fire `OnHostStatusChanged` |
| `IPerCallHostResolver` | lookup in `_tagsByName` | Each call routes to the device of the referenced tag |
| `IAlarmSource` | polled `cnc_rdalmmsg2` via `FocasAlarmProjection` | Opt-in — set `AlarmProjection.Enabled=true`; diffs `(AlarmNumber, Type)` between ticks |
Ack is a no-op — FANUC clears alarms on its own once the underlying condition
resolves, so `AcknowledgeAsync` swallows the batch rather than surfacing
`BadNotSupported`.
## Fixed node tree
Enable a pre-defined hierarchy of CNC nodes populated automatically from
`cnc_sysinfo` + `cnc_rdaxisname` + `cnc_rddynamic2` + related FWLIB calls,
so operators get an out-of-the-box view of identity / axes / program /
timers without declaring per-address tags.
```jsonc
"Config": {
"Devices": [ ... ],
"Tags": [ ... ],
"FixedTree": {
"Enabled": true,
"PollInterval": "00:00:00.250", // fast — per-axis dynamic reads
"ProgramPollInterval": "00:00:01", // medium — program + mode changes
"TimerPollInterval": "00:00:30" // slow — cumulative counters
}
}
```
What gets populated (all under `FOCAS/{deviceHostAddress}/`):
| Subtree | Nodes | Source call |
|---------|-------|-------------|
| `Identity/` | `SeriesNumber`, `Version`, `MaxAxes`, `CncType`, `MtType`, `AxisCount` | `cnc_sysinfo` once at bootstrap |
| `Axes/{name}/` | `AbsolutePosition`, `MachinePosition`, `RelativePosition`, `DistanceToGo`, `ServoLoad` — one folder per discovered axis | `cnc_rdaxisname` once + `cnc_rddynamic2` + `cnc_rdsvmeter` per tick |
| `Axes/FeedRate/Actual`, `Axes/SpindleSpeed/Actual` | Current feed + spindle RPM | `cnc_rddynamic2` |
| `Spindle/{name}/` | `Load` (percentage), `MaxRpm` — one folder per discovered spindle | `cnc_rdspdlname` once + `cnc_rdspload` + `cnc_rdspmaxrpm` |
| `Program/` | `Name` (filename), `ONumber`, `Number`, `MainNumber`, `Sequence`, `BlockCount` | `cnc_exeprgname2` + `cnc_rdblkcount` + cached `cnc_rddynamic2` |
| `OperationMode/` | `Mode` (int), `ModeText` ("AUTO", "MDI", "EDIT", …) | `cnc_rdopmode` |
| `Timers/` | `PowerOnSeconds`, `OperatingSeconds`, `CuttingSeconds`, `CycleSeconds` | `cnc_rdtimer` × 4 |
### Per-series node suppression
The driver probes each optional call once at bootstrap. If the target CNC
returns `EW_FUNC` / `EW_NOOPT` / `EW_VERSION` on the wire, the
corresponding subtree is **not emitted** — the operator doesn't see nodes
that will only ever return `BadDeviceFailure`. Capability suppression
covers `Spindle/`, `Program/` + `OperationMode/`, `Timers/`, and
per-axis `ServoLoad` independently. Identity + `Axes/*` position reads
(which every Fanuc CNC supports) are always emitted.
Position values are scaled integers (matching FOCAS's convention). The
managed side exposes them as `Float64` OPC UA nodes; a future
`cnc_getfigure` integration will add per-axis decimal scaling. Until
then, treat the raw integer as the value the CNC reports and scale on
the client side if decimal precision matters.
**Still user-authored**: `PARAM:6711`, `MACRO:500`, `R100` etc. — specific
numbers whose meaning is MTB-specific. Those go under the device folder
alongside the fixed subtree.
## Alarm projection
Alarm surfacing is **disabled by default** because the polling cost is wasted
on sites that don't consume CNC alarms. Opt in per driver instance:
```jsonc
"Config": {
"Devices": [ ... ],
"Tags": [ ... ],
"AlarmProjection": {
"Enabled": true,
"PollInterval": "00:00:02"
}
}
```
Every alarm transition fires `OnAlarmEvent` with:
- `SourceNodeId` = the device host address (FOCAS has no per-node alarm model;
the CNC exposes a single flat active-alarm list per session)
- `ConditionId` = `"{host}#{Type}:{AlarmNumber}"`
- `AlarmType` = projected from FANUC's `ALM_TYPE_*` (e.g. `Overtravel`, `Servo`,
`Parameter`, `MacroAlarm`)
- `Severity` = Overtravel / Servo / PulseCode → `Critical`; Parameter / Macro
`Medium`; everything else → `High`
Cleared alarms fire a second event with `" (cleared)"` appended to the message
so downstream consumers can ignore the clear if they only care about raises.
## Handle recycling
FANUC CNCs have a finite FWLIB handle pool (~510 concurrent connections) and
certain series have documented handle-leak bugs that manifest after long uptime.
The driver can proactively close + reopen each device's session on a cadence to
release its slot back to the pool:
```jsonc
"Config": {
"Devices": [ ... ],
"HandleRecycle": {
"Enabled": true,
"Interval": "01:00:00"
}
}
```
Disabled by default — a healthy CNC + driver doesn't need it. Enable when field
experience shows handle exhaustion. Typical tuning: 30 min for sites running
multiple OtOpcUa instances against the same CNC (they share the pool); 6 h for a
single-client deployment. Reads / writes during recycle simply wait for the
reconnect rather than failing — worst case, an operator sees a brief read
latency spike once per cadence.
## Testing
- **Unit tests** — `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/` cover the
driver surface via `FakeFocasClient`. Includes the alarm-projection raise /
clear diffing tests.
- **Integration tests** — `tests/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.IntegrationTests/`
hold the Docker simulator scaffold (Stream B / C of the simulator plan —
`docs/v2/implementation/focas-simulator-plan.md`).
- **E2E script** — `scripts/e2e/test-focas.ps1` stages Host + Proxy + a real
CNC (or the simulator) and exercises connect → read → write → subscribe
round-trips. See [`docs/drivers/FOCAS-Test-Fixture.md`](FOCAS-Test-Fixture.md)
for the coverage map.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| `BadCommunicationError` on every read | CNC unreachable on TCP:8193 | Check firewall / LAN reachability; FOCAS Ethernet option must be licensed on the CNC side |
| Every read returns `BadNotWritable` on writes | Expected — OtOpcUa is read-only against FOCAS | If you actually need writes, open a feature request — the driver's managed wire client doesn't expose the write commands |
| `BadOutOfRange` on reads for a macro/parameter | Config address outside the declared `Series` range | Check `docs/v2/focas-version-matrix.md` — either fix the address or widen the `Series` |
| Alarm events never fire | `AlarmProjection.Enabled` left at default (false) | Set it to `true` in the driver config |
## Further reading
- [`docs/v2/driver-specs.md §7`](../v2/driver-specs.md) — full OPC UA node
mapping, pre-defined tag set, per-API notes
- [`docs/v2/focas-version-matrix.md`](../v2/focas-version-matrix.md) —
per-series macro / parameter / PMC range table
- [`docs/v2/implementation/focas-wire-protocol.md`](../v2/implementation/focas-wire-protocol.md)
— captured FOCAS2 wire semantics (magic prefix, handshake, command-id table)
- [upstream `Focas.Wire`](https://github.com/Ladder99/focas-mock/tree/main/dotnet/Focas.Wire)
— the managed client implementation OtOpcUa consumes as a NuGet dependency

View File

@@ -26,7 +26,7 @@ Driver type metadata is registered at startup in `DriverTypeRegistry` (`src/ZB.M
| AB CIP | `Driver.AbCip` | A | libplctag CIP | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IAlarmSource | ControlLogix / CompactLogix. Tag discovery uses the `@tags` walker to enumerate controller-scoped + program-scoped symbols; UDT member resolution via the UDT template reader |
| AB Legacy | `Driver.AbLegacy` | A | libplctag PCCC | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver | SLC 500 / MicroLogix. File-based addressing (`N7:0`, `F8:0`) — no symbol table, tag list is user-authored in the config DB |
| TwinCAT | `Driver.TwinCAT` | B | Beckhoff `TwinCAT.Ads` (`TcAdsClient`) | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver | The only native-notification driver outside Galaxy — ADS delivers `ValueChangedCallback` events the driver forwards straight to `ISubscribable.OnDataChange` without polling. Symbol tree uploaded via `SymbolLoaderFactory` |
| FOCAS | `Driver.FOCAS` | C | FANUC FOCAS2 (`Fwlib32.dll` P/Invoke) | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver | Tier C — FOCAS DLL has crash modes that warrant process isolation. CNC-shaped data model (axes, spindle, PMC, macros, alarms) not a flat tag map |
| [FOCAS](FOCAS.md) | `Driver.FOCAS` | A | Pure-managed `FocasWireClient` FOCAS/2 Ethernet binary protocol on TCP:8193, inlined into the driver assembly | IDriver, ITagDiscovery, IReadable, ISubscribable, IHostConnectivityProbe, IPerCallHostResolver, IAlarmSource | Read-only by design (WriteAsync returns `BadNotWritable`). CNC-shaped data model (axes, spindle, PMC, macros, alarms) not a flat tag map. Previously Tier-C (Host + P/Invoke + shim DLL); retired in the 2026-04-24 migration when the managed wire client landed |
| OPC UA Client | `Driver.OpcUaClient` | B | OPCFoundation `Opc.Ua.Client` | IDriver, ITagDiscovery, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe | Gateway/aggregation driver. Opens a single `Session` against a remote OPC UA server and re-exposes its address space. Owns its own `ApplicationConfiguration` (distinct from `Client.Shared`) because it's always-on with keep-alive + `TransferSubscriptions` across SDK reconnect, not an interactive CLI |
## Per-driver documentation
@@ -35,6 +35,9 @@ Driver type metadata is registered at startup in `DriverTypeRegistry` (`src/ZB.M
- [Galaxy.md](Galaxy.md) — COM bridge, STA pump, IPC, runtime probes
- [Galaxy-Repository.md](Galaxy-Repository.md) — ZB SQL reader, `LocalPlatform` scope filter, change detection
- **FOCAS** has a short getting-started doc because the Tier-C two-project deployment + backend-selection env var + alarm projection opt-in all need explaining up front:
- [FOCAS.md](FOCAS.md) — deployment, config, capability surface, alarm projection, troubleshooting
- **All other drivers** share a single per-driver specification in [docs/v2/driver-specs.md](../v2/driver-specs.md) — addressing, data-type maps, connection settings, and quirks live there. That file is the authoritative per-driver reference; this index points at it rather than duplicating.
## Test-fixture coverage maps

View File

@@ -6,7 +6,7 @@ enforces at driver init time. Every row cites the Fanuc FOCAS Developer
Kit function whose documented input range determines the ceiling.
**Why this exists** — we have no FOCAS hardware on the bench and no
working simulator. Fwlib32 returns `EW_NUMBER` / `EW_PARAM` when you
working simulator. FWLIB (Fwlib64, or Fwlib32 on legacy deployments) returns `EW_NUMBER` / `EW_PARAM` when you
hand it an address outside the controller's supported range; the
driver would map that to a per-read `BadOutOfRange` at steady state.
Catching at `InitializeAsync` with this matrix surfaces operator
@@ -140,6 +140,6 @@ matrix: Macro variable #50000 is outside the documented range
This validation closes the cheap half of the FOCAS hardware-free
stability gap — config errors now fail at load instead of per-read.
The expensive half is Tier-C process isolation so that a crashing
`Fwlib32.dll` doesn't take the main OPC UA server down with it. See
`Fwlib64.dll` doesn't take the main OPC UA server down with it. See
[`docs/v2/implementation/focas-isolation-plan.md`](implementation/focas-isolation-plan.md)
for that plan (task #220).

View File

@@ -1,10 +1,21 @@
# FOCAS Tier-C isolation — plan for task #220
> **Status**: PRs AE shipped. Architecture is in place; the only
> remaining FOCAS work is the hardware-dependent production
> integration of `Fwlib32.dll` into a real `IFocasBackend`
> (`FwlibHostedBackend`), which needs an actual CNC on the bench
> and is tracked as a follow-up on #220.
> **Status**: **FULLY SHIPPED** (code). PRs AE shipped the architecture; the
> 2026-04-23 follow-up shipped the production `Fwlib64FocasBackend` wrapping
> the licensed `Fwlib64.dll`. Only the wire-level live-boot against real
> hardware remains (task #222 / requires a bench CNC).
>
> **Major update 2026-04-23 — Host retargeted to .NET 10 x64 + Fwlib64**:
> Both `Fwlib32.dll` and `Fwlib64.dll` are licensed for this project. The
> original plan put the Host on .NET 4.8 x86 because Fwlib32 was assumed.
> With Fwlib64 available, the Host moves to `net10.0-windows` x64 — same
> runtime as the rest of the fleet. **Tier-C isolation stays anyway** — the
> blast-radius argument against a closed-source vendor P/Invoke is independent
> of bitness. Galaxy (forced x86 by MXAccess COM) is a pure bitness forcing;
> FOCAS is a pure blast-radius choice. Body of this document still reflects
> the original x86 assumptions in a few places — read them as historical
> design context; the current shape is in `docs/drivers/FOCAS-Test-Fixture.md`
> and `exit-gate-phase-3.md`.
>
> **Pre-reqs shipped**: version matrix + pre-flight validation
> (PR #168 — the cheap half of the hardware-free stability gap).

View File

@@ -1,7 +1,7 @@
# v2 Release Readiness
> **Last updated**: 2026-04-19 (all three release blockers CLOSED — Phase 6.3 Streams A/C core shipped)
> **Status**: **RELEASE-READY (code-path)** for v2 GA — all three code-path release blockers are closed. Remaining work is manual (client interop matrix, deployment checklist signoff, OPC UA CTT pass) + hardening follow-ups; see exit-criteria checklist below.
> **Last updated**: 2026-04-24 (Phase 5 driver complement closed — AB CIP, AB Legacy, TwinCAT, FOCAS all shipped; FOCAS Tier-C retired for a pure-managed in-process client)
> **Status**: **RELEASE-READY (code-path)** for v2 GA. All three original code-path release blockers remain closed. Phase 5 is now complete. Remaining work is manual (live-hardware validations, client interop matrix, deployment checklist signoff, OPC UA CTT pass) + hardening follow-ups; see exit-criteria checklist below.
This doc is the single view of where v2 stands against its release criteria. Update it whenever a deferred follow-up closes or a new release blocker is discovered.
@@ -14,23 +14,25 @@ This doc is the single view of where v2 stands against its release criteria. Upd
| Phase 2 — Galaxy driver split (Proxy/Host/Shared) | ✓ | Shipped |
| Phase 3 — OPC UA server + LDAP + security profiles | ✓ | Shipped |
| Phase 4 — Redundancy scaffold (entities + endpoints) | ✓ | Shipped (runtime closes in 6.3) |
| Phase 5 — Drivers | ⚠ partial | Galaxy / Modbus / S7 / OpcUaClient shipped; AB CIP / AB Legacy / TwinCAT / FOCAS deferred (task #120) |
| Phase 6.1 — Resilience & Observability | ✓ | **SHIPPED** (PRs #7883) |
| Phase 6.2 — Authorization runtime | ◐ core | **SHIPPED (core)** (PRs #8488); dispatch wiring + Admin UI deferred |
| Phase 6.3 — Redundancy runtime | ◐ core | **SHIPPED (core)** (PRs #8990); coordinator + UA-node wiring + Admin UI + interop deferred |
| Phase 6.4 — Admin UI completion | ◐ data layer | **SHIPPED (data layer)** (PRs #9192); Blazor UI + OPC 40010 address-space wiring deferred |
| Phase 5 — Drivers | ✓ | **Shipped** Galaxy, Modbus (+ DL205/S7/MELSEC profiles), S7 native, OPC UA Client, AB CIP, AB Legacy, TwinCAT ADS, FOCAS (managed wire client) |
| Phase 6.1 — Resilience & Observability | ✓ | Shipped (PRs #7883) |
| Phase 6.2 — Authorization runtime | ◐ core | Core shipped (PRs #8488, #94 dispatch wiring); finer-grained Browse/Subscribe/Alarm/Call gating + 3-user interop matrix deferred |
| Phase 6.3 — Redundancy runtime | ◐ core | Core shipped (PRs #8990, #9899); peer-probe HostedServices, OPC UA variable-node binding, `sp_PublishGeneration` lease wrap, client interop matrix deferred |
| Phase 6.4 — Admin UI completion | ◐ data layer + Identification | Data layer + OPC 40010 Identification folder shipped (PRs #9192, Identification audit close-out 2026-04-23); Blazor UI pieces deferred |
**Aggregate test counts:** 906 baseline (pre-Phase-6) → **1159 passing** across Phase 6. One pre-existing Client.CLI `SubscribeCommandTests.Execute_PrintsSubscriptionMessage` flake tracked separately.
**Driver integration-test counts** (end-to-end against live or simulated targets): Modbus 26, FOCAS 9, AbCip 7, OpcUaClient 3, S7 3, AbLegacy 2, TwinCAT 2. Plus Galaxy's separate cross-FX parity/stability suite.
**Aggregate test counts** (2026-04-19 baseline): 1159 passing across the solution. One pre-existing Client.CLI `SubscribeCommandTests.Execute_PrintsSubscriptionMessage` flake tracked separately. Rerun `dotnet test ZB.MOM.WW.OtOpcUa.slnx` after the FOCAS migration commits land to refresh the number.
## Release blockers (must close before v2 GA)
Ordered by severity + impact on production fitness.
All code-path release blockers are closed. The remaining items are live-hardware / manual validations listed under exit criteria.
### ~~Security — Phase 6.2 dispatch wiring~~ (task #143 — **CLOSED** 2026-04-19, PR #94)
**Closed**. `AuthorizationGate` + `NodeScopeResolver` now thread through `OpcUaApplicationHost → OtOpcUaServer → DriverNodeManager`. `OnReadValue` + `OnWriteValue` + all four HistoryRead paths call `gate.IsAllowed(identity, operation, scope)` before the invoker. Production deployments activate enforcement by constructing `OpcUaApplicationHost` with an `AuthorizationGate(StrictMode: true)` + populating the `NodeAcl` table.
**Closed**. `AuthorizationGate` + `NodeScopeResolver` thread through `OpcUaApplicationHost → OtOpcUaServer → DriverNodeManager`. `OnReadValue` + `OnWriteValue` + all four HistoryRead paths call `gate.IsAllowed(identity, operation, scope)` before the invoker. Production deployments activate enforcement by constructing `OpcUaApplicationHost` with an `AuthorizationGate(StrictMode: true)` + populating the `NodeAcl` table.
Additional Stream C surfaces (not release-blocking, hardening only):
Remaining Stream C surfaces (hardening, not release-blocking):
- Browse + TranslateBrowsePathsToNodeIds gating with ancestor-visibility logic per `acl-design.md` §Browse.
- CreateMonitoredItems + TransferSubscriptions gating with per-item `(AuthGenerationId, MembershipVersion)` stamp so revoked grants surface `BadUserAccessDenied` within one publish cycle (decision #153).
@@ -39,42 +41,51 @@ Additional Stream C surfaces (not release-blocking, hardening only):
- Finer-grained scope resolution — current `NodeScopeResolver` returns a flat cluster-level scope. Joining against the live Configuration DB to populate UnsArea / UnsLine / Equipment path is tracked as Stream C.12.
- 3-user integration matrix covering every operation × allow/deny.
These are additional hardening — the three highest-value surfaces (Read / Write / HistoryRead) are now gated, which covers the base-security gap for v2 GA.
### ~~Config fallback — Phase 6.1 Stream D wiring~~ (task #136 — **CLOSED** 2026-04-19, PR #96)
**Closed**. `SealedBootstrap` consumes `ResilientConfigReader` + `GenerationSealedCache` + `StaleConfigFlag` end-to-end: bootstrap calls go through the timeout → retry → fallback-to-sealed pipeline; every central-DB success writes a fresh sealed snapshot so the next cache-miss has a known-good fallback; `StaleConfigFlag.IsStale` is now consumed by `HealthEndpointsHost.usingStaleConfig` so `/healthz` body reports reality.
**Closed**. `SealedBootstrap` consumes `ResilientConfigReader` + `GenerationSealedCache` + `StaleConfigFlag` end-to-end; `/healthz` surfaces the stale flag.
Production activation: Program.cs switches `NodeBootstrap → SealedBootstrap` + constructs `OpcUaApplicationHost` with the `StaleConfigFlag` as an optional ctor parameter.
Remaining follow-ups (hardening, not release-blocking):
Remaining follow-ups (hardening):
- A `HostedService` that polls `sp_GetCurrentGenerationForCluster` periodically so peer-published generations land in this node's cache without a restart.
- Richer snapshot payload via `sp_GetGenerationContent` so fallback can serve the full generation content (DriverInstance enumeration, ACL rows, etc.) from the sealed cache alone.
- Richer snapshot payload via `sp_GetGenerationContent` so fallback can serve full generation content (DriverInstance enumeration, ACL rows, etc.) from the sealed cache alone.
### ~~Redundancy — Phase 6.3 Streams A/C core~~ (tasks #145 + #147 — **CLOSED** 2026-04-19, PRs #9899)
**Closed**. The runtime orchestration layer now exists end-to-end:
- `RedundancyCoordinator` reads `ClusterNode` + peer list at startup (Stream A shipped in PR #98). Invariants enforced: 1-2 nodes (decision #83), unique ApplicationUri (#86), ≤1 Primary in Warm/Hot (#84). Startup fails fast on violation; runtime refresh logs + flips `IsTopologyValid=false` so the calculator falls to band 2 without tearing down.
- `RedundancyStatePublisher` orchestrates topology + apply lease + recovery state + peer reachability through `ServiceLevelCalculator` + emits `OnStateChanged` / `OnServerUriArrayChanged` edge-triggered events (Stream C core shipped in PR #99). The OPC UA `ServiceLevel` Byte variable + `ServerUriArray` String[] variable subscribe to these events.
**Closed**. `RedundancyCoordinator` + `RedundancyStatePublisher` + `PeerReachabilityTracker` orchestrate topology + apply lease + recovery state + peer reachability through `ServiceLevelCalculator` + emit `OnStateChanged` / `OnServerUriArrayChanged` edge-triggered events.
Remaining Phase 6.3 surfaces (hardening, not release-blocking):
- `PeerHttpProbeLoop` + `PeerUaProbeLoop` HostedServices that poll the peer + write to `PeerReachabilityTracker` on each tick. Without these the publisher sees `PeerReachability.Unknown` for every peer → Isolated-Primary band (230) even when the peer is up. Safe default (retains authority) but not the full non-transparent-redundancy UX.
- OPC UA variable-node wiring layer: bind the `ServiceLevel` Byte node + `ServerUriArray` String[] node to the publisher's events via `BaseDataVariable.OnReadValue` / direct value push. Scoped follow-up on the Opc.Ua.Server stack integration.
- `PeerHttpProbeLoop` + `PeerUaProbeLoop` HostedServices populating `PeerReachabilityTracker` on each tick. Without these the publisher sees `PeerReachability.Unknown` → Isolated-Primary band (230). Safe default but not the full non-transparent-redundancy UX.
- OPC UA variable-node wiring: bind `ServiceLevel` Byte + `ServerUriArray` String[] to the publisher's events via `BaseDataVariable.OnReadValue` / direct value push.
- `sp_PublishGeneration` wraps its apply in `await using var lease = coordinator.BeginApplyLease(...)` so the `PrimaryMidApply` band (200) fires during actual publishes (task #148 part 2).
- Client interop matrix validation — Ignition / Kepware / Aveva OI Gateway (Stream F, task #150). Manual + doc-only work; doesn't block code ship.
- Client interop matrix — Ignition / Kepware / Aveva OI Gateway (Stream F, task #150). Manual + doc-only.
### Remaining drivers (task #120)
### ~~Phase 5 driver complement~~ (task #120 — **CLOSED** 2026-04-24)
AB CIP, AB Legacy, TwinCAT ADS, FOCAS drivers are planned but unshipped. Decision pending on whether these are release-blocking for v2 GA or can slip to a v2.1 follow-up.
**Closed**. All four deferred drivers shipped:
- **AB CIP** (PRs #202222) — `Driver.AbCip`, `Driver.AbCip.IntegrationTests` (7 tests), AB CIP Cli. Live-boot verified against a ControlLogix rig.
- **AB Legacy** (PRs #202, #223) — `Driver.AbLegacy`, `Driver.AbLegacy.IntegrationTests` (2 tests), AB Legacy Cli. PCCC cip-path workaround for SLC/MicroLogix.
- **TwinCAT ADS** (PRs #205, this branch `task-galaxy-e2e`) — `Driver.TwinCAT`, `Driver.TwinCAT.IntegrationTests` (2 tests), TwinCAT Cli. TCBSD/ESXi fixture for e2e since local Hyper-V / TwinCAT RTIME are mutually exclusive on the dev box.
- **FOCAS** (PRs #173, #199 + this session's migration) — `Driver.FOCAS` with an **in-process managed `FocasWireClient`** that speaks FOCAS/2 over TCP directly. Tier-C isolation retired — `Driver.FOCAS.Host` + `Driver.FOCAS.Shared` + `FwlibNative` P/Invoke + shim DLL + NSSM service all deleted. `Driver.FOCAS.IntegrationTests` covers 9 scenarios (fixed tree identity/axes/program/timers/spindle + user-authored PARAM/MACRO/PMC reads, Browse, Subscribe, IAlarmSource raise/clear, Probe transitions).
Decision recorded: FOCAS is **read-only** against the CNC by design — writes return `BadNotWritable`. See `docs/drivers/FOCAS.md` + `docs/drivers/FOCAS-Test-Fixture.md` for the deployment + coverage map.
## Nice-to-haves (not release-blocking)
- **Admin UI** — Phase 6.1 Stream E.2/E.3 (`/hosts` column refresh), Phase 6.2 Stream D (`RoleGrantsTab` + `AclsTab` Probe), Phase 6.3 Stream E (`RedundancyTab`), Phase 6.4 Streams A/B UI pieces, Stream C DiffViewer, Stream D `IdentificationFields.razor`. Tasks #134, #144, #149, #153, #155, #156, #157.
- **Background services** — Phase 6.1 Stream B.4 `ScheduledRecycleScheduler` HostedService (task #137), Phase 6.1 Stream A analyzer (task #135 — Roslyn analyzer asserting every capability surface routes through `CapabilityInvoker`).
- **Multi-host dispatch** — Phase 6.1 Stream A follow-up (task #135). Currently every driver gets a single pipeline keyed on `driver.DriverInstanceId`; multi-host drivers (Modbus with N PLCs) need per-PLC host resolution so failing PLCs trip per-PLC breakers without poisoning siblings. Decision #144 requires this but we haven't wired it yet.
- **Multi-host dispatch** — Phase 6.1 Stream A follow-up (task #135). Every driver currently gets a single pipeline keyed on `driver.DriverInstanceId`; multi-host drivers (Modbus with N PLCs) need per-PLC host resolution so failing PLCs trip per-PLC breakers without poisoning siblings. Decision #144 requires this but not wired.
- **Phase 7** — scripting + alarming + historian sink (plan drafted 2026-04-20 in `docs/v2/implementation/phase-7-*.md`). Out of scope for v2 GA.
## Live-hardware validations (task #54 + task family)
The code ships; these tasks remain open as lab/field verification:
- **#54** — FOCAS live-CNC wire-level smoke against a real FANUC control. The mock's wire responder is PDU-verified against `fwlibe64.dll` upstream but OtOpcUa's managed client has not been pointed at a production CNC.
- **AB CIP live-boot** — already passed on a ControlLogix rig (PR #222). Continue to run ahead of each release.
- **TwinCAT wire-live** — TCBSD/ESXi fixture covers the common path; production PLC verification remains lab-gated.
## Running the release-readiness check
@@ -82,7 +93,12 @@ AB CIP, AB Legacy, TwinCAT ADS, FOCAS drivers are planned but unshipped. Decisio
pwsh ./scripts/compliance/phase-6-all.ps1
```
This meta-runner invokes each `phase-6-N-compliance.ps1` script in sequence and reports an aggregate PASS/FAIL. It is the single-command verification that what we claim is shipped still compiles + tests pass + the plan-level invariants are still satisfied.
This meta-runner invokes each `phase-6-N-compliance.ps1` script in sequence and reports an aggregate PASS/FAIL:
- `phase-6-1-compliance.ps1` — Resilience & Observability
- `phase-6-2-compliance.ps1` — Authorization runtime
- `phase-6-3-compliance.ps1` — Redundancy runtime
- `phase-6-4-compliance.ps1` — Admin UI completion
Exit 0 = every phase passes its compliance checks + no test-count regression.
@@ -92,18 +108,23 @@ v2 GA requires all of the following:
- [ ] All four Phase 6.N compliance scripts exit 0.
- [ ] `dotnet test ZB.MOM.WW.OtOpcUa.slnx` passes with ≤ 1 known-flake failure.
- [ ] Release blockers listed above all closed (or consciously deferred to v2.1 with a written decision).
- [x] Release blockers listed above all closed.
- [x] Phase 5 driver complement shipped (Galaxy, Modbus, S7, OpcUaClient, AbCip, AbLegacy, TwinCAT, FOCAS).
- [ ] Production deployment checklist (separate doc) signed off by Fleet Admin.
- [ ] At least one end-to-end integration run against the live Galaxy on the dev box succeeds.
- [ ] FOCAS live-CNC wire-level smoke (#54) runs clean against a real FANUC control.
- [ ] OPC UA conformance test (CTT or UA Compliance Test Tool) passes against the live endpoint.
- [ ] Non-transparent redundancy cutover validated with at least one production client (Ignition 8.3 recommended — see decision #85).
## Change log
- **2026-04-19** — Release blocker #3 **closed** (PRs #9899). Phase 6.3 Streams A + C core shipped: `ClusterTopologyLoader` + `RedundancyCoordinator` + `RedundancyStatePublisher` + `PeerReachabilityTracker`. Code-path release blockers all closed; remaining Phase 6.3 surfaces (peer-probe HostedServices, OPC UA variable-node binding, sp_PublishGeneration lease wrap, client interop matrix) are hardening follow-ups.
- **2026-04-19** — Release blocker #2 **closed** (PR #96). `SealedBootstrap` consumes `ResilientConfigReader` + `GenerationSealedCache` + `StaleConfigFlag`; `/healthz` now surfaces the stale flag. Remaining follow-ups (periodic poller + richer snapshot payload) downgraded to hardening.
- **2026-04-19** — Release blocker #1 **closed** (PR #94). `AuthorizationGate` wired into `DriverNodeManager` Read / Write / HistoryRead dispatch. Remaining Stream C surfaces (Browse / Subscribe / Alarm / Call + finer-grained scope resolution) downgraded to hardening follow-ups — no longer release-blocking.
- **2026-04-19** — Phase 6.4 data layer merged (PRs #9192). Phase 6 core complete. Capstone doc created.
- **2026-04-24** — Phase 5 driver complement closed (task #120 CLOSED). AB CIP, AB Legacy, TwinCAT, FOCAS all shipped. FOCAS migration: retired the Tier-C split (`Driver.FOCAS.Host` + `Driver.FOCAS.Shared` + `FwlibNative` + shim DLL deleted) in favour of a pure-managed in-process `FocasWireClient` inlined into `Driver.FOCAS`; driver is now read-only against the CNC by design. Integration test matrix grew to cover Browse / Subscribe / IAlarmSource / Probe end-to-end.
- **2026-04-23** — Phase 6.4 audit close-out. IdentificationFolderBuilder + OPC 40010 Identification folder verified against the shipped code.
- **2026-04-20** — Phase 7 plan drafted (`phase-7-scripting-and-alarming.md`, `phase-7-e2e-smoke.md`). Out of scope for v2 GA.
- **2026-04-19** — Release blocker #3 closed (PRs #9899). Phase 6.3 Streams A + C core shipped: `ClusterTopologyLoader` + `RedundancyCoordinator` + `RedundancyStatePublisher` + `PeerReachabilityTracker`. Code-path release blockers all closed; remaining Phase 6.3 surfaces (peer-probe HostedServices, OPC UA variable-node binding, `sp_PublishGeneration` lease wrap, client interop matrix) are hardening follow-ups.
- **2026-04-19** — Release blocker #2 closed (PR #96). `SealedBootstrap` consumes `ResilientConfigReader` + `GenerationSealedCache` + `StaleConfigFlag`; `/healthz` surfaces the stale flag. Remaining follow-ups (periodic poller + richer snapshot payload) downgraded to hardening.
- **2026-04-19** — Release blocker #1 closed (PR #94). `AuthorizationGate` wired into `DriverNodeManager` Read / Write / HistoryRead dispatch. Remaining Stream C surfaces (Browse / Subscribe / Alarm / Call + finer-grained scope resolution) downgraded to hardening follow-ups — no longer release-blocking.
- **2026-04-19** — Phase 6.4 data layer merged (PRs #9192). Phase 6 core complete.
- **2026-04-19** — Phase 6.3 core merged (PRs #8990). `ServiceLevelCalculator` + `RecoveryStateManager` + `ApplyLeaseRegistry` land as pure logic; coordinator / UA-node wiring / Admin UI / interop deferred.
- **2026-04-19** — Phase 6.2 core merged (PRs #8488). `AuthorizationGate` + `TriePermissionEvaluator` + `LdapGroupRoleMapping` land; dispatch wiring + Admin UI deferred.
- **2026-04-19** — Phase 6.1 shipped (PRs #7883). Polly resilience + Tier A/B/C stability + health endpoints + LiteDB generation-sealed cache + Admin `/hosts` data layer all live.