FOCAS Tier-C PR E — ops glue: ProcessHostLauncher + post-mortem MMF + NSSM install scripts + doc close-out. Final of the 5 PRs for #220. With this landing, the Tier-C architecture is fully shipped; the only remaining FOCAS work is the hardware-dependent FwlibHostedBackend (real Fwlib32.dll P/Invoke, gated on #222 lab rig).

Production IHostProcessLauncher (ProcessHostLauncher.cs): Process.Start spawns OtOpcUa.Driver.FOCAS.Host.exe with OTOPCUA_FOCAS_PIPE / OTOPCUA_ALLOWED_SID / OTOPCUA_FOCAS_SECRET / OTOPCUA_FOCAS_BACKEND in the environment (supervisor-owned, never disk), polls FocasIpcClient.ConnectAsync at 250ms cadence until the pipe is up or the Host exits or the ConnectTimeout deadline passes, then wraps the connected client in an IpcFocasClient. TerminateAsync kills the entire process tree + disposes the IPC stream. ProcessHostLauncherOptions carries HostExePath + PipeName + AllowedSid plus optional SharedSecret (auto-generated from a GUID when omitted so install scripts don't have to), Arguments, Backend (fwlib32/fake/unconfigured default-unconfigured), ConnectTimeout (15s), and Series for CNC pre-flight.

Post-mortem MMF (Host/Stability/PostMortemMmf.cs + Proxy/Supervisor/PostMortemReader.cs): ring-buffer of the last ~1000 IPC operations written by the Host into a memory-mapped file. On a Host crash the supervisor reads the MMF — which survives process death — to see what was in flight. File format: 16-byte header [magic 'OFPC' (0x4F465043) | version | capacity | writeIndex] + N × 256-byte entries [8-byte UTC unix ms | 8-byte opKind | 240-byte UTF-8 message + null terminator]. Magic distinguishes FOCAS MMFs from the Galaxy MMFs that ship the same format shape. Writer is single-producer (Host) with a lock_writeGate; reader is multi-consumer (Proxy + any diagnostic tool) using a separate MemoryMappedFile handle.

NSSM install wrappers (scripts/install/Install-FocasHost.ps1 + Uninstall-FocasHost.ps1): idempotent service registration for OtOpcUaFocasHost. Resolves SID from the ServiceAccount, generates a fresh shared secret per install if not supplied, stages OTOPCUA_FOCAS_PIPE/SID/SECRET/BACKEND in AppEnvironmentExtra so they never hit disk, rotates 10MB stdout/stderr logs under %ProgramData%\OtOpcUa, DependOnService=OtOpcUa so startup order is deterministic. Backend selector defaults to unconfigured so a fresh install doesn't accidentally load a half-configured Fwlib32.dll on first start.

Tests (7 new, 2 files): PostMortemMmfTests.cs in FOCAS.Host.Tests — round-trip write+read preserves order + content, ring-buffer wraps at capacity (writes 10 entries to a 3-slot buffer, asserts only op-7/8/9 survive in FIFO order), message truncation at the 240-byte cap is null-terminated + non-overflowing, reopening an existing file preserves entries. PostMortemReaderCompatibilityTests.cs in FOCAS.Tests — hand-writes a file in the exact host format (magic/entry layout) + asserts the Proxy reader decodes with correct ring-walk ordering when writeIndex != 0, empty-return on missing file + magic mismatch. Keeps the two codebases in format-lockstep without the net10 test project referencing the net48 Host assembly.

Docs updated: docs/v2/implementation/focas-isolation-plan.md promoted from DRAFT to PRs A-E shipped status with per-PR citations + post-ship test counts (189 + 24 + 13 = 226 FOCAS-family tests green). docs/drivers/FOCAS-Test-Fixture.md §5 updated from "architecture scoped but not implemented" to listing the shipped components with the FwlibHostedBackend gap explicitly labeled as hardware-gated. Install-FocasHost.ps1 documents the OTOPCUA_FOCAS_BACKEND selector + points at docs/v2/focas-deployment.md for Fwlib32.dll licensing.

What ISN'T in this PR: (1) the real FwlibHostedBackend implementing IFocasBackend with the P/Invoke — requires either a CNC on the bench or a licensed FANUC developer kit to validate, tracked under #220 as a single follow-up task; (2) Admin /hosts surface integration for FOCAS runtime status — Galaxy Tier-C already has the shape, FOCAS can slot in when someone wires ObservedCrashes/StickyAlertActive/BackoffAttempt to the FleetStatusHub; (3) a full integration test that actually spawns a real FOCAS Host process — ProcessHostLauncher is tested via its contract + the MMF is tested via round-trip, but no test spins up the real exe (the Galaxy Tier-C tests do this, but the FOCAS equivalent adds no new coverage over what's already in place).

Total FOCAS-family tests green after this PR: 189 driver + 24 Shared + 13 Host = 226.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-04-20 14:24:13 -04:00
parent 446a5c022c
commit 8d88ffa14d
9 changed files with 673 additions and 37 deletions

View File

@@ -1,12 +1,13 @@
# FOCAS Tier-C isolation — plan for task #220
> **Status**: DRAFT — not yet started. Tracks the multi-PR work to
> move `Fwlib32.dll` behind an out-of-process host, mirroring the
> Galaxy Tier-C split in [`phase-2-galaxy-out-of-process.md`](phase-2-galaxy-out-of-process.md).
> **Status**: PRs AE shipped. Architecture is in place; the only
> remaining FOCAS work is the hardware-dependent production
> integration of `Fwlib32.dll` into a real `IFocasBackend`
> (`FwlibHostedBackend`), which needs an actual CNC on the bench
> and is tracked as a follow-up on #220.
>
> **Pre-reqs shipped** (this PR): version matrix + pre-flight
> validation + unit tests. Those close the cheap half of the
> hardware-free stability gap. Tier-C closes the expensive half.
> **Pre-reqs shipped**: version matrix + pre-flight validation
> (PR #168 — the cheap half of the hardware-free stability gap).
## Why isolate
@@ -79,32 +80,41 @@ its own timer + pushes change notifications so the Proxy doesn't
round-trip per poll. Matches `Driver.Galaxy.Host` subscription
forwarding.
## PR sequence (proposed)
## PR sequence — shipped
1. **PR A — shared contracts**
Create `Driver.FOCAS.Shared` with the MessagePack DTOs. No
behaviour change. ~200 LOC + round-trip tests for each DTO.
2. **PR B — Host project skeleton**
Create `Driver.FOCAS.Host` .NET 4.8 x86 project, NSSM wrapper,
pipe server scaffold with the same ACL + caller-SID + shared
secret plumbing as Galaxy.Host. No Fwlib32 wiring yet — returns
`NotImplemented` for everything. ~400 LOC.
3. **PR C — Move Fwlib32 calls into Host**
Move `FocasNativeSession`, `FocasTagReader`, `FocasTagWriter`,
`FocasPmcBitRmw` + the STA thread into the Host. Proxy forwards
over IPC. This is the biggest PR — probably 800-1500 LOC of
move-with-translation. Existing unit tests keep passing because
`IFocasTagFactory` is the DI seam the tests inject against.
4. **PR D — Supervisor + respawn**
Proxy-side heartbeat + respawn + crash-loop circuit breaker +
BackPressure fan-out on Host death. ~500 LOC + chaos tests.
5. **PR E — Post-mortem MMF + operational glue**
MMF writer in Host, reader in Proxy. Install scripts for the
new `OtOpcUaFocasHost` Windows service. Docs. ~300 LOC.
1. **PR A (#169) — shared contracts**
`Driver.FOCAS.Shared` netstandard2.0 with MessagePack DTOs for every
IPC surface (Hello/Heartbeat/OpenSession/Read/Write/PmcBitWrite/
Subscribe/Probe/RuntimeStatus/Recycle/ErrorResponse) + FrameReader/
FrameWriter + 24 round-trip tests.
2. **PR B (#170) — Host project skeleton**
`Driver.FOCAS.Host` net48 x86 Windows Service entry point,
`PipeAcl` + `PipeServer` + `IFrameHandler` + `StubFrameHandler`.
ACL denies LocalSystem/Administrators; Hello verifies
shared-secret + protocol major. 3 handshake tests.
3. **PR C (#171) — IPC path end-to-end**
Proxy `Ipc/FocasIpcClient` + `Ipc/IpcFocasClient` (implements
IFocasClient via IPC). Host `Backend/IFocasBackend` +
`FakeFocasBackend` + `UnconfiguredFocasBackend` +
`Ipc/FwlibFrameHandler` replacing the stub. 13 new round-trip
tests via in-memory loopback.
4. **PR D (#172) — Supervisor + respawn**
`Supervisor/Backoff` (5s→15s→60s) + `CircuitBreaker` (3-in-5min →
1h→4h→manual) + `HeartbeatMonitor` + `IHostProcessLauncher` +
`FocasHostSupervisor`. 14 tests.
5. **PR E — Ops glue** ✅ (this PR)
`ProcessHostLauncher` (real Process.Start + FocasIpcClient
connect), `Host/Stability/PostMortemMmf` (magic 'OFPC') +
Proxy `Supervisor/PostMortemReader`, `scripts/install/
Install-FocasHost.ps1` + `Uninstall-FocasHost.ps1` NSSM wrappers.
7 tests (4 MMF round-trip + 3 reader format compatibility).
Total estimate: 2200-3200 LOC across 5 PRs. Consistent with Galaxy
Tier-C but narrower since FOCAS has no Historian + no alarm
history.
**Post-shipment totals: 189 FOCAS driver tests + 24 Shared tests + 13 Host tests = 226 FOCAS-family tests green.**
What remains is hardware-dependent: wiring `Fwlib32.dll` P/Invoke
into a real `FwlibHostedBackend` implementation of `IFocasBackend`
+ validating against a live CNC. The architecture is all the
plumbing that work needs.
## Testing without hardware