FOCAS Tier-C PR E — ops glue: ProcessHostLauncher + post-mortem MMF + NSSM install scripts + doc close-out. Final of the 5 PRs for #220. With this landing, the Tier-C architecture is fully shipped; the only remaining FOCAS work is the hardware-dependent FwlibHostedBackend (real Fwlib32.dll P/Invoke, gated on #222 lab rig).

Production IHostProcessLauncher (ProcessHostLauncher.cs): Process.Start spawns OtOpcUa.Driver.FOCAS.Host.exe with OTOPCUA_FOCAS_PIPE / OTOPCUA_ALLOWED_SID / OTOPCUA_FOCAS_SECRET / OTOPCUA_FOCAS_BACKEND in the environment (supervisor-owned, never disk), polls FocasIpcClient.ConnectAsync at 250ms cadence until the pipe is up or the Host exits or the ConnectTimeout deadline passes, then wraps the connected client in an IpcFocasClient. TerminateAsync kills the entire process tree + disposes the IPC stream. ProcessHostLauncherOptions carries HostExePath + PipeName + AllowedSid plus optional SharedSecret (auto-generated from a GUID when omitted so install scripts don't have to), Arguments, Backend (fwlib32/fake/unconfigured default-unconfigured), ConnectTimeout (15s), and Series for CNC pre-flight.

Post-mortem MMF (Host/Stability/PostMortemMmf.cs + Proxy/Supervisor/PostMortemReader.cs): ring-buffer of the last ~1000 IPC operations written by the Host into a memory-mapped file. On a Host crash the supervisor reads the MMF — which survives process death — to see what was in flight. File format: 16-byte header [magic 'OFPC' (0x4F465043) | version | capacity | writeIndex] + N × 256-byte entries [8-byte UTC unix ms | 8-byte opKind | 240-byte UTF-8 message + null terminator]. Magic distinguishes FOCAS MMFs from the Galaxy MMFs that ship the same format shape. Writer is single-producer (Host) with a lock_writeGate; reader is multi-consumer (Proxy + any diagnostic tool) using a separate MemoryMappedFile handle.

NSSM install wrappers (scripts/install/Install-FocasHost.ps1 + Uninstall-FocasHost.ps1): idempotent service registration for OtOpcUaFocasHost. Resolves SID from the ServiceAccount, generates a fresh shared secret per install if not supplied, stages OTOPCUA_FOCAS_PIPE/SID/SECRET/BACKEND in AppEnvironmentExtra so they never hit disk, rotates 10MB stdout/stderr logs under %ProgramData%\OtOpcUa, DependOnService=OtOpcUa so startup order is deterministic. Backend selector defaults to unconfigured so a fresh install doesn't accidentally load a half-configured Fwlib32.dll on first start.

Tests (7 new, 2 files): PostMortemMmfTests.cs in FOCAS.Host.Tests — round-trip write+read preserves order + content, ring-buffer wraps at capacity (writes 10 entries to a 3-slot buffer, asserts only op-7/8/9 survive in FIFO order), message truncation at the 240-byte cap is null-terminated + non-overflowing, reopening an existing file preserves entries. PostMortemReaderCompatibilityTests.cs in FOCAS.Tests — hand-writes a file in the exact host format (magic/entry layout) + asserts the Proxy reader decodes with correct ring-walk ordering when writeIndex != 0, empty-return on missing file + magic mismatch. Keeps the two codebases in format-lockstep without the net10 test project referencing the net48 Host assembly.

Docs updated: docs/v2/implementation/focas-isolation-plan.md promoted from DRAFT to PRs A-E shipped status with per-PR citations + post-ship test counts (189 + 24 + 13 = 226 FOCAS-family tests green). docs/drivers/FOCAS-Test-Fixture.md §5 updated from "architecture scoped but not implemented" to listing the shipped components with the FwlibHostedBackend gap explicitly labeled as hardware-gated. Install-FocasHost.ps1 documents the OTOPCUA_FOCAS_BACKEND selector + points at docs/v2/focas-deployment.md for Fwlib32.dll licensing.

What ISN'T in this PR: (1) the real FwlibHostedBackend implementing IFocasBackend with the P/Invoke — requires either a CNC on the bench or a licensed FANUC developer kit to validate, tracked under #220 as a single follow-up task; (2) Admin /hosts surface integration for FOCAS runtime status — Galaxy Tier-C already has the shape, FOCAS can slot in when someone wires ObservedCrashes/StickyAlertActive/BackoffAttempt to the FleetStatusHub; (3) a full integration test that actually spawns a real FOCAS Host process — ProcessHostLauncher is tested via its contract + the MMF is tested via round-trip, but no test spins up the real exe (the Galaxy Tier-C tests do this, but the FOCAS equivalent adds no new coverage over what's already in place).

Total FOCAS-family tests green after this PR: 189 driver + 24 Shared + 13 Host = 226.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-04-20 14:24:13 -04:00
parent 446a5c022c
commit 8d88ffa14d
9 changed files with 673 additions and 37 deletions

View File

@@ -69,14 +69,32 @@ covers the common address shapes; per-model quirks are not stressed.
- Parameter range enforcement (CNC rejects out-of-range writes)
- MTB (machine tool builder) custom screens that expose non-standard data
### 5. Tier-C process isolation behavior
### 5. Tier-C process isolation — architecture shipped, Fwlib32 integration hardware-gated
Per driver-stability.md, FOCAS should run process-isolated because
`Fwlib32.dll` has documented crash modes. The test suite runs in-process +
only exercises the happy path + mapped error codes — a native access
violation from the DLL would take the test host down. The process-isolation
path (similar to Galaxy's out-of-process Host) has been scoped but not
implemented.
The Tier-C architecture is now in place as of PRs #169#173 (FOCAS
PR AE, task #220):
- `Driver.FOCAS.Shared` carries MessagePack IPC contracts
- `Driver.FOCAS.Host` (.NET 4.8 x86 Windows service via NSSM) accepts
a connection on a strictly-ACL'd named pipe + dispatches frames to
an `IFocasBackend`
- `Driver.FOCAS.Ipc.IpcFocasClient` implements the `IFocasClient` DI
seam by forwarding over IPC — swap the DI registration and the
driver runs Tier-C with zero other changes
- `Driver.FOCAS.Supervisor.FocasHostSupervisor` owns the spawn +
heartbeat + respawn + 3-in-5min crash-loop breaker + sticky alert
- `Driver.FOCAS.Host.Stability.PostMortemMmf`
`Driver.FOCAS.Supervisor.PostMortemReader` — ring-buffer of the
last ~1000 IPC operations survives a Host crash
The one remaining gap is the production `FwlibHostedBackend`: an
`IFocasBackend` implementation that wraps the licensed
`Fwlib32.dll` P/Invoke. That's hardware-gated on task #222 — we
need a CNC on the bench (or the licensed FANUC developer kit DLL
with a test harness) to validate it. Until then, the Host ships
`FakeFocasBackend` + `UnconfiguredFocasBackend`. Setting
`OTOPCUA_FOCAS_BACKEND=fake` lets operators smoke-test the whole
Tier-C pipeline end-to-end without any CNC.
## When to trust FOCAS tests, when to reach for a rig