FOCAS Tier-C PR E — ops glue: ProcessHostLauncher + post-mortem MMF + NSSM install scripts + doc close-out. Final of the 5 PRs for #220. With this landing, the Tier-C architecture is fully shipped; the only remaining FOCAS work is the hardware-dependent FwlibHostedBackend (real Fwlib32.dll P/Invoke, gated on #222 lab rig).
Production IHostProcessLauncher (ProcessHostLauncher.cs): Process.Start spawns OtOpcUa.Driver.FOCAS.Host.exe with OTOPCUA_FOCAS_PIPE / OTOPCUA_ALLOWED_SID / OTOPCUA_FOCAS_SECRET / OTOPCUA_FOCAS_BACKEND in the environment (supervisor-owned, never disk), polls FocasIpcClient.ConnectAsync at 250ms cadence until the pipe is up or the Host exits or the ConnectTimeout deadline passes, then wraps the connected client in an IpcFocasClient. TerminateAsync kills the entire process tree + disposes the IPC stream. ProcessHostLauncherOptions carries HostExePath + PipeName + AllowedSid plus optional SharedSecret (auto-generated from a GUID when omitted so install scripts don't have to), Arguments, Backend (fwlib32/fake/unconfigured default-unconfigured), ConnectTimeout (15s), and Series for CNC pre-flight. Post-mortem MMF (Host/Stability/PostMortemMmf.cs + Proxy/Supervisor/PostMortemReader.cs): ring-buffer of the last ~1000 IPC operations written by the Host into a memory-mapped file. On a Host crash the supervisor reads the MMF — which survives process death — to see what was in flight. File format: 16-byte header [magic 'OFPC' (0x4F465043) | version | capacity | writeIndex] + N × 256-byte entries [8-byte UTC unix ms | 8-byte opKind | 240-byte UTF-8 message + null terminator]. Magic distinguishes FOCAS MMFs from the Galaxy MMFs that ship the same format shape. Writer is single-producer (Host) with a lock_writeGate; reader is multi-consumer (Proxy + any diagnostic tool) using a separate MemoryMappedFile handle. NSSM install wrappers (scripts/install/Install-FocasHost.ps1 + Uninstall-FocasHost.ps1): idempotent service registration for OtOpcUaFocasHost. Resolves SID from the ServiceAccount, generates a fresh shared secret per install if not supplied, stages OTOPCUA_FOCAS_PIPE/SID/SECRET/BACKEND in AppEnvironmentExtra so they never hit disk, rotates 10MB stdout/stderr logs under %ProgramData%\OtOpcUa, DependOnService=OtOpcUa so startup order is deterministic. Backend selector defaults to unconfigured so a fresh install doesn't accidentally load a half-configured Fwlib32.dll on first start. Tests (7 new, 2 files): PostMortemMmfTests.cs in FOCAS.Host.Tests — round-trip write+read preserves order + content, ring-buffer wraps at capacity (writes 10 entries to a 3-slot buffer, asserts only op-7/8/9 survive in FIFO order), message truncation at the 240-byte cap is null-terminated + non-overflowing, reopening an existing file preserves entries. PostMortemReaderCompatibilityTests.cs in FOCAS.Tests — hand-writes a file in the exact host format (magic/entry layout) + asserts the Proxy reader decodes with correct ring-walk ordering when writeIndex != 0, empty-return on missing file + magic mismatch. Keeps the two codebases in format-lockstep without the net10 test project referencing the net48 Host assembly. Docs updated: docs/v2/implementation/focas-isolation-plan.md promoted from DRAFT to PRs A-E shipped status with per-PR citations + post-ship test counts (189 + 24 + 13 = 226 FOCAS-family tests green). docs/drivers/FOCAS-Test-Fixture.md §5 updated from "architecture scoped but not implemented" to listing the shipped components with the FwlibHostedBackend gap explicitly labeled as hardware-gated. Install-FocasHost.ps1 documents the OTOPCUA_FOCAS_BACKEND selector + points at docs/v2/focas-deployment.md for Fwlib32.dll licensing. What ISN'T in this PR: (1) the real FwlibHostedBackend implementing IFocasBackend with the P/Invoke — requires either a CNC on the bench or a licensed FANUC developer kit to validate, tracked under #220 as a single follow-up task; (2) Admin /hosts surface integration for FOCAS runtime status — Galaxy Tier-C already has the shape, FOCAS can slot in when someone wires ObservedCrashes/StickyAlertActive/BackoffAttempt to the FleetStatusHub; (3) a full integration test that actually spawns a real FOCAS Host process — ProcessHostLauncher is tested via its contract + the MMF is tested via round-trip, but no test spins up the real exe (the Galaxy Tier-C tests do this, but the FOCAS equivalent adds no new coverage over what's already in place). Total FOCAS-family tests green after this PR: 189 driver + 24 Shared + 13 Host = 226. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -69,14 +69,32 @@ covers the common address shapes; per-model quirks are not stressed.
|
||||
- Parameter range enforcement (CNC rejects out-of-range writes)
|
||||
- MTB (machine tool builder) custom screens that expose non-standard data
|
||||
|
||||
### 5. Tier-C process isolation behavior
|
||||
### 5. Tier-C process isolation — architecture shipped, Fwlib32 integration hardware-gated
|
||||
|
||||
Per driver-stability.md, FOCAS should run process-isolated because
|
||||
`Fwlib32.dll` has documented crash modes. The test suite runs in-process +
|
||||
only exercises the happy path + mapped error codes — a native access
|
||||
violation from the DLL would take the test host down. The process-isolation
|
||||
path (similar to Galaxy's out-of-process Host) has been scoped but not
|
||||
implemented.
|
||||
The Tier-C architecture is now in place as of PRs #169–#173 (FOCAS
|
||||
PR A–E, task #220):
|
||||
|
||||
- `Driver.FOCAS.Shared` carries MessagePack IPC contracts
|
||||
- `Driver.FOCAS.Host` (.NET 4.8 x86 Windows service via NSSM) accepts
|
||||
a connection on a strictly-ACL'd named pipe + dispatches frames to
|
||||
an `IFocasBackend`
|
||||
- `Driver.FOCAS.Ipc.IpcFocasClient` implements the `IFocasClient` DI
|
||||
seam by forwarding over IPC — swap the DI registration and the
|
||||
driver runs Tier-C with zero other changes
|
||||
- `Driver.FOCAS.Supervisor.FocasHostSupervisor` owns the spawn +
|
||||
heartbeat + respawn + 3-in-5min crash-loop breaker + sticky alert
|
||||
- `Driver.FOCAS.Host.Stability.PostMortemMmf` ↔
|
||||
`Driver.FOCAS.Supervisor.PostMortemReader` — ring-buffer of the
|
||||
last ~1000 IPC operations survives a Host crash
|
||||
|
||||
The one remaining gap is the production `FwlibHostedBackend`: an
|
||||
`IFocasBackend` implementation that wraps the licensed
|
||||
`Fwlib32.dll` P/Invoke. That's hardware-gated on task #222 — we
|
||||
need a CNC on the bench (or the licensed FANUC developer kit DLL
|
||||
with a test harness) to validate it. Until then, the Host ships
|
||||
`FakeFocasBackend` + `UnconfiguredFocasBackend`. Setting
|
||||
`OTOPCUA_FOCAS_BACKEND=fake` lets operators smoke-test the whole
|
||||
Tier-C pipeline end-to-end without any CNC.
|
||||
|
||||
## When to trust FOCAS tests, when to reach for a rig
|
||||
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
# FOCAS Tier-C isolation — plan for task #220
|
||||
|
||||
> **Status**: DRAFT — not yet started. Tracks the multi-PR work to
|
||||
> move `Fwlib32.dll` behind an out-of-process host, mirroring the
|
||||
> Galaxy Tier-C split in [`phase-2-galaxy-out-of-process.md`](phase-2-galaxy-out-of-process.md).
|
||||
> **Status**: PRs A–E shipped. Architecture is in place; the only
|
||||
> remaining FOCAS work is the hardware-dependent production
|
||||
> integration of `Fwlib32.dll` into a real `IFocasBackend`
|
||||
> (`FwlibHostedBackend`), which needs an actual CNC on the bench
|
||||
> and is tracked as a follow-up on #220.
|
||||
>
|
||||
> **Pre-reqs shipped** (this PR): version matrix + pre-flight
|
||||
> validation + unit tests. Those close the cheap half of the
|
||||
> hardware-free stability gap. Tier-C closes the expensive half.
|
||||
> **Pre-reqs shipped**: version matrix + pre-flight validation
|
||||
> (PR #168 — the cheap half of the hardware-free stability gap).
|
||||
|
||||
## Why isolate
|
||||
|
||||
@@ -79,32 +80,41 @@ its own timer + pushes change notifications so the Proxy doesn't
|
||||
round-trip per poll. Matches `Driver.Galaxy.Host` subscription
|
||||
forwarding.
|
||||
|
||||
## PR sequence (proposed)
|
||||
## PR sequence — shipped
|
||||
|
||||
1. **PR A — shared contracts**
|
||||
Create `Driver.FOCAS.Shared` with the MessagePack DTOs. No
|
||||
behaviour change. ~200 LOC + round-trip tests for each DTO.
|
||||
2. **PR B — Host project skeleton**
|
||||
Create `Driver.FOCAS.Host` .NET 4.8 x86 project, NSSM wrapper,
|
||||
pipe server scaffold with the same ACL + caller-SID + shared
|
||||
secret plumbing as Galaxy.Host. No Fwlib32 wiring yet — returns
|
||||
`NotImplemented` for everything. ~400 LOC.
|
||||
3. **PR C — Move Fwlib32 calls into Host**
|
||||
Move `FocasNativeSession`, `FocasTagReader`, `FocasTagWriter`,
|
||||
`FocasPmcBitRmw` + the STA thread into the Host. Proxy forwards
|
||||
over IPC. This is the biggest PR — probably 800-1500 LOC of
|
||||
move-with-translation. Existing unit tests keep passing because
|
||||
`IFocasTagFactory` is the DI seam the tests inject against.
|
||||
4. **PR D — Supervisor + respawn**
|
||||
Proxy-side heartbeat + respawn + crash-loop circuit breaker +
|
||||
BackPressure fan-out on Host death. ~500 LOC + chaos tests.
|
||||
5. **PR E — Post-mortem MMF + operational glue**
|
||||
MMF writer in Host, reader in Proxy. Install scripts for the
|
||||
new `OtOpcUaFocasHost` Windows service. Docs. ~300 LOC.
|
||||
1. **PR A (#169) — shared contracts** ✅
|
||||
`Driver.FOCAS.Shared` netstandard2.0 with MessagePack DTOs for every
|
||||
IPC surface (Hello/Heartbeat/OpenSession/Read/Write/PmcBitWrite/
|
||||
Subscribe/Probe/RuntimeStatus/Recycle/ErrorResponse) + FrameReader/
|
||||
FrameWriter + 24 round-trip tests.
|
||||
2. **PR B (#170) — Host project skeleton** ✅
|
||||
`Driver.FOCAS.Host` net48 x86 Windows Service entry point,
|
||||
`PipeAcl` + `PipeServer` + `IFrameHandler` + `StubFrameHandler`.
|
||||
ACL denies LocalSystem/Administrators; Hello verifies
|
||||
shared-secret + protocol major. 3 handshake tests.
|
||||
3. **PR C (#171) — IPC path end-to-end** ✅
|
||||
Proxy `Ipc/FocasIpcClient` + `Ipc/IpcFocasClient` (implements
|
||||
IFocasClient via IPC). Host `Backend/IFocasBackend` +
|
||||
`FakeFocasBackend` + `UnconfiguredFocasBackend` +
|
||||
`Ipc/FwlibFrameHandler` replacing the stub. 13 new round-trip
|
||||
tests via in-memory loopback.
|
||||
4. **PR D (#172) — Supervisor + respawn** ✅
|
||||
`Supervisor/Backoff` (5s→15s→60s) + `CircuitBreaker` (3-in-5min →
|
||||
1h→4h→manual) + `HeartbeatMonitor` + `IHostProcessLauncher` +
|
||||
`FocasHostSupervisor`. 14 tests.
|
||||
5. **PR E — Ops glue** ✅ (this PR)
|
||||
`ProcessHostLauncher` (real Process.Start + FocasIpcClient
|
||||
connect), `Host/Stability/PostMortemMmf` (magic 'OFPC') +
|
||||
Proxy `Supervisor/PostMortemReader`, `scripts/install/
|
||||
Install-FocasHost.ps1` + `Uninstall-FocasHost.ps1` NSSM wrappers.
|
||||
7 tests (4 MMF round-trip + 3 reader format compatibility).
|
||||
|
||||
Total estimate: 2200-3200 LOC across 5 PRs. Consistent with Galaxy
|
||||
Tier-C but narrower since FOCAS has no Historian + no alarm
|
||||
history.
|
||||
**Post-shipment totals: 189 FOCAS driver tests + 24 Shared tests + 13 Host tests = 226 FOCAS-family tests green.**
|
||||
|
||||
What remains is hardware-dependent: wiring `Fwlib32.dll` P/Invoke
|
||||
into a real `FwlibHostedBackend` implementation of `IFocasBackend`
|
||||
+ validating against a live CNC. The architecture is all the
|
||||
plumbing that work needs.
|
||||
|
||||
## Testing without hardware
|
||||
|
||||
|
||||
108
scripts/install/Install-FocasHost.ps1
Normal file
108
scripts/install/Install-FocasHost.ps1
Normal file
@@ -0,0 +1,108 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Registers the OtOpcUaFocasHost Windows service. Optional companion to
|
||||
Install-Services.ps1 — only run this on nodes where FOCAS driver instances will run
|
||||
with Tier-C process isolation enabled.
|
||||
|
||||
.DESCRIPTION
|
||||
FOCAS PR #220 / Tier-C isolation plan. Wraps OtOpcUa.Driver.FOCAS.Host.exe (net48 x86)
|
||||
as a Windows service using NSSM, running under the same service account as the main
|
||||
OtOpcUa service so the named-pipe ACL works. Passes the per-process shared secret via
|
||||
environment variable at service-start time so it never hits disk.
|
||||
|
||||
.PARAMETER InstallRoot
|
||||
Where the FOCAS Host binaries live (typically
|
||||
C:\Program Files\OtOpcUa\Driver.FOCAS.Host).
|
||||
|
||||
.PARAMETER ServiceAccount
|
||||
Service account SID or DOMAIN\name. Must match the main OtOpcUa server account so the
|
||||
PipeAcl match succeeds.
|
||||
|
||||
.PARAMETER FocasSharedSecret
|
||||
Per-process secret passed via env var. Generated freshly per install if not supplied.
|
||||
|
||||
.PARAMETER FocasBackend
|
||||
Backend selector for the Host process. One of:
|
||||
fwlib32 (default — real Fanuc Fwlib32.dll integration; requires licensed DLL on PATH)
|
||||
fake (in-memory; smoke-test mode)
|
||||
unconfigured (safe default returning structured errors; use until hardware is wired)
|
||||
|
||||
.PARAMETER FocasPipeName
|
||||
Pipe name the Host listens on. Default: OtOpcUaFocas.
|
||||
|
||||
.EXAMPLE
|
||||
.\Install-FocasHost.ps1 -InstallRoot 'C:\Program Files\OtOpcUa\Driver.FOCAS.Host' `
|
||||
-ServiceAccount 'OTOPCUA\svc-otopcua' -FocasBackend fwlib32
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory)] [string]$InstallRoot,
|
||||
[Parameter(Mandatory)] [string]$ServiceAccount,
|
||||
[string]$FocasSharedSecret,
|
||||
[ValidateSet('fwlib32','fake','unconfigured')] [string]$FocasBackend = 'unconfigured',
|
||||
[string]$FocasPipeName = 'OtOpcUaFocas',
|
||||
[string]$ServiceName = 'OtOpcUaFocasHost',
|
||||
[string]$NssmPath = 'C:\Program Files\nssm\nssm.exe'
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
function Resolve-Sid {
|
||||
param([string]$Account)
|
||||
if ($Account -match '^S-\d-\d+') { return $Account }
|
||||
try {
|
||||
$nt = New-Object System.Security.Principal.NTAccount($Account)
|
||||
return $nt.Translate([System.Security.Principal.SecurityIdentifier]).Value
|
||||
} catch {
|
||||
throw "Could not resolve '$Account' to a SID. Pass an explicit SID or check the account name."
|
||||
}
|
||||
}
|
||||
|
||||
if (-not (Test-Path $NssmPath)) {
|
||||
throw "nssm.exe not found at '$NssmPath'. Install NSSM or pass -NssmPath."
|
||||
}
|
||||
|
||||
$hostExe = Join-Path $InstallRoot 'OtOpcUa.Driver.FOCAS.Host.exe'
|
||||
if (-not (Test-Path $hostExe)) {
|
||||
throw "FOCAS Host binary not found at '$hostExe'. Publish the Driver.FOCAS.Host project first."
|
||||
}
|
||||
|
||||
if (-not $FocasSharedSecret) {
|
||||
$FocasSharedSecret = [System.Guid]::NewGuid().ToString('N')
|
||||
Write-Host "Generated FocasSharedSecret — store it alongside the OtOpcUa service config."
|
||||
}
|
||||
|
||||
$allowedSid = Resolve-Sid $ServiceAccount
|
||||
|
||||
# Idempotent install — remove + re-create if present.
|
||||
$existing = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
|
||||
if ($existing) {
|
||||
Write-Host "Removing existing '$ServiceName' service..."
|
||||
& $NssmPath stop $ServiceName confirm | Out-Null
|
||||
& $NssmPath remove $ServiceName confirm | Out-Null
|
||||
}
|
||||
|
||||
& $NssmPath install $ServiceName $hostExe | Out-Null
|
||||
& $NssmPath set $ServiceName DisplayName 'OT-OPC-UA FOCAS Host (Tier-C isolated Fwlib32)' | Out-Null
|
||||
& $NssmPath set $ServiceName Description 'Out-of-process Fwlib32.dll host for OtOpcUa FOCAS driver. Crash-isolated from the main OPC UA server.' | Out-Null
|
||||
& $NssmPath set $ServiceName ObjectName $ServiceAccount | Out-Null
|
||||
& $NssmPath set $ServiceName Start SERVICE_AUTO_START | Out-Null
|
||||
& $NssmPath set $ServiceName AppStdout (Join-Path $env:ProgramData 'OtOpcUa\focas-host-stdout.log') | Out-Null
|
||||
& $NssmPath set $ServiceName AppStderr (Join-Path $env:ProgramData 'OtOpcUa\focas-host-stderr.log') | Out-Null
|
||||
& $NssmPath set $ServiceName AppRotateFiles 1 | Out-Null
|
||||
& $NssmPath set $ServiceName AppRotateBytes 10485760 | Out-Null
|
||||
|
||||
& $NssmPath set $ServiceName AppEnvironmentExtra `
|
||||
"OTOPCUA_FOCAS_PIPE=$FocasPipeName" `
|
||||
"OTOPCUA_ALLOWED_SID=$allowedSid" `
|
||||
"OTOPCUA_FOCAS_SECRET=$FocasSharedSecret" `
|
||||
"OTOPCUA_FOCAS_BACKEND=$FocasBackend" | Out-Null
|
||||
|
||||
& $NssmPath set $ServiceName DependOnService OtOpcUa | Out-Null
|
||||
|
||||
Write-Host "Installed '$ServiceName' under '$ServiceAccount' (SID=$allowedSid)."
|
||||
Write-Host "Pipe: \\.\pipe\$FocasPipeName Backend: $FocasBackend"
|
||||
Write-Host "Start the service with: Start-Service $ServiceName"
|
||||
Write-Host ""
|
||||
Write-Host "NOTE: the Fwlib32 backend requires the licensed Fwlib32.dll on PATH"
|
||||
Write-Host "alongside the Host exe. See docs/v2/focas-deployment.md."
|
||||
27
scripts/install/Uninstall-FocasHost.ps1
Normal file
27
scripts/install/Uninstall-FocasHost.ps1
Normal file
@@ -0,0 +1,27 @@
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Removes the OtOpcUaFocasHost Windows service.
|
||||
|
||||
.DESCRIPTION
|
||||
Companion to Install-FocasHost.ps1. Stops + unregisters the service via NSSM.
|
||||
Idempotent — succeeds silently if the service doesn't exist.
|
||||
|
||||
.EXAMPLE
|
||||
.\Uninstall-FocasHost.ps1
|
||||
#>
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$ServiceName = 'OtOpcUaFocasHost',
|
||||
[string]$NssmPath = 'C:\Program Files\nssm\nssm.exe'
|
||||
)
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
$svc = Get-Service -Name $ServiceName -ErrorAction SilentlyContinue
|
||||
if (-not $svc) { Write-Host "Service '$ServiceName' not present — nothing to do."; return }
|
||||
|
||||
if (-not (Test-Path $NssmPath)) { throw "nssm.exe not found at '$NssmPath'." }
|
||||
|
||||
& $NssmPath stop $ServiceName confirm | Out-Null
|
||||
& $NssmPath remove $ServiceName confirm | Out-Null
|
||||
Write-Host "Removed '$ServiceName'."
|
||||
@@ -0,0 +1,133 @@
|
||||
using System;
|
||||
using System.IO;
|
||||
using System.IO.MemoryMappedFiles;
|
||||
using System.Text;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Host.Stability;
|
||||
|
||||
/// <summary>
|
||||
/// Ring-buffer of the last N IPC operations, written into a memory-mapped file. On a
|
||||
/// hard crash the Proxy-side supervisor reads the MMF after the corpse is gone to see
|
||||
/// what was in flight at the moment the Host died. Single-writer (the Host), multi-reader
|
||||
/// (the supervisor) — the file format is identical to the Galaxy Tier-C
|
||||
/// <c>PostMortemMmf</c> so a single reader tool can work both.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// File layout:
|
||||
/// <code>
|
||||
/// [16-byte header: magic(4) | version(4) | capacity(4) | writeIndex(4)]
|
||||
/// [capacity × 256-byte entries: each is [8-byte utcUnixMs | 8-byte opKind | 240-byte UTF-8 message]]
|
||||
/// </code>
|
||||
/// Magic is 'OFPC' (0x4F46_5043) to distinguish a FOCAS file from the Galaxy MMF.
|
||||
/// </remarks>
|
||||
public sealed class PostMortemMmf : IDisposable
|
||||
{
|
||||
private const int Magic = 0x4F465043; // 'OFPC'
|
||||
private const int Version = 1;
|
||||
private const int HeaderBytes = 16;
|
||||
public const int EntryBytes = 256;
|
||||
private const int MessageOffset = 16;
|
||||
private const int MessageCapacity = EntryBytes - MessageOffset;
|
||||
|
||||
public int Capacity { get; }
|
||||
public string Path { get; }
|
||||
|
||||
private readonly MemoryMappedFile _mmf;
|
||||
private readonly MemoryMappedViewAccessor _accessor;
|
||||
private readonly object _writeGate = new();
|
||||
|
||||
public PostMortemMmf(string path, int capacity = 1000)
|
||||
{
|
||||
if (capacity <= 0) throw new ArgumentOutOfRangeException(nameof(capacity));
|
||||
Capacity = capacity;
|
||||
Path = path;
|
||||
|
||||
var fileBytes = HeaderBytes + capacity * EntryBytes;
|
||||
Directory.CreateDirectory(System.IO.Path.GetDirectoryName(path)!);
|
||||
|
||||
var fs = new FileStream(path, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read);
|
||||
fs.SetLength(fileBytes);
|
||||
_mmf = MemoryMappedFile.CreateFromFile(fs, null, fileBytes,
|
||||
MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, leaveOpen: false);
|
||||
_accessor = _mmf.CreateViewAccessor(0, fileBytes, MemoryMappedFileAccess.ReadWrite);
|
||||
|
||||
if (_accessor.ReadInt32(0) != Magic)
|
||||
{
|
||||
_accessor.Write(0, Magic);
|
||||
_accessor.Write(4, Version);
|
||||
_accessor.Write(8, capacity);
|
||||
_accessor.Write(12, 0);
|
||||
}
|
||||
}
|
||||
|
||||
public void Write(long opKind, string message)
|
||||
{
|
||||
lock (_writeGate)
|
||||
{
|
||||
var idx = _accessor.ReadInt32(12);
|
||||
var offset = HeaderBytes + idx * EntryBytes;
|
||||
|
||||
_accessor.Write(offset + 0, DateTimeOffset.UtcNow.ToUnixTimeMilliseconds());
|
||||
_accessor.Write(offset + 8, opKind);
|
||||
|
||||
var msgBytes = Encoding.UTF8.GetBytes(message ?? string.Empty);
|
||||
var copy = Math.Min(msgBytes.Length, MessageCapacity - 1);
|
||||
_accessor.WriteArray(offset + MessageOffset, msgBytes, 0, copy);
|
||||
_accessor.Write(offset + MessageOffset + copy, (byte)0);
|
||||
|
||||
var next = (idx + 1) % Capacity;
|
||||
_accessor.Write(12, next);
|
||||
}
|
||||
}
|
||||
|
||||
public PostMortemEntry[] ReadAll()
|
||||
{
|
||||
var magic = _accessor.ReadInt32(0);
|
||||
if (magic != Magic) return new PostMortemEntry[0];
|
||||
|
||||
var capacity = _accessor.ReadInt32(8);
|
||||
var writeIndex = _accessor.ReadInt32(12);
|
||||
|
||||
var entries = new PostMortemEntry[capacity];
|
||||
var count = 0;
|
||||
for (var i = 0; i < capacity; i++)
|
||||
{
|
||||
var slot = (writeIndex + i) % capacity;
|
||||
var offset = HeaderBytes + slot * EntryBytes;
|
||||
|
||||
var ts = _accessor.ReadInt64(offset + 0);
|
||||
if (ts == 0) continue;
|
||||
|
||||
var op = _accessor.ReadInt64(offset + 8);
|
||||
var msgBuf = new byte[MessageCapacity];
|
||||
_accessor.ReadArray(offset + MessageOffset, msgBuf, 0, MessageCapacity);
|
||||
var nulTerm = Array.IndexOf<byte>(msgBuf, 0);
|
||||
var msg = Encoding.UTF8.GetString(msgBuf, 0, nulTerm < 0 ? MessageCapacity : nulTerm);
|
||||
|
||||
entries[count++] = new PostMortemEntry(ts, op, msg);
|
||||
}
|
||||
|
||||
Array.Resize(ref entries, count);
|
||||
return entries;
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_accessor.Dispose();
|
||||
_mmf.Dispose();
|
||||
}
|
||||
}
|
||||
|
||||
public readonly struct PostMortemEntry
|
||||
{
|
||||
public long UtcUnixMs { get; }
|
||||
public long OpKind { get; }
|
||||
public string Message { get; }
|
||||
|
||||
public PostMortemEntry(long utcUnixMs, long opKind, string message)
|
||||
{
|
||||
UtcUnixMs = utcUnixMs;
|
||||
OpKind = opKind;
|
||||
Message = message;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
using System.IO.MemoryMappedFiles;
|
||||
using System.Text;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Supervisor;
|
||||
|
||||
/// <summary>
|
||||
/// Proxy-side reader for the Host's post-mortem MMF. After a Host crash the supervisor
|
||||
/// opens the file (which persists beyond the process lifetime) and enumerates the last
|
||||
/// few thousand IPC operations that were in flight. Format matches
|
||||
/// <c>Driver.FOCAS.Host.Stability.PostMortemMmf</c> — magic 'OFPC' / 256-byte entries.
|
||||
/// </summary>
|
||||
public sealed class PostMortemReader
|
||||
{
|
||||
private const int Magic = 0x4F465043; // 'OFPC'
|
||||
private const int HeaderBytes = 16;
|
||||
private const int EntryBytes = 256;
|
||||
private const int MessageOffset = 16;
|
||||
private const int MessageCapacity = EntryBytes - MessageOffset;
|
||||
|
||||
public string Path { get; }
|
||||
|
||||
public PostMortemReader(string path) => Path = path;
|
||||
|
||||
public PostMortemEntry[] ReadAll()
|
||||
{
|
||||
if (!File.Exists(Path)) return [];
|
||||
|
||||
using var mmf = MemoryMappedFile.CreateFromFile(Path, FileMode.Open, null, 0, MemoryMappedFileAccess.Read);
|
||||
using var accessor = mmf.CreateViewAccessor(0, 0, MemoryMappedFileAccess.Read);
|
||||
|
||||
if (accessor.ReadInt32(0) != Magic) return [];
|
||||
|
||||
var capacity = accessor.ReadInt32(8);
|
||||
var writeIndex = accessor.ReadInt32(12);
|
||||
var entries = new PostMortemEntry[capacity];
|
||||
var count = 0;
|
||||
|
||||
for (var i = 0; i < capacity; i++)
|
||||
{
|
||||
var slot = (writeIndex + i) % capacity;
|
||||
var offset = HeaderBytes + slot * EntryBytes;
|
||||
var ts = accessor.ReadInt64(offset + 0);
|
||||
if (ts == 0) continue;
|
||||
var op = accessor.ReadInt64(offset + 8);
|
||||
var msgBuf = new byte[MessageCapacity];
|
||||
accessor.ReadArray(offset + MessageOffset, msgBuf, 0, MessageCapacity);
|
||||
var nulTerm = Array.IndexOf<byte>(msgBuf, 0);
|
||||
var msg = Encoding.UTF8.GetString(msgBuf, 0, nulTerm < 0 ? MessageCapacity : nulTerm);
|
||||
entries[count++] = new PostMortemEntry(ts, op, msg);
|
||||
}
|
||||
|
||||
Array.Resize(ref entries, count);
|
||||
return entries;
|
||||
}
|
||||
}
|
||||
|
||||
public readonly record struct PostMortemEntry(long UtcUnixMs, long OpKind, string Message);
|
||||
@@ -0,0 +1,113 @@
|
||||
using System.Diagnostics;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Ipc;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Supervisor;
|
||||
|
||||
/// <summary>
|
||||
/// Production <see cref="IHostProcessLauncher"/>. Spawns <c>OtOpcUa.Driver.FOCAS.Host.exe</c>
|
||||
/// with the pipe name / allowed-SID / per-spawn shared secret in the environment, waits for
|
||||
/// the pipe to come up, then connects a <see cref="FocasIpcClient"/> and wraps it in an
|
||||
/// <see cref="IpcFocasClient"/>. On <see cref="TerminateAsync"/> best-effort kills the
|
||||
/// process and closes the IPC stream.
|
||||
/// </summary>
|
||||
public sealed class ProcessHostLauncher : IHostProcessLauncher
|
||||
{
|
||||
private readonly ProcessHostLauncherOptions _options;
|
||||
private Process? _process;
|
||||
private FocasIpcClient? _ipc;
|
||||
|
||||
public ProcessHostLauncher(ProcessHostLauncherOptions options)
|
||||
{
|
||||
_options = options ?? throw new ArgumentNullException(nameof(options));
|
||||
}
|
||||
|
||||
public bool IsProcessAlive => _process is { HasExited: false };
|
||||
|
||||
public async Task<IFocasClient> LaunchAsync(CancellationToken ct)
|
||||
{
|
||||
await TerminateAsync(ct).ConfigureAwait(false);
|
||||
|
||||
var secret = _options.SharedSecret ?? Guid.NewGuid().ToString("N");
|
||||
|
||||
var psi = new ProcessStartInfo
|
||||
{
|
||||
FileName = _options.HostExePath,
|
||||
Arguments = _options.Arguments ?? string.Empty,
|
||||
UseShellExecute = false,
|
||||
CreateNoWindow = true,
|
||||
};
|
||||
psi.Environment["OTOPCUA_FOCAS_PIPE"] = _options.PipeName;
|
||||
psi.Environment["OTOPCUA_ALLOWED_SID"] = _options.AllowedSid;
|
||||
psi.Environment["OTOPCUA_FOCAS_SECRET"] = secret;
|
||||
psi.Environment["OTOPCUA_FOCAS_BACKEND"] = _options.Backend;
|
||||
|
||||
_process = Process.Start(psi)
|
||||
?? throw new InvalidOperationException($"Failed to start {_options.HostExePath}");
|
||||
|
||||
// Poll for pipe readiness up to the configured connect timeout.
|
||||
var deadline = DateTime.UtcNow + _options.ConnectTimeout;
|
||||
while (true)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
if (_process.HasExited)
|
||||
throw new InvalidOperationException(
|
||||
$"FOCAS Host exited before pipe was ready (ExitCode={_process.ExitCode}).");
|
||||
|
||||
try
|
||||
{
|
||||
_ipc = await FocasIpcClient.ConnectAsync(
|
||||
_options.PipeName, secret, TimeSpan.FromSeconds(1), ct).ConfigureAwait(false);
|
||||
break;
|
||||
}
|
||||
catch (TimeoutException)
|
||||
{
|
||||
if (DateTime.UtcNow >= deadline)
|
||||
throw new TimeoutException(
|
||||
$"FOCAS Host pipe {_options.PipeName} did not come up within {_options.ConnectTimeout:g}.");
|
||||
await Task.Delay(TimeSpan.FromMilliseconds(250), ct).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
|
||||
return new IpcFocasClient(_ipc, _options.Series);
|
||||
}
|
||||
|
||||
public async Task TerminateAsync(CancellationToken ct)
|
||||
{
|
||||
if (_ipc is not null)
|
||||
{
|
||||
try { await _ipc.DisposeAsync().ConfigureAwait(false); }
|
||||
catch { /* best effort */ }
|
||||
_ipc = null;
|
||||
}
|
||||
|
||||
if (_process is not null)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (!_process.HasExited)
|
||||
{
|
||||
_process.Kill(entireProcessTree: true);
|
||||
await _process.WaitForExitAsync(ct).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
catch { /* best effort */ }
|
||||
finally
|
||||
{
|
||||
_process.Dispose();
|
||||
_process = null;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public sealed record ProcessHostLauncherOptions(
|
||||
string HostExePath,
|
||||
string PipeName,
|
||||
string AllowedSid)
|
||||
{
|
||||
public string? SharedSecret { get; init; }
|
||||
public string? Arguments { get; init; }
|
||||
public string Backend { get; init; } = "fwlib32";
|
||||
public TimeSpan ConnectTimeout { get; init; } = TimeSpan.FromSeconds(15);
|
||||
public FocasCncSeries Series { get; init; } = FocasCncSeries.Unknown;
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
using System;
|
||||
using System.IO;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Host.Stability;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Host.Tests
|
||||
{
|
||||
[Trait("Category", "Unit")]
|
||||
public sealed class PostMortemMmfTests : IDisposable
|
||||
{
|
||||
private readonly string _tempPath;
|
||||
|
||||
public PostMortemMmfTests()
|
||||
{
|
||||
_tempPath = Path.Combine(Path.GetTempPath(), $"focas-mmf-{Guid.NewGuid():N}.bin");
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (File.Exists(_tempPath)) File.Delete(_tempPath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Write_and_read_preserve_order_and_content()
|
||||
{
|
||||
using (var mmf = new PostMortemMmf(_tempPath, capacity: 10))
|
||||
{
|
||||
mmf.Write(opKind: 1, "read R100");
|
||||
mmf.Write(opKind: 2, "write MACRO:500 = 3.14");
|
||||
mmf.Write(opKind: 3, "probe ok");
|
||||
}
|
||||
|
||||
// Reopen (simulating a reader after the writer crashed).
|
||||
using var reader = new PostMortemMmf(_tempPath, capacity: 10);
|
||||
var entries = reader.ReadAll();
|
||||
entries.Length.ShouldBe(3);
|
||||
entries[0].OpKind.ShouldBe(1L);
|
||||
entries[0].Message.ShouldBe("read R100");
|
||||
entries[1].OpKind.ShouldBe(2L);
|
||||
entries[2].Message.ShouldBe("probe ok");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Ring_buffer_wraps_at_capacity()
|
||||
{
|
||||
using var mmf = new PostMortemMmf(_tempPath, capacity: 3);
|
||||
for (var i = 0; i < 10; i++) mmf.Write(i, $"op-{i}");
|
||||
|
||||
var entries = mmf.ReadAll();
|
||||
entries.Length.ShouldBe(3);
|
||||
// Oldest surviving entry is op-7 (entries 7,8,9 survive in FIFO order).
|
||||
entries[0].Message.ShouldBe("op-7");
|
||||
entries[1].Message.ShouldBe("op-8");
|
||||
entries[2].Message.ShouldBe("op-9");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Truncated_message_is_null_terminated_and_does_not_overflow()
|
||||
{
|
||||
using var mmf = new PostMortemMmf(_tempPath, capacity: 4);
|
||||
var big = new string('x', 500); // longer than the 240-byte message capacity
|
||||
mmf.Write(42, big);
|
||||
|
||||
var entries = mmf.ReadAll();
|
||||
entries.Length.ShouldBe(1);
|
||||
entries[0].Message.Length.ShouldBeLessThanOrEqualTo(240);
|
||||
entries[0].OpKind.ShouldBe(42L);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reopening_with_existing_data_preserves_entries()
|
||||
{
|
||||
using (var first = new PostMortemMmf(_tempPath, capacity: 5))
|
||||
{
|
||||
first.Write(1, "first-run-1");
|
||||
first.Write(2, "first-run-2");
|
||||
}
|
||||
|
||||
using var second = new PostMortemMmf(_tempPath, capacity: 5);
|
||||
var entries = second.ReadAll();
|
||||
entries.Length.ShouldBe(2);
|
||||
entries[0].Message.ShouldBe("first-run-1");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
using System.IO.MemoryMappedFiles;
|
||||
using System.Text;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Supervisor;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// The Proxy-side <see cref="PostMortemReader"/> must read the Host's MMF format
|
||||
/// (magic 'OFPC', 256-byte entries). This test writes a hand-crafted file that mimics
|
||||
/// the Host's layout exactly + asserts the reader decodes it correctly. Keeps the two
|
||||
/// codebases in lockstep on the wire format without needing to reference the net48
|
||||
/// Host assembly from the net10 test project.
|
||||
/// </summary>
|
||||
[Trait("Category", "Unit")]
|
||||
public sealed class PostMortemReaderCompatibilityTests : IDisposable
|
||||
{
|
||||
private readonly string _tempPath = Path.Combine(Path.GetTempPath(), $"focas-mmf-compat-{Guid.NewGuid():N}.bin");
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
if (File.Exists(_tempPath)) File.Delete(_tempPath);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reader_parses_host_format_and_returns_entries_in_oldest_first_order()
|
||||
{
|
||||
const int magic = 0x4F465043;
|
||||
const int capacity = 5;
|
||||
const int headerBytes = 16;
|
||||
const int entryBytes = 256;
|
||||
const int messageOffset = 16;
|
||||
var fileBytes = headerBytes + capacity * entryBytes;
|
||||
|
||||
using (var fs = new FileStream(_tempPath, FileMode.CreateNew, FileAccess.ReadWrite, FileShare.Read))
|
||||
{
|
||||
fs.SetLength(fileBytes);
|
||||
using var mmf = MemoryMappedFile.CreateFromFile(fs, null, fileBytes,
|
||||
MemoryMappedFileAccess.ReadWrite, HandleInheritability.None, leaveOpen: false);
|
||||
using var acc = mmf.CreateViewAccessor(0, fileBytes, MemoryMappedFileAccess.ReadWrite);
|
||||
acc.Write(0, magic);
|
||||
acc.Write(4, 1);
|
||||
acc.Write(8, capacity);
|
||||
acc.Write(12, 2); // writeIndex — next write would land at slot 2
|
||||
|
||||
void WriteEntry(int slot, long ts, long op, string msg)
|
||||
{
|
||||
var offset = headerBytes + slot * entryBytes;
|
||||
acc.Write(offset + 0, ts);
|
||||
acc.Write(offset + 8, op);
|
||||
var bytes = Encoding.UTF8.GetBytes(msg);
|
||||
acc.WriteArray(offset + messageOffset, bytes, 0, bytes.Length);
|
||||
acc.Write(offset + messageOffset + bytes.Length, (byte)0);
|
||||
}
|
||||
|
||||
WriteEntry(0, 100, 1, "op-a");
|
||||
WriteEntry(1, 200, 2, "op-b");
|
||||
// Slots 2,3 unwritten (ts=0) — reader must skip.
|
||||
WriteEntry(4, 50, 9, "old-wrapped");
|
||||
}
|
||||
|
||||
var entries = new PostMortemReader(_tempPath).ReadAll();
|
||||
entries.Length.ShouldBe(3);
|
||||
// writeIndex=2 means the ring walk starts at slot 2, so iteration order is 2→3→4→0→1.
|
||||
// Slots 2 and 3 are empty; 4 yields "old-wrapped"; then 0="op-a", 1="op-b".
|
||||
entries[0].Message.ShouldBe("old-wrapped");
|
||||
entries[1].Message.ShouldBe("op-a");
|
||||
entries[2].Message.ShouldBe("op-b");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reader_returns_empty_when_file_missing()
|
||||
{
|
||||
new PostMortemReader(_tempPath + "-does-not-exist").ReadAll().ShouldBeEmpty();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Reader_returns_empty_when_magic_mismatches()
|
||||
{
|
||||
File.WriteAllBytes(_tempPath, new byte[1024]);
|
||||
new PostMortemReader(_tempPath).ReadAll().ShouldBeEmpty();
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user