lmxopcua/docs/v2/implementation/phase-2-galaxy-out-of-process.md

Phase 2 — Galaxy Out-of-Process Refactor (Tier C)

Status: DRAFT — implementation plan for Phase 2 of the v2 build (plan.md §6, driver-stability.md §"Galaxy — Deep Dive").

Branch: v2/phase-2-galaxy Estimated duration: 6–8 weeks (largest refactor phase; Tier C protections + IPC are the bulk) Predecessor: Phase 1 (phase-1-configuration-and-admin-scaffold.md) Successor: Phase 3 (Modbus TCP driver)

Phase Objective

Move Galaxy / MXAccess from the legacy in-process OtOpcUa.Host project into the Tier C out-of-process topology specified in driver-stability.md:

1. Driver.Galaxy.Shared — .NET Standard 2.0 IPC message contracts (MessagePack DTOs)
2. Driver.Galaxy.Host — .NET 4.8 x86 separate Windows Service that owns MxAccessBridge, GalaxyRepository, alarm tracking, GalaxyRuntimeProbeManager, the Wonderware Historian SDK, the STA thread + Win32 message pump, and all Tier C cross-cutting protections (memory watchdog, scheduled recycle, post-mortem MMF, IPC ACL + caller SID verification, per-process shared secret)
3. Driver.Galaxy.Proxy — .NET 10 in-process driver implementing every capability interface (IDriver, ITagDiscovery, IRediscoverable, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe), forwarding each call over named-pipe IPC and owning the supervisor (heartbeat, host liveness, respawn with backoff, crash-loop circuit breaker, fan-out of Bad quality on host death)
4. Retire the legacy OtOpcUa.Host project — its responsibilities now live in OtOpcUa.Server (built in Phase 1) for OPC UA hosting and OtOpcUa.Driver.Galaxy.Host for Galaxy-specific runtime

Parity, not regression. The phase exit gate is: the v1 IntegrationTests suite passes byte-for-byte against the v2 Galaxy.Proxy + Galaxy.Host topology, and a scripted Client.CLI walkthrough produces equivalent output to v1 (decision #56). Anything different — quality codes, browse paths, alarm shapes, history responses — is a parity defect.

This phase also closes the four 2026-04-13 stability findings (commits c76ab8f and 7310925) by adding regression tests to the parity suite per driver-specs.md Galaxy "Operational Stability Notes".

Scope — What Changes

Concern	Change
Project layout	3 new projects: Driver.Galaxy.Shared (.NET Standard 2.0), Driver.Galaxy.Host (.NET 4.8 x86), Driver.Galaxy.Proxy (.NET 10)
OtOpcUa.Host (legacy in-process)	Retired. Galaxy-specific code moves to Driver.Galaxy.Host; the small remainder (TopShelf wrapper, Program.cs) was already replaced by OtOpcUa.Server in Phase 1
MXAccess COM access	Now lives only in Driver.Galaxy.Host (.NET 4.8 x86, STA thread + Win32 message pump). Main server (OtOpcUa.Server, .NET 10 x64) never references ArchestrA.MxAccess
Wonderware Historian SDK	Same — only in Driver.Galaxy.Host
Galaxy DB queries	GalaxyRepository moves to Driver.Galaxy.Host; the SQL connection string lives in the Galaxy DriverConfig JSON
OPC UA address space build for Galaxy	Driven by Driver.Galaxy.Proxy calls into IAddressSpaceBuilder (Phase 1 API) — Proxy fetches the hierarchy via IPC, streams nodes to the builder
Subscriptions, reads, writes, alarms, history	All forwarded over named-pipe IPC via MessagePack contracts in Driver.Galaxy.Shared
Tier C cross-cutting protections	All wired up per driver-stability.md §"Cross-Cutting Protections" → "Isolated host only (Tier C)" + the Galaxy deep dive
Windows service installer	Two services per Galaxy-using cluster node: OtOpcUa (the main server) + OtOpcUaGalaxyHost (the Galaxy host). Installer scripts updated.
appsettings.json (legacy Galaxy config sections)	Migrated into the central config DB under DriverInstance.DriverConfig JSON for the Galaxy driver instance. Local appsettings.json keeps only Cluster.NodeId + ClusterId + DB conn (per decision #18)

Scope — What Does NOT Change

Item	Reason
OPC UA wire behavior visible to clients	Parity is the gate. Clients see the same browse paths, quality codes, alarm shapes, and history responses as v1
Galaxy hierarchy mapping (gobject parents → OPC UA folders)	Galaxy uses the SystemPlatform-kind namespace; UNS rules don't apply (decision #108). Tag.FolderPath mirrors v1 LmxOpcUa exactly
Galaxy EquipmentClassRef integration	Galaxy is SystemPlatform-namespace; no Equipment rows are created for Galaxy tags. Equipment-namespace work is for the native-protocol drivers in Phase 3+
Any non-Galaxy driver	Phase 3+
OtOpcUa.Server lifecycle / configuration substrate / Admin UI	Built in Phase 1; Phase 2 only adds the Galaxy.Proxy as a DriverInstance
Wonderware Historian dependency	Stays optional, loaded only when Historian.Enabled = true in the Galaxy DriverConfig

Entry Gate Checklist

• Phase 1 exit gate cleared (Configuration + Admin + Server + Core.Abstractions all green; Galaxy still in-process via legacy Host)
• v2 branch is clean
• Phase 1 PR merged
• Dev Galaxy reachable for parity testing — same Galaxy that v1 tests against
• v1 IntegrationTests baseline pass count + duration recorded (this is the parity bar)
• Client.CLI walkthrough script captured against v1 and saved as reference output
• All Phase 2-relevant docs reviewed: plan.md §3–4, §5a (LmxNodeManager reusability), driver-stability.md §"Out-of-Process Driver Pattern (Generalized)" + §"Galaxy — Deep Dive (Tier C)", driver-specs.md §1 (Galaxy)
• Decisions cited or implemented by Phase 2 read at least skim-level: #11, #24, #25, #28, #29, #32, #34, #44, #46–47, #55–56, #62, #63–69, #76, #102 (the Tier C IPC ACL + recycle decisions are all relevant)
• Confirmation that the four 2026-04-13 stability findings (c76ab8f, 7310925) have existing v1 tests that will be the regression net for the v2 split

Evidence file: docs/v2/implementation/entry-gate-phase-2.md.

Task Breakdown

Five work streams (A–E). Stream A is the foundation; B and C run partly in parallel after A; D depends on B + C; E is the parity gate at the end.

Stream A — Driver.Galaxy.Shared (1 week)

Task A.1 — Create the project

src/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Shared/ (.NET Standard 2.0 — must be consumable by both .NET 10 Proxy and .NET 4.8 Host per decision #25). Single dependency: MessagePack NuGet (decision #32).

Task A.2 — IPC message contracts

Define the MessagePack DTOs covering every Galaxy operation the Proxy will forward:

• Lifecycle: OpenSessionRequest, OpenSessionResponse, CloseSessionRequest, Heartbeat (separate channel per decision §"Heartbeat between proxy and host")
• Discovery: DiscoverGalaxyHierarchyRequest, GalaxyObjectInfo, GalaxyAttributeInfo (these are not the v1 Domain types — they're the IPC-shape with MessagePack attributes; the Proxy maps to/from DriverAttributeInfo from Core.Abstractions)
• Read / Write: ReadValuesRequest, ReadValuesResponse, WriteValuesRequest, WriteValuesResponse (carries DataValue shape per decision #13: value + StatusCode + timestamps)
• Subscriptions: SubscribeRequest, UnsubscribeRequest, OnDataChangeNotification (server-pushed)
• Alarms: AlarmSubscribeRequest, AlarmEvent, AlarmAcknowledgeRequest
• History: HistoryReadRequest, HistoryReadResponse
• Probe: HostConnectivityStatus, RuntimeStatusChangeNotification
• Recycle / control: RecycleHostRequest, RecycleStatusResponse

Length-prefixed framing per decision #28; MessagePack body inside each frame.

Acceptance:

• All contracts compile against .NET Standard 2.0
• Unit test project asserts each contract round-trips through MessagePack serialize → deserialize byte-for-byte
• Reflection test asserts no contract references System.Text.Json or anything not in BCL/MessagePack

Task A.3 — Versioning + capability negotiation

Add a top-of-stream Hello message exchanged on connection: protocol version, supported features. Future-proofs for adding new operations without breaking older Hosts.

Acceptance:

• Proxy refuses to talk to a Host advertising a major version it doesn't understand; logs the mismatch
• Host refuses to accept a Proxy from an unknown major version

Stream B — Driver.Galaxy.Host (3–4 weeks)

Task B.1 — Create the project + move Galaxy code

src/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host/ (.NET 4.8, x86 platform target — required for MXAccess COM per decision #23).

Move from legacy OtOpcUa.Host:

• MxAccessBridge.cs and supporting types
• GalaxyRepository.cs and SQL queries
• Alarm tracking infrastructure
• GalaxyRuntimeProbeManager.cs
• MxDataTypeMapper.cs, SecurityClassificationMapper.cs
• Historian plugin loader and IHistorianDataSource (only loaded when Historian.Enabled = true)
• Configuration types (MxAccessConfiguration, GalaxyRepositoryConfiguration, HistorianConfiguration, GalaxyScope) — these now read from the JSON DriverConfig rather than appsettings.json

Driver.Galaxy.Host does not reference Core.Abstractions (decision §5 dependency graph) — it's a closed unit, IPC-fronted.

Acceptance:

• Project builds against .NET 4.8 x86
• All moved files have their namespace updated to ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host.*
• v1 unit tests for these classes (still in OtOpcUa.Host.Tests) move to a new OtOpcUa.Driver.Galaxy.Host.Tests project and pass

Task B.2 — STA thread + Win32 message pump

Per driver-stability.md Galaxy deep dive:

• Single STA thread per Host process owns all LMXProxyServer instances
• Work item dispatch via PostThreadMessage(WM_APP)
• WM_QUIT shutdown only after all outstanding work items complete
• Pump health probe: no-op work item every 10s, timeout = wedged-pump signal that triggers recycle

This is essentially v1's StaComThread lifted from LmxProxy.Host reference (per CLAUDE.md "Reference Implementation" section).

Acceptance:

• Pump starts, dispatches work items, exits cleanly on WM_QUIT
• Pump-wedged simulation (work item that infinite-loops) triggers the 10s timeout and posts a recycle event
• COM call from non-STA thread fails fast with a recognizable error (regression net for cross-apartment bugs)

Task B.3 — MxAccessHandle : SafeHandle for COM lifetime

Wrap each LMXProxyServer connection in a SafeHandle subclass (decision #65 + Galaxy deep dive):

• ReleaseHandle() calls Marshal.ReleaseComObject until refcount = 0, then UnregisterProxy
• Subscription handles wrapped per item; RemoveAdvise → RemoveItem ordering enforced
• CriticalFinalizerObject for finalizer ordering during AppDomain unload
• Pre-shutdown drain: cancel all subscriptions cleanly via the STA pump, in order, before pump exit

Acceptance:

• Unit test asserts a leaked handle (no Dispose) is released by the finalizer
• Shutdown test asserts no orphan COM refs after Host exits cleanly
• Stress test: 1000 subscribe/unsubscribe cycles → handle table empty at the end

Task B.4 — Subscription registry + reconnect

Per driver-stability.md Galaxy deep dive §"Subscription State and Reconnect":

• In-memory registry of (Item, AdviseId, OwningHost) for every subscription
• Reconnect order: register proxy → re-add items → re-advise
• Cross-host quality clear gated on host-status check (closes 2026-04-13 finding)

Acceptance:

• Disconnect simulation: kill TCP to MXAccess; subscriptions go Bad; reconnect; subscriptions restore in correct order
• Multi-host test: stop AppEngine A while AppEngine B is running; verify A's subscriptions go Bad but B's stay Good (closes the cross-host quality clear regression)

Task B.5 — Connection health probe (GalaxyRuntimeProbeManager rebuild)

Lift the existing GalaxyRuntimeProbeManager into the new project. Behaviors per driver-stability.md:

• Subscribe to per-host runtime-status synthetic attribute
• Bad-quality fan-out scoped to the host's subtree (not Galaxy-wide)
• Failed probe subscription does not leave a phantom entry that Tick() flips to Stopped (closes 2026-04-13 finding)

Acceptance:

• Probe failure simulation → no phantom entry; Tick() does not flip arbitrary subscriptions to Stopped (regression test for the finding)
• Probe transitions Stopped → Running → Stopped → Running over 5 minutes; quality fan-out happens correctly each transition

Task B.6 — Named-pipe IPC server with mandatory ACL

Per decision #76 + driver-stability.md §"IPC Security":

• Pipe ACL on creation: ReadWrite | Synchronize granted only to the OtOpcUa server's service principal SID; LocalSystem and Administrators explicitly denied
• Caller identity verification on each new connection: GetImpersonationUserName() cross-checked against configured server service SID; mismatches dropped before any RPC frame is read
• Per-process shared secret: passed by the supervisor at spawn time, required on first frame of every connection
• Heartbeat pipe: separate from data-plane pipe, same ACL

Acceptance:

• Unit test: pipe ACL enumeration shows only the configured SID + Synchronize/ReadWrite
• Integration test: connection from a non-server-SID local process is dropped with audit log entry
• Integration test: connection without correct shared secret on first frame is dropped
• Defense-in-depth test: even if ACL is misconfigured (manually overridden), shared-secret check catches the wrong client

Task B.7 — Memory watchdog with Galaxy-specific thresholds

Per driver-stability.md Galaxy deep dive §"Memory Watchdog Thresholds":

• Sample RSS every 30s
• Warning: 1.5× baseline OR baseline + 200 MB (whichever larger)
• Soft recycle: 2× baseline OR baseline + 200 MB (whichever larger)
• Hard ceiling: 1.5 GB → force-kill
• Slope: > 5 MB/min sustained 30 min → soft recycle

Acceptance:

• Unit test against a mock RSS source: each threshold triggers the correct action
• Integration test with the FaultShim (Stream B.10): leak simulation crosses the soft-recycle threshold and triggers soft recycle path

Task B.8 — Recycle policy with WM_QUIT escalation

Per driver-stability.md Galaxy deep dive §"Recycle Policy (COM-specific)":

• 15s grace for in-flight COM calls (longer than FOCAS because legitimate MXAccess bulk reads take seconds)
• Per-handle: RemoveAdvise → RemoveItem → ReleaseComObject → UnregisterProxy, on the STA thread
• WM_QUIT posted only after all of the above complete
• If STA pump doesn't exit within 5s of WM_QUIT → Environment.Exit(2) (hard exit)
• Soft recycle scheduled daily at 03:00 local; recycle frequency cap 1/hour

Acceptance:

• Soft recycle test: in-flight call returns within grace → clean exit (Exit(0))
• Soft recycle test: in-flight call exceeds grace → hard exit (Exit(2)); supervisor records as unclean recycle
• Wedged-pump test: pump doesn't drain after WM_QUIT → Exit(2) within 5s
• Frequency cap test: trigger 2 soft recycles within an hour → second is blocked, alert raised

Task B.9 — Post-mortem MMF writer

Per driver-stability.md Galaxy deep dive §"Post-Mortem Log Contents":

• Ring buffer of last 1000 IPC operations
• Plus Galaxy-specific snapshots: STA pump state (thread ID, last dispatched timestamp, queue depth), active subscription count by host, MxAccessHandle refcount snapshot, last 100 probe results, last redeploy event, Galaxy DB connection state, Historian connection state if HDA enabled
• Memory-mapped file at %ProgramData%\OtOpcUa\driver-postmortem\galaxy.mmf
• On graceful shutdown: flush ring + snapshots to a rotating log
• On hard crash: supervisor reads the MMF after the corpse is gone

Acceptance:

• Round-trip test: write 1000 operations → read back → assert order + content
• Hard-crash test: kill the process mid-operation → supervisor reads the MMF → ring tail shows the operation that was in flight

Task B.10 — Driver.Galaxy.FaultShim (test-only)

Per driver-stability.md §"Test Coverage for Galaxy Stability" — analogous to FOCAS FaultShim:

• Test-only managed assembly substituted for ArchestrA.MxAccess.dll via assembly binding
• Injects: COM exception at chosen call site, subscription that never fires OnDataChange, Marshal.ReleaseComObject returning unexpected refcount, STA pump deadlock simulation
• Production builds load the real ArchestrA.MxAccess from GAC

Acceptance:

• FaultShim binds successfully under test configuration
• Each fault scenario triggers the expected protection (memory watchdog → recycle, supervisor → respawn, etc.)

Stream C — Driver.Galaxy.Proxy (1.5 weeks, can parallel with B after A done)

Task C.1 — Create the project + capability interface implementation

src/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Proxy/ (.NET 10). Dependencies: Core.Abstractions (Phase 1) + Driver.Galaxy.Shared (Stream A) + MessagePack.

Implement every interface listed in Phase Objective above. Each method:

• Marshals arguments into the matching IPC contract
• Sends over the data-plane pipe
• Awaits the response (with timeout per Polly per decision #34)
• Maps the response into the Core.Abstractions shape (DataValue, DriverAttributeInfo, etc.)
• Surfaces failures as the appropriate StatusCode

Acceptance:

• Each interface method has a unit test against a mock IPC channel: happy path + IPC timeout path + IPC error path
• IRediscoverable opt-in works: when Galaxy.Host signals a redeploy, Proxy invokes the Core's rediscovery flow (not full restart)

Task C.2 — Heartbeat sender + host liveness

Per driver-stability.md §"Heartbeat between proxy and host":

• 2s cadence (decision #72) on the dedicated heartbeat pipe
• 3 consecutive missed responses = host declared dead (6s detection)
• On host-dead: fan out Bad quality on all Galaxy-namespace nodes; ask supervisor to respawn

Acceptance:

• Heartbeat round-trip test against a mock host
• Missed-heartbeat test: stop the mock host's heartbeat responder → 3 misses → supervisor respawn requested
• GC pause test: simulate a 700ms GC pause on the proxy side → no false positive (single missed beat absorbed by 3-miss tolerance)

Task C.3 — Supervisor with respawn-with-backoff + crash-loop circuit breaker

Per driver-stability.md §"Crash-loop circuit breaker" + Galaxy §"Recovery Sequence After Crash":

• Backoff: 5s → 15s → 60s (capped)
• Crash-loop: 3 crashes / 5 min → escalating cooldown (1h → 4h → 24h manual)
• Sticky alert that doesn't auto-clear when cooldown elapses
• On respawn after recycle: reuse cached time_of_last_deploy watermark to skip full DB rediscovery if unchanged

Acceptance:

• Respawn test: kill host process → supervisor respawns within 5s → host re-establishes
• Crash-loop test: force 3 crashes within 5 minutes → 4th respawn blocked, alert raised, manual reset clears alert
• Cooldown escalation test: trip → 1h auto-reset → re-trip within 10 min → 4h cooldown → re-trip → 24h manual

Task C.4 — Address space build via IAddressSpaceBuilder

When the Proxy is asked to discover its tags, it issues DiscoverGalaxyHierarchyRequest to the Host, receives the gobject tree + attributes, and streams them to IAddressSpaceBuilder (Phase 1 API per decision #52). Galaxy uses the SystemPlatform-kind namespace; tags use FolderPath (v1-style) — no Equipment rows are created.

Acceptance:

• Build a Galaxy address space via the Proxy → byte-equivalent OPC UA browse output to v1
• Memory test: large Galaxy (4000+ attributes) → Proxy peak RAM stays under 200 MB during build

Stream D — Retire legacy OtOpcUa.Host (1 week, depends on B + C)

Task D.1 — Delete legacy Host project

Once Galaxy.Host + Galaxy.Proxy are functional, the legacy OtOpcUa.Host project's responsibilities are split:

• Galaxy-specific code → Driver.Galaxy.Host (already moved in Stream B)
• TopShelf wrapper, Program.cs, generic OPC UA hosting → already replaced by OtOpcUa.Server in Phase 1
• Anything else (configuration types, generic helpers) → moved to OtOpcUa.Server or OtOpcUa.Configuration as appropriate

Delete the project from the solution. Update .slnx and any references.

Acceptance:

• ls src/ shows OtOpcUa.Host is gone
• dotnet build OtOpcUa.slnx succeeds with OtOpcUa.Host no longer in the build graph
• All previously-OtOpcUa.Host.Tests tests are either moved to the appropriate new test project or deleted as obsolete

Task D.2 — Update Windows service installer scripts

Two services per cluster node when Galaxy is configured:

• OtOpcUa (the main OtOpcUa.Server) — already installable per Phase 1
• OtOpcUaGalaxyHost (the Driver.Galaxy.Host) — new service registration

Installer must:

• Install both services with the correct service-account SIDs (Galaxy.Host's pipe ACL must grant the OtOpcUa service principal)
• Set the supervisor's per-process secret in the registry or a protected file before first start
• Honor service dependency: Galaxy.Host should be configured to start before OtOpcUa, or OtOpcUa retries until Galaxy.Host is up

Acceptance:

• Install both services on a test box → both start successfully
• Uninstall both → no leftover registry / file system state
• Service-restart cycle: stop OtOpcUa.Server → Galaxy.Host stays up → start OtOpcUa.Server → reconnects to Galaxy.Host pipe

Task D.3 — Migrate Galaxy appsettings.json config to central config DB

Galaxy-specific config sections (MxAccess, Galaxy, Historian) move into the DriverInstance.DriverConfig JSON for the Galaxy driver instance in the Configuration DB. The local appsettings.json keeps only Cluster.NodeId + ClusterId + DB conn (per decision #18).

Migration script: for each existing v1 appsettings.json, generate the equivalent DriverConfig JSON and either insert via Admin UI or via a one-shot SQL script.

Acceptance:

• Migration script runs against a v1 dev appsettings.json → produces a JSON blob that loads into the Galaxy DriverConfig field
• The Galaxy driver instance starts with the migrated config and serves the same address space as v1

Stream E — Parity validation (1 week, gate)

Task E.1 — Run v1 IntegrationTests against v2 Galaxy topology

Per decision #56:

• The same v1 IntegrationTests suite runs against the v2 build with Galaxy.Proxy + Galaxy.Host instead of in-process Galaxy
• All tests must pass
• Pass count = v1 baseline; failure count = 0; skip count = v1 baseline
• Test duration may increase (IPC round-trip latency); document the deviation

Acceptance:

• Test report shows pass/fail/skip counts identical to v1 baseline
• Per-test duration regression report: any test that takes >2× v1 baseline is flagged for review (may be an IPC bottleneck)

Task E.2 — Scripted Client.CLI walkthrough parity

Per decision #56:

• Execute the captured Client.CLI script (recorded at Phase 2 entry gate against v1) against the v2 Galaxy topology
• Diff the output against v1 reference
• Differences allowed only in: timestamps, latency-measurement output. Any value, quality, browse path, or alarm shape difference = parity defect

Acceptance:

• Walkthrough completes without errors
• Output diff vs v1: only timestamp / latency lines differ

Task E.3 — Regression tests for the four 2026-04-13 stability findings

Per driver-specs.md Galaxy "Operational Stability Notes": each of the four findings closed in commits c76ab8f and 7310925 should have a regression test in the Phase 2 parity suite:

• Phantom probe subscription flipping Tick() to Stopped (covered by Task B.5)
• Cross-host quality clear wiping sibling state during recovery (covered by Task B.4)
• Sync-over-async on the OPC UA stack thread → guard against new instances in GenericDriverNodeManager
• Fire-and-forget alarm tasks racing shutdown → guard via the pre-shutdown drain ordering in Task B.3

Acceptance:

• Each of the four scenarios has a named test in the parity suite
• Each test fails on a hand-introduced regression (revert the v1 fix, see test fail)

Task E.4 — Adversarial review of the Phase 2 diff

Per implementation/overview.md exit gate:

• Run /codex:adversarial-review --base v2 on the merged Phase 2 diff
• Findings closed or explicitly deferred with rationale and ticket link

Compliance Checks (run at exit gate)

phase-2-compliance.ps1:

Schema compliance

N/A for Phase 2 — no schema changes (Configuration DB schema is unchanged from Phase 1).

Decision compliance

For each decision number Phase 2 implements (#11, #24, #25, #28, #29, #32, #34, #44, #46–47, #55–56, #62, #63–69, #76, #102, plus the Galaxy-specific #62), verify at least one citation exists in source, tests, or migrations:

$decisions = @(11, 24, 25, 28, 29, 32, 34, 44, 46, 47, 55, 56, 62, 63..69, 76, 102, 122, 123, 124)
foreach ($d in $decisions) {
    $hits = git grep "decision #$d" -- 'src/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.*/' 'tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.*/'
    if (-not $hits) { Write-Error "Decision #$d has no citation"; exit 1 }
}

Visual compliance

N/A — no Admin UI changes in Phase 2 (Galaxy is just another DriverInstance in the Drivers tab).

Behavioral compliance — parity smoke test

The parity suite (Stream E) is the smoke test:

1. v1 IntegrationTests pass count = baseline, fail count = 0
2. Client.CLI walkthrough output matches v1 (modulo timestamps/latency)
3. Four regression tests for 2026-04-13 findings pass

Stability compliance

For Phase 2 (introduces the first Tier C driver in production form):

• Galaxy.Host implements every Tier C cross-cutting protection from driver-stability.md:
  • SafeHandle wrappers for COM (Task B.3) ✓
  • Memory watchdog with Galaxy thresholds (Task B.7) ✓
  • Bounded operation queues per device (already in Core, Phase 1) ✓
  • Heartbeat between proxy and host on separate channel (Tasks A.2, B.6, C.2) ✓
  • Scheduled recycling with WM_QUIT escalation to hard exit (Task B.8) ✓
  • Crash-loop circuit breaker (Task C.3) ✓
  • Post-mortem MMF readable after hard crash (Task B.9) ✓
  • IPC ACL + caller SID verification + per-process shared secret (Task B.6) ✓

Each protection has at least one regression test. The compliance script enumerates and verifies presence:

$protections = @(
    @{Name="SafeHandle for COM"; Test="MxAccessHandleFinalizerReleasesCom"},
    @{Name="Memory watchdog"; Test="WatchdogTriggersRecycleAtThreshold"},
    @{Name="Heartbeat detection"; Test="ThreeMissedHeartbeatsDeclaresHostDead"},
    @{Name="WM_QUIT escalation"; Test="WedgedPumpEscalatesToHardExit"},
    @{Name="Crash-loop breaker"; Test="ThreeCrashesInFiveMinutesOpensCircuit"},
    @{Name="Post-mortem MMF"; Test="MmfSurvivesHardCrashAndIsReadable"},
    @{Name="Pipe ACL enforcement"; Test="NonServerSidConnectionRejected"},
    @{Name="Shared secret"; Test="ConnectionWithoutSecretRejected"}
)
foreach ($p in $protections) {
    $hits = dotnet test --filter "FullyQualifiedName~$($p.Test)" --no-build --logger "console;verbosity=quiet"
    if ($LASTEXITCODE -ne 0) { Write-Error "Stability protection '$($p.Name)' has no passing test '$($p.Test)'"; exit 1 }
}

Documentation compliance

• Any deviation from the Galaxy deep dive in driver-stability.md reflected back; new decisions added with supersedes notes if needed
• driver-specs.md §1 (Galaxy) updated to reflect the actual implementation if the IPC contract or recycle behavior differs from the design doc

Completion Checklist

Stream A — Driver.Galaxy.Shared

• Project created (.NET Standard 2.0, MessagePack-only dependency)
• All IPC contracts defined and round-trip tested
• Hello-message version negotiation implemented
• Reflection test confirms no .NET 10-only types leaked in

Stream B — Driver.Galaxy.Host

• Project created (.NET 4.8 x86)
• All Galaxy-specific code moved from legacy Host
• STA thread + Win32 pump implemented; pump health probe wired up
• MxAccessHandle : SafeHandle for COM lifetime
• Subscription registry + reconnect with cross-host quality scoping
• GalaxyRuntimeProbeManager rebuilt; phantom-probe regression test passes
• Named-pipe IPC server with mandatory ACL + caller SID verification + per-process secret
• Memory watchdog with Galaxy-specific thresholds
• Recycle policy with 15s grace + WM_QUIT escalation to hard exit
• Post-mortem MMF writer + supervisor reader
• FaultShim test-only assembly for fault injection

Stream C — Driver.Galaxy.Proxy

• Project created (.NET 10, depends on Core.Abstractions + Galaxy.Shared)
• All capability interfaces implemented (IDriver, ITagDiscovery, IRediscoverable, IReadable, IWritable, ISubscribable, IAlarmSource, IHistoryProvider, IHostConnectivityProbe)
• Heartbeat sender on dedicated channel; missed-heartbeat detection
• Supervisor with respawn-with-backoff + crash-loop circuit breaker (escalating cooldown 1h/4h/24h)
• Address space build via IAddressSpaceBuilder produces byte-equivalent v1 output

Stream D — Retire legacy OtOpcUa.Host

• Legacy OtOpcUa.Host project deleted from solution
• Windows service installer registers two services (OtOpcUa + OtOpcUaGalaxyHost)
• Galaxy appsettings.json config migrated into central DB DriverConfig
• Migration script tested against v1 dev config

Stream E — Parity validation

• v1 IntegrationTests pass with count = baseline, failures = 0
• Client.CLI walkthrough output matches v1 (modulo timestamps/latency)
• All four 2026-04-13 stability findings have passing regression tests
• Per-test duration regression report: no test >2× v1 baseline (or flagged for review)

Cross-cutting

• phase-2-compliance.ps1 runs and exits 0
• All 8 Tier C stability protections have named, passing tests
• Adversarial review of the phase diff — findings closed or deferred with rationale
• PR opened against v2, includes: link to this doc, link to exit-gate record, compliance script output, parity test report, adversarial review output
• Reviewer signoff (one reviewer beyond the implementation lead)
• exit-gate-phase-2.md recorded

Risks and Mitigations

Risk	Likelihood	Impact	Mitigation
IPC round-trip latency makes parity tests fail on timing assumptions	High	Medium	Per-test duration regression report identifies hot tests; tune timeouts in test config rather than in production code
MessagePack contract drift between Proxy and Host during development	Medium	High	Hello-message version negotiation rejects mismatched majors loudly; CI builds both projects in the same job
STA pump health probe is itself flaky and triggers spurious recycles	Medium	High	Probe interval tunable; default 10s gives 1000ms+ slack on a healthy pump; monitor via post-mortem MMF for false positives
Pipe ACL misconfiguration on installer leaves the IPC accessible to local users	Low	Critical	Defense-in-depth shared secret catches the case; ACL enumeration test in installer integration test
Galaxy.Host process recycle thrash if Galaxy or DB is intermittently unavailable	Medium	Medium	Crash-loop circuit breaker with escalating cooldown caps the thrash; Polly retry on the data path inside Host (not via supervisor restart) handles transient errors
Migration of appsettings.json Galaxy config to DB blob breaks existing deployments	Medium	Medium	Migration script is idempotent and dry-run-able; deploy script asserts central DB has the migrated config before stopping legacy Host
Phase 2 takes longer than 8 weeks	High	Medium	Mid-gate review at 4 weeks — if Stream B isn't past Task B.6 (IPC + ACL), defer Stream B.10 (FaultShim) to Phase 2.5 follow-up
Wonderware Historian SDK incompatibility with .NET 4.8 x86 in the new project layout	Low	High	Move and validate Historian loader as part of Task B.1 — early signal if SDK has any project-shape sensitivity
Hard-exit on wedged pump leaks COM resources	Accepted	Low	Documented intent: hard exit is the only safe response; OS process exit reclaims fds and the OS COM cleanup is best-effort. CNC equivalent in FOCAS deep dive accepts the same trade-off

Out of Scope (do not do in Phase 2)

• Any non-Galaxy driver (Phase 3+)
• UNS / Equipment-namespace work for Galaxy (Galaxy is SystemPlatform-namespace; no Equipment rows for Galaxy tags per decision #108)
• Equipment-class template integration with the schemas repo (Galaxy doesn't use EquipmentClassRef)
• Push-from-DB notification (decision #96 — v2.1)
• Any change to OPC UA wire behavior visible to clients (parity is the gate)
• Consumer cutover (ScadaBridge, Ignition, System Platform IO) — out of v2 scope, separate integration-team track per implementation/overview.md
• Removing the v1 deployment from production (a v2 release decision, not Phase 2)