Three root-cause fixes to get an elevated dev-box shell past session open through to real MXAccess reads: 1. PipeAcl — drop BUILTIN\Administrators deny ACE. UAC's filtered token carries the Admins SID as deny-only, so the deny fired even from non-elevated admin-account shells. The per-connection SID check in PipeServer.VerifyCaller remains the real authorization boundary. 2. PipeServer — swap the Hello-read / VerifyCaller order. ImpersonateNamedPipeClient returns ERROR_CANNOT_IMPERSONATE until at least one frame has been read from the pipe; reading Hello first satisfies that rule. Previously the ACL deny-first path masked this race — removing the deny ACE exposed it. 3. GalaxyIpcClient — add a background reader + single pending-response slot. A RuntimeStatusChange event between OpenSessionRequest and OpenSessionResponse used to satisfy the caller's single ReadFrameAsync and fail CallAsync with "Expected OpenSessionResponse, got RuntimeStatusChange". The reader now routes response kinds (and ErrorResponse) to the pending TCS and everything else to a handler the driver registers in InitializeAsync. The Proxy was already set up to raise managed events from RaiseDataChange / RaiseAlarmEvent / OnHostConnectivityUpdate — those helpers had no caller until now. 4. RedundancyPublisherHostedService — swallow BadServerHalted while polling host.Server.CurrentInstance. StandardServer throws that code during startup rather than returning null, so the first poll attempt crashed the BackgroundService (and the host) before OnServerStarted ran. This race was latent behind the Galaxy init failure above. Updates docs that described the Admins deny ACE + mandatory non-elevated shells, and drops the admin-skip guards from every Galaxy integration + E2E fixture that had them (IpcHandshakeIntegrationTests, EndToEndIpcTests, ParityFixture, LiveStackFixture, HostSubprocessParityTests). Adds GalaxyIpcClientRoutingTests covering the router's request/response match, ErrorResponse, event-between-call, idle event, and peer-close paths. Verified live on the dev box against the p7-smoke cluster (gen 6): driver registered=1 failedInit=0, Phase 7 bridge subscribed, OPC UA server up on 4840, MXAccess read round-trip returns real data with Status=0x00000000. Task #112 — partial: Galaxy live stack is functional end-to-end. The supplied test-galaxy.ps1 script still fails because the UNS walker encodes TagConfig JSON as the tag's NodeId instead of the seeded TagId (pre-existing; separate issue from this commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
166 lines
7.2 KiB
Markdown
166 lines
7.2 KiB
Markdown
# Galaxy test fixture
|
|
|
|
Coverage map + gap inventory for the Galaxy driver — out-of-process Host
|
|
(net48 x86 MXAccess COM) + Proxy (net10) + Shared protocol.
|
|
|
|
**TL;DR: Galaxy has the richest test harness in the fleet** — real Host
|
|
subprocess spawn, real ZB SQL queries, IPC parity checks against the v1
|
|
LmxProxy reference, + live-smoke tests when MXAccess runtime is actually
|
|
installed. Gaps are live-plant + failover-shaped: the E2E suite covers the
|
|
representative ~50-tag deployment but not large-site discovery stress, real
|
|
Rockwell/Siemens PLC enumeration through MXAccess, or ZB SQL Always-On
|
|
replica failover.
|
|
|
|
## What the fixture is
|
|
|
|
Multi-project test topology:
|
|
|
|
- **E2E parity** —
|
|
`tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.E2E/ParityFixture.cs` spawns the
|
|
production `OtOpcUa.Driver.Galaxy.Host.exe` as a subprocess, opens the
|
|
named-pipe IPC, connects `GalaxyProxyDriver` + runs hierarchy / stability
|
|
parity tests against both.
|
|
- **Host.Tests** —
|
|
`tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host.Tests/` — direct Host process
|
|
testing (18+ test classes covering alarm discovery, AVEVA prerequisite
|
|
checks, IPC dispatcher, alarm tracker, probe manager, historian
|
|
cluster/quality/wiring, history read, OPC UA attribute mapping,
|
|
subscription lifecycle, reconnect, multi-host proxy, ADS address routing,
|
|
expression evaluation) + `GalaxyRepositoryLiveSmokeTests` that hit real
|
|
ZB SQL.
|
|
- **Proxy.Tests** — `GalaxyProxyDriver` client contract tests.
|
|
- **Shared.Tests** — shared protocol + address model.
|
|
- **TestSupport** — test helpers reused across the above.
|
|
|
|
## How tests skip
|
|
|
|
- **E2E parity**: `ParityFixture.SkipIfUnavailable()` runs at class init and
|
|
checks Windows-only, ZB SQL reachable on `localhost:1433`, Host EXE built
|
|
in the expected `bin/` folder. Any miss → tests skip.
|
|
- **Live-smoke** (`GalaxyRepositoryLiveSmokeTests`): `Assert.Skip` when ZB
|
|
unreachable. A `per project_galaxy_host_installed` memory on this repo's
|
|
dev box notes the MXAccess runtime is installed. The pipe ACL allows the
|
|
configured SID outright; elevation of the caller doesn't matter because
|
|
the per-connection SID check in `PipeServer.VerifyCaller` only compares
|
|
user SIDs (not group membership or integrity level).
|
|
- **Unit** tests (Shared, Proxy contract, most Host.Tests) have no skip —
|
|
they run anywhere.
|
|
|
|
## What it actually covers
|
|
|
|
### E2E parity suite
|
|
|
|
- `HierarchyParityTests` — Host address-space hierarchy vs v1 LmxProxy
|
|
reference (same ZB, same Galaxy, same shape)
|
|
- `StabilityFindingsRegressionTests` — probe subscription failure
|
|
handling + host-status mutation guard from the v1 stability findings
|
|
backlog
|
|
|
|
### Host.Tests (representative)
|
|
|
|
- Alarm discovery → subsystem setup
|
|
- AVEVA prerequisite checks (runtime installed, platform deployed, etc.)
|
|
- IPC dispatcher — request/response routing over the named pipe
|
|
- Alarm tracker state machine
|
|
- Probe manager — per-runtime probe subscription + reconnect
|
|
- Historian cluster / quality / wiring — Aveva Historian integration
|
|
- OPC UA attribute mapping
|
|
- Subscription lifecycle + reconnect
|
|
- Multi-host proxy routing
|
|
- ADS address routing + expression evaluation (Galaxy's legacy expression
|
|
language)
|
|
|
|
### Live-smoke
|
|
|
|
- `GalaxyRepositoryLiveSmokeTests` — real SQL against ZB database, verifies
|
|
the ZB schema + `LocalPlatform` scope filter + change-detection query
|
|
shape match production.
|
|
|
|
### Capability surfaces hit
|
|
|
|
All of them: `IDriver`, `IReadable`, `IWritable`, `ITagDiscovery`,
|
|
`ISubscribable`, `IHostConnectivityProbe`, `IPerCallHostResolver`,
|
|
`IAlarmSource`, `IHistoryProvider`. Galaxy is the only driver where every
|
|
interface sees both contract + real-integration coverage.
|
|
|
|
## What it does NOT cover
|
|
|
|
### 1. MXAccess COM by default
|
|
|
|
The E2E parity suite backs subscriptions via the DB-only path; MXAccess COM
|
|
integration opts in via a separate live-smoke. So "does the MXAccess STA
|
|
pump correctly handle real Wonderware runtime events" is exercised only
|
|
when the operator runs live smoke on a machine with MXAccess installed.
|
|
|
|
### 2. Real Rockwell / Siemens PLC enumeration
|
|
|
|
Galaxy runtime talks to PLCs through MXAccess (Device Integration Objects).
|
|
The CI parity suite uses a representative ~50-tag deployment; large sites
|
|
(1000+ tag hierarchies, multi-Galaxy replication, deeply-nested templates)
|
|
are not stressed.
|
|
|
|
### 3. ZB SQL Always-On failover
|
|
|
|
Live-smoke hits a single SQL instance. Real production ZB often runs on
|
|
Always-On availability groups; replica failover behavior is not tested.
|
|
|
|
### 4. Galaxy replication / backup-restore
|
|
|
|
Galaxy supports backup + partial replication across platforms — these
|
|
rewrite the ZB schema in ways that change the contained_name vs tag_name
|
|
mapping. Not exercised.
|
|
|
|
### 5. Historian failover
|
|
|
|
Aveva Historian can be clustered. `historian cluster / quality` tests
|
|
verify the cluster-config query; they don't exercise actual failover
|
|
(primary dies → secondary takes over mid-HistoryRead).
|
|
|
|
### 6. AVEVA runtime version matrix
|
|
|
|
MXAccess COM contract varies subtly across System Platform 2017 / 2020 /
|
|
2023. The live-smoke runs against whatever version is installed on the dev
|
|
box; CI has no AVEVA installed at all (licensing + footprint).
|
|
|
|
## When to trust the Galaxy suite, when to reach for a live plant
|
|
|
|
| Question | E2E parity | Live-smoke | Real plant |
|
|
| --- | --- | --- | --- |
|
|
| "Does Host spawn + IPC round-trip work?" | yes | yes | yes |
|
|
| "Does the ZB schema query match production shape?" | partial | yes | yes |
|
|
| "Does MXAccess COM handle runtime reconnect correctly?" | no | yes | yes |
|
|
| "Does the driver scale to 1000+ tags on one Galaxy?" | no | partial | yes (required) |
|
|
| "Does historian failover mid-read return a clean error?" | no | no | yes (required) |
|
|
| "Does System Platform 2023's MXAccess differ from 2020?" | no | partial | yes (required) |
|
|
| "Does ZB Always-On replica failover preserve generation?" | no | no | yes (required) |
|
|
|
|
## Follow-up candidates
|
|
|
|
1. **System Platform 2023 live-smoke matrix** — set up a second dev box
|
|
running SP2023; run the same live-smoke against both to catch COM-contract
|
|
drift early.
|
|
2. **Synthetic large-site fixture** — script a ZB populator that creates a
|
|
1000-Equipment / 20000-tag hierarchy, run the parity suite against it.
|
|
Catches O(N) → O(N²) discovery regressions.
|
|
3. **Historian failover scripted test** — with a two-node AVEVA Historian
|
|
cluster, tear down primary mid-HistoryRead + verify the driver's failover
|
|
behavior + error surface.
|
|
4. **ZB Always-On CI** — SQL Server 2022 on Linux supports Always-On;
|
|
could stand up a two-replica group for replica-failover coverage.
|
|
|
|
This is already the best-tested driver; the remaining work is site-scale
|
|
+ production-topology coverage, not capability coverage.
|
|
|
|
## Key fixture / config files
|
|
|
|
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.E2E/ParityFixture.cs` — E2E fixture
|
|
that spawns Host + connects Proxy
|
|
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.Host.Tests/GalaxyRepositoryLiveSmokeTests.cs`
|
|
— live ZB smoke with `Assert.Skip` gate
|
|
- `tests/ZB.MOM.WW.OtOpcUa.Driver.Galaxy.TestSupport/` — shared helpers
|
|
- `docs/drivers/Galaxy.md` — COM bridge + STA pump + IPC architecture
|
|
- `docs/drivers/Galaxy-Repository.md` — ZB SQL reader + `LocalPlatform`
|
|
scope filter + change detection
|
|
- `docs/v2/aveva-system-platform-io-research.md` — MXAccess + Wonderware
|
|
background
|