Commit Graph

69 Commits

Author SHA1 Message Date
Joseph Doherty 1ccc237cb6 docs(test): record why these assert descriptors, not a built provider
v2-ci / build (push) Successful in 4m21s
v2-ci / unit-tests (push) Failing after 10m15s
Akka.TestKit is the family convention for non-end-to-end Akka testing and would
be the right tool here, but Akka.TestKit.Xunit2 is xunit-v2-only and this project
is on xunit.v3 - adding it produces CS0433 type conflicts. Directory.Packages.props
already documents the same constraint for the three projects deliberately held on
xunit v2 for TestKit. Revisit when Akka ships an xunit.v3 TestKit.
2026-07-18 12:19:50 -04:00
Joseph Doherty b3d1a26f38 fix(secrets): consume Secrets 0.2.1 - Akka replication was inert in 0.2.0
0.2.0's AddZbSecretsAkkaReplication never bound its own ISecretReplicator: it
called AddZbSecrets first, which TryAdds NoOpSecretReplicator, so the package's
own TryAdd was silently discarded. Replication published into a no-op sink and no
actor was ever spawned - no exception, no log line. Found by the Task 6 wiring
work, which is the first code that ever built a container around that extension.
Fixed upstream in 0.2.1.

Test consequence worth recording: the replication-enabled registration tests
previously passed against a plain ActorSystem only BECAUSE of that bug - the
no-op never touched Akka. With 0.2.1 they resolve a real replicator, which spawns
the actor, whose PreStart needs DistributedPubSub and therefore a joined cluster,
so provider-based assertions hang. Standing a real cluster up inside this test
assembly was attempted and did not work, so these assertions are now made against
the ServiceCollection, which is precisely where the defect lives. Actor creation
and convergence remain covered upstream by the library's TwoNodeClusterReplication
tests against a genuine 2-node cluster.

7 registration tests pass in 23ms. Full solution builds 0 errors; no warnings
originate from these files.
2026-07-18 12:12:02 -04:00
Joseph Doherty 3336ec08c7 feat(secrets): opt-in Akka cluster secret replication (default OFF; upstream blocker documented)
Routes the host's secrets registration through a new AddOtOpcUaSecrets extension
that gates the ISecretStore implementation on Secrets:Replication:Enabled.

Opt-in gate (default FALSE)
  This call decides which ISecretStore every node resolves — including driver-role
  nodes with no auth/AdminUI, where a wrong store surfaces as drivers failing to
  open sessions rather than as a failing test. With the flag false the wiring is
  the pre-existing AddZbSecrets(config, "Secrets") call, unchanged, so current
  behavior is byte-identical. With it true, AddZbSecretsAkkaReplication replaces
  that call (it invokes AddZbSecrets internally; calling both would double-register).

  Extracted to a named extension specifically so the registration is testable:
  Program.cs is top-level statements and cannot be exercised by a container test,
  which is how a "registered but never resolvable" defect ships unnoticed.

Serializer HOCON
  AkkaSecretsReplication.SerializationConfig is merged into the ActorSystem config
  inside the AddAkka configurator, conditionally on the same gate — a non-replicating
  node carries no bindings for messages it will never see. Merged via
  AddHocon(..., HoconAddMode.Append), Akka.Hosting's fallback merge and the same mode
  the existing base-config merge uses; a raw Config.WithFallback would fight the
  builder's own assembly.

Lazy-actor mitigation
  The replication actor is created lazily on first ISecretStore resolution, so a node
  that never touches a secret would never announce a manifest and would silently never
  converge. SecretReplicationStarter (IHostedService) resolves the store once at
  startup to make participation unconditional.

KNOWN BLOCKER — replication is currently NON-FUNCTIONAL; do not enable
  ZB.MOM.WW.Secrets.Replicator.AkkaDotNet 0.2.0 never binds its own ISecretReplicator.
  AddZbSecretsAkkaReplication calls AddZbSecrets FIRST, which does
  TryAddSingleton<ISecretReplicator, NoOpSecretReplicator>(); the package's own
  TryAddSingleton<ISecretReplicator>(AkkaSecretReplicator) that follows is therefore
  a no-op. Verified empirically in a built container: with Enabled=true,
  ISecretReplicator resolves to NoOpSecretReplicator, so ReplicatingSecretStore
  publishes into a sink and no actor is ever spawned.

  Consequence: the startup hook cannot create the actor, and the test asserting it
  does is committed Skipped with the evidence. Not worked around here — the fix
  belongs upstream (AddSingleton, or register before calling AddZbSecrets).
  Because the flag defaults false, this commit is inert in production.

Tests: SecretsReplicationRegistrationTests (new) — disabled path resolves plain
SqliteSecretStore and needs no ActorSystem; enabled path resolves
ReplicatingSecretStore AND the undecorated concrete SqliteSecretStore the decorator
is built from (the exact registration gap that shipped once); startup hook registered
only when enabled. Red before wiring (4 assertion failures), green after: 6 pass,
1 skipped (blocker above).

Build: 861 warnings / 0 errors, unchanged from baseline (full --no-incremental A/B).
Host.IntegrationTests: 123 pass, 6 skip, 1 fail — AbCip_Green_AgainstSim, verified
pre-existing on the stashed tree (fixture-gated).

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
2026-07-18 11:15:03 -04:00
Joseph Doherty 872cf7e37a test(secrets): register ISecretResolver in driver-factory resilience tests (Task 10 fix)
v2-ci / build (push) Successful in 3m36s
v2-ci / unit-tests (push) Failing after 8m35s
The full-suite gate caught a real regression: AddOtOpcUaDriverFactories' registration
now resolves ISecretResolver (for Galaxy/OpcUaClient secret: refs, Tasks 7-8), so the
two ResilienceInvokerFactoryRegistrationTests that build a minimal container and resolve
the invoker factory threw 'No service for ISecretResolver'. The real host always
registers it via AddZbSecrets (unconditional, Task 3), and GetRequiredService correctly
fails-fast for a genuinely misconfigured host — so the fix is to complete the test
container with a stub resolver, matching production composition. Test-only.
2026-07-16 18:27:59 -04:00
Joseph Doherty 1badc10445 test(v3-batch4): add Calculation to the driver-probe idempotency key set
Pre-existing stale test surfaced by Batch 4's full gate (Batch 2/3 gates were
live-/run and never ran Host.IntegrationTests): Batch 2 registered CalculationProbe
in DriverFactoryBootstrap (9 probes) but AddOtOpcUaDriverProbes_is_idempotent's
AdminUiDriverTypeKeys still listed 8, so distinctTypes(9) != Length(8). Calculation
is a real DriverTypeNames.Calculation pseudo-driver with a probe; add it and refresh
the now-stale *DriverPage.razor comment (that routed flow retired in Batch 2). Not a
Batch-4 code change — a test-correctness fix.

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 13:22:27 -04:00
Joseph Doherty b8208b3312 test+docs(v3-batch4-wp5): 2-node dual-namespace harness tests + address-space docs
Tests:
- OpcUaServer.IntegrationTests/DualNamespaceAddressSpaceTests.cs (NEW, over-the-wire,
  offline-safe): both namespace URIs registered + distinct; Raw + UNS subtrees browse
  and read; UNS variable Organizes-references its raw node; single-source fan-out parity
  (identical value/quality/timestamp on both NodeIds); HistoryRead via either NodeId ->
  GoodNoData under the shared tagname; WriteOperate gate symmetric across both NodeIds.
- Host.IntegrationTests/EquipmentNamespaceMaterializationTests.cs (extended): full deploy
  -> persisted-artifact -> ParseComposition round-trip carrying both realms, sealing across
  the redundant 2-node cluster (redundancy non-interference). In-memory harness, offline.

Docs (dual-namespace reality):
- CLAUDE.md: new "v3 OPC UA Address Space (Batch 4)" section + Batch-4 testing paragraph.
- docs/Uns.md: address-space projection (two namespaces, Organizes edge, effective-name leaf).
- docs/Historian.md: dual-registration (both NodeIds -> one tagname); updated CLI examples.
- docs/ScriptedAlarms.md + docs/AlarmTracking.md: multi-notifier fan-out, ConditionId=RawPath.
- docs/ScriptEditor.md: dual-namespace clarification (script tag-path semantics unchanged).

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 13:11:34 -04:00
Joseph Doherty 2e0743ad25 fix(v3-batch4-wp3): realm-qualified write routing + dormant discovery guard + self-correction/byte-parity tests (Wave B review H1/M1/M2/L1/L3)
H1 (HIGH): write-routing key now (AddressSpaceRealm, bareId), not bare-only.
A raw s=<RawPath> and a UNS s=<Area/Line/Equip/Eff> can collide as bare
strings; the bare-only key let a colliding raw+UNS pair route to the WRONG
driver ref (last-writer-wins). The realm the node manager resolves (RealmOf)
is now threaded through IOpcUaNodeWriteGateway.WriteAsync -> RouteNodeWrite ->
_driverRefByNodeId keyed by (realm, bareId). New regression test:
Colliding_raw_and_uns_bare_ids_route_to_their_own_driver_by_realm.

M1 (MEDIUM): discovered-node injection made coherently DORMANT. HandleDiscoveredNodes
hard-short-circuits (single enforcement point; _discoveredByDriver never
populates so the re-inject tail is inert too), with a clear log pointing at the
/raw browse-commit flow. New pin: Discovered_nodes_are_ignored_dormant_in_v3;
the 16+2 v2 injection scenarios re-pointed to an accurate skip reason
(DiscoveryInjectionDormantV3).

M2 (MEDIUM): realm-qualified dual-node self-correction tests —
Failed_uns_write_reverts_uns_node_and_leaves_raw_node_untouched +
Raw_realm_revert_reverts_raw_node_only (the second fails if the realm is dropped).

L1: removed the = AddressSpaceRealm.Uns defaults from the consequential
node-manager mutation methods (WriteValue/WriteAlarmCondition/MaterialiseAlarmCondition/
EnsureFolder/EnsureVariable/UpdateFolderDisplayName/UpdateTagAttributes/
RaiseNodesAddedModelChange/Remove*/RevertOptimisticWriteIfNeeded) + the
AttributeValueUpdate/AlarmStateUpdate records, so the compiler forces explicit
realm; read-only accessors + internal builders retain their defaults.

L3: fixed the stale VirtualTagHostActor class comment (V3NodeIds.Uns, not the
retired EquipmentNodeIds.Variable).

Also: DeploymentArtifactRawUnsParityTests — Raw/UNS node-set byte-parity
round-trip between AddressSpaceComposer.Compose and DeploymentArtifact.ParseComposition.

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 11:30:13 -04:00
Joseph Doherty f4f3e17e3e feat(v3-batch4-wp1): composer/planner emit both Raw + UNS subtrees
Un-darken the address-space composition for the v3 dual namespace.

Composer (AddressSpaceComposer):
- Raw subtree: RawContainerNode (Folder/Driver/Device/TagGroup as
  Object/Folder nodes, keyed s=<RawPath>, parent-before-child) + RawTagPlan
  (raw tag Variables keyed (realm=Raw, s=<RawPath>) carrying DataType /
  writable / historize+historian tagname / array shape). Native-alarm intent
  attaches at the RAW tag (ConditionId = RawPath) with the list of referencing
  equipment UNS paths (one alarm at the raw tag, not one per equipment).
- UNS subtree: UnsReferenceVariable per UnsTagReference, keyed
  (realm=Uns, s=<Area>/<Line>/<Equipment>/<EffectiveName>), carrying its
  backing RawPath (Organizes UNS->Raw target + fan-out) and inheriting
  DataType/AccessLevel from the backing raw tag. Effective name =
  DisplayNameOverride else backing raw Tag.Name.
- All RawPaths flow through one shared RawTopology (RawPathResolver +
  memoised container paths), byte-parity with EquipmentReferenceMap.
- Every new node carries an AddressSpaceRealm (so WP3's applier picks the
  namespace at the call site). Additive/defaulted model changes only — the
  un-migrated AddressSpaceApplier still compiles.

Planner (AddressSpacePlanner.Compute) + AddressSpacePlan:
- Per-realm diff sets: RawContainers + RawTags keyed by RawPath (NodeId) so a
  rename = remove+add; UnsReferenceVariables keyed by the stable
  UnsTagReferenceId so a backing-tag rename re-points (BackingRawPath moves,
  NodeId stable) and a display-name-override edit is UNS-only.
- PINNED: raw-tag rename = remove+add in Raw AND re-point in UNS.

Classifier folds the new sets in (adds->PureAdd, removes->PureRemove,
changed->Rebuild safe default).

Tests: AddressSpaceComposerDualNamespaceTests (7),
AddressSpacePlannerDualNamespaceTests (8, incl. the rename pin),
AddressSpaceChangeClassifierDualNamespaceTests (8); reactivated
EquipmentNamespaceMaterializationTests' Batch-4-pending case, re-authored to
drive the composer over the seeded config (the artifact-decode seam is WP3).

Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
2026-07-16 09:30:12 -04:00
Joseph Doherty 2c7fc65bae test(v3): migrate OpcUaServer.Tests + Host.IntegrationTests to v3 dark address space
Migrate the server composer/applier + 2-node harness tests off the retired
Namespace/NamespaceKind + equipment→driver/device binding schema onto the v3
greenfield shape (raw-only Tag; DriverInstance-RawFolderId; equipment via UnsLine).

OpcUaServer.Tests (134 build errors -> green, 335 pass / 4 skip):
- LIVE (migrated, passing): VirtualTag-historize + ScriptedAlarm composition,
  composer purity/hierarchy, applier MaterialiseHierarchy + MaterialiseEquipmentTags
  idempotency (hand-fed applier, not dark).
- SKIP dark-until-Batch-4: {{equip}} token expansion, applier equipment-namespace
  E2E (equipment-tag variable materialization).
- RETIRE-with-comment / skip: alias-tag (Tag.EquipmentId binding gone) + device-host
  equipment binding (architecturally removed).
- Add local DarkAddressSpaceReasons Skip-reason constants (mirrors Runtime.Tests).

Host.IntegrationTests (green, in-memory): drop deleted-Namespace seeds from
MultiClusterScoping + DriverReconnect; reshape EquipmentNamespaceMaterialization to
assert the DARK contract (folder nodes decode, EquipmentTags empty despite a seeded
v3 raw tag) with the full raw-tag->EquipmentTag materialization kept as a
Batch-4-pending Skip.

Tests-only; no src/ changes.
2026-07-15 21:41:04 -04:00
Joseph Doherty b0eb653bad fix(redundancy): 2-node SBR exit-and-rejoin recovery — watchdog + restart supervision + both-node seeds (#459)
v2-ci / unit-tests (pull_request) Failing after 12m9s
v2-ci / build (pull_request) Successful in 5m12s
Corrects the #459 finding. 2-node keep-oldest recovery works fine (the ScadaBridge
sister project proves it); OtOpcUa was missing the supervision pieces that make it
automatic, and docs/Redundancy.md wrongly claimed in-place oldest-crash failover.

Mechanism (confirmed on a 2-container rig + by decompiling Akka KeepOldest.OldestDecision):
on an OLDEST-node crash keep-oldest downs the LONE survivor (DownReachable including
myself) — down-if-alone can't rescue a lone survivor (its branch needs >=2 survivors).
Recovery is exit-and-rejoin: run-coordinated-shutdown-when-down terminates the node and
the service supervisor restarts it. My earlier 'total outage' was a docker-dev artifact
(no restart policy); production Install-Services.ps1 already has sc.exe failure restart.

Changes (ScadaBridge parity):
- ActorSystemTerminationWatchdog (Host, registered after AddAkka): watches
  ActorSystem.WhenTerminated and on an unexpected self-down calls StopApplication so the
  process exits (supervisor restarts it) instead of idling with a dead actor system.
  Distinguishes graceful shutdown via _stopRequested + ApplicationStopping. 3 unit tests.
- docker-dev: restart: unless-stopped on the host anchor (models production supervision) +
  both redundancy peers in SeedNodes so a restarted node re-forms via either peer.
- docs/Redundancy.md: rewrote the split-brain recovery section — younger-loss = in-place
  fast failover; oldest-loss = exit-and-rejoin under supervision (not in-place); the three
  requirements (supervisor + watchdog + both-node seeds); flagged HardKillFailoverTests as
  non-representative (Transport.Shutdown, not a real crash). Instant in-place takeover on
  ANY single loss needs 3+ members.

Cluster.Tests 29/29 (SBR guards), watchdog tests 3/3, full solution builds.
Live re-verify of the watchdog image pending (host docker disk full).
2026-07-15 10:52:36 -04:00
Joseph Doherty 4d2dcff5df test(ldap): live GLAuth outage/recovery gate (R2-08 03/S2)
v2-ci / build (pull_request) Successful in 3m59s
v2-ci / unit-tests (pull_request) Failing after 11m55s
Closes the R2-08 S2 live leg. The offline LdapAuthResilienceTests prove the
directory-outage circuit with fakes + an unroutable-host blackhole; this drives
the SAME circuit through the production auth path (OtOpcUaLdapAuthService +
LdapOpcUaUserAuthenticator) against a REAL GLAuth that is paused mid-run and
unpaused — and doubles as the first end-to-end verification that the PR #451
GLAuth swap (retired bitnami/openldap) binds correctly.

Triple-gated (GLAUTH_LIVE_HOST + GLAUTH_OUTAGE_START_CMD/STOP_CMD), skips clean
offline (verified: 1 skipped, 8 ms). try/finally always unpauses GLAuth.

Verified GREEN against 10.100.0.35:3894 (12 s): healthy binds (alice+bob, mapped
roles) -> docker pause -> 3 bounded failures -> circuit opens -> sub-second
"unavailable" fast-deny -> docker unpause -> half-open probe closes -> binds
succeed again. A boundary AuthTimeout feeds the circuit (authenticator :123-130),
so the docker-pause timeout shape is a faithful outage.
2026-07-15 07:45:07 -04:00
Joseph Doherty 859d63178a fix(harness): make deploy E2E tests pass against real SQL (seed cluster FKs)
v2-ci / build (pull_request) Successful in 4m23s
v2-ci / unit-tests (pull_request) Failing after 10m14s
Integration-sweep follow-up #6 (SQL leg). The sweep flagged 7 heavy 2-node
deploy/failover/reconnect E2E tests as 'time out under amd64-emulated SQL' and
suggested raising deadlines. That diagnosis was WRONG: run against real
(native-amd64) SQL on the Docker host, they still hang — even at 4x deadline.
Root cause is that these tests had never actually run against real SQL, only the
EF in-memory provider, which does NOT enforce foreign keys.

Two distinct defects, both hidden by in-memory:

1. Missing FK-parent seed (5 tests). The deploy records a NodeDeploymentState row
   per node, FK'd to ClusterNode (FK_NodeDeploymentState_ClusterNode_NodeId). The
   tests never seeded ServerCluster + ClusterNode, so on SQL each node-state INSERT
   throws and the deploy never seals (waits the full deadline regardless of size).
   Added TwoNodeClusterHarness.SeedDefaultClusterAsync (seeds a ServerCluster +
   a ClusterNode per node; Warm/NodeCount=2 to satisfy the SQL CHECK constraint
   CK_ServerCluster_RedundancyMode_NodeCount AND the ClusterEnabledNodeCountMismatch
   validator). Called it in DeployHappyPath x2 / Failover / FleetDiagnostics /
   EquipmentNamespace; fixed DriverReconnect to seed node B + NodeCount=2.

2. Stale EquipmentId (EquipmentNamespace test; pre-existing on BOTH providers).
   Seeded EquipmentId='eq-1', but the later-added EquipmentIdNotDerived validator
   requires EquipmentId == DeriveEquipmentId(EquipmentUuid) ('EQ-'+first 12 hex).
   Fixed the seed to a canonical id + matching UUID. Only surfaced once the FK fix
   let the deploy reach equipment validation.

Also added OTOPCUA_HARNESS_SQL_HOST / OTOPCUA_HARNESS_LDAP_HOST overrides so the
harness fixtures can run on the native-amd64 Docker host (avoids arm64 mssql
emulation entirely).

Verified: the 5 SQL-backed classes go 11/11 green vs native SQL; in-memory default
unregressed (12/12). RoslynVirtualTagEvaluatorTests racing test is a pre-existing
flaky race (not SQL-backed; passes in isolation) — left as-is.
2026-07-15 06:43:15 -04:00
Joseph Doherty 3a48eb0c70 fix(harness): swap retired bitnami/openldap for GLAuth in the Host real-LDAP mode
v2-ci / build (pull_request) Successful in 4m13s
v2-ci / unit-tests (pull_request) Failing after 10m18s
The Host.IntegrationTests opt-in real-LDAP mode (OTOPCUA_HARNESS_USE_LDAP=1)
used bitnami/openldap:2.6, gone since Bitnami deprecated their free image
catalog in 2025. Replaced it with GLAuth — the same LDAP server the rest of
the project already uses (docker-dev + the live OPC UA data-plane auth, both
against the shared GLAuth on :3893) — removing the lone OpenLDAP outlier and
the gone-image breakage.

- Added tests/.../Host.IntegrationTests/glauth/config.toml: baseDN dc=zb,dc=local,
  a search-capable serviceaccount, dev users alice (full access) / bob (read-only),
  and role groups with the same gidnumbers as scadaproj/infra/glauth so GroupToRole
  maps identically.
- Compose ldap service -> glauth/glauth:latest, mounting the config, host :3894 -> :3893.
- Repointed the harness real-LDAP override from the OpenLDAP cn=admin/ldapadmin to
  GLAuth's cn=serviceaccount/serviceaccount123.

Live-verified against a deployed container on :3894: the serviceaccount bind
searches and returns alice + her 5 memberOf groups; alice/bob bind with the
correct password; a wrong password yields Invalid credentials (49) — exactly the
OtOpcUaLdapAuthService flow (proven identical to the data-plane's shared-GLAuth path).

NB: real-LDAP mode is opt-in; the default Host run uses StubLdapAuthService, so
this never blocked the default suite. Integration-sweep follow-up #6 (LDAP leg);
the amd64-emulated-SQL deadline leg remains open.
2026-07-15 06:03:50 -04:00
Joseph Doherty 62a49779ac test(host): unreachable-directory activation fails bounded; document the SDK impersonation threading contract (03/S2) 2026-07-13 12:00:35 -04:00
Joseph Doherty 0c785415a2 feat(host): LDAP directory-outage circuit — fail-fast denial during sustained outage (03/S2) 2026-07-13 11:56:45 -04:00
Joseph Doherty 21abe42a90 feat(host): hard wall-clock bound + bounded concurrency on OPC UA LDAP authentication (03/S2) 2026-07-13 11:52:10 -04:00
Joseph Doherty 09faa05f06 feat(security): LDAP auth timeout/concurrency/outage-circuit options; project ConnectionTimeoutMs into the shared library (03/S2) 2026-07-13 11:38:37 -04:00
Joseph Doherty ce6d9e21d3 test(host): red repro — LDAP authenticator has no wall-clock bound on session activation (03/S2) 2026-07-13 11:36:32 -04:00
Joseph Doherty d62b140ccd Merge R2-04 Failure-visibility trio (arch-review round 2) [PR #433]
v2-ci / build (push) Successful in 3m48s
v2-ci / unit-tests (push) Failing after 9m14s
Findings 01/S-1 (AddressSpaceApplyOutcome failure field + apply.failed logging,
no optimistic success), 06/S-1 (Galaxy write fails closed -> BadCommunicationError
+ #5 revert, no knowingly-lost raw Write), 03/S4 (PrimaryGatePolicy default-deny
unknown-role-multi-driver on all gates + scripted-alarm emit gate). T13/T15 (2-node
live gates) deferred -- T15 is the behavior-affecting S4 live gate, must run in heavy
pass. Clean merge, build clean.
2026-07-13 11:19:24 -04:00
Joseph Doherty 68588da05f test(r2-04): 2-node delivery-path guard for the primary write gate (deferred-live) 2026-07-13 11:13:31 -04:00
Joseph Doherty 12b2ce078b fix(r2-10): register the tracker's production reader on driver nodes + wiring guard 2026-07-13 10:42:18 -04:00
Joseph Doherty 37f180c5a3 fix(r2-10): periodic DPS publisher for tracker snapshots 2026-07-13 10:40:54 -04:00
Joseph Doherty 901d4e6ed3 Merge R2-03 Virtual-Tag failure quality (arch-review round 2) [PR #430]
v2-ci / build (push) Successful in 4m2s
v2-ci / unit-tests (push) Failing after 8m20s
Findings 02/S13 (VT failure/timeout degrades node to Bad quality, no stale-Good
forever), 02/S12 (evaluator retry-once on disposed-race), 02/P7 (compile-cache
clear gated on changed expression set). T11/T12 deferred. STATUS.md conflict
resolved additively. Build clean.
2026-07-13 10:20:43 -04:00
Joseph Doherty d67900e455 test(host): red repro — apply-boundary cache clear races in-flight VT evaluation into ObjectDisposedException (02/S12) 2026-07-13 10:09:37 -04:00
Joseph Doherty 05c35c93d9 test(host): pin sequential clear-then-recompile contract of RoslynVirtualTagEvaluator (02/S12 baseline) 2026-07-13 09:55:44 -04:00
Joseph Doherty 35d8abee84 fix(r2-06): RED — ServerHistorianOptionsValidator unit + wiring tests (net-new type; does not compile) (06/S-11) 2026-07-13 09:53:21 -04:00
Joseph Doherty 45455b7677 Merge branch 'fix/archreview-crit2-vt-timeout' 2026-07-09 06:07:33 -04:00
Joseph Doherty 0ed30483b4 Merge branch 'fix/archreview-crit1-split-brain-resolver' 2026-07-09 06:07:33 -04:00
Joseph Doherty 8a8b9ec564 test(archreview #10/#13): guard prod DI binds the REAL invoker factory
AddOtOpcUaDriverFactories (the driver-node bootstrap) must bind the real
DriverCapabilityInvokerFactory — not the NullDriverCapabilityInvokerFactory
pass-through — and it must mint a real CapabilityInvoker per instance. This is the
deterministic DI half of the #10 live gate: a refactor dropping back to the
pass-through would make the whole retry/breaker/bulkhead pipeline inert in production
while every pass-through-defaulting unit test stays green. 3 tests (real factory bound,
real invoker minted, shared singletons), non-fixtured. Host.IntegrationTests.
2026-07-08 23:16:23 -04:00
Joseph Doherty a25c9ed097 test(cluster): hard-kill failover integration test on the 2-node harness (arch-review 03/S1)
The load-bearing integration half of Critical 1. Adds TwoNodeClusterHarness.HardKillNodeAAsync
(canonical in-process crash sim: shut down node A's remoting transport via
IRemoteActorRefProvider.Transport.Shutdown() — associations drop, heartbeats/gossip stop, NO
graceful Cluster.Leave, so node B's failure detector marks A Unreachable) + a
WaitForNodeBSoleDriverLeaderAsync helper (waits for a STABLE takeover: B sole Up member + driver
role-leader). HardKillFailoverTests crashes the oldest node and asserts the survivor takes over —
the exact scenario the graceful StopNodeBAsync path (Cluster.Leave, no downing decision) cannot
exercise.

Live-verified in-process (~35s: acceptable-heartbeat-pause 10s + stable-after 15s + convergence).
Negative control confirmed the test BITES: forcing explicit NoDowning
(akka.cluster.downing-provider-class = "") makes it time out with the node stuck Up-but-Unreachable.

FINDING surfaced by the negative control: removing Critical 1's typed
ClusterOptions.SplitBrainResolver did NOT break failover, because Akka.Cluster.Hosting's
WithClustering applies SplitBrainResolverOption.Default when the option is null — enabling the SBR
downing provider that reads the akka.conf keep-oldest block (present on master BEFORE Critical 1).
So the cluster was NOT running NoDowning before Critical 1; the typed option reinforces the akka.conf
strategy rather than being the sole activator, and Critical 1's 'HOCON inert / NoDowning' premise is
inaccurate. The test guards the failover OUTCOME regardless of activation path. Comments corrected to
reflect this; flagged for review.
2026-07-08 18:24:49 -04:00
Joseph Doherty 7fd44f0fa7 fix(vtags): real script timeout + ALC-safe compile cache (arch-review 02/U2,U3)
U2 (Critical): RoslynVirtualTagEvaluator ran scripts with a CancellationToken
that a synchronous Roslyn script can never observe, so one CPU-bound/infinite-
loop virtual-tag script hung the owning VirtualTagActor forever (no timeout, no
recovery). Route evaluation through the existing TimedScriptEvaluator (Task.Run
+ WaitAsync) so the wall-clock budget is real; a runaway script now returns Bad
within the timeout instead of wedging the actor.

U3 (High): the live path cached evaluators in a raw ConcurrentDictionary never
cleared on republish, leaking a collectible AssemblyLoadContext per edited
script forever. Swap in the purpose-built CompiledScriptCache (Lazy single-
compile, dispose-on-Clear) and add an apply-boundary clear: a new narrow
IScriptCacheOwner seam the VirtualTagHostActor calls on each ApplyVirtualTags
generation (mirrors ScriptedAlarmEngine's per-generation clear).

Tests: infinite-loop-fails-within-timeout (bounded by WaitAsync so a regression
fails instead of hanging), happy-path-after-wrapping, ClearCompiledScripts-
empties-cache, and a host-actor wiring guard asserting each apply generation
calls ClearCompiledScripts (theme #1 production-caller assertion). Host.Integration
VT suite 18/18, Runtime VT suite 12/12.
2026-07-08 16:21:47 -04:00
Joseph Doherty 0b4b2e4cfd refactor(historian-gateway): retire Wonderware historian projects (gateway is sole backend)
The HistorianGateway driver is now the sole historian read/write+alarm backend, so the
Wonderware sidecar projects are dead code. Removes the 5 Wonderware projects (driver,
.Client, .Client.Contracts, + their 2 test projects) from the solution and tree, and fully
retires the vestigial 'Historian.Wonderware' driver type (UI/probe-only; it had no driver
factory): the Host probe registration, the AdminUI driver-config surface (driver page,
tag-config editor/model/validator entry, address picker/builder, driver-type catalog +
dropdown + edit-router entries), and their tests. Prunes the now-unused Wonderware
connection fields (Host/Port/UseTls/ServerCertThumbprint/SharedSecret) from
AlarmHistorianOptions (keeping Enabled + the SQLite store-and-forward knobs) and refreshes
the stale XML docs that named Wonderware as the production backend.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 19:25:21 -04:00
Joseph Doherty 4b14feb373 fix(drivers): serialize driver-config enums as strings in AdminUI pages + probes
AdminUI driver-instance pages serialized enum config fields (S7 CpuType,
Modbus DataType/Region, AbCip PlcFamily, ...) as JSON *numbers* because each
page's _jsonOpts lacked a JsonStringEnumConverter. The driver factories,
however, deserialize into string-typed DTOs (+ lenient ParseEnum) and throw
when binding a JSON number to a string? — so an AdminUI-authored config
containing any enum field produced a blob the driver could not parse,
faulting the driver on deploy. Proven end-to-end for S7 and Modbus; latent
for AbCip/AbLegacy/TwinCAT/FOCAS/Galaxy/Historian. Only OpcUaClient was safe
(its factory + probe already carried the converter).

Add JsonStringEnumConverter to all 9 driver-instance pages' _jsonOpts and the
8 missing driver probes' _opts (factories unchanged — already string-via-
ParseEnum; strictly more permissive, also lets pages load hand-seeded
string-enum configs back into the form).

Also fix DriverProbeHandshakeE2eTests.AbCip_Green_AgainstSim to probe a real
sim tag (TestDINT) — the no-tags @raw_cpu_type fallback is rejected by the
ab_server sim with ErrorBadParam (a real ControlLogix returns ErrorNotFound,
which the probe treats as reachable; hardware-gated follow-up).

Tests: reflection guard over all driver pages' _jsonOpts (AdminUI.Tests);
factory round-trip + numeric-form-throws guards for S7 and Modbus.

Found by running the never-before-run FB-9/FB-10 live verifies.
2026-06-19 04:52:47 -04:00
Joseph Doherty 40e8a23e7c refactor(opcuaserver): rename Phase7* address-space pipeline to AddressSpace*
v2-ci / build (push) Failing after 37s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
The OPC UA address-space build pipeline was named after a v2-roadmap
milestone number rather than its domain. Rename the family to describe
what it does (build/diff/apply the OPC UA address space):

  Phase7Composer          -> AddressSpaceComposer
  Phase7CompositionResult -> AddressSpaceComposition
  Phase7Planner           -> AddressSpacePlanner
  Phase7Plan              -> AddressSpacePlan
  Phase7Applier           -> AddressSpaceApplier
  Phase7ApplyOutcome      -> AddressSpaceApplyOutcome

The 9 Phase7*Tests suites follow suit; Phase7ScriptingEntitiesTests ->
ScriptingEntitiesTests (it tests the scripting migration, not the
pipeline). Log-message prefixes move to the new class names.

Pure mechanical rename, no behavioral change. EF migration classes/IDs
(AddPhase7ScriptingTables, ExtendComputeGenerationDiffWithPhase7) are
immutable and left untouched, as are historical design docs.

Build clean; OpcUaServer 261/261, Runtime 272/272, ScriptingEntities
12/12 green.
2026-06-18 19:16:28 -04:00
Joseph Doherty 0414a04892 test(adminui): review fixes — locked-snapshot assert, stable fake health, InitializeCount property 2026-06-18 09:18:37 -04:00
Joseph Doherty f87ad5ae8f test(adminui): E2E deployed-driver Healthy->Reconnecting->Healthy transition on Reconnect 2026-06-18 09:12:15 -04:00
Joseph Doherty ffb725e4c1 test(harness): production-fidelity DI (AddOtOpcUaRuntime) + opt-in fake driver factory 2026-06-18 08:55:35 -04:00
Joseph Doherty a40c77de87 fix(adminui): flip remaining ModbusTcp test seeds + doc comments to Modbus (review) 2026-06-16 19:51:56 -04:00
Joseph Doherty 1164d423b6 fix(probe): Galaxy gRPC ping — drop invalid Retry, treat MxGatewayAuth exceptions as reachable (live /run)
v2-ci / build (push) Failing after 44s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Two bugs caught by live verification against the mxaccessgw at 10.100.0.48:5120:
- MaxAttempts=1 produced an invalid Polly RetryStrategyOptions -> the probe failed
  on every real gateway. Removed the Retry override (matches GalaxyDriver); fail-fast
  is already guaranteed by the TCP preflight + the per-call deadline.
- A rejected key surfaces as a typed MxGatewayAuthenticationException, not a raw
  RpcException, so 'auth-rejection = reachable' was bypassed. Catch the typed auth/
  authorization exceptions -> Ok=true.
Adds DriverProbeHandshakeE2eTests: direct-probe, skip-gated cross-protocol green/red
discrimination (Modbus, OpcUaClient, Galaxy + a local real OPC UA server).
2026-06-16 07:32:59 -04:00
Joseph Doherty 57355405a6 chore(security): drop dead audit suppressions; patch OpenTelemetry + Tmds.DBus CVEs
All five suppressed advisories are now resolved at baseline/resolved versions,
so every NuGetAuditSuppress is removed repo-wide:
- System.Security.Cryptography.Xml (GHSA-37gx-xxp4-5rgx / GHSA-w3x6-4m5h-cxqf)
  -> fixed by the .NET 10 baseline (10.0.6)
- OPCFoundation Opc.Ua.Core (GHSA-h958-fxgg-g7w3) -> fixed at resolved 1.5.378.106

Two were still live and are now patched via direct security pins:
- OpenTelemetry.Api 1.9.0 -> 1.15.3 (GHSA-g94r-2vxg-569j) pinned in Cluster;
  Runtime/ControlPlane/AdminUI + tests inherit via project reference
- Tmds.DBus.Protocol 0.20.0 -> 0.21.3 (GHSA-xrw6-gwf8-vvr9) pinned in Client.UI

Also correct the Historian sidecar runtime comments (x86 -> x64, matching the
csproj PlatformTarget). Solution audit: 0 vulnerable packages; full build clean.
2026-06-12 09:03:42 -04:00
Joseph Doherty fc0d43a3dc refactor(scripted-alarms): retire orphaned ScriptedAlarmActor + F9b evaluator (T11) 2026-06-10 15:22:26 -04:00
Joseph Doherty b5748288df test(scriptlog): prove bridge→broadcaster delivery off the script-logs DPS topic
Composes the one Layer-0 hop existing tests left uncovered together:
ScriptLogSignalRBridge subscribing to the script-logs DPS topic and
fanning a ScriptLogEntry out to the IInProcessBroadcaster<ScriptLogEntry>
singleton resolved from the SAME DI container the /script-log page injects.
Mirrors DriverStatusHubE2eTests. Confirms the server-side topic→page chain
delivers end-to-end (only the live Blazor circuit remains manual).
2026-06-10 13:53:34 -04:00
Joseph Doherty bd2dd05a0c feat(scripting): evaluators log through root script logger → script-log page (F8) 2026-06-10 12:03:51 -04:00
Joseph Doherty b73ce75402 harden(vtag): exclude backslash from passthrough capture + parity tests (A review) 2026-06-07 15:31:54 -04:00
Joseph Doherty 08d7477860 feat(vtag): passthrough fast-path skips Roslyn for mirror scripts (A) 2026-06-07 15:26:20 -04:00
Joseph Doherty e4a3f07c99 test(integration): poll per-node diagnostics to remove timing fragility (review) 2026-06-07 03:37:04 -04:00
Joseph Doherty ec9599e234 test(integration): multi-cluster deploy scopes drivers per node 2026-06-07 03:31:08 -04:00
Joseph Doherty ad7f9e731f feat(admin): headless POST /api/deployments REST endpoint (API-key gated)
v2-ci / build (push) Failing after 50s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
A thin gateway over the admin-operations cluster singleton so CI/scripts can trigger a
deployment without the Blazor button. Forwards to the same IAdminOperationsClient.
StartDeploymentAsync; mounted on admin-role nodes. Auth is a fixed-time X-Api-Key check
against Security:DeployApiKey (orthogonal to the cookie-only web auth); AllowAnonymous so the
auth fallback doesn't 401 it, self-disabling (503) until the key is set. Outcome->status:
202/200/409/422. Unit tests for the key check + outcome mapping; HTTP E2E (real auth + real
deploy via the 2-node harness). Documented in docs/security.md.
2026-06-06 15:54:51 -04:00
Joseph Doherty a5d857d5b2 test(opcua): E2E deploy of an Equipment namespace through the real ConfigComposer
v2-ci / build (push) Failing after 46s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Seed a 1-area/1-line/1-equipment/1-tag Equipment namespace, StartDeployment via the
in-process 2-node harness, and assert the persisted artifact decodes (ParseComposition)
to the equipment signal (FullName from TagConfig) + friendly UNS folder names. Covers the
ConfigComposer -> ArtifactBlob -> ParseComposition.EquipmentTags seam the unit tests only
approximated with hand-built JSON. (OPC UA browse is covered against a real SDK node manager
in Phase7ApplierHierarchyTests; the cluster harness binds the no-op sink.)
2026-06-06 15:22:25 -04:00
Joseph Doherty c1619d95f5 feat(auth)!: OtOpcUa canonical control-plane roles + config-DB migration (Task 1.7)
Standardize the control-plane admin role VALUES on the canonical six
(ZB.MOM.WW.Auth CanonicalRole). OtOpcUa uses four:
  ConfigViewer   -> Viewer
  ConfigEditor   -> Designer
  FleetAdmin     -> Administrator
  DriverOperator -> Operator   (appsettings-only string role)

This is a rename, not a permission change: enforcement semantics are
preserved (whoever could deploy/administer/operate before still can).

- AdminRole enum members renamed (persisted as string names via
  HasConversion<string>); RoleGrants.razor dropdown default updated.
- EF DATA migration CanonicalizeAdminRoles rewrites existing
  LdapGroupRoleMapping.Role rows old->new (Up) and back (Down); schema /
  model snapshot byte-identical (no pending model changes).
- Enforcement role STRINGS canonicalized:
  * Security policies keep their NAMES ("DriverOperator"/"FleetAdmin")
    but require canonical roles: RequireRole("Operator","Administrator")
    and RequireRole("Administrator").
  * Deployments.razor [Authorize(Roles="Administrator,Designer")].
  * DevStub now grants "Administrator"; LdapOptions/doc-comment examples
    canonicalized.
- Data-plane authorization (NodePermissions/NodeAcl/IPermissionEvaluator/
  TriePermissionEvaluator/UserAuthorizationState) UNTOUCHED.
- New CanonicalAdminRolesTests pins canonical claim values end-to-end and
  the real registered policies; existing role-string tests updated.
2026-06-02 07:30:00 -04:00