Commit Graph

32 Commits

Author SHA1 Message Date
Joseph Doherty f9f1b8fcee fix(localdb): phase-2 live gate — 4 production defects found and fixed
Gate record: docs/plans/2026-07-20-localdb-phase2-live-gate.md.
Checks 1, 2, 5, 6 pass. Checks 3 and 4 are NOT satisfied — the defect they
were meant to confirm turned out to be the opposite of what the plan assumed.

Three defects crash-looped every driver node before check 1 could even run:

1. An empty ServerHistorian:ApiKey kills the host. ServerHistorianOptions-
   Validator exists to turn exactly that class of failure into a named
   OptionsValidationException, but its documented fail tier explicitly excluded
   ApiKey on the reasoning that a keyless client "degrades — the gateway rejects
   calls". It does not: the client validates its own options at construction, so
   the process dies during Akka startup and never makes a call.

2. UseTls disagreeing with the endpoint scheme kills the host too, in both
   directions (both messages confirmed in the shipped client assembly). Moving an
   endpoint from https to http without clearing UseTls is an ordinary migration
   slip.

3. Plaintext h2c was UNREACHABLE. HistorianGatewayClientAdapter forwarded the
   TLS-only options unconditionally, and AllowUntrustedServerCertificate defaults
   to false, so it always sent RequireCertificateValidation=true — which the
   client rejects outright when UseTls=false. Every http:// deployment crashed,
   though the scheme is documented as the supported way to select h2c, and the
   only workaround was to assert a certificate posture for a connection that has
   no certificate.

The fourth was the blocker, and it is Phase 2's own:

4. The drain gate deferred to a Primary that cannot deliver. Redundancy roles are
   elected CLUSTER-WIDE; the alarm queue is PAIR-LOCAL. On the rig the elected
   driver Primary is central-1 — it carries the driver Akka role, replicates
   nobody's LocalDb and does not even run the alarm historian — so every driver
   node logged "Historian drain suspended", including the two site-b nodes that
   have no peer at all. Nothing drained anywhere, where before Phase 2 it drained
   fine. The cost is not a duplicate; it is the buffer growing to the capacity
   wall and evicting the audit trail it exists to protect.

   Fixed in three layers: a separate ShouldDrainAlarmHistory policy (unknown role
   drains; the two gates now deliberately disagree, and a test pins that); peer-
   host matching in DriverHostActor so a node stands down only for a Primary
   holding its rows; and AddAlarmHistorian short-circuiting the gate when
   replication is unconfigured — testing BOTH Replication:PeerAddress and
   SyncListenPort, since only the dialing half sets the former while both halves
   share the queue.

   Every one of these follows from the asymmetry: a false allow costs a duplicate
   row, which at-least-once delivery already accepts and payload-hash ids
   collapse; a false deny loses data silently.

A third vacuous test, caught by the same delete-the-guard discipline: the
role-view tests stayed green with the guard removed, because AwaitAssert polls
until an assertion passes and the assertion was "reads open" — which is the
SEEDED value, satisfied at the first poll before the actor processed anything.
They now assert the sequence of published values through a recording view; the
control then goes red for exactly the cases that matter.

Migration evidence: 11 legacy rows across two deliberately overlapping files
converged to exactly 9 identical rows on both nodes, proving D-6's payload-hash
identity on real nodes rather than in a fixture.

Open design fork, recorded in the gate doc rather than decided here: a pair
cannot currently identify its own Primary, so both halves drain. Safe in every
topology — nothing loses data — but the gate's de-duplication benefit is
unrealised until roles are scoped per pair.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 06:13:01 -04:00
Joseph Doherty 2e4ccf7fe9 chore(localdb): phase-2 DoD sweep
Build: 0 errors solution-wide, and 0 warnings from every project this branch
touches. The ~816 solution-wide warnings are pre-existing xUnit1051 /
OTOPCUA0001 / CS86xx in untouched driver + client test projects.

Tests: full solution run compared against a full run on a detached worktree at
the pre-branch baseline 2e46d054. The two failure SETS are identical -- all 13
tests, same names, zero new failures. Net +26 tests: +3 Core.AlarmHistorian
(drain gate), +6 Runtime (role view), +17 Host.IntegrationTests (migrator +
convergence). Set comparison rather than counts, because the suite carries
standing environment- and load-dependent failures a count would hide.

The greps found real drift Task 6 missed -- eight live sites still naming the
deleted SqliteStoreAndForwardSink, including the AdminUI /alarms/historian
panel text, which is user-visible, and a <see cref> in HistorianAdapterActor
that resolved to nothing without warning. All repointed at
LocalDbStoreAndForwardSink; CLAUDE.md's alarm-history paragraph now also
records the LocalDb buffer and the primary-gated drain, and drops DatabasePath
from the knob list. docs/AlarmTracking.md still promised an
AlarmHistorianOptions.Validate() startup warning for a relative DatabasePath
and an empty SharedSecret; both branches are gone, so it now says so.

Code references to AlarmHistorian:DatabasePath reduce to exactly two intentional
ones: AlarmSfLegacyMigrator.LegacyPathKey and its test. No `new SqliteConnection`
remains anywhere in Core.AlarmHistorian.

Recon doc gains the durable verification record: guard-deletion evidence for
both vacuous passes, the two exact-set replicated-table pins (both assert set
equality, so an added or a dropped registration fails), and the baseline test
comparison with a per-failure account of why each of the 13 is not this
branch's.

Stops here per the plan. Task 8's live gate needs explicit go-ahead; nothing on
this branch is to be merged.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 05:03:57 -04:00
Joseph Doherty 71379816e7 feat(localdb): alarm S&F on LocalDb with primary-gated drain (cutover)
Moves the alarm store-and-forward buffer out of its own alarm-historian.db
and into the node's consolidated LocalDb, where it replicates to the
redundant pair peer. A node that dies holding undelivered alarm history no
longer takes it to the grave.

Tasks 2 and 3 land together, as the plan anticipated. They are not separable:
the gate is a constructor argument of the rewritten sink, and a commit that
replicated the queue without gating the drain would be a commit in which both
nodes of a pair deliver every alarm event, continuously.

That drain gate is the load-bearing part of the change, and the recon
explains why it is new work rather than a refinement. Exactly-once delivery
across a pair is enforced today on the ENQUEUE side, by
HistorianAdapterActor: only the Primary enqueues, so the Secondary's queue is
empty and its ungated drain has nothing to send. Replicating the table
destroys that invariant.

The gate is a Func<bool> the caller supplies, because the drain runs on a
timer the sink owns rather than on a mailbox, and Core.AlarmHistorian cannot
reference PrimaryGatePolicy in Runtime. Runtime supplies it via a new
IRedundancyRoleView singleton that DriverHostActor publishes its Primary-gate
verdict to -- the same verdict the inbound-write and native-ack gates use, so
there is no second notion of am-I-the-Primary to drift.

Two failure modes are deliberately closed:

- The view is seeded OPEN, matching the policy's own answer for an unknown
  role with no driver peer. A deployment that runs no redundancy never
  publishes to it, and defaulting closed would silently stop its alarm
  history forever.

- A gate that throws is read as not-now, never as permission, and a closed
  gate reports the new HistorianDrainState.NotPrimary rather than Idle. A
  Secondary's rising queue is supposed to look different from a stalled
  drain, and if BOTH nodes report NotPrimary the pair is misconfigured and
  says so instead of quietly filling toward the capacity ceiling.

Row ids are a hash of the payload rather than fresh GUIDs. Both adapters
accept the same fanned transition in the window before the first redundancy
snapshot arrives, and under last-writer-wins an equal key converges those two
accepts into one row instead of duplicating them.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
2026-07-21 04:13:00 -04:00
Joseph Doherty 1becf59168 feat(localdb): DriverHostActor stores applied artifacts in the pair-local cache
New Props/ctor parameter appended LAST at all three positions - the forwarding
list inside Props.Create is positional and compiles into an expression tree, so a
mis-ordered argument is usually type-compatible and binds the wrong dependency at
runtime. All 26 existing test construction sites use named arguments and needed no
change.

ReconcileDrivers now returns the blob it loaded. It catches its own DB failures and
returns without rethrowing, so ApplyAndAck can reach its success path having spawned
zero drivers; caching on 'the apply succeeded' would persist an empty artifact as
last-known-good and boot the node into an empty address space during the next
outage. Verified: weakening the guard to null-only turns EmptyArtifact_IsNotCached
red.

The store runs in its own try/catch because an Applied ACK has already been sent by
then - an escaping exception would land in ApplyAndAck's catch and emit a second,
contradictory Failed ACK for a deployment the fleet believes is live.
2026-07-20 11:06:57 -04:00
Joseph Doherty 56ddf45e1c chore(core): flatten Historian sub-namespace to Core.Abstractions root
v2-ci / build (pull_request) Successful in 4m32s
v2-ci / unit-tests (pull_request) Failing after 10m2s
Four continuous-historization types (IHistorizationOutbox, HistorizationOutboxEntry,
IHistorianValueWriter/HistorizationValue, HistorizationCommitMode) drifted into a
ZB.MOM.WW.OtOpcUa.Core.Abstractions.Historian sub-namespace while the other six files
in the same folder correctly use the root namespace. This violated decision #59's
flat-public-surface rule and left InterfaceIndependenceTests.AllPublicTypes_LiveInRootNamespace
red on master (pre-existing, predates round-2 remediation).

Move all four to the root namespace and drop/retarget the now-redundant
'using ...Core.Abstractions.Historian;' in the ~11 consumers. No behavioral change.

Verified: Core.Abstractions.Tests 129/129 (was 128/129), Runtime.Tests Historian+DI 72/72,
Gateway driver unit 102/102, full solution build 0 errors.
2026-07-13 18:29:01 -04:00
Joseph Doherty bacea1a44f fix(archreview #10): wire CapabilityInvoker into prod dispatch via a Polly-free seam
Remediates RESILIENCE-DISPATCH-GAP (surfaced by the 07/C-1 analyzer): the Phase 6.1
CapabilityInvoker resilience pipeline (retry / breaker / bulkhead / telemetry) was
constructed only in tests — the production dispatch layer called driver-capability
methods directly, bypassing it entirely.

The filed plan (thread CapabilityInvoker directly into DriverInstanceActor) is
INFEASIBLE: Runtime is deliberately Polly-free (references Core.Abstractions, not
Core — the same boundary IDriverFactory documents). So this introduces a seam,
mirroring IDriverFactory exactly:

- Core.Abstractions: IDriverCapabilityInvoker (+ NullDriverCapabilityInvoker
  pass-through) and IDriverCapabilityInvokerFactory (+ null factory).
- Core: CapabilityInvoker now implements the interface; new
  DriverCapabilityInvokerFactory builds a per-instance invoker over the
  process-singleton pipeline builder + status tracker + tier resolver.
- Runtime: DriverInstanceActor takes an IDriverCapabilityInvoker (default =
  pass-through) and routes all 6 dispatch sites (write, alarm-ack, subscribe,
  unsubscribe, alarm-subscribe, discover) through it; per-host key resolved via
  IPerCallHostResolver for single-ref calls, driver-instance key for bulk calls.
  DriverHostActor builds + injects the real invoker per spawned driver.
- Host: DriverFactoryBootstrap registers the tracker, pipeline builder, and
  concrete factory (needs DriverFactoryRegistry.GetTier). Runtime SCE resolves the
  factory from DI like IDriverFactory; pass-through on nodes without it bound.
- Analyzer: IDriverCapabilityInvoker.ExecuteAsync/ExecuteWriteAsync are now
  recognized wrapper homes (interface-typed invoker calls must suppress OTOPCUA0001
  too). All 6 RESILIENCE-DISPATCH-GAP pragmas removed — the analyzer (an error in
  Runtime) is now the standing regression guard.

Also adds retry / breaker-open / breaker-close LOGGING to the pipeline builder —
the operator-facing observability surface (the Admin /hosts reader, Phase 6.1
Stream E.2/E.3, was never built, so the pipeline otherwise runs silently).

Scope notes:
- Tier-DEFAULT policy only: DriverInstanceSpec (the deploy artifact) does not carry
  the per-instance ResilienceConfig JSON, so overrides aren't applied yet (tracked
  follow-up: plumb ResilienceConfig through the composer/artifact).
- GenericDriverNodeManager's call is NOT wired: that class is test-only scaffolding
  (zero production references — the Server has its own address-space path); its
  pragma is re-annotated accordingly, not left as a "tracked gap".

Verification (unit + analyzer; live behavioral gate still pending — see FOLLOWUP-10):
- Negative control: unwrapping any site fails the Runtime build with OTOPCUA0001.
- New Runtime guard (recording invoker): write routes via ExecuteWriteAsync with the
  IPerCallHostResolver host; subscribe via ExecuteAsync — proves runtime routing.
- New analyzer test: the interface is a valid wrapper home.
- New pipeline-builder test: retry events are logged.
- Full solution builds clean (0 errors); Runtime.Tests 357, Core.Tests 238,
  Analyzers.Tests 32 all green; pass-through default keeps existing dispatch tests
  byte-for-byte unchanged.
2026-07-08 21:33:16 -04:00
Joseph Doherty 9cad9ed0fc docs(xmldoc): fill missing XML docs + strip tracking-ID comments across src
v2-ci / build (push) Failing after 41s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Adds <summary>/<param>/<returns>/<inheritdoc> where missing and removes
project bookkeeping IDs (task/tracking refs) from shipped code comments,
so the docs read cleanly and CommentChecker is quiet except for known
false positives (PLC/protocol terms, event/IEqualityComparer inheritdoc).
Doc/comment-only; no logic changed; solution builds clean.
2026-07-07 12:38:39 -04:00
Joseph Doherty 257214f775 fix(historian-gateway): wire dormant GatewayTagProvisioner + provisioning observability/docs
PR #423 shipped GatewayTagProvisioner + unit tests but never registered it in
DI nor passed it into the AddressSpaceApplier, so deploying historized tags used
the no-op NullHistorianProvisioning and never called the gateway's EnsureTags
(confirmed live on wonder-app-vd03: zero EnsureTags calls on a historized deploy).
Addresses HISTORIAN-GATEWAY-INTEGRATION-ISSUES.md.

Issue 1 (wire provisioner):
- Runtime: AddHistorianProvisioning extension (gated on ServerHistorian:Enabled,
  mirrors AddServerHistorian) + NullHistorianProvisioning TryAdd default in
  AddOtOpcUaRuntime; WithOtOpcUaRuntimeActors resolves IHistorianProvisioning and
  passes it into the applier.
- Gateway driver: GatewayHistorian.CreateProvisioner factory (mirrors CreateDataSource).
- Host: Program.cs calls AddHistorianProvisioning after AddServerHistorian.
- Tests: AddHistorianProvisioningTests (config-gated registration + the
  register->resolve->applier->EnsureTags chain).

Issue 2 (observability): AddressSpaceApplier logs the provisioning tally on every
successful dispatch (was gated behind Failed/Skipped > 0), including dispatched=N
so a dispatched=N/requested=0 line flags the dormant no-op. +2 tests.

Issue 3 (30s HistoryRead on unprovisioned tags): root coupling fixed by Issue 1;
documented the CallTimeout knob + coupling. Default left at 30s pending the
multi-data-point investigation the issue requests (lowering risks truncating
legitimate large reads).

Issue 4 (docs): docs/Historian.md gains a "Tag auto-provisioning (EnsureTags)"
section and CLAUDE.md a wiring/gating note (both stress ServerHistorian:Enabled).
Sibling scadaproj/CLAUDE.md carries no false claim -> unchanged.

Pre-existing Serilog observation: anchor CWD to AppContext.BaseDirectory before
AddZbSerilog so the relative file sink stops landing in C:\Windows\System32 under
the Windows-service CWD.

Builds 0-error; Runtime.Tests 355, OpcUaServer.Tests 329, Gateway.Tests 99 (+4
live-skipped) all green.
2026-06-27 23:24:29 -04:00
Joseph Doherty 2982cc4bb5 feat(historian-gateway): feed historized refs to the recorder on deploy (close continuous-historization ref-feed gap)
v2-ci / build (pull_request) Failing after 39s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (pull_request) Has been skipped
The ContinuousHistorizationRecorder was spawned with an EMPTY historized-ref
set, so it registered interest in nothing and historized nothing. This feeds it
the currently-historized tag refs on every address-space deploy/redeploy so its
DependencyMuxActor interest converges to exactly the historized set (the same
refs the EnsureTags provisioning hook resolves: override-or-FullName).

Design — delta convergence (the plan is a pure DIFF):
- New seam IHistorizedTagSubscriptionSink (Core.Abstractions/Historian) with a
  Null no-op singleton, mirroring how IHistorianProvisioning decouples the T15
  hook. AddressSpaceApplier gains a DEFAULTED ctor param (Null sink) so all ~80
  existing call sites + the production site compile unchanged.
- Apply() only ever sees a plan diff (an incremental/surgical apply carries a
  delta, not the full set), so the applier feeds an add/remove DELTA computed
  from AddedEquipmentTags / RemovedEquipmentTags / ChangedEquipmentTags. The
  recorder keeps the full set and re-registers it. The feed is a single
  non-blocking Tell behind the sink, wrapped in try/catch so a faulting feed
  never blocks or breaks a deploy (same discipline as the provisioning hook).
- Recorder.UpdateHistorizedRefs(added, removed) converges the tracked set, then
  — only when it actually changed — sends ONE RegisterInterest with the full set
  (the mux's RegisterInterest is a full-REPLACE) or one UnregisterInterest when
  it drains to empty (the mux has no per-ref unregister). An unchanged delta is
  a no-op (no mux churn).
- DI: the recorder is now spawned BEFORE the applier so the adapter
  (ActorHistorizedTagSubscriptionSink) can wrap its IActorRef; the Null sink is
  used when continuous historization is off/unwired.

Tests: recorder convergence (add-from-empty, add+remove converge, idempotent,
drain-to-empty unregisters); applier feeds resolved added refs, removed+renamed
deltas, and survives a throwing sink. Build clean (0 warnings on touched
projects); Runtime/OpcUaServer/Gateway/AdminUI suites green.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 23:21:18 -04:00
Joseph Doherty 0b4b2e4cfd refactor(historian-gateway): retire Wonderware historian projects (gateway is sole backend)
The HistorianGateway driver is now the sole historian read/write+alarm backend, so the
Wonderware sidecar projects are dead code. Removes the 5 Wonderware projects (driver,
.Client, .Client.Contracts, + their 2 test projects) from the solution and tree, and fully
retires the vestigial 'Historian.Wonderware' driver type (UI/probe-only; it had no driver
factory): the Host probe registration, the AdminUI driver-config surface (driver page,
tag-config editor/model/validator entry, address picker/builder, driver-type catalog +
dropdown + edit-router entries), and their tests. Prunes the now-unused Wonderware
connection fields (Host/Port/UseTls/ServerCertThumbprint/SharedSecret) from
AlarmHistorianOptions (keeping Enabled + the SQLite store-and-forward knobs) and refreshes
the stale XML docs that named Wonderware as the production backend.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 19:25:21 -04:00
Joseph Doherty 2a5c717755 feat(historian-gateway): wire ContinuousHistorizationRecorder into DI + hosted lifecycle + meters
Bind ContinuousHistorizationOptions (Enabled/OutboxPath/CommitMode/
CommitIntervalMs/DrainBatchSize/DrainIntervalSeconds/Capacity/backoff) with a
warn-only Validate(); gated on Enabled AND the ServerHistorian gateway being
configured, the Host registers the durable FasterLogHistorizationOutbox (container
-disposed) + a gateway-backed GatewayHistorianValueWriter, and binds outbox
depth/dropped observable gauges on the central scraped meter. WithOtOpcUaRuntimeActors
spawns the recorder (over the same dependency-mux ref) when the options + writer +
outbox resolve, registering ContinuousHistorizationRecorderKey. Spawned with an EMPTY
historized-ref set: the deployed address space builds later, so ref population is a
documented follow-on (a later SetHistorizedRefs feed) — T18 wires the actor + outbox
+ writer + meters; the ref feed is the known remaining gap.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 18:47:20 -04:00
Joseph Doherty 1a6eb7efe6 test(historian-gateway): cover MaxTieClusterOverfetch warning + refresh AddServerHistorian doc
Addresses Task 9 review: add the enabled+nonpositive MaxTieClusterOverfetch warning
test; update the AddServerHistorian XML doc to describe the gateway-backed data source
(the alarm-path Wonderware doc stays until T13).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 17:09:45 -04:00
Joseph Doherty 40e8a23e7c refactor(opcuaserver): rename Phase7* address-space pipeline to AddressSpace*
v2-ci / build (push) Failing after 37s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
The OPC UA address-space build pipeline was named after a v2-roadmap
milestone number rather than its domain. Rename the family to describe
what it does (build/diff/apply the OPC UA address space):

  Phase7Composer          -> AddressSpaceComposer
  Phase7CompositionResult -> AddressSpaceComposition
  Phase7Planner           -> AddressSpacePlanner
  Phase7Plan              -> AddressSpacePlan
  Phase7Applier           -> AddressSpaceApplier
  Phase7ApplyOutcome      -> AddressSpaceApplyOutcome

The 9 Phase7*Tests suites follow suit; Phase7ScriptingEntitiesTests ->
ScriptingEntitiesTests (it tests the scripting migration, not the
pipeline). Log-message prefixes move to the new class names.

Pure mechanical rename, no behavioral change. EF migration classes/IDs
(AddPhase7ScriptingTables, ExtendComputeGenerationDiffWithPhase7) are
immutable and left untouched, as are historical design docs.

Build clean; OpcUaServer 261/261, Runtime 272/272, ScriptingEntities
12/12 green.
2026-06-18 19:16:28 -04:00
Joseph Doherty fcb3801415 fix(historian): dead-letter poison events after maxAttempts (finding 002) 2026-06-16 05:25:43 -04:00
Joseph Doherty 4c78dcd358 feat(redundancy): wire dbHealth into OpcUaPublishActor + spawn PeerProbeSupervisor per node 2026-06-15 13:33:34 -04:00
Joseph Doherty 4501f12669 feat(vtags): wire IHistoryWriter through DriverHostActor (Null default; durable sink infra-gated) (H5d, stillpending §1) 2026-06-15 10:38:49 -04:00
Joseph Doherty 83e1318425 refactor(historian): align ServerHistorianOptions with AlarmHistorian (Port default, Validate list, log context) 2026-06-14 20:23:22 -04:00
Joseph Doherty a6f1f4ef15 feat(historian): AddServerHistorian DI + Host wiring of IHistorianDataSource 2026-06-14 20:17:10 -04:00
Joseph Doherty 5ea6e9d7d9 fix(historian): validate non-positive drain/capacity/retention knobs (review) + log prefix 2026-06-11 13:09:13 -04:00
Joseph Doherty f215982b93 feat(historian): drain/capacity/retention config knobs + startup config-warning validation 2026-06-11 13:04:16 -04:00
Joseph Doherty 943c621371 feat(historian): config-gated SqliteStoreAndForward→Wonderware sink (AddAlarmHistorian) 2026-06-11 11:30:31 -04:00
Joseph Doherty 0742946108 feat(redundancy): gate alarm historization on Primary (A2, defensive — actor currently unfed)
HistorianAdapterActor now subscribes to the redundancy-state DPS topic,
caches the local node's RedundancyRole, and SKIPS the durable-sink enqueue
when the local node is Secondary or Detached. Unknown/null role default-writes
so single-node deploys and the boot window never silently drop historization.
GetStatus stays ungated.

PREMISE: verified the actor is registered but FED BY NOTHING in production —
there is no AlarmHistorianEvent producer and nothing resolves its registry key
to Tell it. This is a FORWARD-LOOKING / DEFENSIVE guard, not a fix for a live
double-write: the moment a per-node feeder lands (engine -> historian, expected
as a per-node cluster broadcast like the alerts topic), only the Primary will
write to the durable sink (exactly-once across all alarm sources).

Mirrors the sibling A1 treatment of ScriptedAlarmHostActor (06c4155) and
OpcUaPublishActor's redundancy-state handler. localNode threaded through
HistorianAdapterActor.Props from ServiceCollectionExtensions (roleInfo.LocalNode).
2026-06-11 08:57:41 -04:00
Joseph Doherty 5256761368 feat(scripted-alarms): spawn + apply ScriptedAlarmHostActor in DriverHostActor (T10) 2026-06-10 15:17:29 -04:00
Joseph Doherty 2bfe18abcf chore(runtime): warn on missing VirtualTag evaluator; document Stale-recovery VirtualTag behaviour
Log a WARNING on startup when IVirtualTagEvaluator is not registered so a DI misconfig on a
driver-role node is visible in logs instead of silently evaluating all VirtualTags to NoChange.
Add a comment in PushDesiredSubscriptions noting that TryRecoverFromStale does not call this
method, so VirtualTags remain empty after a Stale recovery until the next deployment dispatch
(intentional, consistent with driver recovery).
2026-06-07 05:46:24 -04:00
Joseph Doherty 397f9b783a feat(runtime): spawn+apply VirtualTagHostActor on deploy apply and restore 2026-06-07 05:41:04 -04:00
Joseph Doherty 4203b84d51 feat(runtime): publish DriverHealthChanged via DriverInstanceActor
- IDriverHealthPublisher in Core.Abstractions + NullDriverHealthPublisher
  no-op for tests/dev-stub paths.
- AkkaDriverHealthPublisher in Runtime forwards to the cluster-wide
  `driver-health` DPS topic.
- DriverInstanceActor instrumented to publish snapshots on every
  observable state change + a periodic 30s heartbeat so the AdminUI
  snapshot store warms up for newly-joined SignalR clients.
- Sliding 5-minute Faulted-count tracked per actor via Queue<DateTime>.
- DriverHostActor.SpawnChild threads clusterId (_localNode.Value) and
  the health publisher down to every DriverInstanceActor child.
- ServiceCollectionExtensions.AddOtOpcUaRuntime registers
  AkkaDriverHealthPublisher as IDriverHealthPublisher singleton.
2026-05-28 10:22:44 -04:00
Joseph Doherty 64e3fbe035 docs: backfill XML documentation across 756 files
v2-ci / build (push) Failing after 1m43s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
Adds <summary>, <param>, <typeparam>, and <inheritdoc/> tags to public
members surfaced by commentchecker — resolves 5,847 of 5,869 issues
(99.6%) across three /fixdocs passes.
2026-05-28 08:10:17 -04:00
Joseph Doherty 50787823d3 feat(host,runtime): #108 Host DI bindings — OPC UA server + deferred sink
v2-ci / build (push) Failing after 45s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (push) Has been skipped
Wires the OPC UA SDK into the fused Host's lifecycle on driver-role
nodes + spawns OpcUaPublishActor with the proper sink/publisher/dbFactory/
applier resolution. The full read+write data path is now live in
production: Deploy → DriverHost → OpcUaPublish → SDK NodeManager →
subscribed OPC UA clients.

DeferredAddressSpaceSink (Commons.OpcUa):
  - Thread-safe wrapper IOpcUaAddressSpaceSink that delegates to an
    inner sink swapped in at runtime. Needed because Akka actors
    resolve the sink at construction time, but the production sink
    (SdkAddressSpaceSink wrapping OtOpcUaNodeManager) only exists
    after the SDK StandardServer has started.
  - Defaults to NullOpcUaAddressSpaceSink so calls before swap are
    safe; SetSink(null) reverts (for graceful shutdown).

OtOpcUaServerHostedService (Host.OpcUa):
  - IHostedService that owns the OPC UA SDK lifecycle. Reads
    OpcUaApplicationHostOptions from the 'OpcUa' config section,
    creates an OtOpcUaSdkServer, boots it through OpcUaApplicationHost,
    then swaps a real SdkAddressSpaceSink into the DeferredAddressSpaceSink
    singleton.
  - SDK boot failure is logged + non-fatal — the rest of the host
    (admin UI, driver actors) keeps running. Stop reverts to null sink.

WithOtOpcUaRuntimeActors (Runtime):
  - Now spawns OpcUaPublishActor (new actor) + threads its ActorRef
    into DriverHostActor's Props so successful applies trigger the
    address-space rebuild pipeline.
  - Phase7Applier is constructed here from the resolved sink + a
    logger; OpcUaPublishActor takes both.
  - Prepends the opcua-synchronized-dispatcher HOCON so the extension
    is self-contained — consumers (Host, tests) don't need to redeclare
    the dispatcher block.
  - New OpcUaPublishActorKey + OpcUaPublishActorName for actor-registry
    resolution.
  - AddOtOpcUaRuntime now also TryAddSingleton's NullOpcUaAddressSpaceSink
    + NullServiceLevelPublisher so admin-only nodes (or tests that
    don't bind the Deferred sink) stay safe.

Host.Program.cs (driver-role only):
  - Binds DeferredAddressSpaceSink as singleton + as IOpcUaAddressSpaceSink
  - AddHostedService<OtOpcUaServerHostedService>()

Tests: OpcUaServer 24 -> 28 (+4 DeferredAddressSpaceSink unit tests),
Runtime 69 -> 69 (existing ServiceCollectionExtensionsTests extended
to verify the new mux + publish actor registration).

All 6 v2 test suites green: 177 tests passing.

Closes #108. Engine-wiring is now production-bound end-to-end on
driver-role nodes — Deploy reaches real OPC UA Variable nodes that
subscribed clients see.
2026-05-26 10:02:15 -04:00
Joseph Doherty 7fa863f6da feat(runtime): #113 DependencyMuxActor — drivers → virtual-tag fan-out
v2-ci / build (push) Failing after 36s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (push) Has been skipped
End-to-end data path is now wired on the read side: driver subscriptions
fire AttributeValuePublished → DriverHostActor → DependencyMuxActor →
DependencyValueChanged to every interested VirtualTagActor. Previously
the publish hit a dead-letter at the host.

DependencyMuxActor:
  - Per-node fan-out router. Maintains tagRef → Set<IActorRef> with a
    reverse subscriber → refs index so unregister/replace are O(refs).
  - Watches subscribers; Terminated triggers automatic unregister so
    dead virtual-tag actors stop receiving publishes.
  - Re-register replaces the prior interest set — no stale-ref leaks
    on actor restart.
  - Drops publishes for refs with no interested subscribers.

VirtualTagActor:
  - New Props params: dependencyRefs + mux ActorRef.
  - PreStart sends RegisterInterest to the mux; PostStop sends
    UnregisterInterest. Default both null so older callers stay quiet.

DriverHostActor:
  - New dependencyMux Props param. Steady + Applying states now
    receive AttributeValuePublished from their DriverInstance children
    and forward to the mux. Null mux is a no-op (dev/Mac).

ServiceCollectionExtensions:
  - WithOtOpcUaRuntimeActors spawns DependencyMuxActor before
    DriverHostActor and threads its ActorRef into the host's Props.
    New DependencyMuxActorKey + DependencyMuxActorName.

Tests: Runtime 57 -> 63 (+6):
- Mux forwards to only subscribers interested in each ref
- Publish for unregistered ref is dropped silently
- Unregister stops forwarding
- Re-register replaces prior interest set
- VirtualTagActor PreStart registration drives end-to-end eval
  (uses AwaitAssert to race-safely settle the PreStart Tell)
- DriverHostActor forwards AttributeValuePublished through to mux

All 6 v2 test suites green: 163 tests passing.

F8 (#79) state updated — dep subscribe seam shipped, Core.VirtualTags
production engine binding (compile + ITagUpstreamSource subscribe) is
the residual.
2026-05-26 09:43:06 -04:00
Joseph Doherty da141497f8 feat(runtime): F7 spawn lifecycle + F20 ShouldStub gate
DriverHostActor.ApplyAndAck now reads the deployment artifact and
reconciles its set of DriverInstanceActor children — spawn the missing,
ApplyDelta to those with changed config, stop the removed/disabled.
The diff lives in pure DriverSpawnPlanner so it can be unit-tested
without an ActorSystem.

Adds IDriverFactory in Core.Abstractions (consumed by Runtime) +
DriverFactoryRegistryAdapter in Core.Hosting that wraps the existing
v1 DriverFactoryRegistry — Runtime stays decoupled from Polly/Serilog,
the Host wires the adapter once driver assemblies have registered.

ShouldStub(type, roles) is now actually called on every spawn — Galaxy
+ Wonderware-Historian boot stubbed on macOS/Linux or whenever the host
carries the dev role. Missing factory ⇒ stub fallback, never a crash.

Tests: 24 → 34 in Runtime (+10):
- DriverSpawnPlannerTests x7 (diff cases, type change ⇒ stop+respawn)
- DeploymentArtifactTests  x5 (empty/malformed/missing fields tolerant)
- DriverHostActorReconcileTests x4 (spawn count, stub fallback,
  ShouldStub gate, second-apply stops the removed)
All 6 v2 test suites green: 120 tests passing.

Closes F20 (ShouldStub wired). F7 marked partial — subscription
publishing + write path still stubbed in DriverInstanceActor itself.
2026-05-26 08:57:16 -04:00
Joseph Doherty 686138123f feat(runtime): F11 — HistorianAdapterActor wired to IAlarmHistorianSink
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been cancelled
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been cancelled
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been cancelled
v2-ci / build (push) Has been cancelled
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been cancelled
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been cancelled
v2-ci / integration (push) Has been cancelled
Reshapes the placeholder buffered-counter actor into a thin fire-and-forget
bridge over the existing IAlarmHistorianSink contract. Default sink is
NullAlarmHistorianSink; production deployments override the DI binding to
SqliteStoreAndForwardSink wrapping WonderwareHistorianClient (the v1
components in src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Wonderware*
are reused verbatim — actor is just a mailbox-friendly entry point).

- HistorianAdapterActor.Props(IAlarmHistorianSink?) — null defaults to NullAlarmHistorianSink
- Receive<AlarmHistorianEvent>: fire-and-forget sink.EnqueueAsync
- Receive<GetStatus>: returns sink.GetStatus() (queue depth + drain state)
- ServiceCollectionExtensions.AddOtOpcUaRuntime registers the default sink
- WithOtOpcUaRuntimeActors spawns the actor + registers HistorianAdapterActorKey
- Program.cs calls AddOtOpcUaRuntime when hasDriver

Tests: 2 new (forward-to-sink + GetStatus). Runtime suite 17 → 18.
2026-05-26 07:18:08 -04:00
Joseph Doherty 09d6676e1f feat(runtime): WithOtOpcUaRuntimeActors extension for driver-role node startup (F19)
Mirrors WithOtOpcUaControlPlaneSingletons for the driver role. Spawns
DriverHostActor + DbHealthProbeActor on the host's ActorSystem and
registers both under marker keys. Host's Program.cs now calls it when
the node carries the driver role, so driver-only and admin+driver
deployments both auto-bootstrap the per-node actors.

Integration test covers the registration round-trip via Microsoft.Extensions.Hosting
+ Akka.Hosting AddAkka.
2026-05-26 06:09:37 -04:00