HistorianAdapterActor now subscribes to the redundancy-state DPS topic,
caches the local node's RedundancyRole, and SKIPS the durable-sink enqueue
when the local node is Secondary or Detached. Unknown/null role default-writes
so single-node deploys and the boot window never silently drop historization.
GetStatus stays ungated.
PREMISE: verified the actor is registered but FED BY NOTHING in production —
there is no AlarmHistorianEvent producer and nothing resolves its registry key
to Tell it. This is a FORWARD-LOOKING / DEFENSIVE guard, not a fix for a live
double-write: the moment a per-node feeder lands (engine -> historian, expected
as a per-node cluster broadcast like the alerts topic), only the Primary will
write to the durable sink (exactly-once across all alarm sources).
Mirrors the sibling A1 treatment of ScriptedAlarmHostActor (06c4155) and
OpcUaPublishActor's redundancy-state handler. localNode threaded through
HistorianAdapterActor.Props from ServiceCollectionExtensions (roleInfo.LocalNode).
T21: add an AdminUI path for acknowledging/shelving alarms that routes
through the admin-pinned AdminOperationsActor cluster singleton, which
republishes onto the same 'alarm-commands' DPS topic the OPC UA method
path (T18) and the engine subscriber (T19) use. The broadcast + the
ScriptedAlarmHostActor ownership filter handle cross-node routing, so
the singleton needs no knowledge of which node owns the alarm.
- Commons: AcknowledgeAlarmCommand/ShelveAlarmCommand (+ result records)
and a shared AlarmCommandsTopic const; ScriptedAlarmHostActor now
re-exports that const (mirrors the DriverControlTopic pattern).
- AdminOperationsActor: two handlers map the control-plane messages to
AlarmCommand (Acknowledge / OneShotShelve / TimedShelve / Unshelve,
threading User/Comment/UnshelveAtUtc) and publish via the DPS mediator.
- IAdminOperationsClient + AdminOperationsClient: typed Acknowledge/Shelve
ask wrappers mirroring StartDeploymentAsync.
- Alerts.razor: per-row DriverOperator-gated Ack/Shelve/Unshelve controls;
operator name from AuthenticationState. Timed-shelve datetime UI deferred.
- 5 TestKit tests (mediator-probe subscribed to alarm-commands) verifying
each kind's mapping + reply; 56/56 ControlPlane tests green.
Validate AddComment up-front (IsNullOrWhiteSpace guard + Warning log) so
a blank-comment command is cleanly rejected before reaching the engine
rather than faulting inside ApplyAddComment and being silently swallowed
by the outer catch. Mirrors the existing TimedShelve missing-UnshelveAtUtc
pattern.
Also fix two stale inline comments: the "async void crash" note on
TimedShelve now correctly says "fault escaping async Task → supervision
restart", and the ownership-filter now documents the benign race with a
concurrent LoadAsync clearing the loaded set.
Tests: AlarmCommand_add_comment_empty_text_is_rejected_not_driven (Theory
— empty string + whitespace) and AlarmCommand_add_comment_nonempty_drives_engine
(positive path, asserts CommentAdded transition on alerts topic).
Subscribe the host to the cluster alarm-commands DPS topic in PreStart and
drive the matching ScriptedAlarmEngine op per inbound AlarmCommand. An
ownership filter (engine.LoadedAlarmIds) ignores commands for alarms this
node does not own; TimedShelve without UnshelveAtUtc and unknown operations
are logged + rejected (never thrown); op failures are caught + logged so a
faulting op can't fault the actor. Re-projection is left to the engine's
existing OnEvent -> OnEngineEmission path.
Handler is a Task-returning ReceiveAsync (the project's AK2003 analyzer
forbids an async-void Receive delegate), giving ordered awaited async on the
actor thread. Adds 3 TestKit tests: ack drives the engine with mapped args,
unowned command ignored, missing-UnshelveAtUtc TimedShelve rejected not
thrown.
Wire the materialised AlarmConditionState method handlers so a client calling
Acknowledge/Confirm/Shelve/AddComment is gated on the AlarmAck data-plane role
and, when allowed, routed back to the scripted-alarm engine via a new
`alarm-commands` DistributedPubSub topic.
- Commons: new AlarmCommand DTO (AlarmId/Operation/User/Comment/UnshelveAtUtc).
- ScriptedAlarmHostActor: add AlarmCommandsTopic const.
- OtOpcUaNodeManager: settable AlarmCommandRouter + wire OnAcknowledge/OnConfirm/
OnAddComment/OnShelve/OnTimedUnshelve. Each resolves the principal off
ISessionOperationContext.UserIdentity as RoleCarryingUserIdentity, fails closed
(BadUserAccessDenied) when the AlarmAck role is absent or no identity, else maps
+ routes an AlarmCommand and returns Good. OnShelve discriminates OneShotShelve/
TimedShelve/Unshelve from the SDK flags; TimedShelve expiry = UtcNow + ms.
No Akka/IActorRef handle — only the Action<AlarmCommand> delegate. T20 de-dup
note left; WriteAlarmCondition untouched.
- OpcUaServer.Security: OpcUaDataPlaneRoles.AlarmAck shared const (the role was a
bare string everywhere; introduced one symbol for the gate + tests).
- OtOpcUaSdkServer: SetAlarmCommandRouter pass-through.
- Host: boot wiring publishes each command via mediator.Tell(Publish(...)) using a
lazy ActorSystem accessor (mirrors DpsScriptLogPublisher).
- Tests: 11 new gate + mapping tests (OpcUaServer.Tests 88->99, all green).
Default HistorizeToAveva/Retain/Enabled to the entity defaults (true) when a
field is absent/null/non-boolean so a partial blob decodes identically to the
composer's view of a default-constructed ScriptedAlarm (byte-parity), and only
call GetBoolean for a genuine true/false token. Add direct ExtractAlarmDependencyRefs
unit tests (overlap dedup + reserved {{equip}} exclusion).
Concrete ITagUpstreamSource the scripted-alarm host actor pushes
DependencyValueChanged values into and ScriptedAlarmEngine reads/subscribes
from. Thread-safe: ConcurrentDictionary value cache + per-path ImmutableList
observer lists with atomic add/remove and capture-then-invoke fan-out.
ReadTag of an unknown path returns a Bad-quality (0x80000000) snapshot stamped
via the injected clock. Adds the Core.ScriptedAlarms project reference Runtime
needs to see the interface.
ParseComposition(blob, nodeId, onInconsistency?) detects a kept equipment whose
UNS line belongs to another cluster (a same-cluster-invariant violation that
would orphan the equipment folder) and reports it via an optional callback,
wired to OpcUaPublishActor's logger. Detection-only; the upstream draft
validator remains the authority. Adds two unit tests.
Log a WARNING on startup when IVirtualTagEvaluator is not registered so a DI misconfig on a
driver-role node is visible in logs instead of silently evaluating all VirtualTags to NoChange.
Add a comment in PushDesiredSubscriptions noting that TryRecoverFromStale does not call this
method, so VirtualTags remain empty after a Stale recovery until the next deployment dispatch
(intentional, consistent with driver recovery).
Context.Watch each spawned child; OnChildTerminated evicts it from _children so the
next ApplyVirtualTags (still containing that vtagId) falls through the ContainsKey
guard and re-spawns a fresh VirtualTagActor. Adds a spawn-site Debug log, moves the
TODO about in-place plan mutation to the skip-existing branch where it belongs, and
adds a deterministic TestKit test (Child_is_respawned_after_unexpected_termination)
that kills the first child, drains its UnregisterInterest from the mux probe, re-applies,
and asserts a second distinct RegisterInterest arrives.
Two bundle-review fixes + idempotency coverage:
- CRITICAL: the planner ignored EquipmentTags, so an incremental deploy changing only
equipment tags produced an empty plan and HandleRebuild short-circuited before
materialising them. Add TagId to EquipmentTagPlan + Added/Removed/ChangedEquipmentTags
to Phase7Plan (diffed by TagId, in IsEmpty, driving Apply's needsRebuild) — mirroring
the GalaxyTags treatment.
- IMPORTANT: equipment variable NodeId was the raw driver FullName, which collides across
identical machines (e.g. two PLCs both exposing register 40001) — the second variable
was silently dropped. NodeId is now folder-scoped (parent/Name); FullName stays on
EquipmentTagPlan for the later values-routing milestone.
- Task 4: SDK-backed idempotency test (double-apply -> single variable); restart-safety
confirmed (RestoreApplied reuses the same RebuildAddressSpace -> HandleRebuild path).
- Minor: align composer equipment-tag sort with the artifact decoder (coalesce FolderPath).
Equipment folder DisplayName was the colloquial MachineCode; the live rebuild (artifact
ReadEquipmentNode) + composer now use the UNS level-5 Name segment, matching Area/Line
folders + EquipmentNodeWalker. NodeId stays the logical EquipmentId so browse-path
resolution + ACLs are unaffected.
Add Phase7Applier.MaterialiseEquipmentTags — a sink-based pass (Task-0 decision A) that
ensures each EquipmentTagPlan's Variable (NodeId = FullName) under its existing equipment
folder, nesting any FolderPath as a sub-folder. Wire it into OpcUaPublishActor.HandleRebuild
after the Galaxy pass. Variables start BadWaitingForInitialData; never re-creates equipment
folders (decision #4).
Add EquipmentTagPlan + an init-only EquipmentTags member on Phase7CompositionResult
(mirror of GalaxyTags). Populate it compose-side (Tag.EquipmentId != null AND owning
namespace Kind == Equipment) and artifact-decode-side via BuildEquipmentTagPlans, with
FullName extracted from Tag.TagConfig. Init-only member (not a 7th positional param) so
existing convenience constructors + call sites are untouched.
Two ordering/lifecycle gaps surfaced once tag values began streaming:
1. OpcUaPublishActor.HandleRebuild loaded the latest *Sealed* artifact, but the
rebuild fires at apply time — before this deployment seals — so it materialised
the PREVIOUS revision while SubscribeBulk subscribed to the applied one. The two
disagreed (4 variables materialised vs 396 subscribed) and every config needed
two deploys. RebuildAddressSpace now carries the applied DeploymentId and the
rebuild loads that exact artifact.
2. On restart a node recovered its revision from NodeDeploymentState but left the
driver children + address space empty (and an identical-config redeploy no-ops on
the unchanged revision), so a rebuilt node served nothing until a config change.
Bootstrap now calls RestoreApplied: re-spawn drivers, rebuild from the applied
artifact, re-push SubscribeBulk — no re-ack.
Verified live: recreating the driver nodes auto-restores all 396 galaxy mirror
tags across 40 machines with Good live values, no deploy required.
Materialised SystemPlatform/Galaxy variables previously stayed
BadWaitingForInitialData because nothing told the driver to subscribe
(OpcUaPublishActor TODO 'on a future SubscribeBulk pass') and published
values were only forwarded to the VirtualTag mux, never the OPC UA sink.
DriverHostActor now, after each apply, groups the deployment's galaxy tag
MXAccess refs by driver and sends DriverInstanceActor.SetDesiredSubscriptions;
the actor retains the set and (re)subscribes on every Connected entry, so
values resume after reconnects/redeploys (closes the F8b/#113 gap). Published
values are also forwarded to OpcUaPublishActor as AttributeValueUpdate
(NodeId == galaxy MxAccessRef) so the materialised variable shows live data.
Verified live in docker-dev: galaxy TestMachine_001 tags go Good with a
changing TestChangingInt. +1 unit test.
- Topic-name drift fix: DriverHealthChanged.TopicName and
DriverControlTopic.Name now live on the message contracts in
Commons. AkkaDriverHealthPublisher, DriverStatusSignalRBridge,
DriverHostActor, and AdminOperationsActor all delegate to the
single constant so a rename can't silently desynchronise
publisher and subscriber.
- DriverStatusPanel._opResultClearTimer switched from
System.Timers.Timer to System.Threading.Timer + awaited
DisposeAsync. Prevents an in-flight 8s clear-callback from
invoking StateHasChanged on a component whose hub has already
been released.
- PublishHealthSnapshot deduplicates against the last published
(state, lastSuccess, lastError, errorCount) fingerprint. The
30s heartbeat no longer floods the SignalR layer with identical
Healthy snapshots — newly-joined clients still warm up via the
snapshot store on JoinDriver.
- DriverInstanceSpec carries ClusterId from the deployment artifact;
DriverHostActor threads the real cluster identity into
DriverInstanceActor instead of the local NodeId. Old pre-PR
artifacts without a ClusterId field fall back to the NodeId so
in-flight deployments keep working.
- DriverHostActor.ChildEntry holds the full DriverInstanceSpec
(was only carrying DriverType + LastConfigJson). Restart respawns
preserve RowId, Name, Enabled, ClusterId — no placeholder values.
- Drop the unnecessary _faultLock on DriverInstanceActor — every
read/write site runs inside an Akka message handler which is
single-threaded per actor instance.
- DriverStatusPanel.DisposeAsync awaits Timer.DisposeAsync so an
in-flight 5s tick can't invoke StateHasChanged on a component
whose hub has already been torn down.
- RestartDriver / ReconnectDriver messages + AdminOperationsActor
handlers (broadcast via driver-control DPS topic; audited via
ConfigEdits).
- DriverHostActor subscribes to driver-control; locates the
matching child DriverInstanceActor and stops+respawns it
(Restart) or sends it a ForceReconnect internal message
(Reconnect — re-enters Reconnecting state without full stop).
DriverInstanceSpec constructor call uses named args to handle
the full 6-parameter signature.
- New DriverOperator authorization policy mapped to DriverOperator
or FleetAdmin role; documented in docs/security.md. Map LDAP
group via GroupToRole (e.g. "ot-driver-operator": "DriverOperator").
- DriverStatusPanel renders Reconnect + Restart buttons when the
user holds the DriverOperator policy (hidden otherwise). Restart
requires an in-page Razor confirm block (no JS confirm, keeps
SignalR event loop unblocked). Both buttons show a spinner and
are disabled during in-flight; result chip auto-clears after 8s.
Username sourced from AuthenticationStateProvider.
Reconnect resolves to "ForceReconnect" (re-enter Reconnecting,
not full stop+respawn) — transport drops and retries while actor
and in-memory state are preserved. All DriverInstanceActor states
handle ForceReconnect safely (no-op when already in transition).
- IDriverHealthPublisher in Core.Abstractions + NullDriverHealthPublisher
no-op for tests/dev-stub paths.
- AkkaDriverHealthPublisher in Runtime forwards to the cluster-wide
`driver-health` DPS topic.
- DriverInstanceActor instrumented to publish snapshots on every
observable state change + a periodic 30s heartbeat so the AdminUI
snapshot store warms up for newly-joined SignalR clients.
- Sliding 5-minute Faulted-count tracked per actor via Queue<DateTime>.
- DriverHostActor.SpawnChild threads clusterId (_localNode.Value) and
the health publisher down to every DriverInstanceActor child.
- ServiceCollectionExtensions.AddOtOpcUaRuntime registers
AkkaDriverHealthPublisher as IDriverHealthPublisher singleton.
Adds <summary>, <param>, <typeparam>, and <inheritdoc/> tags to public
members surfaced by commentchecker — resolves 5,847 of 5,869 issues
(99.6%) across three /fixdocs passes.
Closes the gap where Tag rows with EquipmentId=NULL + Namespace.Kind=SystemPlatform
(Galaxy hierarchy) existed in ConfigDb but were never surfaced in the OPC UA
address space. Now they materialise as Variable nodes under a folder named for
their FolderPath, browseable through any OPC UA client.
Layers touched:
- IOpcUaAddressSpaceSink: new EnsureVariable(nodeId, parentFolderId, displayName,
dataType) signature on the sink interface, NullSink, DeferredSink, SdkSink.
- OtOpcUaNodeManager.EnsureVariable: creates a BaseDataVariableState parented
under the named folder (or root), initial Value=null +
StatusCode=BadWaitingForInitialData; resolves Tag.DataType strings to the
matching OPC UA built-in NodeId. Idempotent.
- Phase7CompositionResult: new GalaxyTags collection of GalaxyTagPlan records
carrying (TagId, DriverInstanceId, FolderPath, DisplayName, DataType,
MxAccessRef). Constructor overloads keep existing call sites compiling.
- Phase7Composer.Compose: now takes Tag + Namespace inputs, filters for
SystemPlatform-namespace tags with EquipmentId=NULL, emits GalaxyTagPlan
rows with MXAccess ref "FolderPath.Name".
- Phase7Plan: new AddedGalaxyTags / RemovedGalaxyTags / ChangedGalaxyTags
collections + GalaxyTagDelta record; IsEmpty + needsRebuild updated.
- Phase7Planner.Compute: diffs GalaxyTags by TagId via existing DiffById helper.
- DeploymentArtifact.ParseComposition: reads the Tags + Namespaces +
DriverInstances arrays the ConfigComposer already emits, applies the same
SystemPlatform filter, returns the same GalaxyTagPlan list as the composer
so artifact-side and compose-side plans agree.
- Phase7Applier: new MaterialiseGalaxyTags pass that ensures one folder per
distinct FolderPath then one Variable per tag. NodeId for the variable is
"<FolderPath>.<Name>" matching the MXAccess ref so the future Galaxy
SubscribeBulk wiring can address them directly.
- OpcUaPublishActor.RebuildAddressSpace: invokes MaterialiseGalaxyTags after
MaterialiseHierarchy. _lastApplied initialiser updated for the new ctor.
- seed-clusters.sql: pre-existing TestMachine_001.TestAlarm001..003 rows
needed no change — the composer/applier now picks them up automatically.
Verified end-to-end via docker-dev: deploy click → driver-a logs
"Phase7Applier: Galaxy tags materialised (tags=3, folders=1)" → OPC UA Client
CLI browses the three Variable nodes under TestMachine_001 folder. Reads
return BadWaitingForInitialData status (expected — Galaxy driver's
SubscribeBulk wiring to push values into the nodes is the remaining
follow-up).
User confirmed the mxaccessgw client (Galaxy driver) doesn't need Windows
— only the gateway worker has that constraint. This wires the Galaxy
driver into the docker-dev fleet:
- docker-compose.yml: GALAXY_MXGW_API_KEY env var on every host service
(admin nodes harmlessly ignore it; driver-role nodes pick it up when
the seeded DriverInstance resolves ApiKeySecretRef=env:GALAXY_MXGW_API_KEY).
Default value matches the key the operator provided; override via shell
env (GALAXY_MXGW_API_KEY=... docker compose up -d) to rotate without
editing compose.
- seed-clusters.sql: now creates a SystemPlatform Namespace
(MAIN-galaxy, urn:zb:docker-dev:galaxy) plus a GalaxyMxGateway
DriverInstance (MAIN-galaxy-mxgw) in the MAIN cluster pointing at
http://10.100.0.48:5120 with UseTls=false. Idempotent via IF NOT EXISTS.
- DriverInstanceActor.ShouldStub: clarified the doc comment — only the
legacy "Galaxy" type name and "Historian.Wonderware" are Windows-only;
the v2 "GalaxyMxGateway" driver is .NET 10 cross-platform (gRPC to an
external gateway) and is NOT stubbed.
- README: documents the final operator step — sign in, click "Deploy
current configuration" on /deployments to materialise the seeded
Galaxy driver into a running gRPC connection. Raw DriverInstance rows
don't spawn drivers on their own; the v2 lifecycle requires a sealed
Deployment first.
Phase7Composer now carries UnsAreaProjection + UnsLineProjection lists so
the applier can materialise the full UNS topology in the OPC UA address
space. New IOpcUaAddressSpaceSink.EnsureFolder(folderNodeId, parentNodeId,
displayName) seam (no-op default, recorded in tests, forwarded by
DeferredAddressSpaceSink, implemented by SdkAddressSpaceSink). The SDK-
side OtOpcUaNodeManager gains an EnsureFolder API that creates
FolderState nodes with proper parent linkage; RebuildAddressSpace now
clears folders too so re-applies don't accumulate stale topology.
Phase7Applier.MaterialiseHierarchy walks composition.UnsAreas →
composition.UnsLines → composition.EquipmentNodes, calling EnsureFolder
with the correct parent at each level. Idempotent — calling twice with
the same composition is a no-op. OpcUaPublishActor.HandleRebuild invokes
it after Phase7Applier.Apply so OPC UA clients browsing the server now
see Area/Line/Equipment as proper folders rather than flat tag ids.
DeploymentArtifact.ParseComposition reads UnsAreas + UnsLines from the
JSON snapshot the ControlPlane emits, populating the new fields when
present.
Phase7Composer.Compose now accepts UnsAreas + UnsLines; a 3-arg overload
preserves the old signature for legacy callers + existing tests. The
Phase7CompositionResult convenience ctor likewise keeps the planner
tests working without UNS data.
3 new hierarchy tests (pure unit + boot-verify against a real
OtOpcUaSdkServer); OpcUaServer suite is 48/48 green (was 45, +3),
Runtime 74/74 unchanged.
Closes#85.
Joseph Doherty