60 s labelled a slow-but-clean large rebuild degraded. The deadline moves to a
shared Commons DeployTimings constant that ConfigPublishCoordinator reads, and
the node's timeout is deadline - 15 s margin, so the held ack still lands in
time. A timed-out ack's reason now says the outcome was not reported in time,
distinct from a real sink failure.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
The apply writes the node's row Applied/null before the address-space outcome
is known; central adds the reason only when it ACCEPTS the ack, so a discarded
(stale/timed-out) ack left a fused node's row reading clean. The prefix moves
to ApplyAck.DegradedReasonPrefix so both writers spell the row identically.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
ApplyAck gains trailing Degraded/DegradedReason (defaulted, so an old node's
DPS JSON or gRPC ack decodes exactly as before). mesh_transport.proto adds
ApplyAckMessage.degraded = 6 / optional degraded_reason = 7. OpcUaPublishActor
replies RebuildCompleted(Degraded, Reason) on every handler path when the
RebuildAddressSpace carries ReportOutcome.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
#542 filed FleetStatusBroadcaster as blind to site nodes since the Phase 6 per-cluster mesh
split: it derives "the fleet" from central's own Akka gossip ring, which after the split is
just central's own pair. Scoping the cross-mesh rewrite found the deeper truth — the whole
chain is dead end to end, so this deletes it rather than reviving it.
- Zero consumers. FleetStatusBroadcaster published FleetStatusChanged on the DPS
"fleet-status" topic; FleetStatusSignalRBridge forwarded it to FleetStatusHub
("/hubs/fleet-status"); nothing anywhere connected. The AdminUI bans self-connecting
HubConnections, no browser JS subscribes, and /fleet (Fleet.razor) reads
NodeDeploymentState from the ConfigDb plus IClusterRoleInfo on a 5 s poll instead.
- Its second input never existed. DriverHostStatusHeartbeat had no production sender —
only the broadcaster's own test told it — so CurrentRevision was always null and the
30 s staleness -> Degraded flip could never fire.
- Post-Phase-6 DPS is pair-local, so even the central-only view it did produce was an
accident of topology rather than a design.
The fleet visibility that actually works is unchanged and already cross-mesh-correct:
/fleet over NodeDeploymentState, and the four Phase 5 per-node telemetry dials.
Precedent for deleting inert machinery rather than activating it: #522 (driver stability
tiers), #507 (discovered-node injection), #525 (GetMemoryFootprint).
Deleted: FleetStatusBroadcaster (+ its nested DriverHostStatusHeartbeat), its admin
singleton registration and FleetStatusBroadcasterKey, FleetStatusSignalRBridge and its
key, FleetStatusHub and its route mapping, and the FleetStatusChanged / FleetNodeStatus /
FleetNodeHealth messages. Both Fleet/ directories survive — ControlPlane keeps
ClusterNodeAddressReconciler, Commons keeps GetDiagnostics. Docs swept where they claimed
a fleet-status channel exists (Telemetry.md's Phase 5 non-migrated channels,
Configuration.md, CLAUDE.md); completed-program plan docs under docs/plans and docs/v2 are
left as the historical record they are.
Build 0 errors, 868 warnings — byte-identical to master's rebuild baseline.
ControlPlane.Tests 231/231, AdminUI.Tests 1109/1109.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
MeshPaths.CentralCommunication / .NodeCommunication (the "/user/..." full-path constants) had
ZERO production consumers after the ClusterClient deletion: both gRPC services resolve their
comm actor through ActorRegistry by typed key, never by ActorSelection over a path. Only the
path-probe test still read them, while the class doc went on calling them "the wire contract
of the central↔node boundary" — a claim that stopped being true the moment nothing addressed
an actor across a process boundary by name.
- The two full-path constants are deleted; the two Name constants remain.
- MeshPaths' doc now says what it is (the local actor-naming convention the registry-keyed
services are spawned under) and records what it stopped being, so the next reader does not
re-derive a contract from the file's history.
- MeshCommActorPathTests composes "/user/" + Name itself — the absolute path is that test's
own addressing detail now, not something the product declares. Renamed
Both_mesh_comm_actors_exist_at_their_contract_paths ->
..._are_spawned_under_their_declared_names, because what it still pins is that the two
registration sites actually ran on a real booted node; a rename is now a compile error, so
"contract" was the wrong word for what would break.
- NodeCommunicationActor's summary no longer cross-references the deleted constant or calls
the name an address. CentralCommunicationActor was checked and makes no such claim.
MeshGrpcBoundaryTests replaces the MeshClusterClientBoundaryTests deleted in step 1. Three
self-seeded single-member clusters sharing the ActorSystem name, joined ONLY by gRPC, driving
the REAL CentralCommunicationActor -> MeshGrpcCommandDialer -> Kestrel h2c ->
MeshCommandAuthInterceptor -> NodeCommandGrpcService -> NodeCommunicationActor -> EventStream:
1. No_cross_mesh_Akka_association_forms_between_central_and_either_site — the program's own
point, kept as a test rather than a live-gate note: each cluster stays single-member and
none ever holds another's address as a member.
2. One_dispatch_through_central_reaches_a_node_in_every_cluster_mesh — the fan, with each
site holding its OWN per-cluster PSK, so a dial that selected the wrong cluster's key
would be refused at the far side.
3. Falsifiability_control_a_dial_carrying_the_wrong_cluster_key_delivers_nothing — without
it, (2) cannot distinguish "the gRPC boundary carried it" from "something else in the
process did". The same node then accepts the right key, so the silence is the auth gate
and not a dead endpoint.
All three run fully offline — no Docker fixture — and pass.
Comment sweep across every remaining clusterclient|receptionist hit in src/ and tests/:
the program label becomes "mesh gRPC command plane", and every comment that described the
deleted transport as live now describes what actually runs (Dps = pair-local default,
Grpc = cross-mesh). mesh_transport.proto's header/service comments are reworded; no message,
field or enum is touched — the contract is locked. MeshCommActorPathTests is reshaped rather
than retired: the two paths still exist and still matter, but what a mismatch costs is now a
misleading Unavailable from the gRPC ingress rather than a silently dropped receptionist send.
The only surviving mentions of the old names are the ones that must survive: the fail-closed
rules that name "ClusterClient" as a retired value refused at boot, their tests, and this
test class's own account of what it replaces.
A driver builds its RawPath -> definition table at init and tolerates a
per-tag miss: an unmappable TagConfig, an unparseable address, a
capability-matrix rejection. That tolerance is right -- one bad tag must not
take down the other few thousand. What was missing is any statement of the
fact afterwards. A skip is not an error and happens before any read, so the
driver stays Healthy, errors/5 min stays 0, Last error stays empty, and Last
read advances live off the tags that did map. On the wonder Makino in #540
that reading held while focas-z-34184 ignored 20 of its 22 tags.
New optional capability ISkippedTagReporter, backed by a shared
SkippedTagLedger so the reset/commit/cap rules are one implementation rather
than one per driver. The report -- authored count, exact skipped count, and a
capped sample of RawPath + reason -- rides DriverHealthChanged to a new Tags
column on /hosts, over the Phase 5 telemetry stream. No table: a driver-only
node has no ConfigDb to write to, the lesson from #521.
The issue named eight drivers. It is twelve -- every driver in
DriverTypeNames.All has an authored-tag skip path. Calculation (no scriptId),
MTConnect (no readable DataItem id), Galaxy (no attributeRef) and OpcUaClient
(no nodeId) were unlisted, and OpcUaClient's was the quietest of the twelve:
NamespaceMap drops such an entry with a bare `continue` and not even a log
line. SkippedTagReporterFleetTests pins the invariant reflectively over every
IDriver in src/Drivers/, so a thirteenth cannot re-introduce the silence.
Sql drops a tag at TWO points and its report is the union of both: a blob
that does not map, AND an identifier the 8.1 catalog gate rejects against the
live catalog. The second is what schema drift produces -- a renamed column, a
dropped view -- months after a deployment sealed green, and reporting only
the first would have that driver claim "0 skipped" while polling half its
tags, which is worse than reporting nothing because the column would then be
actively reassuring.
Deliberately additive to driver state, never a change of it. Latching
non-Healthy for a config problem would satisfy both halves of the #527
comms-loss conjunction -- 90 s without a successful read AND non-Healthy --
and a driver whose tags were ALL skipped has no successful reads by
construction, so it would reconnect-loop forever against a healthy device.
Three details that are load-bearing rather than incidental:
- The skip digest is part of the publish fingerprint. A skipped tag leaves
every other fingerprint component identical to a clean driver's, so without
it PublishHealthSnapshot's dedup swallows exactly the publish carrying the
signal -- the trap that already bit the rediscovery and host-status
channels. The digest covers the sample, not just the counts: a redeploy
that fixes one tag and breaks another leaves both counts unchanged.
- The count is exact; only the sample is capped, and the tooltip says so when
it is showing fewer than it counted. A truncated list that looked complete
would have an operator fix twenty tags and believe the job done.
- The ledger stages and commits, so an init that throws part-way leaves the
previous report standing. A partial snapshot would read as "we now skip
fewer tags", the opposite of the truth.
The proto field needs no companion presence flag, unlike host_statuses: a
singular message field carries real presence in proto3, and a has_ flag would
collide with protoc's generated HasSkippedTags.
Verified red-first three ways: dropping SkippedTagDigest from the fingerprint
fails the two dedup tests, removing the seam from one driver fails the fleet
guard, and dropping the catalog-gate leg fails the Sql union test. Full
solution builds with zero new warnings against 4e22f024.
`IHostConnectivityProbe` was a dead surface: eleven drivers implement it, `GetHostStatuses()`
had ZERO production call sites, and `OnHostStatusChanged` had no subscriber outside the Galaxy
driver's own aggregator. Per-host connectivity was computed by every driver and read by nobody.
The issue offered "build the publisher or delete it". The publisher as its entity doc described
it — driver nodes upserting `DriverHostStatus` rows — is not buildable: per-cluster mesh Phase 4
gates `AddOtOpcUaConfigDb` on the `admin` role, so a driver-only node has no ConfigDb connection
to write rows with. So the capability is kept and the transport changed.
`DriverHealthChanged.HostStatuses` now carries the probe result to `/hosts` as a Hosts column.
That channel already reached the page, already survives the mesh split via the Phase 5 gRPC
telemetry stream, and already replays a last-value snapshot on re-subscribe — so per-host state
re-primes after a reconnect without a durable store. Both halves of the interface finally do
what they are for: the event triggers a prompt publish, the pull is the source of truth.
The point of the column is the case the driver-level state chip structurally cannot express: a
multi-device driver stays aggregate-Healthy while ONE of its devices is unreachable.
Two traps, both pinned by tests that were falsified against the prod code:
- The host digest MUST be in the publish fingerprint. On a single-host-down transition every
other fingerprint component is unchanged, so the dedup would swallow exactly the publish
carrying the news — the trap that already bit the rediscovery signal. Removing it turns the
guard test red, verified.
- null (no probe) must stay distinct from empty (probe with no hosts). proto3 cannot tell an
absent repeated field from an empty one, hence the explicit `has_host_statuses` flag;
collapsing them would render every probe-less driver as one whose devices are all fine.
Dropped: the DriverHostStatus entity, enum, DbSet, model config and table (migration
DropDriverHostStatusTable — empty on every deployment, so the scaffolder's data-loss warning is
moot, and Down() recreates it exactly).
Found en route, NOT fixed here: `DriverInstanceResilienceStatus` is the identical defect — no
writer, no reader, only a DbSet declaration, while the live data rides the
`driver-resilience-status` telemetry channel. Its doc-comment now states that rather than
describing the sampler and AdminUI join that were never built. Filed as #524 rather than widening
this schema change beyond what was asked.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Nine drivers raise IRediscoverable.OnRediscoveryNeeded when they observe that
a remote's tag set may have changed — a Galaxy redeploy, a TwinCAT
symbol-version bump, an MTConnect agent restart, a Sparkplug rebirth. Nothing
in src/ subscribed, so every raise went into the void. Drivers even wrap the
invoke in try/catch for "subscriber threw"; there has never been a subscriber.
DriverInstanceActor now attaches the event in PreStart and detaches in
PostStop, mirroring AttachAlarmSource/DetachAlarmSource. Attach is per-actor,
not per-connect: the raise has no connection affinity and can land while the
driver is between connects. The detach is load-bearing rather than tidy — the
IDriver instance outlives the actor when the host respawns a child around the
same driver object, so a missing -= accumulates one handler per respawn, each
holding a dead Self.
The signal rides the existing driver-health snapshot to /hosts as a
"re-browse" chip. It is ADVISORY: the served address space is deliberately not
rebuilt, because v3 authors raw tags explicitly through the /raw
browse-commit flow and a runtime graft would materialise nodes no operator
approved and no deployment artifact records.
Two things worth knowing:
- PublishHealthSnapshot dedups on a health fingerprint, and a rediscovery
raise on an otherwise-healthy driver changes none of the four fields it
hashed. Without adding the timestamp to that tuple the dedup swallows the
exact publish carrying the signal. Reverting that one line turns 3 of the 5
new tests red, which is how it was confirmed rather than assumed.
- The new test stub implements IDriver from scratch instead of deriving from
the shared StubDriver, whose GetHealth() returns DateTime.UtcNow — its
fingerprint differs on every call, so the dedup never engages and the dedup
test would have passed vacuously either way.
The planned discovery-result drift detector was NOT built. Modbus, S7, MQTT,
AbLegacy and Sql all echo authored config from DiscoverAsync rather than
browsing the device, so a discovered-vs-authored diff is permanently empty for
them — it would have been another plausible-looking inert seam, which is the
class this audit exists to remove. IRediscoverable is the trustworthy signal
because the driver asserts it deliberately.
DriverHealthChanged, IDriverHealthPublisher.Publish and HostsDriverRow gain
defaulted optional parameters, so no existing call site changed. telemetry.proto
gains fields 8/9 and both Phase-5 mappers carry them.
Runtime.Tests 512 passed / 31 skipped.
Phase 2 Task 3. Central-side end of the boundary: routes every central->node
command out over DPS or ClusterClient per MeshTransport:Mode, and forwards
inbound ApplyAcks to the deploy coordinator singleton (Forward, not Tell, so
the coordinator sees the node rather than this relay).
Three things here are load-bearing and easy to get wrong later:
SendToAll, never Send. Today's DPS publish reaches EVERY DriverHostActor and
the node side has no ClusterId or node filter to compensate -- scoping happens
later, inside the artifact. ClusterClient.Send delivers to exactly ONE
registered actor, so it would deploy to a single node while every other node
silently kept its old config, and the deployment could still seal green.
Sabotage-verified: swapping to Send reddens exactly that one test.
Exactly ONE ClusterClient, fleet-wide. A receptionist serves its whole cluster,
so SendToAll reaches every registered node-comm actor in the mesh regardless of
which contact point was dialled. One client per application Cluster -- the
shape Phase 6 wants -- would fan each command out once per cluster while the
fleet is still one mesh: N x duplicate DispatchDeployment and ApplyAck. Marked
TODO(Phase 6).
The contact set does NOT scope delivery. Excluding a maintenance-mode node from
the contacts does not stop it receiving commands; it still receives them, as it
does under today's broadcast. The filter is about which receptionists are worth
dialling, not about who gets the message. Stated in the code because the
opposite is the natural assumption.
Also carries the sister project's shipped fixes: per-row address parsing so one
malformed ClusterNode cannot abort the refresh and leave the contact set
half-built (regression test orders the bad row FIRST), the cache entry cleared
before recreate so a failed create cannot leave commands routing into a
stopping actor, and a Status.Failure handler so a DB outage is a Warning rather
than a silent debug-level unhandled message.
Two test-harness bugs found and fixed while writing this, both mine: SubscribeAck
goes to the SENDER (so the mediator subscribe needed an explicit sender), and
EventFilter matches case-INSENSITIVELY, so a "was NOT" filter also caught this
actor's own "was not created" warning.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Phase 2 Task 2. Two small pieces the comm actors are built on.
MeshCommand carries a command plus the DPS topic it would have been published
on, so the admin singletons stop knowing which transport is in force. Without
it the dark switch would have to be threaded through five publish sites across
two actors, each free to drift. Central-side only -- it never crosses the wire,
so it carries no serialization contract.
DefaultMeshClusterClientFactory appends a generation counter to the actor name.
Context.Stop is asynchronous and the name stays reserved until termination
completes, so recreating the same name inside one message handler throws
InvalidActorNameException -- which is exactly what a contact-set change does
(stop the old client, create the replacement, one handler). Ported from the
sister project, where it was a shipped bug fix rather than foresight.
Sabotage-verified: freezing the name to a constant reddens both name tests with
InvalidActorNameException.
Dropped a third test I had written -- it asserted ClusterClientSettings
directly and never touched the factory, so it was coverage theatre. Replaced
with a comment saying where contact propagation IS covered (the Task 8 boundary
test, the only place it can actually fail).
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
RedundancyStateActor derived the driver Primary from ClusterState.RoleLeader("driver").
Akka offers two different notions of a "first" member and they are not the same
one: role leader is the lowest-ADDRESSED Up member (host, then port), while
ClusterSingletonManager places singletons on the OLDEST — lowest up-number.
They agree on a freshly-formed cluster, which is why every existing test passed.
They diverge after any restart: the restarted node re-joins as the youngest while
keeping its address, so if it holds the lower address it becomes role leader while
the singletons stay put. The snapshot would then name a Primary that is not
hosting the work, and every Primary-gated surface follows it — inbound device
writes, native-alarm acks, the fleet-wide alerts emit, and the alarm-history
drain would all enable on the wrong node while the node actually running the
singletons stayed gated off.
BuildSnapshot now selects the oldest Up member carrying the driver role, matching
singleton placement. Leaving members are excluded: a node handing its singletons
over must not be named Primary.
NodeRedundancyState.IsRoleLeaderForDriver is renamed IsDriverPrimary, and
NodeHealthInputs.IsDriverRoleLeader likewise. Keeping the old names would have
left the wire contract asserting a derivation the code no longer uses — the same
drift that made this defect invisible.
Proven by a real two-node cluster rather than a mock. RedundancyPrimaryElectionTests
binds the first-joining node to the HIGHER port, so oldest and lowest-address name
different nodes, and includes a fixture assertion that the divergence actually
occurred — without it the real assertion could pass for the wrong reason. Positive
control: restoring the RoleLeader derivation turns exactly the two election tests
red while the fixture check stays green.
Runtime.Tests 440 passed, ControlPlane.Tests 82, Cluster.Tests 36.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Materialize each native alarm ONCE at the raw tag (ConditionId = RawPath, Raw
realm); wire the single condition as an SDK event notifier of each referencing
equipment's UNS folder so one ReportEvent fans to every root without
re-reporting per root (which would break Server-object dedup + Part 9 ack
correlation).
- New sink method WireAlarmNotifiers(alarmNodeId, alarmRealm,
notifierFolderNodeIds, notifierFolderRealm) on IOpcUaAddressSpaceSink,
forwarded through DeferredAddressSpaceSink + SdkAddressSpaceSink +
NullOpcUaAddressSpaceSink (the forwarding-trap guard); auto-covered by the
DeferredSinkForwardingReflectionTests realm + forwarding guards + a
hand-written forward test.
- OtOpcUaNodeManager: the normative AddNotifier(isInverse) pair + idempotent
EnsureFolderIsEventNotifier per equipment folder; tracked per condition in
_alarmNotifierWiring. Teardown symmetry: RemoveNotifier(bidirectional:true) on
RebuildAddressSpace, RemoveAlarmConditionNode, and RemoveEquipmentSubtree so no
inverse-notifier entry leaks across redeploys.
- AddressSpaceApplier.MaterialiseRawSubtree wires notifiers for each native alarm
tag, resolving its ReferencingEquipmentPaths (Area/Line/Equipment) to the
EquipmentId folder NodeIds via BuildEquipmentIdByFolderPath.
- AlarmTransitionEvent gains ReferencingEquipmentPaths (empty default); /alerts
renders the referencing-equipment list as display metadata.
- Un-skipped + rewrote the native-alarm dark tests
(DriverHostActorNativeAlarmTests x6, DriverHostActorNativeAlarmAckRoutingTests
x1) for the v3 raw-condition model; new NodeManagerMultiNotifierAlarmTests
proves multi-notifier wiring + teardown symmetry (no leaked duplicates after a
re-trip) + applier wiring test.
Claude-Session: https://claude.ai/code/session_01LVneM3eh1UtJxEisFXgmox
T21: add an AdminUI path for acknowledging/shelving alarms that routes
through the admin-pinned AdminOperationsActor cluster singleton, which
republishes onto the same 'alarm-commands' DPS topic the OPC UA method
path (T18) and the engine subscriber (T19) use. The broadcast + the
ScriptedAlarmHostActor ownership filter handle cross-node routing, so
the singleton needs no knowledge of which node owns the alarm.
- Commons: AcknowledgeAlarmCommand/ShelveAlarmCommand (+ result records)
and a shared AlarmCommandsTopic const; ScriptedAlarmHostActor now
re-exports that const (mirrors the DriverControlTopic pattern).
- AdminOperationsActor: two handlers map the control-plane messages to
AlarmCommand (Acknowledge / OneShotShelve / TimedShelve / Unshelve,
threading User/Comment/UnshelveAtUtc) and publish via the DPS mediator.
- IAdminOperationsClient + AdminOperationsClient: typed Acknowledge/Shelve
ask wrappers mirroring StartDeploymentAsync.
- Alerts.razor: per-row DriverOperator-gated Ack/Shelve/Unshelve controls;
operator name from AuthenticationState. Timed-shelve datetime UI deferred.
- 5 TestKit tests (mediator-probe subscribed to alarm-commands) verifying
each kind's mapping + reply; 56/56 ControlPlane tests green.
Deep-adopt the shared audit record. Deletes the bespoke 8-field positional
Commons AuditEvent and repoints the writer path at ZB.MOM.WW.Audit.AuditEvent
(0.1.0, feed-mapped via dohertj2-gitea). Adds the package reference to both
Commons and ControlPlane.
- AuditWriterActor now implements IAuditWriter: WriteAsync(evt, ct) is a
best-effort, never-throwing entry point that Self.Tell()s the event onto the
same batching/dedup/flush pipeline and returns Task.CompletedTask. Existing
Receive<AuditEvent> + 500/5s batching + two-layer dedup unchanged.
- Flush mapping updated for the canonical field types: OccurredAtUtc is now
DateTimeOffset (.UtcDateTime into the datetime2 column), SourceNode is string?
(was NodeId.Value), CorrelationId is Guid? (stored null when null). Outcome is
NOT yet persisted (column lands in Task 2.2).
- New AuditOutcomeMapper.FromAction maps the OtOpcUa action vocabulary to the
required canonical Outcome: OpcUaAccessDenied / CrossClusterNamespaceAttempt ->
Denied; config verbs (DraftCreated/Edited, Published, RolledBack, NodeApplied,
ClusterCreated, NodeAdded, CredentialAdded/Disabled, ExternalIdReleased) ->
Success. OtOpcUa emits no Failure events.
The Akka message shape changed, but the structured audit path is dormant (zero
production emit/Tell sites; all live audit flows through the bespoke SP path),
so there is no rolling-deploy wire-compat concern. Tested-not-exercised by
design.
ControlPlane.Tests: 44/44 green (AuditWriterActor suite rewritten to construct
the canonical record + assert the Outcome derivation table + the WriteAsync
best-effort/mailbox-routing contract + null SourceNode/CorrelationId handling).
- Topic-name drift fix: DriverHealthChanged.TopicName and
DriverControlTopic.Name now live on the message contracts in
Commons. AkkaDriverHealthPublisher, DriverStatusSignalRBridge,
DriverHostActor, and AdminOperationsActor all delegate to the
single constant so a rename can't silently desynchronise
publisher and subscriber.
- DriverStatusPanel._opResultClearTimer switched from
System.Timers.Timer to System.Threading.Timer + awaited
DisposeAsync. Prevents an in-flight 8s clear-callback from
invoking StateHasChanged on a component whose hub has already
been released.
- PublishHealthSnapshot deduplicates against the last published
(state, lastSuccess, lastError, errorCount) fingerprint. The
30s heartbeat no longer floods the SignalR layer with identical
Healthy snapshots — newly-joined clients still warm up via the
snapshot store on JoinDriver.
- RestartDriver / ReconnectDriver messages + AdminOperationsActor
handlers (broadcast via driver-control DPS topic; audited via
ConfigEdits).
- DriverHostActor subscribes to driver-control; locates the
matching child DriverInstanceActor and stops+respawns it
(Restart) or sends it a ForceReconnect internal message
(Reconnect — re-enters Reconnecting state without full stop).
DriverInstanceSpec constructor call uses named args to handle
the full 6-parameter signature.
- New DriverOperator authorization policy mapped to DriverOperator
or FleetAdmin role; documented in docs/security.md. Map LDAP
group via GroupToRole (e.g. "ot-driver-operator": "DriverOperator").
- DriverStatusPanel renders Reconnect + Restart buttons when the
user holds the DriverOperator policy (hidden otherwise). Restart
requires an in-page Razor confirm block (no JS confirm, keeps
SignalR event loop unblocked). Both buttons show a spinner and
are disabled during in-flight; result chip auto-clears after 8s.
Username sourced from AuthenticationStateProvider.
Reconnect resolves to "ForceReconnect" (re-enter Reconnecting,
not full stop+respawn) — transport drops and retries while actor
and in-memory state are preserved. All DriverInstanceActor states
handle ForceReconnect safely (no-op when already in transition).
- IDriverProbe abstraction in Core.Abstractions; one impl per driver
type, resolved by DriverType string. Phase 7.3 + 7.4 add concrete
probes for the 9 supported driver types.
- TestDriverConnect / TestDriverConnectResult messages.
- AdminOperationsActor.HandleTestDriverConnectAsync looks up the probe
by DriverType, runs it with a [1,60]s clamped timeout, and returns
success/latency or failure/message. Probes that throw or time out
surface as soft failures.
Final F15 batch wires up the SignalR-backed live pages, ports the bulk
equipment importer, and progressively enhances the Script source editor
with Monaco.
Message contracts:
- Commons.Messages.Alerts.AlarmTransitionEvent — fires on every alarm
state transition; published on the `alerts` DPS topic by future
ScriptedAlarmActor (F9) emits.
- Commons.Messages.Logging.ScriptLogEntry — one log line emitted by a
hosted script; published on the `script-logs` DPS topic by future
VirtualTagActor (F8) + ScriptedAlarmActor (F9) emits.
(Folder named "Logging" to dodge .gitignore's "logs/" rule.)
SignalR plumbing:
- AlertHub gains MethodName + bridge actor (AlertSignalRBridge)
- ScriptLogHub introduced; ScriptLogSignalRBridge follows the same
DPS-subscribe → IHubContext fan-out pattern as FleetStatusSignalRBridge
- WithOtOpcUaSignalRBridges now spawns all three bridges
- MapOtOpcUaHubs maps /hubs/script-log alongside the existing hubs
Pages:
- /alerts live alarm tail, 200-row capacity
- /script-log live script-log tail with level + script
filter, 500-row capacity
- /clusters/{id}/equipment/import — CSV bulk Equipment add with preview
(Name/MachineCode/UnsLineId/Driver +
optional ZTag/SAPID/Manufacturer/Model;
skips rows whose MachineCode already
exists in the fleet)
- ScriptEdit progressively enhanced with Monaco editor via JSInterop —
the textarea remains Blazor's source of truth and Monaco syncs into it
on every keystroke so @bind keeps working; falls back gracefully if
the CDN is unreachable.
MainLayout nav gains a "Live" section (Deployments, Alerts, Alarms
historian) and a "Scripts" link under Scripting. ClusterEquipment
surfaces the new Import CSV button.
Tally: F15 ships ~42 razor pages + 3 SignalR hubs + 3 bridge actors.
Microsoft.AspNetCore.SignalR.Client added (was already in central PM).
All 104 v2 tests remain green.
- New Commons.Messages.Fleet.GetDiagnostics request record.
- DriverHostActor handles GetDiagnostics in all three states (Steady, Applying,
Stale); replies with a NodeDiagnosticsSnapshot built from _currentRevision
+ the local NodeId. Drivers list is empty until F7 wires the per-instance
children.
- FleetDiagnosticsClient now resolves the target via ActorSelection at
akka.tcp://{system}@{nodeId}/user/driver-host and Asks with a 3s timeout.
On timeout/peer-down it returns an empty snapshot so the UI degrades
gracefully rather than throwing.
Two new integration tests in Host.IntegrationTests:
- GetDiagnostics_returns_snapshot_with_target_NodeId verifies the
cross-node Ask/Reply works.
- GetDiagnostics_after_deploy_reports_current_revision exercises the
end-to-end path: AdminOps starts a deployment, both DriverHostActors
apply, then diagnostics reports the new revision on both nodes.
All 98 v2 tests pass (was 96 + 2 new).
Adds the empty project skeletons that subsequent v2 tasks fill in:
src/Core/ZB.MOM.WW.OtOpcUa.Commons (types, interfaces, message contracts)
src/Core/ZB.MOM.WW.OtOpcUa.Cluster (Akka.Hosting + cluster wiring)
src/Server/ZB.MOM.WW.OtOpcUa.Security (cookie+JWT auth, LDAP)
src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane (admin-role cluster singletons)
src/Server/ZB.MOM.WW.OtOpcUa.Runtime (per-node driver actors)
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer (OPC UA SDK application host)
src/Server/ZB.MOM.WW.OtOpcUa.AdminUI (Razor class library)
src/Server/ZB.MOM.WW.OtOpcUa.Host (single fused web binary)
Each project sets TreatWarningsAsErrors=true in its own csproj (per the
Directory.Build.props deviation note in the previous commit). NuGetAuditSuppress
entries cover transitive vulnerability advisories the new strictness surfaces:
- GHSA-g94r-2vxg-569j (OpenTelemetry.Api 1.9.0 via Akka.Cluster.Hosting/Tools)
- GHSA-h958-fxgg-g7w3 (Opc.Ua.Core 1.5.374.126 via OpcUaServer)
- GHSA-37gx-xxp4-5rgx + GHSA-w3x6-4m5h-cxqf (legacy advisories already accepted)
OpcUaServer pins OPCFoundation.NetStandard.Opc.Ua.Configuration to 1.5.374.126
via VersionOverride to match Opc.Ua.Server's transitive Opc.Ua.Core (same
constraint as the legacy Server project).
Runtime does NOT project-reference any concrete Driver.* assemblies; drivers
load reflectively at runtime (Phase 6). Runtime gets the IDriver contract
through Core.Abstractions instead.
Host's Microsoft.Extensions.Hosting.WindowsServices is conditional on the
Windows OS so the project builds on macOS dev machines.
Build verification: dotnet build -> 438 warnings (all pre-existing xUnit1051
in legacy Server.Tests/Admin.Tests), 0 errors. Closes Task 9 (build green
smoke check, no separate commit).
Joseph Doherty