fix(alarms): #647 review — Raise on an active condition is a re-statement; FOCAS outage quality + health

MAJOR-1: a deploy that changes a driver's config respawns it, and the fresh instance's empty told
record re-sent Raise for every still-active alarm, un-acking acknowledged conditions and writing a
duplicate "Activated" alerts row. NativeAlarmProjector now treats Raise on an already-active
condition as a re-statement (keeps Acked; ForwardNativeAlarm skips the alerts row); only Retrigger
(or inactive->active) un-acks. Sources whose upstream ack state is authoritative send Retrigger for
an unacked re-statement: OpcUaClient on first sight (no prior told), Galaxy for an unacked active
snapshot entry. Runtime E2E drives a FRESH FOCAS driver after a redeploy and asserts the ack survives.

MAJOR-2: a failed FOCAS alarm read annotates each subscribed condition BadNoCommunication once per
outage (#626 quality-only path) with one Warning (MINOR-6), lifted to Good on recovery; failed alarm
reads degrade driver health with LastSuccessfulRead frozen (successful ones advance it), so an
alarm-only driver trips the #527 comms-loss watchdog.

MINOR-3 CLAUDE.md #630 bullet; MINOR-4 split line; MINOR-5 a non-empty cnc_rdalmmsg2 payload that
fits neither stride throws; MINOR-7 value-tag + parsed-address tables are built and swapped, never
cleared in place under the retained poll engine. Docs corrected (FOCAS.md, AlarmTracking.md, CLAUDE.md).

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-08 20:18:03 -04:00
parent 482bafdfac
commit 89961af6e6
17 changed files with 516 additions and 66 deletions
@@ -1877,12 +1877,14 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers, IWithUnboundedS
foreach (var n in nodeIds)
{
var nodeId = n.NodeId;
var snapshot = _nativeAlarmProjector.Project(nodeId, msg.Args);
var snapshot = _nativeAlarmProjector.Project(nodeId, msg.Args, out var restatement);
_opcUaPublishActor.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.AlarmStateUpdate(
nodeId, snapshot, msg.Args.SourceTimestampUtc, n.Realm));
// Only the Primary publishes the cluster-wide alerts transition (see the gate above).
if (!serviceAlertsAsPrimary) continue;
// Only the Primary publishes the cluster-wide alerts transition (see the gate above). A re-statement
// (Gitea #647 review: a Raise on an already-active condition, sent by a driver a deploy respawned) is
// not a transition — the condition was already Active, so a second "Activated" row would be a duplicate.
if (!serviceAlertsAsPrimary || restatement) continue;
var meta = _alarmMetaByNodeId.TryGetValue(nodeId, out var m)
? m : (EquipmentId: nodeId, Name: nodeId, AlarmType: "AlarmCondition", HistorizeToAveva: (bool?)null,
@@ -18,13 +18,29 @@ public sealed class NativeAlarmProjector
/// <param name="nodeId">The materialised condition node's id (the projection's state key).</param>
/// <param name="e">The native alarm transition.</param>
/// <returns>The full Part 9 condition snapshot to write to the node.</returns>
public AlarmConditionSnapshot Project(string nodeId, AlarmEventArgs e)
public AlarmConditionSnapshot Project(string nodeId, AlarmEventArgs e) => Project(nodeId, e, out _);
/// <summary>
/// Project an alarm transition, reporting whether it was a <b>re-statement</b> (Gitea #647 review): a
/// <see cref="AlarmTransitionKind.Raise"/> on a condition already projected Active. That is what a driver
/// whose told-record died with its instance sends after a deploy respawns it — the alarm it re-reads is the
/// one the server already shows, so the acknowledgement is KEPT and the host writes no second "Activated"
/// row. Only <see cref="AlarmTransitionKind.Retrigger"/> (or inactive → active) un-acknowledges; a source
/// that knows the alarm is a genuinely new statement must send Retrigger.
/// </summary>
/// <param name="nodeId">The materialised condition node's id (the projection's state key).</param>
/// <param name="e">The native alarm transition.</param>
/// <param name="restatement">True when <paramref name="e"/> re-stated an already-active condition.</param>
/// <returns>The full Part 9 condition snapshot to write to the node.</returns>
public AlarmConditionSnapshot Project(string nodeId, AlarmEventArgs e, out bool restatement)
{
var prev = _prior.TryGetValue(nodeId, out var p) ? p : (Active: false, Acked: true);
// #626: a source with a finer scale than the four buckets supplies the exact Part 9 severity.
var sev = e.OpcUaSeverity is { } exact ? (ushort)Math.Clamp(exact, 1, 1000) : MapSeverity(e.Severity);
restatement = e.Kind == AlarmTransitionKind.Raise && prev.Active;
var (active, acked) = e.Kind switch
{
AlarmTransitionKind.Raise when prev.Active => (true, prev.Acked),
AlarmTransitionKind.Raise or AlarmTransitionKind.Retrigger => (true, false),
AlarmTransitionKind.Acknowledge => (prev.Active, true),
AlarmTransitionKind.Clear => (false, prev.Acked),