549c656489
A deploy that triggers a FULL address-space rebuild clears `_alarmConditions` and re-materialises every condition node fresh — inactive, acked, confirmed, unshelved. The engine reloads its persisted state and re-derives Active from the predicate, so it ends up correct; but there is no transition to report, so `LoadAsync` yields `EmissionKind.None` and `OnEngineEmission` drops it. Nothing writes the node. Live-reproduced on the docker-dev rig against the pre-fix image: an alarm that had fired and cleared but was still UNACKNOWLEDGED came back from a rebuild reporting Acknowledged with Retain=false — so it disappeared from ConditionRefresh entirely, while the engine still held Unacknowledged. An operator's outstanding alarm silently vanishes from the alarm list on deploy. `ScriptedAlarmHostActor.ReassertConditionNodes` closes it. After each load it reads the new `ScriptedAlarmEngine.GetProjections()` and sends one `AlarmStateUpdate` per alarm NOT in the Part 9 no-event position. Load-bearing properties: - Node-only. It sends `AlarmStateUpdate` and nothing else — never the `alerts` topic, never the telemetry hub. `alerts` is the historization path, so a row per alarm per deploy would append a duplicate historian/AVEVA record every time anyone deploys. This is why it reads a projection rather than re-emitting: `ScriptedAlarmProjection` carries no `EmissionKind`, so there is nothing for the emission machinery — or a future refactor — to mistake for a transition. The issue's sketch said `GetState(alarmId)` + `LoadedAlarmIds`; that alone would have clobbered the node's severity and message, so the projection also carries the definition's severity, the resolved message template, and the last-observed worst-input quality. - Only non-normal alarms. One in the no-event position already matches what the materialise built, so a never-fired alarm stays byte-identical to a deploy without this pass (including its Message, which the materialise seeds with the display name), and an all-normal deploy writes nothing at all. - Timestamp is the condition's own `LastTransitionUtc`, not the deploy instant. - No duplicate Part 9 events: `WriteAlarmCondition`'s delta-gate compares against the node's CURRENT state, so a re-assert onto a freshly materialised node is a genuine delta (fires once — correct, the rebuild reset what clients see) while one onto a surgically-preserved node is suppressed. Tests: 6 new host tests + 6 new engine tests. Falsifiability checked by disabling the call — 4 of the 6 host tests go red; the other 2 are absence guards against an over-broad fix and pass either way by design. Live gate (docker-dev, central-2, pre-fix image vs. rebuilt image): - pre-fix: condition absent from ConditionRefresh after a rebuild; - post-fix: present, Unacknowledged, Retain=True, stamped with its own LastTransitionUtc rather than the restart instant; - /alerts stayed empty across two re-asserts, with the panel proven live by a real ACTIVATED transition immediately afterwards — no duplicate history.
181 lines
8.2 KiB
C#
181 lines
8.2 KiB
C#
using Serilog;
|
|
using Shouldly;
|
|
using Xunit;
|
|
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
|
using ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms;
|
|
using ZB.MOM.WW.OtOpcUa.Core.Scripting;
|
|
|
|
namespace ZB.MOM.WW.OtOpcUa.Core.ScriptedAlarms.Tests;
|
|
|
|
/// <summary>
|
|
/// #487 — <see cref="ScriptedAlarmEngine.GetProjections"/>: the read that lets a caller restore an
|
|
/// alarm's condition node after a full address-space rebuild, without waiting for the alarm's next
|
|
/// transition (which for a still-active alarm with static dependencies may never come).
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// The load that motivates this is deliberately the one that emits NOTHING: an alarm whose persisted
|
|
/// state is already Active and whose predicate is still true produces <see cref="EmissionKind.None"/>,
|
|
/// so the transition stream cannot carry its state. These tests pin both halves — the projection does
|
|
/// report the state, and reading it does not fire an emission.
|
|
/// </remarks>
|
|
[Trait("Category", "Unit")]
|
|
public sealed class ScriptedAlarmProjectionTests
|
|
{
|
|
private static ScriptedAlarmDefinition Alarm(
|
|
string id = "HighTemp",
|
|
string predicate = """return (int)ctx.GetTag("Temp").Value > 100;""",
|
|
string msg = "Temp is {Temp}",
|
|
AlarmSeverity sev = AlarmSeverity.High) =>
|
|
new(AlarmId: id,
|
|
EquipmentPath: "Plant/Line1/Reactor",
|
|
AlarmName: id,
|
|
Kind: AlarmKind.AlarmCondition,
|
|
Severity: sev,
|
|
MessageTemplate: msg,
|
|
PredicateScriptSource: predicate);
|
|
|
|
/// <summary>An already-Active persisted state, as a store would hold it across a restart/redeploy.</summary>
|
|
private static AlarmConditionState PersistedActive(
|
|
string alarmId = "HighTemp",
|
|
AlarmAckedState acked = AlarmAckedState.Unacknowledged,
|
|
DateTime? lastTransitionUtc = null) =>
|
|
new(alarmId,
|
|
AlarmEnabledState.Enabled,
|
|
AlarmActiveState.Active,
|
|
acked,
|
|
AlarmConfirmedState.Confirmed,
|
|
ShelvingState.Unshelved,
|
|
lastTransitionUtc ?? DateTime.UtcNow,
|
|
LastActiveUtc: lastTransitionUtc ?? DateTime.UtcNow,
|
|
LastClearedUtc: null,
|
|
LastAckUtc: null, LastAckUser: null, LastAckComment: null,
|
|
LastConfirmUtc: null, LastConfirmUser: null, LastConfirmComment: null,
|
|
Comments: []);
|
|
|
|
private static ScriptedAlarmEngine Build(FakeUpstream up, IAlarmStateStore store)
|
|
{
|
|
var logger = new LoggerConfiguration().CreateLogger();
|
|
return new ScriptedAlarmEngine(up, store, new ScriptLoggerFactory(logger), logger);
|
|
}
|
|
|
|
/// <summary>The core of #487: a reload of a still-active alarm emits NOTHING, yet the projection
|
|
/// reports it Active — so a caller has a way to restore the node the rebuild reset.</summary>
|
|
[Fact]
|
|
public async Task A_still_active_alarm_emits_nothing_on_reload_but_projects_Active()
|
|
{
|
|
var up = new FakeUpstream();
|
|
up.Set("Temp", 150); // predicate still true — no transition to report
|
|
var store = new InMemoryAlarmStateStore();
|
|
await store.SaveAsync(PersistedActive(), TestContext.Current.CancellationToken);
|
|
|
|
using var eng = Build(up, store);
|
|
var emissions = new List<ScriptedAlarmEvent>();
|
|
eng.OnEvent += (_, e) => { lock (emissions) emissions.Add(e); }; // subscribe BEFORE the load
|
|
|
|
await eng.LoadAsync([Alarm()], TestContext.Current.CancellationToken);
|
|
|
|
// The premise. If this ever starts emitting, the bug's shape has changed and the re-assert
|
|
// in ScriptedAlarmHostActor may have become a duplicate rather than the only source of truth.
|
|
lock (emissions)
|
|
{
|
|
emissions.ShouldBeEmpty("a still-active alarm has no transition to report on reload");
|
|
}
|
|
|
|
var projection = eng.GetProjections().ShouldHaveSingleItem();
|
|
projection.AlarmId.ShouldBe("HighTemp");
|
|
projection.Condition.Active.ShouldBe(AlarmActiveState.Active);
|
|
projection.Condition.Acked.ShouldBe(AlarmAckedState.Unacknowledged);
|
|
}
|
|
|
|
/// <summary>The projection carries the definition's severity and the message template resolved against
|
|
/// the live value cache — so a re-asserted node is indistinguishable from one a real transition wrote.</summary>
|
|
[Fact]
|
|
public async Task Projection_carries_definition_severity_and_a_resolved_message()
|
|
{
|
|
var up = new FakeUpstream();
|
|
up.Set("Temp", 150);
|
|
var store = new InMemoryAlarmStateStore();
|
|
await store.SaveAsync(PersistedActive(), TestContext.Current.CancellationToken);
|
|
|
|
using var eng = Build(up, store);
|
|
await eng.LoadAsync([Alarm(sev: AlarmSeverity.Critical)], TestContext.Current.CancellationToken);
|
|
|
|
var projection = eng.GetProjections().ShouldHaveSingleItem();
|
|
projection.Severity.ShouldBe(AlarmSeverity.Critical);
|
|
projection.Message.ShouldBe("Temp is 150", "the template resolves against the current value cache");
|
|
}
|
|
|
|
/// <summary>Reading projections is a pure read — it must never raise <see cref="ScriptedAlarmEngine.OnEvent"/>,
|
|
/// or every deploy would append a duplicate historian / alerts row.</summary>
|
|
[Fact]
|
|
public async Task Reading_projections_never_fires_an_emission()
|
|
{
|
|
var up = new FakeUpstream();
|
|
up.Set("Temp", 150);
|
|
var store = new InMemoryAlarmStateStore();
|
|
await store.SaveAsync(PersistedActive(), TestContext.Current.CancellationToken);
|
|
|
|
using var eng = Build(up, store);
|
|
await eng.LoadAsync([Alarm()], TestContext.Current.CancellationToken);
|
|
|
|
var emissions = 0;
|
|
eng.OnEvent += (_, _) => Interlocked.Increment(ref emissions);
|
|
|
|
for (var i = 0; i < 5; i++) eng.GetProjections().ShouldNotBeEmpty();
|
|
|
|
Volatile.Read(ref emissions).ShouldBe(0);
|
|
}
|
|
|
|
/// <summary>An alarm that is genuinely normal projects the no-event position, so a caller can tell
|
|
/// "nothing to restore" from "restore this".</summary>
|
|
[Fact]
|
|
public async Task A_normal_alarm_projects_the_no_event_position()
|
|
{
|
|
var up = new FakeUpstream();
|
|
up.Set("Temp", 50); // predicate false
|
|
using var eng = Build(up, new InMemoryAlarmStateStore());
|
|
|
|
await eng.LoadAsync([Alarm()], TestContext.Current.CancellationToken);
|
|
|
|
var projection = eng.GetProjections().ShouldHaveSingleItem();
|
|
projection.Condition.Active.ShouldBe(AlarmActiveState.Inactive);
|
|
projection.Condition.Acked.ShouldBe(AlarmAckedState.Acknowledged);
|
|
projection.Condition.Enabled.ShouldBe(AlarmEnabledState.Enabled);
|
|
projection.Condition.Shelving.Kind.ShouldBe(ShelvingKind.Unshelved);
|
|
}
|
|
|
|
/// <summary>The worst-input quality rides the projection, so restoring a node whose script inputs are
|
|
/// Bad does not clobber its Quality back to Good (#478's invariant, on the re-assert path).</summary>
|
|
[Fact]
|
|
public async Task Projection_carries_the_last_observed_worst_input_quality()
|
|
{
|
|
var up = new FakeUpstream();
|
|
up.Set("Temp", 150);
|
|
var store = new InMemoryAlarmStateStore();
|
|
await store.SaveAsync(PersistedActive(), TestContext.Current.CancellationToken);
|
|
|
|
using var eng = Build(up, store);
|
|
await eng.LoadAsync([Alarm()], TestContext.Current.CancellationToken);
|
|
eng.GetProjections().ShouldHaveSingleItem().WorstInputStatusCode.ShouldBe(0u, "inputs start Good");
|
|
|
|
up.Push("Temp", 150, 0x80000000u); // input goes Bad — condition freezes, quality is annotated
|
|
|
|
// The push re-evaluates on a background worker; poll for the tracked bucket to move.
|
|
var deadline = DateTime.UtcNow.AddSeconds(10);
|
|
while (eng.GetProjections()[0].WorstInputStatusCode == 0u && DateTime.UtcNow < deadline)
|
|
{
|
|
await Task.Delay(20, TestContext.Current.CancellationToken);
|
|
}
|
|
|
|
(eng.GetProjections()[0].WorstInputStatusCode >> 30).ShouldBe(2u, "a Bad input projects Bad quality");
|
|
}
|
|
|
|
/// <summary>Before the first load there is nothing to project — the read is safe, not a throw.</summary>
|
|
[Fact]
|
|
public void Projections_are_empty_before_the_first_load()
|
|
{
|
|
using var eng = Build(new FakeUpstream(), new InMemoryAlarmStateStore());
|
|
eng.GetProjections().ShouldBeEmpty();
|
|
}
|
|
}
|