fix(site-runtime): seed republished overrides into Script Actors without firing scripts; alarms still evaluate (review 2026-10-21 01/SR-U1, decision M1)

HandleOverridesLoaded now sends children a site-local AttributeValueSeed
(stream still gets the AttributeValueChanged). ScriptActor updates its
snapshot, Conditional truth value and Expression last result (seed-only eval,
never fires); AlarmActor evaluates it as a normal change.
This commit is contained in:
Joseph Doherty
2026-10-22 09:03:10 -04:00
parent 999653a6b9
commit 3c11510c52
6 changed files with 174 additions and 14 deletions
+1 -1
View File
@@ -239,7 +239,7 @@ flowchart TD
> **Routed multi-attribute reads and writes are one message each (review 2026-10-21 01/SR-P4).** `Route.To().GetAttributes()` / `SetAttributes()` used to cost one Ask — a temp actor, a mailbox round trip and a 30 s timer — per attribute. The Deployment Manager now sends the Instance Actor a single site-local `InstanceGetAttributesRequest` / `InstanceSetAttributesRequest` (`SiteRuntime/Messages/InstanceAttributeBatch.cs`, never on the wire). Reads are answered in one pass from one consistent point in the mailbox. Writes are applied in order exactly as the equivalent single `SetStaticAttributeCommand`s (static → in-memory + coalesced persist, data-sourced → DCL write, unknown/poison values rejected); data-sourced writes are issued concurrently and the reply waits only for those, each still bounded by its own 30 s DCL Ask (the batch Ask allows 35 s so a device timeout surfaces as that attribute's own failure). Both replies carry the per-attribute `GetAttributeResponse` / `SetStaticAttributeResponse`, so the routed response is unchanged: a failed device write or unknown attribute is still reported by name.
> **Loaded overrides are republished to the children and the stream (review 2026-10-21 01/SR-U1).** The override load is asynchronous and lands *after* `PreStart` has already created the Script/Alarm Actors, whose trigger snapshots are seeded from the attribute set at that moment — the deploy **defaults**. Static attributes never re-emit a change on their own, so before this fix every trigger on an instance that restarted, failed over, was enabled or was started by the reconcile sweep evaluated an overridden attribute at its default indefinitely (an Expression alarm on `Mode == "Auto"` never raised), and the Debug View live stream showed the default, while `GetAttribute` answered correctly. The load now republishes each override that changed the in-memory value or quality as an ordinary `AttributeValueChanged` (quality from the load, no `SourceExecutionId`) through the same fan-out every change uses, with waiter evaluation **off** — nothing can legitimately be waiting on a replay. Consequence, accepted deliberately: a ValueChange/Conditional script keyed on an overridden static attribute fires once as the actor starts, exactly as one keyed on a data-sourced attribute already fires on that attribute's first DCL value. A static write that lands before the load is newer than the stored row, so the load skips that attribute rather than rolling it back. Overrides for attributes the deployed config no longer defines are still applied (unchanged) but not republished.
> **Loaded overrides are republished to the children and the stream (review 2026-10-21 01/SR-U1).** The override load is asynchronous and lands *after* `PreStart` has already created the Script/Alarm Actors, whose trigger snapshots are seeded from the attribute set at that moment — the deploy **defaults**. Static attributes never re-emit a change on their own, so before this fix every trigger on an instance that restarted, failed over, was enabled or was started by the reconcile sweep evaluated an overridden attribute at its default indefinitely (an Expression alarm on `Mode == "Auto"` never raised), and the Debug View live stream showed the default, while `GetAttribute` answered correctly. The load now republishes each override that changed the in-memory value or quality (quality from the load, no `SourceExecutionId`), with waiter evaluation **off** — nothing can legitimately be waiting on a replay. The site stream / Debug View receive it as an ordinary `AttributeValueChanged`; the children receive it as a site-local **`AttributeValueSeed`** (decision 2026-10-22): **Script Actors adopt the value without running** — the snapshot, a Conditional trigger's truth value and an Expression trigger's last result (evaluated, never fired) are brought up to date, so a restart/failover never re-fires ValueChange/Conditional/Expression scripts and the next real change is judged against the real value; a WhileTrue trigger whose condition already holds is seeded true without starting its re-fire timer. **Alarm Actors evaluate the seed like any change**, so an alarm whose condition already holds raises. A static write that lands before the load is newer than the stored row, so the load skips that attribute rather than rolling it back. Overrides for attributes the deployed config no longer defines are still applied (unchanged) but not republished.
> **Stopping an instance answers its pending waits (review 2026-10-21 01/SR-M3).** `PostStop` cancels every registered `WaitForAttribute` timeout and replies `Matched=false, TimedOut=false, ErrorMessage="… abandoned: instance stopped"`, so a script parked in `Attributes.WaitAsync` across a redeploy/disable/delete learns why immediately instead of burning its whole wait plus the Ask timeout and reporting a time-out, and no `WaitForAttributeTimeout` lands in dead letters (a monitored health metric).
@@ -246,6 +246,9 @@ public class AlarmActor : ReceiveActor
// Handle attribute value changes
Receive<AttributeValueChanged>(HandleAttributeValueChanged);
// Review 2026-10-21 01/SR-U1: an override seeded at instance start is evaluated like any
// change — an alarm whose condition already holds must raise.
Receive<Messages.AttributeValueSeed>(seed => HandleAttributeValueChanged(seed.Change));
// Handle alarm execution completion (also releases the in-flight slot)
Receive<AlarmExecutionCompleted>(HandleAlarmExecutionCompleted);
@@ -1831,24 +1831,32 @@ public class InstanceActor : ReceiveActor
/// otherwise an "any-change" waiter (or a waiter whose target equals the unchanged
/// value) would spuriously fire even though nothing actually changed.
/// </param>
private void PublishAndNotifyChildren(AttributeValueChanged changed, bool evaluateWaiters = true)
/// <param name="seedOnly">
/// Review 2026-10-21 01/SR-U1: children receive an <see cref="AttributeValueSeed"/> instead of
/// the change — Script Actors adopt the value without launching a run, Alarm Actors evaluate
/// it normally. The site stream still gets the ordinary change.
/// </param>
private void PublishAndNotifyChildren(
AttributeValueChanged changed, bool evaluateWaiters = true, bool seedOnly = false)
{
// Publish to site-wide stream
_streamManager?.PublishAttributeValueChanged(changed);
object toChildren = seedOnly ? new AttributeValueSeed(changed) : changed;
// Route the change to exactly the children interested in it (P2): the all-changes
// bucket (Expression scripts/alarms + fail-open children) plus the children monitoring
// this specific attribute. Children still gate on their own trigger config — routing
// is a fan-out optimization, not the correctness check.
foreach (var child in _allChangeChildren)
{
child.Tell(changed);
child.Tell(toChildren);
}
if (_childrenByMonitoredAttribute.TryGetValue(changed.AttributeName, out var interested))
{
foreach (var child in interested)
{
child.Tell(changed);
child.Tell(toChildren);
}
}
@@ -1942,9 +1950,10 @@ public class InstanceActor : ReceiveActor
/// evaluated against the default for the actor's whole lifetime while <c>GetAttribute</c>
/// answered with the override. Routing each changed value through
/// <see cref="PublishAndNotifyChildren"/> re-seeds every child snapshot and the site stream in
/// one place. Consequence, deliberately accepted: a ValueChange/Conditional script keyed on an
/// overridden static attribute fires once as the actor starts — exactly as one keyed on a
/// data-sourced attribute already fires on that attribute's first DCL value.
/// one place. The children get it as an <see cref="AttributeValueSeed"/> (user decision M1):
/// Script Actors adopt the value and their trigger state WITHOUT running — a restart must not
/// re-fire ValueChange/Conditional/Expression scripts — while Alarm Actors evaluate it, so an
/// alarm whose condition already holds raises.
/// </para>
/// <para>
/// Waiters are NOT evaluated: none can be waiting on a value the instance already held, and a
@@ -2035,7 +2044,8 @@ public class InstanceActor : ReceiveActor
new AttributeValueChanged(
_instanceUniqueName, name, name, _attributes[name],
_attributeQualities.GetValueOrDefault(name) ?? "Good", DateTimeOffset.UtcNow),
evaluateWaiters: false);
evaluateWaiters: false,
seedOnly: true);
}
_logger.LogDebug(
@@ -117,6 +117,8 @@ public class ScriptActor : ReceiveActor, IWithTimers
// against the latest snapshot. Both flags are touched only on the actor thread.
private bool _evalInFlight;
private bool _evalPending;
// 01/SR-U1: true when the pending evaluation was requested only by seeds (never fires).
private bool _evalPendingSeedOnly;
// WhileTrue trigger state: the most recent truth value of a Conditional
// trigger's comparison, used to detect false->true / true->false edges.
@@ -205,6 +207,8 @@ public class ScriptActor : ReceiveActor, IWithTimers
// Handle attribute value changes for value-change and conditional triggers
Receive<AttributeValueChanged>(HandleAttributeValueChanged);
// Review 2026-10-21 01/SR-U1: a value seeded at instance start updates trigger state only.
Receive<Messages.AttributeValueSeed>(HandleAttributeValueSeed);
// Handle interval tick
Receive<IntervalTick>(_ => TrySpawnExecution(null));
@@ -338,6 +342,31 @@ public class ScriptActor : ReceiveActor, IWithTimers
}
}
/// <summary>
/// Review 2026-10-21 01/SR-U1 (user decision M1): adopts a value the Instance Actor seeded at
/// start (a persisted static override loaded after this actor was created with the deploy
/// default) WITHOUT launching a run. The snapshot and the trigger's edge state are brought up
/// to date — a Conditional's truth value, an Expression's last result (evaluated, never fired) —
/// so the NEXT real change is judged against the real value: no spurious edge from the replay,
/// and no missed edge afterwards. A WhileTrue trigger whose condition already holds is seeded
/// true without starting its re-fire timer; it fires on its next false→true edge.
/// </summary>
private void HandleAttributeValueSeed(Messages.AttributeValueSeed seed)
{
var changed = seed.Change;
_attributeSnapshot[changed.AttributeName] = changed.Value;
if (_triggerConfig is ConditionalTriggerConfig conditional &&
conditional.AttributeName == changed.AttributeName)
{
_conditionState = EvaluateCondition(conditional, changed.Value);
}
else if (_triggerConfig is ExpressionTriggerConfig)
{
StartExpressionEvaluation(seedOnly: true);
}
}
/// <summary>
/// Starts an off-dispatcher evaluation of the compiled trigger expression (P1).
/// The expression previously ran synchronously on the actor's dispatcher thread
@@ -359,10 +388,20 @@ public class ScriptActor : ReceiveActor, IWithTimers
/// the work is queued — so gate-wait time burns the same budget and a saturated gate
/// yields a timely false instead of an unbounded stall.</para>
/// </summary>
private void StartExpressionEvaluation()
/// <param name="seedOnly">
/// 01/SR-U1: the result only seeds <see cref="_lastExpressionResult"/> and never fires. A
/// coalesced pending evaluation stays seed-only unless a real change also asked for it.
/// </param>
private void StartExpressionEvaluation(bool seedOnly = false)
{
if (_compiledTriggerExpression == null || _triggerConfig is not ExpressionTriggerConfig) return;
if (_evalInFlight) { _evalPending = true; return; } // coalesce bursts: one in flight, one pending
if (_evalInFlight)
{
// coalesce bursts: one in flight, one pending (seed-only only if every request was)
_evalPendingSeedOnly = _evalPending ? _evalPendingSeedOnly && seedOnly : seedOnly;
_evalPending = true;
return;
}
_evalInFlight = true;
var snapshot = new Dictionary<string, object?>(_attributeSnapshot); // point-in-time copy, actor thread
@@ -407,7 +446,7 @@ public class ScriptActor : ReceiveActor, IWithTimers
cts.Dispose();
}
}).PipeTo(self,
success: r => new ExpressionEvalResult(r),
success: r => new ExpressionEvalResult(r, seedOnly),
failure: ex => new ExpressionEvalFailed(ex));
}
@@ -422,13 +461,21 @@ public class ScriptActor : ReceiveActor, IWithTimers
_evalInFlight = false;
if (_triggerConfig is ExpressionTriggerConfig exprConfig)
{
if (exprConfig.Mode == TriggerMode.WhileTrue)
if (msg.SeedOnly)
{
// 01/SR-U1: a seed adopts the truth value and never fires.
}
else if (exprConfig.Mode == TriggerMode.WhileTrue)
HandleWhileTrueTransition(msg.Result, _lastExpressionResult);
else if (msg.Result && !_lastExpressionResult)
TrySpawnExecution(null);
_lastExpressionResult = msg.Result;
}
if (_evalPending) { _evalPending = false; StartExpressionEvaluation(); }
if (_evalPending)
{
_evalPending = false;
StartExpressionEvaluation(_evalPendingSeedOnly);
}
}
/// <summary>
@@ -873,7 +920,7 @@ public class ScriptActor : ReceiveActor, IWithTimers
/// carries the boolean truth value so the OnTrue/WhileTrue edge logic runs on
/// the actor thread.
/// </summary>
private sealed record ExpressionEvalResult(bool Result);
private sealed record ExpressionEvalResult(bool Result, bool SeedOnly = false);
/// <summary>
/// Piped back to self when the off-dispatcher evaluation TASK itself faults
@@ -0,0 +1,19 @@
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming;
namespace ZB.MOM.WW.ScadaBridge.SiteRuntime.Messages;
/// <summary>
/// Site-local: an attribute value an Instance Actor's children must ADOPT without treating it
/// as a change that fires scripts (review 2026-10-21 01/SR-U1, user decision M1). Sent in place of
/// a plain <see cref="AttributeValueChanged"/> when the persisted static overrides load at actor
/// start — the children were created before that load and seeded with the deploy defaults.
///
/// <para>
/// Script Actors update their attribute snapshot and trigger state (Conditional truth value,
/// Expression last result) but do NOT launch a run; Alarm Actors evaluate it exactly as a normal
/// change, so an alarm whose condition already holds raises. The site stream / Debug View still
/// receive the ordinary <see cref="Change"/>.
/// </para>
/// </summary>
/// <param name="Change">The value being seeded, in the ordinary change shape.</param>
public sealed record AttributeValueSeed(AttributeValueChanged Change);
@@ -172,4 +172,85 @@ public class InstanceActorOverrideRepublishTests : TestKit, IDisposable
Assert.False(response.TimedOut);
Assert.Contains("instance stopped", response.ErrorMessage);
}
private static ResolvedScript FiringScript(string name, string triggerType, object triggerConfig, string flag) => new()
{
CanonicalName = name,
TriggerType = triggerType,
TriggerConfiguration = JsonSerializer.Serialize(triggerConfig),
Code = $"await Instance.SetAttribute(\"{flag}\", \"1\"); return null;"
};
private string AttributeOf(IActorRef instance, string instanceName, string attribute)
{
instance.Tell(new GetAttributeRequest("g", instanceName, attribute, DateTimeOffset.UtcNow));
return ExpectMsg<GetAttributeResponse>(TimeSpan.FromSeconds(15)).Value?.ToString() ?? "";
}
/// <summary>
/// User decision M1: an override seeded at actor start updates trigger state but does NOT run
/// ValueChange / Conditional / Expression scripts; an Expression ALARM still raises; and a
/// later real change fires all three scripts.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task SeededOverride_RaisesAlarm_ButFiresNoScript_UntilARealChange()
{
const string name = "PumpSeed";
await _storage.SetStaticOverrideAsync(name, "Level", "50");
const string over20 = "int.Parse(Attributes[\"Level\"].ToString()) > 20";
var config = new FlattenedConfiguration
{
InstanceUniqueName = name,
Attributes =
[
new ResolvedAttribute { CanonicalName = "Level", Value = "10", DataType = "Int32" },
new ResolvedAttribute { CanonicalName = "FiredVC", Value = "0", DataType = "String" },
new ResolvedAttribute { CanonicalName = "FiredCond", Value = "0", DataType = "String" },
new ResolvedAttribute { CanonicalName = "FiredExpr", Value = "0", DataType = "String" }
],
Scripts =
[
FiringScript("VC", "ValueChange", new { attributeName = "Level" }, "FiredVC"),
FiringScript("Cond", "Conditional",
new { attributeName = "Level", @operator = ">", threshold = 20 }, "FiredCond"),
FiringScript("Expr", "Expression", new { expression = over20 }, "FiredExpr")
],
Alarms =
[
new ResolvedAlarm
{
CanonicalName = "LevelHigh", TriggerType = "Expression", PriorityLevel = 500,
TriggerConfiguration = JsonSerializer.Serialize(new { expression = over20 })
}
]
};
var instance = StartInstance(name, config);
// The alarm raises from the seeded value alone — no other write.
AwaitAssert(() => Assert.Equal(AlarmState.Active, AlarmStateOf(instance, name, "LevelHigh")),
TimeSpan.FromSeconds(30), TimeSpan.FromMilliseconds(100));
Assert.Equal("50", AttributeOf(instance, name, "Level"));
await Task.Delay(TimeSpan.FromSeconds(1.5));
Assert.Equal("0", AttributeOf(instance, name, "FiredVC"));
Assert.Equal("0", AttributeOf(instance, name, "FiredCond"));
Assert.Equal("0", AttributeOf(instance, name, "FiredExpr"));
// Real changes fire them: down (Expression edge resets) then up again.
instance.Tell(new SetStaticAttributeCommand("w1", name, "Level", "5", DateTimeOffset.UtcNow));
ExpectMsg<SetStaticAttributeResponse>(TimeSpan.FromSeconds(15));
await Task.Delay(500);
instance.Tell(new SetStaticAttributeCommand("w2", name, "Level", "60", DateTimeOffset.UtcNow));
ExpectMsg<SetStaticAttributeResponse>(TimeSpan.FromSeconds(15));
AwaitAssert(() =>
{
Assert.Equal("1", AttributeOf(instance, name, "FiredVC"));
Assert.Equal("1", AttributeOf(instance, name, "FiredCond"));
Assert.Equal("1", AttributeOf(instance, name, "FiredExpr"));
}, TimeSpan.FromSeconds(20), TimeSpan.FromMilliseconds(200));
}
}