fix(scripting): only timeouts are quarantine strikes; alarm Bad annotation once per bucket change (fix round 1 I1, I3)

- I1 ScriptRunGuard: a run refused because the previous one is still
  executing is a Bad publish plus otopcua.script.run_refused, and no longer
  a quarantine strike. Only a timeout (and an allocation breach) counts;
  a refusal neither adds to nor resets the timeout streak; a completed run
  (value or throw) resets it. On a 1 Hz trigger with a 2 s budget a single
  4 s overrun is one timeout plus two refusals and must not quarantine.
  Tests updated across the guard, the alarm engine, the Calculation
  evaluator and the VT evaluator, plus
  Refusals_neither_count_toward_nor_reset_the_timeout_streak.
- I3 ScriptedAlarmEngine: the quality bucket is tracked only after the
  predicate completes (or in the inputs-not-ready path), so the tracked
  bucket is the last one annotated. A refused or quarantined predicate
  used to re-track the input bucket Good before the attempt and then flip
  it Bad, emitting QualityChanged on every push. The run-guard tests now
  count emissions (exactly 1 across N pushes, refused and quarantined
  cases), with a healthy sibling's dispatch count as the positive control
  that every push was processed. Both were red before the fix.

Refs #734

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
This commit is contained in:
Joseph Doherty
2026-10-22 13:26:12 -04:00
parent 909b061d3a
commit 6aa5edaf30
10 changed files with 161 additions and 67 deletions
@@ -76,7 +76,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
/// <summary>
/// Per-alarm run quarantine (keyed by alarm id, source identity = predicate source). Refuses a
/// predicate run while the alarm's previous (timed-out, abandoned) run is still executing and
/// quarantines after three consecutive non-completions or an allocation-budget breach — one orphan
/// quarantines after three consecutive timeouts (a refusal is not a strike) or an allocation-budget breach — one orphan
/// per alarm instead of one per upstream push. Reset on every <see cref="LoadAsync"/> (a redeploy).
/// </summary>
private readonly ScriptRunGuard _guard;
@@ -798,7 +798,6 @@ public sealed class ScriptedAlarmEngine : IDisposable
// short-circuit so an outright-Bad input is still observed. A bucket change with no state
// transition is delivered out of band as a QualityChanged emission (see below).
var worstStatus = WorstInputStatus(scratch.ReadCache);
var qualityBucketChanged = TrackQualityBucket(state.Definition.AlarmId, worstStatus);
// Cold-start guard — skip the predicate when any referenced upstream tag has no
// cached value yet (the upstream subscription hasn't delivered its first push).
@@ -811,7 +810,7 @@ public sealed class ScriptedAlarmEngine : IDisposable
// The condition is frozen (can't trust its state), but its source quality just changed
// bucket — annotate it out of band so a comms-lost / Bad-input scripted condition reports
// Bad, mirroring the native OT path.
if (qualityBucketChanged)
if (TrackQualityBucket(state.Definition.AlarmId, worstStatus))
pendingEmissions.Add(BuildQualityEmission(state, seed, worstStatus));
return seed;
}
@@ -851,6 +850,12 @@ public sealed class ScriptedAlarmEngine : IDisposable
}
scratch.LastNonCompletion = null;
// Track the bucket only once the predicate has COMPLETED (I3). The tracked bucket is the last
// bucket this alarm was annotated with; tracking the input bucket before the attempt flipped it to
// Good on every push, so a refused / quarantined predicate (which annotates Bad) re-emitted
// QualityChanged on every push instead of once per real bucket change.
var qualityBucketChanged = TrackQualityBucket(state.Definition.AlarmId, worstStatus);
var result = Part9StateMachine.ApplyPredicate(seed, predicateTrue, nowUtc);
var transition = result.Emission != EmissionKind.None
? BuildEmission(state, result.State, result.Emission, worstStatus)