dohertj2/lmxopcua
1
0
Fork 0
Code Issues 36 Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix(alarms): engine cache prune + bounded dispose, source Warning, cached dead-letter depth, script-log rate limit (E3)

Browse Source 
- R05:1/12 ScriptedAlarmEngine: _valueCache is cleared in LoadAsync (it is
  re-seeded for the referenced paths) so retired tags no longer accrete across
  deploys; OnUpstreamChange no longer writes the cache after Dispose.
- R05:3 ScriptedAlarmEngine.Dispose drains in-flight work for at most 5 s
  (DisposeDrainTimeout), then cancels it through a shutdown token every
  DrainDirtyAsync/ShelvingCheckAsync now runs on, logs a Warning and gives
  one second of grace — a stalled IAlarmStateStore.SaveAsync can no longer
  hang host shutdown. A save that completes inside the bound is still drained
  (positive control test).
- R05:6 ScriptedAlarmSource logs a throwing OnAlarmEvent subscriber at
  Warning (naming the alarm) instead of swallowing it.
- R05:7 LocalDbStoreAndForwardSink keeps a cached dead-letter counter in step
  with every dead-letter/revive/purge path and resyncs it with the queued
  counter on the same cadence; GetStatus no longer runs a COUNT(*) scan.
- R05:9 ScriptLogRateLimiter (20 per source per minute, fixed window, one
  summary entry naming the dropped count when the window rolls) behind
  ScriptLogTopicSink (key = script|vtag|alarm) and ScriptLogCompanionSink
  (key = script name).

Refs #734

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
This commit is contained in:
Joseph Doherty
2026-10-22 10:17:30 -04:00
parent 2e9160317f
commit 6cbf763719
10 changed files with 683 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/Core/ZB.MOM.WW.OtOpcUa.Core.Scripting/ScriptLogTopicSink.cs
+41 -8
View File
@@ -25,25 +25,37 @@ namespace ZB.MOM.WW.OtOpcUa.Core.Scripting;
/// property so pipelines that pre-date this sink continue to work correctly. If
/// neither property is present <c>ScriptId</c> defaults to <c>"unknown"</c>.
/// </para>
/// <para>
/// <b>Rate-limited per source.</b> A script that logs at Information on every evaluation of a 1 Hz
/// input put one rendered string per second per script onto the cluster bus; 200 such scripts shed
/// everyone's entries out of the bounded telemetry channels. A <see cref="ScriptLogRateLimiter"/>
/// keyed by the entry's identity (script + virtual tag + alarm) admits 20 entries per minute per
/// source and publishes one summary entry naming the dropped count when the window rolls.
/// </para>
/// </remarks>
public sealed class ScriptLogTopicSink : ILogEventSink
{
private readonly IScriptLogPublisher _publisher;
private readonly LogEventLevel _min;
private readonly ScriptLogRateLimiter _limiter;
/// <summary>Initializes a new instance of the <see cref="ScriptLogTopicSink"/> class.</summary>
/// <param name="publisher">The publisher that routes entries to the cluster bus. Must not be <c>null</c>.</param>
/// <param name="min">Minimum log level to forward. Events below this level are discarded (default: <see cref="LogEventLevel.Information"/>).</param>
/// <param name="limiter">Per-source rate limiter; null ⇒ the default 20 per minute.</param>
/// <exception cref="ArgumentNullException">Thrown when <paramref name="publisher"/> is <c>null</c>.</exception>
public ScriptLogTopicSink(IScriptLogPublisher publisher, LogEventLevel min = LogEventLevel.Information)
public ScriptLogTopicSink(IScriptLogPublisher publisher, LogEventLevel min = LogEventLevel.Information,
ScriptLogRateLimiter? limiter = null)
{
_publisher = publisher ?? throw new ArgumentNullException(nameof(publisher));
_min = min;
_limiter = limiter ?? new ScriptLogRateLimiter();
}
/// <summary>
/// Converts the <paramref name="logEvent"/> to a <see cref="ScriptLogEntry"/> and
/// publishes it, unless the event is below the configured minimum level.
/// publishes it, unless the event is below the configured minimum level or the source is over its
/// per-minute budget.
/// </summary>
/// <param name="logEvent">The Serilog event to process. Silently ignored when <c>null</c>.</param>
public void Emit(LogEvent logEvent)
@@ -56,15 +68,36 @@ public sealed class ScriptLogTopicSink : ILogEventSink
? s
: null;
var scriptId = P(ScriptLoggerFactory.ScriptIdProperty)
?? P(ScriptLoggerFactory.ScriptNameProperty)
?? "unknown";
var virtualTagId = P(ScriptLoggerFactory.VirtualTagIdProperty);
var alarmId = P(ScriptLoggerFactory.AlarmIdProperty);
var equipmentId = P(ScriptLoggerFactory.EquipmentIdProperty);
var admission = _limiter.Admit(string.Concat(scriptId, "|", virtualTagId, "|", alarmId));
if (admission.SuppressedInPreviousWindow > 0)
{
_publisher.Publish(new ScriptLogEntry(
ScriptId: scriptId,
Level: nameof(LogEventLevel.Warning),
Message: $"{admission.SuppressedInPreviousWindow} script-log entr{(admission.SuppressedInPreviousWindow == 1 ? "y" : "ies")} " +
$"from this source were suppressed in the previous {_limiter.Window.TotalSeconds:F0} s " +
$"(limit {ScriptLogRateLimiter.DefaultLimitPerWindow}/window); see scripts-*.log for the full stream",
TimestampUtc: logEvent.Timestamp.UtcDateTime,
VirtualTagId: virtualTagId,
AlarmId: alarmId,
EquipmentId: equipmentId));
}
if (!admission.Admitted) return;
_publisher.Publish(new ScriptLogEntry(
ScriptId: P(ScriptLoggerFactory.ScriptIdProperty)
?? P(ScriptLoggerFactory.ScriptNameProperty)
?? "unknown",
ScriptId: scriptId,
Level: logEvent.Level.ToString(),
Message: logEvent.RenderMessage(),
TimestampUtc: logEvent.Timestamp.UtcDateTime,
VirtualTagId: P(ScriptLoggerFactory.VirtualTagIdProperty),
AlarmId: P(ScriptLoggerFactory.AlarmIdProperty),
EquipmentId: P(ScriptLoggerFactory.EquipmentIdProperty)));
VirtualTagId: virtualTagId,
AlarmId: alarmId,
EquipmentId: equipmentId));
}
}