dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs(comments): strip internal task/milestone/bundle bookkeeping from code comments

Browse Source 
Remove project bookkeeping citations from shipped code comments across the
solution: hyphenated task IDs (WP-14, StoreAndForward-025), milestone/task/
issue refs (M3, Task 4, Audit Log #23, #21), Bundle X task-bundle labels,
and C/D/K/S/T phase labels.

Comment text only — no code logic, string/log literals, or XML-doc structure
changed. Genuine descriptions are preserved (only the citation is stripped),
and technical lookalikes are retained (UTF-8, SHA-256, T00:00:00, M365,
UTC-5, pre-C4/pre-C5 schema versions). Flagged by the new CommentChecker
TaskReferenceInComment / TrackingReferenceInComment checks plus targeted
grep passes; full solution builds clean, append-only guard tests pass.
This commit is contained in:
Joseph Doherty
2026-07-07 11:03:26 -04:00
parent 67005ca4c0
commit 9cff87fe85
435 changed files with 2338 additions and 2547 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/IStoreAndForwardSiteContext.cs
+2 -2
View File
@@ -3,7 +3,7 @@ namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// Optional ambient site context the Store-and-Forward service consults at
/// construction time. Carries the site identifier the S&amp;F retry loop
/// stamps onto cached-call audit telemetry (Audit Log #23 / M3 Bundle F).
/// stamps onto cached-call audit telemetry.
/// </summary>
/// <remarks>
/// <para>
@@ -17,7 +17,7 @@ namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// The Host registers a trivial adapter that forwards to the same
/// <c>NodeOptions.SiteId</c> the existing <c>ISiteIdentityProvider</c> reads.
/// Resolution is optional: when no binding is registered the S&amp;F service
/// stamps an empty site id, preserving the legacy pre-M3 behaviour exactly.
/// stamps an empty site id, preserving the legacy behaviour exactly.
/// </para>
/// </remarks>
public interface IStoreAndForwardSiteContext
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/NotificationForwarder.cs
+2 -14
View File
@@ -50,7 +50,7 @@ public sealed class NotificationForwarder
/// failure. Sourced from host configuration.
/// </param>
/// <param name="logger">
/// Optional logger. StoreAndForward-018: a corrupt buffered payload is logged at
/// Optional logger. A corrupt buffered payload is logged at
/// Warning before being discarded so an operator has a forensic trail of the row
/// that vanished from the buffer.
/// </param>
@@ -76,18 +76,6 @@ public sealed class NotificationForwarder
/// <returns>A task that resolves to <c>true</c> when central accepts (or the payload is corrupt and discarded); throws on a transient forward failure so the engine retries.</returns>
public async Task<bool> DeliverAsync(StoreAndForwardMessage message)
{
// StoreAndForward-018: an unreadable payload cannot be fixed by retrying.
// The design doc explicitly forbids parking notifications ("notifications do
// not park — they are retried at the fixed forward interval until central
// acks"; Component-StoreAndForward.md). The earlier behaviour returned false
// here, which the S&F engine interprets as a permanent failure and parks
// the row — contradicting the invariant and surfacing the row in the
// central UI's parked-message list. The correct outcome for a corrupt-payload
// notification is to DISCARD: log a Warning with the buffered row id +
// payload preview for forensics, then return true so the engine clears the
// buffer via its standard success-path cleanup. The buffered row is
// unrecoverable; retrying or parking would both make the queue worse, not
// better.
if (!TryBuildSubmit(message, out var submit))
{
_logger.LogWarning(
@@ -122,7 +110,7 @@ public sealed class NotificationForwarder
/// forwarded to central, returning <c>false</c> if the payload is unreadable.
///
/// The buffered payload IS a serialized <see cref="NotificationSubmit"/> written by
/// the site <c>Notify.Send</c> enqueue path (Task 19). Its
/// the site <c>Notify.Send</c> enqueue path. Its
/// <see cref="NotificationSubmit.NotificationId"/> is the central idempotency key —
/// it was generated by the script, equals the buffered row's
/// <see cref="StoreAndForwardMessage.Id"/>, and is stable across every retry. The
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/ParkedMessageHandlerActor.cs
+3 -6
View File
@@ -30,7 +30,7 @@ public class ParkedMessageHandlerActor : ReceiveActor
Receive<ParkedMessageRetryRequest>(HandleRetry);
Receive<ParkedMessageDiscardRequest>(HandleDiscard);
// Task 5 (#22): central→site Retry/Discard relay for parked cached
// Central→site Retry/Discard relay for parked cached
// operations. The cached call's S&F buffer message id is the
// TrackedOperationId, so these reuse the same parked-message primitive
// as HandleRetry/HandleDiscard, keyed off the tracked id.
@@ -43,9 +43,6 @@ public class ParkedMessageHandlerActor : ReceiveActor
var sender = Sender;
var siteId = _siteId;
// StoreAndForward-007: idiomatic PipeTo with explicit success/failure
// projections instead of ContinueWith. Both projections touch only locals
// (captured before the await), so they are safe to run off the actor thread.
_service.GetParkedMessagesAsync(category: null, msg.PageNumber, msg.PageSize)
.PipeTo(
sender,
@@ -103,7 +100,7 @@ public class ParkedMessageHandlerActor : ReceiveActor
}
/// <summary>
/// Task 5 (#22): executes a central-relayed Retry of a parked cached call.
/// Executes a central-relayed Retry of a parked cached call.
/// The tracked id is the S&amp;F buffer message id, so this reuses
/// <see cref="StoreAndForwardService.RetryParkedMessageAsync"/> — which only
/// touches rows that are actually <c>Parked</c> (a non-parked or unknown
@@ -125,7 +122,7 @@ public class ParkedMessageHandlerActor : ReceiveActor
}
/// <summary>
/// Task 5 (#22): executes a central-relayed Discard of a parked cached call.
/// Executes a central-relayed Discard of a parked cached call.
/// Mirrors <see cref="HandleRetryParkedOperation"/>; Discard removes the
/// parked S&amp;F buffer row (only when it is actually <c>Parked</c>).
/// </summary>
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/ReplicationService.cs
+9 -13
View File
@@ -4,7 +4,7 @@ using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// WP-11: Async replication of buffer operations to standby node.
/// Async replication of buffer operations to standby node.
///
/// - Forwards add/remove/park operations to standby via a replication handler.
/// - No ack wait (fire-and-forget per design).
@@ -39,7 +39,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11: Replicates an enqueue operation to standby (fire-and-forget).
/// Replicates an enqueue operation to standby (fire-and-forget).
/// </summary>
/// <param name="message">The message that was enqueued on the active node.</param>
public void ReplicateEnqueue(StoreAndForwardMessage message)
@@ -53,7 +53,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11: Replicates a remove operation to standby (fire-and-forget).
/// Replicates a remove operation to standby (fire-and-forget).
/// </summary>
/// <param name="messageId">The identifier of the message to remove from the standby buffer.</param>
public void ReplicateRemove(string messageId)
@@ -67,7 +67,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11: Replicates a park operation to standby (fire-and-forget).
/// Replicates a park operation to standby (fire-and-forget).
/// </summary>
/// <param name="message">The message that was parked on the active node.</param>
public void ReplicatePark(StoreAndForwardMessage message)
@@ -81,7 +81,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11 / StoreAndForward-016: Replicates an operator-initiated requeue (a parked
/// Replicates an operator-initiated requeue (a parked
/// message moved back to the pending queue) to standby (fire-and-forget). The
/// carried message reflects the active node's post-requeue state (Pending,
/// retry_count = 0) so the standby's copy can be brought into sync.
@@ -98,7 +98,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11: Applies a replicated operation received from the active node.
/// Applies a replicated operation received from the active node.
/// Used by the standby node to keep its SQLite in sync.
/// </summary>
/// <param name="operation">The replication operation to apply.</param>
@@ -124,9 +124,6 @@ public class ReplicationService
break;
case ReplicationOperationType.Requeue when operation.Message != null:
// StoreAndForward-016: an operator retried a parked message on the
// active node; mirror that on the standby by moving its row back to
// Pending with retry_count = 0 so a failover preserves the retry.
operation.Message.Status = StoreAndForwardMessageStatus.Pending;
operation.Message.RetryCount = 0;
await storage.UpdateMessageAsync(operation.Message);
@@ -144,7 +141,6 @@ public class ReplicationService
}
catch (Exception ex)
{
// WP-11: No ack wait — log and move on
_logger.LogDebug(ex,
"Replication of {OpType} for message {MessageId} failed (best-effort)",
operation.OperationType, operation.MessageId);
@@ -154,7 +150,7 @@ public class ReplicationService
}
/// <summary>
/// WP-11: Represents a buffer operation to be replicated to standby.
/// Represents a buffer operation to be replicated to standby.
/// </summary>
public record ReplicationOperation(
ReplicationOperationType OperationType,
@@ -162,7 +158,7 @@ public record ReplicationOperation(
StoreAndForwardMessage? Message);
/// <summary>
/// WP-11: Types of buffer operations that are replicated.
/// Types of buffer operations that are replicated.
/// </summary>
public enum ReplicationOperationType
{
@@ -170,7 +166,7 @@ public enum ReplicationOperationType
Remove,
Park,
/// <summary>
/// StoreAndForward-016: an operator moved a parked message back to the pending
/// An operator moved a parked message back to the pending
/// queue. The standby resets its matching row to Pending with retry_count = 0.
/// </summary>
Requeue
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/ServiceCollectionExtensions.cs
+3 -8
View File
@@ -30,11 +30,11 @@ public static class ServiceCollectionExtensions
var options = sp.GetRequiredService<IOptions<StoreAndForwardOptions>>().Value;
var logger = sp.GetRequiredService<ILogger<StoreAndForwardService>>();
var replication = sp.GetRequiredService<ReplicationService>();
// Audit Log #23 (M3 Bundle F): Wire the cached-call lifecycle
// Wire the cached-call lifecycle
// observer + site identity through DI so the S&F retry loop emits
// per-attempt + terminal telemetry under the same TrackedOperationId
// the script-thread CachedSubmit row used. Both bindings are
// optional — when null the legacy pre-M3 retry behaviour is
// optional — when null the legacy retry behaviour is
// preserved exactly (tests, central nodes without sites, hosts
// that haven't called AddAuditLog).
//
@@ -44,13 +44,8 @@ public static class ServiceCollectionExtensions
// ISiteIdentityProvider — HealthMonitoring already references S&F.
var cachedCallObserver = sp.GetService<ICachedCallLifecycleObserver>();
var siteContext = sp.GetService<IStoreAndForwardSiteContext>();
// StoreAndForward-023: pass null/empty through unchanged — the
// service constructor normalises it to UnknownSiteSentinel so a
// host without an IStoreAndForwardSiteContext registration is
// observable in the central audit log instead of producing a
// silent empty-string SourceSite.
var siteId = siteContext?.SiteId ?? string.Empty;
// M1.7: optional site operational-event log. Resolved through
// Optional site operational-event log. Resolved through
// GetService so a host (or test) that has not called
// AddSiteEventLogging simply gets null and the S&F activity stays
// a no-op for site-event purposes.
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardMessage.cs
+6 -7
View File
@@ -3,7 +3,7 @@ using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// WP-9: Represents a single store-and-forward message as stored in SQLite.
/// Represents a single store-and-forward message as stored in SQLite.
/// Maps to the sf_messages table.
/// </summary>
public class StoreAndForwardMessage
@@ -11,7 +11,7 @@ public class StoreAndForwardMessage
/// <summary>Unique message ID (GUID).</summary>
public string Id { get; set; } = string.Empty;
/// <summary>WP-9: Category: ExternalSystem, Notification, or CachedDbWrite.</summary>
/// <summary>Category: ExternalSystem, Notification, or CachedDbWrite.</summary>
public StoreAndForwardCategory Category { get; set; }
/// <summary>Target system name (external system, notification list, or DB connection).</summary>
@@ -23,7 +23,7 @@ public class StoreAndForwardMessage
/// <summary>
/// Number of retry-sweep attempts performed so far. The initial (immediate or
/// caller-made) delivery attempt is attempt 0 and is not counted here; this
/// field counts only background retry attempts (StoreAndForward-003).
/// field counts only background retry attempts.
/// </summary>
public int RetryCount { get; set; }
@@ -52,12 +52,11 @@ public class StoreAndForwardMessage
/// <summary>
/// Instance that originated this message (for S&amp;F-survives-delete behavior).
/// WP-13: Messages are NOT cleared when instance is deleted.
/// </summary>
public string? OriginInstanceName { get; set; }
/// <summary>
/// Audit Log #23 (ExecutionId Task 4): the originating script execution's
/// The originating script execution's
/// per-run correlation id, threaded from <c>ScriptRuntimeContext</c> through
/// the cached-call enqueue path. Carried so the store-and-forward retry loop
/// can stamp it onto the per-attempt / terminal cached-call audit rows
@@ -69,7 +68,7 @@ public class StoreAndForwardMessage
public Guid? ExecutionId { get; set; }
/// <summary>
/// Audit Log #23 (ExecutionId Task 4): the originating script identifier,
/// The originating script identifier,
/// threaded alongside <see cref="ExecutionId"/> from the cached-call enqueue
/// path so the retry-loop audit rows carry the same <c>SourceScript</c>
/// provenance the script-side cached rows already carry. <c>null</c> when not
@@ -78,7 +77,7 @@ public class StoreAndForwardMessage
public string? SourceScript { get; set; }
/// <summary>
/// Audit Log #23 (ParentExecutionId Task 6): the <c>ExecutionId</c> of the
/// The <c>ExecutionId</c> of the
/// inbound-API request that spawned the originating script execution,
/// threaded alongside <see cref="ExecutionId"/> from the cached-call enqueue
/// path. Carried so the store-and-forward retry loop can stamp it onto the
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardOptions.cs
+5 -19
View File
@@ -1,39 +1,25 @@
namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// WP-9/10: Configuration options for the Store-and-Forward Engine.
/// Configuration options for the Store-and-Forward Engine.
/// </summary>
public class StoreAndForwardOptions
{
/// <summary>Path to the SQLite database for S&amp;F message persistence.</summary>
public string SqliteDbPath { get; set; } = "./data/store-and-forward.db";
/// <summary>WP-11: Whether to replicate buffer operations to standby node.</summary>
/// <summary>Whether to replicate buffer operations to standby node.</summary>
public bool ReplicationEnabled { get; set; } = true;
/// <summary>WP-10: Default retry interval for messages without per-source settings.</summary>
/// <summary>Default retry interval for messages without per-source settings.</summary>
public TimeSpan DefaultRetryInterval { get; set; } = TimeSpan.FromSeconds(30);
/// <summary>
/// WP-10: Default maximum retry count before parking. Applied when an
/// Default maximum retry count before parking. Applied when an
/// <c>EnqueueAsync</c> caller does not pass an explicit <c>maxRetries</c>.
/// <para>
/// <b>StoreAndForward-019:</b> this default is enforced uniformly across
/// every category, including <see cref="Commons.Types.Enums.StoreAndForwardCategory.Notification"/>:
/// once the buffered message's retry count reaches this cap the engine
/// parks the row. The Component-StoreAndForward.md "notifications do not
/// park" wording reflects the operational <i>intent</i> when central is
/// reachable on the normal cadence; under a sustained central outage that
/// exceeds <c>DefaultMaxRetries × forward-interval</c> a buffered
/// notification <i>will</i> park and surface in the parked-message UI,
/// matching the rest of the system's bounded-retry-then-park behaviour.
/// Callers that genuinely require unbounded retry must pass
/// <c>maxRetries: 0</c> on <c>EnqueueAsync</c> (the documented "no limit"
/// escape hatch — see <c>StoreAndForwardService.EnqueueAsync</c>).
/// </para>
/// </summary>
public int DefaultMaxRetries { get; set; } = 50;
/// <summary>WP-10: Interval for the background retry timer sweep.</summary>
/// <summary>Interval for the background retry timer sweep.</summary>
public TimeSpan RetryTimerInterval { get; set; } = TimeSpan.FromSeconds(10);
}
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardService.cs
+67 -173
View File
@@ -8,7 +8,7 @@ using ZB.MOM.WW.ScadaBridge.SiteEventLogging;
namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// WP-9/10: Core store-and-forward service.
/// Core store-and-forward service.
///
/// Lifecycle:
/// 1. Caller attempts immediate delivery via IDeliveryHandler
@@ -18,14 +18,14 @@ namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// message is retried until delivered and is never parked for retry exhaustion)
/// 5. Permanent failures are returned to caller immediately (never buffered)
///
/// WP-10: Fixed retry interval (not exponential). Per-source-entity retry settings.
/// Fixed retry interval (not exponential). Per-source-entity retry settings.
/// Background timer-based retry sweep.
///
/// WP-12: Parked messages queryable, retryable, and discardable.
/// Parked messages queryable, retryable, and discardable.
///
/// WP-14: Buffer depth reported as health metric. Activity logged to site event log.
/// Buffer depth reported as health metric. Activity logged to site event log.
///
/// WP-15: CachedCall idempotency is the caller's responsibility.
/// CachedCall idempotency is the caller's responsibility.
/// This service does not deduplicate — if the same message is enqueued twice,
/// it will be delivered twice. Callers using ExternalSystem.CachedCall() must
/// design their payloads to be idempotent (e.g., include unique request IDs
@@ -38,14 +38,14 @@ public class StoreAndForwardService
private readonly ReplicationService? _replication;
private readonly ILogger<StoreAndForwardService> _logger;
/// <summary>
/// Audit Log #23 (M3 Bundle E — Task E4): site-side observer notified
/// Site-side observer notified
/// after every cached-call delivery attempt. Optional — when null no
/// telemetry is emitted; the legacy pre-M3 retry loop behaviour is
/// telemetry is emitted; the legacy retry loop behaviour is
/// preserved exactly.
/// </summary>
private readonly ICachedCallLifecycleObserver? _cachedCallObserver;
/// <summary>
/// M1.7: optional site operational-event log. When non-null the service maps
/// Optional site operational-event log. When non-null the service maps
/// its own buffer/retry/park activity (the same activity that drives
/// <see cref="OnActivity"/>) onto site events — <c>store_and_forward</c> for the
/// cached-call categories and <c>notification</c> for the site's
@@ -54,26 +54,14 @@ public class StoreAndForwardService
/// </summary>
private readonly ISiteEventLogger? _siteEventLogger;
/// <summary>
/// Audit Log #23 (M3 Bundle E — Task E4): site id stamped onto the
/// Site id stamped onto the
/// cached-call attempt context so the audit bridge can build the
/// <see cref="SiteCallOperational"/> half of the telemetry packet.
/// <para>
/// <b>StoreAndForward-023:</b> an empty-string site id must never reach
/// downstream consumers — the central audit pipeline keys
/// <c>(SourceSite, TrackedOperationId)</c> off this value, so an empty
/// string degrades correlation to a per-id-only index and breaks the
/// per-site routing of <c>RetryParkedOperation</c>/<c>DiscardParkedOperation</c>
/// commands. The constructor normalises a null/empty/whitespace
/// <paramref name="siteId"/> argument to <see cref="UnknownSiteSentinel"/>
/// so a misconfigured host (no <c>IStoreAndForwardSiteContext</c>
/// registered) produces a distinctive marker in the central audit log
/// rather than silently merging multiple sites into the empty bucket.
/// </para>
/// </summary>
private readonly string _siteId;
/// <summary>
/// StoreAndForward-023: distinctive marker stamped onto cached-call audit
/// Distinctive marker stamped onto cached-call audit
/// telemetry when the host has not registered an
/// <see cref="IStoreAndForwardSiteContext"/>. Chosen with a leading <c>$</c>
/// so it cannot collide with a real site id (which is a configuration
@@ -84,7 +72,7 @@ public class StoreAndForwardService
public const string UnknownSiteSentinel = "$unknown-site";
/// <summary>
/// M1.7: the detail-string prefix written by <see cref="EnqueueAsync"/>
/// The detail-string prefix written by <see cref="EnqueueAsync"/>
/// when an immediate forward attempt throws and the message is buffered for
/// the retry sweep. <see cref="EmitSiteEvent"/> matches on this same prefix
/// to distinguish a forward <i>failure</i> (logged) from a routine
@@ -98,7 +86,7 @@ public class StoreAndForwardService
private int _retryInProgress;
/// <summary>
/// StoreAndForward-024: the in-flight retry sweep <see cref="Task"/>, or
/// The in-flight retry sweep <see cref="Task"/>, or
/// <c>null</c> when no sweep is currently running. Captured when the timer
/// callback starts a sweep so <see cref="StopAsync"/> can wait for it to
/// finish before the host disposes downstream dependencies
@@ -110,7 +98,7 @@ public class StoreAndForwardService
private Task? _sweepTask;
/// <summary>
/// StoreAndForward-024: how long <see cref="StopAsync"/> waits for an
/// How long <see cref="StopAsync"/> waits for an
/// in-flight retry sweep to finish before returning. The default — 10 s —
/// is generous enough to let a typical sweep over the buffered queue drain,
/// but bounded so a hung downstream call (a stuck SQLite write, a
@@ -122,7 +110,7 @@ public class StoreAndForwardService
private static readonly TimeSpan SweepShutdownWaitTimeout = TimeSpan.FromSeconds(10);
/// <summary>
/// WP-14 (telemetry): cached count of messages currently buffered for
/// Cached count of messages currently buffered for
/// forwarding — i.e. rows in <see cref="StoreAndForwardMessageStatus.Pending"/>,
/// the live store-and-forward queue waiting to be delivered. This backs the
/// <c>scadabridge.store_and_forward.queue.depth</c> observable gauge.
@@ -145,7 +133,7 @@ public class StoreAndForwardService
private long _bufferedCount;
/// <summary>
/// Test seam (WP-14 telemetry): simulates a concurrent pre-seed
/// Test seam: simulates a concurrent pre-seed
/// <see cref="BufferAsync"/> increment landing on <see cref="_bufferedCount"/>
/// before <see cref="StartAsync"/> seeds it, so a test can prove the seed uses
/// <see cref="Interlocked.Add"/> (additive) rather than Exchange (clobbering).
@@ -154,7 +142,7 @@ public class StoreAndForwardService
Interlocked.Increment(ref _bufferedCount);
/// <summary>
/// WP-14 (telemetry): an instance field that guards against a single instance
/// An instance field that guards against a single instance
/// registering the queue-depth provider (and re-seeding the counter) more than
/// once — e.g. a second <see cref="StartAsync"/> on the same instance. It does NOT
/// coordinate across instances: the gauge slot in <see cref="ScadaBridgeTelemetry"/>
@@ -164,7 +152,7 @@ public class StoreAndForwardService
private int _queueDepthProviderRegistered;
/// <summary>
/// StoreAndForward-025: the exact provider delegate this instance registered with
/// The exact provider delegate this instance registered with
/// the process-global <see cref="ScadaBridgeTelemetry"/> gauge slot, retained so
/// <see cref="StopAsync"/> can deregister it by identity (compare-and-clear). Holding
/// the same reference both registers and clears the slot; the identity check ensures a
@@ -174,7 +162,7 @@ public class StoreAndForwardService
private Func<long>? _queueDepthProvider;
/// <summary>
/// WP-10: Delivery handler delegate. The return value / exception is interpreted
/// Delivery handler delegate. The return value / exception is interpreted
/// the same way on both the immediate-delivery path (<see cref="EnqueueAsync"/>)
/// and the background retry path (<c>RetryMessageAsync</c>):
/// <list type="bullet">
@@ -192,7 +180,7 @@ public class StoreAndForwardService
private readonly Dictionary<StoreAndForwardCategory, Func<StoreAndForwardMessage, Task<bool>>> _deliveryHandlers = new();
/// <summary>
/// WP-14: Event callback for logging S&amp;F activity to site event log.
/// Event callback for logging S&amp;F activity to site event log.
/// </summary>
public event Action<string, StoreAndForwardCategory, string>? OnActivity;
@@ -206,7 +194,7 @@ public class StoreAndForwardService
/// <param name="cachedCallObserver">Optional observer for cached call lifecycle events.</param>
/// <param name="siteId">The site identifier this service belongs to.</param>
/// <param name="siteEventLogger">
/// M1.7: optional site operational-event log. When non-null, buffer/retry/park
/// Optional site operational-event log. When non-null, buffer/retry/park
/// activity is mirrored to site events (<c>store_and_forward</c> /
/// <c>notification</c> by category). Optional with a <c>null</c> default so the
/// many direct-construction tests still compile unchanged.
@@ -225,17 +213,13 @@ public class StoreAndForwardService
_logger = logger;
_replication = replication;
_cachedCallObserver = cachedCallObserver;
// StoreAndForward-023: normalise an empty / whitespace site id to the
// distinctive UnknownSiteSentinel so downstream consumers (the central
// audit pipeline keying off SourceSite) never see an empty string and
// a misconfigured host is recognisable in the central log.
_siteId = string.IsNullOrWhiteSpace(siteId) ? UnknownSiteSentinel : siteId;
_siteEventLogger = siteEventLogger;
// M1.7: ride the existing activity hook to emit site operational events.
// Ride the existing activity hook to emit site operational events.
// RaiseActivity already isolates a throwing subscriber, so a failing
// event log can never be misclassified as a transient delivery failure
// (StoreAndForward-009). Only subscribe when a logger is wired so the
// event log can never be misclassified as a transient delivery failure.
// Only subscribe when a logger is wired so the
// legacy (test/central) construction path stays a no-op.
if (_siteEventLogger != null)
{
@@ -244,7 +228,7 @@ public class StoreAndForwardService
}
/// <summary>
/// M1.7: maps one store-and-forward activity to a site operational event,
/// Maps one store-and-forward activity to a site operational event,
/// following the Site Event Logging spec's per-category scope
/// (Component-SiteEventLogging.md §"Events Logged"):
/// <list type="bullet">
@@ -340,39 +324,17 @@ public class StoreAndForwardService
{
await _storage.InitializeAsync();
// WP-14 (telemetry): seed the cached buffered-message count from the
// store exactly once (the gauge callback cannot run an async COUNT), then
// register the sync, non-blocking provider with the process-global
// ScadaBridgeTelemetry gauge. Both steps are inside the one-time guard so a
// second StartAsync on the same instance cannot double-seed.
//
// The seed is an Interlocked.Add — NOT an Exchange — to avoid a startup race:
// between the await above returning and this point, a concurrent BufferAsync
// could already have Interlocked.Increment'd _bufferedCount. Exchange would
// clobber that increment (losing a +1); Add preserves it. _bufferedCount
// starts at 0 and only BufferAsync increments it before the seed, so
// 0 + pending + (any concurrent increments) is the correct live count.
var pending = await _storage.GetMessageCountByStatusAsync(
StoreAndForwardMessageStatus.Pending);
if (Interlocked.CompareExchange(ref _queueDepthProviderRegistered, 1, 0) == 0)
{
Interlocked.Add(ref _bufferedCount, pending);
// StoreAndForward-025: retain the exact delegate so StopAsync can
// deregister it by identity (compare-and-clear) without stomping a
// newer instance that may have re-registered into the global slot.
var provider = (Func<long>)(() => Interlocked.Read(ref _bufferedCount));
_queueDepthProvider = provider;
ScadaBridgeTelemetry.SetQueueDepthProvider(provider);
}
_retryTimer = new Timer(
// StoreAndForward-024: capture the sweep Task on each tick so
// StopAsync can await any in-flight invocation before the host
// disposes _storage/_replication underneath it. The RetryPending
// path is self-guarded against overlapping sweeps via the
// _retryInProgress Interlocked flag, so unconditionally re-assigning
// the field here cannot lose a still-running task (the new tick
// will short-circuit if one is already running).
_ => Volatile.Write(ref _sweepTask, RetryPendingMessagesAsync()),
null,
_options.RetryTimerInterval,
@@ -386,24 +348,6 @@ public class StoreAndForwardService
/// <summary>
/// Stops the background retry timer and waits (bounded) for any in-flight
/// retry sweep to finish before returning.
///
/// StoreAndForward-024: prior to this fix, <see cref="StopAsync"/> only
/// disposed the timer — a sweep already inside
/// <see cref="RetryPendingMessagesAsync"/> continued running against
/// <see cref="_storage"/> and <see cref="_replication"/> after this method
/// returned, and could then NRE / throw on a disposed dependency once the
/// DI container ran its own shutdown. We now await the captured sweep task
/// (with a bounded <see cref="SweepShutdownWaitTimeout"/> so a hung
/// dependency cannot block host shutdown indefinitely) before returning.
///
/// StoreAndForward-025: after the timer is disposed and the in-flight sweep has
/// drained, the queue-depth provider this instance registered with the process-global
/// <see cref="ScadaBridgeTelemetry"/> gauge is deregistered by identity (compare-and-
/// clear) — otherwise a stopped service would report a frozen depth forever and the
/// provider closure would pin this dead instance for the process lifetime. The clear is
/// identity-checked so a newer instance that already re-registered the global slot is
/// not stomped, and <see cref="_queueDepthProviderRegistered"/> is reset so a later
/// <see cref="StartAsync"/> on this instance re-registers cleanly.
/// </summary>
/// <returns>A task representing the asynchronous stop operation.</returns>
public async Task StopAsync()
@@ -445,33 +389,22 @@ public class StoreAndForwardService
}
}
// StoreAndForward-025: release the process-global queue-depth gauge provider so a
// stopped service stops reporting a frozen depth and the closure no longer pins
// this dead instance. Identity-checked (compare-and-clear) so a successor
// instance's provider is left intact; reset the one-time guard so a later
// StartAsync re-registers.
var provider = _queueDepthProvider;
if (provider is not null)
{
ScadaBridgeTelemetry.ClearQueueDepthProvider(provider);
_queueDepthProvider = null;
}
// StoreAndForward-028: reset the cached depth alongside the registration guard.
// StartAsync re-seeds _bufferedCount from the durable Pending count under this
// same guard; without resetting here, a same-instance Stop->Start would ADD the
// re-read count on top of the leftover in-memory value, double-counting the gauge
// (~2N). Reset to zero so the restart seeds from a clean base. Buffered rows
// remain durable in SQLite, so the re-seed restores the true count.
Interlocked.Exchange(ref _bufferedCount, 0);
Interlocked.Exchange(ref _queueDepthProviderRegistered, 0);
}
/// <summary>
/// WP-10: Enqueues a message for store-and-forward delivery.
/// Enqueues a message for store-and-forward delivery.
/// Attempts immediate delivery first. On transient failure, buffers for retry.
/// On permanent failure (handler returns false), returns false immediately.
///
/// WP-10: Retry-count lifecycle — the immediate (or caller-made) delivery attempt
/// Retry-count lifecycle — the immediate (or caller-made) delivery attempt
/// is attempt 0 and is not counted; the background retry sweep increments
/// <see cref="StoreAndForwardMessage.RetryCount"/> on each retry. A buffered
/// message is parked once <c>RetryCount</c> reaches <paramref name="maxRetries"/>
@@ -482,14 +415,14 @@ public class StoreAndForwardService
/// that want delivery abandoned after a bounded number of attempts must pass a
/// positive <paramref name="maxRetries"/>.
///
/// WP-15: CachedCall idempotency note — this method does not deduplicate.
/// CachedCall idempotency note — this method does not deduplicate.
/// The caller (e.g., ExternalSystem.CachedCall()) is responsible for ensuring
/// that the remote system can handle duplicate deliveries safely.
/// </summary>
/// <param name="category">Message category (selects the delivery handler).</param>
/// <param name="target">Target system name (external system / notification list / DB connection).</param>
/// <param name="payloadJson">JSON-serialized call payload, treated opaquely.</param>
/// <param name="originInstanceName">Instance that originated the message (WP-13: survives instance deletion).</param>
/// <param name="originInstanceName">Instance that originated the message (survives instance deletion).</param>
/// <param name="maxRetries">
/// Maximum background retry-sweep attempts before the message is parked.
/// <b><c>0</c> = no limit</b> — the message is retried on every sweep until
@@ -511,23 +444,23 @@ public class StoreAndForwardService
/// inside the payload, and it is the id the forwarder submits to central.
/// </param>
/// <param name="executionId">
/// Audit Log #23 (ExecutionId Task 4): the originating script execution's
/// The originating script execution's
/// per-run correlation id. Threaded onto the buffered row so the retry-loop
/// cached-call audit rows carry it. <c>null</c> for callers (notifications,
/// pre-Task-4 callers) that do not supply one.
/// cached-call audit rows carry it. <c>null</c> for callers (e.g. notifications)
/// that do not supply one.
/// </param>
/// <param name="sourceScript">
/// Audit Log #23 (ExecutionId Task 4): the originating script identifier,
/// The originating script identifier,
/// threaded onto the buffered row alongside <paramref name="executionId"/>
/// so the retry-loop audit rows carry the same provenance the script-side
/// cached rows do. <c>null</c> when not known.
/// </param>
/// <param name="parentExecutionId">
/// Audit Log #23 (ParentExecutionId Task 6): the <c>ExecutionId</c> of the
/// The <c>ExecutionId</c> of the
/// inbound-API request that spawned the originating script execution.
/// Threaded onto the buffered row alongside <paramref name="executionId"/>
/// so the retry-loop cached-call audit rows carry it. <c>null</c> for a
/// non-routed run and for callers (notifications, pre-Task-6 callers) that
/// non-routed run and for callers (e.g. notifications) that
/// do not supply one.
/// </param>
/// <returns>A task that resolves to a result indicating whether the message was delivered or buffered.</returns>
@@ -582,7 +515,7 @@ public class StoreAndForwardService
{
// Transient failure — buffer for retry. The immediate attempt is
// attempt 0; RetryCount tracks only sweep retries, so it stays 0
// here (StoreAndForward-003).
// here.
_logger.LogWarning(ex,
"Immediate delivery to {Target} failed (transient), buffering for retry",
target);
@@ -600,7 +533,7 @@ public class StoreAndForwardService
// delivery itself — buffer for the background retry sweep to deliver.
// The initial attempt (caller-made, or skipped because no handler is
// registered) is attempt 0; RetryCount tracks only sweep retries and
// therefore stays 0 here (StoreAndForward-003).
// therefore stays 0 here.
if (!attemptImmediateDelivery)
{
message.LastAttemptAt = DateTimeOffset.UtcNow;
@@ -613,21 +546,18 @@ public class StoreAndForwardService
}
/// <summary>
/// Persists a message to the local SQLite buffer and (WP-11) replicates the
/// Persists a message to the local SQLite buffer and replicates the
/// add to the standby node so a failover does not lose the buffered message.
/// </summary>
private async Task BufferAsync(StoreAndForwardMessage message)
{
await _storage.EnqueueAsync(message);
_replication?.ReplicateEnqueue(message);
// WP-14 (telemetry): a freshly buffered row is Pending → grows the live
// queue depth. Bumped after the durable write so the gauge never leads the
// store.
Interlocked.Increment(ref _bufferedCount);
}
/// <summary>
/// WP-10: Background retry sweep. Processes all pending messages that are due for retry.
/// Background retry sweep. Processes all pending messages that are due for retry.
/// </summary>
/// <returns>A task representing the asynchronous retry sweep.</returns>
internal async Task RetryPendingMessagesAsync()
@@ -666,7 +596,7 @@ public class StoreAndForwardService
return;
}
// Audit Log #23 (M3 Bundle E — Tasks E4/E5): measure per-attempt
// Measure per-attempt
// duration so the audit row carries a meaningful DurationMs. Captured
// around the handler invocation only — storage / replication overhead
// is excluded.
@@ -681,12 +611,11 @@ public class StoreAndForwardService
{
await _storage.RemoveMessageAsync(message.Id);
_replication?.ReplicateRemove(message.Id);
// WP-14 (telemetry): a delivered row leaves the Pending queue.
Interlocked.Decrement(ref _bufferedCount);
RaiseActivity("Delivered", message.Category,
$"Delivered to {message.Target} after {message.RetryCount} retries");
// M3: terminal Delivered observer notification — the audit
// Terminal Delivered observer notification — the audit
// bridge maps this to Attempted + CachedResolve(Delivered).
await NotifyCachedCallObserverAsync(
message,
@@ -699,9 +628,6 @@ public class StoreAndForwardService
}
// Permanent failure on retry — park immediately.
// StoreAndForward-005: the sweep observed this row as Pending; only commit
// the park if it is still Pending so a concurrent operator action that
// moved it (retry/discard) is not silently overwritten.
message.Status = StoreAndForwardMessageStatus.Parked;
message.LastAttemptAt = DateTimeOffset.UtcNow;
message.LastError = "Permanent failure (handler returned false)";
@@ -714,14 +640,12 @@ public class StoreAndForwardService
message.Id);
return;
}
// WP-14 (telemetry): the row committed Pending→Parked, leaving the live
// forward queue. Only counted when the conditional update actually won.
Interlocked.Decrement(ref _bufferedCount);
_replication?.ReplicatePark(message);
RaiseActivity("Parked", message.Category,
$"Permanent failure for {message.Target}: handler returned false");
// M3: terminal PermanentFailure observer notification — the
// Terminal PermanentFailure observer notification — the
// audit bridge maps this to Attempted(Failed) + CachedResolve(Parked).
await NotifyCachedCallObserverAsync(
message,
@@ -741,8 +665,6 @@ public class StoreAndForwardService
if (message.MaxRetries > 0 && message.RetryCount >= message.MaxRetries)
{
// StoreAndForward-005: conditional park — see the permanent-failure
// branch above for rationale.
message.Status = StoreAndForwardMessageStatus.Parked;
var parked = await _storage.UpdateMessageIfStatusAsync(
message, StoreAndForwardMessageStatus.Pending);
@@ -753,8 +675,6 @@ public class StoreAndForwardService
message.Id);
return;
}
// WP-14 (telemetry): the row committed Pending→Parked, leaving the
// live forward queue. Only counted when the conditional update won.
Interlocked.Decrement(ref _bufferedCount);
_replication?.ReplicatePark(message);
RaiseActivity("Parked", message.Category,
@@ -763,7 +683,7 @@ public class StoreAndForwardService
"Message {MessageId} parked after {MaxRetries} retries to {Target}",
message.Id, message.MaxRetries, message.Target);
// M3: terminal ParkedMaxRetries observer notification — the
// Terminal ParkedMaxRetries observer notification — the
// audit bridge maps this to Attempted(Failed) + CachedResolve(Parked).
await NotifyCachedCallObserverAsync(
message,
@@ -775,9 +695,6 @@ public class StoreAndForwardService
}
else
{
// StoreAndForward-005: the retry-count increment is also conditional
// on the row still being Pending so it cannot clobber an operator
// action that ran during the failed delivery.
if (!await _storage.UpdateMessageIfStatusAsync(
message, StoreAndForwardMessageStatus.Pending))
{
@@ -789,7 +706,7 @@ public class StoreAndForwardService
RaiseActivity("Retried", message.Category,
$"Retry {message.RetryCount}/{message.MaxRetries} for {message.Target}: {ex.Message}");
// M3: per-attempt TransientFailure observer notification —
// Per-attempt TransientFailure observer notification —
// the audit bridge maps this to Attempted(Failed).
await NotifyCachedCallObserverAsync(
message,
@@ -803,20 +720,20 @@ public class StoreAndForwardService
}
/// <summary>
/// Audit Log #23 (M3 Bundle E — Tasks E4/E5): notify the registered
/// Notify the registered
/// <see cref="ICachedCallLifecycleObserver"/> of the just-completed
/// attempt. Only fires for cached-call categories
/// (<see cref="StoreAndForwardCategory.ExternalSystem"/> and
/// <see cref="StoreAndForwardCategory.CachedDbWrite"/>); the
/// <see cref="StoreAndForwardCategory.Notification"/> category has its
/// own central-side audit pipeline (Notification Outbox / #21) and must
/// own central-side audit pipeline (Notification Outbox) and must
/// not surface on this hook.
/// </summary>
/// <remarks>
/// Best-effort: an observer that throws is logged and swallowed so a
/// failing audit pipeline cannot corrupt S&amp;F retry bookkeeping
/// (alog.md §7 contract). Messages whose ids are not valid GUIDs (pre-M3
/// callers that didn't thread a TrackedOperationId in) are silently
/// (alog.md §7 contract). Messages whose ids are not valid GUIDs (callers
/// that didn't thread a TrackedOperationId in) are silently
/// skipped — the observer requires a parseable id by contract.
/// </remarks>
private async Task NotifyCachedCallObserverAsync(
@@ -847,17 +764,6 @@ public class StoreAndForwardService
if (!TrackedOperationId.TryParse(message.Id, out var trackedId))
{
// StoreAndForward-022: previously a silent skip — but a non-GUID
// message id means a caller bypassed the audit hot path with zero
// feedback. The drop is still best-effort (S&F retry bookkeeping
// must never depend on the audit pipeline) but it is now observable
// via a Warning so a misconfigured caller can be diagnosed.
// Engine-minted ids (Guid.NewGuid().ToString("N")) and the current
// caller set (NotificationOutbox enqueue with NotificationId,
// cached-call enqueue with TrackedOperationId.ToString()) all
// parse — this log line fires only when a future caller supplies a
// non-GUID id, which is exactly when the silent-drop was hardest
// to diagnose.
_logger.LogWarning(
"Cached-call audit observer skipped: message id {MessageId} is not a parseable TrackedOperationId (category {Category}, outcome {Outcome}). " +
"Audit lifecycle for this operation will have no rows.",
@@ -881,18 +787,17 @@ public class StoreAndForwardService
OccurredAtUtc: DateTime.SpecifyKind(occurredAtUtc, DateTimeKind.Utc),
DurationMs: durationMs,
SourceInstanceId: message.OriginInstanceName,
// Audit Log #23 (ExecutionId Task 4): the buffered message
// carries the originating script execution's ExecutionId +
// SourceScript; surface them on the context so the bridge can
// stamp the retry-loop cached audit rows. Null on rows buffered
// before Task 4 (back-compat).
// The buffered message carries the originating script
// execution's ExecutionId + SourceScript; surface them on the
// context so the bridge can stamp the retry-loop cached audit
// rows. Null on rows buffered before this field existed
// (back-compat).
ExecutionId: message.ExecutionId,
SourceScript: message.SourceScript,
// Audit Log #23 (ParentExecutionId Task 6): the buffered
// message also carries the spawning inbound-API request's
// ExecutionId; surface it so the bridge stamps it onto the
// retry-loop cached rows. Null for a non-routed run and on
// rows buffered before Task 6 (back-compat).
// The buffered message also carries the spawning inbound-API
// request's ExecutionId; surface it so the bridge stamps it
// onto the retry-loop cached rows. Null for a non-routed run
// and on rows buffered before this field existed (back-compat).
ParentExecutionId: message.ParentExecutionId);
}
catch (Exception buildEx)
@@ -922,7 +827,7 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-12: Gets parked messages for central query (Pattern 8).
/// Gets parked messages for central query (Pattern 8).
/// </summary>
/// <param name="category">Optional category filter, or null for all categories.</param>
/// <param name="pageNumber">The page number (1-based).</param>
@@ -937,14 +842,14 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-12: Retries a parked message (moves back to pending queue).
/// Retries a parked message (moves back to pending queue).
///
/// StoreAndForward-016: an operator requeue is a buffer state change and is
/// An operator requeue is a buffer state change and is
/// replicated to the standby (as a <see cref="ReplicationOperationType.Requeue"/>)
/// so a failover preserves the operator's retry intent.
/// StoreAndForward-017: the activity-log entry carries the message's true
/// The activity-log entry carries the message's true
/// category rather than a hard-coded one.
/// StoreAndForward-020: the parked row is captured <i>before</i> the local
/// The parked row is captured <i>before</i> the local
/// requeue write rather than re-read after it, so a concurrent
/// <c>RemoveMessageAsync</c> or <c>DiscardParkedMessageAsync</c> running
/// between the two storage calls cannot leave the standby in <c>Parked</c>
@@ -955,13 +860,6 @@ public class StoreAndForwardService
/// <returns>True if successfully retried, false otherwise.</returns>
public async Task<bool> RetryParkedMessageAsync(string messageId)
{
// StoreAndForward-020: capture the parked row up front so the standby
// gets a Requeue even if a concurrent writer (a sweep delete after a
// successful delivery, or an operator discard) removes the row between
// the local update and the re-load. The storage call below is
// conditional on status = Parked, so if the row has already moved we
// return false here without replicating — the standby's matching row
// will be reconciled by whichever other operator path won the race.
var captured = await _storage.GetMessageByIdAsync(messageId);
if (captured is null || captured.Status != StoreAndForwardMessageStatus.Parked)
{
@@ -974,9 +872,6 @@ public class StoreAndForwardService
return false;
}
// WP-14 (telemetry): an operator requeue moves Parked→Pending, re-adding the
// row to the live forward queue. Counted only when the conditional storage
// update actually flipped the row.
Interlocked.Increment(ref _bufferedCount);
// The active node just rewrote this row to Pending with retry_count = 0
@@ -997,12 +892,12 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-12: Permanently discards a parked message.
/// Permanently discards a parked message.
///
/// StoreAndForward-016: an operator discard is a buffer removal and is replicated
/// An operator discard is a buffer removal and is replicated
/// to the standby (as a <see cref="ReplicationOperationType.Remove"/>) so the
/// discarded message does not reappear after a failover.
/// StoreAndForward-017: the activity-log entry carries the message's true
/// The activity-log entry carries the message's true
/// category rather than a hard-coded one.
/// </summary>
/// <param name="messageId">The identifier of the message to discard.</param>
@@ -1011,7 +906,6 @@ public class StoreAndForwardService
{
// Capture the category before the row is deleted so the activity log is
// labelled correctly.
// WP-14 (telemetry): Parked rows are not in _bufferedCount; discarding a Parked row needs no counter adjustment.
var message = await _storage.GetMessageByIdAsync(messageId);
var success = await _storage.DiscardParkedMessageAsync(messageId);
if (success)
@@ -1024,7 +918,7 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-14: Gets buffer depth by category for health reporting.
/// Gets buffer depth by category for health reporting.
/// </summary>
/// <returns>A dictionary of buffer depths by category.</returns>
public async Task<Dictionary<StoreAndForwardCategory, int>> GetBufferDepthAsync()
@@ -1033,7 +927,7 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-13: Gets count of S&amp;F messages for a given instance (for verifying survival on deletion).
/// Gets count of S&amp;F messages for a given instance (for verifying survival on deletion).
/// </summary>
/// <param name="instanceName">The instance name to query.</param>
/// <returns>The number of messages originating from the instance.</returns>
@@ -1056,7 +950,7 @@ public class StoreAndForwardService
}
/// <summary>
/// WP-14: Raises the S&amp;F activity notification. StoreAndForward-009: the
/// Raises the S&amp;F activity notification. The
/// delegate is snapshotted (so a concurrent unsubscribe cannot NRE) and every
/// subscriber invocation is wrapped so a slow/throwing subscriber (e.g. the site
/// event log) cannot abort the caller. Crucially, a subscriber exception raised
src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardStorage.cs
+19 -53
View File
@@ -5,22 +5,9 @@ using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
namespace ZB.MOM.WW.ScadaBridge.StoreAndForward;
/// <summary>
/// WP-9: SQLite persistence layer for store-and-forward messages.
/// SQLite persistence layer for store-and-forward messages.
/// Uses direct Microsoft.Data.Sqlite (not EF Core) for lightweight site-side storage.
/// No max buffer size per design decision.
///
/// StoreAndForward-008: every method opens a fresh <see cref="SqliteConnection"/> for
/// the duration of the call rather than holding a long-lived connection. This is a
/// deliberate trade-off, not an oversight: Microsoft.Data.Sqlite maintains an internal
/// connection pool keyed on the connection string, so <c>OpenAsync</c> on a previously
/// used connection string reuses a pooled handle instead of performing a real file
/// open. The retry sweep therefore relies on that pool for acceptable performance —
/// it calls <see cref="RemoveMessageAsync"/> / <see cref="UpdateMessageIfStatusAsync"/>
/// once per due message, and with no max buffer size (by design) the buffer can grow
/// large. The connection-per-call style keeps each method self-contained and
/// transaction-scoped; if profiling ever shows the pooled open to be a bottleneck on
/// the hot retry path, the remedy is a batched sweep API that opens one connection (and
/// one transaction) per sweep.
/// </summary>
public class StoreAndForwardStorage
{
@@ -71,7 +58,7 @@ public class StoreAndForwardStorage
";
await command.ExecuteNonQueryAsync();
// Audit Log #23 (ExecutionId Task 4): additively add the execution_id /
// Additively add the execution_id /
// source_script columns. CREATE TABLE IF NOT EXISTS above does NOT add
// columns to a table that already exists from before these fields, so a
// databases created by an older build needs the columns ALTER-ed in.
@@ -82,7 +69,7 @@ public class StoreAndForwardStorage
await AddColumnIfMissingAsync(connection, "execution_id", "TEXT");
await AddColumnIfMissingAsync(connection, "source_script", "TEXT");
// Audit Log #23 (ParentExecutionId Task 6): additively add the
// Additively add the
// parent_execution_id column the same way — a sibling to execution_id.
// Nullable with no default, so any row buffered before this migration
// reads back ParentExecutionId = null (back-compat).
@@ -92,7 +79,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// Audit Log #23 (ExecutionId Task 4): adds a column to <c>sf_messages</c>
/// Adds a column to <c>sf_messages</c>
/// only when it is not already present. SQLite lacks <c>ADD COLUMN IF NOT
/// EXISTS</c>, so the schema is probed via <c>PRAGMA table_info</c> first.
/// Idempotent — safe to run on every <see cref="InitializeAsync"/>.
@@ -143,7 +130,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-9: Enqueues a new message with Pending status.
/// Enqueues a new message with Pending status.
/// </summary>
/// <param name="message">The message to enqueue.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -174,13 +161,13 @@ public class StoreAndForwardStorage
cmd.Parameters.AddWithValue("@status", (int)message.Status);
cmd.Parameters.AddWithValue("@lastError", (object?)message.LastError ?? DBNull.Value);
cmd.Parameters.AddWithValue("@origin", (object?)message.OriginInstanceName ?? DBNull.Value);
// Audit Log #23 (ExecutionId Task 4): the execution id is stored as its
// The execution id is stored as its
// canonical string form ("D") so it round-trips cleanly through the
// TEXT column; null when not a cached call / not threaded.
cmd.Parameters.AddWithValue("@executionId",
message.ExecutionId.HasValue ? message.ExecutionId.Value.ToString("D") : DBNull.Value);
cmd.Parameters.AddWithValue("@sourceScript", (object?)message.SourceScript ?? DBNull.Value);
// Audit Log #23 (ParentExecutionId Task 6): the parent execution id is
// The parent execution id is
// stored as its canonical string form ("D") so it round-trips cleanly
// through the TEXT column; null when not a routed cached call.
cmd.Parameters.AddWithValue("@parentExecutionId",
@@ -190,7 +177,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-10: Gets all messages that are due for retry (Pending status, last attempt older than retry interval).
/// Gets all messages that are due for retry (Pending status, last attempt older than retry interval).
/// </summary>
/// <returns>A task that resolves to the list of messages due for retry, ordered by creation time ascending.</returns>
public async Task<List<StoreAndForwardMessage>> GetMessagesForRetryAsync()
@@ -216,7 +203,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-10: Updates a message after a delivery attempt.
/// Updates a message after a delivery attempt.
/// </summary>
/// <param name="message">The message with updated retry count, status, and last error.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -245,17 +232,10 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-10: Updates a message after a delivery attempt, but only if the row is still
/// Updates a message after a delivery attempt, but only if the row is still
/// in the expected status. Returns true if the row was updated, false if it had
/// already been changed (e.g. an operator retried or discarded the message) and so
/// was skipped.
///
/// StoreAndForward-005: the retry sweep uses this for its state-changing writes so
/// it cannot clobber a concurrent operator action (RetryParkedMessageAsync /
/// DiscardParkedMessageAsync). Those operator operations are themselves SQL-
/// conditional on <c>status = Parked</c>; making the sweep's writes conditional on
/// the status the sweep observed closes the sweep-vs-management race rather than
/// relying only on the in-process overlapping-sweep guard.
/// </summary>
/// <param name="message">The message with the updated values to persist.</param>
/// <param name="expectedStatus">The status the row must currently have for the update to proceed.</param>
@@ -289,7 +269,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-10: Removes a successfully delivered message.
/// Removes a successfully delivered message.
/// </summary>
/// <param name="messageId">The id of the message to remove.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
@@ -306,13 +286,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-12: Gets all parked messages, optionally filtered by category, with pagination.
///
/// StoreAndForward-006: the COUNT(*) and the paged SELECT run inside a single
/// transaction so they observe one consistent snapshot. Without it, a concurrent
/// enqueue/park/discard arriving between the two statements yields a TotalCount
/// inconsistent with the returned page (flickering totals / off-by-one page math
/// in the paginated UI).
/// Gets all parked messages, optionally filtered by category, with pagination.
/// </summary>
/// <param name="category">Optional category filter; null returns parked messages from all categories.</param>
/// <param name="pageNumber">1-based page number.</param>
@@ -363,14 +337,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-12: Moves a parked message back to pending for retry.
///
/// StoreAndForward-010: <c>last_attempt_at</c> is reset to NULL so the re-queued
/// message is unambiguously due on the next retry sweep. An operator-initiated
/// retry means "attempt this again now"; leaving the stale parked timestamp in
/// place would make the message's retry timing depend on the configured retry
/// interval relative to the original (pre-park) attempt — "try immediately" only
/// by accident, and a long interval would instead delay the operator's retry.
/// Moves a parked message back to pending for retry.
/// </summary>
/// <param name="messageId">The id of the parked message to move back to Pending.</param>
/// <returns>A task that resolves to <c>true</c> if the message was found and reset to Pending; <c>false</c> if not found or not in Parked status.</returns>
@@ -394,7 +361,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-12: Permanently discards a parked message.
/// Permanently discards a parked message.
/// </summary>
/// <param name="messageId">The id of the parked message to discard.</param>
/// <returns>A task that resolves to <c>true</c> if the message was found and deleted; <c>false</c> if not found or not in Parked status.</returns>
@@ -413,7 +380,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-14: Gets buffer depth by category (count of pending messages per category).
/// Gets buffer depth by category (count of pending messages per category).
/// </summary>
/// <returns>A task that resolves to a dictionary mapping each category to its pending message count.</returns>
public async Task<Dictionary<StoreAndForwardCategory, int>> GetBufferDepthByCategoryAsync()
@@ -442,7 +409,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// WP-13: Verifies messages are NOT deleted when an instance is deleted.
/// Verifies messages are NOT deleted when an instance is deleted.
/// Returns the count of messages for a given origin instance.
/// </summary>
/// <param name="instanceName">The origin instance name to count messages for.</param>
@@ -537,7 +504,7 @@ public class StoreAndForwardStorage
Status = (StoreAndForwardMessageStatus)reader.GetInt32(9),
LastError = reader.IsDBNull(10) ? null : reader.GetString(10),
OriginInstanceName = reader.IsDBNull(11) ? null : reader.GetString(11),
// Audit Log #23 (ExecutionId Task 4): rows persisted before the
// Rows persisted before the
// additive migration have no execution_id / source_script value;
// IsDBNull guards keep those reading back as null (back-compat).
// Guid.TryParse (not Parse) guards the retry sweep: a corrupt
@@ -545,7 +512,7 @@ public class StoreAndForwardStorage
// than throwing FormatException and aborting the whole sweep.
ExecutionId = ParseGuidColumn(reader, 12),
SourceScript = reader.IsDBNull(13) ? null : reader.GetString(13),
// Audit Log #23 (ParentExecutionId Task 6): rows persisted
// Rows persisted
// before the additive migration have no parent_execution_id
// value; the IsDBNull guard inside ParseGuidColumn keeps those
// reading back as null (back-compat). Guid.TryParse (not Parse)
@@ -557,8 +524,7 @@ public class StoreAndForwardStorage
}
/// <summary>
/// Audit Log #23 (ExecutionId Task 4 / ParentExecutionId Task 6):
/// defensively reads a nullable GUID column (<c>execution_id</c> or
/// Defensively reads a nullable GUID column (<c>execution_id</c> or
/// <c>parent_execution_id</c>). A <c>null</c> value (legacy pre-migration
/// rows) and a malformed non-null value both yield <c>null</c> — a corrupt
/// id must not throw and abort the retry sweep, which reads many rows.