fix(notifications): list sort index + CAS operator Retry/Discard (P5-3, P5-4)

P5-3: add IX_Notifications_Created_NotificationId (CreatedAt DESC,
NotificationId DESC) — the Central UI Notification Outbox list page sorts on
exactly that pair, but no existing index leads unconditionally on CreatedAt,
so an unfiltered/lightly filtered page fell back to a full clustered-index
scan + sort on every COUNT + page request. Mirrors the Medium wave's
IX_SiteCalls_Created_TrackedOperationId migration + SQL twin pattern
faithfully, including the handwritten docs/plans/sql script.

P5-4: operator Retry/Discard was read-check-write keyed on NotificationId
only — a detached PipeTo task interleaving with the dispatcher's own write
(or a second concurrent operator action) could land between the read and
the write and get silently clobbered by the unconditional UpdateAsync,
stamping a Delivered row Discarded with stale fields. Adds
INotificationOutboxRepository.TryUpdateFromParkedAsync, a single-statement
compare-and-swap (WHERE Status == Parked) mirroring
SecuredWriteRepository.TryMarkApprovedAsync's CAS shape. NotificationOutboxActor's
RetryAsync/DiscardAsync now use the CAS write and, on a lost race, re-read
to report "notification already progressed to <Status>" instead of silently
overwriting whatever is there now.

Proof of failure: reverting only the actor's CAS usage while keeping the
repository addition reproduces the silent-overwrite bug — 7 tests fail,
including two new deterministic (non-sleep) race-proof tests that mock the
exact GetByIdAsync -> lost-CAS -> re-read sequence the finding describes.
P5-3 is perf-only; verified via migration/SQL-twin parity instead (no
runtime proof-of-failure is meaningful for an index).
This commit is contained in:
Joseph Doherty
2026-08-19 16:20:18 -04:00
parent 11675285af
commit 1cea01d924
12 changed files with 2607 additions and 32 deletions
@@ -1,4 +1,5 @@
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Notifications;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Notifications;
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
@@ -83,6 +84,43 @@ public interface INotificationOutboxRepository
/// </returns>
Task<bool> UpdateAsync(Notification n, CancellationToken cancellationToken = default);
/// <summary>
/// Compare-and-swap variant of <see cref="UpdateAsync"/> for the operator
/// Retry/Discard one-shots: persists <paramref name="n"/>'s delivery-state
/// columns (same narrow column set as <see cref="UpdateAsync"/>) ONLY if the
/// row's current <c>Status</c> in the database is still
/// <see cref="NotificationStatus.Parked"/>. Commits internally — this call
/// is its own transaction.
/// </summary>
/// <remarks>
/// <b>Why this exists (arch-review 2026-08-19 P5-4).</b> The operator retry/
/// discard handlers are read-check-write across two round trips:
/// <see cref="GetByIdAsync"/> reads the row, the handler checks its
/// in-memory <c>Status</c>, then a write persists the transition. A detached
/// <c>PipeTo</c> task can interleave with the dispatcher's own write (or a
/// second concurrent operator action) between those two round trips, so the
/// in-memory check is a fast-path rejection, not a correctness guard — a
/// blind <see cref="UpdateAsync"/> in that window can stamp a row that has
/// already moved on (e.g. the dispatcher delivered it) with the operator's
/// stale fields, silently overwriting a <c>Delivered</c> row as
/// <c>Discarded</c>. The <c>WHERE Status = Parked</c> predicate below makes
/// the <c>Parked-&gt;(Pending|Discarded)</c> transition atomic at the row
/// level, mirroring <c>SecuredWriteRepository.TryMarkApprovedAsync</c>'s
/// single-statement compare-and-swap. A row that has already progressed
/// (dispatcher won the race, or a second operator action won it) and a row
/// that has vanished (retention purge) both report zero rows affected;
/// callers cannot tell those apart from the return value alone and must
/// re-read to report which honestly.
/// </remarks>
/// <param name="n">The notification whose delivery state should be persisted, if still Parked.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// <see langword="true"/> when a row with that <c>NotificationId</c> was
/// found with <c>Status == Parked</c> and its delivery state was persisted;
/// <see langword="false"/> otherwise (not found, or no longer Parked).
/// </returns>
Task<bool> TryUpdateFromParkedAsync(Notification n, CancellationToken cancellationToken = default);
/// <summary>
/// Returns a page of notifications matching <paramref name="filter"/>, ordered by
/// <c>CreatedAt</c> descending, together with the total matching count.