fix(centralui): pending-updates rail refreshes as soon as a deployment finishes — cache drops its snapshot on Success/Failed and signals the indicator, instead of serving the pre-deploy count for up to TTL + one poll

This commit is contained in:
Joseph Doherty
2026-10-22 17:11:18 -04:00
parent 3e1435dfac
commit 0e5402915b
8 changed files with 305 additions and 12 deletions
+1 -1
View File
@@ -19,7 +19,7 @@ Central cluster only. Sites have no user interface.
- **The decision.** *Approve* is `Caution` (amber): approving executes a real write to live plant equipment, so it is not a benign primary action, but it is not destroying anything either. *Reject* **keeps** `Destructive` (red): it is terminal and irreversible for the request — the Operator must resubmit — and holding it red is what keeps the two opposite decisions visually distinct. Dropping *Reject* to a neutral button was considered and rejected: it would simply invert the bias, making declining look like the safe default.
- Colour is never the only signal — the button always carries its verb (see the label contract above), so the mapping is an accessibility affordance, not the meaning.
- **`DialogSeverity` is an additive overload**, not a replacement: the boolean form remains the signature for the ~34 existing call sites, which are genuinely two-state. Adding a fifth optional parameter to the boolean form instead would have silently un-matched every existing four-argument NSubstitute stub — the row-28 gotcha, where a stub that stops matching returns a null `Task` and the caller NREs with no compile error. Reach for the overload only when `Caution` is what you mean.
- **Pending updates rail indicator (2026-09-14, `PendingUpdatesIndicator`, design `docs/plans/2026-09-14-pending-updates-design.md`).** A block in the rail footer, placed above the user name (`MainLayout`'s `RailFooter`, `.rail-user`), summarising the fleet-wide `PendingUpdatesSnapshot` from Component-DeploymentManager.md's *Pending-updates aggregate*. Rendered only for a user holding **Deployer or Administrator** — the two roles that can act on it — and hidden for everyone else. States: *Checking for updates…* (first load), *Up to date* (muted, check icon), **Pending updates** with a warning count pill and a breakdown line (`N instances`, linking to `/deployment/topology` for a Deployer; `M sites`, linking to `/admin/sites` for an Administrator, plain text otherwise), or *Updates unavailable* (error styling, tooltip carries the failure message) on a faulted evaluation. A tooltip on the block reads *as of HH:mm UTC*. It polls `IPendingUpdatesCache.GetAsync()` every 30 s through `PollGate` — cheap, since the cache's own 60 s TTL means the fleet is actually re-flattened at most once per TTL fleet-wide, no matter how many circuits (or how often each one) poll it — disposes its timer with the circuit, and renders through `InvokeAsync`. A **site-scoped** user sees counts narrowed to their permitted sites only (`PendingUpdatesSnapshot.ForSites`, the same scope `SiteScopeService` applies to Topology), so the indicator and the Sites page never disagree about which sites a given operator can see.
- **Pending updates rail indicator (2026-09-14, `PendingUpdatesIndicator`, design `docs/plans/2026-09-14-pending-updates-design.md`).** A block in the rail footer, placed above the user name (`MainLayout`'s `RailFooter`, `.rail-user`), summarising the fleet-wide `PendingUpdatesSnapshot` from Component-DeploymentManager.md's *Pending-updates aggregate*. Rendered only for a user holding **Deployer or Administrator** — the two roles that can act on it — and hidden for everyone else. States: *Checking for updates…* (first load), *Up to date* (muted, check icon), **Pending updates** with a warning count pill and a breakdown line (`N instances`, linking to `/deployment/topology` for a Deployer; `M sites`, linking to `/admin/sites` for an Administrator, plain text otherwise), or *Updates unavailable* (error styling, tooltip carries the failure message) on a faulted evaluation. A tooltip on the block reads *as of HH:mm UTC*. It polls `IPendingUpdatesCache.GetAsync()` every 30 s through `PollGate` — cheap, since the cache's own 60 s TTL means the fleet is actually re-flattened at most once per TTL fleet-wide, no matter how many circuits (or how often each one) poll it — disposes its timer with the circuit, and renders through `InvokeAsync`. **A finished deployment refreshes it at once (2026-10-22):** the cache subscribes to `IDeploymentStatusNotifier`, drops its memoized snapshot on every `Success`/`Failed` status (a flight already running when that happens still answers its callers but is not memoized — `SingleFlightMemo.Invalidate`), and raises `IPendingUpdatesCache.Invalidated` once completions have been quiet for 1 s, so a bulk deploy's burst notifies once; the indicator re-reads on that signal (a reload the `PollGate` drops mid-poll is re-run once). Before this, a Deploy all left the pre-deploy count on the rail for up to the TTL plus one poll (~90 s). Only deployments run on the same central node are seen — one run through the peer still clears within the TTL. A **site-scoped** user sees counts narrowed to their permitted sites only (`PendingUpdatesSnapshot.ForSites`, the same scope `SiteScopeService` applies to Topology), so the indicator and the Sites page never disagree about which sites a given operator can see.
- **Rail nav scroll pane (2026-09-14).** `NavMenu.razor` wraps its items in a `<div class="rail-scroll">`; `site.css` (≥ lg only) fills the sticky `100vh` `#theme-rail` (`#theme-rail > .side-rail { height: 100% }`) and makes `.rail-scroll` the flexible, independently-scrolling region (`flex: 1 1 auto; min-height: 0; overflow-y: auto`), so a long expanded nav scrolls under the brand bar instead of pushing the rail footer (and now the pending-updates indicator) off-screen. Below lg the rail is inline and scrolls with the page, unchanged. The `ZB.MOM.WW.Theme` kit itself is untouched — this is app-local CSS/markup layered on top of it (deferred-work register row 123 proposes upstreaming it for the kit's other consumers).
## Review hardening (2026-09-09)
@@ -140,6 +140,23 @@
// the same circuit) must move this block, not leave it showing what the
// previous principal was allowed to see.
AuthStateProvider.AuthenticationStateChanged += OnAuthenticationStateChanged;
PendingUpdates.Invalidated += OnPendingUpdatesInvalidated;
}
// Set when the cache reports a finished deployment. A reload that arrives while a poll is
// already running is dropped by the PollGate, so the running poll re-checks this flag and
// goes round once more — otherwise the post-deploy count could wait out a full interval.
private int _reloadRequested;
private void OnPendingUpdatesInvalidated()
{
if (_disposed || !_visible)
{
return;
}
Interlocked.Exchange(ref _reloadRequested, 1);
_ = RunLoadAsync();
}
protected override async Task OnParametersSetAsync()
@@ -267,9 +284,20 @@
/// </summary>
internal Task PollOnceAsync() => RunLoadAsync();
private Task RunLoadAsync() => _pollGate.RunAsync(
LoadAsync,
error => Logger.LogDebug(error, "Pending-updates indicator poll tick failed."));
private async Task RunLoadAsync()
{
await _pollGate.RunAsync(
LoadAsync,
error => Logger.LogDebug(error, "Pending-updates indicator poll tick failed."));
// A reload requested while that poll was running was dropped by the gate: run once more.
if (!_disposed && Interlocked.Exchange(ref _reloadRequested, 0) == 1)
{
await _pollGate.RunAsync(
LoadAsync,
error => Logger.LogDebug(error, "Pending-updates indicator reload failed."));
}
}
private async Task LoadAsync()
{
@@ -342,6 +370,7 @@
{
_disposed = true;
AuthStateProvider.AuthenticationStateChanged -= OnAuthenticationStateChanged;
PendingUpdates.Invalidated -= OnPendingUpdatesInvalidated;
lock (_timerLock)
{
@@ -34,4 +34,12 @@ public interface IPendingUpdatesCache
/// <param name="cancellationToken">Cancels this caller's wait only, never the shared flight.</param>
/// <returns>The memoized (or freshly evaluated) fleet-wide snapshot.</returns>
Task<PendingUpdatesSnapshot> GetAsync(bool forceRefresh = false, CancellationToken cancellationToken = default);
/// <summary>
/// Raised (on a thread-pool thread) once deployments have finished on this node and the
/// memoized snapshot was dropped — after a short quiet period, so a bulk deploy's burst of
/// completions raises it once. Subscribers re-read via <see cref="GetAsync"/> instead of
/// waiting out their poll.
/// </summary>
event Action? Invalidated;
}
@@ -1,5 +1,6 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.DeploymentManager;
using ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
@@ -14,8 +15,16 @@ namespace ZB.MOM.WW.ScadaBridge.CentralUI.Services;
/// <see cref="IServiceScopeFactory"/> — the evaluator is scoped (it opens repositories), so
/// every flight resolves it from a fresh scope, off any circuit-scoped context.
/// </para>
/// <para>
/// A finished deployment (<see cref="DeploymentStatus.Success"/> or
/// <see cref="DeploymentStatus.Failed"/>, from <see cref="IDeploymentStatusNotifier"/>) drops the
/// memoized snapshot at once and, after <see cref="DefaultInvalidationQuietPeriod"/> without
/// another completion, raises <see cref="Invalidated"/>. Before this, a Deploy all left the rail
/// showing the pre-deploy count for up to the TTL plus one poll. Only deployments run on THIS
/// node are seen; one run through the peer central still clears within the TTL.
/// </para>
/// </summary>
public sealed class PendingUpdatesCache : IPendingUpdatesCache
public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly SingleFlightMemo<PendingUpdatesSnapshot> _memo;
@@ -23,18 +32,50 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache
/// <summary>Negative TTL after a faulted fleet evaluation, so polling does not hammer a sick DB.</summary>
internal static readonly TimeSpan FaultBackoff = TimeSpan.FromSeconds(10);
/// <summary>How long completions must stop arriving before <see cref="Invalidated"/> is raised.</summary>
internal static readonly TimeSpan DefaultInvalidationQuietPeriod = TimeSpan.FromSeconds(1);
private readonly IDeploymentStatusNotifier _statusNotifier;
private readonly TimeSpan _invalidationQuietPeriod;
private readonly Timer _invalidatedTimer;
private volatile bool _disposed;
/// <inheritdoc/>
public event Action? Invalidated;
/// <summary>
/// Initializes the shared pending-updates cache.
/// </summary>
/// <param name="scopeFactory">Opens a fresh DI scope per fleet evaluation.</param>
/// <param name="options">Supplies <see cref="DeploymentManagerOptions.PendingUpdatesCacheTtl"/>.</param>
public PendingUpdatesCache(IServiceScopeFactory scopeFactory, IOptions<DeploymentManagerOptions> options)
/// <param name="statusNotifier">Reports finished deployments, which invalidate the snapshot.</param>
public PendingUpdatesCache(
IServiceScopeFactory scopeFactory,
IOptions<DeploymentManagerOptions> options,
IDeploymentStatusNotifier statusNotifier)
: this(scopeFactory, options, statusNotifier, DefaultInvalidationQuietPeriod)
{
}
/// <summary>Initializes the cache with an explicit quiet period (tests).</summary>
/// <param name="scopeFactory">Opens a fresh DI scope per fleet evaluation.</param>
/// <param name="options">Supplies <see cref="DeploymentManagerOptions.PendingUpdatesCacheTtl"/>.</param>
/// <param name="statusNotifier">Reports finished deployments, which invalidate the snapshot.</param>
/// <param name="invalidationQuietPeriod">How long completions must stop before <see cref="Invalidated"/> is raised.</param>
internal PendingUpdatesCache(
IServiceScopeFactory scopeFactory,
IOptions<DeploymentManagerOptions> options,
IDeploymentStatusNotifier statusNotifier,
TimeSpan invalidationQuietPeriod)
{
_scopeFactory = scopeFactory ?? throw new ArgumentNullException(nameof(scopeFactory));
ArgumentNullException.ThrowIfNull(options);
_statusNotifier = statusNotifier ?? throw new ArgumentNullException(nameof(statusNotifier));
_invalidationQuietPeriod = invalidationQuietPeriod;
_memo = new SingleFlightMemo<PendingUpdatesSnapshot>(
options.Value.PendingUpdatesCacheTtl, faultBackoff: FaultBackoff);
_invalidatedTimer = new Timer(_ => RaiseInvalidated(), null, Timeout.Infinite, Timeout.Infinite);
_statusNotifier.StatusChanged += OnDeploymentStatusChanged;
}
/// <inheritdoc/>
@@ -42,6 +83,42 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache
bool forceRefresh = false, CancellationToken cancellationToken = default) =>
_memo.GetAsync(EvaluateAsync, forceRefresh, cancellationToken);
/// <summary>Stops listening for deployment status changes.</summary>
public void Dispose()
{
_disposed = true;
_statusNotifier.StatusChanged -= OnDeploymentStatusChanged;
_invalidatedTimer.Dispose();
}
private void OnDeploymentStatusChanged(DeploymentStatusChange change)
{
if (_disposed || change.Status is not (DeploymentStatus.Success or DeploymentStatus.Failed))
{
return;
}
// Drop the snapshot at once so no caller is served the pre-deploy count, and (re)arm the
// quiet-period timer so a bulk deploy's burst of completions notifies subscribers once.
_memo.Invalidate();
try
{
_invalidatedTimer.Change(_invalidationQuietPeriod, Timeout.InfiniteTimeSpan);
}
catch (ObjectDisposedException)
{
// Disposed between the check and the re-arm.
}
}
private void RaiseInvalidated()
{
if (!_disposed)
{
Invalidated?.Invoke();
}
}
private async Task<PendingUpdatesSnapshot> EvaluateAsync()
{
await using var scope = _scopeFactory.CreateAsyncScope();
@@ -39,6 +39,12 @@ internal sealed class SingleFlightMemo<T>
/// <summary>Wall-clock instant until which a faulted <see cref="_current"/> is re-served.</summary>
private DateTimeOffset _faultedUntil;
/// <summary>
/// Bumped by <see cref="Invalidate"/>. A flight records the value it started under and is
/// memoized only if no invalidation happened while it ran.
/// </summary>
private long _generation;
/// <summary>
/// Initializes a memo slot.
/// </summary>
@@ -57,6 +63,21 @@ internal sealed class SingleFlightMemo<T>
/// </summary>
public int FlightCount { get; private set; }
/// <summary>
/// Marks the memoized value stale, so the next <see cref="GetAsync"/> starts a new flight.
/// A flight already in progress still answers the callers waiting on it, but its result is
/// not memoized: it may have read the state this invalidation reports as changed.
/// </summary>
public void Invalidate()
{
lock (_lock)
{
_generation++;
_freshUntil = default;
_faultedUntil = default;
}
}
/// <summary>
/// Returns the memoized value, starting a flight if none is fresh.
/// </summary>
@@ -86,6 +107,7 @@ internal sealed class SingleFlightMemo<T>
TaskCompletionSource<T>? started = null;
Task<T> result;
long generation = 0;
lock (_lock)
{
@@ -115,6 +137,7 @@ internal sealed class SingleFlightMemo<T>
started = new TaskCompletionSource<T>(TaskCreationOptions.RunContinuationsAsynchronously);
_current = started.Task;
FlightCount++;
generation = _generation;
result = started.Task;
}
}
@@ -123,20 +146,22 @@ internal sealed class SingleFlightMemo<T>
// never be able to re-enter this slot on the calling thread while the lock is held.
if (started is not null)
{
_ = RunAsync(factory, started, ttlOverride ?? _ttl);
_ = RunAsync(factory, started, ttlOverride ?? _ttl, generation);
}
return cancellationToken.CanBeCanceled ? result.WaitAsync(cancellationToken) : result;
}
private async Task RunAsync(Func<Task<T>> factory, TaskCompletionSource<T> completion, TimeSpan ttl)
private async Task RunAsync(Func<Task<T>> factory, TaskCompletionSource<T> completion, TimeSpan ttl, long generation)
{
try
{
var value = await factory().ConfigureAwait(false);
lock (_lock)
{
_freshUntil = _clock() + ttl;
// Invalidated while running: answer this flight's callers, but do not serve it
// to anyone else.
_freshUntil = generation == _generation ? _clock() + ttl : default;
}
completion.SetResult(value);
}
@@ -126,6 +126,26 @@ public class PendingUpdatesIndicatorTests : BunitContext
return host.FindComponent<PendingUpdatesIndicator>();
}
/// <summary>
/// Verifies that the rail re-reads as soon as the cache reports it was invalidated (a
/// deployment finished), rather than waiting out its 30 s poll: after a successful Deploy
/// all the old count used to linger for up to the cache TTL plus one poll interval.
/// </summary>
[Fact]
public void CacheInvalidated_RereadsWithoutWaitingForThePoll()
{
var cache = Substitute.For<IPendingUpdatesCache>();
cache.GetAsync(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(
Task.FromResult(Snapshot(SiteRow(1, staleInstances: 9))),
Task.FromResult(Snapshot(SiteRow(1, staleInstances: 0))));
var cut = RenderFor(cache, Role(Roles.Deployer));
cut.WaitForAssertion(() => Assert.Contains("9 instances", cut.Markup));
cache.Invalidated += Raise.Event<Action>();
cut.WaitForAssertion(() => Assert.Contains("Up to date", cut.Markup), TimeSpan.FromSeconds(5));
}
/// <summary>Verifies that a user with only the Viewer role sees no pending-updates indicator.</summary>
[Fact]
public void Viewer_RendersNothing()
@@ -1,6 +1,8 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Options;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.CentralUI.Services;
using ZB.MOM.WW.ScadaBridge.DeploymentManager;
using ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
@@ -21,6 +23,8 @@ public class PendingUpdatesCacheTests : IDisposable
private readonly IPendingUpdatesEvaluator _evaluator = Substitute.For<IPendingUpdatesEvaluator>();
private readonly ServiceProvider _provider;
private readonly DeploymentStatusNotifier _statusNotifier =
new(NullLogger<DeploymentStatusNotifier>.Instance);
/// <summary>Initializes a service provider that resolves the substitute pending-updates evaluator.</summary>
public PendingUpdatesCacheTests()
@@ -35,8 +39,15 @@ public class PendingUpdatesCacheTests : IDisposable
Microsoft.Extensions.Options.Options.Create(
new DeploymentManagerOptions { PendingUpdatesCacheTtl = ttl ?? TimeSpan.FromHours(1) });
private PendingUpdatesCache CreateSut(TimeSpan? ttl = null) =>
new(_provider.GetRequiredService<IServiceScopeFactory>(), Options(ttl));
private PendingUpdatesCache CreateSut(TimeSpan? ttl = null, TimeSpan? invalidationQuietPeriod = null) =>
new(
_provider.GetRequiredService<IServiceScopeFactory>(),
Options(ttl),
_statusNotifier,
invalidationQuietPeriod ?? TimeSpan.FromMilliseconds(20));
private void Finish(int instanceId, DeploymentStatus status) =>
_statusNotifier.NotifyStatusChanged(new DeploymentStatusChange($"dep-{instanceId}", instanceId, status));
private static PendingUpdatesSnapshot Snapshot() =>
new(T0, Array.Empty<SitePendingUpdates>());
@@ -138,6 +149,86 @@ public class PendingUpdatesCacheTests : IDisposable
await _evaluator.Received(2).EvaluateAsync(Arg.Any<CancellationToken>());
}
/// <summary>
/// Verifies that a finished deployment drops the memoized snapshot — a deploy is exactly the
/// event that changes the answer, so serving the pre-deploy count for the rest of the TTL
/// window left the rail showing stale "pending" counts after a successful Deploy all.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[InlineData(DeploymentStatus.Success)]
[InlineData(DeploymentStatus.Failed)]
public async Task FinishedDeployment_InvalidatesTheMemoizedSnapshot(DeploymentStatus status)
{
_evaluator.EvaluateAsync(Arg.Any<CancellationToken>()).Returns(_ => Snapshot());
using var sut = CreateSut();
await sut.GetAsync();
Finish(1, status);
await sut.GetAsync();
await _evaluator.Received(2).EvaluateAsync(Arg.Any<CancellationToken>());
}
/// <summary>Verifies that a deployment that has only started does not invalidate the snapshot.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task StartedDeployment_DoesNotInvalidate()
{
_evaluator.EvaluateAsync(Arg.Any<CancellationToken>()).Returns(_ => Snapshot());
using var sut = CreateSut();
await sut.GetAsync();
Finish(1, DeploymentStatus.InProgress);
await sut.GetAsync();
await _evaluator.Received(1).EvaluateAsync(Arg.Any<CancellationToken>());
}
/// <summary>
/// Verifies that a burst of completions — one per instance in a bulk deploy — raises
/// <c>Invalidated</c> once, after the burst goes quiet, so every open rail re-reads once
/// instead of once per instance.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task BurstOfFinishedDeployments_RaisesInvalidatedOnceAfterItGoesQuiet()
{
using var sut = CreateSut(invalidationQuietPeriod: TimeSpan.FromMilliseconds(100));
var raised = 0;
var signal = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
sut.Invalidated += () =>
{
Interlocked.Increment(ref raised);
signal.TrySetResult();
};
for (var i = 1; i <= 9; i++)
{
Finish(i, DeploymentStatus.Success);
}
await signal.Task.WaitAsync(TimeSpan.FromSeconds(5));
await Task.Delay(300);
Assert.Equal(1, Volatile.Read(ref raised));
}
/// <summary>Verifies that the cache stops listening to deployment status changes once disposed.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Dispose_UnsubscribesFromStatusChanges()
{
var sut = CreateSut();
var raised = 0;
sut.Invalidated += () => Interlocked.Increment(ref raised);
sut.Dispose();
Finish(1, DeploymentStatus.Success);
await Task.Delay(150);
Assert.Equal(0, Volatile.Read(ref raised));
}
/// <summary>Releases the resources held by the test fixture.</summary>
public void Dispose() => _provider.Dispose();
}
@@ -199,4 +199,47 @@ public class SingleFlightMemoTests
released.SetResult(99);
Assert.Equal(99, await patient);
}
/// <summary>Verifies that <c>Invalidate</c> makes a still-fresh value stale, so the next call queries again.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Invalidate_DiscardsAFreshValue()
{
var invocations = 0;
var memo = new SingleFlightMemo<int>(TimeSpan.FromHours(1), () => T0);
var first = await memo.GetAsync(() => Task.FromResult(++invocations));
memo.Invalidate();
var second = await memo.GetAsync(() => Task.FromResult(++invocations));
Assert.Equal(1, first);
Assert.Equal(2, second);
Assert.Equal(2, memo.FlightCount);
}
/// <summary>
/// Verifies that a flight already running when <c>Invalidate</c> is called still answers its
/// callers but is not memoized: it may have read the state the invalidation says changed.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Invalidate_DuringAFlight_ResultIsServedButNotMemoized()
{
var released = new TaskCompletionSource<int>(TaskCreationOptions.RunContinuationsAsynchronously);
var invocations = 0;
var memo = new SingleFlightMemo<int>(TimeSpan.FromHours(1), () => T0);
var inFlight = memo.GetAsync(() =>
{
invocations++;
return released.Task;
});
memo.Invalidate();
released.SetResult(7);
Assert.Equal(7, await inFlight);
var next = await memo.GetAsync(() => Task.FromResult(++invocations));
Assert.Equal(2, next);
Assert.Equal(2, memo.FlightCount);
}
}