2807b5a3ea
Row 65 — trigger FIRED, fixed. BundleUnlockAttempt rows were pruned only by
nodes actively serving unlocks, so a cluster that stopped importing entirely
left its last rows in place forever. The row's own trigger was "Transport gains
a central scheduler a prune tick could ride" — it has one now:
BundleSessionEvictionService, the central-only 1-minute session sweep added by
the transport-session fixes. It now also ticks the new
IBundleUnlockAttemptWindow.PruneAsync. Both drivers share the ONE 15-minute CAS
cadence claim, so ticking on a 1-minute sweep costs at most one bounded DELETE
per quarter hour and the two can never prune concurrently. PruneAsync is
contractually non-throwing (unlike TryRegisterAttemptAsync, which throws and
lets the limiter own the fail-open decision) — a background sweep has no caller
to report to and must survive a database blip. Both Transport docs updated;
three tests in FleetWideUnlockThrottleTests, incl. the retention sweep running
with no unlock traffic anywhere in the test, which is the whole point.
Row 58 — closed with a pin, and the OLD pin was found blind. The existing
Deployments_Dispose_UnsubscribesFromNotifier is behavioural: it is satisfied by
the _disposed guard alone, so deleting the `-=` from Dispose leaves it green
while the singleton notifier keeps a delegate pinning the circuit-scoped
component. Verified by mutation — with the `-=` removed the old test still
passed and only the two new pins failed. Added:
* a structural retention pin asserting the notifier's invocation list actually
shrinks to 0 on Dispose;
* a source scan pinning the row's stated TRIGGER ("a second page subscribes,
or any subscriber lands without a matching Dispose unsubscribe") — every
CentralUI file that subscribes must also unsubscribe and be IDisposable,
with a non-empty guard so the scan cannot pass vacuously.
Row 60 — closed with a pin, and the register's premise corrected.
GlobalArtifactSnapshot does NOT hold EF-materialised entities: it is projected
into pure Commons DTOs before the fan-out. The EF state that genuinely crosses
into the parallel send is the tracked Site entities the lambda closes over, and
that is safe only while lazy loading is off. Pinned in ConfigurationDatabase
.Tests two ways — no proxies extension on the built DbContextOptions, and no
EntityFrameworkCore.Proxies package reference anywhere in the repo (the
structurally stronger half: UseLazyLoadingProxies() is not a compilable call
without it). The real invariant is now spelled out at the fan-out itself.
Row 59 — no product change; added the missing tripwire remark stating the
constraint the safety argument rests on (serial by design, one scoped DbContext,
do not Task.WhenAll; if it gets slow the fix is a scope PER SITE, not a shared
scope under fan-out). Line numbers in the register have drifted to :528-545.
Rows 36, 37, 64: verification only, no change — see the sweep report.
134 lines
6.5 KiB
C#
134 lines
6.5 KiB
C#
using Microsoft.Extensions.Hosting;
|
|
using Microsoft.Extensions.Logging;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Transport;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Observability;
|
|
|
|
namespace ZB.MOM.WW.ScadaBridge.Transport.Import;
|
|
|
|
/// <summary>
|
|
/// T-007: periodic background sweep that drives <see cref="IBundleSessionStore.EvictExpired"/>
|
|
/// so abandoned import sessions clear from memory on their own, without needing a
|
|
/// new <see cref="IBundleSessionStore.Get"/> to trigger lazy eviction. Each session
|
|
/// owns the decrypted bundle content (potentially up to ~100 MB of secrets — DB
|
|
/// connection strings, SMTP credentials, external-system auth configs), and the
|
|
/// design contract is "bundles are not retained server-side after ApplyAsync
|
|
/// commits". This service keeps abandoned / failed sessions from pinning that
|
|
/// plaintext for the full 30-minute TTL when no other traffic flows.
|
|
/// <para>
|
|
/// <b>Register row 65: it also ticks the bundle-unlock attempt prune.</b> That retention
|
|
/// sweep used to ride the unlock path alone, on the reasoning that Transport owned no
|
|
/// central scheduler — this service is that scheduler, so a cluster that stops importing
|
|
/// entirely no longer leaves its last <c>BundleUnlockAttempt</c> rows in place
|
|
/// indefinitely. The window keeps its own 15-minute cadence claim, so ticking it on the
|
|
/// 1-minute session sweep costs at most one bounded DELETE per quarter hour, and the two
|
|
/// drivers can never prune concurrently.
|
|
/// </para>
|
|
/// </summary>
|
|
internal sealed class BundleSessionEvictionService : BackgroundService
|
|
{
|
|
private static readonly TimeSpan SweepInterval = TimeSpan.FromMinutes(1);
|
|
|
|
private readonly IBundleSessionStore _sessionStore;
|
|
private readonly IBundleUnlockAttemptWindow _unlockAttemptWindow;
|
|
private readonly ILogger<BundleSessionEvictionService> _logger;
|
|
|
|
/// <summary>
|
|
/// Register row 53: the open-session gauge callback, held in a field so
|
|
/// <see cref="StopAsync"/> can pass the SAME delegate reference to
|
|
/// <see cref="ScadaBridgeTelemetry.ClearImportSessionCountProvider"/> — the clear is
|
|
/// an identity-checked compare-and-clear, so a stale stop cannot unregister a newer
|
|
/// instance's provider. Mirrors StoreAndForwardService's queue-depth registration.
|
|
/// </summary>
|
|
private readonly Func<long> _openSessionCountProvider;
|
|
|
|
/// <summary>
|
|
/// Initializes a new instance of <see cref="BundleSessionEvictionService"/>.
|
|
/// </summary>
|
|
/// <param name="sessionStore">The bundle session store to sweep for expired sessions.</param>
|
|
/// <param name="unlockAttemptWindow">Shared unlock-attempt window whose retention prune this sweep also ticks (register row 65).</param>
|
|
/// <param name="logger">Logger for sweep diagnostics.</param>
|
|
public BundleSessionEvictionService(
|
|
IBundleSessionStore sessionStore,
|
|
IBundleUnlockAttemptWindow unlockAttemptWindow,
|
|
ILogger<BundleSessionEvictionService> logger)
|
|
{
|
|
_sessionStore = sessionStore ?? throw new ArgumentNullException(nameof(sessionStore));
|
|
_unlockAttemptWindow = unlockAttemptWindow ?? throw new ArgumentNullException(nameof(unlockAttemptWindow));
|
|
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
|
_openSessionCountProvider = () => _sessionStore.OpenSessionCount;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Register row 53: registers the open-import-sessions gauge provider before the
|
|
/// sweep loop starts. This service is the Transport component's only lifecycle seam
|
|
/// (the session store itself is a plain singleton), and it is registered by
|
|
/// <c>AddTransport</c>, which only the Central branch of the host calls — so the
|
|
/// gauge reports 0 on a site node, where there are no import sessions to count.
|
|
/// </summary>
|
|
/// <param name="cancellationToken">Host startup cancellation token.</param>
|
|
/// <returns>A task that completes when the base <see cref="BackgroundService"/> has started.</returns>
|
|
public override Task StartAsync(CancellationToken cancellationToken)
|
|
{
|
|
ScadaBridgeTelemetry.SetImportSessionCountProvider(_openSessionCountProvider);
|
|
return base.StartAsync(cancellationToken);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Deregisters this instance's gauge provider on graceful stop so the gauge falls
|
|
/// back to 0 rather than reading through a dead store. The clear is identity-checked,
|
|
/// so a late stop cannot clobber a successor that already re-registered.
|
|
/// </summary>
|
|
/// <param name="cancellationToken">Host shutdown cancellation token.</param>
|
|
/// <returns>A task that completes when the base <see cref="BackgroundService"/> has stopped.</returns>
|
|
public override async Task StopAsync(CancellationToken cancellationToken)
|
|
{
|
|
ScadaBridgeTelemetry.ClearImportSessionCountProvider(_openSessionCountProvider);
|
|
await base.StopAsync(cancellationToken).ConfigureAwait(false);
|
|
}
|
|
|
|
/// <inheritdoc />
|
|
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
|
|
{
|
|
while (!stoppingToken.IsCancellationRequested)
|
|
{
|
|
try
|
|
{
|
|
await Task.Delay(SweepInterval, stoppingToken).ConfigureAwait(false);
|
|
}
|
|
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
|
{
|
|
return;
|
|
}
|
|
|
|
try
|
|
{
|
|
_sessionStore.EvictExpired();
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_logger.LogWarning(ex, "Bundle session sweep failed; will retry on next interval.");
|
|
}
|
|
|
|
// Register row 65: retention tick for the fleet-wide unlock-attempt window.
|
|
// PruneAsync is contractually non-throwing and self-limits to its own
|
|
// 15-minute cadence, so this is a cheap no-op on most sweeps. Kept separate
|
|
// from the session sweep's try so neither maintenance duty can mask the other.
|
|
try
|
|
{
|
|
await _unlockAttemptWindow
|
|
.PruneAsync(BundleUnlockRateLimiter.DefaultWindow, stoppingToken)
|
|
.ConfigureAwait(false);
|
|
}
|
|
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
|
|
{
|
|
return;
|
|
}
|
|
catch (Exception ex)
|
|
{
|
|
_logger.LogWarning(
|
|
ex, "Bundle-unlock attempt prune tick failed; will retry on next interval.");
|
|
}
|
|
}
|
|
}
|
|
}
|