refactor(deployment): injectable terminal-purge rate limiter; test the swallowed failure (ScadaBridge#90)

DeploymentService.TryPurgeTerminalDeploymentRecordsAsync rate-limited its
opportunistic retention sweep with a process-wide static tick field, so only
one test in the assembly could drive the sweep and its swallow-on-failure path
was untested.

The limit now lives in TerminalPurgeRateLimiter: a TimeProvider-driven
compare-and-swap claim (MinInterval 6 h) registered by AddDeploymentManager as
a DI singleton, so every scoped DeploymentService still shares one limit per
node. DeploymentService takes it as an optional trailing constructor parameter;
a hand-built instance gets its own limiter. The sweep's cutoff now comes from
the same clock. Behaviour is otherwise unchanged.

Evidence (TerminalPurgeRateLimiterTests):
- claim once per interval on an injected clock; 16 concurrent claimants ->
  exactly one wins;
- two services with their own limiters both sweep, two sharing one sweep once;
- cutoff = injected now - retention;
- a failing artifact batch is logged as a warning with the exception and
  swallowed, the already-purged terminal rows are still reported, the next
  call inside the interval does not retry and the first call after it does;
  a first-batch failure returns 0 and skips the artifact purge;
- AddDeploymentManager registers the limiter as a singleton shared across scopes.
Mutation-checked: a process-wide static limiter fails 5 tests (incl. the
existing artifact-purge test); narrowing the catch fails both failure tests.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:20:46 -05:00
parent 6070cc2b9d
commit 51d2f6c906
6 changed files with 311 additions and 23 deletions
@@ -270,7 +270,12 @@ window the table grows for the life of the system and every unfiltered deploymen
query degrades with age rather than with page size.
`TerminalDeploymentRecordRetention` (default **365 days**, deliberately generous
because deployment history is operator forensics) bounds it, swept opportunistically
and rate-limited on the deployment-list read path.
and rate-limited on the deployment-list read path: at most one sweep per 6 hours per
central node, enforced by the DI-singleton `TerminalPurgeRateLimiter` (a
`TimeProvider`-driven compare-and-swap claim shared by every scoped
`DeploymentService`; Gitea #90 — formerly a process-wide static). A failed sweep is
logged as a warning and swallowed — it never fails the operator's query — and still
consumes the interval, so the next attempt waits the full 6 hours.
Only **terminal** rows (`Success` / `Failed`) are eligible. An `InProgress` row is
never purged regardless of age: it is precisely the row the query-before-redeploy