fix(historian-gateway): cancellation-safe alarm writer + dispose-safe outbox + provisioner polish + outbox tests

I-1: GatewayAlarmHistorianWriter no longer dead-letters events cancelled
mid-drain at shutdown. WriteBatchAsync short-circuits remaining events to
RetryPlease once cancellation is requested, and SendOneAsync catches
OperationCanceledException (when the token is cancelled) -> RetryPlease,
so in-flight events stay queued instead of being permanently dropped.

I-2: FasterLogHistorizationOutbox.Dispose now guards the awaited periodic
loop with a broad catch (Exception) after the OperationCanceledException
catch, so a non-Faster teardown fault (e.g. ObjectDisposedException) can
never escape Dispose.

M-1: GatewayTagProvisioner skips the empty EnsureTags round-trip when every
request is non-historizable (early return).

M-2: GatewayTagProvisioner handles plain shutdown cancellation quietly
(Debug, not Warning), counting the unsent batch as Failed, never throwing.

M-3/M-4: Added remove-last-entry (TailAddress truncation branch) and
FIFO implicit-ack (RemoveAsync acks up to and including the target)
durability tests, both reopen-and-survive.

M-5: Clarifying comment in RecoverState on the transient over-capacity
rebuild after a crash between append-commit and drop-truncation-commit.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
This commit is contained in:
Joseph Doherty
2026-06-26 17:47:20 -04:00
parent 0be79219fc
commit 22711444cc
6 changed files with 127 additions and 1 deletions
@@ -65,6 +65,15 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter
for (var i = 0; i < batch.Count; i++)
{
if (cancellationToken.IsCancellationRequested)
{
// Shutdown mid-drain: short-circuit the remaining events to RetryPlease rather than
// calling the gateway with a cancelled token. They stay queued for retry next startup
// — a cancellation must NEVER dead-letter an in-flight event (silent data loss).
outcomes[i] = HistorianWriteOutcome.RetryPlease;
continue;
}
outcomes[i] = await SendOneAsync(batch[i], cancellationToken).ConfigureAwait(false);
}
@@ -79,6 +88,13 @@ public sealed class GatewayAlarmHistorianWriter : IAlarmHistorianWriter
.ConfigureAwait(false);
return MapAck(ack);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
// Cancellation mid-send at shutdown is NOT a poison event. Map to RetryPlease so the
// event stays queued for next startup rather than being dead-lettered (data loss).
_logger.LogDebug("Alarm SendEvent cancelled at shutdown; will retry.");
return HistorianWriteOutcome.RetryPlease;
}
catch (Exception exception)
{
// NEVER throw out of the writer — the drain worker expects a per-event outcome. Classify