fix(historian): the recorder died on its first value — live gate for continuous historization (#491)
Ran #491's live gate against a real ZB.MOM.WW.HistorianGateway talking to a real AVEVA Historian, and it found a production-fatal defect on the first value it ever saw. ContinuousHistorizationRecorder.OnValueChangedAsync touched `Self` after awaiting IHistorizationOutbox.AppendAsync(...).ConfigureAwait(false). ConfigureAwait(false) discards Akka's ActorTaskScheduler synchronization context, so `Self` — which reads ActorBase.Context — threw NotSupportedException. The throw escaped into supervision, which RESTARTED the actor; because production seeds an EMPTY ref set (the deployed set arrives later via UpdateHistorizedRefs), the restart silently discarded every historized ref. Net effect: one value captured, then nothing, with no further error to show for it. Fixed by capturing `IActorRef self = Self` before the await — the idiom the GetStatus handler already applied to Sender. Thirteen unit tests missed it because InMemoryOutbox.AppendAsync returned ValueTask.CompletedTask: a completed ValueTask never yields, so the continuation ran inline with the context still installed. The production FasterLog outbox fsyncs per entry and does yield. The first regression test was itself vacuous — `await Task.Yield()` posts back to the CAPTURED context, so the append still completed on the actor thread and reverting the fix left the test green. The fake must complete on a POOL thread (Task.Delay(1).ConfigureAwait(false)), as a real fsync does; with that, reverting the fix reddens exactly one test. The rig could not run this gate at all: docker-dev carried ServerHistorian__* but no ContinuousHistorization__* keys, so the recorder could never spawn. Both centrals now carry them (default OFF — the recorder needs ContinuousHistorization:Enabled AND ServerHistorian:Enabled), with the outbox on the per-node named volume so it survives a container recreate. ServerHistorian__UseTls was hard-coded true and is now parameterized; a plaintext h2c gateway was unreachable without it. Gate results: outbox append, drain, WriteLiveValues accepted by a real historian, and restart convergence (capture resumes after a restart with no new deployment) all pass. The read-back hop is NOT proven — the LAN historian accepts writes but serves no reads under a "License violation: Database tag count exceeds license tag count", and the licensed corporate one needs the VPN. EnsureTags provisioning is likewise unexercised there (RemoteGrpc-only, and that CAP is not listening). #491 stays open for the read-back hop alone. Also recorded, not fixed: the recorder has NO Primary gate, unlike the alarm store-and-forward sink, so both nodes of a redundant pair send duplicate live values. Gating it would introduce a failover window with no capture — that is a design decision, not a cleanup. Evidence: docs/plans/2026-08-01-continuous-historization-live-gate.md Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
This commit is contained in:
@@ -247,6 +247,21 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
|
||||
|
||||
var timestampUtc = DateTime.SpecifyKind(msg.TimestampUtc, DateTimeKind.Utc);
|
||||
|
||||
// Capture Self BEFORE the first await. Self reads ActorBase.Context, and the awaits below use
|
||||
// ConfigureAwait(false) — which drops Akka's ActorTaskScheduler synchronization context, so any
|
||||
// continuation that actually resumes off the actor thread finds NO ActorContext and throws
|
||||
// NotSupportedException. This is the same idiom the GetStatus handler already applies to Sender.
|
||||
//
|
||||
// It only bites when the append genuinely YIELDS: the production FasterLog outbox fsyncs per entry
|
||||
// (PerEntry commit mode), so it does; a fake outbox returning ValueTask.CompletedTask completes
|
||||
// synchronously, the continuation runs inline with the context still installed, and the bug is
|
||||
// invisible. That is exactly how it reached production — see the yielding fake in
|
||||
// ContinuousHistorizationRecorderTests. Live-caught on the docker-dev rig (Gitea #491), where the
|
||||
// throw escaped the handler into supervision: the actor restarted, and because production seeds an
|
||||
// EMPTY ref set (the deployed set arrives later via UpdateHistorizedRefs), the restart silently
|
||||
// dropped every historized ref — so historization stopped dead after the FIRST value.
|
||||
IActorRef self = Self;
|
||||
|
||||
// One outbox entry PER historian name: a single mux fan-out must reach every aliased historized tag
|
||||
// (each its own historian name), not just one — collapsing to one name here would silently drop the
|
||||
// rest. Snapshot the names first: although the mailbox is suspended across the awaits below (so the
|
||||
@@ -290,8 +305,9 @@ public sealed class ContinuousHistorizationRecorder : ReceiveActor, IWithTimers
|
||||
}
|
||||
|
||||
// Nudge a prompt drain attempt; the DrainTick handler de-dups (already draining) and honours
|
||||
// any active backoff window, so this never bypasses a failure cooldown.
|
||||
Self.Tell(DrainTick.Instance);
|
||||
// any active backoff window, so this never bypasses a failure cooldown. Uses the pre-await
|
||||
// capture, never `Self` — see the note above.
|
||||
self.Tell(DrainTick.Instance);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
Reference in New Issue
Block a user