Commit Graph

2 Commits

Author SHA1 Message Date
Joseph Doherty e77ea4aa73 docs(historian): close #491 — read-back and EnsureTags proven against wonder-sql-vd03
v2-ci / build (push) Successful in 4m11s
v2-ci / unit-tests (push) Failing after 14m17s
The 2026-08-01 run closed every leg except read-back, which no reachable historian could
serve that day (the LAN instance accepts writes but serves no reads under a licence tag-count
violation; the licensed one needed the VPN). With the VPN up the run was repeated against
wonder-sql-vd03 and the remaining legs pass:

- EnsureTags provisioning: ensured=2, failed=0 — the path the LAN box could not exercise at
  all (it is RemoteGrpc-only and that CAP was not listening).
- Read-back: two Modbus registers carrying DISTINCT NON-ZERO constants were historized, and
  each read back as its own value (HR20 -> 8224, HR3 -> 120, quality 192). A first attempt
  read back 0, which proves nothing — the same trap #536 recorded, so the proof was rebuilt
  on values a crossed or fabricated answer could not fake.

Found during the run and FILED rather than fixed:

- #538 — one permanently-failing tag re-writes every healthy tag on every drain pass, forever.
  DrainOnceAsync writes EVERY per-tag group and then acks all-or-nothing, so a tag whose
  historian tag does not exist stops the FIFO head advancing while the healthy tags' values
  are re-submitted every drain interval; the outbox never truncates (Capacity defaults to 0 =
  unbounded, observed at 8.3 MB). The all-or-nothing ack is deliberate — RemoveAsync truncates
  THROUGH an id, so partial acking could drop older unwritten entries — so the fix changes the
  outbox contract and does not belong in this commit.

Also re-confirmed live: EnsureTags provisioning fires only for newly-ADDED historized tags, so
flipping an existing tag's historianTagname provisions nothing (Issue 5 of
2026-06-27-historian-gateway-integration-issues.md).

No production code changes — the recorder fix shipped in ad744734.

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-04 11:29:15 -04:00
Joseph Doherty ad74473452 fix(historian): the recorder died on its first value — live gate for continuous historization (#491)
v2-ci / build (push) Successful in 4m54s
v2-ci / unit-tests (push) Failing after 16m42s
Ran #491's live gate against a real ZB.MOM.WW.HistorianGateway talking to a real AVEVA
Historian, and it found a production-fatal defect on the first value it ever saw.

ContinuousHistorizationRecorder.OnValueChangedAsync touched `Self` after awaiting
IHistorizationOutbox.AppendAsync(...).ConfigureAwait(false). ConfigureAwait(false) discards
Akka's ActorTaskScheduler synchronization context, so `Self` — which reads ActorBase.Context —
threw NotSupportedException. The throw escaped into supervision, which RESTARTED the actor;
because production seeds an EMPTY ref set (the deployed set arrives later via
UpdateHistorizedRefs), the restart silently discarded every historized ref. Net effect: one
value captured, then nothing, with no further error to show for it. Fixed by capturing
`IActorRef self = Self` before the await — the idiom the GetStatus handler already applied to
Sender.

Thirteen unit tests missed it because InMemoryOutbox.AppendAsync returned
ValueTask.CompletedTask: a completed ValueTask never yields, so the continuation ran inline
with the context still installed. The production FasterLog outbox fsyncs per entry and does
yield. The first regression test was itself vacuous — `await Task.Yield()` posts back to the
CAPTURED context, so the append still completed on the actor thread and reverting the fix left
the test green. The fake must complete on a POOL thread (Task.Delay(1).ConfigureAwait(false)),
as a real fsync does; with that, reverting the fix reddens exactly one test.

The rig could not run this gate at all: docker-dev carried ServerHistorian__* but no
ContinuousHistorization__* keys, so the recorder could never spawn. Both centrals now carry
them (default OFF — the recorder needs ContinuousHistorization:Enabled AND
ServerHistorian:Enabled), with the outbox on the per-node named volume so it survives a
container recreate. ServerHistorian__UseTls was hard-coded true and is now parameterized; a
plaintext h2c gateway was unreachable without it.

Gate results: outbox append, drain, WriteLiveValues accepted by a real historian, and restart
convergence (capture resumes after a restart with no new deployment) all pass. The read-back
hop is NOT proven — the LAN historian accepts writes but serves no reads under a "License
violation: Database tag count exceeds license tag count", and the licensed corporate one needs
the VPN. EnsureTags provisioning is likewise unexercised there (RemoteGrpc-only, and that CAP
is not listening). #491 stays open for the read-back hop alone.

Also recorded, not fixed: the recorder has NO Primary gate, unlike the alarm store-and-forward
sink, so both nodes of a redundant pair send duplicate live values. Gating it would introduce a
failover window with no capture — that is a design decision, not a cleanup.

Evidence: docs/plans/2026-08-01-continuous-historization-live-gate.md

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-08-01 19:57:37 -04:00