F1 (HIGH) DebugStreamBridgeActor: the 5-minute orphan net measured the MAILBOX
(SetReceiveTimeout), and once stream events were correctly marked
INotInfluenceReceiveTimeout nothing recurring reset it — the snapshot lands once
and GrpcStreamStable once — so every healthy session self-terminated at ~6 min
with a false "Site disconnected". Replaced with a periodic self-tick
(ConsumerLivenessCheckInterval, 30s) over a consumer-last-seen stamp renewed only
by DebugStreamConsumerAlive, which DebugStreamService Tells on a shared timer to
every session still in its registry (holding a session there IS "a consumer is
attached" — both the Blazor view and the SignalR hub release it on
dispose/disconnect, and it works headless). Reverting the wrapper was rejected: it
would restore the quiet-instance orphan bug.
F2 (MED) SiteStreamGrpcClient: the RpcException(Cancelled) filter now requires
cts.IsCancellationRequested. A peer-originated / channel-dispose Cancelled fired
none of onError/onCompleted/onConnected, leaving SiteAlarmAggregatorActor with
_streamDown=false forever (IsLive stuck true, reconcile reopen guard never fired).
F3 (MED) SiteStreamGrpcClient: a header TIMEOUT is no longer reported as
connected — that shape is exactly what an unreachable site produces, and it
cleared _streamDown, consumed _seedOnConnect and launched a full snapshot fan-out
at a dead site. AwaitHeadersAsync returns bool; the first received event is the
fallback connected signal, fired at most once from headers OR first event.
F4 (LOW-MED) SqliteAuditWriter.MarkReconciledUpToAsync: the blanket below-cursor
UPDATE retired late-stamped inserts that were never served (then age-purged —
silent loss). The flip is now bounded by insertion order: a Pending row retires
only if its rowid is at or below the high-water mark of rows this instance has
served from ReadPendingSinceAsync (clamped on purge, since SQLite reuses rowids);
Forwarded rows are exempt (central ACKed them over the push path). At-least-once
is unchanged.
F5 (LOW) Documented the liveness dependency (a served row never covered by a later
cursor stays Pending forever; PurgeExpiredAsync never purges Pending) in
ISiteAuditQueue + Component-AuditLog.md, and added a cheap site-health signal:
SiteAuditBacklogReporter logs a rate-limited warning when the existing
oldest-pending metric exceeds 24h.
F6 (MED) SiteAlarmAggregatorActor: _fanoutSinceLastTick was armed by the
reconcile's OWN fan-out, so steady state ran fan-out→skip→fan-out→skip — one
reconcile per 2x interval (120s), halving the not-reporting refresh and the alarm
reconcile backstop. The skip is now armed only by connect/failover-driven seeds
(initial, _seedOnConnect, and a re-seed queued behind one).
Tests: Communication.Tests 691 passed (+13), AuditLog.Tests 382 passed (+5).
Phase 4 of the ClusterClient -> gRPC migration deleted the Akka transports
but left the naming behind: `ClusterClientSiteAuditClient` was transport-
agnostic and worked unchanged, so it survived the deletion under a name
that now describes a transport the repo no longer has. Same for a scatter
of doc-comments still framing gRPC as "the new transport" beside an Akka
one that is gone.
Renames it to `SiteCommunicationAuditClient` (and its test file) and
rewrites the stale comments to describe the single transport that exists.
Also tightens CLAUDE.md: drops the self-describing directory listing and
the 27-component enumeration in favour of the non-obvious parts only.
Behaviour-neutral: names and prose only. Recorded as the Phase 4
follow-up in docs/plans/2026-07-22-clusterclient-to-grpc-plan.md.
- MockSiteStreamGrpcClient.SubscribeCalls and UnsubscribedCorrelationIds
switched from bare List<T> to lock-guarded backing fields with snapshot
accessors, eliminating the actor-thread/test-thread data race (matches
the existing lock(events) pattern for ReceivedEvents)
- AttributeKey and AlarmKey null-guard each component with ?? string.Empty
so a null SourceReference/AlarmName/etc. cannot silently collide with an
empty-string component in the dedup dictionary
- On_Snapshot_Opens_GrpcStream renamed to
On_Snapshot_Does_Not_Open_Additional_GrpcStream; assertion updated to
confirm exactly one subscribe (the PreStart stream-first open) with no
second subscribe after snapshot delivery
- _stopped ordering in InstanceNotFound path moved after CleanupGrpc()
for consistency with DebugStreamTerminated and ReceiveTimeout handlers
Re-architect DebugStreamBridgeActor from snapshot-first to stream-first so no
attribute/alarm event occurring during the snapshot-build + network-transit
window is lost (#26).
Lifecycle change:
- PreStart now opens the gRPC subscription FIRST (alongside sending the
SubscribeDebugViewRequest), so live events start flowing immediately.
- Phase model via a single _snapshotDelivered flag (mutated only on the actor
thread). While buffering (snapshot not yet delivered), AttributeValueChanged/
AlarmStateChanged are appended to an ordered _preSnapshotBuffer instead of
being delivered. After snapshot+flush, the same handlers pass through directly.
- On DebugViewSnapshot: deliver snapshot, then flush the buffer in arrival order
with per-entity dedup, then set _snapshotDelivered=true (pass-through).
Dedup rule (exactly-once):
- Identity: attributes by (InstanceUniqueName, AttributePath, AttributeName);
alarms by (InstanceUniqueName, AlarmName, SourceReference) so native
per-condition alarms are not conflated. Keys joined with a NUL delimiter
(declared as an escaped char constant; no raw NUL in source) so distinct
identities never collide on a space within a name.
- Boundary: a buffered event whose timestamp is <= the snapshot's timestamp for
the same entity is already reflected -> DROP; strictly-newer (>) -> DELIVER;
entity absent from the snapshot -> DELIVER (genuine gap-window event).
Preserved paths:
- M2.11 InstanceNotFound: with stream-first the gRPC stream is already open, so
the not-found path now tears it down (CleanupGrpc) + clears the buffer, does
NOT enter pass-through, delivers the not-found snapshot, and stops cleanly.
- Reconnect (ReconnectGrpcStream -> OpenGrpcStream) does not touch the phase
flag: a mid-session reconnect resumes pass-through; a reconnect during the
buffering phase stays buffering until the snapshot arrives.
- Communication-008 retry/stability/stop/terminate + ReceiveTimeout orphan net
unchanged. Duplicate/late snapshot after delivery is ignored defensively.
Tests: 10 new M2.18 tests (stream-first ordering, gap-window buffering, dedup
drop/deliver for attrs + alarms, ordering, pass-through, InstanceNotFound
teardown, reconnect-during-buffering, reconnect-after-snapshot) + revised the
M2.11 not-found test to assert stream teardown. Full DebugStreamBridgeActor
class green: 23/23.
- Add DebugStreamBridgeActorTests: On_InstanceNotFound_Snapshot_Forwards_To_OnEvent_Does_Not_Open_Stream_And_Terminates — asserts _onEvent receives the not-found snapshot, SubscribeCalls remains empty, and the actor terminates cleanly via Watch/ExpectTerminated.
- Add comment in DebugStreamBridgeActor near Context.Stop(Self) explaining that the subsequent StopDebugStream Tell from DebugStreamService.StopStream produces a benign expected dead-letter.
- Reword not-found toast in DebugView.razor to "Instance not found on the selected site — check the deployment target." (accurate when the instance may be deployed to a different site).
Joseph Doherty