The In-process page feeds table still described the alarms page's subscription as
provider status only, contradicting the three passages updated in 7b6dfba.
The loop's catch-all comment (and the cadence bullet that repeated it) claimed the
monitor completes a subscriber's stream on restart. It does not: ClearCache pushes
snapshot_status(false) through the still-open channel, and a subscriber is only
completed-with-error on a failed TryWrite, or removed by its own disposal.
The SnapshotStatus arm's priming claim had no test behind it — every push test
attached to an untruncated feed and pushed the edge itself. ScriptedAlarmFeed now
replays an optional open sequence, and a new test attaches to a feed already
primed provider_status then snapshot_status(truncated) and asserts the banner
comes up with no edge pushed after render.
Audited every Server-0xx finding whose Resolution described a
documentation-only or comment-only change, and spot-checked the doc
sub-claims of otherwise test-backed resolutions. 20 entries annotated
in place (append-only; no historical resolution text rewritten).
Two corrections had not survived and are re-applied:
Server-040: the MapGroupsToRoles lookup-precedence comment moved intact
into DashboardGroupRoleMapping (792e3f9) and was then deleted wholesale
by fca978d, a sweep meant only to strip (Server-NNN) tracking markers.
That also removed a later, substantive paragraph recording that the
shared ZB.MOM.WW.Auth.Ldap provider pre-strips groups to short RDN
names, so a full-DN GroupToRole key is unsupported. Both paragraphs
restored, minus the tracking IDs.
Server-009: the WAL / busy_timeout note vanished when the Storage
section of docs/Authentication.md was rewritten to delegate
connection-factory detail to ZB.MOM.WW.Auth.ApiKeys. The behavior is
still live in the library (confirmed against 0.2.1), so the fix is
prose-only.
Server-011/014/022/023 are annotated as moot rather than regressed:
the IAlarmRpcDispatcher trio was deleted in dc9c0c9 and no stale
'not yet wired' / 'PR A.6/A.7' prose survives in Server source.
Server-038's documented v1 ACL gap was later closed by
IDashboardSessionAcl, so its remarks are current.
Comment/doc-only; no logic changes.
The truncated-snapshot caveat moved from poll-only to push-driven. The page
already held an in-process alarm-feed subscription for the provider badge; it
now also handles the feed's snapshot_status frame, so a capped provider fetch is
caveated when the monitor decides it rather than up to three seconds later.
The poll's assignment stays as the reconcile baseline — both sources read the
same monitor verdict, and the frame is consumed, never synthesized page-side.
StreamAsync primes every subscriber with a snapshot_status frame at open, so a
page attaching mid-truncation needs no priming logic of its own; the loop is
renamed StatusFeedLoopAsync because it now feeds two indicators, not one.
The form split tags with the shared ParseList and attached the result verbatim,
so "team-a, TEAM-A" persisted as two entries and the constraints column read
dashboard_tags=[team-a, TEAM-A] — one grant reported as two on the page whose
job is to show what a key was granted. Enforcement never saw it (a session holds
its tags in a case-insensitive set), which is exactly why the display was the
only place it could surface.
De-duplicates ordinal-ignore-case at the attach point only, first spelling
winning, matching ApiKeyAdminCommandLineParser.ParseDashboardTags. ParseList is
untouched: the five glob lists are matched literally, so near-duplicates there
are not necessarily the same rule and must survive verbatim — pinned by a test.
The help text claimed to mirror the CLI flag; it now claims only the shared
separators and the dedupe, since the form still drops an empty segment silently
where the CLI hard-fails. A browser form has no exit code to fail with, so that
difference stays, and Authorization.md now records it.
The constraints column enumerated only the eight positional ApiKeyConstraints
members, so a key whose sole recorded policy was a dashboard tag summarised to
an empty string and rendered as "-" — the same cell a key with no policy at
all gets. ApiKeyConstraints.IsEmpty counts DashboardTags, so that key is not
unconstrained, and the column was quietly telling operators otherwise about a
grant that decides who can watch a session's events.
The create form had no dashboard-tags input either, so tagged keys could only
be minted from the apikey create-key CLI. Adds the field beside the other
constraint lists (same ParseList separators) and attaches it through the
record's init-only member, since it postdates the eight-member constructor.
CreateModel, OpenCreateDialog and TryBuildCreateRequest widen to internal for
the new render tests: the create form is behind a click and static rendering
cannot dispatch one. That is the assembly's existing InternalsVisibleTo seam.
Review found AlarmDispatcher.SnapshotActiveAlarms reading the snapshot and the
truncation verdict through two independent lock acquisitions, defended by a
comment claiming read-order made a race "widen only, never narrow". That claim
was false: a not-truncated -> truncated poll landing between the two reads pairs
a stale false with a capped snapshot, which is exactly the false all-clear the
feature exists to prevent. It was safe only because AlarmCommandHandler
STA-serializes consumer calls — an accident of the call graph, not an invariant.
Made the invariant structural instead of documented. IMxAccessAlarmConsumer now
exposes ONE accessor, `IReadOnlyList<MxAlarmSnapshotRecord> SnapshotActiveAlarms(
out bool truncated)`, which implementations must satisfy from a single
acquisition of the lock guarding the retained snapshot — mirroring the write
side, where FoldFetch already updates snapshot and verdict together. The
separate LastSnapshotTruncated property is gone from every layer, so there is no
second read left to pair badly. `out` over a result struct follows the file's
established idiom (FoldFetch, ParseSnapshotXml).
The same threading applies one level up: IAlarmCommandHandler.QueryActive now
carries `out bool snapshotTruncated`, so MxAccessCommandExecutor stamps the reply
payload from the value the records were stamped with rather than reading the
state a second time.
Direct tests for the three hops that were only covered end-to-end:
- AlarmDispatcherTests: truncated consumer snapshot stamps FromTruncatedSnapshot
on every mapped record, with a complete-snapshot control, plus an assertion
that the independent per-record Degraded flag is not dragged along.
- AlarmCommandHandlerTests: the verdict delegates through the dispatcher
(Theory over both values), and survives a prefix filter that removes every
record — the case the per-record flag cannot cover.
- AlarmCommandExecutorTests: the reply payload's SnapshotTruncated comes from the
handler (Theory over both values), including the zero-record case.
The WnWrapAlarmConsumer truncation tests now assert through
SnapshotActiveAlarms(out ...) rather than an internal field, because the pairing
is the contract.
Also: GatewayAlarmMonitor's _snapshotTruncated comment now says "as of the last
full reconcile" rather than implying it tracks the current _alarms contents,
which live transitions keep moving via ApplyTransition between passes.
Detection heuristic still untouched (fetchedRecordCount >= maxAlarmsPerFetch);
no @COUNT parsing, per docs/AlarmProbeFindings.md. Still additive gateway
metadata about our fetch mechanics, not MXAccess behavior — not a parity
deviation, and no event is synthesized.
Gateway: NonWindows.slnx builds clean (0 warnings); ~Alarm filter 107/107 pass.
Worker + Worker.Tests are windev-gated; the signature change was reviewed by
inspection across all 7 IMxAccessAlarmConsumer implementers, all 3
IAlarmCommandHandler implementers, and every call site.
Follow-up to the per-session event ACL. Part of that change rode into 693a78d
via a concurrent agent's pathspec-less commit; this commit carries the review
fixes and uses pathspecs on the commit itself so it cannot recur in either
direction.
Gating the page's subscribe seam made AttachEvents asynchronous — it awaits the
authentication state — and that await is a suspension point the synchronous
version did not have. On a rapid A -> B navigation the suspended A continuation
resumes after B's parameter set has run to completion, re-reads the live
SessionId (now B's), and attaches B a SECOND time. The ACL is not bypassed —
the newer attach already cleared that same session — but the fields holding B's
first subscription are overwritten in place, so nothing ever disposes it: its
EventsHubViewerRegistry entry is never released, which keeps the mirror cloning
events for a session the page is no longer watching through that handle, and
its pump is never cancelled. A resource leak the ACL work introduced.
OnParametersSetAsync now claims a monotonic _attachGeneration synchronously,
before its first await, and AttachEventsAsync re-checks it after the await and
before any field write or Subscribe call. A stale attach returns rather than
detaching: it owns nothing, and tearing down there would destroy the newer
attach's subscription. DetachEventsAsync needs no such guard — it captures and
nulls the live fields synchronously before it awaits, so a resumed detach only
unwinds what it already took ownership of. Same dispatcher-owned identity idea
as the existing ReferenceEquals guards in PumpEventsAsync and
MarkDisconnectedAsync, one level up.
The interleaving is not expressible with the static HtmlRenderer idiom the other
page tests use: it renders a root component once and exposes no parameter-update
seam. The new test therefore adds a minimal Renderer subclass whose only job is
to mount a component and drive a second SetParametersAsync into it while the
first is parked on a gated AuthenticationStateProvider. That subclass is the
lone reason for a narrowly scoped BL0006 suppression, justified in place: it is
test-only scaffolding that never ships, and the cost of the warning coming true
is a compile break in one test file on an SDK bump. Confirmed non-vacuous by
mutation — with the generation check disabled the test goes red on the doubled
subscription and the two passing ACL tests stay green.
Two decision-table corners are now pinned rather than implied. Admin x
nonexistent session id resolves to ALLOW, because the admin bypass is evaluated
before the registry lookup; a plausible "look the session up first, it reads
better" refactor would flip it, so a test documents the ordering. EventsHub's
remarks said "an unknown session id is denied" without qualification, which read
as universal; they now state that the bypass is checked first and every rule
below it is a non-Admin rule.
HubTokenServiceTests gains the truly-absent-field case: a hand-built payload
JSON with no Tags key at all, protected through the same purpose, which is the
shape every in-flight token has across the deploy that introduces the field. The
existing test covered present-but-empty, which does not exercise the null
coalesce that stands between a legacy token and a crash on the hub auth path.
ProtectorPurpose became internal so the test cannot drift from the real purpose
string.
Tag-count cardinality cap considered and recorded as a deliberate non-goal.
Build 0 warnings / 0 errors; 48 filtered (ACL/hub/token/page) and 257 dashboard
tests pass.
The truncation-cliff fix made alarm transitions truncation-safe but silent:
when GetXmlCurrentAlarms2 returns exactly maxAlmCnt records the worker
suppresses absence-implies-Clear inference and says so only in a rate-limited
stderr warning. No client and no operator could tell a complete active set
from a capped one.
Two additive proto3 booleans carry the verdict out:
- QueryActiveAlarmsReplyPayload.snapshot_truncated = 2 (worker IPC reply)
- ActiveAlarmSnapshot.from_truncated_snapshot = 16 (per record)
The per-record field is not an aesthetic choice. QueryActiveAlarms returns a
bare `stream ActiveAlarmSnapshot` with no envelope, header, or trailer, so a
per-record boolean is the only carrier that stays wire-compatible; an envelope
message would change every existing client's stream element type. The reply
payload states it too because a prefix filter can leave zero records and a
truncated fetch with nothing to report still has to say so. The flag means
"this set may be incomplete", never "this record is unreliable" — it is
independent of the subtag-fallback `degraded` field.
Detection is deliberately UNCHANGED: IsTruncatedFetch remains
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe (docs/AlarmProbeFindings.md,
ce5d8ae) could not verify whether ALARM_RECORDS/@COUNT reports the total active
count or only the records in the reply, so @COUNT is not parsed for detection;
switching to it stays blocked on probe evidence. The probe's comment
annotations in WnWrapAlarmConsumer.cs are preserved.
Reset semantics: not latched. WnWrapAlarmConsumer.FoldFetch replaces the
verdict on every poll under the same lock as the snapshot merge, so the first
sub-cap fetch clears it; GatewayAlarmMonitor.ClearCache drops it with the cache
generation it describes. A caveat that never turns off is one operators learn
to ignore.
Flow: WnWrapAlarmConsumer.LastSnapshotTruncated -> AlarmDispatcher (stamps every
record) / IAlarmCommandHandler (payload) -> MxAccessCommandExecutor reply ->
GatewayAlarmMonitor._snapshotTruncated -> IGatewayAlarmService.SnapshotTruncated
-> DashboardAlarmQueryResult -> AlarmsPage warning banner (render-side only; the
poll loop and DisposeAsync drain are untouched). The public QueryActiveAlarms
RPC forwards worker snapshots unmodified, so the per-record flag needed no
mapper change — a test pins that.
Parity: this describes OUR fetch mechanics — additive gateway metadata — not
MXAccess provider behavior. No event is synthesized and no MXAccess-observable
semantics change, so it is not a parity deviation.
Tests: worker LastSnapshotTruncated set/reset/consecutive-burst (windev-run);
gateway end-to-end truncated reply -> monitor -> public stream, with the
complete-reply control as the load-bearing assertion; AlarmsPage banner
present/absent. Docs: gateway.md alarm surface, docs/DesignDecisions.md entry.
Adds a dashboard event-visibility tag to ApiKeyConstraints, riding in the existing
constraints JSON blob so no auth-store schema migration is needed (design
docs/plans/2026-07-10-dashboard-session-acl-tst15.md sections 3/3.1, open call
settled per its own recommendation). The tag is visibility-only: no read, write,
browse, or subscribe path consults it, and HasRead/HasWriteConstraints ignore it.
GatewaySession gains an immutable, ordinal-ignore-case Tags set stamped at
construction from the owning API key, forwarded by MxAccessGatewayService.OpenSession
from the resolved ApiKeyIdentity — never from the wire request, so a client cannot
label its own session with another tenant's tag. ISessionManager gains a tag-carrying
OpenSessionAsync overload whose default implementation forwards to the tagless one, so
an implementation that does not model tags opens an untagged (least visible) session.
apikey create-key gains --dashboard-tags team-a,team-b (repeatable, trimmed,
de-duplicated; an empty segment is rejected rather than dropped) and list-keys prints
the tags column. No enforcement yet — the EventsHub ACL that consumes the tag is a
later change.
Groundwork for the per-session dashboard event ACL (docs/plans/2026-07-10-dashboard-session-acl-tst15.md 3.2): a dashboard group can now grant visibility tags, and untagged sessions default to AdminOnly. Enforcement lands with the EventsHub ACL; nothing consumes the grant yet.
GroupToTag is deliberately uncoupled from GroupToRole - a group may appear in either map, both, or neither - and is validated for shape only. Tags gate dashboard event visibility, never data access.
ISessionManager.ReadEventsAsync had zero production call sites: the worker
event channel is drained once by GatewaySession.MapWorkerEventsAsync (the
distributor pump), and every consumer — gRPC subscribers, the dashboard
mirror, the alarm monitor — attaches to the distributor. The interface
member, SessionManager's forwarder, and GatewaySession.ReadEventsAsync are
gone; IWorkerClient/WorkerClient.ReadEventsAsync is untouched, it is the
live worker-channel claim.
No test was removed or rewired: nothing invoked the member through the
interface. Nine ISessionManager test fakes carried a required-member stub
(seven threw NotSupportedException or yielded nothing; EventStreamServiceTests
and GatewaySessionDashboardMirrorTests forwarded to the session; the two
MxAccessGatewayService fakes yielded their Events list) — all nine stubs were
deleted. The MxAccessGatewayService suites' streaming tests already run
through FakeEventStreamService, which reads the same Events list, so their
coverage is unchanged; only the now-inaccurate doc comments on Events /
LastReadEventsSessionId were reworded.
The MapWorkerEventsAsync comment no longer describes a twin to keep in step;
it now states the single-reader claim directly. docs/Sessions.md drops
ReadEventsAsync from the SessionManager member list and from the Run-state
prose. The 2026-08-15 deferred-remediation as-built note records the removal.
The remediation reviews approved every task but left a tail of small notes.
This lands the gateway-side half of them.
Hardening (behavior changes, all narrow):
- BuildFilteredWriteBulkCommand's unreachable default case failed OPEN: a
fifth bulk-write kind added upstream without a filter case here would have
shipped the DENIED entries to the worker while reporting them denied to the
caller. It now throws UnreachableException.
- SqliteCanonicalAuditStore.ListRecentAsync no longer throws on a row it
cannot date. The retention sweep deliberately preserves such rows (SQLite's
datetime() yields NULL, so the DELETE never matches), which guaranteed the
dashboard's recent-audit view would meet one eventually and lose the whole
page to it. The row is now reported at DateTimeOffset.MinValue with every
other column intact, behind an optional logger.
- The audit drain loop's finally now completes the channel writer alongside
detaching the drain, so a producer that raced past the attached check takes
the write-through branch instead of stranding its event in a buffer nobody
reads until shutdown. TryComplete is idempotent, so StopAsync is unaffected.
Tests:
- MapCommandReply ownership (Assert.Same on the inner reply), mirroring the
existing MapEvent ownership test.
- Redactor key-id length boundary at exactly 64 and 65 characters, pinning
which way it fails. Nothing validates key-id length at creation, so
docs/Diagnostics.md's "which no issued key id does" is now stated as the
heuristic it is.
- ApiKeyFailureLimiter.Reset with a PartitionResolution whose partition was
evicted between the Check and the Reset: inert, and clears nobody else's
block.
- Constraint-cache concurrency stress: the cap is enforced by the inserting
thread, so overshoot must be transient and proportional to the in-flight
inserters, and the cache must settle at or under the cap.
- ListRecentAsync against a raw-SQL undateable row.
Comment/doc accuracy:
- EventsHubViewerRegistry.ReleaseConnection records that it relies on
SignalR's default sequential per-connection dispatch
(MaximumParallelInvocationsPerClient = 1).
- A PERF(followup) note on Invoke's double session resolve and why removing it
needs a SessionManager overload.
- SessionEventDistributor: the volatile-field comment named the pump as the
lock-free reader, but the pump's single capture point is inside _replayLock;
the genuinely lock-free reader is SubscriberCount. OnSubscriberOverflow's
"cannot be observed here" now excepts the DisposeAsync abandon path. The
churn test names its ConcurrentDictionary bucket-order assumption and that a
violation surfaces as a read timeout, not a silent pass.
- The two "restores the sequential drain's behavior" claims (SessionManager,
docs/Sessions.md) were wrong: the sequential drain leaked too, because
KillWorkerAsync's entry ThrowIfCancellationRequested aborted the whole loop
on the first session for zero kills. Reworded to "fixes a leak the
sequential drain also had", with the sweep-bound/shutdown-unbound
ParallelOptions asymmetry explained.
- ISessionManager.ShutdownAsync's token doc: it degrades the drain to a kill
sweep rather than cancelling it, with the bounded overrun stated.
SessionShutdownHostedService.StopAsync records that its cancellation-logging
branch is now unreachable.
Decrement was decrement-first with a single non-retried repair CAS. From zero,
two unmatched decrements (SignalR calls OnDisconnectedAsync for a connection
whose OnConnectedAsync faulted) capture -1 and -2; a real Increment then makes
the count -1, and the first decrementer's stale CompareExchange(0, -1) matches
and resets to zero — erasing a live connection, so the idle gate freezes an open
dashboard. The same lost race also made Decrement report 0 when it had not
written 0.
Clamping now happens inside the compare-and-swap: read, clamp, publish, retry on
loss. A lost race re-reads the fresh value instead of repairing a stale one.
The counter moves to its own file per the one-public-type-per-file convention and
gains direct tests: the zero floor under concurrent unmatched decrements, matched
pairs settling at zero, and an interleaved connect/disconnect stress round. The
stress test asserts the observable invariants only — the specific interleaving
cannot be forced through the public API (verified: the previous implementation
passes it), which its remarks now state rather than implying a reproducer. A hub
wiring test is skipped for the EventsHub reason: driving Hub.OnConnectedAsync
needs caller-clients and connection-context fakes, and the overrides are two
lines of delegation to the tested type.
Also documents that the API-key refresh's pre-gate time check races benignly.
The snapshot publisher broadcast to Clients.All on every ~1s tick forever, with
zero viewers. Each tick cost a session-registry snapshot and sort, a metrics
snapshot that copies dictionaries under the global metrics lock, a full rebuild
of EffectiveGatewayConfiguration, and a SQLite read of the API key table.
DashboardSnapshotHub now counts live connections into the singleton
DashboardSnapshotHubConnectionCounter (clamped at zero, since SignalR can call
OnDisconnectedAsync for a connection whose OnConnectedAsync faulted). The
publisher drives the snapshot enumerator by hand instead of await foreach: with
no connections it does not call MoveNextAsync at all, so the producing iterator
stays suspended at its yield and no snapshot is built — the gate removes the
build, not just the broadcast. It re-checks once a second, so the first viewer
resumes the tick within about one interval; that viewer is seeded immediately by
DashboardPageBase's synchronous GetSnapshot() and by the hub's OnConnectedAsync.
Two per-tick costs are bounded independently of the gate: the effective
configuration is startup-static (options are bound once at boot and never
reloaded), so it is built once and cached; and the API key summaries refresh at
most every 15s, since the list only changes when an operator creates, rotates,
or revokes a key. Only a successful refresh restarts the interval, so a failed
or timed-out read is still retried on the next tick with the previous summaries
left on screen.
Joseph Doherty