The remediation reviews approved every task but left a tail of small notes.
This lands the gateway-side half of them.
Hardening (behavior changes, all narrow):
- BuildFilteredWriteBulkCommand's unreachable default case failed OPEN: a
fifth bulk-write kind added upstream without a filter case here would have
shipped the DENIED entries to the worker while reporting them denied to the
caller. It now throws UnreachableException.
- SqliteCanonicalAuditStore.ListRecentAsync no longer throws on a row it
cannot date. The retention sweep deliberately preserves such rows (SQLite's
datetime() yields NULL, so the DELETE never matches), which guaranteed the
dashboard's recent-audit view would meet one eventually and lose the whole
page to it. The row is now reported at DateTimeOffset.MinValue with every
other column intact, behind an optional logger.
- The audit drain loop's finally now completes the channel writer alongside
detaching the drain, so a producer that raced past the attached check takes
the write-through branch instead of stranding its event in a buffer nobody
reads until shutdown. TryComplete is idempotent, so StopAsync is unaffected.
Tests:
- MapCommandReply ownership (Assert.Same on the inner reply), mirroring the
existing MapEvent ownership test.
- Redactor key-id length boundary at exactly 64 and 65 characters, pinning
which way it fails. Nothing validates key-id length at creation, so
docs/Diagnostics.md's "which no issued key id does" is now stated as the
heuristic it is.
- ApiKeyFailureLimiter.Reset with a PartitionResolution whose partition was
evicted between the Check and the Reset: inert, and clears nobody else's
block.
- Constraint-cache concurrency stress: the cap is enforced by the inserting
thread, so overshoot must be transient and proportional to the in-flight
inserters, and the cache must settle at or under the cap.
- ListRecentAsync against a raw-SQL undateable row.
Comment/doc accuracy:
- EventsHubViewerRegistry.ReleaseConnection records that it relies on
SignalR's default sequential per-connection dispatch
(MaximumParallelInvocationsPerClient = 1).
- A PERF(followup) note on Invoke's double session resolve and why removing it
needs a SessionManager overload.
- SessionEventDistributor: the volatile-field comment named the pump as the
lock-free reader, but the pump's single capture point is inside _replayLock;
the genuinely lock-free reader is SubscriberCount. OnSubscriberOverflow's
"cannot be observed here" now excepts the DisposeAsync abandon path. The
churn test names its ConcurrentDictionary bucket-order assumption and that a
violation surfaces as a read timeout, not a silent pass.
- The two "restores the sequential drain's behavior" claims (SessionManager,
docs/Sessions.md) were wrong: the sequential drain leaked too, because
KillWorkerAsync's entry ThrowIfCancellationRequested aborted the whole loop
on the first session for zero kills. Reworded to "fixes a leak the
sequential drain also had", with the sweep-bound/shutdown-unbound
ParallelOptions asymmetry explained.
- ISessionManager.ShutdownAsync's token doc: it degrades the drain to a kill
sweep rather than cancelling it, with the bounded overrun stated.
SessionShutdownHostedService.StopAsync records that its cancellation-logging
branch is now unreachable.
Decrement was decrement-first with a single non-retried repair CAS. From zero,
two unmatched decrements (SignalR calls OnDisconnectedAsync for a connection
whose OnConnectedAsync faulted) capture -1 and -2; a real Increment then makes
the count -1, and the first decrementer's stale CompareExchange(0, -1) matches
and resets to zero — erasing a live connection, so the idle gate freezes an open
dashboard. The same lost race also made Decrement report 0 when it had not
written 0.
Clamping now happens inside the compare-and-swap: read, clamp, publish, retry on
loss. A lost race re-reads the fresh value instead of repairing a stale one.
The counter moves to its own file per the one-public-type-per-file convention and
gains direct tests: the zero floor under concurrent unmatched decrements, matched
pairs settling at zero, and an interleaved connect/disconnect stress round. The
stress test asserts the observable invariants only — the specific interleaving
cannot be forced through the public API (verified: the previous implementation
passes it), which its remarks now state rather than implying a reproducer. A hub
wiring test is skipped for the EventsHub reason: driving Hub.OnConnectedAsync
needs caller-clients and connection-context fakes, and the overrides are two
lines of delegation to the tested type.
Also documents that the API-key refresh's pre-gate time check races benignly.
The snapshot publisher broadcast to Clients.All on every ~1s tick forever, with
zero viewers. Each tick cost a session-registry snapshot and sort, a metrics
snapshot that copies dictionaries under the global metrics lock, a full rebuild
of EffectiveGatewayConfiguration, and a SQLite read of the API key table.
DashboardSnapshotHub now counts live connections into the singleton
DashboardSnapshotHubConnectionCounter (clamped at zero, since SignalR can call
OnDisconnectedAsync for a connection whose OnConnectedAsync faulted). The
publisher drives the snapshot enumerator by hand instead of await foreach: with
no connections it does not call MoveNextAsync at all, so the producing iterator
stays suspended at its yield and no snapshot is built — the gate removes the
build, not just the broadcast. It re-checks once a second, so the first viewer
resumes the tick within about one interval; that viewer is seeded immediately by
DashboardPageBase's synchronous GetSnapshot() and by the hub's OnConnectedAsync.
Two per-tick costs are bounded independently of the gate: the effective
configuration is startup-static (options are bound once at boot and never
reloaded), so it is built once and cached; and the API key summaries refresh at
most every 15s, since the list only changes when an operator creates, rotates,
or revokes a key. Only a successful refresh restarts the interval, so a failed
or timed-out read is still retried on the next tick with the previous summaries
left on screen.
DashboardEventBroadcaster.Publish ran a deep protobuf Clone (redaction is on by
default) and a group SendAsync for every event of every session, before anything
checked whether a dashboard client was actually watching. In the steady state the
session:{id} group is empty, so that work was thrown away per event.
SignalR does not expose group membership, so EventsHub now mirrors its own
add/remove into a singleton EventsHubViewerRegistry, and OnDisconnectedAsync
releases everything a dropped connection held (SessionDetailsPage disposes the
connection rather than unsubscribing). Publish returns early when the session has
no viewers, before the redaction clone. Watched sessions behave exactly as before.
Lazy mirror-lease start/stop was deliberately not attempted — it entangles the
dashboard with SessionEventDistributor subscribe lifetime for no saving beyond
this gate; recorded in docs/GatewayDashboardDesign.md.
Family-wide nav sweep: the Secrets management page should be linked from
each app's UI, visible to Administrator-role users only.
The link already existed in MainLayout's Admin section. The gate did not:
the rail rendered every item for every visitor, including a Viewer and the
anonymous-localhost read-only identity. Not an access hole — the mounted
page carries [Authorize(Policy = "secrets:manage")], so a Viewer clicking
through was denied — but a dead link presented as a live one. There was
also no existing role-gated nav pattern to follow; the rail's only
AuthorizeView was the footer's signed-in/signed-out split.
Gated on the POLICY rather than a role literal, so nav visibility cannot
drift from what the page enforces. In this host the two are equivalent:
GatewayOptionsValidator constrains Dashboard:GroupToRole values to
Administrator or Viewer, so the shared library's other manage-granting
roles (secrets-manager, secrets-reveal) are unreachable. The policy form
stays correct if that ever relaxes, where a role literal would then hide
the link from users who can use the page.
API Keys is deliberately left ungated. It looks like the same case and is
not: ApiKeysPage renders for a Viewer with write affordances hidden, so
hiding its link would remove legitimate read access. The secrets page has
no read-only mode. The rule is "gate the link when the page denies the
role outright", not "gate everything under Admin".
Coverage: three tests pin the policy's verdict per principal
(Administrator admitted, Viewer refused, unauthenticated refused), and
/admin/secrets joins the canonical route list — it is the one nav
destination mounted from an RCL rather than declared here, so a routing
regression could remove it without touching this repo's pages. The
principal helper sets an authentication type deliberately: without one
the role assertions would pass vacuously for the wrong reason.
Not a rendering test — the suite has no component-testing harness, and
adding one to assert a single AuthorizeView would be a large dependency
for a small claim.
Build 0 warnings / 0 errors; suite 895/895.
Auth 0.1.5 -> 0.2.0, Health 0.2.0 -> 0.3.0, Secrets/.Abstractions/.Ui
0.6.1 -> 0.6.2. Theme, GalaxyRepository, Audit, Configuration, Telemetry
and Telemetry.Serilog were already at the newest version on the feed.
Checked against the shared-lib source rather than the version numbers,
because these packages are versioned as a family and a bump is not by
itself evidence that the package changed:
- Auth 0.2.0 is the only one carrying content for us: LDAP backup-DC
failover (FallbackServers, endpoint walk with sticky preference,
boot-time entry validation). Purely additive; the default is empty,
which leaves single-endpoint behaviour unchanged.
- Health 0.3.0 carries a breaking change, but every line of it is in
ZB.MOM.WW.Health.Akka, which we do not reference. No commit touched
the core ZB.MOM.WW.Health package between 0.2.0 and 0.3.0.
- Secrets 0.6.2 is a message-only change: one validator string literal
gains mounted-volume guidance. SecretsStorePathRules is untouched.
The four non-csproj files are not a separate feature. Configuration/
LdapOptions is a deliberate shadow of the shared type and carries an
explicit warning to mirror any new upstream field, because AddZbLdapAuth
binds the whole MxGateway:Ldap section onto the shared options. So
FallbackServers is live on our config surface the moment the package
lands, and without the mirror an operator could configure a backup DC
that works but is invisible on the dashboard's Settings page. The
Settings row renders "none" when empty, since that is the answer someone
who believes a backup DC is configured actually needs.
Entry syntax is deliberately NOT re-validated here: the shared validator
already fails the boot on a malformed entry and owns the (internal)
parser, so a second copy would drift. Note both validators skip entirely
when Ldap:Enabled is false.
Verified the binder is non-strict (ErrorOnUnknownConfiguration is unused
anywhere in the tree), so the upgrade could not break startup on a
newly-recognised key either way.
Build 0 warnings / 0 errors; gateway suite 892/892, unchanged. The live
LDAP tests are opt-in and were not run, so the failover path itself is
covered only by the shared library's own tests.
Family-wide admin-UI cleanup pass (scadaproj admin_ui_cleanup.md) applied to the
Blazor dashboard. Behaviour is unchanged throughout — no @onclick, disabled,
binding, auth gate, or arm->confirm flow was touched.
Uncontrolled error text is now truncated at the render site. Fault messages,
Galaxy load errors, and browse-tree load failures were rendered in full into
fixed-width table cells, where a long exception string blows out the column.
Each site gets DashboardDisplay.Abbreviate plus a title attribute carrying the
untruncated text, so nothing becomes unreachable. Abbreviate is length-checked
rather than a bare range slice: `value[..n]` on a shorter string throws and
takes the whole page render down with it. The two detail views whose entire
purpose is to show one fault in full — SessionDetailsPage and GalaxyPage's
Last Error — are deliberately left untruncated.
Two classes referenced from markup had no definition anywhere in the sheet.
.browse-stale-banner was inert; .tree-load-status was a real visual defect —
loading and failed-to-load rows sit among .tree-row siblings and carry the same
leading .tree-toggle-empty spacer, but that spacer only takes its width as a
flex item, so without a flex container those rows lost their indent.
Confirm/cancel pairs in ConfirmDialog and the API-key create form are now
btn-groups with role="group" and an aria-label, replacing margin-spaced loose
buttons.
Removes a paragraph on GalaxyPage naming internal RPCs (DiscoverHierarchy,
GetLastDeployTime) — implementation detail with no meaning to a dashboard
operator.
Verified in a real browser, not bUnit: full build clean, 879/879 tests, and a
live gate against a running dashboard with a genuine ~250-char SqlClient
exception as the erroring row. Results per check, including the checks that
could NOT be exercised without an x86 worker, are recorded in
docs/plans/2026-08-11-dashboard-ui-sweeps.md.
That plan doc also records a correction: this app is NOT Bootstrap-free. The
sweep brief said it was, citing the scadaproj index; libman.json pins
bootstrap 5.3.3 and App.razor:7 links it ahead of the theme. The stale claim had
already cost this app one skipped family sweep (scadaproj#2, the /admin/secrets
modal), so that modal was live-gated here too and passes.
Resolve all CommentChecker findings across the gateway server, worker, tests,
and .NET client (314 -> 0 real issues): add missing <returns>/<summary>/<param>
on public and test members, convert Stream/interface overrides to <inheritdoc/>,
and remove internal task/issue tracking IDs (SEC-*, IPC-*, WRK-*, GWC-*, TST-*,
Client.Dotnet-*) from shipped code documentation while preserving the design
rationale prose. Shipped comments should not carry internal bookkeeping, and
complete XML docs keep the analyzer/TreatWarningsAsErrors gate and generated API
docs clean. The 6 remaining flags are heuristic false positives (MD5, UTC-4,
capacity-1, near-1601) left intact so real documentation is not corrupted.
Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
SEC-25 (near-term hardening; full per-session EventsHub ACL stays deferred to
roadmap item 12): DashboardEventBroadcaster now redacts tag values from a deep
CLONE of the event before mirroring to SignalR when Dashboard:ShowTagValues is
false (the default). Clears MxEvent.value (covers OnDataChange/OnWriteComplete/
OperationComplete/OnBufferedDataChange — their bodies are empty discriminators,
values ride in the top-level field incl. buffered arrays) and the alarm body's
current_value/limit_value. Source event never mutated (shared with the gRPC
path + replay ring) - verified by a source-not-mutated test. Makes the formerly
dead ShowTagValues flag live for the mirror. EventsHub TODO(per-session-acl)
kept and tied to roadmap item 12. Tests: DashboardEventBroadcasterTests (3).
SEC-30: trim docs/Diagnostics.md to mark opt-in command-value logging as NOT
YET IMPLEMENTED (no LogCommandValues knob, RedactCommandValue has no call
sites, no values logged); wiring deferred pending secured-bulk redaction
coverage (SEC-13). No option/call sites added.
Server build clean (0 warnings); Dashboard tests 152/152.
Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
Restore the __Host- browser guarantees for the default secure deployment
without breaking plaintext/dev:
- DashboardServiceCollectionExtensions PostConfigure now resolves the cookie
name as: explicit MxGateway:Dashboard:CookieName override wins; else
__Host-MxGatewayDashboard when SecurePolicy==Always (RequireHttpsCookie
true); else the plain MxGatewayDashboard default. Guard: never apply the
__Host- prefix without Secure (browsers silently drop it).
- DashboardAuthenticationDefaults: add SecureCookieName const; keep the plain
CookieName as the non-secure fallback.
- Docs corrected to the actual conditional contract (five stale claims):
gateway.md, GatewayProcessDesign.md, ImplementationPlanGateway.md,
GatewayDashboardDesign.md, CLAUDE.md; GatewayConfiguration.md phrasing
tightened.
- Test: DashboardCookieOptionsTests asserts the name flips with
RequireHttpsCookie and that an explicit override wins.
Server build clean (0 warnings); Dashboard tests 149/149.
Claude-Session: https://claude.ai/code/session_01DMXXvNuPekkkrTEyPNxEkW
Gateway-side follow-up to the shared auth-lib expiry core (delivered via G-2):
- apikey create-key gains optional --expires — absolute ISO-8601 UTC or relative
<N>d/<N>h from now; omitted means non-expiring (opt-in, unchanged default).
Threaded ApiKeyAdminCommand -> parser -> runner into the library's
CreateKeyAsync(..., expiresUtc, ...). list-keys shows an expiry column and an
active/expired/revoked status.
- Dashboard API Keys page surfaces expiry: an Expires column (Never when unset)
and a status badge reading Expired (red) / Expiring (<=7d, amber) / Revoked /
Active. DashboardApiKeySummary.ExpiresUtc projected in DashboardSnapshotService;
StatusBadge maps the new states.
Tests: parser (absolute/relative/invalid/none) + end-to-end past-expiry rejection
through the live verifier; dashboard summary suite green. Docs: Authentication.md
(verification-flow expiry step, CLI table + examples, dashboard badge).
Closes the tracked SEC-10 polish (SEC-10 core was already Done).
- SEC-02: DashboardAuthorizationHandler restricts the loopback and
Authentication:Mode=Disabled bypasses to read-only. They now satisfy only a
Viewer-bearing requirement (AnyDashboardRole), never AdminOnly, so anonymous
localhost can view the dashboard but cannot reach API-key CRUD or session
Close/Kill at the policy layer (previously guarded only by service re-checks).
- SEC-12: DashboardSessionAdminService emits canonical AuditEvents through
IAuditWriter (actions dashboard-close-session / dashboard-kill-worker, category
SessionAdmin) on Success/Failure/Denied, mirroring the API-key audit path so
destructive session actions leave durable, queryable rows.
- SEC-20: drop the unbounded session_id tag from the exported
mxgateway.heartbeats.failed counter (per-session detail stays in the snapshot/log).
Docs updated same-change: CLAUDE.md (read-only loopback + 5-min bearer),
GatewayDashboardDesign.md (bypass scoping + session-admin audit), Metrics.md.
Server build clean; 30/30 targeted + 295/295 Dashboard/Security/App/Metrics sweep.
SEC-07: add QueryActiveAlarmsRequest -> events:read scope arm; fix two tests that
constructed StreamAlarmsRequest instead of QueryActiveAlarmsRequest.
SEC-05: shorten hub-token lifetime 30m -> 5m; document that the ?access_token= query
carriage must never be request-logged.
SEC-08: gateway-side CachingApiKeyVerifier (short TTL, keyed on a hash of the presented
secret) skips the per-call store read+last_used write; CoalescingMarkApiKeyStore bounds
last_used writes to <=1/key/min; identity constraints are cached. Invalidation is wired
at the gateway admin sites (revoke/rotate/delete); short TTL backstops out-of-process CLI.
SEC-11: fixed-window rate limit on POST /auth/login + a per-peer (key-id) failure limiter
checked before VerifyAsync; new MxGateway:Security options bound + validated.
Also fixes regressions from the SEC-01/04/06 commit (c185f62) that a narrow test filter
missed (all now covered by a full-suite checkpoint):
- Rooting check is cross-platform: accepts Windows C:\/UNC forms on Unix so the shipped
appsettings path validates on the macOS dev box, still rejecting bare filenames.
- AddGatewayConfiguration TryAdds a non-production IHostEnvironment fallback so the validator
resolves in minimal test/tooling containers; the real host + apikey CLI register the actual
environment first (TryAdd no-op there).
- Test assembly defaults ASPNETCORE_ENVIRONMENT=Development (ModuleInitializer) so full-host
tests exercise wiring instead of tripping the SEC-04/06 production guards.
- GatewayOptionsTests asserts the SEC-01 CommonApplicationData-derived default (platform-correct).
archreview: SEC-07/05/08/11 (P1). Verified: NonWindows build clean; full gateway suite
747 passed / 42 failed, where all 42 are the pre-existing macOS named-pipe-harness failures
(Unix-socket path limit) and 0 are validation/regression failures.
Resolves Server-059, Tests-041, Server-060 (2026-06-25 re-review).
DashboardSnapshotService memoized the whole Galaxy summary keyed on the cache
Sequence, but the shared library bumps Sequence only on a heavy refresh: the
steady-state tick, the refresh-failure path, and the age-based status getter all
replace the entry via 'previous with { ... }' at the SAME Sequence. The dashboard
therefore froze LastQueriedAt and, during a Galaxy SQL outage, kept showing
Healthy with no error for the whole outage.
Split DashboardGalaxySummaryProjector into ComputeBreakdown (the O(N) template/
category work, the only sequence-bound part) and BuildSummary (cheap volatile
fields copied fresh). ResolveGalaxySummary now memoizes only the breakdown by
Sequence and rebuilds the summary from the current entry each tick. Removed the
redundant DashboardGalaxyProjector wrapper.
Tests: same-sequence status/error/timestamp now reflected (the regression);
memoization-hit and sequence-invalidation guards; GatewayApplicationTests asserts
the DI container resolves IGalaxyBrowseScopeProvider to GatewayBrowseScopeProvider
(pins the registration-order invariant over the library's no-op default).
(1) GatewaySession.StartDashboardMirror: publish _dashboardMirrorLease and _dashboardMirrorTask
atomically under one _syncRoot section; if the session is already Closing/Closed/Faulted,
dispose the just-created lease and return without starting the mirror task so nothing is orphaned.
(2) WaitUntilAsync test helper: catch OperationCanceledException and call Assert.Fail with the
timeout duration and predicate source text instead of letting the exception propagate raw.
(3) New SessionEventDistributorTests.InternalSubscriberOverflow_HandlerSeesIsOnlySubscriberFalse:
verifies CountExternalSubscribers excludes the internal subscriber, so isOnlySubscriber==false
even when the internal subscriber is the only registered subscriber.
(4) SubscriberOverflowHandler delegate gains isInternal parameter; overflow metric label is
"dashboard-mirror" for internal subscribers and "grpc-event-stream" for external ones.
(5) DashboardEventBroadcaster.Publish: wrap SendAsync Task acquisition in try/catch so a
synchronous throw cannot escape the never-throw Publish interface contract.
Add a nullable string? OwnerKeyId property to GatewaySession that captures
the API key identifier (KeyId) of the authenticated caller that opened the
session. Wire it through ISessionManager.OpenSessionAsync → SessionManager
→ GatewaySession constructor. The gRPC service passes identityAccessor
.Current?.KeyId; internal callers (GatewayAlarmMonitor, DashboardLiveDataService)
pass null. Covers the positive and null cases with two new TDD-first tests.
- docs/plans/2026-06-14-deferred-followups.md: mark D1 as executed
(commit 4af24b9; metric emitted at DashboardSnapshotService.cs:198);
note D2 resolved as no-op; D3-D5 remain pending
- docs/AlarmClientDiscovery.md §5: rewrite STA "production fix needed"
to past tense — alarms now route through GatewayAlarmMonitor/worker STA
- EventsHub.cs: replace stale "publisher side is a future follow-up"
comment; DashboardEventBroadcaster is live and DI-registered
- CLAUDE.md: fix all project-name drift (src/MxGateway.* →
src/ZB.MOM.WW.MxGateway.*; MxGateway.sln → ZB.MOM.WW.MxGateway.slnx;
clients/dotnet/MxGateway.Client.sln → ZB.MOM.WW.MxGateway.Client.slnx)
- GalaxyRepositoryGrpcService.cs: remove dead MapSqlException method and
its IDE0051 suppression pragma; drop now-unused ILogger ctor param and
Microsoft.Data.SqlClient using; build confirmed 0 warnings/errors
Render Fallback:Mode=ForceSubtag as a cyan 'Subtag monitoring (forced)'
badge, distinct from the amber failover 'degraded' badge, so an intentional
configuration isn't shown as a fault. Distinguished by the shared
AlarmProviderReasons.ForcedSubtag reason carried on the provider-status feed.
The themed Blazor <LoginCard> page (Components/Pages/Login.razor, @page "/login")
registers a Razor Components endpoint that matches ALL HTTP methods. The credential
form POSTed to /login, where MapPost("/login") also matched — so every POST /login
threw Microsoft.AspNetCore.Routing.Matching.AmbiguousMatchException (HTTP 500),
breaking dashboard login for every user. It was latent because the dashboard was only
ever reached via the AllowAnonymousLocalhost bypass on the host box.
Move the credential POST to a distinct /auth/login route (mirroring ScadaBridge, which
never collided because it posts to /auth/login). GET /login stays the Blazor page; the
cookie LoginPath stays /login. Adds a registration assertion pinning DashboardLoginPost
to /auth/login as the regression guard.
Files: Login.razor (LoginCard Action), DashboardEndpointRouteBuilderExtensions (MapPost
route), GatewayApplicationTests (route assertion).
The dashboard auth cookie name was hardcoded to the constant
DashboardAuthenticationDefaults.CookieName (MxGatewayDashboard). Browser
cookies are scoped by host+path but NOT by port, so two gateway instances
sharing a hostname would clobber each other's dashboard session under the
shared name.
Add DashboardOptions.CookieName (MxGateway:Dashboard:CookieName); null/blank
keeps the canonical default. Applied in the existing dashboard cookie
PostConfigure (runs after the inline AddCookie default, so it wins). Behaviour
is unchanged when unset. Adds a Tests case for the override.
Repoint the server-rendered sign-in/fallback HTML (DashboardEndpointRouteBuilderExtensions) from /css/theme.css to the kit's _content/ZB.MOM.WW.Theme/css/{theme,layout}.css, mirroring ThemeHead, since that static page cannot use the Razor component.
Joseph Doherty