feat(alarms): structural degraded-status signal for truncated alarm snapshots

The truncation-cliff fix made alarm transitions truncation-safe but silent:
when GetXmlCurrentAlarms2 returns exactly maxAlmCnt records the worker
suppresses absence-implies-Clear inference and says so only in a rate-limited
stderr warning. No client and no operator could tell a complete active set
from a capped one.

Two additive proto3 booleans carry the verdict out:

- QueryActiveAlarmsReplyPayload.snapshot_truncated = 2 (worker IPC reply)
- ActiveAlarmSnapshot.from_truncated_snapshot = 16 (per record)

The per-record field is not an aesthetic choice. QueryActiveAlarms returns a
bare `stream ActiveAlarmSnapshot` with no envelope, header, or trailer, so a
per-record boolean is the only carrier that stays wire-compatible; an envelope
message would change every existing client's stream element type. The reply
payload states it too because a prefix filter can leave zero records and a
truncated fetch with nothing to report still has to say so. The flag means
"this set may be incomplete", never "this record is unreliable" — it is
independent of the subtag-fallback `degraded` field.

Detection is deliberately UNCHANGED: IsTruncatedFetch remains
`fetchedRecordCount >= maxAlarmsPerFetch`. The live probe (docs/AlarmProbeFindings.md,
ce5d8ae) could not verify whether ALARM_RECORDS/@COUNT reports the total active
count or only the records in the reply, so @COUNT is not parsed for detection;
switching to it stays blocked on probe evidence. The probe's comment
annotations in WnWrapAlarmConsumer.cs are preserved.

Reset semantics: not latched. WnWrapAlarmConsumer.FoldFetch replaces the
verdict on every poll under the same lock as the snapshot merge, so the first
sub-cap fetch clears it; GatewayAlarmMonitor.ClearCache drops it with the cache
generation it describes. A caveat that never turns off is one operators learn
to ignore.

Flow: WnWrapAlarmConsumer.LastSnapshotTruncated -> AlarmDispatcher (stamps every
record) / IAlarmCommandHandler (payload) -> MxAccessCommandExecutor reply ->
GatewayAlarmMonitor._snapshotTruncated -> IGatewayAlarmService.SnapshotTruncated
-> DashboardAlarmQueryResult -> AlarmsPage warning banner (render-side only; the
poll loop and DisposeAsync drain are untouched). The public QueryActiveAlarms
RPC forwards worker snapshots unmodified, so the per-record flag needed no
mapper change — a test pins that.

Parity: this describes OUR fetch mechanics — additive gateway metadata — not
MXAccess provider behavior. No event is synthesized and no MXAccess-observable
semantics change, so it is not a parity deviation.

Tests: worker LastSnapshotTruncated set/reset/consecutive-burst (windev-run);
gateway end-to-end truncated reply -> monitor -> public stream, with the
complete-reply control as the load-bearing assertion; AlarmsPage banner
present/absent. Docs: gateway.md alarm surface, docs/DesignDecisions.md entry.
This commit is contained in:
Joseph Doherty
2026-08-17 04:18:34 -04:00
parent b8b7b69ba0
commit 693a78db7d
41 changed files with 2217 additions and 309 deletions
@@ -0,0 +1,218 @@
using System.Runtime.CompilerServices;
using System.Security.Claims;
using System.Threading.Channels;
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Authorization;
using Microsoft.AspNetCore.Components.Web;
using Microsoft.AspNetCore.Components.Web.HtmlRendering;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using ZB.MOM.WW.MxGateway.Contracts.Proto;
using ZB.MOM.WW.MxGateway.Server.Dashboard;
using ZB.MOM.WW.MxGateway.Server.Dashboard.Components.Pages;
using ZB.MOM.WW.MxGateway.Server.Dashboard.Hubs;
namespace ZB.MOM.WW.MxGateway.Tests.Gateway.Dashboard;
/// <summary>
/// Covers the ACL gate on the session-details page's in-process subscribe seam (SEC-25 / TST-15).
/// </summary>
/// <remarks>
/// <para>
/// The 2026-08 in-process feed refactor gave the dashboard a second way to subscribe to a
/// session's events — <see cref="IDashboardSessionEventSubscriber"/>, used by this page — so
/// gating the hub join alone would leave the page as an ungated path to the same feed. The
/// denial assertion here is the one that proves the second seam is closed: it asserts that
/// <see cref="IDashboardSessionEventSubscriber.Subscribe"/> is never called, not merely that the
/// panel renders differently.
/// </para>
/// <para>
/// Rendered through the framework's static <see cref="HtmlRenderer"/>, the same idiom
/// <c>SecretsNavRenderTests</c> uses — no component-testing package, because the assertions are
/// about the emitted markup and the calls the lifecycle makes, not about interactivity.
/// </para>
/// </remarks>
public sealed class SessionDetailsPageEventAclTests
{
private const string SessionId = "session-1";
// Matched without the trailing possessive so the assertion does not depend on how the
// renderer escapes the apostrophe.
private const string DeniedMessage = "Not authorized for this session";
private const string WaitingMarker = "Waiting for events.";
/// <summary>A denied caller gets the message and no subscription is opened.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Page_WhenAclDenies_RendersMessageAndDoesNotSubscribe()
{
RecordingEventSubscriber subscriber = new();
string html = await RenderAsync(subscriber, allow: false);
Assert.Contains(DeniedMessage, html, StringComparison.Ordinal);
Assert.DoesNotContain(WaitingMarker, html, StringComparison.Ordinal);
Assert.Empty(subscriber.SubscribedSessionIds);
}
/// <summary>
/// The control for the denial above: an allowed caller subscribes and sees the ordinary
/// waiting state. Without this, a page that failed to render its events panel at all would
/// satisfy the "no subscription" assertion and the suite would report a working gate over a
/// broken panel.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Page_WhenAclAllows_SubscribesAndRendersWaitingState()
{
RecordingEventSubscriber subscriber = new();
string html = await RenderAsync(subscriber, allow: true);
Assert.Equal([SessionId], subscriber.SubscribedSessionIds);
Assert.Contains(WaitingMarker, html, StringComparison.Ordinal);
Assert.DoesNotContain(DeniedMessage, html, StringComparison.Ordinal);
}
private static async Task<string> RenderAsync(RecordingEventSubscriber subscriber, bool allow)
{
ServiceCollection services = new();
services.AddLogging();
services.AddSingleton<IDashboardSnapshotService>(new StubSnapshotService());
services.AddSingleton<IDashboardSnapshotFeed>(new IdleSnapshotFeed());
services.AddSingleton<IDashboardSessionAdminService>(new NonManagingSessionAdminService());
services.AddSingleton<IDashboardSessionEventSubscriber>(subscriber);
services.AddSingleton<IDashboardSessionAcl>(new StubSessionAcl(allow));
services.AddSingleton<AuthenticationStateProvider>(new StubAuthenticationStateProvider());
await using ServiceProvider provider = services.BuildServiceProvider();
await using HtmlRenderer renderer = new(provider, provider.GetRequiredService<ILoggerFactory>());
return await renderer.Dispatcher.InvokeAsync(async () =>
{
HtmlRootComponent output = await renderer.RenderComponentAsync<SessionDetailsPage>(
ParameterView.FromDictionary(new Dictionary<string, object?>
{
[nameof(SessionDetailsPage.SessionId)] = SessionId,
}));
return output.ToHtmlString();
});
}
private sealed class StubSessionAcl(bool allow) : IDashboardSessionAcl
{
/// <inheritdoc />
public bool CanViewSession(ClaimsPrincipal? principal, string sessionId) => allow;
}
private sealed class RecordingEventSubscriber : IDashboardSessionEventSubscriber
{
/// <summary>Gets the session ids <see cref="Subscribe"/> was called with, in order.</summary>
public List<string> SubscribedSessionIds { get; } = [];
/// <inheritdoc />
public IDashboardEventSubscription Subscribe(string sessionId)
{
SubscribedSessionIds.Add(sessionId);
return new IdleSubscription();
}
// A subscription whose channel never yields and never completes, so the page's pump parks
// exactly as it would against a quiet session.
private sealed class IdleSubscription : IDashboardEventSubscription
{
private readonly Channel<MxEvent> _channel = Channel.CreateUnbounded<MxEvent>();
/// <inheritdoc />
public ChannelReader<MxEvent> Reader => _channel.Reader;
/// <inheritdoc />
public void Dispose() => _channel.Writer.TryComplete();
}
}
private sealed class StubSnapshotService : IDashboardSnapshotService
{
/// <inheritdoc />
public DashboardSnapshot GetSnapshot() => new(
GeneratedAt: DateTimeOffset.UnixEpoch,
GatewayStartedAt: DateTimeOffset.UnixEpoch,
GatewayUptime: TimeSpan.Zero,
GatewayStatus: "Healthy",
GatewayVersion: "test",
Sessions:
[
new DashboardSessionSummary(
SessionId: SessionId,
BackendName: "backend",
State: SessionState.Ready,
ClientIdentity: "client",
ClientSessionName: "client-session",
ClientCorrelationId: "correlation",
OpenedAt: DateTimeOffset.UnixEpoch,
LastClientActivityAt: DateTimeOffset.UnixEpoch,
LeaseExpiresAt: null,
WorkerProcessId: null,
WorkerState: null,
LastWorkerHeartbeatAt: null,
EventsReceived: 0,
LastFault: null),
],
Workers: [],
Metrics: [],
Faults: [],
ApiKeys: [],
Configuration: null!,
Galaxy: null!);
/// <inheritdoc />
public IAsyncEnumerable<DashboardSnapshot> WatchSnapshotsAsync(CancellationToken cancellationToken) =>
new IdleSnapshotFeed().WatchAsync(cancellationToken);
}
// Parks until the page is disposed, so the base page's watch loop neither spins nor pushes a
// second snapshot mid-assertion.
private sealed class IdleSnapshotFeed : IDashboardSnapshotFeed
{
/// <inheritdoc />
public async IAsyncEnumerable<DashboardSnapshot> WatchAsync(
[EnumeratorCancellation] CancellationToken cancellationToken)
{
await Task.Delay(Timeout.Infinite, cancellationToken).ConfigureAwait(false);
yield break;
}
}
private sealed class NonManagingSessionAdminService : IDashboardSessionAdminService
{
/// <inheritdoc />
public bool CanManage(ClaimsPrincipal user) => false;
/// <inheritdoc />
public Task<DashboardSessionAdminResult> CloseSessionAsync(
ClaimsPrincipal user,
string sessionId,
CancellationToken cancellationToken) =>
Task.FromResult(DashboardSessionAdminResult.Fail("not supported"));
/// <inheritdoc />
public Task<DashboardSessionAdminResult> KillWorkerAsync(
ClaimsPrincipal user,
string sessionId,
CancellationToken cancellationToken) =>
Task.FromResult(DashboardSessionAdminResult.Fail("not supported"));
}
private sealed class StubAuthenticationStateProvider : AuthenticationStateProvider
{
/// <inheritdoc />
public override Task<AuthenticationState> GetAuthenticationStateAsync() =>
Task.FromResult(new AuthenticationState(new ClaimsPrincipal(new ClaimsIdentity(
[new Claim(ClaimTypes.Name, "viewer-user"), new Claim(ClaimTypes.Role, DashboardRoles.Viewer)],
authenticationType: "test",
nameType: ClaimTypes.Name,
roleType: ClaimTypes.Role))));
}
}