Joseph Doherty
a2f0086fa1
ROW 80 (observability correctness). An unreplicated site published
LocalDbReplicationConnected=false / LocalDbOplogBacklog=0, contradicting the
nullable semantics every doc and the proto already state ("null means no
reading, NOT disconnected with an empty backlog").
Root cause: the replication engine is registered unconditionally and idles
without a peer, so ISyncStatus answers false/0 on a node that was never meant to
replicate, and LocalDbReplicationStatusReporter passed that straight through.
Two consequences: an operator read connected:false on a healthy site as a fault,
and central's null branches — row 33's conditional KPI emission and
sustained-badge anchor exclusion — never received the null they exist for, so
rig site-b/site-c emitted a permanently misleading
localDbReplicationConnected=0 series.
Fix mirrors the SiteEventLogReplicationCheck precedent: the Host evaluates
SiteLocalDbSetup.ReplicationIsConfigured ONCE and passes it to
AddLocalDbReplicationHealthBridge as a third provider; unconfigured
short-circuits the probe to (null, null). HealthMonitoring never reads LocalDb
config itself, and the same captured value now also feeds
SiteEventLogReplicationCheck so the two surfaces cannot disagree.
ISiteHealthCollector.SetLocalDbReplicationStatus widens to bool? — the
non-nullable bool was the type-level root cause (null was inexpressible).
false now means exactly one thing: configured, peer session down = a real fault.
Wire/central/UI needed no change: the proto already carries BoolValue/Int64Value
wrappers and the aggregator + KPI source already branch on null. This makes code
match doc, so no doc claim was weakened.
Watch the trap that caused this: configured-ness is PeerAddress OR ApiKey (only
the dialling half sets a peer address). The pre-existing SiteLocalDbWiringTests
fixture sets ApiKey and IS a replicating node, so its false/0 assertion is
correct and is kept, with a comment saying so; the new
UnreplicatedSiteLocalDbHealthTests fixture is the genuinely-unreplicated one.
ROW 73 (silent loss of self-heal). AkkaHostedService skips
SiteReconciliationActor when no IDeploymentConfigFetcher is resolvable, and that
skip was a lone boot Warning — a node with no self-heal path at all went on
reporting itself perfectly healthy forever.
Adds tri-state SiteHealthReport.StartupReconcilerActive (null = not reported,
i.e. the central self-report or a pre-row-73 site build; true = running;
false = booted without it), latched once at boot by the Host in both branches.
Additive proto field 29 (BoolValue, vendored C# regenerated via
docker/regen-proto.sh centralcontrol; csproj restored, verified empty diff).
The skip branch also raises its log to Error — no configuration makes the
fetcher legitimately absent on a site node — and writes a best-effort
deployment/Error site event so the evidence outlives the boot log.
Central treatment: a "No self-heal" WARNING badge on the site tile, not a
failure. Config still reaches the node via LocalDb replication while it is up;
what it lost is recovery from a deploy that landed while it was down — degraded,
not offline. Only the explicit false is badged (badging null would light up the
whole fleet the moment central is upgraded ahead of the sites), and offline
sites are skipped, matching the replication-backlog badge.
Fail-before evidence: with the row-80 gate behaviourally reverted,
UnreplicatedSite_HealthReport_ReportsNullForBothLocalDbFields fails
"Expected: null / Actual: False" and 3 of the 8 new reporter unit tests fail.
Tests: HealthMonitoring 128, Host 518, CentralUI 1051, Communication 717,
Commons 715, SiteEventLogging 81 — all green.
651 lines
28 KiB
C#
651 lines
28 KiB
C#
using Google.Protobuf;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Audit;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Audit;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Deployment;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Health;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Notification;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Types;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
|
|
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
|
|
using ZB.MOM.WW.ScadaBridge.Communication.Grpc;
|
|
|
|
namespace ZB.MOM.WW.ScadaBridge.Communication.Tests.Grpc;
|
|
|
|
/// <summary>
|
|
/// Round-trip golden tests for <see cref="CentralControlDtoMapper"/> — the DTO bridge
|
|
/// for the seven <c>CentralControlService</c> RPCs (Phase 1A of the ClusterClient→gRPC
|
|
/// migration).
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// Every message gets a pair: a FULLY-POPULATED case that proves no field is dropped,
|
|
/// and a MINIMAL case that proves every nullable/optional/empty-collection member comes
|
|
/// back as null-or-empty rather than as a zero-valued stand-in. The minimal cases are
|
|
/// the ones that matter: a per-field unit test happily passes while a whole optional
|
|
/// branch is silently never written, which is exactly the class of bug the round-trip
|
|
/// guard in PLAN-05 T8 caught five times over.
|
|
/// </para>
|
|
/// <para>
|
|
/// When a round-trip here fails, the mapper is wrong — not the test.
|
|
/// </para>
|
|
/// </remarks>
|
|
public class CentralControlDtoMapperTests
|
|
{
|
|
private static readonly DateTimeOffset SampleInstant =
|
|
new(2026, 7, 22, 9, 30, 15, 250, TimeSpan.Zero);
|
|
|
|
// -----------------------------------------------------------------------
|
|
// NotificationSubmit / Ack
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Fact]
|
|
public void NotificationSubmit_RoundTrip_FullyPopulated_PreservesEveryField()
|
|
{
|
|
var original = new NotificationSubmit(
|
|
NotificationId: Guid.NewGuid().ToString(),
|
|
ListName: "plant-ops",
|
|
Subject: "Tank 4 overfill",
|
|
Body: "Level exceeded 95% for 5 minutes.",
|
|
SourceSiteId: "site-a",
|
|
SourceInstanceId: "Tank04",
|
|
SourceScript: "OnLevelHigh",
|
|
SiteEnqueuedAt: SampleInstant,
|
|
OriginExecutionId: Guid.NewGuid(),
|
|
OriginParentExecutionId: Guid.NewGuid(),
|
|
SourceNode: "node-b");
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
// Every member is a scalar, so record value-equality is a true deep compare.
|
|
Assert.Equal(original, roundTripped);
|
|
}
|
|
|
|
[Fact]
|
|
public void NotificationSubmit_RoundTrip_AllOptionalsNull_StayNull()
|
|
{
|
|
var original = new NotificationSubmit(
|
|
NotificationId: Guid.NewGuid().ToString(),
|
|
ListName: "plant-ops",
|
|
Subject: "s",
|
|
Body: "b",
|
|
SourceSiteId: "site-a",
|
|
SourceInstanceId: null,
|
|
SourceScript: null,
|
|
SiteEnqueuedAt: SampleInstant,
|
|
OriginExecutionId: null,
|
|
OriginParentExecutionId: null,
|
|
SourceNode: null);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(original, roundTripped);
|
|
Assert.Null(roundTripped.SourceInstanceId);
|
|
Assert.Null(roundTripped.SourceScript);
|
|
Assert.Null(roundTripped.OriginExecutionId);
|
|
Assert.Null(roundTripped.OriginParentExecutionId);
|
|
Assert.Null(roundTripped.SourceNode);
|
|
}
|
|
|
|
[Fact]
|
|
public void NotificationSubmit_NonUtcOffset_NormalizesToTheSameInstant()
|
|
{
|
|
// Documented contract: a protobuf Timestamp is an instant, so the offset
|
|
// component of a DateTimeOffset does not survive. Every producer stamps UTC
|
|
// (Notify.Send uses DateTimeOffset.UtcNow), so the instant is what matters.
|
|
var melbourne = new DateTimeOffset(2026, 7, 22, 19, 30, 15, TimeSpan.FromHours(10));
|
|
|
|
var original = new NotificationSubmit(
|
|
"id", "list", "s", "b", "site-a", null, null, melbourne);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(melbourne.UtcDateTime, roundTripped.SiteEnqueuedAt.UtcDateTime);
|
|
Assert.Equal(TimeSpan.Zero, roundTripped.SiteEnqueuedAt.Offset);
|
|
}
|
|
|
|
[Fact]
|
|
public void NotificationSubmitAck_RoundTrip_Accepted_And_Rejected()
|
|
{
|
|
var accepted = new NotificationSubmitAck("n1", Accepted: true, Error: null);
|
|
var rejected = new NotificationSubmitAck("n2", Accepted: false, Error: "list not found");
|
|
|
|
Assert.Equal(accepted, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(accepted))));
|
|
Assert.Equal(rejected, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(rejected))));
|
|
Assert.Null(CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(accepted))).Error);
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// NotificationStatusQuery / Response
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Fact]
|
|
public void NotificationStatusQuery_RoundTrip_PreservesEveryField()
|
|
{
|
|
var original = new NotificationStatusQuery("corr-1", Guid.NewGuid().ToString());
|
|
|
|
Assert.Equal(original, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original))));
|
|
}
|
|
|
|
[Fact]
|
|
public void NotificationStatusResponse_RoundTrip_FullyPopulated_PreservesEveryField()
|
|
{
|
|
var original = new NotificationStatusResponse(
|
|
CorrelationId: "corr-1",
|
|
Found: true,
|
|
Status: "Delivered",
|
|
RetryCount: 3,
|
|
LastError: "smtp 421",
|
|
DeliveredAt: SampleInstant);
|
|
|
|
Assert.Equal(original, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original))));
|
|
}
|
|
|
|
[Fact]
|
|
public void NotificationStatusResponse_RoundTrip_NotFound_LeavesOptionalsNull()
|
|
{
|
|
var original = new NotificationStatusResponse(
|
|
CorrelationId: "corr-1",
|
|
Found: false,
|
|
Status: "Unknown",
|
|
RetryCount: 0,
|
|
LastError: null,
|
|
DeliveredAt: null);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(original, roundTripped);
|
|
Assert.Null(roundTripped.LastError);
|
|
Assert.Null(roundTripped.DeliveredAt);
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Audit ingest envelopes (rows delegate to AuditEventDtoMapper / SiteCallDtoMapper)
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Fact]
|
|
public void IngestAuditEventsCommand_RoundTrip_PreservesOrderAndRows()
|
|
{
|
|
var first = NewAuditEvent(sourceScript: "OnDemand");
|
|
var second = NewAuditEvent(sourceScript: "OnTrigger");
|
|
var original = new IngestAuditEventsCommand([first, second]);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(2, roundTripped.Events.Count);
|
|
Assert.Equal(first.AsRow().EventId, roundTripped.Events[0].AsRow().EventId);
|
|
Assert.Equal("OnDemand", roundTripped.Events[0].AsRow().SourceScript);
|
|
Assert.Equal(second.AsRow().EventId, roundTripped.Events[1].AsRow().EventId);
|
|
Assert.Equal("OnTrigger", roundTripped.Events[1].AsRow().SourceScript);
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestAuditEventsCommand_RoundTrip_EmptyBatch_StaysEmpty()
|
|
{
|
|
var roundTripped = CentralControlDtoMapper.FromDto(
|
|
Wire(CentralControlDtoMapper.ToDto(new IngestAuditEventsCommand([]))));
|
|
|
|
Assert.Empty(roundTripped.Events);
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestCachedTelemetryCommand_RoundTrip_PreservesBothHalvesOfEachPacket()
|
|
{
|
|
var audit = NewAuditEvent(sourceScript: "OnDemand");
|
|
var siteCall = NewSiteCall();
|
|
var original = new IngestCachedTelemetryCommand([new CachedTelemetryEntry(audit, siteCall)]);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
var entry = Assert.Single(roundTripped.Entries);
|
|
Assert.Equal(audit.AsRow().EventId, entry.Audit.AsRow().EventId);
|
|
|
|
// IngestedAtUtc is central-set inside the dual-write transaction and is
|
|
// deliberately off the wire, so it is the one member excluded from the compare.
|
|
Assert.Equal(siteCall with { IngestedAtUtc = default }, entry.SiteCall with { IngestedAtUtc = default });
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestCachedTelemetryCommand_RoundTrip_NullableSiteCallFields_StayNull()
|
|
{
|
|
var siteCall = NewSiteCall() with
|
|
{
|
|
SourceNode = null,
|
|
LastError = null,
|
|
HttpStatus = null,
|
|
TerminalAtUtc = null,
|
|
};
|
|
var original = new IngestCachedTelemetryCommand(
|
|
[new CachedTelemetryEntry(NewAuditEvent(), siteCall)]);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
var entry = Assert.Single(roundTripped.Entries);
|
|
Assert.Null(entry.SiteCall.SourceNode);
|
|
Assert.Null(entry.SiteCall.LastError);
|
|
Assert.Null(entry.SiteCall.HttpStatus);
|
|
Assert.Null(entry.SiteCall.TerminalAtUtc);
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestCachedTelemetryCommand_RoundTrip_EmptyBatch_StaysEmpty()
|
|
{
|
|
var roundTripped = CentralControlDtoMapper.FromDto(
|
|
Wire(CentralControlDtoMapper.ToDto(new IngestCachedTelemetryCommand([]))));
|
|
|
|
Assert.Empty(roundTripped.Entries);
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestAck_RoundTrip_PreservesIdsAndOrder()
|
|
{
|
|
var ids = new[] { Guid.NewGuid(), Guid.NewGuid(), Guid.NewGuid() };
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromIngestAck(
|
|
Wire(CentralControlDtoMapper.ToIngestAck(ids)));
|
|
|
|
Assert.Equal(ids, roundTripped);
|
|
}
|
|
|
|
[Fact]
|
|
public void IngestAck_RoundTrip_NothingAccepted_StaysEmpty()
|
|
{
|
|
// An empty ack is meaningful — "zero rows persisted" leaves the site's rows
|
|
// Pending — so it must not be indistinguishable from a dropped field.
|
|
Assert.Empty(CentralControlDtoMapper.FromIngestAck(
|
|
Wire(CentralControlDtoMapper.ToIngestAck([]))));
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Startup reconciliation
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Fact]
|
|
public void ReconcileSiteRequest_RoundTrip_PreservesInventoryMap()
|
|
{
|
|
var original = new ReconcileSiteRequest(
|
|
SiteIdentifier: "site-a",
|
|
NodeId: "node-a",
|
|
LocalNameToRevisionHash: new Dictionary<string, string>
|
|
{
|
|
["Plant.Tank04"] = "hash-1",
|
|
["Plant.Pump01"] = "hash-2",
|
|
});
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(original.SiteIdentifier, roundTripped.SiteIdentifier);
|
|
Assert.Equal(original.NodeId, roundTripped.NodeId);
|
|
Assert.Equal(
|
|
original.LocalNameToRevisionHash.OrderBy(kv => kv.Key),
|
|
roundTripped.LocalNameToRevisionHash.OrderBy(kv => kv.Key));
|
|
}
|
|
|
|
[Fact]
|
|
public void ReconcileSiteRequest_RoundTrip_EmptyInventory_StaysEmpty()
|
|
{
|
|
// A node with nothing deployed is the normal first-boot case, and central
|
|
// must see an empty inventory rather than a missing one.
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(
|
|
new ReconcileSiteRequest("site-a", "node-a", new Dictionary<string, string>()))));
|
|
|
|
Assert.Empty(roundTripped.LocalNameToRevisionHash);
|
|
}
|
|
|
|
[Fact]
|
|
public void ReconcileSiteResponse_RoundTrip_PreservesGapOrphansAndUrl()
|
|
{
|
|
var original = new ReconcileSiteResponse(
|
|
Gap:
|
|
[
|
|
new ReconcileGapItem("Plant.Tank04", "dep-1", "hash-1", IsEnabled: true, "token-1"),
|
|
new ReconcileGapItem("Plant.Pump01", "dep-2", "hash-2", IsEnabled: false, "token-2"),
|
|
],
|
|
OrphanNames: ["Plant.Retired01", "Plant.Retired02"],
|
|
CentralFetchBaseUrl: "http://scadabridge-central-a:5000");
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(original.Gap, roundTripped.Gap);
|
|
Assert.Equal(original.OrphanNames, roundTripped.OrphanNames);
|
|
Assert.Equal(original.CentralFetchBaseUrl, roundTripped.CentralFetchBaseUrl);
|
|
}
|
|
|
|
[Fact]
|
|
public void ReconcileSiteResponse_RoundTrip_NoGap_StaysEmpty()
|
|
{
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(
|
|
new ReconcileSiteResponse([], [], "http://central:5000"))));
|
|
|
|
Assert.Empty(roundTripped.Gap);
|
|
Assert.Empty(roundTripped.OrphanNames);
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Site health
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Fact]
|
|
public void SiteHealthReport_RoundTrip_FullyPopulated_PreservesEveryField()
|
|
{
|
|
var original = FullyPopulatedHealthReport();
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
AssertHealthReportsEqual(original, roundTripped);
|
|
}
|
|
|
|
[Fact]
|
|
public void SiteHealthReport_RoundTrip_MinimalReport_LeavesEveryOptionalNullOrEmpty()
|
|
{
|
|
// The shape a producer emits before any reporter has run: no endpoints map, no
|
|
// tag-quality map, no cluster-node list, no audit backlog, no nullable gauges.
|
|
var original = new SiteHealthReport(
|
|
SiteId: "site-a",
|
|
SequenceNumber: 1,
|
|
ReportTimestamp: SampleInstant,
|
|
DataConnectionStatuses: new Dictionary<string, ConnectionHealth>(),
|
|
TagResolutionCounts: new Dictionary<string, TagResolutionStatus>(),
|
|
ScriptErrorCount: 0,
|
|
AlarmEvaluationErrorCount: 0,
|
|
StoreAndForwardBufferDepths: new Dictionary<string, int>(),
|
|
DeadLetterCount: 0,
|
|
DeployedInstanceCount: 0,
|
|
EnabledInstanceCount: 0,
|
|
DisabledInstanceCount: 0);
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
AssertHealthReportsEqual(original, roundTripped);
|
|
|
|
Assert.Empty(roundTripped.DataConnectionStatuses);
|
|
Assert.Empty(roundTripped.TagResolutionCounts);
|
|
Assert.Empty(roundTripped.StoreAndForwardBufferDepths);
|
|
Assert.Null(roundTripped.DataConnectionEndpoints);
|
|
Assert.Null(roundTripped.DataConnectionTagQuality);
|
|
Assert.Null(roundTripped.ClusterNodes);
|
|
Assert.Null(roundTripped.SiteAuditBacklog);
|
|
Assert.Null(roundTripped.OldestParkedMessageAgeSeconds);
|
|
Assert.Null(roundTripped.ScriptOldestBusyAgeSeconds);
|
|
Assert.Null(roundTripped.LocalDbReplicationConnected);
|
|
Assert.Null(roundTripped.LocalDbOplogBacklog);
|
|
Assert.Null(roundTripped.StartupReconcilerActive);
|
|
}
|
|
|
|
[Fact]
|
|
public void SiteHealthReport_RoundTrip_EmptyNullableCollections_StayEmptyNotNull()
|
|
{
|
|
// The distinction the three wrapper messages exist for. Null means "this node
|
|
// does not report the signal"; empty means "it reports it, and there is
|
|
// nothing in it". proto3 cannot express presence on repeated/map fields, so
|
|
// collapsing one into the other here would be invisible until an operator
|
|
// misread the health page.
|
|
var original = FullyPopulatedHealthReport() with
|
|
{
|
|
DataConnectionEndpoints = new Dictionary<string, string>(),
|
|
DataConnectionTagQuality = new Dictionary<string, TagQualityCounts>(),
|
|
ClusterNodes = [],
|
|
};
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.NotNull(roundTripped.DataConnectionEndpoints);
|
|
Assert.Empty(roundTripped.DataConnectionEndpoints);
|
|
Assert.NotNull(roundTripped.DataConnectionTagQuality);
|
|
Assert.Empty(roundTripped.DataConnectionTagQuality);
|
|
Assert.NotNull(roundTripped.ClusterNodes);
|
|
Assert.Empty(roundTripped.ClusterNodes);
|
|
}
|
|
|
|
[Fact]
|
|
public void SiteHealthReport_RoundTrip_FalseAndZeroGauges_StayFalseAndZero()
|
|
{
|
|
// The mirror of the null case: LocalDbReplicationConnected=false ("configured
|
|
// but disconnected") and LocalDbOplogBacklog=0 ("healthy, nothing queued") are
|
|
// real values that must not decay into null on the wire.
|
|
var original = FullyPopulatedHealthReport() with
|
|
{
|
|
LocalDbReplicationConnected = false,
|
|
LocalDbOplogBacklog = 0,
|
|
OldestParkedMessageAgeSeconds = 0d,
|
|
ScriptOldestBusyAgeSeconds = 0d,
|
|
// Register row 73: false ("this node booted with no self-heal") is the value
|
|
// central badges on, so it must not decay into null ("not reported") en route.
|
|
StartupReconcilerActive = false,
|
|
};
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.False(roundTripped.LocalDbReplicationConnected);
|
|
Assert.Equal(0L, roundTripped.LocalDbOplogBacklog);
|
|
Assert.False(roundTripped.StartupReconcilerActive);
|
|
Assert.Equal(0d, roundTripped.OldestParkedMessageAgeSeconds);
|
|
Assert.Equal(0d, roundTripped.ScriptOldestBusyAgeSeconds);
|
|
}
|
|
|
|
[Fact]
|
|
public void SiteHealthReport_RoundTrip_BacklogWithEmptyQueue_KeepsNullOldestPending()
|
|
{
|
|
var original = FullyPopulatedHealthReport() with
|
|
{
|
|
SiteAuditBacklog = new SiteAuditBacklogSnapshot(0, null, 4096),
|
|
};
|
|
|
|
var roundTripped = CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original)));
|
|
|
|
Assert.Equal(new SiteAuditBacklogSnapshot(0, null, 4096), roundTripped.SiteAuditBacklog);
|
|
}
|
|
|
|
[Fact]
|
|
public void SiteHealthReportAck_RoundTrip_Accepted_And_Rejected()
|
|
{
|
|
var accepted = new SiteHealthReportAck("site-a", 42, Accepted: true);
|
|
var rejected = new SiteHealthReportAck("site-a", 43, Accepted: false, Error: "aggregator down");
|
|
|
|
Assert.Equal(accepted, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(accepted))));
|
|
Assert.Equal(rejected, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(rejected))));
|
|
Assert.Null(CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(accepted))).Error);
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Heartbeat + ConnectionHealth enum
|
|
// -----------------------------------------------------------------------
|
|
|
|
[Theory]
|
|
[InlineData(true)]
|
|
[InlineData(false)]
|
|
public void HeartbeatMessage_RoundTrip_PreservesEveryField(bool isActive)
|
|
{
|
|
var original = new HeartbeatMessage("site-a", "scadabridge-site-a-node-b", isActive, SampleInstant);
|
|
|
|
Assert.Equal(original, CentralControlDtoMapper.FromDto(Wire(CentralControlDtoMapper.ToDto(original))));
|
|
}
|
|
|
|
[Theory]
|
|
[InlineData(ConnectionHealth.Connected)]
|
|
[InlineData(ConnectionHealth.Disconnected)]
|
|
[InlineData(ConnectionHealth.Connecting)]
|
|
[InlineData(ConnectionHealth.Error)]
|
|
public void ConnectionHealth_RoundTrip_EveryValueSurvives(ConnectionHealth health)
|
|
{
|
|
Assert.Equal(health, CentralControlDtoMapper.FromDto(CentralControlDtoMapper.ToDto(health)));
|
|
}
|
|
|
|
[Fact]
|
|
public void ConnectionHealth_Unspecified_DecodesToErrorNotConnected()
|
|
{
|
|
// Fail-safe direction: a wire value this build does not recognise must never
|
|
// render as "healthy" on the central health page.
|
|
Assert.Equal(
|
|
ConnectionHealth.Error,
|
|
CentralControlDtoMapper.FromDto(ConnectionHealthEnum.ConnectionHealthUnspecified));
|
|
}
|
|
|
|
[Fact]
|
|
public void ConnectionHealth_EveryEnumMemberIsMapped()
|
|
{
|
|
// Guards the ArgumentOutOfRangeException path: adding a ConnectionHealth value
|
|
// without extending the mapper should fail here, not silently on a rig.
|
|
foreach (var health in Enum.GetValues<ConnectionHealth>())
|
|
{
|
|
var wire = CentralControlDtoMapper.ToDto(health);
|
|
Assert.NotEqual(ConnectionHealthEnum.ConnectionHealthUnspecified, wire);
|
|
}
|
|
}
|
|
|
|
// -----------------------------------------------------------------------
|
|
// Fixtures
|
|
// -----------------------------------------------------------------------
|
|
|
|
private static SiteHealthReport FullyPopulatedHealthReport() =>
|
|
new(
|
|
SiteId: "site-a",
|
|
SequenceNumber: 987654321L,
|
|
ReportTimestamp: SampleInstant,
|
|
DataConnectionStatuses: new Dictionary<string, ConnectionHealth>
|
|
{
|
|
["opc-main"] = ConnectionHealth.Connected,
|
|
["mx-gateway"] = ConnectionHealth.Error,
|
|
["opc-backup"] = ConnectionHealth.Connecting,
|
|
["legacy"] = ConnectionHealth.Disconnected,
|
|
},
|
|
TagResolutionCounts: new Dictionary<string, TagResolutionStatus>
|
|
{
|
|
["opc-main"] = new(TotalSubscribed: 120, SuccessfullyResolved: 118),
|
|
},
|
|
ScriptErrorCount: 4,
|
|
AlarmEvaluationErrorCount: 2,
|
|
StoreAndForwardBufferDepths: new Dictionary<string, int>
|
|
{
|
|
["erp"] = 17,
|
|
["mes"] = 0,
|
|
},
|
|
DeadLetterCount: 5,
|
|
DeployedInstanceCount: 30,
|
|
EnabledInstanceCount: 28,
|
|
DisabledInstanceCount: 2,
|
|
NodeRole: "Active",
|
|
NodeHostname: "scadabridge-site-a-node-a",
|
|
DataConnectionEndpoints: new Dictionary<string, string>
|
|
{
|
|
["opc-main"] = "opc.tcp://opcua:4840",
|
|
},
|
|
DataConnectionTagQuality: new Dictionary<string, TagQualityCounts>
|
|
{
|
|
["opc-main"] = new(Good: 100, Bad: 3, Uncertain: 15),
|
|
},
|
|
ParkedMessageCount: 6,
|
|
ClusterNodes:
|
|
[
|
|
new NodeStatus("scadabridge-site-a-node-a", IsOnline: true, "Active"),
|
|
new NodeStatus("scadabridge-site-a-node-b", IsOnline: false, "Standby"),
|
|
],
|
|
SiteAuditWriteFailures: 7,
|
|
AuditRedactionFailure: 8,
|
|
SiteAuditBacklog: new SiteAuditBacklogSnapshot(
|
|
PendingCount: 42,
|
|
OldestPendingUtc: new DateTime(2026, 7, 22, 8, 0, 0, DateTimeKind.Utc),
|
|
OnDiskBytes: 1_234_567L),
|
|
SiteEventLogWriteFailures: 9L,
|
|
OldestParkedMessageAgeSeconds: 3600.5d)
|
|
{
|
|
ScriptQueueDepth = 11,
|
|
ScriptBusyThreads = 3,
|
|
ScriptOldestBusyAgeSeconds = 12.25d,
|
|
LocalDbReplicationConnected = true,
|
|
LocalDbOplogBacklog = 250L,
|
|
StartupReconcilerActive = true,
|
|
};
|
|
|
|
/// <summary>
|
|
/// Compares two reports member by member. <see cref="SiteHealthReport"/> is a record,
|
|
/// but its dictionary/list members compare by reference under record equality, so
|
|
/// <c>Assert.Equal(a, b)</c> would pass trivially for the scalars and never look
|
|
/// inside the collections — the exact blind spot these goldens exist to close.
|
|
/// </summary>
|
|
private static void AssertHealthReportsEqual(SiteHealthReport expected, SiteHealthReport actual)
|
|
{
|
|
Assert.Equal(expected.SiteId, actual.SiteId);
|
|
Assert.Equal(expected.SequenceNumber, actual.SequenceNumber);
|
|
Assert.Equal(expected.ReportTimestamp, actual.ReportTimestamp);
|
|
Assert.Equal(
|
|
expected.DataConnectionStatuses.OrderBy(kv => kv.Key),
|
|
actual.DataConnectionStatuses.OrderBy(kv => kv.Key));
|
|
Assert.Equal(
|
|
expected.TagResolutionCounts.OrderBy(kv => kv.Key),
|
|
actual.TagResolutionCounts.OrderBy(kv => kv.Key));
|
|
Assert.Equal(expected.ScriptErrorCount, actual.ScriptErrorCount);
|
|
Assert.Equal(expected.AlarmEvaluationErrorCount, actual.AlarmEvaluationErrorCount);
|
|
Assert.Equal(
|
|
expected.StoreAndForwardBufferDepths.OrderBy(kv => kv.Key),
|
|
actual.StoreAndForwardBufferDepths.OrderBy(kv => kv.Key));
|
|
Assert.Equal(expected.DeadLetterCount, actual.DeadLetterCount);
|
|
Assert.Equal(expected.DeployedInstanceCount, actual.DeployedInstanceCount);
|
|
Assert.Equal(expected.EnabledInstanceCount, actual.EnabledInstanceCount);
|
|
Assert.Equal(expected.DisabledInstanceCount, actual.DisabledInstanceCount);
|
|
Assert.Equal(expected.NodeRole, actual.NodeRole);
|
|
Assert.Equal(expected.NodeHostname, actual.NodeHostname);
|
|
Assert.Equal(
|
|
expected.DataConnectionEndpoints?.OrderBy(kv => kv.Key),
|
|
actual.DataConnectionEndpoints?.OrderBy(kv => kv.Key));
|
|
Assert.Equal(
|
|
expected.DataConnectionTagQuality?.OrderBy(kv => kv.Key),
|
|
actual.DataConnectionTagQuality?.OrderBy(kv => kv.Key));
|
|
Assert.Equal(expected.ParkedMessageCount, actual.ParkedMessageCount);
|
|
Assert.Equal(expected.ClusterNodes, actual.ClusterNodes);
|
|
Assert.Equal(expected.SiteAuditWriteFailures, actual.SiteAuditWriteFailures);
|
|
Assert.Equal(expected.AuditRedactionFailure, actual.AuditRedactionFailure);
|
|
Assert.Equal(expected.SiteAuditBacklog, actual.SiteAuditBacklog);
|
|
Assert.Equal(expected.SiteEventLogWriteFailures, actual.SiteEventLogWriteFailures);
|
|
Assert.Equal(expected.OldestParkedMessageAgeSeconds, actual.OldestParkedMessageAgeSeconds);
|
|
Assert.Equal(expected.ScriptQueueDepth, actual.ScriptQueueDepth);
|
|
Assert.Equal(expected.ScriptBusyThreads, actual.ScriptBusyThreads);
|
|
Assert.Equal(expected.ScriptOldestBusyAgeSeconds, actual.ScriptOldestBusyAgeSeconds);
|
|
Assert.Equal(expected.LocalDbReplicationConnected, actual.LocalDbReplicationConnected);
|
|
Assert.Equal(expected.LocalDbOplogBacklog, actual.LocalDbOplogBacklog);
|
|
Assert.Equal(expected.StartupReconcilerActive, actual.StartupReconcilerActive);
|
|
}
|
|
|
|
/// <summary>
|
|
/// Serializes a wire message and parses it back, so every round-trip below crosses a
|
|
/// real protobuf encode/decode rather than only exercising the mapper's object graph.
|
|
/// This is what proves the three collection wrapper messages keep their presence
|
|
/// when empty — an empty message encodes as a tag with zero-length payload, and a
|
|
/// mapper that used a bare <c>repeated</c>/<c>map</c> field instead would decode an
|
|
/// empty collection back as null with no test-visible difference at the object level.
|
|
/// </summary>
|
|
/// <typeparam name="T">Generated protobuf message type.</typeparam>
|
|
/// <param name="message">The message to send through an encode/decode cycle.</param>
|
|
/// <returns>An independent instance parsed from <paramref name="message"/>'s bytes.</returns>
|
|
private static T Wire<T>(T message) where T : IMessage<T>, new() =>
|
|
new MessageParser<T>(() => new T()).ParseFrom(message.ToByteArray());
|
|
|
|
private static ZB.MOM.WW.Audit.AuditEvent NewAuditEvent(string? sourceScript = null) =>
|
|
ScadaBridgeAuditEventFactory.Create(
|
|
channel: AuditChannel.ApiOutbound,
|
|
kind: AuditKind.ApiCallCached,
|
|
status: AuditStatus.Forwarded,
|
|
eventId: Guid.NewGuid(),
|
|
occurredAtUtc: new DateTime(2026, 7, 22, 9, 0, 0, DateTimeKind.Utc),
|
|
target: "ERP.GetOrder",
|
|
sourceSiteId: "site-a",
|
|
sourceNode: "node-a",
|
|
sourceScript: sourceScript);
|
|
|
|
private static SiteCall NewSiteCall() => new()
|
|
{
|
|
TrackedOperationId = TrackedOperationId.New(),
|
|
Channel = "ApiOutbound",
|
|
Target = "ERP.GetOrder",
|
|
SourceSite = "site-a",
|
|
SourceNode = "node-a",
|
|
Status = "Delivered",
|
|
RetryCount = 2,
|
|
LastError = "transient 503",
|
|
HttpStatus = 200,
|
|
CreatedAtUtc = new DateTime(2026, 7, 22, 8, 0, 0, DateTimeKind.Utc),
|
|
UpdatedAtUtc = new DateTime(2026, 7, 22, 8, 5, 0, DateTimeKind.Utc),
|
|
TerminalAtUtc = new DateTime(2026, 7, 22, 8, 10, 0, DateTimeKind.Utc),
|
|
IngestedAtUtc = new DateTime(2026, 7, 22, 8, 10, 1, DateTimeKind.Utc),
|
|
};
|
|
}
|