Files
ScadaBridge/tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/SiteCommandSamples.cs
T
Joseph Doherty 41a5ecf14a feat(external-systems): deploy per-system retry policy to sites + CLI/management surface (register row 27)
An external system's MaxRetries/RetryDelay existed on the central entity but never
reached a site and had no management surface, so every ExternalSystem.CachedCall
buffered with the S&F engine default (DefaultMaxRetries 50 x DefaultRetryInterval
30s ~= 25 min to park) whatever central had configured. The sibling
DatabaseConnectionArtifact had always carried its equivalents.

The consumption end was already wired: ExternalSystemClient.CachedCallAsync already
read system.MaxRetries/RetryDelay and passed them to EnqueueAsync with the 0-as-unset
convention. Only the transport and persistence were missing.

Commons
- ExternalSystemArtifact gains MaxRetries/RetryDelay as optional positional params
  (0 / TimeSpan.Zero = unset), matching DatabaseConnectionArtifact rather than going
  nullable, so it maps 1:1 onto the entity and the proto3 scalars.
- Create/UpdateExternalSystemCommand gain nullable MaxRetries/RetryDelay, following
  the existing TimeoutSeconds "omit to preserve" convention. Both additive.

Central
- ManagementActor create maps both; update preserves-if-null.
- ArtifactDeploymentService projects them onto the artifact.

Wire
- site_command.proto ExternalSystemArtifactDto gains max_retries=7 /
  retry_delay=8 (google.protobuf.Duration), mirroring DatabaseConnectionArtifactDto.
  Additive: a pre-existing site decodes them as 0/absent, which IS the unset value,
  so old behaviour is preserved. Vendored C# regenerated via docker/regen-proto.sh
  sitecommand; csproj Protobuf item re-commented, no message types added or removed.

Site
- external_systems gains max_retries / retry_delay_ms via AddColumnIfMissing,
  DEFAULT 0 = unset. Storage upsert, repository SELECT/mapping (new ordinals 6/7
  appended so existing ones are untouched), apply path, and the legacy migrator's
  explicit column list all carry them.

Replicated-table decision: external_systems IS in SiteLocalDbSetup.ReplicatedTables,
so this was checked against the AckTime/native_alarm_state precedent. Real columns
are correct here, not a JSON bag: the table has no general-purpose config column
(auth_configuration and method_definitions are both semantically specific),
timeout_seconds already set the additive-column precedent on this exact table, and
sibling database_connections stores the same two values as plain columns. Verified in
LocalDb source that RegisterReplicated re-reads PRAGMA table_info every boot and its
install script is DROP-then-CREATE, so capture triggers rebuild over the new column
set automatically, and SiteStorageSchema.Apply runs before registration on the same
boot. The only cost is the ReplicatedTable.ComputeDigest change, which the sync
handshake compares fail-closed - a mixed-version pair refuses to sync loudly instead
of diverging silently. Both-nodes-together is already the standing site-pair rule, so
no new operational constraint; recorded in topology-guide.md.

Also fixed: ArtifactDiff.CompareExternalSystem never compared MaxRetries/RetryDelay,
so a bundle import that changed a system's retry budget previewed as "Identical" -
harmless while the values were inert, a live behaviour change now.

No EF migration: the central ExternalSystemDefinitions columns already exist
(InitialSchema). Central UI already had both form fields.

Tests: site storage round-trip + unset-default, site apply path, central artifact
projection, proto round-trip (via the shared sample), management create/preserve,
CLI option parse + alias, and diff reporting. Fail-before verified on the two core
tests (expected 7 got 0 on the read path; expected 9 got 0 on the apply path).

Docs: Component-ExternalSystemGateway, Component-StoreAndForward, Component-CLI,
components/StoreAndForward, components/SiteRuntime, deployment/topology-guide,
CLI README (which was also missing --timeout-seconds entirely).
2026-08-15 12:56:12 -04:00

309 lines
17 KiB
C#

using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Protocol;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Artifacts;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.DataConnection;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.DebugView;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Deployment;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.InboundApi;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Lifecycle;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Management;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.RemoteQuery;
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming;
using ZB.MOM.WW.ScadaBridge.Commons.Types;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Alarms;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
namespace ZB.MOM.WW.ScadaBridge.Communication.Tests;
/// <summary>
/// Golden fixtures for the site-command round-trip suite: for every command,
/// reply and nested type the mapper handles, at least one MAXIMALLY populated
/// sample and one MINIMAL sample (every nullable null, every optional
/// collection empty or absent).
/// </summary>
/// <remarks>
/// <para>
/// The two-sample rule is the point. A single "typical" sample proves only that
/// the happy path survives; the pairs are what catch a dropped optional field or
/// a null/empty collapse — the class of silent data loss the transport
/// round-trip guard exposed in PLAN-05 T8, which per-field unit tests had all
/// missed.
/// </para>
/// <para>
/// <see cref="SiteCommandDtoMapperGoldenTests"/> drives this table by
/// reflection and FAILS when the mapper gains a type that has no sample here, so
/// the coverage cannot silently drift as commands are added.
/// </para>
/// </remarks>
internal static class SiteCommandSamples
{
private static readonly DateTimeOffset T1 = new(2026, 7, 22, 13, 45, 12, 345, TimeSpan.Zero);
private static readonly DateTimeOffset T2 = new(2026, 1, 2, 3, 4, 5, TimeSpan.Zero);
private static readonly DateTime D1 = new(2026, 3, 4, 5, 6, 7, DateTimeKind.Utc);
private static readonly DateTime D2 = new(2027, 3, 4, 5, 6, 7, DateTimeKind.Utc);
private static readonly Guid G1 = Guid.Parse("11111111-2222-3333-4444-555555555555");
/// <summary>
/// Every CLR type the mapper round-trips, mapped to its golden samples.
/// Enums are excluded — they get their own exhaustive test.
/// </summary>
public static IReadOnlyDictionary<Type, object[]> All { get; } = Build();
private static Dictionary<Type, object[]> Build()
{
var samples = new Dictionary<Type, object[]>();
void Add<T>(params T[] values) where T : notnull =>
samples[typeof(T)] = values.Cast<object>().ToArray();
// ── Lifecycle commands ──
Add(new RefreshDeploymentCommand(
"dep-1", "Site1.Pump1", "hash-abc", "multi-role", T1, "http://central:5000", "tok-1"));
Add(new EnableInstanceCommand("cmd-1", "Site1.Pump1", T1));
Add(new DisableInstanceCommand("cmd-2", "Site1.Pump1", T2));
Add(new DeleteInstanceCommand("cmd-3", "Site1.Pump1", T1));
Add(new DeploymentStateQueryRequest("corr-1", "Site1.Pump1", T1));
Add(
// Full: every artifact collection populated.
new DeployArtifactsCommand(
"dep-2",
[SharedScript(), new SharedScriptArtifact("s2", "return 2;", null, null)],
[ExternalSystem()],
[new DatabaseConnectionArtifact("db", "Server=x;", 3, TimeSpan.FromSeconds(7.5))],
[new NotificationListArtifact("ops", ["a@x.com", "b@x.com"])],
[DataConnection()],
[Smtp()],
T1),
// Minimal: every collection NULL — must not come back as empty lists.
new DeployArtifactsCommand("dep-3", null, null, null, null, null, null, T2),
// Boundary: every collection present but EMPTY — must not come back null.
new DeployArtifactsCommand("dep-4", [], [], [], [], [], [], T1));
Add(SharedScript(), new SharedScriptArtifact("bare", "", null, null));
Add(ExternalSystem(), new ExternalSystemArtifact("bare", "http://x", "None", null, null, 0));
Add(new DatabaseConnectionArtifact("db", "Server=x;", 3, TimeSpan.FromMilliseconds(1500)),
new DatabaseConnectionArtifact("db2", "", 0, TimeSpan.Zero));
Add(new NotificationListArtifact("ops", ["a@x.com"]), new NotificationListArtifact("empty", []));
Add(DataConnection(), new DataConnectionArtifact("bare", "OpcUa", null, null, 0));
Add(Smtp(), new SmtpConfigurationArtifact("bare", "smtp", 25, "None", "f@x.com", null, null, null));
// ── Lifecycle replies ──
Add(new DeploymentStatusResponse("dep-1", "Site1.Pump1", DeploymentStatus.Success, null, T1),
new DeploymentStatusResponse("dep-1", "Site1.Pump1", DeploymentStatus.Failed, "boom", T2));
Add(new InstanceLifecycleResponse("cmd-1", "Site1.Pump1", true, null, T1),
new InstanceLifecycleResponse("cmd-1", "Site1.Pump1", false, "nope", T2));
Add(new DeploymentStateQueryResponse("corr-1", "Site1.Pump1", true, "dep-1", "hash-abc", T1),
new DeploymentStateQueryResponse("corr-1", "Site1.Pump1", false, null, null, T2));
Add(new ArtifactDeploymentResponse("dep-2", "site-a", true, null, T1),
new ArtifactDeploymentResponse("dep-2", "site-a", false, "handler missing", T2));
// ── OPC UA commands ──
Add(new BrowseNodeCommand("conn", "ns=2;s=Root", "cursor-1", "site-a"),
new BrowseNodeCommand("conn", null, null, null));
Add(new SearchAddressSpaceCommand("conn", "pump", 4, 200, "site-a"),
new SearchAddressSpaceCommand("conn", "", 0, 0, null));
Add(new ReadTagValuesCommand("conn", ["a", "b"]), new ReadTagValuesCommand("conn", []));
Add(new VerifyEndpointCommand("conn", "OpcUa", "{\"url\":\"opc.tcp://x\"}", "site-a"),
new VerifyEndpointCommand("conn", "OpcUa", "{}", null));
Add(new TrustServerCertCommand("conn", "ZGVy", "AABB", "site-a"),
new TrustServerCertCommand("conn", "ZGVy", "AABB", null));
Add(new ListServerCertsCommand("site-a"), new ListServerCertsCommand(null));
Add(new RemoveServerCertCommand("AABB", "site-a"), new RemoveServerCertCommand("AABB", null));
Add(new WriteTagRequest("corr-2", "conn", "ns=2;s=Speed", 42.5d, T1),
new WriteTagRequest("corr-2", "conn", "ns=2;s=Speed", null, T2),
new WriteTagRequest("corr-2", "conn", "ns=2;s=Flag", true, T1),
new WriteTagRequest("corr-2", "conn", "ns=2;s=Name", "manual", T1));
// ── OPC UA replies ──
Add(new BrowseNodeResult([Node(), NodeBare()], true, null, "cursor-2"),
new BrowseNodeResult([], false, new BrowseFailure(BrowseFailureKind.Timeout, "timed out"), null));
Add(Node(), NodeBare());
Add(new BrowseFailure(BrowseFailureKind.ConnectionNotConnected, "not connected"),
new BrowseFailure(BrowseFailureKind.ServerError, ""));
Add(new SearchAddressSpaceResult([new AddressSpaceMatch(Node(), "/Root/Pump1")], true, null),
new SearchAddressSpaceResult([], false, new BrowseFailure(BrowseFailureKind.NotBrowsable, "no")));
Add(new AddressSpaceMatch(Node(), "/Root/Pump1"), new AddressSpaceMatch(NodeBare(), ""));
Add(new ReadTagValuesResult([Outcome(), OutcomeFailed()], null),
new ReadTagValuesResult([], new ReadTagValuesFailure(ReadTagValuesFailureKind.ConnectionNotFound, "gone")));
Add(Outcome(), OutcomeFailed());
Add(new ReadTagValuesFailure(ReadTagValuesFailureKind.Timeout, "slow"),
new ReadTagValuesFailure(ReadTagValuesFailureKind.ServerError, ""));
Add(new VerifyEndpointResult(true, null, null, null),
new VerifyEndpointResult(false, VerifyFailureKind.UntrustedCertificate, "untrusted", Cert()),
new VerifyEndpointResult(false, VerifyFailureKind.Unreachable, "refused", null));
Add(Cert());
Add(new CertTrustResult(true, null, [Trusted(), TrustedRejected()]),
new CertTrustResult(true, null, null),
new CertTrustResult(false, "partial failure", []));
Add(Trusted(), TrustedRejected());
Add(new WriteTagResponse("corr-2", true, null, T1),
new WriteTagResponse("corr-2", false, "denied", T2));
// ── Query commands ──
Add(new EventLogQueryRequest(
"corr-3", "site-a", T1, T2, "Lifecycle", "Warning", "Site1.Pump1", "restart", "cursor-3", 50, T1),
new EventLogQueryRequest("corr-3", "site-a", null, null, null, null, null, null, null, 25, T2));
Add(new DebugSnapshotRequest("Site1.Pump1", "corr-4"));
Add(new SubscribeDebugViewRequest("Site1.Pump1", "corr-5"));
Add(new UnsubscribeDebugViewRequest("Site1.Pump1", "corr-6"));
// ── Query replies ──
Add(new EventLogQueryResponse("corr-3", "site-a", [Entry(), EntryBare()], "cursor-4", true, true, null, T1),
new EventLogQueryResponse("corr-3", "site-a", [], null, false, false, "handler missing", T2));
Add(Entry(), EntryBare());
Add(new DebugViewSnapshot("Site1.Pump1", [AttrValue(), AttrValueNull()], [ComputedAlarm(), NativeAlarm()], T1),
new DebugViewSnapshot("Site1.Pump1", [], [], T2, InstanceNotFound: true));
Add(AttrValue(), AttrValueNull(), AttrValueList());
Add(ComputedAlarm(), NativeAlarm());
Add(new AlarmConditionState(true, false, true, AlarmShelveState.TimedShelved, true, 900),
new AlarmConditionState(false, true, null, AlarmShelveState.Unshelved, false, 0));
// ── Parked commands ──
Add(new ParkedMessageQueryRequest("corr-7", "site-a", 2, 25, T1));
Add(new ParkedMessageRetryRequest("corr-8", "site-a", "msg-1", T1));
Add(new ParkedMessageDiscardRequest("corr-9", "site-a", "msg-1", T2));
Add(new RetryParkedOperation("corr-10", new TrackedOperationId(G1)),
new RetryParkedOperation("corr-10", default));
Add(new DiscardParkedOperation("corr-11", new TrackedOperationId(G1)),
new DiscardParkedOperation("corr-11", default));
// ── Parked replies ──
Add(new ParkedMessageQueryResponse("corr-7", "site-a", [Parked(), ParkedBare()], 2, 1, 25, true, null, T1),
new ParkedMessageQueryResponse("corr-7", "site-a", [], 0, 1, 25, false, "handler missing", T2));
Add(Parked(), ParkedBare());
Add(new ParkedMessageRetryResponse("corr-8", true),
new ParkedMessageRetryResponse("corr-8", false, "not parked"));
Add(new ParkedMessageDiscardResponse("corr-9", true),
new ParkedMessageDiscardResponse("corr-9", false, "not parked"));
Add(new ParkedOperationActionAck("corr-10", true),
new ParkedOperationActionAck("corr-10", false, "handler missing"));
// ── Route commands ──
Add(new RouteToCallRequest("corr-12", "Site1.Pump1", "Start", Parameters(), T1, G1),
new RouteToCallRequest("corr-12", "Site1.Pump1", "Start", null, T2),
new RouteToCallRequest("corr-12", "Site1.Pump1", "Start", new Dictionary<string, object?>(), T1));
Add(new RouteToGetAttributesRequest("corr-13", "Site1.Pump1", ["Speed", "Flow"], T1, G1),
new RouteToGetAttributesRequest("corr-13", "Site1.Pump1", [], T2));
Add(new RouteToSetAttributesRequest(
"corr-14", "Site1.Pump1", new Dictionary<string, string> { ["Speed"] = "10" }, T1, G1),
new RouteToSetAttributesRequest("corr-14", "Site1.Pump1", new Dictionary<string, string>(), T2));
Add(new RouteToWaitForAttributeRequest(
"corr-15", "Site1.Pump1", "Speed", "10", TimeSpan.FromSeconds(30), T1, G1, true),
new RouteToWaitForAttributeRequest(
"corr-15", "Site1.Pump1", "Speed", null, TimeSpan.Zero, T2),
// "" is a legitimate wait target and must NOT collapse to null.
new RouteToWaitForAttributeRequest(
"corr-15", "Site1.Pump1", "Speed", "", TimeSpan.FromMinutes(1), T1));
// ── Route replies ──
Add(new RouteToCallResponse("corr-12", true, 17L, null, T1),
new RouteToCallResponse("corr-12", false, null, "script faulted", T2));
Add(new RouteToGetAttributesResponse("corr-13", Parameters(), true, null, T1),
new RouteToGetAttributesResponse("corr-13", new Dictionary<string, object?>(), false, "no instance", T2));
Add(new RouteToSetAttributesResponse("corr-14", true, null, T1),
new RouteToSetAttributesResponse("corr-14", false, "locked", T2));
Add(new RouteToWaitForAttributeResponse("corr-15", true, 10, "Good", false, true, null, T1),
new RouteToWaitForAttributeResponse("corr-15", false, null, null, true, true, null, T2));
// ── Failover ──
Add(new TriggerSiteFailover("corr-16", "site-a"));
Add(new SiteFailoverAck("corr-16", true, "akka.tcp://scadabridge@node-b:8081", null),
new SiteFailoverAck("corr-16", false, null, "no standby available"));
return samples;
}
private static SharedScriptArtifact SharedScript() =>
new("Calc", "return 1;", "{\"p\":\"int\"}", "{\"r\":\"int\"}");
// MaxRetries/RetryDelay are non-default on purpose: they are the deferred-work-row-27
// additions (proto fields 7-8), and the generic round-trip assertion is what pins them
// to the wire. A sample left at the defaults would round-trip vacuously.
private static ExternalSystemArtifact ExternalSystem() =>
new("Mes", "https://mes/api", "ApiKey", "{\"key\":\"x\"}", "[{\"name\":\"Post\"}]", 45,
MaxRetries: 12, RetryDelay: TimeSpan.FromSeconds(20));
private static DataConnectionArtifact DataConnection() =>
new("Plc1", "OpcUa", "{\"url\":\"opc.tcp://a\"}", "{\"url\":\"opc.tcp://b\"}", 5);
private static SmtpConfigurationArtifact Smtp() =>
new("Default", "smtp.host", 587, "Basic", "from@x.com", "user", "secret", "{\"tenant\":\"t\"}");
private static BrowseNode Node() => new("ns=2;s=Pump1.Speed", "Speed", BrowseNodeClass.Variable, false, "Double", -1, true);
private static BrowseNode NodeBare() => new("ns=2;s=Root", "Root", BrowseNodeClass.Object, true);
private static TagReadOutcome Outcome() => new("ns=2;s=Speed", true, 12.5d, "Good", T1, null);
private static TagReadOutcome OutcomeFailed() => new("ns=2;s=Bad", false, null, "Bad", T2, "no such node");
private static ServerCertInfo Cert() => new("AABB", "CN=server", "CN=issuer", D1, D2, "ZGVy");
private static TrustedCertInfo Trusted() => new("AABB", "CN=server", "CN=issuer", D1, D2, false);
private static TrustedCertInfo TrustedRejected() => new("CCDD", "CN=other", "CN=issuer", D1, D2, true);
private static EventLogEntry Entry() =>
new(G1.ToString("D"), T1, "Lifecycle", "Warning", "Site1.Pump1", "InstanceActor", "restarted", "{\"n\":1}");
private static EventLogEntry EntryBare() =>
new(Guid.Empty.ToString("D"), T2, "System", "Info", null, "Host", "started", null);
private static ParkedMessageEntry Parked() =>
new("msg-1", "Mes", "Post", "500 from server", 7, T1, T2, 10, StoreAndForwardCategory.CachedDbWrite, "Site1.Pump1");
private static ParkedMessageEntry ParkedBare() =>
new("msg-2", "Mes", "Post", "", 0, T2, T2);
private static AttributeValueChanged AttrValue() =>
new("Site1.Pump1", "Pump1.Speed", "Speed", 12.5d, "Good", T1);
private static AttributeValueChanged AttrValueNull() =>
new("Site1.Pump1", "Pump1.Speed", "Speed", null, "Bad", T2);
private static AttributeValueChanged AttrValueList() =>
new("Site1.Pump1", "Pump1.Trend", "Trend", new List<object?> { 1, 2.5d, "x", null }, "Good", T1);
/// <summary>A computed alarm: <c>Condition</c> left implicit, which is the common case.</summary>
private static AlarmStateChanged ComputedAlarm() =>
new("Site1.Pump1", "HighSpeed", AlarmState.Active, 700, T1)
{
Level = AlarmLevel.HighHigh,
Message = "Speed critically high"
};
/// <summary>A mirrored native alarm: every native enrichment field populated, <c>Condition</c> explicit.</summary>
private static AlarmStateChanged NativeAlarm() =>
new("Site1.Pump1", "Tank01.Level.HiHi", AlarmState.Active, 850, T2)
{
Level = AlarmLevel.None,
Message = "Level high",
Kind = AlarmKind.NativeOpcUa,
Condition = new AlarmConditionState(true, false, false, AlarmShelveState.OneShotShelved, true, 850),
SourceReference = "Tank01.Level.HiHi",
AlarmTypeName = "AnalogLimitAlarm.HiHi",
Category = "Process",
OperatorUser = "multi-role",
OperatorComment = "ack'd at panel",
OriginalRaiseTime = T1,
CurrentValue = "91.2",
LimitValue = "90.0",
NativeSourceCanonicalName = "Tank01.TankAlarms",
IsConfiguredPlaceholder = true
};
private static Dictionary<string, object?> Parameters() => new()
{
["count"] = 3,
["ratio"] = 1.25d,
["name"] = "batch-1",
["enabled"] = true,
["missing"] = null,
["when"] = T1,
["id"] = G1,
["items"] = new List<object?> { 1L, "two", null },
["nested"] = new Dictionary<string, object?> { ["inner"] = 5f }
};
}