Files
ScadaBridge/tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Repositories/SiteRepositoryTests.cs
T
Joseph Doherty 41a5ecf14a feat(external-systems): deploy per-system retry policy to sites + CLI/management surface (register row 27)
An external system's MaxRetries/RetryDelay existed on the central entity but never
reached a site and had no management surface, so every ExternalSystem.CachedCall
buffered with the S&F engine default (DefaultMaxRetries 50 x DefaultRetryInterval
30s ~= 25 min to park) whatever central had configured. The sibling
DatabaseConnectionArtifact had always carried its equivalents.

The consumption end was already wired: ExternalSystemClient.CachedCallAsync already
read system.MaxRetries/RetryDelay and passed them to EnqueueAsync with the 0-as-unset
convention. Only the transport and persistence were missing.

Commons
- ExternalSystemArtifact gains MaxRetries/RetryDelay as optional positional params
  (0 / TimeSpan.Zero = unset), matching DatabaseConnectionArtifact rather than going
  nullable, so it maps 1:1 onto the entity and the proto3 scalars.
- Create/UpdateExternalSystemCommand gain nullable MaxRetries/RetryDelay, following
  the existing TimeoutSeconds "omit to preserve" convention. Both additive.

Central
- ManagementActor create maps both; update preserves-if-null.
- ArtifactDeploymentService projects them onto the artifact.

Wire
- site_command.proto ExternalSystemArtifactDto gains max_retries=7 /
  retry_delay=8 (google.protobuf.Duration), mirroring DatabaseConnectionArtifactDto.
  Additive: a pre-existing site decodes them as 0/absent, which IS the unset value,
  so old behaviour is preserved. Vendored C# regenerated via docker/regen-proto.sh
  sitecommand; csproj Protobuf item re-commented, no message types added or removed.

Site
- external_systems gains max_retries / retry_delay_ms via AddColumnIfMissing,
  DEFAULT 0 = unset. Storage upsert, repository SELECT/mapping (new ordinals 6/7
  appended so existing ones are untouched), apply path, and the legacy migrator's
  explicit column list all carry them.

Replicated-table decision: external_systems IS in SiteLocalDbSetup.ReplicatedTables,
so this was checked against the AckTime/native_alarm_state precedent. Real columns
are correct here, not a JSON bag: the table has no general-purpose config column
(auth_configuration and method_definitions are both semantically specific),
timeout_seconds already set the additive-column precedent on this exact table, and
sibling database_connections stores the same two values as plain columns. Verified in
LocalDb source that RegisterReplicated re-reads PRAGMA table_info every boot and its
install script is DROP-then-CREATE, so capture triggers rebuild over the new column
set automatically, and SiteStorageSchema.Apply runs before registration on the same
boot. The only cost is the ReplicatedTable.ComputeDigest change, which the sync
handshake compares fail-closed - a mixed-version pair refuses to sync loudly instead
of diverging silently. Both-nodes-together is already the standing site-pair rule, so
no new operational constraint; recorded in topology-guide.md.

Also fixed: ArtifactDiff.CompareExternalSystem never compared MaxRetries/RetryDelay,
so a bundle import that changed a system's retry budget previewed as "Identical" -
harmless while the values were inert, a live behaviour change now.

No EF migration: the central ExternalSystemDefinitions columns already exist
(InitialSchema). Central UI already had both form fields.

Tests: site storage round-trip + unset-default, site apply path, central artifact
projection, proto round-trip (via the shared sample), management create/preserve,
CLI option parse + alias, and diff reporting. Fail-before verified on the two core
tests (expected 7 got 0 on the read path; expected 9 got 0 on the apply path).

Docs: Component-ExternalSystemGateway, Component-StoreAndForward, Component-CLI,
components/StoreAndForward, components/SiteRuntime, deployment/topology-guide,
CLI README (which was also missing --timeout-seconds entirely).
2026-08-15 12:56:12 -04:00

360 lines
16 KiB
C#

using Microsoft.Extensions.Logging.Abstractions;
using ZB.MOM.WW.ScadaBridge.SiteRuntime.Persistence;
using ZB.MOM.WW.ScadaBridge.SiteRuntime.Repositories;
using ZB.MOM.WW.ScadaBridge.TestSupport;
namespace ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests.Repositories;
/// <summary>
/// SiteRuntime-006 / SiteRuntime-007 regression tests for the site-local repositories.
///
/// SiteRuntime-006: the repositories must obtain a SQLite connection through
/// <see cref="SiteStorageService.CreateConnection"/>, not by reading a private field
/// via reflection.
///
/// SiteRuntime-007: the synthetic integer IDs derived from entity names must be stable
/// across process restarts (a freshly-constructed service/repository), so an ID handed
/// to a caller still resolves the same entity later.
/// </summary>
public class SiteRepositoryTests : IDisposable
{
private readonly TestLocalDb _localDb;
public SiteRepositoryTests()
{
_localDb = TestLocalDb.CreateTemp("site-repo-test");
}
public void Dispose()
{
// Dispose first — the master connection anchors the WAL, so the sidecars
// cannot be removed while it is open.
var path = _localDb.Path;
_localDb.Dispose();
TestLocalDb.DeleteFiles(path);
GC.SuppressFinalize(this);
}
/// <summary>
/// A brand-new <see cref="SiteStorageService"/> instance over the same site database.
/// The service now takes an <c>ILocalDb</c> instead of a connection string, so the
/// single fixture database is shared while each call still yields a fresh service
/// object — which is what the restart tests below actually vary (the synthetic IDs are
/// derived per service/repository instance, not per connection).
/// </summary>
private SiteStorageService NewStorage()
=> new(_localDb.Db, NullLogger<SiteStorageService>.Instance);
/// <summary>
/// SiteRuntime-006: an external system stored via <see cref="SiteStorageService"/>
/// can be read back through the repository — proving the repository's connection
/// (now obtained from <see cref="SiteStorageService.CreateConnection"/>) is valid.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_RoundTripsStoredDefinition()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"WeatherApi", "https://api.example.com", "ApiKey", "{\"key\":\"x\"}", null);
var repo = new SiteExternalSystemRepository(storage);
var all = await repo.GetAllExternalSystemsAsync();
Assert.Single(all);
Assert.Equal("WeatherApi", all[0].Name);
Assert.Equal("https://api.example.com", all[0].EndpointUrl);
}
/// <summary>
/// ExternalSystemGateway (Timeout): the per-system <c>TimeoutSeconds</c> stored via
/// <see cref="SiteStorageService.StoreExternalSystemAsync"/> must round-trip through the
/// site SQLite store and back out via the repository, so site-side calls honor it.
/// </summary>
[Fact]
public async Task SiteStorage_ExternalSystem_TimeoutSeconds_RoundTrips()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"WeatherApi", "https://api.example.com", "ApiKey", "{\"key\":\"x\"}", null, 7);
var repo = new SiteExternalSystemRepository(storage);
var found = await repo.GetExternalSystemByNameAsync("WeatherApi");
Assert.NotNull(found);
Assert.Equal(7, found!.TimeoutSeconds);
}
/// <summary>
/// Deferred-work row 27: the per-system Store-and-Forward retry policy
/// (<c>MaxRetries</c>/<c>RetryDelay</c>) must round-trip through the site SQLite store,
/// so <c>ExternalSystemClient.CachedCallAsync</c> can hand it to the engine instead of
/// silently falling back to <c>DefaultMaxRetries</c> (50) x <c>DefaultRetryInterval</c> (30 s).
/// Before the fix the site table had no such columns and this read back 0 / TimeSpan.Zero.
/// </summary>
[Fact]
public async Task SiteStorage_ExternalSystem_RetryPolicy_RoundTrips()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"MesApi", "https://mes.example.com", "ApiKey", null, null,
timeoutSeconds: 0, maxRetries: 7, retryDelay: TimeSpan.FromSeconds(90));
var repo = new SiteExternalSystemRepository(storage);
var found = await repo.GetExternalSystemByNameAsync("MesApi");
Assert.NotNull(found);
Assert.Equal(7, found!.MaxRetries);
Assert.Equal(TimeSpan.FromSeconds(90), found.RetryDelay);
}
/// <summary>
/// The unset case stays unset: a system deployed without a retry policy must read back as
/// 0 / <see cref="TimeSpan.Zero"/>, which is the value the gateway translates to null so the
/// engine defaults apply. This is what makes the new columns behaviourally invisible until
/// an operator actually configures a policy.
/// </summary>
[Fact]
public async Task SiteStorage_ExternalSystem_RetryPolicy_DefaultsToUnset()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"PlainApi", "https://plain.example.com", "ApiKey", null, null);
var repo = new SiteExternalSystemRepository(storage);
var found = await repo.GetExternalSystemByNameAsync("PlainApi");
Assert.NotNull(found);
Assert.Equal(0, found!.MaxRetries);
Assert.Equal(TimeSpan.Zero, found.RetryDelay);
}
/// <summary>
/// SiteRuntime-007: the synthetic ID for an external system must be identical when
/// the storage service and repository are re-created (simulating a process restart).
/// With the old <see cref="string.GetHashCode()"/> the ID was randomized per process
/// and a by-ID lookup after a restart would fail.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_SyntheticId_IsStableAcrossRestart()
{
var storage1 = NewStorage();
await storage1.InitializeAsync();
await storage1.StoreExternalSystemAsync(
"StableSystem", "https://x", "None", null, null);
var repo1 = new SiteExternalSystemRepository(storage1);
var idBeforeRestart = (await repo1.GetAllExternalSystemsAsync())[0].Id;
// Simulate a process restart — brand-new service + repository instances.
var storage2 = NewStorage();
var repo2 = new SiteExternalSystemRepository(storage2);
var idAfterRestart = (await repo2.GetAllExternalSystemsAsync())[0].Id;
Assert.Equal(idBeforeRestart, idAfterRestart);
// And the by-ID lookup must succeed using the pre-restart ID.
var found = await repo2.GetExternalSystemByIdAsync(idBeforeRestart);
Assert.NotNull(found);
Assert.Equal("StableSystem", found.Name);
}
// ── ExternalSystemGateway-011: name-keyed repository lookups ──
/// <summary>
/// ExternalSystemGateway-011: the site repository's name-keyed external-system
/// lookup returns the matching row, and the same synthetic ID as the by-ID path.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_GetByName_ReturnsMatchingDefinition()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"Alpha", "https://alpha.test", "ApiKey", "{\"key\":\"x\"}", null);
await storage.StoreExternalSystemAsync(
"Beta", "https://beta.test", "Basic", null, null);
var repo = new SiteExternalSystemRepository(storage);
var found = await repo.GetExternalSystemByNameAsync("Beta");
Assert.NotNull(found);
Assert.Equal("Beta", found!.Name);
Assert.Equal("https://beta.test", found.EndpointUrl);
// The by-name path must produce the same synthetic ID as the by-id path.
var byId = await repo.GetExternalSystemByIdAsync(found.Id);
Assert.NotNull(byId);
Assert.Equal("Beta", byId!.Name);
}
/// <summary>
/// ExternalSystemGateway-011: a missing name resolves to <c>null</c>.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_GetByName_MissingName_ReturnsNull()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync(
"Alpha", "https://alpha.test", "ApiKey", null, null);
var repo = new SiteExternalSystemRepository(storage);
Assert.Null(await repo.GetExternalSystemByNameAsync("DoesNotExist"));
}
/// <summary>
/// ExternalSystemGateway-011: the site repository's name-keyed method lookup
/// returns the method scoped to its parent system, or <c>null</c> for a miss.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_GetMethodByName_ResolvesScopedToSystem()
{
var storage = NewStorage();
await storage.InitializeAsync();
var methodDefs = "[{\"Name\":\"getData\",\"HttpMethod\":\"GET\",\"Path\":\"/data\"}]";
await storage.StoreExternalSystemAsync(
"WeatherApi", "https://api.example.com", "ApiKey", null, methodDefs);
var repo = new SiteExternalSystemRepository(storage);
var system = await repo.GetExternalSystemByNameAsync("WeatherApi");
Assert.NotNull(system);
var method = await repo.GetMethodByNameAsync(system!.Id, "getData");
Assert.NotNull(method);
Assert.Equal("getData", method!.Name);
Assert.Equal("GET", method.HttpMethod);
Assert.Null(await repo.GetMethodByNameAsync(system.Id, "noSuchMethod"));
}
/// <summary>
/// ExternalSystemGateway-011: the site repository's name-keyed database-connection
/// lookup returns the matching row, or <c>null</c> for a miss.
/// </summary>
[Fact]
public async Task DatabaseConnectionRepository_GetByName_ReturnsMatchingDefinition()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreDatabaseConnectionAsync(
"Plant", "Server=plant;Database=p;", maxRetries: 3, retryDelay: TimeSpan.FromSeconds(2));
await storage.StoreDatabaseConnectionAsync(
"Historian", "Server=hist;Database=h;", maxRetries: 0, retryDelay: TimeSpan.FromSeconds(5));
var repo = new SiteExternalSystemRepository(storage);
var found = await repo.GetDatabaseConnectionByNameAsync("Historian");
Assert.NotNull(found);
Assert.Equal("Historian", found!.Name);
Assert.Equal("Server=hist;Database=h;", found.ConnectionString);
Assert.Equal(0, found.MaxRetries);
Assert.Null(await repo.GetDatabaseConnectionByNameAsync("DoesNotExist"));
}
// NotificationRepository_SyntheticId_IsStableAcrossRestart was removed 2026-07-10
// (arch-review 08 §1.3/#23) along with the vestigial SiteNotificationRepository —
// notification config is central-only and never lives on a site. The synthetic-ID
// stability guarantee is still exercised by ExternalSystemRepository_SyntheticId_IsStableAcrossRestart.
// ── WP2.6a: ExternalSystemDefinitionCache (arch-review misc — site external-system resolution) ──
/// <summary>
/// WP2.6a: two repository instances sharing one <see cref="ExternalSystemDefinitionCache"/>
/// (the production DI shape — one singleton cache, many scoped repository instances) must
/// see the SAME cached snapshot: a row written after the cache was already populated by
/// the first repository is invisible to the second until the cache is invalidated.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_SharedCache_HitServesStaleSnapshotUntilInvalidated()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync("Alpha", "https://alpha.test", "None", null, null);
var cache = new ExternalSystemDefinitionCache();
var repo1 = new SiteExternalSystemRepository(storage, cache);
var repo2 = new SiteExternalSystemRepository(storage, cache);
// repo1 populates the shared cache.
var initial = await repo1.GetAllExternalSystemsAsync();
Assert.Single(initial);
// A new system is written directly to storage — repo2, sharing the same
// cache, must NOT see it yet (cache hit serves the stale snapshot).
await storage.StoreExternalSystemAsync("Beta", "https://beta.test", "None", null, null);
var stillStale = await repo2.GetAllExternalSystemsAsync();
Assert.Single(stillStale);
// After invalidation, the next read reloads from storage and sees both rows.
cache.InvalidateAll();
var fresh = await repo2.GetAllExternalSystemsAsync();
Assert.Equal(2, fresh.Count);
}
/// <summary>
/// WP2.6a: the by-ID lookups (system and method) must resolve correctly out of the
/// cached snapshot — proving the O(1) id-index build in <c>LoadSnapshotAsync</c> is
/// wired correctly for both entity kinds, not just the by-name path already covered
/// by ExternalSystemGateway-011.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_SharedCache_ByIdLookups_ResolveSystemAndMethod()
{
var storage = NewStorage();
await storage.InitializeAsync();
var methodDefs = "[{\"Name\":\"getData\",\"HttpMethod\":\"GET\",\"Path\":\"/data\"}]";
await storage.StoreExternalSystemAsync(
"WeatherApi", "https://api.example.com", "ApiKey", null, methodDefs);
var cache = new ExternalSystemDefinitionCache();
var repo = new SiteExternalSystemRepository(storage, cache);
var system = await repo.GetExternalSystemByNameAsync("WeatherApi");
Assert.NotNull(system);
var byId = await repo.GetExternalSystemByIdAsync(system!.Id);
Assert.NotNull(byId);
Assert.Equal("WeatherApi", byId!.Name);
var methods = await repo.GetMethodsByExternalSystemIdAsync(system.Id);
Assert.Single(methods);
var methodById = await repo.GetExternalSystemMethodByIdAsync(methods[0].Id);
Assert.NotNull(methodById);
Assert.Equal("getData", methodById!.Name);
Assert.Null(await repo.GetExternalSystemByIdAsync(-1));
Assert.Null(await repo.GetExternalSystemMethodByIdAsync(-1));
}
/// <summary>
/// WP2.6a: a repository constructed via the single-arg (no shared cache) constructor
/// gets its own private cache — a second such repository over the same storage must
/// see rows written after the first repository's cache was already populated,
/// preserving the pre-cache "always fresh" behavior for callers that opt out of
/// sharing.
/// </summary>
[Fact]
public async Task ExternalSystemRepository_PrivateCache_DoesNotShareAcrossInstances()
{
var storage = NewStorage();
await storage.InitializeAsync();
await storage.StoreExternalSystemAsync("Alpha", "https://alpha.test", "None", null, null);
var repo1 = new SiteExternalSystemRepository(storage);
Assert.Single(await repo1.GetAllExternalSystemsAsync());
await storage.StoreExternalSystemAsync("Beta", "https://beta.test", "None", null, null);
// A brand-new repository instance (its own private cache, unpopulated) sees both rows.
var repo2 = new SiteExternalSystemRepository(storage);
Assert.Equal(2, (await repo2.GetAllExternalSystemsAsync()).Count);
}
}