dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

refactor(sf,site): both stores take ILocalDb instead of a connection string

Browse Source 
Tasks 5 and 6 of the Phase 2 plan, committed together because their test
fallout is entangled — several fixtures construct both stores.

StoreAndForwardStorage and SiteStorageService now take ILocalDb. Connections
come from ILocalDb.CreateConnection(), which hands out an already-open,
pragma-configured connection carrying the zb_hlc_next() UDF the capture triggers
call; a raw connection would lack the UDF and every write to a replicated table
would fail closed. Deleted with the connection strings: S&F's
EnsureDatabaseDirectoryExists and its per-open busy_timeout pragma, and the site
service's BusyTimeoutFloorSeconds normalization — LocalDb owns all of it now.

DI: AddSiteRuntime's string overload is gone (nothing left to supply), so the
Host calls the no-arg form. ScadaBridge:Database:SiteDbPath and
StoreAndForwardOptions.SqliteDbPath survive only as the migrator's source
locations in Tasks 8/9.

Two things the plan did not anticipate, both worth reading:

1. FOUND A REAL LATENT DEFECT, from Phase 1, now fixed. The plan assumed
   directory creation simply moved to LocalDb along with file ownership. It did
   not: the LocalDb library never creates the parent directory, and
   SqliteLocalDb opens the file eagerly in its constructor — so a missing
   directory is a hard boot failure ("SQLite Error 14: unable to open database
   file"), not a degraded start. The default site config points at the RELATIVE
   path ./data/site-localdb.db, so any site node without a pre-existing data/
   directory fails to boot. The docker rig escapes only because its volume mount
   happens to create /app/data — a coincidence that would have hidden this until
   a bare-metal or fresh deployment. This has been latent since Phase 1 made
   LocalDb:Path required; deleting S&F's EnsureDatabaseDirectoryExists here
   would have widened it. Re-established the guarantee at the layer that now
   owns the path (SiteLocalDbDirectory.Ensure, called before AddZbLocalDb) and
   pinned it with SiteLocalDbDirectoryTests. Non-vacuity is not assumed: two
   tests written against the wrong assumption failed with exactly this
   SQLite Error 14 before the fix existed.

2. Test fallout was ~7x the plan's estimate. The plan named "fixtures" in one
   project; the constructor change actually reaches 40 files across 7 test
   projects, and most used Mode=Memory;Cache=Shared — which LocalDb has no
   equivalent for, so every one had to move to a real temp file. Rather than
   copy the Phase 1 TestLocalDb fixture into 7 projects, added a shared
   tests/ZB.MOM.WW.ScadaBridge.TestSupport library (not a test project) so the
   WAL-sidecar cleanup and the "real, not stubbed" rationale live in one place.

Retargeted rather than deleted, in both directions: the S&F WAL test now asserts
against the LocalDb-backed store (WAL genuinely is LocalDb's job), while the
directory-creation test moved to Host.Tests (that guarantee is NOT LocalDb's).
SiteStorageServiceTests.Initialize_EnablesWalJournalMode got the same treatment.
DeploymentManagerMediumFindingsTests induced a persistence failure via an
unopenable path, which no longer reaches the assertion since the fixture now
throws first; it induces the same failure shape via an uninitialized store.

Verified: full solution build 0 warnings; SiteRuntime 532, Host 318,
AuditLog 355, ExternalSystemGateway 142, HealthMonitoring 97,
StoreAndForward 153 — 1597 passed, 0 failed.

Claude-Session: https://claude.ai/code/session_01BL2Vu1ESDQ9SCN4gVKkdts
This commit is contained in:
Joseph Doherty
2026-07-20 03:05:45 -04:00
parent 3dfb288b74
commit f2efeb37b7
57 changed files with 1368 additions and 848 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs
+53 -32
View File
@@ -1,24 +1,60 @@
using System.Net;
using System.Net.Http.Headers;
using System.Text;
using Microsoft.Data.Sqlite;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using ZB.MOM.WW.LocalDb;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.ExternalSystems;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.StoreAndForward;
using ZB.MOM.WW.ScadaBridge.TestSupport;
namespace ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests;
/// <summary>
/// WP-6/7: Tests for ExternalSystemClient — HTTP client, call modes, error handling.
/// </summary>
public class ExternalSystemClientTests
public class ExternalSystemClientTests : IDisposable
{
private readonly IExternalSystemRepository _repository = Substitute.For<IExternalSystemRepository>();
private readonly IHttpClientFactory _httpClientFactory = Substitute.For<IHttpClientFactory>();
/// <summary>
/// Temp local databases opened by the Store-and-Forward tests, torn down in
/// <see cref="Dispose"/>.
/// </summary>
private readonly List<TestLocalDb> _localDbs = new();
/// <summary>
/// Opens a real <c>ILocalDb</c> over a fresh temp file for a test that needs a live
/// <see cref="StoreAndForwardStorage"/>. The store takes an <c>ILocalDb</c> and LocalDb
/// has no in-memory mode, so each test gets its own file rather than a shared-cache
/// in-memory database held open by a keep-alive connection.
/// </summary>
private TestLocalDb CreateLocalDb(string prefix)
{
var localDb = TestLocalDb.CreateTemp(prefix);
_localDbs.Add(localDb);
return localDb;
}
/// <summary>
/// Disposes every temp local database and then deletes its files — in that order,
/// because the master connection LocalDb holds anchors the WAL sidecars.
/// </summary>
public void Dispose()
{
foreach (var localDb in _localDbs)
{
var path = localDb.Path;
localDb.Dispose();
TestLocalDb.DeleteFiles(path);
}
GC.SuppressFinalize(this);
}
/// <summary>
/// Configures the repository substitute for the name-keyed resolution path used by
/// <c>ExternalSystemClient</c> (ExternalSystemGateway-011). A <c>null</c> system or
@@ -275,11 +311,8 @@ public class ExternalSystemClientTests
_httpClientFactory.CreateClient(Arg.Any<string>()).Returns(httpClient);
// A real S&F service with a registered delivery handler that counts invocations.
var dbName = $"EsgDoubleDispatch_{Guid.NewGuid():N}";
var connStr = $"Data Source={dbName};Mode=Memory;Cache=Shared";
using var keepAlive = new SqliteConnection(connStr);
keepAlive.Open();
var storage = new StoreAndForwardStorage(connStr, NullLogger<StoreAndForwardStorage>.Instance);
var localDb = CreateLocalDb("EsgDoubleDispatch");
var storage = new StoreAndForwardStorage(localDb.Db, NullLogger<StoreAndForwardStorage>.Instance);
await storage.InitializeAsync();
var sfOptions = new StoreAndForwardOptions
{
@@ -422,11 +455,8 @@ public class ExternalSystemClientTests
var httpClient = new HttpClient(new MockHttpMessageHandler(HttpStatusCode.InternalServerError, "boom"));
_httpClientFactory.CreateClient(Arg.Any<string>()).Returns(httpClient);
var dbName = $"EsgRetry_{Guid.NewGuid():N}";
var connStr = $"Data Source={dbName};Mode=Memory;Cache=Shared";
using var keepAlive = new SqliteConnection(connStr);
keepAlive.Open();
var storage = new StoreAndForwardStorage(connStr, NullLogger<StoreAndForwardStorage>.Instance);
var localDb = CreateLocalDb("EsgRetry");
var storage = new StoreAndForwardStorage(localDb.Db, NullLogger<StoreAndForwardStorage>.Instance);
await storage.InitializeAsync();
// S&F defaults deliberately different from the system's settings.
var sfOptions = new StoreAndForwardOptions
@@ -454,7 +484,7 @@ public class ExternalSystemClientTests
var depth = await storage.GetBufferDepthByCategoryAsync();
Assert.Equal(1, depth[ZB.MOM.WW.ScadaBridge.Commons.Types.Enums.StoreAndForwardCategory.ExternalSystem]);
var buffered = ReadBufferedRetrySettings(connStr);
var buffered = ReadBufferedRetrySettings(localDb.Db);
Assert.Equal(7, buffered.MaxRetries);
Assert.Equal((long)TimeSpan.FromSeconds(42).TotalMilliseconds, buffered.RetryIntervalMs);
@@ -466,10 +496,10 @@ public class ExternalSystemClientTests
}
private static (int MaxRetries, long RetryIntervalMs, Guid? ExecutionId, string? SourceScript)
ReadBufferedRetrySettings(string connStr)
ReadBufferedRetrySettings(ILocalDb localDb)
{
using var conn = new SqliteConnection(connStr);
conn.Open();
// CreateConnection hands back an already-open, pragma-configured connection.
using var conn = localDb.CreateConnection();
using var cmd = conn.CreateCommand();
cmd.CommandText =
"SELECT max_retries, retry_interval_ms, execution_id, source_script FROM sf_messages";
@@ -505,11 +535,8 @@ public class ExternalSystemClientTests
var httpClient = new HttpClient(new MockHttpMessageHandler(HttpStatusCode.InternalServerError, "boom"));
_httpClientFactory.CreateClient(Arg.Any<string>()).Returns(httpClient);
var dbName = $"EsgRetryZero_{Guid.NewGuid():N}";
var connStr = $"Data Source={dbName};Mode=Memory;Cache=Shared";
using var keepAlive = new SqliteConnection(connStr);
keepAlive.Open();
var storage = new StoreAndForwardStorage(connStr, NullLogger<StoreAndForwardStorage>.Instance);
var localDb = CreateLocalDb("EsgRetryZero");
var storage = new StoreAndForwardStorage(localDb.Db, NullLogger<StoreAndForwardStorage>.Instance);
await storage.InitializeAsync();
var sfOptions = new StoreAndForwardOptions
{
@@ -525,7 +552,7 @@ public class ExternalSystemClientTests
await client.CachedCallAsync("TestAPI", "postData");
var (maxRetries, _, _, _) = ReadBufferedRetrySettings(connStr);
var (maxRetries, _, _, _) = ReadBufferedRetrySettings(localDb.Db);
// Must be the bounded S&F default, never 0 — a stored 0 would mean retry-forever.
Assert.Equal(99, maxRetries);
Assert.NotEqual(0, maxRetries);
@@ -651,11 +678,8 @@ public class ExternalSystemClientTests
var httpClient = new HttpClient(new HangingHttpMessageHandler(TimeSpan.FromMinutes(10)));
_httpClientFactory.CreateClient(Arg.Any<string>()).Returns(httpClient);
var dbName = $"EsgCancel_{Guid.NewGuid():N}";
var connStr = $"Data Source={dbName};Mode=Memory;Cache=Shared";
using var keepAlive = new SqliteConnection(connStr);
keepAlive.Open();
var storage = new StoreAndForwardStorage(connStr, NullLogger<StoreAndForwardStorage>.Instance);
var localDb = CreateLocalDb("EsgCancel");
var storage = new StoreAndForwardStorage(localDb.Db, NullLogger<StoreAndForwardStorage>.Instance);
await storage.InitializeAsync();
var sfOptions = new StoreAndForwardOptions
{
@@ -1060,11 +1084,8 @@ public class ExternalSystemClientTests
.Returns(_ => new HttpClient(new MockHttpMessageHandler(HttpStatusCode.OK, hugeBody)));
// A real S&F service so we can assert the oversized response is NOT buffered.
var dbName = $"EsgOversize_{Guid.NewGuid():N}";
var connStr = $"Data Source={dbName};Mode=Memory;Cache=Shared";
using var keepAlive = new SqliteConnection(connStr);
keepAlive.Open();
var storage = new StoreAndForwardStorage(connStr, NullLogger<StoreAndForwardStorage>.Instance);
var localDb = CreateLocalDb("EsgOversize");
var storage = new StoreAndForwardStorage(localDb.Db, NullLogger<StoreAndForwardStorage>.Instance);
await storage.InitializeAsync();
var sfOptions = new StoreAndForwardOptions
{