Files
ScadaBridge/tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/StaleInstanceProbeTests.cs
T
Joseph Doherty e0e4b24679 fix(deploy+cli): review findings — honest CLI timeouts, watermark-complete staleness, phase-2 staging, lock-safe cancellation
Six adversarial-review findings, each verified against the code first.

F1 (HIGH) CLI HttpClient capped every call at min(30s, caller timeout),
silently truncating deploy site's 5-minute BulkDeployTimeout and the
5-minute bundle export/preview/import calls — which printed a fake
"504 Request timed out" while the server kept working. HttpClient.Timeout
is now Timeout.InfiniteTimeSpan (the per-call CTS is the single overall
deadline, connect included) with the connect phase bounded separately on
SocketsHttpHandler.ConnectTimeout. The env override is renamed to
SCADABRIDGE_HTTP_CONNECT_TIMEOUT_SECONDS to match its new meaning.

F2 (HIGH) StaleInstanceProbe's process-static memo served stale hashes
because nothing bumped the watermark on three paths:
  (a) BundleImporter commits through the raw DbContext, so no import ever
      moved the watermark — a second import overwriting the same template
      could be OMITTED from ImportResult.StaleInstanceIds. It now bumps
      once per apply ATTEMPT: after the commit, and after the rollback too
      (the probe runs pre-commit, so a rolled-back attempt leaves memos for
      state that never landed; bumping on both paths is the simplest
      correct shape, versus threading transaction awareness through a
      process-static cache).
  (b) CollectWatermarkBumps' default: arm silently no-op'd, contradicting
      its own doc. It now sets unattributed=true — an over-broad bump costs
      extra work, a missed one produces stale work.
  (c) DataConnection edits route through SiteRepository.SaveChangesAsync,
      which had no watermark at all, yet Protocol/Primary+Backup config/
      FailoverRetryCount are revision-hash inputs. It now bumps (BumpAll —
      a connection has no owning template) after a commit that touched one.

F3 (MED) CLI TemplateTableProjection read child ARRAYS, but ListTemplates
now returns database-projected TemplateSummary rows, so template list
printed all zeros. It now prefers the *Count scalars and falls back to
array length (template get still returns full entities). --detail help
text and README corrected: a listing cannot yield definitions, so --detail
renders the raw summary payload and template get --id is the full dump.

F4 (MED) DeploySiteAsync staged every PendingDeployment in phase 1 against
a 5-min TTL while phase 2 reached them one batch at a time, so tail
instances' fetch tokens could expire before their command was sent.
Staging moved into phase 2, immediately before each send; prepare keeps
its flatten/validate/record work. The staging write is the phase's only
repository touch and is serialised behind a 1-permit semaphore, so the
non-thread-safe DbContext constraint holds and the sends stay concurrent.

F5 (MED, latent) DeploySiteAsync leaked every held operation lock if
cancelled — a wedged per-instance semaphore is permanent for the process.
Phase 2 no longer throws (cancellation is recorded as a per-instance
outcome so phase 3 still runs), and an escape from phase 1 or 3 now
unwinds every unfinalised entry: Failed status + lock release.

F6 (LOW) ScriptCompileVerdictCache's promotion wrote hot directly,
bypassing SegmentCapacity (true ceiling 3x against a documented 2x).
Promotion now goes through Store, keeping generational semantics; _hot
and _cold are volatile.

Tests: CLI 396, DeploymentManager 133, ManagementService 494,
TemplateEngine 478, ScriptAnalysis 60, Transport 157, Transport
integration 106, ConfigurationDatabase 366 — all green, 0 build warnings.
The F2/F4/F5 regression tests were each confirmed to FAIL with their fix
reverted.
2026-08-14 23:51:04 -04:00

100 lines
3.8 KiB
C#

using NSubstitute;
using ZB.MOM.WW.ScadaBridge.Commons.Types;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Flattening;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
using ZB.MOM.WW.ScadaBridge.TemplateEngine.Flattening;
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests;
/// <summary>
/// The staleness fast path: <see cref="StaleInstanceProbe"/> memoises a computed
/// revision hash against the <see cref="ITemplateGraphWatermark"/> readings it was
/// computed under, and the memo is PROCESS-STATIC. That makes the watermark the
/// only thing standing between a cached hash and a wrong answer — so these tests
/// pin that the memo is served only while the watermark has not moved, and is
/// dropped the moment it has.
///
/// <para>
/// The failure this guards against is a real one: a data-connection edit, or a
/// bundle import committing through the raw <c>DbContext</c>, changes the flattened
/// output without going through the repository path that derives bumps from the
/// change tracker. If that write does not bump, the memo below keeps answering with
/// the old hash forever.
/// </para>
/// </summary>
public class StaleInstanceProbeTests
{
private const int InstanceId = 1;
private readonly TemplateGraphWatermark _watermark = new();
private readonly IFlatteningPipeline _pipeline = Substitute.For<IFlatteningPipeline>();
private string _currentHash = "sha256:first";
private int _flattenCount;
public StaleInstanceProbeTests()
{
// Process-static memo: start every test from a clean slate.
StaleInstanceProbe.ClearMemos();
_pipeline.CreateSession().Returns(_ => new FlattenSession(_watermark));
_pipeline
.FlattenAndValidateAsync(InstanceId, Arg.Any<CancellationToken>(), Arg.Any<bool>(), Arg.Any<FlattenSession?>())
.Returns(_ =>
{
_flattenCount++;
var config = new FlattenedConfiguration { InstanceUniqueName = "Inst-01" };
return Result<FlatteningPipelineResult>.Success(
new FlatteningPipelineResult(config, _currentHash, ValidationResult.Success()));
});
}
private StaleInstanceProbe CreateProbe() => new(_pipeline, _watermark);
[Fact]
public async Task UnchangedWatermark_ServesMemoisedHash_WithoutReflattening()
{
var probe = CreateProbe();
Assert.Equal("sha256:first", await probe.GetCurrentRevisionHashAsync(InstanceId));
Assert.Equal(1, _flattenCount);
Assert.Equal("sha256:first", await probe.GetCurrentRevisionHashAsync(InstanceId));
Assert.Equal(1, _flattenCount);
}
/// <summary>
/// <c>BumpAll</c> is the unattributed fallback used by everything that changes a
/// flattening input without an owning template id — data-connection saves and
/// bundle imports both rely on it. It must invalidate the memo.
/// </summary>
[Fact]
public async Task BumpAll_InvalidatesTheMemo()
{
var probe = CreateProbe();
await probe.GetCurrentRevisionHashAsync(InstanceId);
// The underlying config drifted (e.g. a data connection was repointed).
_currentHash = "sha256:second";
// Without a bump the stale hash would still be served...
Assert.Equal("sha256:first", await probe.GetCurrentRevisionHashAsync(InstanceId));
_watermark.BumpAll();
Assert.Equal("sha256:second", await probe.GetCurrentRevisionHashAsync(InstanceId));
Assert.Equal(2, _flattenCount);
}
[Fact]
public async Task BumpInstance_InvalidatesTheMemoForThatInstance()
{
var probe = CreateProbe();
await probe.GetCurrentRevisionHashAsync(InstanceId);
_currentHash = "sha256:second";
_watermark.BumpInstance(InstanceId);
Assert.Equal("sha256:second", await probe.GetCurrentRevisionHashAsync(InstanceId));
}
}