feat(pending-updates): short-circuit an unchanged fleet on a DB rowversion watermark (#62)

Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every
deployed instance on each cache-TTL expiry even when nothing had changed.

- Migration AddConfigGraphRowVersions: a non-concurrency rowversion column
  + index on the 18 configuration tables the evaluation reads (Templates +
  5 member tables, Instances + 4 override/binding tables, DataConnections,
  DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions,
  ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in
  the model; SQLite test contexts are untouched. Script:
  docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated
  central-pair migration window (#107 item 4).
- IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of
  MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which
  raise no rowversion). rowversion is engine-bumped for any writer, so a
  peer central node's edit moves it — not the per-process watermark MG-3
  removed.
- The secret store (outside the DB) gets its own stamp: SHA-256 over
  ListAsync(includeDeleted: true) metadata.
- Evaluator re-serves the last CLEAN snapshot while watermark + secret
  stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default
  10, validated >= 1, 1 = off) is full regardless — the safety net for a
  same-KEK re-wrap or an input table missing from the set. Always full when
  the token is unavailable/unreadable, after an evaluation that could not
  compute a site's artifact hash, on Re-check, and after a deploy completion.
  It only re-serves a real earlier answer, never manufactures one.

Tests: SQL Server (real migrated DB) — token stable without writes, moves
for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local
EF write, a detached Update still saves (not a concurrency token), and a
guard that every table the evaluator's repository reads touch is covered.
Evaluator — reuse without flatten (re-served answer still shows the stale
instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes,
unclean evaluation not reused, cache Re-check and deploy completion force
full. Mutation-proven: token compare, clean check, cache force flag, row
count, and an uncovered read each turn tests red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 22:38:55 -05:00
parent 63ec45e2e3
commit a4806576af
20 changed files with 3963 additions and 9 deletions
@@ -0,0 +1,97 @@
using System.Data;
using System.Globalization;
using System.Text;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
/// <summary>
/// SQL Server <see cref="IConfigGraphWatermarkReader"/> (Gitea #62): ONE query returning
/// <c>MAX(RowVersion)</c> and <c>COUNT_BIG(*)</c> for every table in
/// <see cref="ConfigGraphRowVersions.EntityTypes"/>, folded into an opaque token.
///
/// <para>
/// Why both numbers: <c>rowversion</c> is a database-wide monotonic counter bumped by the
/// engine on every INSERT and UPDATE — whoever issues it (the peer central node, a bundle
/// import, an <c>ExecuteUpdate</c>, a hand-run script) — so any insert or update raises some
/// table's maximum. A DELETE raises nothing, so the per-table row count catches it; a
/// delete paired with an insert is caught by the insert's new maximum.
/// </para>
/// </summary>
public sealed class ConfigGraphWatermarkReader : IConfigGraphWatermarkReader
{
private readonly ScadaBridgeDbContext _context;
/// <summary>Creates the reader over the scoped configuration context.</summary>
/// <param name="context">The configuration database context.</param>
public ConfigGraphWatermarkReader(ScadaBridgeDbContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
/// <inheritdoc />
public async Task<string?> ReadAsync(CancellationToken cancellationToken = default)
{
// The columns only exist under SQL Server (ScadaBridgeDbContext.OnModelCreating).
if (!_context.Database.IsSqlServer())
return null;
var sql = BuildQuery(_context);
var connection = _context.Database.GetDbConnection();
var openedHere = connection.State != ConnectionState.Open;
if (openedHere)
await _context.Database.OpenConnectionAsync(cancellationToken);
try
{
await using var command = connection.CreateCommand();
command.CommandText = sql;
command.Transaction = _context.Database.CurrentTransaction?.GetDbTransaction();
var token = new StringBuilder();
await using var reader = await command.ExecuteReaderAsync(cancellationToken);
while (await reader.ReadAsync(cancellationToken))
{
token.Append(reader.GetString(0)).Append(':');
if (!reader.IsDBNull(1))
token.Append(Convert.ToHexString((byte[])reader.GetValue(1)));
token.Append(':')
.Append(reader.GetInt64(2).ToString(CultureInfo.InvariantCulture))
.Append(';');
}
return token.ToString();
}
finally
{
if (openedHere)
await _context.Database.CloseConnectionAsync();
}
}
/// <summary>
/// Builds the single <c>UNION ALL</c> query from the EF model's table mappings, so a
/// renamed table follows the model rather than a hard-coded name.
/// </summary>
/// <param name="context">The context whose model supplies the table names.</param>
/// <returns>The SQL text.</returns>
internal static string BuildQuery(DbContext context)
{
var parts = new List<string>(ConfigGraphRowVersions.EntityTypes.Count);
foreach (var type in ConfigGraphRowVersions.EntityTypes)
{
var entity = context.Model.FindEntityType(type)
?? throw new InvalidOperationException($"Entity {type.Name} is not in the model.");
var table = entity.GetTableName()
?? throw new InvalidOperationException($"Entity {type.Name} is not mapped to a table.");
var schema = entity.GetSchema() ?? "dbo";
parts.Add(
$"SELECT N'{table}' AS [T], MAX([{ConfigGraphRowVersions.ColumnName}]) AS [V], COUNT_BIG(*) AS [C] " +
$"FROM [{schema}].[{table}]");
}
// ORDER BY: a UNION ALL has no guaranteed row order, and the token is compared as a string.
return string.Join(" UNION ALL ", parts) + " ORDER BY [T];";
}
}