dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(pending-updates): short-circuit an unchanged fleet on a DB rowversion watermark (#62)

Browse Source 
Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every
deployed instance on each cache-TTL expiry even when nothing had changed.

- Migration AddConfigGraphRowVersions: a non-concurrency rowversion column
  + index on the 18 configuration tables the evaluation reads (Templates +
  5 member tables, Instances + 4 override/binding tables, DataConnections,
  DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions,
  ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in
  the model; SQLite test contexts are untouched. Script:
  docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated
  central-pair migration window (#107 item 4).
- IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of
  MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which
  raise no rowversion). rowversion is engine-bumped for any writer, so a
  peer central node's edit moves it — not the per-process watermark MG-3
  removed.
- The secret store (outside the DB) gets its own stamp: SHA-256 over
  ListAsync(includeDeleted: true) metadata.
- Evaluator re-serves the last CLEAN snapshot while watermark + secret
  stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default
  10, validated >= 1, 1 = off) is full regardless — the safety net for a
  same-KEK re-wrap or an input table missing from the set. Always full when
  the token is unavailable/unreadable, after an evaluation that could not
  compute a site's artifact hash, on Re-check, and after a deploy completion.
  It only re-serves a real earlier answer, never manufactures one.

Tests: SQL Server (real migrated DB) — token stable without writes, moves
for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local
EF write, a detached Update still saves (not a concurrency token), and a
guard that every table the evaluator's repository reads touch is covered.
Evaluator — reuse without flatten (re-served answer still shows the stale
instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes,
unclean evaluation not reused, cache Re-check and deploy completion force
full. Mutation-proven: token compare, clean check, cache force flag, row
count, and an uncovered read each turn tests red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 22:38:55 -05:00
parent 63ec45e2e3
commit a4806576af
20 changed files with 3963 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IConfigGraphWatermarkReader.cs
+30
View File
@@ -0,0 +1,30 @@
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
/// <summary>
/// Reads a cheap, DATABASE-side change token over every configuration-database table that
/// feeds the pending-updates evaluation — instance staleness (the flatten inputs plus the
/// deployed snapshots) and the per-site artifact revision hash (Gitea #62).
///
/// <para>
/// The token is derived from SQL Server <c>rowversion</c> columns, so it moves on ANY
/// insert or update of those tables — including a write made by the peer central node, a
/// bundle import, a raw <c>ExecuteUpdate</c> or a hand-edited row — without any write path
/// having to remember to bump it. Each table contributes <c>MAX(rowversion)</c> AND its row
/// count, because a delete never raises a rowversion: a removed override or template member
/// is caught by the count. This is deliberately NOT the per-process template-graph
/// watermark: that one never sees a peer-node edit (arch-review 2026-09-09 MG-3).
/// </para>
/// </summary>
public interface IConfigGraphWatermarkReader
{
/// <summary>
/// Reads the current change token in one round trip.
/// </summary>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// An opaque token that is equal across two reads only when none of the covered tables
/// changed in between; <see langword="null"/> when the provider cannot produce one (a
/// non-SQL-Server test provider), in which case callers must not short-circuit.
/// </returns>
Task<string?> ReadAsync(CancellationToken cancellationToken = default);
}
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ConfigGraphRowVersions.cs
+92
View File
@@ -0,0 +1,92 @@
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Deployment;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.ExternalSystems;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Scripts;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase;
/// <summary>
/// Gitea #62: the <c>rowversion</c> shadow columns behind
/// <see cref="Commons.Interfaces.Repositories.IConfigGraphWatermarkReader"/>.
///
/// <para>
/// <b>The covered set is exactly what <c>PendingUpdatesEvaluator</c> reads.</b> Instance
/// staleness re-flattens each deployed instance (<c>FlatteningPipeline.FlattenCoreAsync</c>:
/// the instance with its four override/binding collections, the template inheritance and
/// composition graph with its five member collections, the site's data connections) and
/// compares against the deployed snapshots; the artifact hash
/// (<c>ArtifactDeploymentService.ComputeCurrentArtifactHashesAsync</c>) reads shared
/// scripts, external systems + their methods, database connections and data connections,
/// and compares against the hash stamped on each site row. The secret store, which also
/// feeds the artifact hash, is outside this database and has its own stamp in the evaluator.
/// A table added to either read path MUST be added here, or a change to it is only picked
/// up by the evaluator's periodic forced full evaluation.
/// </para>
///
/// <para>
/// The columns are NOT concurrency tokens: they exist only to be compared as a maximum, and
/// making them tokens would change the conflict behaviour of every save on these tables
/// (a detached <c>Update</c> would carry a null original value and fail). They are SQL
/// Server only — <see cref="ScadaBridgeDbContext"/> applies them only under that provider,
/// so the SQLite test contexts never see a column SQLite cannot generate.
/// </para>
/// </summary>
public static class ConfigGraphRowVersions
{
/// <summary>The shadow property / column name on every covered table.</summary>
public const string ColumnName = "RowVersion";
/// <summary>The entity types whose tables are covered by the watermark.</summary>
public static IReadOnlyList<Type> EntityTypes { get; } =
[
// Flatten inputs: the template graph.
typeof(Template),
typeof(TemplateAttribute),
typeof(TemplateAlarm),
typeof(TemplateScript),
typeof(TemplateComposition),
typeof(TemplateNativeAlarmSource),
// Flatten inputs: the instance and its overrides/bindings.
typeof(Instance),
typeof(InstanceAttributeOverride),
typeof(InstanceAlarmOverride),
typeof(InstanceConnectionBinding),
typeof(InstanceNativeAlarmSourceOverride),
// Flatten input AND artifact input.
typeof(DataConnection),
// The deployed side of instance staleness.
typeof(DeployedConfigSnapshot),
// Artifact inputs, and the site row carrying the stamped artifact hash.
typeof(Site),
typeof(SharedScript),
typeof(ExternalSystemDefinition),
typeof(ExternalSystemMethod),
typeof(DatabaseConnectionDefinition),
];
/// <summary>
/// Adds the non-concurrency <c>rowversion</c> shadow column (and an index on it, so
/// <c>MAX</c> is a seek and <c>COUNT_BIG</c> scans the narrowest index) to every covered table.
/// </summary>
/// <param name="modelBuilder">The model builder.</param>
public static void Apply(ModelBuilder modelBuilder)
{
ArgumentNullException.ThrowIfNull(modelBuilder);
foreach (var type in EntityTypes)
{
var entity = modelBuilder.Entity(type);
// Spelled out rather than IsRowVersion(): that also makes the column a concurrency
// token, and switching the token off afterwards drops the provider's rowversion
// type mapping (the column would scaffold as a nullable varbinary(900)).
entity.Property<byte[]>(ColumnName)
.HasColumnType("rowversion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.IsConcurrencyToken(false);
entity.HasIndex(ColumnName);
}
}
}
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/20261110032839_AddConfigGraphRowVersions.Designer.cs
Generated
+2381
View File
File diff suppressed because it is too large Load Diff
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/20261110032839_AddConfigGraphRowVersions.cs
+393
View File
@@ -0,0 +1,393 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
{
/// <summary>
/// Gitea #62: a <c>rowversion</c> column (+ index) on the 18 configuration tables that feed
/// the pending-updates evaluation — the flatten inputs (Templates and their attribute /
/// alarm / script / composition / native-alarm-source members, Instances and their four
/// override/binding tables, DataConnections), the deployed side (DeployedConfigSnapshots)
/// and the artifact-hash inputs (Sites, SharedScripts, ExternalSystemDefinitions,
/// ExternalSystemMethods, DatabaseConnectionDefinitions). <c>PendingUpdatesEvaluator</c>
/// short-circuits when MAX(RowVersion) + COUNT_BIG(*) over them is unchanged.
/// <para>
/// Purely additive: SQL Server fills a rowversion for every existing row as the column is
/// added (a size-of-data operation on each table — these are configuration tables, small
/// next to AuditLog). Old binaries ignore the column (it is never a concurrency token and
/// is never written by the application). Ships in the coordinated central-pair migration
/// window with the other pending migrations (ScadaBridge#107 item 4).
/// </para>
/// </summary>
public partial class AddConfigGraphRowVersions : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "TemplateScripts",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "Templates",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "TemplateNativeAlarmSources",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "TemplateCompositions",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "TemplateAttributes",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "TemplateAlarms",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "Sites",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "SharedScripts",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "Instances",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "InstanceNativeAlarmSourceOverrides",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "InstanceConnectionBindings",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "InstanceAttributeOverrides",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "InstanceAlarmOverrides",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "ExternalSystemMethods",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "ExternalSystemDefinitions",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "DeployedConfigSnapshots",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "DataConnections",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.AddColumn<byte[]>(
name: "RowVersion",
table: "DatabaseConnectionDefinitions",
type: "rowversion",
rowVersion: true,
nullable: false);
migrationBuilder.CreateIndex(
name: "IX_TemplateScripts_RowVersion",
table: "TemplateScripts",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_Templates_RowVersion",
table: "Templates",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_TemplateNativeAlarmSources_RowVersion",
table: "TemplateNativeAlarmSources",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_TemplateCompositions_RowVersion",
table: "TemplateCompositions",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_TemplateAttributes_RowVersion",
table: "TemplateAttributes",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_TemplateAlarms_RowVersion",
table: "TemplateAlarms",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_Sites_RowVersion",
table: "Sites",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_SharedScripts_RowVersion",
table: "SharedScripts",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_Instances_RowVersion",
table: "Instances",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_InstanceNativeAlarmSourceOverrides_RowVersion",
table: "InstanceNativeAlarmSourceOverrides",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_InstanceConnectionBindings_RowVersion",
table: "InstanceConnectionBindings",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_InstanceAttributeOverrides_RowVersion",
table: "InstanceAttributeOverrides",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_InstanceAlarmOverrides_RowVersion",
table: "InstanceAlarmOverrides",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_ExternalSystemMethods_RowVersion",
table: "ExternalSystemMethods",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_ExternalSystemDefinitions_RowVersion",
table: "ExternalSystemDefinitions",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_DeployedConfigSnapshots_RowVersion",
table: "DeployedConfigSnapshots",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_DataConnections_RowVersion",
table: "DataConnections",
column: "RowVersion");
migrationBuilder.CreateIndex(
name: "IX_DatabaseConnectionDefinitions_RowVersion",
table: "DatabaseConnectionDefinitions",
column: "RowVersion");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropIndex(
name: "IX_TemplateScripts_RowVersion",
table: "TemplateScripts");
migrationBuilder.DropIndex(
name: "IX_Templates_RowVersion",
table: "Templates");
migrationBuilder.DropIndex(
name: "IX_TemplateNativeAlarmSources_RowVersion",
table: "TemplateNativeAlarmSources");
migrationBuilder.DropIndex(
name: "IX_TemplateCompositions_RowVersion",
table: "TemplateCompositions");
migrationBuilder.DropIndex(
name: "IX_TemplateAttributes_RowVersion",
table: "TemplateAttributes");
migrationBuilder.DropIndex(
name: "IX_TemplateAlarms_RowVersion",
table: "TemplateAlarms");
migrationBuilder.DropIndex(
name: "IX_Sites_RowVersion",
table: "Sites");
migrationBuilder.DropIndex(
name: "IX_SharedScripts_RowVersion",
table: "SharedScripts");
migrationBuilder.DropIndex(
name: "IX_Instances_RowVersion",
table: "Instances");
migrationBuilder.DropIndex(
name: "IX_InstanceNativeAlarmSourceOverrides_RowVersion",
table: "InstanceNativeAlarmSourceOverrides");
migrationBuilder.DropIndex(
name: "IX_InstanceConnectionBindings_RowVersion",
table: "InstanceConnectionBindings");
migrationBuilder.DropIndex(
name: "IX_InstanceAttributeOverrides_RowVersion",
table: "InstanceAttributeOverrides");
migrationBuilder.DropIndex(
name: "IX_InstanceAlarmOverrides_RowVersion",
table: "InstanceAlarmOverrides");
migrationBuilder.DropIndex(
name: "IX_ExternalSystemMethods_RowVersion",
table: "ExternalSystemMethods");
migrationBuilder.DropIndex(
name: "IX_ExternalSystemDefinitions_RowVersion",
table: "ExternalSystemDefinitions");
migrationBuilder.DropIndex(
name: "IX_DeployedConfigSnapshots_RowVersion",
table: "DeployedConfigSnapshots");
migrationBuilder.DropIndex(
name: "IX_DataConnections_RowVersion",
table: "DataConnections");
migrationBuilder.DropIndex(
name: "IX_DatabaseConnectionDefinitions_RowVersion",
table: "DatabaseConnectionDefinitions");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "TemplateScripts");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "Templates");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "TemplateNativeAlarmSources");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "TemplateCompositions");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "TemplateAttributes");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "TemplateAlarms");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "Sites");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "SharedScripts");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "Instances");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "InstanceNativeAlarmSourceOverrides");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "InstanceConnectionBindings");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "InstanceAttributeOverrides");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "InstanceAlarmOverrides");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "ExternalSystemMethods");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "ExternalSystemDefinitions");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "DeployedConfigSnapshots");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "DataConnections");
migrationBuilder.DropColumn(
name: "RowVersion",
table: "DatabaseConnectionDefinitions");
}
}
}
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/ScadaBridgeDbContextModelSnapshot.cs
+126
View File
@@ -229,6 +229,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(100)
.HasColumnType("nvarchar(100)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("DeploymentId");
@@ -236,6 +241,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("InstanceId")
.IsUnique();
b.HasIndex("RowVersion");
b.ToTable("DeployedConfigSnapshots");
});
@@ -408,11 +415,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<TimeSpan>("RetryDelay")
.HasColumnType("time");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("Name")
.IsUnique();
b.HasIndex("RowVersion");
b.ToTable("DatabaseConnectionDefinitions");
});
@@ -449,6 +463,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<TimeSpan>("RetryDelay")
.HasColumnType("time");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("TimeoutSeconds")
.ValueGeneratedOnAdd()
.HasColumnType("int")
@@ -459,6 +478,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("Name")
.IsUnique();
b.HasIndex("RowVersion");
b.ToTable("ExternalSystemDefinitions");
});
@@ -496,8 +517,15 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(4000)
.HasColumnType("nvarchar(4000)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("ExternalSystemDefinitionId", "Name")
.IsUnique();
@@ -586,6 +614,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<int?>("AreaId")
.HasColumnType("int");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("SiteId")
.HasColumnType("int");
@@ -606,6 +639,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("AreaId");
b.HasIndex("RowVersion");
b.HasIndex("SiteId");
b.HasIndex("TemplateId");
@@ -638,12 +673,19 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<int?>("PriorityLevelOverride")
.HasColumnType("int");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<string>("TriggerConfigurationOverride")
.HasMaxLength(4000)
.HasColumnType("nvarchar(4000)");
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("InstanceId", "AlarmCanonicalName")
.IsUnique();
@@ -673,8 +715,15 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<string>("OverrideValue")
.HasColumnType("nvarchar(max)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("InstanceId", "AttributeName")
.IsUnique();
@@ -704,10 +753,17 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<int>("InstanceId")
.HasColumnType("int");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("DataConnectionId");
b.HasIndex("RowVersion");
b.HasIndex("InstanceId", "AttributeName")
.IsUnique();
@@ -736,6 +792,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<bool?>("IsMesReportableOverride")
.HasColumnType("bit");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<string>("SourceCanonicalName")
.IsRequired()
.HasMaxLength(400)
@@ -747,6 +808,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("InstanceId", "SourceCanonicalName")
.IsUnique();
@@ -1175,11 +1238,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(4000)
.HasColumnType("nvarchar(4000)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("Name")
.IsUnique();
b.HasIndex("RowVersion");
b.ToTable("SharedScripts");
});
@@ -1384,11 +1454,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(50)
.HasColumnType("nvarchar(50)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("SiteId")
.HasColumnType("int");
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("SiteId", "Name")
.IsUnique();
@@ -1435,6 +1512,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(500)
.HasColumnType("nvarchar(500)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<string>("SiteIdentifier")
.IsRequired()
.HasMaxLength(100)
@@ -1445,6 +1527,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("Name")
.IsUnique();
b.HasIndex("RowVersion");
b.HasIndex("SiteIdentifier")
.IsUnique();
@@ -1480,6 +1564,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<int?>("ParentTemplateId")
.HasColumnType("int");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.HasKey("Id");
b.HasIndex("FolderId");
@@ -1490,6 +1579,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("ParentTemplateId");
b.HasIndex("RowVersion");
b.ToTable("Templates");
});
@@ -1528,6 +1619,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.Property<int>("PriorityLevel")
.HasColumnType("int");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("TemplateId")
.HasColumnType("int");
@@ -1542,6 +1638,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("TemplateId", "Name")
.IsUnique();
@@ -1587,6 +1685,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(200)
.HasColumnType("nvarchar(200)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("TemplateId")
.HasColumnType("int");
@@ -1595,6 +1698,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("TemplateId", "Name")
.IsUnique();
@@ -1617,6 +1722,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(200)
.HasColumnType("nvarchar(200)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("TemplateId")
.HasColumnType("int");
@@ -1624,6 +1734,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasIndex("ComposedTemplateId");
b.HasIndex("RowVersion");
b.HasIndex("TemplateId", "InstanceName")
.IsUnique();
@@ -1696,6 +1808,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(200)
.HasColumnType("nvarchar(200)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<string>("SourceReference")
.IsRequired()
.HasMaxLength(1000)
@@ -1706,6 +1823,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("TemplateId", "Name")
.IsUnique();
@@ -1752,6 +1871,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
.HasMaxLength(4000)
.HasColumnType("nvarchar(4000)");
b.Property<byte[]>("RowVersion")
.IsRequired()
.ValueGeneratedOnAddOrUpdate()
.HasColumnType("rowversion");
b.Property<int>("TemplateId")
.HasColumnType("int");
@@ -1765,6 +1889,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
b.HasKey("Id");
b.HasIndex("RowVersion");
b.HasIndex("TemplateId", "Name")
.IsUnique();
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/ConfigGraphWatermarkReader.cs
+97
View File
@@ -0,0 +1,97 @@
using System.Data;
using System.Globalization;
using System.Text;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
/// <summary>
/// SQL Server <see cref="IConfigGraphWatermarkReader"/> (Gitea #62): ONE query returning
/// <c>MAX(RowVersion)</c> and <c>COUNT_BIG(*)</c> for every table in
/// <see cref="ConfigGraphRowVersions.EntityTypes"/>, folded into an opaque token.
///
/// <para>
/// Why both numbers: <c>rowversion</c> is a database-wide monotonic counter bumped by the
/// engine on every INSERT and UPDATE — whoever issues it (the peer central node, a bundle
/// import, an <c>ExecuteUpdate</c>, a hand-run script) — so any insert or update raises some
/// table's maximum. A DELETE raises nothing, so the per-table row count catches it; a
/// delete paired with an insert is caught by the insert's new maximum.
/// </para>
/// </summary>
public sealed class ConfigGraphWatermarkReader : IConfigGraphWatermarkReader
{
private readonly ScadaBridgeDbContext _context;
/// <summary>Creates the reader over the scoped configuration context.</summary>
/// <param name="context">The configuration database context.</param>
public ConfigGraphWatermarkReader(ScadaBridgeDbContext context)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
}
/// <inheritdoc />
public async Task<string?> ReadAsync(CancellationToken cancellationToken = default)
{
// The columns only exist under SQL Server (ScadaBridgeDbContext.OnModelCreating).
if (!_context.Database.IsSqlServer())
return null;
var sql = BuildQuery(_context);
var connection = _context.Database.GetDbConnection();
var openedHere = connection.State != ConnectionState.Open;
if (openedHere)
await _context.Database.OpenConnectionAsync(cancellationToken);
try
{
await using var command = connection.CreateCommand();
command.CommandText = sql;
command.Transaction = _context.Database.CurrentTransaction?.GetDbTransaction();
var token = new StringBuilder();
await using var reader = await command.ExecuteReaderAsync(cancellationToken);
while (await reader.ReadAsync(cancellationToken))
{
token.Append(reader.GetString(0)).Append(':');
if (!reader.IsDBNull(1))
token.Append(Convert.ToHexString((byte[])reader.GetValue(1)));
token.Append(':')
.Append(reader.GetInt64(2).ToString(CultureInfo.InvariantCulture))
.Append(';');
}
return token.ToString();
}
finally
{
if (openedHere)
await _context.Database.CloseConnectionAsync();
}
}
/// <summary>
/// Builds the single <c>UNION ALL</c> query from the EF model's table mappings, so a
/// renamed table follows the model rather than a hard-coded name.
/// </summary>
/// <param name="context">The context whose model supplies the table names.</param>
/// <returns>The SQL text.</returns>
internal static string BuildQuery(DbContext context)
{
var parts = new List<string>(ConfigGraphRowVersions.EntityTypes.Count);
foreach (var type in ConfigGraphRowVersions.EntityTypes)
{
var entity = context.Model.FindEntityType(type)
?? throw new InvalidOperationException($"Entity {type.Name} is not in the model.");
var table = entity.GetTableName()
?? throw new InvalidOperationException($"Entity {type.Name} is not mapped to a table.");
var schema = entity.GetSchema() ?? "dbo";
parts.Add(
$"SELECT N'{table}' AS [T], MAX([{ConfigGraphRowVersions.ColumnName}]) AS [V], COUNT_BIG(*) AS [C] " +
$"FROM [{schema}].[{table}]");
}
// ORDER BY: a UNION ALL has no guaranteed row order, and the token is compared as a string.
return string.Join(" UNION ALL ", parts) + " ORDER BY [T];";
}
}
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ScadaBridgeDbContext.cs
+6
View File
@@ -186,6 +186,12 @@ public class ScadaBridgeDbContext : DbContext, IDataProtectionKeyContext
ApplySecretColumnEncryption(modelBuilder);
// Gitea #62: rowversion change-detection columns on the pending-updates input tables.
// SQL Server only (true for production and the design-time factory, so the migration
// and snapshot carry them); SQLite cannot generate a rowversion.
if (Database.IsSqlServer())
ConfigGraphRowVersions.Apply(modelBuilder);
NeutralizeSqlServerComputedColumnsForNonSqlServerProviders(modelBuilder);
}
src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ServiceCollectionExtensions.cs
+1
View File
@@ -82,6 +82,7 @@ public static class ServiceCollectionExtensions
services.AddScoped<ICentralUiRepository, CentralUiRepository>();
services.AddScoped<ITemplateEngineRepository, TemplateEngineRepository>();
services.AddScoped<IDeploymentManagerRepository, DeploymentManagerRepository>();
services.AddScoped<IConfigGraphWatermarkReader, ConfigGraphWatermarkReader>();
services.AddScoped<ISiteRepository, SiteRepository>();
services.AddScoped<IExternalSystemRepository, ExternalSystemRepository>();
services.AddScoped<INotificationRepository, NotificationRepository>();
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentManagerOptions.cs
+12
View File
@@ -92,4 +92,16 @@ public class DeploymentManagerOptions
/// </para>
/// </summary>
public TimeSpan PendingUpdatesCacheTtl { get; set; } = TimeSpan.FromSeconds(60);
/// <summary>
/// Gitea #62: the pending-updates evaluator skips the fleet flatten and re-serves its last
/// snapshot while the database-side config watermark (rowversion MAX + row count over every
/// input table) and the secret-store stamp are both unchanged — but at most this many
/// evaluations in a row: every Nth evaluation is a forced full one, the safety net for
/// anything the two stamps cannot see (a KEK re-wrap that keeps its KEK id, an input table
/// someone forgets to add to the watermark). With the default 60 s TTL, 10 bounds such a
/// miss to about ten minutes of a fleet being looked at. <c>1</c> disables the short-circuit
/// (every evaluation is full). Must be at least 1.
/// </summary>
public int PendingUpdatesFullEvaluationEvery { get; set; } = 10;
}
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentManagerOptionsValidator.cs
+5
View File
@@ -49,6 +49,11 @@ public sealed class DeploymentManagerOptionsValidator : OptionsValidatorBase<Dep
$"pending-updates snapshot, and a non-positive value would re-flatten the whole fleet on " +
$"every poll of every open circuit.");
builder.RequireThat(options.PendingUpdatesFullEvaluationEvery >= 1,
$"ScadaBridge:DeploymentManager:PendingUpdatesFullEvaluationEvery must be at least 1 " +
$"(was {options.PendingUpdatesFullEvaluationEvery}); it is how many pending-updates " +
$"evaluations may re-serve an unchanged snapshot before a forced full one (1 = always full).");
builder.RequireThat(options.TerminalDeploymentRecordRetention >= TimeSpan.Zero,
$"ScadaBridge:DeploymentManager:TerminalDeploymentRecordRetention must not be negative " +
$"(was {options.TerminalDeploymentRecordRetention}); use TimeSpan.Zero to disable purging of " +
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/PendingUpdates/PendingUpdatesCache.cs
+17 -2
View File
@@ -39,6 +39,10 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
private readonly Timer _invalidatedTimer;
private volatile bool _disposed;
// Gitea #62: set by an explicit Re-check and by a deploy completion so the next evaluation
// ignores the evaluator's unchanged-watermark short-circuit and re-flattens the fleet.
private volatile bool _forceFullNext;
/// <inheritdoc/>
public event Action? Invalidated;
@@ -79,8 +83,12 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
/// <inheritdoc/>
public Task<PendingUpdatesSnapshot> GetAsync(
bool forceRefresh = false, CancellationToken cancellationToken = default) =>
_memo.GetAsync(EvaluateAsync, forceRefresh, cancellationToken);
bool forceRefresh = false, CancellationToken cancellationToken = default)
{
if (forceRefresh)
_forceFullNext = true;
return _memo.GetAsync(EvaluateAsync, forceRefresh, cancellationToken);
}
/// <summary>Stops listening for deployment status changes.</summary>
public void Dispose()
@@ -99,6 +107,7 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
// Drop the snapshot at once so no caller is served the pre-deploy count, and (re)arm the
// quiet-period timer so a bulk deploy's burst of completions notifies subscribers once.
_forceFullNext = true;
_memo.Invalidate();
try
{
@@ -121,6 +130,12 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
private async Task<PendingUpdatesSnapshot> EvaluateAsync()
{
await using var scope = _scopeFactory.CreateAsyncScope();
if (_forceFullNext)
{
_forceFullNext = false;
scope.ServiceProvider.GetService<PendingUpdatesShortCircuitState>()?.Reset();
}
return await scope.ServiceProvider.GetRequiredService<IPendingUpdatesEvaluator>().EvaluateAsync();
}
}
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/PendingUpdates/PendingUpdatesEvaluator.cs
+129
View File
@@ -1,9 +1,14 @@
using System.Diagnostics;
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
@@ -29,6 +34,23 @@ namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
/// answer that must never be manufactured — it would silently hide a fleet waiting
/// on a deploy.
/// </para>
///
/// <para>
/// <b>Short-circuit when nothing changed (Gitea #62).</b> Before the fleet pass it reads a
/// change token — the database-side <see cref="IConfigGraphWatermarkReader"/> (rowversion
/// MAX + row count over every input table, so a peer central node's edit moves it) plus a
/// stamp over the secret store's metadata (name, revision, updated time, tombstone, KEK id,
/// content type, description — every field of a secret that feeds the artifact hash except
/// the wrap envelope). When that token equals the one read before the last CLEAN full
/// evaluation, the previous snapshot is re-served instead of re-flattening the fleet. A
/// full evaluation is still forced every
/// <see cref="DeploymentManagerOptions.PendingUpdatesFullEvaluationEvery"/>th time, the
/// safety net for what the two stamps cannot see (a KEK re-wrap that keeps its KEK id does
/// not bump a secret's revision). The short-circuit is off — every evaluation is full —
/// when the watermark is unavailable (non-SQL-Server provider), when the secret store
/// cannot be listed, and after an evaluation that could not compute a site's artifact hash
/// (a secret-store outage); it only ever re-serves a real earlier answer.
/// </para>
/// </summary>
public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
{
@@ -37,6 +59,10 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
private readonly ArtifactDeploymentService _artifactDeploymentService;
private readonly DeploymentService _deploymentService;
private readonly ILogger<PendingUpdatesEvaluator> _logger;
private readonly IConfigGraphWatermarkReader? _watermarkReader;
private readonly ISecretStore? _secretStore;
private readonly PendingUpdatesShortCircuitState? _shortCircuit;
private readonly int _fullEvaluationEvery;
/// <summary>
/// Initializes a new evaluator.
@@ -55,12 +81,43 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
ArtifactDeploymentService artifactDeploymentService,
DeploymentService deploymentService,
ILogger<PendingUpdatesEvaluator> logger)
: this(siteRepository, deploymentRepository, artifactDeploymentService, deploymentService, logger,
watermarkReader: null, secretStore: null, shortCircuit: null, options: null)
{
}
/// <summary>
/// Initializes a new evaluator that may short-circuit an unchanged fleet (Gitea #62).
/// </summary>
/// <param name="siteRepository">Repository the site list is read from.</param>
/// <param name="deploymentRepository">Repository the slim deployed-instance list is read from.</param>
/// <param name="artifactDeploymentService">Computes each site's current artifact revision hash.</param>
/// <param name="deploymentService">Evaluates instance staleness in one batched flatten session.</param>
/// <param name="logger">Logger instance.</param>
/// <param name="watermarkReader">Database-side change token; null disables the short-circuit.</param>
/// <param name="secretStore">The central secret store feeding the artifact hash; null disables the short-circuit.</param>
/// <param name="shortCircuit">The process-wide baseline; null disables the short-circuit.</param>
/// <param name="options">Supplies <see cref="DeploymentManagerOptions.PendingUpdatesFullEvaluationEvery"/>.</param>
public PendingUpdatesEvaluator(
ISiteRepository siteRepository,
IDeploymentManagerRepository deploymentRepository,
ArtifactDeploymentService artifactDeploymentService,
DeploymentService deploymentService,
ILogger<PendingUpdatesEvaluator> logger,
IConfigGraphWatermarkReader? watermarkReader,
ISecretStore? secretStore,
PendingUpdatesShortCircuitState? shortCircuit,
IOptions<DeploymentManagerOptions>? options)
{
_siteRepository = siteRepository;
_deploymentRepository = deploymentRepository;
_artifactDeploymentService = artifactDeploymentService;
_deploymentService = deploymentService;
_logger = logger;
_watermarkReader = watermarkReader;
_secretStore = secretStore;
_shortCircuit = shortCircuit;
_fullEvaluationEvery = Math.Max(1, options?.Value.PendingUpdatesFullEvaluationEvery ?? 1);
}
/// <inheritdoc />
@@ -68,6 +125,17 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
{
var stopwatch = Stopwatch.StartNew();
// Read BEFORE the fleet pass: a change landing mid-pass then makes the next token differ,
// so the worst case is one extra full evaluation, never a missed change.
var token = await ReadChangeTokenAsync(cancellationToken);
if (token is not null && _shortCircuit!.TryReuse(token, _fullEvaluationEvery, out var unchanged))
{
_logger.LogDebug(
"Pending-updates snapshot reused: config watermark and secret-store stamp unchanged");
// Re-verified as current just now, so it carries this check's time.
return unchanged with { ComputedAt = DateTimeOffset.UtcNow };
}
var sites = await _siteRepository.GetAllSitesAsync(cancellationToken);
// Slim, untracked projection: identity, site and state are all this pass reads.
var instances = await _deploymentRepository.GetDeployedInstanceSummariesAsync(cancellationToken);
@@ -105,6 +173,14 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
var snapshot = new PendingUpdatesSnapshot(DateTimeOffset.UtcNow, rows);
// Only a CLEAN evaluation becomes the baseline. A site missing from the hash map means its
// secret lookup failed (it reads Out of date); re-serving that would pin a transient outage.
if (_shortCircuit is not null)
{
var clean = sites.All(s => currentArtifactHashes.ContainsKey(s.Id));
_shortCircuit.RecordFull(clean ? token : null, snapshot);
}
// One line per evaluation — this runs at most once per cache TTL, so it is
// the cheapest record of what the fleet looked like and what the sweep cost.
_logger.LogInformation(
@@ -120,6 +196,59 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
return snapshot;
}
/// <summary>
/// The combined change token: the database watermark and the secret-store stamp. Null —
/// meaning "evaluate in full" — when the short-circuit is not wired, the watermark is
/// unavailable, or either read fails. A failed read is logged, never thrown: the full
/// evaluation that follows reports any real outage through its own path.
/// </summary>
private async Task<string?> ReadChangeTokenAsync(CancellationToken cancellationToken)
{
if (_watermarkReader is null || _secretStore is null || _shortCircuit is null || _fullEvaluationEvery <= 1)
return null;
try
{
var watermark = await _watermarkReader.ReadAsync(cancellationToken);
if (watermark is null)
return null;
var secrets = await _secretStore.ListAsync(includeDeleted: true, cancellationToken);
return watermark + "|" + StampSecrets(secrets);
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
_logger.LogWarning(ex, "Pending-updates change token unavailable; evaluating in full");
return null;
}
}
/// <summary>
/// SHA-256 over the secret store's metadata rows, name-sorted. Covers everything about a
/// secret that reaches the artifact hash except the wrap envelope bytes, which a same-KEK
/// re-wrap can change without moving any of these fields (the periodic full evaluation
/// covers that).
/// </summary>
/// <param name="secrets">The store's metadata, tombstones included.</param>
/// <returns>The hex stamp.</returns>
internal static string StampSecrets(IReadOnlyList<SecretMetadata> secrets)
{
var text = new StringBuilder();
foreach (var s in secrets.OrderBy(s => s.Name.Value, StringComparer.Ordinal))
{
text.Append(s.Name.Value).Append('\u001f')
.Append(s.Revision.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
.Append(s.UpdatedUtc.UtcTicks.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
.Append(s.CreatedUtc.UtcTicks.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
.Append(s.IsDeleted ? '1' : '0').Append('\u001f')
.Append(s.KekId).Append('\u001f')
.Append(s.ContentType.ToString()).Append('\u001f')
.Append(s.Description).Append('\u001e');
}
return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(text.ToString())));
}
/// <summary>
/// Decides a site's artifact state by comparing the hash stamped by its last
/// successful artifact deploy against the hash the current definitions would
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/PendingUpdates/PendingUpdatesShortCircuitState.cs
+81
View File
@@ -0,0 +1,81 @@
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
/// <summary>
/// Gitea #62: the process-wide baseline the (scoped) <see cref="PendingUpdatesEvaluator"/>
/// short-circuits against — the change token read just BEFORE the last clean full
/// evaluation, and the snapshot that evaluation produced.
///
/// <para>
/// <b>The token is never a per-process signal.</b> It is the database-side config watermark
/// plus the secret-store stamp, so an edit made through the peer central node moves it
/// exactly as a local one does (arch-review 2026-09-09 MG-3 removed a shortcut on the
/// per-process template-graph watermark for that reason). The only thing this object holds
/// per process is the last answer, and that answer is only ever re-served — never a
/// manufactured "nothing pending".
/// </para>
/// </summary>
public sealed class PendingUpdatesShortCircuitState
{
private readonly object _lock = new();
private string? _token;
private PendingUpdatesSnapshot? _snapshot;
private int _reusesSinceFull;
/// <summary>
/// Returns the baseline snapshot when <paramref name="token"/> equals the baseline token
/// and fewer than <paramref name="fullEvaluationEvery"/> − 1 reuses have happened since
/// the last full evaluation; counts the reuse.
/// </summary>
/// <param name="token">The change token read for this evaluation.</param>
/// <param name="fullEvaluationEvery">Every Nth evaluation must be full.</param>
/// <param name="snapshot">The re-servable snapshot, when this returns <c>true</c>.</param>
/// <returns><c>true</c> when the full evaluation may be skipped.</returns>
public bool TryReuse(string token, int fullEvaluationEvery, out PendingUpdatesSnapshot snapshot)
{
ArgumentNullException.ThrowIfNull(token);
lock (_lock)
{
if (_snapshot is not null
&& _token is not null
&& string.Equals(_token, token, StringComparison.Ordinal)
&& _reusesSinceFull < fullEvaluationEvery - 1)
{
_reusesSinceFull++;
snapshot = _snapshot;
return true;
}
snapshot = null!;
return false;
}
}
/// <summary>
/// Records a full evaluation. A <see langword="null"/> <paramref name="token"/> (no
/// watermark available, or the evaluation was not clean) clears the baseline, so the next
/// evaluation is full too.
/// </summary>
/// <param name="token">The change token read before the evaluation, or null.</param>
/// <param name="snapshot">The snapshot the evaluation produced.</param>
public void RecordFull(string? token, PendingUpdatesSnapshot snapshot)
{
ArgumentNullException.ThrowIfNull(snapshot);
lock (_lock)
{
_token = token;
_snapshot = token is null ? null : snapshot;
_reusesSinceFull = 0;
}
}
/// <summary>Drops the baseline so the next evaluation is full (deploy completion, Re-check).</summary>
public void Reset()
{
lock (_lock)
{
_token = null;
_snapshot = null;
_reusesSinceFull = 0;
}
}
}
src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ServiceCollectionExtensions.cs
+19 -1
View File
@@ -1,10 +1,13 @@
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Transport;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
using ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager;
@@ -92,7 +95,22 @@ public static class ServiceCollectionExtensions
// it reads through the same per-request DbContext as the services it composes;
// the singleton single-flight cache below runs it in its own DI scope so the
// fleet is flattened at most once per cache TTL.
services.AddScoped<IPendingUpdatesEvaluator, PendingUpdatesEvaluator>();
// Gitea #62: the evaluator short-circuits an unchanged fleet against a DATABASE-side
// rowversion watermark + a secret-store stamp; the baseline it compares against is
// process-wide, so it lives in a singleton. The watermark reader and the secret store
// are optional (a host without the configuration database or a secret store simply
// always evaluates in full).
services.AddSingleton<PendingUpdatesShortCircuitState>();
services.AddScoped<IPendingUpdatesEvaluator>(sp => new PendingUpdatesEvaluator(
sp.GetRequiredService<ISiteRepository>(),
sp.GetRequiredService<IDeploymentManagerRepository>(),
sp.GetRequiredService<ArtifactDeploymentService>(),
sp.GetRequiredService<DeploymentService>(),
sp.GetRequiredService<ILogger<PendingUpdatesEvaluator>>(),
sp.GetService<IConfigGraphWatermarkReader>(),
sp.GetService<ISecretStore>(),
sp.GetRequiredService<PendingUpdatesShortCircuitState>(),
sp.GetService<IOptions<DeploymentManagerOptions>>()));
// Process-wide memo over the evaluator, shared by the Central UI (rail indicator,
// Sites page, Health tiles) and the management API's GetPendingUpdatesCommand (CLI