feat(pending-updates): short-circuit an unchanged fleet on a DB rowversion watermark (#62)
Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every deployed instance on each cache-TTL expiry even when nothing had changed. - Migration AddConfigGraphRowVersions: a non-concurrency rowversion column + index on the 18 configuration tables the evaluation reads (Templates + 5 member tables, Instances + 4 override/binding tables, DataConnections, DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions, ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in the model; SQLite test contexts are untouched. Script: docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated central-pair migration window (#107 item 4). - IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which raise no rowversion). rowversion is engine-bumped for any writer, so a peer central node's edit moves it — not the per-process watermark MG-3 removed. - The secret store (outside the DB) gets its own stamp: SHA-256 over ListAsync(includeDeleted: true) metadata. - Evaluator re-serves the last CLEAN snapshot while watermark + secret stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default 10, validated >= 1, 1 = off) is full regardless — the safety net for a same-KEK re-wrap or an input table missing from the set. Always full when the token is unavailable/unreadable, after an evaluation that could not compute a site's artifact hash, on Re-check, and after a deploy completion. It only re-serves a real earlier answer, never manufactures one. Tests: SQL Server (real migrated DB) — token stable without writes, moves for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local EF write, a detached Update still saves (not a concurrency token), and a guard that every table the evaluator's repository reads touch is covered. Evaluator — reuse without flatten (re-served answer still shows the stale instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes, unclean evaluation not reused, cache Re-check and deploy completion force full. Mutation-proven: token compare, clean check, cache force flag, row count, and an uncovered read each turn tests red. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
@@ -72,8 +72,26 @@ Three Central UI changes, requested together:
|
||||
(default 60 s, validated > 0), running the evaluator in its own DI scope. `GetAsync(forceRefresh)`.
|
||||
Consequence: however many circuits poll, the fleet is flattened at most once per TTL, and not at
|
||||
all when nobody is looking.
|
||||
- Not in scope (register rows): a template-graph-watermark short-circuit so an unchanged fleet costs
|
||||
no re-flatten; a `deploy` CLI/`Health` KPI surface for the counts.
|
||||
- Not in scope (register rows): a `deploy` CLI/`Health` KPI surface for the counts.
|
||||
- **Unchanged-fleet short-circuit (Gitea #62, register row 124, owner decision (b) 2026-11-09).** Before
|
||||
the fleet pass the evaluator reads a change token = `IConfigGraphWatermarkReader` (ONE query:
|
||||
`MAX(RowVersion)` + `COUNT_BIG(*)` per table over the 18 input tables — Templates + their five member
|
||||
tables, Instances + their four override/binding tables, DataConnections, DeployedConfigSnapshots, Sites,
|
||||
SharedScripts, ExternalSystemDefinitions, ExternalSystemMethods, DatabaseConnectionDefinitions; migration
|
||||
`AddConfigGraphRowVersions`, list in `ConfigGraphRowVersions`) **plus** a SHA-256 stamp over the secret
|
||||
store's `ListAsync(includeDeleted: true)` metadata (name, revision, created/updated, tombstone, KEK id,
|
||||
content type, description). Equal to the token read before the last *clean* full evaluation → the
|
||||
previous snapshot is re-served (with a fresh `ComputedAt`). rowversion is database-wide and engine-bumped,
|
||||
so a peer central node's edit, a bundle import or an `ExecuteUpdate` all move it; the count catches a
|
||||
delete (which raises no rowversion). It is deliberately not the per-process template-graph watermark
|
||||
(MG-3). Safety net: every `PendingUpdatesFullEvaluationEvery`th evaluation (default 10 ≈ 10 min at the
|
||||
60 s TTL; `1` disables the short-circuit) is full regardless — it covers a same-KEK re-wrap (wrap
|
||||
envelope changes, revision does not) and any input table someone forgets to add (a SQL Server guard
|
||||
test fails if a repository read used by the evaluator touches an uncovered table). Always full: watermark
|
||||
unavailable (non-SQL-Server provider) or unreadable, secret listing fails, the previous evaluation could
|
||||
not compute a site's artifact hash (secret-store outage — re-serving it would pin a transient *Out of
|
||||
date*), a *Re-check* (`forceRefresh`), and the evaluation after a deploy completion. It never
|
||||
manufactures an answer — it only re-serves a real earlier one.
|
||||
|
||||
### 3.3 Sites page (Admin)
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
-- AddConfigGraphRowVersions (2026-11-10, ScadaBridge#62). A rowversion column + index on the 18 configuration
|
||||
-- tables feeding the pending-updates evaluation, so PendingUpdatesEvaluator can skip the fleet flatten when
|
||||
-- MAX(RowVersion)+COUNT_BIG(*) over them is unchanged. Additive: SQL Server fills every existing row as the
|
||||
-- column is added; old binaries ignore the column. Apply after AddAuditLogHashChain in the coordinated
|
||||
-- central-pair migration window (ScadaBridge#107 item 4).
|
||||
BEGIN TRANSACTION;
|
||||
ALTER TABLE [TemplateScripts] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Templates] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateNativeAlarmSources] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateCompositions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateAttributes] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateAlarms] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Sites] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [SharedScripts] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Instances] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceNativeAlarmSourceOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceConnectionBindings] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceAttributeOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceAlarmOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [ExternalSystemMethods] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [ExternalSystemDefinitions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DeployedConfigSnapshots] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DataConnections] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DatabaseConnectionDefinitions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
CREATE INDEX [IX_TemplateScripts_RowVersion] ON [TemplateScripts] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Templates_RowVersion] ON [Templates] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateNativeAlarmSources_RowVersion] ON [TemplateNativeAlarmSources] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateCompositions_RowVersion] ON [TemplateCompositions] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateAttributes_RowVersion] ON [TemplateAttributes] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateAlarms_RowVersion] ON [TemplateAlarms] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Sites_RowVersion] ON [Sites] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_SharedScripts_RowVersion] ON [SharedScripts] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Instances_RowVersion] ON [Instances] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceNativeAlarmSourceOverrides_RowVersion] ON [InstanceNativeAlarmSourceOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceConnectionBindings_RowVersion] ON [InstanceConnectionBindings] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceAttributeOverrides_RowVersion] ON [InstanceAttributeOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceAlarmOverrides_RowVersion] ON [InstanceAlarmOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_ExternalSystemMethods_RowVersion] ON [ExternalSystemMethods] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_ExternalSystemDefinitions_RowVersion] ON [ExternalSystemDefinitions] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DeployedConfigSnapshots_RowVersion] ON [DeployedConfigSnapshots] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DataConnections_RowVersion] ON [DataConnections] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DatabaseConnectionDefinitions_RowVersion] ON [DatabaseConnectionDefinitions] ([RowVersion]);
|
||||
|
||||
INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion])
|
||||
VALUES (N'20261110032839_AddConfigGraphRowVersions', N'10.0.13');
|
||||
|
||||
COMMIT;
|
||||
GO
|
||||
|
||||
@@ -569,9 +569,23 @@ at once; one run through the peer central clears within the TTL. See
|
||||
the Sites-page badge/re-check UI and the rail-footer indicator
|
||||
(Component-CentralUI.md → *Site & Data Connection Management* and *Technology*).
|
||||
|
||||
Not in scope for this slice (deferred-work register rows 124–125): a
|
||||
template-graph-watermark short-circuit so an unchanged fleet costs no re-flatten
|
||||
per TTL, and a `deploy` CLI / Health-dashboard KPI surface for the counts.
|
||||
**Unchanged-fleet short-circuit (Gitea #62, register row 124).** Inside a flight the
|
||||
evaluator first reads a change token — `IConfigGraphWatermarkReader` (one query of
|
||||
`MAX(RowVersion)` + `COUNT_BIG(*)` over the 18 tables the evaluation reads, the
|
||||
`rowversion` columns added by migration `AddConfigGraphRowVersions`; the covered set is
|
||||
`ConfigGraphRowVersions.EntityTypes`) plus a SHA-256 stamp over the secret store's
|
||||
metadata listing — and re-serves the last clean snapshot when it is unchanged.
|
||||
`rowversion` is bumped by the engine for any writer, so an edit through the peer central
|
||||
node is detected (the per-process template-graph watermark would not be — arch-review
|
||||
2026-09-09 MG-3). Every `ScadaBridge:DeploymentManager:PendingUpdatesFullEvaluationEvery`th
|
||||
evaluation (default 10, validated ≥ 1; 1 disables the short-circuit) is full regardless,
|
||||
the safety net for a same-KEK secret re-wrap and for an input table missing from the set.
|
||||
A *Re-check* (`forceRefresh`), a deploy completion, an unavailable watermark or secret
|
||||
listing, and an evaluation that could not compute a site's artifact hash all force a full
|
||||
evaluation. Only a real earlier answer is ever re-served.
|
||||
|
||||
Not in scope for this slice (deferred-work register row 125): a `deploy` CLI /
|
||||
Health-dashboard KPI surface for the counts.
|
||||
|
||||
## Instance Lifecycle Commands
|
||||
|
||||
|
||||
Reference in New Issue
Block a user