feat(pending-updates): short-circuit an unchanged fleet on a DB rowversion watermark (#62)
Owner decision (b), 2026-11-09. PendingUpdatesEvaluator re-flattened every deployed instance on each cache-TTL expiry even when nothing had changed. - Migration AddConfigGraphRowVersions: a non-concurrency rowversion column + index on the 18 configuration tables the evaluation reads (Templates + 5 member tables, Instances + 4 override/binding tables, DataConnections, DeployedConfigSnapshots, Sites, SharedScripts, ExternalSystemDefinitions, ExternalSystemMethods, DatabaseConnectionDefinitions). SQL Server only in the model; SQLite test contexts are untouched. Script: docs/plans/sql/AddConfigGraphRowVersions.sql. Joins the coordinated central-pair migration window (#107 item 4). - IConfigGraphWatermarkReader / ConfigGraphWatermarkReader: ONE query of MAX(RowVersion) + COUNT_BIG(*) per table (count catches deletes, which raise no rowversion). rowversion is engine-bumped for any writer, so a peer central node's edit moves it — not the per-process watermark MG-3 removed. - The secret store (outside the DB) gets its own stamp: SHA-256 over ListAsync(includeDeleted: true) metadata. - Evaluator re-serves the last CLEAN snapshot while watermark + secret stamp are unchanged; every PendingUpdatesFullEvaluationEvery'th (default 10, validated >= 1, 1 = off) is full regardless — the safety net for a same-KEK re-wrap or an input table missing from the set. Always full when the token is unavailable/unreadable, after an evaluation that could not compute a site's artifact hash, on Re-check, and after a deploy completion. It only re-serves a real earlier answer, never manufactures one. Tests: SQL Server (real migrated DB) — token stable without writes, moves for a peer-connection INSERT/UPDATE and a non-max DELETE, moves for a local EF write, a detached Update still saves (not a concurrency token), and a guard that every table the evaluator's repository reads touch is covered. Evaluator — reuse without flatten (re-served answer still shows the stale instance), watermark move, 4 secret changes, every-Nth, 5 no-token modes, unclean evaluation not reused, cache Re-check and deploy completion force full. Mutation-proven: token compare, clean check, cache force flag, row count, and an uncovered read each turn tests red. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
@@ -141,7 +141,7 @@ spec for each is `docs/requirements/Component-<Name>.md`, and `README.md` carrie
|
||||
- Per-instance operation lock covers all mutating commands (deploy, disable, enable, delete).
|
||||
- **Bulk site deployment (`DeploymentService.DeploySiteAsync`, arch-review WP2.5)** redeploys every out-of-date instance at a site through one flatten session (a template chain shared by N instances is walked once) in three phases — serial Prepare, bounded-parallel Send (`SiteDeploymentMaxParallelism`, default 4), serial Finalize — still **not all-or-nothing**: any one instance's failure doesn't block the rest. **Disabled instances are SKIPPED by bulk deploy as of 2026-08-19 (arch-review P7-2)** — they are reported in the summary's additive `SkippedInstances` list, never flattened/sent/re-enabled (single-instance deploy remains the explicit operator path that deploys+enables a Disabled instance); **Up-to-date instances are ALSO skipped as of 2026-08-20 (arch-review P7-6)** — a redeploy is not idempotent at the site (it resets static overrides and clears native alarm state), so `DeploySiteAsync` pre-filters through `IStaleInstanceProbe` (trustworthy since P7-4 fixed its memo timing) and reports unchanged instances in `SkippedInstances` with reason `up-to-date`; the filter is conservative (every uncertainty deploys: missing snapshot, `ShouldQuerySiteBeforeRedeploy`, null/faulting probe). Additive `MgmtDeploySiteCommand.Force` / CLI `deploy site --force` restores the unconditional push for what central cannot see (restored volume, rebuilt node) — **Disabled still wins under `--force`**. Also P7-5: the three phases share one scoped DbContext and a failed `SaveChanges` leaves the entity tracked, so one permanent failure used to cascade to every later instance and strand records `InProgress`; `MarkDeploymentFailedAsync` now detaches pending changes and retries the `Failed` write once on the clean tracker; Prepare's per-instance reconcile also runs under a short `SiteDeploymentReconcileTimeout` (5s) with a once-per-batch unreachable-site circuit-breaker, so a down site no longer costs 30s × N serial before the first send (P7-3). The `PendingDeployment` staging row (5-minute TTL) is written per-instance immediately before that instance's round-trip in the Send phase, not upfront in Prepare, so a large batch's tail instances don't have their TTL clock already half-spent by the time they're reached.
|
||||
- **Area-subtree bulk deploy + Topology first-layer preload (Gitea #38/#39/#40, 2026-10-22).** `DeploymentService.DeployAreaAsync(areaId, user, force)` / `MgmtDeployArea` / CLI `deploy area --id <areaId> [--force]` / the Topology **Deploy all…** context item (site and area nodes) run the same `DeploySiteAsync` core narrowed to the area plus every descendant area (one cycle-safe in-memory `ParentAreaId` walk shared with the confirm-count `PreviewSubtreeDeployAsync`), with identical Disabled/up-to-date/`force` rules and summary; authorised by the area's owning site (fail-closed on an unknown area for scoped callers); area-less instances are reachable only by a site deploy. `DeploymentService` now depends on `ITemplateEngineRepository`. The page runs Deploy all/Deploy in their **own DI scope** and pauses the live tick while an action runs — a minutes-long deploy on the circuit-scoped DbContext collides with the 15 s refresh. Topology now pre-loads every permitted site's areas and opens sites expanded when no saved tree state exists (instances stay lazy). #38: the inline area rename bound on `onchange`, so Enter always committed the old name — it binds `oninput` now; never rely on `onchange` for an Enter-to-commit input.
|
||||
- **Site artifact revision + fleet-wide pending-updates indicator (2026-09-14, `docs/plans/2026-09-14-pending-updates-design.md`).** `Site.ArtifactRevisionHash`/`ArtifactsDeployedAt` (migration `AddSiteArtifactRevision`) are stamped by `ArtifactDeploymentService` only on a site whose artifact deploy succeeds, so a redeployed fleet's artifacts are trackable per site the same way instance staleness already is. `PendingUpdatesEvaluator` combines that with `DeploymentService.GetInstanceStalenessAsync` (one shared `FlattenSession` for the whole fleet) into a `PendingUpdatesSnapshot`, held behind a process-wide singleton `IPendingUpdatesCache` (`ScadaBridge:DeploymentManager:PendingUpdatesCacheTtl`, default 60s) so N Blazor circuits cost one fleet flatten per TTL, not one each. Surfaces: the Sites page's per-site **Artifacts** badge (`Current`/`Out of date`/`Never deployed`) + instance-staleness line + header **Re-check**, and a rail-footer **pending updates** indicator (Deployer/Administrator only, 30s poll, site-scoped). **Upgrade consequence:** every site reads *Never deployed* until one artifact deploy runs post-upgrade — deliberate, not a bug. Also shipped: the rail nav now sits in its own scroll pane (`NavMenu`'s `.rail-scroll` + `site.css`) so a long nav can't push the footer/indicator off-screen; the `ZB.MOM.WW.Theme` kit itself is untouched (register row 123 proposes upstreaming it).
|
||||
- **Site artifact revision + fleet-wide pending-updates indicator (2026-09-14, `docs/plans/2026-09-14-pending-updates-design.md`).** `Site.ArtifactRevisionHash`/`ArtifactsDeployedAt` (migration `AddSiteArtifactRevision`) are stamped by `ArtifactDeploymentService` only on a site whose artifact deploy succeeds, so a redeployed fleet's artifacts are trackable per site the same way instance staleness already is. `PendingUpdatesEvaluator` combines that with `DeploymentService.GetInstanceStalenessAsync` (one shared `FlattenSession` for the whole fleet) into a `PendingUpdatesSnapshot`, held behind a process-wide singleton `IPendingUpdatesCache` (`ScadaBridge:DeploymentManager:PendingUpdatesCacheTtl`, default 60s) so N Blazor circuits cost one fleet flatten per TTL, not one each — **and an unchanged fleet costs none (Gitea #62):** the evaluator re-serves its last clean snapshot while a DB-side token (`MAX(RowVersion)`+`COUNT_BIG(*)` over the 18 input tables, migration `AddConfigGraphRowVersions`, list `ConfigGraphRowVersions` — add any new input table there) plus a secret-store metadata stamp is unchanged, with a forced full evaluation every `PendingUpdatesFullEvaluationEvery` (10) as the safety net; Re-check and deploy completion force a full one. Surfaces: the Sites page's per-site **Artifacts** badge (`Current`/`Out of date`/`Never deployed`) + instance-staleness line + header **Re-check**, and a rail-footer **pending updates** indicator (Deployer/Administrator only, 30s poll, site-scoped). **Upgrade consequence:** every site reads *Never deployed* until one artifact deploy runs post-upgrade — deliberate, not a bug. Also shipped: the rail nav now sits in its own scroll pane (`NavMenu`'s `.rail-scroll` + `site.css`) so a long nav can't push the footer/indicator off-screen; the `ZB.MOM.WW.Theme` kit itself is untouched (register row 123 proposes upstreaming it).
|
||||
- **Site secrets ride the artifact deploy; the gRPC secrets hub is retired in ScadaBridge (2026-09-28, `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md`).** `Deploy Artifacts` now ships each site **only the secrets it references** — `SiteSecretsCollector` scans a site's data-connection JSON for `secret:<name>` (`SecretReferenceScanner`, protocol-agnostic, bootstrap names like `SB-GRPC-PSK-*` excluded) and resolves each against the central `ISecretStore`; a missing/tombstoned secret fails **that site's** deploy before anything is sent (fail-closed), ciphertext only ever crosses the wire, and secrets factor into `ArtifactRevisionHashService`'s staleness hash (a missing secret hashes as a distinguishable marker, so an unresolvable reference never reads *Current*; a site with no secret references hashes identically whether the field is null or empty). A pre-feature site (answers with no `SecretsApplied`) or a partial apply is also treated as a deploy failure and is never stamped; one `DeploySiteSecrets`/`SiteSecrets` audit row (`secrets` name + revision, `missing` and `removed` names, never ciphertext — `removed` comes from the site's `SecretsRemoved` reply, so a removal-only deploy is audited too) is written per site deploy that carried, lacked or retired secrets, on both the sweep and the per-site retry paths. A central secret-store read failure fails only the sites that reference a secret and leaves them *Out of date* in the pending-updates check — it no longer aborts the sweep or the fleet-wide check — and both artifact-deploy CLI commands exit 1 when any site failed. Site-side: `deployed_secrets` is the 11th LocalDb-replicated table (ciphertext rows only, upsert-then-soft-remove); the per-node `DeployedSecretsMirror` (`SiteRuntime:DeployedSecretsMirrorInterval`, default 15s, also run synchronously right after an active-node apply) reconciles it into that node's local `ISecretStore`, ledgers what it mirrored in the node-local `mirrored_secrets` table so an out-of-band local secret sharing a deployed name is never silently touched (a mirror-owned name deleted or edited locally is healed back to the deployed value on the next pass, even at an unchanged revision), and trial-decrypts every pass to catch a KEK mismatch. Each node also records its current mirror failures (names + messages only) in the 12th replicated table `secret_mirror_status` — written when its failure set changes, once per start-up and every `SecretMirrorStatusRefreshInterval` (10 min) as a heartbeat; rows not refreshed for `SecretMirrorStatusStaleAfter` (1 h: a decommissioned/renamed/down node) are dropped from the report and deleted — and the active node forwards every node's row as `SiteHealthReport.SecretMirrorStatuses` (proto field 33), so a **standby** KEK mismatch shows as a *Secret mirror failing* badge on the Health dashboard instead of only in the standby's log (review finding M1). Secrets are mirrored before the deploy's data connections reach the DCL; a rotation with unchanged connection config takes effect on the next (re)connect. **A site pair using deployed secrets must run LocalDb replication** — otherwise only the node active at deploy time holds them. Central runs one shared secret store, `Secrets:Store:Provider=SqlServer` (Central role only; a clustered central left on `Sqlite` gets a boot Warning) — the pre-host `${secret:}` expander reads the same store the running node serves. **ALL secrets replication is retired in ScadaBridge** — the pull-only gRPC hub and the SQL-Server replication mode both — `Secrets:Replication:Enabled` set to anything but empty/`false` now fails `StartupValidator` on either role with a message pointing at Deploy Artifacts / `Secrets:Store:Provider=SqlServer`.
|
||||
- **Create Instance picks its template from the Templates folder tree** (2026-08-21): the flat `<select>` is gone. The Template field is a read-only folder-qualified path box (`TemplatePathFormatter`, "Folder / Sub / Name") + a **Choose…** button, both opening `TemplatePickerDialog` — an `IDialogService.ShowAsync` body wrapping the **same shared `TemplateFolderTree`** the Templates page renders, with its own search, expand/collapse-all, templates-only selection and double-click-to-pick. It opens on the current pick via the tree's additive **`RevealKeys`** parameter (ancestors of the named `f:`/`t:` keys start expanded when no `Filter` is active; an active filter's reveal set still wins; callers passing nothing are byte-identical). `InstanceCreate` gained `[SupplyParameterFromQuery] TemplateId`, which the Templates tree's new **Deployer-gated "New Instance…"** context-menu item navigates with; combined with `?siteId=` it populates the Connection Bindings card on first render. **The trap for anyone extending the picker:** in `TreeViewSelectionMode.Single` the inner `TreeView` reads the *singular* `SelectedKey`/`SelectedKeyChanged` pair, but `TemplateFolderTree` forwards only the plural (Checkbox-mode) `SelectedKeys` pair — so a Single-mode caller cannot get a row highlight through the shared tree and must own selection in its own `NodeContent`, as the picker does. Design: `docs/plans/2026-08-21-template-picker-design.md`.
|
||||
- **Create Instance binds module groups at creation** (2026-08-21): the create page's Connection Bindings card flattens template+site through the additive `IFlatteningPipeline.PreviewAsync` (an unsaved instance — no validation, no hash, nothing persisted, instance repo never touched) and reuses `BindingGroupBuilder`; `NodeBrowserDialog.SelectableNodeClasses` lets it pick **Object** nodes (default stays Variable-only for every existing caller). Create = `CreateInstanceAsync` → `SetConnectionBindingsAsync` (the existing upsert, no new server surface) → always navigate to Configure with `?created=1&bound=N|&bindError=…`, because the row exists whether or not the second call succeeded. Partial binding is legal: a group missing a connection or an Object is skipped and stays unbound. Design: `docs/plans/2026-08-21-create-instance-bindings-design.md`.
|
||||
|
||||
@@ -72,8 +72,26 @@ Three Central UI changes, requested together:
|
||||
(default 60 s, validated > 0), running the evaluator in its own DI scope. `GetAsync(forceRefresh)`.
|
||||
Consequence: however many circuits poll, the fleet is flattened at most once per TTL, and not at
|
||||
all when nobody is looking.
|
||||
- Not in scope (register rows): a template-graph-watermark short-circuit so an unchanged fleet costs
|
||||
no re-flatten; a `deploy` CLI/`Health` KPI surface for the counts.
|
||||
- Not in scope (register rows): a `deploy` CLI/`Health` KPI surface for the counts.
|
||||
- **Unchanged-fleet short-circuit (Gitea #62, register row 124, owner decision (b) 2026-11-09).** Before
|
||||
the fleet pass the evaluator reads a change token = `IConfigGraphWatermarkReader` (ONE query:
|
||||
`MAX(RowVersion)` + `COUNT_BIG(*)` per table over the 18 input tables — Templates + their five member
|
||||
tables, Instances + their four override/binding tables, DataConnections, DeployedConfigSnapshots, Sites,
|
||||
SharedScripts, ExternalSystemDefinitions, ExternalSystemMethods, DatabaseConnectionDefinitions; migration
|
||||
`AddConfigGraphRowVersions`, list in `ConfigGraphRowVersions`) **plus** a SHA-256 stamp over the secret
|
||||
store's `ListAsync(includeDeleted: true)` metadata (name, revision, created/updated, tombstone, KEK id,
|
||||
content type, description). Equal to the token read before the last *clean* full evaluation → the
|
||||
previous snapshot is re-served (with a fresh `ComputedAt`). rowversion is database-wide and engine-bumped,
|
||||
so a peer central node's edit, a bundle import or an `ExecuteUpdate` all move it; the count catches a
|
||||
delete (which raises no rowversion). It is deliberately not the per-process template-graph watermark
|
||||
(MG-3). Safety net: every `PendingUpdatesFullEvaluationEvery`th evaluation (default 10 ≈ 10 min at the
|
||||
60 s TTL; `1` disables the short-circuit) is full regardless — it covers a same-KEK re-wrap (wrap
|
||||
envelope changes, revision does not) and any input table someone forgets to add (a SQL Server guard
|
||||
test fails if a repository read used by the evaluator touches an uncovered table). Always full: watermark
|
||||
unavailable (non-SQL-Server provider) or unreadable, secret listing fails, the previous evaluation could
|
||||
not compute a site's artifact hash (secret-store outage — re-serving it would pin a transient *Out of
|
||||
date*), a *Re-check* (`forceRefresh`), and the evaluation after a deploy completion. It never
|
||||
manufactures an answer — it only re-serves a real earlier one.
|
||||
|
||||
### 3.3 Sites page (Admin)
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
-- AddConfigGraphRowVersions (2026-11-10, ScadaBridge#62). A rowversion column + index on the 18 configuration
|
||||
-- tables feeding the pending-updates evaluation, so PendingUpdatesEvaluator can skip the fleet flatten when
|
||||
-- MAX(RowVersion)+COUNT_BIG(*) over them is unchanged. Additive: SQL Server fills every existing row as the
|
||||
-- column is added; old binaries ignore the column. Apply after AddAuditLogHashChain in the coordinated
|
||||
-- central-pair migration window (ScadaBridge#107 item 4).
|
||||
BEGIN TRANSACTION;
|
||||
ALTER TABLE [TemplateScripts] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Templates] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateNativeAlarmSources] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateCompositions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateAttributes] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [TemplateAlarms] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Sites] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [SharedScripts] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [Instances] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceNativeAlarmSourceOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceConnectionBindings] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceAttributeOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [InstanceAlarmOverrides] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [ExternalSystemMethods] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [ExternalSystemDefinitions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DeployedConfigSnapshots] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DataConnections] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
ALTER TABLE [DatabaseConnectionDefinitions] ADD [RowVersion] rowversion NOT NULL;
|
||||
|
||||
CREATE INDEX [IX_TemplateScripts_RowVersion] ON [TemplateScripts] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Templates_RowVersion] ON [Templates] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateNativeAlarmSources_RowVersion] ON [TemplateNativeAlarmSources] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateCompositions_RowVersion] ON [TemplateCompositions] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateAttributes_RowVersion] ON [TemplateAttributes] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_TemplateAlarms_RowVersion] ON [TemplateAlarms] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Sites_RowVersion] ON [Sites] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_SharedScripts_RowVersion] ON [SharedScripts] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_Instances_RowVersion] ON [Instances] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceNativeAlarmSourceOverrides_RowVersion] ON [InstanceNativeAlarmSourceOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceConnectionBindings_RowVersion] ON [InstanceConnectionBindings] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceAttributeOverrides_RowVersion] ON [InstanceAttributeOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_InstanceAlarmOverrides_RowVersion] ON [InstanceAlarmOverrides] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_ExternalSystemMethods_RowVersion] ON [ExternalSystemMethods] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_ExternalSystemDefinitions_RowVersion] ON [ExternalSystemDefinitions] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DeployedConfigSnapshots_RowVersion] ON [DeployedConfigSnapshots] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DataConnections_RowVersion] ON [DataConnections] ([RowVersion]);
|
||||
|
||||
CREATE INDEX [IX_DatabaseConnectionDefinitions_RowVersion] ON [DatabaseConnectionDefinitions] ([RowVersion]);
|
||||
|
||||
INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion])
|
||||
VALUES (N'20261110032839_AddConfigGraphRowVersions', N'10.0.13');
|
||||
|
||||
COMMIT;
|
||||
GO
|
||||
|
||||
@@ -569,9 +569,23 @@ at once; one run through the peer central clears within the TTL. See
|
||||
the Sites-page badge/re-check UI and the rail-footer indicator
|
||||
(Component-CentralUI.md → *Site & Data Connection Management* and *Technology*).
|
||||
|
||||
Not in scope for this slice (deferred-work register rows 124–125): a
|
||||
template-graph-watermark short-circuit so an unchanged fleet costs no re-flatten
|
||||
per TTL, and a `deploy` CLI / Health-dashboard KPI surface for the counts.
|
||||
**Unchanged-fleet short-circuit (Gitea #62, register row 124).** Inside a flight the
|
||||
evaluator first reads a change token — `IConfigGraphWatermarkReader` (one query of
|
||||
`MAX(RowVersion)` + `COUNT_BIG(*)` over the 18 tables the evaluation reads, the
|
||||
`rowversion` columns added by migration `AddConfigGraphRowVersions`; the covered set is
|
||||
`ConfigGraphRowVersions.EntityTypes`) plus a SHA-256 stamp over the secret store's
|
||||
metadata listing — and re-serves the last clean snapshot when it is unchanged.
|
||||
`rowversion` is bumped by the engine for any writer, so an edit through the peer central
|
||||
node is detected (the per-process template-graph watermark would not be — arch-review
|
||||
2026-09-09 MG-3). Every `ScadaBridge:DeploymentManager:PendingUpdatesFullEvaluationEvery`th
|
||||
evaluation (default 10, validated ≥ 1; 1 disables the short-circuit) is full regardless,
|
||||
the safety net for a same-KEK secret re-wrap and for an input table missing from the set.
|
||||
A *Re-check* (`forceRefresh`), a deploy completion, an unavailable watermark or secret
|
||||
listing, and an evaluation that could not compute a site's artifact hash all force a full
|
||||
evaluation. Only a real earlier answer is ever re-served.
|
||||
|
||||
Not in scope for this slice (deferred-work register row 125): a `deploy` CLI /
|
||||
Health-dashboard KPI surface for the counts.
|
||||
|
||||
## Instance Lifecycle Commands
|
||||
|
||||
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
namespace ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
|
||||
/// <summary>
|
||||
/// Reads a cheap, DATABASE-side change token over every configuration-database table that
|
||||
/// feeds the pending-updates evaluation — instance staleness (the flatten inputs plus the
|
||||
/// deployed snapshots) and the per-site artifact revision hash (Gitea #62).
|
||||
///
|
||||
/// <para>
|
||||
/// The token is derived from SQL Server <c>rowversion</c> columns, so it moves on ANY
|
||||
/// insert or update of those tables — including a write made by the peer central node, a
|
||||
/// bundle import, a raw <c>ExecuteUpdate</c> or a hand-edited row — without any write path
|
||||
/// having to remember to bump it. Each table contributes <c>MAX(rowversion)</c> AND its row
|
||||
/// count, because a delete never raises a rowversion: a removed override or template member
|
||||
/// is caught by the count. This is deliberately NOT the per-process template-graph
|
||||
/// watermark: that one never sees a peer-node edit (arch-review 2026-09-09 MG-3).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public interface IConfigGraphWatermarkReader
|
||||
{
|
||||
/// <summary>
|
||||
/// Reads the current change token in one round trip.
|
||||
/// </summary>
|
||||
/// <param name="cancellationToken">Cancellation token.</param>
|
||||
/// <returns>
|
||||
/// An opaque token that is equal across two reads only when none of the covered tables
|
||||
/// changed in between; <see langword="null"/> when the provider cannot produce one (a
|
||||
/// non-SQL-Server test provider), in which case callers must not short-circuit.
|
||||
/// </returns>
|
||||
Task<string?> ReadAsync(CancellationToken cancellationToken = default);
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Deployment;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.ExternalSystems;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Scripts;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #62: the <c>rowversion</c> shadow columns behind
|
||||
/// <see cref="Commons.Interfaces.Repositories.IConfigGraphWatermarkReader"/>.
|
||||
///
|
||||
/// <para>
|
||||
/// <b>The covered set is exactly what <c>PendingUpdatesEvaluator</c> reads.</b> Instance
|
||||
/// staleness re-flattens each deployed instance (<c>FlatteningPipeline.FlattenCoreAsync</c>:
|
||||
/// the instance with its four override/binding collections, the template inheritance and
|
||||
/// composition graph with its five member collections, the site's data connections) and
|
||||
/// compares against the deployed snapshots; the artifact hash
|
||||
/// (<c>ArtifactDeploymentService.ComputeCurrentArtifactHashesAsync</c>) reads shared
|
||||
/// scripts, external systems + their methods, database connections and data connections,
|
||||
/// and compares against the hash stamped on each site row. The secret store, which also
|
||||
/// feeds the artifact hash, is outside this database and has its own stamp in the evaluator.
|
||||
/// A table added to either read path MUST be added here, or a change to it is only picked
|
||||
/// up by the evaluator's periodic forced full evaluation.
|
||||
/// </para>
|
||||
///
|
||||
/// <para>
|
||||
/// The columns are NOT concurrency tokens: they exist only to be compared as a maximum, and
|
||||
/// making them tokens would change the conflict behaviour of every save on these tables
|
||||
/// (a detached <c>Update</c> would carry a null original value and fail). They are SQL
|
||||
/// Server only — <see cref="ScadaBridgeDbContext"/> applies them only under that provider,
|
||||
/// so the SQLite test contexts never see a column SQLite cannot generate.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public static class ConfigGraphRowVersions
|
||||
{
|
||||
/// <summary>The shadow property / column name on every covered table.</summary>
|
||||
public const string ColumnName = "RowVersion";
|
||||
|
||||
/// <summary>The entity types whose tables are covered by the watermark.</summary>
|
||||
public static IReadOnlyList<Type> EntityTypes { get; } =
|
||||
[
|
||||
// Flatten inputs: the template graph.
|
||||
typeof(Template),
|
||||
typeof(TemplateAttribute),
|
||||
typeof(TemplateAlarm),
|
||||
typeof(TemplateScript),
|
||||
typeof(TemplateComposition),
|
||||
typeof(TemplateNativeAlarmSource),
|
||||
// Flatten inputs: the instance and its overrides/bindings.
|
||||
typeof(Instance),
|
||||
typeof(InstanceAttributeOverride),
|
||||
typeof(InstanceAlarmOverride),
|
||||
typeof(InstanceConnectionBinding),
|
||||
typeof(InstanceNativeAlarmSourceOverride),
|
||||
// Flatten input AND artifact input.
|
||||
typeof(DataConnection),
|
||||
// The deployed side of instance staleness.
|
||||
typeof(DeployedConfigSnapshot),
|
||||
// Artifact inputs, and the site row carrying the stamped artifact hash.
|
||||
typeof(Site),
|
||||
typeof(SharedScript),
|
||||
typeof(ExternalSystemDefinition),
|
||||
typeof(ExternalSystemMethod),
|
||||
typeof(DatabaseConnectionDefinition),
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// Adds the non-concurrency <c>rowversion</c> shadow column (and an index on it, so
|
||||
/// <c>MAX</c> is a seek and <c>COUNT_BIG</c> scans the narrowest index) to every covered table.
|
||||
/// </summary>
|
||||
/// <param name="modelBuilder">The model builder.</param>
|
||||
public static void Apply(ModelBuilder modelBuilder)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(modelBuilder);
|
||||
foreach (var type in EntityTypes)
|
||||
{
|
||||
var entity = modelBuilder.Entity(type);
|
||||
// Spelled out rather than IsRowVersion(): that also makes the column a concurrency
|
||||
// token, and switching the token off afterwards drops the provider's rowversion
|
||||
// type mapping (the column would scaffold as a nullable varbinary(900)).
|
||||
entity.Property<byte[]>(ColumnName)
|
||||
.HasColumnType("rowversion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.IsConcurrencyToken(false);
|
||||
entity.HasIndex(ColumnName);
|
||||
}
|
||||
}
|
||||
}
|
||||
+2381
File diff suppressed because it is too large
Load Diff
+393
@@ -0,0 +1,393 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
{
|
||||
/// <summary>
|
||||
/// Gitea #62: a <c>rowversion</c> column (+ index) on the 18 configuration tables that feed
|
||||
/// the pending-updates evaluation — the flatten inputs (Templates and their attribute /
|
||||
/// alarm / script / composition / native-alarm-source members, Instances and their four
|
||||
/// override/binding tables, DataConnections), the deployed side (DeployedConfigSnapshots)
|
||||
/// and the artifact-hash inputs (Sites, SharedScripts, ExternalSystemDefinitions,
|
||||
/// ExternalSystemMethods, DatabaseConnectionDefinitions). <c>PendingUpdatesEvaluator</c>
|
||||
/// short-circuits when MAX(RowVersion) + COUNT_BIG(*) over them is unchanged.
|
||||
/// <para>
|
||||
/// Purely additive: SQL Server fills a rowversion for every existing row as the column is
|
||||
/// added (a size-of-data operation on each table — these are configuration tables, small
|
||||
/// next to AuditLog). Old binaries ignore the column (it is never a concurrency token and
|
||||
/// is never written by the application). Ships in the coordinated central-pair migration
|
||||
/// window with the other pending migrations (ScadaBridge#107 item 4).
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public partial class AddConfigGraphRowVersions : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "TemplateScripts",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "Templates",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "TemplateNativeAlarmSources",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "TemplateCompositions",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "TemplateAttributes",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "TemplateAlarms",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "Sites",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "SharedScripts",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "Instances",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "InstanceNativeAlarmSourceOverrides",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "InstanceConnectionBindings",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "InstanceAttributeOverrides",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "InstanceAlarmOverrides",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "ExternalSystemMethods",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "ExternalSystemDefinitions",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "DeployedConfigSnapshots",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "DataConnections",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.AddColumn<byte[]>(
|
||||
name: "RowVersion",
|
||||
table: "DatabaseConnectionDefinitions",
|
||||
type: "rowversion",
|
||||
rowVersion: true,
|
||||
nullable: false);
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_TemplateScripts_RowVersion",
|
||||
table: "TemplateScripts",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Templates_RowVersion",
|
||||
table: "Templates",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_TemplateNativeAlarmSources_RowVersion",
|
||||
table: "TemplateNativeAlarmSources",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_TemplateCompositions_RowVersion",
|
||||
table: "TemplateCompositions",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_TemplateAttributes_RowVersion",
|
||||
table: "TemplateAttributes",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_TemplateAlarms_RowVersion",
|
||||
table: "TemplateAlarms",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Sites_RowVersion",
|
||||
table: "Sites",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_SharedScripts_RowVersion",
|
||||
table: "SharedScripts",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Instances_RowVersion",
|
||||
table: "Instances",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_InstanceNativeAlarmSourceOverrides_RowVersion",
|
||||
table: "InstanceNativeAlarmSourceOverrides",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_InstanceConnectionBindings_RowVersion",
|
||||
table: "InstanceConnectionBindings",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_InstanceAttributeOverrides_RowVersion",
|
||||
table: "InstanceAttributeOverrides",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_InstanceAlarmOverrides_RowVersion",
|
||||
table: "InstanceAlarmOverrides",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_ExternalSystemMethods_RowVersion",
|
||||
table: "ExternalSystemMethods",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_ExternalSystemDefinitions_RowVersion",
|
||||
table: "ExternalSystemDefinitions",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_DeployedConfigSnapshots_RowVersion",
|
||||
table: "DeployedConfigSnapshots",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_DataConnections_RowVersion",
|
||||
table: "DataConnections",
|
||||
column: "RowVersion");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_DatabaseConnectionDefinitions_RowVersion",
|
||||
table: "DatabaseConnectionDefinitions",
|
||||
column: "RowVersion");
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_TemplateScripts_RowVersion",
|
||||
table: "TemplateScripts");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_Templates_RowVersion",
|
||||
table: "Templates");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_TemplateNativeAlarmSources_RowVersion",
|
||||
table: "TemplateNativeAlarmSources");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_TemplateCompositions_RowVersion",
|
||||
table: "TemplateCompositions");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_TemplateAttributes_RowVersion",
|
||||
table: "TemplateAttributes");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_TemplateAlarms_RowVersion",
|
||||
table: "TemplateAlarms");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_Sites_RowVersion",
|
||||
table: "Sites");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_SharedScripts_RowVersion",
|
||||
table: "SharedScripts");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_Instances_RowVersion",
|
||||
table: "Instances");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_InstanceNativeAlarmSourceOverrides_RowVersion",
|
||||
table: "InstanceNativeAlarmSourceOverrides");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_InstanceConnectionBindings_RowVersion",
|
||||
table: "InstanceConnectionBindings");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_InstanceAttributeOverrides_RowVersion",
|
||||
table: "InstanceAttributeOverrides");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_InstanceAlarmOverrides_RowVersion",
|
||||
table: "InstanceAlarmOverrides");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_ExternalSystemMethods_RowVersion",
|
||||
table: "ExternalSystemMethods");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_ExternalSystemDefinitions_RowVersion",
|
||||
table: "ExternalSystemDefinitions");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_DeployedConfigSnapshots_RowVersion",
|
||||
table: "DeployedConfigSnapshots");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_DataConnections_RowVersion",
|
||||
table: "DataConnections");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_DatabaseConnectionDefinitions_RowVersion",
|
||||
table: "DatabaseConnectionDefinitions");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "TemplateScripts");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "Templates");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "TemplateNativeAlarmSources");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "TemplateCompositions");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "TemplateAttributes");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "TemplateAlarms");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "Sites");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "SharedScripts");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "Instances");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "InstanceNativeAlarmSourceOverrides");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "InstanceConnectionBindings");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "InstanceAttributeOverrides");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "InstanceAlarmOverrides");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "ExternalSystemMethods");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "ExternalSystemDefinitions");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "DeployedConfigSnapshots");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "DataConnections");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "RowVersion",
|
||||
table: "DatabaseConnectionDefinitions");
|
||||
}
|
||||
}
|
||||
}
|
||||
+126
@@ -229,6 +229,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(100)
|
||||
.HasColumnType("nvarchar(100)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("DeploymentId");
|
||||
@@ -236,6 +241,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.HasIndex("InstanceId")
|
||||
.IsUnique();
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.ToTable("DeployedConfigSnapshots");
|
||||
});
|
||||
|
||||
@@ -408,11 +415,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<TimeSpan>("RetryDelay")
|
||||
.HasColumnType("time");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("Name")
|
||||
.IsUnique();
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.ToTable("DatabaseConnectionDefinitions");
|
||||
});
|
||||
|
||||
@@ -449,6 +463,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<TimeSpan>("RetryDelay")
|
||||
.HasColumnType("time");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("TimeoutSeconds")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("int")
|
||||
@@ -459,6 +478,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.HasIndex("Name")
|
||||
.IsUnique();
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.ToTable("ExternalSystemDefinitions");
|
||||
});
|
||||
|
||||
@@ -496,8 +517,15 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(4000)
|
||||
.HasColumnType("nvarchar(4000)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("ExternalSystemDefinitionId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
@@ -586,6 +614,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<int?>("AreaId")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("SiteId")
|
||||
.HasColumnType("int");
|
||||
|
||||
@@ -606,6 +639,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasIndex("AreaId");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("SiteId");
|
||||
|
||||
b.HasIndex("TemplateId");
|
||||
@@ -638,12 +673,19 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<int?>("PriorityLevelOverride")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<string>("TriggerConfigurationOverride")
|
||||
.HasMaxLength(4000)
|
||||
.HasColumnType("nvarchar(4000)");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("InstanceId", "AlarmCanonicalName")
|
||||
.IsUnique();
|
||||
|
||||
@@ -673,8 +715,15 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<string>("OverrideValue")
|
||||
.HasColumnType("nvarchar(max)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("InstanceId", "AttributeName")
|
||||
.IsUnique();
|
||||
|
||||
@@ -704,10 +753,17 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<int>("InstanceId")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("DataConnectionId");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("InstanceId", "AttributeName")
|
||||
.IsUnique();
|
||||
|
||||
@@ -736,6 +792,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<bool?>("IsMesReportableOverride")
|
||||
.HasColumnType("bit");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<string>("SourceCanonicalName")
|
||||
.IsRequired()
|
||||
.HasMaxLength(400)
|
||||
@@ -747,6 +808,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("InstanceId", "SourceCanonicalName")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1175,11 +1238,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(4000)
|
||||
.HasColumnType("nvarchar(4000)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("Name")
|
||||
.IsUnique();
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.ToTable("SharedScripts");
|
||||
});
|
||||
|
||||
@@ -1384,11 +1454,18 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(50)
|
||||
.HasColumnType("nvarchar(50)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("SiteId")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("SiteId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1435,6 +1512,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(500)
|
||||
.HasColumnType("nvarchar(500)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<string>("SiteIdentifier")
|
||||
.IsRequired()
|
||||
.HasMaxLength(100)
|
||||
@@ -1445,6 +1527,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.HasIndex("Name")
|
||||
.IsUnique();
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("SiteIdentifier")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1480,6 +1564,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<int?>("ParentTemplateId")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("FolderId");
|
||||
@@ -1490,6 +1579,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasIndex("ParentTemplateId");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.ToTable("Templates");
|
||||
});
|
||||
|
||||
@@ -1528,6 +1619,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
b.Property<int>("PriorityLevel")
|
||||
.HasColumnType("int");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("TemplateId")
|
||||
.HasColumnType("int");
|
||||
|
||||
@@ -1542,6 +1638,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("TemplateId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1587,6 +1685,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(200)
|
||||
.HasColumnType("nvarchar(200)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("TemplateId")
|
||||
.HasColumnType("int");
|
||||
|
||||
@@ -1595,6 +1698,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("TemplateId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1617,6 +1722,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(200)
|
||||
.HasColumnType("nvarchar(200)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("TemplateId")
|
||||
.HasColumnType("int");
|
||||
|
||||
@@ -1624,6 +1734,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasIndex("ComposedTemplateId");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("TemplateId", "InstanceName")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1696,6 +1808,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(200)
|
||||
.HasColumnType("nvarchar(200)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<string>("SourceReference")
|
||||
.IsRequired()
|
||||
.HasMaxLength(1000)
|
||||
@@ -1706,6 +1823,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("TemplateId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
@@ -1752,6 +1871,11 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(4000)
|
||||
.HasColumnType("nvarchar(4000)");
|
||||
|
||||
b.Property<byte[]>("RowVersion")
|
||||
.IsRequired()
|
||||
.ValueGeneratedOnAddOrUpdate()
|
||||
.HasColumnType("rowversion");
|
||||
|
||||
b.Property<int>("TemplateId")
|
||||
.HasColumnType("int");
|
||||
|
||||
@@ -1765,6 +1889,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RowVersion");
|
||||
|
||||
b.HasIndex("TemplateId", "Name")
|
||||
.IsUnique();
|
||||
|
||||
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
using System.Data;
|
||||
using System.Globalization;
|
||||
using System.Text;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Storage;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
|
||||
|
||||
/// <summary>
|
||||
/// SQL Server <see cref="IConfigGraphWatermarkReader"/> (Gitea #62): ONE query returning
|
||||
/// <c>MAX(RowVersion)</c> and <c>COUNT_BIG(*)</c> for every table in
|
||||
/// <see cref="ConfigGraphRowVersions.EntityTypes"/>, folded into an opaque token.
|
||||
///
|
||||
/// <para>
|
||||
/// Why both numbers: <c>rowversion</c> is a database-wide monotonic counter bumped by the
|
||||
/// engine on every INSERT and UPDATE — whoever issues it (the peer central node, a bundle
|
||||
/// import, an <c>ExecuteUpdate</c>, a hand-run script) — so any insert or update raises some
|
||||
/// table's maximum. A DELETE raises nothing, so the per-table row count catches it; a
|
||||
/// delete paired with an insert is caught by the insert's new maximum.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class ConfigGraphWatermarkReader : IConfigGraphWatermarkReader
|
||||
{
|
||||
private readonly ScadaBridgeDbContext _context;
|
||||
|
||||
/// <summary>Creates the reader over the scoped configuration context.</summary>
|
||||
/// <param name="context">The configuration database context.</param>
|
||||
public ConfigGraphWatermarkReader(ScadaBridgeDbContext context)
|
||||
{
|
||||
_context = context ?? throw new ArgumentNullException(nameof(context));
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public async Task<string?> ReadAsync(CancellationToken cancellationToken = default)
|
||||
{
|
||||
// The columns only exist under SQL Server (ScadaBridgeDbContext.OnModelCreating).
|
||||
if (!_context.Database.IsSqlServer())
|
||||
return null;
|
||||
|
||||
var sql = BuildQuery(_context);
|
||||
var connection = _context.Database.GetDbConnection();
|
||||
var openedHere = connection.State != ConnectionState.Open;
|
||||
if (openedHere)
|
||||
await _context.Database.OpenConnectionAsync(cancellationToken);
|
||||
try
|
||||
{
|
||||
await using var command = connection.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
command.Transaction = _context.Database.CurrentTransaction?.GetDbTransaction();
|
||||
|
||||
var token = new StringBuilder();
|
||||
await using var reader = await command.ExecuteReaderAsync(cancellationToken);
|
||||
while (await reader.ReadAsync(cancellationToken))
|
||||
{
|
||||
token.Append(reader.GetString(0)).Append(':');
|
||||
if (!reader.IsDBNull(1))
|
||||
token.Append(Convert.ToHexString((byte[])reader.GetValue(1)));
|
||||
token.Append(':')
|
||||
.Append(reader.GetInt64(2).ToString(CultureInfo.InvariantCulture))
|
||||
.Append(';');
|
||||
}
|
||||
|
||||
return token.ToString();
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (openedHere)
|
||||
await _context.Database.CloseConnectionAsync();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Builds the single <c>UNION ALL</c> query from the EF model's table mappings, so a
|
||||
/// renamed table follows the model rather than a hard-coded name.
|
||||
/// </summary>
|
||||
/// <param name="context">The context whose model supplies the table names.</param>
|
||||
/// <returns>The SQL text.</returns>
|
||||
internal static string BuildQuery(DbContext context)
|
||||
{
|
||||
var parts = new List<string>(ConfigGraphRowVersions.EntityTypes.Count);
|
||||
foreach (var type in ConfigGraphRowVersions.EntityTypes)
|
||||
{
|
||||
var entity = context.Model.FindEntityType(type)
|
||||
?? throw new InvalidOperationException($"Entity {type.Name} is not in the model.");
|
||||
var table = entity.GetTableName()
|
||||
?? throw new InvalidOperationException($"Entity {type.Name} is not mapped to a table.");
|
||||
var schema = entity.GetSchema() ?? "dbo";
|
||||
parts.Add(
|
||||
$"SELECT N'{table}' AS [T], MAX([{ConfigGraphRowVersions.ColumnName}]) AS [V], COUNT_BIG(*) AS [C] " +
|
||||
$"FROM [{schema}].[{table}]");
|
||||
}
|
||||
|
||||
// ORDER BY: a UNION ALL has no guaranteed row order, and the token is compared as a string.
|
||||
return string.Join(" UNION ALL ", parts) + " ORDER BY [T];";
|
||||
}
|
||||
}
|
||||
@@ -186,6 +186,12 @@ public class ScadaBridgeDbContext : DbContext, IDataProtectionKeyContext
|
||||
|
||||
ApplySecretColumnEncryption(modelBuilder);
|
||||
|
||||
// Gitea #62: rowversion change-detection columns on the pending-updates input tables.
|
||||
// SQL Server only (true for production and the design-time factory, so the migration
|
||||
// and snapshot carry them); SQLite cannot generate a rowversion.
|
||||
if (Database.IsSqlServer())
|
||||
ConfigGraphRowVersions.Apply(modelBuilder);
|
||||
|
||||
NeutralizeSqlServerComputedColumnsForNonSqlServerProviders(modelBuilder);
|
||||
}
|
||||
|
||||
|
||||
@@ -82,6 +82,7 @@ public static class ServiceCollectionExtensions
|
||||
services.AddScoped<ICentralUiRepository, CentralUiRepository>();
|
||||
services.AddScoped<ITemplateEngineRepository, TemplateEngineRepository>();
|
||||
services.AddScoped<IDeploymentManagerRepository, DeploymentManagerRepository>();
|
||||
services.AddScoped<IConfigGraphWatermarkReader, ConfigGraphWatermarkReader>();
|
||||
services.AddScoped<ISiteRepository, SiteRepository>();
|
||||
services.AddScoped<IExternalSystemRepository, ExternalSystemRepository>();
|
||||
services.AddScoped<INotificationRepository, NotificationRepository>();
|
||||
|
||||
@@ -92,4 +92,16 @@ public class DeploymentManagerOptions
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public TimeSpan PendingUpdatesCacheTtl { get; set; } = TimeSpan.FromSeconds(60);
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #62: the pending-updates evaluator skips the fleet flatten and re-serves its last
|
||||
/// snapshot while the database-side config watermark (rowversion MAX + row count over every
|
||||
/// input table) and the secret-store stamp are both unchanged — but at most this many
|
||||
/// evaluations in a row: every Nth evaluation is a forced full one, the safety net for
|
||||
/// anything the two stamps cannot see (a KEK re-wrap that keeps its KEK id, an input table
|
||||
/// someone forgets to add to the watermark). With the default 60 s TTL, 10 bounds such a
|
||||
/// miss to about ten minutes of a fleet being looked at. <c>1</c> disables the short-circuit
|
||||
/// (every evaluation is full). Must be at least 1.
|
||||
/// </summary>
|
||||
public int PendingUpdatesFullEvaluationEvery { get; set; } = 10;
|
||||
}
|
||||
|
||||
@@ -49,6 +49,11 @@ public sealed class DeploymentManagerOptionsValidator : OptionsValidatorBase<Dep
|
||||
$"pending-updates snapshot, and a non-positive value would re-flatten the whole fleet on " +
|
||||
$"every poll of every open circuit.");
|
||||
|
||||
builder.RequireThat(options.PendingUpdatesFullEvaluationEvery >= 1,
|
||||
$"ScadaBridge:DeploymentManager:PendingUpdatesFullEvaluationEvery must be at least 1 " +
|
||||
$"(was {options.PendingUpdatesFullEvaluationEvery}); it is how many pending-updates " +
|
||||
$"evaluations may re-serve an unchanged snapshot before a forced full one (1 = always full).");
|
||||
|
||||
builder.RequireThat(options.TerminalDeploymentRecordRetention >= TimeSpan.Zero,
|
||||
$"ScadaBridge:DeploymentManager:TerminalDeploymentRecordRetention must not be negative " +
|
||||
$"(was {options.TerminalDeploymentRecordRetention}); use TimeSpan.Zero to disable purging of " +
|
||||
|
||||
@@ -39,6 +39,10 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
|
||||
private readonly Timer _invalidatedTimer;
|
||||
private volatile bool _disposed;
|
||||
|
||||
// Gitea #62: set by an explicit Re-check and by a deploy completion so the next evaluation
|
||||
// ignores the evaluator's unchanged-watermark short-circuit and re-flattens the fleet.
|
||||
private volatile bool _forceFullNext;
|
||||
|
||||
/// <inheritdoc/>
|
||||
public event Action? Invalidated;
|
||||
|
||||
@@ -79,8 +83,12 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
|
||||
|
||||
/// <inheritdoc/>
|
||||
public Task<PendingUpdatesSnapshot> GetAsync(
|
||||
bool forceRefresh = false, CancellationToken cancellationToken = default) =>
|
||||
_memo.GetAsync(EvaluateAsync, forceRefresh, cancellationToken);
|
||||
bool forceRefresh = false, CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (forceRefresh)
|
||||
_forceFullNext = true;
|
||||
return _memo.GetAsync(EvaluateAsync, forceRefresh, cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>Stops listening for deployment status changes.</summary>
|
||||
public void Dispose()
|
||||
@@ -99,6 +107,7 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
|
||||
|
||||
// Drop the snapshot at once so no caller is served the pre-deploy count, and (re)arm the
|
||||
// quiet-period timer so a bulk deploy's burst of completions notifies subscribers once.
|
||||
_forceFullNext = true;
|
||||
_memo.Invalidate();
|
||||
try
|
||||
{
|
||||
@@ -121,6 +130,12 @@ public sealed class PendingUpdatesCache : IPendingUpdatesCache, IDisposable
|
||||
private async Task<PendingUpdatesSnapshot> EvaluateAsync()
|
||||
{
|
||||
await using var scope = _scopeFactory.CreateAsyncScope();
|
||||
if (_forceFullNext)
|
||||
{
|
||||
_forceFullNext = false;
|
||||
scope.ServiceProvider.GetService<PendingUpdatesShortCircuitState>()?.Reset();
|
||||
}
|
||||
|
||||
return await scope.ServiceProvider.GetRequiredService<IPendingUpdatesEvaluator>().EvaluateAsync();
|
||||
}
|
||||
}
|
||||
|
||||
+129
@@ -1,9 +1,14 @@
|
||||
using System.Diagnostics;
|
||||
using System.Globalization;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
|
||||
using ZB.MOM.WW.Secrets.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
|
||||
|
||||
@@ -29,6 +34,23 @@ namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
|
||||
/// answer that must never be manufactured — it would silently hide a fleet waiting
|
||||
/// on a deploy.
|
||||
/// </para>
|
||||
///
|
||||
/// <para>
|
||||
/// <b>Short-circuit when nothing changed (Gitea #62).</b> Before the fleet pass it reads a
|
||||
/// change token — the database-side <see cref="IConfigGraphWatermarkReader"/> (rowversion
|
||||
/// MAX + row count over every input table, so a peer central node's edit moves it) plus a
|
||||
/// stamp over the secret store's metadata (name, revision, updated time, tombstone, KEK id,
|
||||
/// content type, description — every field of a secret that feeds the artifact hash except
|
||||
/// the wrap envelope). When that token equals the one read before the last CLEAN full
|
||||
/// evaluation, the previous snapshot is re-served instead of re-flattening the fleet. A
|
||||
/// full evaluation is still forced every
|
||||
/// <see cref="DeploymentManagerOptions.PendingUpdatesFullEvaluationEvery"/>th time, the
|
||||
/// safety net for what the two stamps cannot see (a KEK re-wrap that keeps its KEK id does
|
||||
/// not bump a secret's revision). The short-circuit is off — every evaluation is full —
|
||||
/// when the watermark is unavailable (non-SQL-Server provider), when the secret store
|
||||
/// cannot be listed, and after an evaluation that could not compute a site's artifact hash
|
||||
/// (a secret-store outage); it only ever re-serves a real earlier answer.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
{
|
||||
@@ -37,6 +59,10 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
private readonly ArtifactDeploymentService _artifactDeploymentService;
|
||||
private readonly DeploymentService _deploymentService;
|
||||
private readonly ILogger<PendingUpdatesEvaluator> _logger;
|
||||
private readonly IConfigGraphWatermarkReader? _watermarkReader;
|
||||
private readonly ISecretStore? _secretStore;
|
||||
private readonly PendingUpdatesShortCircuitState? _shortCircuit;
|
||||
private readonly int _fullEvaluationEvery;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new evaluator.
|
||||
@@ -55,12 +81,43 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
ArtifactDeploymentService artifactDeploymentService,
|
||||
DeploymentService deploymentService,
|
||||
ILogger<PendingUpdatesEvaluator> logger)
|
||||
: this(siteRepository, deploymentRepository, artifactDeploymentService, deploymentService, logger,
|
||||
watermarkReader: null, secretStore: null, shortCircuit: null, options: null)
|
||||
{
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new evaluator that may short-circuit an unchanged fleet (Gitea #62).
|
||||
/// </summary>
|
||||
/// <param name="siteRepository">Repository the site list is read from.</param>
|
||||
/// <param name="deploymentRepository">Repository the slim deployed-instance list is read from.</param>
|
||||
/// <param name="artifactDeploymentService">Computes each site's current artifact revision hash.</param>
|
||||
/// <param name="deploymentService">Evaluates instance staleness in one batched flatten session.</param>
|
||||
/// <param name="logger">Logger instance.</param>
|
||||
/// <param name="watermarkReader">Database-side change token; null disables the short-circuit.</param>
|
||||
/// <param name="secretStore">The central secret store feeding the artifact hash; null disables the short-circuit.</param>
|
||||
/// <param name="shortCircuit">The process-wide baseline; null disables the short-circuit.</param>
|
||||
/// <param name="options">Supplies <see cref="DeploymentManagerOptions.PendingUpdatesFullEvaluationEvery"/>.</param>
|
||||
public PendingUpdatesEvaluator(
|
||||
ISiteRepository siteRepository,
|
||||
IDeploymentManagerRepository deploymentRepository,
|
||||
ArtifactDeploymentService artifactDeploymentService,
|
||||
DeploymentService deploymentService,
|
||||
ILogger<PendingUpdatesEvaluator> logger,
|
||||
IConfigGraphWatermarkReader? watermarkReader,
|
||||
ISecretStore? secretStore,
|
||||
PendingUpdatesShortCircuitState? shortCircuit,
|
||||
IOptions<DeploymentManagerOptions>? options)
|
||||
{
|
||||
_siteRepository = siteRepository;
|
||||
_deploymentRepository = deploymentRepository;
|
||||
_artifactDeploymentService = artifactDeploymentService;
|
||||
_deploymentService = deploymentService;
|
||||
_logger = logger;
|
||||
_watermarkReader = watermarkReader;
|
||||
_secretStore = secretStore;
|
||||
_shortCircuit = shortCircuit;
|
||||
_fullEvaluationEvery = Math.Max(1, options?.Value.PendingUpdatesFullEvaluationEvery ?? 1);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
@@ -68,6 +125,17 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
{
|
||||
var stopwatch = Stopwatch.StartNew();
|
||||
|
||||
// Read BEFORE the fleet pass: a change landing mid-pass then makes the next token differ,
|
||||
// so the worst case is one extra full evaluation, never a missed change.
|
||||
var token = await ReadChangeTokenAsync(cancellationToken);
|
||||
if (token is not null && _shortCircuit!.TryReuse(token, _fullEvaluationEvery, out var unchanged))
|
||||
{
|
||||
_logger.LogDebug(
|
||||
"Pending-updates snapshot reused: config watermark and secret-store stamp unchanged");
|
||||
// Re-verified as current just now, so it carries this check's time.
|
||||
return unchanged with { ComputedAt = DateTimeOffset.UtcNow };
|
||||
}
|
||||
|
||||
var sites = await _siteRepository.GetAllSitesAsync(cancellationToken);
|
||||
// Slim, untracked projection: identity, site and state are all this pass reads.
|
||||
var instances = await _deploymentRepository.GetDeployedInstanceSummariesAsync(cancellationToken);
|
||||
@@ -105,6 +173,14 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
|
||||
var snapshot = new PendingUpdatesSnapshot(DateTimeOffset.UtcNow, rows);
|
||||
|
||||
// Only a CLEAN evaluation becomes the baseline. A site missing from the hash map means its
|
||||
// secret lookup failed (it reads Out of date); re-serving that would pin a transient outage.
|
||||
if (_shortCircuit is not null)
|
||||
{
|
||||
var clean = sites.All(s => currentArtifactHashes.ContainsKey(s.Id));
|
||||
_shortCircuit.RecordFull(clean ? token : null, snapshot);
|
||||
}
|
||||
|
||||
// One line per evaluation — this runs at most once per cache TTL, so it is
|
||||
// the cheapest record of what the fleet looked like and what the sweep cost.
|
||||
_logger.LogInformation(
|
||||
@@ -120,6 +196,59 @@ public sealed class PendingUpdatesEvaluator : IPendingUpdatesEvaluator
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The combined change token: the database watermark and the secret-store stamp. Null —
|
||||
/// meaning "evaluate in full" — when the short-circuit is not wired, the watermark is
|
||||
/// unavailable, or either read fails. A failed read is logged, never thrown: the full
|
||||
/// evaluation that follows reports any real outage through its own path.
|
||||
/// </summary>
|
||||
private async Task<string?> ReadChangeTokenAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
if (_watermarkReader is null || _secretStore is null || _shortCircuit is null || _fullEvaluationEvery <= 1)
|
||||
return null;
|
||||
|
||||
try
|
||||
{
|
||||
var watermark = await _watermarkReader.ReadAsync(cancellationToken);
|
||||
if (watermark is null)
|
||||
return null;
|
||||
|
||||
var secrets = await _secretStore.ListAsync(includeDeleted: true, cancellationToken);
|
||||
return watermark + "|" + StampSecrets(secrets);
|
||||
}
|
||||
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
|
||||
{
|
||||
_logger.LogWarning(ex, "Pending-updates change token unavailable; evaluating in full");
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// SHA-256 over the secret store's metadata rows, name-sorted. Covers everything about a
|
||||
/// secret that reaches the artifact hash except the wrap envelope bytes, which a same-KEK
|
||||
/// re-wrap can change without moving any of these fields (the periodic full evaluation
|
||||
/// covers that).
|
||||
/// </summary>
|
||||
/// <param name="secrets">The store's metadata, tombstones included.</param>
|
||||
/// <returns>The hex stamp.</returns>
|
||||
internal static string StampSecrets(IReadOnlyList<SecretMetadata> secrets)
|
||||
{
|
||||
var text = new StringBuilder();
|
||||
foreach (var s in secrets.OrderBy(s => s.Name.Value, StringComparer.Ordinal))
|
||||
{
|
||||
text.Append(s.Name.Value).Append('\u001f')
|
||||
.Append(s.Revision.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
|
||||
.Append(s.UpdatedUtc.UtcTicks.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
|
||||
.Append(s.CreatedUtc.UtcTicks.ToString(CultureInfo.InvariantCulture)).Append('\u001f')
|
||||
.Append(s.IsDeleted ? '1' : '0').Append('\u001f')
|
||||
.Append(s.KekId).Append('\u001f')
|
||||
.Append(s.ContentType.ToString()).Append('\u001f')
|
||||
.Append(s.Description).Append('\u001e');
|
||||
}
|
||||
|
||||
return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(text.ToString())));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Decides a site's artifact state by comparing the hash stamped by its last
|
||||
/// successful artifact deploy against the hash the current definitions would
|
||||
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #62: the process-wide baseline the (scoped) <see cref="PendingUpdatesEvaluator"/>
|
||||
/// short-circuits against — the change token read just BEFORE the last clean full
|
||||
/// evaluation, and the snapshot that evaluation produced.
|
||||
///
|
||||
/// <para>
|
||||
/// <b>The token is never a per-process signal.</b> It is the database-side config watermark
|
||||
/// plus the secret-store stamp, so an edit made through the peer central node moves it
|
||||
/// exactly as a local one does (arch-review 2026-09-09 MG-3 removed a shortcut on the
|
||||
/// per-process template-graph watermark for that reason). The only thing this object holds
|
||||
/// per process is the last answer, and that answer is only ever re-served — never a
|
||||
/// manufactured "nothing pending".
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public sealed class PendingUpdatesShortCircuitState
|
||||
{
|
||||
private readonly object _lock = new();
|
||||
private string? _token;
|
||||
private PendingUpdatesSnapshot? _snapshot;
|
||||
private int _reusesSinceFull;
|
||||
|
||||
/// <summary>
|
||||
/// Returns the baseline snapshot when <paramref name="token"/> equals the baseline token
|
||||
/// and fewer than <paramref name="fullEvaluationEvery"/> − 1 reuses have happened since
|
||||
/// the last full evaluation; counts the reuse.
|
||||
/// </summary>
|
||||
/// <param name="token">The change token read for this evaluation.</param>
|
||||
/// <param name="fullEvaluationEvery">Every Nth evaluation must be full.</param>
|
||||
/// <param name="snapshot">The re-servable snapshot, when this returns <c>true</c>.</param>
|
||||
/// <returns><c>true</c> when the full evaluation may be skipped.</returns>
|
||||
public bool TryReuse(string token, int fullEvaluationEvery, out PendingUpdatesSnapshot snapshot)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(token);
|
||||
lock (_lock)
|
||||
{
|
||||
if (_snapshot is not null
|
||||
&& _token is not null
|
||||
&& string.Equals(_token, token, StringComparison.Ordinal)
|
||||
&& _reusesSinceFull < fullEvaluationEvery - 1)
|
||||
{
|
||||
_reusesSinceFull++;
|
||||
snapshot = _snapshot;
|
||||
return true;
|
||||
}
|
||||
|
||||
snapshot = null!;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Records a full evaluation. A <see langword="null"/> <paramref name="token"/> (no
|
||||
/// watermark available, or the evaluation was not clean) clears the baseline, so the next
|
||||
/// evaluation is full too.
|
||||
/// </summary>
|
||||
/// <param name="token">The change token read before the evaluation, or null.</param>
|
||||
/// <param name="snapshot">The snapshot the evaluation produced.</param>
|
||||
public void RecordFull(string? token, PendingUpdatesSnapshot snapshot)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(snapshot);
|
||||
lock (_lock)
|
||||
{
|
||||
_token = token;
|
||||
_snapshot = token is null ? null : snapshot;
|
||||
_reusesSinceFull = 0;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Drops the baseline so the next evaluation is full (deploy completion, Re-check).</summary>
|
||||
public void Reset()
|
||||
{
|
||||
lock (_lock)
|
||||
{
|
||||
_token = null;
|
||||
_snapshot = null;
|
||||
_reusesSinceFull = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,13 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.DependencyInjection.Extensions;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Transport;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
|
||||
using ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
|
||||
using ZB.MOM.WW.Secrets.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager;
|
||||
|
||||
@@ -92,7 +95,22 @@ public static class ServiceCollectionExtensions
|
||||
// it reads through the same per-request DbContext as the services it composes;
|
||||
// the singleton single-flight cache below runs it in its own DI scope so the
|
||||
// fleet is flattened at most once per cache TTL.
|
||||
services.AddScoped<IPendingUpdatesEvaluator, PendingUpdatesEvaluator>();
|
||||
// Gitea #62: the evaluator short-circuits an unchanged fleet against a DATABASE-side
|
||||
// rowversion watermark + a secret-store stamp; the baseline it compares against is
|
||||
// process-wide, so it lives in a singleton. The watermark reader and the secret store
|
||||
// are optional (a host without the configuration database or a secret store simply
|
||||
// always evaluates in full).
|
||||
services.AddSingleton<PendingUpdatesShortCircuitState>();
|
||||
services.AddScoped<IPendingUpdatesEvaluator>(sp => new PendingUpdatesEvaluator(
|
||||
sp.GetRequiredService<ISiteRepository>(),
|
||||
sp.GetRequiredService<IDeploymentManagerRepository>(),
|
||||
sp.GetRequiredService<ArtifactDeploymentService>(),
|
||||
sp.GetRequiredService<DeploymentService>(),
|
||||
sp.GetRequiredService<ILogger<PendingUpdatesEvaluator>>(),
|
||||
sp.GetService<IConfigGraphWatermarkReader>(),
|
||||
sp.GetService<ISecretStore>(),
|
||||
sp.GetRequiredService<PendingUpdatesShortCircuitState>(),
|
||||
sp.GetService<IOptions<DeploymentManagerOptions>>()));
|
||||
|
||||
// Process-wide memo over the evaluator, shared by the Central UI (rail indicator,
|
||||
// Sites page, Health tiles) and the management API's GetPendingUpdatesCommand (CLI
|
||||
|
||||
+198
@@ -0,0 +1,198 @@
|
||||
using System.Text.RegularExpressions;
|
||||
using Microsoft.Data.SqlClient;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
|
||||
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
|
||||
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests.Migrations;
|
||||
using Xunit;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #62: the database-side config watermark behind the pending-updates short-circuit,
|
||||
/// against a real migrated SQL Server database (the <c>AddConfigGraphRowVersions</c>
|
||||
/// migration included). The "peer node" in these tests is a separate raw
|
||||
/// <see cref="SqlConnection"/> that never touches this process's DbContext — the exact shape
|
||||
/// of a write made by the other central node, which a per-process watermark never sees
|
||||
/// (arch-review 2026-09-09 MG-3).
|
||||
/// </summary>
|
||||
public class ConfigGraphWatermarkReaderTests : IClassFixture<MsSqlMigrationFixture>
|
||||
{
|
||||
private readonly MsSqlMigrationFixture _fixture;
|
||||
|
||||
/// <summary>Initializes the suite with the shared MS SQL migration fixture.</summary>
|
||||
/// <param name="fixture">The shared MS SQL migration fixture.</param>
|
||||
public ConfigGraphWatermarkReaderTests(MsSqlMigrationFixture fixture) => _fixture = fixture;
|
||||
|
||||
private ScadaBridgeDbContext CreateContext(SqlReaderCommandRecorder? recorder = null)
|
||||
{
|
||||
var builder = new DbContextOptionsBuilder<ScadaBridgeDbContext>().UseSqlServer(_fixture.ConnectionString);
|
||||
if (recorder is not null)
|
||||
builder.AddInterceptors(recorder);
|
||||
return new ScadaBridgeDbContext(builder.Options);
|
||||
}
|
||||
|
||||
private async Task<string> ReadAsync()
|
||||
{
|
||||
await using var context = CreateContext();
|
||||
var token = await new ConfigGraphWatermarkReader(context).ReadAsync();
|
||||
Assert.NotNull(token);
|
||||
return token!;
|
||||
}
|
||||
|
||||
private void PeerExecute(string sql, params SqlParameter[] parameters)
|
||||
{
|
||||
using var connection = _fixture.OpenConnection();
|
||||
using var command = connection.CreateCommand();
|
||||
command.CommandText = sql;
|
||||
command.Parameters.AddRange(parameters);
|
||||
command.ExecuteNonQuery();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that the token is stable with no writes, and moves for a peer-node insert, update
|
||||
/// and delete made on a connection this process's context never saw.
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task Token_IsStableWithoutWrites_AndMovesForPeerInsertUpdateAndDelete()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
var ident = "wm-" + Guid.NewGuid().ToString("N")[..8];
|
||||
|
||||
var t0 = await ReadAsync();
|
||||
Assert.Equal(t0, await ReadAsync()); // no write → identical token (the short-circuit case)
|
||||
|
||||
// Peer INSERT.
|
||||
PeerExecute("INSERT INTO [Sites] ([Name], [SiteIdentifier]) VALUES (@n, @i);",
|
||||
new SqlParameter("@n", ident), new SqlParameter("@i", ident));
|
||||
var t1 = await ReadAsync();
|
||||
Assert.NotEqual(t0, t1);
|
||||
Assert.Equal(t1, await ReadAsync());
|
||||
|
||||
// Peer UPDATE of a column no application write path ever "bumps" anything for.
|
||||
PeerExecute("UPDATE [Sites] SET [Description] = N'edited by the peer' WHERE [SiteIdentifier] = @i;",
|
||||
new SqlParameter("@i", ident));
|
||||
var t2 = await ReadAsync();
|
||||
Assert.NotEqual(t1, t2);
|
||||
|
||||
// Peer DELETE of a row that is NOT the table's newest — so MAX(RowVersion) stays put and
|
||||
// only the row count can catch it. A second, newer row is inserted first to guarantee that.
|
||||
var newer = ident + "-b";
|
||||
PeerExecute("INSERT INTO [Sites] ([Name], [SiteIdentifier]) VALUES (@n, @i);",
|
||||
new SqlParameter("@n", newer), new SqlParameter("@i", newer));
|
||||
var t3 = await ReadAsync();
|
||||
PeerExecute("DELETE FROM [Sites] WHERE [SiteIdentifier] = @i;", new SqlParameter("@i", ident));
|
||||
var t4 = await ReadAsync();
|
||||
Assert.NotEqual(t3, t4);
|
||||
}
|
||||
|
||||
/// <summary>Verifies that an EF write through this process's context also moves the token.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task Token_MovesForALocalEfWrite_AndTheRowVersionIsNotAConcurrencyToken()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
var ident = "wm-" + Guid.NewGuid().ToString("N")[..8];
|
||||
var before = await ReadAsync();
|
||||
|
||||
int id;
|
||||
await using (var context = CreateContext())
|
||||
{
|
||||
var site = new Site(ident, ident);
|
||||
context.Sites.Add(site);
|
||||
await context.SaveChangesAsync();
|
||||
id = site.Id;
|
||||
}
|
||||
|
||||
var afterInsert = await ReadAsync();
|
||||
Assert.NotEqual(before, afterInsert);
|
||||
|
||||
// A detached Update of a stale copy must still save: the column is change detection only,
|
||||
// never an optimistic-concurrency check that would fail a save the old schema accepted.
|
||||
await using (var context = CreateContext())
|
||||
{
|
||||
context.Sites.Update(new Site(ident, ident) { Id = id, Description = "detached update" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
Assert.NotEqual(afterInsert, await ReadAsync());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Guard for the covered set: every table the pending-updates evaluator's repository reads
|
||||
/// touch must carry the watermark column, so a new input table cannot silently escape it.
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task EveryTableThePendingUpdatesReadsTouch_IsCoveredByTheWatermark()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
// Seed one site → template → instance so the split-query child reads actually execute
|
||||
// (EF skips a split query's child statements when the root row is absent).
|
||||
var ident = "wm-" + Guid.NewGuid().ToString("N")[..8];
|
||||
int siteId, templateId, instanceId;
|
||||
await using (var seed = CreateContext())
|
||||
{
|
||||
var site = new Site(ident, ident);
|
||||
var template = new ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates.Template(ident);
|
||||
seed.Sites.Add(site);
|
||||
seed.Templates.Add(template);
|
||||
await seed.SaveChangesAsync();
|
||||
var instance = new ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances.Instance(ident)
|
||||
{
|
||||
SiteId = site.Id,
|
||||
TemplateId = template.Id,
|
||||
};
|
||||
seed.Instances.Add(instance);
|
||||
await seed.SaveChangesAsync();
|
||||
(siteId, templateId, instanceId) = (site.Id, template.Id, instance.Id);
|
||||
}
|
||||
|
||||
var recorder = new SqlReaderCommandRecorder();
|
||||
await using var context = CreateContext(recorder);
|
||||
var watermark = new TemplateGraphWatermark();
|
||||
var sites = new SiteRepository(context, watermark);
|
||||
var templates = new TemplateEngineRepository(context, watermark);
|
||||
var deployments = new DeploymentManagerRepository(context);
|
||||
var external = new ExternalSystemRepository(context);
|
||||
|
||||
// PendingUpdatesEvaluator → ArtifactDeploymentService.ComputeCurrentArtifactHashesAsync
|
||||
await sites.GetAllSitesAsync();
|
||||
await templates.GetAllSharedScriptsAsync();
|
||||
await external.GetAllExternalSystemsAsync();
|
||||
await external.GetAllDatabaseConnectionsAsync();
|
||||
await deployments.GetAllExternalSystemMethodsAsync();
|
||||
await sites.GetDataConnectionsBySiteIdAsync(siteId);
|
||||
// → DeploymentService.GetInstanceStalenessAsync / FlatteningPipeline.FlattenCoreAsync
|
||||
await deployments.GetDeployedInstanceSummariesAsync();
|
||||
await deployments.GetDeployedRevisionHashesAsync();
|
||||
await deployments.GetDeployedSnapshotsByInstanceIdsAsync([instanceId]);
|
||||
await templates.GetInstanceByIdAsync(instanceId);
|
||||
await templates.GetTemplateWithChildrenAsync(templateId);
|
||||
|
||||
var covered = ConfigGraphRowVersions.EntityTypes
|
||||
.Select(t => context.Model.FindEntityType(t)!.GetTableName()!)
|
||||
.ToHashSet(StringComparer.OrdinalIgnoreCase);
|
||||
var read = recorder.Commands
|
||||
.SelectMany(c => Regex.Matches(c, @"(?:FROM|JOIN)\s+(?:\[\w+\]\.)?\[(\w+)\]").Select(m => m.Groups[1].Value))
|
||||
.ToHashSet(StringComparer.OrdinalIgnoreCase);
|
||||
|
||||
Assert.NotEmpty(read);
|
||||
Assert.Empty(read.Except(covered));
|
||||
// Positive control: the regex really does see the tables (a vacuous empty set would pass above).
|
||||
Assert.Contains("Templates", read);
|
||||
Assert.Contains("InstanceAttributeOverrides", read);
|
||||
Assert.Contains("DeployedConfigSnapshots", read);
|
||||
}
|
||||
|
||||
/// <summary>Verifies that a non-SQL-Server provider yields no token, so callers never short-circuit on it.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task NonSqlServerProvider_ReturnsNull()
|
||||
{
|
||||
await using var context = SqliteTestHelper.CreateInMemoryContext();
|
||||
Assert.Null(await new ConfigGraphWatermarkReader(context).ReadAsync());
|
||||
}
|
||||
}
|
||||
+254
@@ -0,0 +1,254 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Microsoft.Extensions.Options;
|
||||
using NSubstitute;
|
||||
using NSubstitute.ExceptionExtensions;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Instances;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Sites;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
|
||||
using ZB.MOM.WW.ScadaBridge.Communication;
|
||||
using ZB.MOM.WW.ScadaBridge.DeploymentManager.PendingUpdates;
|
||||
using ZB.MOM.WW.ScadaBridge.TemplateEngine.Flattening;
|
||||
using ZB.MOM.WW.Secrets.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Gitea #62: <see cref="PendingUpdatesEvaluator"/> skips the fleet flatten while the
|
||||
/// database-side config watermark and the secret-store stamp are unchanged, and never
|
||||
/// otherwise. "Did a full evaluation happen" is observed as a call to the flatten seam
|
||||
/// (<see cref="IFlatteningPipeline.ComputeRevisionHashAsync"/>) — the cost being avoided.
|
||||
/// The watermark here is a substitute; the real reader's peer-node detection is proven
|
||||
/// against SQL Server in <c>ConfigGraphWatermarkReaderTests</c>.
|
||||
/// </summary>
|
||||
public class PendingUpdatesShortCircuitTests
|
||||
{
|
||||
private readonly ISiteRepository _siteRepo = Substitute.For<ISiteRepository>();
|
||||
private readonly IDeploymentManagerRepository _deploymentRepo = Substitute.For<IDeploymentManagerRepository>();
|
||||
private readonly IFlatteningPipeline _pipeline = Substitute.For<IFlatteningPipeline>();
|
||||
private readonly IConfigGraphWatermarkReader _watermark = Substitute.For<IConfigGraphWatermarkReader>();
|
||||
private readonly ISecretStore _secrets = Substitute.For<ISecretStore>();
|
||||
private readonly PendingUpdatesShortCircuitState _state = new();
|
||||
private string _token = "wm-1";
|
||||
private List<SecretMetadata> _secretRows = [Secret("db-pass", revision: 1)];
|
||||
|
||||
/// <summary>Wires one site with one deployed, up-to-date instance.</summary>
|
||||
public PendingUpdatesShortCircuitTests()
|
||||
{
|
||||
_pipeline.CreateSession().Returns(_ => new FlattenSession(new TemplateGraphWatermark()));
|
||||
_siteRepo.GetAllSitesAsync(Arg.Any<CancellationToken>()).Returns([new Site("Alpha", "site-a") { Id = 1 }]);
|
||||
_deploymentRepo.GetDeployedInstanceSummariesAsync(Arg.Any<CancellationToken>())
|
||||
.Returns([new Instance("Inst7") { Id = 7, SiteId = 1, State = InstanceState.Enabled }]);
|
||||
_deploymentRepo.GetDeployedRevisionHashesAsync(Arg.Any<CancellationToken>())
|
||||
.Returns(new Dictionary<int, string> { [7] = "h7" });
|
||||
_pipeline.ComputeRevisionHashAsync(7, Arg.Any<FlattenSession?>(), Arg.Any<CancellationToken>())
|
||||
.Returns(Result<string>.Success("h7"));
|
||||
_watermark.ReadAsync(Arg.Any<CancellationToken>()).Returns(_ => _token);
|
||||
_secrets.ListAsync(true, Arg.Any<CancellationToken>()).Returns(_ => (IReadOnlyList<SecretMetadata>)_secretRows);
|
||||
}
|
||||
|
||||
private static SecretMetadata Secret(string name, long revision, string kek = "kek-1") => new()
|
||||
{
|
||||
Name = new SecretName(name),
|
||||
ContentType = SecretContentType.Text,
|
||||
KekId = kek,
|
||||
Revision = revision,
|
||||
CreatedUtc = new DateTimeOffset(2026, 11, 1, 0, 0, 0, TimeSpan.Zero),
|
||||
UpdatedUtc = new DateTimeOffset(2026, 11, 1, 0, 0, 0, TimeSpan.Zero).AddMinutes(revision),
|
||||
};
|
||||
|
||||
private PendingUpdatesEvaluator Evaluator(int every = 10, bool wired = true)
|
||||
{
|
||||
var comms = new CommunicationService(Options.Create(new CommunicationOptions()), NullLogger<CommunicationService>.Instance);
|
||||
var options = Options.Create(new DeploymentManagerOptions { PendingUpdatesFullEvaluationEvery = every });
|
||||
var artifacts = new ArtifactDeploymentService(
|
||||
_siteRepo, _deploymentRepo, Substitute.For<ITemplateEngineRepository>(), Substitute.For<IExternalSystemRepository>(),
|
||||
comms, Substitute.For<IAuditService>(), options, NullLogger<ArtifactDeploymentService>.Instance,
|
||||
new ArtifactRevisionHashService(), new SiteSecretsCollector(_secrets));
|
||||
var deployments = new DeploymentService(
|
||||
_deploymentRepo, _siteRepo, Substitute.For<ITemplateEngineRepository>(), _pipeline, comms,
|
||||
new OperationLockManager(), Substitute.For<IAuditService>(), new DiffService(), new RevisionHashService(),
|
||||
new DeploymentStatusNotifier(NullLogger<DeploymentStatusNotifier>.Instance),
|
||||
options, Options.Create(new CommunicationOptions()), NullLogger<DeploymentService>.Instance);
|
||||
return wired
|
||||
? new PendingUpdatesEvaluator(_siteRepo, _deploymentRepo, artifacts, deployments,
|
||||
NullLogger<PendingUpdatesEvaluator>.Instance, _watermark, _secrets, _state, options)
|
||||
: new PendingUpdatesEvaluator(_siteRepo, _deploymentRepo, artifacts, deployments,
|
||||
NullLogger<PendingUpdatesEvaluator>.Instance);
|
||||
}
|
||||
|
||||
private int FlattenCalls() => _pipeline.ReceivedCalls()
|
||||
.Count(c => c.GetMethodInfo().Name == nameof(IFlatteningPipeline.ComputeRevisionHashAsync));
|
||||
|
||||
/// <summary>Verifies that an unchanged watermark and secret stamp re-serve the snapshot without flattening.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task Unchanged_ReservesTheSnapshot_WithoutAFleetFlatten()
|
||||
{
|
||||
// A stale instance, so the re-served answer is visibly "something pending", not a blank one.
|
||||
_pipeline.ComputeRevisionHashAsync(7, Arg.Any<FlattenSession?>(), Arg.Any<CancellationToken>())
|
||||
.Returns(Result<string>.Success("h7-edited"));
|
||||
_deploymentRepo.GetDeployedSnapshotsByInstanceIdsAsync(Arg.Any<IReadOnlyCollection<int>>(), Arg.Any<CancellationToken>())
|
||||
.Returns(new Dictionary<int, Commons.Entities.Deployment.DeployedConfigSnapshot>
|
||||
{
|
||||
[7] = new("d", "h7", """{"InstanceUniqueName":"Inst7"}""") { InstanceId = 7 },
|
||||
});
|
||||
var evaluator = Evaluator();
|
||||
|
||||
var first = await evaluator.EvaluateAsync();
|
||||
var second = await evaluator.EvaluateAsync();
|
||||
|
||||
Assert.Equal(1, FlattenCalls());
|
||||
Assert.Equal(1, first.StaleInstanceCount);
|
||||
Assert.Equal(first.Sites, second.Sites);
|
||||
Assert.True(second.HasPendingUpdates);
|
||||
Assert.True(second.ComputedAt >= first.ComputedAt);
|
||||
}
|
||||
|
||||
/// <summary>Verifies that a moved database watermark (e.g. a peer-node edit) forces a full evaluation.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task WatermarkMoved_ForcesAFullEvaluation()
|
||||
{
|
||||
var evaluator = Evaluator();
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
_token = "wm-2"; // the shared DB's rowversion/count moved — whichever node wrote it
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
Assert.Equal(2, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that a secret-store change (revision, tombstone, re-wrap under a new KEK) forces a full evaluation.</summary>
|
||||
/// <param name="change">Which secret change to apply.</param>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Theory]
|
||||
[InlineData("revision")]
|
||||
[InlineData("added")]
|
||||
[InlineData("deleted")]
|
||||
[InlineData("kek")]
|
||||
public async Task SecretStoreChanged_ForcesAFullEvaluation(string change)
|
||||
{
|
||||
var evaluator = Evaluator();
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
_secretRows = change switch
|
||||
{
|
||||
"revision" => [Secret("db-pass", revision: 2)],
|
||||
"added" => [Secret("db-pass", revision: 1), Secret("api-key", revision: 1)],
|
||||
"deleted" => [Secret("db-pass", revision: 1) with { IsDeleted = true }],
|
||||
_ => [Secret("db-pass", revision: 1, kek: "kek-2")],
|
||||
};
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
Assert.Equal(2, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that every Nth evaluation is full even when nothing changed (the safety net).</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task EveryNthEvaluation_IsFull_EvenWhenUnchanged()
|
||||
{
|
||||
var evaluator = Evaluator(every: 3);
|
||||
|
||||
for (var i = 0; i < 7; i++)
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
// Evaluations 1, 4 and 7 are full; 2, 3, 5, 6 reuse.
|
||||
Assert.Equal(3, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that an unavailable watermark, a failing secret-store listing, or N=1 never short-circuit.</summary>
|
||||
/// <param name="mode">Which unavailability to simulate.</param>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Theory]
|
||||
[InlineData("no-watermark")]
|
||||
[InlineData("watermark-throws")]
|
||||
[InlineData("secret-list-throws")]
|
||||
[InlineData("every-1")]
|
||||
[InlineData("unwired")]
|
||||
public async Task NoTrustworthyToken_EveryEvaluationIsFull(string mode)
|
||||
{
|
||||
switch (mode)
|
||||
{
|
||||
case "no-watermark": _watermark.ReadAsync(Arg.Any<CancellationToken>()).Returns((string?)null); break;
|
||||
case "watermark-throws": _watermark.ReadAsync(Arg.Any<CancellationToken>()).ThrowsAsync(new TimeoutException()); break;
|
||||
case "secret-list-throws": _secrets.ListAsync(true, Arg.Any<CancellationToken>()).ThrowsAsync(new InvalidOperationException("store down")); break;
|
||||
}
|
||||
|
||||
var evaluator = Evaluator(every: mode == "every-1" ? 1 : 10, wired: mode != "unwired");
|
||||
await evaluator.EvaluateAsync();
|
||||
await evaluator.EvaluateAsync();
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
Assert.Equal(3, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that an evaluation whose artifact hash could not be computed (a secret-store read
|
||||
/// failure while resolving a site's secret reference) is not reused, so the transient
|
||||
/// Out-of-date answer clears on the very next evaluation.
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task UncleanEvaluation_IsNotReused()
|
||||
{
|
||||
_siteRepo.GetDataConnectionsBySiteIdAsync(1, Arg.Any<CancellationToken>())
|
||||
.Returns([new DataConnection("GW1", "MxGateway", 1) { PrimaryConfiguration = """{"ApiKey":"secret:db-pass"}""" }]);
|
||||
_secrets.GetAsync(Arg.Any<SecretName>(), Arg.Any<CancellationToken>()).ThrowsAsync(new TimeoutException("store down"));
|
||||
var evaluator = Evaluator();
|
||||
|
||||
await evaluator.EvaluateAsync();
|
||||
await evaluator.EvaluateAsync();
|
||||
|
||||
Assert.Equal(2, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that the cache's Re-check (forceRefresh) bypasses the short-circuit.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task CacheForceRefresh_BypassesTheShortCircuit()
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
services.AddSingleton(_state);
|
||||
services.AddScoped<IPendingUpdatesEvaluator>(_ => Evaluator());
|
||||
await using var provider = services.BuildServiceProvider();
|
||||
using var cache = new PendingUpdatesCache(
|
||||
provider.GetRequiredService<IServiceScopeFactory>(),
|
||||
Options.Create(new DeploymentManagerOptions()),
|
||||
new DeploymentStatusNotifier(NullLogger<DeploymentStatusNotifier>.Instance));
|
||||
|
||||
await cache.GetAsync();
|
||||
await cache.GetAsync(forceRefresh: true); // unchanged token, but the operator asked: full
|
||||
Assert.Equal(2, FlattenCalls());
|
||||
|
||||
await cache.GetAsync(forceRefresh: true);
|
||||
Assert.Equal(3, FlattenCalls());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that a deploy completion makes the next evaluation full.</summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[Fact]
|
||||
public async Task DeployCompletion_MakesTheNextEvaluationFull()
|
||||
{
|
||||
var notifier = new DeploymentStatusNotifier(NullLogger<DeploymentStatusNotifier>.Instance);
|
||||
var services = new ServiceCollection();
|
||||
services.AddSingleton(_state);
|
||||
services.AddScoped<IPendingUpdatesEvaluator>(_ => Evaluator());
|
||||
await using var provider = services.BuildServiceProvider();
|
||||
using var cache = new PendingUpdatesCache(
|
||||
provider.GetRequiredService<IServiceScopeFactory>(),
|
||||
Options.Create(new DeploymentManagerOptions()),
|
||||
notifier);
|
||||
|
||||
await cache.GetAsync();
|
||||
notifier.NotifyStatusChanged(new DeploymentStatusChange("dep-7", 7, DeploymentStatus.Success));
|
||||
await cache.GetAsync();
|
||||
|
||||
Assert.Equal(2, FlattenCalls());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user