perf(templates): descendant reconcile loads only the subtree + chain (ScadaBridge#86)

TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.

The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.

Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
  subtree, an ancestor above the edited template, a composed module,
  look-alike unrelated trees, a parent cycle and a dangling parent, the
  closure-backed reconcile issues exactly the writes (add/update/delete,
  audit, save) of the old whole-graph reconcile, and the resolver answers
  identically for every descendant. Fails with the old service (whole-graph
  load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
  / 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
  one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
  GetAllTemplatesAsync wired to throw; existing propagation tests now run
  against the closure via ServeReconcileClosureFromFullGraph.

ResyncInheritedMembersAsync (operator action) still reads the whole graph.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:01:56 -05:00
parent 0eac956d25
commit c29b99c63b
10 changed files with 718 additions and 31 deletions
+15 -10
View File
@@ -219,17 +219,22 @@ names, locks and edges: `InstanceService.SetAlarmOverrideAsync` (resolving one a
name) and `TemplateDeletionService.CanDeleteTemplateAsync` (inheritor / derived / name) and `TemplateDeletionService.CanDeleteTemplateAsync` (inheritor / derived /
composer checks). `TemplateFolderService.DeleteFolderAsync`, which only counts composer checks). `TemplateFolderService.DeleteFolderAsync`, which only counts
templates per `FolderId`, reads the lighter `GetTemplateStubsAsync` identity projection templates per `FolderId`, reads the lighter `GetTemplateStubsAsync` identity projection
(review 2026-10-21 04/P6, 08/P1). Template member add/update paths still load the whole (review 2026-10-21 04/P6, 08/P1). Template member add/update paths read only the
analysis projection for collision detection; narrowing that to the edited template's edited template's inheritance chain + composition closure through
chain + composition closure needs a closure-scoped repository read that does not exist `GetTemplateAnalysisClosureAsync` (the same projection, one query per closure depth —
yet. Gitea #66).
The tracked, body-bearing `GetAllTemplatesAsync` remains the read for the two paths The tracked, body-bearing shape remains the read for the two paths that genuinely need
that genuinely need it — the inheritance reconciler (which compares and copies it — the inheritance reconciler (which compares and copies script bodies and writes
script bodies and writes through the loaded entities) and the flattener (which must through the loaded entities) and the flattener (which must observe rows an in-flight
observe rows an in-flight bundle import has staged on the shared change tracker). bundle import has staged on the shared change tracker). The automatic post-mutation
`ReconcileDescendantsAsync` additionally accepts an already-loaded tracked graph so `ReconcileDescendantsAsync` reads only `GetTemplateReconcileClosureAsync` — the edited
a caller holding one does not force a second load. template's derived subtree plus its own inheritance chain, tracked and fully included,
one load per subtree depth and chain level (Gitea #86). The reconcile is
inheritance-only, so every descendant's chain lies inside that set; a template with no
children costs a single empty probe query. It still accepts an already-loaded tracked
graph from a caller holding one. The operator `ResyncInheritedMembersAsync` and the
flattener keep reading the whole tracked graph via `GetAllTemplatesAsync`.
### Native Alarm Source Resolution ### Native Alarm Source Resolution
@@ -69,6 +69,20 @@ public interface ITemplateEngineRepository
/// <returns>A task that resolves to the closure templates without script bodies.</returns> /// <returns>A task that resolves to the closure templates without script bodies.</returns>
Task<IReadOnlyList<Template>> GetTemplateAnalysisClosureAsync(int templateId, CancellationToken cancellationToken = default); Task<IReadOnlyList<Template>> GetTemplateAnalysisClosureAsync(int templateId, CancellationToken cancellationToken = default);
/// <summary> /// <summary>
/// Returns the TRACKED, fully-loaded templates (the <see cref="GetAllTemplatesAsync"/> shape:
/// attributes, alarms, scripts with bodies, compositions, native alarm sources) that the
/// inherited-member reconcile of <paramref name="templateId"/>'s descendants reads: its whole
/// derived subtree plus its own inheritance chain (see
/// <see cref="ZB.MOM.WW.ScadaBridge.Commons.Types.Templates.TemplateReconcileClosure"/>).
/// Backs <c>TemplateService.ReconcileDescendantsAsync</c> (Gitea #86), which previously
/// loaded every template on each base-member add/update/delete. Returns an empty list when the
/// template has no children — there is nothing to reconcile — after a single probe query.
/// </summary>
/// <param name="templateId">The template whose member set changed.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that resolves to the subtree followed by the template's chain.</returns>
Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default);
/// <summary>
/// Returns an untracked identity/edge projection of every template (no children), ordered by name. /// Returns an untracked identity/edge projection of every template (no children), ordered by name.
/// </summary> /// </summary>
/// <param name="cancellationToken">Cancellation token.</param> /// <param name="cancellationToken">Cancellation token.</param>
@@ -0,0 +1,103 @@
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
/// <summary>
/// Computes the set of templates the inherited-member reconcile reads when a base member of one
/// template changes (Gitea #86): the template's whole <b>derived subtree</b> (children,
/// grandchildren, …) plus the template's own <b>inheritance chain</b> (itself, its parent,
/// grandparent, …).
///
/// <para>
/// This is the descendant-direction counterpart of <see cref="TemplateAnalysisClosure"/> (#66).
/// <c>TemplateService.ReconcileDescendantsAsync</c> reconciles every descendant <c>D</c> by
/// resolving <c>D</c>'s inheritance chain (<c>TemplateInheritanceResolver.Resolve</c>), which is
/// inheritance only — compositions are never followed. <c>D</c>'s chain is the path from
/// <c>D</c> up to the edited template (all inside the subtree) followed by the edited template's
/// own chain, and every member winner originates on that chain. So the subtree plus the root's
/// chain is everything the reconcile can read; the rest of the library is irrelevant.
/// </para>
///
/// <para>
/// The walk is breadth-first by batch: one <c>loadChildren</c> call per subtree depth and one
/// <c>loadByIds</c> call per ancestor level, so a store-backed loader issues one query per
/// depth rather than one per template. When the template has no children — the common case —
/// the walk stops after the single children probe and returns an empty list, because there is
/// nothing to reconcile. Cycles terminate (each id is loaded at most once) and a dangling parent
/// edge simply ends the chain, exactly as the resolver's chain builder does.
/// </para>
/// </summary>
public static class TemplateReconcileClosure
{
/// <summary>
/// Collects the derived subtree and inheritance chain of <paramref name="templateId"/>.
/// </summary>
/// <param name="templateId">The template whose member set changed.</param>
/// <param name="loadChildren">Loads every template whose parent is one of the given ids.</param>
/// <param name="loadByIds">Loads the templates with the given ids (unknown ids are omitted).</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// The subtree (breadth-first) followed by the root's chain, each template once; an empty
/// list when the template has no children.
/// </returns>
public static async Task<IReadOnlyList<Template>> CollectAsync(
int templateId,
Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadChildren,
Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadByIds,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(loadChildren);
ArgumentNullException.ThrowIfNull(loadByIds);
var loaded = new Dictionary<int, Template>();
var result = new List<Template>();
// The root is "visited" from the start so a cycle back to it does not re-expand it; it is
// loaded (if it exists) by the chain walk below.
var expanded = new HashSet<int> { templateId };
// 1. Derived subtree, one children query per depth.
var frontier = new List<int> { templateId };
while (frontier.Count > 0)
{
cancellationToken.ThrowIfCancellationRequested();
var children = await loadChildren(frontier, cancellationToken).ConfigureAwait(false);
var next = new List<int>();
foreach (var child in children)
{
if (child.Id == templateId || !loaded.TryAdd(child.Id, child))
continue;
result.Add(child);
if (expanded.Add(child.Id))
next.Add(child.Id);
}
frontier = next;
}
if (result.Count == 0)
return result; // no descendants: nothing to reconcile, so load nothing else
// 2. The root's own inheritance chain, one query per level. A level already loaded as
// part of the subtree means a parent cycle — the chain ends there.
var requested = new HashSet<int>();
int? nextId = templateId;
while (nextId is int id && requested.Add(id))
{
cancellationToken.ThrowIfCancellationRequested();
if (loaded.TryGetValue(id, out var already))
{
nextId = already.ParentTemplateId;
continue;
}
var batch = await loadByIds(new[] { id }, cancellationToken).ConfigureAwait(false);
var template = batch.FirstOrDefault(t => t.Id == id);
if (template is null)
break; // dangling parent edge (or an unknown root)
loaded[id] = template;
result.Add(template);
nextId = template.ParentTemplateId;
}
return result;
}
}
@@ -80,14 +80,7 @@ public class TemplateEngineRepository : ITemplateEngineRepository
/// <inheritdoc /> /// <inheritdoc />
public async Task<IReadOnlyList<Template>> GetAllTemplatesAsync(CancellationToken cancellationToken = default) public async Task<IReadOnlyList<Template>> GetAllTemplatesAsync(CancellationToken cancellationToken = default)
{ {
return await _context.Templates return await FullTemplateGraph().ToListAsync(cancellationToken);
.Include(t => t.Attributes)
.Include(t => t.Alarms)
.Include(t => t.Scripts)
.Include(t => t.Compositions)
.Include(t => t.NativeAlarmSources)
.AsSplitQuery()
.ToListAsync(cancellationToken);
} }
/// <inheritdoc /> /// <inheritdoc />
@@ -143,6 +136,44 @@ public class TemplateEngineRepository : ITemplateEngineRepository
cancellationToken); cancellationToken);
} }
/// <inheritdoc />
// One tracked, fully-included query per subtree depth (children of the frontier) and one per
// ancestor level — the same entity shape as GetAllTemplatesAsync, because the reconcile
// writes through these entities and compares script bodies (Gitea #86).
public Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default)
{
return TemplateReconcileClosure.CollectAsync(
templateId,
async (parentIds, ct) =>
{
var idList = parentIds.ToList();
return await FullTemplateGraph()
.Where(t => t.ParentTemplateId != null && idList.Contains(t.ParentTemplateId.Value))
.ToListAsync(ct);
},
async (ids, ct) =>
{
var idList = ids.ToList();
return await FullTemplateGraph()
.Where(t => idList.Contains(t.Id))
.ToListAsync(ct);
},
cancellationToken);
}
// The tracked, fully-included template shape shared by GetAllTemplatesAsync and
// GetTemplateReconcileClosureAsync.
private IQueryable<Template> FullTemplateGraph()
{
return _context.Templates
.Include(t => t.Attributes)
.Include(t => t.Alarms)
.Include(t => t.Scripts)
.Include(t => t.Compositions)
.Include(t => t.NativeAlarmSources)
.AsSplitQuery();
}
// The analysis projection shared by GetAllTemplatesForAnalysisAsync and // The analysis projection shared by GetAllTemplatesForAnalysisAsync and
// GetTemplateAnalysisClosureAsync (see the note on the former for why it is untracked and // GetTemplateAnalysisClosureAsync (see the note on the former for why it is untracked and
// drops TemplateScript.Code). // drops TemplateScript.Code).
@@ -1268,18 +1268,14 @@ public class TemplateService
/// <param name="user">Username for the audit row.</param> /// <param name="user">Username for the audit row.</param>
/// <param name="cancellationToken">Cancellation token.</param> /// <param name="cancellationToken">Cancellation token.</param>
/// <param name="loadedTemplates"> /// <param name="loadedTemplates">
/// The full, TRACKED template graph if the caller already has one. Reconciling /// A TRACKED template graph the caller already has, containing at least the
/// writes through these entities and compares script bodies, so this must be /// template's derived subtree and inheritance chain. Reconciling writes through
/// the tracked <c>GetAllTemplatesAsync</c> shape — never the no-tracking, /// these entities and compares script bodies, so this must be the tracked
/// body-free analysis projection. When <see langword="null"/> the graph is /// <c>GetAllTemplatesAsync</c> shape — never the no-tracking, body-free analysis
/// loaded here. /// projection. When <see langword="null"/> only the reconcile closure (the
/// /// subtree plus the template's chain, Gitea #86) is loaded here — never the
/// <para> /// whole library, and nothing beyond one probe query when the template has no
/// Passing it matters because every member Add/Update/Delete on /// children.
/// <see cref="TemplateService"/> used to load the whole graph once for its own
/// checks and then again inside this method. Handing the graph over collapses
/// that to one load per operation.
/// </para>
/// </param> /// </param>
/// <returns>A task that represents the asynchronous operation.</returns> /// <returns>A task that represents the asynchronous operation.</returns>
public async Task ReconcileDescendantsAsync( public async Task ReconcileDescendantsAsync(
@@ -1288,7 +1284,12 @@ public class TemplateService
CancellationToken cancellationToken = default, CancellationToken cancellationToken = default,
IReadOnlyList<Template>? loadedTemplates = null) IReadOnlyList<Template>? loadedTemplates = null)
{ {
var all = loadedTemplates ?? await _repository.GetAllTemplatesAsync(cancellationToken); // The descendants' chains run through templateId and on up its own chain, and
// the resolver is inheritance-only, so the subtree + the template's chain is
// everything the reconcile reads (#86).
var all = loadedTemplates
?? await _repository.GetTemplateReconcileClosureAsync(templateId, cancellationToken)
?? Array.Empty<Template>();
var descendants = GetDescendantIdsBreadthFirst(templateId, all); var descendants = GetDescendantIdsBreadthFirst(templateId, all);
if (descendants.Count == 0) if (descendants.Count == 0)
return; return;
@@ -0,0 +1,125 @@
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests;
/// <summary>
/// Gitea #86: <see cref="TemplateEngineRepository.GetTemplateReconcileClosureAsync"/> reads only a
/// template's derived subtree + inheritance chain — tracked and fully included, the same shape as
/// <see cref="TemplateEngineRepository.GetAllTemplatesAsync"/> — instead of the whole library.
/// Asserted against real SQL on SQLite with the reader commands recorded.
/// </summary>
public class TemplateReconcileClosureRepositoryTests : IDisposable
{
private const int UnrelatedTemplates = 40;
private const string Body = "/* script body the reconcile compares and copies */";
private readonly SqlReaderCommandRecorder _recorder = new();
private readonly ScadaBridgeDbContext _context;
private readonly TemplateEngineRepository _repository;
/// <summary>Initializes the test class with an in-memory context and repository.</summary>
public TemplateReconcileClosureRepositoryTests()
{
_context = SqliteTestHelper.CreateInMemoryContext(_recorder);
_repository = new TemplateEngineRepository(_context, new TemplateGraphWatermark());
}
/// <summary>Releases the resources held by this test class.</summary>
public void Dispose()
{
_context.Database.CloseConnection();
_context.Dispose();
}
private async Task<Template> AddAsync(string name, int? parentId = null)
{
var t = new Template(name) { ParentTemplateId = parentId };
t.Attributes.Add(new TemplateAttribute($"{name}Attr1"));
t.Attributes.Add(new TemplateAttribute($"{name}Attr2"));
t.Scripts.Add(new TemplateScript($"{name}Script", Body));
_context.Templates.Add(t);
await _context.SaveChangesAsync();
return t;
}
private sealed record Seeded(int Root, int Base, int Mid, int Leaf, int Leaf3, int Sibling, int Module);
private async Task<Seeded> SeedAsync()
{
var root = await AddAsync("Root");
var @base = await AddAsync("Base", root.Id);
var mid = await AddAsync("Mid", @base.Id);
var leaf = await AddAsync("Leaf", mid.Id);
var leaf3 = await AddAsync("Leaf3", @base.Id);
var sibling = await AddAsync("Sibling", root.Id); // Base's sibling: not in Base's closure
var module = await AddAsync("Module");
_context.Set<TemplateComposition>().Add(
new TemplateComposition("m") { TemplateId = @base.Id, ComposedTemplateId = module.Id });
await _context.SaveChangesAsync();
for (var i = 0; i < UnrelatedTemplates; i++)
await AddAsync($"Unrelated{i}");
_context.ChangeTracker.Clear();
_recorder.Reset();
return new Seeded(root.Id, @base.Id, mid.Id, leaf.Id, leaf3.Id, sibling.Id, module.Id);
}
/// <summary>
/// The closure is exactly the subtree + chain — 5 templates instead of 47 — loaded tracked
/// with script bodies and every child collection, in one load per subtree depth + chain level.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_ReadsOnlySubtreeAndChain_TrackedAndFullyIncluded()
{
var ids = await SeedAsync();
var whole = await _repository.GetAllTemplatesAsync();
var wholeQueries = _recorder.Commands.Count;
_context.ChangeTracker.Clear();
_recorder.Reset();
var closure = await _repository.GetTemplateReconcileClosureAsync(ids.Base);
var closureQueries = _recorder.Commands.Count;
Assert.Equal(
new[] { ids.Root, ids.Base, ids.Mid, ids.Leaf, ids.Leaf3 }.OrderBy(i => i),
closure.Select(t => t.Id).OrderBy(i => i));
Assert.Equal(7 + UnrelatedTemplates, whole.Count);
Assert.Equal(10, closure.Sum(t => t.Attributes.Count));
// Four full loads — children of {Base} and of {Mid, Leaf3}; chain Base, Root — each one
// split-query set like the whole-graph load, plus the empty children-of-{Leaf} probe whose
// includes never run. More round trips than the single whole-graph load, but each is
// bounded by the closure, not the library.
Assert.Equal(4 * wholeQueries + 1, closureQueries);
// Same shape the reconcile writes through: tracked, script bodies present, compositions loaded.
Assert.Equal(5, _context.ChangeTracker.Entries<Template>().Count());
Assert.All(closure.SelectMany(t => t.Scripts), s => Assert.Equal(Body, s.Code));
Assert.Single(closure.Single(t => t.Id == ids.Base).Compositions);
Assert.DoesNotContain(closure, t => t.Id == ids.Module || t.Id == ids.Sibling);
}
/// <summary>
/// The common case — a template nobody derives from — costs one children probe that returns
/// no rows, and loads and tracks nothing.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_NoDescendants_SingleProbe_LoadsNothing()
{
var ids = await SeedAsync();
var closure = await _repository.GetTemplateReconcileClosureAsync(ids.Leaf);
Assert.Empty(closure);
Assert.Single(_recorder.Commands);
Assert.Empty(_context.ChangeTracker.Entries());
Assert.Empty(await _repository.GetTemplateReconcileClosureAsync(987654));
}
}
@@ -37,4 +37,33 @@ internal static class AnalysisClosureMock
ct); ct);
}); });
} }
/// <summary>
/// Gitea #86: the descendant reconcile now asks for the reconcile closure (derived subtree +
/// the template's chain) instead of <c>GetAllTemplatesAsync</c>. Wires it to run the real
/// closure walk over whatever the full tracked-graph setup returns, so every existing
/// propagation scenario exercises the service against the reduced set.
/// </summary>
/// <param name="repoMock">The repository mock.</param>
/// <param name="closureRequests">Optional sink recording each requested closure root.</param>
public static void ServeReconcileClosureFromFullGraph(
this Mock<ITemplateEngineRepository> repoMock,
List<int>? closureRequests = null)
{
async Task<IReadOnlyList<Template>> All(CancellationToken token) =>
await repoMock.Object.GetAllTemplatesAsync(token) ?? Array.Empty<Template>();
repoMock
.Setup(r => r.GetTemplateReconcileClosureAsync(It.IsAny<int>(), It.IsAny<CancellationToken>()))
.Returns((int templateId, CancellationToken ct) =>
{
closureRequests?.Add(templateId);
return TemplateReconcileClosure.CollectAsync(
templateId,
async (parentIds, token) => (await All(token))
.Where(t => t.ParentTemplateId is int p && parentIds.Contains(p)).ToList(),
async (ids, token) => (await All(token)).Where(t => ids.Contains(t.Id)).ToList(),
ct);
});
}
} }
@@ -24,6 +24,7 @@ public class TemplateAuditShapeTests
{ {
_repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1); _repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
_repoMock.ServeClosureFromFullGraph(); _repoMock.ServeClosureFromFullGraph();
_repoMock.ServeReconcileClosureFromFullGraph();
_auditMock _auditMock
.Setup(a => a.LogAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(), .Setup(a => a.LogAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<object?>(), It.IsAny<CancellationToken>())) It.IsAny<string>(), It.IsAny<string>(), It.IsAny<object?>(), It.IsAny<CancellationToken>()))
@@ -0,0 +1,314 @@
using System.Text.Json;
using System.Text.Json.Serialization;
using Moq;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
namespace ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests;
/// <summary>
/// Gitea #86: <c>TemplateService.ReconcileDescendantsAsync</c> now reconciles against the
/// template's derived subtree + inheritance chain instead of the whole tracked library. These
/// tests pin that the closure is (a) sufficient — for every template of a graph with a deep
/// subtree, an ancestor chain above the edited template, a composed module, unrelated look-alike
/// trees, a parent cycle and a dangling parent, the reconcile issues exactly the same writes and
/// the resolver answers identically for every descendant — and (b) actually smaller, loading
/// nothing beyond one probe when there are no descendants.
/// </summary>
public class TemplateReconcileClosureTests
{
private const int Root = 1, Base = 2, Mid = 3, Leaf1 = 4, Leaf2 = 5, Leaf3 = 6, GreatLeaf = 7, Mod = 8;
private const int U1 = 9, U2 = 10, CycA = 11, CycB = 12, CycC = 13, Dang = 14, DangKid = 15;
private static readonly JsonSerializerOptions Json = new()
{
ReferenceHandler = ReferenceHandler.IgnoreCycles,
WriteIndented = false,
};
/// <summary>Builds a fresh, independently mutable copy of the fixture graph.</summary>
private static List<Template> BuildGraph()
{
var memberId = 1000;
Template T(int id, string name, int? parent = null)
=> new(name) { Id = id, ParentTemplateId = parent };
void Attr(Template t, string name, string? value, bool inherited = false, bool locked = false) =>
t.Attributes.Add(new TemplateAttribute(name)
{
Id = memberId++, TemplateId = t.Id, Value = value, DataType = DataType.String,
IsInherited = inherited, IsLocked = locked,
});
void Script(Template t, string name, string code, bool inherited = false) =>
t.Scripts.Add(new TemplateScript(name, code)
{
Id = memberId++, TemplateId = t.Id, TriggerType = "Startup", IsInherited = inherited,
});
void Alarm(Template t, string name, int priority, bool inherited = false) =>
t.Alarms.Add(new TemplateAlarm(name)
{
Id = memberId++, TemplateId = t.Id, PriorityLevel = priority,
TriggerType = AlarmTriggerType.ValueMatch, IsInherited = inherited,
});
var root = T(Root, "Root");
Attr(root, "R", "root"); // inherited from ABOVE the edited template
Script(root, "Init", "root-body");
var @base = T(Base, "Base", Root);
Attr(@base, "A", "1");
Attr(@base, "L", "locked", locked: true);
Attr(@base, "New", "added-after-derivation"); // no placeholder anywhere yet
Script(@base, "Calc", "base-body-v2");
Alarm(@base, "Hi", 700);
@base.Compositions.Add(new TemplateComposition("m") { Id = memberId++, TemplateId = Base, ComposedTemplateId = Mod });
var mid = T(Mid, "Mid", Base);
Attr(mid, "A", "mid-override"); // own override, never touched
Attr(mid, "R", "stale-root", inherited: true);
Attr(mid, "L", "locked", inherited: true);
Script(mid, "Calc", "base-body-v1", inherited: true); // stale body
Alarm(mid, "Hi", 500, inherited: true); // stale priority
var leaf1 = T(Leaf1, "Leaf1", Mid);
Attr(leaf1, "A", "1", inherited: true); // stale: Mid overrides A
Attr(leaf1, "Gone", "orphan", inherited: true); // orphan placeholder
var leaf2 = T(Leaf2, "Leaf2", Mid);
Attr(leaf2, "L", "override-attempt"); // locked upstream
var leaf3 = T(Leaf3, "Leaf3", Base);
var greatLeaf = T(GreatLeaf, "GreatLeaf", Leaf1);
Script(greatLeaf, "Calc", "own-body");
var mod = T(Mod, "Mod");
Attr(mod, "ModOnly", "x");
// Unrelated look-alike tree: same member names, must never be loaded.
var u1 = T(U1, "U1");
Attr(u1, "A", "u");
Attr(u1, "New", "u");
var u2 = T(U2, "U2", U1);
Attr(u2, "A", "stale", inherited: true);
// Parent cycle with a hanging child, and a dangling parent edge.
var cycA = T(CycA, "CycA", CycB);
Attr(cycA, "CA", "a");
var cycB = T(CycB, "CycB", CycA);
Attr(cycB, "CB", "b");
var cycC = T(CycC, "CycC", CycA);
var dang = T(Dang, "Dang", 998);
Attr(dang, "D", "d");
var dangKid = T(DangKid, "DangKid", Dang);
return new List<Template>
{
root, @base, mid, leaf1, leaf2, leaf3, greatLeaf, mod, u1, u2, cycA, cycB, cycC, dang, dangKid,
};
}
private static Task<IReadOnlyList<Template>> Closure(
IReadOnlyList<Template> graph,
int id,
List<string>? calls = null) =>
TemplateReconcileClosure.CollectAsync(
id,
(parentIds, _) =>
{
calls?.Add("children:" + string.Join(",", parentIds.OrderBy(i => i)));
return Task.FromResult<IReadOnlyList<Template>>(
graph.Where(t => t.ParentTemplateId is int p && parentIds.Contains(p)).ToList());
},
(ids, _) =>
{
calls?.Add("ids:" + string.Join(",", ids.OrderBy(i => i)));
return Task.FromResult<IReadOnlyList<Template>>(graph.Where(t => ids.Contains(t.Id)).ToList());
});
// Independent re-statement of the service's descendant walk over a given template list.
private static List<int> Descendants(int rootId, IReadOnlyList<Template> all)
{
var result = new List<int>();
var seen = new HashSet<int> { rootId };
var queue = new Queue<int>(new[] { rootId });
while (queue.Count > 0)
{
var id = queue.Dequeue();
foreach (var child in all.Where(t => t.ParentTemplateId == id))
{
if (seen.Add(child.Id))
{
result.Add(child.Id);
queue.Enqueue(child.Id);
}
}
}
return result;
}
/// <summary>Every fixture template id.</summary>
/// <returns>One row per template.</returns>
public static IEnumerable<object[]> EveryTemplate() => BuildGraph().Select(t => new object[] { t.Id });
/// <summary>
/// The closure holds the same descendant set as the whole graph, and the inheritance resolver
/// gives every descendant the identical resolved member set from it.
/// </summary>
/// <param name="templateId">The edited template.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[MemberData(nameof(EveryTemplate))]
public async Task Resolver_AnswersIdenticallyForEveryDescendant(int templateId)
{
var graph = BuildGraph();
var closure = await Closure(graph, templateId);
var wholeDescendants = Descendants(templateId, graph);
Assert.Equal(wholeDescendants.OrderBy(i => i), Descendants(templateId, closure).OrderBy(i => i));
foreach (var d in wholeDescendants)
{
Assert.Equal(
JsonSerializer.Serialize(TemplateInheritanceResolver.Resolve(d, graph), Json),
JsonSerializer.Serialize(TemplateInheritanceResolver.Resolve(d, closure), Json));
}
}
/// <summary>
/// End to end through <see cref="TemplateService.ReconcileDescendantsAsync"/>: the closure-backed
/// reconcile issues exactly the writes (adds / updates / deletes / audit / save) the old
/// whole-graph reconcile did, for every template of the fixture.
/// </summary>
/// <param name="templateId">The edited template.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[MemberData(nameof(EveryTemplate))]
public async Task Reconcile_IssuesIdenticalWrites_FromClosureAndWholeGraph(int templateId)
{
var expected = await RecordReconcileWrites(templateId, useClosure: false);
var actual = await RecordReconcileWrites(templateId, useClosure: true);
Assert.Equal(expected, actual);
}
/// <summary>The equivalence above is not vacuous: editing Base really adds, updates and removes rows.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Fixture_ReconcilingBase_WritesAddsUpdatesAndDeletes()
{
var writes = await RecordReconcileWrites(Base, useClosure: true);
Assert.Contains(writes, w => w.StartsWith("AddTemplateAttributeAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAttributeAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateScriptAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAlarmAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("DeleteTemplateAttributeAsync", StringComparison.Ordinal));
// Root's member reaches Mid only through Base's own chain — the ancestor walk is load-bearing.
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAttributeAsync", StringComparison.Ordinal)
&& w.Contains("\"Value\":\"root\"", StringComparison.Ordinal));
}
/// <summary>The closure is the subtree plus the chain — no module, no unrelated tree, no siblings.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_IsSubtreePlusChainOnly()
{
var graph = BuildGraph();
var closure = await Closure(graph, Base);
Assert.Equal(
new[] { Root, Base, Mid, Leaf1, Leaf2, Leaf3, GreatLeaf },
closure.Select(t => t.Id).OrderBy(i => i));
Assert.True(closure.Count < graph.Count);
// A mid-tree edit loads its own subtree and chain, not its siblings.
Assert.Equal(
new[] { Root, Base, Mid, Leaf1, Leaf2, GreatLeaf },
(await Closure(graph, Mid)).Select(t => t.Id).OrderBy(i => i));
}
/// <summary>A template with no children costs one children probe and loads nothing.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_NoDescendants_OneProbeAndEmpty()
{
var calls = new List<string>();
Assert.Empty(await Closure(BuildGraph(), Leaf2, calls));
Assert.Equal(new[] { "children:5" }, calls);
Assert.Empty(await Closure(BuildGraph(), 424242));
}
/// <summary>One children batch per subtree depth, one id batch per ancestor level, no id twice.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_LoadsOneBatchPerDepth()
{
var calls = new List<string>();
await Closure(BuildGraph(), Base, calls);
Assert.Equal(
new[] { "children:2", "children:3,6", "children:4,5", "children:7", "ids:2", "ids:1" },
calls);
}
/// <summary>Parent cycles terminate and a dangling parent ends the chain.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_TerminatesOnCyclesAndDanglingParents()
{
var graph = BuildGraph();
Assert.Equal(new[] { CycA, CycB, CycC }, (await Closure(graph, CycA)).Select(t => t.Id).OrderBy(i => i));
Assert.Equal(new[] { Dang, DangKid }, (await Closure(graph, Dang)).Select(t => t.Id).OrderBy(i => i));
}
/// <summary>
/// Runs the reconcile for <paramref name="templateId"/> on a fresh copy of the graph and returns
/// every write the service issued, rendered as sorted strings (sibling order is not part of the
/// contract — each descendant is reconciled independently).
/// </summary>
private static async Task<List<string>> RecordReconcileWrites(int templateId, bool useClosure)
{
var graph = BuildGraph();
var repo = new Mock<ITemplateEngineRepository>();
var audit = new Mock<IAuditService>();
repo.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
if (useClosure)
{
repo.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()))
.ThrowsAsync(new InvalidOperationException("reconcile loaded the whole tracked graph"));
repo.Setup(r => r.GetTemplateReconcileClosureAsync(templateId, It.IsAny<CancellationToken>()))
.Returns((int id, CancellationToken _) => Closure(graph, id));
}
else
{
repo.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(graph);
}
var service = new TemplateService(repo.Object, audit.Object);
if (useClosure)
await service.ReconcileDescendantsAsync(templateId, "tester");
else
await service.ReconcileDescendantsAsync(templateId, "tester", default, graph);
static string Render(Moq.IInvocation i) =>
i.Method.Name + "(" + string.Join(", ", i.Arguments
.Where(a => a is not CancellationToken)
.Select(a => JsonSerializer.Serialize(a, Json))) + ")";
var writes = repo.Invocations
.Where(i => i.Method.Name.StartsWith("Add", StringComparison.Ordinal)
|| i.Method.Name.StartsWith("Update", StringComparison.Ordinal)
|| i.Method.Name.StartsWith("Delete", StringComparison.Ordinal)
|| i.Method.Name == nameof(ITemplateEngineRepository.SaveChangesAsync))
.Select(Render)
.Concat(audit.Invocations.Select(Render))
.OrderBy(s => s, StringComparer.Ordinal)
.ToList();
return writes;
}
}
@@ -5,6 +5,7 @@ using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories; using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services; using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums; using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
namespace ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests; namespace ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests;
@@ -22,6 +23,7 @@ public class TemplateServiceTests
_auditMock = new Mock<IAuditService>(); _auditMock = new Mock<IAuditService>();
_repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1); _repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
_repoMock.ServeClosureFromFullGraph(_closureRequests); _repoMock.ServeClosureFromFullGraph(_closureRequests);
_repoMock.ServeReconcileClosureFromFullGraph();
_service = new TemplateService(_repoMock.Object, _auditMock.Object); _service = new TemplateService(_repoMock.Object, _auditMock.Object);
} }
@@ -2405,4 +2407,66 @@ public class TemplateServiceTests
_repoMock.Verify(r => r.GetTemplateAnalysisClosureAsync(2, It.IsAny<CancellationToken>()), Times.Exactly(3)); _repoMock.Verify(r => r.GetTemplateAnalysisClosureAsync(2, It.IsAny<CancellationToken>()), Times.Exactly(3));
_repoMock.Verify(r => r.GetAllTemplatesForAnalysisAsync(It.IsAny<CancellationToken>()), Times.Never); _repoMock.Verify(r => r.GetAllTemplatesForAnalysisAsync(It.IsAny<CancellationToken>()), Times.Never);
} }
// ========================================================================
// Gitea #86: descendant reconcile loads only the subtree + chain
// ========================================================================
/// <summary>
/// A base member add/update/delete propagates to the derived subtree from the reconcile
/// closure alone: the whole tracked graph is wired to throw and is never loaded, while an
/// unrelated template sharing the member name is never touched.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task MemberMutations_PropagateFromReconcileClosure_NeverLoadWholeTrackedGraph()
{
var baseT = BaseTemplate(7, "Base");
baseT.Attributes.Add(new TemplateAttribute("A") { Id = 70, TemplateId = 7, DataType = DataType.String, Value = "1" });
var child = DerivedTemplate(8, "Child", 7);
child.Attributes.Add(new TemplateAttribute("A") { Id = 80, TemplateId = 8, DataType = DataType.String, Value = "1", IsInherited = true });
var grandChild = DerivedTemplate(9, "GrandChild", 8);
var unrelated = BaseTemplate(50, "Unrelated");
unrelated.Attributes.Add(new TemplateAttribute("A") { Id = 500, TemplateId = 50, DataType = DataType.String, Value = "x" });
var library = new List<Template> { unrelated, baseT, child, grandChild };
_repoMock.Setup(r => r.GetTemplateByIdAsync(7, It.IsAny<CancellationToken>())).ReturnsAsync(baseT);
_repoMock.Setup(r => r.GetTemplateAttributeByIdAsync(70, It.IsAny<CancellationToken>()))
.ReturnsAsync(baseT.Attributes.Single());
_repoMock.Setup(r => r.GetAllTemplatesForAnalysisAsync(It.IsAny<CancellationToken>())).ReturnsAsync(library);
_repoMock.Setup(r => r.GetTemplateReconcileClosureAsync(It.IsAny<int>(), It.IsAny<CancellationToken>()))
.Returns((int id, CancellationToken ct) => TemplateReconcileClosure.CollectAsync(
id,
(parents, _) => Task.FromResult<IReadOnlyList<Template>>(
library.Where(t => t.ParentTemplateId is int p && parents.Contains(p)).ToList()),
(ids, _) => Task.FromResult<IReadOnlyList<Template>>(library.Where(t => ids.Contains(t.Id)).ToList()),
ct));
_repoMock.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()))
.ThrowsAsync(new InvalidOperationException("reconcile loaded the whole tracked graph"));
var added = new List<TemplateAttribute>();
_repoMock.Setup(r => r.AddTemplateAttributeAsync(It.IsAny<TemplateAttribute>(), It.IsAny<CancellationToken>()))
.Callback<TemplateAttribute, CancellationToken>((a, _) =>
{
added.Add(a);
if (a.TemplateId == 7) baseT.Attributes.Add(a); // simulate persistence of the base's own member
})
.Returns(Task.CompletedTask);
var updated = new List<TemplateAttribute>();
_repoMock.Setup(r => r.UpdateTemplateAttributeAsync(It.IsAny<TemplateAttribute>(), It.IsAny<CancellationToken>()))
.Callback<TemplateAttribute, CancellationToken>((a, _) => updated.Add(a))
.Returns(Task.CompletedTask);
var add = await _service.AddAttributeAsync(7, new TemplateAttribute("C") { DataType = DataType.String, Value = "c" }, "admin");
var update = await _service.UpdateAttributeAsync(70, new TemplateAttribute("A") { DataType = DataType.String, Value = "2" }, "admin");
Assert.True(add.IsSuccess, add.IsFailure ? add.Error : null);
Assert.True(update.IsSuccess, update.IsFailure ? update.Error : null);
Assert.Contains(added, a => a.TemplateId == 8 && a.Name == "C" && a.IsInherited);
Assert.Contains(added, a => a.TemplateId == 9 && a.Name == "C" && a.IsInherited); // two levels down
Assert.Contains(updated, a => a.TemplateId == 8 && a.Name == "A" && a.Value == "2");
Assert.DoesNotContain(added.Concat(updated), a => a.TemplateId == 50);
_repoMock.Verify(r => r.GetTemplateReconcileClosureAsync(7, It.IsAny<CancellationToken>()), Times.Exactly(2));
_repoMock.Verify(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()), Times.Never);
}
} }