perf(templates): descendant reconcile loads only the subtree + chain (ScadaBridge#86)

TemplateService.ReconcileDescendantsAsync - run after every base member
add/update/delete, template and native-alarm-source alike - loaded the whole
tracked template graph (every template, member and script body) on each call,
which after #66 was the dominant cost of those paths. It now calls the new
ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked,
fully-included shape as GetAllTemplatesAsync, restricted to the edited
template's derived subtree plus its own inheritance chain. The reconcile is
inheritance-only (TemplateInheritanceResolver never follows compositions) and
every descendant's chain runs through the edited template, so that set is
everything it reads.

The walk (Commons TemplateReconcileClosure, the descendant-direction twin of
#66's TemplateAnalysisClosure) loads one children batch per subtree depth and
one batch per ancestor level, terminates on parent cycles and stops at
dangling parents. A template with no children - the common case - costs one
empty probe and loads nothing.

Evidence:
- TemplateReconcileClosureTests: for every template of a graph with a deep
  subtree, an ancestor above the edited template, a composed module,
  look-alike unrelated trees, a parent cycle and a dangling parent, the
  closure-backed reconcile issues exactly the writes (add/update/delete,
  audit, save) of the old whole-graph reconcile, and the resolver answers
  identically for every descendant. Fails with the old service (whole-graph
  load wired to throw) and with the ancestor walk removed (mutation-checked).
- TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates
  / 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs
  one probe query and tracks nothing.
- TemplateServiceTests: attribute add + update propagate two levels down with
  GetAllTemplatesAsync wired to throw; existing propagation tests now run
  against the closure via ServeReconcileClosureFromFullGraph.

ResyncInheritedMembersAsync (operator action) still reads the whole graph.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 13:01:56 -05:00
parent 0eac956d25
commit c29b99c63b
10 changed files with 718 additions and 31 deletions
+15 -10
View File
@@ -219,17 +219,22 @@ names, locks and edges: `InstanceService.SetAlarmOverrideAsync` (resolving one a
name) and `TemplateDeletionService.CanDeleteTemplateAsync` (inheritor / derived /
composer checks). `TemplateFolderService.DeleteFolderAsync`, which only counts
templates per `FolderId`, reads the lighter `GetTemplateStubsAsync` identity projection
(review 2026-10-21 04/P6, 08/P1). Template member add/update paths still load the whole
analysis projection for collision detection; narrowing that to the edited template's
chain + composition closure needs a closure-scoped repository read that does not exist
yet.
(review 2026-10-21 04/P6, 08/P1). Template member add/update paths read only the
edited template's inheritance chain + composition closure through
`GetTemplateAnalysisClosureAsync` (the same projection, one query per closure depth —
Gitea #66).
The tracked, body-bearing `GetAllTemplatesAsync` remains the read for the two paths
that genuinely need it — the inheritance reconciler (which compares and copies
script bodies and writes through the loaded entities) and the flattener (which must
observe rows an in-flight bundle import has staged on the shared change tracker).
`ReconcileDescendantsAsync` additionally accepts an already-loaded tracked graph so
a caller holding one does not force a second load.
The tracked, body-bearing shape remains the read for the two paths that genuinely need
it — the inheritance reconciler (which compares and copies script bodies and writes
through the loaded entities) and the flattener (which must observe rows an in-flight
bundle import has staged on the shared change tracker). The automatic post-mutation
`ReconcileDescendantsAsync` reads only `GetTemplateReconcileClosureAsync` — the edited
template's derived subtree plus its own inheritance chain, tracked and fully included,
one load per subtree depth and chain level (Gitea #86). The reconcile is
inheritance-only, so every descendant's chain lies inside that set; a template with no
children costs a single empty probe query. It still accepts an already-loaded tracked
graph from a caller holding one. The operator `ResyncInheritedMembersAsync` and the
flattener keep reading the whole tracked graph via `GetAllTemplatesAsync`.
### Native Alarm Source Resolution
@@ -69,6 +69,20 @@ public interface ITemplateEngineRepository
/// <returns>A task that resolves to the closure templates without script bodies.</returns>
Task<IReadOnlyList<Template>> GetTemplateAnalysisClosureAsync(int templateId, CancellationToken cancellationToken = default);
/// <summary>
/// Returns the TRACKED, fully-loaded templates (the <see cref="GetAllTemplatesAsync"/> shape:
/// attributes, alarms, scripts with bodies, compositions, native alarm sources) that the
/// inherited-member reconcile of <paramref name="templateId"/>'s descendants reads: its whole
/// derived subtree plus its own inheritance chain (see
/// <see cref="ZB.MOM.WW.ScadaBridge.Commons.Types.Templates.TemplateReconcileClosure"/>).
/// Backs <c>TemplateService.ReconcileDescendantsAsync</c> (Gitea #86), which previously
/// loaded every template on each base-member add/update/delete. Returns an empty list when the
/// template has no children — there is nothing to reconcile — after a single probe query.
/// </summary>
/// <param name="templateId">The template whose member set changed.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>A task that resolves to the subtree followed by the template's chain.</returns>
Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default);
/// <summary>
/// Returns an untracked identity/edge projection of every template (no children), ordered by name.
/// </summary>
/// <param name="cancellationToken">Cancellation token.</param>
@@ -0,0 +1,103 @@
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
/// <summary>
/// Computes the set of templates the inherited-member reconcile reads when a base member of one
/// template changes (Gitea #86): the template's whole <b>derived subtree</b> (children,
/// grandchildren, …) plus the template's own <b>inheritance chain</b> (itself, its parent,
/// grandparent, …).
///
/// <para>
/// This is the descendant-direction counterpart of <see cref="TemplateAnalysisClosure"/> (#66).
/// <c>TemplateService.ReconcileDescendantsAsync</c> reconciles every descendant <c>D</c> by
/// resolving <c>D</c>'s inheritance chain (<c>TemplateInheritanceResolver.Resolve</c>), which is
/// inheritance only — compositions are never followed. <c>D</c>'s chain is the path from
/// <c>D</c> up to the edited template (all inside the subtree) followed by the edited template's
/// own chain, and every member winner originates on that chain. So the subtree plus the root's
/// chain is everything the reconcile can read; the rest of the library is irrelevant.
/// </para>
///
/// <para>
/// The walk is breadth-first by batch: one <c>loadChildren</c> call per subtree depth and one
/// <c>loadByIds</c> call per ancestor level, so a store-backed loader issues one query per
/// depth rather than one per template. When the template has no children — the common case —
/// the walk stops after the single children probe and returns an empty list, because there is
/// nothing to reconcile. Cycles terminate (each id is loaded at most once) and a dangling parent
/// edge simply ends the chain, exactly as the resolver's chain builder does.
/// </para>
/// </summary>
public static class TemplateReconcileClosure
{
/// <summary>
/// Collects the derived subtree and inheritance chain of <paramref name="templateId"/>.
/// </summary>
/// <param name="templateId">The template whose member set changed.</param>
/// <param name="loadChildren">Loads every template whose parent is one of the given ids.</param>
/// <param name="loadByIds">Loads the templates with the given ids (unknown ids are omitted).</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// The subtree (breadth-first) followed by the root's chain, each template once; an empty
/// list when the template has no children.
/// </returns>
public static async Task<IReadOnlyList<Template>> CollectAsync(
int templateId,
Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadChildren,
Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadByIds,
CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(loadChildren);
ArgumentNullException.ThrowIfNull(loadByIds);
var loaded = new Dictionary<int, Template>();
var result = new List<Template>();
// The root is "visited" from the start so a cycle back to it does not re-expand it; it is
// loaded (if it exists) by the chain walk below.
var expanded = new HashSet<int> { templateId };
// 1. Derived subtree, one children query per depth.
var frontier = new List<int> { templateId };
while (frontier.Count > 0)
{
cancellationToken.ThrowIfCancellationRequested();
var children = await loadChildren(frontier, cancellationToken).ConfigureAwait(false);
var next = new List<int>();
foreach (var child in children)
{
if (child.Id == templateId || !loaded.TryAdd(child.Id, child))
continue;
result.Add(child);
if (expanded.Add(child.Id))
next.Add(child.Id);
}
frontier = next;
}
if (result.Count == 0)
return result; // no descendants: nothing to reconcile, so load nothing else
// 2. The root's own inheritance chain, one query per level. A level already loaded as
// part of the subtree means a parent cycle — the chain ends there.
var requested = new HashSet<int>();
int? nextId = templateId;
while (nextId is int id && requested.Add(id))
{
cancellationToken.ThrowIfCancellationRequested();
if (loaded.TryGetValue(id, out var already))
{
nextId = already.ParentTemplateId;
continue;
}
var batch = await loadByIds(new[] { id }, cancellationToken).ConfigureAwait(false);
var template = batch.FirstOrDefault(t => t.Id == id);
if (template is null)
break; // dangling parent edge (or an unknown root)
loaded[id] = template;
result.Add(template);
nextId = template.ParentTemplateId;
}
return result;
}
}
@@ -80,14 +80,7 @@ public class TemplateEngineRepository : ITemplateEngineRepository
/// <inheritdoc />
public async Task<IReadOnlyList<Template>> GetAllTemplatesAsync(CancellationToken cancellationToken = default)
{
return await _context.Templates
.Include(t => t.Attributes)
.Include(t => t.Alarms)
.Include(t => t.Scripts)
.Include(t => t.Compositions)
.Include(t => t.NativeAlarmSources)
.AsSplitQuery()
.ToListAsync(cancellationToken);
return await FullTemplateGraph().ToListAsync(cancellationToken);
}
/// <inheritdoc />
@@ -143,6 +136,44 @@ public class TemplateEngineRepository : ITemplateEngineRepository
cancellationToken);
}
/// <inheritdoc />
// One tracked, fully-included query per subtree depth (children of the frontier) and one per
// ancestor level — the same entity shape as GetAllTemplatesAsync, because the reconcile
// writes through these entities and compares script bodies (Gitea #86).
public Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default)
{
return TemplateReconcileClosure.CollectAsync(
templateId,
async (parentIds, ct) =>
{
var idList = parentIds.ToList();
return await FullTemplateGraph()
.Where(t => t.ParentTemplateId != null && idList.Contains(t.ParentTemplateId.Value))
.ToListAsync(ct);
},
async (ids, ct) =>
{
var idList = ids.ToList();
return await FullTemplateGraph()
.Where(t => idList.Contains(t.Id))
.ToListAsync(ct);
},
cancellationToken);
}
// The tracked, fully-included template shape shared by GetAllTemplatesAsync and
// GetTemplateReconcileClosureAsync.
private IQueryable<Template> FullTemplateGraph()
{
return _context.Templates
.Include(t => t.Attributes)
.Include(t => t.Alarms)
.Include(t => t.Scripts)
.Include(t => t.Compositions)
.Include(t => t.NativeAlarmSources)
.AsSplitQuery();
}
// The analysis projection shared by GetAllTemplatesForAnalysisAsync and
// GetTemplateAnalysisClosureAsync (see the note on the former for why it is untracked and
// drops TemplateScript.Code).
@@ -1268,18 +1268,14 @@ public class TemplateService
/// <param name="user">Username for the audit row.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <param name="loadedTemplates">
/// The full, TRACKED template graph if the caller already has one. Reconciling
/// writes through these entities and compares script bodies, so this must be
/// the tracked <c>GetAllTemplatesAsync</c> shape — never the no-tracking,
/// body-free analysis projection. When <see langword="null"/> the graph is
/// loaded here.
///
/// <para>
/// Passing it matters because every member Add/Update/Delete on
/// <see cref="TemplateService"/> used to load the whole graph once for its own
/// checks and then again inside this method. Handing the graph over collapses
/// that to one load per operation.
/// </para>
/// A TRACKED template graph the caller already has, containing at least the
/// template's derived subtree and inheritance chain. Reconciling writes through
/// these entities and compares script bodies, so this must be the tracked
/// <c>GetAllTemplatesAsync</c> shape — never the no-tracking, body-free analysis
/// projection. When <see langword="null"/> only the reconcile closure (the
/// subtree plus the template's chain, Gitea #86) is loaded here — never the
/// whole library, and nothing beyond one probe query when the template has no
/// children.
/// </param>
/// <returns>A task that represents the asynchronous operation.</returns>
public async Task ReconcileDescendantsAsync(
@@ -1288,7 +1284,12 @@ public class TemplateService
CancellationToken cancellationToken = default,
IReadOnlyList<Template>? loadedTemplates = null)
{
var all = loadedTemplates ?? await _repository.GetAllTemplatesAsync(cancellationToken);
// The descendants' chains run through templateId and on up its own chain, and
// the resolver is inheritance-only, so the subtree + the template's chain is
// everything the reconcile reads (#86).
var all = loadedTemplates
?? await _repository.GetTemplateReconcileClosureAsync(templateId, cancellationToken)
?? Array.Empty<Template>();
var descendants = GetDescendantIdsBreadthFirst(templateId, all);
if (descendants.Count == 0)
return;
@@ -0,0 +1,125 @@
using Microsoft.EntityFrameworkCore;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests;
/// <summary>
/// Gitea #86: <see cref="TemplateEngineRepository.GetTemplateReconcileClosureAsync"/> reads only a
/// template's derived subtree + inheritance chain — tracked and fully included, the same shape as
/// <see cref="TemplateEngineRepository.GetAllTemplatesAsync"/> — instead of the whole library.
/// Asserted against real SQL on SQLite with the reader commands recorded.
/// </summary>
public class TemplateReconcileClosureRepositoryTests : IDisposable
{
private const int UnrelatedTemplates = 40;
private const string Body = "/* script body the reconcile compares and copies */";
private readonly SqlReaderCommandRecorder _recorder = new();
private readonly ScadaBridgeDbContext _context;
private readonly TemplateEngineRepository _repository;
/// <summary>Initializes the test class with an in-memory context and repository.</summary>
public TemplateReconcileClosureRepositoryTests()
{
_context = SqliteTestHelper.CreateInMemoryContext(_recorder);
_repository = new TemplateEngineRepository(_context, new TemplateGraphWatermark());
}
/// <summary>Releases the resources held by this test class.</summary>
public void Dispose()
{
_context.Database.CloseConnection();
_context.Dispose();
}
private async Task<Template> AddAsync(string name, int? parentId = null)
{
var t = new Template(name) { ParentTemplateId = parentId };
t.Attributes.Add(new TemplateAttribute($"{name}Attr1"));
t.Attributes.Add(new TemplateAttribute($"{name}Attr2"));
t.Scripts.Add(new TemplateScript($"{name}Script", Body));
_context.Templates.Add(t);
await _context.SaveChangesAsync();
return t;
}
private sealed record Seeded(int Root, int Base, int Mid, int Leaf, int Leaf3, int Sibling, int Module);
private async Task<Seeded> SeedAsync()
{
var root = await AddAsync("Root");
var @base = await AddAsync("Base", root.Id);
var mid = await AddAsync("Mid", @base.Id);
var leaf = await AddAsync("Leaf", mid.Id);
var leaf3 = await AddAsync("Leaf3", @base.Id);
var sibling = await AddAsync("Sibling", root.Id); // Base's sibling: not in Base's closure
var module = await AddAsync("Module");
_context.Set<TemplateComposition>().Add(
new TemplateComposition("m") { TemplateId = @base.Id, ComposedTemplateId = module.Id });
await _context.SaveChangesAsync();
for (var i = 0; i < UnrelatedTemplates; i++)
await AddAsync($"Unrelated{i}");
_context.ChangeTracker.Clear();
_recorder.Reset();
return new Seeded(root.Id, @base.Id, mid.Id, leaf.Id, leaf3.Id, sibling.Id, module.Id);
}
/// <summary>
/// The closure is exactly the subtree + chain — 5 templates instead of 47 — loaded tracked
/// with script bodies and every child collection, in one load per subtree depth + chain level.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_ReadsOnlySubtreeAndChain_TrackedAndFullyIncluded()
{
var ids = await SeedAsync();
var whole = await _repository.GetAllTemplatesAsync();
var wholeQueries = _recorder.Commands.Count;
_context.ChangeTracker.Clear();
_recorder.Reset();
var closure = await _repository.GetTemplateReconcileClosureAsync(ids.Base);
var closureQueries = _recorder.Commands.Count;
Assert.Equal(
new[] { ids.Root, ids.Base, ids.Mid, ids.Leaf, ids.Leaf3 }.OrderBy(i => i),
closure.Select(t => t.Id).OrderBy(i => i));
Assert.Equal(7 + UnrelatedTemplates, whole.Count);
Assert.Equal(10, closure.Sum(t => t.Attributes.Count));
// Four full loads — children of {Base} and of {Mid, Leaf3}; chain Base, Root — each one
// split-query set like the whole-graph load, plus the empty children-of-{Leaf} probe whose
// includes never run. More round trips than the single whole-graph load, but each is
// bounded by the closure, not the library.
Assert.Equal(4 * wholeQueries + 1, closureQueries);
// Same shape the reconcile writes through: tracked, script bodies present, compositions loaded.
Assert.Equal(5, _context.ChangeTracker.Entries<Template>().Count());
Assert.All(closure.SelectMany(t => t.Scripts), s => Assert.Equal(Body, s.Code));
Assert.Single(closure.Single(t => t.Id == ids.Base).Compositions);
Assert.DoesNotContain(closure, t => t.Id == ids.Module || t.Id == ids.Sibling);
}
/// <summary>
/// The common case — a template nobody derives from — costs one children probe that returns
/// no rows, and loads and tracks nothing.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_NoDescendants_SingleProbe_LoadsNothing()
{
var ids = await SeedAsync();
var closure = await _repository.GetTemplateReconcileClosureAsync(ids.Leaf);
Assert.Empty(closure);
Assert.Single(_recorder.Commands);
Assert.Empty(_context.ChangeTracker.Entries());
Assert.Empty(await _repository.GetTemplateReconcileClosureAsync(987654));
}
}
@@ -37,4 +37,33 @@ internal static class AnalysisClosureMock
ct);
});
}
/// <summary>
/// Gitea #86: the descendant reconcile now asks for the reconcile closure (derived subtree +
/// the template's chain) instead of <c>GetAllTemplatesAsync</c>. Wires it to run the real
/// closure walk over whatever the full tracked-graph setup returns, so every existing
/// propagation scenario exercises the service against the reduced set.
/// </summary>
/// <param name="repoMock">The repository mock.</param>
/// <param name="closureRequests">Optional sink recording each requested closure root.</param>
public static void ServeReconcileClosureFromFullGraph(
this Mock<ITemplateEngineRepository> repoMock,
List<int>? closureRequests = null)
{
async Task<IReadOnlyList<Template>> All(CancellationToken token) =>
await repoMock.Object.GetAllTemplatesAsync(token) ?? Array.Empty<Template>();
repoMock
.Setup(r => r.GetTemplateReconcileClosureAsync(It.IsAny<int>(), It.IsAny<CancellationToken>()))
.Returns((int templateId, CancellationToken ct) =>
{
closureRequests?.Add(templateId);
return TemplateReconcileClosure.CollectAsync(
templateId,
async (parentIds, token) => (await All(token))
.Where(t => t.ParentTemplateId is int p && parentIds.Contains(p)).ToList(),
async (ids, token) => (await All(token)).Where(t => ids.Contains(t.Id)).ToList(),
ct);
});
}
}
@@ -24,6 +24,7 @@ public class TemplateAuditShapeTests
{
_repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
_repoMock.ServeClosureFromFullGraph();
_repoMock.ServeReconcileClosureFromFullGraph();
_auditMock
.Setup(a => a.LogAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<object?>(), It.IsAny<CancellationToken>()))
@@ -0,0 +1,314 @@
using System.Text.Json;
using System.Text.Json.Serialization;
using Moq;
using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
namespace ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests;
/// <summary>
/// Gitea #86: <c>TemplateService.ReconcileDescendantsAsync</c> now reconciles against the
/// template's derived subtree + inheritance chain instead of the whole tracked library. These
/// tests pin that the closure is (a) sufficient — for every template of a graph with a deep
/// subtree, an ancestor chain above the edited template, a composed module, unrelated look-alike
/// trees, a parent cycle and a dangling parent, the reconcile issues exactly the same writes and
/// the resolver answers identically for every descendant — and (b) actually smaller, loading
/// nothing beyond one probe when there are no descendants.
/// </summary>
public class TemplateReconcileClosureTests
{
private const int Root = 1, Base = 2, Mid = 3, Leaf1 = 4, Leaf2 = 5, Leaf3 = 6, GreatLeaf = 7, Mod = 8;
private const int U1 = 9, U2 = 10, CycA = 11, CycB = 12, CycC = 13, Dang = 14, DangKid = 15;
private static readonly JsonSerializerOptions Json = new()
{
ReferenceHandler = ReferenceHandler.IgnoreCycles,
WriteIndented = false,
};
/// <summary>Builds a fresh, independently mutable copy of the fixture graph.</summary>
private static List<Template> BuildGraph()
{
var memberId = 1000;
Template T(int id, string name, int? parent = null)
=> new(name) { Id = id, ParentTemplateId = parent };
void Attr(Template t, string name, string? value, bool inherited = false, bool locked = false) =>
t.Attributes.Add(new TemplateAttribute(name)
{
Id = memberId++, TemplateId = t.Id, Value = value, DataType = DataType.String,
IsInherited = inherited, IsLocked = locked,
});
void Script(Template t, string name, string code, bool inherited = false) =>
t.Scripts.Add(new TemplateScript(name, code)
{
Id = memberId++, TemplateId = t.Id, TriggerType = "Startup", IsInherited = inherited,
});
void Alarm(Template t, string name, int priority, bool inherited = false) =>
t.Alarms.Add(new TemplateAlarm(name)
{
Id = memberId++, TemplateId = t.Id, PriorityLevel = priority,
TriggerType = AlarmTriggerType.ValueMatch, IsInherited = inherited,
});
var root = T(Root, "Root");
Attr(root, "R", "root"); // inherited from ABOVE the edited template
Script(root, "Init", "root-body");
var @base = T(Base, "Base", Root);
Attr(@base, "A", "1");
Attr(@base, "L", "locked", locked: true);
Attr(@base, "New", "added-after-derivation"); // no placeholder anywhere yet
Script(@base, "Calc", "base-body-v2");
Alarm(@base, "Hi", 700);
@base.Compositions.Add(new TemplateComposition("m") { Id = memberId++, TemplateId = Base, ComposedTemplateId = Mod });
var mid = T(Mid, "Mid", Base);
Attr(mid, "A", "mid-override"); // own override, never touched
Attr(mid, "R", "stale-root", inherited: true);
Attr(mid, "L", "locked", inherited: true);
Script(mid, "Calc", "base-body-v1", inherited: true); // stale body
Alarm(mid, "Hi", 500, inherited: true); // stale priority
var leaf1 = T(Leaf1, "Leaf1", Mid);
Attr(leaf1, "A", "1", inherited: true); // stale: Mid overrides A
Attr(leaf1, "Gone", "orphan", inherited: true); // orphan placeholder
var leaf2 = T(Leaf2, "Leaf2", Mid);
Attr(leaf2, "L", "override-attempt"); // locked upstream
var leaf3 = T(Leaf3, "Leaf3", Base);
var greatLeaf = T(GreatLeaf, "GreatLeaf", Leaf1);
Script(greatLeaf, "Calc", "own-body");
var mod = T(Mod, "Mod");
Attr(mod, "ModOnly", "x");
// Unrelated look-alike tree: same member names, must never be loaded.
var u1 = T(U1, "U1");
Attr(u1, "A", "u");
Attr(u1, "New", "u");
var u2 = T(U2, "U2", U1);
Attr(u2, "A", "stale", inherited: true);
// Parent cycle with a hanging child, and a dangling parent edge.
var cycA = T(CycA, "CycA", CycB);
Attr(cycA, "CA", "a");
var cycB = T(CycB, "CycB", CycA);
Attr(cycB, "CB", "b");
var cycC = T(CycC, "CycC", CycA);
var dang = T(Dang, "Dang", 998);
Attr(dang, "D", "d");
var dangKid = T(DangKid, "DangKid", Dang);
return new List<Template>
{
root, @base, mid, leaf1, leaf2, leaf3, greatLeaf, mod, u1, u2, cycA, cycB, cycC, dang, dangKid,
};
}
private static Task<IReadOnlyList<Template>> Closure(
IReadOnlyList<Template> graph,
int id,
List<string>? calls = null) =>
TemplateReconcileClosure.CollectAsync(
id,
(parentIds, _) =>
{
calls?.Add("children:" + string.Join(",", parentIds.OrderBy(i => i)));
return Task.FromResult<IReadOnlyList<Template>>(
graph.Where(t => t.ParentTemplateId is int p && parentIds.Contains(p)).ToList());
},
(ids, _) =>
{
calls?.Add("ids:" + string.Join(",", ids.OrderBy(i => i)));
return Task.FromResult<IReadOnlyList<Template>>(graph.Where(t => ids.Contains(t.Id)).ToList());
});
// Independent re-statement of the service's descendant walk over a given template list.
private static List<int> Descendants(int rootId, IReadOnlyList<Template> all)
{
var result = new List<int>();
var seen = new HashSet<int> { rootId };
var queue = new Queue<int>(new[] { rootId });
while (queue.Count > 0)
{
var id = queue.Dequeue();
foreach (var child in all.Where(t => t.ParentTemplateId == id))
{
if (seen.Add(child.Id))
{
result.Add(child.Id);
queue.Enqueue(child.Id);
}
}
}
return result;
}
/// <summary>Every fixture template id.</summary>
/// <returns>One row per template.</returns>
public static IEnumerable<object[]> EveryTemplate() => BuildGraph().Select(t => new object[] { t.Id });
/// <summary>
/// The closure holds the same descendant set as the whole graph, and the inheritance resolver
/// gives every descendant the identical resolved member set from it.
/// </summary>
/// <param name="templateId">The edited template.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[MemberData(nameof(EveryTemplate))]
public async Task Resolver_AnswersIdenticallyForEveryDescendant(int templateId)
{
var graph = BuildGraph();
var closure = await Closure(graph, templateId);
var wholeDescendants = Descendants(templateId, graph);
Assert.Equal(wholeDescendants.OrderBy(i => i), Descendants(templateId, closure).OrderBy(i => i));
foreach (var d in wholeDescendants)
{
Assert.Equal(
JsonSerializer.Serialize(TemplateInheritanceResolver.Resolve(d, graph), Json),
JsonSerializer.Serialize(TemplateInheritanceResolver.Resolve(d, closure), Json));
}
}
/// <summary>
/// End to end through <see cref="TemplateService.ReconcileDescendantsAsync"/>: the closure-backed
/// reconcile issues exactly the writes (adds / updates / deletes / audit / save) the old
/// whole-graph reconcile did, for every template of the fixture.
/// </summary>
/// <param name="templateId">The edited template.</param>
/// <returns>A task that represents the asynchronous operation.</returns>
[Theory]
[MemberData(nameof(EveryTemplate))]
public async Task Reconcile_IssuesIdenticalWrites_FromClosureAndWholeGraph(int templateId)
{
var expected = await RecordReconcileWrites(templateId, useClosure: false);
var actual = await RecordReconcileWrites(templateId, useClosure: true);
Assert.Equal(expected, actual);
}
/// <summary>The equivalence above is not vacuous: editing Base really adds, updates and removes rows.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Fixture_ReconcilingBase_WritesAddsUpdatesAndDeletes()
{
var writes = await RecordReconcileWrites(Base, useClosure: true);
Assert.Contains(writes, w => w.StartsWith("AddTemplateAttributeAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAttributeAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateScriptAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAlarmAsync", StringComparison.Ordinal));
Assert.Contains(writes, w => w.StartsWith("DeleteTemplateAttributeAsync", StringComparison.Ordinal));
// Root's member reaches Mid only through Base's own chain — the ancestor walk is load-bearing.
Assert.Contains(writes, w => w.StartsWith("UpdateTemplateAttributeAsync", StringComparison.Ordinal)
&& w.Contains("\"Value\":\"root\"", StringComparison.Ordinal));
}
/// <summary>The closure is the subtree plus the chain — no module, no unrelated tree, no siblings.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_IsSubtreePlusChainOnly()
{
var graph = BuildGraph();
var closure = await Closure(graph, Base);
Assert.Equal(
new[] { Root, Base, Mid, Leaf1, Leaf2, Leaf3, GreatLeaf },
closure.Select(t => t.Id).OrderBy(i => i));
Assert.True(closure.Count < graph.Count);
// A mid-tree edit loads its own subtree and chain, not its siblings.
Assert.Equal(
new[] { Root, Base, Mid, Leaf1, Leaf2, GreatLeaf },
(await Closure(graph, Mid)).Select(t => t.Id).OrderBy(i => i));
}
/// <summary>A template with no children costs one children probe and loads nothing.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_NoDescendants_OneProbeAndEmpty()
{
var calls = new List<string>();
Assert.Empty(await Closure(BuildGraph(), Leaf2, calls));
Assert.Equal(new[] { "children:5" }, calls);
Assert.Empty(await Closure(BuildGraph(), 424242));
}
/// <summary>One children batch per subtree depth, one id batch per ancestor level, no id twice.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_LoadsOneBatchPerDepth()
{
var calls = new List<string>();
await Closure(BuildGraph(), Base, calls);
Assert.Equal(
new[] { "children:2", "children:3,6", "children:4,5", "children:7", "ids:2", "ids:1" },
calls);
}
/// <summary>Parent cycles terminate and a dangling parent ends the chain.</summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task Closure_TerminatesOnCyclesAndDanglingParents()
{
var graph = BuildGraph();
Assert.Equal(new[] { CycA, CycB, CycC }, (await Closure(graph, CycA)).Select(t => t.Id).OrderBy(i => i));
Assert.Equal(new[] { Dang, DangKid }, (await Closure(graph, Dang)).Select(t => t.Id).OrderBy(i => i));
}
/// <summary>
/// Runs the reconcile for <paramref name="templateId"/> on a fresh copy of the graph and returns
/// every write the service issued, rendered as sorted strings (sibling order is not part of the
/// contract — each descendant is reconciled independently).
/// </summary>
private static async Task<List<string>> RecordReconcileWrites(int templateId, bool useClosure)
{
var graph = BuildGraph();
var repo = new Mock<ITemplateEngineRepository>();
var audit = new Mock<IAuditService>();
repo.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
if (useClosure)
{
repo.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()))
.ThrowsAsync(new InvalidOperationException("reconcile loaded the whole tracked graph"));
repo.Setup(r => r.GetTemplateReconcileClosureAsync(templateId, It.IsAny<CancellationToken>()))
.Returns((int id, CancellationToken _) => Closure(graph, id));
}
else
{
repo.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(graph);
}
var service = new TemplateService(repo.Object, audit.Object);
if (useClosure)
await service.ReconcileDescendantsAsync(templateId, "tester");
else
await service.ReconcileDescendantsAsync(templateId, "tester", default, graph);
static string Render(Moq.IInvocation i) =>
i.Method.Name + "(" + string.Join(", ", i.Arguments
.Where(a => a is not CancellationToken)
.Select(a => JsonSerializer.Serialize(a, Json))) + ")";
var writes = repo.Invocations
.Where(i => i.Method.Name.StartsWith("Add", StringComparison.Ordinal)
|| i.Method.Name.StartsWith("Update", StringComparison.Ordinal)
|| i.Method.Name.StartsWith("Delete", StringComparison.Ordinal)
|| i.Method.Name == nameof(ITemplateEngineRepository.SaveChangesAsync))
.Select(Render)
.Concat(audit.Invocations.Select(Render))
.OrderBy(s => s, StringComparer.Ordinal)
.ToList();
return writes;
}
}
@@ -5,6 +5,7 @@ using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Repositories;
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Enums;
using ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
namespace ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests;
@@ -22,6 +23,7 @@ public class TemplateServiceTests
_auditMock = new Mock<IAuditService>();
_repoMock.Setup(r => r.SaveChangesAsync(It.IsAny<CancellationToken>())).ReturnsAsync(1);
_repoMock.ServeClosureFromFullGraph(_closureRequests);
_repoMock.ServeReconcileClosureFromFullGraph();
_service = new TemplateService(_repoMock.Object, _auditMock.Object);
}
@@ -2405,4 +2407,66 @@ public class TemplateServiceTests
_repoMock.Verify(r => r.GetTemplateAnalysisClosureAsync(2, It.IsAny<CancellationToken>()), Times.Exactly(3));
_repoMock.Verify(r => r.GetAllTemplatesForAnalysisAsync(It.IsAny<CancellationToken>()), Times.Never);
}
// ========================================================================
// Gitea #86: descendant reconcile loads only the subtree + chain
// ========================================================================
/// <summary>
/// A base member add/update/delete propagates to the derived subtree from the reconcile
/// closure alone: the whole tracked graph is wired to throw and is never loaded, while an
/// unrelated template sharing the member name is never touched.
/// </summary>
/// <returns>A task that represents the asynchronous operation.</returns>
[Fact]
public async Task MemberMutations_PropagateFromReconcileClosure_NeverLoadWholeTrackedGraph()
{
var baseT = BaseTemplate(7, "Base");
baseT.Attributes.Add(new TemplateAttribute("A") { Id = 70, TemplateId = 7, DataType = DataType.String, Value = "1" });
var child = DerivedTemplate(8, "Child", 7);
child.Attributes.Add(new TemplateAttribute("A") { Id = 80, TemplateId = 8, DataType = DataType.String, Value = "1", IsInherited = true });
var grandChild = DerivedTemplate(9, "GrandChild", 8);
var unrelated = BaseTemplate(50, "Unrelated");
unrelated.Attributes.Add(new TemplateAttribute("A") { Id = 500, TemplateId = 50, DataType = DataType.String, Value = "x" });
var library = new List<Template> { unrelated, baseT, child, grandChild };
_repoMock.Setup(r => r.GetTemplateByIdAsync(7, It.IsAny<CancellationToken>())).ReturnsAsync(baseT);
_repoMock.Setup(r => r.GetTemplateAttributeByIdAsync(70, It.IsAny<CancellationToken>()))
.ReturnsAsync(baseT.Attributes.Single());
_repoMock.Setup(r => r.GetAllTemplatesForAnalysisAsync(It.IsAny<CancellationToken>())).ReturnsAsync(library);
_repoMock.Setup(r => r.GetTemplateReconcileClosureAsync(It.IsAny<int>(), It.IsAny<CancellationToken>()))
.Returns((int id, CancellationToken ct) => TemplateReconcileClosure.CollectAsync(
id,
(parents, _) => Task.FromResult<IReadOnlyList<Template>>(
library.Where(t => t.ParentTemplateId is int p && parents.Contains(p)).ToList()),
(ids, _) => Task.FromResult<IReadOnlyList<Template>>(library.Where(t => ids.Contains(t.Id)).ToList()),
ct));
_repoMock.Setup(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()))
.ThrowsAsync(new InvalidOperationException("reconcile loaded the whole tracked graph"));
var added = new List<TemplateAttribute>();
_repoMock.Setup(r => r.AddTemplateAttributeAsync(It.IsAny<TemplateAttribute>(), It.IsAny<CancellationToken>()))
.Callback<TemplateAttribute, CancellationToken>((a, _) =>
{
added.Add(a);
if (a.TemplateId == 7) baseT.Attributes.Add(a); // simulate persistence of the base's own member
})
.Returns(Task.CompletedTask);
var updated = new List<TemplateAttribute>();
_repoMock.Setup(r => r.UpdateTemplateAttributeAsync(It.IsAny<TemplateAttribute>(), It.IsAny<CancellationToken>()))
.Callback<TemplateAttribute, CancellationToken>((a, _) => updated.Add(a))
.Returns(Task.CompletedTask);
var add = await _service.AddAttributeAsync(7, new TemplateAttribute("C") { DataType = DataType.String, Value = "c" }, "admin");
var update = await _service.UpdateAttributeAsync(70, new TemplateAttribute("A") { DataType = DataType.String, Value = "2" }, "admin");
Assert.True(add.IsSuccess, add.IsFailure ? add.Error : null);
Assert.True(update.IsSuccess, update.IsFailure ? update.Error : null);
Assert.Contains(added, a => a.TemplateId == 8 && a.Name == "C" && a.IsInherited);
Assert.Contains(added, a => a.TemplateId == 9 && a.Name == "C" && a.IsInherited); // two levels down
Assert.Contains(updated, a => a.TemplateId == 8 && a.Name == "A" && a.Value == "2");
Assert.DoesNotContain(added.Concat(updated), a => a.TemplateId == 50);
_repoMock.Verify(r => r.GetTemplateReconcileClosureAsync(7, It.IsAny<CancellationToken>()), Times.Exactly(2));
_repoMock.Verify(r => r.GetAllTemplatesAsync(It.IsAny<CancellationToken>()), Times.Never);
}
}