perf(templates): descendant reconcile loads only the subtree + chain (ScadaBridge#86)

TemplateService.ReconcileDescendantsAsync - run after every base member add/update/delete, template and native-alarm-source alike - loaded the whole tracked template graph (every template, member and script body) on each call, which after #66 was the dominant cost of those paths. It now calls the new ITemplateEngineRepository.GetTemplateReconcileClosureAsync: the same tracked, fully-included shape as GetAllTemplatesAsync, restricted to the edited template's derived subtree plus its own inheritance chain. The reconcile is inheritance-only (TemplateInheritanceResolver never follows compositions) and every descendant's chain runs through the edited template, so that set is everything it reads. The walk (Commons TemplateReconcileClosure, the descendant-direction twin of #66's TemplateAnalysisClosure) loads one children batch per subtree depth and one batch per ancestor level, terminates on parent cycles and stops at dangling parents. A template with no children - the common case - costs one empty probe and loads nothing. Evidence: - TemplateReconcileClosureTests: for every template of a graph with a deep subtree, an ancestor above the edited template, a composed module, look-alike unrelated trees, a parent cycle and a dangling parent, the closure-backed reconcile issues exactly the writes (add/update/delete, audit, save) of the old whole-graph reconcile, and the resolver answers identically for every descendant. Fails with the old service (whole-graph load wired to throw) and with the ancestor walk removed (mutation-checked). - TemplateReconcileClosureRepositoryTests (SQLite, recorded SQL): 5 templates / 10 attributes instead of 47 / 94, tracked with script bodies; a leaf costs one probe query and tracks nothing. - TemplateServiceTests: attribute add + update propagate two levels down with GetAllTemplatesAsync wired to throw; existing propagation tests now run against the closure via ServeReconcileClosureFromFullGraph. ResyncInheritedMembersAsync (operator action) still reads the whole graph. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:01:56 -05:00
parent 0eac956d25
commit c29b99c63b
10 changed files with 718 additions and 31 deletions
@@ -69,6 +69,20 @@ public interface ITemplateEngineRepository
    /// <returns>A task that resolves to the closure templates without script bodies.</returns>
    Task<IReadOnlyList<Template>> GetTemplateAnalysisClosureAsync(int templateId, CancellationToken cancellationToken = default);
    /// <summary>
+    /// Returns the TRACKED, fully-loaded templates (the <see cref="GetAllTemplatesAsync"/> shape:
+    /// attributes, alarms, scripts with bodies, compositions, native alarm sources) that the
+    /// inherited-member reconcile of <paramref name="templateId"/>'s descendants reads: its whole
+    /// derived subtree plus its own inheritance chain (see
+    /// <see cref="ZB.MOM.WW.ScadaBridge.Commons.Types.Templates.TemplateReconcileClosure"/>).
+    /// Backs <c>TemplateService.ReconcileDescendantsAsync</c> (Gitea #86), which previously
+    /// loaded every template on each base-member add/update/delete. Returns an empty list when the
+    /// template has no children — there is nothing to reconcile — after a single probe query.
+    /// </summary>
+    /// <param name="templateId">The template whose member set changed.</param>
+    /// <param name="cancellationToken">Cancellation token.</param>
+    /// <returns>A task that resolves to the subtree followed by the template's chain.</returns>
+    Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default);
+    /// <summary>
    /// Returns an untracked identity/edge projection of every template (no children), ordered by name.
    /// </summary>
    /// <param name="cancellationToken">Cancellation token.</param>
@@ -0,0 +1,103 @@
+using ZB.MOM.WW.ScadaBridge.Commons.Entities.Templates;
+
+namespace ZB.MOM.WW.ScadaBridge.Commons.Types.Templates;
+
+/// <summary>
+/// Computes the set of templates the inherited-member reconcile reads when a base member of one
+/// template changes (Gitea #86): the template's whole <b>derived subtree</b> (children,
+/// grandchildren, …) plus the template's own <b>inheritance chain</b> (itself, its parent,
+/// grandparent, …).
+///
+/// <para>
+/// This is the descendant-direction counterpart of <see cref="TemplateAnalysisClosure"/> (#66).
+/// <c>TemplateService.ReconcileDescendantsAsync</c> reconciles every descendant <c>D</c> by
+/// resolving <c>D</c>'s inheritance chain (<c>TemplateInheritanceResolver.Resolve</c>), which is
+/// inheritance only — compositions are never followed. <c>D</c>'s chain is the path from
+/// <c>D</c> up to the edited template (all inside the subtree) followed by the edited template's
+/// own chain, and every member winner originates on that chain. So the subtree plus the root's
+/// chain is everything the reconcile can read; the rest of the library is irrelevant.
+/// </para>
+///
+/// <para>
+/// The walk is breadth-first by batch: one <c>loadChildren</c> call per subtree depth and one
+/// <c>loadByIds</c> call per ancestor level, so a store-backed loader issues one query per
+/// depth rather than one per template. When the template has no children — the common case —
+/// the walk stops after the single children probe and returns an empty list, because there is
+/// nothing to reconcile. Cycles terminate (each id is loaded at most once) and a dangling parent
+/// edge simply ends the chain, exactly as the resolver's chain builder does.
+/// </para>
+/// </summary>
+public static class TemplateReconcileClosure
+{
+    /// <summary>
+    /// Collects the derived subtree and inheritance chain of <paramref name="templateId"/>.
+    /// </summary>
+    /// <param name="templateId">The template whose member set changed.</param>
+    /// <param name="loadChildren">Loads every template whose parent is one of the given ids.</param>
+    /// <param name="loadByIds">Loads the templates with the given ids (unknown ids are omitted).</param>
+    /// <param name="cancellationToken">Cancellation token.</param>
+    /// <returns>
+    /// The subtree (breadth-first) followed by the root's chain, each template once; an empty
+    /// list when the template has no children.
+    /// </returns>
+    public static async Task<IReadOnlyList<Template>> CollectAsync(
+        int templateId,
+        Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadChildren,
+        Func<IReadOnlyCollection<int>, CancellationToken, Task<IReadOnlyList<Template>>> loadByIds,
+        CancellationToken cancellationToken = default)
+    {
+        ArgumentNullException.ThrowIfNull(loadChildren);
+        ArgumentNullException.ThrowIfNull(loadByIds);
+
+        var loaded = new Dictionary<int, Template>();
+        var result = new List<Template>();
+        // The root is "visited" from the start so a cycle back to it does not re-expand it; it is
+        // loaded (if it exists) by the chain walk below.
+        var expanded = new HashSet<int> { templateId };
+
+        // 1. Derived subtree, one children query per depth.
+        var frontier = new List<int> { templateId };
+        while (frontier.Count > 0)
+        {
+            cancellationToken.ThrowIfCancellationRequested();
+            var children = await loadChildren(frontier, cancellationToken).ConfigureAwait(false);
+            var next = new List<int>();
+            foreach (var child in children)
+            {
+                if (child.Id == templateId || !loaded.TryAdd(child.Id, child))
+                    continue;
+                result.Add(child);
+                if (expanded.Add(child.Id))
+                    next.Add(child.Id);
+            }
+            frontier = next;
+        }
+
+        if (result.Count == 0)
+            return result; // no descendants: nothing to reconcile, so load nothing else
+
+        // 2. The root's own inheritance chain, one query per level. A level already loaded as
+        //    part of the subtree means a parent cycle — the chain ends there.
+        var requested = new HashSet<int>();
+        int? nextId = templateId;
+        while (nextId is int id && requested.Add(id))
+        {
+            cancellationToken.ThrowIfCancellationRequested();
+            if (loaded.TryGetValue(id, out var already))
+            {
+                nextId = already.ParentTemplateId;
+                continue;
+            }
+
+            var batch = await loadByIds(new[] { id }, cancellationToken).ConfigureAwait(false);
+            var template = batch.FirstOrDefault(t => t.Id == id);
+            if (template is null)
+                break; // dangling parent edge (or an unknown root)
+            loaded[id] = template;
+            result.Add(template);
+            nextId = template.ParentTemplateId;
+        }
+
+        return result;
+    }
+}
@@ -80,14 +80,7 @@ public class TemplateEngineRepository : ITemplateEngineRepository
    /// <inheritdoc />
    public async Task<IReadOnlyList<Template>> GetAllTemplatesAsync(CancellationToken cancellationToken = default)
    {
-        return await _context.Templates
-            .Include(t => t.Attributes)
-            .Include(t => t.Alarms)
-            .Include(t => t.Scripts)
-            .Include(t => t.Compositions)
-            .Include(t => t.NativeAlarmSources)
-            .AsSplitQuery()
-            .ToListAsync(cancellationToken);
+        return await FullTemplateGraph().ToListAsync(cancellationToken);
    }

    /// <inheritdoc />
@@ -143,6 +136,44 @@ public class TemplateEngineRepository : ITemplateEngineRepository
            cancellationToken);
    }

+    /// <inheritdoc />
+    // One tracked, fully-included query per subtree depth (children of the frontier) and one per
+    // ancestor level — the same entity shape as GetAllTemplatesAsync, because the reconcile
+    // writes through these entities and compares script bodies (Gitea #86).
+    public Task<IReadOnlyList<Template>> GetTemplateReconcileClosureAsync(int templateId, CancellationToken cancellationToken = default)
+    {
+        return TemplateReconcileClosure.CollectAsync(
+            templateId,
+            async (parentIds, ct) =>
+            {
+                var idList = parentIds.ToList();
+                return await FullTemplateGraph()
+                    .Where(t => t.ParentTemplateId != null && idList.Contains(t.ParentTemplateId.Value))
+                    .ToListAsync(ct);
+            },
+            async (ids, ct) =>
+            {
+                var idList = ids.ToList();
+                return await FullTemplateGraph()
+                    .Where(t => idList.Contains(t.Id))
+                    .ToListAsync(ct);
+            },
+            cancellationToken);
+    }
+
+    // The tracked, fully-included template shape shared by GetAllTemplatesAsync and
+    // GetTemplateReconcileClosureAsync.
+    private IQueryable<Template> FullTemplateGraph()
+    {
+        return _context.Templates
+            .Include(t => t.Attributes)
+            .Include(t => t.Alarms)
+            .Include(t => t.Scripts)
+            .Include(t => t.Compositions)
+            .Include(t => t.NativeAlarmSources)
+            .AsSplitQuery();
+    }
+
    // The analysis projection shared by GetAllTemplatesForAnalysisAsync and
    // GetTemplateAnalysisClosureAsync (see the note on the former for why it is untracked and
    // drops TemplateScript.Code).
@@ -1268,18 +1268,14 @@ public class TemplateService
    /// <param name="user">Username for the audit row.</param>
    /// <param name="cancellationToken">Cancellation token.</param>
    /// <param name="loadedTemplates">
-    /// The full, TRACKED template graph if the caller already has one. Reconciling
-    /// writes through these entities and compares script bodies, so this must be
-    /// the tracked <c>GetAllTemplatesAsync</c> shape — never the no-tracking,
-    /// body-free analysis projection. When <see langword="null"/> the graph is
-    /// loaded here.
-    ///
-    /// <para>
-    /// Passing it matters because every member Add/Update/Delete on
-    /// <see cref="TemplateService"/> used to load the whole graph once for its own
-    /// checks and then again inside this method. Handing the graph over collapses
-    /// that to one load per operation.
-    /// </para>
+    /// A TRACKED template graph the caller already has, containing at least the
+    /// template's derived subtree and inheritance chain. Reconciling writes through
+    /// these entities and compares script bodies, so this must be the tracked
+    /// <c>GetAllTemplatesAsync</c> shape — never the no-tracking, body-free analysis
+    /// projection. When <see langword="null"/> only the reconcile closure (the
+    /// subtree plus the template's chain, Gitea #86) is loaded here — never the
+    /// whole library, and nothing beyond one probe query when the template has no
+    /// children.
    /// </param>
    /// <returns>A task that represents the asynchronous operation.</returns>
    public async Task ReconcileDescendantsAsync(
@@ -1288,7 +1284,12 @@ public class TemplateService
        CancellationToken cancellationToken = default,
        IReadOnlyList<Template>? loadedTemplates = null)
    {
-        var all = loadedTemplates ?? await _repository.GetAllTemplatesAsync(cancellationToken);
+        // The descendants' chains run through templateId and on up its own chain, and
+        // the resolver is inheritance-only, so the subtree + the template's chain is
+        // everything the reconcile reads (#86).
+        var all = loadedTemplates
+                  ?? await _repository.GetTemplateReconcileClosureAsync(templateId, cancellationToken)
+                  ?? Array.Empty<Template>();
        var descendants = GetDescendantIdsBreadthFirst(templateId, all);
        if (descendants.Count == 0)
            return;