perf(deploy): flatten-session caching, bulk DeploySiteAsync, paged management queries

2026-08-14 21:14:22 -04:00
parent ee193cd2bb
commit 48b3c40a7f
59 changed files with 3051 additions and 231 deletions
@@ -137,6 +137,64 @@ When an instance is deployed, the Template Engine resolves the full configuratio
 5. Resolve data connection bindings — replace connection name references with concrete connection details from the site.
 6. Output a flat structure: list of attributes with resolved values and data source addresses, list of alarms with resolved trigger definitions, list of scripts with resolved code and triggers.

+### Flatten-session caching and the graph watermark (WP2.5)
+
+The flatten above is driven per instance, so an unmemoised implementation re-walks
+the same template chain — one query per link, plus the compositions of every
+template it reaches and of every composed chain those reach — once per instance,
+and re-issues the three session-global queries (shared scripts, the shared-schema
+library, the target site's data connections) each time too.
+
+A **`FlattenSession`** memoises all of that for the lifetime of ONE
+flatten/validate operation. Callers that flatten a batch (`DeploySiteAsync`) pass
+one shared session; a caller that passes none gets a private single-use session,
+which still collapses the repeated composed-chain loads inside a single instance's
+flatten.
+
+Cache validity is decided by **`ITemplateGraphWatermark`**, a process-wide set of
+monotonic counters bumped by the configuration-database unit of work — the only
+place every template-graph writer funnels through (`TemplateService`, the
+`ManagementActor` native-alarm-source handlers, and the Transport bundle importer
+all commit via the same `SaveChangesAsync`, and the change tracker is inspected
+pre-commit to attribute each change to its owning template or instance):
+
+- a memoised **template** is keyed on `(id, template version)`;
+- a memoised **chain** is valid only while BOTH the graph's `StructureVersion`
+  (bumped on add/remove, re-parent, or any composition change — i.e. anything that
+  can alter chain MEMBERSHIP) and every member's own version are unchanged. Both
+  halves are load-bearing: structure alone misses an ordinary member edit, member
+  versions alone miss a re-parent.
+
+Sessions are short-lived by construction, so the design's "template state is
+captured at the time of flatten" guarantee is unchanged — the session narrows the
+capture window, it never widens it.
+
+The watermark is **in-memory and process-local**, deliberately not persisted and
+not replicated between central nodes. A restart, a failover, or any mutation simply
+misses and falls back to the authoritative full flatten, so the watermark can only
+ever cause EXTRA work, never stale work.
+
+The same watermark backs a **staleness fast path** in `StaleInstanceProbe`: a
+previously computed revision hash is reused when the instance's version, the
+structure version, and the version of every template that flatten walked are all
+unchanged. This matters most where the probe is called per-instance across a whole
+bundle import or a fleet-wide staleness sweep.
+
+### Design-time analysis reads
+
+The design-time checks — acyclicity (`CycleDetector`), naming collisions
+(`CollisionDetector`) and canonical-name resolution (`TemplateResolver`) — read the
+whole template graph but write through none of it and read no script bodies. They
+use `GetAllTemplatesForAnalysisAsync`: `AsNoTracking`, with `TemplateScript.Code`
+projected away.
+
+The tracked, body-bearing `GetAllTemplatesAsync` remains the read for the two paths
+that genuinely need it — the inheritance reconciler (which compares and copies
+script bodies and writes through the loaded entities) and the flattener (which must
+observe rows an in-flight bundle import has staged on the shared change tracker).
+`ReconcileDescendantsAsync` additionally accepts an already-loaded tracked graph so
+a caller holding one does not force a second load.
+
 ### Native Alarm Source Resolution

 The `FlatteningService` resolves native alarm sources alongside alarms, emitting a `ResolvedNativeAlarmSource` (CanonicalName, ConnectionName, SourceReference, ConditionFilter *(optional)*, and `Source` ∈ `Template` | `Inherited` | `Composed` | `Override`) for each. The resolved set is attached to `FlattenedConfiguration.NativeAlarmSources`.