ScadaBridge/docs/components/TemplateEngine.md

Template Engine

The Template Engine models the machine blueprints — templates — from which all deployed instances are created. It enforces inheritance, composition, locking, naming rules, and acyclicity at authoring time, then flattens the resulting graph plus instance overrides into a concrete, revision-hashed FlattenedConfiguration that the Deployment Manager sends to sites.

Overview

Template Engine (#1) runs on the central cluster only. Sites receive flattened output and have no awareness of template structure. The component code lives in src/ZB.MOM.WW.ScadaBridge.TemplateEngine/, organized as follows:

• Root — TemplateService, SharedScriptService, TemplateResolver, CycleDetector, CollisionDetector, LockEnforcer, TemplateNaming — core authoring operations and graph invariant enforcement.
• Flattening/ — FlatteningService, RevisionHashService, DiffService — produce and compare the deployment-ready representation.
• Validation/ — ValidationService, SemanticValidator, ScriptCompiler, CSharpDelimiterScanner — pre-deployment and on-demand correctness checks.
• Services/ — InstanceService, SiteService, AreaService, TemplateFolderService, TemplateDeletionService — domain operations that depend on the scoped ITemplateEngineRepository.

The single DI entry point is ServiceCollectionExtensions.AddTemplateEngine. TemplateService and SharedScriptService are scoped; the flattening and validation utilities are transient; static helpers (CycleDetector, CollisionDetector, LockEnforcer, TemplateResolver) are not registered.

Key Concepts

Template graph

The full set of templates forms a directed graph with two independent edge types:

• Inheritance — Template.ParentTemplateId (nullable int?). A null value means no parent; a non-null value sets the defining ancestor. The parent is set at creation time and is immutable thereafter; UpdateTemplateAsync rejects any attempt to change it.
• Composition — TemplateComposition rows, each pointing from an owner TemplateId to a ComposedTemplateId with a slot name (InstanceName). Only base (non-derived) templates may be composed.

Both edge types are enforced acyclic on every mutating call. CycleDetector provides three checks:

• DetectInheritanceCycle — walks the proposed parent chain upward looking for the template being modified.
• DetectCompositionCycle — BFS from the proposed composed template through its own compositions.
• DetectCrossGraphCycle — BFS across both inheritance and composition edges simultaneously, catching cycles that neither pure check alone would find.

AddCompositionAsync runs DetectCompositionCycle and DetectCrossGraphCycle before any composition is written — DetectInheritanceCycle does not run on the composition path. Because the graph can contain not-yet-saved templates (Id = 0), CycleDetector.BuildLookup uses TryAdd rather than ToDictionary so duplicate Ids do not throw.

Derived templates and the composition model

When AddCompositionAsync composes template B into template A under slot name Pump, the engine calls CreateCascadedCompositionAsync, which:

1. Creates a derived Template (IsDerived = true, ParentTemplateId = B.Id, Name = "Pump") as the slot-owned backing record.
2. Copies B's attributes, alarms, and scripts onto the derived template with IsInherited = true.
3. Creates a TemplateComposition row linking A to the derived template.
4. Sets derived.OwnerCompositionId so the slot can be deleted as a unit.
5. Recurses into B's own compositions to replicate them under the new derived template.

Derived templates are hidden from the main template tree and cannot be directly composed or deleted by name — removal goes through DeleteCompositionAsync, which calls CascadeDeleteDerivedAsync to tear the whole subtree down.

Canonical naming and path qualification

Members of composed modules are addressed by path-qualified canonical names: [ModuleInstanceName].[MemberName]. For deeper nesting the dot chain extends: Pump.Motor.Speed. Direct members of the owning template carry no prefix.

TemplateResolver.ResolveAllMembers builds a Dictionary<string, ResolvedTemplateMember> in inheritance-chain order (root first) then adds composed-module members with their prefix. The last writer on a given canonical name wins — child overrides shadow parent definitions. FindMemberByCanonicalName is the entry point for lock checks during UpdateAttribute/Alarm/ScriptAsync.

TemplateNaming.QualifiedName computes the full dotted path of a derived template at read time by walking the OwnerCompositionId chain — the derived template stores only its contained name (InstanceName), so the full path is never stored and cannot drift.

Locking and override granularity

LockEnforcer enforces three classes of rules for attributes, alarms, and scripts:

Rule	Mechanism
IsLocked = true blocks all downstream overrides	ValidateLockChange: once set, cannot be cleared (one-way ratchet).
LockedInDerived = true blocks derived-template overrides of that specific member	ValidateLockedInDerivedChange: also a one-way ratchet — cannot be cleared on a base template.
Fixed fields cannot change at any level	ValidateAttributeOverride / ValidateAlarmOverride / ValidateScriptOverride.

Attribute fixed fields: DataType, DataSourceReference. Overridable: Value, Description.

Alarm fixed fields: Name, TriggerType. Overridable: PriorityLevel, TriggerConfiguration, Description, OnTriggerScriptId.

Script fixed fields: Name. Overridable: Code, TriggerType, TriggerConfiguration, MinTimeBetweenRuns, ParameterDefinitions, ReturnDefinition.

Intermediate locking is permitted: a child template can lock an unlocked member inherited from its parent. Unlocking is never permitted at any level.

Naming collisions

Adding a member (attribute, alarm, script, or composition) triggers CollisionDetector.DetectCollisions on a speculative clone of the template. The detector collects all canonical names — direct members plus path-qualified composed-module members plus inherited members — groups them by canonical name, and reports any group where two entries come from different origin descriptions. A collision is a design-time error and blocks the operation.

Because each composition slot has a unique InstanceName prefix, members from different slots can never collide by canonical name. Collisions arise only when a directly defined member shares an unqualified name with an inherited or composed member under the same owner.

Flattening

FlatteningService.Flatten takes the instance, its template inheritance chain (most-derived first), a composition map, per-composed-template chains, and available data connections, and produces a FlattenedConfiguration. The resolution order is:

1. Instance overrides (highest priority, respects locks).
2. Most-derived template in the inheritance chain.
3. Parent templates, walking to the root.
4. Composed module members, path-qualified.

The nine steps in order:

1. Validate LockedInDerived is not violated across each chain.
2. Resolve attributes from the inheritance chain (base-to-derived; IsInherited placeholders never shadow live base values).
3. Resolve composed-module attributes with path-qualified canonical names, recursively.
4. Apply InstanceAttributeOverride records (locked attributes are silently skipped).
5. Apply InstanceConnectionBinding records (data-sourced attributes only).
6. Resolve alarms; for HiLo trigger type, merge setpoints key-by-key so a derived template can override just hi while inheriting loLo.
7. Resolve scripts; wire ScriptScope (self- and parent-path) into each composed script so Attributes["X"] resolves to the right path-prefix at runtime. Then resolve alarm OnTriggerScriptId FKs to canonical script names (ResolveAlarmScriptReferences).
8. Resolve native alarm source bindings (TemplateNativeAlarmSource), apply InstanceNativeAlarmSourceOverride.
9. Collect connection configurations — iterate resolved attributes, and for each attribute that has a bound data connection, populate FlattenedConfiguration.Connections with the corresponding ConnectionConfig (protocol, primary and backup JSON, failover retry count).

Revision hash

RevisionHashService.ComputeHash produces a deterministic sha256:<hex> string over the canonical JSON serialization of the FlattenedConfiguration. Volatile fields (GeneratedAtUtc) are excluded. Collections are sorted by CanonicalName before hashing. Internal Hashable* records declare their properties in alphabetical order because System.Text.Json emits them in declaration order — out-of-order additions would silently break determinism. The hash covers resolved attributes, alarms, scripts, and connection configurations. NativeAlarmSources is not included in the hash: changes to native alarm source bindings do not alter the revision hash and therefore do not trigger a re-deploy on their own. The hash is included in the deployment identity and lets the Deployment Manager detect whether a re-flatten has changed anything before pushing to sites.

Diff

DiffService.ComputeDiff compares two FlattenedConfiguration snapshots by canonical name, producing Added, Removed, and Changed entries for attributes, alarms, and scripts. ComputeConnectionsDiff produces the same shape for data-connection configurations. The diff is used by the Deployment Manager to decide whether a full redeploy is needed.

Concurrent editing

Template edits use last-write-wins — there is no optimistic concurrency token on Template or its member rows. Two simultaneous edits to the same template produce one winner. This is by design. The same policy applies to instances and is documented in the InstanceService class comment: "Concurrent edits are last-write-wins — there is no version token or conflict detection on instance state." Optimistic concurrency (RowVersion) applies to deployment status records in the Deployment Manager, not to template or instance authoring.

Architecture

Service map

AddTemplateEngine()
├── TemplateService        (scoped)  — template + member CRUD, collision/acyclicity pre-checks
├── SharedScriptService    (scoped)  — system-wide shared script CRUD + syntax validation
├── InstanceService        (scoped)  — instance CRUD, overrides, connection bindings
├── SiteService            (scoped)  — site CRUD
├── AreaService            (scoped)  — area CRUD
├── TemplateFolderService  (scoped)  — folder hierarchy, sibling-name uniqueness, acyclicity on move
├── TemplateDeletionService(scoped)  — deletion constraints; called from TemplateService.DeleteTemplateAsync
├── FlatteningService      (transient)
├── RevisionHashService    (transient)
├── DiffService            (transient)
├── ValidationService      (transient) — full pipeline: 8 stages merged into one ValidationResult
├── SemanticValidator      (transient) — call-target, argument-count, operand-type, cross-call rules
└── ScriptCompiler         (transient) — advisory forbidden-API scan + delimiter balance check

Static helpers — CycleDetector, CollisionDetector, LockEnforcer, TemplateResolver, TemplateNaming — are not registered with DI.

Validation pipeline

ValidationService.Validate runs eight stages in sequence and merges results via ValidationResult.Merge:

Stage	Category	Outcome
ValidateFlatteningSuccess	FlatteningFailure	Error on missing name; warning on empty configuration
ValidateNamingCollisions	NamingCollision	Error per duplicate canonical name within entity type
ValidateScriptCompilation	ScriptCompilation	Error per script that fails ScriptCompiler.TryCompile
ValidateAlarmTriggerReferences	AlarmTriggerReference	Error when attributeName / attribute key not in flattened attributes
ValidateScriptTriggerReferences	ScriptTriggerReference	Same check for script triggers
ValidateExpressionTriggers	ScriptTriggerReference / AlarmTriggerReference	Blank warning, syntax error, or missing Attributes["X"] reference
ValidateConnectionBindingCompleteness	ConnectionBinding	Warning per data-sourced attribute with no binding
SemanticValidator.Validate	Multiple	Call targets, argument counts, RangeViolation/HiLo operand types, on-trigger script existence, cross-call violations, native alarm source completeness

ValidationResult is defined in Commons: IsValid is true when Errors is empty; Warnings do not block deployment. Each ValidationEntry carries a ValidationCategory, a human-readable Message, and an optional EntityName (canonical name of the offending entity).

Key entity types (defined in Commons)

Type	Namespace	Role
Template	Commons.Entities.Templates	Base and derived template rows; IsDerived distinguishes slot-owned derived templates
TemplateAttribute	same	Attribute definition with IsInherited, LockedInDerived, DataType, DataSourceReference
TemplateAlarm	same	Alarm definition; TriggerType and Name are fixed fields
TemplateScript	same	Script definition; Name is a fixed field
TemplateComposition	same	Slot row linking owner to composed (or derived) template by InstanceName
TemplateNativeAlarmSource	same	Read-only native alarm binding; SourceReference is a raw connection address
FlattenedConfiguration	Commons.Types.Flattening	Deployment-ready snapshot; fields Attributes, Alarms, Scripts, NativeAlarmSources, Connections
ResolvedAttribute / ResolvedAlarm / ResolvedScript / ResolvedNativeAlarmSource	same	Flattened member records carrying CanonicalName and Source provenance
ConfigurationDiff / DiffEntry<T>	same	Diff output keyed by canonical name
ValidationResult / ValidationEntry / ValidationCategory	same	Validation output

Usage

Authoring a template with inheritance and composition

The normal flow goes through TemplateService methods; each call validates graph invariants before persisting and audits after:

// Create a base template
var base = await templateService.CreateTemplateAsync(
    "MotorBase", description: null, parentTemplateId: null, user: "alice");

// Add an attribute to the base
await templateService.AddAttributeAsync(base.Value.Id, new TemplateAttribute("Speed")
{
    DataType = DataType.Float,
    DataSourceReference = "/Motor/Speed"
}, user: "alice");

// Create a child that inherits from the base
var child = await templateService.CreateTemplateAsync(
    "PumpMotor", description: null, parentTemplateId: base.Value.Id, user: "alice");

// Compose a feature module (AlarmsModule) into the base — acyclicity and collision
// checks run; a derived template is auto-created to back the slot
await templateService.AddCompositionAsync(
    templateId: base.Value.Id,
    composedTemplateId: alarmsModule.Id,
    instanceName: "Alarms",
    user: "alice");

After composition, Alarms.HighTemp is the canonical name of HighTemp from AlarmsModule as it appears in the flattened output.

Resolving members and checking overrides

// Returns all effective members with canonical names for templateId
var members = await templateService.ResolveTemplateMembersAsync(templateId);

// Returns TemplateResolver.ResolvedTemplateMember with CanonicalName, MemberType,
// IsLocked, and ModulePath (null for direct members, slot prefix for composed members).
foreach (var m in members)
{
    Console.WriteLine($"{m.CanonicalName} ({m.MemberType}) locked={m.IsLocked}");
}

Flattening and hashing

FlatteningService and RevisionHashService are transient services called by the Deployment Manager, not directly from the Central UI. The caller builds the template chain (most-derived first) from the repository and passes it:

var flatResult = flatteningService.Flatten(
    instance,
    templateChain,          // IReadOnlyList<Template>, index 0 = instance's template
    compositionMap,         // Dictionary<int, IReadOnlyList<TemplateComposition>>
    composedTemplateChains, // Dictionary<int, IReadOnlyList<Template>>
    dataConnections);       // Dictionary<int, DataConnection>

if (flatResult.IsSuccess)
{
    var hash = revisionHashService.ComputeHash(flatResult.Value);
    // hash: "sha256:3a7f..."
}

Validating before deployment

var validationResult = validationService.Validate(flatResult.Value, sharedScripts);
if (!validationResult.IsValid)
{
    foreach (var err in validationResult.Errors)
        logger.LogError("{Category} {Entity}: {Msg}", err.Category, err.EntityName, err.Message);
}

Dependencies & Interactions

• Commons (#16) — owns all entity types (Template, TemplateAttribute, TemplateAlarm, TemplateScript, TemplateComposition, TemplateNativeAlarmSource), the flattening types (FlattenedConfiguration, ResolvedAttribute, ResolvedAlarm, ResolvedScript, ResolvedNativeAlarmSource, ConfigurationDiff), the ValidationResult/ValidationEntry/ValidationCategory hierarchy, the ITemplateEngineRepository interface, and the IAuditService interface. Template Engine imports from Commons; it never holds a direct EF Core dependency.
• Configuration Database (#17) — provides the ITemplateEngineRepository implementation backed by the central MS SQL database. TemplateService, InstanceService, and all Services/ classes resolve this via constructor injection. EF Core migrations for template tables live in this project.
• Deployment Manager (#2) — consumes FlatteningService, RevisionHashService, DiffService, and ValidationService to prepare deployment packages. It also calls TemplateDeletionService.CanDeleteTemplateAsync to check constraints before removing a template that has active deployments.
• Site Runtime (#3) — receives the FlattenedConfiguration (via the Deployment Manager) and uses ResolvedScript.Scope to set the path-prefix context for script attribute accessors. ResolvedNativeAlarmSource records drive the site's NativeAlarmActor bindings.
• Central UI (#9) — the primary authoring surface. All template CRUD, instance management, shared script editing, and folder organization go through the Management Service, which delegates to TemplateService, InstanceService, and SharedScriptService. The Central UI calls ValidateAsync on-demand so designers see errors before deployment.
• Management Service (#18) — the Akka.NET actor that exposes template operations over the cluster boundary. TemplateService and related services are injected into its DI scope per request.
• Transport (#24) — exports and imports templates as encrypted bundles. On import, the Transport component calls TemplateService.CreateTemplateAsync (and member-add methods) for each template in the bundle; acyclicity and collision checks run identically to manual authoring.
• Design spec: Component-TemplateEngine.md.

Troubleshooting

Composition fails with a cycle error

CycleDetector.DetectCrossGraphCycle rejects edges that would create a cycle across either inheritance or composition edges. The most common trigger is composing a template that already (transitively) includes the owner — for example, A composes B, and then trying to compose A into B. The error message identifies the template by name. Remove or restructure the graph to break the circular dependency.

LockedInDerived cannot be cleared

LockEnforcer.ValidateLockedInDerivedChange enforces a one-way ratchet: once a base template sets LockedInDerived = true on a member, the flag cannot be cleared. This is intentional — clearing it retroactively would make previously blocked derived overrides legal without any visible signal to derived-template authors. The only remediation is to create a new base template without the flag.

Revision hash changes unexpectedly between deploys

The SHA-256 hash covers resolved attributes, alarms, scripts, and connection configurations. Changes to any of those members anywhere in the inheritance or composition chain — including in parent templates or feature modules the instance does not directly own — will change the hash. Note that NativeAlarmSources is not part of the hash, so native alarm source binding changes alone do not change the revision hash. Use DiffService.ComputeDiff to identify exactly which canonical names changed and why.

Naming collision on composed member add

CollisionDetector.DetectCollisions fires when a member's unqualified name collides with a direct or inherited member on the same owner template. Because composed members carry a slot-name prefix, a collision can arise between a directly defined member (Speed) and a composed member that resolves to the same unqualified name in an ancestor. The fix is to rename one of the conflicting members before adding the composition.

Semantic validation: CallScript target not found

SemanticValidator.ExtractCallTargets uses a substring scan for CallScript("name", ...) and CallShared("name", ...). If the target name does not match any resolved script canonical name, the error CallTargetNotFound is reported. Check that the call uses the full canonical name, including any composition prefix (e.g., Alarms.HandleFault, not just HandleFault).

Related Documentation

• Template Engine design specification
• Commons
• Configuration Database
• Deployment Manager
• Site Runtime
• Central UI
• Management Service
• Transport