ScadaBridge/docs/requirements/Component-TemplateEngine.md

# Component: Template Engine

## Purpose

The Template Engine is the core modeling component that lives on the central cluster. It manages the definition, inheritance, composition, and resolution of machine templates — the blueprints from which all machine instances are created. It handles flattening templates into deployable configurations, calculating diffs between deployed and current states, and performing comprehensive pre-deployment validation.

## Location

Central cluster only. Sites receive flattened output and have no awareness of templates.

## Responsibilities

- Store and manage template definitions (attributes, alarms, scripts) in the configuration database.
- Enforce inheritance (is-a) relationships between templates.
- Enforce composition (has-a) relationships, including recursive nesting of feature modules.
- Detect and reject naming collisions when composing feature modules (design-time error).
- Resolve the attribute chain: Instance → Child Template → Parent Template → Composing Template → Composed Module.
- Enforce locking rules — locked members cannot be overridden downstream, intermediate levels can lock previously unlocked members, and nothing can unlock what's locked above.
- Support adding new attributes, alarms, and scripts in child templates.
- Prevent removal of inherited members.
- Flatten a fully resolved template + instance overrides into a deployable configuration (no template structure, just concrete attribute values with resolved data connection bindings).
- Calculate diffs between deployed and template-derived configurations.
- Perform comprehensive pre-deployment validation (see Validation section).
- Provide on-demand validation for Design users during template authoring.
- Enforce template deletion constraints — templates cannot be deleted if any instances or child templates reference them.
- Organize templates into nested folders (`TemplateFolder` entity) and validate folder hierarchy invariants (acyclicity, sibling uniqueness, non-empty-on-delete).
- Resolve and surface the full multi-level effective inherited member set for a template (read-only authoring view), including origin annotation, locked state, merged trigger config, and a staleness summary comparing the child's effective member set against the base.

## Key Entities

### Template
- Has a unique name/ID.
- Optionally extends a parent template (inheritance).
- Contains zero or more composed feature modules (composition).
- Defines attributes, alarms, and scripts as first-class members.
- Cannot be deleted if referenced by instances or child templates.
- Concurrent editing uses **last-write-wins** — no pessimistic locking or conflict detection.
- May belong to a `TemplateFolder` via nullable `FolderId`, or live at the tree root when null.

### TemplateFolder
- Hierarchical organizational entity with a self-referencing `ParentFolderId` (null at the root).
- Sibling folder names are unique (case-insensitive) within the same parent.
- Folders carry **no semantic meaning** for template resolution, flattening, validation, or inheritance — they exist purely for UI organization.
- Folder deletion is blocked if the folder contains any subfolders or templates.
- The folder graph is enforced acyclic on move (a folder cannot become its own descendant).
- Each folder carries an integer **`SortOrder`** (assigned distinctly on create; default 0) used for sibling reorder. `ReorderTemplateFolderCommand` swaps the `SortOrder` of two adjacent siblings; the UI exposes Move-up / Move-down menu items in the folder context menu. Drag-drop reorganization is deliberately deferred.

### Attribute
- Name, Value, Data Type (Boolean, Integer, Float, String), Lock Flag, Description.
- Optional Data Source Reference — a **relative path** within a data connection (e.g., `/Motor/Speed`). The template defines *what* to read but not *where* to read it from. The connection binding is an instance-level concern.
- Value may be empty if intended to be set at instance level or via data connection binding.

### Alarm
- Name, Description, Priority Level (0–1000), Lock Flag.
- Trigger Definition: Value Match, Range Violation, Rate of Change, HiLo, or Expression.
- Optional On-Trigger Script reference.
- Expression triggers carry an **`analysisKind`** field (`Advisory` | `Strict`) in the trigger-config JSON. `Advisory` (default) keeps the current behavior where a blank expression is a non-blocking advisory finding. `Strict` escalates a blank expression to a **deploy-blocking error**. The `analysisKind` is set per-trigger in the UI or via the CLI `--trigger-kind` option.

### Native Alarm Source (`TemplateNativeAlarmSource`)
- A read-only binding that mirrors **native alarms** raised by an upstream system — OPC UA Alarms & Conditions or the MxAccess Gateway — rather than alarms evaluated by the Site Runtime from attribute values.
- Fields: Name, Description *(optional)*, ConnectionName (the data connection that carries the native alarms), SourceReference (a raw connection address — an OPC UA `SourceNode` nodeId, or an MxAccess object/area), ConditionFilter *(optional — when null, mirror **all** conditions under the source)*, and the standard locking flags (`IsLocked`, `IsInherited`, `LockedInDerived`).
- `SourceReference` is a **raw connection address**, not a relative attribute path — the Template Engine does not interpret or rewrite it (contrast with an attribute's `DataSourceReference`).
- Defined on a template as a first-class member via `Template.NativeAlarmSources`.
- Resolved native alarm sources drive the Site Runtime's **NativeAlarmActor** (see Interactions); the Template Engine only models and flattens them.

### Script (Template-Level)
- Name, Lock Flag, C# source code.
- Trigger configuration: Interval, Value Change, Conditional, Expression, or invoked by alarm/other script. Conditional and Expression triggers also carry a fire mode — **OnTrue** (fire as the condition becomes true) or **WhileTrue** (re-fire on a timer while it stays true).
- Expression triggers carry an **`analysisKind`** field (`Advisory` | `Strict`) in the trigger-config JSON, with the same semantics as alarm Expression triggers above.
- Optional minimum time between runs — also the re-fire cadence for a WhileTrue trigger.
- **Parameter Definition** *(optional)*: Defines input parameters (name and data type per parameter). Scripts without parameters accept no arguments. Parameters that take a JSON `object` or `list` value may carry a JSON Schema definition (plain schema or `{"$ref":"lib:Name"}` pointing to the schema library) used by the Central UI value-entry form and Monaco hover/completion.
- **Return Value Definition** *(optional)*: Defines the structure of the script's return value (field names and data types). Supports single objects and lists of objects. Scripts without a return definition return void.

### Instance
- Associated with a specific template and a specific site.
- Assigned to an area within the site.
- Can override non-locked attribute values (no adding/removing attributes).
- Can override non-locked native alarm source bindings via `Instance.NativeAlarmSourceOverrides` (see Override Granularity) — no adding/removing sources.
- Bound to data connections at instance creation — **per-attribute binding** where each attribute with a data source reference individually selects its data connection.
- Can be in **enabled** or **disabled** state.
- Can be **deleted** — deletion is blocked if the site is unreachable.

### Area
- Hierarchical groupings per site (parent-child).
- Stored in the configuration database.
- Used for filtering/organizing instances in the UI.

## Composed Member Addressing

When a template composes a feature module, members from that module are addressed using a **path-qualified canonical name**: `[ModuleInstanceName].[MemberName]`. For nested compositions, the path extends: `[OuterModule].[InnerModule].[MemberName]`.

- All internal references (triggers, scripts, diffs, stream topics, UI display) use canonical names.
- The composing template's own members (not from a module) have no prefix — they are top-level names.
- Naming collision detection operates on canonical names, so two modules can define the same member name as long as their module instance names differ.

### Derived template naming

A composition slot is materialized as its own *derived* template. A derived
template stores a **contained name** — the composition slot's instance name
(e.g. `Pump`), unique only within its owner. The **qualified name**
(`Motor Controller.Pump`, or `Motor Controller.Pump.TempSensor` when nested) is
*computed* on read by walking the owner-composition chain — it is not stored.
Only base (user-authored) templates are globally unique by name; a derived
template's uniqueness is the slot-name uniqueness within its owner. The Central
UI shows the contained name as the template title and the qualified path as a
breadcrumb.

## Override Granularity

Override and lock rules apply per entity type at the following granularity:

- **Attributes**: Value and Description are overridable. Data Type is fixed by the defining level. `DataSourceReference` on a template attribute defines the **default** physical address for that attribute. Instances may override per attribute via `InstanceConnectionBinding.DataSourceReferenceOverride`; the override replaces the template default at flattening time. When the override is null (the default), the template value is used. Lock applies to the entire attribute (when locked, no fields can be overridden).
- **Alarms**: Priority Level, Trigger Definition (thresholds/ranges/rates), Description, and On-Trigger Script reference are overridable. Name and Trigger Type (Value Match vs. Range vs. Rate of Change) are fixed. Lock applies to the entire alarm.
- **Native alarm sources**: An instance overrides a non-locked source via `InstanceNativeAlarmSourceOverride`, keyed by `SourceCanonicalName`. `ConnectionNameOverride`, `SourceReferenceOverride`, and `ConditionFilterOverride` are individually overridable — each is applied only when non-null; a null field **keeps the inherited value**. Name is fixed. Lock applies to the entire source.
- **Scripts**: C# source code, Trigger configuration, minimum time between runs, and parameter/return definitions are overridable. Name is fixed. Lock applies to the entire script.
- **Composed module members**: A composing template or child template can override non-locked members inside a composed module using the canonical path-qualified name.

## Naming Collision Detection

When a template composes two or more feature modules, the system must check for naming collisions across:
- Attribute names
- Alarm names
- Script names

If any composed module introduces a name that already exists (from another composed module or from the composing template itself), this is a **design-time error**. The template cannot be saved until the conflict is resolved. Collision detection is performed recursively for nested module compositions.

A derived template stores `IsInherited` placeholder rows that mirror the members it inherits from its parent chain. These placeholders are **excluded** from collision detection on the template's own member set and on the inherited-parent walk — the inheritance walk already re-adds those members under the parent's origin, so counting the placeholder copies as well would report a spurious self-collision against the very member they mirror. (Placeholders are only counted when a composed module is itself a derived template, because the composed-module walk does not climb that module's parent chain and the placeholders are then the sole representation of its inherited members.) Without this exclusion there is no supported way to add an attribute, alarm, script, or composition to a derived template — every add is blocked by false collisions on the pre-existing inherited rows.

## Flattening Process

When an instance is deployed, the Template Engine resolves the full configuration:

1. Start with the base template's attributes, alarms, and scripts.
2. Walk the inheritance chain, applying overrides at each level (respecting locks).
3. Resolve composed feature modules, applying overrides from composing templates (respecting locks).
4. Apply instance-level overrides (respecting locks).
5. Resolve data connection bindings — replace connection name references with concrete connection details from the site.
6. Output a flat structure: list of attributes with resolved values and data source addresses, list of alarms with resolved trigger definitions, list of scripts with resolved code and triggers.

### Native Alarm Source Resolution

The `FlatteningService` resolves native alarm sources alongside alarms, emitting a `ResolvedNativeAlarmSource` (CanonicalName, ConnectionName, SourceReference, ConditionFilter *(optional)*, and `Source` ∈ `Template` | `Inherited` | `Composed` | `Override`) for each. The resolved set is attached to `FlattenedConfiguration.NativeAlarmSources`.

- **Inheritance**: resolution walks the chain base → derived; a derived-level source wins over the base unless the base level locked it.
- **Composition**: a composed module's sources are path-qualified to the canonical name `[ModuleInstanceName].[Name]`, subject to the same naming-collision checks as other members. Because `SourceReference` is a raw connection address (not an attribute path), composition performs **no attribute-reference rewriting** on it.
- **Instance overrides**: `InstanceNativeAlarmSourceOverride` applies its non-null fields (`ConnectionNameOverride`, `SourceReferenceOverride`, `ConditionFilterOverride`) over the inherited/composed result and sets `Source = Override`.

## Inheritance Resolve — Authoring View

`GetResolvedTemplateMembersCommand` (handled by `TemplateInheritanceResolver`) returns the **full multi-level effective inherited member set** for a template — the set of members a child template would see if it were the leaf of the inheritance chain. This is a **read-only** query; it does not mutate any template or instance, and it has no effect on the deploy pipeline.

The resolved result carries, per member:
- **Canonical name** and **member kind** (Attribute, Alarm, Script, NativeAlarmSource).
- **Origin** — which template in the chain (by name and ID) first defined the member.
- **Locked** — whether the member is locked at the level it was last defined, preventing further override.
- **Merged HiLo trigger config** — for alarms with a HiLo trigger, the merged setpoint object (base + any intermediate override).
- **Staleness summary** — a flag indicating whether the inherited effective value differs from what the requesting template's own copy has stored (i.e., the base has changed since the child was last edited). This drives the read-only "base changed" banner in `TemplateEdit`.

The resolver is consumed only by the Central UI `TemplateEdit` page. It is not part of the flattening pipeline and is not called during deployment.

## Inherited-Member Propagation & Resync

A derived template stores `IsInherited` **placeholder rows** mirroring every member it inherits (attributes, alarms, scripts, native alarm sources). These placeholders are what the editor's editable member tabs render and what the staleness summary above compares against. They are materialized when the derived template is created (compose / inherit), and kept in sync by two mechanisms so the stored rows never drift incomplete:

- **Auto-propagation.** Whenever a member is **added, updated, or removed** on a template, the change is propagated to that template's entire derived subtree: a missing inherited placeholder is materialized, a drifted one is re-synced to the live effective value, and an orphaned one (its base member removed) is deleted. This runs automatically as part of the member-mutating commands, so children stay complete going forward.
- **Resync (operator repair).** `ResyncInheritedMembersAsync` (CLI `template resync-members`, and a **Resync** button on the editor's "base changed" banner) reconciles a template **and its subtree** on demand — repairing templates that drifted before auto-propagation existed (e.g. base members added after a child was derived). Resyncing a base repairs every derivation; resyncing a leaf repairs just it.

Both delegate to one **order-independent reconcile** that compares a template's stored inherited rows against the inheritance resolver's effective set (the same precedence + HiLo merge the editor preview and deploy use) and only ever touches `IsInherited` placeholder rows — never an authored override (`IsInherited == false`). Because the resolver ignores placeholder rows when picking winners, reconciling one template never changes what another resolves, so the operation needs no particular ordering. The effective value mirrored into each placeholder matches the staleness comparison key per member type, so after a reconcile the "base changed" banner clears. Reconcile is best-effort housekeeping for the *stored authoring rows*: a deploy always re-resolves the chain fresh regardless, so a not-yet-resynced template still deploys correctly.

## Diff Calculation

The Template Engine can compare:
- The **currently deployed** flat configuration of an instance.
- The **current template-derived** flat configuration (what the instance would look like if redeployed now).

The diff output identifies added, removed, and changed attributes/alarms/scripts.

## Pre-Deployment Validation

Before a deployment is sent to a site, the Template Engine performs comprehensive validation:

- **Flattening**: The full template hierarchy resolves and flattens without errors.
- **Naming collision detection**: No duplicate attribute, alarm, or script names in the flattened configuration.
- **Script compilation and trust check**: All instance scripts and alarm on-trigger scripts are compiled via `ScriptCompiler.TryCompile`, which performs a real Roslyn type-checking compile (using `ScriptCompileSurface` from the Script Analysis component as the globals type) and an authoritative forbidden-API check (via `ScriptTrustValidator.FindViolations`). Scripts with compilation errors, type errors, or forbidden-API violations are rejected. This gate is authoritative — not advisory — meaning alias, `using static`, and `global::` bypasses are caught by semantic symbol resolution.
- **Alarm trigger references**: Alarm trigger definitions reference attributes that exist in the flattened configuration.
- **Script trigger references**: Script triggers (value change, conditional) reference attributes that exist in the flattened configuration.
- **Data connection binding completeness**: Every attribute with a data source reference has a data connection binding assigned on the instance, and the bound data connection name exists as a defined connection at the instance's site.
- **Exception**: Validation does **not** verify that data source relative paths resolve to real tags on physical devices — that is a runtime concern.

### Semantic Validation

Beyond compilation, the Template Engine performs static semantic checks:

- **Script call targets**: `Instance.CallScript()` and `Scripts.CallShared()` targets must reference scripts that exist in the flattened configuration or shared script library.
- **Argument compatibility**: Parameter count and data types at call sites must match the target script's parameter definitions.
- **Return type compatibility**: If a script call's return value is used, the return type definition must match the caller's expectations.
- **Trigger operand types**: Alarm triggers and script conditional triggers must reference attributes with compatible data types (e.g., Range Violation requires numeric attributes).
- **Native alarm sources** (`ValidationCategory.NativeAlarmSourceInvalid`): `SemanticValidator.Validate` flags a `ResolvedNativeAlarmSource` when its `SourceReference` is empty, its `ConnectionName` is empty, or — when the caller supplies the alarm-capable connection set — its connection is unknown or not alarm-capable (protocol ∉ {`OpcUa`, `MxGateway`}). The alarm-capable connection set is an **optional, additive third parameter** to `Validate`; the empty-field checks always run, and the connection-binding check runs only when the set is provided.

### Graph Acyclicity

The Template Engine enforces that inheritance and composition graphs are **acyclic**:

- A template cannot inherit from itself or from any descendant in its inheritance chain.
- A template cannot compose itself or any ancestor/descendant that would create a circular composition.
- These checks are performed on save.

### Flattened Configuration Revision

Each flattened configuration output includes a **revision hash** (computed from the resolved content). This hash is used for:

- Staleness detection: comparing the deployed revision to the current template-derived revision without a full diff.
- Diff correlation: ensuring diffs are computed against a consistent baseline.

The override flows into the flattened attribute's `DataSourceReference` and therefore participates in the revision hash — changes to an instance's binding overrides re-deploy as expected.

### On-Demand Validation

The same validation logic is available to Design users in the Central UI without triggering a deployment. This allows template authors to check their work for errors during authoring.

### Shared Script Validation

For shared scripts, pre-compilation validation is performed before deployment. Since shared scripts have no instance context, validation uses `RoslynScriptCompiler.ParseDiagnostics` for syntax errors and `ScriptTrustValidator.FindViolations` for the forbidden-API check. Full type-checking compilation is not performed (no globals type is available), but syntax and trust violations are caught and reported.

## Dependencies

- **Configuration Database (MS SQL)**: Stores all templates, instances, areas, and their relationships.
- **Security & Auth**: Enforces Design role for template authoring, Deployment role for instance management.
- **Configuration Database (via IAuditService)**: All template and instance changes are audit logged.
- **Script Analysis (#25)**: `ScriptCompiler.TryCompile` uses `RoslynScriptCompiler` and `ScriptTrustValidator.FindViolations` from this component for the deploy-gate script compilation and forbidden-API check. `ValidationService.CheckExpressionSyntax` uses `ScriptTrustValidator` and `TriggerCompileSurface` for expression trigger validation.

## Interactions

- **Deployment Manager**: Requests flattened configurations, diffs, and validation results from the Template Engine.
- **Central UI**: Provides the data model for template authoring, instance management, and on-demand validation. Native alarm source CRUD (template-level definitions and instance-level overrides) is exposed via the Management Service / CLI / Central UI alongside attributes and alarms.
- **Site Runtime (#3)**: Consumes each `ResolvedNativeAlarmSource` in the flattened configuration to drive its **NativeAlarmActor**, which mirrors the native OPC UA A&C / MxAccess Gateway alarms identified by the resolved connection, source reference, and condition filter.
- **Transport (#24)**: Reads templates, attributes, alarms, scripts, and composition relationships for bundle export; writes the same via repositories during bundle import.