From 84a050420fd2a7870eb3abeb1e288b335abac43a Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Tue, 10 Nov 2026 05:02:06 -0500 Subject: [PATCH] fix(transport): carry derived composition-slot templates and rebuild them through the compose flow (ScadaBridge#118) Bundles did not record that a template was a derived slot template, so the importer re-created every slot as a BASE template named after the slot. - Export: TemplateDto gains an additive, null-omitted `Derivation` (owner path + base path by stable names); a derived template carries only its own rows (overrides/additions, not inherited placeholders). The closure pulls in each derived template's base/owner; base orders first. schemaVersion 1.1 -> 1.2, bundleFormatVersion stays 1. - Import: base templates go through the ordinary passes; derived templates are ordered topologically (DerivedTemplatePlan; cycle / duplicate / malformed = preview Blocker + SemanticValidationException before anything is staged), their owner and base resolved by path (#100 root rule), and the slot is created with TemplateService.AddCompositionAsync, synced in place when it already exists with the same base, or deleted+recomposed when the base differs; own rows applied as overrides, then ResyncInheritedMembersAsync. Missing owner/base fails that record (BundleImportDerivedTemplateUnresolved audit + ImportResult warning), never creating a base in its place. - Preview/diff: derived rows are keyed by qualified name and compared against the target slot at the same owner path (own rows + base), so an unchanged slot is Identical and a same-named base template is never matched. - Pre-#118 bundles import exactly as before (new fixture pre-118-derived-plaintext.scadabundle, written by a13aabc7). The field is in content, pinned by the signed manifest's contentHash. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV --- docs/requirements/Component-Transport.md | 18 +- .../Export/DependencyResolver.cs | 34 +- .../Import/ArtifactDiff.cs | 65 +- .../Import/BundleImporter.Derived.cs | 457 ++++++++++++ .../Import/BundleImporter.cs | 100 ++- .../Import/DerivedTemplatePlan.cs | 127 ++++ .../Serialization/EntityDtos.cs | 51 ++ .../Serialization/EntitySerializer.cs | 81 ++- .../Serialization/ManifestBuilder.cs | 5 +- .../pre-118-derived-plaintext.scadabundle | Bin 0 -> 892 bytes .../Import/DerivedTemplateTransportTests.cs | 678 ++++++++++++++++++ .../Export/DependencyResolverTests.cs | 25 + .../Import/DerivedTemplatePlanTests.cs | 74 ++ .../Serialization/ManifestBuilderTests.cs | 2 +- 14 files changed, 1676 insertions(+), 41 deletions(-) create mode 100644 src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.Derived.cs create mode 100644 src/ZB.MOM.WW.ScadaBridge.Transport/Import/DerivedTemplatePlan.cs create mode 100644 tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Fixtures/pre-118-derived-plaintext.scadabundle create mode 100644 tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/DerivedTemplateTransportTests.cs create mode 100644 tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Import/DerivedTemplatePlanTests.cs diff --git a/docs/requirements/Component-Transport.md b/docs/requirements/Component-Transport.md index 4f732261..06ce579b 100644 --- a/docs/requirements/Component-Transport.md +++ b/docs/requirements/Component-Transport.md @@ -59,7 +59,7 @@ zip-bomb guard, and the exporter never emits per-script files. ```json { "bundleFormatVersion": 1, - "schemaVersion": "1.1", + "schemaVersion": "1.2", "createdAtUtc": "2026-05-24T12:34:56Z", "sourceEnvironment": "dev-cluster-a", "exportedBy": "alice@corp.example", @@ -106,6 +106,7 @@ The manifest is plaintext so the import wizard can preview bundle contents and s - Unknown top-level entity types in `contents[]` surface in the import preview as "skipped — unsupported in this version" rather than failing the whole import. - `bundleFormatVersion` newer than what the importer supports produces a hard refusal at upload with a clear upgrade message. - `schemaVersion` minor increments are **additive-only**. T18 bumped it `1.0 → 1.1` (adding the `sites`/`dataConnections`/`instances` arrays). `bundleFormatVersion` stays `1`; `ManifestValidator` gates solely on `bundleFormatVersion`, so a pre-T18 importer still accepts a 1.1 bundle and simply lists the new site/instance/connection entries as "skipped — unsupported in this version". +- ScadaBridge#118 bumped it `1.1 → 1.2`: a derived (composition-slot) template carries a `Derivation` object (see [Derived templates](#derived-composition-slot-templates-scadabridge118)). It is omitted on base templates (`WhenWritingNull`), so a base template's JSON is unchanged; an older importer ignores the unknown property and imports the slot the old (wrong) way; a 1.0/1.1 bundle has no derivation anywhere and imports exactly as before (pinned by `Fixtures/pre-118-derived-plaintext.scadabundle`). `bundleFormatVersion` stays `1`. The field lives in `content.json`/`content.enc`, whose SHA-256 is the manifest's `contentHash`, so a signed bundle's signature over `manifest.json` covers it (an edited derivation fails the hash check, or — with a recomputed hash — the signature). ## Architecture @@ -314,6 +315,16 @@ Site identifiers and connection names are environment-specific, so a bundle expo - `InstanceConnectionBinding.DataConnectionId` is resolved via the connection map, with a **Pass-2** that also binds connections which already exist in the target but were not carried in the bundle. - `InstanceNativeAlarmSourceOverride.ConnectionNameOverride` is rewritten to the **mapped target** connection name. +### Derived (composition-slot) templates (ScadaBridge#118) + +Composing a template into a slot creates a slot-owned **derived** template (`IsDerived`, `ParentTemplateId` = the base, `OwnerCompositionId` = the slot) named after the slot. Before #118 the bundle did not say so, and the importer re-created every such template as a **base** template named after its slot. Owner decision (2026-11-10): carry the relationship and rebuild the slot through the normal compose flow. + +- **Export.** A derived template's `TemplateDto` carries `Derivation` = `{ OwnerTemplateName, SlotPath, BaseTemplateName, BaseSlotPath }`: its owner path (the base template at the root of the owner chain plus the slot names down to it — its qualified name, e.g. `Pump.M1` or `Station.P.M1`) and its base path (a base template, or for a cascade-created nested slot the source-side derived template, e.g. `Pump.M1`). Its member lists carry only its **own** rows (`IsInherited == false`: overrides and additions), not the inherited placeholders. The dependency closure pulls in each derived template's base (and owner), and the manifest order puts the base first. +- **Identity.** A derived template is previewed and resolved under its **qualified name** (`Template` / `Pump.M1`), compared against the target's template at the same owner path — never against a base template that shares the slot name. The diff compares the derived-ness, the base (by qualified name) and the target slot's own rows only, so a re-import of an unchanged slot is `Identical` (auto-Skip). A Rename resolution on a derived row is treated as Add/Overwrite (the slot name is fixed by the owner). +- **Order.** `DerivedTemplatePlan` orders derived templates after their owner and base (topological — the bundle order does not matter). A cycle, a duplicate qualified name or a malformed derivation is a preview Blocker and a `SemanticValidationException` before anything is staged. +- **Apply.** The ordinary template passes see base templates only (and the owner's composition pass leaves the rows of its bundled derived slots alone). After the acyclicity guard, each derived template's owner and base are resolved by path — root by the #100 rule (exact name, base template first, a case-variant base as fallback; a name held only by derived templates is never a root) then slot by slot — in the bundle-applied/target state. A missing slot is created with `TemplateService.AddCompositionAsync` (the Central UI / management API compose flow: derived row, placeholders, nested cascade slots, cycle and collision checks); an existing slot with the same base is synced **in place** (same row, so nested slots deriving from it keep their FK); one with another base is removed with `DeleteCompositionAsync` and recomposed. The bundled own rows are then applied as overrides, own rows the bundle no longer carries revert to placeholders, and `ResyncInheritedMembersAsync` reconciles the placeholders. Derived templates skip the import-time one-template semantic validation (their inherited members are not in the DTO); the trust gate still runs over their scripts. +- **Missing owner/base.** If either resolves in neither the bundle nor the target (e.g. the base was Skipped), that record fails: a `BundleImportDerivedTemplateUnresolved` audit row and an `ImportResult.Warnings` entry name it, and no base template is created in its place. + ### Stale-Instance Signaling There is no explicit stale-mark write. Overwriting a template during import changes its content, which naturally changes the flattened-config hash that `DeploymentService.CompareAsync` computes at query time. The Deployments page already surfaces any instance whose computed hash differs from the deployed snapshot — no new mechanism is required. @@ -405,6 +416,7 @@ Import flows through the same audited repository methods the UI and CLI use, so | Imported alarm references missing on-trigger script | `BundleImportAlarmScriptUnresolved` (warning; alarm FK left null) | | Imported template's composition references missing target template | `BundleImportCompositionUnresolved` (warning; composition row not written) | | Imported derived template's base template missing (unresolved / skipped) | `BundleImportBaseTemplateUnresolved` (warning; `ParentTemplateId` left null) | +| Imported derived composition-slot template's owner or base missing (#118) | `BundleImportDerivedTemplateUnresolved` (warning; the slot is not created and no base template is created instead) | **Correlation:** every per-entity row written during an import carries a new optional `BundleImportId` column (the GUID of the parent `BundleImported` summary row). The repository layer (`CentralUiRepository.QueryConfigurationAuditAsync`) already accepts a `BundleImportId` filter parameter. The Configuration Audit Log Viewer UI surface — a "Bundle Import" filter dropdown and a hyperlink on the `BundleImported` summary row that pre-populates that filter — is a deferred UI follow-up (tracked under Transport-012 in the code review backlog). Until then, an operator can drive the same filter via the CLI `audit query --bundle-import-id `. @@ -476,7 +488,7 @@ The `manifest.json` file is always present in the ZIP root and is never encrypte ```json { "bundleFormatVersion": 1, - "schemaVersion": "1.1", + "schemaVersion": "1.2", "createdAtUtc": "2026-05-24T12:34:56Z", "sourceEnvironment": "dev-cluster-a", "exportedBy": "alice@corp.example", @@ -543,7 +555,7 @@ The `manifest.json` file is always present in the ZIP root and is never encrypte | Field | Description | |---|---| | `bundleFormatVersion` | Integer. Importer hard-refuses any value other than the code constant `ManifestBuilder.CurrentBundleFormatVersion`. Stays `1` through T18. The schema and format versions are code constants by design — there is no `Transport:SchemaVersionMajor` option (removed 2026-10-22; it was validated but never read). | -| `schemaVersion` | Semver string. Minor increments are additive-only and accepted by older importers. T18 bumped this `1.0 → 1.1` (adding the `sites`/`dataConnections`/`instances` content arrays); a pre-T18 importer lists those new entries as "skipped — unsupported in this version" since `ManifestValidator` gates only on `bundleFormatVersion`. | +| `schemaVersion` | Semver string. Minor increments are additive-only and accepted by older importers. T18 bumped this `1.0 → 1.1` (adding the `sites`/`dataConnections`/`instances` content arrays); a pre-T18 importer lists those new entries as "skipped — unsupported in this version" since `ManifestValidator` gates only on `bundleFormatVersion`. ScadaBridge#118 bumped it `1.1 → 1.2` (the additive `Derivation` on derived composition-slot templates). | | `createdAtUtc` | ISO-8601 UTC timestamp of when the export was created. | | `sourceEnvironment` | The `SourceEnvironment` name of the exporting cluster (from `TransportOptions`). Displayed in the import wizard and required to be retyped at the confirm step. | | `exportedBy` | Authenticated username of the person who performed the export. | diff --git a/src/ZB.MOM.WW.ScadaBridge.Transport/Export/DependencyResolver.cs b/src/ZB.MOM.WW.ScadaBridge.Transport/Export/DependencyResolver.cs index ce8e0ef0..7a8e0e8b 100644 --- a/src/ZB.MOM.WW.ScadaBridge.Transport/Export/DependencyResolver.cs +++ b/src/ZB.MOM.WW.ScadaBridge.Transport/Export/DependencyResolver.cs @@ -422,11 +422,21 @@ public sealed class DependencyResolver IReadOnlyCollection