Transport: derived (composition-slot) templates round-trip as BASE templates — IsDerived/OwnerCompositionId are not in the bundle #118

New Issue

2026-11-10T04:38:20-05:00

dohertj2 commented

2026-11-10 04:38:20 -05:00

Found while fixing #100 item 1 (code reading; not reproduced end to end).

TemplateDto (src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntityDtos.cs) carries no IsDerived / OwnerCompositionId.
The exporter's composition closure (DependencyResolver.ExpandTemplateClosureAsync) pulls every ComposedTemplateId into the bundle, and the compose flow points a composition at its slot-owned derived template (TemplateService.BuildDerivedTemplate(source, instanceName) — the derived template is named after the composition slot).
The importer builds every bundled template as new Template(name) { Description = … } (BundleImporter.cs, the Add path), so an imported derived template lands as a base template named after the slot (e.g. Motor1), visible in the template tree, inside the filtered unique name index, and no longer cascade-owned by its composition.

Consequences to confirm: two compositions with the same slot name in one bundle produce two TemplateDtos with the same name (collide on import); a slot name equal to an existing base template name collides with it (or Overwrites it); deleting the composition on the target no longer deletes its slot template.

This is a design decision for the owner (carry IsDerived + owner (TemplateName, InstanceName) additively in the DTO and recreate the derived row through the compose flow, vs. flatten derived templates out of the bundle). #100 item 1 only fixed by-name reference resolution against the target (base-first, ambiguous derived-only names unresolved); it does not address this.

Found while fixing #100 item 1 (code reading; not reproduced end to end). - `TemplateDto` (`src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntityDtos.cs`) carries no `IsDerived` / `OwnerCompositionId`. - The exporter's composition closure (`DependencyResolver.ExpandTemplateClosureAsync`) pulls every `ComposedTemplateId` into the bundle, and the compose flow points a composition at its slot-owned **derived** template (`TemplateService.BuildDerivedTemplate(source, instanceName)` — the derived template is named after the composition slot). - The importer builds every bundled template as `new Template(name) { Description = … }` (`BundleImporter.cs`, the Add path), so an imported derived template lands as a **base** template named after the slot (e.g. `Motor1`), visible in the template tree, inside the filtered unique name index, and no longer cascade-owned by its composition. Consequences to confirm: two compositions with the same slot name in one bundle produce two `TemplateDto`s with the same name (collide on import); a slot name equal to an existing base template name collides with it (or Overwrites it); deleting the composition on the target no longer deletes its slot template. This is a design decision for the owner (carry `IsDerived` + owner `(TemplateName, InstanceName)` additively in the DTO and recreate the derived row through the compose flow, vs. flatten derived templates out of the bundle). #100 item 1 only fixed by-name reference resolution against the target (base-first, ambiguous derived-only names unresolved); it does not address this.

dohertj2 added the type:bug priority:p2 labels 2026-11-10 04:38:20 -05:00

dohertj2 referenced this issue

2026-11-10 04:38:47 -05:00

Import: GetIdByNameAsync derived-template ambiguity; intra-record exact-name lookups; reuse the PSK pub-sub invalidation for D1/D3 #100

dohertj2 commented

2026-11-10 04:39:52 -05:00

Owner decision 2026-11-10: carry the derived/base template relationship in the bundle and rebuild the derived slot through the normal compose flow on import (recommended option). Implementation in progress.

**Owner decision 2026-11-10:** carry the derived/base template relationship in the bundle and rebuild the derived slot through the normal compose flow on import (recommended option). Implementation in progress.

dohertj2 commented

2026-11-10 05:03:49 -05:00

Fixed on main @ 84a05042. Implemented the owner decision: the bundle now carries the derived/base relationship, and the import rebuilds the slot through the normal compose flow.

Format. TemplateDto gets an additive field, Derivation = { OwnerTemplateName, SlotPath, BaseTemplateName, BaseSlotPath }. It names the owner path and the base path by stable names. Examples: Pump.M1, and for a nested slot the cascade created, Station.P.M1 with base Pump.M1.

The field is omitted on base templates (WhenWritingNull), so a base template's JSON is unchanged.
A derived template carries only its own rows (IsInherited == false, meaning overrides and additions). It does not carry the inherited placeholders.
schemaVersion 1.1 → 1.2. bundleFormatVersion stays 1, so no format bump: older importers ignore the property.
The field sits in content.json/content.enc. The manifest's contentHash covers that content, so the #45 signature over manifest.json covers it too.
The export closure now pulls in each derived template's base and owner, and orders the base first.

Import.

The ordinary template passes see base templates only. The owner's composition pass no longer deletes or writes its bundled derived slots.
DerivedTemplatePlan orders derived templates topologically after their owner and base, so bundle order does not matter.
A cycle, a duplicate qualified name, or a malformed derivation is a preview Blocker and a SemanticValidationException, raised before anything is staged.
After the acyclicity guard, owner and base are resolved by path. The root uses the #100 rule: exact name, base template first, a case-variant base as fallback, and a name held only by derived templates is never a root. Each slot is then matched by composition.
Then:
- Missing slot: created with TemplateService.AddCompositionAsync.
- Existing slot, same base: synced in place. It keeps the same row id, so nested slots that derive from it keep their FK.
- Existing slot, different base: removed with DeleteCompositionAsync and recomposed.
The bundled own rows are applied as overrides. Own rows the bundle no longer carries revert to placeholders, and ResyncInheritedMembersAsync then reconciles them.
Missing owner or base (for example, the base was Skipped and is not on the target): that record fails, with a BundleImportDerivedTemplateUnresolved audit row and an ImportResult.Warnings entry. No base template is created in its place.

Differential / identical-skip. Derived rows are previewed and resolved under their qualified name (Template / Pump.M1). Each is compared against the target's template at the same owner path, never against a base template with the same name. The comparison covers derived-ness, the base (qualified) and the target slot's own rows. An unchanged slot reads Identical, and a changed override reads Modified on that row only.

Compatibility. New fixture Fixtures/pre-118-derived-plaintext.scadabundle, exported by a13aabc7 (pre-fix code), contains a Pump whose M1 slot is a derived Motor with Speed overridden to 5. It imports exactly as before: M1 arrives as a base template inheriting Motor, and Pump composes it. The pre-signing-* fixtures and LegacyBundleCompatibilityTests are unchanged and pass.

Tests. DerivedTemplateTransportTests (10), DerivedTemplatePlanTests (3) and one DependencyResolverTests case cover:

a round trip of a base+derived pair (the slot stays derived, owned and inheriting, with its override, its addition and its inherited placeholders);
identity (Identical on re-import, Modified on one override, a same-named base left untouched);
a nested cascade slot;
derived templates listed before their base and owner;
a missing base, and a base present only on the target;
in-place Overwrite with stale-row revert, while the owner is also Overwritten;
a cycle;
the pre-#118 fixture;
a signed bundle carrying the field (it verifies, and editing the derivation fails the hash check or, with a recomputed hash, the signature).

Mutation-proved, every mutation killed:

the serializer drops Derivation: 7 tests fail;
derived templates go through the base pass: 5 fail;
the plan uses bundle order: 2 fail;
no cycle check: the cycle test fails;
the diff compares placeholders: the identity test fails;
the composition pass deletes derived slots: the Overwrite test fails;
no stale revert: the Overwrite test fails;
the missing-base audit is dropped: the missing-base test fails.

Evidence (after rebasing onto fb04b54d): dotnet build ZB.MOM.WW.ScadaBridge.slnx 0 warnings / 0 errors. All passed: Transport.Tests 236, Transport.IntegrationTests 161, TemplateEngine.Tests 617, ManagementService.Tests 790, CentralUI.Tests 1540, CLI.Tests 592 (TemplateEngine and CLI were run before the rebase). Not deployed. The design is documented in Component-Transport.md, "Derived (composition-slot) templates (ScadaBridge#118)".

Seen on the way, not changed (TemplateEngine): once a derived slot holds an override, TemplateService.AddAttributeAsync on that slot fails with "Naming collision on 'Speed': Attribute from M1, Attribute from parent 'Motor'". So an addition is only possible before the first override. Worth a separate look.

Fixed on main @ 84a05042. Implemented the owner decision: the bundle now carries the derived/base relationship, and the import rebuilds the slot through the normal compose flow. **Format.** `TemplateDto` gets an additive field, `Derivation = { OwnerTemplateName, SlotPath, BaseTemplateName, BaseSlotPath }`. It names the owner path and the base path by stable names. Examples: `Pump.M1`, and for a nested slot the cascade created, `Station.P.M1` with base `Pump.M1`. - The field is omitted on base templates (`WhenWritingNull`), so a base template's JSON is unchanged. - A derived template carries only its **own** rows (`IsInherited == false`, meaning overrides and additions). It does not carry the inherited placeholders. - `schemaVersion` 1.1 → 1.2. `bundleFormatVersion` stays 1, so no format bump: older importers ignore the property. - The field sits in `content.json`/`content.enc`. The manifest's `contentHash` covers that content, so the #45 signature over `manifest.json` covers it too. - The export closure now pulls in each derived template's base and owner, and orders the base first. **Import.** - The ordinary template passes see base templates only. The owner's composition pass no longer deletes or writes its bundled derived slots. - `DerivedTemplatePlan` orders derived templates topologically after their owner and base, so bundle order does not matter. - A cycle, a duplicate qualified name, or a malformed derivation is a preview Blocker and a `SemanticValidationException`, raised before anything is staged. - After the acyclicity guard, owner and base are resolved by path. The root uses the #100 rule: exact name, base template first, a case-variant base as fallback, and a name held only by derived templates is never a root. Each slot is then matched by composition. - Then: - **Missing slot:** created with `TemplateService.AddCompositionAsync`. - **Existing slot, same base:** synced in place. It keeps the same row id, so nested slots that derive from it keep their FK. - **Existing slot, different base:** removed with `DeleteCompositionAsync` and recomposed. - The bundled own rows are applied as overrides. Own rows the bundle no longer carries revert to placeholders, and `ResyncInheritedMembersAsync` then reconciles them. - **Missing owner or base** (for example, the base was Skipped and is not on the target): that record fails, with a `BundleImportDerivedTemplateUnresolved` audit row and an `ImportResult.Warnings` entry. No base template is created in its place. **Differential / identical-skip.** Derived rows are previewed and resolved under their qualified name (`Template` / `Pump.M1`). Each is compared against the target's template at the same owner path, never against a base template with the same name. The comparison covers derived-ness, the base (qualified) and the target slot's own rows. An unchanged slot reads Identical, and a changed override reads Modified on that row only. **Compatibility.** New fixture `Fixtures/pre-118-derived-plaintext.scadabundle`, exported by a13aabc7 (pre-fix code), contains a Pump whose M1 slot is a derived Motor with Speed overridden to 5. It imports exactly as before: `M1` arrives as a base template inheriting Motor, and Pump composes it. The `pre-signing-*` fixtures and `LegacyBundleCompatibilityTests` are unchanged and pass. **Tests.** `DerivedTemplateTransportTests` (10), `DerivedTemplatePlanTests` (3) and one `DependencyResolverTests` case cover: - a round trip of a base+derived pair (the slot stays derived, owned and inheriting, with its override, its addition and its inherited placeholders); - identity (Identical on re-import, Modified on one override, a same-named base left untouched); - a nested cascade slot; - derived templates listed before their base and owner; - a missing base, and a base present only on the target; - in-place Overwrite with stale-row revert, while the owner is also Overwritten; - a cycle; - the pre-#118 fixture; - a signed bundle carrying the field (it verifies, and editing the derivation fails the hash check or, with a recomputed hash, the signature). **Mutation-proved**, every mutation killed: - the serializer drops `Derivation`: 7 tests fail; - derived templates go through the base pass: 5 fail; - the plan uses bundle order: 2 fail; - no cycle check: the cycle test fails; - the diff compares placeholders: the identity test fails; - the composition pass deletes derived slots: the Overwrite test fails; - no stale revert: the Overwrite test fails; - the missing-base audit is dropped: the missing-base test fails. **Evidence** (after rebasing onto fb04b54d): `dotnet build ZB.MOM.WW.ScadaBridge.slnx` 0 warnings / 0 errors. All passed: Transport.Tests 236, Transport.IntegrationTests 161, TemplateEngine.Tests 617, ManagementService.Tests 790, CentralUI.Tests 1540, CLI.Tests 592 (TemplateEngine and CLI were run before the rebase). Not deployed. The design is documented in Component-Transport.md, "Derived (composition-slot) templates (ScadaBridge#118)". **Seen on the way, not changed (TemplateEngine):** once a derived slot holds an override, `TemplateService.AddAttributeAsync` on that slot fails with "Naming collision on 'Speed': Attribute from M1, Attribute from parent 'Motor'". So an addition is only possible before the first override. Worth a separate look.

dohertj2 closed this issue

2026-11-10 05:03:50 -05:00

dohertj2 referenced this issue

2026-11-10 05:04:15 -05:00

TemplateEngine: adding an attribute to a derived slot that already has an override fails with a false naming collision #119

dohertj2 referenced this issue

2026-11-10 05:07:42 -05:00

TemplateEngine: adding an attribute to a derived slot that already has an override fails with a false naming collision #119

dohertj2 referenced this issue from a commit

2026-11-10 05:07:43 -05:00

fix(templates): override rows on a derived template are not second definitions in the collision check (ScadaBridge#119)

Sign in to join this conversation.