docs: sync Transport/TemplateEngine/DeploymentManager specs + CLAUDE.md with the arch-review fix wave (plan 05)
#05-T25 (terminal doc sweep + deferred-decision records). Component-Transport: scripts/ dir marked not-implemented; transported-entity list gains LockedInDerived / script cadence-timeout / real AreaName / native-alarm-sources (round-trip-guard verified); LineDiffer input cap (T22); rename call-site limitation made load-bearing; preview→apply no-optimistic-concurrency window + session cap recorded. Component- TemplateEngine: alarm-override granularity corrected (only TriggerConfig+PriorityLevel instance-overridable, NOT Description/OnTriggerScript). Component-DeploymentManager: OperationLockManager per-node invariant (routing rejection owned by plans 01/07) + post-success audit isolation (T17). CLAUDE.md #24 blurb + Transport bullet updated (trust gate, severity split, ArtifactDiff identity fix, T8 fidelity fixes, T14 plan-06 handoff). Component-ScriptAnalysis fifth-call-site + deny-list already synced in T20/T21. No ../scadaproj/CLAUDE.md change needed — no wire-relationship/stack shift. Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
This commit is contained in:
Joseph Doherty
@@ -94,7 +94,7 @@ Other peers in the `scadaproj` family (see `scadaproj/CLAUDE.md` for details): `
|
||||
21. Notification Outbox — Central component ingesting store-and-forwarded notifications, `Notifications` audit table, dispatcher loop, retry/parking, delivery KPIs.
|
||||
22. Site Call Audit — Central component auditing site cached calls (`CachedCall`/`CachedWrite`); `SiteCalls` audit table, telemetry ingest, reconciliation, KPIs, central→site Retry/Discard relay; sites remain the source of truth.
|
||||
23. Audit Log — Central append-only AuditLog table spanning every script-trust-boundary action (outbound API sync+cached, outbound DB sync+cached, notifications, inbound API). Site SQLite hot-path + gRPC telemetry + reconciliation; combined telemetry with Site Call Audit; central direct-write for Notification Outbox dispatch + Inbound API; monthly partitioning, 365-day retention.
|
||||
24. Transport — File-based, encrypted bundle export/import via Central UI. Templates, system artifacts, central-only configuration, plus site/instance-scoped config (`Site`s, site `DataConnection`s, `Instance`s) reconciled across environments by a `BundleNameMap` name-mapping subsystem. Per-conflict resolution with a per-line Myers diff. Correlated audit via `BundleImportId`. Never touches site runtime nodes (imported instances land `NotDeployed`).
|
||||
24. Transport — File-based, encrypted bundle export/import via Central UI. Templates (with **all** child collections — attributes/alarms/scripts/compositions/**native-alarm-sources**, field-complete incl. `LockedInDerived` + script cadence/timeout), system artifacts, central-only configuration, plus site/instance-scoped config (`Site`s, site `DataConnection`s, `Instance`s + real `AreaName` by name) reconciled across environments by a `BundleNameMap` name-mapping subsystem. Per-conflict resolution with a per-line Myers diff. Runs the **script trust gate** at import review (5th `ScriptTrustValidator` call site — forbidden-API scripts rejected pre-runtime; template-script name-resolution findings are advisory warnings, ApiMethod findings hard-block). Correlated audit via `BundleImportId`. Never touches site runtime nodes (imported instances land `NotDeployed`).
|
||||
25. Script Analysis — Shared authoritative script-trust analyzer: unified forbidden-API deny-list (`ScriptTrustPolicy`), fused semantic+syntactic validator (`ScriptTrustValidator`), Roslyn compile wrapper (`RoslynScriptCompiler`), and compile-only globals stubs (`ScriptCompileSurface`/`TriggerCompileSurface`); consumed by Template Engine, Site Runtime, Inbound API, and Central UI.
|
||||
26. KPI History — Reusable central KPI-history backbone: tall/EAV `KpiSample` store in central MS SQL, `KpiHistoryRecorderActor` cluster singleton (`kpi-history-recorder`, not readiness-gated) sampling DI-registered `IKpiSampleSource`s every minute, bucketed query (`GetRawSeriesAsync` + `KpiSeriesBucketer`) + scoped `KpiHistoryQueryService`, and a reusable custom-SVG `KpiTrendChart`; ships trends for Notification Outbox, Site Call Audit, Audit Log, and Site Health.
|
||||
27. DelmiaNotifier — Standalone external client tool (NOT a cluster component, NOT in the Host): a compact Native-AOT (`win-x64`) console app (`WWNotifier.exe`) that DELMIA Apriso shells out to per recipe download. POSTs to the Inbound API `DelmiaRecipeDownload` method (`X-API-Key`, key from `SCADABRIDGE_API_KEY`), with connect-failure-only failover across a comma-list of base URLs, and reports the legacy `YES`/`NO` + exit-code stdout contract — a drop-in replacement for the legacy `WWNotifier` (see `docs/former-api-specs/dnc/`). Zero-dependency BCL-only, `System.Text.Json` source-gen. Project README: `src/ZB.MOM.WW.ScadaBridge.DelmiaNotifier/README.md`; design: `docs/plans/2026-06-26-delmia-recipe-notifier-design.md`.
|
||||
@@ -149,7 +149,7 @@ Other peers in the `scadaproj` family (see `scadaproj/CLAUDE.md` for details): `
|
||||
- Last-write-wins for concurrent template editing (no optimistic concurrency on templates).
|
||||
- Optimistic concurrency on deployment status records.
|
||||
- Naming collisions in composed feature modules are design-time errors.
|
||||
- Transport (#24, M8): bundles now move site/instance config (`Site`s, site-scoped `DataConnection`s, `Instance`s + override children + `Area` by name), not just central-only config — but still never touch site runtime nodes (imported instances land `NotDeployed`). A `BundleNameMap` name-mapping subsystem (Commons: `SiteMapping`/`ConnectionMapping`/`MappingAction`) reconciles environment-specific identifiers (sites by `SiteIdentifier`, connections by site-qualified `{SiteIdentifier}/{Name}`, instances by `UniqueName`): auto-match → operator override via the import-wizard Map step / CLI `--map-site`/`--map-connection`/`--create-missing-*` → blocker rows for the unresolved. D3 "carry full config (encrypted secrets)": Site addresses travel; `DataConnection` config rides the encrypted `SecretsBlock` (presence-only in diffs). Per-line Myers diff for code fields via the pure `LineDiffer` (`ArtifactDiff` embeds a size-capped structured `lineDiff`); `ImportResult.StaleInstanceIds` is real via the `IStaleInstanceProbe` seam (Commons; implemented in DeploymentManager). `schemaVersion` 1.0→1.1 (additive); `bundleFormatVersion` stays 1; no new EF tables/columns.
|
||||
- Transport (#24, M8): bundles now move site/instance config (`Site`s, site-scoped `DataConnection`s, `Instance`s + override children + `Area` by name), not just central-only config — but still never touch site runtime nodes (imported instances land `NotDeployed`). A `BundleNameMap` name-mapping subsystem (Commons: `SiteMapping`/`ConnectionMapping`/`MappingAction`) reconciles environment-specific identifiers (sites by `SiteIdentifier`, connections by site-qualified `{SiteIdentifier}/{Name}`, instances by `UniqueName`): auto-match → operator override via the import-wizard Map step / CLI `--map-site`/`--map-connection`/`--create-missing-*` → blocker rows for the unresolved. D3 "carry full config (encrypted secrets)": Site addresses travel; `DataConnection` config rides the encrypted `SecretsBlock` (presence-only in diffs). Per-line Myers diff for code fields via the pure `LineDiffer` (`ArtifactDiff` embeds a size-capped structured `lineDiff`; input capped at `MaxInputLines`=4000 → summary-only, so a crafted bundle can't OOM the active node); `ImportResult.StaleInstanceIds` is real via the `IStaleInstanceProbe` seam (Commons; implemented in DeploymentManager). `schemaVersion` 1.0→1.1 (additive); `bundleFormatVersion` stays 1; no new EF tables/columns. **Arch-review 05 (plan 05):** template child fidelity completed (round-trip-guard-verified: `LockedInDerived`, native-alarm-sources, script cadence/timeout, real `AreaName`, ES retry config + methods-on-Add, notification recipients, folder parent edges); import runs the **script trust gate** (template/shared/ApiMethod bodies **and** Expression-trigger bodies) as a hard pre-runtime gate + a name-resolution severity split (template=advisory `ConflictKind.Warning`+`ImportResult.Warnings`, ApiMethod=hard error); `ArtifactDiff` resolves folder/base/composition names (unchanged structured templates classify Identical again); `MaxConcurrentImportSessions`=8 session cap. Rename does NOT rewrite call sites (deploy gate on the target is the real check). Plan-06 handoff: `ScriptArtifactsChanged`/`IScriptArtifactChangeBus` seam + Host in-process bus + Transport post-commit publisher shipped (consumers = plan 06).
|
||||
|
||||
### Store-and-Forward
|
||||
- Fixed retry interval, no max buffer size. Only transient failures buffered.
|
||||
|
||||
@@ -91,6 +91,16 @@ All mutating operations on a single instance (deploy, disable, enable, delete) s
|
||||
- Only one mutating operation per instance can be in-flight at a time. A second operation is rejected with an "operation in progress" error.
|
||||
- **Different instances**: Operations on different instances can proceed **in parallel**, even at the same site. Each tracks status independently. This supports the bulk "deploy all out-of-date instances" operation efficiently.
|
||||
|
||||
> **`OperationLockManager` scope invariant (recorded decision, arch-review 05).** The
|
||||
> per-instance locks are **per-node, in-memory**. Mutual exclusion therefore holds only
|
||||
> because management traffic reaches a single node — the **active** central node, via
|
||||
> Traefik's active-node routing. A direct mutation sent to the **standby** central port
|
||||
> would take a *different* in-memory lock and could race the active node. Structurally
|
||||
> rejecting mutating operations on the non-active node (so the standby can never accept
|
||||
> them) is **not** owned here — it belongs to the cluster-routing / management-surface
|
||||
> plans (01/07). The Deployment Manager records the invariant and assumes single-active-node
|
||||
> delivery; it does not itself enforce node identity.
|
||||
|
||||
### Allowed State Transitions
|
||||
|
||||
| Current State | Deploy | Disable | Enable | Delete |
|
||||
@@ -119,6 +129,7 @@ When deploying artifacts (shared scripts, external system definitions, etc.) to
|
||||
|
||||
- Only the **current deployment status** per instance is stored in the configuration database (pending, in-progress, success, failed).
|
||||
- No deployment history table — the audit log (via IAuditService) already captures every deployment action with who, what, when, and result.
|
||||
- **Post-success audit isolation (arch-review 05, Task 17):** the audit write that records a successful deployment runs **outside** the deployment's own success path — an audit-write failure (DB blip, transient error) is logged and swallowed, and NEVER flips a genuinely-successful deployment to failed or triggers a spurious rollback. Audit is best-effort; the deployment's own apply result is authoritative (mirrors the system-wide "audit-write failure never aborts the user-facing action" rule).
|
||||
- The Deployment Manager uses current status to determine staleness (is this instance up-to-date?) and display deployment results in the UI.
|
||||
|
||||
## Deployment Scope
|
||||
|
||||
@@ -110,7 +110,7 @@ breadcrumb.
|
||||
Override and lock rules apply per entity type at the following granularity:
|
||||
|
||||
- **Attributes**: Value and Description are overridable. Data Type is fixed by the defining level. `DataSourceReference` on a template attribute defines the **default** physical address for that attribute. Instances may override per attribute via `InstanceConnectionBinding.DataSourceReferenceOverride`; the override replaces the template default at flattening time. When the override is null (the default), the template value is used. Lock applies to the entire attribute (when locked, no fields can be overridden).
|
||||
- **Alarms**: Priority Level, Trigger Definition (thresholds/ranges/rates), Description, and On-Trigger Script reference are overridable. Name and Trigger Type (Value Match vs. Range vs. Rate of Change) are fixed. Lock applies to the entire alarm.
|
||||
- **Alarms**: Priority Level and Trigger Definition (thresholds/ranges/rates) are instance-overridable. Description and On-Trigger Script reference are **not** instance-overridable — `InstanceAlarmOverride` carries only `TriggerConfigurationOverride` and `PriorityLevelOverride`, so an instance cannot re-point an alarm's on-trigger script or reword its description (those are template-level authoring decisions). Name and Trigger Type (Value Match vs. Range vs. Rate of Change) are fixed. Lock applies to the entire alarm. (Recorded decision, arch-review 05: this narrower granularity is the implemented behavior; adding Description/OnTriggerScript override columns is future feature work, not spec debt.)
|
||||
- **Native alarm sources**: An instance overrides a non-locked source via `InstanceNativeAlarmSourceOverride`, keyed by `SourceCanonicalName`. `ConnectionNameOverride`, `SourceReferenceOverride`, and `ConditionFilterOverride` are individually overridable — each is applied only when non-null; a null field **keeps the inherited value**. Name is fixed. Lock applies to the entire source.
|
||||
- **Scripts**: C# source code, Trigger configuration, minimum time between runs, and parameter/return definitions are overridable. Name is fixed. Lock applies to the entire script.
|
||||
- **Composed module members**: A composing template or child template can override non-locked members inside a composed module using the canonical path-qualified name.
|
||||
|
||||
@@ -20,8 +20,8 @@ As of M8 (T18), Transport is no longer limited to central-only configuration: it
|
||||
- Resolve artifact dependencies at export time: base templates, shared scripts, external systems, template folders, notification lists, SMTP configs, API keys, API methods.
|
||||
- Move **site-scoped configuration** (T18): `Site` definitions, site-scoped `DataConnection`s (protocol connections — distinct from External-System `DatabaseConnection`s), and `Instance`s along with their `InstanceAttributeOverride` / `InstanceAlarmOverride` / `InstanceNativeAlarmSourceOverride` / `InstanceConnectionBinding` children and `Area` membership (carried by name).
|
||||
- Reconcile cross-environment site identifiers and connection names through the **name-mapping subsystem** (`BundleNameMap`): auto-match by identifier/name, operator override via the import-wizard Map step or CLI flags, and per-conflict create-or-bind resolution (see "Name Mapping").
|
||||
- Compute a **per-line (Myers) diff** for code fields on Modified artifacts (T20) via the pure `LineDiffer`, embedding a size-capped structured line diff in each `ArtifactDiff`.
|
||||
- Serialize and deserialize all transportable entity types to/from bundle DTOs, carving secret fields into an isolated `SecretsBlock`. A template travels with **all** of its child collections — `TemplateAttribute`, `TemplateAlarm`, `TemplateScript`, `TemplateComposition`, and `TemplateNativeAlarmSource` (including `IsInherited` placeholder rows, which the collision detector depends on) — each field-faithful on Add and reconciled add/update/delete on Overwrite. Carrying the native alarm sources is what keeps an imported `InstanceNativeAlarmSourceOverride` from dangling against a source the target template would otherwise lack.
|
||||
- Compute a **per-line (Myers) diff** for code fields on Modified artifacts (T20) via the pure `LineDiffer`, embedding a size-capped structured line diff in each `ArtifactDiff`. Two independent caps guard the diff: a 400-line **output** cap on the emitted hunk list, and — as of arch-review 05 (Task 22) — a `MaxInputLines` **input** cap (4000 combined lines) that short-circuits the O((N+M)²) Myers trace to a summary-only result (`Truncated = true`, add/remove totals only) so a bloated or crafted bundle cannot OOM the active central node from the import preview.
|
||||
- Serialize and deserialize all transportable entity types to/from bundle DTOs, carving secret fields into an isolated `SecretsBlock`. A template travels with **all** of its child collections — `TemplateAttribute`, `TemplateAlarm`, `TemplateScript`, `TemplateComposition`, and `TemplateNativeAlarmSource` (including `IsInherited` placeholder rows, which the collision detector depends on) — each field-faithful on Add and reconciled add/update/delete on Overwrite. Field completeness is verified by a reflection round-trip guard over every writable property (Add + Overwrite), and includes the fields earlier revisions dropped: the derived-lock flag `LockedInDerived` on attributes/alarms/scripts, script cadence/timeout (`MinTimeBetweenRuns` / `ExecutionTimeoutSeconds`), and — for instances — the real `AreaName` (the exporter previously emitted `null`, half-shipping the Area-by-name feature). Carrying the native alarm sources is what keeps an imported `InstanceNativeAlarmSourceOverride` from dangling against a source the target template would otherwise lack.
|
||||
- Encrypt content with AES-256-GCM + PBKDF2-SHA256 (600 000 iterations) when a passphrase is supplied; leave content plaintext with a UI warning and an `UnencryptedBundleExport` audit event when none is given.
|
||||
- Validate `manifest.json` on upload: format version gating, SHA-256 content hash verification.
|
||||
- Manage in-memory `BundleSession` objects: 30-minute TTL, 3-strike passphrase lockout per session.
|
||||
@@ -41,13 +41,15 @@ As of M8 (T18), Transport is no longer limited to central-only configuration: it
|
||||
bundle.scadabundle
|
||||
├── manifest.json # required, NOT encrypted
|
||||
├── content.json # plaintext artifact data (when no passphrase)
|
||||
├── content.enc # encrypted artifact data (when passphrase set)
|
||||
└── scripts/ # optional: large script bodies as files
|
||||
├── template-{id}-{name}.cs
|
||||
└── shared-{id}-{name}.cs
|
||||
└── content.enc # encrypted artifact data (when passphrase set)
|
||||
```
|
||||
|
||||
Exactly one of `content.json` or `content.enc` is present.
|
||||
Exactly one of `content.json` or `content.enc` is present. Script bodies (template
|
||||
scripts, shared scripts, ApiMethod scripts) travel **inline** inside the `content`
|
||||
blob, not as separate `scripts/` files — a well-formed bundle has exactly two zip
|
||||
entries (`manifest.json` + one content blob). A separate `scripts/` directory is
|
||||
**not implemented**: `TransportOptions.MaxBundleEntryCount` (default 4) is a
|
||||
zip-bomb guard, and the exporter never emits per-script files.
|
||||
|
||||
### `manifest.json` (plaintext)
|
||||
|
||||
@@ -131,7 +133,9 @@ flowchart TD
|
||||
class T muted
|
||||
```
|
||||
|
||||
The component is central-hosted. It is registered in `ZB.MOM.WW.ScadaBridge.Host` for central roles only, never for site roles — even when a bundle carries site-scoped config (T18), Transport only writes the central configuration database, never the site clusters. All persistence flows through existing audited repository interfaces in `ZB.MOM.WW.ScadaBridge.ConfigurationDatabase` — the component does not call `DbContext.SaveChangesAsync` directly. `BundleSessionStore` is in-process on the active central node (matching Blazor Server circuit affinity): 30-minute TTL, eviction on expiry, 3-strike passphrase lockout per session.
|
||||
The component is central-hosted. It is registered in `ZB.MOM.WW.ScadaBridge.Host` for central roles only, never for site roles — even when a bundle carries site-scoped config (T18), Transport only writes the central configuration database, never the site clusters. All persistence flows through existing audited repository interfaces in `ZB.MOM.WW.ScadaBridge.ConfigurationDatabase` — the component does not call `DbContext.SaveChangesAsync` directly. `BundleSessionStore` is in-process on the active central node (matching Blazor Server circuit affinity): 30-minute TTL, eviction on expiry, 3-strike passphrase lockout per session, and a `MaxConcurrentImportSessions` cap (default 8; arch-review 05 Task 24) bounding the decrypted-content footprint since each open session pins a fully-decrypted bundle in memory.
|
||||
|
||||
**Preview→apply window (recorded decision).** The diff/preview an operator reviews is a point-in-time snapshot; there is **no optimistic concurrency** between preview and apply. If the target config changes in the window between `PreviewAsync` and `ApplyAsync` (another admin edits a template, say), the apply proceeds against the current state — the resolutions are keyed by `(EntityType, Name)`, so they still target the right rows, but the operator may not have seen the latest state in the diff. This is accepted for v1 (single-operator import is the expected workflow, and the terminal Confirm step already gates a typo-resistant environment-name entry). The bundle-format `version` fields are reserved for a future optimistic-concurrency check but are not yet enforced (Underdeveloped 5).
|
||||
|
||||
## Export Flow
|
||||
|
||||
@@ -288,6 +292,8 @@ Site identifiers and connection names are environment-specific, so a bundle expo
|
||||
|
||||
**Template second-pass rewires.** Bundles carry template-to-template references by **name**, not id (ids are environment-specific). After the templates are flushed, the importer runs three name-resolution passes over the imported templates so every cross-template FK points at the freshly-persisted target row: the **alarm on-trigger-script** links (`TemplateAlarm.OnTriggerScriptId`), the **composition** edges (`TemplateComposition`), and the **inheritance** edges (`Template.ParentTemplateId`, resolved from `BaseTemplateName` through the same rename/skip resolution map — a `null` base clears a stale edge on Overwrite). Each pass is best-effort: an unresolvable reference leaves the FK null and emits a warning audit row (`BundleImportAlarmScriptUnresolved` / `BundleImportCompositionUnresolved` / `BundleImportBaseTemplateUnresolved`) rather than aborting the import.
|
||||
|
||||
**Rename call-site limitation (load-bearing).** A `Rename` resolution renames the entity being imported (e.g. a SharedScript `Foo` → `Bar`), but it does **not** rewrite call sites inside importing scripts — a script body that calls `CallShared("Foo")` still says `Foo` after import. So the target DB can end up with scripts referencing the old name. This is deliberate: Pass 1 name resolution registers **both** the original and renamed names as resolvable (so the import isn't blocked), and the authoritative check is the target's own **deploy-time validation** (`ScriptCompiler.TryCompile` in the Template Engine deploy gate), which recompiles against the actual persisted catalog. Operators renaming a shared dependency must fix the calling scripts on the target (or re-export with the dependency included under its original name).
|
||||
|
||||
**FK rewiring at apply.** Inside the single EF transaction the importer resolves or creates the target sites and connections **first**, then upserts instances (forced to `NotDeployed`) and rewires their foreign keys against the resolved targets:
|
||||
|
||||
- `InstanceConnectionBinding.DataConnectionId` is resolved via the connection map, with a **Pass-2** that also binds connections which already exist in the target but were not carried in the bundle.
|
||||
|
||||
Reference in New Issue
Block a user