diff --git a/docs/plans/2026-05-24-transport-design.md b/docs/plans/2026-05-24-transport-design.md new file mode 100644 index 0000000..61b085c --- /dev/null +++ b/docs/plans/2026-05-24-transport-design.md @@ -0,0 +1,423 @@ +# Transport — Bundle Export / Import (Component #24) — Design + +**Status:** Approved (brainstorming complete, awaiting implementation plan). +**Author session date:** 2026-05-24. +**Target component number:** #24 (Transport). + +--- + +## 1. Purpose + +Provide a file-based, encrypted, environment-agnostic way to **promote configuration artifacts from one ScadaLink cluster to another** (e.g., dev → staging → prod) through the Central UI. + +A user with the `Design` role on the source cluster exports a selected set of templates and supporting artifacts to a `.scadabundle` file. A user with the `Admin` role on the target cluster uploads the bundle, reviews a diff, resolves conflicts, and applies it. Import is **config-only**: it updates the central configuration database and marks affected instances stale; the user redeploys to sites via the existing Deployments page. + +Transport does **not** touch site nodes, does not move runtime state, and does not move site-scoped artifacts. + +--- + +## 2. Location + +- New project: `src/ScadaLink.Transport/` +- New tests: `tests/ScadaLink.Transport.Tests/`, `tests/ScadaLink.Transport.IntegrationTests/` +- New design doc: `docs/requirements/Component-Transport.md` (created as part of implementation). +- Central UI pages: `src/ScadaLink.CentralUI/Components/Pages/Design/TransportExport.razor`, `TransportImport.razor`. +- EF migration in `src/ScadaLink.ConfigurationDatabase/Migrations/` (adds `BundleImportId` column to `ConfigurationAuditLog`). + +--- + +## 3. Scope + +### 3.1 In scope — transportable artifact groups + +| Group | Entities | +|---|---| +| Templates | `Template`, `TemplateAttribute`, `TemplateAlarm`, `TemplateScript`, `TemplateComposition`, `TemplateFolder` | +| System artifacts | `SharedScript`, `ExternalSystemDefinition` + `ExternalSystemMethod`, `DatabaseConnectionDefinition` | +| Central-only | `NotificationList` + recipients, `SmtpConfiguration`, `ApiKey`, `ApiMethod` | + +### 3.2 Out of scope (never bundled) + +- **Site-scoped:** `Site`, `Area`, `Instance`, `InstanceAttributeOverride`, `InstanceAlarmOverride`, `InstanceConnectionBinding`, `DataConnection`. +- **Operational state:** `DeploymentRecord`, `SystemArtifactDeploymentRecord`, `DeployedConfigSnapshot`, all audit tables, S&F buffers. +- **Cluster identity / auth state:** LDAP mappings, site-scope rules. + +Rationale: environments do not share stable site identities (dev "Site-Dev-1" vs. prod "Plant-North"). Migrating instances would require a name-mapping subsystem, deferred until there is concrete user demand. + +### 3.3 Non-goals (explicit v1 exclusions) + +- Direct cluster-to-cluster pull (file-based only). +- Auto-redeploy on import. +- Bundle signing (separate keypair); manifest SHA-256 is sufficient for v1 tamper detection. +- Differential / incremental bundles — every export is a full snapshot of selected artifacts. +- Server-side bundle library / browsing. + +--- + +## 4. Bundle Format + +### 4.1 File layout + +`.scadabundle` is a renamed `.zip`: + +``` +bundle.scadabundle +├── manifest.json # required, NOT encrypted +├── content.json # plaintext artifact data (when no passphrase) +├── content.enc # encrypted artifact data (when passphrase set) +└── scripts/ # optional: large script bodies as files + ├── template-{id}-{name}.cs + └── shared-{id}-{name}.cs +``` + +Exactly one of `content.json` or `content.enc` is present. + +### 4.2 `manifest.json` (plaintext) + +```json +{ + "bundleFormatVersion": 1, + "schemaVersion": "1.0", + "createdAtUtc": "2026-05-24T12:34:56Z", + "sourceEnvironment": "dev-cluster-a", + "exportedBy": "alice@corp.example", + "scadaLinkVersion": "1.4.2", + "contentHash": "sha256:...", + "encryption": { + "algorithm": "AES-256-GCM", + "kdf": "PBKDF2-SHA256", + "iterations": 600000, + "saltB64": "...", + "ivB64": "..." + }, + "summary": { + "templates": 12, "templateFolders": 3, "sharedScripts": 4, + "externalSystems": 2, "dbConnections": 1, + "notificationLists": 1, "smtpConfigs": 0, "apiKeys": 2, "apiMethods": 5 + }, + "contents": [ + { "type": "Template", "name": "Pump", "version": 5, + "dependsOn": ["SharedScript:PumpUtils"] }, + { "type": "Template", "name": "Pump.WaterPump", "version": 3, + "dependsOn": ["Template:Pump", "ExternalSystem:HistorianAPI"] } + ] +} +``` + +The manifest is plaintext so the import wizard can preview bundle contents and source provenance **before** the user supplies a passphrase. + +### 4.3 `content.json` / `content.enc` + +- One top-level array per entity type, POCO shapes serialized via `System.Text.Json`. +- Secret fields (API key hashes, SMTP password, external system credentials, DB connection passwords) live in a nested `secrets` block on each affected entity. +- The whole `content` blob is AES-256-GCM encrypted with a key derived via PBKDF2-SHA256 (600 000 iterations) from the user's passphrase, with per-bundle random salt and per-encryption random IV. +- Unencrypted bundles are allowed but UI warns and audit-tags them `UnencryptedBundleExport`. + +### 4.4 Forward compatibility + +- Unknown top-level entity types in `contents[]` surface in the import preview as "skipped — unsupported in this version" rather than failing the whole import. +- `bundleFormatVersion` newer than what the importer supports → hard refusal at upload with a clear upgrade message. + +--- + +## 5. Architecture (Option A — new component) + +``` +ScadaLink.Transport +├── IBundleExporter +│ ExportAsync(ExportSelection, Passphrase?, ct) → Stream +├── IBundleImporter +│ LoadAsync(stream, Passphrase?, ct) → BundleSession +│ PreviewAsync(sessionId, ct) → ImportPreview +│ ApplyAsync(sessionId, resolutions, ct) → ImportResult +├── DependencyResolver +├── BundleSerializer (manifest + content JSON; ZIP packer) +├── SecretEncryptor (AES-256-GCM + PBKDF2) +├── BundleSessionStore (in-memory, TTL'd) +└── ManifestValidator (schema/version gating, hash check) +``` + +- Component is **central-only**. Registered in `ScadaLink.Host` for central roles, never for site roles. +- All persistence goes through **existing audited repository interfaces** in `ScadaLink.ConfigurationDatabase` — no raw `DbContext.SaveChangesAsync` from this component. +- `BundleSessionStore` is in-process on the active central node (matches Blazor Server circuit affinity). 30-minute TTL, GC on expiry, 3-strike passphrase lockout per session. + +Rejected alternatives: +- **Option B (extend Template Engine + Deployment Manager):** spreads bundle format and encryption knowledge across components with no shared owner. Cross-artifact dependency resolution has no clean home. +- **Option C (Central-UI-only service):** CLI cannot reuse it (CLI talks to the Management Service). Violates the codebase pattern of keeping UI thin. + +--- + +## 6. Export Flow + +### 6.1 UI — 4-step wizard under the Design nav group + +**Step 1 — Select artifacts.** Templates rendered as a **tree** matching the existing Templates page (reuse the tree component from `docs/plans/2026-03-23-treeview-component.md`, refactored into a shared `TemplateFolderTree.razor` with a new "checkbox selection" mode). Tri-state checkboxes on folders (`☑` all, `☐` none, `▣` partial). Search filters the tree in place. Other artifact groups (shared scripts, external systems, notification lists, SMTP configs, API keys, API methods) are flat lists — no folder hierarchy in their data model. + +**Step 2 — Review dependencies.** The resolver expands the user's selection along these edges: + +- `Template A` composes `Template B` → include `B`. +- `Template` references `SharedScript` (by name) → include the script. +- `Template` references `ExternalSystem` → include the def and its methods. +- `ApiMethod` references `SharedScript` → include the script. +- `NotificationList` references `SmtpConfiguration` → include the SMTP config. +- Any folder containing a selected template is included so the structure is reproducible on import. + +User can toggle "include all dependencies" off (with warning that the bundle may produce an invalid import). + +**Step 3 — Encryption.** Passphrase + confirm. Strength meter. Explicit warning lists every secret field being encrypted. Optional "Export without encryption" path is logged with the `UnencryptedBundleExport` audit flag. + +**Step 4 — Download.** Generated filename pattern: `scadabundle-{sourceEnv}-{yyyy-MM-dd-HHmmss}.scadabundle`. SHA-256 displayed for out-of-band verification. + +### 6.2 Backend + +``` +User (Design role) ─► Central UI Export wizard + │ + ▼ + IBundleExporter + │ + ├─► DependencyResolver ─► repositories (read) + ├─► EntitySerializer ─► content.json + ├─► SecretEncryptor ─► content.enc (if passphrase) + ├─► ManifestBuilder ─► manifest.json + ▼ + ZIP packer → temp file → browser download + │ + ▼ + IAuditService.LogAsync(BundleExported …) +``` + +Audit event: `BundleExported` — caller, artifact count, content hash, encrypted yes/no, bundle filename. + +Authorization: `RequireDesign` on both the Razor page and `IBundleExporter.ExportAsync` (defense in depth). + +--- + +## 7. Import Flow + +### 7.1 UI — 5-step wizard under the Design nav group + +**Step 1 — Upload.** Drag-and-drop or browse. On selection, manifest is parsed and displayed (source env, exporter, timestamp, content count, SHA-256, encrypted yes/no). Manifest hash validated against `content` blob. + +**Step 2 — Passphrase** (skipped if bundle is unencrypted). 3-wrong-attempt lockout invalidates the session. + +**Step 3 — Diff & resolve conflicts.** For each artifact in the bundle, compare to existing by name: + +- **Identical** (field-by-field) → marked, auto-skipped, cannot be selected. +- **Modified** → shows side-by-side or `+/-/~` line diff. User picks **Skip / Overwrite / Rename**. +- **New** → marked `+ Add`. User can opt to skip individually. + +Bulk "Apply to all" at the top (Overwrite / Skip / Rename), overridable per row. Summary line at bottom: "5 add · 2 overwrite · 1 skip · 0 rename". + +Bundle references that cannot be satisfied in either the bundle or the target DB (e.g., a template references a shared script that's neither in the bundle nor pre-existing) appear as **blocker rows** — Apply is disabled until they are resolved (typically by skipping the dependent artifact or re-exporting with dependencies). + +**Step 4 — Confirm.** Final summary plus a **"3 instances will become stale"** warning enumerating affected instances. User types the **source environment name** to confirm (typo-resistant gate at the prod boundary). + +**Step 5 — Result.** Counts, link to `BundleImported` audit row, link to Deployments page filtered to the newly stale instances. + +### 7.2 Backend + +``` +User (Admin role) ─► uploads bundle + │ + ▼ + IBundleImporter.LoadAsync + · verify SHA-256 (manifest vs content) + · check bundleFormatVersion supported + · decrypt content.enc with passphrase (if encrypted) + · deserialize entities + · open BundleSession (30-min TTL) + │ + ▼ + PreviewAsync → diff vs target DB → ImportPreview + │ + ▼ (user reviews + resolves conflicts) + │ + ApplyAsync (single EF transaction) + · run pre-deployment semantic validator (Template Engine) + · apply resolutions (add / overwrite / skip / rename) + · upsert TemplateFolder hierarchy + · stale-mark affected Instances + · IAuditService.LogAsync(BundleImported …) + · commit + │ + ▼ + ImportResult → UI step 5 + │ + ▼ + "View on Deployments →" (existing page) +``` + +Authorization: `RequireAdmin` on both the Razor page and `IBundleImporter.*` entrypoints. + +### 7.3 Stale-instance signaling + +When an applied conflict overwrites a template (or a template composed by other templates), every `Instance` whose direct or composed template was overwritten gets its `DeployedRevisionHash` set to `stale` (reusing the existing revision-drift mechanism in Deployment Manager). The Deployments page already surfaces stale instances and offers redeploy; no new UI is needed. + +--- + +## 8. Error Handling + +| Where | Failure | Surfaced as | +|---|---|---| +| Upload | Not a zip / missing `manifest.json` | Step 1 error: "Not a valid ScadaLink bundle" | +| Upload | `bundleFormatVersion` newer than supported | Step 1 error: "Bundle was created by ScadaLink v{x}; upgrade this cluster" | +| Upload | Content hash mismatch | Step 1 error: "Bundle integrity check failed — file may be corrupt" | +| Unlock | Wrong passphrase | Step 2 error; 3rd wrong attempt invalidates session, audit `BundleImportUnlockFailed` | +| Preview | Bundle references shared script not in bundle and not in target DB | Listed as a blocker row in Step 3; cannot Apply until resolved | +| Apply | Semantic validation fails (call target type mismatch, etc.) | Modal: "Validation failed — N errors", per-error list, no DB writes | +| Apply | DB transaction fails | Rollback; full transactional guarantee — nothing partial lands; `BundleImportFailed` audit row written outside the rolled-back transaction | +| Session | TTL expired | Step 3+ refresh prompts re-upload | + +Imports are **all-or-nothing per bundle.** A bundle either applies fully or not at all. + +--- + +## 9. Security + +- **AES-256-GCM** for content encryption with **PBKDF2-SHA256 / 600 000 iterations** (OWASP 2023+ guidance), per-bundle random salt, random IV per encryption. GCM auth tag verified before decryption — wrong passphrase fails cleanly. +- **Passphrase never persisted.** Lives only inside the export/import service call path; discarded after use. No environment variable, no log line. +- **Failed-unlock rate limit:** per-session 3-strike lockout; per-IP-per-hour cap (default 10, configurable) to deter brute force against a stolen bundle. Each failed attempt produces a `BundleImportUnlockFailed` audit row. +- **Bundle size cap** on upload (default 100 MB, configurable) to bound memory. +- **In-transit:** existing HTTPS to Central UI; no new channel. +- **Audit trail is the chain of custody.** Every export, every import (including aborted ones at validation), every unlock failure is audit-logged with source env, content hash, encrypted yes/no, and artifact summary. +- **Defense in depth on authorization:** `RequireDesign` (export) and `RequireAdmin` (import) enforced both on the Razor page and inside `ScadaLink.Transport` service entrypoints. UI is not the only gate. +- **Bundles are not retained server-side** after download (export) or after `ApplyAsync` commits (import). + +--- + +## 10. Configuration Audit Trail + +Import flows through the **same audited repository methods** the UI and CLI use, so every artifact mutated by `ApplyAsync` emits the existing per-entity `ConfigurationAuditLog` row: + +| Action during import | `ConfigurationAuditLog` rows emitted | +|---|---| +| Template added | `TemplateCreated` + `TemplateAttributeCreated` (×N) + `TemplateScriptCreated` (×N) + … | +| Template overwritten | `TemplateUpdated` + per-field rows (`TemplateAttributeAdded`, `TemplateScriptUpdated`, …) | +| Template skipped | (no rows) | +| Template renamed-on-import | `TemplateCreated` with the new name (existing row untouched) | +| External system overwritten | `ExternalSystemDefinitionUpdated` + per-method rows | +| Notification list added | `NotificationListCreated` + per-recipient rows | +| API key added | `ApiKeyCreated` | +| Instance marked stale | `InstanceMarkedStale` (existing event) | + +**Correlation:** every per-entity row written during an import carries a new optional `BundleImportId` column (the GUID of the parent `BundleImported` summary row). The existing `ConfigurationAuditLog` Central UI page gains a **Bundle Import** filter that surfaces all rows for a given import. The `BundleImported` summary row links to the filtered view. + +**Schema change:** one EF migration adds: + +- `BundleImportId uniqueidentifier NULL` on `ConfigurationAuditLog`. +- Non-clustered index `IX_ConfigurationAuditLog_BundleImportId`. + +**Transactional guarantee:** the EF transaction wraps both the entity writes and the audit log writes (existing pattern). A rollback removes both. The `BundleImported` summary row is the final write inside the transaction, so partial audit trails are impossible. A failed import emits a single `BundleImportFailed` row outside the rolled-back transaction. + +--- + +## 11. Observability + +- Structured log events: `transport.export.completed`, `transport.import.applied`, `transport.import.failed`, `transport.import.unlock_failed`. +- Counters: bundle size (bytes), artifact count, decryption failures per hour, sessions opened/expired. +- New `BundleImported` and `BundleImportFailed` audit row types appear in the existing audit log page filter. + +--- + +## 12. Authorization Summary + +| Operation | Required role | Enforced at | +|---|---|---| +| Open Export page | `RequireDesign` | Razor page authorize attribute | +| `IBundleExporter.ExportAsync` | `RequireDesign` | Service entrypoint | +| Open Import page | `RequireAdmin` | Razor page authorize attribute | +| `IBundleImporter.LoadAsync` / `PreviewAsync` / `ApplyAsync` | `RequireAdmin` | Service entrypoint | +| `ConfigurationAuditLog` "Bundle Import" filter | `RequireAdmin` or `Audit` | Existing audit page logic | + +--- + +## 13. CLI (Deferred) + +The `ScadaLink.Transport` library is callable from both Razor pages and `ScadaLink.CLI`. CLI commands are **not** built in v1 but the design leaves a clean path: + +``` +scadalink transport export \ + --templates Pump,Pump.WaterPump \ + --shared-scripts PumpUtils \ + --out bundle.scadabundle \ + --passphrase-file /run/secrets/p + +scadalink transport import bundle.scadabundle \ + --passphrase-file /run/secrets/p \ + --on-conflict overwrite|skip|rename \ + --dry-run +``` + +Same auth model via the Management API. + +--- + +## 14. Test Plan + +| Layer | Project | Coverage | +|---|---|---| +| Unit | `tests/ScadaLink.Transport.Tests` (new) | `BundleSerializer` round-trip, `DependencyResolver` topology + cycle handling, `SecretEncryptor` (encrypt → decrypt round-trip, wrong passphrase fails, tampered ciphertext fails GCM auth tag), `ManifestBuilder` schema-version gating, `ImportPreview` diff for each entity type (identical / modified / new), per-conflict resolution merge logic, stale-instance cascade through composed templates | +| Integration | `tests/ScadaLink.Transport.IntegrationTests` (new) | End-to-end against in-memory + real SQL Server: export → import on same DB (no-op idempotency), export → wipe → import (full restore), export → modify-target → import under each conflict resolution, validation failure rolls back cleanly, audit rows correlated by `BundleImportId` | +| UI | `tests/ScadaLink.CentralUI.Tests` | Razor page authorization (`RequireDesign` for export, `RequireAdmin` for import), wizard step navigation, session TTL expiry, tri-state checkbox tree behavior | +| Manual | docker cluster | Cross-cluster: bring up two clusters, export from one, import to the other, verify stale instances surface on Deployments page, redeploy works | + +--- + +## 15. Deployment + +- New component registered in `ScadaLink.Host` for central roles only. +- No new ports, no new database (uses existing central MS SQL). +- One EF migration: `BundleImportId` nullable `uniqueidentifier` column on `ConfigurationAuditLog` + supporting index. +- `bash docker/deploy.sh` picks up the change with the standard rebuild + restart — no compose changes. + +--- + +## 16. Documentation Deliverables + +- **New:** `docs/requirements/Component-Transport.md` — full component design doc following the standard structure (Purpose, Location, Responsibilities, detailed design sections, Dependencies, Interactions). +- **Updated:** `README.md` component table — add row #24: "Transport — Bundle export/import for templates, shared scripts, external systems, central-only artifacts; AES-256-GCM encryption; per-conflict resolution on import; correlated audit trail." +- **Updated cross-references in:** + - `Component-TemplateEngine.md` — mention Transport as a consumer of template + composition + script entities. + - `Component-DeploymentManager.md` — mention stale-instance signaling from Transport. + - `Component-SecurityAuth.md` — record the role mapping (`RequireDesign` export, `RequireAdmin` import). + - `Component-ConfigurationDatabase.md` — record the new `BundleImportId` column + index. + - `Component-CentralUI.md` — record the two new pages under the Design nav group. +- `Component-Transport.md` includes the bundle JSON schema in an appendix for downstream tooling. + +--- + +## 17. Rollout + +- Ship in **one PR / branch**: design doc + component code + UI pages + EF migration + tests + README + cross-reference updates. +- Backward compatible — no behavior change for users who never open the new pages. +- **No feature flag** (per `CLAUDE.md` guidance — don't add flags for hypothetical toggling). + +--- + +## 18. Open Questions / Future Work + +- **Site-scoped artifact transport** (Instances, Areas, bindings, DataConnections). Requires a name-mapping subsystem (source-env-site → target-env-site). Deferred until concrete demand. +- **Direct cluster-to-cluster pull** as an alternative to file handoff. Same `ScadaLink.Transport` library can back it; needs cross-env auth design. +- **Bundle signing** with an asymmetric keypair (separate from passphrase encryption) for stronger non-repudiation. Manifest content hash is sufficient for v1 tamper detection. +- **CLI commands** (`scadalink transport export/import`). Shape pre-decided in §13; not built in v1. +- **Differential bundles** ("only what changed since last export"). YAGNI for v1. + +--- + +## 19. Brainstorming Decisions Captured + +(For traceability to the brainstorming session that produced this doc.) + +| Question | Decision | +|---|---| +| Transport model | File-based export/import (.scadabundle) | +| Artifact groups | Templates + composition; system artifacts; central-only — site-scoped excluded | +| Secrets | Encrypted in bundle via passphrase (AES-256-GCM + PBKDF2 600k) | +| Selection granularity | Per-artifact with auto-included dependencies | +| Selection UI | Tree view for templates (reuse Templates-page tree); flat lists for other groups | +| Conflict resolution | Interactive per-artifact diff + Skip / Overwrite / Rename | +| Post-import behavior | Config-only; user redeploys via existing Deployments page | +| Authorization | `Design` to export, `Admin` to import | +| Audit | Per-entity rows via existing audited repositories, correlated by `BundleImportId` | +| Architecture | New component `ScadaLink.Transport` (Option A) |