Reflect this session's implementation work in the Transport (#24)
component spec:
- New 'CLI' section covering bundle export / preview / import
commands, the base64-over-JSON wire format, the 200 MB request-body
cap, and the 5-minute per-command timeout. Authorization table +
Interactions section updated to mention ManagementActor handlers.
- Import wizard nav placement corrected from Design to Admin (already
the case in code; the spec lagged).
- Blocker-scan heuristic boundaries documented under Import Flow:
the '.' skip, the DataSourceReference exclusion, and the
KnownNonReferenceNames denylist. Both DetectBlockersAsync and
RunSemanticValidationAsync Pass 1 share the filter.
ApplyTemplateFoldersAsync staged new folders via AddFolderAsync but
never called SaveChanges -- so on relational providers (MSSQL) the
new folders sit in the change tracker with Id=0 until the outer
SaveChanges much later in ApplyAsync. The previous attempt to read
the folder name->id map via _templateRepo.GetAllFoldersAsync only
saw pre-existing rows, so newly-bundled folders couldn't satisfy a
template's FolderName reference. Add an intermediate SaveChanges
inside ApplyTemplatesAsync before the folder query.
ApplyTemplatesAsync built the Template entity via BuildTemplate() but
never read the DTO's FolderName -- so every imported template landed
at the root regardless of which folder it lived in on the source
cluster. ApplyTemplateFoldersAsync had already flushed the folder
rows by that point; the FK just wasn't being set.
Resolve folder-name -> persisted FolderId from the same name table
(via _templateRepo.GetAllFoldersAsync after the folder pass), honour
TemplateFolder Rename resolutions, and set FolderId on Add /
Overwrite / Rename paths alike. The audit-row 'after' state now
includes FolderId so the action's effect is visible in the
configuration audit log.
PreviewAsync can emit multiple ImportPreviewItem rows for the same
(EntityType, Name) -- one per modified member of a template, for
example. ApplyAsync internally calls .ToDictionary() on the
resolutions list and throws ArgumentException on duplicate keys.
The Central UI's BuildDefaultResolutions already dedupes via a
dictionary assignment (last-write-wins). Mirror that in the CLI
handler so 'bundle import' tolerates the duplicate-rows shape the
preview returns.
Inbound API scripts access request parameters via the Parameters
runtime API root (Parameters.x or Parameters["x"]). The blocker
heuristic was flagging it as a missing SharedScript or ExternalSystem
on bundles that include ApiMethod scripts. Same surgical fix as the
other entries on KnownNonReferenceNames.
Three new CLI commands automate the Transport feature end-to-end:
scadalink bundle export --output FILE --passphrase X [--all | --templates A,B ...] [--include-dependencies] [--source-environment NAME]
scadalink bundle preview --input FILE --passphrase X
scadalink bundle import --input FILE --passphrase X [--on-conflict skip|overwrite|rename]
Wire format: bundle bytes travel as base64 inside the existing
/management JSON envelope -- no new endpoints, no streaming plumbing.
The 100 MB raw cap inflates to ~140 MB base64; per-request body size
on the management endpoint is raised to 200 MB via the
IHttpMaxRequestBodySizeFeature.
Server side: three new command records in
ScadaLink.Commons.Messages.Management (auto-discovered by the
existing ManagementCommandRegistry), ManagementActor dispatch and
role rules (Export=Design, Preview/Import=Admin), and three
handlers that delegate to the existing IBundleExporter /
IBundleImporter services with name-keyed selection resolution.
Per-bundle CLI timeout bumped to 5 min for large exports.
Conflict policy on import is a single global flag for all Modified
rows; Identical rows always Skip, New rows always Add, Blocker rows
abort. Rename mints a per-bundle timestamp suffix.
SecurityConfiguration.HasData declares 4 LdapGroupMapping seed rows
(Admin / Design / Deployment-All / Deployment-SiteA) but the
InitialSchema migration only INSERTs the Admin row -- the other three
were never captured into a migration. A fresh ScadaLinkConfig2 starts
with multi-role getting Admin only, no Design or Deployment access.
(The same divergence exists on primary's ScadaLinkConfig, but it has
the rows from earlier history.)
Insert the missing three idempotently from seed-sites.sh so env2's
fresh deploys end up role-aligned with the running primary cluster.
The longer-term fix is a new EF migration that captures the HasData
diff -- intentionally not done here to avoid touching the primary
cluster's existing rows.
RunSemanticValidationAsync's Pass 1 minimal-name-resolution scan
duplicated DetectBlockersAsync's heuristic but had the same two bugs
fixed in the previous two commits: it was scanning
TemplateAttribute.DataSourceReference (an OPC UA address-space path,
not script source) and it was missing the KnownNonReferenceNames
denylist. As a result, an import that passed the diff-step blocker
check would still fail at Apply with the same 30+ identifiers
reappearing as "Bundle semantic validation failed" errors.
Apply the same two fixes here so the diff preview and the Apply-time
validator agree.
DetectBlockersAsync was feeding TemplateAttribute.DataSourceReference
into the identifier scanner alongside script bodies, but that field is
an OPC UA node-address path (e.g. "ns=3;s=Tank.Level") owned by the
device, not script source. The dot delimiter inside the path tripped
the heuristic into flagging the address segment ("Tank", "Sensor",
"TestChildObject", "DevAppEngine") as a missing SharedScript or
ExternalSystem reference -- a 100% false-positive class on any
template catalog with OPC-UA-mapped attributes.
Drop the DataSourceReference scan entirely. Attribute.Value is still
scanned because it can carry a design-time default expression that
calls into runtime APIs. Add a regression test pinning the new behavior.
The DetectBlockersAsync heuristic was catching every PascalCase
"Identifier(" or "Identifier." token in script bodies and treating it
as a candidate SharedScript or ExternalSystem reference. On a normal
template catalog this surfaced 30+ blocker rows for .NET stdlib
(DateTimeOffset, Convert, ToString, Dispose, UtcNow...), ScadaLink
runtime API roots (Notify, Database, ExternalSystem, Scripts...), and
SQL keywords inside string literals (COUNT), blocking the import.
Two surgical fixes:
1. Skip identifiers preceded by `.` so `obj.Method()` no longer flags
`Method` as a top-level reference.
2. Maintain a `KnownNonReferenceNames` denylist for the small set of
well-known stdlib / runtime / SQL tokens that can never be
user-defined SharedScripts or ExternalSystems.
The documented use case -- a top-level free-standing call to a missing
SharedScript or ExternalSystem (e.g. `MissingHelper()` at the start of
an expression, or `ErpSystem.Call(...)` where ErpSystem is the
external-system identifier) -- still produces a blocker row, pinned
by the existing test plus a new noise-filter regression test.
Step 1's Next button was wrapped in `@if (_session is not null)`, which
hid it for encrypted bundles where the first LoadAsync call legitimately
leaves _session null until the passphrase is supplied at Step 2.
Trigger the Next-button region on `_bundleBytes is not null` instead,
showing a placeholder notice when the manifest isn't decrypted yet so
the user has a visible affordance to advance to the passphrase step.
OnFileSelectedAsync called TryLoadAsync with a null passphrase to peek
the manifest, but the outer `catch (Exception)` surfaced the expected
"Passphrase required for encrypted bundle" ArgumentException as a fatal
"Failed to read bundle" error -- blocking the user from ever advancing
to the passphrase step. Catch ArgumentException specifically and let
the wizard advance normally on the next click.
11-task plan (T0-T10) covering the sibling docker-env2/ directory:
SQL setup script + mount, Traefik config, central/site appsettings,
docker-compose, lifecycle scripts, .gitignore, READMEs and cross-refs,
verification checklist, and a manual smoke test. No application code
changes -- pure deploy tooling. Most tasks (T0-T9) are independent
and parallel-ready; T5 is gated on T0 + T4; T10 gates on all of T0-T9.
Brainstorming output for a sibling docker-env2/ tree that brings up a
minimal second cluster (2 central + 1 site x 2 nodes + Traefik) on the
same machine alongside the primary docker/ stack. Shares the existing
scadalink-net network and scadalink-mssql container but uses separate
logical databases (ScadaLinkConfig2 / ScadaLinkMachineData2) so the
Transport (#24) feature can be exercised end-to-end with real
cross-environment exports and imports.
The asserted 'LDAP credentials' tagline was deliberately removed from
Login.razor in f973f49 but the test was not updated alongside. Drop
the test — it asserts on UI text that no longer exists by design.
Add an unconditional alert-info banner in the Notification Lists fieldset
(Step 1) explaining that SMTP configurations are not auto-included as
dependencies and must be selected separately.
Add TimeSpan? MinTimeBetweenRuns to TemplateScriptDto and int MaxRetries /
TimeSpan RetryDelay to ExternalSystemDto; wire both directions in
EntitySerializer. Extends the existing script round-trip assertion and adds
Roundtrip_external_system_preserves_retry_config.
- NavMenu: move Import Bundle out of the nested RequireDesign/RequireAdmin
double-gate into the top-level Admin section so an Admin-only user sees it
without needing the Design role; Export Bundle stays in the Design section.
- TransportImport: inject IAuditService + ScadaLinkDbContext; emit a
BundleImportUnlockFailed audit row (best-effort, swallowed on failure) on
every wrong-passphrase attempt in SubmitPassphraseAsync, with attempt
number and error reason in afterState.
- docker central-node-a/b appsettings: add ScadaLink:Transport section with
SourceEnvironment = "docker-cluster" so the importer picks up a non-null
environment name in the audit trail.
- CentralUI.Tests: register IAuditService mock + SQLite in-memory
ScadaLinkDbContext in TransportImportPageTests to satisfy the two new injects.
Implements Task T21 of the Transport feature. A four-step Blazor wizard
(Select → Review → Encrypt → Download) under /design/transport/export,
gated on AuthorizationPolicies.RequireDesign:
1. Select — TemplateFolderTree (checkbox-mode) plus flat checkbox
lists for shared scripts, external systems, DB connections,
notification lists, SMTP configs, API keys, API methods.
2. Review — runs DependencyResolver, surfaces seed vs auto-included.
"Include all dependencies" toggle re-resolves on flip.
3. Encrypt — passphrase + confirm with strength meter, secret-count
warning over the resolved closure, explicit unencrypted
opt-out path (calls BundleExporter with passphrase=null
so the audit row tags UnencryptedBundleExport).
4. Download— calls IBundleExporter.ExportAsync, streams bytes to the
browser via JS interop (wwwroot/js/transport.js), displays
filename + size + SHA-256 + encryption status.
Source environment is sourced from new TransportOptions.SourceEnvironment
(bound from ScadaLink:Transport:SourceEnvironment, defaults "scadalink"),
filename pattern scadabundle-{env}-{yyyy-MM-dd-HHmmss}.scadabundle.
Tests (bUnit + policy): step 1 group rendering, step 2 dependency
expansion (Pump composes Motor), step 4 full walkthrough verifying
ExportAsync receives the selected ids + authenticated identity, and a
RequireDesign policy-deny test for users without the Design role. Also
unit-pins the filename-sanitisation contract.
Address one Blocker and three Important findings from code review of
2c34f12 (BundleImporter.ApplyAsync):
- BLOCKER: wrap RollbackAsync in nested try/catch so a rollback fault
does not swallow the BundleImportFailed audit row. Dispose the
failed transaction before the audit-write so the new SaveChangesAsync
uses a fresh implicit transaction instead of enlisting in the broken
one. Surface the rollback exception's message on the failure row
alongside the original cause, and swallow audit-write faults per the
design's best-effort-audit invariant. Add regression integration
test using a SQLite transaction interceptor that throws on rollback.
- Document re-entrancy assumption on IAuditCorrelationContext: scoped
lifetime, single circuit, concurrent imports within a shared scope
must serialize externally.
- Document repository audit responsibility on BundleImporter: repos
are thin EF wrappers; ApplyAsync writes audit rows explicitly. If
repos ever start emitting audit rows, the explicit calls here must
be removed to avoid double-logging.
- Document BundleSessionStore thread-safety: ConcurrentDictionary
primitives are safe under concurrent callers; BundleSession itself
is not thread-safe.
