ScadaBridge/docs/components/Transport.md

Transport

The Transport component provides file-based, encrypted bundle export and import of central configuration artifacts between ScadaBridge environments via the Central UI. It is purely central — no site nodes are touched, no runtime state moves, and no site-scoped artifacts travel in a bundle.

Overview

Transport (#24) is a central-only component that lives in src/ZB.MOM.WW.ScadaBridge.Transport/, split into three functional areas:

• Export/ — BundleExporter, DependencyResolver, ResolvedExport. The export pipeline resolves artifact dependencies, serializes entities to wire-shaped DTOs, optionally encrypts the content, and produces a ZIP-formatted .scadabundle stream.
• Import/ — BundleImporter, BundleSessionStore, BundleSessionEvictionService, ArtifactDiff, BundleUnlockRateLimiter. The import pipeline validates the bundle envelope, decrypts it, diffs against the target environment, and applies operator-chosen conflict resolutions in a single EF transaction.
• Serialization/ and Encryption/ — BundleSerializer, EntitySerializer, ManifestBuilder, ManifestValidator, BundleSecretEncryptor, BundleManifestAad. Stateless helpers that handle ZIP packing/unpacking, DTO projection, SHA-256 hashing, and AES-256-GCM authenticated encryption.

The single DI entry point is ServiceCollectionExtensions.AddTransport, registered by Host for central roles only. Stateless helpers are singletons; the exporter and importer are scoped because they reach into per-request EF Core scopes and audited repositories.

Key Concepts

Bundle format

A .scadabundle file is a ZIP archive with exactly two entries:

bundle.scadabundle
├── manifest.json     # always plaintext; never encrypted
└── content.json      # plaintext artifact data (no passphrase)
 OR content.enc       # AES-256-GCM ciphertext (passphrase supplied)

manifest.json is always plaintext so the import wizard can display source provenance and artifact counts before the operator supplies a passphrase. BundleManifest carries: BundleFormatVersion, SchemaVersion, CreatedAtUtc, SourceEnvironment, ExportedBy, ScadaBridgeVersion, ContentHash (sha256:<hex> of the raw content bytes), optional Encryption metadata, a Summary (artifact counts by type), and a Contents list (one ManifestContentEntry per artifact with its dependsOn edges).

BundleFormatVersion is an integer gate: the importer requires BundleFormatVersion to equal ManifestBuilder.CurrentBundleFormatVersion (currently 1) and rejects any other value — higher or lower — with ManifestValidationResult.UnsupportedFormatVersion. TransportOptions.SchemaVersionMajor is not read during import. Unknown entity types in Contents produce a preview-row classification of "unsupported" rather than aborting the whole import.

Encrypted vs plaintext bundles

When a passphrase is supplied, BundleSecretEncryptor derives a 256-bit key via PBKDF2-SHA256 (default 600,000 iterations, configurable) from a fresh 16-byte random salt, then encrypts the UTF-8 JSON content using AES-256-GCM with a fresh 12-byte nonce. Output format is ciphertext ‖ GCM-tag (16-byte tag appended). The salt and nonce are stored in manifest.json; the passphrase is never persisted. An unencrypted export is permitted but produces an UnencryptedBundleExport audit event rather than BundleExported.

AAD binding (T-005)

BundleManifestAad.Compute produces AES-GCM Associated Authenticated Data by SHA-256-hashing a canonicalized form of the manifest with ContentHash zeroed and Encryption nulled. This binds SourceEnvironment, ExportedBy, Summary, and Contents to the GCM authentication tag. Tampering with any of those fields on a stolen bundle yields an AuthenticationTagMismatchException on decryption, making the Step-4 "type the source environment to confirm" gate tamper-evident.

BundleSession and session lifecycle

After LoadAsync validates and decrypts a bundle, the plaintext content is stored in a BundleSession held by the singleton BundleSessionStore (a ConcurrentDictionary<Guid, BundleSession>). Sessions have a 30-minute TTL (BundleSessionTtlMinutes). BundleSessionEvictionService sweeps the store every minute so abandoned sessions — and the secrets they carry — are released without waiting for the next Get call. ApplyAsync explicitly zeros and removes the session on both success and failure (T-007).

Conflict resolution and BundleImportId correlation

PreviewAsync compares each bundle artifact against the target environment using ArtifactDiff, classifying items as Identical, Modified, New, or Blocker. The operator assigns a ResolutionAction (Add, Overwrite, Skip, Rename) per item. ApplyAsync honours those resolutions in a single EF transaction and threads a new BundleImportId GUID through every per-entity audit row via the scoped IAuditCorrelationContext. This makes every configuration row written by a bundle import queryable as a group from the audit log.

Architecture

Export pipeline

BundleExporter.ExportAsync orchestrates these steps in sequence:

1. DependencyResolver.ResolveAsync — expands the operator's ExportSelection to the full transitive closure, then topologically sorts templates (base-before-derived via Kahn's algorithm).
2. EntitySerializer.ToBundleContent — projects EF entity POCOs to wire-shaped DTOs (BundleContentDto), carving secret fields (connection strings, credentials, OAuth tokens) into per-entity SecretsBlock records.
3. ManifestBuilder.Build — stamps BundleFormatVersion, SchemaVersion, SHA-256 ContentHash, Summary, and Contents into a BundleManifest.
4. BundleSerializer.Pack — serializes the manifest and content into a ZIP stream. When a passphrase is present, BundleSecretEncryptor.Encrypt runs with a fresh salt and nonce; Pack re-stamps ContentHash and Encryption in the manifest against the ciphertext it actually writes.
5. Audit — IAuditService.LogAsync writes one BundleExported (or UnencryptedBundleExport) row with the SHA-256 of the full ZIP stream as EntityId.

public sealed class BundleExporter : IBundleExporter
{
    public async Task<Stream> ExportAsync(
        ExportSelection selection,
        string user,
        string sourceEnvironment,
        string? passphrase,
        CancellationToken cancellationToken = default)
    {
        var resolved   = await _resolver.ResolveAsync(selection, cancellationToken);
        var aggregate  = new EntityAggregate(/* resolved collections */);
        var contentDto = _entitySerializer.ToBundleContent(aggregate);
        var summary    = new BundleSummary(/* counts from resolved */);
        EncryptionMetadata? encryptionSeed = passphrase is null ? null
            : new EncryptionMetadata("AES-256-GCM", "PBKDF2-SHA256",
                _options.Value.Pbkdf2Iterations, string.Empty, string.Empty);
        var manifest = _manifestBuilder.Build(sourceEnvironment, user, assemblyVersion,
            encryptionSeed, summary, resolved.ContentManifest,
            _bundleSerializer.SerializeContentBytes(contentDto));
        var zipStream = _bundleSerializer.Pack(contentDto, manifest, passphrase, _encryptor);
        var bundleHash = ComputeStreamSha256(zipStream);
        await _auditService.LogAsync(user,
            passphrase is null ? "UnencryptedBundleExport" : "BundleExported",
            "Bundle", bundleHash, sourceEnvironment, /* afterState */, cancellationToken);
        zipStream.Position = 0;
        return zipStream;
    }
}

Dependency expansion

DependencyResolver walks five dependency edge types when ExportSelection.IncludeDependencies is true:

Edge	Mechanism
Template composes Template	TemplateComposition.ComposedTemplateId (BFS over composition graph)
Template references SharedScript	Name-scan of TemplateScript.Code, TemplateAttribute.Value, and TemplateAttribute.DataSourceReference
Template references ExternalSystem	Name-scan of TemplateScript.Code, TemplateAttribute.DataSourceReference, and TemplateAttribute.Value
ApiMethod references SharedScript	Name-scan of ApiMethod.Script
Template folder ancestor chain	Always included regardless of IncludeDependencies

ExternalSystemMethod records always travel with their parent ExternalSystemDefinition. Templates are emitted in topological order (base-before-derived) so the importer can Apply* them in sequence without forward-reference gaps.

Inbound API keys are explicitly excluded — per re-architecture C4, keys are environment-specific and must be re-issued on the target cluster. ApiMethod definitions travel without key bindings.

Import pipeline

BundleImporter is a three-phase service:

Phase 1 — LoadAsync: copies the upload stream to a seekable MemoryStream, enforces the bundle size cap (MaxBundleSizeMb), validates the ZIP envelope (entry count, per-entry decompressed size, compression ratio — all before decompression), reads and validates manifest.json (format version, SHA-256 content hash), and decrypts content.enc when Encryption is present. The decrypted bytes are stored in a new BundleSession.

Passphrase lockout operates at two levels: per-bundle (3-strike counter in BundleSessionStore, keyed by ContentHash so a second browser tab sharing the same bundle bytes cannot reset the counter) and per-IP-per-hour (BundleUnlockRateLimiter, default 10 attempts). A successful decrypt clears the per-bundle counter.

// From BundleImporter.LoadAsync — decrypt path (simplified)
var aad = Encryption.BundleManifestAad.Compute(manifest);
try
{
    decryptedContent = _encryptor.Decrypt(contentBytes, manifest.Encryption, passphrase, aad);
}
catch (CryptographicException)
{
    var newCount = _sessionStore.IncrementUnlockFailureCount(manifest.ContentHash);
    if (newCount >= maxAttempts)
        throw new BundleLockedException(manifest.ContentHash, newCount);
    throw;
}
_sessionStore.ClearUnlockFailures(manifest.ContentHash);

Phase 2 — PreviewAsync: deserializes the session's plaintext bytes to BundleContentDto and calls ArtifactDiff.Compare* methods for each entity type. Diff results use ConflictKind (Identical, Modified, New, Blocker). Modified items carry a FieldDiffJson payload with changed-field names and old/new values; script bodies record a line-count delta rather than full text to keep the diff compact. DetectBlockersAsync scans script bodies for unresolvable SharedScript or ExternalSystem name references.

Phase 3 — ApplyAsync: runs semantic validation first (a name-resolution scan plus the full SemanticValidator from TemplateEngine), then applies all resolutions inside one EF transaction. The correlation GUID is set on IAuditCorrelationContext.BundleImportId before any writes so that every IAuditService.LogAsync call during the apply picks it up automatically. Three SaveChangesAsync calls handle forward references: an intermediate flush inside ApplyTemplatesAsync materializes folder identity values so that template FolderId foreign keys can be wired correctly; a second flush after all Apply* helpers materializes row identities before ResolveAlarmScriptLinksAsync and ResolveCompositionEdgesAsync run; a third flush commits the BundleImported audit row just before CommitAsync. All three flushes operate inside the same outer transaction. On failure, the transaction rolls back, BundleImportId is cleared, and a BundleImportFailed row is written outside the rolled-back transaction before the exception propagates.

// From BundleImporter.ApplyAsync — correlation + transaction pattern
_correlationContext.BundleImportId = bundleImportId;
await using var tx = await _dbContext.Database.BeginTransactionAsync(ct);
try
{
    var errors = await RunSemanticValidationAsync(content, resolutionMap, ct);
    if (errors.Count > 0) throw new SemanticValidationException(errors);

    await ApplyTemplateFoldersAsync(/* ... */);
    await ApplyTemplatesAsync(/* ... */);
    // ... other entity types ...
    await _dbContext.SaveChangesAsync(ct);     // flush for FK resolution
    await ResolveAlarmScriptLinksAsync(/* ... */);
    await ResolveCompositionEdgesAsync(/* ... */);

    await _auditService.LogAsync(user, "BundleImported", "Bundle",
        bundleImportId.ToString(), session.Manifest.SourceEnvironment, /* afterState */, ct);
    await _dbContext.SaveChangesAsync(ct);
    await tx.CommitAsync(ct);
    ZeroDecryptedContent(session);
    _sessionStore.Remove(sessionId);
    return new ImportResult(BundleImportId: bundleImportId, /* counts */);
}
catch
{
    await tx.RollbackAsync(ct);
    _correlationContext.BundleImportId = null;
    await _auditService.LogAsync(user, "BundleImportFailed", /* ... */);
    ZeroDecryptedContent(session);
    _sessionStore.Remove(sessionId);
    throw;
}

Usage

The Central UI surfaces Transport through two wizard pages (see Central UI):

• Export (/design/transport/export, Design role): a 4-step wizard — select artifacts, review resolved dependencies, set a passphrase, download the .scadabundle file.
• Import (/design/transport/import, Admin role): a 5-step wizard — upload bundle, enter passphrase, review the diff and set per-artifact resolutions, confirm (operator types the source environment name), view the result and navigate to the Deployments page for any newly stale instances.

The same operations are available via the CLI:

scadabridge bundle export  --output FILE --passphrase X [--templates A,B] \
                            [--include-dependencies] [--source-environment NAME]

scadabridge bundle preview --input FILE --passphrase X

scadabridge bundle import  --input FILE --passphrase X [--on-conflict skip|overwrite|rename]

CLI commands route through ManagementActor handlers (ExportBundleCommand, PreviewBundleCommand, ImportBundleCommand), which delegate to the same IBundleExporter / IBundleImporter scoped services. Bundle bytes ride the existing /management JSON envelope as base64.

After import, template changes propagate to deployed instances through revision-hash drift detection in DeploymentService.CompareAsync. Transport does not write a stale marker — the existing Deployments page surfaces affected instances automatically.

Configuration

TransportOptions is bound from the ScadaBridge:Transport section.

Key	Default	Description
SourceEnvironment	"scadabridge"	Environment label stamped in manifest.json and used in export filenames.
SchemaVersionMajor	1	Major schema version stamped in exported manifests. Not read by the importer; import version-gating uses ManifestBuilder.CurrentBundleFormatVersion directly.
BundleSessionTtlMinutes	30	TTL for an in-progress import session.
MaxBundleSizeMb	100	Upload size cap; enforced before any decompression.
MaxBundleEntryCount	4	Maximum ZIP entries (a valid bundle has exactly 2).
MaxBundleEntryDecompressedMb	200	Per-entry decompressed size cap (ZIP-bomb defence).
MaxBundleEntryCompressionRatio	50	Per-entry compression ratio cap (ZIP-bomb defence).
MaxUnlockAttemptsPerSession	3	Per-bundle passphrase strike limit.
MaxUnlockAttemptsPerIpPerHour	10	Per-IP trailing-hour unlock attempts.
Pbkdf2Iterations	600000	PBKDF2-SHA256 iteration count for key derivation.

SourceEnvironment should be set per environment (e.g., dev-cluster, prod-cluster) so the import wizard's confirmation gate works correctly.

Dependencies & Interactions

• Commons (#16) — owns BundleManifest, ExportSelection, ImportPreview, ImportResolution, ImportResult, BundleSession, EncryptionMetadata, BundleSummary, ManifestContentEntry, ConflictKind, and the IBundleExporter, IBundleImporter, IBundleSessionStore, IAuditCorrelationContext interfaces. Transport implementations bind to these contracts; Commons defines nothing Transport-specific beyond the DTOs and interfaces.
• Configuration Database (#17) — supplies all repository implementations (ITemplateEngineRepository, IExternalSystemRepository, INotificationRepository, IInboundApiRepository), IAuditService for per-entity audit rows, the IAuditCorrelationContext implementation (AuditCorrelationContext) registered as scoped, the ScadaBridgeDbContext used for the import transaction, and the EF migration that adds BundleImportId uniqueidentifier NULL (with index IX_AuditLogEntries_BundleImportId) to AuditLogEntries.
• Template Engine (#1) — provides SemanticValidator, invoked inside ApplyAsync before the transaction commits. The importer feeds each imported TemplateDto through the validator alongside the combined in-bundle + pre-existing SharedScript catalog; validation errors surface as SemanticValidationException and roll back the entire import.
• Audit Log / Configuration Audit — every export produces a BundleExported or UnencryptedBundleExport row; every import produces a BundleImported summary row (or BundleImportFailed on rollback). Per-entity rows written by Apply* helpers carry BundleImportId so operators can query all configuration changes from a single import as a group. BundleImportUnlockFailed rows are written on passphrase failures. Warning rows BundleImportAlarmScriptUnresolved and BundleImportCompositionUnresolved are written when second-pass FK rewire cannot resolve a name.
• Central UI — hosts the Export Bundle page under the Design nav group and the Import Bundle page under the Admin nav group; the import result page links to the Deployments page and to the filtered Configuration Audit Log Viewer pre-populated with the completed BundleImportId.
• Security & Auth (#10) — enforces RequireDesign on export and RequireAdmin on import, both at the Razor page layer and inside the IBundleExporter / IBundleImporter service entrypoints (defense in depth).
• Deployment Manager (#2) — not directly called by Transport; template overwrites naturally change the flattened-config hash that DeploymentService.CompareAsync reads, causing affected instances to surface as stale on the Deployments page.

Troubleshooting

Bundle upload rejected at format version check

LoadAsync throws NotSupportedException when manifest.json carries a bundleFormatVersion that does not equal ManifestBuilder.CurrentBundleFormatVersion (currently 1). Any non-matching value — whether higher or lower — is rejected. Upgrade the target cluster or re-export from a version that produces format version 1.

Content hash mismatch on upload

LoadAsync throws InvalidDataException("Bundle content hash does not match manifest — file may be corrupt."). The ZIP was corrupted in transit. Compare the SHA-256 shown in the export wizard's Step 4 against the downloaded file and re-export if they differ.

Session expired between diff and apply

PreviewAsync or ApplyAsync throws when the BundleSession is not found. The 30-minute TTL elapsed while the operator was reviewing the diff. Re-upload the bundle to start a new session.

Apply rolls back with SemanticValidationException

A template's scripts reference a SharedScript or ExternalSystem that exists neither in the bundle nor in the target environment, or a type mismatch exists in a call argument. The exception lists per-template errors. Either re-export with the missing dependency included, or pre-create the missing artifact in the target environment before importing.

Passphrase lockout

After 3 wrong passphrase attempts against the same bundle (keyed by ContentHash), BundleImporter.LoadAsync throws BundleLockedException. The session is unusable. Re-upload the bundle file to get a new session with a fresh counter. A BundleImportUnlockFailed audit row is written on each failed attempt.

BundleImportAlarmScriptUnresolved / BundleImportCompositionUnresolved warnings

These audit rows appear when the second-pass rewire (ResolveAlarmScriptLinksAsync / ResolveCompositionEdgesAsync) cannot match a name to a persisted row. The import commits — the FK is left null / the composition row is skipped — but the warning signals an incomplete import. Re-examine the bundle's dependency graph and re-export with the missing artifacts included.

Related Documentation

• Transport design specification
• Central UI
• Audit Log
• Template Engine
• Configuration Database
• Commons
• Security
• Deployment Manager