Joseph Doherty
9dbd5e96f7
ConfigComposer gzips the serialised snapshot (CompressionLevel.Fastest) immediately after SerializeToUtf8Bytes and BEFORE taking the SHA-256, so the compressed bytes ARE the artifact everywhere: Deployment.ArtifactBlob, the LocalDb cache chunks, the gRPC fetch stream, and the RevisionHash a FetchAndCache node verifies against. The invariant the deploy path rests on — RevisionHash == SHA-256(stored/wire bytes) — is preserved, which is what leaves the fetcher, the cache and the serve path untouched: nothing between the composer and DeploymentArtifact.Open ever looks inside the blob. Read side: Open() sniffs the gzip magic 1F 8B (valid JSON can never start with 0x1F, so no framing of our own is needed) and inflates through a size guard — MaxDecompressedBytes, 512 MiB, internal + mutable so the guard can be exercised without inflating half a gigabyte per test run. Pre-upgrade uncompressed blobs take the identical path they always did. Every decompression failure — corrupt stream, truncated stream, over-limit inflation — is reported as JsonException, the same exception family the malformed-JSON path already owns. So the lenient static wrappers still degrade to empty results and the apply paths still fail the apply and keep last-known-good (#485); no caller's decision changes because the reason the bytes are unusable changed. gzip output is not guaranteed byte-stable across .NET releases. Harmless here: the hash is computed FROM the emitted bytes at compose time and never re-derived from the JSON, so the worst an implementation change costs is one spurious "changed" revision after a runtime upgrade. Snapshot determinism is unaffected — compression wraps the final bytes and reorders nothing. Tests: gzip/uncompressed parity through both the instance parses and the statics, corrupt-gzip leniency through the wrappers, the size guard (with a falsifiability control proving the same bytes inflate fine under the production limit), and composer-side blob-starts-with-1F8B + RevisionHash == SHA256(blob) with a control proving the hash is NOT over the JSON.
117 lines
7.5 KiB
C#
117 lines
7.5 KiB
C#
using System.IO.Compression;
|
||
using System.Security.Cryptography;
|
||
using System.Text.Json;
|
||
using Microsoft.EntityFrameworkCore;
|
||
using ZB.MOM.WW.OtOpcUa.Configuration;
|
||
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
|
||
|
||
namespace ZB.MOM.WW.OtOpcUa.ControlPlane.AdminOperations;
|
||
|
||
/// <summary>
|
||
/// Pure snapshot composer: reads the current live-edit state from <see cref="OtOpcUaConfigDbContext"/>
|
||
/// and serialises it into a deterministic, gzip-compressed byte[] artifact + SHA-256 hex revision hash.
|
||
/// Determinism comes from sorting every collection by its natural key before serialising, so two snapshots
|
||
/// over the same DB state always produce the same hash regardless of EF row ordering; compression wraps the
|
||
/// final bytes and reorders nothing. The hash is taken over the COMPRESSED bytes — see <see cref="Pack"/>.
|
||
/// </summary>
|
||
public static class ConfigComposer
|
||
{
|
||
public sealed record ConfigArtifact(byte[] Blob, string RevisionHash);
|
||
|
||
private static readonly JsonSerializerOptions JsonOptions = new()
|
||
{
|
||
WriteIndented = false,
|
||
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.Never,
|
||
};
|
||
|
||
/// <summary>Reads the current configuration and returns a deterministic snapshot blob with revision hash.</summary>
|
||
/// <param name="db">The configuration database context.</param>
|
||
/// <param name="ct">The cancellation token for the operation.</param>
|
||
/// <returns>A deterministic <see cref="ConfigArtifact"/> containing the serialised snapshot bytes and its revision hash.</returns>
|
||
public static async Task<ConfigArtifact> SnapshotAndFlattenAsync(
|
||
OtOpcUaConfigDbContext db, CancellationToken ct = default)
|
||
{
|
||
// v3 greenfield snapshot: the Raw tree (RawFolders → DriverInstances(+RawFolderId) → Devices(+DeviceConfig)
|
||
// → TagGroups → Tags(DeviceId/TagGroupId, no EquipmentId/FolderPath/DriverInstanceId)) plus the UNS
|
||
// projection (UnsAreas/UnsLines/Equipment/UnsTagReferences) and the retained VirtualTags/ScriptedAlarms/
|
||
// PollGroups/NodeAcls/Scripts. The Namespaces snapshot is RETIRED (the two OPC UA namespaces are implicit).
|
||
// RevisionHash inputs follow the blob automatically — adding/removing a table shifts the SHA-256.
|
||
var snapshot = new
|
||
{
|
||
Clusters = await db.ServerClusters.AsNoTracking().OrderBy(x => x.ClusterId).ToListAsync(ct),
|
||
Nodes = await db.ClusterNodes.AsNoTracking().OrderBy(x => x.NodeId).ToListAsync(ct),
|
||
RawFolders = await db.RawFolders.AsNoTracking().OrderBy(x => x.RawFolderId).ToListAsync(ct),
|
||
DriverInstances = await db.DriverInstances.AsNoTracking().OrderBy(x => x.DriverInstanceId).ToListAsync(ct),
|
||
Devices = await db.Devices.AsNoTracking().OrderBy(x => x.DeviceId).ToListAsync(ct),
|
||
TagGroups = await db.TagGroups.AsNoTracking().OrderBy(x => x.TagGroupId).ToListAsync(ct),
|
||
Equipment = await db.Equipment.AsNoTracking().OrderBy(x => x.EquipmentId).ToListAsync(ct),
|
||
Tags = await db.Tags.AsNoTracking().OrderBy(x => x.TagId).ToListAsync(ct),
|
||
UnsTagReferences = await db.UnsTagReferences.AsNoTracking().OrderBy(x => x.UnsTagReferenceId).ToListAsync(ct),
|
||
PollGroups = await db.PollGroups.AsNoTracking().OrderBy(x => x.PollGroupId).ToListAsync(ct),
|
||
UnsAreas = await db.UnsAreas.AsNoTracking().OrderBy(x => x.UnsAreaId).ToListAsync(ct),
|
||
UnsLines = await db.UnsLines.AsNoTracking().OrderBy(x => x.UnsLineId).ToListAsync(ct),
|
||
NodeAcls = await db.NodeAcls.AsNoTracking().OrderBy(x => x.NodeAclId).ToListAsync(ct),
|
||
// #520 stage 4: the data-plane credentials travel with the config, because a driver-only node
|
||
// has no ConfigDb to read them from (per-cluster mesh Phase 4) and would otherwise deny every
|
||
// UserName token. Only the peppered HMAC of each secret ships — never the secret — and the
|
||
// pepper is node configuration that is deliberately NOT in the artifact, so a stolen blob
|
||
// still cannot be turned into a working credential. This is also why revoking a key reaches a
|
||
// driver-only node only on the next deployment; see the ApiKey entity's remarks.
|
||
ApiKeys = await db.ApiKeys.AsNoTracking().OrderBy(x => x.ApiKeyId).ToListAsync(ct),
|
||
Scripts = await db.Scripts.AsNoTracking().OrderBy(x => x.ScriptId).ToListAsync(ct),
|
||
VirtualTags = await db.VirtualTags.AsNoTracking().OrderBy(x => x.VirtualTagId).ToListAsync(ct),
|
||
ScriptedAlarms = await db.ScriptedAlarms.AsNoTracking().OrderBy(x => x.ScriptedAlarmId).ToListAsync(ct),
|
||
};
|
||
|
||
return Pack(JsonSerializer.SerializeToUtf8Bytes(snapshot, JsonOptions));
|
||
}
|
||
|
||
/// <summary>
|
||
/// Compress the serialised snapshot and hash the COMPRESSED bytes — the emitted artifact and its
|
||
/// revision hash, in one step, so the two can never be derived from different byte sequences.
|
||
/// </summary>
|
||
/// <remarks>
|
||
/// <para>
|
||
/// <b>Compression happens before hashing on purpose.</b> The invariant the whole deploy path
|
||
/// rests on is <c>RevisionHash == SHA-256(stored/wire bytes)</c>: a <c>FetchAndCache</c> node
|
||
/// verifies the bytes it fetched against the hash its dispatch already carried, and the LocalDb
|
||
/// cache stores those same bytes. Hashing the uncompressed JSON instead would break every one of
|
||
/// those checks. Nothing between here and <c>DeploymentArtifact.Open</c> looks inside the blob,
|
||
/// which is what makes the change invisible to the cache and the gRPC serve path.
|
||
/// </para>
|
||
/// <para>
|
||
/// gzip with the standard magic <c>1F 8B</c> and no framing of our own: valid JSON can never
|
||
/// start with <c>0x1F</c>, so the read side sniffs the first two bytes and pre-upgrade
|
||
/// uncompressed rows keep parsing unchanged. <see cref="CompressionLevel.Fastest"/> because the
|
||
/// artifact is highly repetitive JSON — the marginal ratio from a slower level is small next to
|
||
/// the ~10–20x the first level already delivers.
|
||
/// </para>
|
||
/// <para>
|
||
/// gzip output is not guaranteed byte-stable across .NET releases. Harmless here: the hash is
|
||
/// computed from the bytes emitted at compose time and never re-derived from the JSON, so an
|
||
/// implementation change costs at most one spurious "changed" revision after a runtime upgrade.
|
||
/// Determinism of the snapshot itself is unaffected — compression wraps the final bytes and
|
||
/// reorders nothing.
|
||
/// </para>
|
||
/// </remarks>
|
||
/// <param name="json">The deterministic UTF-8 JSON snapshot.</param>
|
||
/// <returns>The compressed artifact blob and the SHA-256 hex digest OF THAT BLOB.</returns>
|
||
internal static ConfigArtifact Pack(byte[] json)
|
||
{
|
||
using var output = new MemoryStream(json.Length / 8);
|
||
using (var gzip = new GZipStream(output, CompressionLevel.Fastest, leaveOpen: true))
|
||
{
|
||
gzip.Write(json, 0, json.Length);
|
||
}
|
||
|
||
var blob = output.ToArray();
|
||
return new ConfigArtifact(blob, Convert.ToHexStringLower(SHA256.HashData(blob)));
|
||
}
|
||
|
||
/// <summary>Returns the SHA-256 hex digest of the supplied artifact bytes (lowercase, no prefix).</summary>
|
||
/// <param name="blob">The bytes to hash.</param>
|
||
/// <returns>The lowercase hex-encoded SHA-256 digest.</returns>
|
||
public static string HashOf(ReadOnlySpan<byte> blob) =>
|
||
Convert.ToHexStringLower(SHA256.HashData(blob));
|
||
}
|