perf(controlplane): gzip artifact blob inside the hashed bytes (~10-20x smaller)
ConfigComposer gzips the serialised snapshot (CompressionLevel.Fastest) immediately after SerializeToUtf8Bytes and BEFORE taking the SHA-256, so the compressed bytes ARE the artifact everywhere: Deployment.ArtifactBlob, the LocalDb cache chunks, the gRPC fetch stream, and the RevisionHash a FetchAndCache node verifies against. The invariant the deploy path rests on — RevisionHash == SHA-256(stored/wire bytes) — is preserved, which is what leaves the fetcher, the cache and the serve path untouched: nothing between the composer and DeploymentArtifact.Open ever looks inside the blob. Read side: Open() sniffs the gzip magic 1F 8B (valid JSON can never start with 0x1F, so no framing of our own is needed) and inflates through a size guard — MaxDecompressedBytes, 512 MiB, internal + mutable so the guard can be exercised without inflating half a gigabyte per test run. Pre-upgrade uncompressed blobs take the identical path they always did. Every decompression failure — corrupt stream, truncated stream, over-limit inflation — is reported as JsonException, the same exception family the malformed-JSON path already owns. So the lenient static wrappers still degrade to empty results and the apply paths still fail the apply and keep last-known-good (#485); no caller's decision changes because the reason the bytes are unusable changed. gzip output is not guaranteed byte-stable across .NET releases. Harmless here: the hash is computed FROM the emitted bytes at compose time and never re-derived from the JSON, so the worst an implementation change costs is one spurious "changed" revision after a runtime upgrade. Snapshot determinism is unaffected — compression wraps the final bytes and reorders nothing. Tests: gzip/uncompressed parity through both the instance parses and the statics, corrupt-gzip leniency through the wrappers, the size guard (with a falsifiability control proving the same bytes inflate fine under the production limit), and composer-side blob-starts-with-1F8B + RevisionHash == SHA256(blob) with a control proving the hash is NOT over the JSON.
This commit is contained in:
@@ -1,3 +1,6 @@
|
||||
using System.IO.Compression;
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Configuration.Entities;
|
||||
@@ -86,6 +89,43 @@ public sealed class ConfigComposerTests : ControlPlaneActorTestBase
|
||||
artifact.RevisionHash.ShouldMatch("^[0-9a-f]{64}$");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Task 8: the emitted blob is gzip AND the revision hash is the SHA-256 of THOSE bytes. The second
|
||||
/// half is the load-bearing one — a <c>FetchAndCache</c> node verifies the bytes it fetched against
|
||||
/// the <c>RevisionHash</c> its dispatch carried, so hashing the uncompressed JSON instead would fail
|
||||
/// every fetch in the fleet while every hash-shape test above stayed green.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task Blob_is_gzip_and_the_revision_hash_is_over_the_compressed_bytes()
|
||||
{
|
||||
var f = NewInMemoryDbFactory();
|
||||
await using (var db = f.CreateDbContext())
|
||||
{
|
||||
db.ServerClusters.Add(NewCluster("c1"));
|
||||
await db.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var artifact = await ConfigComposer.SnapshotAndFlattenAsync(f.CreateDbContext());
|
||||
|
||||
artifact.Blob[0].ShouldBe((byte)0x1F);
|
||||
artifact.Blob[1].ShouldBe((byte)0x8B);
|
||||
artifact.RevisionHash.ShouldBe(Convert.ToHexStringLower(SHA256.HashData(artifact.Blob)));
|
||||
// …which is exactly what ConfigComposer.HashOf (the fetcher's derivation) computes over the same bytes.
|
||||
artifact.RevisionHash.ShouldBe(ConfigComposer.HashOf(artifact.Blob));
|
||||
|
||||
// Falsifiability control: the hash is NOT over the JSON. Decompress and hash that — it must differ,
|
||||
// or "hash of the compressed bytes" would be an untested claim on a blob that happened to round-trip.
|
||||
using var inflated = new MemoryStream();
|
||||
await using (var gz = new GZipStream(new MemoryStream(artifact.Blob), CompressionMode.Decompress))
|
||||
{
|
||||
await gz.CopyToAsync(inflated);
|
||||
}
|
||||
var json = inflated.ToArray();
|
||||
Encoding.UTF8.GetString(json).ShouldStartWith("{");
|
||||
Convert.ToHexStringLower(SHA256.HashData(json)).ShouldNotBe(artifact.RevisionHash);
|
||||
artifact.Blob.Length.ShouldBeLessThan(json.Length);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// v3: verifies that <see cref="ConfigComposer.SnapshotAndFlattenAsync"/> serialises a
|
||||
/// <see cref="Device"/>'s endpoint into the artifact blob and that
|
||||
|
||||
Reference in New Issue
Block a user