docs: record surgical-deploy semantics + artifact compression ops notes (plan 2026-08-15)

CLAUDE.md — a new "How a deploy reaches that address space" block under the v3
Batch 4 address-space section. The blanket ChangedRawTags/ChangedUnsReferenceVariables
=> Rebuild rule is gone, so the doc now names both halves explicitly: what still
rebuilds (alarm-bearing raw-tag deltas, raw-tag identity changes, raw container
renames/removals, changed equipment/alarms) and what is now surgical (non-alarm
attribute deltas, every UnsReferenceDelta). Also records the three semantics that
are easy to break by accident: mirror repair rides UnsReferenceVariable's inherited
backing fields (omitting a newly-inherited field silently reopens the staleness),
a degraded non-rebuild apply deliberately does not advance _lastApplied, and the
raw-realm announce gap is no longer neutral now that surgical adds keep
subscriptions alive.

CLAUDE.md ConfigSource — the RevisionHash == SHA-256(artifact bytes) paragraph is
still true and stays; it gains the clause that the blob is gzip-compressed since
2026-08-15 and the hash covers the COMPRESSED bytes, which is precisely why the
fetcher, cache and serve path were untouched.

docs/Uns.md + docs/Raw.md — one section each on what a deploy now tears down, from
the authoring surface's point of view, including the mirror repair semantics
(identity-preserving backing edits update in place and keep their MonitoredItems;
re-points and effective-name edits remove+recreate that one node).

docs/Configuration.md — "Artifact compression (2026-08-15)" under ConfigSource/
ConfigServe: gzip inside the hashed bytes, magic-sniffed on read, pre-upgrade
uncompressed rows still parse, plus the two operational facts an operator needs.
The first deploy after upgrade re-hashes every revision (same config, new bytes,
new RevisionHash) so it is a one-time full re-dispatch rather than a no-op; and
rollout is ordered — nodes must run artifact-sniffing code BEFORE central starts
emitting compressed blobs, since a pre-upgrade node handed a gzip artifact fails
the apply and keeps last-known-good until it is upgraded.

Plan doc — a "Follow-ups (recorded, not implemented)" section covering the seven
items surfaced during implementation and review: the raw-realm announce gap, the
RemoveEquipmentSubtree dangling-inverse-edge exposure, the missing end-to-end test
for a degraded remove-bearing apply's ratcheted rebuild, the scripted-alarm
thin-plan-ids-against-rich-rows blindness, the absent monotonic dispatch sequence,
LocalDb store ordering under concurrent StoreAsync, and threading the in-hand blob
through RestoreApplied.

tasks.json — all nine tasks completed.

Docs only; no source changes.
This commit is contained in:
Joseph Doherty
2026-08-15 18:02:21 -04:00
parent d787502d54
commit 0b537b5dec
6 changed files with 229 additions and 10 deletions
+33
View File
@@ -217,6 +217,39 @@ See the [ClusterClient→gRPC migration design](plans/2026-08-06-clusterclient-t
**`RevisionHash == SHA-256(artifact bytes)` is load-bearing.** `ConfigComposer` computes the deployment's revision hash as the lowercase-hex SHA-256 of the exact artifact bytes, so the node verifies a fetch against the `RevisionHash` the dispatch already carries — no hash rides in the proto. If a future change makes the revision hash anything else, every fetch fails verification.
#### Artifact compression (2026-08-15)
The artifact blob is **gzip-compressed, inside the hashed bytes**. `ConfigComposer.Pack` gzips the
serialised snapshot (`CompressionLevel.Fastest`, ~10–20x on this highly repetitive JSON) immediately
after `SerializeToUtf8Bytes` and **then** takes the SHA-256, so the compressed bytes *are* the artifact
everywhere — `Deployment.ArtifactBlob`, the LocalDb cache chunks, the gRPC fetch stream, and the
`RevisionHash` a `FetchAndCache` node verifies against. The invariant above is preserved exactly, which
is why the fetcher, the cache and the serve path are untouched: nothing between the composer and
`DeploymentArtifact.Open` looks inside the blob.
Decompression lives in exactly one place, `DeploymentArtifact.Open`, which **sniffs the gzip magic
`1F 8B`** at offset 0 — valid JSON can never start with `0x1F`, so no framing of our own is needed and
**pre-upgrade uncompressed rows keep parsing unchanged**. Inflation runs under a 512 MiB guard so a
corrupt or hostile blob cannot OOM the node, and every decompression failure (corrupt, truncated,
over-limit) is reported as `JsonException` — the same exception family the malformed-JSON path already
owns, so no caller's decision changes.
Two operational facts:
- **The first deploy after the upgrade re-hashes every revision.** The same configuration now serialises
to different (compressed) bytes, so it produces a different `RevisionHash`. Expect a **one-time full
re-dispatch to every node**, not the usual no-op — this is normal, and it happens once.
- **Rollout ordering matters: upgrade the NODES first.** A node must be running artifact-sniffing code
*before* central starts emitting compressed blobs. A pre-upgrade node handed a gzip artifact cannot
parse it and **fails the apply**, which means it keeps serving last-known-good (the #485 contract) —
degraded but not destructive, and it recovers as soon as it is upgraded. Rolling back central is
likewise safe in the other direction, since sniffing code still reads uncompressed blobs.
(gzip output is not guaranteed byte-stable across .NET releases. Harmless here: the hash is computed
from the bytes emitted at compose time and never re-derived from the JSON, so an implementation change
costs at most one spurious "changed" revision after a runtime upgrade. Snapshot determinism is
unaffected — compression wraps the final bytes and reorders nothing.)
On the docker-dev rig the central pair carries `ConfigServe` (port `4055`, dev key) and stays `Direct`; the four site nodes carry `ConfigSource` (endpoints + matching key) defaulting to `Direct`. Flip only the site nodes with `OTOPCUA_CONFIG_MODE=FetchAndCache` at `docker compose up` — central keeps SQL and models the target topology.
**`FetchAndCache` is mandatory on a driver-only node — per-cluster mesh Phase 4.** `Program.cs` now