dohertj2/lmxopcua
1
0
Fork 0
Code Issues 32 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0dbbb5ac6becb07dec047f95e25a6e33a2e16f7d
lmxopcua/docs/plans/2026-08-15-surgical-deploys.md
T

32 KiB
Raw Blame History

Surgical Deploys Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.

Goal: Make routine config edits non-events instead of fleet-wide outages: a raw-tag attribute edit or a UNS-reference add/remove/re-point no longer tears down the whole address space (killing every client subscription), the artifact is read from SQL once per apply instead of three times, fully JSON-parsed once per actor instead of ~8–11 times, only the delta is materialised instead of the whole composition, blocking apply I/O leaves the hot actor mailboxes, and the artifact blob is deflate-compressed inside the hashed bytes.

Architecture: Three prongs. (1) Surgical routing: AddressSpaceChangeClassifier learns that a non-alarm RawTagDelta and any UnsReferenceDelta are surgical-eligible; AddressSpaceApplier applies raw-tag attribute edits in place via the existing ISurgicalAddressSpaceSink.UpdateTagAttributes (Raw realm) and handles UNS-reference deltas as remove(previous-node)+recreate(current) — the recreate rides the delta materialise pass. Alarm-bearing raw tags keep the full-rebuild path (the WP4 notifier-wiring guard). (2) Parse/read-once: Direct mode threads the already-read blob into the rebuild + subscription pushes (the FetchAndCache path is the template, DriverHostActor.cs:1650-1654); a new DeploymentArtifact.Open() returns a ParsedDeploymentArtifact that parses the JSON once and serves all consumers; the materialise passes gain subset overloads so non-rebuild applies materialise only the plan's added/changed nodes. (3) Mechanics: the LocalDb cache write leaves the mailbox via PipeTo, the Applying-state dispatch deferral stops busy-spinning, and ConfigComposer emits a deflate-compressed blob (magic-sniffed on the read side, hash computed over the compressed bytes so RevisionHash == SHA-256(stored/wire bytes) stays true).

Tech Stack: C# / .NET 10, System.Text.Json (JsonDocument), Akka.NET, System.IO.Compression, xUnit + Shouldly.

Branch: perf/surgical-deploys (branched off perf/value-pipeline-batching — plans 1 and 2 touch the same actors; do NOT branch off master).

Context for implementers (read before any task)

Verified facts — do not re-derive:

  • The rebuild cliff: AddressSpaceChangeClassifier.Classify (src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/AddressSpaceChangeClassifier.cs:69-78) routes ANY non-empty ChangedRawTags / ChangedUnsReferenceVariables (among others) to Rebuild. RawTagPlan.ReferencingEquipmentPaths participates in record equality (AddressSpaceComposer.cs, RawTagPlan.Equals), so adding/removing a UnsTagReference flips the backing raw tag to Changed ⇒ full rebuild. The WP4 comment block (AddressSpaceChangeClassifier.cs:62-68) is the binding guard: any surgical ChangedRawTags path for ALARM tags must re-wire notifiers — we sidestep it by keeping alarm-bearing deltas on the Rebuild path.
  • The applier already has every surgical primitive: ISurgicalAddressSpaceSink (src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/ISurgicalAddressSpaceSink.cs) — UpdateTagAttributes(nodeId, writable, historianTagname, dataType, isArray, arrayLength, realm) (realm-aware, returns false ⇒ caller rebuilds), RemoveVariableNode, RemoveAlarmConditionNode, RemoveEquipmentSubtree, UpdateFolderDisplayName. The applier's surgical branch is AddressSpaceApplier.Apply (AddressSpaceApplier.cs:151-224); its false/throw ⇒ full-rebuild ratchet must be preserved for every new surgical operation.
  • Apply flow (publish actor): OpcUaPublishActor.HandleRebuild (OpcUaPublishActor.cs:361-512): load blob (msg.Artifact wins; else ConfigDb) → FeedDataPlaneAuthority(artifact) (MUST stay ahead of the plan.IsEmpty return — #520 stage 4) → ParseComposition → AddressSpacePlanner.Compute(_lastApplied, composition) → plan.IsEmpty return → _applier.Apply(plan) → six full-composition materialise passes (:456-480) → AnnounceAddedNodes when !outcome.RebuildCalled. _lastApplied advances only after Apply.
  • Apply flow (host actor): Direct mode ApplyAndAck (DriverHostActor.cs:1680-1754): ReconcileDrivers(deploymentId) reads the blob from SQL and RETURNS it (appliedBlob), then RebuildAddressSpace(correlation, deploymentId) (NO bytes ⇒ publish actor re-reads SQL at OpcUaPublishActor.cs:578-593) and PushDesiredSubscriptions(deploymentId) (re-reads SQL again). FetchAndCache mode HandleFetchedForApply (:1609-1674) already does it right: ReconcileDriversFromBlob(id, bytes) → RebuildAddressSpace(corr, id, appliedBlob) → PushDesiredSubscriptionsFromArtifact(id, appliedBlob) → CacheAppliedArtifact.
  • Parse entry points (src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DeploymentArtifact.cs): ParseDriverInstances(ReadOnlySpan<byte>) :89, ResolveClusterScope(ReadOnlySpan<byte>, string) :513, ParseDriverInstances(blob, nodeId) :555 (scope+parse), ParseDataPlaneAuthority(blob, nodeId) :591, ParseComposition(blob) :893, ParseComposition(blob, nodeId, onInconsistency) :955 (scope+parse+BuildClusterSets :1018). EVERY one does JsonDocument.Parse(blob.ToArray()) — a full copy per parse. The private Build* helpers already take JsonElement root, which is what makes the parse-once refactor mechanical.
  • Per-apply parse count today: host: ReconcileDrivers* (scope+instances = 2) + PushDesiredSubscriptions* (scope+composition+clustersets = 3) + CacheAppliedArtifact (scope = 1); publish actor: FeedDataPlaneAuthority (authority+scope = 2) + ParseComposition (3). Target: ONE JsonDocument per actor per apply.
  • Blocking I/O on mailboxes: CacheAppliedArtifact does .GetAwaiter().GetResult() (DriverHostActor.cs:1901-1904, comment at :1870-1875 acknowledges it); the Applying-state deferral re-Forwards to Self in a hot loop (DriverHostActor.cs:944-955); BeginFetchAndCacheApply (:1571-1602) is the PipeTo template.
  • Compression constraint (CLAUDE.md, load-bearing): RevisionHash == SHA-256(artifact bytes) — ConfigComposer computes the hash of the exact blob, GrpcDeploymentArtifactFetcher verifies fetched bytes against it, chunked LocalDb cache stores those bytes. Compressing BEFORE hashing preserves the invariant everywhere (hash of the compressed bytes). The dispatch stays payload-free.
  • Invariants to preserve:
    1. #485 "no bytes is no answer" — a null/empty artifact fails the apply and keeps last-known-good; never let compression or parse-once break that path.
    2. FeedDataPlaneAuthority runs ahead of plan.IsEmpty — a key-revocation-only artifact must still land.
    3. The full-rebuild ratchet — any surgical false/throw falls back to SafeRebuild().
    4. _lastApplied advances only on a successful Apply — the retry diffs against what is actually materialised.
    5. Deterministic composition — ConfigComposer's deterministic sort is what makes revision-hash no-ops real; compression must not reorder anything (it can't — it wraps the final bytes).
    6. The classifier stays default-closed — any delta shape not explicitly whitelisted still lands on Rebuild.
  • Key records: RawTagPlan(TagId, NodeId, ParentNodeId, DriverInstanceId, Name, DataType, Writable, Alarm, ReferencingEquipmentPaths, IsHistorized, HistorianTagname, IsArray, ArrayLength, Realm) with custom Equals (AddressSpaceComposer.cs:141+); UnsReferenceVariable(UnsTagReferenceId, EquipmentId, NodeId, EffectiveName, BackingRawPath, DataType, Writable, Realm) (:204+); AddressSpacePlan.RawTagDelta(Previous, Current) / UnsReferenceDelta(Previous, Current) (AddressSpacePlan.cs:134-138).
  • Materialise passes (AddressSpaceApplier.cs): MaterialiseHierarchy(composition) :643, MaterialiseRawSubtree(composition) :682 (needs BuildEquipmentIdByFolderPath(composition) for alarm notifier wiring), MaterialiseUnsReferences(composition) :776 (builds rawByPath from composition.RawTags for inherited shape/historian), MaterialiseScriptedAlarms, MaterialiseEquipmentTags, MaterialiseEquipmentVirtualTags (in the :820+ region). All idempotent loops over composition lists.
  • Existing tests to keep green: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AddressSpaceChangeClassifierTests.cs + AddressSpaceChangeClassifierDualNamespaceTests.cs (some assert ChangedRawTags ⇒ Rebuild / ChangedUnsReferenceVariables ⇒ Rebuild — those assertions CHANGE deliberately in Task 4; update them to assert the new contract, alarm-delta cases still Rebuild), applier tests, OpcUaPublishActorApplyFailureTests, DriverHostActorFetchAndCacheBootstrapTests, tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests/DeploymentArtifactProtoTests.cs, and tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests/SubscriptionSurvivalTests.cs (the live gate shape for "subscriptions survive").
  • Commit style: feat(...) / perf(...) / refactor(...) with scope; one commit per task.

Task 1: Direct mode threads the already-read blob

Classification: small Estimated implement time: ~3 min Parallelizable with: Task 2

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs:1726-1730
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/ — extend the existing apply-path test class that exercises ApplyAndAck (grep for it; the FetchAndCache tests show the probe idiom)

Step 1: Failing test — drive a Direct-mode apply with a real blob in the fake ConfigDb; assert the publish-actor probe's RebuildAddressSpace message carries Artifact != null with the exact blob bytes (today it carries null).

Step 3: Implement — in ApplyAndAck, replace lines :1726 and :1729:

_opcUaPublishActor?.Tell(new ZB.MOM.WW.OtOpcUa.Runtime.OpcUa.OpcUaPublishActor.RebuildAddressSpace(correlation, deploymentId, appliedBlob));
PushDesiredSubscriptionsFromArtifact(deploymentId, appliedBlob);

(appliedBlob is non-null here — the null case returned at :1706.) Leave PushDesiredSubscriptions(deploymentId) in place for any other caller; if it becomes call-site-dead, delete it and its SQL read. Update the comment block: the publish actor no longer reads central SQL on the Direct path either — LoadArtifact/LoadLatestArtifact remain only as the legacy/dev fallback for a bytes-less RebuildAddressSpace.

Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests — PASS.

Step 5: git add -A && git commit -m "perf(runtime): Direct-mode apply threads the in-hand artifact (1 SQL read per apply, not 3)"

Task 2: DeploymentArtifact.Open() — parse once

Classification: high-risk (config-parsing seam; behavior must be byte-identical) Estimated implement time: ~5 min Parallelizable with: Task 1

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DeploymentArtifact.cs
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/ParsedDeploymentArtifactTests.cs (create)

Step 1: Failing test — build a representative artifact JSON (reuse/borrow the builder from existing DeploymentArtifact tests — grep ParseComposition under tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests for the fixture); assert parity: using var parsed = DeploymentArtifact.Open(blob); then parsed.ResolveClusterScope(node), parsed.ParseDriverInstances(node), parsed.ParseComposition(node, onInconsistency), parsed.ParseDataPlaneAuthority(node) each equal the static counterparts' results (deep equality on the records; compositions compare with their record equality).

Step 3: Implement

  • Add:
/// <summary>One-parse view over an artifact blob: a single JsonDocument serves every consumer of an
/// apply (driver reconcile, subscription push, composition, data-plane authority, cluster scope) instead
/// of each entry point re-parsing — at production sizes each static parse was a full LOH copy + document
/// build, and one apply performed ~8-11 of them. Dispose when the apply completes.</summary>
public sealed class ParsedDeploymentArtifact : IDisposable
{
    private readonly JsonDocument _doc;
    private readonly byte[] _blob;                 // decoded (post-decompression) bytes
    private ClusterScope? _scope;                  // memoized per node — every consumer resolves the same node
    private string? _scopeNodeId;
    internal ParsedDeploymentArtifact(JsonDocument doc, byte[] blob) { _doc = doc; _blob = blob; }
    public JsonElement Root => _doc.RootElement;
    public void Dispose() => _doc.Dispose();
}
  • public static ParsedDeploymentArtifact Open(byte[] blob) — parse ONCE (JsonDocument.Parse(blob) — no .ToArray() copy; the caller's array is not mutated afterward on any call path).
  • Move the body of each static entry point into an instance method on ParsedDeploymentArtifact operating on Root (the private Build* helpers already take JsonElement root — pass Root straight in). Memoize ResolveClusterScope(nodeId) per node id (_scope/_scopeNodeId) since ParseDriverInstances/ParseComposition/ParseDataPlaneAuthority each begin with the same scope resolution.
  • Re-express every existing static as a thin wrapper: using var parsed = Open(blob.ToArray-if-span-else-array); return parsed.X(...) — public signatures may stay ReadOnlySpan<byte> for compatibility (the wrapper pays one copy; the hot path stops using the statics in Task 3). Do not change any output shape.

Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests + dotnet test tests/Core/ZB.MOM.WW.OtOpcUa.Commons.Tests — PASS.

Step 5: git commit -m "refactor(runtime): ParsedDeploymentArtifact — one JsonDocument per apply"

Task 3: Actors consume Open()

Classification: standard Estimated implement time: ~4 min Parallelizable with: none (needs 1 + 2)

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs (ReconcileDriversFromBlob, PushDesiredSubscriptionsFromArtifact, CacheAppliedArtifact call sites within one apply)
  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/OpcUa/OpcUaPublishActor.cs (HandleRebuild + FeedDataPlaneAuthority)

Implement (behavior-preserving):

  • Host: in ApplyAndAck / HandleFetchedForApply, using var parsed = DeploymentArtifact.Open(appliedBlob); and pass parsed down (add internal overloads ReconcileDriversFromParsed, PushDesiredSubscriptionsFromParsed, CacheAppliedArtifact(deploymentId, revision, blob, parsed) — cache still stores the BYTES; it only needs parsed for the scope lookup it currently re-parses for). The existing blob-based methods stay as wrappers for the boot paths.
  • Publish actor: HandleRebuild opens once after the #485 guard; FeedDataPlaneAuthority(parsed, artifactBytes) (the SHA-256 at :535 still hashes the raw bytes); the composition parse uses parsed.ParseComposition(...).
  • Tests: the Task 2 parity tests + the existing apply suites ARE the net; add one assertion-level test only if a seam demands it.

Run both Runtime + OpcUaServer test projects. Commit: perf(runtime): apply path parses the artifact once per actor

Task 4: Classifier — surgical eligibility for raw-tag and UNS-reference deltas

Classification: high-risk (the outage-per-edit fix; default-closed semantics) Estimated implement time: ~5 min Parallelizable with: none

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/AddressSpaceChangeClassifier.cs
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/AddressSpaceChangeClassifierTests.cs + AddressSpaceChangeClassifierDualNamespaceTests.cs (update the deliberately-changed assertions; add the new cases)

Step 1: Write the failing tests (these define the new contract):

  1. Reference_only_raw_tag_delta_is_not_a_rebuild — a RawTagDelta whose ONLY difference is ReferencingEquipmentPaths, Alarm == null both sides ⇒ kind is NOT Rebuild (with no other deltas ⇒ AttributeOnly).
  2. Attribute_raw_tag_delta_is_not_a_rebuild — non-alarm delta differing in Writable/IsHistorized/HistorianTagname/DataType/IsArray/ArrayLength ⇒ not Rebuild.
  3. Alarm_bearing_raw_tag_delta_still_rebuilds — Alarm != null on either side ⇒ Rebuild (the WP4 guard).
  4. Identity_field_raw_tag_delta_still_rebuilds — e.g. DriverInstanceId or Name differs ⇒ Rebuild (default-closed).
  5. Uns_reference_delta_is_not_a_rebuild — a UnsReferenceDelta (re-point or display-name case) ⇒ not Rebuild.
  6. Combination: UNS-ref ADD + the induced reference-only raw-tag delta ⇒ PureAdd (the headline case — one added UNS variable, zero teardown).

Step 3: Implement

  • Add the predicate (mirror TagDeltaIsSurgicalEligible's whitelist-via-with idiom, which auto-closes over future fields):
// v3 surgical raw-tag eligibility: a NON-alarm raw tag whose only differences are the in-place-updatable
// attributes (Writable / IsHistorized / HistorianTagname / DataType / IsArray / ArrayLength) and/or its
// ReferencingEquipmentPaths. The referencing set is node-inert for a VALUE tag — it only drives native-alarm
// notifier wiring (the WP4 guard), which is why Alarm != null on EITHER side disqualifies. Identity fields
// (NodeId is the diff key; TagId / DriverInstanceId / Name / ParentNodeId / Realm) still force a rebuild via
// the with-equality default-closed pattern.
internal static bool RawTagDeltaIsSurgicalEligible(AddressSpacePlan.RawTagDelta d) =>
    d.Previous.Alarm is null && d.Current.Alarm is null &&
    (d.Previous with
    {
        Writable = d.Current.Writable,
        IsHistorized = d.Current.IsHistorized,
        HistorianTagname = d.Current.HistorianTagname,
        DataType = d.Current.DataType,
        IsArray = d.Current.IsArray,
        ArrayLength = d.Current.ArrayLength,
        ReferencingEquipmentPaths = d.Current.ReferencingEquipmentPaths,
    }).Equals(d.Current);
  • In Classify rule 2 (:69-78): replace plan.ChangedRawTags.Count > 0 with plan.ChangedRawTags.Any(d => !RawTagDeltaIsSurgicalEligible(d)), and DELETE plan.ChangedUnsReferenceVariables.Count > 0 from the rebuild condition (every UnsReferenceDelta is handled surgically in Tasks 5-6: remove-previous + recreate-current).
  • Rewrite the WP4 comment block to state the NEW contract: alarm-bearing raw-tag deltas still rebuild (notifier wiring); value-tag deltas are surgical; the applier owns UpdateTagAttributes for them; UNS-ref deltas are remove+recreate.

Step 4: dotnet test tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests --filter "FullyQualifiedName~AddressSpaceChangeClassifier" — PASS (including the updated legacy assertions).

Step 5: git commit -m "feat(opcuaserver): classifier — non-alarm raw-tag + UNS-reference deltas are surgical, not rebuilds"

Task 5: Applier — apply the new surgical deltas

Classification: high-risk (rebuild ratchet semantics) Estimated implement time: ~5 min Parallelizable with: none (needs 4)

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/AddressSpaceApplier.cs (Apply, :151-224)
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/ — extend the applier test class (find it via grep AddressSpaceApplier in that project; use its recording-fake surgical sink)

Step 1: Failing tests

  1. Surgical_raw_tag_delta_updates_in_place_raw_realm — plan with one eligible RawTagDelta (historize flip); fake surgical sink records UpdateTagAttributes(d.Current.NodeId, writable, historian, dataType, isArray, arrayLength, AddressSpaceRealm.Raw) with historian resolved as MaterialiseRawSubtree resolves it (override else NodeId, AddressSpaceApplier.cs:748-750) and writable = Writable && !IsArray; NO RebuildAddressSpace call.
  2. Reference_only_raw_tag_delta_is_a_noop_update — only ReferencingEquipmentPaths differs ⇒ either no sink call or one idempotent UpdateTagAttributes (assert no rebuild; prefer SKIPPING the call when the with-comparison excluding the referencing set shows attribute equality — implement that skip).
  3. Uns_reference_delta_removes_previous_node — plan with one UnsReferenceDelta; sink records terminal-Bad WriteValue(Previous.NodeId, ..., Uns) then RemoveVariableNode(Previous.NodeId, Uns); no rebuild. (The recreate happens in the publish actor's delta materialise pass — Task 6 — NOT here.)
  4. Surgical_false_falls_back_to_rebuild — surgical sink returns false for the raw-tag update ⇒ RebuildAddressSpace called (ratchet preserved).

Step 3: Implement in Apply:

  • Compute var surgicalRawTagDeltas = plan.ChangedRawTags.Where(AddressSpaceChangeClassifier.RawTagDeltaIsSurgicalEligible).ToList(); (when kind != Rebuild this equals ALL of ChangedRawTags — the classifier guarantees it; still filter, defense-in-depth) and var unsRefDeltas = plan.ChangedUnsReferenceVariables;
  • Extend the surgical-branch trigger (:171): surgicalTagDeltas.Count > 0 || renamedFolders.Count > 0 || hasRemovePass || surgicalRawTagDeltas.Count > 0 || unsRefDeltas.Count > 0.
  • Inside the ISurgicalAddressSpaceSink branch, after the existing equipment-tag loop:
    • Raw-tag loop: skip when attribute-equal apart from ReferencingEquipmentPaths (test 2); else surgical.UpdateTagAttributes(d.Current.NodeId, d.Current.Writable && !d.Current.IsArray, d.Current.IsHistorized ? (string.IsNullOrWhiteSpace(d.Current.HistorianTagname) ? d.Current.NodeId : d.Current.HistorianTagname) : null, d.Current.DataType, d.Current.IsArray, d.Current.ArrayLength, AddressSpaceRealm.Raw) with the same try/catch + allApplied ratchet as the equipment-tag loop.
    • UNS-ref loop: SafeWriteValue(d.Previous.NodeId, null, OpcUaStatus.Bad, ts, AddressSpaceRealm.Uns); then SafeRemoveVariable(surgical, d.Previous.NodeId, AddressSpaceRealm.Uns) into the ratchet. Doc-comment WHY remove+recreate (a re-point must move the Organizes edge and the sink has no RemoveReference; a display-name edit changes the NodeId; remove+recreate covers both, at the cost of that one node's monitored items — vs. every subscription server-wide today).
  • changedCount accounting already includes both sets (:115-121) — verify, don't double-count.
  • Also update the FeedHistorizedRefs changed-raw-tags handling: it already handles plan.ChangedRawTags (:571-578) — now those deltas reach it on the surgical path too; confirm the logic is delta-shape-agnostic (it is) and leave it.

Step 4: run the OpcUaServer.Tests project — PASS.

Step 5: git commit -m "feat(opcuaserver): applier applies raw-tag + UNS-reference deltas surgically"

Task 6: Delta materialisation (publish actor materialises the plan, not the world)

Classification: high-risk (the O(fleet)-per-apply fix) Estimated implement time: ~5 min Parallelizable with: none (needs 5)

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/AddressSpaceApplier.cs (subset overloads on the six passes)
  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/OpcUa/OpcUaPublishActor.cs:456-480
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests/ applier tests + tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/OpcUa/ publish-actor tests

Step 1: Failing tests

  1. Pure_add_materialises_only_the_added_nodes — publish-actor-level: apply composition A (N nodes), then A+1 tag; the fake sink's EnsureVariable/EnsureFolder calls after the second apply cover ONLY the added tag (+ its added containers), not all N. (This is the headline test.)
  2. Rebuild_still_materialises_the_full_composition — force a rebuild-kind delta; full-composition passes run as today.
  3. Uns_reference_delta_recreates_current_node — a re-point delta: after Task 5's remove, the UNS pass re-ensures Current.NodeId with shape inherited from the FULL composition's rawByPath (backing tag NOT in the delta), plus the AddReference Organizes edge to the NEW backing path.
  4. Applier-level: MaterialiseRawSubtree(full, subsetTags, subsetContainers) wires alarm notifiers using the FULL composition's equipment map.

Step 3: Implement

  • Applier: give the passes subset parameters, full-composition context retained for lookups:
    • MaterialiseRawSubtree(AddressSpaceComposition composition, IReadOnlyList<RawContainerNode>? onlyContainers = null, IReadOnlyList<RawTagPlan>? onlyTags = null) — iterate onlyContainers ?? composition.RawContainers / onlyTags ?? composition.RawTags; BuildEquipmentIdByFolderPath(composition) stays full.
    • MaterialiseUnsReferences(AddressSpaceComposition composition, IReadOnlyList<UnsReferenceVariable>? only = null) — iterate only ?? composition.UnsReferenceVariables; rawByPath stays full.
    • MaterialiseScriptedAlarms / MaterialiseEquipmentTags / MaterialiseEquipmentVirtualTags: same only pattern over their respective lists.
    • MaterialiseHierarchy stays full-composition ALWAYS (folders are O(areas+lines+equipment), cheap, and parents must exist for any add).
  • Publish actor (:456-480): after var outcome = _applier.Apply(plan); branch on outcome.RebuildCalled:
    • Rebuild ⇒ the six passes exactly as today (full composition).
    • Non-rebuild ⇒ MaterialiseHierarchy(composition) full; then subsets: containers = plan.AddedRawContainers, raw tags = plan.AddedRawTags, UNS refs = plan.AddedUnsReferenceVariables concat plan.ChangedUnsReferenceVariables.Select(d => d.Current), scripted alarms = plan.AddedAlarms, equipment tags = plan.AddedEquipmentTags, vtags = plan.AddedEquipmentVirtualTags. Skip a pass entirely when its subset is empty.
    • Comment WHY: the passes are idempotent but not free — each call is a lock acquisition; a one-tag add on a 50k-node server was ~50k sink calls.
  • AnnounceAddedNodes stays as-is (known pre-existing gap: raw/UNS adds are not announced — out of scope, note it in the comment).

Step 4: OpcUaServer.Tests + Runtime.Tests — PASS (watch OpcUaPublishActorApplyFailureTests + DarkAddressSpaceReasons for full-pass assumptions; update only assertions that pinned the OLD full-pass behavior explicitly, preserving their intent).

Step 5: git commit -m "perf(opcuaserver): non-rebuild applies materialise the delta, not the composition"

Task 7: Blocking I/O off the host mailbox

Classification: high-risk (actor lifecycle) Estimated implement time: ~5 min Parallelizable with: Task 8

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs (CacheAppliedArtifact :1896-1904 region; Applying-state deferral :944-955)
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/ — extend the apply/dispatch test classes

Part A — async cache write. CacheAppliedArtifact's .GetAwaiter().GetResult() becomes fire-and-forget with observed completion:

_artifactCache.StoreAsync(...).PipeTo(Self,
    success: () => new ArtifactCached(deploymentId),
    failure: ex => new ArtifactCacheFailed(deploymentId, ex.Message));

(private records; handlers just log Info/Warning — the cache is a boot fallback, not apply-critical; the docstring at :1870-1875 already debates exactly this and the apply outcome does not depend on it). CHECK first whether BootstrapFromCache (:788, :859) races a concurrent store — it runs in PreStart before any apply, so no; note that in the comment. Keep the store idempotent-skip (IsAlreadyCachedAsync) inside the async flow. Failing test: apply completes and acks BEFORE a deliberately-slow fake cache resolves; a follow-up ArtifactCached self-message is observed (TestKit probe or log assertion per the project's idiom).

Part B — stash instead of forward-spin. The Applying-state deferral for a DIFFERENT deployment (:944-955): make DriverHostActor implement IWithUnboundedStash (public IStash Stash { get; set; }), replace Self.Forward(msg) with Stash.Stash(), and call Stash.UnstashAll() at every Become(Steady) transition (both finally blocks + the fetch-failure path). Failing test: send dispatch B while A's fetch is in flight (slow fake fetcher); assert no busy-loop (the log does NOT contain the deferral line more than once) and B applies after A completes.

Step 4: Runtime.Tests — PASS. Step 5: git commit -m "perf(runtime): apply-path cache write off-mailbox; stash deferred dispatches"

Task 8: Compressed artifact blob

Classification: high-risk (wire/storage format; hash invariant) Estimated implement time: ~5 min Parallelizable with: Task 7

Files:

  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/AdminOperations/ConfigComposer.cs (emit compressed; hash unchanged in derivation — SHA-256 of the emitted bytes)
  • Modify: src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DeploymentArtifact.cs (Open() + static wrappers sniff-and-decompress)
  • Test: tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests/Drivers/ParsedDeploymentArtifactTests.cs + the ConfigComposer test class (grep SnapshotAndFlattenAsync under tests/Server)

Format decision (locked): gzip via System.IO.Compression.GZipStream (CompressionLevel.Fastest), detected by the gzip magic bytes 0x1F 0x8B at offset 0 (a JSON artifact can never start with 0x1F). No custom framing. Compression happens INSIDE ConfigComposer immediately after SerializeToUtf8Bytes, BEFORE the revision hash is computed — so RevisionHash == SHA-256(stored bytes) holds and GrpcDeploymentArtifactFetcher's verification (GrpcDeploymentArtifactFetcher.cs:89-90) is untouched. The LocalDb cache + gRPC serve stores/ships the compressed bytes as-is (they never look inside). Decompression lives in exactly one place: DeploymentArtifact.Open() (and thus every static wrapper). OpcUaPublishActor.FeedDataPlaneAuthority's SHA at OpcUaPublishActor.cs:535 hashes the RAW (compressed) bytes it received — verify that call site hashes the bytes as-carried, not a decompressed copy.

Step 1: Failing tests

  1. Open_reads_gzip_compressed_artifacts — compress a fixture blob, parity-assert ParseComposition vs the uncompressed parse.
  2. Open_still_reads_uncompressed_artifacts (backward compat — pre-upgrade rows).
  3. Composer_emits_gzip_and_hash_of_compressed_bytes — ArtifactBlob starts with 1F 8B; the deployment's RevisionHash equals SHA256(ArtifactBlob) lowercase hex.

Step 3: Implement — decompress with a size guard (refuse decompressed output > 512 MB — a corrupt/hostile blob must not OOM the node; treat over-limit exactly like unparseable JSON: the existing catch paths). Note in a comment: gzip output is not guaranteed byte-stable across .NET releases; the hash is computed from the emitted bytes at compose time and never re-derived, so the only consequence of an impl change is one spurious "changed" revision after a runtime upgrade.

Step 4: Runtime.Tests + ControlPlane tests (if the composer test project is DB-backed and the DB is unreachable, run the artifact round-trip tests and note the skip). Step 5: git commit -m "perf(controlplane): gzip artifact blob inside the hashed bytes (~10-20x smaller)"

Task 9: Full verification + docs

Classification: standard Estimated implement time: ~5 min Parallelizable with: none (last)

Files:

  • Modify: CLAUDE.md — the "v3 OPC UA Address Space" / classifier-adjacent statements that now describe the old behavior (grep CLAUDE.md for ChangedRawTags and Rebuild); keep the edits surgical and truthful to the new contract (alarm-bearing deltas still rebuild).
  • Modify: docs/Uns.md + docs/Raw.md — one short paragraph each: reference add/remove and attribute edits are now surgical; what still rebuilds (alarm deltas, container renames, identity changes).

Steps: full dotnet build ZB.MOM.WW.OtOpcUa.slnx; run every non-DB test project (same list as plan 1 Task 12 + tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests); record skipped DB-backed suites in the commit message. Commit: docs: record surgical-deploy semantics (plan 2026-08-15).

Explicit non-goals

  • No surgical raw-container (folder/device/group) renames — still rebuild (no surgical folder-remove on the sink; pre-existing).
  • No alarm-bearing raw-tag surgical path — the WP4 notifier-rewire stays owned by the rebuild.
  • No central-side deploy-start dedup (AdminOperationsActor triple DB read) — separate follow-up.
  • No AnnounceAddedNodes extension to raw/UNS adds — pre-existing gap, noted in comments.
  • No cross-actor shared ParsedDeploymentArtifact — one parse per actor is the target; sharing a disposable across mailboxes isn't worth the ownership complexity.
  • No artifact schema changes — the JSON inside the (now compressed) blob is byte-identical.
Reference in New Issue View Git Blame Copy Permalink