T13b: the isolated otopcua-t13b pair with OpcUa:Redundancy:Mode absent, on origin/masterf826af8e(it containsc569f806) and A/B on c569f806^ (687256d6). Every check passes on both builds: - levels 250/240; - Good data on both nodes; - writes, Acknowledge, AddComment and alarm delivery on the Secondary; - graceful stop: the survivor reaches 250 259 ms after Leaving (pre-fix 270 ms); - crash: 250 at +43.6 s (pre-fix +42.8 s), with no demotion at auto-down; - rejoin at 240 with no failback. The role-decision changes are gated on IsWarm and do not run under Unset. Plan L5, the design's section 4.3 and the Warm table in Redundancy.md now say a Leaving/Exiting node reports ServiceLevel 0, not 90 (ComputeWarm returns 0 for any member that is not Up/Joining; T13 note A). Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
26 KiB
Warm non-transparent redundancy (Gitea #717): implementation plan
Status: APPROVED by the owner 2026-11-13 (#717), together with 2026-11-11-warm-redundancy-717-design.md
(referred to below as "the spec", with § references). The spec's §9 records the owner's answers: writes and Part 9
methods are refused BadInvalidState under Warm, promote-while-unreachable is on by default, the standby status is
BadOutOfService, there is no ConfigDb migration, and the bands are as drafted.
Every task ships dark: OpcUa:Redundancy:Mode defaults to Unset, so merging any task changes no deployed
behaviour.
Conventions:
- Wire tests use a real OPC Foundation client session against a loopback
OtOpcUaSdkServer, intests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests(the home ofDualEndpointTests,ConditionRefreshDeferralWireTestsandFolderConditionRefreshWireTests). - Actor tests go in
tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Testsandtests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests. - Every gate gets an allow leg as well as a deny leg (the vacuous-test rule), and a mutation check: revert the production line and confirm the test fails.
T0: SDK 2.0.1 re-proof spikes (throwaway, separately approved)
The 1.5.378.106 spikes do not carry over (spec §10). Write throwaway loopback fixtures against 2.0.1, outside
src/. No product edits. Each fixture is rerun independently and its evidence is recorded in
docs/plans/evidence/2026-11-xx-717-sdk2/.
- a. Metadata graph. Build it via
CreateMainNodeManagerFactory+AddBehaviourToPredefinedNodeAsync. Check Read/Browse/Translate ofi=2296(HasTypeDefinition 2039),i=3709, andi=11314for both Warm and None. The passivei=11314visit must resolve to the constructed instance. - b. Read projection coverage. Cover Read, the initial sample on create, the change fan-out after
ClearChangeMasks, ResendData, queue size 1 vs. N, an AggregateFilter item, and Sampling → Reporting. Decide betweenOnSimpleReadValueand aReadValueAttributeoverride, and record which async callback paths 2.0.1 takes. - c. Provenance. Stamp items through the 2.0.1
MonitoredItemFactory/MonitoredItemCreateDecision. Exercise ModifyMonitoredItems handle change, delete/reuse, TransferSubscriptions (IMonitoredItemTransferCoordinator), and a restore. - d. Publish/Republish. Override
PublishAsync/RepublishAsyncwith the 2.0.1 signatures (RequestLifetime). Cover the first-delivery choice, first delivery through Republish, an ack followed byBadMessageNotAvailable, and a cancelled or failed response. - e. Events. Exercise the stamp/deliver split under the #855 per-item folder path and the #854 Server-object cut. Run the promotion refresh on all three item kinds, and a client ConditionRefresh/2 while Standby.
Exit: a short results note per fixture, and an owner go/no-go. A "no" on (c) or (d) re-opens D4's mechanism. D4 itself stays an owner decision.
T0 run 2026-11-13: (a) and (b) pass; (c) and (d) are no-go as written. SDK 2.0.1 hands every Publish and Republish
a fresh clone of the cached message and pools the notification payloads, so provenance and the first-delivery choice
cannot be keyed by object. An alternative, keyed by (SubscriptionId, SequenceNumber) with a clone-surviving
provenance token, passed the same spike. The owner adopted it on 2026-11-13; spec §4.6 and T2/T7 below describe
it. Evidence: docs/plans/evidence/2026-11-13-717-sdk2/RESULTS.md.
T1: Configuration, validation, effective self URI
Files:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OpcUaApplicationHost.cs: options at:29-121. AddOpcUaRedundancyOptions Redundancy(Mode;PromoteWhilePrimaryUnreachable, defaulttrue, owner answer Q2).src/Server/ZB.MOM.WW.OtOpcUa.Host/Configuration/OpcUaApplicationHostOptionsValidator.cs.- A post-load check in
StartAsync(:218-250) against the effectiveApplicationConfiguration.ApplicationUri.
Rules (spec §3):
Hotand unknown values fail host start.Warmrequires at least one peer, all URIs distinct, self not listed as a peer, and thedriverrole.Nonerequires no peers.
Tests:
- A validator matrix.
- An XML
ApplicationConfigPathwhose URI differs from the options value, proving the effective URI is used. - A boot-banner log assertion.
T2: Standard redundancy metadata graph
Files:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/OtOpcUaSdkServer.cs: in the existingOnNodeManagerStartedAsyncoverride, install the T0a graph per mode throughIPredefinedNodeSubtypeReplacer.ReplacePredefinedInstanceSubtypeAsync(noCreateMainNodeManagerFactoryoverride is needed on 2.0.1).- Under
Warm, the initialServer.ServiceLevelis set to 1 in the same pass (spec §4.3).Unsetmust not touch the graph. UnderNonethe SDK's orphani=11314(T0a finding) is deleted so it is absent by NodeId too.
Tests (wire):
- Per mode, check the
i=3709value, thei=2296type definition, the presence or absence ofi=11314, its value order[effectiveSelf, peers…], and that it is read-only (a Write returnsBadNotWritable). ServerArray(i=2254) is unchanged; extendDualEndpointTests.- A Warm server's ServiceLevel reads 1 before the first publish.
- Security profiles and startup are unaffected under
Unset(golden read of today's values).
T3: Ordered redundancy snapshots (spec §4.2)
Files:
src/Core/ZB.MOM.WW.OtOpcUa.Commons/Messages/Redundancy/RedundancyStateChanged.cs(:9-11): add an optionalSnapshotRevision? Revision(additive).RedundancyStateActor.cs:- stamp the revision in
PublishIfDirty(:105-113); - set
IncarnationTickinPreStart; - read the host's
UpNumberfromCluster.SelfMember.
- stamp the revision in
- Subscribers: accept only a newer revision in
OpcUaPublishActor.HandleRedundancyStateChanged(:1494-1498) andDriverHostActor.OnRedundancyStateChanged(:2555). - An unrevisioned snapshot keeps today's handling for the existing gates.
Tests:
- Ordering unit tests: an older singleton incarnation's heartbeat after a newer snapshot is dropped; a same-process singleton restart advances.
- A real 2-node cluster handover test (pattern:
RedundancyPrimaryElectionTests). It holds back the old incarnation's last publish through the_broadcastOverrideseam (:111) and asserts the subscriber keeps the new Primary. - Serialization round-trip of the extended message, plus a mixed-version decode with the field absent.
- The pinned assumption: a removed node cannot rejoin without a new
ActorSystem.
T4: Warm role verdict and ServiceLevel bands (spec §4.1, §4.3)
Files:
- New
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Redundancy/WarmRoleAuthority.cs: a pure function of membership view, accepted snapshot,EndpointListeningandSelfMemberstatus. src/Core/ZB.MOM.WW.OtOpcUa.Cluster/Redundancy/ServiceLevelCalculator.cs: addComputeWarm(inputs, isActive).Computeitself is unchanged.OpcUaPublishActor.RecomputeServiceLevel(:1576-1636) branches on mode. It subscribes toMemberUp/Removedand to reachability (owner answer Q2) so the verdict re-derives on membership changes, not only on snapshots.
Tests:
- A truth table covering every row of §4.3 and the invariant max(Standby) < min(Active serving).
- Membership/snapshot disagreement gives Standby, plus the 20 s Warning.
- Boot with an unknown role gives Standby; a lone node goes Active after its own snapshot.
None/Unsetproduce byte-identical output to today'sServiceLevelCalculatorTests.- DB-less mode (#721 timer) is covered.
T5: Standby projection seam and transition sequencer (spec §4.7)
Files:
- New
src/Core/ZB.MOM.WW.OtOpcUa.Commons/OpcUa/IStandbyProjection.cs, with a deferred/late-bound adapter that mirrorsDeferredServiceLevelPublisherand retains the latest state across binding (the #719 lesson). - Bound in
src/Server/ZB.MOM.WW.OtOpcUa.Host/OpcUa/OtOpcUaServerHostedService.csnext to the ServiceLevel binding. OpcUaPublishActorruns promotion and demotion as one serialized handler, in the §4.7 step order.
Tests (actor, with a recording fake):
- The exact step order for both directions.
- A ServiceLevel publish failure halts a promotion before the Active level and retries; a demotion continues.
- A burst of alternating verdicts converges on the last one.
- Binding after the first transition replays the current state.
T6: Per-tag read projection (spec §4.4)
Files:
src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/EquipmentVariableState.cs: the hook decided in T0b, reading a node-manager-wide volatile projection flag. StoredValueis never modified.OtOpcUaNodeManagerimplementsIStandbyProjection.SetDataProjection.- Owner answer Q1: under Warm, the ordinary-tag write handler refuses a write on the Standby with
BadInvalidStatebefore it reaches the node-write gateway.Unset/Nonekeep the #5 optimistic path.
Tests (wire):
- Warm Standby: a Write returns
BadInvalidStateand the gateway is never called; on the Active node the same Write reaches the gateway; underUnseta node that is not Primary still takes today's optimistic path. - Standby Read of a raw tag, a UNS reference, a virtual tag and a Calculation tag returns
BadOutOfService/null. - A Standby Read of a condition child,
i=2267ori=3709stays Good. - A Read with an IndexRange on an array tag is projected; a DataEncoding request is projected.
- Acquisition continues: flip to Active and the latest value is read without any new driver publish.
- ACL on: a denied node is still
BadUserAccessDeniedand the projection never leaks existence.
T7: Ordinary-tag marker and first-delivery decision record (spec §4.6, D4)
The T0-adopted alternative (owner, 2026-11-13). The 1.5 object-keyed provenance and choice table are not used.
Files:
OtOpcUaNodeManager: replacem_monitoredItemManagerwith aMonitoredNodeMonitoredItemManagersubclass that re-implementsIMonitoredItemManager.CreateMonitoredItemand creates a custom sampled item for ordinary-tag (EquipmentVariableState)Valueitems only. ItsPublishruns base first and then setsMonitoredItemNotification.Messageto anOrdinaryTagMarker(ClientHandle, DataValue).OtOpcUaSdkServer.cs:PublishAsync/RepublishAsyncoverrides and a newFirstDeliveryRecordkeyed by(SubscriptionId, SequenceNumber), pruned byAvailableSequenceNumbersand bounded.- Wire into
OtOpcUaNodeManager.CreateMonitoredItems/OnMonitoredItemCreatedwithout disturbing ACL filtering or_liveItems.
Tests (wire):
- Queued Good samples, a ClientHandle change, then demotion: the first Publish carries
BadOutOfService, and a Republish returns the same projected payload. - Prebuilt sequences (
maxNotificationsPerPublish=1), then demotion: the next Publish is projected, and Republish returns the recorded projected choice. A Republish of a prebuilt, never-sent number isBadMessageNotAvailable. - Good delivered while Active, then demotion: Republish returns the original Good; after an ack it is
BadMessageNotAvailable. - A mixed subscription (ordinary tag +
i=2267+ condition-child item): only ordinary-tag items are rewritten. - The marker is validated: a notification whose marker names another handle or value is not rewritten (unit).
- SDK-field regression pin: the helper survives
CoreUtils.Cloneand is not encoded by the binary encoder. - The record is bounded, and pruned against
AvailableSequenceNumbers. - Concurrent Publish and role flip, by a stress loop with a deterministic barrier, not sleeps.
T8: Promotion and demotion of live data items
Files:
OtOpcUaNodeManager:IStandbyProjection.QueueCurrentToLiveItems(bool projected)iterates_liveItems. It reuses the QueueValue-with-ignoreFilterspattern (:7303), not detach orMonitoringMode.Disabled; the disabled-mode-drops-queue lesson applies. ACL-suppressed items are skipped.
Tests (wire):
- Demotion delivers exactly one
BadOutOfServiceper item, then silence while values keep changing internally. - Promotion delivers the current Good value without a driver publish.
- An ACL-stood-down item receives nothing either way.
- A DeadbandFilter item still receives the promotion value.
T9: Alarm event suppression and promotion refresh (spec §4.5, D3, D5)
Files (OtOpcUaNodeManager):
- Split
ReportConditionEvent(:1376) into stamp and deliver; deliver is gated by the projection flag. - Audit all callers (
:1254,:1341,:2842,:4659) andWireMethodTransitionReporting. IStandbyProjection.RefreshAllEventItems()runs the existingConditionRefresh(:6586) Server-object path andRefreshDirectItems(:6974) over every live event item, from_allEventsItemsand_eventItems.ReportToServerObject(:6060) is untouched.- Owner answer Q4: under Warm, every Part 9 condition method (native or scripted) is refused
BadInvalidStateon the Standby before it changes condition state.
Tests (wire, ACL off and on):
- Warm Standby: Acknowledge and a shelving method on a scripted condition return
BadInvalidStateand route no command; the Active node accepts them;Unsetkeeps today's scripted-method acceptance. - A new alarm on the Standby reaches no Server-object, folder or direct item.
- Model-change and audit events are still delivered on the Standby.
- A client ConditionRefresh and ConditionRefresh2 on the Standby return Start, retained, End (D5).
- Promotion gives Start, retained, End on all three item kinds, followed by new events.
- An ACL-denied condition is absent from the promotion refresh.
- A condition raised and cleared (Retain=false) while Standby is not replayed.
- An event reported while Active but deferred (#838, via
NotifierStall) is still delivered after demotion. - A Standby-stamped EventId is accepted by Acknowledge after promotion.
- The
RingEventQueueFactorypath is untouched (RingEventMonitoredItemQueueParityTestsstay green). - Re-run
EventQueueOverflowAclWireTestsandConditionMethodEventTests.
T10: Deploy-time mode cross-check and observability (spec §3, §7)
Files:
src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DeploymentArtifact.cs: read this node's clusterRedundancyModefrom theClustersarray.OpcUaPublishActorcompares it with the local mode at apply.src/Core/ZB.MOM.WW.OtOpcUa.Commons/Observability/OtOpcUaTelemetry.cs: add the six instruments.
Tests:
- A mismatch logs an Error and records the metric.
- Each counter fires on its path. Use a
MeterListenerfiltered by instrument instance, never by meter name alone (the #503 lesson).
T11: ScadaBridge prerequisite (cross-repo)
- Filed as ScadaBridge#138 (2026-11-13). A ScadaBridge issue (
type:feature,cross-repo,priority:p1, linked to #717). The DCL becomes ServiceLevel-aware: monitori=2267, and fail over when it is below 200 for a configurable window while the other endpoint reports higher. Also add a test for an unsolicited RefreshStart/End on existing alarm items (spec §5). - File the lmxopcua companion as
blocked+cross-repo. - The Warm flip in T13/rollout is gated on that issue being deployed.
T12: Documentation and index
docs/Redundancy.md: rewrite the:9block, add a Warm contract section, update the tiers table. Remove the generic "standby / no device contact / exactly-once" wording that contradicts acquisition and at-least-once (c7794).docs/OpcUaServer.md:70,docs/Configuration.md(the new keys), anddocs/security.md(ns=0 stays ungated; projection order vs. ACL)..claude/rules/cluster-topology.md§Redundancy.docs/plans/phase-6-3-redundancy-interop-plan.mdA2/A4/A5b: under Warm, A2 expects 90 and A4/A5b become testable. Adddocs/v2/redundancy-interop-playbook.md.- Umbrella: update the OtOpcUa row and
sister-project-history/otopcua.mdin../scadaprojon itsmain, pushed (rootCLAUDE.mdprocedure).
T13: Live gate on docker-dev (central pair, :4840 / :4841)
Overlay: docker-dev/docker-compose.warm.yml, central-1/2 only. It sets OpcUa__Redundancy__Mode=Warm. The base
compose file already gives each central its own OpcUa__ApplicationUri=urn:OtOpcUa:central-1|2 (matching
seed-clusters.sql) and its partner in OpcUa__PeerApplicationUris__0 (lmxopcua#801, after spec §2 was written). Login is disabled on the rig, so drive
everything yourself. Use otopcua-cli from src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI against both ports.
| Leg | Procedure | Pass |
|---|---|---|
| L1 metadata | redundancy on :4840 and :4841 |
RedundancySupport=Warm. ServerUriArray is self first and differs per node. Same URIs in both. |
| L2 bands | read -n i=2267 on both |
Primary 250, Standby 90 |
| L3 standby data | read and subscribe of a raw tag and its UNS reference on the Standby |
BadOutOfService/null; the Primary shows Good at the same moment |
| L4 standby alarms | alarms on both. Trip a scripted alarm via its source tag. |
Event on the Primary only. A ConditionRefresh on the Standby lists it as retained (D5). |
| L5 manual failover | AdminUI manual failover (docs/Redundancy.md §Manual failover). Keep the CLI subscriptions open on both. |
Old Primary: SL drops to 0 (a Leaving/Exiting member is not Up/Joining, so ServiceLevelCalculator.ComputeWarm returns 0; T13 note A), then BadOutOfService, with no Good delivered after the drop. New Primary: Good values, then RefreshStart / retained / RefreshEnd, then SL 250, in that order (timestamps). |
| L6 crash | docker kill central-<primary> |
The survivor goes Active once the Primary is unreachable (PromoteWhilePrimaryUnreachable, owner answer Q2). Record the measured gap. |
| L7 rejoin | Restart the killed node | It comes back Standby (90) and never shows 250 or 255. The boot read is 1 or 90. |
| L8 standalone | site-b-2 (or a one-node overlay) with Mode=None |
RedundancySupport=None, no i=11314, data Good |
| L9 Unset control | Another node left on Unset |
Byte-identical to pre-change: SL 240/250, SDK-default i=3709 |
| L10 writes/history | Write and HistoryRead on the Standby | The write is refused BadInvalidState (owner answer Q1); HistoryRead is Good. |
| L11 transfer | A UserName session's subscription on the Standby is taken over by a second session of the same identity (TransferSubscriptions), then the node is demoted/promoted | The transferred items keep their marker; a demotion still projects; a promotion delivers Good. |
| L12 durable restore | A durable subscription, if the rig's SDK configuration enables them, across a node restart | The restored items are created through the custom item (projected on the Standby). If durable subscriptions are not enabled on the rig, record that and leave the leg open. |
Record evidence in docs/plans/2026-11-xx-warm-redundancy-717-live-gate.md. Do not mark G8/G9 (#615); they
need CTT and a production client (Ignition 8.3 / ScadaBridge after T11).
Rollout
- Merge T1–T12 dark (
Unsetdefault). The full CI floor applies. - Deploy the ScadaBridge T11 change everywhere ScadaBridge reads OtOpcUa.
- Upgrade both nodes of a pair to a revision-capable build (T3) before flipping either one.
- Flip both nodes of one docker-dev pair, then run T13. After that, flip one non-production site pair.
- Production is a separate, owner-approved change per pair. Flip the Secondary first, then the Primary, in one window.
A mixed window (one Warm, one Unset) is tolerable: the Unset node serves Good at 240/250. But do not leave it
that way, because ServerUriArray then exists on only one node.
Rollback
Set OpcUa:Redundancy:Mode=Unset on both nodes and restart them. The setting is startup-static, so there is
nothing persistent to undo: no schema changes and no artifact changes. The Revision field stays and is harmless,
because unrevisioned handling is preserved. Rolling back ScadaBridge is independent; its ServiceLevel-aware
failover also works against Unset servers.
Implementation status (2026-11-13)
T1–T10 and T12 are implemented on master, default-off (OpcUa:Redundancy:Mode defaults to Unset, pinned by
WarmRedundancyWireTests.Unset_is_todays_server_exactly and OpcUaPublishActorWarmTests.Unset_never_drives_the_projection_and_keeps_todays_levels).
T11 is ScadaBridge#138 (separate). T13 ran 2026-11-13 on an isolated two-node pair (not the shared docker-dev rig):
every applicable leg passes after c569f806, which fixes two crash-failover defects the gate found. The survivor no
longer demotes when auto-down removes the Primary, and the verdict reads its own cluster events instead of the lagging
read view. Evidence: docs/plans/evidence/2026-11-13-717-t13/RESULTS.md. L8 (None) was not run, and L12 is not
applicable (durable subscriptions are off). G8/G9 (#615) stay open.
Test homes: WarmStandbyUnitTests (OpcUaServer.Tests: marker, rewrite, decision record, the SDK-field regression pin,
option rules), WarmRedundancyWireTests (OpcUaServer.IntegrationTests: metadata per mode, read projection, write and
Part 9 refusal, the three T0 demotion timings, live-item queue, alarm suppression/refresh, effective-URI XML load),
WarmRedundancyRevisionTests + WarmServiceLevelBandTests (ControlPlane.Tests), WarmRoleAuthorityTests +
OpcUaPublishActorWarmTests (Runtime.Tests), OpcUaRedundancyValidationTests (Host.IntegrationTests). The deferred
depth below adds WarmSingletonHandoverTests (ControlPlane.Tests), WarmRedundancyMetricsTests (Runtime.Tests),
WarmStandbyCloneMatrixTests (OpcUaServer.Tests), WarmRedundancyWireTests.Deferred.cs and WarmRedundancyPooledWireTests
(OpcUaServer.IntegrationTests).
Deferred coverage added (2026-11-13, in-process, no docker). Each test was mutation-checked against a deliberately broken implementation and failed. No product bug was found.
- T3:
WarmSingletonHandoverTests(ControlPlane.Tests). A real 2-node cluster runs the redundancy singleton underClusterSingletonManageron both nodes. The old incarnation's last publish is held back through the broadcast seam, its node is downed, and the singleton hands over. The held publish is then delivered to a real WarmOpcUaPublishActoron the survivor. That publish is the older revision but has the higher sequence. The survivor stays Active. The second test pins that a removed node cannot rejoin from its old ActorSystem. A new ActorSystem on the same address rejoins as a younger member, with a higher up-number. Mutations killed:- the gate accepts older revisions;
- the revision drops the up-number and incarnation (sequence only);
- for the pin, skipping the Down (a harness negative control).
- T6:
A_DataEncoding_read_and_item_are_projected_…(wire). On the Standby, a Binary or XML DataEncoding read of a structured (EUInformation) tag is projected. An encoding a scalar cannot take still returns the base read's ownBadDataEncodingUnsupported. An item created withDefaultXmlgets the projected value on the Standby, and on promotion it receives the value read through its own encoding (TypeIdi=888). Mutations killed:- DataEncoding reads are not projected;
- the promotion queue ignores the item's DataEncoding.
- T7:
An_item_deleted_between_queue_and_publish_…(wire). The deleted item's queued samples never surface. Its sibling is still first-delivered projected, and a Republish gives the same result.WarmRedundancyPooledWireTests.A_client_handle_reused_after_delete_…(wire, non-parallel collection). A marked payload goes back through SDK 2.0.1's process-wide payload pool, and its ClientHandle is reused by ani=2267item. That item stays Good. The test first asserts that the pool really hands out a marked payload.WarmStandbyCloneMatrixTests(unit). One delivered clone carries two DataChangeNotifications (one with DiagnosticInfos), an EventNotificationList and a StatusChangeNotification. Only marked values are rewritten. Diagnostics, events and status pass through in order. Neither the clone nor the cached message is mutated, and the result encodes. Two clones project to identical bytes. A recycled payload's stale marker, rented through the real pool with the same handle, never validates.- Mutations killed:
- the marker is not validated (wire and unit);
- the marker is not set;
- DiagnosticInfos are dropped;
- non-data payloads are dropped;
- the rewrite happens in place;
- only the first DataChangeNotification is rewritten.
- T8:
An_ACL_stood_down_item_receives_nothing_…(wire). An item thatRevalidateMonitoredItemsstood down gets nothing from a demotion or a promotion (QueueCurrentToLiveItemsreturns 1, not 2). Its permitted sibling gets both. Once the grant is restored, the next demotion reaches the item. Mutation killed: stood-down items are not skipped. - T10:
WarmRedundancyMetricsTests(Runtime.Tests, non-parallel collection) checks four things:warm_transition, with direction and reason (agreed/snapshot), and no count on a same-role snapshot;- the
warm_rolegauge, -1, then 0, then 1, then 0; snapshot_rejectedat the publish actor forolder_revisionandunrevisioned;- at
DriverHostActor, a stale snapshot is dropped under Warm and counted, and its role view does not move. The Unset control does move. This is the first actor test of the driver host's ordering gate.
WarmRedundancyPooledWireTestschecksstandby.projected_notifications: it equals the rewritten count, counts again on Republish, and stays 0 while Active. It also checksstandby.suppressed_alarm_events: one per withheld transition, none once Active.- Every listener selects by instrument instance.
- Mutations killed: each counter's
Addwas removed, the direction tag inverted, the gauge never set, and the driver-host gate ignored.
Still open (live-only, T13): the durable restore, TransferSubscriptions under one identity, and the rig legs.