R1: NativeAlarmProjector treats Raise-on-Active as a re-statement only when the message matches what
the condition shows; a different alarm appearing during a respawn un-acks and writes its alerts row.
FOCAS condition messages are a deterministic function of the matched set (tested across fresh driver
instances and reordered lists); Runtime E2E now covers both the same alarm and a new one during a
respawn. Residual (source message determinism) documented in AlarmTracking.md.
R2: a non-OK cnc_rdalmmsg2 return is thrown as FocasWireException{Rc} and classified as a CNC
refusal: conditions get a quality-only BadNotSupported once, one Warning, polling continues, and
driver health is untouched (no 90 s watchdog reconnect loop on a production control). Only transport
faults degrade (falsification control in the test).
R3: alarm-path health is tracked apart from the value path and combined in GetHealth: worst state,
later LastSuccessfulRead. A good value read can no longer mask a failing alarm read (no per-poll
/hosts flip) and a good alarm read cannot mask a value Degraded.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
MAJOR-1: a deploy that changes a driver's config respawns it, and the fresh instance's empty told
record re-sent Raise for every still-active alarm, un-acking acknowledged conditions and writing a
duplicate "Activated" alerts row. NativeAlarmProjector now treats Raise on an already-active
condition as a re-statement (keeps Acked; ForwardNativeAlarm skips the alerts row); only Retrigger
(or inactive->active) un-acks. Sources whose upstream ack state is authoritative send Retrigger for
an unacked re-statement: OpcUaClient on first sight (no prior told), Galaxy for an unacked active
snapshot entry. Runtime E2E drives a FRESH FOCAS driver after a redeploy and asserts the ack survives.
MAJOR-2: a failed FOCAS alarm read annotates each subscribed condition BadNoCommunication once per
outage (#626 quality-only path) with one Warning (MINOR-6), lifted to Good on recovery; failed alarm
reads degrade driver health with LastSuccessfulRead frozen (successful ones advance it), so an
alarm-only driver trips the #527 comms-loss watchdog.
MINOR-3 CLAUDE.md #630 bullet; MINOR-4 split line; MINOR-5 a non-empty cnc_rdalmmsg2 payload that
fits neither stride throws; MINOR-7 value-tag + parsed-address tables are built and swapped, never
cleared in place under the retained poll engine. Docs corrected (FOCAS.md, AlarmTracking.md, CLAUDE.md).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
An ALARM: address (*, <Type>, <Type>/<n>, <Type>/<lo>-<hi>, T<code>, OTHER) on a raw tag that
also carries the cross-driver alarm object is ONE condition with ConditionId == RawPath and a
Kind (Raise / Retrigger / Clear), so DriverHostActor.ForwardNativeAlarm routes it and
NativeAlarmProjector activates it. Fixes all seven triage defects:
1. SubscribeAlarmsAsync treated the host's RawPaths as device hosts -> nothing polled.
2. Decorated "{host}#{Type}:{Number}" ConditionId -> every event dropped.
3. No Kind -> projector kept prior state.
4. WireFocasClient.ReadAlarmsAsync answered [] on failure -> mass clear + un-acked re-raise.
It now throws; a failed device read keeps every condition's state.
5. Acks lost on reconnect -> told-record dedup (MTConnect #626 pattern) held for the driver's
lifetime; same-set re-subscribe / re-init re-raise nothing; changed set prunes leavers only.
6. Subscribe threw while AlarmProjection.Enabled=false -> the tag is the opt-in; flag is a no-op.
7. Alarm tags could not be authored -> ALARM: grammar (FocasAlarmSelector, Contracts) backs the
editor, the address picker, the deploy-time inspector and driver init.
Severity: authored alarm.severity -> exact OpcUaSeverity; else the 16i-era type bucket. The type
table stays the only one (moved to Contracts, documented 16i-era; 30i codes not invented; raw
T<code> selector + raw code in every message). Read-only: ack is server-side. Also clears the
value table on a re-initialise (#630 gap that let a removed/moved tag keep reading).
E2E: Runtime.Tests FocasNativeAlarmEndToEndTests drives the real driver through the real
DriverHostActor and asserts the condition goes Active then inactive.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
MTConnectConditionAlarmWireTests drives a scripted Agent's Fault/Normal through the real
driver, NativeAlarmProjector and sink to a client event subscription on the raw device
folder. docs/drivers/MTConnect.md replaces 'CONDITION v1: plain String' with the alarm
mapping, opt-in authoring, multi-condition, ack, replay and comms-loss behaviour, and a
live-check recipe; EVENT vocabularies as enumerations marked won't-fix. AlarmTracking.md
lists the MTConnect source and the two new AlarmEventArgs fields.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
New() ran FindNodeInAddressSpace under Lock, taking other node managers' locks (lock-order
risk); the three AlarmConditionType_ShelvingState_* ids are now a static set. Adds wire tests:
TimedShelve/Unshelve by the condition's own instance method ids (object = condition or its
ShelvingState) succeed and report, and a ShelvedStateMachineType_* id call is pinned as
BadMethodInvalid with a positive control.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Create(assignNodeIds: true) asked CustomNodeManager2.New, whose default returns the
node's current id, so every condition child kept the type's ns=0 instance-declaration id
(or an SDK-generated ns=0 number): translate of a child path returned BadNoMatch and a
Read of a browsed child id was BadNodeIdUnknown or served from the TYPE (UnshelveTime 0
on a condition timed-shelved for a minute), bypassing the data-plane ACL.
OtOpcUaNodeManager.New now derives {conditionId}/{browsePath} in the condition's
namespace, so the ACL scope of a child is its condition's scope plus the path. The
ShelvingState methods adopt their original AlarmConditionType_ShelvingState_* id as
MethodDeclarationId so clients calling them by that id still resolve. Condition removal
and rebuild forget the whole PredefinedNodes subtree so a removed condition's children
stop answering.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
MINOR-3: new DriverHostActorPrimaryRoleSourceTests case drives the
deny flip with a real ClusterEvent.MemberUp on the EventStream (how the
cluster's domain-event publisher reaches a Cluster.Subscribe subscriber)
instead of a direct RepublishPrimaryVerdict tell; it fails when the
Cluster.Subscribe(IMemberEvent) call is deleted.
MINOR-2: the published verdict MIRRORS the live gate; before this node has
joined its member count is 0 and both allow (seconds; up to the bootstrap
guard's 25 s probe), so the Sparkplug primary-host claim can be held on
both nodes in that window (native ack is unaffected - view and route
agree). Behaviour deliberately unchanged: denying until self is Up would
change the live gate too and delay writes/acks on a single-node cluster's
own join. Comments + AlarmTracking.md / Redundancy.md / Mqtt.md corrected.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Review R1 M2: since #597 the SDK timer fires at real expiry, and OnTimedUnshelve routed an operator-style
Unshelve by 'system' from both pair nodes, pre-empting the engine's AutoUnshelve. The engine owns expiry,
so the callback now routes nothing. Integration test: an engine stand-in that acts on a routed Unshelve and
expires late, plus a wrapped OnTimedUnshelve (positive control: the SDK timer fired).
Review R1 m1: Part9StateMachine.ApplyAcknowledge records a commented re-ack as AddComment (persisted and
replicated) instead of a NoOp; an uncommented re-ack is still a no-op.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
- Native-condition Acknowledge on a non-Primary node returns BadInvalidState (nothing
routed/applied/reported) via NativeAlarmAckPrimaryGate, fed from
IDriverPrimaryRoleSource by OtOpcUaServerHostedService.
- A re-ack carrying a comment reports one comment event (Acked/Confirmed unchanged).
- TimedShelve applies its requested duration and the scripted projection carries
UnshelveAtUtc, so ShelvingState/UnshelveTime counts down instead of ~24.8 days.
- Native re-ack still routed to the driver (deliberate: documented retry path), pinned.
- AdminUI /alerts client ack/shelve path test; docs/AlarmTracking.md decisions incl.
accepted SDK EventsMonitored residual (item 6).
- Also #599/03/S20: RemoveEquipmentSubtree severs inverse Organizes edges.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Review of 167a48bf (FIX-FIRST): making native Confirmed/Shelving method-owned produced over-the-wire
regressions — a native TimedShelve never expired, a native OneShotShelve was silently undone by
SetActiveState, an OPC UA ack left a native alarm Unconfirmed forever (ScadaBridge maps ConfirmedState),
and any local state was per-node, never reaching the redundant peer.
- Reverted: the driver projection owns native Confirmed/Shelving again.
- Native Confirm / OneShotShelve / TimedShelve / Unshelve return BadNotSupported with no event, as native
Enable/Disable already did. Scripted alarms keep every method.
- A native Acknowledge re-asserts ConfirmedState=true and the projection's Retain rule (Active || !Acked)
before its single report. The native condition starts Confirmed.
- The method path reports only when the state actually moved (before/after). A re-ack is silent and
restores the Confirmed/Retain that the SDK flips, so a scripted node never drifts from its engine.
AddComment always reports.
- Docs: native alarms support Acknowledge/AddComment only; the Secondary-node native-ack flap is documented.
Tests: refused native methods (5), native ack stays Confirmed through echo/clear/reactivate, scripted
TimedShelve expires with exactly one unshelve event, re-ack silent (native + scripted). Each fix falsified.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
#587 — a client Acknowledge/Confirm/AddComment/Shelve/Enable/Disable emitted no condition event: the SDK's
ReportStateChange is gated on AutoReportStateChanges (never set), and the engine re-projection found the node
already in the new state and was suppressed by the delta-gate. WireMethodTransitionReporting now wraps each
method's SDK handler and, on Good, reports exactly once via ReportConditionEvent, under Lock so the routed
re-projection cannot race in ahead of the SDK applying the state. Independent of the SDK's imprecise
AreEventsMonitored bookkeeping. The shelve methods' state is applied by the wrapper (the SDK never applies
it). Native conditions' Confirmed/Shelving are method-owned (NativeAlarmProjector's constants would re-report
every ack and undo every shelve). MaxTimeShelved set to int.MaxValue ms at materialise — Create() left it 0
and every TimedShelve failed BadShelvingTimeOutOfRange. Wrong "SDK auto-fires (E2)" comments fixed.
#588 — with ACL on, GeneralModelChange and AuditWriteUpdate events no longer go to Server.ReportEvent
(ungated to every i=2253 item): ReportToServerObject delivers them per item, a model change narrowed to the
Affected nodes the session may Browse (dropped when none), an audit write only with Read on its source plus
the SDK audit/encryption guard. Flag off: unchanged.
#586 review: materialise raises only the new condition's monitored unit (was a full O(N) reconcile per
condition ⇒ O(N²) deploys under Lock).
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
Server-object (all-events) items bypassed the ACL entirely: the SDK attached them to every root
notifier's fan-out via SubscribeToAllEvents, and a root notifier with no direct subscriber forwarded
to the core Server object, which this node manager never sees.
- SubscribeToAllEvents override records the item and attaches it to nothing.
- Each condition's own OnReportEvent (fires once per event) queues the event to each recorded item
whose session holds Subscribe on the condition (RawPath) or any notifier folder it reports through
-- the per-folder path's question; otherwise mirrors MonitoredNode2.OnReportEvent.
- The root-notifier forward to the core Server object (OnReportEvent override) is dropped.
- ConditionRefresh on an i=2253 item is filtered the same way (once per condition).
- Revalidation is implicit: every event re-asks the authorizer (per-generation cache, key expiry and
revocation re-checked per call). Inert with Security:Acl:Enabled=false.
Over-the-wire tests: no-grant vs granted (positive control), created-before-deploy, revoke/restore by
deployment, key expiry with no tick, refresh, and flag-off. Falsified: disabling the filter, or
restoring the forward, turns the enforcement tests red.
Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
DriverInstanceActor.DisconnectObserved had no production sender. The only paths
into Reconnecting were a failed *initial* connect and an operator's Force
Reconnect, so a driver that connected and then lost its device stayed Connected
indefinitely: publishing Bad for every tag, never re-initialising, and NOT
recovering when the device returned. PublishHealthSnapshot read GetHealth() for
reporting only and never acted on a driver reporting itself unhealthy.
Both comms-loss quality paths -- #477's native-condition annotation and #481's
scripted-alarm input fan-out -- hang off ConnectivityChanged, so both were
correctly wired to a signal that never fired.
The Connected health tick now evaluates a watchdog and self-sends
DisconnectObserved when the driver reports a non-Healthy state AND its
LastSuccessfulRead has been frozen for DefaultCommsLossTimeout (90s).
The conjunction is the design, and the issue's own proposal -- "non-Healthy for
N consecutive polls" -- is wrong. Degraded means *connected but some data is
bad*; several drivers write health per tag inside the read loop
(ModbusDriver.ReadAsync), so one unreadable address latches Degraded forever
against a perfectly reachable device, and a state-only watchdog would reconnect
that driver every 90s indefinitely -- worse than the bug. LastSuccessfulRead is
the honest evidence: preserved across Degraded/Faulted writes, advanced by any
success. The state check remains the second conjunct so an idle Healthy driver
with nothing subscribed is not declared dead for not reading.
Verified across all 12 drivers that a sustained failure does reach a non-Healthy
state -- nine poll drivers via Degraded, MQTT via live connection state, Galaxy
via its reconnect supervisor. CalculationDriver reports Healthy while running
and so can never trip, which is correct: it has no transport.
Also: the Reconnecting retry now calls ShutdownAsync before InitializeAsync.
ModbusDriver (_transport) and MqttDriver (_connection) overwrite an
IAsyncDisposable field inside InitializeAsync without disposing the previous
one, so re-initialising without a teardown leaks a socket per attempt -- latent
while only a manual Force Reconnect reached this path, but the watchdog makes it
automatic and unbounded (~360 leaks for an hour-long outage at the 10s retry).
Tests: 8 new in DriverInstanceActorCommsLossWatchdogTests, each absence
assertion carrying a positive control (a proven count of processed health-poll
ticks). Falsified five ways -- dropping either conjunct reds exactly the test
written for it, and removing the teardown, the null baseline, or the watchdog
call each reds its own guard.
Live gate PASSED on docker-dev (Modbus + otopcua-pymodbus-standard), run twice:
- fixture stopped -> watchdog fires at ~108s, "declaring comms lost"
- fan-out observed at Debug: "pushed BadNoCommunication for 2 mux ref(s)"
(#481 Layer 4) and the #477 native-condition annotation on a driver
owning conditions
- fixture restarted -> reconnected in ~8s with NO node restart; both the raw
(ns=2) and UNS (ns=3) NodeIds read Good with fresh timestamps
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
The docker-dev gate for #481 could not close on its headline scenario, and the
reason is worth recording next to the feature rather than only on the tracker:
DriverInstanceActor.DisconnectObserved has no production sender, so a driver
whose device dies mid-run is never detected. Filed as #527 with the live
evidence (Modbus stayed Connected >10 min publishing Bad, and did not recover
when its fixture returned until the node was restarted).
What the gate DID establish: the change deploys and runs (scripted alarm
materialised on central-1 against the seeded Modbus tag), and the #478 Layer 3
path does carry Bad into scripted input quality for Modbus, because that driver
publishes Bad per tag rather than going silent.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Layer 4 of the condition-Quality chain (#477 -> #478). #478 gave a scripted
condition the worst quality of its input tags, but only for inputs whose driver
PUBLISHES a Bad data change. A poll driver whose device goes unreachable
publishes nothing — it raises ConnectivityChanged(false) and goes silent — so
the engine held its last Good value, WorstInputStatus stayed Good, and worse,
AreInputsReady stayed true, so the predicate went on evaluating a stale value
after the source had died.
`DriverHostActor.FanCommsLossToDependencyMux` pushes a synthetic
BadNoCommunication with a null value into the dependency mux for every value ref
the disconnected driver owns.
The issue asked for a driver -> all-tag-refs map to be built for this. It
already exists: `_nodeIdByDriverRef`, keyed (DriverInstanceId, RawPath) and
rebuilt from the composition on every apply. Alarm-bearing tags live in the
separate `_alarmNodeIdByDriverRef` and are deliberately excluded — a native
condition is not a mux value and takes the Part 9 quality path instead.
Nothing is pushed on reconnect, deliberately: publishing Good would assert a
value quality this actor has not observed. Recovery rides the driver's own
republish, and the mechanism was verified rather than assumed — the
Connected -> Reconnecting edge calls DetachSubscription, which nulls the
subscription handle, so the reconnect's ResubscribeDesired builds a FRESH
subscription whose PollGroupEngine initial push runs with forceRaise: true.
Without that detach the poll engine would compare against its retained
LastValues, see an unchanged value, raise nothing, and strand these refs Bad
forever — the value-dedup trap that bit the v3 Batch 4 virtual-tag reassert.
Closes#478 Finding 3, which the above depends on. WorstInputStatus skipped
every input whose VALUE was null as "no data yet" — and Bad-with-no-value is
exactly the shape a comms-lost driver produces, so the skip was hiding the case
that matters most; without this half the fan-out above would be silently inert.
The upstream source now stamps a never-pushed path BadWaitingForInitialData —
the honest Part 4 code for that state — and the engine skips that specific code
rather than every null value.
Deleting the null-skip outright is the obvious fix and is wrong in the other
direction: it makes every not-yet-published input drag its condition Bad at
deploy, flooding the quality path with load-time annotations, which is what the
skip existed to prevent. Both mistakes are pinned by a test pair that goes red
under the opposite error, and both were falsified by reverting the production
code:
old null-skip restored -> A_pushed_Bad_carrying_no_value RED
null-skip deleted outright -> An_input_that_was_never_pushed RED
fan-out call removed -> the three DriverHostActor tests RED (1 of 7)
The never-pushed test was vacuous on first writing and is worth recording: the
quality bucket only emits on CHANGE, so the load-time evaluation is the one that
flashes Bad, and subscribing to OnEvent after LoadAsync made the assertion
unfalsifiable — it passed against the broken behaviour. It now subscribes first.
`FakeUpstream.ReadTag` returned 0x80340000 (BadNotConnected) for an unset path,
a different Bad code from production's placeholder. Harmless while any Bad
behaved alike; load-bearing now, so it is aligned and pinned by a reflection
guard asserting the engine's private constant agrees with what the production
source stamps.
Behaviour change to weigh, flagged in the issue and confirmed: virtual tags
computed from a comms-lost input now go Bad too. That follows from using the
shared mux and reads as correct, but it is broader than scripted alarms.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
Build: 0 errors solution-wide, and 0 warnings from every project this branch
touches. The ~816 solution-wide warnings are pre-existing xUnit1051 /
OTOPCUA0001 / CS86xx in untouched driver + client test projects.
Tests: full solution run compared against a full run on a detached worktree at
the pre-branch baseline 2e46d054. The two failure SETS are identical -- all 13
tests, same names, zero new failures. Net +26 tests: +3 Core.AlarmHistorian
(drain gate), +6 Runtime (role view), +17 Host.IntegrationTests (migrator +
convergence). Set comparison rather than counts, because the suite carries
standing environment- and load-dependent failures a count would hide.
The greps found real drift Task 6 missed -- eight live sites still naming the
deleted SqliteStoreAndForwardSink, including the AdminUI /alarms/historian
panel text, which is user-visible, and a <see cref> in HistorianAdapterActor
that resolved to nothing without warning. All repointed at
LocalDbStoreAndForwardSink; CLAUDE.md's alarm-history paragraph now also
records the LocalDb buffer and the primary-gated drain, and drops DatabasePath
from the knob list. docs/AlarmTracking.md still promised an
AlarmHistorianOptions.Validate() startup warning for a relative DatabasePath
and an empty SharedSecret; both branches are gone, so it now says so.
Code references to AlarmHistorian:DatabasePath reduce to exactly two intentional
ones: AlarmSfLegacyMigrator.LegacyPathKey and its test. No `new SqliteConnection`
remains anywhere in Core.AlarmHistorian.
Recon doc gains the durable verification record: guard-deletion evidence for
both vacuous passes, the two exact-set replicated-table pins (both assert set
equality, so an added or a dropped registration fails), and the baseline test
comparison with a per-failure account of why each of the 13 is not this
branch's.
Stops here per the plan. Task 8's live gate needs explicit go-ahead; nothing on
this branch is to be merged.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Enables the alarm store-and-forward sink on all four driver nodes of the
docker-dev rig -- the replicating site-a pair and the default-OFF site-b pin
-- so the live gate has a real buffer to watch converge, and can see that
site-b's sink works as a plain node-local queue with no peer traffic.
Two rig details worth stating rather than rediscovering. The endpoint is
deliberately unresolvable: there is no HistorianGateway here, so every drain
attempt fails, which is exactly the historian-outage state the buffer exists
for. But it still has to be a syntactically valid absolute http(s) URI or the
host refuses to start, because ServerHistorianOptionsValidator is
consumer-gated -- AlarmHistorian:Enabled=true makes the endpoint required
even while ServerHistorian:Enabled=false. And MaxAttempts is raised far above
the production default of 10, which against a permanently unreachable gateway
would dead-letter the entire queue about five minutes in, turning a buffering
test into a dead-letter test.
Docs record what an operator now has to know: that a rising queue depth on a
Secondary is correct and on BOTH nodes is not (that shape is a redundancy
snapshot naming neither node -- check node identity before suspecting the
sink), that delivery is at-least-once across a failover by design with no
dedup layer, and that AlarmHistorian:DatabasePath is removed but should be
left in place through the upgrade because the migrator still reads it to find
the file to copy.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Post-implementation review (HIGH finding) noted #478's mux-delivered
input-quality path does not cover a driver comms-loss: a poll driver
(Modbus/S7) whose device goes unreachable emits only ConnectivityChanged and
goes silent on the value feed, so a scripted alarm keeps the last Good value.
The code as shipped faithfully implements #478's written scope (worst of input
tags' qualities via the dependency mux). The comms-loss bridge for scripted
alarms (symmetric of native #477-L2, plus the null-value/cold-start asymmetry
and its VT-quality ripple) is tracked as #481. Docs updated in
AlarmTracking.md + the design doc.
Layer 3 of #477: a scripted alarm's condition Quality now reflects the WORST
quality across its input tags, mirroring the native OT semantic (#477 L2).
Plumbing (quality was silently discarded twice on the live path):
- VirtualTagActor.DependencyValueChanged gains Quality (defaulted Good); the
DependencyMuxActor forwards the published AttributeValuePublished.Quality it
already carried; ScriptedAlarmHostActor.OnDependencyChanged pushes the real
quality into the engine (was hardcoded 0u/Good).
Engine (Core.ScriptedAlarms):
- ScriptedAlarmEngine computes worst-of-input quality each eval (skipping
not-yet-published inputs, which are a readiness concern, not a quality signal)
and carries it on ScriptedAlarmEvent.WorstInputStatusCode.
- A real transition carries the current worst quality so ToSnapshot's full
snapshot doesn't clobber quality back to Good (e.g. transition while Uncertain).
- A Bad input freezes the condition (no transition), like a comms-lost native
driver; a quality-bucket change with no transition emits the new
EmissionKind.QualityChanged, routed to the existing #477-L2
AlarmQualityUpdate -> WriteAlarmQuality node path (quality only, no /alerts
row, no historian write). ScriptedAlarmSource skips QualityChanged so it never
fabricates a phantom IAlarmSource event.
Host: ToSnapshot maps WorstInputStatusCode -> OpcUaQuality; OnEngineEmission
routes QualityChanged out of band.
Tests (TDD, RED-first): engine worst-carry + Bad/restore QualityChanged +
unchanged-bucket-no-emit; source swallows QualityChanged; mux forwards quality;
host Bad-dep -> AlarmQualityUpdate(no alerts) + transition snapshot carries worst.
Docs: AlarmTracking.md Layer-3 section + design doc.
Closes#478
Part 9 ConditionType.Quality was never assigned; default(StatusCode)==Good
so every native + scripted condition reported Good unconditionally — a
comms-lost device still showed a healthy, inactive, Good condition (a
wrong-VALUE bug, distinct from the null-value #473/#475). Clients (and HMIs
bucketing on IsGood) could not tell "genuinely inactive" from "lost contact".
Layer 1 — make Quality a real, plumbed field:
- AlarmConditionSnapshot gains OpcUaQuality Quality (default Good).
- MaterialiseAlarmCondition sets it (native BadWaitingForInitialData, scripted Good).
- WriteAlarmCondition projects snapshot.Quality; the delta-gate gains a Quality
member so a quality-bucket change fires a Part 9 event.
Layer 2 — drive native quality from driver connectivity (a comms-lost driver
emits no alarm transitions, and an alarm-bearing raw tag has no value variable,
so quality can't come from either existing channel):
- DriverInstanceActor Tells parent ConnectivityChanged on Connected/Reconnecting.
- DriverHostActor fans it to every native condition the driver owns as
OpcUaPublishActor.AlarmQualityUpdate (Good on connect, Bad on disconnect).
- New dedicated IOpcUaAddressSpaceSink.WriteAlarmQuality sets ONLY Quality and
fires only on a bucket change — never touches Active/Acked/Retain (an active
alarm that loses comms stays active). Not a full-snapshot re-projection, so it
can't clobber severity/message and works for a never-fired condition.
Forwarded through DeferredAddressSpaceSink (F10b trap; auto-verified by the
reflection forwarding guard). Ungated by redundancy role; no /alerts row.
Scripted conditions stay Good; worst-of-input quality deferred to #478 (Layer 3).
Tests: node-level (materialise/project/no-clobber/unknown-node no-op),
NativeAlarmProjector, DriverInstanceActor connectivity emission, DriverHostActor
fan-out, OpcUaPublishActor routing, and the wire-level guard
(Condition_event_Quality_tracks_source_connectivity_on_the_wire) — RED-verified
against a simulated pre-fix always-Good server. Existing DriverInstanceActor
parent probes ignore the new ConnectivityChanged.
Docs: docs/AlarmTracking.md §"Condition source-data Quality (#477)";
design doc docs/plans/2026-07-17-alarm-condition-quality-477-design.md.
MaterialiseAlarmCondition never assigned the mandatory Part 9 ConditionType
classification fields, so every condition event — native and scripted — shipped
ConditionClassId = NodeId.Null (i=0) and ConditionClassName = empty text. Same
mechanism as #473: Create() builds the mandatory children from the type's
embedded definition but leaves them unset, and nothing downstream synthesises
them (ReportEvent / InstanceStateSnapshot copy children verbatim). An HMI
bucketing alarms by condition class dropped every OtOpcUa alarm as unclassified.
Report BaseConditionClassType — Part 9's "no condition class modelled" value.
This is the honest report: we hold no classification at the materialise seam.
Deliberately NOT ProcessConditionClassType (the SDK sample's pick), which would
assert a classification we cannot back and would be actively wrong for a Galaxy
alarm whose upstream category is Safety/Diagnostics — trading a detectable null
for an undetectable lie. Real per-alarm classification needs the driver's
AlarmCategory carried to this deploy-time seam (it lives only on the runtime
AlarmEventArgs transition today) and is a separate feature.
Guards, both observed RED against the pre-fix server:
- NativeAlarmEventIdentityFieldDeliveryTests: wire-level, its own select clause
(the #473 test's clause mirrors ScadaBridge's exactly and its indices are
load-bearing, so it is left untouched). The class fields are declared on
ConditionType, not BaseEventType, so they are selected against that type.
- NodeManagerAlarmSourceFieldsTests: node-level, native (Raw) + scripted (Uns).
Stacked on #473 (PR #474) — merge after it.
MaterialiseAlarmCondition never assigned the three mandatory BaseEventType
identity fields, so all three arrived null on every condition event — native
and scripted. The SDK does not synthesise them on this path: Create() builds
the children from the type definition but leaves them unset, the auto-filling
BaseEventState.Initialize overload is only used for transient events, and
ReportEvent / InstanceStateSnapshot copy children verbatim. A conforming
client could not attribute an alarm to its source.
EventType = the concrete materialised type (TypeDefinitionId)
SourceNode = the condition's own NodeId (== ConditionId) — the condition IS
the source; an alarm-bearing raw tag materialises only the
condition, with no sibling value variable
SourceName = the same identifying id string: RawPath (native) /
ScriptedAlarmId (scripted)
SourceName carries the unique id rather than the leaf name: the leaf collides
across devices (HR200 on two PLCs) and is already carried by ConditionName, so
nothing is lost. Documented in docs/AlarmTracking.md, including that clients
must key on ConditionId and must not compose SourceName with ConditionName,
and that SourceName is NOT a live<->history join key (the alarm-history writer
stamps it with the EquipmentPath — a pre-existing divergence, now called out).
Tests: NativeAlarmEventIdentityFieldDeliveryTests is the wire-level guard —
a real client subscription using the standard [EventType, SourceNode,
SourceName, Time, Message, Severity] select clause, verified to fail against
the pre-fix server. NodeManagerAlarmSourceFieldsTests guards the node across
both realms, the base-type fallback, and the kind-swap re-materialise.
The HistoryRead events projection is a separate path (it projects historian
rows, not node fields) and is unaffected — its EventType => Variant.Null
assertions still hold.
HistorianGateway is now the sole historian backend (read + alarm SendEvent +
continuous WriteLiveValues). Document the final state and retire the Wonderware
sidecar from the docs/config/labels:
- CLAUDE.md: rewrite the Historian section — ServerHistorian /
ContinuousHistorization / AlarmHistorian config keys, the IHistorianProvisioning
EnsureTags hook, the GatewayAlarmHistorianWriter SendEvent path + ReadEvents
dependency on gateway RuntimeDb:EventReadsEnabled=true, gateway-side
prerequisites (RuntimeDb flags + historian:read/write/tags:write scopes),
migration note, and two KNOWN-LIMITATION callouts (live-validation gate +
empty historized-ref-set recorder follow-on).
- appsettings.json: fix the stale ServerHistorian block (Host/Port/SharedSecret/
ServerCertThumbprint -> Endpoint/ApiKey/UseTls/AllowUntrustedServerCertificate/
CaCertificatePath/CallTimeout, keep MaxTieClusterOverfetch); add a disabled
ContinuousHistorization block; prune the orphaned Wonderware keys from
AlarmHistorian (keep the SQLite knobs). ApiKey env-supplied via
ServerHistorian__ApiKey (commented; valid strict JSON via _comment keys).
- README.md + docs (Historian.md, AlarmHistorian.md, Configuration.md,
ServiceHosting.md, DriverLifecycle.md, drivers/README.md, Uns.md, VirtualTags.md,
AlarmTracking.md, Client.UI.md, README.md, TestConnectProbes.md): retire the
Wonderware historian backend from current-backend descriptions; fix the stale
ServerHistorian/AlarmHistorian config tables (now gateway shape); convert
drivers/Historian.Wonderware.md to a retired stub pointing at the gateway.
- Source/UI labels (descriptive text only, no behavior change):
OtOpcUaServerHostedService.cs, HistoryPaging.cs, OtOpcUaSdkServer.cs,
HistorianAdapterActor.cs, VirtualTagModal.razor, ScriptedAlarmModal.razor,
AlarmsHistorian.razor now name the HistorianGateway backend.
Build clean (0 errors); AdminUI.Tests green (514 passed).
Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
Records T17-T22 as shipped: RoleCarryingUserIdentity, Part 9 method handlers gated on AlarmAck
role, alarm-commands DPS topic, ScriptedAlarmHostActor dispatch, WriteAlarmCondition delta-gate,
AdminUI /alerts Acknowledge/Shelve/Unshelve buttons via AdminOperationsActor singleton, and
Client.CLI ack/confirm/shelve commands. Corrects stale "Not started" / "Partial" entries in
phase-7-status.md (Stream G OPC UA method binding row and C.6 row and Gap 1 body) and adds
the alarm-commands topic to Runtime.md. Removes untracked scratch files resume.md and pending.md.
ORPHAN DECISION: Keep as live doc (path: keep-and-fix).
Rationale: the file carries unique v2 current content describing
the alarms-over-gateway epic architecture; docs/ScriptedAlarms.md
cross-references it explicitly. The orphan symptom is that
docs/README.md still indexes docs/v1/AlarmTracking.md — wiring
this top-level file into README.md is a follow-up task.
STRUCTURAL (dimension 2):
- docs/AlarmTracking.md line 138: Security.md → security.md (CASE-MISMATCH
from links-report.md rows 1–2). Verified: docs/security.md exists
(inode 77517627); docs/Security.md is the same file on APFS
case-insensitive FS, but the checker requires exact on-disk casing.
check_links.py: zero rows for docs/AlarmTracking.md after fix.
CODE-REALITY (dimension 4):
- line 16 table: `Phase7EngineComposer` / `Phase7EngineComposer.RouteToHistorianAsync`
→ no such class exists. Real class is `Phase7Composer`
(src/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer/Phase7Composer.cs).
Scripted-alarm historian routing goes through ScriptedAlarmActor →
HistorianAdapterActor → IAlarmHistorianSink, not a RouteToHistorianAsync
method. Fixed to: Phase7Composer / ScriptedAlarmActor transitions →
HistorianAdapterActor → IAlarmHistorianSink.
- lines 107–123 "Historian write-back" section: referenced
`Phase7Composer.ResolveHistorianSink` (method doesn't exist in
current Phase7Composer.cs), `GalaxyProxyDriver` / `GalaxyHistorianWriter`
(retired in PR 7.2 — no such class in codebase), and `aahClientManaged`
as a direct call (now mediated through WonderwareHistorianClient).
Current architecture: NullAlarmHistorianSink default registered in
ServiceCollectionExtensions.AddOtOpcUaRuntime(); production override
is SqliteStoreAndForwardSink wrapping WonderwareHistorianClient; bridge
is HistorianAdapterActor (src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Historian/
HistorianAdapterActor.cs). Section rewritten to match code reality.
- line 108: "Program.cs" as NullAlarmHistorianSink registration site →
actual site is ServiceCollectionExtensions.cs, not Program.cs.
STALE-STATUS (dimension 3): no blocked/pending/not-yet banners found
in the top-level file; it was already written as current-state fact.
Galaxy native alarms work end-to-end (verified 2026-05-31) and the
doc correctly describes that as delivered.
CODE-BUG-FLAGS: none. All stale references were doc-side errors; the
production code is correct.
UNVERIFIABLE CLAIMS: AlarmConditionService, DriverNodeManager, ConditionSink,
DriverAlarmSourceAcknowledger, DriverWritableAcknowledger — these are
mentioned by name in the doc but their .cs files were not found in the
search. They may live under a path not searched, or may be internal
implementation details within existing files. These claims are plausible
given the architecture and were not changed.
Add #pragma warning disable xUnit1051 at the top of ContractsWireParityTests.cs.
The xUnit1051 analyser fires on MessagePack's Serialize/Deserialize overloads that
have an optional CancellationToken parameter; these are synchronous parity tests
where the token is not meaningful — the suppression is scoped to this file only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite src/ and tests/ project paths in docs, CLAUDE.md, README.md, and
test-fixture READMEs to the new module-folder layout (Core/Server/Drivers/
Client/Tooling). References to retired v1 projects (Galaxy.Host/Proxy/Shared,
the legacy monolithic test projects) are left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sixteenth PR of the alarms-over-gateway epic
(docs/plans/alarms-over-gateway.md). Closes the documentation sweep
the plan calls for.
- docs/AlarmTracking.md — promoted top-level v2-final architecture
doc (was a worktree-only draft pre-epic). Covers the three alarm
sources (Galaxy MxAccess driver-native / Galaxy sub-attribute
fallback / scripted alarms), how they converge on
AlarmConditionService, the Acknowledge routing decision in
DriverNodeManager (driver-native preferred over IWritable
sub-attribute fallback), the sidecar historian write-back path
for non-Galaxy producers, and cross-references to the plan +
v1 archive.
- docs/v1/AlarmTracking.md — banner pointing readers at the v2
doc; preserved as historical record.
- docs/drivers/Galaxy.md — capability list updated to include
IAlarmSource (now eight capabilities, restored by B.2). Replaced
the "IAlarmSource retired in 7.2" sentence with the restoration
note + cross-link to docs/AlarmTracking.md.
- docs/plans/alarms-over-gateway.md — completion banner at the
top of the plan, marking 14 of 16 PRs shipped 2026-04-30 and
noting that A.2 + A.4 + D.1 are the hardware-gated follow-up.
Memory entries updated separately:
- project_alarms_over_gateway_epic.md (new) — epic summary +
per-PR digest.
- project_galaxy_via_mxgateway.md — added "Alarms restored"
bullet pointing at the new architecture.
- project_server_history_alarm_subsystems.md — bullet 2 updated
to describe the new ack-routing decision (B.3) + bullet 3
added describing the historian write-back path that B.4 + C.1
+ C.2 light up.
- MEMORY.md index — new pointer entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit (three parallel agent passes) found 43 markdown files carrying
stale references to the deleted Galaxy.Host/Proxy/Shared projects
after the v2-mxgw merge. This commit lands the prioritized fixes.
Track 1 — high-traffic in-place rewrites (3 files, ~454 lines deleted)
- README.md (202 → 91 lines): drops .NET 4.8 / x86 / TopShelf install
text; leads with the multi-driver .NET 10 server identity and points
at scripts/install/Install-Services.ps1 and the parity rig.
- docs/v2/driver-specs.md §1 Galaxy (~289 → ~66 lines): replaces the
Tier-C out-of-process spec with a Tier-A in-process description
matching the current GalaxyDriver code, with the four-section
GalaxyDriverOptions JSON shape pulled verbatim from
Config/GalaxyDriverOptions.cs.
- docs/drivers/Galaxy.md (211 → 92 lines): full rewrite around the
current Browse/Runtime/Health/Config sub-folders.
Track 2 — historical banners (5 files)
- lmx_mxgw.md, lmx_mxgw_impl.md, lmx_backend.md,
docs/v2/Galaxy.ParityMatrix.md,
docs/v2/implementation/phase-2-galaxy-out-of-process.md each get a
"✅ Completed 2026-04-30 — historical record" banner block. lmx_mxgw.md
also fixes two dead links (`docs/Galaxy.Driver.md` and
`docs/v2/Galaxy.Driver.md`) → `docs/drivers/Galaxy.md`.
Track 3 — v1 archive sweep (10 git mv + 1 new index + 2 in-place scrubs)
- Moved 10 v1 docs under docs/v1/ preserving subpath structure:
AlarmTracking, Configuration, DataTypeMapping, HistoricalDataAccess,
Subscriptions (top-level); drivers/Galaxy-Repository,
drivers/Galaxy-Test-Fixture; reqs/GalaxyRepositoryReqs,
reqs/MxAccessClientReqs, reqs/ServiceHostReqs.
- New docs/v1/README.md is the shared archive banner + per-file table.
- docs/README.md repointed to the v1 paths and updated to reflect the
v2 two-process deploy shape (Server + Admin + optional
OtOpcUaWonderwareHistorian).
- docs/v2/Galaxy.ParityRig.md got a historical banner + four inline
scrubs marking the OtOpcUaGalaxyHost service / Driver.Galaxy.Host
EXE / Driver.Galaxy.ParityTests project as deleted-in-PR-7.2.
The repo's live-reading surface (README + CLAUDE.md + docs/v2/) now
describes only the post-PR-7.2 architecture. v1 docs are preserved as
a labelled archive under docs/v1/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit of docs/ against src/ surfaced shipped features without current-reference
coverage (FOCAS CLI, Core.Scripting+VirtualTags, Core.ScriptedAlarms,
Core.AlarmHistorian), an out-of-date driver count + capability matrix, ADR-002's
virtual-tag dispatch not reflected in data-path docs, broken cross-references,
and OpcUaServerReqs declaring OPC-020..022 that were never scoped. This commit
closes all of those so operators + integrators can stay inside docs/ without
falling back to v2/implementation/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite seven core-architecture docs to match the shipped multi-driver platform.
The v1 single-driver LmxNodeManager framing is replaced with the Core +
capability-interface model — Galaxy is now one driver of seven, and each doc
points at the current class names + source paths.
What changed per file:
- OpcUaServer.md — OtOpcUaServer as StandardServer host; per-driver
DriverNodeManager + CapabilityInvoker wiring; Config-DB-driven configuration
(sp_PublishGeneration, DraftRevisionToken, Admin UI); Phase 6.2
AuthorizationGate integration.
- AddressSpace.md — GenericDriverNodeManager.BuildAddressSpaceAsync walks
ITagDiscovery.DiscoverAsync and streams DriverAttributeInfo through
IAddressSpaceBuilder; CapturingBuilder registers alarm-condition sinks;
per-driver NodeId schemes replace the fixed ns=1;s=ZB root.
- ReadWriteOperations.md — OnReadValue / OnWriteValue dispatch to
IReadable.ReadAsync / IWritable.WriteAsync through CapabilityInvoker,
honoring WriteIdempotentAttribute (#143); two-layer authorization
(WriteAuthzPolicy + Phase 6.2 AuthorizationGate).
- Subscriptions.md — ISubscribable.SubscribeAsync/UnsubscribeAsync is the
capability surface; STA-thread story is now Galaxy-specific (StaPump inside
Driver.Galaxy.Host), other drivers are free-threaded.
- AlarmTracking.md — IAlarmSource is optional; AlarmSurfaceInvoker wraps
Subscribe/Ack/Unsubscribe with fan-out by IPerCallHostResolver and the
no-retry AlarmAcknowledge pipeline (#143); CapturingBuilder registers sinks
at build time.
- DataTypeMapping.md — DriverDataType + SecurityClassification are the
driver-agnostic enums; per-driver mappers (GalaxyProxyDriver inline,
AbCipDataType, ModbusDriver, etc.); SecurityClassification is metadata only,
ACL enforcement is at the server layer.
- IncrementalSync.md — IRediscoverable covers backend-change signals;
sp_ComputeGenerationDiff + DiffViewer drive generation-level change
detection; IDriver.ReinitializeAsync is the in-process recovery path.
Add ServerCapabilities/OperationLimits node, enable diagnostics, add OnModifyMonitoredItemsComplete
override for DA compliance. Wire shelving, enable/disable, confirm, and addcomment handlers on
alarm conditions with LocalTime/Quality event fields for Part 9 compliance. Add Aes128/Aes256
security profiles, X.509 certificate authentication, and AUDIT-prefixed auth logging. Fix flaky
probe monitor test. Update docs for all changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Map LDAP groups to custom OPC UA role NodeIds on RoleBasedIdentity.GrantedRoleIds
during authentication, replacing the username-to-role side cache. Split ReadWrite
into WriteOperate/WriteTune/WriteConfigure so write access is gated per Galaxy
security classification. AnonymousCanWrite now behaves consistently regardless
of LDAP state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously alarms were only reported to the immediate parent node and the Server node.
Now ReportEventUpNotifierChain walks the full parent chain so clients subscribed at
TestArea see alarms from TestMachine_001, and EventNotifier is set on all ancestors
of alarm-containing nodes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Joseph Doherty