dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
7804fe7958db6b6a06e8d3ebbd8381598df17eb3
ScadaBridge/docs/requirements/Component-DataConnectionLayer.md
T

44 KiB
Raw Blame History

Component: Data Connection Layer

Purpose

The Data Connection Layer provides a uniform interface for reading from and writing to physical machines at site clusters. It abstracts protocol-specific details behind a common interface, manages subscriptions, and delivers live tag value updates to Instance Actors. It is a clean data pipe — it performs no evaluation of triggers, alarm conditions, or business logic.

Location

Site clusters only. Central does not interact with machines directly.

Responsibilities

  • Manage data connections defined centrally and deployed to sites as part of artifact deployment (OPC UA servers). Data connection definitions are stored in local SQLite after deployment.
  • Establish and maintain connections to data sources based on deployed instance configurations.
  • Subscribe to tag paths as requested by Instance Actors (based on attribute data source references in the flattened configuration).
  • Deliver tag value updates to the requesting Instance Actors.
  • Support writing values to machines (when Instance Actors forward SetAttribute write requests for data-connected attributes).
  • Report data connection health status to the Health Monitoring component.

Common Interface

All protocol adapters implement the same interface:

IDataConnection : IAsyncDisposable
├── Connect(connectionDetails) → void
├── Disconnect() → void
├── Subscribe(tagPath, callback) → subscriptionId
├── Unsubscribe(subscriptionId) → void
├── Read(tagPath) → value
├── ReadBatch(tagPaths) → values
├── Write(tagPath, value) → void
├── WriteBatch(values) → void
├── WriteBatchAndWait(values, flagPath, flagValue, responsePath, responseValue, timeout) → bool
├── Status → ConnectionHealth
└── Disconnected → event Action?

The Disconnected event is raised by an adapter when it detects an unexpected connection loss (server offline, network failure, keep-alive timeout). The DataConnectionActor subscribes to this event to trigger the reconnection state machine. Additional protocols can be added by implementing this interface.

Batch Capability Seam

A protocol whose wire form can subscribe MANY tags in one round trip additionally implements the optional capability interface IBatchSubscribableConnection (same pattern as IBrowsableDataConnection / IAlarmSubscribableConnection):

IBatchSubscribableConnection
├── SubscribeBatchAsync(tagPaths, callback) → IReadOnlyList<TagSubscribeResult>
└── UnsubscribeBatchAsync(subscriptionIds) → void
  • One shared callback, plain tag list. Every tag in a batch shares one callback; SubscriptionCallback already carries the tag path, so per-tag delegates would carry nothing.
  • Partial-failure contract. A per-tag fault (bad node id, unresolvable path) is a result row with Success:false and never aborts the batch — mirroring the gateway's SubscribeResult and OPC UA's per-monitored-item create status. A thrown exception means the whole batch failed; the actor classifies it exactly as the per-tag path does and drives the Reconnecting state machine on a connection-level fault.
  • Implementing the interface also asserts that ReadBatchAsync / WriteBatchAsync are true bulk service calls rather than loops over the single-tag methods. Both shipped adapters (OpcUaDataConnection, MxGatewayDataConnection) implement it; an adapter that does not keeps the historical per-tag behaviour, so the capability is purely additive.
  • Single-tag SubscribeAsync / ReadAsync / WriteAsync are kept (heartbeat monitor, interactive read/write paths) and delegate to the batch form as a batch of one.

Sizing: at the 37,500-tag site target (docs/deployment/topology-guide.md), the pre-batch path issued one adapter round trip — and, on OPC UA, one ApplyChanges — per tag on every subscribe and every reconnect.

Common Value Type

All protocols produce the same value tuple consumed by Instance Actors. Before the first value update arrives from the DCL, data-sourced attributes are held at uncertain quality by the Instance Actor (see Site Runtime — Initialization):

Concept ScadaBridge Design
Value container TagValue(Value, Quality, Timestamp)
Quality QualityCode enum: Good / Bad / Uncertain
Timestamp DateTimeOffset (UTC)
Value type object?

Supported Protocols

OPC UA

  • Uses the OPC Foundation .NET Standard Library (OPCFoundation.NetStandard.Opc.Ua.Client).
  • Session-based connection with endpoint discovery, certificate handling, and configurable security modes.
  • Subscriptions via OPC UA Monitored Items with data change notifications (1000ms sampling, queue size 10, discard-oldest).
  • Read/Write via OPC UA Read/Write services with StatusCode-based quality mapping.
  • Disconnect detection via Session.KeepAlive event (see Disconnect Detection Pattern below).

MxGateway

  • Connects to the MxAccess Gateway (AVEVA/Wonderware MXAccess-backed Galaxy) over gRPC using the ZB.MOM.WW.MxGateway.Client NuGet package (from the Gitea feed); ZB.MOM.WW.MxGateway.Contracts is pulled in transitively.
  • Session-based: OpenSession + Register on connect; AddItem + Advise per subscription; value changes arrive on the gateway's server-streaming event feed (StreamEvents), resumable via worker_sequence.
  • Read/Write via ReadBulk / WriteBulk; writes carry a configurable WriteUserId. Quality maps the OPC-style quality byte (≥192 Good, ≥64 Uncertain, else Bad), with a failing MXAccess status proxy treated as Bad.
  • Galaxy hierarchy browse via the separate GalaxyRepositoryClient — objects are navigable nodes (keyed by Galaxy gobject id), attributes are selectable leaves (keyed by full tag reference). Browse is lazy and attribute-light: navigation uses BrowseChildren with include_attributes=false (child objects only), and an object's own attributes are fetched only when it is expanded, via DiscoverHierarchy(root=<object>, max_depth=0) scoped to that single object. This keeps each browse level's reply small; inlining every child's full attribute set could exceed the Akka remote frame and silently drop the reply.
  • Disconnect detection: a fault on the event stream raises IDataConnection.Disconnected, driving the same reconnection state machine as OPC UA.
  • Implemented as MxGatewayDataConnection over an IMxGatewayClient seam; the seam is decoupled from the generated gRPC types (only RealMxGatewayClient references them), so the adapter is fully unit-testable with a fake.

Endpoint Redundancy

Data connections support an optional backup endpoint for automatic failover when the active endpoint becomes unreachable. Both endpoints use the same protocol.

Entity fields:

Field Type Notes
PrimaryConfiguration string? (max 4000) Required. Renamed from Configuration
BackupConfiguration string? (max 4000) Optional. Null = no backup
FailoverRetryCount int (default 3) Retries on active endpoint before switching

Failover state machine:

%%{init: {'theme':'base', 'themeVariables': {'textColor':'#111111','lineColor':'#555555','edgeLabelBackground':'#ffffff','fontSize':'15px'}}}%%
flowchart TD
    connected(["Connected"])
    pushbad["push bad quality"]
    retry["retry active endpoint<br/>(5s)"]
    decide{"N failures<br/>(≥ FailoverRetryCount)?"}
    switch["switch to other endpoint"]
    dispose["dispose adapter,<br/>create fresh adapter<br/>with other config"]
    reconnect["reconnect"]
    resub["ReSubscribeAll"]

    connected -->|disconnect| pushbad
    pushbad --> retry
    retry --> decide
    decide -->|"no (retry again)"| retry
    decide -->|yes| switch
    switch --> dispose
    dispose --> reconnect
    reconnect --> resub
    resub -->|back to Connected| connected

    classDef start fill:#d5e8d4,stroke:#82b366,color:#111111;
    classDef proc fill:#dae8fc,stroke:#6c8ebf,color:#111111;
    classDef dec fill:#fff2cc,stroke:#d6b656,color:#111111;
    classDef warn fill:#ffe6cc,stroke:#d79b00,color:#111111;
    classDef bad fill:#f8cecc,stroke:#b85450,color:#111111;
    class connected start
    class pushbad bad
    class retry,reconnect,resub proc
    class decide dec
    class switch,dispose warn
  • Round-robin: primary → backup → primary → backup. No preferred endpoint after first failover — the connection stays on whichever endpoint is working.
  • No auto-failback: The connection remains on the active endpoint until it fails.
  • Single-endpoint connections (no backup): Retry indefinitely on the same endpoint, preserving existing behavior.
  • Adapter lifecycle on failover: The actor disposes the current IDataConnection adapter and creates a fresh one via DataConnectionFactory.Create() with the other endpoint's configuration. Clean slate — no stale state.

Health reporting:

  • DataConnectionHealthReport includes ActiveEndpoint: "Primary", "Backup", or "Primary (no backup)".

Site event log entries:

  • DataConnectionFailover (Warning) — connection name, from-endpoint, to-endpoint, failure count.
  • DataConnectionRestored (Info) — connection name, active endpoint.

See 2026-03-22-primary-backup-data-connections-design.md for the full design.

Connection Configuration Reference

All settings are parsed from the data connection's configuration JSON dictionaries (PrimaryConfiguration and optional BackupConfiguration, stored as IDictionary<string, string> connection details). Both endpoints use the same protocol-specific keys. Invalid numeric values fall back to defaults silently.

OPC UA Settings

Key Type Default Description
endpoint / EndpointUrl string opc.tcp://localhost:4840 OPC UA server endpoint URL
SessionTimeoutMs int 60000 OPC UA session timeout in milliseconds
OperationTimeoutMs int 15000 Transport operation timeout in milliseconds
PublishingIntervalMs int 1000 Subscription publishing interval in milliseconds
KeepAliveCount int 10 Keep-alive frames before session timeout
LifetimeCount int 30 Subscription lifetime in publish intervals
MaxNotificationsPerPublish int 100 Max notifications batched per publish cycle
SamplingIntervalMs int 1000 Per-item server sampling rate in milliseconds
QueueSize int 10 Per-item notification buffer size
MaxMonitoredItemsPerSubscription int 5000 Monitored-item budget per OPC UA subscription. Above it the adapter shards onto an additional subscription on the same session (37,500 tags → 8 shards). Per endpoint, because item-count ceilings are a property of the target server.
SecurityMode string None Preferred endpoint security: None, Sign, or SignAndEncrypt
AutoAcceptUntrustedCerts bool true Accept untrusted server certificates

MxGateway Settings

Key Type Default Description
Endpoint string http://localhost:5000 Gateway base URL
ApiKey string — Sent to the gateway as authorization: Bearer <key>
ClientName string scadabridge (when blank) MXAccess client registration name
WriteUserId int 0 MXAccess user id applied to every write-back (0 = no user context)
UseTls bool false Use TLS to a secured gateway
CaFile string — Path to the CA certificate (TLS only)
ServerName string — TLS server-name override
ReadTimeoutMs int 5000 ReadBulk per-call timeout in milliseconds

Secret handling for ApiKey follows the same at-rest treatment and log/telemetry redaction as the OPC UA UserIdentity username/password fields.

Shared Settings (appsettings.json)

These are configured via DataConnectionOptions in appsettings.json, not per-connection:

Setting Default Description
ReconnectInterval 5s Fixed interval between reconnection attempts
TagResolutionRetryInterval 10s Floor interval for the unresolved-tag retry; the retry backs off from here
TagResolutionRetryMaxInterval 5m Ceiling for the exponential tag-resolution backoff
WriteTimeout 30s Timeout for write operations
SeedReadTimeout 30s Per-chunk timeout for seed reads on the initial-subscribe (and reconnect re-seed) path. A hung device read is treated as a failed seed: retried up to SeedReadMaxAttempts, then the tag stays Uncertain until a change notification arrives.
SeedOverallTimeout 120s Wall-clock deadline for the WHOLE seed (all chunks, all retry rounds). Replaces the old 30s-per-tag serial worst case; remaining tags are logged and stay Uncertain.
SeedReadBatchSize 250 Tags per seed-read chunk
SeedReadMaxParallelism 4 Seed-read chunks in flight concurrently
SubscribeBatchSize 500 Tags per adapter subscribe round trip (initial subscribe, reconnect re-subscribe, resolution probes)
SubscribeBatchDelay 50ms Delay BETWEEN reconnect re-subscribe chunks. Not applied on the instance-driven subscribe path — the Deployment Manager already staggers instance startup there.
QualityFlushInterval 1s Coalescing window for pushing tag-quality counters to the health collector
MxSupervisoryAdviseParallelism 16 Max in-flight supervisory advise commands on the MxGateway bulk-subscribe path when the endpoint has no write-user context

Subscription Management

  • When an Instance Actor is created (as part of the Site Runtime actor hierarchy), it registers its data source references with the Data Connection Layer.
  • The DCL subscribes to the tag paths using the concrete connection details from the flattened configuration.
  • Tag value updates are delivered directly to the requesting Instance Actor.
  • When an Instance Actor is stopped (due to disable, delete, or redeployment), the DCL cleans up the associated subscriptions.
  • When a new Instance Actor is created for a redeployment, subscriptions are established fresh based on the new configuration.

Batching, chunking and pacing

Three subscribe paths, deliberately paced differently:

Path Shape
Instance-driven subscribe (SubscribeTagsRequest, i.e. site failover / deploy) One SubscribeBatchAsync per SubscribeBatchSize chunk with no added delay. The Deployment Manager already staggers instance startup (SiteRuntimeOptions.StartupBatchSize/StartupBatchDelayMs), so requests arrive pre-spaced at ~75 tags each — double-staggering would only slow failover.
Reconnect re-subscribe (ReSubscribeAll) Sequential chunks of SubscribeBatchSize with SubscribeBatchDelay between them, all inside ONE background task; each chunk reports its per-tag results back to the actor as its own message. Replaces a tight loop that fired one fire-and-forget task per tag.
Tag-resolution probe The whole not-in-flight unresolved set, chunked at SubscribeBatchSize, as one probe round per timer tick.

Seeding (the initial-value read that keeps a STATIC tag from staying Uncertain) issues chunked bulk reads — SeedReadBatchSize per chunk, SeedReadMaxParallelism chunks in flight, SeedReadTimeout per chunk — under a single SeedOverallTimeout deadline covering every chunk and retry round.

OPC UA subscription sharding. RealOpcUaClient places monitored items on the first subscription shard with free capacity (MaxMonitoredItemsPerSubscription), creating a shard when all are full and deleting one once it empties. A batch is N × AddItem plus one ApplyChanges per touched shard. Alarms & Conditions event items are pinned to a dedicated event shard, so ConditionRefresh has a single subscription id to target and QueueSize:1000 event items never consume data-shard capacity. Every structural mutation (AddItem / RemoveItem / ApplyChanges / Create / Delete) and every shard-list mutation serializes behind ONE per-client gate — the SDK Subscription is not safe under concurrent structural mutation, and the subscribe task, resolution probe and alarm subscribe are otherwise unordered.

MxGateway. A batch is one SubscribeBulk gateway command (AddItem + Advise per tag in a single worker pass), replacing the historical 2 RPCs per tag. When the endpoint has no write-user context (WriteUserId == 0) the worker offers no BULK supervisory advise, so the client issues one AddItemBulk and then pipelines the per-item supervisory advises with a bounded window (MxSupervisoryAdviseParallelism). A cross-repo follow-up (an additive supervisory flag on SubscribeBulkCommand in mxaccessgw) would collapse that to one RPC as well.

Write-Back Support

  • When a script calls Instance.SetAttribute for an attribute with a data source reference, the Instance Actor sends a write request to the DCL.
  • The DCL writes the value to the physical device via the appropriate protocol.
  • The existing subscription picks up the confirmed new value from the device and delivers it back to the Instance Actor as a standard value update.
  • The Instance Actor's in-memory value is not updated until the device confirms the write.

Browsing the address space

DCL is a clean data pipe on the hot path. Browse is an opt-in capability for protocols that support it, exposed via IBrowsableDataConnection. Only consumed by management/UI (the tag picker on the instance configure page); Instance Actors never call it. The browse path is protocol-agnostic: the same command/service/dialog serve every browsable protocol.

  • OpcUaDataConnection and MxGatewayDataConnection both implement IBrowsableDataConnection; other/custom protocols do not (and return a NotBrowsable failure).
  • DataConnectionManagerActor handles BrowseNodeCommand (fields: ConnectionName, ParentNodeId, and an optional opaque ContinuationToken for paging) and replies with BrowseNodeResult (children + Truncated + an optional continuation token + structured BrowseFailure?). The Central UI facade is IBrowseService/BrowseService, backing the NodeBrowserDialog tag picker.
  • Browse type-info: each child BrowseNode carries optional best-effort type metadata — DataType (friendly name), ValueRank (scalar/array), and Writable. DataType is populated by both OPC UA and MxGateway: RealOpcUaClient batch-reads built-in data-type node ids and maps them to friendly names for Variable nodes; MxGatewayDataConnection surfaces each attribute leaf's GalaxyAttribute.data_type_name (free-form Galaxy type text) as DataType. ValueRank and Writable are OPC-UA-only; MxGateway leaves them unset.
  • Node ids are opaque protocol-specific strings: OPC UA uses NodeIds; MxGateway uses Galaxy gobject ids for navigable objects and full tag references for selectable attribute leaves.
  • OPC UA namespace binding (Gitea #14): a NodeId's ns=<index> is meaningful only against one server's namespace table, so a server that adds/removes/reorders a namespace silently re-points every stored binding — best case BadNodeIdUnknown, worst case the index names a different namespace holding a colliding identifier and the binding resolves to the wrong node with Good quality. The namespace URI is the stable identity. OpcUaNodeReference is therefore the single translation seam between stored references and the wire:
    • Resolve(reference, session.NamespaceUris) accepts both ns=<index>;s=<id> (existing bindings, unchanged) and the durable nsu=<namespace-uri>;s=<id>, mapping the URI to the live index at use time. It is used at every parse site — subscribe, read, write, alarm-subscribe, browse. A URI absent from the server's NamespaceArray throws with a message naming the URI, rather than binding to whatever sits at that index; a svr= reference to another server is rejected outright.
    • ToDurable(expandedNodeId, session.NamespaceUris) is what the browser emits, so what the picker shows is what gets stored and newly-authored bindings are index-proof from the start. Namespace 0 is exempt (spec-fixed) and keeps its short form. It also closes a round-trip gap: the browse previously emitted ExpandedNodeId.ToString(), which for a URI- or server-index-carrying reference produced a string NodeId.Parse could not read back.
    • Bindings stored before this change keep their ns= form and keep working; they are only as durable as the server's namespace order. Re-authoring against a picker (which now emits nsu=) is what makes them durable — see docs/plans/ and Gitea #14 for the OtOpcUa v3.0 cutover, where v2's sole custom namespace and v3's raw namespace both sit at index 2, so v2-era bindings resolve against v3 without error while meaning something else.
    • OtOpcUa v3.0 dual-namespace cutover: v3.0 splits OtOpcUa's address space into two namespaces, raw (https://zb.com/otopcua/raw) and uns (https://zb.com/otopcua/uns), where v2 exposed one. ScadaBridge is already namespace-URI-durable against this split: OpcUaNodeReference resolves a stored nsu=<uri>;s=... binding against the live session NamespaceArray at use time, so it is index-proof regardless of which namespace a given tag lands in or how many namespaces the server publishes. The node-browser picker already nudges operators toward the durable form, since ToDurable is what it emits for newly-authored bindings. See docs/plans/2026-07-23-otopcua-v3-dual-namespace-cutover-scope.md for the full cutover scope.
  • Browse runs against the live session; no caching at DCL.
  • Frame-size guard: the reply crosses the site→central Akka frame (default 128 KB) on a temp Ask actor; an oversized reply is silently discarded by remoting, hanging the picker. The child handler caps each BrowseNodeResult to a byte budget (~100 KB) before replying, OR-ing the adapter's own truncation signal into Truncated. This is protocol-agnostic (every adapter's reply funnels through it). Per-protocol upstream caps narrow the window first: OPC UA requests at most 500 references per node (continuation point → Truncated); MxGateway relies on the gateway's BrowseChildren page cap.
  • BrowseNext paging (OPC UA): a Truncated level no longer forces manual node-id entry. When the OPC UA browse is truncated, the adapter returns the session's continuation point as the reply's opaque ContinuationToken; a follow-up BrowseNodeCommand carrying that token calls Session.BrowseNext to fetch the next page (the picker exposes this as "Load more"). Continuation points are session-bound and can expire — BadContinuationPointInvalid is caught and the browse restarts from a fresh first page rather than failing. Manual node-id entry remains as a fallback when the site or its session is offline.

Address-space search

A second opt-in capability seam, IAddressSpaceSearchable (in Commons, mirroring the IBrowsableDataConnection / IAlarmSubscribableConnection pattern; implemented by the OPC UA adapter, consumed by management/UI only):

IAddressSpaceSearchable
└── SearchAddressSpaceAsync(query, maxDepth, maxResults, ct) → matches
  • DataConnectionManagerActor handles SearchAddressSpaceCommand (ConnectionName, Query, MaxDepth, MaxResults); the OPC UA adapter does a bounded recursive browse (depth + result caps) from the Objects folder, matching a case-insensitive substring against each node's DisplayName and root-relative path, and returns matches as AddressSpaceMatch (the BrowseNode plus its full path). When a cap is hit the result flags it so the UI can prompt "showing first N — refine".
  • The Central UI facade is BrowseService.SearchAsync (Design role), surfaced as the search box in NodeBrowserDialog.
  • StubOpcUaClient carries a canned browse/search implementation so the picker and its bUnit/unit tests run without a live OPC UA server.

Endpoint verification (OPC UA)

Before saving or deploying an OPC UA data connection, the Central UI can ask the target site to probe the configured endpoint — this exercises connectivity (and TLS trust) against the live config, including edited-but-unsaved values.

  • DataConnectionManagerActor handles VerifyEndpointCommand (SiteId, protocol, config JSON). The site spins up a temporary RealOpcUaClient from the submitted config, attempts discovery + a session with a short timeout (a few seconds), then disconnects and disposes the client. The reply is a VerifyEndpointResult (Success, a typed FailureKind, an error message, and an optional captured server cert).
  • The probe forces AutoAcceptUntrustedCerts = false and hooks the certificate-validation callback so it can capture an untrusted server certificate (Subject / Issuer / Thumbprint / NotBefore / NotAfter / DER) for the UI to display. Critically, the probe never trusts the cert — the validation callback always rejects (Accept = false); trusting is a separate, explicit, Admin-gated action that writes the cert into the site PKI store (see the cert-trust path in Component-SiteRuntime.md). The probe is read-only with respect to the trust boundary.

Native Alarm Mirroring

Some data sources publish their own alarms — OPC UA Alarms & Conditions servers and the MxAccess Gateway. The DCL can mirror these native alarms into the Site Runtime as a read-only feed: ScadaBridge reflects source alarm state but never acknowledges, confirms, shelves, or otherwise writes back to the source. This complements (does not replace) ScadaBridge's own computed alarms; it feeds the Site Runtime's NativeAlarmActor peer subsystem.

Like browse, this is an opt-in capability for protocols that support it. It does not touch the hot value path — alarm transitions flow over a separate per-connection feed.

Capability Seam

Mirroring is exposed via the optional IAlarmSubscribableConnection capability interface (in Commons), which an IDataConnection implementation may also implement (mirroring the IBrowsableDataConnection pattern; consumed by the DataConnectionActor only):

IAlarmSubscribableConnection
├── SubscribeAlarmsAsync(sourceReference, conditionFilter?, callback, ct) → subscriptionId
└── UnsubscribeAlarmsAsync(subscriptionId, ct) → void

The AlarmTransitionCallback delivers a protocol-neutral NativeAlarmTransition per transition. On every (re)subscribe the adapter replays a snapshot of currently-active conditions (Snapshot… records terminated by a SnapshotComplete sentinel) so consumers can reconcile state after a reconnect.

Protocol Adapters

  • OPC UA (OpcUaDataConnection + RealOpcUaClient): a single event MonitoredItem (AttributeId = EventNotifier) on the Server object, with an EventFilter selecting EventType / SourceNode / Severity plus the ConditionType / AcknowledgeableConditionType / AlarmConditionType state fields. ConditionRefresh is invoked on subscribe to replay active conditions as the snapshot. The OPC UA field → NativeAlarmTransition mapping is isolated in the pure helper OpcUaAlarmMapper, unit-testable without a live server.
  • MxGateway (MxGatewayDataConnection + RealMxGatewayClient): mirrors over the gateway package's StreamAlarmsAsync — a resumable background stream whose reconnect re-sends a snapshot. The field mapping lives in MxGatewayAlarmMapper.

Other/custom protocols do not implement the capability; a subscribe request against such a connection is replied to with a failure (SubscribeAlarmsResponse.Success = false).

Connection Actor Behavior

The DataConnectionActor opens one alarm feed per source (not per subscriber) and routes incoming transitions to instance subscribers by source-object reference — a prefix match of the transition's SourceObjectReference (falling back to SourceReference) against each subscriber's registered SourceReference. Subscribers (the Site Runtime's NativeAlarmActor instances) are ref-counted per source, so the underlying feed is opened once and torn down only when the last subscriber for that source unsubscribes.

The registered SourceReference is a NodeId for OPC UA (the picker, CSV and config all store NodeIds), so the transition's routing identity must live in the same space or nothing matches. Gitea #17: the OPC UA adapter previously set SourceObjectReference from the event's SourceName (a plain name — the RawPath for OtOpcUa), which never prefix-matches a NodeId binding, so every native OPC UA alarm transition was silently dropped. Because each OPC UA feed is opened for exactly one binding, the adapter now tags every transition on that feed with the binding string verbatim (via the pure OpcUaAlarmMapper.BuildIdentity), making the routing key an exact match independent of whether the binding is stored as ns=<index> or the durable nsu=<uri> form. The Server-object aggregate feed (empty binding) keeps an empty routing identity, so it reaches only "mirror everything" subscribers and never leaks into a specific-node binding. The per-condition SourceReference key stays the human-readable SourceName.ConditionName, so persistence and display are unchanged. MxGateway is unaffected — its bindings are object names and its mapper already emits matching names.

  • State gating: SubscribeAlarmsRequest is handled only in the Connected state; requests arriving while Connecting/Reconnecting are stashed (standard Become/Stash) and processed on entering Connected.
  • Capability check: if _adapter is not IAlarmSubscribableConnection, the actor replies SubscribeAlarmsResponse(Success = false, ...).
  • Reconnect handling: on entering Reconnecting, the actor pushes a NativeAlarmSourceUnavailable to every alarm subscriber (consumers mark mirrored alarms uncertain rather than clearing them). On successful reconnection it re-subscribes the feed; the adapter re-emits a snapshot, reconciling state.
  • Shared condition filter (last-subscriber-wins): because the feed is opened once per source, it carries a single condition filter. A second subscriber that registers a different filter for the same source overwrites it (last writer wins) and the actor logs a warning — co-subscribers to one source are expected to agree on the filter.
  • Routing index: subscribed sources are bucketed by the first path segment of their reference (everything before the first .), and a transition is matched only against the bucket for its own first segment, plus a residue list of sources that carry no separator at all (a prefix shorter than one segment, which can match other buckets). This is sound because a source reference containing a separator can only prefix a reference sharing its entire first segment. The per-source StartsWith test and the condition-type gate inside the bucket are unchanged, so routing decisions are identical to the previous linear scan over every subscribed source; the SnapshotComplete sentinel still bypasses the index entirely and is broadcast to every subscriber.
  • Gateway union filter (MxGateway): StreamAlarms carries a single alarm_filter_prefix, so the adapter opens the stream on the longest common prefix of the currently subscribed source references and restarts it only when a NEW source falls outside that prefix (the source then replays a fresh snapshot, which NativeAlarmActor already handles). An unsubscribe never restarts the stream — the prefix it leaves behind is at worst too broad, which costs bandwidth, not correctness. The prefix is a bandwidth optimisation only; the actor's per-source + condition-type gate remains authoritative, mirroring the OPC UA server-side WhereClause stance.

Protocol-Neutral Types & Messages

All defined in Commons so the feed is identical across protocols:

Type Shape
NativeAlarmTransition SourceReference, SourceObjectReference, AlarmTypeName, Kind, Condition, Category, Description, Message, OperatorUser, OperatorComment, OriginalRaiseTime?, TransitionTime, CurrentValue, LimitValue, AckTime?
AlarmConditionState Active, Acknowledged, Confirmed? (null when not confirmable), Shelve, Suppressed, Severity (0–1000)
AlarmTransitionKind (enum) Snapshot, SnapshotComplete, Raise, Acknowledge, Clear, Retrigger, StateChange

OperatorUser / OperatorComment and CurrentValue / LimitValue are display-only mirrors from the source.

Ack Timestamp (AckTime)

AckTime is the instant a condition was acknowledged, mirrored end-to-end (transition → AlarmStateChanged → AlarmStateUpdate field 24 → site native_alarm_state) so MES can report a real AckDT and the Alarms.CurrentAsync() script accessor can surface it. Added additively as a trailing optional parameter — every pre-existing positional construction still compiles and yields null.

One rule decides whether it is set: the condition must be BOTH active and acknowledged. That single predicate gives all three required behaviours — null while unacknowledged, cleared on re-raise (a re-raise transition arrives unacknowledged), and no phantom ack on a return-to-normal (which matters for MxGateway, where INACTIVE maps to Acknowledged = true).

Provenance differs by protocol, and the difference is deliberate — the value is never fabricated:

Source Ack instant used Accuracy
OPC UA A&C AcknowledgeableConditionType/AckedState/TransitionTime — SelectClause index 18, appended after the limit fields so indices 0–17 keep their meaning The source's own ack instant
OPC UA A&C, server omits the field The event's Time field When the DCL observed the acknowledged state
MxAccess Gateway The transition's own timestamp (TransitionTimestamp, or the DCL's receipt time when the gateway omits it); on a re-subscribe snapshot, LastTransitionTimestamp for an ACTIVE_ACKED entry When the system saw the ack — the gateway feed carries no dedicated ack timestamp

The decision lives in the pure mappers (OpcUaAlarmMapper.DeriveAckTime, MxGatewayAlarmMapper.DeriveAckTime), so it is unit-tested without a live server or gateway.

Design record: docs/plans/2026-06-30-mes-alarm-status-api.md §6.4.

Messages:

  • SubscribeAlarmsRequest / SubscribeAlarmsResponse — instance (via the DCL manager) subscribes a source binding to native alarms; the response carries success + an optional error message.
  • UnsubscribeAlarmsRequest — cancels a native alarm subscription for an instance + source.
  • NativeAlarmTransitionUpdate(ConnectionName, Transition) — DCL → instance: one routed transition (including snapshot replay).
  • NativeAlarmSourceUnavailable(ConnectionName, SourceReference, Timestamp) — DCL → instance: the feed for a source became unavailable (connection lost).

Value Update Message Format

Each value update delivered to an Instance Actor includes:

  • Tag path: The relative path of the attribute's data source reference.
  • Value: The new value from the device.
  • Quality: Data quality indicator (good, bad, uncertain).
  • Timestamp: When the value was read from the device.

Connection Actor Model

Each data connection is managed by a dedicated connection actor that uses the Akka.NET Become/Stash pattern to model its lifecycle as a state machine:

  • Connecting: The actor attempts to establish the connection. Subscription requests and write commands received during this phase are stashed (buffered in the actor's stash).
  • Connected: The actor is actively servicing subscriptions. On entering this state, all stashed messages are unstashed and processed.
  • Reconnecting: The connection was lost. The actor transitions back to a connecting-like state, stashing new requests while it retries.

This pattern ensures no messages are lost during connection transitions and is the standard Akka.NET approach for actors with I/O lifecycle dependencies.

OPC UA-specific notes: The RealOpcUaClient uses the OPC Foundation SDK's Session.KeepAlive event for proactive disconnect detection. The SDK sends keep-alive requests at the subscription's KeepAliveCount × PublishingInterval (default: 10s). When keep-alive fails, the ConnectionLost event fires, triggering the same reconnection flow. On reconnection, the DCL re-creates the OPC UA session and subscription, then re-adds all monitored items.

Connection Lifecycle & Reconnection

The DCL manages connection lifecycle automatically:

  1. Connection drop detection: When a connection to a data source is lost, the DCL immediately pushes a value update with quality bad for every tag subscribed on that connection. Instance Actors see the staleness immediately and publish it to the site stream / debug view; script and alarm actors are deliberately NOT re-notified on a quality-only change (no value changed — re-evaluating triggers would cause spurious firings). Scripts observe quality on their next attribute read; a computed alarm holds its last-known state until fresh values arrive.
  2. Auto-reconnect with fixed interval: The DCL retries the connection at a configurable fixed interval (e.g., every 5 seconds). The retry interval is a shared setting for all data connections (ReconnectInterval in the Shared Settings table). This is consistent with the fixed-interval retry philosophy used throughout the system. Individual gRPC/OPC UA operations (reads, writes) fail immediately to the caller on error; there is no operation-level retry within the adapter.
  3. Connection state transitions: The DCL tracks each connection's state as connected, disconnected, or reconnecting. All transitions are logged to Site Event Logging.
  4. Transparent re-subscribe: On successful reconnection, the DCL automatically re-establishes all previously active subscriptions for that connection. Instance Actors require no action — they simply see quality return to good as fresh values arrive from restored subscriptions.
  5. Clean client replacement on (re)connect: On every (re)connect the adapter detaches and disposes the previous protocol client (session, keep-alive timers, event loop) before creating a new one — a stale client can neither leak nor signal a disconnect against the new session.

Disconnect Detection Pattern

Each adapter implements the IDataConnection.Disconnected event to proactively signal connection loss to the DataConnectionActor. Detection uses two complementary paths:

Proactive detection (server goes offline between operations):

  • OPC UA: The OPC Foundation SDK fires Session.KeepAlive events at regular intervals. RealOpcUaClient hooks this event; when ServiceResult.IsBad(e.Status) (server unreachable, keep-alive timeout), it fires ConnectionLost. The OpcUaDataConnection adapter translates this into IDataConnection.Disconnected.

Reactive detection (failure discovered during an operation):

  • Both adapters wrap ReadAsync (and by extension ReadBatchAsync) with exception handling. If a read throws a non-cancellation exception, the adapter calls RaiseDisconnected() and re-throws. The DataConnectionActor's existing error handling catches the exception while the disconnect event triggers the reconnection state machine.

Event marshalling: The DataConnectionActor subscribes to _adapter.Disconnected in PreStart(). Since Disconnected may fire from a background thread (gRPC stream task, OPC UA keep-alive timer), the handler sends an AdapterDisconnected message to Self, marshalling the notification onto the actor's message loop. This triggers BecomeReconnecting() → bad quality push → retry timer.

Once-only guard: OpcUaDataConnection uses a volatile bool _disconnectFired flag to ensure RaiseDisconnected() fires exactly once per connection session. The flag resets on successful reconnection (ConnectAsync).

Write Failure Handling

Writes to physical devices are synchronous from the script's perspective:

  • If the write fails (connection down, device rejection, timeout), the error is returned to the calling script. Script authors can catch and handle write errors (log, notify, retry, etc.).
  • Write failures are also logged to Site Event Logging.
  • There is no store-and-forward for device writes — these are real-time control operations. Buffering stale setpoints for later application would be dangerous in an industrial context.

Tag Path Resolution

When the DCL subscribes to a tag path from the flattened configuration but the path does not exist on the physical device (e.g., typo in the template, device firmware changed, device still booting):

  1. The failure is logged to Site Event Logging.
  2. The attribute is marked with quality bad.
  3. The DCL retries resolution with exponential backoff, accommodating devices that come online in stages or load modules after startup: the interval starts at TagResolutionRetryInterval (10s), doubles after a round in which nothing resolved, and is capped at TagResolutionRetryMaxInterval (5m). It resets to the floor as soon as any tag resolves or the connection reconnects, so a device that is merely slow to boot is still picked up quickly while a dead one is not probed at full width forever. Each round probes the whole unresolved set in SubscribeBatchSize chunks, not one call per tag.
  4. The retry timer is single-shot, rescheduled on probe completion — a periodic timer cannot back off, and rescheduling only on completion is what keeps a fan-out of failures from resetting the clock (the anti-starvation property previously defended by an IsTimerActive gate, which is retained).
  5. On successful resolution, the subscription activates normally and quality reflects the live value from the device.

Note: Pre-deployment validation at central does not verify that tag paths resolve to real tags on physical devices — that is a runtime concern handled here.

Health Reporting

The DCL reports the following metrics to the Health Monitoring component via the existing periodic heartbeat:

  • Connection status: connected, disconnected, or reconnecting per data connection.
  • Tag resolution counts: Per connection, the number of total subscribed tags vs. successfully resolved tags. This gives operators visibility into misconfigured templates without needing to open the debug view for individual instances.
  • Tag quality counters are pushed on a genuine quality transition only, coalesced onto a QualityFlushInterval (1s) single-shot timer. Counter arithmetic still runs per message; only the collector push is deferred, and a value whose quality is unchanged moves no counter at all. Health reports poll at 30s, so the coalescing loses nothing. Three paths flush synchronously because their correctness depends on it: the bad-quality push on disconnect, unsubscribe, and the reconnect counter reset.

Dependencies

  • Site Runtime (Instance Actors): Receives subscription registrations and delivers value updates. Receives write requests.
  • Site Runtime (NativeAlarmActor): For alarm-subscribable connections, receives SubscribeAlarmsRequest/UnsubscribeAlarmsRequest and delivers NativeAlarmTransitionUpdate / NativeAlarmSourceUnavailable (read-only native alarm mirroring).
  • Health Monitoring: Reports connection status.
  • Site Event Logging: Logs connection status changes.

Interactions

  • Site Runtime (Instance Actors): Bidirectional — delivers value updates, receives subscription registrations and write-back commands.
  • Site Runtime (NativeAlarmActor): Bidirectional — receives alarm subscribe/unsubscribe requests, delivers native alarm transitions and source-unavailable notifications (read-only; no ack-back to the source).
  • Health Monitoring: Reports connection health periodically.
  • Site Event Logging: Logs connection/disconnection events.
Reference in New Issue View Git Blame Copy Permalink