Commit Graph

32 Commits

Author SHA1 Message Date
Joseph Doherty
c9b236e507 fix(data-connection): resolve DataConnectionLayer-006..012 — quality-counter reconciliation, per-tag batch reads, configurable failover threshold, dedup retry, stale-callback guard, secure cert default 2026-05-16 21:11:24 -04:00
Joseph Doherty
fccd3274d3 fix(data-connection-layer): resolve DataConnectionLayer-002/003/004/005 — Resume supervision, concurrent dicts, subscribe-failure classification, write timeout 2026-05-16 19:40:40 -04:00
Joseph Doherty
239bee3bc4 fix(data-connection): resolve DataConnectionLayer-001 — off-thread actor state mutation
HandleSubscribe spawned a Task.Run that mutated DataConnectionActor private
state (_subscriptionIds, _subscriptionsByInstance, _totalSubscribed,
_resolvedTags, _unresolvedTags) from a thread-pool thread, racing the actor's
own message loop — a data race on non-thread-safe Dictionary/HashSet and
non-atomic counters.

Restructured HandleSubscribe to follow the actor's existing PipeTo(Self)
pattern: the background task now performs only adapter I/O and pipes a
SubscribeCompleted message to Self; all subscription-state mutation happens
in the new HandleSubscribeCompleted handler on the actor thread (wired into
the Connected, Connecting and Reconnecting states).

Adds DCL001_ConcurrentSubscribes_DoNotCorruptSubscriptionCounters (30x30
concurrent subscribes) which fails against the pre-fix code and passes after.
2026-05-16 18:26:43 -04:00
Joseph Doherty
9c60592632 build: adopt NuGet Central Package Management
Move all package versions into Directory.Packages.props so every project
resolves a single consistent version. Consolidates the Roslyn packages
(Microsoft.CodeAnalysis.CSharp.Scripting/Workspaces) onto 5.0.0, which
resolves the pre-existing NU1608 version-skew error in the test projects.
2026-05-16 15:56:30 -04:00
Joseph Doherty
8faaa8fe2b feat(dcl): Layer D OpcUaGlobalOptions for app-wide identity + cert paths
New deployment-wide options bound from the "OpcUa" section of appsettings.json:
- ApplicationName (default "ScadaLink-DCL")
- TrustedIssuerStorePath / TrustedPeerStorePath / RejectedCertificateStorePath

Empty paths fall back to Path.GetTempPath()/ScadaLink/pki/* so dev runs work
without explicit config — same defaults the hardcoded values previously used.

Wiring:
- ServiceCollectionExtensions binds OpcUaGlobalOptions to the OpcUa section.
- DataConnectionFactory takes IOptions<OpcUaGlobalOptions> and constructs
  RealOpcUaClientFactory with the snapshot.
- RealOpcUaClient(globalOptions) replaces the hardcoded ApplicationName and
  the three CertificateTrustList store paths in ApplicationConfiguration.
- Parameterless ctors on factory and client preserved for the existing test
  suite (32/32 DCL tests still green).
2026-05-12 02:27:58 -04:00
Joseph Doherty
e6a5b558f3 feat(dcl): Layer C runtime wires new OPC UA settings through to OPC SDK
OpcUaConnectionOptions record gains DiscardOldest, SubscriptionPriority,
SubscriptionDisplayName, TimestampsToReturn, plus OpcUaDeadbandOptions and
OpcUaUserIdentityOptions nullable sub-records.

OpcUaDataConnection.ConnectAsync copies all new fields from the typed
OpcUaEndpointConfig (including the Deadband and UserIdentity sub-objects)
into the OpcUaConnectionOptions record.

RealOpcUaClient:
- BuildUserIdentity translates TokenType into Opc.Ua.UserIdentity:
  Anonymous → null, UsernamePassword → new UserIdentity(name, utf8(pass)),
  X509Certificate → new UserIdentity(X509CertificateLoader.LoadPkcs12FromFile(...)).
- Subscription uses opts.SubscriptionDisplayName and opts.SubscriptionPriority.
- MonitoredItem.DiscardOldest is opts.DiscardOldest (was hardcoded true).
- BuildDataChangeFilter materializes a DataChangeFilter when Deadband is set.
- ReadAsync uses MapTimestampsToReturn for opts.TimestampsToReturn (was hardcoded Source).

X509CertificateLoader replaces obsolete X509Certificate2(string,string) ctor
(SYSLIB0057 on .NET 10). UserIdentity(string,byte[]) ctor used because the
(string,string) overload was removed in OPC Foundation 1.5.378.106.
2026-05-12 02:26:15 -04:00
Joseph Doherty
f98d29fc36 refactor(dcl): OpcUaDataConnection uses OpcUaEndpointConfig via FromFlatDict 2026-05-12 00:57:09 -04:00
Joseph Doherty
ec1d8f1393 chore(deps): bump packages flagged by NU190x advisories
Restore inside the docker build was failing because TreatWarningsAsErrors
promotes NU1902/NU1903/NU1904 (vulnerable package warnings) to errors.
Bump the flagged packages to advisory-free versions:

- MailKit                                          4.15.1 -> 4.16.0    (GHSA-9j88-vvj5-vhgr)
- Microsoft.AspNetCore.DataProtection.EFCore       10.0.5 -> 10.0.7    (GHSA-9mv3-2cwr-p262, transitively pulls fixed System.Security.Cryptography.Xml — GHSA-37gx-xxp4-5rgx, GHSA-w3x6-4m5h-cxqf)
- OpenTelemetry.Api  (transitive via Akka.Hosting) 1.9.0  -> 1.15.3    (GHSA-g94r-2vxg-569j, GHSA-8785-wc3w-h8q6) — added as a direct PackageReference in ScadaLink.Host to override the Akka.Hosting pin

To resolve the NU1605 downgrade chain triggered by DataProtection.EFCore
10.0.7 (which transitively requires Microsoft.EntityFrameworkCore >= 10.0.7
and friends), bump every Microsoft.* 10.0.5 reference across src/ and
tests/ to 10.0.7 in lockstep.
2026-05-08 09:34:17 -04:00
Joseph Doherty
9dccf8e72f deprecate(lmxproxy): move all LmxProxy code, tests, and docs to deprecated/
LmxProxy is no longer needed. Moved the entire lmxproxy/ workspace, DCL
adapter files, and related docs to deprecated/. Removed LmxProxy registration
from DataConnectionFactory, project reference from DCL, protocol option from
UI, and cleaned up all requirement docs.
2026-04-08 15:56:23 -04:00
Joseph Doherty
de7c4067e4 feat(dcl): add debug-level logging for heartbeat subscription callbacks 2026-03-24 16:19:39 -04:00
Joseph Doherty
5fdeaf613f feat(dcl): failover on repeated unstable connections (connect-then-stale pattern)
Previously, failover only triggered when ConnectAsync failed consecutively.
If a connection succeeded but went stale quickly (e.g., heartbeat timeout),
the failure counter reset on each successful connect and failover never
triggered.

Added a separate _consecutiveUnstableDisconnects counter that increments
when a connection lasts less than StableConnectionThreshold (60s) before
disconnecting. When this counter reaches failoverRetryCount, the actor
fails over to the backup endpoint. Stable connections (lasting >60s)
reset this counter.

The original connection-failure failover path is unchanged.
2026-03-24 16:19:39 -04:00
Joseph Doherty
0d03aec4f2 feat(dcl): log connection disconnect events to site event log 2026-03-24 16:19:39 -04:00
Joseph Doherty
d4397910f0 feat(dcl): add StaleTagMonitor for heartbeat-based disconnect detection
Composable StaleTagMonitor class in Commons fires a Stale event when no
value is received within a configurable max silence period. Integrated
into both LmxProxyDataConnection and OpcUaDataConnection adapters via
optional HeartbeatTagPath/HeartbeatMaxSilence connection config keys.
When stale, the adapter fires Disconnected triggering the standard
reconnect cycle. 10 unit tests cover timer behavior.
2026-03-24 16:19:39 -04:00
Joseph Doherty
65cc7b69cd feat(health): wire up NodeHostname, ConnectionEndpoint, TagQuality, ParkedMessageCount collectors
- AkkaHostedService: SetNodeHostname from NodeOptions
- DataConnectionActor: UpdateConnectionEndpoint on state transitions,
  track per-tag quality counts and UpdateTagQuality on value changes
- HealthReportSender: query StoreAndForwardStorage for parked message count
- StoreAndForwardStorage: add GetParkedMessageCountAsync()
2026-03-24 16:19:39 -04:00
Joseph Doherty
ecf4b434c2 refactor(dcl): simplify ValueFormatter now that SDK returns native .NET arrays
The LmxProxy client's ExtractArrayValue now returns proper .NET arrays
(bool[], int[], DateTime[], etc.) instead of ArrayValue objects. Removed
the reflection-based FormatArrayContainer logic — IEnumerable handling
is sufficient for all array types.
2026-03-22 15:15:38 -04:00
Joseph Doherty
dcdf79afdc fix(dcl): format ArrayValue objects as comma-separated strings for display
ArrayValue from LmxProxy client was showing as type name in debug views.
Added ValueFormatter utility and NormalizeValue in LmxProxyDataConnection
to convert arrays at the adapter boundary. DateTime arrays remain as
"System.DateTime[]" due to server-side v1 string serialization.
2026-03-22 14:46:15 -04:00
Joseph Doherty
801c0c1df2 feat(dcl): add active endpoint to health reports and log failover events
Add ActiveEndpoint field to DataConnectionHealthReport showing which
endpoint is active (Primary, Backup, or Primary with no backup configured).
Log failover transitions and connection restoration events to the site
event log via ISiteEventLogger, passed as an optional parameter through
the actor hierarchy for backwards compatibility.
2026-03-22 08:34:05 -04:00
Joseph Doherty
da290fa4f8 feat(dcl): add failover state machine to DataConnectionActor with round-robin endpoint switching 2026-03-22 08:30:03 -04:00
Joseph Doherty
46304678da feat(dcl): extend CreateConnectionCommand with backup config and failover retry count
Update CreateConnectionCommand to carry PrimaryConnectionDetails,
BackupConnectionDetails, and FailoverRetryCount. Update all callers:
DataConnectionManagerActor, DataConnectionActor, DeploymentManagerActor,
FlatteningService, and ConnectionConfig. The actor stores both configs
but continues using primary only — failover logic comes in Task 3.
2026-03-22 08:24:39 -04:00
Joseph Doherty
5ec7f35150 feat(dcl): replace hand-rolled LmxProxy gRPC client with real LmxProxyClient library
Switches from v1 string-based proto stubs to the production LmxProxyClient
(v2 native TypedValue protocol) via project reference. Deletes 6k+ lines of
generated proto code. Preserves ILmxProxyClient adapter interface for testability.
2026-03-22 07:55:50 -04:00
Joseph Doherty
1a540f4f0a feat: add HTTP Management API, migrate CLI from Akka ClusterClient to HTTP
Replace the CLI's Akka.NET ClusterClient transport with a simple HTTP client
targeting a new POST /management endpoint on the Central Host. The endpoint
handles Basic Auth, LDAP authentication, role resolution, and ManagementActor
dispatch in a single round-trip — eliminating the CLI's Akka, LDAP, and
Security dependencies.

Also fixes DCL ReSubscribeAll losing subscriptions on repeated reconnect by
deriving the tag list from _subscriptionsByInstance instead of _subscriptionIds.
2026-03-20 23:55:31 -04:00
Joseph Doherty
7740a3bcf9 feat: add JoeAppEngine OPC UA nodes, fix DCL auto-reconnect and quality push
- Add JoeAppEngine folder to OPC UA nodes.json (BTCS, AlarmCntsBySeverity, Scheduler/ScanTime)
- Fix DataConnectionActor: capture Self in PreStart for use from non-actor threads,
  preventing Self.Tell failure in Disconnected event handler
- Implement InstanceActor.HandleConnectionQualityChanged to mark attributes Bad on disconnect
- Fix LmxFakeProxy TagMapper to serialize arrays as JSON instead of "System.Int32[]"
- Allow DataType and DataSourceReference updates in TemplateService.UpdateAttributeAsync
- Update test_infra_opcua.md with JoeAppEngine documentation
2026-03-19 13:27:54 -04:00
Joseph Doherty
e837eae2cc feat: wire real LmxProxy gRPC client into Data Connection Layer
Replace stub ILmxProxyClient with production proto-generated gRPC client
(RealLmxProxyClient) that connects to LmxProxy servers with x-api-key
metadata header authentication. Includes pre-generated proto stubs for
ARM64 Docker compatibility, updated adapter with proper quality mapping
(Good/Uncertain/Bad), subscription via server-streaming RPC, and 20 unit
tests covering all operations. Updated Component-DataConnectionLayer.md
to reflect the actual implementation.
2026-03-18 11:57:18 -04:00
Joseph Doherty
f063fb1ca3 fix: wire DCL tag value delivery, alarm evaluation, and snapshot timestamps
Three runtime bugs fixed:
- DataConnectionActor: TagValueReceived/TagResolutionSucceeded/Failed not
  handled in any Become state — OPC UA values went to dead letters. Added
  initial read after subscribe to seed current values immediately.
- AlarmActor: ParseEvalConfig expected "attributeName"/"matchValue"/"min"/
  "max" keys but seed data uses "attribute"/"value"/"high"/"low". Added
  support for both conventions and !=prefix for not-equal matching.
- InstanceActor: snapshots reported all alarms (including unevaluated) with
  correct priorities and source timestamps instead of current UTC. Removed
  bogus Vibration template attribute that shadowed Speed's tag mapping.
2026-03-18 07:36:48 -04:00
Joseph Doherty
75a6636a2c fix: wire DCL connection state changes into ISiteHealthCollector
DataConnectionActor now calls UpdateConnectionHealth() on state
transitions (Connecting/Connected/Reconnecting) and UpdateTagResolution()
on connection establishment. DataConnectionManagerActor calls
RemoveConnection() on actor removal. Health reports now include
data connection statuses when instances are deployed with bindings.
2026-03-18 00:20:02 -04:00
Joseph Doherty
1b06a4971e Fix OPC UA adapter: pass connection details, certificate stores, endpoint discovery
- DataConnectionActor now stores and passes connection details to adapter ConnectAsync
- DataConnectionManagerActor passes connection details when creating actor
- RealOpcUaClient uses DiscoveryClient for endpoint selection with no-security preference
- Added certificate trust store paths to prevent TrustedIssuerCertificates error
- Sanitize connection names for Akka actor paths (replace spaces)
2026-03-17 12:19:44 -04:00
Joseph Doherty
8e1d0816b3 Complete OPC UA data flow: binding UI, flattening connections, real OPC UA client
- Add connection binding UI to Instances page (per-attribute and bulk assign)
- FlatteningService populates Connections dict from bound data connections
- Real OPC UA client using OPC Foundation SDK for live tag subscriptions
- DataConnectionFactory uses RealOpcUaClientFactory by default
- OpcUaDataConnection supports both "endpoint" and "EndpointUrl" config keys
2026-03-17 11:40:39 -04:00
Joseph Doherty
dfb809a909 Wire DCL to Instance Actors for OPC UA tag value flow
- Add TagValueUpdate/ConnectionQualityChanged handlers to InstanceActor
- InstanceActor subscribes to DCL on PreStart based on DataSourceReference
- DeploymentManagerActor creates DCL connections on deploy and passes DCL ref
- AkkaHostedService creates DCL Manager Actor for tag subscriptions
- Move CreateConnectionCommand to Commons for cross-project access
- Add ConnectionConfig to FlattenedConfiguration for deployment packaging
2026-03-17 11:21:11 -04:00
Joseph Doherty
389f5a0378 Phase 3B: Site I/O & Observability — Communication, DCL, Script/Alarm actors, Health, Event Logging
Communication Layer (WP-1–5):
- 8 message patterns with correlation IDs, per-pattern timeouts
- Central/Site communication actors, transport heartbeat config
- Connection failure handling (no central buffering, debug streams killed)

Data Connection Layer (WP-6–14, WP-34):
- Connection actor with Become/Stash lifecycle (Connecting/Connected/Reconnecting)
- OPC UA + LmxProxy adapters behind IDataConnection
- Auto-reconnect, bad quality propagation, transparent re-subscribe
- Write-back, tag path resolution with retry, health reporting
- Protocol extensibility via DataConnectionFactory

Site Runtime (WP-15–25, WP-32–33):
- ScriptActor/ScriptExecutionActor (triggers, concurrent execution, blocking I/O dispatcher)
- AlarmActor/AlarmExecutionActor (ValueMatch/RangeViolation/RateOfChange, in-memory state)
- SharedScriptLibrary (inline execution), ScriptRuntimeContext (API)
- ScriptCompilationService (Roslyn, forbidden API enforcement, execution timeout)
- Recursion limit (default 10), call direction enforcement
- SiteStreamManager (per-subscriber bounded buffers, fire-and-forget)
- Debug view backend (snapshot + stream), concurrency serialization
- Local artifact storage (4 SQLite tables)

Health Monitoring (WP-26–28):
- SiteHealthCollector (thread-safe counters, connection state)
- HealthReportSender (30s interval, monotonic sequence numbers)
- CentralHealthAggregator (offline detection 60s, online recovery)

Site Event Logging (WP-29–31):
- SiteEventLogger (SQLite, 6 event categories, ISO 8601 UTC)
- EventLogPurgeService (30-day retention, 1GB cap)
- EventLogQueryService (filters, keyword search, keyset pagination)

541 tests pass, zero warnings.
2026-03-16 20:57:25 -04:00
Joseph Doherty
8c2091dc0a Phase 0 WP-0.10–0.12: Host skeleton, options classes, sample configs, and execution framework
- WP-0.10: Role-based Host startup (Central=WebApplication, Site=generic Host),
  15 component AddXxx() extension methods, MapCentralUI/MapInboundAPI stubs
- WP-0.11: 12 per-component options classes with config binding
- WP-0.12: Sample appsettings for central and site topologies
- Add execution procedure and checklist template to generate_plans.md
- Add phase-0-checklist.md for execution tracking
- Resolve all 21 open questions from plan generation
- Update IDataConnection with batch ops and IAsyncDisposable
57 tests pass, zero warnings.
2026-03-16 18:59:07 -04:00
Joseph Doherty
fed5f5a82c Add .gitignore and remove tracked build artifacts (bin/obj) 2026-03-16 18:38:00 -04:00
Joseph Doherty
34190e1347 Phase 0 WP-0.1: Create .NET 10 solution structure with all 17 component projects
17 source projects (Commons + Host + 15 components) and 17 xUnit test projects.
SLNX format, net10.0, nullable enabled, warnings as errors. All components
reference Commons; Host references all components. Builds and tests clean.
2026-03-16 18:37:36 -04:00