17aea09fca2fd220e6e8bb82c5e57bd87b4dfd03
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
428b8db024 |
test(dcl+communication+storeandforward+kpihistory+host): close register row-40-class sleeps + row-32 provider leak (arch-review 2026-08-19 P8-8 + P8-9)
P8-8: replace the remaining wall-clock sleep-then-assert instances outside Communication.Tests' Medium-wave sweep with the same row-40 house pattern — continuous polling (AssertHoldsFor) for ABSENCE assertions instead of a single end-of-window sample, which cannot distinguish "stayed quiet throughout" from "hadn't happened yet when the one sample landed" and cannot catch a value that flips wrong and back before that sample. - DataConnectionActorBatchTests: QualityCounters_ArePushedOnTransitionOnly_AndCoalesced (:206/:214 class) converted to AssertHoldsFor; ResolutionProbes_BackOff... (:181/:184, a genuine real-time backoff-RATE assertion, not a sync-primitive stand-in) converted to AssertRoundsWithinWindow, which enforces the ceiling continuously (fails fast on a no-backoff regression) while the floor still necessarily waits out the window. - SiteAlarmLiveCacheServiceTests (:452-454, :470-472): converted to the project's own AssertHoldsFor (mirrors SiteAlarmAggregatorActorTests', same assembly). - StoreAndForwardServiceTests (:891-892): async AssertHoldsForAsync variant (the class runs on the async test thread). - KpiHistoryRecorderActorTests (:1043-1049): two skip-path FoldCount==0 absence checks converted. Each helper is a private per-file copy, matching the Medium-wave T17 precedent (SiteAlarmAggregatorActorTests.AssertHoldsFor) rather than a shared TestSupport utility. Proof of the mechanism: a standalone (non-repo) harness run 100 trials of a real background thread flipping a boolean true then back to false inside a 300ms window — Thread.Sleep(300)+one sample missed the violation 100/100 times (false pass), continuous 20ms polling caught it 100/100 times. Most of the actual conversions here guard monotonic counters (their single-sample variant would eventually see the final value too), so each converted test carries an inline honesty note on what specifically the conversion buys for that assertion (fail-fast vs. true flip-back detection). P8-9: ScadaBridgeTelemetryTests.EmitHelpers_DoNotThrow called SetQueueDepthProvider(() => 5) against the process-global row-32 provider slot with no clear and no [Collection] — a landmine for any future same-process reader of scadabridge.store_and_forward.queue.depth. Fixed with compare-and-clear in a finally via the existing (production, untouched) ClearQueueDepthProvider. Searched this assembly and the whole test tree for a second same-process reader that asserts on the value: none exists today (MetricsEndpointTests scrapes /metrics but only asserts on the response shape, and the only value-asserting test, QueueDepthGaugeTests, lives in StoreAndForward.Tests — a separate test-host process the leak cannot reach), so no [Collection] was added; the test's doc comment records the search and flags it for re-check if that changes. Test-only change, zero production edits. Full suites green: DCL.Tests 342/342, Communication.Tests 729/729, StoreAndForward.Tests 135/135, KpiHistory.Tests 54/54, Host.Tests 522/522. Each converted test additionally looped 20x in isolation with zero flakes. |
||
|
Joseph Doherty
|
6823e7e3fa |
fix(dcl): P2-5 seed probe-resolved tags; P2-6 close the Connecting/Reconnecting AdapterDisconnected gap
P2-5 — the tag-resolution PROBE was the only subscribe path that did not seed. HandleSubscribe carries seeds back on SubscribeCompleted and ReSubscribeAll re-seeds after a reconnect, because a fresh advise only yields a value on the source's next CHANGE. HandleRetryTagResolution did neither, so a tag that resolved LATE (device still booting, item created after deployment) and then never changed had no value path at all. OPC UA masks it with the initial data-change notification on subscription creation; an MxGateway advise can legitimately return VT_EMPTY, so a static tag stayed Uncertain for the life of the connection. New SeedNewlyResolvedTags reads the round's newly resolved tags and delivers them through the normal generation-guarded TagValueReceived path — directly rather than on the completion message (the probe only covers tags already registered in _instancesByTag, so the initial subscribe's ordering hazard does not apply), and as its own fire-and-forget task so the seed reads never delay the BatchSubscribeCompleted that re-arms the retry timer. P2-6 — only the Connected state handled AdapterDisconnected; in Connecting and Reconnecting it fell through to Unhandled. A connect that succeeds and drops again before its piped ConnectResult is applied therefore delivered the disconnect FIRST, where it was discarded, and ConnectResult(true) then declared the connection up on a dead adapter. An adapter raises Disconnected at most once per successful connect (OpcUaDataConnection's once-only _disconnectFired, reset only by the next successful ConnectAsync), so a connection with no subscriptions — nothing to fail at connection level and drive the reconnect machine — sat Connected on a dead adapter until the process restarted. Both states now record the disconnect in _disconnectDuringConnectAttempt (cleared at the top of HandleAttemptConnect, so it only ever refers to the attempt in flight) and both result handlers demote such a "success" to the ordinary connect-FAILURE tail — counting toward the failover budget and re-arming the retry timer. A flag rather than Stash: the stash has no selective drop, so a stashed disconnect would be replayed after the next successful connect and cost a spurious reconnect cycle. AdapterDisconnected is now stamped with the adapter generation (hooked via a per-adapter closure instead of a method group; SwapAdapter hooks AFTER the bump), and IsStaleDisconnect drops a notification from a swapped-out adapter in all three states — the disconnect twin of IsStaleConnectResult. SwapAdapter's unhook cannot stop an event already executing on a background thread, so without the fence an endpoint the operator has just reconfigured AWAY from could knock the healthy replacement into a reconnect. Tests: DataConnectionProbeSeedTests + DataConnectionDisconnectRaceTests (4 new). The race is reproduced by ORDERING, not timing — the connect is held on a gate, Disconnected is raised while the actor sits idle, and a GetHealthReport round trip is the mailbox barrier; no sleeps. All 4 were proved to fail against the pre-fix behaviour via a targeted revert of the four behaviour sites (the stale-generation test cannot be proved by a plain revert — the record carried no generation before this change, so the test would not compile; noted in its doc comment). Full DCL suite 346/346 green. |
||
|
Joseph Doherty
|
26ab6f8f09 |
fix(dcl): generation-fence ConnectResult + roll back failed OPC UA subscribe batches (arch-review 2026-08-19 P2-3 + P2-4)
P2-3. ConnectResult was the last piped completion with no adapter generation on
it, so a connect launched against generation N and answered after the adapter had
been swapped (failover, or the row-97 in-place reconfigure) was applied to its
REPLACEMENT. A stale success drove ReSubscribeAll + BecomeConnected against an
adapter that had never connected, and the replacement's own result then landed in
the Connected state where nothing handles it. With a bad new endpoint the batch
subscribes failed at connection level with a log line only, no AdapterDisconnected
ever fired, and the idempotent-reconfigure guard refused a re-send of the same
configuration — only a restart recovered the connection. A stale FAILURE was as
wrong the other way: it counted toward failover and re-armed the reconnect timer
underneath the attempt genuinely in flight.
HandleAttemptConnect now stamps the generation and both handlers open with the
same guard the subscribe leg got in
|
||
|
Joseph Doherty
|
c4d3c8b875 |
fix(dcl): carry adapter generation on SubscribeCompleted/AlarmSubscribeCompleted; drop stale completions and release captured-adapter handles (arch-review 2026-08-19 P2-1 High)
An in-flight subscribe launched against adapter generation N can take up to SeedOverallTimeout (120s) to answer. If the adapter is swapped in that window (failover, or the row-97 in-place reconfigure), the completion was applied unconditionally: it stored a dead session's handle under the tag key, so the LIVE completion for the same tag read as a duplicate and had its working monitored item released (tag resolved forever, no data); it re-stamped the old endpoint's seed values with the CURRENT generation, bypassing the stale-data guard; and a connection-level failure row from the dead adapter knocked the healthy new connection back into Reconnecting. The alarm twin released the new live alarm feed the same way. Both completions now carry the generation AND the adapter instance they ran against, and both handlers drop a completion whose generation no longer matches - no handle store, no seed application, no per-tag or connection-level failure processing, no duplicate-release logic. Handles the dropped completion did create are released best-effort against the CAPTURED adapter (handles are adapter-scoped; MxGateway's are plain ints that collide across sessions), and the existing release sites in both handlers were switched to the message's adapter for the same reason. The in-flight markers are deliberately left alone (ReSubscribeAll/ReSubscribeAllAlarms clear them wholesale and may already have re-registered the tag/source), and the requester gets a failure reply so its retry timer re-sends. Tests: DataConnectionStaleCompletionTests - stale handle not stored / live item not released, release lands on the old adapter only, seed values not delivered, stale connection-level failure does not re-enter Reconnecting, stale alarm completion does not release the live feed. All five fail against the unfixed handlers. |
||
|
Joseph Doherty
|
7c1268d85c |
feat(dcl+siteruntime): reconfigure a LIVE data connection in place — register row 97
`DataConnectionManagerActor.HandleCreateConnection` is create-only (warns and
returns for a name it already holds), so `EnsureDclConnection`'s documented
"a changed config re-issues the create so the DCL adopts it" was never true on
ANY node: a corrected endpoint/credential/backup/failover-count took effect only
when the connection was next built from scratch (node restart, singleton
failover, remove-then-re-add), while the DM's hash cache recorded it as applied.
Rows 89/90 made that honest with a Warning; this closes it.
New `ReconfigureConnectionCommand` (Commons, same shape as the create):
- `DeploymentManagerActor.EnsureDclConnection` now has three outcomes off its one
per-name hash slot — unchanged -> nothing at all, new -> create, CHANGED ->
reconfigure — and logs "Reconfigured DCL connection {Connection}
(protocol={Protocol}) in place" at Information in place of the old Warning. The
winner-resolution / arbitration / tombstone logic is untouched; only what
happens after a change is detected changed.
- `DataConnectionManagerActor` FORWARDS the command to the owning connection
actor, falling back to a create for a name it does not hold, so the command is
safe whatever the manager's state.
- `DataConnectionActor` handles it in ALL THREE states and never stashes it — a
connection pointed at a wrong or dead endpoint lives in Connecting/Reconnecting,
which is exactly when a correction arrives; stashed it would wait on the broken
configuration forever. Handling replaces protocol/primary/backup/failover-count,
re-arms endpoint selection to Primary (the new config's primary is the intent,
whatever the old config had failed over to), and calls the new shared
`SwapAdapter` helper — now used by both failover sites too, so the swap exists
once: unhook + fire-and-forget dispose -> _factory.Create -> rewire -> bump
_adapterGeneration LAST. `BecomeReconnecting(afterReconfigure: true)` then
reconnects immediately (StartSingleTimer replaces any armed reconnect, so no
racing attempts) with the unstable-disconnect accounting and failover check
suppressed — the link was dropped on purpose, and counting it as an unstable
cycle could trip a failover onto the NEW backup before the new primary had been
tried once. ReSubscribeAll/ReSubscribeAllAlarms then re-issue everything from
the untouched subscription registries. Idempotent: a reconfigure carrying the
configuration already in force is a no-op.
Stop-and-recreate was rejected as destructive: subscriptions live inside
DataConnectionActor and InstanceActor cancels its subscribe-retry timer on first
success, so a rebuilt actor would have no subscribers and nothing would ever
re-subscribe.
Tests: new DataConnectionReconfigureTests (7; 5 fail / 2 pass against a
behavioural revert). DclConnectionArbitrationTests gains two row-97 tests and its
CDC-arrival test now asserts the second push is a reconfigure; its ping-pong
guard also asserts zero reconfigures, since a flip would now show up there.
SiteRuntime-010 renamed to ..._ReissuesReconfigureCommand. 3 fail against a
reverted EnsureDclConnection. DataConnectionLayer.Tests 327/327,
SiteRuntime.Tests 643/643, solution build 0 warnings.
Docs: Component-DataConnectionLayer.md gains an "In-place reconfiguration"
section under Endpoint Redundancy; Component-SiteRuntime.md, and the
docs/components/ mirrors, updated; the arbitration decision memo gains an
appended §9 postscript resolving its residual 1 (residuals 2 and 3 stay open).
|
||
|
Joseph Doherty
|
491df111ea |
fix(dcl): derive tag-resolution health counts from per-tag authoritative state
Closes arch-review remediation residual #1 (DCL unsubscribe-during-reconnect
count staleness).
DataConnectionActor tracked TotalSubscribedTags/ResolvedTags as two int fields
incremented and decremented at five independent sites. ReSubscribeAll clears the
very maps those decrements key off (_subscriptionIds, _unresolvedTags) while
deliberately preserving _subscriptionsByInstance, so an unsubscribe landing
inside a reconnect window matched NEITHER decrement branch: the total leaked +1
per subscribe/reconnect/unsubscribe churn cycle, permanently and cumulatively.
The
|
||
|
Joseph Doherty
|
37f13e2eaa | fix(dcl): discard in-flight subscribe results for unsubscribed tags; release the orphaned handle | ||
|
Joseph Doherty
|
d15c5f02ea | perf(dcl): batch subscribe/read/write seam, bounded reconnect, sharded subscriptions | ||
|
Joseph Doherty
|
74dd26eebd | feat: OPC UA address-space search plumbing — actor + comm + BrowseService (T15) | ||
|
Joseph Doherty
|
9ec2450ad5 | feat: thread BrowseNext continuation token through actor + BrowseService (T15) | ||
|
Joseph Doherty
|
2cfe0de927 | feat(dcl): BrowseNext continuation paging + StubOpcUaClient canned browse (T15) | ||
|
Joseph Doherty
|
4b6ff49822 |
fix(dcl+centralui): MxGateway tag browse — lazy attributes, frame-size cap, wider scrollable picker
Expanding a Galaxy object in the tag picker hung on "loading…": the browse reply inlined every child's full attribute set (~152 KB), exceeding Akka's 128 KB remote frame, and remoting silently discarded the oversized reply. Browse path (DataConnectionLayer): - RealMxGatewayClient: navigation now uses BrowseChildren(include_attributes= false) — child objects only — and an object's own attributes load lazily via DiscoverHierarchy(root, max_depth=0) when it's expanded. Payload drops from ~152 KB/level to a few KB. Seam contract unchanged. - DataConnectionActor.CapBrowseChildren: protocol-agnostic byte-budget cap (~100 KB) on every BrowseNodeResult before it crosses the site→central frame, OR-ing the adapter's own Truncated flag. Byte budget, not a count — the only bound that holds regardless of NodeId/attribute-name length. - RealOpcUaClient: requestedMaxReferencesPerNode 1000 → 500 to narrow the window before the byte budget applies. - Graceful gRPC Unimplemented handling → NotSupportedException → BrowseFailureKind.NotBrowsable with an actionable message (older gateway builds lacking BrowseChildren). Picker UI (CentralUI): - NodeBrowserDialog: modal-lg → modal-xl; new scoped .razor.css caps the tree at 55vh with its own scrollbar so manual entry + Select/Cancel stay visible. - Protocol-agnostic failure messages (was hardcoded "OPC UA …"); renamed the leftover opcua-browser-tree class to node-browser-tree. Tests: new frame-budget cap test + NotSupported=>NotBrowsable mapping test; DCL suite 88/88. Doc: Component-DataConnectionLayer.md records the lazy attribute-light browse and the frame-size guard. |
||
|
Joseph Doherty
|
9b7916bb2e |
refactor(browse): rename BrowseOpcUaNode* to protocol-agnostic BrowseNode*
Renames BrowseOpcUaNodeCommand/Result -> BrowseNodeCommand/Result and CommunicationService.BrowseOpcUaNodeAsync -> BrowseNodeAsync across Commons, Communication, SiteRuntime, DCL actors, and CentralUI. Wire manifest name follows (BrowseOpcUaNode -> BrowseNode). Browse regression tests green. |
||
|
Joseph Doherty
|
2a7dee4afa |
feat(centralui+dcl): Test Bindings popup — one-shot live read of bound tags
Adds a Test Bindings button to the Connection Bindings table on the Configure
Instance page that opens a modal showing the live current value of every bound
attribute. Reuses the routing path that the OPC UA tag browser landed on:
Central: TestBindingsDialog → IBindingTester → CommunicationService
→ ReadTagValuesCommand → SiteEnvelope (Ask)
Site: SiteCommunicationActor → DeploymentManagerActor singleton
→ DataConnectionManagerActor → child DataConnectionActor
→ _adapter.ReadBatchAsync
Split mirrors the browse handler:
• Manager owns ConnectionNotFound (only it sees the per-site connection set).
• Child owns ConnectionNotConnected (pre-call status check, never stash —
read is interactive design-time), Timeout (OperationCanceledException),
ServerError (any other exception). Per-tag failures from ReadBatchAsync
become failure TagReadOutcomes without aborting the batch.
CentralUI:
• IBindingTester / BindingTester — Design-role guard via HasClaim against
JwtTokenService.RoleClaimType (not IsInRole — see
|
||
|
Joseph Doherty
|
d285174597 |
feat(dcl+ui): rename BrowseOpcUaNode -> ConnectionName-keyed; implement site handler + dialog failure mapping
- BrowseOpcUaNodeCommand: int DataConnectionId -> string ConnectionName (site DataConnectionManagerActor indexes children by name; CentralUI already has the connection name in scope via the dropdown — no extra plumbing across the trust boundary). - IOpcUaBrowseService / OpcUaBrowseService: parameter renamed accordingly. - OpcUaBrowserDialog: collapse the duplicate ConnectionName parameters (display label and routing key are the same string). - Task 10: DataConnectionManagerActor forwards BrowseOpcUaNodeCommand to its child by name (owns ConnectionNotFound); DataConnectionActor adds the receive across all three lifecycle states (Connecting / Connected / Reconnecting) and maps adapter outcomes to BrowseFailureKind (NotBrowsable / ConnectionNotConnected / Timeout / ServerError). - Task 17: SetFailure in OpcUaBrowserDialog implements the full BrowseFailureKind switch with friendly UI messages. - Tests: DataConnectionManagerBrowseHandlerTests covers ConnectionNotFound, NotBrowsable, success, and ConnectionNotConnectedException paths. |