dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

DB-less ServiceLevel does not age stale role snapshots without incoming messages #721

New Issue
Closed
opened 2026-10-13 13:38:40 -04:00 by dohertj2 · 6 comments
dohertj2 commented 2026-10-13 13:38:40 -04:00
Owner
Copy Link
Copy Source

DB-less driver nodes can keep advertising their last healthy ServiceLevel after the stored redundancy snapshot becomes stale if no further snapshot or matching peer-probe result arrives.

Source evidence at PR #720 head c8e3e279b4 (no runtime reproduction claimed):

  • Runtime/ServiceCollectionExtensions.cs:424–435 omits DbHealthProbe when dbFactory is null; :559–567 constructs OpcUaPublishActor with dbHealthProbe=null and dbLess=true.
  • Runtime/OpcUa/OpcUaPublishActor.cs:348–365 starts its periodic HealthTick only when a DB probe exists. The tick currently unconditionally asks that probe (:1013–1021).
  • DB-less calculation checks snapshot age against the stale window (:1082–1091), but recomputation occurs only on incoming snapshot, DB sample or matching peer-probe result (:994–1036). No independent expiry trigger exists for this production path.

A single surviving DB-less node receiving one healthy Primary snapshot can therefore retain its last healthy advertised level if the snapshot source and matching probes cease. The existing calculation intends stale snapshots to demote to 200, but elapsed time alone does not invoke it. Peer-verdict expiry also needs timely reevaluation under the existing policy.

Acceptance:

  • Reproduce the DB-less no-probe/no-new-message expiry deterministically, including healthy initial publication and stale-window crossing.
  • Recompute freshness periodically without asking a null DB probe; verify later fresh snapshots recover.
  • Preserve existing DB-backed timeout behavior, existing health arithmetic, election and mutation policies. Do not silently reuse the device-write pre-join allowance as client-publication authority.
  • Verify missing/local-detached snapshots and actor shutdown/timer cleanup. Avoid live device calls or sleep-based timing regressions where supported virtual time/clock seams can be used.

This is independent of #719 cache/binding recovery and #717 Warm policy. Neither an implementation nor a new staleness/readiness policy is approved yet. Read-only audit evidence retained in the ongoing issue-sweep workspace.

DB-less driver nodes can keep advertising their last healthy ServiceLevel after the stored redundancy snapshot becomes stale if no further snapshot or matching peer-probe result arrives. Source evidence at PR #720 head c8e3e279b4f9f1e8d420bb8adf83c5428e1b1b90 (no runtime reproduction claimed): - Runtime/ServiceCollectionExtensions.cs:424–435 omits DbHealthProbe when dbFactory is null; :559–567 constructs OpcUaPublishActor with dbHealthProbe=null and dbLess=true. - Runtime/OpcUa/OpcUaPublishActor.cs:348–365 starts its periodic HealthTick only when a DB probe exists. The tick currently unconditionally asks that probe (:1013–1021). - DB-less calculation checks snapshot age against the stale window (:1082–1091), but recomputation occurs only on incoming snapshot, DB sample or matching peer-probe result (:994–1036). No independent expiry trigger exists for this production path. A single surviving DB-less node receiving one healthy Primary snapshot can therefore retain its last healthy advertised level if the snapshot source and matching probes cease. The existing calculation intends stale snapshots to demote to 200, but elapsed time alone does not invoke it. Peer-verdict expiry also needs timely reevaluation under the existing policy. Acceptance: - Reproduce the DB-less no-probe/no-new-message expiry deterministically, including healthy initial publication and stale-window crossing. - Recompute freshness periodically without asking a null DB probe; verify later fresh snapshots recover. - Preserve existing DB-backed timeout behavior, existing health arithmetic, election and mutation policies. Do not silently reuse the device-write pre-join allowance as client-publication authority. - Verify missing/local-detached snapshots and actor shutdown/timer cleanup. Avoid live device calls or sleep-based timing regressions where supported virtual time/clock seams can be used. This is independent of #719 cache/binding recovery and #717 Warm policy. Neither an implementation nor a new staleness/readiness policy is approved yet. Read-only audit evidence retained in the ongoing issue-sweep workspace.
dohertj2 commented 2026-10-13 14:30:25 -04:00
Author
Owner
Copy Link
Copy Source

The approved fix is implemented and committed locally as 1e71a25e; it starts periodic freshness recomputation for DB-less nodes without asking a null DB probe. The existing calculator, DB-backed Ask/timeout, legacy role-only behavior, election and mutation policies are preserved. UTC defaults to DateTime.UtcNow.

Verification: the original timer regressions failed with follower [240] instead of [240,200] and Primary [250] instead of [250,210]. Restored targeted tests passed 33/33; independently changing each strict age comparison to >= failed its boundary tests (2 snapshot cases, 1 peer-expiry case). The full Runtime suite passed 833, failed 0, with the 3 existing retired-v3 skips. Root independently reran all 6 new cases: 6 passed. Independent review found no actionable issues.

The reviewed #722 remarks are integrated as local commit f478b844. Both fixes remain unpushed while #719 final-head CI run983 is queued behind active run980; publishing now could supersede that pending gate. Next: integrate verified main after #719, publish the bounded batch, run required hosted CI on its final head, then merge and close only after success. No Warm implementation or live calls were performed.

The approved fix is implemented and committed locally as 1e71a25e; it starts periodic freshness recomputation for DB-less nodes without asking a null DB probe. The existing calculator, DB-backed Ask/timeout, legacy role-only behavior, election and mutation policies are preserved. UTC defaults to DateTime.UtcNow. Verification: the original timer regressions failed with follower [240] instead of [240,200] and Primary [250] instead of [250,210]. Restored targeted tests passed 33/33; independently changing each strict age comparison to >= failed its boundary tests (2 snapshot cases, 1 peer-expiry case). The full Runtime suite passed 833, failed 0, with the 3 existing retired-v3 skips. Root independently reran all 6 new cases: 6 passed. Independent review found no actionable issues. The reviewed #722 remarks are integrated as local commit f478b844. Both fixes remain unpushed while #719 final-head CI run983 is queued behind active run980; publishing now could supersede that pending gate. Next: integrate verified main after #719, publish the bounded batch, run required hosted CI on its final head, then merge and close only after success. No Warm implementation or live calls were performed.
dohertj2 commented 2026-10-13 15:11:35 -04:00
Author
Owner
Copy Link
Copy Source

Prepared a local integration of approved #721/#722 fixes with #719 final head8828e999 while its required hosted CI runs. Local merge commit bc80b53e is not pushed.

Fresh integrated build and targeted actor suite passed35/0failed/0skipped; individual TRX outcomes confirm all35 passed. This includes six new freshness cases and the existing actor/publication regressions, including #719 failure/retry cases. No compiler warning lines were observed in this build. Independent two-parent source review found only the expected approved changes, with no actionable P1/P2 findings.

The earlier complete Runtime result833passed/3existing skips belongs to the pre-integration #721 tree. The current integrated result is35targeted tests, not another full-suite claim. Required final-head hosted CI will cover the published integration before merge/closure. #719 final CI983 remains active; publication still waits for that gate and its main merge. No live calls or Warm implementation.

Prepared a local integration of approved #721/#722 fixes with #719 final head8828e999 while its required hosted CI runs. Local merge commit bc80b53e is not pushed. Fresh integrated build and targeted actor suite passed35/0failed/0skipped; individual TRX outcomes confirm all35 passed. This includes six new freshness cases and the existing actor/publication regressions, including #719 failure/retry cases. No compiler warning lines were observed in this build. Independent two-parent source review found only the expected approved changes, with no actionable P1/P2 findings. The earlier complete Runtime result833passed/3existing skips belongs to the pre-integration #721 tree. The current integrated result is35targeted tests, not another full-suite claim. Required final-head hosted CI will cover the published integration before merge/closure. #719 final CI983 remains active; publication still waits for that gate and its main merge. No live calls or Warm implementation.
dohertj2 commented 2026-10-13 16:00:00 -04:00
Author
Owner
Copy Link
Copy Source

Published reviewed fixes in PR #724, head e1feb9fbb6, based on merged #719 master 1bf8c614. The ancestry merge preserves tree 618eb2d3e7 exactly: integrated actor suite 35/35, independent review clear; earlier full Runtime 833 passed/3 existing skips before integration. No redundant local full-solution rerun or live calls.

Required final-head hosted run #985/job3606 is queued behind main run #984/job3605. Merge/closure wait for #985 success, floor and artifacts. Production Warm behavior is outside this fix.

Published reviewed fixes in PR #724, head e1feb9fbb6e3a1cb78c95cdb31ee24635ab11a33, based on merged #719 master 1bf8c614. The ancestry merge preserves tree 618eb2d3e795955b8dc1a0f8af4628fcca2e5e05 exactly: integrated actor suite 35/35, independent review clear; earlier full Runtime 833 passed/3 existing skips before integration. No redundant local full-solution rerun or live calls. Required final-head hosted run #985/job3606 is queued behind main run #984/job3605. Merge/closure wait for #985 success, floor and artifacts. Production Warm behavior is outside this fix.
dohertj2 commented 2026-10-13 17:10:20 -04:00
Author
Owner
Copy Link
Copy Source

A final acceptance audit found that the received-snapshot missing-local-entry branch was only source-verified: the existing detached-node test did not execute that branch. Entirely absent snapshot startup also lacked dedicated timer-period verification. The approved #721 scope already requests missing/detached snapshot verification, so narrow test-only coverage is being added before closure. Production policy remains unchanged: missing local entry publishes zero; no snapshot/local identity returns without publication. Warm startup policy remains #717. Current hosted run 985 tests published e1feb9f; if retained verification changes are published, their new exact head will require its own CI gate. No merge or closure has been performed.

A final acceptance audit found that the received-snapshot missing-local-entry branch was only source-verified: the existing detached-node test did not execute that branch. Entirely absent snapshot startup also lacked dedicated timer-period verification. The approved #721 scope already requests missing/detached snapshot verification, so narrow test-only coverage is being added before closure. Production policy remains unchanged: missing local entry publishes zero; no snapshot/local identity returns without publication. Warm startup policy remains #717. Current hosted run 985 tests published e1feb9f; if retained verification changes are published, their new exact head will require its own CI gate. No merge or closure has been performed.
dohertj2 commented 2026-10-13 17:20:17 -04:00
Author
Owner
Copy Link
Copy Source

Published the reviewed test-only acceptance addition in commit c4b4046832 on PR #724. Retained diff: one freshness fixture, 78 added lines; production source restored and unchanged. Focused tests 2/2 passed; complete scrubbed Runtime suite 837 passed, zero failed, three existing retired-v3 skips. Both added cases individually passed; guard and timer-registration mutations each failed both tests as intended. Final build zero warnings/errors; independent review no P1/P2. Missing-local-entry zero/retry/recovery and wholly absent snapshot no-publication timer behavior now have direct runtime evidence. PR final tree is 973c6799d1. Hosted run985 remains evidence for the earlier e1feb head and cannot authorize merge of this new head. Required new-head CI and merge remain pending; #721/#722 stay open. #725/#726 local commits stay held until #724 clears its gate. No CI cancellation/restart/rerun or live calls.

Published the reviewed test-only acceptance addition in commit c4b404683257da195cdf20e45a923a1b336be460 on PR #724. Retained diff: one freshness fixture, 78 added lines; production source restored and unchanged. Focused tests 2/2 passed; complete scrubbed Runtime suite 837 passed, zero failed, three existing retired-v3 skips. Both added cases individually passed; guard and timer-registration mutations each failed both tests as intended. Final build zero warnings/errors; independent review no P1/P2. Missing-local-entry zero/retry/recovery and wholly absent snapshot no-publication timer behavior now have direct runtime evidence. PR final tree is 973c6799d17d239fe7208010a6197788b6cc06a2. Hosted run985 remains evidence for the earlier e1feb head and cannot authorize merge of this new head. Required new-head CI and merge remain pending; #721/#722 stay open. #725/#726 local commits stay held until #724 clears its gate. No CI cancellation/restart/rerun or live calls.
dohertj2 commented 2026-10-13 19:04:06 -04:00
Author
Owner
Copy Link
Copy Source

Merged PR #724 as c89912a93b after hosted run 986 passed all ten steps at final head c4b4046832. The merged tree exactly equals the tested head tree (973c6799d1). CI verified 14,656 executed tests across 62 TRX files, 573 skips, the 12,000-test floor and artifact upload. #721 and #722 are resolved. The independently reproduced intermittent ConditionRefresh failure remains tracked in open #728. No live calls or Warm implementation.

Merged PR #724 as c89912a93b87b1742dd049980780d5eb3f816c20 after hosted run 986 passed all ten steps at final head c4b404683257da195cdf20e45a923a1b336be460. The merged tree exactly equals the tested head tree (973c6799d17d239fe7208010a6197788b6cc06a2). CI verified 14,656 executed tests across 62 TRX files, 573 skips, the 12,000-test floor and artifact upload. #721 and #722 are resolved. The independently reproduced intermittent ConditionRefresh failure remains tracked in open #728. No live calls or Warm implementation.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
auto-managed
queue management
 cross-driver
queue management
 driver/abcip
queue management
 driver/ablegacy
queue management
 driver/focas
queue management
 driver/mqtt
MQTT / Sparkplug B broker-subscribe driver
 driver/opcuaclient
queue management
 driver/s7
queue management
 driver/twincat
queue management
 phase/1
queue management
 phase/2
queue management
 phase/3
queue management
 phase/4
queue management
 phase/5
queue management
 phase/6
queue management
 queue/blocked
queue management
 queue/done
queue management
 queue/failed
queue management
 queue/in-progress
queue management
 queue/queued
queue management
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
dohertj2 (dohertj2)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#721
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?