Bootstrap retry cancellation test races a wall-clock deadline #710

Closed
opened 2026-10-13 09:43:08 -04:00 by dohertj2 · 3 comments
Owner

Full capped #707 solution run reproduced CentralCommunicationActorTests.Bootstrap_retries_stop_once_the_gate_is_published: expected factory.Calls2, actual3 at line775. ControlPlane277passed/1failed, terminalprojectfailure. The fixture relies on wall-clock Thread.Sleep100 followed by manual refresh before a1-second retry timer, then Thread.Sleep2500. Seeing the second provider resolve does not guarantee the test thread runs before that retry expires. This failure was already among18 in CI952; resource caps alone did not remove it.

Production cancels its keyed retry timer on first gate publication; no production cancellation defect is established. Proposed test-only readiness: use an isolated Akka TestScheduler/local actor system for this fact, explicitly observe failed initial resolution, publish while virtual retry is pending, advance beyond the cancelled deadline, and preserve exact2load/1publication assertions. A mutation without timer cancellation must fail this test, proving cancellation coverage. Leave other actor harnesses and production timeouts unchanged.

Acceptance: deterministic positive/negative timer controls, affected/full ControlPlane and solution tests, independent review and resulting greenCI beforeclosing. #707 keeps CIresource bounds and remaining failures. Primary scheduler docs: https://getakka.net/articles/configuration/modules/akka.testkit.html .

Full capped #707 solution run reproduced CentralCommunicationActorTests.Bootstrap_retries_stop_once_the_gate_is_published: expected factory.Calls2, actual3 at line775. ControlPlane277passed/1failed, terminalprojectfailure. The fixture relies on wall-clock Thread.Sleep100 followed by manual refresh before a1-second retry timer, then Thread.Sleep2500. Seeing the second provider resolve does not guarantee the test thread runs before that retry expires. This failure was already among18 in CI952; resource caps alone did not remove it. Production cancels its keyed retry timer on first gate publication; no production cancellation defect is established. Proposed test-only readiness: use an isolated Akka TestScheduler/local actor system for this fact, explicitly observe failed initial resolution, publish while virtual retry is pending, advance beyond the cancelled deadline, and preserve exact2load/1publication assertions. A mutation without timer cancellation must fail this test, proving cancellation coverage. Leave other actor harnesses and production timeouts unchanged. Acceptance: deterministic positive/negative timer controls, affected/full ControlPlane and solution tests, independent review and resulting greenCI beforeclosing. #707 keeps CIresource bounds and remaining failures. Primary scheduler docs: https://getakka.net/articles/configuration/modules/akka.testkit.html .
Author
Owner

Approved test-only fix committed locally as 8ff9f38b. An isolated virtual scheduler and explicit initial-failure/publication barriers replace wall-clock ordering assumptions. Removing production retry cancellation caused the new test to fail on an unexpected third load; production was restored. Full ControlPlane assembly passed 278 tests with no failures/skips. Independent review found no actionable defects. Combined solution and hosted CI are still publication/closure gates; no production behavior change.

Approved test-only fix committed locally as 8ff9f38b. An isolated virtual scheduler and explicit initial-failure/publication barriers replace wall-clock ordering assumptions. Removing production retry cancellation caused the new test to fail on an unexpected third load; production was restored. Full ControlPlane assembly passed 278 tests with no failures/skips. Independent review found no actionable defects. Combined solution and hosted CI are still publication/closure gates; no production behavior change.
Author
Owner

Combined reviewed test/CI fixes published in PR #711, head 76bdca90e9. Complete local coverage: 14,631 passed, 572 skipped, zero failures across 62 assembly TRX files; 12,000-test floor passed. Completed results from the interrupted invocation were retained and only five unfinished assemblies resumed with identical settings/code. Cancellation mutation verified RED; independent combined review clear. Hosted CI run 975 is active; issues stay open pending CI and merge.

Combined reviewed test/CI fixes published in PR #711, head 76bdca90e94e814e06ef10b78271e2b492b2ba32. Complete local coverage: 14,631 passed, 572 skipped, zero failures across 62 assembly TRX files; 12,000-test floor passed. Completed results from the interrupted invocation were retained and only five unfinished assemblies resumed with identical settings/code. Cancellation mutation verified RED; independent combined review clear. Hosted CI run 975 is active; issues stay open pending CI and merge.
Author
Owner

Merged PR #711 as 19bc13bf6a after hosted CI run 975 passed on exact head 76bdca90e9. Merged tree equals tested tree. Restore, Release build, README drift check, serialized full test step, 12,000 executed-test floor and TRX upload all succeeded; gate counted 14,630 executed tests across 62 reports. Local coverage also passed with 14,631 tests and 572 skips. Existing skip-summary diagnostic defect is tracked separately in #712. No retries or assertion deadline changes.

Merged PR #711 as 19bc13bf6a408e795de656f97b9bc5c2b976f5e5 after [hosted CI run 975](https://gitea.dohertylan.com/dohertj2/lmxopcua/actions/runs/975) passed on exact head 76bdca90e94e814e06ef10b78271e2b492b2ba32. Merged tree equals tested tree. Restore, Release build, README drift check, serialized full test step, 12,000 executed-test floor and TRX upload all succeeded; gate counted 14,630 executed tests across 62 reports. Local coverage also passed with 14,631 tests and 572 skips. Existing skip-summary diagnostic defect is tracked separately in #712. No retries or assertion deadline changes.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#710