50 sub-second presence budgets remain in Runtime.Tests (the class behind three of #500's five defects) #502

New Issue

2026-07-25T18:39:57-04:00

dohertj2 commented

2026-07-25 18:39:57 -04:00

Split out of #500, which was root-caused to five separate timing/ordering defects in Runtime.Tests. Three of those five were a tight presence budget: an AwaitAssert/AwaitCondition duration that is comfortable on an idle machine but marginal when the assembly runs its 44 Akka test classes ~14-way parallel.

While fixing them I swept the project for the same shape. 50 polling assertions still carry a sub-second duration, concentrated heavily in one file:

file	count	duration
`OpcUa/OpcUaPublishActorTests.cs`	27	500 ms
`Health/PeerProbeSupervisorTests.cs`	13	500 ms
`Drivers/DriverHostActorDiscoveryTests.cs`	5	300 ms
`OpcUa/OpcUaPublishActorRebuildTests.cs`	2	500 ms
`Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs`	2	800 ms
`VirtualTags/VirtualTagActorTests.cs`	1	500 ms

These were deliberately NOT bulk-raised as part of #500. None of them has been observed failing across roughly 90 instrumented full-assembly runs, so raising all 50 would have been a large speculative diff across files unrelated to the reported fault. Filing instead so the decision is explicit rather than an omission.

The principle, if this is picked up. A presence budget is an upper bound before giving up, not a wait: AwaitAssert polls and returns the instant the condition holds. Raising it therefore costs nothing on the happy path and cannot make a genuinely failing assertion pass — the only thing it changes is how long a real breakage takes to report. That is the opposite of an absence window (ExpectNoMsg, settle-then-assert), where every millisecond is spent on every run and the length is a real calibration decision. HistorianAdapterActorTests now has both, named separately (AssertTimeout vs Settle) with the distinction documented — that is the pattern to copy.

Suggested shape: one documented constant per class (or a shared harness constant on RuntimeActorTestBase) rather than 50 literals, so the next person changing it changes it once.

Caveat — two of the 50 are a different bug. DriverHostActorNativeAlarmAckRoutingTests.cs:90 and :116 use AwaitAssert for an absence assertion, so they return instantly and prove nothing regardless of the number. Those are #501; raising their budget would not fix them and would just make them slower.

Split out of #500, which was root-caused to five separate timing/ordering defects in `Runtime.Tests`. Three of those five were a **tight presence budget**: an `AwaitAssert`/`AwaitCondition` duration that is comfortable on an idle machine but marginal when the assembly runs its 44 Akka test classes ~14-way parallel. While fixing them I swept the project for the same shape. **50 polling assertions still carry a sub-second duration**, concentrated heavily in one file: | file | count | duration | |---|---|---| | `OpcUa/OpcUaPublishActorTests.cs` | 27 | 500 ms | | `Health/PeerProbeSupervisorTests.cs` | 13 | 500 ms | | `Drivers/DriverHostActorDiscoveryTests.cs` | 5 | 300 ms | | `OpcUa/OpcUaPublishActorRebuildTests.cs` | 2 | 500 ms | | `Drivers/DriverHostActorNativeAlarmAckRoutingTests.cs` | 2 | 800 ms | | `VirtualTags/VirtualTagActorTests.cs` | 1 | 500 ms | **These were deliberately NOT bulk-raised as part of #500.** None of them has been observed failing across roughly 90 instrumented full-assembly runs, so raising all 50 would have been a large speculative diff across files unrelated to the reported fault. Filing instead so the decision is explicit rather than an omission. **The principle, if this is picked up.** A *presence* budget is an upper bound before giving up, not a wait: `AwaitAssert` polls and returns the instant the condition holds. Raising it therefore costs nothing on the happy path and **cannot** make a genuinely failing assertion pass — the only thing it changes is how long a real breakage takes to report. That is the opposite of an *absence* window (`ExpectNoMsg`, settle-then-assert), where every millisecond is spent on every run and the length is a real calibration decision. `HistorianAdapterActorTests` now has both, named separately (`AssertTimeout` vs `Settle`) with the distinction documented — that is the pattern to copy. Suggested shape: one documented constant per class (or a shared harness constant on `RuntimeActorTestBase`) rather than 50 literals, so the next person changing it changes it once. **Caveat — two of the 50 are a different bug.** `DriverHostActorNativeAlarmAckRoutingTests.cs:90` and `:116` use `AwaitAssert` for an *absence* assertion, so they return instantly and prove nothing regardless of the number. Those are #501; raising their budget would not fix them and would just make them slower.

dohertj2 referenced this issue

2026-07-25 20:20:36 -04:00

Intermittent single-test failure in Runtime.Tests under full-sweep load (unidentified) #500

dohertj2 commented

2026-07-25 20:20:37 -04:00

Superseded by the fix for #500 (154171f4). I filed this when I had decided not to bulk-raise the remaining sub-second presence budgets, on the grounds that none had been observed failing. Three further 30-run verification rounds then each surfaced a different member of that family, in three different files — so the decision recorded here was overturned by evidence and the work was done as part of #500: RuntimeActorTestBase.PresenceBudget (15 s) with 57 call sites routed through it.

The two DriverHostActorNativeAlarmAckRoutingTests sites listed here are still open as #501 — they are an absence-assertion misuse, not a budget problem, so a larger number would not fix them.

Superseded by the fix for #500 (`154171f4`). I filed this when I had decided **not** to bulk-raise the remaining sub-second presence budgets, on the grounds that none had been observed failing. Three further 30-run verification rounds then each surfaced a different member of that family, in three different files — so the decision recorded here was overturned by evidence and the work was done as part of #500: `RuntimeActorTestBase.PresenceBudget` (15 s) with 57 call sites routed through it. The two `DriverHostActorNativeAlarmAckRoutingTests` sites listed here are still open as #501 — they are an absence-assertion misuse, not a budget problem, so a larger number would not fix them.

dohertj2 closed this issue

2026-07-25 20:20:37 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#502