F1: PendingEval.SoftExpired runs on the pipe reader and completed the
caller's outcome inline (production outcomes have no async continuations, by
design, because results resume on the ResultDispatcher). The caller's code
(the runner's Cancel write, the seam's publish) therefore ran on the reader,
which could hold up heartbeats (a false silence kill) and, with blocking I/O,
write to the pipe from the reader. The soft outcome is now posted to the
dispatcher like a result. Test: a 1.5 s continuation on a soft timeout with a
300 ms silence limit resumes on ScriptWorker[t].Results and the worker lives
(before: it resumed on the Reader).
F2: after a transport timeout ("worker unresponsive"), a late soft notice
armed G but nobody sent Cancel, so a polling script could not end itself and
the whole worker was killed at notice + G with a kill strike. PendingEval now
exposes LateSoftExpiry (asynchronous continuations, cancelled when the run
answers); the runner sends the Cancel and counts the timeout
(NoteTimeout) from it. A genuinely slow script whose notice is delayed past
T + G is thus still a timeout. Test: stall past T + G, thaw, a polling
script overruns T: the worker survives G and the key is quarantined for
timeouts, not a hard deadline (before: the worker was killed).
The soft deadline T was armed on the host at submit, so the pipe round trip
and the time a starved worker waits to be scheduled counted against the
script: under load a trivial script reported 'script timed out', in Shadow
mode that was a script mismatch, and at T + G the host killed the worker and
struck the key.
- Protocol (additive, negotiated): Hello/Ready.worker_soft_clock and
EvalResults.soft_expired_eval_ids. protocol_version stays 1; a worker that
does not echo the flag keeps the pre-#827 host clock.
- Worker: the watchdog runs each run's soft clock from the moment an
evaluation thread starts it and reports the crossing (woken early for
budgets shorter than its 50 ms poll). It still never cancels by itself;
its fail-fast is anchored at the soft notice + G + margin.
- Host (PendingEval): the soft notice answers the caller and arms the kill G
later. The host's own timer becomes a transport deadline, T + G from submit
(restarted at a deferred run's start); passing it with no result and no
notice yields TransportTimedOut -> Unavailable 'script worker unresponsive'
(BadResourceUnavailable, recovery = the run's completion), no strike, no
kill, no Cancel. A re-trigger refused by that still-unanswered run is
Unavailable too, not 'previous run still running'. Counted as
otopcua.script.ipc.timeouts{kind}.
- Frame writer: coalescing now carries soft_expired_eval_ids (it silently
dropped them); the host processes soft notices before results in a frame.
- Shadow: unresponsive is shadow_unavailable (Unavailable), never a mismatch.
Tests: WorkerSoftClockTests suspends the real worker (SIGSTOP via libc kill /
NtSuspendProcess) - a stall past T still returns 7; a stall past T + G is a
runner fault with nothing struck; a late-starting runaway is still timed out
at T from its start and killed G after; Shadow stays match /
shadow_unavailable; both clocks enforce T and G; PendingEval state machine.
Joseph Doherty