docs(scripting): T starts at the run's start but is wall clock from there; document the late-notice, ceiling and counter rules (#827 review F4)
The proto comment, VirtualTags.md and CLAUDE.md said the soft deadline measures the script's own execution and that starvation never counts. Only the time before the run STARTS is excluded; from the start it is wall-clock time, as in process, so a GC pause or a starved / suspended worker during the run still counts. A mid-run freeze test is not added: it races the script's own return against the watchdog's notice; documented as not covered. Also documents F2 (late notice cancels and counts), F3 (progress ceiling, killed_unresponsive) and F5 (answer = stalled | retrigger).
This commit is contained in:
+21
-12
@@ -58,18 +58,27 @@ processes:
|
||||
`script timed out after …` failure and the run token is cancelled inside the worker; at `T + G`
|
||||
(`Scripting:Worker:HardGrace`, default `max(1 s, T)`) the worker is **killed** (`Process.Kill(entireProcessTree)`,
|
||||
plus a worker-side fail-fast watchdog) — the thread, its memory and its ALC go with it.
|
||||
- **`T` is the script's own run time (lmxopcua#827).** As in process (where the guard's clock starts when the
|
||||
thread pool picks the run up), `T` starts when a worker evaluation thread starts the run — not at submit — so the
|
||||
pipe round trip and the time a starved worker waits to be scheduled never count against the script. The worker runs
|
||||
that clock and reports the crossing (`EvalResults.soft_expired_eval_ids`); the host answers its caller, sends the
|
||||
cancel, and arms the kill `G` after the notice. What the host still times on its own is a **transport deadline**,
|
||||
`T + G` from submit (restarted at a queued run's start): no result and no soft notice by then is the runner's fault —
|
||||
the tag publishes **`BadResourceUnavailable`** (`"script worker unresponsive"`), nothing is struck or quarantined,
|
||||
the worker is not killed (a really hung one goes silent and the heartbeat monitor kills it), and the seam
|
||||
re-evaluates when the worker answers. A re-trigger while that run is still unanswered is the runner's fault too, not
|
||||
`previous run still running`. Counted as `otopcua.script.ipc.timeouts{kind}`. The clock is negotiated in the handshake
|
||||
(`Hello`/`Ready.worker_soft_clock`): a worker that does not echo it keeps the host clock from submit (the pre-#827
|
||||
contract), so a mismatched host/worker pair degrades to the old behaviour instead of losing its deadlines.
|
||||
- **`T` starts when the run starts (lmxopcua#827).** As in process (where the guard's clock starts when the thread
|
||||
pool picks the run up), `T` starts when a worker evaluation thread starts the run — not at submit — so the pipe
|
||||
round trip and the time a starved worker takes to **start** the run never count against the script. From the start
|
||||
on it is wall-clock time, exactly as in process: a GC pause, a CPU-starved or suspended worker **during** the run
|
||||
still counts against `T` (not separated, and not tested beyond that equivalence). The worker runs that clock and
|
||||
reports the crossing (`EvalResults.soft_expired_eval_ids`); the host resumes its caller on the result dispatcher
|
||||
(never on the pipe reader), sends the cancel, and arms the kill `G` after the notice.
|
||||
- **Transport deadline.** What the host still times on its own is `T + G` from submit (restarted at a queued run's
|
||||
start): no result and no soft notice by then is the runner's fault — the tag publishes **`BadResourceUnavailable`**
|
||||
(`"script worker unresponsive"`), nothing is struck, the worker is not killed, and the seam re-evaluates when the
|
||||
worker answers. A re-trigger while that run is still unanswered gets the same answer, not
|
||||
`previous run still running`. If the worker then reports the script over `T` after all (slow **and** late — a
|
||||
notice delayed past `T + G` lands here too), the run still gets its cancel and counts as a timeout. Counted as
|
||||
`otopcua.script.ipc.timeouts{kind, answer = stalled | retrigger}`.
|
||||
- **Progress ceiling.** The host does not depend on the worker's notices to end a run: while evaluations are
|
||||
outstanding, a worker that keeps heartbeating but produces no result, start or soft notice for
|
||||
`3 × the largest outstanding T + G + 3 heartbeat intervals` is killed (`killed_unresponsive`, no culprit,
|
||||
attributed like a heartbeat kill). A worker that stops heartbeating is killed by the silence rule as before.
|
||||
- **Negotiated.** The clock is agreed in the handshake (`Hello`/`Ready.worker_soft_clock`): a worker that does not
|
||||
echo it keeps the host clock from submit (the pre-#827 contract), so a mismatched host/worker pair degrades to the
|
||||
old behaviour instead of losing its deadlines.
|
||||
- **Attribution and quarantine** live in the host and survive restarts: a deadline kill strikes exactly its key
|
||||
(other evaluations in flight are victims, not strikes); a crash with no deadline crossed makes every key in flight
|
||||
a suspect that must complete `Scripting:Worker:IsolationProbationRuns` (20) clean runs in the isolation worker,
|
||||
|
||||
Reference in New Issue
Block a user