fix(deploy): derive the apply-outcome timeout from central's apply deadline (105 s) and name a timeout reason distinctly (#601 review)
60 s labelled a slow-but-clean large rebuild degraded. The deadline moves to a shared Commons DeployTimings constant that ConfigPublishCoordinator reads, and the node's timeout is deadline - 15 s margin, so the held ack still lands in time. A timed-out ack's reason now says the outcome was not reported in time, distinct from a real sink failure. Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
+10
-3
@@ -598,9 +598,16 @@ deployment green fleet-wide. Now:
|
||||
is clean), and only then is the ack sent. It is a plain `Tell` + reply message — no `Ask`, no `await`, so
|
||||
nothing resumes off the actor thread.
|
||||
- **A missing outcome is degraded, never clean.** If no reply arrives within
|
||||
`DriverHostActor.DefaultApplyOutcomeTimeout` (60 s — inside the coordinator's 2-minute apply deadline,
|
||||
which also has to cover the artifact read/fetch), the ack goes out `Applied` + degraded
|
||||
(*"did not report an outcome within 60s"*). A node with no publish actor wired acks clean at once, as before.
|
||||
`DriverHostActor.DefaultApplyOutcomeTimeout` the ack goes out `Applied` + degraded, with a reason starting
|
||||
*"address-space apply outcome not reported in time"* — distinct from a real sink failure (which carries
|
||||
`failedNodes=` / "rebuild failed" / "threw"), so an operator can tell a slow or lost apply from a broken one.
|
||||
The timeout is **derived**, not a free constant: `DeployTimings.ApplyDeadline` (Commons, 2 min — the same
|
||||
value `ConfigPublishCoordinator.DefaultApplyDeadline` reads) minus `DeployTimings.ApplyOutcomeSafetyMargin`
|
||||
(15 s) = **105 s**, so a slow-but-clean rebuild of a large address space is not labelled degraded and the
|
||||
ack still reaches central before its deadline. Change the deadline there and both sides move together.
|
||||
Accepted edge: the timer starts after the artifact read + driver reconcile, so if those alone exceed the
|
||||
15 s margin a never-answered apply can still time the deployment out. A node with no publish actor wired
|
||||
acks clean at once, as before.
|
||||
- **The ack carries an ADDITIVE degraded flag on both transports.** `ApplyAck` gained trailing
|
||||
`Degraded = false, DegradedReason = null` (an old node's DPS JSON decodes clean); `ApplyAckMessage` gained
|
||||
`bool degraded = 6; optional string degraded_reason = 7;` (an old node's gRPC ack decodes clean). The
|
||||
|
||||
Reference in New Issue
Block a user