fix(drivers): don't tear down field I/O when a deploy's artifact reads back empty (#485)

The driver-side half of the same mistake the address-space fix corrected.
ReconcileDrivers and PushDesiredSubscriptionsFromArtifact both read the
artifact as `...FirstOrDefault() ?? Array.Empty<byte>()`. Their THROW paths
already returned early, but empty bytes flowed onwards as a real answer: zero
driver specs, so DriverSpawnPlanner planned every running child for StopChild,
and then an empty desired set dropped each surviving driver's live
subscription handle. A node lost its entire field I/O and still ACKed Applied.

Same rule as the address space: no bytes is no answer, not "a configuration
with no drivers". Nothing legitimate produces a zero-length blob — deleting
the last driver still deploys a JSON document with empty arrays. Both sites
now skip and keep what is running. The two guards are independent because
PushDesiredSubscriptions does its OWN ConfigDb read, so the row can go missing
between them.

Also corrects the ReconcileDrivers doc comment, which claimed an empty blob
made it "effectively a no-op" — with children running it was the opposite.

Tests: DriverHostActorUnreadableArtifactTests, RED-first (verified failing —
after the empty dispatch the driver list was empty). A zero-length ArtifactBlob
reproduces byte-for-byte what the missing-row case delivers, so the race does
not have to be constructed.

Two controls, both load-bearing:
- a READABLE driverless artifact still stops the driver, so the guard keys on
  "the read gave us nothing", not on "fewer drivers than before";
- dropping a driver's LAST TAG still clears its subscription. This one was
  added after the absence assertion was caught passing on a race: the
  unsubscribe is an async self-tell, so with the second guard deleted the test
  still went green. The control observes that same unsubscribe arriving well
  inside the settle window, which is what makes the absence meaningful — with
  the guard deleted the suite now correctly goes RED.

SubscribableStubDriver gains an UnsubscribeCount for that observation.

Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
Joseph Doherty
2026-07-21 02:40:14 -04:00
parent c7f5b9cfa3
commit b0c031f09f
4 changed files with 275 additions and 3 deletions
@@ -1618,9 +1618,9 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
/// <summary>
/// Read the deployment artifact + reconcile the set of running <see cref="DriverInstanceActor"/>
/// children. Spawn missing, ApplyDelta on config change, stop removed/disabled drivers.
/// When the artifact blob is empty (legacy ControlPlane tests, smoke fixtures) or the
/// configured <see cref="IDriverFactory"/> can't materialise any of the requested
/// types, this is effectively a no-op.
/// When the artifact blob is empty (legacy ControlPlane tests, smoke fixtures) the reconcile is
/// SKIPPED outright — see the issue #485 guard below — and when the configured
/// <see cref="IDriverFactory"/> can't materialise any of the requested types this is a no-op.
/// </summary>
/// <returns>
/// The artifact blob that was reconciled, or <see langword="null"/> when it could not be
@@ -1653,6 +1653,20 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
return null;
}
// Issue #485 — no bytes is no answer, not "a configuration with no drivers". Parsing zero bytes
// yields zero specs, and DriverSpawnPlanner then plans EVERY running child for StopChild: a missing
// deployment row (or a half-written one) silently takes this node's entire field I/O down while the
// apply still ACKs Applied. Nothing legitimate produces a zero-length blob — deleting the last
// driver still deploys a JSON document with empty arrays — so treat it exactly like the load
// failure above and leave the running children alone.
if (blob.Length == 0)
{
_log.Warning(
"DriverHost {Node}: artifact for {Id} read back empty; skipping reconcile and keeping the {Count} running driver(s)",
_localNode, deploymentId, _children.Count);
return null;
}
var specs = DeploymentArtifact.ParseDriverInstances(blob, _localNode.Value);
var snapshots = _children.ToDictionary(
kv => kv.Key,
@@ -1814,6 +1828,18 @@ public sealed class DriverHostActor : ReceiveActor, IWithTimers
/// </remarks>
private void PushDesiredSubscriptionsFromArtifact(DeploymentId deploymentId, byte[] blob)
{
// Issue #485 — the same "no bytes is no answer" rule as ReconcileDrivers. An empty artifact parses
// to a composition with no raw tags, which hands every child an EMPTY desired set — and an empty set
// drops the live subscription handle rather than re-subscribing. Reachable independently of the
// reconcile guard: this method does its OWN ConfigDb read, so the row can go missing between the two.
if (blob.Length == 0)
{
_log.Warning(
"DriverHost {Node}: artifact for {Id} read back empty; skipping SubscribeBulk and keeping the live subscriptions",
_localNode, deploymentId);
return;
}
AddressSpaceComposition composition;
try
{