fix(siteruntime): startup reconcile starts the healed InstanceActor (register row 67)
SiteReconciliationActor's node-STARTUP fetch healed the config FILE but never the RUNTIME. Instance Actors are created from exactly two paths -- the Deployment Manager's boot-time SQLite load and an explicit deploy/enable command -- and the reconcile pass runs AFTER that boot load has already read storage. So a config reconcile restored started no actor: an enabled, successfully-deployed instance whose config was missing at boot (lost volume, first boot after a restore, rehomed data dir) stayed dead until the NEXT restart, whose load finally picked it up. Proven live on site-a 2026-08-15: "Reconcile: fetched + stored config" with no "InstanceActor started" after it, and a debug snapshot answering instanceNotFound. The pass now raises a fire-and-forget EnsureInstanceStartedCommand (new Commons Deployment message) for every ENABLED instance it healed, routed through the deployment-manager singleton PROXY so Instance Actors stay under the singleton no matter which node's reconcile found the gap. The handler is deliberately advisory: - idempotent -- no-op when the actor is already running, which both nodes' reconcile actors legitimately trigger for the same instance; - enabled-state-aware -- it re-reads the durable row, which is the authority on both the config body and enabled/disabled, so a disable landing after the fetch still wins and the instance stays stopped; - it defers to any in-flight deploy/redeploy/warm for the instance, which keeps the deploy gate's ordering guarantees and the all-or-nothing-per-instance rule intact -- this never applies a config, it only starts one already persisted; - a missing row is a quiet no-op, the normal case on an unreplicated site whose peer node healed its own file. The heal is no longer silent: a successful reconcile start emits a Warning log line and an operational `deployment` site event, because reaching it means the node booted without a config for an instance that should have been running.
This commit is contained in:
@@ -1024,6 +1024,11 @@ akka {{
|
||||
// guarded-writes it (orphans are logged, never deleted). Requires the HTTP
|
||||
// config fetcher; if it is somehow absent the self-heal is skipped (best-effort —
|
||||
// replication remains the primary path and the next restart retries).
|
||||
// The dmProxy is passed so the pass heals the RUNTIME too (register row 67): each
|
||||
// enabled instance it restores gets an EnsureInstanceStartedCommand routed to the
|
||||
// singleton, which starts the Instance Actor if the boot-time load missed it. Going
|
||||
// through the PROXY (not a local ActorOf) keeps Instance Actors under the singleton
|
||||
// regardless of which node's reconcile pass found the gap.
|
||||
if (deploymentConfigFetcher != null)
|
||||
{
|
||||
var reconcileLogger = _serviceProvider.GetRequiredService<ILoggerFactory>()
|
||||
@@ -1037,7 +1042,8 @@ akka {{
|
||||
_nodeOptions.NodeName,
|
||||
reconcileLogger,
|
||||
null,
|
||||
null)),
|
||||
null,
|
||||
dmProxy)),
|
||||
"site-reconciliation");
|
||||
_logger.LogInformation(
|
||||
"SiteReconciliationActor created (per-node startup self-heal) for site {SiteId} node {Node}",
|
||||
|
||||
Reference in New Issue
Block a user