fix(site-runtime): reconcile artifact deletions on apply — central deletes no longer orphan site rows

The artifact apply (DeploymentManagerActor.HandleDeployArtifacts) was
upsert-only, so deleting an external system (or shared script, DB connection,
data connection) centrally never removed the site's SQLite row — a deleted
external system stayed callable from site scripts forever. Central always
ships the COMPLETE set of each artifact class (ArtifactDeploymentService
GetAll* snapshots; the wire's presence-tracking wrapper lists preserve
null-vs-empty), so the site now applies upsert-then-reconcile: after storing
the incoming set, SiteStorageService.DeleteRowsExceptAsync removes any stored
row absent from it, per artifact table. A null list still means 'field not
shipped' and touches nothing.

Runtime cleanup rides along: a reconciled-away shared script is unregistered
from the compiled SharedScriptLibrary (a stale delegate would stay callable
until restart), and a removed data connection is evicted from the DCL hash
cache and its live connection actor stopped via the previously-caller-less
RemoveConnectionCommand — both on the actor thread via the extended
ApplyArtifactDataConnectionsToDcl message. All four tables are
RegisterReplicated, so the deletes reach the standby as ordinary CDC row
tombstones.

Tests: storage-level reconcile per table (incl. empty-set-deletes-all and
idempotency) in ArtifactStorageTests; actor-level pins in
DeploymentManagerActorTests (orphan delete, null-set no-op, library
unregistration, DCL stop for the removed connection only). Docs:
Component-DeploymentManager + Component-SiteRuntime record the
full-set/reconcile semantics.
This commit is contained in:
Joseph Doherty
2026-08-01 10:54:18 -04:00
parent 0123b68719
commit 2d03f2d507
6 changed files with 342 additions and 6 deletions
@@ -760,6 +760,93 @@ public class SiteStorageService
return results;
}
// ── Artifact set reconciliation ──
/// <summary>
/// Deletes shared scripts absent from the supplied full set and returns the deleted names.
/// </summary>
/// <param name="keepNames">The complete set of shared script names that should exist.</param>
/// <returns>A task that resolves to the names of the rows that were deleted.</returns>
public Task<List<string>> DeleteSharedScriptsExceptAsync(IReadOnlyCollection<string> keepNames)
=> DeleteRowsExceptAsync("shared_scripts", keepNames);
/// <summary>
/// Deletes external system definitions absent from the supplied full set and returns the deleted names.
/// </summary>
/// <param name="keepNames">The complete set of external system names that should exist.</param>
/// <returns>A task that resolves to the names of the rows that were deleted.</returns>
public Task<List<string>> DeleteExternalSystemsExceptAsync(IReadOnlyCollection<string> keepNames)
=> DeleteRowsExceptAsync("external_systems", keepNames);
/// <summary>
/// Deletes database connection definitions absent from the supplied full set and returns the deleted names.
/// </summary>
/// <param name="keepNames">The complete set of database connection names that should exist.</param>
/// <returns>A task that resolves to the names of the rows that were deleted.</returns>
public Task<List<string>> DeleteDatabaseConnectionsExceptAsync(IReadOnlyCollection<string> keepNames)
=> DeleteRowsExceptAsync("database_connections", keepNames);
/// <summary>
/// Deletes data connection definitions absent from the supplied full set and returns the deleted names.
/// </summary>
/// <param name="keepNames">The complete set of data connection names that should exist.</param>
/// <returns>A task that resolves to the names of the rows that were deleted.</returns>
public Task<List<string>> DeleteDataConnectionDefinitionsExceptAsync(IReadOnlyCollection<string> keepNames)
=> DeleteRowsExceptAsync("data_connection_definitions", keepNames);
/// <summary>
/// Deletes every row of an artifact table whose <c>name</c> is not in
/// <paramref name="keepNames"/>, returning the deleted names.
///
/// Central always ships the COMPLETE system-wide set of each artifact class on an
/// artifact deployment, so a stored row absent from the incoming set means the
/// artifact was deleted centrally. The store methods are upsert-only, which used to
/// leave such rows orphaned on the site forever (a centrally-deleted external
/// system stayed callable from site scripts indefinitely); the artifact apply now
/// reconciles by calling this after upserting. An empty <paramref name="keepNames"/>
/// legitimately deletes every row — central saying "none of these exist anymore".
/// </summary>
private async Task<List<string>> DeleteRowsExceptAsync(string table, IReadOnlyCollection<string> keepNames)
{
await using var connection = OpenConnection();
await using var transaction = connection.BeginTransaction();
var keepParams = keepNames.Select((name, i) => (Key: $"@k{i}", Value: name)).ToList();
var predicate = keepParams.Count == 0
? string.Empty
: $" WHERE name NOT IN ({string.Join(", ", keepParams.Select(p => p.Key))})";
var removed = new List<string>();
await using (var select = connection.CreateCommand())
{
select.Transaction = transaction;
select.CommandText = $"SELECT name FROM {table}{predicate}";
foreach (var (key, value) in keepParams)
select.Parameters.AddWithValue(key, value);
await using var reader = await select.ExecuteReaderAsync();
while (await reader.ReadAsync())
removed.Add(reader.GetString(0));
}
if (removed.Count > 0)
{
await using var delete = connection.CreateCommand();
delete.Transaction = transaction;
delete.CommandText = $"DELETE FROM {table}{predicate}";
foreach (var (key, value) in keepParams)
delete.Parameters.AddWithValue(key, value);
await delete.ExecuteNonQueryAsync();
_logger.LogInformation(
"Artifact reconcile removed {Count} orphaned row(s) from {Table}: {Names}",
removed.Count, table, string.Join(", ", removed));
}
transaction.Commit();
return removed;
}
}
/// <summary>