fix(localdb): close both Phase 1 live-gate follow-ups
1. Wiped-node back-fill (live gate check 4). Fixed upstream in ZB.MOM.WW.LocalDb 0.1.2 and pinned here. The gate's diagnosis was slightly off: the oplog cap was not the gate. Snapshot detection measured the peer's gap against the oldest surviving oplog row and read an empty oplog as "no gap possible" — and an empty oplog is the steady state of a converged pair, since ack-pruning deletes everything the peer confirmed. The healthy state was the one state that could not heal a wiped peer. Now measured against last_acked_seq when the oplog is empty. Pinned at this level too, not just the library's: the pair harness grows a WipePassiveAsync (a NEW database — a rebuilt node comes back with a new node id and a zero watermark), and the new scenario asserts the emptied oplog as its precondition before wiping, then requires the deployment artifact to come back byte-identical with no new deploy. Verified RED against the pinned 0.1.1 (times out waiting for back-fill) and green on 0.1.2. 2. Oplog growth on default-OFF nodes (live gate check 8), fixed at the source: StoreAsync now skips entirely when the pointer already names this deployment/revision, the SHA matches the bytes, and the expected chunk count is present. Re-caching an artifact the node already holds — every restart's boot-from-cache, every RestoreApplied — writes nothing and mints no oplog rows, where before it cost a delete plus an insert per chunk plus a pointer update, all identical to what was already there. Identity is over the bytes and the chunks are counted, both deliberately: a re-composed artifact can carry the same ids with different bytes, and a pointer can name a deployment whose chunks are missing, where skipping would make an unreadable cache permanent. Both guards verified RED against a naive pointer-only skip. The gate's stated bound was also wrong and is corrected in the doc: growth was never headed for the 1M row cap. AddZbLocalDbReplication is registered unconditionally, so MaintenanceBackgroundService runs on default-OFF nodes and the 7-day MaxOplogAge cap prunes. Runtime.Tests 412/0/31, Host.IntegrationTests LocalDb 45/45. Pre-existing and unrelated: AbCip_Green_AgainstSim (needs the AB CIP docker fixture, red on a clean master too) and a flaky Roslyn race test (green 2/2 in isolation). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
This commit is contained in:
@@ -63,6 +63,14 @@ public sealed class LocalDbDeploymentArtifactCache : IDeploymentArtifactCache
|
||||
var cachedAtUtc = FormatTimestamp(DateTimeOffset.UtcNow);
|
||||
var chunkCount = (artifact.Length + ChunkSize - 1) / ChunkSize;
|
||||
|
||||
if (await IsAlreadyCachedAsync(clusterId, deploymentId, revisionHash, sha, chunkCount, ct))
|
||||
{
|
||||
_logger.LogDebug(
|
||||
"Deployment {DeploymentId} (revision {RevisionHash}) is already cached intact for cluster {ClusterId}; skipping the re-store.",
|
||||
deploymentId, revisionHash, clusterId);
|
||||
return;
|
||||
}
|
||||
|
||||
// One transaction for the whole store. A pointer that commits without its chunks — or
|
||||
// chunks that commit without the pointer — is a cache that reads back as a corrupt hit
|
||||
// rather than a clean miss, which is the one outcome this type exists to prevent.
|
||||
@@ -155,6 +163,74 @@ public sealed class LocalDbDeploymentArtifactCache : IDeploymentArtifactCache
|
||||
await tx.CommitAsync(ct);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Whether this exact artifact is already cached <b>and readable</b> for the cluster, so
|
||||
/// storing it again would rewrite every row to its current value.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why this exists.</b> Every write to a replicated table fires a capture trigger and
|
||||
/// mints an oplog row. Re-caching an artifact the node already holds — which is what a
|
||||
/// restart's boot-from-cache and every <c>RestoreApplied</c> recovery does — would
|
||||
/// therefore cost a delete plus an insert per chunk plus a pointer update, all carrying
|
||||
/// values identical to what is already there. On a replicating pair that is pure churn;
|
||||
/// on an unpaired (default-OFF) node there is no peer to ack those rows, so they
|
||||
/// accumulate in the oplog until the library's age cap prunes them. The Phase 1 live gate
|
||||
/// saw exactly that (check 8).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Identity is over the bytes, not the ids.</b> The pointer must match on
|
||||
/// <c>artifact_sha256</c> as well as the deployment id and revision hash: a re-composed
|
||||
/// artifact can legitimately carry the same ids with different bytes, and keeping the
|
||||
/// stale copy would serve a boot-from-cache node the wrong address space while every
|
||||
/// id-shaped signal claimed it was right.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>The chunk count is load-bearing.</b> A pointer can name a deployment whose chunks
|
||||
/// are missing or partial — a half-replicated artifact, or a prune that raced the
|
||||
/// pointer. Skipping on the pointer alone would make that state permanent, because the
|
||||
/// re-store that repairs it is the call being skipped.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Accepted consequence:</b> a skipped store leaves <c>applied_at_utc</c> at the
|
||||
/// moment the artifact was <i>first</i> cached rather than last confirmed. Refreshing it
|
||||
/// would mint the very oplog row this check exists to avoid. Nothing reads it as a
|
||||
/// liveness signal; retention ranks distinct deployments, which are unaffected.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private async Task<bool> IsAlreadyCachedAsync(
|
||||
string clusterId, string deploymentId, string revisionHash, string sha, int chunkCount,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var matches = await _db.QueryAsync(
|
||||
"""
|
||||
SELECT (
|
||||
SELECT COUNT(*)
|
||||
FROM deployment_artifacts
|
||||
WHERE cluster_id = @ClusterId
|
||||
AND deployment_id = @DeploymentId
|
||||
AND chunk_count = @ChunkCount
|
||||
)
|
||||
FROM deployment_pointer
|
||||
WHERE cluster_id = @ClusterId
|
||||
AND deployment_id = @DeploymentId
|
||||
AND revision_hash = @RevisionHash
|
||||
AND artifact_sha256 = @Sha
|
||||
""",
|
||||
r => r.GetInt32(0),
|
||||
new
|
||||
{
|
||||
ClusterId = clusterId,
|
||||
DeploymentId = deploymentId,
|
||||
RevisionHash = revisionHash,
|
||||
Sha = sha,
|
||||
ChunkCount = chunkCount,
|
||||
},
|
||||
ct);
|
||||
|
||||
return matches.Count == 1 && matches[0] == chunkCount;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public async Task<CachedDeploymentArtifact?> GetCurrentAsync(
|
||||
string clusterId, CancellationToken ct = default)
|
||||
|
||||
Reference in New Issue
Block a user