fix(deploy+cli): review findings — honest CLI timeouts, watermark-complete staleness, phase-2 staging, lock-safe cancellation
Six adversarial-review findings, each verified against the code first.
F1 (HIGH) CLI HttpClient capped every call at min(30s, caller timeout),
silently truncating deploy site's 5-minute BulkDeployTimeout and the
5-minute bundle export/preview/import calls — which printed a fake
"504 Request timed out" while the server kept working. HttpClient.Timeout
is now Timeout.InfiniteTimeSpan (the per-call CTS is the single overall
deadline, connect included) with the connect phase bounded separately on
SocketsHttpHandler.ConnectTimeout. The env override is renamed to
SCADABRIDGE_HTTP_CONNECT_TIMEOUT_SECONDS to match its new meaning.
F2 (HIGH) StaleInstanceProbe's process-static memo served stale hashes
because nothing bumped the watermark on three paths:
(a) BundleImporter commits through the raw DbContext, so no import ever
moved the watermark — a second import overwriting the same template
could be OMITTED from ImportResult.StaleInstanceIds. It now bumps
once per apply ATTEMPT: after the commit, and after the rollback too
(the probe runs pre-commit, so a rolled-back attempt leaves memos for
state that never landed; bumping on both paths is the simplest
correct shape, versus threading transaction awareness through a
process-static cache).
(b) CollectWatermarkBumps' default: arm silently no-op'd, contradicting
its own doc. It now sets unattributed=true — an over-broad bump costs
extra work, a missed one produces stale work.
(c) DataConnection edits route through SiteRepository.SaveChangesAsync,
which had no watermark at all, yet Protocol/Primary+Backup config/
FailoverRetryCount are revision-hash inputs. It now bumps (BumpAll —
a connection has no owning template) after a commit that touched one.
F3 (MED) CLI TemplateTableProjection read child ARRAYS, but ListTemplates
now returns database-projected TemplateSummary rows, so template list
printed all zeros. It now prefers the *Count scalars and falls back to
array length (template get still returns full entities). --detail help
text and README corrected: a listing cannot yield definitions, so --detail
renders the raw summary payload and template get --id is the full dump.
F4 (MED) DeploySiteAsync staged every PendingDeployment in phase 1 against
a 5-min TTL while phase 2 reached them one batch at a time, so tail
instances' fetch tokens could expire before their command was sent.
Staging moved into phase 2, immediately before each send; prepare keeps
its flatten/validate/record work. The staging write is the phase's only
repository touch and is serialised behind a 1-permit semaphore, so the
non-thread-safe DbContext constraint holds and the sends stay concurrent.
F5 (MED, latent) DeploySiteAsync leaked every held operation lock if
cancelled — a wedged per-instance semaphore is permanent for the process.
Phase 2 no longer throws (cancellation is recorded as a per-instance
outcome so phase 3 still runs), and an escape from phase 1 or 3 now
unwinds every unfinalised entry: Failed status + lock release.
F6 (LOW) ScriptCompileVerdictCache's promotion wrote hot directly,
bypassing SegmentCapacity (true ceiling 3x against a documented 2x).
Promotion now goes through Store, keeping generational semantics; _hot
and _cold are volatile.
Tests: CLI 396, DeploymentManager 133, ManagementService 494,
TemplateEngine 478, ScriptAnalysis 60, Transport 157, Transport
integration 106, ConfigurationDatabase 366 — all green, 0 build warnings.
The F2/F4/F5 regression tests were each confirmed to FAIL with their fix
reverted.
This commit is contained in:
@@ -49,7 +49,12 @@ public static class ScriptCompileVerdictCache
|
||||
/// <summary>
|
||||
/// Upper bound on entries in the hot generation. The cache holds at most
|
||||
/// <c>2 × SegmentCapacity</c> entries in total (hot + cold), preserving the
|
||||
/// previous 4096-entry ceiling.
|
||||
/// previous 4096-entry ceiling. Enforced on EVERY insert into hot — fresh
|
||||
/// verdicts and cold-generation promotions alike, both of which go through
|
||||
/// <see cref="Store"/>. (Hot can momentarily exceed the bound by the number of
|
||||
/// threads that pass the pre-rotation check concurrently; the bound is a
|
||||
/// memory ceiling, not a hard invariant, and that overshoot is bounded by
|
||||
/// concurrency rather than by working-set size.)
|
||||
/// </summary>
|
||||
private const int SegmentCapacity = 2048;
|
||||
|
||||
@@ -61,8 +66,13 @@ public static class ScriptCompileVerdictCache
|
||||
/// </summary>
|
||||
private static readonly object RotateGate = new();
|
||||
|
||||
private static ConcurrentDictionary<string, (bool Ok, string? Error)> _hot = new();
|
||||
private static ConcurrentDictionary<string, (bool Ok, string? Error)> _cold = new();
|
||||
// Volatile: both fields are REPLACED wholesale by a rotation under
|
||||
// RotateGate, while readers on GetOrAdd/Store run lock-free. Without the
|
||||
// volatile read a reader is free to cache the reference (or observe the
|
||||
// rotation's two writes out of order) and keep serving — or worse, keep
|
||||
// inserting into — a generation the rotation has already retired.
|
||||
private static volatile ConcurrentDictionary<string, (bool Ok, string? Error)> _hot = new();
|
||||
private static volatile ConcurrentDictionary<string, (bool Ok, string? Error)> _cold = new();
|
||||
private static long _hits;
|
||||
private static long _evictions;
|
||||
|
||||
@@ -110,11 +120,18 @@ public static class ScriptCompileVerdictCache
|
||||
if (cold.TryGetValue(key, out verdict))
|
||||
{
|
||||
// Promote: the entry is in active use, so it must survive the next
|
||||
// rotation. Writing to the CURRENT hot generation (re-read, in case a
|
||||
// rotation happened since the snapshot) is what makes this an LRU
|
||||
// rather than a fixed-lifetime cache.
|
||||
// rotation. Goes through Store (rather than a direct `_hot[key] =`)
|
||||
// so a promotion obeys SegmentCapacity like any other insert — a
|
||||
// direct write let hot grow past the segment bound whenever a working
|
||||
// set larger than SegmentCapacity was being re-read, taking the true
|
||||
// ceiling to 3 × SegmentCapacity against a documented 2 ×. Store
|
||||
// writes to the CURRENT hot generation (re-read inside, in case a
|
||||
// rotation happened since the snapshot above), which is what makes
|
||||
// this an LRU rather than a fixed-lifetime cache; a rotation
|
||||
// triggered BY this promotion still keeps the entry, because the
|
||||
// insert lands in the fresh hot generation afterwards.
|
||||
Interlocked.Increment(ref _hits);
|
||||
_hot[key] = verdict;
|
||||
Store(key, verdict);
|
||||
return verdict;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user