{ "planPath": "docs/plans/2026-07-22-mesh-phase3-config-fetch-and-cache.md", "note": "Per-cluster mesh Phase 3. Decisions pre-made by the user: gRPC fetch RPC (NOT token-gated HTTP); shared node key (NOT per-deployment token, so no migration and DispatchDeployment is UNCHANGED); config-flagged dark switch ConfigSource:Mode=Direct default; both pair nodes fetch (no Primary gating). Five recon facts + the phase-boundary rule (Phase 3 = config READS only; NodeDeploymentState write / DbHealthProbe / EfAlarmConditionStateStore stay for Phase 4) are in the plan header. RevisionHash == SHA-256(blob) is load-bearing for fetch verification.", "tasks": [ { "id": 0, "subject": "Task 0: ConfigSource/ConfigServe options + validator + registration", "status": "completed", "classification": "small", "parallelizableWith": [ 1 ], "commit": "9d1e60c0", "note": "One sabotage (accept empty endpoints) reddened its test; file was untracked so restored by hand not git checkout." }, { "id": 1, "subject": "Task 1: deployment_artifact.v1 proto + first in-repo Grpc.Tools codegen", "status": "completed", "classification": "standard", "parallelizableWith": [ 0 ], "commit": "62e8f54f", "note": "First locally-compiled proto in the repo. Grpc.Tools 2.76.0 pinned (aligned with the other Grpc 2.76.0 refs); GrpcServices=Both. Commons builds clean, codegen verified." }, { "id": 2, "subject": "Task 2: central DeploymentArtifactService \u2014 stream sealed artifact, NotFound-hide the rest", "status": "completed", "classification": "standard", "blockedBy": [ 1 ], "parallelizableWith": [ 4 ], "commit": "7d7de482", "note": "Impl RENAMED to DeploymentArtifactGrpcService \u2014 the generated container class is itself named DeploymentArtifactService and its nested base collided. Aliased the base. Sabotage dropping the sealed+empty guard reddened 2 tests." }, { "id": 3, "subject": "Task 3: config-serve auth interceptor + dedicated h2c listener wiring", "status": "completed", "classification": "high-risk", "blockedBy": [ 0, 2 ], "commit": "4d88f67c", "note": "Merged the LocalDb-sync + ConfigServe listener blocks into ONE that re-applies the existing HTTP surface exactly once and adds both dedicated h2c ports; a fused admin+driver node double-binding the HTTP port throws 'address already in use' (sabotage-proven). AddGrpc moved out of the hasDriver block to hasDriver||hasAdmin with both path-scoped interceptors. StatusCode.Unauthenticated (LocalDb uses PermissionDenied). Coexistence proven by a minimal-WebApplication shape test, mirroring LocalDbSyncListenerTests; real end-to-end fetch is Task 8." }, { "id": 4, "subject": "Task 4: node gRPC artifact fetcher \u2014 SHA-verified reassembly, endpoint failover, null-on-any-failure", "status": "completed", "classification": "standard", "blockedBy": [ 1 ], "parallelizableWith": [ 2 ], "commit": "7a8fb5f1", "note": "Uniform invariant: return the first endpoint's bytes that reassemble AND verify (SHA-256 lowercase-hex == dispatched RevisionHash); else null. RpcException / empty-stream / hash-mismatch all -> try-next, never throw. Func seam keeps gRPC out of unit tests (fake subclasses the generated client's protected ctor). Namespace is .DeploymentCache NOT .Deployment (else it shadows Configuration.Entities.Deployment across the whole test assembly via enclosing-namespace lookup) \u2014 same trap LocalDbDeploymentArtifactCache already dodged. Added Grpc.Net.Client to Runtime.csproj. xUnit v2 here: CancellationToken.None, not TestContext.Current." }, { "id": 5, "subject": "Task 5: FetchAndCache apply path \u2014 fetch->cache->apply-from-bytes, null=apply-failure", "status": "completed", "classification": "high-risk", "blockedBy": [ 4 ], "commit": "648f173b", "note": "DispatchDeployment (rev differs) in FetchAndCache -> BeginFetchAndCacheApply PipeTo(Self) FetchedForApply(bytes?), never a blocking await in a receive. HandleFetchedForApply applies synchronously mirroring ApplyAndAck: null bytes = FAILURE (keep last-known-good, don't advance rev, ack Failed); good bytes reconcile+rebuild+push FROM the bytes (OpcUaPublishActor never reads SQL) then cache. A faulted fetch task maps to null via PipeTo failure:. Extracted ReconcileDriversFromBlob (Direct/FetchAndCache/boot all share it; #485 empty guard moved in); ApplyCachedArtifact now reuses it. Props gains fetchAndCacheMode+artifactFetcher LAST. Idempotent re-dispatch is the existing _currentRevision short-circuit (no separate cache-contains check added; peer-replication-missed-fetch is covered by Task 6 boot + StoreAsync idempotency). Sabotages: advance-rev-on-null reddens test b; drop _currentRevision guard reddens test c." }, { "id": 6, "subject": "Task 6: FetchAndCache bootstrap from the LocalDb pointer; no driver-side config SQL read", "status": "completed", "classification": "high-risk", "blockedBy": [ 5 ], "commit": "6d004738", "note": "Bootstrap() branches to BootstrapFromCache() in FetchAndCache: GetCurrentUnkeyedAsync -> set _currentRevision -> ApplyCachedArtifact (no SQL, no re-ack). _isRunningFromCache STAYS false (normal op, not degraded). Empty/unreadable cache -> Steady-no-revision (first dispatch fetches), NEVER Stale (Stale = 'SQL down, retry'; FetchAndCache has no config SQL read to recover). TryRecoverFromStale gains a defensive FetchAndCache early-return (unreachable anyway \u2014 retry-db timer only starts in Stale). KEY test lever: UpsertNodeDeploymentState SWALLOWS db failures, so a FetchAndCache dispatch runs end-to-end under a ThrowingDbFactory \u2014 the ack proves Steady-not-Stale AND no-SQL-at-boot in one shot. NodeDiagnosticsSnapshot lives in Commons.Interfaces not Messages.Fleet." }, { "id": 7, "subject": "Task 7: #485 empty/corrupt/mismatch coverage on the FetchAndCache paths", "status": "pending", "classification": "high-risk", "blockedBy": [ 6 ] }, { "id": 8, "subject": "Task 8: real two-Host gRPC artifact-fetch boundary test + wrong-key falsifiability control", "status": "pending", "classification": "high-risk", "blockedBy": [ 3, 4 ], "parallelizableWith": [ 9 ] }, { "id": 9, "subject": "Task 9: docker-dev rig config + docs + program-plan flip", "status": "pending", "classification": "small", "blockedBy": [ 5 ], "parallelizableWith": [ 8 ] }, { "id": 10, "subject": "Task 10: live gate on docker-dev (site nodes -> FetchAndCache; step 4 = stop central SQL mid-fetch = the headline gate)", "status": "pending", "classification": "high-risk", "blockedBy": [ 8, 9 ] } ], "lastUpdated": "2026-07-23T01:30:00Z" }