fix(site-secrets): contain secret-store outages per site; audit removal-only deploys

Secret-store outage (final review M5): SiteSecretsCollector returns a store
read failure as SiteSecretsCollection.StoreError instead of throwing. A deploy
now fails only the sites that reference a secret (nothing sent, not stamped,
DeploySiteSecrets row with the error), and ComputeCurrentArtifactHashesAsync
omits only those sites (they read Out of date) instead of the exception
aborting the whole sweep and the fleet-wide pending-updates check.

Removal-only deploys: the site reply gains SecretsRemoved (proto field 7,
names only) - the deployed_secrets rows the deploy soft-removed. Central
carries it on SiteArtifactResult and writes the DeploySiteSecrets row with a
removed list whenever it is non-empty, so dropping a site's last secret:
reference is audited centrally rather than only in site logs.

docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md

@@ -237,11 +237,32 @@ accurate as the spec:
     next (re)connect (no forced reconnect).
   - *One `secret:` parser* (`SecretReferenceScanner.TryParseSecretReference`) shared by the scanner
     and `MxGatewayDataConnection`, so a padded/case-variant reference resolves the shipped name.
-  - *Audit shape (§3.7 deviation).* The `DeploySiteSecrets` row carries `secrets` and `missing` —
-    there is **no `removed` field** — and a removal-only deploy (empty list shipped) writes **no
-    central row**; removals are logged on the site by `DeployedSecretsMirror`.
+  - *Audit shape (§3.7).* The `DeploySiteSecrets` row carries `secrets`, `missing` and `removed`.
+    (Shipped first without `removed`, so a removal-only deploy wrote no central row; closed by the
+    follow-ups below.)
   - *Deployment prerequisite:* a site pair using deployed secrets must run LocalDb replication
     (topology guide), since on an unreplicated pair only the node active at deploy time gets them.
+- **Follow-ups closed after merge (2026-09-28).**
+  - *Secret-store outage is per site (review M5).* `SiteSecretsCollector` returns a store read
+    failure as `SiteSecretsCollection.StoreError` instead of throwing. A deploy fails only the
+    sites that reference a secret (nothing is sent to them; they get a `DeploySiteSecrets` row
+    with the error), and `ComputeCurrentArtifactHashesAsync` leaves those sites out of its map,
+    which the pending-updates evaluator reads as *Out of date*. Sites with no secret references
+    never touch the store and are unaffected. Before this, one `GetAsync` exception aborted the
+    whole sweep and the fleet-wide pending-updates check.
+  - *Removal-only deploys are audited.* The site's reply gained `SecretsRemoved` (proto
+    `ArtifactDeploymentResponseDto.secrets_removed = 7`, names only; absent/empty reads as null):
+    the `deployed_secrets` rows that deploy soft-removed. Central carries it on
+    `SiteArtifactResult.SecretsRemoved` and writes the `DeploySiteSecrets` row whenever it is
+    non-empty, with a `removed` list. An old site never sends it, so its removals stay unaudited
+    centrally, as before.
+  - *CLI exit code.* `site deploy-artifacts` and `deploy artifacts` exit `1` when the summary's
+    `failureCount` is non-zero (same rule `deploy site` already had).
+  - *Hub-era site-a secrets (review M8).* The retired hub had copied six `hub-gate-*`/`shared-gate-*`
+    test secrets (2026-08-07) into both rig site-a nodes' local stores. The mirror deliberately
+    never deletes a name it did not ledger, so they were removed once by hand on the rig (both
+    nodes stopped together; see the live-gate doc). QA and vd03 never ran the hub
+    (`Secrets:Replication:Enabled=false`), so nothing like this exists there.
 - **Rig posture.** `docker/docker-compose.yml`'s central nodes now carry
   `Secrets__Store__Provider=SqlServer` plus a shared KEK; site-a shares that KEK (it is the
   secrets-via-artifact-deploy site); site-b/site-c deliberately carry a **mismatched** KEK so the

docs/requirements/Component-DeploymentManager.md

@@ -385,16 +385,25 @@ out below.
   check 5). An old site ignores proto field 9, so the empty list costs it
   nothing, and the hash is unaffected (empty == null above).
 - **Audit.** One best-effort `DeploySiteSecrets` / `SiteSecrets` audit row is
-  written per site deploy that carried secrets or a missing-secret failure
-  (never for a site with no secret references at all) — after-state is `{
-  deploymentId, secrets: [{name, revision}], missing: [name], success }`, which
-  structurally excludes ciphertext. Written on both `DeployCoreAsync` and
-  `RetryForSiteAsync`, including the fail-closed path (audited even though
-  nothing was sent). The row carries `secrets` and `missing` only — there is
-  **no `removed` field** (design §3.7 named one), and a deploy that only
-  removes secrets (the site's last reference dropped, an empty list shipped)
-  writes **no central row**; removals are logged on the site by
-  `DeployedSecretsMirror`.
+  written per site deploy that carried secrets, failed on a missing secret or a
+  secret-store read, or retired previously-deployed secrets (never for a site
+  with no secret involvement at all) — after-state is `{ deploymentId,
+  secrets: [{name, revision}], missing: [name], removed: [name], success,
+  error }`, which structurally excludes ciphertext. Written on both
+  `DeployCoreAsync` and `RetryForSiteAsync`, including the fail-closed paths
+  (audited even though nothing was sent). `removed` is the site's reply field
+  `ArtifactDeploymentResponse.SecretsRemoved` — the `deployed_secrets` rows that
+  deploy soft-removed — so a deploy that only drops the site's last secret
+  reference is still audited centrally. A site that predates the field never
+  reports it.
+- **Central secret-store outage.** `SiteSecretsCollector` returns a store read
+  failure as `StoreError` rather than throwing, so it is contained per site: a
+  site that references a secret fails (`Central secret store could not be read
+  while resolving '<name>' (...)`, nothing sent, not stamped), every other site
+  deploys, and `ComputeCurrentArtifactHashesAsync` omits only the affected sites
+  from its map (they read *Out of date* on the Sites page) instead of failing the
+  fleet-wide pending-updates check. A site with no secret references never
+  touches the store.
 - **Secret rotation and live connections.** The site writes and mirrors the
   shipped secrets before it pushes data connections to the DCL, so a
   connection created by the same deploy resolves a new secret on first

docs/requirements/Component-SiteRuntime.md

@@ -166,6 +166,7 @@ flowchart TD
 - `HandleDeployArtifacts` additionally accepts the command's `Secrets` list (`SecretArtifact`, ciphertext only) and writes the full shipped set to the `deployed_secrets` table via `SiteStorageService.UpsertDeployedSecretsAsync` — upsert every shipped name, soft-remove (`is_removed = 1`) any stored name absent from the set. A `null` `Secrets` field (pre-feature central) touches nothing and answers with no `SecretsApplied`. A current central always ships a list — **empty** when the site references no secrets — so dropping a site's last `secret:` reference removes the secret from both nodes. Storage detail: *Deployed Secrets (Site SQLite / LocalDb)* below.
 - After writing the table, the active node runs `DeployedSecretsMirror.ReconcileAsync` **synchronously**, before replying — so the deploy response's `SecretsApplied` count and any apply/verify error reflect this deploy's own outcome, not a value left over from the mirror's background schedule. `SecretsApplied` counts only the names *this deploy shipped* that applied and verified cleanly; a mirror failure for an unrelated, already-broken name (a different deploy's leftover, or a stale removal) is logged at Warning and excluded — it neither fails this deploy nor reduces its `SecretsApplied`.
 - No `DeployedSecretsMirror` registered (secrets deployment not composed on this node) with a non-empty `Secrets` list fails the deploy with `Secret deployment is not enabled on this site node`; with an empty `Secrets` list it still succeeds (`SecretsApplied = 0`).
+- The reply also carries `SecretsRemoved`: the names `UpsertDeployedSecretsAsync` soft-removed on this deploy (site-wide, whichever node's mirror later deletes them), or `null` when none. Central uses it for the `removed` list of its `DeploySiteSecrets` audit row, so a deploy that only retires secrets is audited centrally.
 - **Ordering against the DCL.** The shipped secrets are written and mirrored **before** the deploy's data-connection definitions are pushed to the DCL (the definitions are still stored in SQLite at the same point, and the push still goes out if the secrets step fails), so a connection created or reconfigured by the same deploy resolves a newly shipped secret on its first connect. A rotation with **unchanged** connection config does not recreate the connection (the DCL push is hash-guarded) — the new value takes effect on that connection's next (re)connect. No forced reconnect is performed.
 - See *Deployed Secrets (Site SQLite / LocalDb)* below for the mirror itself.