feat(audit): archive retention and superseded stale-archive cleanup (ScadaBridge#109)

Owner decisions recorded: the archive destination stays a directory/UNC
(no object-storage client; share + permissions named at enablement),
parquet-tools-only reading (no re-import), unsigned like the chain (#107).

AuditLog:Archive:RetentionDays (default 0 = never delete; else must exceed
AuditLog:RetentionDays, counted from the archived month's end) and
DeleteSupersededStaleArchives (default true) run after every purge tick.
A retried purge's stale Archived archive is deleted only when a later,
verifying, Purged archive of the same partition provably holds every row:
same prior anchor and the stale chain-end hash at its chain-end sequence
(row or tombstone), every unchained row present by EventId; otherwise it is
reported KeptStale. A sole copy left at Archived is never touched. Deletion
removes only the archive name's own three files in its folder, manifest
first, never a manifest-supplied path. `audit archive cleanup [--dry-run]`
= CleanupAuditArchives (Administrator).

AuditLogArchiveCleanupTests (MSSQL) cover retried purge, kept stale copy,
sole copy, retention and the path guard; each safety rule mutation-checked.
Follow-ups filed: #113, #114, #115.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV

docs/components/CLI.md

@@ -24,8 +24,8 @@ Management commands use `POST /management`. Audit query, export, and tree use
 `GET /api/audit/query`, `/api/audit/export`, and `/api/audit/tree`; source-node
 backfill uses `POST /api/audit/backfill-source-node`. `audit verify-chain` sends the
 `VerifyAuditChain` management command (ScadaBridge#41); `audit verify-heads` reads off-box
-exported chain heads and sends `VerifyAuditChainHeads` (ScadaBridge#107); `audit archive list|verify` send
-`ListAuditArchives` / `VerifyAuditArchive` (ScadaBridge#42). `debug stream` uses a persistent
+exported chain heads and sends `VerifyAuditChainHeads` (ScadaBridge#107); `audit archive list|verify|cleanup` send
+`ListAuditArchives` / `VerifyAuditArchive` (ScadaBridge#42) / `CleanupAuditArchives` (ScadaBridge#109). `debug stream` uses a persistent
 SignalR connection. Audit export streams the response without buffering it all.
 
 ### Authentication and roles
@@ -122,7 +122,7 @@ scadabridge instance deploy --id 7
 | `db-connection` | `list`, `get`, `create`, `update`, `delete` | `Designer` |
 | `api-method` | `list`, `get`, `create`, `update`, `delete`, `validate`, `recompile` | `Designer` |
 | `bundle` | `export`, `preview`, `import` | Designer export; Administrator preview/import |
-| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `verify-chain`, `verify-heads`, `archive list`, `archive verify` | Audit roles; Administrator backfill; verify-chain, verify-heads and archive need an unscoped audit role |
+| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `verify-chain`, `verify-heads`, `archive list`, `archive verify`, `archive cleanup` | Audit roles; Administrator backfill and archive cleanup; verify-chain, verify-heads and archive need an unscoped audit role |
 | `audit-config` | `query` (config-change audit trail; was `audit-log` pre-M8) | `Admin` |
 | `health` | `summary`, `pending-updates`, `site`, `event-log`, `parked-messages` | Authenticated reads with server-side scope checks |
 | `debug` | `snapshot`, `stream` | `Deployer` |

docs/requirements/Component-AuditLog.md

@@ -770,8 +770,12 @@ MS SQL for direct-write events). Unredacted secrets never persist.
   - **Where.** `AuditLog:Archive:Directory` — an absolute path (validated at startup)
     that both central nodes reach, because the purge runs on the active node and
     `audit archive list|verify` on whichever node serves the call. Files land in
-    `{Directory}/{YYYY}/`. Object storage (S3 / Azure Blob) is not called directly —
-    mount it as a file system or copy the files on.
+    `{Directory}/{YYYY}/`. **Owner decision (ScadaBridge#109): the destination stays a
+    directory / UNC share — no native object-storage client.** Object storage (S3 / Azure
+    Blob) is reached, if ever, by mounting it as a file system or copying the files on. The
+    share (UNC path) and its permissions are chosen when the archive is enabled on DEV/QA:
+    both central nodes' service account needs modify rights on it (write, rename, and — for
+    cleanup — delete), auditors read-only; the SQL Server login needs none.
   - **What.** Per partition, three files sharing the stem
     `auditlog-{YYYY-MM}-{archivedAt:yyyyMMddTHHmmssfff}Z`: `.parquet` (every row of the
     partition — the ten stored columns, `OccurredAtUtc` both as a UTC microsecond
@@ -798,8 +802,33 @@ MS SQL for direct-write events). Unredacted secrets never persist.
   - **Retention interplay.** Archiving changes nothing about *when* a partition is
     purged (`RetentionDays`). Per-channel overrides still delete their rows earlier,
     so those rows are never archived — the archive carries their tombstones instead,
-    which is what keeps the archived chain verifiable. Archive files have no retention
-    of their own.
+    which is what keeps the archived chain verifiable.
+  - **Archive retention (ScadaBridge#109) — default never delete.**
+    `AuditLog:Archive:RetentionDays` (default `0` = archives are never deleted, until compliance
+    names a period) is counted from the END of the archived month and, when set, must exceed
+    `AuditLog:RetentionDays` (validated at startup) — an archive always outlives the rows' time
+    in the database. After each purge tick the purge actor deletes an expired archive that is
+    `Purged`, or an `Archived` one whose partition no longer holds rows.
+  - **Superseded stale archives (ScadaBridge#109).** A purge whose guard finds the partition
+    changed (51002/51003) leaves its archive at `Archived` (verify reports `NotPurged`, or
+    `Superseded` once a later purge anchored past it) and the next tick archives again. With
+    `AuditLog:Archive:DeleteSupersededStaleArchives` (default `true`) the purge actor deletes
+    such a stale archive only when a LATER archive of the same partition is `Purged`, verifies,
+    and provably holds every row of it: for every chain month, the same earlier anchor and the
+    stale archive's chain-end hash at its chain-end sequence (row or tombstone), and every
+    unchained row present by EventId. Otherwise it is kept and reported `KeptStale` (e.g. an
+    unchained row a per-channel purge removed in between — no tombstone — so the stale archive
+    is that row's only copy). An archive left at `Archived` because the node stopped between the
+    switch and the manifest update has no later purged archive and is never touched.
+  - **Safe deletion.** Cleanup deletes only the three files a well-formed archive name implies
+    (`{name}.manifest.json`, `.parquet`, `.tombstones.parquet`) in that archive's own
+    `{Directory}/{YYYY}` folder — never a path read from a manifest — manifest first, so an
+    interrupted deletion leaves leftovers that are no longer an archive (orphans of an interrupted
+    archive WRITE are not cleaned yet — #114). A cleanup failure is a
+    purge failure on the health surface. `scadabridge audit archive cleanup [--dry-run]`
+    (`CleanupAuditArchives`, Administrator) runs the same pass on demand or reports what it
+    would delete. Pinned by `AuditLogArchiveCleanupTests` (mutation-checked: each safety rule
+    removed fails its test).
   - **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks both
     files' SHA-256 and counts against the manifest, recomputes each archived month's
     chain from the archived rows and tombstones (`AuditChainVerifier`, starting from
@@ -807,8 +836,9 @@ MS SQL for direct-write events). Unredacted secrets never persist.
     `Matches`; `Superseded` (late rows for the month were chained, purged and archived
     again afterwards — that later archive resumes from this one's end); `NotPurged`;
     or `Mismatch` (archive or head altered). The files are plain Parquet — DuckDB,
-    pyarrow or Spark read them directly. There is no re-import into `dbo.AuditLog`:
-    a re-inserted expired row would be purged again by the next tick.
+    pyarrow, Spark or parquet-tools read them directly. **Owner decision (ScadaBridge#109):
+    reading with Parquet tools is the supported access path; there is no re-import into
+    `dbo.AuditLog`** — a re-inserted expired row would be purged again by the next tick.
 - **Sites:** `SiteAuditRetentionService` (site-only `IHostedService`) runs the
   site SQLite retention purge on a timer — first tick after
   `AuditLog:SiteRetention:InitialDelay` (default 5 min, short so a daily-recycled
@@ -880,7 +910,7 @@ MS SQL for direct-write events). Unredacted secrets never persist.
   to the maintenance login: the monthly partition roll-forward (`ALTER PARTITION FUNCTION`,
   needs `ALTER ANY DATASPACE` on the runtime login, or run it as a DBA job) and the one-time
   `audit backfill-source-node` (needs `UPDATE` on `SourceNode` — a DBA-run step in a hardened
-  setup). Migrations themselves are applied by a DBA with the production SQL scripts.
+  setup) — both tracked in #113. Migrations themselves are applied by a DBA with the production SQL scripts.
 - **Authorization.** Reading the Audit Log requires the existing **Audit** role
   extended with a new **OperationalAudit** permission. Per-site row scoping
   reuses the existing site-permission model; bulk export requires an additional
@@ -943,7 +973,7 @@ MS SQL for direct-write events). Unredacted secrets never persist.
     `FileMode.Append` only, one file per node per day; give the service account create +
     append-data rights and nothing else), or `Sink: Syslog` — RFC 5424 (`<PRI>1 … chainhead -
     {json}`, facility 13 "log audit", severity Notice) over UDP (RFC 5426) or TCP (RFC 6587
-    octet counting). A failed write is logged and retried with the same heads next tick.
+    octet counting). A failed write is logged and retried with the same heads next tick (no health counter yet — #115).
     `scadabridge audit verify-heads --path <file|dir>` reads those records back (file-sink
     lines or a syslog collector's log) and the `VerifyAuditChainHeads` command checks each
     against the database: the link stored at the exported sequence (the head, a live row's
@@ -961,9 +991,10 @@ MS SQL for direct-write events). Unredacted secrets never persist.
 - **Parquet archival (T2) — shipped (ScadaBridge#42), default OFF.** See Retention &
   Purge. The archive carries the hash-chain columns and the chain head, so tamper
   evidence survives the purge: `audit archive verify` ties an archive to the checkpoint
-  anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned;
-  a principal able to rewrite both the archive files and the head row can forge a
-  consistent pair.
+  anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned by owner
+  decision (ScadaBridge#107/#109); a principal able to rewrite both the archive files and the
+  head row can forge a consistent pair — unless the month's head was exported off-box first
+  (`AuditLog:HashChain:HeadExport`): `audit verify-heads` then exposes the rewritten anchor.
 - **Site SQLite security.** File permissions: read/write by the ScadaBridge
   service account only. Not backed up off-machine — site SQLite is a buffer,
   not a record.
@@ -1062,7 +1093,9 @@ archive (see Retention & Purge):
     "Directory": "D:\\AuditArchive",   // absolute; required when Enabled; reachable by both centrals
     "RowGroupSize": 50000,            // 1,000 – 1,000,000
     "ReadCommandTimeoutMinutes": 60,  // reading one partition out of SQL Server
-    "ChainLockTimeoutSeconds": 300    // wait for a chaining pass before skipping the tick
+    "ChainLockTimeoutSeconds": 300,   // wait for a chaining pass before skipping the tick
+    "RetentionDays": 0,               // 0 = never delete; else > AuditLog:RetentionDays, from the archived month's end (#109)
+    "DeleteSupersededStaleArchives": true  // delete stale archives a later purged archive provably covers (#109)
   }
 }
 ```

docs/requirements/Component-CLI.md

@@ -319,6 +319,7 @@ scadabridge audit verify-chain --month <YYYY-MM>
 scadabridge audit verify-heads --path <file-or-directory>
 scadabridge audit archive list
 scadabridge audit archive verify --name <archive>
+scadabridge audit archive cleanup [--dry-run]
 ```
 
 - `audit query` — filtered query against the central `AuditLog` table, matching the
@@ -370,6 +371,12 @@ scadabridge audit archive verify --name <archive>
   archived hash chain and compares its end with the purge anchor; exit `0` verified,
   `3` not verified, `1` error, `2` authorization failure. Same gate as `verify-chain`.
   See Component-AuditLog.md, Retention & Purge.
+- `audit archive cleanup [--dry-run]` (ScadaBridge#109) — sends `CleanupAuditArchives`:
+  deletes superseded stale archives (a retried purge's, once a later purged archive provably
+  holds all their rows) and archives past `AuditLog:Archive:RetentionDays` (0 = never), the
+  same pass the purge actor runs after every tick; `--dry-run` only reports. Prints each
+  `DeletedSuperseded` / `DeletedExpired` / `KeptStale` with its reason; exit `0`. Administrator
+  only (it deletes files).
 
 ### Health Commands
 ```

docs/requirements/Component-ManagementService.md

@@ -202,6 +202,7 @@ The two-person authorization workflow for writes through the MxAccess Gateway. B
 - **VerifyAuditChain** (ScadaBridge#41): verifies one UTC calendar month (`Month`, `YYYY-MM`) of the central AuditLog hash chain and returns an `AuditChainVerificationResult` (status, rows verified, tombstones applied, purge anchor, head sequence, unchained rows, first break with partition / `ChainSeq` / `EventId` / kind). Gated on the OperationalAudit roles (`Administrator`, `Viewer`); a site-scoped non-Administrator caller is refused (`ManagementUnauthorized`) because a month's chain spans every site. Long-running Ask timeout. Errors: `INVALID_ARGUMENT` (bad month), `CHAIN_BUSY` (a chaining/purge pass held the chain lock past 60 s). Backs `scadabridge audit verify-chain`.
 - **VerifyAuditChainHeads** (ScadaBridge#107): checks chain heads exported off-box (`Records`, at most 50,000 `AuditChainHeadRecord`s; duplicates collapsed) against the database chain and returns an `AuditChainHeadVerificationResult` (overall `Verified`, one `AuditChainHeadCheck` per distinct exported head: `Matches` / `Extended` / `Purged`, or the failures `Mismatch` / `Regressed` / `MissingHead` / `MissingLink` / `CountMismatch`). Same gate as VerifyAuditChain; long-running Ask timeout. Errors: `INVALID_ARGUMENT` (no records / over the cap), `CHAIN_BUSY`. Backs `scadabridge audit verify-heads`.
 - **ListAuditArchives** / **VerifyAuditArchive** (ScadaBridge#42): list the AuditLog Parquet cold archives under `AuditLog:Archive:Directory` (`AuditArchiveListResult`: enabled flag + every manifest), and verify one by `Name` (`AuditArchiveVerificationResult`: file integrity, per-month chain recomputed from the archived rows, anchor `Matches`/`Superseded`/`NotPurged`/`Mismatch`, overall `Verified`). Same gate as VerifyAuditChain (OperationalAudit roles, site-scoped non-Administrator refused). VerifyAuditArchive uses the long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`, `INVALID_ARGUMENT` (not an archive name), `NOT_FOUND`. Back `scadabridge audit archive list|verify`.
+- **CleanupAuditArchives** (ScadaBridge#109): runs the archive cleanup now (`DryRun` = report only) — superseded stale archives a later purged archive provably covers, and archives past `AuditLog:Archive:RetentionDays` — and returns an `AuditArchiveCleanupResult` (`DryRun`, `RetentionDays`, one item per archive deleted / kept stale, with the action `DeletedExpired` / `DeletedSuperseded` / `KeptStale` and reason). Administrator only; runs on the maintenance-connection archive when registered; long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`. Backs `scadabridge audit archive cleanup`.
 
 ### Transport (Bundle Import / Export)