fix(audit): maintenance-login roll-forward/backfill, archive orphan cleanup, head-export health (ScadaBridge#113 #114 #115)

#113: AuditLogPartitionMaintenance (monthly SPLIT RANGE) and the
backfill-source-node endpoint now use the AuditLog:MaintenanceConnectionString
login when set (unset = unchanged). Migration AuditMaintenanceGrants gives
scadabridge_audit_purger ALTER ANY DATASPACE and column-level UPDATE (SourceNode).
#114: archive cleanup deletes the writer's own *.tmp files and manifest-less
data files older than 24 h inside {Directory}/{YYYY} (DeletedOrphan, dry-run aware).
#115: head-export failures bump scadabridge.audit.chain_head_export.failures and
Degrade the /health/ready entry audit-chain-head-export (failing or stale > 2 intervals).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 23:47:47 -05:00
parent b95ee5a658
commit 533414a16d
19 changed files with 2943 additions and 16 deletions
+25 -8
View File
@@ -823,8 +823,14 @@ MS SQL for direct-write events). Unredacted secrets never persist.
- **Safe deletion.** Cleanup deletes only the three files a well-formed archive name implies
(`{name}.manifest.json`, `.parquet`, `.tombstones.parquet`) in that archive's own
`{Directory}/{YYYY}` folder — never a path read from a manifest — manifest first, so an
interrupted deletion leaves leftovers that are no longer an archive (orphans of an interrupted
archive WRITE are not cleaned yet — #114). A cleanup failure is a
interrupted deletion leaves leftovers that are no longer an archive. **Orphans of an interrupted
archive WRITE (ScadaBridge#114):** the same pass also deletes, reporting action `DeletedOrphan`
(dry-run aware), the writer's own temp files (`{name}.parquet.tmp`,
`{name}.tombstones.parquet.tmp`, `{name}.manifest.json.tmp`) and data files whose
`.manifest.json` was never written — only files named exactly like that for a well-formed
archive name, directly inside a four-digit `{Directory}/{YYYY}` folder whose year matches the
name, not reparse points, and last written more than 24 h ago (a running write is never
touched). Disk-only; no correctness impact. A cleanup failure is a
purge failure on the health surface. `scadabridge audit archive cleanup [--dry-run]`
(`CleanupAuditArchives`, Administrator) runs the same pass on demand or reports what it
would delete. Pinned by `AuditLogArchiveCleanupTests` (mutation-checked: each safety rule
@@ -906,11 +912,17 @@ MS SQL for direct-write events). Unredacted secrets never persist.
`AuditMaintenanceLoginTests` runs exactly this setup on SQL Server: the writer ingests and is
refused the chain UPDATE and the per-channel DELETE; the maintenance login chains, verifies,
reads heads, purges per channel, enumerates partitions, archives and switches out. Not moved
to the maintenance login: the monthly partition roll-forward (`ALTER PARTITION FUNCTION`,
needs `ALTER ANY DATASPACE` on the runtime login, or run it as a DBA job) and the one-time
`audit backfill-source-node` (needs `UPDATE` on `SourceNode` — a DBA-run step in a hardened
setup) — both tracked in #113. Migrations themselves are applied by a DBA with the production SQL scripts.
reads heads, purges per channel, enumerates partitions, archives and switches out. **Also on the
maintenance login (ScadaBridge#113):** the monthly partition roll-forward
(`ALTER PARTITION FUNCTION … SPLIT RANGE`, `AuditLogPartitionMaintenance` now built on
`AuditMaintenanceContext`) and `audit backfill-source-node` (the management endpoint resolves the
keyed maintenance `IAuditLogRepository`). Migration `AuditMaintenanceGrants` gives
`scadabridge_audit_purger` the two grants they need that `ALTER ON SCHEMA::dbo` does not confer:
database-level `ALTER ANY DATASPACE` (roll-forward) and column-level `UPDATE (SourceNode)` on
`dbo.AuditLog` (backfill; no other column becomes updatable, the writer's DENY still wins).
Unset `AuditLog:MaintenanceConnectionString` keeps both on the runtime login as before — which
then needs those same two grants (or db_owner), exactly as the single-login default always did.
`AuditMaintenanceLoginTests` pins both with a writer-refused negative control. Migrations themselves are applied by a DBA with the production SQL scripts.
- **Authorization.** Reading the Audit Log requires the existing **Audit** role
extended with a new **OperationalAudit** permission. Per-site row scoping
reuses the existing site-permission model; bulk export requires an additional
@@ -973,7 +985,12 @@ MS SQL for direct-write events). Unredacted secrets never persist.
`FileMode.Append` only, one file per node per day; give the service account create +
append-data rights and nothing else), or `Sink: Syslog` — RFC 5424 (`<PRI>1 … chainhead -
{json}`, facility 13 "log audit", severity Notice) over UDP (RFC 5426) or TCP (RFC 6587
octet counting). A failed write is logged and retried with the same heads next tick (no health counter yet — #115).
octet counting). A failed write is logged and retried with the same heads next tick, and (ScadaBridge#115)
is also a health signal: counter `scadabridge.audit.chain_head_export.failures` on the shared
ScadaBridge meter, plus a `/health/ready` entry `audit-chain-head-export` that is Degraded (HTTP
200, Overview-visible, not out of rotation) while the last tick failed or no tick has succeeded
for more than two intervals; Healthy when the export is disabled. UDP syslog is fire-and-forget,
so a dropped datagram is invisible by design — use `Protocol: Tcp` where that matters.
`scadabridge audit verify-heads --path <file|dir>` reads those records back (file-sink
lines or a syslog collector's log) and the `VerifyAuditChainHeads` command checks each
against the database: the link stored at the exported sequence (the head, a live row's