Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.
- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
tombstone file = the month's per-channel purge tombstones. Buffered row
groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
chain heads and tombstones in one transaction under the chain lock SHARED,
writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
.tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
month's head and prior anchor) via tmp + rename, manifest last. Archives are
never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
overload. Under the EXCLUSIVE chain lock it re-counts the partition and
re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
both still equal the archive's, so the switch's checkpoint anchor is exactly
the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
from the archived rows and tombstones (AuditChainVerifier), and the chain end
against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
absolute; RowGroupSize, timeouts bounded.
No migration: the purger role already reads AuditLog, the head and tombstones.
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Audit roadmap T1 (alog.md §11.4): one chain per UTC calendar month of
OccurredAtUtc. Each chained row carries ChainSeq (gap-free within its month)
and RowHash = SHA-256(prev.RowHash || canonical(row)); the v1 canonical form
(AuditChainHasher) is pinned by a known-answer vector computed independently.
- Migration AddAuditLogHashChain: nullable ChainSeq/RowHash on dbo.AuditLog
(metadata-only add), AuditLogChainHead (per-month chain end + partition-purge
checkpoint anchor), AuditLogChainTombstone, two partition-aligned filtered
indexes, scadabridge_audit_chainer role with a column-level UPDATE grant.
Production script docs/plans/sql/AddAuditLogHashChain.sql.
- AuditLogHashChain chains unchained rows oldest month first under an
exclusive sp_getapplock (zero timeout, so a dual-active central pair still
yields one chain), with an optimistic head check and a ChainSeq IS NULL
row-count check that roll a pass back instead of leaving a gap. The ingest
path is untouched.
- Purges keep the chain verifiable: the partition switch records the month's
checkpoint anchor and drops its tombstones; the per-channel DELETE writes a
tombstone per chained row in the same transaction (OUTPUT INTO). Both, and
the SourceNode backfill, hold the chain lock; the backfill skips chained
rows. The switch staging table gains the two new columns.
- AuditChainVerifier reports the first break (partition, ChainSeq, EventId,
kind) for edited, deleted, reordered rows and tail truncation.
- AuditLogHashChainService (central, default OFF via AuditLog:HashChain).
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
Comment/XML-doc changes only:
- add summaries to ~10k test methods, fixtures, fakes and helpers
(+ <param> on theories, <returns> on async Task methods,
<inheritdoc/> on interface implementations the checker can resolve)
- strip ~2.6k project-tracking references (WP*, P*-*, row-NN, Task N,
#NN, Component-0NN) from comments, keeping the explanatory text;
reword false positives in comments only
- real docs on fakes implementing BCL/library interfaces
(IDisposable, ILogger, IOptionsMonitor, IAsyncLifetime)
- reword the last 5 src/ false positives (CWE-1236, failures-1,
year-0001, M365)
Only non-comment edits: four one-line nested classes split across lines
so members can carry docs. Build 0 errors / 0 warnings.
Joseph Doherty