Commit Graph

3 Commits

Author SHA1 Message Date
Joseph Doherty c4d7ec9b96 feat(audit): off-box chain-head export, verify-heads, maintenance login (ScadaBridge#107)
Item 1 (owner decision: no signing). AuditLog:HashChain:HeadExport (default
off): both centrals periodically read every month's chain head (LastSeq,
LastHash, purge anchor, chained rows, tombstones; one snapshot under the
chain lock SHARED) and append the changed heads to a sink the DB login cannot
write - File (append-only JSONL per node per day, local or UNC) or Syslog
(RFC 5424 over UDP / TCP octet counting). `audit verify-heads --path` reads
those records back (file or syslog-collector lines) and the new
VerifyAuditChainHeads command checks each against the database: Matches /
Extended / Purged, or Mismatch / Regressed / MissingHead / MissingLink /
CountMismatch (exit 3). AuditChainHeadStoreTests forges a chain with the
production hasher: verify-chain says Intact, verify-heads says Mismatch.

Item 3. AuditLog:MaintenanceConnectionString (${secret:}-capable, unset =
today's single login) runs the chain job, chain verify / head export, the
partition switch-out + archive and the per-channel purge on
AuditMaintenanceContext (keyed audit-maintenance repository/archive, purge
actor prefers them). AuditMaintenanceLoginTests runs the hardened setup on
SQL Server: writer (datareader/datawriter/audit_writer) ingests and is
refused chaining and deleting; the chainer+purger login does the full cycle.
Two-login grants documented in Component-AuditLog.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 23:16:05 -05:00
Joseph Doherty 7be4f19861 feat(audit): Parquet cold archive of expired AuditLog partitions before the purge (ScadaBridge#42)
Audit roadmap T2 (alog.md §15.2): with AuditLog:Archive:Enabled (default off)
the purge actor writes each expired monthly partition to Parquet under an
absolute AuditLog:Archive:Directory before switching it out, so audit rows
outlive RetentionDays.

- Parquet.Net 6.1.0 (fully managed; NuGetAudit clean) in central package
  management.
- AuditArchiveParquet: data file = the ten stored columns, OccurredAtUtc as a
  UTC microsecond timestamp plus exact OccurredAtUtcTicks, and ChainSeq/RowHash;
  tombstone file = the month's per-channel purge tombstones. Buffered row
  groups; the files are plain Parquet (DuckDB/pyarrow read them).
- AuditLogPartitionArchive (IAuditLogArchive): reads the partition's rows,
  chain heads and tombstones in one transaction under the chain lock SHARED,
  writes {Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet,
  .tombstones.parquet and a .manifest.json (counts, SHA-256 per file, each
  month's head and prior anchor) via tmp + rename, manifest last. Archives are
  never overwritten.
- Archive before purge: SwitchOutPartitionAsync gains an internal guarded
  overload. Under the EXCLUSIVE chain lock it re-counts the partition and
  re-reads each month's LastSeq and THROWs 51002/51003 (nothing purged) unless
  both still equal the archive's, so the switch's checkpoint anchor is exactly
  the archive's chain end. The manifest is then marked Purged.
- Fail closed: a partition that cannot be archived is not purged; the failure
  takes the existing AuditLogPurgeFailedEvent / PurgeFailures path. Success
  publishes AuditLogArchivedEvent.
- Verify: file hashes and counts against the manifest, the chain recomputed
  from the archived rows and tombstones (AuditChainVerifier), and the chain end
  against the database anchor (Matches / Superseded / NotPurged / Mismatch).
- AuditLogArchiveOptionsValidator: Directory required when enabled and must be
  absolute; RowGroupSize, timeouts bounded.

No migration: the purger role already reads AuditLog, the head and tombstones.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:12:12 -05:00
Joseph Doherty fd4be80913 feat(audit): SHA-256 hash chain over the central AuditLog (ScadaBridge#41)
Audit roadmap T1 (alog.md §11.4): one chain per UTC calendar month of
OccurredAtUtc. Each chained row carries ChainSeq (gap-free within its month)
and RowHash = SHA-256(prev.RowHash || canonical(row)); the v1 canonical form
(AuditChainHasher) is pinned by a known-answer vector computed independently.

- Migration AddAuditLogHashChain: nullable ChainSeq/RowHash on dbo.AuditLog
  (metadata-only add), AuditLogChainHead (per-month chain end + partition-purge
  checkpoint anchor), AuditLogChainTombstone, two partition-aligned filtered
  indexes, scadabridge_audit_chainer role with a column-level UPDATE grant.
  Production script docs/plans/sql/AddAuditLogHashChain.sql.
- AuditLogHashChain chains unchained rows oldest month first under an
  exclusive sp_getapplock (zero timeout, so a dual-active central pair still
  yields one chain), with an optimistic head check and a ChainSeq IS NULL
  row-count check that roll a pass back instead of leaving a gap. The ingest
  path is untouched.
- Purges keep the chain verifiable: the partition switch records the month's
  checkpoint anchor and drops its tombstones; the per-channel DELETE writes a
  tombstone per chained row in the same transaction (OUTPUT INTO). Both, and
  the SourceNode backfill, hold the chain lock; the backfill skips chained
  rows. The switch staging table gains the two new columns.
- AuditChainVerifier reports the first break (partition, ChainSeq, EventId,
  kind) for edited, deleted, reordered rows and tail truncation.
- AuditLogHashChainService (central, default OFF via AuditLog:HashChain).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:33:11 -05:00