feat(audit): hash-chain cutoff - chain only rows written from first enablement, no backlog (#139)
Owner decision on #107 (2026-11-13): enabling AuditLog:HashChain must not chain the historical backlog. main chained every unchained row oldest-first. - The first chaining pass on a pristine database (no head, no chained row) writes a chain-start anchor: AuditLogChainTombstone (cutoff month, ChainSeq 0), PurgedAtUtc = DB SYSUTCDATETIME(), RowHash = SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks). Written once under the exclusive chain lock, so both centrals agree whatever their clocks/config. No migration: existing table + grants. - Only rows with OccurredAtUtc >= cutoff are chained. The anchor hash is the cutoff month's genesis, binding the cutoff into the chain. - AuditLog:HashChain:ChainBacklog (default false) restores chain-everything, read only on the very first enablement. Legacy chains (heads, no anchor) keep chaining everything. - verify-chain splits unchained rows into UnchainedRows vs PreCutoffRows (+ ChainFromUtc); new breaks ChainedBeforeCutoff (row re-dated before the cutoff, or cutoff moved forward) and CutoffAnchorInvalid (anchor hash / month / multiplicity). Archive verify uses the anchor from its own tombstone file or the DB. - Partition switch-out keeps the anchor (deletes ChainSeq > 0 only); the head store / export / verify-heads tombstone count excludes it. - CLI verify-chain table prints Chain from + Pre-cutoff rows. Tests: AuditLogHashChainCutoffTests (7, MSSQL), 6 verifier unit tests, service/CLI/management wire tests; 9 mutations all killed. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
This commit is contained in:
@@ -936,7 +936,8 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
- **Model.** One chain per **partition = UTC calendar month of `OccurredAtUtc`**
|
||||
(the `ps_AuditLog_Month` month). Each chained row carries `ChainSeq` (1, 2, 3 …,
|
||||
gap-free within its month) and `RowHash = SHA-256(prev.RowHash || canonical(row))`,
|
||||
`prev` being the row at `ChainSeq - 1` (32 zero bytes for the first). The canonical
|
||||
`prev` being the row at `ChainSeq - 1` (32 zero bytes for the first — or, in the month
|
||||
the chain started in, the chain-start anchor below). The canonical
|
||||
v1 form (`AuditChainHasher`) is a domain tag, the month and sequence number, then
|
||||
the ten writable columns in table order, each string length-prefixed with a NULL
|
||||
marker; the computed columns derive from `DetailsJson` and are covered by it. A
|
||||
@@ -960,10 +961,44 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
`OUTPUT … INTO`; verification bridges the gap with it. Both purge paths and the
|
||||
SourceNode backfill hold the same lock exclusively. The backfill skips chained rows
|
||||
(a chained row's `SourceNode` is frozen) — run it before enabling the chain.
|
||||
- **Pre-existing rows** start unchained (`ChainSeq`/`RowHash` NULL). Enabling the job
|
||||
chains the historical backlog oldest month first, so those rows are protected from
|
||||
the moment they are chained, not from when they were written. Verification reports
|
||||
still-unchained rows as `UnchainedRows`, never as a break.
|
||||
- **Chain start: no backlog chaining (ScadaBridge#139; owner decision on #107,
|
||||
2026-11-13).** Enabling the job on a database with history does **not** chain the
|
||||
backlog. The first chaining pass on a pristine database (no chain head, no chained row)
|
||||
writes a **chain-start anchor** and from then on only rows with
|
||||
`OccurredAtUtc >= cutoff` are chained; older rows keep `ChainSeq`/`RowHash` NULL for
|
||||
ever and are out of scope.
|
||||
- *Where the cutoff lives.* One row in `dbo.AuditLogChainTombstone` at `ChainSeq = 0`
|
||||
of the cutoff month (real links start at 1, so slot 0 is "the link before the
|
||||
first"): `PurgedAtUtc` = the cutoff, `RowHash` =
|
||||
`SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks)`, `EventId` =
|
||||
`5cadab1d-c4a1-4c0f-a000-000000000139`. The cutoff is the database's
|
||||
`SYSUTCDATETIME()` at that first pass, written once inside the chaining transaction
|
||||
under the exclusive chain lock — so both central nodes read the same instant whatever
|
||||
their own clocks or config say, and it survives restarts. It is never updated; the
|
||||
partition switch-out keeps it (it deletes only `ChainSeq > 0` tombstones); the head
|
||||
store's tombstone count, the head export and `verify-heads` ignore it. **No
|
||||
migration**: the table and the grants exist (the maintenance login writes it through
|
||||
`scadabridge_audit_purger`'s INSERT on the tombstone table).
|
||||
- *Bound into the chain.* The anchor hash is the genesis the cutoff month's first row
|
||||
links to (instead of 32 zero bytes). Moving the cutoff therefore breaks that month's
|
||||
chain, and the month's exported head anchors the cutoff off-box like any other link.
|
||||
- *`AuditLog:HashChain:ChainBacklog`* (default `false`) restores the old behaviour —
|
||||
chain every row, oldest month first, no anchor — but only on the very first
|
||||
enablement: once an anchor or any chained row exists the decision is fixed and the
|
||||
option is ignored, so two centrals with different values still end up with one
|
||||
chain (whichever chains first decides). A database whose chain was enabled before
|
||||
this change (heads, no anchor) keeps chaining every row, as before.
|
||||
- *Late rows.* The cutoff is on `OccurredAtUtc` (the partition key, indexed), not on
|
||||
ingest time (`IngestedAtUtc` is a non-persisted JSON column that must never be
|
||||
filtered on). A site row dated before the cutoff that reaches central after
|
||||
enablement — the site's store-and-forward backlog at that moment — stays out of
|
||||
scope; so does a row from a node whose clock runs behind the database's by the
|
||||
skew. Both are bounded and reported, never silently dropped.
|
||||
- *Index.* Pre-cutoff rows stay in the filtered `IX_AuditLog_Unchained` until retention
|
||||
purges their partition; the chaining read seeks `OccurredAtUtc >= cutoff`, so they
|
||||
cost storage only, not chaining time.
|
||||
- Verification reports in-scope rows not chained yet as `UnchainedRows` and the
|
||||
backlog as `PreCutoffRows` (plus `ChainFromUtc`); neither is ever a break.
|
||||
- **Verification.** `scadabridge audit verify-chain --month YYYY-MM` sends the
|
||||
`VerifyAuditChain` management command (OperationalAudit roles; refused for a
|
||||
site-scoped caller because a month's chain spans every site; long-running Ask). It
|
||||
@@ -971,8 +1006,28 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
`NotChained` or `Broken` with the **first break**: partition, `ChainSeq`, `EventId`
|
||||
and kind — `HashMismatch` (an edited row, or rows reordered — the sequence number is
|
||||
inside the hashed bytes), `MissingRow` (a deleted row with no tombstone),
|
||||
`DuplicateSequence`, `IncompleteLink`, or `HeadMismatch` (rows deleted from the end).
|
||||
CLI exit code 3 on `Broken`.
|
||||
`DuplicateSequence`, `IncompleteLink`, `HeadMismatch` (rows deleted from the end),
|
||||
`ChainedBeforeCutoff` or `CutoffAnchorInvalid` (below). CLI exit code 3 on `Broken`;
|
||||
the table output also prints `Chain from` and `Pre-cutoff rows`.
|
||||
- **Cutoff-eligibility attacks (ScadaBridge#139).** Because pre-cutoff rows are
|
||||
unprotected by design, an attacker would like to (a) move a chained row's
|
||||
`OccurredAtUtc` back before the cutoff to hide it among them, or (b) move the cutoff
|
||||
forward. (a) with the link left in place is `ChainedBeforeCutoff` (no row before the
|
||||
cutoff is ever chained — this also catches an attacker who re-hashes the row and every
|
||||
later link); with the link cleared too, its sequence number is a `MissingRow` (or
|
||||
`HeadMismatch` at the tail); moved into an earlier month, the source month reports
|
||||
`MissingRow` and the target month `ChainedBeforeCutoff`. (b) editing the anchor's
|
||||
cutoff without its hash, filing it under another month, or adding a second anchor is
|
||||
`CutoffAnchorInvalid` (in every month's verification); re-hashing the anchor to the new
|
||||
cutoff leaves the already-chained rows before it (`ChainedBeforeCutoff`) and changes the
|
||||
cutoff month's genesis (`HashMismatch` at `ChainSeq` 1); deleting the anchor is a
|
||||
`HashMismatch` at `ChainSeq` 1. **Known limits:** pre-cutoff rows themselves carry no
|
||||
tamper evidence (that is the decision), and a backdated row forged *before* the cutoff
|
||||
is indistinguishable from a genuine backlog row — the chain never proved a row genuine,
|
||||
only that a chained row was not changed after chaining. A principal that rewrites the
|
||||
anchor, the cutoff month's whole chain and its head is caught only by the off-box head
|
||||
export, as for any full-chain rewrite. `AuditLogHashChainCutoffTests` and
|
||||
`AuditChainVerifierTests` pin every case above.
|
||||
- **External anchor: off-box head export (ScadaBridge#107 item 1) — default OFF.** Inside the
|
||||
database alone, a principal that can rewrite rows AND recompute every later link AND the
|
||||
head produces a chain `verify-chain` accepts. Owner decision: **no signing**; instead
|
||||
@@ -1006,7 +1061,9 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
head), grants the purger the head/tombstone rights its purge paths need, and DENYs
|
||||
INSERT/UPDATE/DELETE on both chain tables to `scadabridge_audit_writer`.
|
||||
- **Parquet archival (T2) — shipped (ScadaBridge#42), default OFF.** See Retention &
|
||||
Purge. The archive carries the hash-chain columns and the chain head, so tamper
|
||||
Purge. The archive carries the hash-chain columns and the chain head (and, for the
|
||||
cutoff month, the chain-start anchor in its tombstone file, so the archive verifies from
|
||||
its own files and splits its unchained rows into pending vs pre-cutoff), so tamper
|
||||
evidence survives the purge: `audit archive verify` ties an archive to the checkpoint
|
||||
anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned by owner
|
||||
decision (ScadaBridge#107/#109); a principal able to rewrite both the archive files and the
|
||||
@@ -1088,6 +1145,7 @@ Security & Tamper-Evidence):
|
||||
"IntervalSeconds": 60,
|
||||
"BatchSize": 1000,
|
||||
"MaxBatchesPerTick": 100,
|
||||
"ChainBacklog": false, // default: chain only rows from first enablement on (#139); true = chain the backlog, first enablement only
|
||||
"HeadExport": {
|
||||
"Enabled": false, // default off
|
||||
"IntervalMinutes": 60,
|
||||
|
||||
Reference in New Issue
Block a user