docs(audit): record the removal of the hash chain and the two-login hardening
Component-AuditLog (Security & Tamper-Evidence now a removal note, archive format v2, config), Component-CLI, Component-ManagementService, HighLevelReqs AL-11/AL-12, docs/components/CLI, the CLI README, alog.md §11.4/§15 and CLAUDE.md. New production script docs/plans/sql/RemoveAuditLogHashChain.sql (idempotent, from migration 64); AddAuditLogHashChain.sql marked superseded. Historical plans, deployment records and deploy-prep packs are left as written. Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
This commit is contained in:
@@ -776,33 +776,28 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
share (UNC path) and its permissions are chosen when the archive is enabled on DEV/QA:
|
||||
both central nodes' service account needs modify rights on it (write, rename, and — for
|
||||
cleanup — delete), auditors read-only; the SQL Server login needs none.
|
||||
- **What.** Per partition, three files sharing the stem
|
||||
- **What.** Per partition, two files sharing the stem
|
||||
`auditlog-{YYYY-MM}-{archivedAt:yyyyMMddTHHmmssfff}Z`: `.parquet` (every row of the
|
||||
partition — the ten stored columns, `OccurredAtUtc` both as a UTC microsecond
|
||||
timestamp and as exact `OccurredAtUtcTicks`, and `ChainSeq`/`RowHash`, null for
|
||||
rows the chaining job had not reached; computed columns are not stored, they derive
|
||||
from `DetailsJson`), `.tombstones.parquet` (the month's per-channel purge
|
||||
tombstones) and `.manifest.json` (row / chained / unchained / tombstone counts,
|
||||
SHA-256 and size of each file, and each chain month's head — `LastSeq`, `LastHash`,
|
||||
and the anchor an earlier purge left). Each file is written to a `.tmp` sibling,
|
||||
flushed and renamed; the manifest goes last and is the commit marker. Archives are
|
||||
never overwritten. Row groups of `RowGroupSize` (default 50,000) bound memory.
|
||||
- **Consistent with the hash chain.** The partition's rows, heads and tombstones are
|
||||
read in one transaction holding the chain lock SHARED, so no chaining pass or purge
|
||||
moves the chain mid-read. The switch then runs with an archive guard: under the
|
||||
EXCLUSIVE chain lock it re-counts the partition and re-reads each month's
|
||||
`LastSeq`, and THROWs (51002 row count changed / 51003 chain advanced — nothing
|
||||
purged) unless both still equal the archive's. The switch's checkpoint anchor
|
||||
(`PurgedThroughSeq/Hash`) therefore equals the archive's recorded chain end. A
|
||||
stale archive keeps manifest status `Archived`; the next tick writes a fresh one.
|
||||
- **Fail closed.** A partition that cannot be archived (volume full, lock busy past
|
||||
`ChainLockTimeoutSeconds`, …) is not purged: the failure takes the usual
|
||||
`AuditLogPurgeFailedEvent` / `PurgeFailures` path and the table keeps the rows.
|
||||
A successful archive publishes `AuditLogArchivedEvent` before `AuditLogPurgedEvent`.
|
||||
timestamp and as exact `OccurredAtUtcTicks`; computed columns are not stored, they
|
||||
derive from `DetailsJson`) and `.manifest.json` (row count, time range, SHA-256 and size
|
||||
of the data file). Each file is written to a `.tmp` sibling, flushed and renamed; the
|
||||
manifest goes last and is the commit marker. Archives are never overwritten. Row groups
|
||||
of `RowGroupSize` (default 50,000) bound memory. This is layout **format v2**; a format-v1
|
||||
archive (written before 2026-11-15, while the hash chain existed) also has `ChainSeq` /
|
||||
`RowHash` data columns, a `.tombstones.parquet` file and chain fields in its manifest —
|
||||
it still lists, verifies and is cleaned up; the chain parts are ignored.
|
||||
- **Consistent with the purge.** The partition's rows are read in one transaction. The
|
||||
switch then runs with an archive guard: it re-counts the partition and THROWs (51002,
|
||||
nothing purged) unless the count still equals the archive's. A stale archive keeps
|
||||
manifest status `Archived`; the next tick writes a fresh one.
|
||||
- **Fail closed.** A partition that cannot be archived (volume full, read timeout, …) is
|
||||
not purged: the failure takes the usual `AuditLogPurgeFailedEvent` / `PurgeFailures`
|
||||
path and the table keeps the rows. A successful archive publishes `AuditLogArchivedEvent`
|
||||
before `AuditLogPurgedEvent`.
|
||||
- **Retention interplay.** Archiving changes nothing about *when* a partition is
|
||||
purged (`RetentionDays`). Per-channel overrides still delete their rows earlier,
|
||||
so those rows are never archived — the archive carries their tombstones instead,
|
||||
which is what keeps the archived chain verifiable.
|
||||
so those rows are never archived.
|
||||
- **Archive retention (ScadaBridge#109) — default never delete.**
|
||||
`AuditLog:Archive:RetentionDays` (default `0` = archives are never deleted, until compliance
|
||||
names a period) is counted from the END of the archived month and, when set, must exceed
|
||||
@@ -810,23 +805,20 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
in the database. After each purge tick the purge actor deletes an expired archive that is
|
||||
`Purged`, or an `Archived` one whose partition no longer holds rows.
|
||||
- **Superseded stale archives (ScadaBridge#109).** A purge whose guard finds the partition
|
||||
changed (51002/51003) leaves its archive at `Archived` (verify reports `NotPurged`, or
|
||||
`Superseded` once a later purge anchored past it) and the next tick archives again. With
|
||||
changed (51002) leaves its archive at `Archived` and the next tick archives again. With
|
||||
`AuditLog:Archive:DeleteSupersededStaleArchives` (default `true`) the purge actor deletes
|
||||
such a stale archive only when a LATER archive of the same partition is `Purged`, verifies,
|
||||
and provably holds every row of it: for every chain month, the same earlier anchor and the
|
||||
stale archive's chain-end hash at its chain-end sequence (row or tombstone), and every
|
||||
unchained row present by EventId. Otherwise it is kept and reported `KeptStale` (e.g. an
|
||||
unchained row a per-channel purge removed in between — no tombstone — so the stale archive
|
||||
is that row's only copy). An archive left at `Archived` because the node stopped between the
|
||||
switch and the manifest update has no later purged archive and is never touched.
|
||||
- **Safe deletion.** Cleanup deletes only the three files a well-formed archive name implies
|
||||
(`{name}.manifest.json`, `.parquet`, `.tombstones.parquet`) in that archive's own
|
||||
and provably holds every row of it (every row present by EventId). Otherwise it is kept and
|
||||
reported `KeptStale` (e.g. a row a per-channel purge removed in between, so the stale
|
||||
archive is that row's only copy). An archive left at `Archived` because the node stopped
|
||||
between the switch and the manifest update has no later purged archive and is never touched.
|
||||
- **Safe deletion.** Cleanup deletes only the files a well-formed archive name implies
|
||||
(`{name}.manifest.json`, `.parquet`, and a format-v1 archive's `.tombstones.parquet`) in that archive's own
|
||||
`{Directory}/{YYYY}` folder — never a path read from a manifest — manifest first, so an
|
||||
interrupted deletion leaves leftovers that are no longer an archive. **Orphans of an interrupted
|
||||
archive WRITE (ScadaBridge#114):** the same pass also deletes, reporting action `DeletedOrphan`
|
||||
(dry-run aware), the writer's own temp files (`{name}.parquet.tmp`,
|
||||
`{name}.tombstones.parquet.tmp`, `{name}.manifest.json.tmp`) and data files whose
|
||||
`{name}.manifest.json.tmp`, and a format-v1 writer's `{name}.tombstones.parquet.tmp`) and data files whose
|
||||
`.manifest.json` was never written — only files named exactly like that for a well-formed
|
||||
archive name, directly inside a four-digit `{Directory}/{YYYY}` folder whose year matches the
|
||||
name, not reparse points, and last written more than 24 h ago (a running write is never
|
||||
@@ -835,16 +827,14 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
(`CleanupAuditArchives`, Administrator) runs the same pass on demand or reports what it
|
||||
would delete. Pinned by `AuditLogArchiveCleanupTests` (mutation-checked: each safety rule
|
||||
removed fails its test).
|
||||
- **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks both
|
||||
files' SHA-256 and counts against the manifest, recomputes each archived month's
|
||||
chain from the archived rows and tombstones (`AuditChainVerifier`, starting from
|
||||
any earlier anchor), and compares the chain end with the database's current anchor:
|
||||
`Matches`; `Superseded` (late rows for the month were chained, purged and archived
|
||||
again afterwards — that later archive resumes from this one's end); `NotPurged`;
|
||||
or `Mismatch` (archive or head altered). The files are plain Parquet — DuckDB,
|
||||
pyarrow, Spark or parquet-tools read them directly. **Owner decision (ScadaBridge#109):
|
||||
reading with Parquet tools is the supported access path; there is no re-import into
|
||||
`dbo.AuditLog`** — a re-inserted expired row would be purged again by the next tick.
|
||||
- **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks the data
|
||||
file's size and SHA-256 and its row count against the manifest. It proves the file is the
|
||||
one the archive wrote, not that its rows are genuine: archives are unsigned, and there is no
|
||||
tamper evidence since the hash chain was removed (see Security). The files are plain
|
||||
Parquet — DuckDB, pyarrow, Spark or parquet-tools read them directly. **Owner decision
|
||||
(ScadaBridge#109): reading with Parquet tools is the supported access path; there is no
|
||||
re-import into `dbo.AuditLog`** — a re-inserted expired row would be purged again by the
|
||||
next tick.
|
||||
- **Sites:** `SiteAuditRetentionService` (site-only `IHostedService`) runs the
|
||||
site SQLite retention purge on a timer — first tick after
|
||||
`AuditLog:SiteRetention:InitialDelay` (default 5 min, short so a daily-recycled
|
||||
@@ -868,207 +858,41 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
|
||||
## Security & Tamper-Evidence
|
||||
|
||||
- **Append-only enforcement — what actually holds in the default deployment.**
|
||||
By default ScadaBridge runs with **one** connection principal for both the writer and the
|
||||
maintenance/purge paths. (A second, maintenance login is supported but opt-in — see
|
||||
*Two-login hardening* below.) So in the default deployment the append-only invariant is
|
||||
enforced by two application-level controls, not by database permissions:
|
||||
> **Removed 2026-11-15.** Tamper-evidence (hash chain, off-box head export) and the two-login
|
||||
> maintenance hardening were removed on 2026-11-15 at IT security's request, pending a
|
||||
> family-wide general solution; migration `RemoveAuditLogHashChain` drops their schema.
|
||||
> The design they shipped with is kept as history in `docs/plans/` and the ScadaBridge#41,
|
||||
> #107 and #139 issues.
|
||||
|
||||
- **Append-only enforcement — what actually holds.**
|
||||
ScadaBridge runs with **one** connection principal for both the writer and the
|
||||
maintenance/purge paths, so the append-only invariant is enforced by two
|
||||
application-level controls, not by database permissions:
|
||||
1. **CI grep guard** — `AuditLogAppendOnlyGuardTests` scans the ConfigurationDatabase
|
||||
source for any `UPDATE`/`DELETE` DML targeting `dbo.AuditLog` and fails the build
|
||||
on a hit. Exactly three mutations are allow-listed by an explicit
|
||||
`// AUDIT-PURGE-ALLOWED` marker (the per-channel retention `DELETE TOP`, the
|
||||
one-time `SourceNode` sentinel backfill `UPDATE`, and the hash-chain link `UPDATE`
|
||||
that sets only `ChainSeq`/`RowHash` while they are NULL — ScadaBridge#41); every
|
||||
other UPDATE/DELETE trips the guard.
|
||||
on a hit. Exactly two mutations are allow-listed by an explicit
|
||||
`// AUDIT-PURGE-ALLOWED` marker (the per-channel retention `DELETE TOP` and the
|
||||
one-time `SourceNode` sentinel backfill `UPDATE`); every other UPDATE/DELETE trips
|
||||
the guard.
|
||||
2. **Code review** — the marker is deliberately specific so an unrelated mutation
|
||||
cannot inherit the exemption without a reviewer noticing.
|
||||
- **Two-login hardening (ScadaBridge#107 item 3) — opt-in.** Migrations provision three
|
||||
roles: `scadabridge_audit_writer` (`INSERT` + `SELECT` on `AuditLog`; `DENY UPDATE` +
|
||||
`DENY DELETE` so a later `db_datawriter` membership cannot silently re-enable mutation;
|
||||
`DENY INSERT/UPDATE/DELETE` on `AuditLogChainHead` and `AuditLogChainTombstone`),
|
||||
`scadabridge_audit_chainer` (`SELECT` on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`,
|
||||
`SELECT/INSERT/UPDATE` on the head, `SELECT` on tombstones) and `scadabridge_audit_purger`
|
||||
(`SELECT` + `ALTER ON SCHEMA::dbo` + `CREATE TABLE` for the switch-out staging table,
|
||||
`DELETE ON dbo.AuditLog` for the per-channel purge — `FixAuditPurgerRoleGrants`, arch-review 04
|
||||
S3 — and `SELECT/UPDATE` on the head, `SELECT/INSERT/DELETE` on tombstones). Because the writer
|
||||
role's DENYs win over any grant, the chain job and the purges **cannot** run on a login that is
|
||||
in the writer role. Set `AuditLog:MaintenanceConnectionString` (central nodes; `${secret:}`
|
||||
references resolve, as for every pre-host key) and the hash-chain job, chain verification /
|
||||
head export, the partition switch-out (with its Parquet archive) and the per-channel DELETE run
|
||||
on their own connection with that string (`AuditMaintenanceContext`); unset, they run on
|
||||
`ScadaBridge:Database:ConfigurationDb` exactly as before. A DBA provisions:
|
||||
|
||||
| Login | Used by | Database roles |
|
||||
|---|---|---|
|
||||
| runtime (writer) — `ScadaBridge:Database:ConfigurationDb` | everything else, incl. audit ingest | `db_datareader`, `db_datawriter`, `scadabridge_audit_writer` (+ whatever the rest of the app needs) |
|
||||
| maintenance — `AuditLog:MaintenanceConnectionString` | chain job, verify-chain / verify-heads, head export, partition switch-out + archive, per-channel purge | `scadabridge_audit_chainer`, `scadabridge_audit_purger` — and NOT `scadabridge_audit_writer` |
|
||||
|
||||
```sql
|
||||
CREATE USER [svc_scadabridge_audit_maint] FOR LOGIN [svc_scadabridge_audit_maint];
|
||||
ALTER ROLE scadabridge_audit_chainer ADD MEMBER [svc_scadabridge_audit_maint];
|
||||
ALTER ROLE scadabridge_audit_purger ADD MEMBER [svc_scadabridge_audit_maint];
|
||||
ALTER ROLE scadabridge_audit_writer ADD MEMBER [svc_scadabridge_app]; -- the runtime login
|
||||
```
|
||||
|
||||
`AuditMaintenanceLoginTests` runs exactly this setup on SQL Server: the writer ingests and is
|
||||
refused the chain UPDATE and the per-channel DELETE; the maintenance login chains, verifies,
|
||||
reads heads, purges per channel, enumerates partitions, archives and switches out. **Also on the
|
||||
maintenance login (ScadaBridge#113):** the monthly partition roll-forward
|
||||
(`ALTER PARTITION FUNCTION … SPLIT RANGE`, `AuditLogPartitionMaintenance` now built on
|
||||
`AuditMaintenanceContext`) and `audit backfill-source-node` (the management endpoint resolves the
|
||||
keyed maintenance `IAuditLogRepository`). Migration `AuditMaintenanceGrants` gives
|
||||
`scadabridge_audit_purger` the two grants they need that `ALTER ON SCHEMA::dbo` does not confer:
|
||||
database-level `ALTER ANY DATASPACE` (roll-forward) and column-level `UPDATE (SourceNode)` on
|
||||
`dbo.AuditLog` (backfill; no other column becomes updatable, the writer's DENY still wins).
|
||||
Unset `AuditLog:MaintenanceConnectionString` keeps both on the runtime login as before — which
|
||||
then needs those same two grants (or db_owner), exactly as the single-login default always did.
|
||||
`AuditMaintenanceLoginTests` pins both with a writer-refused negative control. Migrations themselves are applied by a DBA with the production SQL scripts.
|
||||
- **Database roles.** Migrations still provision `scadabridge_audit_writer` (`INSERT` +
|
||||
`SELECT` on `AuditLog`; `DENY UPDATE` + `DENY DELETE` so a later `db_datawriter`
|
||||
membership cannot silently re-enable mutation) and `scadabridge_audit_purger` (`SELECT` +
|
||||
`ALTER ON SCHEMA::dbo` + `CREATE TABLE` for the switch-out staging table, `DELETE ON
|
||||
dbo.AuditLog` for the per-channel purge — `FixAuditPurgerRoleGrants`, arch-review 04 S3).
|
||||
The application does not use them: every path, purge and archive included, runs on
|
||||
`ScadaBridge:Database:ConfigurationDb`, which therefore needs the rights of both (or
|
||||
db_owner).
|
||||
- **Authorization.** Reading the Audit Log requires the existing **Audit** role
|
||||
extended with a new **OperationalAudit** permission. Per-site row scoping
|
||||
reuses the existing site-permission model; bulk export requires an additional
|
||||
**AuditExport** permission.
|
||||
- **Payload redaction at write.** See Payload Capture Policy. Unredacted
|
||||
secrets never persist; the safety net over-redacts on misconfiguration.
|
||||
- **Hash-chain tamper evidence (T1) — shipped (ScadaBridge#41), default OFF.**
|
||||
`AuditLog:HashChain:Enabled` (default `false`, plus `IntervalSeconds` 60,
|
||||
`BatchSize` 1000, `MaxBatchesPerTick` 100) starts the central
|
||||
`AuditLogHashChainService` on both central nodes.
|
||||
- **Model.** One chain per **partition = UTC calendar month of `OccurredAtUtc`**
|
||||
(the `ps_AuditLog_Month` month). Each chained row carries `ChainSeq` (1, 2, 3 …,
|
||||
gap-free within its month) and `RowHash = SHA-256(prev.RowHash || canonical(row))`,
|
||||
`prev` being the row at `ChainSeq - 1` (32 zero bytes for the first — or, in the month
|
||||
the chain started in, the chain-start anchor below). The canonical
|
||||
v1 form (`AuditChainHasher`) is a domain tag, the month and sequence number, then
|
||||
the ten writable columns in table order, each string length-prefixed with a NULL
|
||||
marker; the computed columns derive from `DetailsJson` and are covered by it. A
|
||||
known-answer test pins the encoding — changing it invalidates every stored chain.
|
||||
- **Who writes the chain.** Not the hot path: the two central nodes and the site
|
||||
ingest INSERT concurrently, so a write-time chain would need a global serialisation
|
||||
point on every audit write. A **chaining job** links unchained rows (oldest month
|
||||
first, in `OccurredAtUtc, EventId` order within a pass) under an exclusive SQL
|
||||
Server application lock (`sp_getapplock`, zero timeout — the peer node's pass just
|
||||
skips its tick), with an optimistic head check and a `ChainSeq IS NULL` row-count
|
||||
check that roll the pass back rather than leave a gap. This is alog.md §11.4's
|
||||
"chaining job", run every minute instead of only after a month closes, so a row is
|
||||
tamper-evident about a minute after it reaches central. Chain order is chaining
|
||||
order, so late-arriving site rows simply take the next sequence number.
|
||||
- **Purges.** `dbo.AuditLogChainHead` holds each month's `(LastSeq, LastHash)` — which
|
||||
is what exposes deletion of the newest rows — and the checkpoint anchor
|
||||
`(PurgedThroughSeq, PurgedThroughHash)` that the monthly partition switch sets
|
||||
inside its own transaction; verification resumes after it. The per-channel retention
|
||||
DELETE writes a `dbo.AuditLogChainTombstone` (month, `ChainSeq`, `RowHash`,
|
||||
`EventId`) for every chained row it removes, in the same transaction, via
|
||||
`OUTPUT … INTO`; verification bridges the gap with it. Both purge paths and the
|
||||
SourceNode backfill hold the same lock exclusively. The backfill skips chained rows
|
||||
(a chained row's `SourceNode` is frozen) — run it before enabling the chain.
|
||||
- **Chain start: no backlog chaining (ScadaBridge#139; owner decision on #107,
|
||||
2026-11-13).** Enabling the job on a database with history does **not** chain the
|
||||
backlog. The first chaining pass on a pristine database (no chain head, no chained row)
|
||||
writes a **chain-start anchor** and from then on only rows with
|
||||
`OccurredAtUtc >= cutoff` are chained; older rows keep `ChainSeq`/`RowHash` NULL for
|
||||
ever and are out of scope.
|
||||
- *Where the cutoff lives.* One row in `dbo.AuditLogChainTombstone` at `ChainSeq = 0`
|
||||
of the cutoff month (real links start at 1, so slot 0 is "the link before the
|
||||
first"): `PurgedAtUtc` = the cutoff, `RowHash` =
|
||||
`SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks)`, `EventId` =
|
||||
`5cadab1d-c4a1-4c0f-a000-000000000139`. The cutoff is the database's
|
||||
`SYSUTCDATETIME()` at that first pass, written once inside the chaining transaction
|
||||
under the exclusive chain lock — so both central nodes read the same instant whatever
|
||||
their own clocks or config say, and it survives restarts. It is never updated; the
|
||||
partition switch-out keeps it (it deletes only `ChainSeq > 0` tombstones); the head
|
||||
store's tombstone count, the head export and `verify-heads` ignore it. **No
|
||||
migration**: the table and the grants exist (the maintenance login writes it through
|
||||
`scadabridge_audit_purger`'s INSERT on the tombstone table).
|
||||
- *Bound into the chain.* The anchor hash is the genesis the cutoff month's first row
|
||||
links to (instead of 32 zero bytes). Moving the cutoff therefore breaks that month's
|
||||
chain, and the month's exported head anchors the cutoff off-box like any other link.
|
||||
- *`AuditLog:HashChain:ChainBacklog`* (default `false`) restores the old behaviour —
|
||||
chain every row, oldest month first, no anchor — but only on the very first
|
||||
enablement: once an anchor or any chained row exists the decision is fixed and the
|
||||
option is ignored, so two centrals with different values still end up with one
|
||||
chain (whichever chains first decides). A database whose chain was enabled before
|
||||
this change (heads, no anchor) keeps chaining every row, as before.
|
||||
- *Late rows.* The cutoff is on `OccurredAtUtc` (the partition key, indexed), not on
|
||||
ingest time (`IngestedAtUtc` is a non-persisted JSON column that must never be
|
||||
filtered on). A site row dated before the cutoff that reaches central after
|
||||
enablement — the site's store-and-forward backlog at that moment — stays out of
|
||||
scope; so does a row from a node whose clock runs behind the database's by the
|
||||
skew. Both are bounded and reported, never silently dropped.
|
||||
- *Index.* Pre-cutoff rows stay in the filtered `IX_AuditLog_Unchained` until retention
|
||||
purges their partition; the chaining read seeks `OccurredAtUtc >= cutoff`, so they
|
||||
cost storage only, not chaining time.
|
||||
- Verification reports in-scope rows not chained yet as `UnchainedRows` and the
|
||||
backlog as `PreCutoffRows` (plus `ChainFromUtc`); neither is ever a break.
|
||||
- **Verification.** `scadabridge audit verify-chain --month YYYY-MM` sends the
|
||||
`VerifyAuditChain` management command (OperationalAudit roles; refused for a
|
||||
site-scoped caller because a month's chain spans every site; long-running Ask). It
|
||||
takes the lock SHARED, recomputes every link in sequence order and reports `Intact`,
|
||||
`NotChained` or `Broken` with the **first break**: partition, `ChainSeq`, `EventId`
|
||||
and kind — `HashMismatch` (an edited row, or rows reordered — the sequence number is
|
||||
inside the hashed bytes), `MissingRow` (a deleted row with no tombstone),
|
||||
`DuplicateSequence`, `IncompleteLink`, `HeadMismatch` (rows deleted from the end),
|
||||
`ChainedBeforeCutoff` or `CutoffAnchorInvalid` (below). CLI exit code 3 on `Broken`;
|
||||
the table output also prints `Chain from` and `Pre-cutoff rows`.
|
||||
- **Cutoff-eligibility attacks (ScadaBridge#139).** Because pre-cutoff rows are
|
||||
unprotected by design, an attacker would like to (a) move a chained row's
|
||||
`OccurredAtUtc` back before the cutoff to hide it among them, or (b) move the cutoff
|
||||
forward. (a) with the link left in place is `ChainedBeforeCutoff` (no row before the
|
||||
cutoff is ever chained — this also catches an attacker who re-hashes the row and every
|
||||
later link); with the link cleared too, its sequence number is a `MissingRow` (or
|
||||
`HeadMismatch` at the tail); moved into an earlier month, the source month reports
|
||||
`MissingRow` and the target month `ChainedBeforeCutoff`. (b) editing the anchor's
|
||||
cutoff without its hash, filing it under another month, or adding a second anchor is
|
||||
`CutoffAnchorInvalid` (in every month's verification); re-hashing the anchor to the new
|
||||
cutoff leaves the already-chained rows before it (`ChainedBeforeCutoff`) and changes the
|
||||
cutoff month's genesis (`HashMismatch` at `ChainSeq` 1); deleting the anchor is a
|
||||
`HashMismatch` at `ChainSeq` 1. **Known limits:** pre-cutoff rows themselves carry no
|
||||
tamper evidence (that is the decision), and a backdated row forged *before* the cutoff
|
||||
is indistinguishable from a genuine backlog row — the chain never proved a row genuine,
|
||||
only that a chained row was not changed after chaining. A principal that rewrites the
|
||||
anchor, the cutoff month's whole chain and its head is caught only by the off-box head
|
||||
export, as for any full-chain rewrite. `AuditLogHashChainCutoffTests` and
|
||||
`AuditChainVerifierTests` pin every case above.
|
||||
- **External anchor: off-box head export (ScadaBridge#107 item 1) — default OFF.** Inside the
|
||||
database alone, a principal that can rewrite rows AND recompute every later link AND the
|
||||
head produces a chain `verify-chain` accepts. Owner decision: **no signing**; instead
|
||||
`AuditLog:HashChain:HeadExport:Enabled` makes both central nodes, every `IntervalMinutes`
|
||||
(60), read every month's head (`LastSeq`, `LastHash`, the purge anchor, live chained rows,
|
||||
tombstones — one snapshot under the chain lock SHARED) and append the heads that changed
|
||||
since that node's last export (all of them after a restart; `ExportUnchanged` for every
|
||||
tick) to a sink the database login cannot write: `Sink: File` — append-only JSON lines in
|
||||
`{File:Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl` (local or UNC; opened
|
||||
`FileMode.Append` only, one file per node per day; give the service account create +
|
||||
append-data rights and nothing else), or `Sink: Syslog` — RFC 5424 (`<PRI>1 … chainhead -
|
||||
{json}`, facility 13 "log audit", severity Notice) over UDP (RFC 5426) or TCP (RFC 6587
|
||||
octet counting). A failed write is logged and retried with the same heads next tick, and (ScadaBridge#115)
|
||||
is also a health signal: counter `scadabridge.audit.chain_head_export.failures` on the shared
|
||||
ScadaBridge meter, plus a `/health/ready` entry `audit-chain-head-export` that is Degraded (HTTP
|
||||
200, Overview-visible, not out of rotation) while the last tick failed or no tick has succeeded
|
||||
for more than two intervals; Healthy when the export is disabled. UDP syslog is fire-and-forget,
|
||||
so a dropped datagram is invisible by design — use `Protocol: Tcp` where that matters.
|
||||
`scadabridge audit verify-heads --path <file|dir>` reads those records back (file-sink
|
||||
lines or a syslog collector's log) and the `VerifyAuditChainHeads` command checks each
|
||||
against the database: the link stored at the exported sequence (the head, a live row's
|
||||
`RowHash`, a tombstone, or the purge anchor) must still carry the exported hash —
|
||||
`Matches` / `Extended` / `Purged` (removed by a partition switch; verify the archive), or
|
||||
the failures `Mismatch` (rewritten chain), `Regressed` (head rolled back), `MissingHead`,
|
||||
`MissingLink` (chained row deleted outside the purge paths), `CountMismatch` (same head,
|
||||
rows + tombstones changed). `AuditChainHeadStoreTests` forges a chain with the production
|
||||
hasher: `verify-chain` reports it Intact, `verify-heads` reports Mismatch. Coverage is from
|
||||
the first export on: a forgery of links never exported is not detectable this way.
|
||||
- **Roles (DBA hardening).** `AddAuditLogHashChain` adds `scadabridge_audit_chainer`
|
||||
(SELECT on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`, read/write on the
|
||||
head), grants the purger the head/tombstone rights its purge paths need, and DENYs
|
||||
INSERT/UPDATE/DELETE on both chain tables to `scadabridge_audit_writer`.
|
||||
- **Parquet archival (T2) — shipped (ScadaBridge#42), default OFF.** See Retention &
|
||||
Purge. The archive carries the hash-chain columns and the chain head (and, for the
|
||||
cutoff month, the chain-start anchor in its tombstone file, so the archive verifies from
|
||||
its own files and splits its unchained rows into pending vs pre-cutoff), so tamper
|
||||
evidence survives the purge: `audit archive verify` ties an archive to the checkpoint
|
||||
anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned by owner
|
||||
decision (ScadaBridge#107/#109); a principal able to rewrite both the archive files and the
|
||||
head row can forge a consistent pair — unless the month's head was exported off-box first
|
||||
(`AuditLog:HashChain:HeadExport`): `audit verify-heads` then exposes the rewritten anchor.
|
||||
Purge. Archives are unsigned by owner decision (ScadaBridge#109) and carry no tamper
|
||||
evidence: `audit archive verify` proves file integrity against the manifest only.
|
||||
- **Site SQLite security.** File permissions: read/write by the ScadaBridge
|
||||
service account only. Not backed up off-machine — site SQLite is a buffer,
|
||||
not a record.
|
||||
@@ -1132,31 +956,10 @@ the global window; `PerChannelRetentionDays` specifies per-channel windows that
|
||||
are strictly shorter — any channel whose override equals or exceeds the global
|
||||
value is silently ignored (the global partition switch-out already governs it).
|
||||
|
||||
The nested `AuditLog:HashChain` section (ScadaBridge#41) runs the chaining job; its
|
||||
`HeadExport` sub-section (ScadaBridge#107) exports the chain heads off-box, and the root
|
||||
`AuditLog:MaintenanceConnectionString` moves the maintenance paths to a second login (see
|
||||
Security & Tamper-Evidence):
|
||||
|
||||
```jsonc
|
||||
"AuditLog": {
|
||||
"MaintenanceConnectionString": "${secret:SB-AUDIT-MAINT-DB}", // optional; unset = runtime connection
|
||||
"HashChain": {
|
||||
"Enabled": false, // default off
|
||||
"IntervalSeconds": 60,
|
||||
"BatchSize": 1000,
|
||||
"MaxBatchesPerTick": 100,
|
||||
"ChainBacklog": false, // default: chain only rows from first enablement on (#139); true = chain the backlog, first enablement only
|
||||
"HeadExport": {
|
||||
"Enabled": false, // default off
|
||||
"IntervalMinutes": 60,
|
||||
"ExportUnchanged": false, // true = every head every tick
|
||||
"Sink": "File", // File | Syslog
|
||||
"File": { "Directory": "\\\\fileserver\\audit-heads" }, // absolute, local or UNC
|
||||
"Syslog": { "Host": "syslog.example", "Port": 514, "Protocol": "Udp", "Facility": 13, "AppName": "scadabridge", "TimeoutSeconds": 10 }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
`AuditLog:HashChain` (with its `HeadExport` sub-section) and
|
||||
`AuditLog:MaintenanceConnectionString` were removed on 2026-11-15 with the hash chain. A node
|
||||
that still carries them boots normally and logs one startup warning per key
|
||||
(`RetiredAuditSettingsWarning`): delete them.
|
||||
|
||||
The nested `AuditLog:Archive` section (ScadaBridge#42) turns on the Parquet cold
|
||||
archive (see Retention & Purge):
|
||||
@@ -1168,7 +971,6 @@ archive (see Retention & Purge):
|
||||
"Directory": "D:\\AuditArchive", // absolute; required when Enabled; reachable by both centrals
|
||||
"RowGroupSize": 50000, // 1,000 – 1,000,000
|
||||
"ReadCommandTimeoutMinutes": 60, // reading one partition out of SQL Server
|
||||
"ChainLockTimeoutSeconds": 300, // wait for a chaining pass before skipping the tick
|
||||
"RetentionDays": 0, // 0 = never delete; else > AuditLog:RetentionDays, from the archived month's end (#109)
|
||||
"DeleteSupersededStaleArchives": true // delete stale archives a later purged archive provably covers (#109)
|
||||
}
|
||||
@@ -1285,9 +1087,8 @@ orphaned entries) and in the CLI's `audit tree` output.
|
||||
- **[CLI (#19)](Component-CLI.md)** — `scadabridge audit query`,
|
||||
`scadabridge audit export`, `scadabridge audit tree --execution-id <guid>`,
|
||||
`scadabridge audit backfill-source-node --sentinel <s> --before <date>`, and
|
||||
`scadabridge audit verify-chain --month <YYYY-MM>` (hash-chain verification via the
|
||||
`VerifyAuditChain` management command, ScadaBridge#41); same permission requirements
|
||||
as the UI.
|
||||
`scadabridge audit archive list|verify|cleanup` (Parquet archives, ScadaBridge#42/#109);
|
||||
same permission requirements as the UI.
|
||||
- **[KPI History (#26)](Component-KpiHistory.md)** — emits `IKpiSampleSource`
|
||||
(`AuditLogKpiSampleSource`, Global) consumed by the KpiHistory recorder (#26),
|
||||
reusing the existing audit-KPI reads; the resulting `totalEventsLastHour` /
|
||||
|
||||
Reference in New Issue
Block a user