dohertj2/ScadaBridge
1
0
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audit): off-box chain-head export, verify-heads, maintenance login (ScadaBridge#107)

Browse Source 
Item 1 (owner decision: no signing). AuditLog:HashChain:HeadExport (default
off): both centrals periodically read every month's chain head (LastSeq,
LastHash, purge anchor, chained rows, tombstones; one snapshot under the
chain lock SHARED) and append the changed heads to a sink the DB login cannot
write - File (append-only JSONL per node per day, local or UNC) or Syslog
(RFC 5424 over UDP / TCP octet counting). `audit verify-heads --path` reads
those records back (file or syslog-collector lines) and the new
VerifyAuditChainHeads command checks each against the database: Matches /
Extended / Purged, or Mismatch / Regressed / MissingHead / MissingLink /
CountMismatch (exit 3). AuditChainHeadStoreTests forges a chain with the
production hasher: verify-chain says Intact, verify-heads says Mismatch.

Item 3. AuditLog:MaintenanceConnectionString (${secret:}-capable, unset =
today's single login) runs the chain job, chain verify / head export, the
partition switch-out + archive and the per-channel purge on
AuditMaintenanceContext (keyed audit-maintenance repository/archive, purge
actor prefers them). AuditMaintenanceLoginTests runs the hardened setup on
SQL Server: writer (datareader/datawriter/audit_writer) ingests and is
refused chaining and deleting; the chainer+purger login does the full cycle.
Two-login grants documented in Component-AuditLog.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 23:16:05 -05:00
parent 5e7d5389d4
commit c4d7ec9b96
40 changed files with 2769 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/requirements/Component-AuditLog.md
+86 -21
View File
@@ -833,9 +833,9 @@ MS SQL for direct-write events). Unredacted secrets never persist.
## Security & Tamper-Evidence
- **Append-only enforcement — what actually holds in the default deployment.**
ScadaBridge runs with **one** connection principal for both the writer and the
maintenance/purge paths (the runtime does not open a second, lower-privilege
connection for purge). So in the default deployment the append-only invariant is
By default ScadaBridge runs with **one** connection principal for both the writer and the
maintenance/purge paths. (A second, maintenance login is supported but opt-in — see
*Two-login hardening* below.) So in the default deployment the append-only invariant is
enforced by two application-level controls, not by database permissions:
1. **CI grep guard** — `AuditLogAppendOnlyGuardTests` scans the ConfigurationDatabase
source for any `UPDATE`/`DELETE` DML targeting `dbo.AuditLog` and fails the build
@@ -846,20 +846,41 @@ MS SQL for direct-write events). Unredacted secrets never persist.
other UPDATE/DELETE trips the guard.
2. **Code review** — the marker is deliberately specific so an unrelated mutation
cannot inherit the exemption without a reviewer noticing.
- **Optional DBA hardening (two DB roles).** For a deployment that wants
database-level enforcement on top of the application controls, migrations provision
two roles: `scadabridge_audit_writer` (`INSERT` + `SELECT` only; `DENY UPDATE` +
`DENY DELETE` so a later `db_datawriter` membership cannot silently re-enable
mutation) and `scadabridge_audit_purger` (the maintenance principal). To use them,
a DBA provisions **two logins** — the runtime connection mapped to the writer role,
and a separate maintenance job/connection mapped to the purger role. The purger role
now carries the permissions its purge path genuinely needs: `SELECT` + `ALTER ON
SCHEMA::dbo` **plus** `CREATE TABLE` (the switch-out dance CREATEs a staging table)
and a scoped `DELETE ON dbo.AuditLog` (the per-channel retention override is a bounded
row DELETE) — granted by the `FixAuditPurgerRoleGrants` migration (arch-review 04, S3).
Without those two grants the purger role could not actually execute the switch-out or
the per-channel purge; the earlier claim that "row-level `DELETE` is not granted even
to purge" was stale once `PerChannelRetentionDays` shipped.
- **Two-login hardening (ScadaBridge#107 item 3) — opt-in.** Migrations provision three
roles: `scadabridge_audit_writer` (`INSERT` + `SELECT` on `AuditLog`; `DENY UPDATE` +
`DENY DELETE` so a later `db_datawriter` membership cannot silently re-enable mutation;
`DENY INSERT/UPDATE/DELETE` on `AuditLogChainHead` and `AuditLogChainTombstone`),
`scadabridge_audit_chainer` (`SELECT` on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`,
`SELECT/INSERT/UPDATE` on the head, `SELECT` on tombstones) and `scadabridge_audit_purger`
(`SELECT` + `ALTER ON SCHEMA::dbo` + `CREATE TABLE` for the switch-out staging table,
`DELETE ON dbo.AuditLog` for the per-channel purge — `FixAuditPurgerRoleGrants`, arch-review 04
S3 — and `SELECT/UPDATE` on the head, `SELECT/INSERT/DELETE` on tombstones). Because the writer
role's DENYs win over any grant, the chain job and the purges **cannot** run on a login that is
in the writer role. Set `AuditLog:MaintenanceConnectionString` (central nodes; `${secret:}`
references resolve, as for every pre-host key) and the hash-chain job, chain verification /
head export, the partition switch-out (with its Parquet archive) and the per-channel DELETE run
on their own connection with that string (`AuditMaintenanceContext`); unset, they run on
`ScadaBridge:Database:ConfigurationDb` exactly as before. A DBA provisions:
| Login | Used by | Database roles |
|---|---|---|
| runtime (writer) — `ScadaBridge:Database:ConfigurationDb` | everything else, incl. audit ingest | `db_datareader`, `db_datawriter`, `scadabridge_audit_writer` (+ whatever the rest of the app needs) |
| maintenance — `AuditLog:MaintenanceConnectionString` | chain job, verify-chain / verify-heads, head export, partition switch-out + archive, per-channel purge | `scadabridge_audit_chainer`, `scadabridge_audit_purger` — and NOT `scadabridge_audit_writer` |
```sql
CREATE USER [svc_scadabridge_audit_maint] FOR LOGIN [svc_scadabridge_audit_maint];
ALTER ROLE scadabridge_audit_chainer ADD MEMBER [svc_scadabridge_audit_maint];
ALTER ROLE scadabridge_audit_purger ADD MEMBER [svc_scadabridge_audit_maint];
ALTER ROLE scadabridge_audit_writer ADD MEMBER [svc_scadabridge_app]; -- the runtime login
```
`AuditMaintenanceLoginTests` runs exactly this setup on SQL Server: the writer ingests and is
refused the chain UPDATE and the per-channel DELETE; the maintenance login chains, verifies,
reads heads, purges per channel, enumerates partitions, archives and switches out. Not moved
to the maintenance login: the monthly partition roll-forward (`ALTER PARTITION FUNCTION`,
needs `ALTER ANY DATASPACE` on the runtime login, or run it as a DBA job) and the one-time
`audit backfill-source-node` (needs `UPDATE` on `SourceNode` — a DBA-run step in a hardened
setup). Migrations themselves are applied by a DBA with the production SQL scripts.
- **Authorization.** Reading the Audit Log requires the existing **Audit** role
extended with a new **OperationalAudit** permission. Per-site row scoping
reuses the existing site-permission model; bulk export requires an additional
@@ -910,10 +931,29 @@ MS SQL for direct-write events). Unredacted secrets never persist.
inside the hashed bytes), `MissingRow` (a deleted row with no tombstone),
`DuplicateSequence`, `IncompleteLink`, or `HeadMismatch` (rows deleted from the end).
CLI exit code 3 on `Broken`.
- **What it does NOT prove.** The chain is unsigned and anchored only inside the same
database: a principal that can rewrite rows AND recompute every later link AND the
head can produce a consistent forgery. Signing the head (key custody) and anchoring
it outside the database are open owner decisions (ScadaBridge#41).
- **External anchor: off-box head export (ScadaBridge#107 item 1) — default OFF.** Inside the
database alone, a principal that can rewrite rows AND recompute every later link AND the
head produces a chain `verify-chain` accepts. Owner decision: **no signing**; instead
`AuditLog:HashChain:HeadExport:Enabled` makes both central nodes, every `IntervalMinutes`
(60), read every month's head (`LastSeq`, `LastHash`, the purge anchor, live chained rows,
tombstones — one snapshot under the chain lock SHARED) and append the heads that changed
since that node's last export (all of them after a restart; `ExportUnchanged` for every
tick) to a sink the database login cannot write: `Sink: File` — append-only JSON lines in
`{File:Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl` (local or UNC; opened
`FileMode.Append` only, one file per node per day; give the service account create +
append-data rights and nothing else), or `Sink: Syslog` — RFC 5424 (`<PRI>1 … chainhead -
{json}`, facility 13 "log audit", severity Notice) over UDP (RFC 5426) or TCP (RFC 6587
octet counting). A failed write is logged and retried with the same heads next tick.
`scadabridge audit verify-heads --path <file|dir>` reads those records back (file-sink
lines or a syslog collector's log) and the `VerifyAuditChainHeads` command checks each
against the database: the link stored at the exported sequence (the head, a live row's
`RowHash`, a tombstone, or the purge anchor) must still carry the exported hash —
`Matches` / `Extended` / `Purged` (removed by a partition switch; verify the archive), or
the failures `Mismatch` (rewritten chain), `Regressed` (head rolled back), `MissingHead`,
`MissingLink` (chained row deleted outside the purge paths), `CountMismatch` (same head,
rows + tombstones changed). `AuditChainHeadStoreTests` forges a chain with the production
hasher: `verify-chain` reports it Intact, `verify-heads` reports Mismatch. Coverage is from
the first export on: a forgery of links never exported is not detectable this way.
- **Roles (DBA hardening).** `AddAuditLogHashChain` adds `scadabridge_audit_chainer`
(SELECT on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`, read/write on the
head), grants the purger the head/tombstone rights its purge paths need, and DENYs
@@ -987,6 +1027,31 @@ the global window; `PerChannelRetentionDays` specifies per-channel windows that
are strictly shorter — any channel whose override equals or exceeds the global
value is silently ignored (the global partition switch-out already governs it).
The nested `AuditLog:HashChain` section (ScadaBridge#41) runs the chaining job; its
`HeadExport` sub-section (ScadaBridge#107) exports the chain heads off-box, and the root
`AuditLog:MaintenanceConnectionString` moves the maintenance paths to a second login (see
Security & Tamper-Evidence):
```jsonc
"AuditLog": {
"MaintenanceConnectionString": "${secret:SB-AUDIT-MAINT-DB}", // optional; unset = runtime connection
"HashChain": {
"Enabled": false, // default off
"IntervalSeconds": 60,
"BatchSize": 1000,
"MaxBatchesPerTick": 100,
"HeadExport": {
"Enabled": false, // default off
"IntervalMinutes": 60,
"ExportUnchanged": false, // true = every head every tick
"Sink": "File", // File | Syslog
"File": { "Directory": "\\\\fileserver\\audit-heads" }, // absolute, local or UNC
"Syslog": { "Host": "syslog.example", "Port": 514, "Protocol": "Udp", "Facility": 13, "AppName": "scadabridge", "TimeoutSeconds": 10 }
}
}
}
```
The nested `AuditLog:Archive` section (ScadaBridge#42) turns on the Parquet cold
archive (see Retention & Purge):
docs/requirements/Component-CLI.md
+14 -1
View File
@@ -304,7 +304,7 @@ The `scadabridge audit` group targets the centralized Audit Log component (#23)
exposes the UI-equivalent operational audit surface. Permissions follow the same
read-vs-export split the Central UI uses (see Component-AuditLog.md, Security &
Tamper-Evidence, and Security & Auth #10): `audit query`, `audit tree`, and
`audit verify-chain` and `audit archive list|verify` require the `OperationalAudit`
`audit verify-chain`, `audit verify-heads` and `audit archive list|verify` require the `OperationalAudit`
permission; `audit export`
additionally requires `AuditExport`; `audit backfill-source-node` requires the
`Admin` role (maintenance path only). The server enforces permission checks and
@@ -316,6 +316,7 @@ scadabridge audit export --since <t> --until <t> --format csv|jsonl|parquet --ou
scadabridge audit tree --execution-id <guid> [--format table|json]
scadabridge audit backfill-source-node --before <ISO-8601-UTC> [--sentinel <value>] [--batch <n>]
scadabridge audit verify-chain --month <YYYY-MM>
scadabridge audit verify-heads --path <file-or-directory>
scadabridge audit archive list
scadabridge audit archive verify --name <archive>
```
@@ -350,6 +351,18 @@ scadabridge audit archive verify --name <archive>
`0` intact or not chained, `3` broken, `1` error, `2` authorization failure. Requires
`OperationalAudit` and a caller not scoped to specific sites. See
Component-AuditLog.md, Security & Tamper-Evidence.
- `audit verify-heads --path <file-or-directory>` (ScadaBridge#107) — reads the AuditLog
chain heads exported off-box by `AuditLog:HashChain:HeadExport` (JSON lines from the file
sink, or any log a syslog collector wrote — each line's JSON object is taken from its first
`{` to its last `}`; a directory is read recursively for `*.jsonl`/`*.log`/`*.txt`),
collapses duplicates and sends them in the `VerifyAuditChainHeads` management command. The
server checks each exported head against the database chain — `Matches`, `Extended` (chain
grew; the link at the exported sequence still carries the exported hash), `Purged` (removed
by a partition purge; verify the archive), or the failures `Mismatch` (rewritten chain),
`Regressed`, `MissingHead`, `MissingLink`, `CountMismatch`. Exit `0` verified, `3` a
failure, `1` error (no records found, bad input, `CHAIN_BUSY`), `2` authorization failure.
Same gate as `verify-chain`. Run it together with `verify-chain`: a forger who rewrites rows
and re-hashes the chain inside SQL Server passes `verify-chain` but not `verify-heads`.
- `audit archive list` / `audit archive verify --name <archive>` (ScadaBridge#42) —
send the `ListAuditArchives` / `VerifyAuditArchive` management commands for the
AuditLog Parquet cold archives the retention purge writes (`AuditLog:Archive`,
docs/requirements/Component-ManagementService.md
+1
View File
@@ -200,6 +200,7 @@ The two-person authorization workflow for writes through the MxAccess Gateway. B
- **QueryAuditLog**: Legacy configuration-change audit query (filtered by entity type, user, date range, etc.) routed through `/management`. Gated to the `Admin` role; superseded for the centralized Audit Log component (#23) by the dedicated `/api/audit/*` REST endpoints described below.
- **VerifyAuditChain** (ScadaBridge#41): verifies one UTC calendar month (`Month`, `YYYY-MM`) of the central AuditLog hash chain and returns an `AuditChainVerificationResult` (status, rows verified, tombstones applied, purge anchor, head sequence, unchained rows, first break with partition / `ChainSeq` / `EventId` / kind). Gated on the OperationalAudit roles (`Administrator`, `Viewer`); a site-scoped non-Administrator caller is refused (`ManagementUnauthorized`) because a month's chain spans every site. Long-running Ask timeout. Errors: `INVALID_ARGUMENT` (bad month), `CHAIN_BUSY` (a chaining/purge pass held the chain lock past 60 s). Backs `scadabridge audit verify-chain`.
- **VerifyAuditChainHeads** (ScadaBridge#107): checks chain heads exported off-box (`Records`, at most 50,000 `AuditChainHeadRecord`s; duplicates collapsed) against the database chain and returns an `AuditChainHeadVerificationResult` (overall `Verified`, one `AuditChainHeadCheck` per distinct exported head: `Matches` / `Extended` / `Purged`, or the failures `Mismatch` / `Regressed` / `MissingHead` / `MissingLink` / `CountMismatch`). Same gate as VerifyAuditChain; long-running Ask timeout. Errors: `INVALID_ARGUMENT` (no records / over the cap), `CHAIN_BUSY`. Backs `scadabridge audit verify-heads`.
- **ListAuditArchives** / **VerifyAuditArchive** (ScadaBridge#42): list the AuditLog Parquet cold archives under `AuditLog:Archive:Directory` (`AuditArchiveListResult`: enabled flag + every manifest), and verify one by `Name` (`AuditArchiveVerificationResult`: file integrity, per-month chain recomputed from the archived rows, anchor `Matches`/`Superseded`/`NotPurged`/`Mismatch`, overall `Verified`). Same gate as VerifyAuditChain (OperationalAudit roles, site-scoped non-Administrator refused). VerifyAuditArchive uses the long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`, `INVALID_ARGUMENT` (not an archive name), `NOT_FOUND`. Back `scadabridge audit archive list|verify`.
### Transport (Bundle Import / Export)
docs/requirements/HighLevelReqs.md
+1 -1
View File
@@ -480,7 +480,7 @@ Sections 10.1–10.4 cover **configuration-database audit** (config-mutating use
- **AL-9**: The site SQLite Audit Log is purged only when `ForwardState ∈ {Forwarded, Reconciled}` — i.e., a row must be either confirmed-forwarded *or* confirmed-reconciled before it can be removed. A central outage therefore **cannot cause audit loss at sites**.
- **AL-10**: The Central UI exposes an **Audit Log page** with a cross-channel filter (by site, target, status, time range, correlation ID), plus **drill-ins from existing operational pages** (Site Calls, Notification Outbox, Inbound API).
- **AL-11**: Append-only semantics are **enforced via DB roles** (no UPDATE/DELETE granted on the `AuditLog` table to application accounts); a **tamper-evidence hash chain** (per-month SHA-256 chain, default off) shipped later by ScadaBridge#41.
- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export (csv, jsonl, parquet), hash-chain verification (verify-chain reports the first break of a month's hash chain), and listing/verifying the Parquet cold archives (ScadaBridge#42) against the central Audit Log.
- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export (csv, jsonl, parquet), hash-chain verification (verify-chain reports the first break of a month's hash chain; verify-heads checks off-box exported chain heads against the database, ScadaBridge#107), and listing/verifying the Parquet cold archives (ScadaBridge#42) against the central Audit Log.
## 11. Health Monitoring