From b418408c0fbc0756b8110ebc1bd452f80f6d94a0 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Sun, 15 Nov 2026 17:37:33 -0500 Subject: [PATCH] docs(audit): record the removal of the hash chain and the two-login hardening MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Component-AuditLog (Security & Tamper-Evidence now a removal note, archive format v2, config), Component-CLI, Component-ManagementService, HighLevelReqs AL-11/AL-12, docs/components/CLI, the CLI README, alog.md §11.4/§15 and CLAUDE.md. New production script docs/plans/sql/RemoveAuditLogHashChain.sql (idempotent, from migration 64); AddAuditLogHashChain.sql marked superseded. Historical plans, deployment records and deploy-prep packs are left as written. Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8 --- CLAUDE.md | 4 +- alog.md | 31 +- docs/components/CLI.md | 6 +- docs/plans/sql/AddAuditLogHashChain.sql | 2 + docs/plans/sql/RemoveAuditLogHashChain.sql | 94 +++++ docs/requirements/Component-AuditLog.md | 327 ++++-------------- docs/requirements/Component-CLI.md | 35 +- .../Component-ManagementService.md | 9 +- docs/requirements/HighLevelReqs.md | 4 +- src/ZB.MOM.WW.ScadaBridge.CLI/README.md | 77 +---- 10 files changed, 194 insertions(+), 395 deletions(-) create mode 100644 docs/plans/sql/RemoveAuditLogHashChain.sql diff --git a/CLAUDE.md b/CLAUDE.md index 0bb755f2..9aad9828 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -184,8 +184,8 @@ spec for each is `docs/requirements/Component-.md`, and `README.md` carrie - Append-only enforced via DB roles (writer role has INSERT only, no UPDATE/DELETE). - **The set-based ingest probe is `WITH (FORCESEEK)` (#124).** Left alone, `InsertManyIfNotExistsAsync`'s `NOT EXISTS` anti-semi-join scanned the EventId index, and concurrent writers (ingest + reconciliation singletons, dual-write, direct writes) deadlocked on S/IX page locks — 4 writers lost ~45 % of packets to 1205. With the hint: 0 deadlocks, 8 writers 8.6 k rows/s. Reproduce with `ResidualBench audit-contention`. - **Clustered key `(OccurredAtUtc, EventId)` + aligned unique `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` (migration `FlipAuditLogClusteredKey`, #111, B2 of the #78 benchmark).** The Audit Log page's newest-first page/keyset now reads an ordered range instead of the whole table (300k rows: 90,532 → 34 reads). Offline whole-table rebuild — script `docs/plans/sql/FlipAuditLogClusteredKey.sql` (+ `.rollback.sql`), both centrals stopped: the partition-switch staging table now reads the live key order, but binaries older than the flip hard-code the old one. DEV/QA not applied (sizing + window in #111). -- **Parquet cold archive (T2, ScadaBridge#42) — default OFF (`AuditLog:Archive:Enabled` + absolute `Directory`, shared by both centrals).** The purge actor writes each expired partition to `{Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet` (10 stored columns + `ChainSeq`/`RowHash`, tick-exact `OccurredAtUtcTicks`) plus `.tombstones.parquet` and a `.manifest.json` (written last; chain head at archive time, SHA-256 per file), read under the chain lock SHARED; then the switch runs with a guard (under the EXCLUSIVE lock: partition row count + each month's `LastSeq` must still equal the archive's, else THROW 51002/51003 and nothing is purged). Fail closed: a partition that cannot be archived is not purged. `audit archive list|verify` = `ListAuditArchives`/`VerifyAuditArchive` (verify-chain's gate): file hashes, chain recomputed from the archive, chain end vs the DB purge anchor (`Matches`/`Superseded`/`NotPurged`/`Mismatch`), CLI exit 3. `audit export --format parquet` streams the 24 export columns (was 501). No migration. **Owner decisions (#109):** destination stays a directory/UNC (no object-storage client; share + permissions named at enablement), parquet-tools-only reading (no re-import), unsigned like the chain. **Retention** `AuditLog:Archive:RetentionDays` (default 0 = never; else > `AuditLog:RetentionDays`, counted from the month's end) and **superseded stale-archive cleanup** (`DeleteSupersededStaleArchives`, default true: a retried purge's `Archived` archive is deleted only when a later `Purged`, verifying archive provably holds every row — same anchor + chain-end hash, unchained rows by EventId; else `KeptStale`) run after every purge tick; deletion touches only the archive name's own three files, manifest first; `audit archive cleanup [--dry-run]` = `CleanupAuditArchives` (Administrator). -- **Hash-chain tamper evidence (T1, ScadaBridge#41) — default OFF (`AuditLog:HashChain:Enabled`).** One SHA-256 chain per UTC calendar month: `ChainSeq` + `RowHash = SHA-256(prev || canonical(row))` on `AuditLog` (migration `AddAuditLogHashChain`), linked by the central `AuditLogHashChainService` (both nodes, serialised by `sp_getapplock`) — never by the ingest path. `AuditLogChainHead` catches tail deletion and carries the partition-switch checkpoint anchor; the per-channel DELETE writes an `AuditLogChainTombstone` per chained row in the same transaction. `audit verify-chain --month` = management command `VerifyAuditChain` (OperationalAudit, unscoped callers only), first break with partition/ChainSeq/EventId, CLI exit 3. A chained row's `SourceNode` is frozen (backfill skips it). **Unsigned by owner decision (#107); external anchor = off-box head export** (`AuditLog:HashChain:HeadExport`, default off: both centrals append changed per-month heads — month, `LastSeq`/`LastHash`, purge anchor, row + tombstone counts — to append-only JSONL files `{Directory}/{yyyy}/audit-chain-heads-{date}-{node}.jsonl` or RFC 5424 syslog UDP/TCP); `audit verify-heads --path` = `VerifyAuditChainHeads` checks them against the DB (a re-hashed forgery passes verify-chain, fails verify-heads; exit 3). **Two-login hardening (#107 item 3):** `AuditLog:MaintenanceConnectionString` (`${secret:}`-capable, unset = runtime login) runs the chain job, verify/head export, partition switch-out + archive and per-channel purge on `AuditMaintenanceContext` (keyed `audit-maintenance` repository/archive); that login gets `scadabridge_audit_chainer` + `scadabridge_audit_purger`, the runtime login `scadabridge_audit_writer` (whose DENYs block chaining/purging) — pinned by `AuditMaintenanceLoginTests`. Since #113 the monthly roll-forward and `backfill-source-node` use it too (migration `AuditMaintenanceGrants`: purger gets `ALTER ANY DATASPACE` + `UPDATE (SourceNode)`). Head-export failures (#115) bump `scadabridge.audit.chain_head_export.failures` and Degrade `/health/ready` entry `audit-chain-head-export`; archive cleanup (#114) also deletes writer-named orphan temp / manifest-less files older than 24 h (`DeletedOrphan`). +- **Parquet cold archive (T2, ScadaBridge#42) — default OFF (`AuditLog:Archive:Enabled` + absolute `Directory`, shared by both centrals).** The purge actor writes each expired partition to `{Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet` (the 10 stored columns, tick-exact `OccurredAtUtcTicks`) plus a `.manifest.json` (written last; row count + SHA-256/size of the data file) — layout **format v2** since 2026-11-15; a format-v1 archive (written with the hash chain: `ChainSeq`/`RowHash` columns, a `.tombstones.parquet`, chain fields in the manifest) still lists, verifies and is cleaned up. The switch then runs with a guard (the partition row count must still equal the archive's, else THROW 51002 and nothing is purged). Fail closed: a partition that cannot be archived is not purged. `audit archive list|verify` = `ListAuditArchives`/`VerifyAuditArchive` (OperationalAudit, unscoped callers): data-file size, SHA-256 and row count against the manifest — file integrity only, **no tamper evidence**; CLI exit 3. `audit export --format parquet` streams the 24 export columns. **Owner decisions (#109):** destination stays a directory/UNC (no object-storage client), parquet-tools-only reading (no re-import), unsigned. **Retention** `AuditLog:Archive:RetentionDays` (default 0 = never; else > `AuditLog:RetentionDays`, counted from the month's end) and **superseded stale-archive cleanup** (`DeleteSupersededStaleArchives`, default true: a retried purge's `Archived` archive is deleted only when a later `Purged`, verifying archive holds every one of its rows by EventId; else `KeptStale`) run after every purge tick; deletion touches only the archive name's own files, manifest first; `audit archive cleanup [--dry-run]` = `CleanupAuditArchives` (Administrator); it also deletes writer-named orphan temp / manifest-less files older than 24 h (`DeletedOrphan`, #114). +- **Hash-chain tamper evidence and the two-login maintenance hardening: REMOVED 2026-11-15.** Tamper-evidence (hash chain, off-box head export) and the two-login maintenance hardening were removed on 2026-11-15 at IT security's request, pending a family-wide general solution; migration `RemoveAuditLogHashChain` drops their schema (`ChainSeq`/`RowHash`, `AuditLogChainHead`/`AuditLogChainTombstone`, the two chain indexes, `scadabridge_audit_chainer`, and the purger's `AuditMaintenanceGrants` grants; `scadabridge_audit_writer`/`_purger` stay). **Deploy gate:** binaries before this change map the chain columns, so the migration ships with the new binaries, both centrals stopped; the new binaries purge either schema (the staging table copies the chain columns only when the live table has them). A leftover `AuditLog:HashChain:*` or `AuditLog:MaintenanceConnectionString` is ignored with one startup warning (`RetiredAuditSettingsWarning`) — DEV vd03 and QA vq03/vq04 still carry `AuditLog__HashChain__Enabled=true` in the Central service env from the 2026-11-13 enablement; remove it at the next deploy. Everything runs on `ScadaBridge:Database:ConfigurationDb` again. History: ScadaBridge#41/#107/#113/#115/#139 and `docs/plans/`. - Node-of-origin is captured alongside site-of-origin: `SourceNode` (`varchar(64)` NULL) on `AuditLog`, `Notifications`, and `SiteCalls` — `node-a`/`node-b` for site rows (qualified by `SourceSiteId`/`SourceSite`), `central-a`/`central-b` for central direct-write rows. Stamped at the writing node, carried verbatim through telemetry + reconciliation, and indexed via `IX_AuditLog_Node_Occurred (SourceNode, OccurredAtUtc)` on `AuditLog`. - Per-node stuck KPIs (M5.3 T6): Notification Outbox and Site Call Audit expose `PerNodeNotificationKpiRequest`/`PerNodeSiteCallKpiRequest` messages that group stuck/parked/delivered counts by `SourceNode`, surfacing per-node breakdowns on the Health dashboard. - `audit tree --execution-id ` CLI command (M5.3 T8) + `GET /api/audit/tree` endpoint — resolves any node to its chain root and renders the full execution tree; backed by `IAuditLogRepository.GetExecutionTreeAsync`. diff --git a/alog.md b/alog.md index f0d19ee1..678c3674 100644 --- a/alog.md +++ b/alog.md @@ -493,21 +493,11 @@ Audit Log page **Export** button streams CSV (current filter) server-side. Defau ### 11.3 Payload redaction at write See §8.2. Contract: unredacted secrets never persist. Safety net over-redacts on misconfiguration. -### 11.4 Tamper-evidence hash chain (shipped, ScadaBridge#41 — default off) -- Each row gains a `RowHash` column, plus a `ChainSeq` (position in its partition's chain). -- `RowHash = SHA-256(prev.RowHash || canonical(row))` per partition (UTC calendar month). -- Computed by a chaining job — implemented incrementally (every minute, both central - nodes, serialised by a SQL application lock) rather than only after the month closes. -- Purges keep the chain verifiable: the partition switch records a checkpoint anchor on - the month's chain head; the per-channel DELETE leaves a tombstone per chained row. -- Verifiable via `scadabridge audit verify-chain --month YYYY-MM` (first break reported - with partition, `ChainSeq`, `EventId`). Unsigned (owner decision, ScadaBridge#107): instead - the per-month heads are exported off-box (`AuditLog:HashChain:HeadExport`, append-only files - or syslog, default off) and `scadabridge audit verify-heads` checks them against the database. - The chain job and both purge paths can run on a separate maintenance login - (`AuditLog:MaintenanceConnectionString`). -- Default **off** (`AuditLog:HashChain:Enabled`). Full design: Component-AuditLog.md, - Security & Tamper-Evidence. +### 11.4 Tamper-evidence hash chain (REMOVED 2026-11-15) +Shipped by ScadaBridge#41 (with the off-box head export and the two-login maintenance hardening, +ScadaBridge#107/#113), then removed on 2026-11-15 at IT security's request, pending a family-wide +general solution; migration `RemoveAuditLogHashChain` drops its schema. The shipped design is kept +in the ScadaBridge#41/#107/#139 issues and `docs/plans/`. ### 11.5 Site SQLite security - File permissions: read/write by the ScadaBridge service account only. @@ -601,17 +591,16 @@ MS SQL handles this with batched ingest and the time-aligned indexes. New `scadabridge audit` command group: - `scadabridge audit query --site --since --kind [...]` — same filter set as the UI. - `scadabridge audit export --since --until --format csv|jsonl|parquet --output ` — bulk export, server-side streaming. -- `scadabridge audit verify-chain --month ` — hash-chain verification (when §11.4 is enabled). -- `scadabridge audit verify-heads --path ` — off-box exported chain heads vs the database (ScadaBridge#107). +- `scadabridge audit archive list|verify|cleanup` — the Parquet archives (§15.2). Requires the same **OperationalAudit** / **AuditExport** permissions as the UI. ### 15.2 Parquet archival (shipped, ScadaBridge#42 — default off) The retention purge dumps each expiring monthly partition to Parquet before switching it out — enabling indefinite cold retention without bloating MS SQL. Shipped as: - `AuditLog:Archive:Enabled` (default **off**) + `Directory` (absolute path both central nodes reach — a shared volume or UNC share; object storage is mounted, not called). -- Per partition: `auditlog-YYYY-MM-Z.parquet` (the stored columns + `ChainSeq`/`RowHash`), `.tombstones.parquet` (per-channel purge tombstones) and `.manifest.json` (counts, SHA-256 per file, the month's chain head at archive time), under `{Directory}/{YYYY}/`. -- Archive before purge, fail closed: the switch runs only after the archive is durable, and re-checks under the chain lock that the partition still holds exactly the archived rows and chain head; otherwise nothing is purged and the next tick re-archives. -- `scadabridge audit archive list` / `verify --name `: verify recomputes the archived chain and ties its end to the checkpoint anchor the purge left (§11.4). +- Per partition: `auditlog-YYYY-MM-Z.parquet` (the stored columns) and `.manifest.json` (row count, SHA-256 and size of the data file), under `{Directory}/{YYYY}/` (format v2; format-v1 archives from the hash-chain era also carry chain columns and a `.tombstones.parquet`). +- Archive before purge, fail closed: the switch runs only after the archive is durable, and re-checks that the partition still holds exactly the archived row count; otherwise nothing is purged and the next tick re-archives. +- `scadabridge audit archive list` / `verify --name `: verify checks the data file against its manifest (file integrity, not tamper evidence). - `audit export --format parquet` streams the export columns as Parquet. --- @@ -622,7 +611,7 @@ The retention purge dumps each expiring monthly partition to Parquet before swit |---|---|---| | 1 | Component number | **#23 Audit Log** (README matrix + HighLevelReqs). | | 2 | Nav placement | New top-level **Audit** nav group in Central UI. | -| 3 | Hash-chain tamper evidence (§11.4) | Deferred to v1.x; shipped by ScadaBridge#41 (unsigned SHA-256 chain, default off). | +| 3 | Hash-chain tamper evidence (§11.4) | Deferred to v1.x; shipped by ScadaBridge#41 (unsigned SHA-256 chain, default off); removed 2026-11-15 (IT security request). | | 4 | Parquet archival to object storage (§15.2) | Deferred to v1.x; shipped by ScadaBridge#42 (Parquet on a configured directory, default off). | | 5 | Per-channel retention overrides (§12.1) | Deferred to v1.x. v1 uses a single global `RetentionDays`. | | 6 | Default payload cap | **8 KB** for `RequestSummary` / `ResponseSummary`; **64 KB** on error rows (`Status IN ('Failed', 'Parked', 'Discarded')`). | diff --git a/docs/components/CLI.md b/docs/components/CLI.md index 1954b9bf..814d28e8 100644 --- a/docs/components/CLI.md +++ b/docs/components/CLI.md @@ -22,9 +22,7 @@ Direct-to-node access (`http://localhost:9001` for `central-a`, `http://localhos Management commands use `POST /management`. Audit query, export, and tree use `GET /api/audit/query`, `/api/audit/export`, and `/api/audit/tree`; source-node -backfill uses `POST /api/audit/backfill-source-node`. `audit verify-chain` sends the -`VerifyAuditChain` management command (ScadaBridge#41); `audit verify-heads` reads off-box -exported chain heads and sends `VerifyAuditChainHeads` (ScadaBridge#107); `audit archive list|verify|cleanup` send +backfill uses `POST /api/audit/backfill-source-node`. `audit archive list|verify|cleanup` send `ListAuditArchives` / `VerifyAuditArchive` (ScadaBridge#42) / `CleanupAuditArchives` (ScadaBridge#109). `debug stream` uses a persistent SignalR connection. Audit export streams the response without buffering it all. @@ -122,7 +120,7 @@ scadabridge instance deploy --id 7 | `db-connection` | `list`, `get`, `create`, `update`, `delete` | `Designer` | | `api-method` | `list`, `get`, `create`, `update`, `delete`, `validate`, `recompile` | `Designer` | | `bundle` | `export`, `preview`, `import` | Designer export; Administrator preview/import | -| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `verify-chain`, `verify-heads`, `archive list`, `archive verify`, `archive cleanup` | Audit roles; Administrator backfill and archive cleanup; verify-chain, verify-heads and archive need an unscoped audit role | +| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `archive list`, `archive verify`, `archive cleanup` | Audit roles; Administrator backfill and archive cleanup; archive needs an unscoped audit role | | `audit-config` | `query` (config-change audit trail; was `audit-log` pre-M8) | `Admin` | | `health` | `summary`, `pending-updates`, `site`, `event-log`, `parked-messages` | Authenticated reads with server-side scope checks | | `debug` | `snapshot`, `stream` | `Deployer` | diff --git a/docs/plans/sql/AddAuditLogHashChain.sql b/docs/plans/sql/AddAuditLogHashChain.sql index 47dbe686..8dd1cad9 100644 --- a/docs/plans/sql/AddAuditLogHashChain.sql +++ b/docs/plans/sql/AddAuditLogHashChain.sql @@ -1,3 +1,5 @@ +-- SUPERSEDED 2026-11-15: the hash chain was removed at IT security's request; RemoveAuditLogHashChain.sql drops +-- everything this script creates. Kept as the record of what was applied to DEV/QA on 2026-11-13. -- AddAuditLogHashChain (2026-11-09, ScadaBridge#41). AuditLog hash-chain tamper evidence: -- ChainSeq/RowHash on dbo.AuditLog, AuditLogChainHead + AuditLogChainTombstone, two partition-aligned -- filtered indexes, the scadabridge_audit_chainer role. Apply to the central configuration database diff --git a/docs/plans/sql/RemoveAuditLogHashChain.sql b/docs/plans/sql/RemoveAuditLogHashChain.sql new file mode 100644 index 00000000..9721df40 --- /dev/null +++ b/docs/plans/sql/RemoveAuditLogHashChain.sql @@ -0,0 +1,94 @@ +-- RemoveAuditLogHashChain (2026-11-15). Removes the AuditLog hash chain and the two-login maintenance grants at IT +-- security's request: IX_AuditLog_Unchained/IX_AuditLog_ChainSeq, the scadabridge_audit_chainer role (members first), +-- dbo.AuditLogChainHead/AuditLogChainTombstone, AuditLog.ChainSeq/RowHash, and the purger's ALTER ANY DATASPACE + +-- UPDATE (SourceNode) grants. scadabridge_audit_writer/_purger themselves stay. Idempotent (generated with --idempotent +-- from 20261110120000_AuditMaintenanceGrants, migration 64 = DEV vd03 and QA vq03/vq04 on 2026-11-15). +-- DEPLOY GATE: apply with both centrals stopped, together with binaries that contain this migration; older binaries +-- map ChainSeq/RowHash and fail against the migrated table. Remove AuditLog__HashChain__* from the Central service env +-- in the same window (the new binaries ignore it with a warning). Rollback: the migration's Down recreates the schema. +BEGIN TRANSACTION; +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + + IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_ChainSeq' AND object_id = OBJECT_ID('dbo.AuditLog')) + EXEC(N'DROP INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog;'); + IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Unchained' AND object_id = OBJECT_ID('dbo.AuditLog')) + EXEC(N'DROP INDEX IX_AuditLog_Unchained ON dbo.AuditLog;'); +END; + +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + + IF DATABASE_PRINCIPAL_ID('scadabridge_audit_chainer') IS NOT NULL + BEGIN + IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NOT NULL + EXEC(N'REVOKE UPDATE (ChainSeq, RowHash) ON dbo.AuditLog FROM scadabridge_audit_chainer;'); + EXEC(N'REVOKE SELECT ON dbo.AuditLog FROM scadabridge_audit_chainer;'); + + DECLARE @dropMembers nvarchar(max) = N''; + SELECT @dropMembers = @dropMembers + N'ALTER ROLE scadabridge_audit_chainer DROP MEMBER ' + QUOTENAME(m.name) + N';' + CHAR(10) + FROM sys.database_role_members rm + JOIN sys.database_principals r ON r.principal_id = rm.role_principal_id + JOIN sys.database_principals m ON m.principal_id = rm.member_principal_id + WHERE r.name = N'scadabridge_audit_chainer'; + EXEC sp_executesql @dropMembers; + + EXEC(N'DROP ROLE scadabridge_audit_chainer;'); + END +END; + +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + + IF OBJECT_ID('dbo.AuditLogChainHead', 'U') IS NOT NULL + EXEC(N'DROP TABLE dbo.AuditLogChainHead;'); + IF OBJECT_ID('dbo.AuditLogChainTombstone', 'U') IS NOT NULL + EXEC(N'DROP TABLE dbo.AuditLogChainTombstone;'); +END; + +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + + IF COL_LENGTH('dbo.AuditLog', 'ChainSeq') IS NOT NULL + EXEC(N'ALTER TABLE dbo.AuditLog DROP COLUMN ChainSeq;'); + IF COL_LENGTH('dbo.AuditLog', 'RowHash') IS NOT NULL + EXEC(N'ALTER TABLE dbo.AuditLog DROP COLUMN RowHash;'); +END; + +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + + IF DATABASE_PRINCIPAL_ID('scadabridge_audit_purger') IS NOT NULL + BEGIN + EXEC(N'REVOKE ALTER ANY DATASPACE FROM scadabridge_audit_purger;'); + EXEC(N'REVOKE UPDATE (SourceNode) ON dbo.AuditLog FROM scadabridge_audit_purger;'); + END +END; + +IF NOT EXISTS ( + SELECT * FROM [__EFMigrationsHistory] + WHERE [MigrationId] = N'20261115221558_RemoveAuditLogHashChain' +) +BEGIN + INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion]) + VALUES (N'20261115221558_RemoveAuditLogHashChain', N'10.0.13'); +END; + +COMMIT; +GO + diff --git a/docs/requirements/Component-AuditLog.md b/docs/requirements/Component-AuditLog.md index fe7c8841..f0af2960 100644 --- a/docs/requirements/Component-AuditLog.md +++ b/docs/requirements/Component-AuditLog.md @@ -776,33 +776,28 @@ MS SQL for direct-write events). Unredacted secrets never persist. share (UNC path) and its permissions are chosen when the archive is enabled on DEV/QA: both central nodes' service account needs modify rights on it (write, rename, and — for cleanup — delete), auditors read-only; the SQL Server login needs none. - - **What.** Per partition, three files sharing the stem + - **What.** Per partition, two files sharing the stem `auditlog-{YYYY-MM}-{archivedAt:yyyyMMddTHHmmssfff}Z`: `.parquet` (every row of the partition — the ten stored columns, `OccurredAtUtc` both as a UTC microsecond - timestamp and as exact `OccurredAtUtcTicks`, and `ChainSeq`/`RowHash`, null for - rows the chaining job had not reached; computed columns are not stored, they derive - from `DetailsJson`), `.tombstones.parquet` (the month's per-channel purge - tombstones) and `.manifest.json` (row / chained / unchained / tombstone counts, - SHA-256 and size of each file, and each chain month's head — `LastSeq`, `LastHash`, - and the anchor an earlier purge left). Each file is written to a `.tmp` sibling, - flushed and renamed; the manifest goes last and is the commit marker. Archives are - never overwritten. Row groups of `RowGroupSize` (default 50,000) bound memory. - - **Consistent with the hash chain.** The partition's rows, heads and tombstones are - read in one transaction holding the chain lock SHARED, so no chaining pass or purge - moves the chain mid-read. The switch then runs with an archive guard: under the - EXCLUSIVE chain lock it re-counts the partition and re-reads each month's - `LastSeq`, and THROWs (51002 row count changed / 51003 chain advanced — nothing - purged) unless both still equal the archive's. The switch's checkpoint anchor - (`PurgedThroughSeq/Hash`) therefore equals the archive's recorded chain end. A - stale archive keeps manifest status `Archived`; the next tick writes a fresh one. - - **Fail closed.** A partition that cannot be archived (volume full, lock busy past - `ChainLockTimeoutSeconds`, …) is not purged: the failure takes the usual - `AuditLogPurgeFailedEvent` / `PurgeFailures` path and the table keeps the rows. - A successful archive publishes `AuditLogArchivedEvent` before `AuditLogPurgedEvent`. + timestamp and as exact `OccurredAtUtcTicks`; computed columns are not stored, they + derive from `DetailsJson`) and `.manifest.json` (row count, time range, SHA-256 and size + of the data file). Each file is written to a `.tmp` sibling, flushed and renamed; the + manifest goes last and is the commit marker. Archives are never overwritten. Row groups + of `RowGroupSize` (default 50,000) bound memory. This is layout **format v2**; a format-v1 + archive (written before 2026-11-15, while the hash chain existed) also has `ChainSeq` / + `RowHash` data columns, a `.tombstones.parquet` file and chain fields in its manifest — + it still lists, verifies and is cleaned up; the chain parts are ignored. + - **Consistent with the purge.** The partition's rows are read in one transaction. The + switch then runs with an archive guard: it re-counts the partition and THROWs (51002, + nothing purged) unless the count still equals the archive's. A stale archive keeps + manifest status `Archived`; the next tick writes a fresh one. + - **Fail closed.** A partition that cannot be archived (volume full, read timeout, …) is + not purged: the failure takes the usual `AuditLogPurgeFailedEvent` / `PurgeFailures` + path and the table keeps the rows. A successful archive publishes `AuditLogArchivedEvent` + before `AuditLogPurgedEvent`. - **Retention interplay.** Archiving changes nothing about *when* a partition is purged (`RetentionDays`). Per-channel overrides still delete their rows earlier, - so those rows are never archived — the archive carries their tombstones instead, - which is what keeps the archived chain verifiable. + so those rows are never archived. - **Archive retention (ScadaBridge#109) — default never delete.** `AuditLog:Archive:RetentionDays` (default `0` = archives are never deleted, until compliance names a period) is counted from the END of the archived month and, when set, must exceed @@ -810,23 +805,20 @@ MS SQL for direct-write events). Unredacted secrets never persist. in the database. After each purge tick the purge actor deletes an expired archive that is `Purged`, or an `Archived` one whose partition no longer holds rows. - **Superseded stale archives (ScadaBridge#109).** A purge whose guard finds the partition - changed (51002/51003) leaves its archive at `Archived` (verify reports `NotPurged`, or - `Superseded` once a later purge anchored past it) and the next tick archives again. With + changed (51002) leaves its archive at `Archived` and the next tick archives again. With `AuditLog:Archive:DeleteSupersededStaleArchives` (default `true`) the purge actor deletes such a stale archive only when a LATER archive of the same partition is `Purged`, verifies, - and provably holds every row of it: for every chain month, the same earlier anchor and the - stale archive's chain-end hash at its chain-end sequence (row or tombstone), and every - unchained row present by EventId. Otherwise it is kept and reported `KeptStale` (e.g. an - unchained row a per-channel purge removed in between — no tombstone — so the stale archive - is that row's only copy). An archive left at `Archived` because the node stopped between the - switch and the manifest update has no later purged archive and is never touched. - - **Safe deletion.** Cleanup deletes only the three files a well-formed archive name implies - (`{name}.manifest.json`, `.parquet`, `.tombstones.parquet`) in that archive's own + and provably holds every row of it (every row present by EventId). Otherwise it is kept and + reported `KeptStale` (e.g. a row a per-channel purge removed in between, so the stale + archive is that row's only copy). An archive left at `Archived` because the node stopped + between the switch and the manifest update has no later purged archive and is never touched. + - **Safe deletion.** Cleanup deletes only the files a well-formed archive name implies + (`{name}.manifest.json`, `.parquet`, and a format-v1 archive's `.tombstones.parquet`) in that archive's own `{Directory}/{YYYY}` folder — never a path read from a manifest — manifest first, so an interrupted deletion leaves leftovers that are no longer an archive. **Orphans of an interrupted archive WRITE (ScadaBridge#114):** the same pass also deletes, reporting action `DeletedOrphan` (dry-run aware), the writer's own temp files (`{name}.parquet.tmp`, - `{name}.tombstones.parquet.tmp`, `{name}.manifest.json.tmp`) and data files whose + `{name}.manifest.json.tmp`, and a format-v1 writer's `{name}.tombstones.parquet.tmp`) and data files whose `.manifest.json` was never written — only files named exactly like that for a well-formed archive name, directly inside a four-digit `{Directory}/{YYYY}` folder whose year matches the name, not reparse points, and last written more than 24 h ago (a running write is never @@ -835,16 +827,14 @@ MS SQL for direct-write events). Unredacted secrets never persist. (`CleanupAuditArchives`, Administrator) runs the same pass on demand or reports what it would delete. Pinned by `AuditLogArchiveCleanupTests` (mutation-checked: each safety rule removed fails its test). - - **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks both - files' SHA-256 and counts against the manifest, recomputes each archived month's - chain from the archived rows and tombstones (`AuditChainVerifier`, starting from - any earlier anchor), and compares the chain end with the database's current anchor: - `Matches`; `Superseded` (late rows for the month were chained, purged and archived - again afterwards — that later archive resumes from this one's end); `NotPurged`; - or `Mismatch` (archive or head altered). The files are plain Parquet — DuckDB, - pyarrow, Spark or parquet-tools read them directly. **Owner decision (ScadaBridge#109): - reading with Parquet tools is the supported access path; there is no re-import into - `dbo.AuditLog`** — a re-inserted expired row would be purged again by the next tick. + - **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks the data + file's size and SHA-256 and its row count against the manifest. It proves the file is the + one the archive wrote, not that its rows are genuine: archives are unsigned, and there is no + tamper evidence since the hash chain was removed (see Security). The files are plain + Parquet — DuckDB, pyarrow, Spark or parquet-tools read them directly. **Owner decision + (ScadaBridge#109): reading with Parquet tools is the supported access path; there is no + re-import into `dbo.AuditLog`** — a re-inserted expired row would be purged again by the + next tick. - **Sites:** `SiteAuditRetentionService` (site-only `IHostedService`) runs the site SQLite retention purge on a timer — first tick after `AuditLog:SiteRetention:InitialDelay` (default 5 min, short so a daily-recycled @@ -868,207 +858,41 @@ MS SQL for direct-write events). Unredacted secrets never persist. ## Security & Tamper-Evidence -- **Append-only enforcement — what actually holds in the default deployment.** - By default ScadaBridge runs with **one** connection principal for both the writer and the - maintenance/purge paths. (A second, maintenance login is supported but opt-in — see - *Two-login hardening* below.) So in the default deployment the append-only invariant is - enforced by two application-level controls, not by database permissions: +> **Removed 2026-11-15.** Tamper-evidence (hash chain, off-box head export) and the two-login +> maintenance hardening were removed on 2026-11-15 at IT security's request, pending a +> family-wide general solution; migration `RemoveAuditLogHashChain` drops their schema. +> The design they shipped with is kept as history in `docs/plans/` and the ScadaBridge#41, +> #107 and #139 issues. + +- **Append-only enforcement — what actually holds.** + ScadaBridge runs with **one** connection principal for both the writer and the + maintenance/purge paths, so the append-only invariant is enforced by two + application-level controls, not by database permissions: 1. **CI grep guard** — `AuditLogAppendOnlyGuardTests` scans the ConfigurationDatabase source for any `UPDATE`/`DELETE` DML targeting `dbo.AuditLog` and fails the build - on a hit. Exactly three mutations are allow-listed by an explicit - `// AUDIT-PURGE-ALLOWED` marker (the per-channel retention `DELETE TOP`, the - one-time `SourceNode` sentinel backfill `UPDATE`, and the hash-chain link `UPDATE` - that sets only `ChainSeq`/`RowHash` while they are NULL — ScadaBridge#41); every - other UPDATE/DELETE trips the guard. + on a hit. Exactly two mutations are allow-listed by an explicit + `// AUDIT-PURGE-ALLOWED` marker (the per-channel retention `DELETE TOP` and the + one-time `SourceNode` sentinel backfill `UPDATE`); every other UPDATE/DELETE trips + the guard. 2. **Code review** — the marker is deliberately specific so an unrelated mutation cannot inherit the exemption without a reviewer noticing. -- **Two-login hardening (ScadaBridge#107 item 3) — opt-in.** Migrations provision three - roles: `scadabridge_audit_writer` (`INSERT` + `SELECT` on `AuditLog`; `DENY UPDATE` + - `DENY DELETE` so a later `db_datawriter` membership cannot silently re-enable mutation; - `DENY INSERT/UPDATE/DELETE` on `AuditLogChainHead` and `AuditLogChainTombstone`), - `scadabridge_audit_chainer` (`SELECT` on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`, - `SELECT/INSERT/UPDATE` on the head, `SELECT` on tombstones) and `scadabridge_audit_purger` - (`SELECT` + `ALTER ON SCHEMA::dbo` + `CREATE TABLE` for the switch-out staging table, - `DELETE ON dbo.AuditLog` for the per-channel purge — `FixAuditPurgerRoleGrants`, arch-review 04 - S3 — and `SELECT/UPDATE` on the head, `SELECT/INSERT/DELETE` on tombstones). Because the writer - role's DENYs win over any grant, the chain job and the purges **cannot** run on a login that is - in the writer role. Set `AuditLog:MaintenanceConnectionString` (central nodes; `${secret:}` - references resolve, as for every pre-host key) and the hash-chain job, chain verification / - head export, the partition switch-out (with its Parquet archive) and the per-channel DELETE run - on their own connection with that string (`AuditMaintenanceContext`); unset, they run on - `ScadaBridge:Database:ConfigurationDb` exactly as before. A DBA provisions: - - | Login | Used by | Database roles | - |---|---|---| - | runtime (writer) — `ScadaBridge:Database:ConfigurationDb` | everything else, incl. audit ingest | `db_datareader`, `db_datawriter`, `scadabridge_audit_writer` (+ whatever the rest of the app needs) | - | maintenance — `AuditLog:MaintenanceConnectionString` | chain job, verify-chain / verify-heads, head export, partition switch-out + archive, per-channel purge | `scadabridge_audit_chainer`, `scadabridge_audit_purger` — and NOT `scadabridge_audit_writer` | - - ```sql - CREATE USER [svc_scadabridge_audit_maint] FOR LOGIN [svc_scadabridge_audit_maint]; - ALTER ROLE scadabridge_audit_chainer ADD MEMBER [svc_scadabridge_audit_maint]; - ALTER ROLE scadabridge_audit_purger ADD MEMBER [svc_scadabridge_audit_maint]; - ALTER ROLE scadabridge_audit_writer ADD MEMBER [svc_scadabridge_app]; -- the runtime login - ``` - - `AuditMaintenanceLoginTests` runs exactly this setup on SQL Server: the writer ingests and is - refused the chain UPDATE and the per-channel DELETE; the maintenance login chains, verifies, - reads heads, purges per channel, enumerates partitions, archives and switches out. **Also on the - maintenance login (ScadaBridge#113):** the monthly partition roll-forward - (`ALTER PARTITION FUNCTION … SPLIT RANGE`, `AuditLogPartitionMaintenance` now built on - `AuditMaintenanceContext`) and `audit backfill-source-node` (the management endpoint resolves the - keyed maintenance `IAuditLogRepository`). Migration `AuditMaintenanceGrants` gives - `scadabridge_audit_purger` the two grants they need that `ALTER ON SCHEMA::dbo` does not confer: - database-level `ALTER ANY DATASPACE` (roll-forward) and column-level `UPDATE (SourceNode)` on - `dbo.AuditLog` (backfill; no other column becomes updatable, the writer's DENY still wins). - Unset `AuditLog:MaintenanceConnectionString` keeps both on the runtime login as before — which - then needs those same two grants (or db_owner), exactly as the single-login default always did. - `AuditMaintenanceLoginTests` pins both with a writer-refused negative control. Migrations themselves are applied by a DBA with the production SQL scripts. +- **Database roles.** Migrations still provision `scadabridge_audit_writer` (`INSERT` + + `SELECT` on `AuditLog`; `DENY UPDATE` + `DENY DELETE` so a later `db_datawriter` + membership cannot silently re-enable mutation) and `scadabridge_audit_purger` (`SELECT` + + `ALTER ON SCHEMA::dbo` + `CREATE TABLE` for the switch-out staging table, `DELETE ON + dbo.AuditLog` for the per-channel purge — `FixAuditPurgerRoleGrants`, arch-review 04 S3). + The application does not use them: every path, purge and archive included, runs on + `ScadaBridge:Database:ConfigurationDb`, which therefore needs the rights of both (or + db_owner). - **Authorization.** Reading the Audit Log requires the existing **Audit** role extended with a new **OperationalAudit** permission. Per-site row scoping reuses the existing site-permission model; bulk export requires an additional **AuditExport** permission. - **Payload redaction at write.** See Payload Capture Policy. Unredacted secrets never persist; the safety net over-redacts on misconfiguration. -- **Hash-chain tamper evidence (T1) — shipped (ScadaBridge#41), default OFF.** - `AuditLog:HashChain:Enabled` (default `false`, plus `IntervalSeconds` 60, - `BatchSize` 1000, `MaxBatchesPerTick` 100) starts the central - `AuditLogHashChainService` on both central nodes. - - **Model.** One chain per **partition = UTC calendar month of `OccurredAtUtc`** - (the `ps_AuditLog_Month` month). Each chained row carries `ChainSeq` (1, 2, 3 …, - gap-free within its month) and `RowHash = SHA-256(prev.RowHash || canonical(row))`, - `prev` being the row at `ChainSeq - 1` (32 zero bytes for the first — or, in the month - the chain started in, the chain-start anchor below). The canonical - v1 form (`AuditChainHasher`) is a domain tag, the month and sequence number, then - the ten writable columns in table order, each string length-prefixed with a NULL - marker; the computed columns derive from `DetailsJson` and are covered by it. A - known-answer test pins the encoding — changing it invalidates every stored chain. - - **Who writes the chain.** Not the hot path: the two central nodes and the site - ingest INSERT concurrently, so a write-time chain would need a global serialisation - point on every audit write. A **chaining job** links unchained rows (oldest month - first, in `OccurredAtUtc, EventId` order within a pass) under an exclusive SQL - Server application lock (`sp_getapplock`, zero timeout — the peer node's pass just - skips its tick), with an optimistic head check and a `ChainSeq IS NULL` row-count - check that roll the pass back rather than leave a gap. This is alog.md §11.4's - "chaining job", run every minute instead of only after a month closes, so a row is - tamper-evident about a minute after it reaches central. Chain order is chaining - order, so late-arriving site rows simply take the next sequence number. - - **Purges.** `dbo.AuditLogChainHead` holds each month's `(LastSeq, LastHash)` — which - is what exposes deletion of the newest rows — and the checkpoint anchor - `(PurgedThroughSeq, PurgedThroughHash)` that the monthly partition switch sets - inside its own transaction; verification resumes after it. The per-channel retention - DELETE writes a `dbo.AuditLogChainTombstone` (month, `ChainSeq`, `RowHash`, - `EventId`) for every chained row it removes, in the same transaction, via - `OUTPUT … INTO`; verification bridges the gap with it. Both purge paths and the - SourceNode backfill hold the same lock exclusively. The backfill skips chained rows - (a chained row's `SourceNode` is frozen) — run it before enabling the chain. - - **Chain start: no backlog chaining (ScadaBridge#139; owner decision on #107, - 2026-11-13).** Enabling the job on a database with history does **not** chain the - backlog. The first chaining pass on a pristine database (no chain head, no chained row) - writes a **chain-start anchor** and from then on only rows with - `OccurredAtUtc >= cutoff` are chained; older rows keep `ChainSeq`/`RowHash` NULL for - ever and are out of scope. - - *Where the cutoff lives.* One row in `dbo.AuditLogChainTombstone` at `ChainSeq = 0` - of the cutoff month (real links start at 1, so slot 0 is "the link before the - first"): `PurgedAtUtc` = the cutoff, `RowHash` = - `SHA-256("ScadaBridge.AuditLog.ChainCutoff.v1" || YYYY-MM || ticks)`, `EventId` = - `5cadab1d-c4a1-4c0f-a000-000000000139`. The cutoff is the database's - `SYSUTCDATETIME()` at that first pass, written once inside the chaining transaction - under the exclusive chain lock — so both central nodes read the same instant whatever - their own clocks or config say, and it survives restarts. It is never updated; the - partition switch-out keeps it (it deletes only `ChainSeq > 0` tombstones); the head - store's tombstone count, the head export and `verify-heads` ignore it. **No - migration**: the table and the grants exist (the maintenance login writes it through - `scadabridge_audit_purger`'s INSERT on the tombstone table). - - *Bound into the chain.* The anchor hash is the genesis the cutoff month's first row - links to (instead of 32 zero bytes). Moving the cutoff therefore breaks that month's - chain, and the month's exported head anchors the cutoff off-box like any other link. - - *`AuditLog:HashChain:ChainBacklog`* (default `false`) restores the old behaviour — - chain every row, oldest month first, no anchor — but only on the very first - enablement: once an anchor or any chained row exists the decision is fixed and the - option is ignored, so two centrals with different values still end up with one - chain (whichever chains first decides). A database whose chain was enabled before - this change (heads, no anchor) keeps chaining every row, as before. - - *Late rows.* The cutoff is on `OccurredAtUtc` (the partition key, indexed), not on - ingest time (`IngestedAtUtc` is a non-persisted JSON column that must never be - filtered on). A site row dated before the cutoff that reaches central after - enablement — the site's store-and-forward backlog at that moment — stays out of - scope; so does a row from a node whose clock runs behind the database's by the - skew. Both are bounded and reported, never silently dropped. - - *Index.* Pre-cutoff rows stay in the filtered `IX_AuditLog_Unchained` until retention - purges their partition; the chaining read seeks `OccurredAtUtc >= cutoff`, so they - cost storage only, not chaining time. - - Verification reports in-scope rows not chained yet as `UnchainedRows` and the - backlog as `PreCutoffRows` (plus `ChainFromUtc`); neither is ever a break. - - **Verification.** `scadabridge audit verify-chain --month YYYY-MM` sends the - `VerifyAuditChain` management command (OperationalAudit roles; refused for a - site-scoped caller because a month's chain spans every site; long-running Ask). It - takes the lock SHARED, recomputes every link in sequence order and reports `Intact`, - `NotChained` or `Broken` with the **first break**: partition, `ChainSeq`, `EventId` - and kind — `HashMismatch` (an edited row, or rows reordered — the sequence number is - inside the hashed bytes), `MissingRow` (a deleted row with no tombstone), - `DuplicateSequence`, `IncompleteLink`, `HeadMismatch` (rows deleted from the end), - `ChainedBeforeCutoff` or `CutoffAnchorInvalid` (below). CLI exit code 3 on `Broken`; - the table output also prints `Chain from` and `Pre-cutoff rows`. - - **Cutoff-eligibility attacks (ScadaBridge#139).** Because pre-cutoff rows are - unprotected by design, an attacker would like to (a) move a chained row's - `OccurredAtUtc` back before the cutoff to hide it among them, or (b) move the cutoff - forward. (a) with the link left in place is `ChainedBeforeCutoff` (no row before the - cutoff is ever chained — this also catches an attacker who re-hashes the row and every - later link); with the link cleared too, its sequence number is a `MissingRow` (or - `HeadMismatch` at the tail); moved into an earlier month, the source month reports - `MissingRow` and the target month `ChainedBeforeCutoff`. (b) editing the anchor's - cutoff without its hash, filing it under another month, or adding a second anchor is - `CutoffAnchorInvalid` (in every month's verification); re-hashing the anchor to the new - cutoff leaves the already-chained rows before it (`ChainedBeforeCutoff`) and changes the - cutoff month's genesis (`HashMismatch` at `ChainSeq` 1); deleting the anchor is a - `HashMismatch` at `ChainSeq` 1. **Known limits:** pre-cutoff rows themselves carry no - tamper evidence (that is the decision), and a backdated row forged *before* the cutoff - is indistinguishable from a genuine backlog row — the chain never proved a row genuine, - only that a chained row was not changed after chaining. A principal that rewrites the - anchor, the cutoff month's whole chain and its head is caught only by the off-box head - export, as for any full-chain rewrite. `AuditLogHashChainCutoffTests` and - `AuditChainVerifierTests` pin every case above. - - **External anchor: off-box head export (ScadaBridge#107 item 1) — default OFF.** Inside the - database alone, a principal that can rewrite rows AND recompute every later link AND the - head produces a chain `verify-chain` accepts. Owner decision: **no signing**; instead - `AuditLog:HashChain:HeadExport:Enabled` makes both central nodes, every `IntervalMinutes` - (60), read every month's head (`LastSeq`, `LastHash`, the purge anchor, live chained rows, - tombstones — one snapshot under the chain lock SHARED) and append the heads that changed - since that node's last export (all of them after a restart; `ExportUnchanged` for every - tick) to a sink the database login cannot write: `Sink: File` — append-only JSON lines in - `{File:Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl` (local or UNC; opened - `FileMode.Append` only, one file per node per day; give the service account create + - append-data rights and nothing else), or `Sink: Syslog` — RFC 5424 (`1 … chainhead - - {json}`, facility 13 "log audit", severity Notice) over UDP (RFC 5426) or TCP (RFC 6587 - octet counting). A failed write is logged and retried with the same heads next tick, and (ScadaBridge#115) - is also a health signal: counter `scadabridge.audit.chain_head_export.failures` on the shared - ScadaBridge meter, plus a `/health/ready` entry `audit-chain-head-export` that is Degraded (HTTP - 200, Overview-visible, not out of rotation) while the last tick failed or no tick has succeeded - for more than two intervals; Healthy when the export is disabled. UDP syslog is fire-and-forget, - so a dropped datagram is invisible by design — use `Protocol: Tcp` where that matters. - `scadabridge audit verify-heads --path ` reads those records back (file-sink - lines or a syslog collector's log) and the `VerifyAuditChainHeads` command checks each - against the database: the link stored at the exported sequence (the head, a live row's - `RowHash`, a tombstone, or the purge anchor) must still carry the exported hash — - `Matches` / `Extended` / `Purged` (removed by a partition switch; verify the archive), or - the failures `Mismatch` (rewritten chain), `Regressed` (head rolled back), `MissingHead`, - `MissingLink` (chained row deleted outside the purge paths), `CountMismatch` (same head, - rows + tombstones changed). `AuditChainHeadStoreTests` forges a chain with the production - hasher: `verify-chain` reports it Intact, `verify-heads` reports Mismatch. Coverage is from - the first export on: a forgery of links never exported is not detectable this way. - - **Roles (DBA hardening).** `AddAuditLogHashChain` adds `scadabridge_audit_chainer` - (SELECT on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`, read/write on the - head), grants the purger the head/tombstone rights its purge paths need, and DENYs - INSERT/UPDATE/DELETE on both chain tables to `scadabridge_audit_writer`. - **Parquet archival (T2) — shipped (ScadaBridge#42), default OFF.** See Retention & - Purge. The archive carries the hash-chain columns and the chain head (and, for the - cutoff month, the chain-start anchor in its tombstone file, so the archive verifies from - its own files and splits its unchained rows into pending vs pre-cutoff), so tamper - evidence survives the purge: `audit archive verify` ties an archive to the checkpoint - anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned by owner - decision (ScadaBridge#107/#109); a principal able to rewrite both the archive files and the - head row can forge a consistent pair — unless the month's head was exported off-box first - (`AuditLog:HashChain:HeadExport`): `audit verify-heads` then exposes the rewritten anchor. + Purge. Archives are unsigned by owner decision (ScadaBridge#109) and carry no tamper + evidence: `audit archive verify` proves file integrity against the manifest only. - **Site SQLite security.** File permissions: read/write by the ScadaBridge service account only. Not backed up off-machine — site SQLite is a buffer, not a record. @@ -1132,31 +956,10 @@ the global window; `PerChannelRetentionDays` specifies per-channel windows that are strictly shorter — any channel whose override equals or exceeds the global value is silently ignored (the global partition switch-out already governs it). -The nested `AuditLog:HashChain` section (ScadaBridge#41) runs the chaining job; its -`HeadExport` sub-section (ScadaBridge#107) exports the chain heads off-box, and the root -`AuditLog:MaintenanceConnectionString` moves the maintenance paths to a second login (see -Security & Tamper-Evidence): - -```jsonc -"AuditLog": { - "MaintenanceConnectionString": "${secret:SB-AUDIT-MAINT-DB}", // optional; unset = runtime connection - "HashChain": { - "Enabled": false, // default off - "IntervalSeconds": 60, - "BatchSize": 1000, - "MaxBatchesPerTick": 100, - "ChainBacklog": false, // default: chain only rows from first enablement on (#139); true = chain the backlog, first enablement only - "HeadExport": { - "Enabled": false, // default off - "IntervalMinutes": 60, - "ExportUnchanged": false, // true = every head every tick - "Sink": "File", // File | Syslog - "File": { "Directory": "\\\\fileserver\\audit-heads" }, // absolute, local or UNC - "Syslog": { "Host": "syslog.example", "Port": 514, "Protocol": "Udp", "Facility": 13, "AppName": "scadabridge", "TimeoutSeconds": 10 } - } - } -} -``` +`AuditLog:HashChain` (with its `HeadExport` sub-section) and +`AuditLog:MaintenanceConnectionString` were removed on 2026-11-15 with the hash chain. A node +that still carries them boots normally and logs one startup warning per key +(`RetiredAuditSettingsWarning`): delete them. The nested `AuditLog:Archive` section (ScadaBridge#42) turns on the Parquet cold archive (see Retention & Purge): @@ -1168,7 +971,6 @@ archive (see Retention & Purge): "Directory": "D:\\AuditArchive", // absolute; required when Enabled; reachable by both centrals "RowGroupSize": 50000, // 1,000 – 1,000,000 "ReadCommandTimeoutMinutes": 60, // reading one partition out of SQL Server - "ChainLockTimeoutSeconds": 300, // wait for a chaining pass before skipping the tick "RetentionDays": 0, // 0 = never delete; else > AuditLog:RetentionDays, from the archived month's end (#109) "DeleteSupersededStaleArchives": true // delete stale archives a later purged archive provably covers (#109) } @@ -1285,9 +1087,8 @@ orphaned entries) and in the CLI's `audit tree` output. - **[CLI (#19)](Component-CLI.md)** — `scadabridge audit query`, `scadabridge audit export`, `scadabridge audit tree --execution-id `, `scadabridge audit backfill-source-node --sentinel --before `, and - `scadabridge audit verify-chain --month ` (hash-chain verification via the - `VerifyAuditChain` management command, ScadaBridge#41); same permission requirements - as the UI. + `scadabridge audit archive list|verify|cleanup` (Parquet archives, ScadaBridge#42/#109); + same permission requirements as the UI. - **[KPI History (#26)](Component-KpiHistory.md)** — emits `IKpiSampleSource` (`AuditLogKpiSampleSource`, Global) consumed by the KpiHistory recorder (#26), reusing the existing audit-KPI reads; the resulting `totalEventsLastHour` / diff --git a/docs/requirements/Component-CLI.md b/docs/requirements/Component-CLI.md index 424057e9..1b6008fd 100644 --- a/docs/requirements/Component-CLI.md +++ b/docs/requirements/Component-CLI.md @@ -305,8 +305,8 @@ The new centralized Audit Log component (#23) is exposed via the `scadabridge au The `scadabridge audit` group targets the centralized Audit Log component (#23) and exposes the UI-equivalent operational audit surface. Permissions follow the same read-vs-export split the Central UI uses (see Component-AuditLog.md, Security & -Tamper-Evidence, and Security & Auth #10): `audit query`, `audit tree`, and -`audit verify-chain`, `audit verify-heads` and `audit archive list|verify` require the `OperationalAudit` +Tamper-Evidence, and Security & Auth #10): `audit query`, `audit tree` and +`audit archive list|verify` require the `OperationalAudit` permission; `audit export` additionally requires `AuditExport`; `audit backfill-source-node` requires the `Admin` role (maintenance path only). The server enforces permission checks and @@ -317,8 +317,6 @@ scadabridge audit query [--since ] [--until ] [--channel ] [--kind ] scadabridge audit export --since --until --format csv|jsonl|parquet --output [--channel ] [--kind ] [--status ] [--site ] [--target ] [--actor ] scadabridge audit tree --execution-id [--format table|json] scadabridge audit backfill-source-node --before [--sentinel ] [--batch ] -scadabridge audit verify-chain --month -scadabridge audit verify-heads --path scadabridge audit archive list scadabridge audit archive verify --name scadabridge audit archive cleanup [--dry-run] @@ -345,33 +343,14 @@ scadabridge audit archive cleanup [--dry-run] rows where `SourceNode IS NULL` and `OccurredAtUtc < --before`, in batches (`--batch`, default 5000). Admin-only maintenance command. Idempotent. Backed by `POST /api/audit/backfill-source-node`. -- `audit verify-chain --month ` (ScadaBridge#41) — verifies the AuditLog - hash chain for one UTC calendar month by sending the `VerifyAuditChain` management - command over `POST /management` (the one `audit` leaf that is a management contract - rather than a REST call). Reports `Intact` / `NotChained` / `Broken`, rows verified, - tombstones applied, the partition-purge anchor, the head sequence and unchained rows; - on `Broken`, the first break's partition, `ChainSeq`, `EventId` and kind. Exit code - `0` intact or not chained, `3` broken, `1` error, `2` authorization failure. Requires - `OperationalAudit` and a caller not scoped to specific sites. See - Component-AuditLog.md, Security & Tamper-Evidence. -- `audit verify-heads --path ` (ScadaBridge#107) — reads the AuditLog - chain heads exported off-box by `AuditLog:HashChain:HeadExport` (JSON lines from the file - sink, or any log a syslog collector wrote — each line's JSON object is taken from its first - `{` to its last `}`; a directory is read recursively for `*.jsonl`/`*.log`/`*.txt`), - collapses duplicates and sends them in the `VerifyAuditChainHeads` management command. The - server checks each exported head against the database chain — `Matches`, `Extended` (chain - grew; the link at the exported sequence still carries the exported hash), `Purged` (removed - by a partition purge; verify the archive), or the failures `Mismatch` (rewritten chain), - `Regressed`, `MissingHead`, `MissingLink`, `CountMismatch`. Exit `0` verified, `3` a - failure, `1` error (no records found, bad input, `CHAIN_BUSY`), `2` authorization failure. - Same gate as `verify-chain`. Run it together with `verify-chain`: a forger who rewrites rows - and re-hashes the chain inside SQL Server passes `verify-chain` but not `verify-heads`. +- `audit verify-chain` and `audit verify-heads` were removed on 2026-11-15 with the AuditLog + hash chain (IT security request; see Component-AuditLog.md, Security & Tamper-Evidence). - `audit archive list` / `audit archive verify --name ` (ScadaBridge#42) — send the `ListAuditArchives` / `VerifyAuditArchive` management commands for the AuditLog Parquet cold archives the retention purge writes (`AuditLog:Archive`, - default off). `verify` checks the files against their manifest, recomputes the - archived hash chain and compares its end with the purge anchor; exit `0` verified, - `3` not verified, `1` error, `2` authorization failure. Same gate as `verify-chain`. + default off). `verify` checks the data file against its manifest (size, SHA-256, row + count); exit `0` verified, `3` not verified, `1` error, `2` authorization failure. + Requires `OperationalAudit` and a caller not scoped to specific sites. See Component-AuditLog.md, Retention & Purge. - `audit archive cleanup [--dry-run]` (ScadaBridge#109) — sends `CleanupAuditArchives`: deletes superseded stale archives (a retried purge's, once a later purged archive provably diff --git a/docs/requirements/Component-ManagementService.md b/docs/requirements/Component-ManagementService.md index 1379b35a..fe486a42 100644 --- a/docs/requirements/Component-ManagementService.md +++ b/docs/requirements/Component-ManagementService.md @@ -199,10 +199,9 @@ The two-person authorization workflow for writes through the MxAccess Gateway. B ### Audit Log - **QueryAuditLog**: Legacy configuration-change audit query (filtered by entity type, user, date range, etc.) routed through `/management`. Gated to the `Admin` role; superseded for the centralized Audit Log component (#23) by the dedicated `/api/audit/*` REST endpoints described below. -- **VerifyAuditChain** (ScadaBridge#41): verifies one UTC calendar month (`Month`, `YYYY-MM`) of the central AuditLog hash chain and returns an `AuditChainVerificationResult` (status, rows verified, tombstones applied, purge anchor, head sequence, unchained rows, first break with partition / `ChainSeq` / `EventId` / kind). Gated on the OperationalAudit roles (`Administrator`, `Viewer`); a site-scoped non-Administrator caller is refused (`ManagementUnauthorized`) because a month's chain spans every site. Long-running Ask timeout. Errors: `INVALID_ARGUMENT` (bad month), `CHAIN_BUSY` (a chaining/purge pass held the chain lock past 60 s). Backs `scadabridge audit verify-chain`. -- **VerifyAuditChainHeads** (ScadaBridge#107): checks chain heads exported off-box (`Records`, at most 50,000 `AuditChainHeadRecord`s; duplicates collapsed) against the database chain and returns an `AuditChainHeadVerificationResult` (overall `Verified`, one `AuditChainHeadCheck` per distinct exported head: `Matches` / `Extended` / `Purged`, or the failures `Mismatch` / `Regressed` / `MissingHead` / `MissingLink` / `CountMismatch`). Same gate as VerifyAuditChain; long-running Ask timeout. Errors: `INVALID_ARGUMENT` (no records / over the cap), `CHAIN_BUSY`. Backs `scadabridge audit verify-heads`. -- **ListAuditArchives** / **VerifyAuditArchive** (ScadaBridge#42): list the AuditLog Parquet cold archives under `AuditLog:Archive:Directory` (`AuditArchiveListResult`: enabled flag + every manifest), and verify one by `Name` (`AuditArchiveVerificationResult`: file integrity, per-month chain recomputed from the archived rows, anchor `Matches`/`Superseded`/`NotPurged`/`Mismatch`, overall `Verified`). Same gate as VerifyAuditChain (OperationalAudit roles, site-scoped non-Administrator refused). VerifyAuditArchive uses the long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`, `INVALID_ARGUMENT` (not an archive name), `NOT_FOUND`. Back `scadabridge audit archive list|verify`. -- **CleanupAuditArchives** (ScadaBridge#109): runs the archive cleanup now (`DryRun` = report only) — superseded stale archives a later purged archive provably covers, and archives past `AuditLog:Archive:RetentionDays` — and returns an `AuditArchiveCleanupResult` (`DryRun`, `RetentionDays`, one item per archive deleted / kept stale, with the action `DeletedExpired` / `DeletedSuperseded` / `KeptStale` and reason). Administrator only; runs on the maintenance-connection archive when registered; long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`. Backs `scadabridge audit archive cleanup`. +- **VerifyAuditChain** / **VerifyAuditChainHeads** were removed on 2026-11-15 with the AuditLog hash chain (IT security request; see Component-AuditLog.md, Security & Tamper-Evidence). +- **ListAuditArchives** / **VerifyAuditArchive** (ScadaBridge#42): list the AuditLog Parquet cold archives under `AuditLog:Archive:Directory` (`AuditArchiveListResult`: enabled flag + every manifest), and verify one by `Name` (`AuditArchiveVerificationResult`: data-file size, SHA-256 and row count against the manifest, overall `Verified`). Gated on the OperationalAudit roles (`Administrator`, `Viewer`); a site-scoped non-Administrator caller is refused (`ManagementUnauthorized`) because an archive holds every site's rows. VerifyAuditArchive uses the long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`, `INVALID_ARGUMENT` (not an archive name), `NOT_FOUND`. Back `scadabridge audit archive list|verify`. +- **CleanupAuditArchives** (ScadaBridge#109): runs the archive cleanup now (`DryRun` = report only) — superseded stale archives a later purged archive provably covers, and archives past `AuditLog:Archive:RetentionDays` — and returns an `AuditArchiveCleanupResult` (`DryRun`, `RetentionDays`, one item per archive deleted / kept stale, with the action `DeletedExpired` / `DeletedSuperseded` / `KeptStale` and reason). Administrator only; long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`. Backs `scadabridge audit archive cleanup`. ### Transport (Bundle Import / Export) @@ -270,7 +269,7 @@ Every incoming message carries the authenticated user's identity and roles. The - **Verifier** role required for: approving / rejecting a secured write (`ApproveSecuredWriteCommand` / `RejectSecuredWriteCommand`). The no-self-approval rule (`Operator ≠ Verifier`) is enforced in the handler, independent of the role check. - **Any of Operator / Verifier / Administrator** required for: listing / querying secured writes (`ListSecuredWritesCommand`). Read-only, but the history exposes process-sensitive tag values, so it is gated any-of rather than open to every authenticated user (arch-review UA1). Enforced as an any-of gate (`GetRequiredRoles` returns the permitted set; the caller must hold at least one). - **Admin** role additionally required for: server-certificate trust/remove/list (`TrustServerCertCommand` / `RemoveServerCertCommand` / `ListServerCertsCommand`). -- **Any of Administrator / Viewer (OperationalAudit)** required for: AuditLog hash-chain verification (`VerifyAuditChainCommand`, ScadaBridge#41); the handler additionally refuses a site-scoped non-Administrator caller. +- **Any of Administrator / Viewer (OperationalAudit)** required for: listing and verifying AuditLog Parquet archives (`ListAuditArchivesCommand` / `VerifyAuditArchiveCommand`, ScadaBridge#42); the handler additionally refuses a site-scoped non-Administrator caller. - **Read-only access** (any authenticated role): health summary, health site, site event log queries, parked message queries. Unauthorized commands receive an `Unauthorized` response message. Failed authorization attempts are not audit logged (consistent with existing behavior). diff --git a/docs/requirements/HighLevelReqs.md b/docs/requirements/HighLevelReqs.md index 2ebc3709..a7636a19 100644 --- a/docs/requirements/HighLevelReqs.md +++ b/docs/requirements/HighLevelReqs.md @@ -479,8 +479,8 @@ Sections 10.1–10.4 cover **configuration-database audit** (config-mutating use - **AL-8**: Central retention defaults to **365 days**, enforced by a **monthly partition switch-and-drop** purge — no row-by-row delete. - **AL-9**: The site SQLite Audit Log is purged only when `ForwardState ∈ {Forwarded, Reconciled}` — i.e., a row must be either confirmed-forwarded *or* confirmed-reconciled before it can be removed. A central outage therefore **cannot cause audit loss at sites**. - **AL-10**: The Central UI exposes an **Audit Log page** with a cross-channel filter (by site, target, status, time range, correlation ID), plus **drill-ins from existing operational pages** (Site Calls, Notification Outbox, Inbound API). -- **AL-11**: Append-only semantics are **enforced via DB roles** (no UPDATE/DELETE granted on the `AuditLog` table to application accounts); a **tamper-evidence hash chain** (per-month SHA-256 chain, default off) shipped later by ScadaBridge#41. -- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export (csv, jsonl, parquet), hash-chain verification (verify-chain reports the first break of a month's hash chain; verify-heads checks off-box exported chain heads against the database, ScadaBridge#107), and listing/verifying the Parquet cold archives (ScadaBridge#42) against the central Audit Log. +- **AL-11**: Append-only semantics are **enforced via DB roles** (no UPDATE/DELETE granted on the `AuditLog` table to application accounts). The tamper-evidence hash chain that ScadaBridge#41 shipped later was removed on 2026-11-15 at IT security's request, pending a family-wide general solution. +- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export (csv, jsonl, parquet), and listing/verifying the Parquet cold archives (ScadaBridge#42) against the central Audit Log. ## 11. Health Monitoring diff --git a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md index 4a9e386f..2abc1141 100644 --- a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md +++ b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md @@ -2096,69 +2096,8 @@ The command is idempotent — running it multiple times converges (only rows whe `SourceNode IS NULL` are eligible; already-set rows are untouched). Backed by `POST /api/audit/backfill-source-node`. Requires `Admin` role. -#### `audit verify-chain` - -Verify the audit log hash chain for a given month. - -```sh -scadabridge --url audit verify-chain --month -``` - -| Option | Required | Default | Description | -|--------|----------|---------|-------------| -| `--month` | yes | — | UTC calendar month to verify, `YYYY-MM` (e.g. `2026-05`) | - -Sends the `VerifyAuditChain` management command (ScadaBridge#41). The server -recomputes every link of that month's AuditLog hash chain and reports the first -break, if any: partition, `ChainSeq`, `EventId` and kind (`HashMismatch` = edited or -reordered row, `MissingRow` = deleted row, `HeadMismatch` = rows deleted from the -end, `DuplicateSequence`, `IncompleteLink`). `--format table` prints a summary; -`json` prints the result object. - -| Exit code | Meaning | -|-----------|---------| -| `0` | `Intact`, or `NotChained` (nothing chained for that month yet) | -| `3` | `Broken` — tamper evidence; see `firstBreak` | -| `1` | Request error (bad month, `CHAIN_BUSY`, server error) | -| `2` | Authorization failure (needs `Administrator` or `Viewer`, not site-scoped) | - -The chain only exists once `AuditLog:HashChain:Enabled` is on for the central nodes -(default off); see `Component-AuditLog.md`, Security & Tamper-Evidence. - -#### `audit verify-heads` - -Check AuditLog chain heads that were exported off-box (ScadaBridge#107) against the -database chain. - -```sh -scadabridge --url audit verify-heads --path -``` - -| Option | Required | Default | Description | -|--------|----------|---------|-------------| -| `--path` | yes | — | An export file, or a directory read recursively for `*.jsonl` / `*.log` / `*.txt` | - -The central nodes write one JSON record per month head (`month`, `lastSeq`, `lastHash`, -purge anchor, chained-row and tombstone counts, `exportedAtUtc`, `node`) when -`AuditLog:HashChain:HeadExport:Enabled` is on — to append-only files -(`{Directory}/{yyyy}/audit-chain-heads-{yyyy-MM-dd}-{node}.jsonl`) or to syslog (RFC 5424, the -JSON record as the message). The CLI reads both forms (the JSON object is taken from each -line's first `{` to its last `}`, so a syslog collector's prefixes do not matter), keeps one -record per distinct (month, sequence, hash), and sends `VerifyAuditChainHeads`. Per record the -server reports `Matches`, `Extended`, `Purged` (the link was removed by a partition purge — -check the month's archive), or a failure: `Mismatch` (the chain was rewritten and re-hashed), -`Regressed` (head rolled back), `MissingHead`, `MissingLink` (a chained row deleted outside the -purge paths), `CountMismatch`. - -| Exit code | Meaning | -|-----------|---------| -| `0` | Every exported head is consistent with the database | -| `3` | At least one failure — tamper evidence | -| `1` | No records found, bad input, `CHAIN_BUSY`, server error | -| `2` | Authorization failure (needs `Administrator` or `Viewer`, not site-scoped) | - -Run it with `audit verify-chain`: `verify-chain` recomputes each month's links; -`verify-heads` proves the database still holds the heads that left the box. +`audit verify-chain` and `audit verify-heads` were removed on 2026-11-15 with the AuditLog +hash chain (IT security request; see `Component-AuditLog.md`, Security & Tamper-Evidence). #### `audit archive list` / `audit archive verify` @@ -2176,17 +2115,15 @@ scadabridge --url audit archive verify --name `list` sends `ListAuditArchives` and prints each archive's name, partition, status (`Archived` = written, purge not confirmed; `Purged` = the partition was switched out -and the archive is the only copy), row counts and size. `verify` sends -`VerifyAuditArchive`: the server checks both Parquet files' SHA-256 against the -manifest, recomputes the archived month's hash chain from the archived rows and -tombstones, and compares the chain end with the checkpoint anchor the purge left in -`AuditLogChainHead` (`Matches`, `Superseded` by a later purge of late rows, -`NotPurged`, or `Mismatch`). +and the archive is the only copy), row count and size. `verify` sends +`VerifyAuditArchive`: the server checks the data file's size and SHA-256 and its row +count against the manifest. It proves the file is the one the archive wrote; archives are +unsigned and carry no tamper evidence. | Exit code | Meaning | |-----------|---------| | `0` | Verified | -| `3` | Not verified — a file was altered, the chain is broken, or the anchor mismatches | +| `3` | Not verified — the data file is missing, altered, or its row count differs from the manifest | | `1` | Request error (`ARCHIVE_NOT_CONFIGURED`, `NOT_FOUND`, `INVALID_ARGUMENT`, server error) | | `2` | Authorization failure (needs `Administrator` or `Viewer`, not site-scoped) |