From 3ff463b6fd4fcff7a51ebd78ff456c6820add5ba Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 9 Nov 2026 17:01:03 -0500 Subject: [PATCH] docs(audit): Parquet cold archive shipped, default off (ScadaBridge#42) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Component-AuditLog Retention & Purge describes the archive: directory and layout, the three files, consistency with the hash chain (SHARED read, guarded switch, anchor = archived chain end), fail-closed purge, retention interplay (per-channel overrides are not archived, their tombstones are), verify/read, and the AuditLog:Archive configuration. alog.md §13.6/§15.2/§16, CLAUDE.md, HighLevelReqs AL-12, Component-CLI, Component-ManagementService, docs/components/{CLI,ManagementService}.md and the CLI README drop the "deferred / 501" wording and document `audit archive list|verify` and the parquet export. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV --- CLAUDE.md | 3 +- alog.md | 13 ++-- docs/components/CLI.md | 7 +- docs/components/ManagementService.md | 2 +- docs/requirements/Component-AuditLog.md | 69 ++++++++++++++++++- docs/requirements/Component-CLI.md | 17 ++++- .../Component-ManagementService.md | 1 + docs/requirements/HighLevelReqs.md | 2 +- src/ZB.MOM.WW.ScadaBridge.CLI/README.md | 40 ++++++++++- 9 files changed, 136 insertions(+), 18 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 838deb50..60503be4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -181,7 +181,8 @@ spec for each is `docs/requirements/Component-.md`, and `README.md` carrie - Payload cap 8 KB by default / 64 KB on error rows; auth headers redacted by default; SQL parameter values captured by default; per-target redaction opt-in. Inbound API: full verbatim capture up to `InboundMaxBytes` (default 1 MiB); request headers stored in `Extra.requestHeaders` (post-redaction); per-method `SkipBodyCapture` flag suppresses bodies while still recording headers + metadata; `AuditInboundCeilingHits` counter surfaced on health snapshot. (M5.3 T7) - Audit-write failure NEVER aborts the user-facing action — audit is best-effort, the action's own success/failure path is authoritative. - 365-day central retention with monthly partition-switch purge; per-channel retention overrides (`AuditLog:PerChannelRetentionDays`) expire rows earlier than the global window via a bounded, batched row DELETE on the purge actor's maintenance path — values must be shorter than the global window (M5.5 T3); 7-day site SQLite retention with a hard `ForwardState` invariant (no row purged until forwarded or reconciled). -- Append-only enforced via DB roles (writer role has INSERT only, no UPDATE/DELETE); Parquet archival (T2) is deferred to v1.x. +- Append-only enforced via DB roles (writer role has INSERT only, no UPDATE/DELETE). +- **Parquet cold archive (T2, ScadaBridge#42) — default OFF (`AuditLog:Archive:Enabled` + absolute `Directory`, shared by both centrals).** The purge actor writes each expired partition to `{Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet` (10 stored columns + `ChainSeq`/`RowHash`, tick-exact `OccurredAtUtcTicks`) plus `.tombstones.parquet` and a `.manifest.json` (written last; chain head at archive time, SHA-256 per file), read under the chain lock SHARED; then the switch runs with a guard (under the EXCLUSIVE lock: partition row count + each month's `LastSeq` must still equal the archive's, else THROW 51002/51003 and nothing is purged). Fail closed: a partition that cannot be archived is not purged. `audit archive list|verify` = `ListAuditArchives`/`VerifyAuditArchive` (verify-chain's gate): file hashes, chain recomputed from the archive, chain end vs the DB purge anchor (`Matches`/`Superseded`/`NotPurged`/`Mismatch`), CLI exit 3. `audit export --format parquet` streams the 24 export columns (was 501). No migration; no re-import path (open decision), object storage not wired (mount it). - **Hash-chain tamper evidence (T1, ScadaBridge#41) — default OFF (`AuditLog:HashChain:Enabled`).** One SHA-256 chain per UTC calendar month: `ChainSeq` + `RowHash = SHA-256(prev || canonical(row))` on `AuditLog` (migration `AddAuditLogHashChain`), linked by the central `AuditLogHashChainService` (both nodes, serialised by `sp_getapplock`) — never by the ingest path. `AuditLogChainHead` catches tail deletion and carries the partition-switch checkpoint anchor; the per-channel DELETE writes an `AuditLogChainTombstone` per chained row in the same transaction. `audit verify-chain --month` = management command `VerifyAuditChain` (OperationalAudit, unscoped callers only), first break with partition/ChainSeq/EventId, CLI exit 3. A chained row's `SourceNode` is frozen (backfill skips it). Unsigned; signing/external anchoring are open. - Node-of-origin is captured alongside site-of-origin: `SourceNode` (`varchar(64)` NULL) on `AuditLog`, `Notifications`, and `SiteCalls` — `node-a`/`node-b` for site rows (qualified by `SourceSiteId`/`SourceSite`), `central-a`/`central-b` for central direct-write rows. Stamped at the writing node, carried verbatim through telemetry + reconciliation, and indexed via `IX_AuditLog_Node_Occurred (SourceNode, OccurredAtUtc)` on `AuditLog`. - Per-node stuck KPIs (M5.3 T6): Notification Outbox and Site Call Audit expose `PerNodeNotificationKpiRequest`/`PerNodeSiteCallKpiRequest` messages that group stuck/parked/delivered counts by `SourceNode`, surfacing per-node breakdowns on the Health dashboard. diff --git a/alog.md b/alog.md index 7027e0ca..1e265223 100644 --- a/alog.md +++ b/alog.md @@ -571,7 +571,7 @@ MS SQL handles this with batched ingest and the time-aligned indexes. ### 13.6 Levers - Reduce `DefaultCapBytes` per §8.1. - Tighten per-channel retention per §12.1 (especially `DbOutbound.DbWrite` read traffic). -- Defer to v1.x: Parquet archival to object storage before purge (§15.2). +- Parquet archival before purge (§15.2) — shipped by ScadaBridge#42, default off. --- @@ -601,8 +601,13 @@ New `scadabridge audit` command group: Requires the same **OperationalAudit** / **AuditExport** permissions as the UI. -### 15.2 Object-storage archival (deferred, v1.x) -A monthly job dumps the closing partition to Parquet on operator-configured object storage before central purge — enabling indefinite cold retention without bloating MS SQL. Flag for v1.x; not in initial scope. +### 15.2 Parquet archival (shipped, ScadaBridge#42 — default off) +The retention purge dumps each expiring monthly partition to Parquet before switching it out — enabling indefinite cold retention without bloating MS SQL. Shipped as: +- `AuditLog:Archive:Enabled` (default **off**) + `Directory` (absolute path both central nodes reach — a shared volume or UNC share; object storage is mounted, not called). +- Per partition: `auditlog-YYYY-MM-Z.parquet` (the stored columns + `ChainSeq`/`RowHash`), `.tombstones.parquet` (per-channel purge tombstones) and `.manifest.json` (counts, SHA-256 per file, the month's chain head at archive time), under `{Directory}/{YYYY}/`. +- Archive before purge, fail closed: the switch runs only after the archive is durable, and re-checks under the chain lock that the partition still holds exactly the archived rows and chain head; otherwise nothing is purged and the next tick re-archives. +- `scadabridge audit archive list` / `verify --name `: verify recomputes the archived chain and ties its end to the checkpoint anchor the purge left (§11.4). +- `audit export --format parquet` streams the export columns as Parquet. --- @@ -613,7 +618,7 @@ A monthly job dumps the closing partition to Parquet on operator-configured obje | 1 | Component number | **#23 Audit Log** (README matrix + HighLevelReqs). | | 2 | Nav placement | New top-level **Audit** nav group in Central UI. | | 3 | Hash-chain tamper evidence (§11.4) | Deferred to v1.x; shipped by ScadaBridge#41 (unsigned SHA-256 chain, default off). | -| 4 | Parquet archival to object storage (§15.2) | Deferred to v1.x. | +| 4 | Parquet archival to object storage (§15.2) | Deferred to v1.x; shipped by ScadaBridge#42 (Parquet on a configured directory, default off). | | 5 | Per-channel retention overrides (§12.1) | Deferred to v1.x. v1 uses a single global `RetentionDays`. | | 6 | Default payload cap | **8 KB** for `RequestSummary` / `ResponseSummary`; **64 KB** on error rows (`Status IN ('Failed', 'Parked', 'Discarded')`). | diff --git a/docs/components/CLI.md b/docs/components/CLI.md index 4024540e..ee500c0f 100644 --- a/docs/components/CLI.md +++ b/docs/components/CLI.md @@ -23,7 +23,8 @@ Direct-to-node access (`http://localhost:9001` for `central-a`, `http://localhos Management commands use `POST /management`. Audit query, export, and tree use `GET /api/audit/query`, `/api/audit/export`, and `/api/audit/tree`; source-node backfill uses `POST /api/audit/backfill-source-node`. `audit verify-chain` sends the -`VerifyAuditChain` management command (ScadaBridge#41). `debug stream` uses a persistent +`VerifyAuditChain` management command (ScadaBridge#41); `audit archive list|verify` send +`ListAuditArchives` / `VerifyAuditArchive` (ScadaBridge#42). `debug stream` uses a persistent SignalR connection. Audit export streams the response without buffering it all. ### Authentication and roles @@ -120,7 +121,7 @@ scadabridge instance deploy --id 7 | `db-connection` | `list`, `get`, `create`, `update`, `delete` | `Designer` | | `api-method` | `list`, `get`, `create`, `update`, `delete`, `validate`, `recompile` | `Designer` | | `bundle` | `export`, `preview`, `import` | Designer export; Administrator preview/import | -| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `verify-chain` | Audit roles; Administrator backfill; verify-chain needs an unscoped audit role | +| `audit` | `query`, `export`, `tree`, `backfill-source-node`, `verify-chain`, `archive list`, `archive verify` | Audit roles; Administrator backfill; verify-chain and archive need an unscoped audit role | | `audit-config` | `query` (config-change audit trail; was `audit-log` pre-M8) | `Admin` | | `health` | `summary`, `pending-updates`, `site`, `event-log`, `parked-messages` | Authenticated reads with server-side scope checks | | `debug` | `snapshot`, `stream` | `Deployer` | @@ -195,7 +196,7 @@ The management URL is resolved in this order: `--url` flag → `SCADABRIDGE_MANA - [Management Service (#18)](./ManagementService.md) — the server-side counterpart. Every CLI command (except the audit REST/placeholder and SignalR stream operations) translates to a named management command dispatched through `POST /management` to the `ManagementActor`. Role enforcement and LDAP authentication are applied there. The `ManagementCommandRegistry` in Commons maps command types to their names; both sides must stay in sync. - [Traefik Proxy (#20)](./TraefikProxy.md) — the recommended connection target. Pointing `--url` at the Traefik address ensures requests reach the active central node without per-command failover logic in the CLI. The `debug stream` command's WebSocket connection (SignalR `/hubs/debug-stream`) also traverses Traefik, which proxies the WebSocket upgrade natively. -- [Audit Log (#23)](./AuditLog.md) — the `audit` command group targets the query, export, tree, and backfill REST endpoints exposed by the Audit Log component, bypassing the management command envelope; `verify-chain` is a local placeholder. The `audit-config` group (formerly `audit-log`) targets the configuration-change audit trail (`IAuditService`) via the standard management envelope. +- [Audit Log (#23)](./AuditLog.md) — the `audit` command group targets the query, export, tree, and backfill REST endpoints exposed by the Audit Log component, bypassing the management command envelope; `verify-chain` and `archive list|verify` are management commands. The `audit-config` group (formerly `audit-log`) targets the configuration-change audit trail (`IAuditService`) via the standard management envelope. - [Security & Auth (#10)](./Security.md) — the server resolves LDAP credentials and maps group memberships to ScadaBridge roles (including `Administrator`, `Designer`, `Deployer`, `Operator`, `Verifier`, and audit roles). The CLI does not interact with LDAP directly. - [Commons (#16)](./Commons.md) — owns the management command record types and the `ManagementCommandRegistry` that maps each type to its wire name. The CLI project references Commons for these contracts. - [Transport (#24)](./Transport.md) — the `bundle` command group drives the Transport feature: `bundle export` requests a base64-encoded bundle from the server and streams it to a local `.scadabundle` file; `bundle preview` uploads a file and returns the diff manifest; `bundle import` uploads a file and applies it with a configurable conflict policy. diff --git a/docs/components/ManagementService.md b/docs/components/ManagementService.md index ebba4ba0..4866faaf 100644 --- a/docs/components/ManagementService.md +++ b/docs/components/ManagementService.md @@ -128,7 +128,7 @@ private void HandleEnvelope(ManagementEnvelope envelope) `AuditEndpoints.MapAuditAPI` registers two GET endpoints that go directly to `IAuditLogRepository`, bypassing the actor: - `GET /api/audit/query` — keyset-paged JSON result. Requires `OperationalAudit` permission (Admin / Audit / AuditReadOnly roles). Accepts `channel`, `kind`, `status`, `sourceSiteId`, `correlationId`, `executionId`, `parentExecutionId`, `fromUtc`, `toUtc`, `pageSize`, and cursor params `afterOccurredAtUtc`/`afterEventId`. Returns `{ events, nextCursor }` where `nextCursor` is explicit `null` on the last page. -- `GET /api/audit/export` — server-side streaming export (CSV or JSONL) of all matching rows, paging the repository internally at 1 000 rows per batch and flushing after each batch. Requires `AuditExport` permission (Admin / Audit roles). `format=parquet` returns HTTP 501 (deferred). +- `GET /api/audit/export` — server-side streaming export (CSV, JSONL or Parquet) of all matching rows, paging the repository internally at 1 000 rows per batch and flushing after each batch (Parquet: after each 10 000-row row group; same 24 columns, content type `application/vnd.apache.parquet` — ScadaBridge#42). Requires `AuditExport` permission (Admin / Audit roles). Both endpoints apply the same HTTP Basic Auth / LDAP / role flow as `/management`. Site-scoped callers have their `sourceSiteId` filter intersected with their `PermittedSiteIds`; an explicit out-of-scope filter returns HTTP 403 rather than silently empty results. diff --git a/docs/requirements/Component-AuditLog.md b/docs/requirements/Component-AuditLog.md index 75f5c94e..9d15852b 100644 --- a/docs/requirements/Component-AuditLog.md +++ b/docs/requirements/Component-AuditLog.md @@ -759,6 +759,51 @@ MS SQL for direct-write events). Unredacted secrets never persist. partition switch-out loop. Values are validated to be in `[30, RetentionDays]`; keys that are not a recognized `AuditChannel` enum name are rejected at startup. +- **Parquet cold archive (T2, ScadaBridge#42) — default OFF.** With + `AuditLog:Archive:Enabled` the purge actor archives every expired partition before + switching it out, so audit rows outlive `RetentionDays` as Parquet files: + - **Where.** `AuditLog:Archive:Directory` — an absolute path (validated at startup) + that both central nodes reach, because the purge runs on the active node and + `audit archive list|verify` on whichever node serves the call. Files land in + `{Directory}/{YYYY}/`. Object storage (S3 / Azure Blob) is not called directly — + mount it as a file system or copy the files on. + - **What.** Per partition, three files sharing the stem + `auditlog-{YYYY-MM}-{archivedAt:yyyyMMddTHHmmssfff}Z`: `.parquet` (every row of the + partition — the ten stored columns, `OccurredAtUtc` both as a UTC microsecond + timestamp and as exact `OccurredAtUtcTicks`, and `ChainSeq`/`RowHash`, null for + rows the chaining job had not reached; computed columns are not stored, they derive + from `DetailsJson`), `.tombstones.parquet` (the month's per-channel purge + tombstones) and `.manifest.json` (row / chained / unchained / tombstone counts, + SHA-256 and size of each file, and each chain month's head — `LastSeq`, `LastHash`, + and the anchor an earlier purge left). Each file is written to a `.tmp` sibling, + flushed and renamed; the manifest goes last and is the commit marker. Archives are + never overwritten. Row groups of `RowGroupSize` (default 50,000) bound memory. + - **Consistent with the hash chain.** The partition's rows, heads and tombstones are + read in one transaction holding the chain lock SHARED, so no chaining pass or purge + moves the chain mid-read. The switch then runs with an archive guard: under the + EXCLUSIVE chain lock it re-counts the partition and re-reads each month's + `LastSeq`, and THROWs (51002 row count changed / 51003 chain advanced — nothing + purged) unless both still equal the archive's. The switch's checkpoint anchor + (`PurgedThroughSeq/Hash`) therefore equals the archive's recorded chain end. A + stale archive keeps manifest status `Archived`; the next tick writes a fresh one. + - **Fail closed.** A partition that cannot be archived (volume full, lock busy past + `ChainLockTimeoutSeconds`, …) is not purged: the failure takes the usual + `AuditLogPurgeFailedEvent` / `PurgeFailures` path and the table keeps the rows. + A successful archive publishes `AuditLogArchivedEvent` before `AuditLogPurgedEvent`. + - **Retention interplay.** Archiving changes nothing about *when* a partition is + purged (`RetentionDays`). Per-channel overrides still delete their rows earlier, + so those rows are never archived — the archive carries their tombstones instead, + which is what keeps the archived chain verifiable. Archive files have no retention + of their own. + - **Verify / read.** `audit archive verify --name` (`VerifyAuditArchive`) checks both + files' SHA-256 and counts against the manifest, recomputes each archived month's + chain from the archived rows and tombstones (`AuditChainVerifier`, starting from + any earlier anchor), and compares the chain end with the database's current anchor: + `Matches`; `Superseded` (late rows for the month were chained, purged and archived + again afterwards — that later archive resumes from this one's end); `NotPurged`; + or `Mismatch` (archive or head altered). The files are plain Parquet — DuckDB, + pyarrow or Spark read them directly. There is no re-import into `dbo.AuditLog`: + a re-inserted expired row would be purged again by the next tick. - **Sites:** `SiteAuditRetentionService` (site-only `IHostedService`) runs the site SQLite retention purge on a timer — first tick after `AuditLog:SiteRetention:InitialDelay` (default 5 min, short so a daily-recycled @@ -868,9 +913,12 @@ MS SQL for direct-write events). Unredacted secrets never persist. (SELECT on `AuditLog`, column-level `UPDATE (ChainSeq, RowHash)`, read/write on the head), grants the purger the head/tombstone rights its purge paths need, and DENYs INSERT/UPDATE/DELETE on both chain tables to `scadabridge_audit_writer`. -- **Parquet archival (T2) — deferred to v1.x.** Long-term cold storage of purged - monthly partitions as Parquet files (suitable for offline analytics) will be - added in a future milestone. (T1 shipped later — see above; T2 is still deferred.) +- **Parquet archival (T2) — shipped (ScadaBridge#42), default OFF.** See Retention & + Purge. The archive carries the hash-chain columns and the chain head, so tamper + evidence survives the purge: `audit archive verify` ties an archive to the checkpoint + anchor still in `AuditLogChainHead`. Like the chain itself, archives are unsigned; + a principal able to rewrite both the archive files and the head row can forge a + consistent pair. - **Site SQLite security.** File permissions: read/write by the ScadaBridge service account only. Not backed up off-machine — site SQLite is a buffer, not a record. @@ -934,6 +982,21 @@ the global window; `PerChannelRetentionDays` specifies per-channel windows that are strictly shorter — any channel whose override equals or exceeds the global value is silently ignored (the global partition switch-out already governs it). +The nested `AuditLog:Archive` section (ScadaBridge#42) turns on the Parquet cold +archive (see Retention & Purge): + +```jsonc +"AuditLog": { + "Archive": { + "Enabled": false, // default off + "Directory": "D:\\AuditArchive", // absolute; required when Enabled; reachable by both centrals + "RowGroupSize": 50000, // 1,000 – 1,000,000 + "ReadCommandTimeoutMinutes": 60, // reading one partition out of SQL Server + "ChainLockTimeoutSeconds": 300 // wait for a chaining pass before skipping the tick + } +} +``` + The nested `AuditLog:Purge` section controls the purge actor cadence and batch size: diff --git a/docs/requirements/Component-CLI.md b/docs/requirements/Component-CLI.md index ad356494..53a525c3 100644 --- a/docs/requirements/Component-CLI.md +++ b/docs/requirements/Component-CLI.md @@ -304,7 +304,8 @@ The `scadabridge audit` group targets the centralized Audit Log component (#23) exposes the UI-equivalent operational audit surface. Permissions follow the same read-vs-export split the Central UI uses (see Component-AuditLog.md, Security & Tamper-Evidence, and Security & Auth #10): `audit query`, `audit tree`, and -`audit verify-chain` require the `OperationalAudit` permission; `audit export` +`audit verify-chain` and `audit archive list|verify` require the `OperationalAudit` +permission; `audit export` additionally requires `AuditExport`; `audit backfill-source-node` requires the `Admin` role (maintenance path only). The server enforces permission checks and returns HTTP 403 (CLI exit code 2) on denial. @@ -315,6 +316,8 @@ scadabridge audit export --since --until --format csv|jsonl|parquet --ou scadabridge audit tree --execution-id [--format table|json] scadabridge audit backfill-source-node --before [--sentinel ] [--batch ] scadabridge audit verify-chain --month +scadabridge audit archive list +scadabridge audit archive verify --name ``` - `audit query` — filtered query against the central `AuditLog` table, matching the @@ -323,8 +326,9 @@ scadabridge audit verify-chain --month JSON (default) or table. - `audit export` — server-side streaming export of the central `AuditLog` to the requested format (`csv`, `jsonl`, `parquet`) written to `--output`. The server - streams rows rather than materializing them in memory; the CLI writes bytes - through to disk. Supports the same scoping filters as `audit query`. + streams rows rather than materializing them in memory (Parquet: one row group of + 10,000 rows at a time, the same 24 columns as csv/jsonl — ScadaBridge#42); the CLI + writes bytes through to disk. Supports the same scoping filters as `audit query`. - `audit tree --execution-id ` (M5.3 T8) — renders the full execution-chain tree for the given `ExecutionId`. The server resolves the root from any node in the chain (walks `ParentExecutionId` to find the root, then traverses downward) @@ -346,6 +350,13 @@ scadabridge audit verify-chain --month `0` intact or not chained, `3` broken, `1` error, `2` authorization failure. Requires `OperationalAudit` and a caller not scoped to specific sites. See Component-AuditLog.md, Security & Tamper-Evidence. +- `audit archive list` / `audit archive verify --name ` (ScadaBridge#42) — + send the `ListAuditArchives` / `VerifyAuditArchive` management commands for the + AuditLog Parquet cold archives the retention purge writes (`AuditLog:Archive`, + default off). `verify` checks the files against their manifest, recomputes the + archived hash chain and compares its end with the purge anchor; exit `0` verified, + `3` not verified, `1` error, `2` authorization failure. Same gate as `verify-chain`. + See Component-AuditLog.md, Retention & Purge. ### Health Commands ``` diff --git a/docs/requirements/Component-ManagementService.md b/docs/requirements/Component-ManagementService.md index e6dba714..7b0db6ac 100644 --- a/docs/requirements/Component-ManagementService.md +++ b/docs/requirements/Component-ManagementService.md @@ -200,6 +200,7 @@ The two-person authorization workflow for writes through the MxAccess Gateway. B - **QueryAuditLog**: Legacy configuration-change audit query (filtered by entity type, user, date range, etc.) routed through `/management`. Gated to the `Admin` role; superseded for the centralized Audit Log component (#23) by the dedicated `/api/audit/*` REST endpoints described below. - **VerifyAuditChain** (ScadaBridge#41): verifies one UTC calendar month (`Month`, `YYYY-MM`) of the central AuditLog hash chain and returns an `AuditChainVerificationResult` (status, rows verified, tombstones applied, purge anchor, head sequence, unchained rows, first break with partition / `ChainSeq` / `EventId` / kind). Gated on the OperationalAudit roles (`Administrator`, `Viewer`); a site-scoped non-Administrator caller is refused (`ManagementUnauthorized`) because a month's chain spans every site. Long-running Ask timeout. Errors: `INVALID_ARGUMENT` (bad month), `CHAIN_BUSY` (a chaining/purge pass held the chain lock past 60 s). Backs `scadabridge audit verify-chain`. +- **ListAuditArchives** / **VerifyAuditArchive** (ScadaBridge#42): list the AuditLog Parquet cold archives under `AuditLog:Archive:Directory` (`AuditArchiveListResult`: enabled flag + every manifest), and verify one by `Name` (`AuditArchiveVerificationResult`: file integrity, per-month chain recomputed from the archived rows, anchor `Matches`/`Superseded`/`NotPurged`/`Mismatch`, overall `Verified`). Same gate as VerifyAuditChain (OperationalAudit roles, site-scoped non-Administrator refused). VerifyAuditArchive uses the long-running Ask timeout. Errors: `ARCHIVE_NOT_CONFIGURED`, `INVALID_ARGUMENT` (not an archive name), `NOT_FOUND`. Back `scadabridge audit archive list|verify`. ### Transport (Bundle Import / Export) diff --git a/docs/requirements/HighLevelReqs.md b/docs/requirements/HighLevelReqs.md index 7b4c95ee..e09b05f5 100644 --- a/docs/requirements/HighLevelReqs.md +++ b/docs/requirements/HighLevelReqs.md @@ -480,7 +480,7 @@ Sections 10.1–10.4 cover **configuration-database audit** (config-mutating use - **AL-9**: The site SQLite Audit Log is purged only when `ForwardState ∈ {Forwarded, Reconciled}` — i.e., a row must be either confirmed-forwarded *or* confirmed-reconciled before it can be removed. A central outage therefore **cannot cause audit loss at sites**. - **AL-10**: The Central UI exposes an **Audit Log page** with a cross-channel filter (by site, target, status, time range, correlation ID), plus **drill-ins from existing operational pages** (Site Calls, Notification Outbox, Inbound API). - **AL-11**: Append-only semantics are **enforced via DB roles** (no UPDATE/DELETE granted on the `AuditLog` table to application accounts); a **tamper-evidence hash chain** (per-month SHA-256 chain, default off) shipped later by ScadaBridge#41. -- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export, and hash-chain verification (verify-chain reports the first break of a month's hash chain) against the central Audit Log. +- **AL-12**: The CLI provides a `scadabridge audit` command group for query, export (csv, jsonl, parquet), hash-chain verification (verify-chain reports the first break of a month's hash chain), and listing/verifying the Parquet cold archives (ScadaBridge#42) against the central Audit Log. ## 11. Health Monitoring diff --git a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md index cc2decd6..f6fa627b 100644 --- a/src/ZB.MOM.WW.ScadaBridge.CLI/README.md +++ b/src/ZB.MOM.WW.ScadaBridge.CLI/README.md @@ -1996,8 +1996,10 @@ scadabridge --url audit export --since