feat(audit): flip AuditLog clustered key to (OccurredAtUtc, EventId) (ScadaBridge#111)
Variant B2 of the #78 benchmark: migration FlipAuditLogClusteredKey drops the twelve non-clustered indexes, re-adds PK_AuditLog as (OccurredAtUtc, EventId) on ps_AuditLog_Month, adds the aligned unique UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc) for the idempotency probe, and rebuilds the twelve indexes unchanged. Guarded (re-runnable) Up, inverse-rebuild Down; production scripts docs/plans/sql/FlipAuditLogClusteredKey{,.rollback}.sql. The partition-switch purge staging table now reads the live clustered key order from sys.index_columns, so it switches either shape (old binaries hard-code the old order: ship with the binaries, both centrals stopped). Evidence: 300k rows, page 1 90,532 -> 34 logical reads (ordered backward clustered scan under Top), keyset page 90,532 -> 46, probe 14 -> 14; FlipAuditLogClusteredKeyMigrationTests pins indexes, grants, Down/Up, the staging key (mutation: hard-coded old key fails SWITCH) and the read ratio. Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
@@ -182,6 +182,7 @@ spec for each is `docs/requirements/Component-<Name>.md`, and `README.md` carrie
|
||||
- Audit-write failure NEVER aborts the user-facing action — audit is best-effort, the action's own success/failure path is authoritative.
|
||||
- 365-day central retention with monthly partition-switch purge; per-channel retention overrides (`AuditLog:PerChannelRetentionDays`) expire rows earlier than the global window via a bounded, batched row DELETE on the purge actor's maintenance path — values must be shorter than the global window (M5.5 T3); 7-day site SQLite retention with a hard `ForwardState` invariant (no row purged until forwarded or reconciled).
|
||||
- Append-only enforced via DB roles (writer role has INSERT only, no UPDATE/DELETE).
|
||||
- **Clustered key `(OccurredAtUtc, EventId)` + aligned unique `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` (migration `FlipAuditLogClusteredKey`, #111, B2 of the #78 benchmark).** The Audit Log page's newest-first page/keyset now reads an ordered range instead of the whole table (300k rows: 90,532 → 34 reads). Offline whole-table rebuild — script `docs/plans/sql/FlipAuditLogClusteredKey.sql` (+ `.rollback.sql`), both centrals stopped: the partition-switch staging table now reads the live key order, but binaries older than the flip hard-code the old one. DEV/QA not applied (sizing + window in #111).
|
||||
- **Parquet cold archive (T2, ScadaBridge#42) — default OFF (`AuditLog:Archive:Enabled` + absolute `Directory`, shared by both centrals).** The purge actor writes each expired partition to `{Directory}/{YYYY}/auditlog-{YYYY-MM}-{stamp}Z.parquet` (10 stored columns + `ChainSeq`/`RowHash`, tick-exact `OccurredAtUtcTicks`) plus `.tombstones.parquet` and a `.manifest.json` (written last; chain head at archive time, SHA-256 per file), read under the chain lock SHARED; then the switch runs with a guard (under the EXCLUSIVE lock: partition row count + each month's `LastSeq` must still equal the archive's, else THROW 51002/51003 and nothing is purged). Fail closed: a partition that cannot be archived is not purged. `audit archive list|verify` = `ListAuditArchives`/`VerifyAuditArchive` (verify-chain's gate): file hashes, chain recomputed from the archive, chain end vs the DB purge anchor (`Matches`/`Superseded`/`NotPurged`/`Mismatch`), CLI exit 3. `audit export --format parquet` streams the 24 export columns (was 501). No migration; no re-import path (open decision), object storage not wired (mount it).
|
||||
- **Hash-chain tamper evidence (T1, ScadaBridge#41) — default OFF (`AuditLog:HashChain:Enabled`).** One SHA-256 chain per UTC calendar month: `ChainSeq` + `RowHash = SHA-256(prev || canonical(row))` on `AuditLog` (migration `AddAuditLogHashChain`), linked by the central `AuditLogHashChainService` (both nodes, serialised by `sp_getapplock`) — never by the ingest path. `AuditLogChainHead` catches tail deletion and carries the partition-switch checkpoint anchor; the per-channel DELETE writes an `AuditLogChainTombstone` per chained row in the same transaction. `audit verify-chain --month` = management command `VerifyAuditChain` (OperationalAudit, unscoped callers only), first break with partition/ChainSeq/EventId, CLI exit 3. A chained row's `SourceNode` is frozen (backfill skips it). Unsigned; signing/external anchoring are open.
|
||||
- Node-of-origin is captured alongside site-of-origin: `SourceNode` (`varchar(64)` NULL) on `AuditLog`, `Notifications`, and `SiteCalls` — `node-a`/`node-b` for site rows (qualified by `SourceSiteId`/`SourceSite`), `central-a`/`central-b` for central direct-write rows. Stamped at the writing node, carried verbatim through telemetry + reconciliation, and indexed via `IX_AuditLog_Node_Occurred (SourceNode, OccurredAtUtc)` on `AuditLog`.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# AuditLog clustered key benchmark and P4 / P6 follow-ups (ScadaBridge#78)
|
||||
|
||||
Date: 2026-11-09. Status: benchmark record + recommendation only; no migration written.
|
||||
Date: 2026-11-09. Status: benchmark record + recommendation. **Implemented 2026-11-10 as migration `FlipAuditLogClusteredKey` (variant B2, #111)** — script `docs/plans/sql/FlipAuditLogClusteredKey.sql` + `.rollback.sql`; the partition-switch staging table now follows the live key order (the "old binaries work on either shape" claim below is wrong for that one statement — binaries before the flip cannot switch a flipped table). Reduced re-run, SQL Server 2022 dev container, 300k rows / 631 MB: page 1 90,532 → 34 logical reads (plan: `Clustered Index Scan … ORDERED BACKWARD` under `Top`, no sort), keyset page 90,532 → 46, idempotency probe 14 → 14; `FlipAuditLogClusteredKeyMigrationTests` pins the same ratio on 20k rows (76 vs 7,420 / 274 vs 7,393).
|
||||
|
||||
## P3 - flip `dbo.AuditLog` clustered key `(EventId, OccurredAtUtc)` -> `(OccurredAtUtc, EventId)`?
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
-- FlipAuditLogClusteredKey ROLLBACK (ScadaBridge#111): the inverse rebuild back to clustered
|
||||
-- (EventId, OccurredAtUtc), UX_AuditLog_EventId_Occurred dropped, the twelve non-clustered
|
||||
-- indexes rebuilt unchanged. Same cost and window as the forward script; a no-op on a table that
|
||||
-- already has the old key. Binaries from FlipAuditLogClusteredKey on purge either shape.
|
||||
-- Run with QUOTED_IDENTIFIER ON (sqlcmd -I, or keep this header).
|
||||
SET QUOTED_IDENTIFIER ON;
|
||||
SET ANSI_NULLS ON;
|
||||
GO
|
||||
BEGIN TRANSACTION;
|
||||
|
||||
IF
|
||||
(SELECT c.name FROM sys.index_columns ic
|
||||
JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = OBJECT_ID('dbo.AuditLog') AND ic.index_id = 1 AND ic.key_ordinal = 1) = N'OccurredAtUtc'
|
||||
BEGIN
|
||||
|
||||
DECLARE @dropSql nvarchar(max) = N'';
|
||||
SELECT @dropSql = @dropSql + N'DROP INDEX ' + QUOTENAME(i.name) + N' ON dbo.AuditLog;' + CHAR(10)
|
||||
FROM sys.indexes i
|
||||
WHERE i.object_id = OBJECT_ID('dbo.AuditLog') AND i.type = 2 AND i.name IN (
|
||||
N'IX_AuditLog_OccurredAtUtc', N'IX_AuditLog_Site_Occurred', N'IX_AuditLog_CorrelationId',
|
||||
N'IX_AuditLog_Channel_Status_Occurred', N'IX_AuditLog_Target_Occurred', N'IX_AuditLog_Execution',
|
||||
N'IX_AuditLog_ParentExecution', N'IX_AuditLog_Node_Occurred', N'IX_AuditLog_Kind_Occurred',
|
||||
N'IX_AuditLog_Actor_Occurred', N'IX_AuditLog_Unchained', N'IX_AuditLog_ChainSeq',
|
||||
N'UX_AuditLog_EventId_Occurred');
|
||||
EXEC sp_executesql @dropSql;
|
||||
ALTER TABLE dbo.AuditLog DROP CONSTRAINT PK_AuditLog;
|
||||
ALTER TABLE dbo.AuditLog ADD CONSTRAINT PK_AuditLog PRIMARY KEY CLUSTERED (EventId, OccurredAtUtc)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_OccurredAtUtc ON dbo.AuditLog (OccurredAtUtc DESC) INCLUDE (Status) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Site_Occurred ON dbo.AuditLog (SourceSiteId ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_CorrelationId ON dbo.AuditLog (CorrelationId) WHERE CorrelationId IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Channel_Status_Occurred ON dbo.AuditLog (Category ASC, Status ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Target_Occurred ON dbo.AuditLog (Target ASC, OccurredAtUtc DESC) WHERE Target IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Execution ON dbo.AuditLog (ExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ParentExecution ON dbo.AuditLog (ParentExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Node_Occurred ON dbo.AuditLog (SourceNode, OccurredAtUtc) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Kind_Occurred ON dbo.AuditLog (Kind ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Actor_Occurred ON dbo.AuditLog (Actor ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained ON dbo.AuditLog (OccurredAtUtc, EventId) WHERE ChainSeq IS NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog (ChainSeq) WHERE ChainSeq IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
END
|
||||
|
||||
DELETE FROM [__EFMigrationsHistory]
|
||||
WHERE [MigrationId] = N'20261110033732_FlipAuditLogClusteredKey';
|
||||
|
||||
COMMIT;
|
||||
GO
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
-- FlipAuditLogClusteredKey (2026-11-10, ScadaBridge#111, variant B2 of
|
||||
-- docs/plans/2026-11-09-auditlog-clustered-key-benchmark.md). dbo.AuditLog clustered key
|
||||
-- (EventId, OccurredAtUtc) -> (OccurredAtUtc, EventId) + aligned unique UX_AuditLog_EventId_Occurred
|
||||
-- (EventId, OccurredAtUtc); the twelve non-clustered indexes are dropped first and rebuilt last,
|
||||
-- unchanged and partition-aligned. Re-runnable (no-op on an already-flipped table).
|
||||
--
|
||||
-- OFFLINE whole-table rebuild: size the window from the real table first
|
||||
-- SELECT SUM(used_page_count)*8/1024 AS MB FROM sys.dm_db_partition_stats WHERE object_id = OBJECT_ID('dbo.AuditLog');
|
||||
-- (local reference: 631 MB / 300k rows rebuilt in ~9 s on an idle dev container; production I/O and
|
||||
-- log growth differ — the whole table is rewritten in one transaction, so the log needs ~2x the
|
||||
-- table size free). Take a backup first. Stop BOTH central nodes (old binaries' partition-switch
|
||||
-- purge cannot switch a flipped table: their staging table hard-codes the old key — msg 4947,
|
||||
-- rolled back, nothing purged). Rollback: FlipAuditLogClusteredKey.rollback.sql (same cost).
|
||||
--
|
||||
-- Run with QUOTED_IDENTIFIER ON (sqlcmd -I, or keep this header): AuditLog has persisted computed
|
||||
-- columns and filtered indexes and SQL Server raises error 1934 otherwise.
|
||||
SET QUOTED_IDENTIFIER ON;
|
||||
SET ANSI_NULLS ON;
|
||||
GO
|
||||
BEGIN TRANSACTION;
|
||||
|
||||
IF
|
||||
(SELECT c.name FROM sys.index_columns ic
|
||||
JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = OBJECT_ID('dbo.AuditLog') AND ic.index_id = 1 AND ic.key_ordinal = 1) = N'EventId'
|
||||
BEGIN
|
||||
|
||||
DECLARE @dropSql nvarchar(max) = N'';
|
||||
SELECT @dropSql = @dropSql + N'DROP INDEX ' + QUOTENAME(i.name) + N' ON dbo.AuditLog;' + CHAR(10)
|
||||
FROM sys.indexes i
|
||||
WHERE i.object_id = OBJECT_ID('dbo.AuditLog') AND i.type = 2 AND i.name IN (
|
||||
N'IX_AuditLog_OccurredAtUtc', N'IX_AuditLog_Site_Occurred', N'IX_AuditLog_CorrelationId',
|
||||
N'IX_AuditLog_Channel_Status_Occurred', N'IX_AuditLog_Target_Occurred', N'IX_AuditLog_Execution',
|
||||
N'IX_AuditLog_ParentExecution', N'IX_AuditLog_Node_Occurred', N'IX_AuditLog_Kind_Occurred',
|
||||
N'IX_AuditLog_Actor_Occurred', N'IX_AuditLog_Unchained', N'IX_AuditLog_ChainSeq',
|
||||
N'UX_AuditLog_EventId_Occurred');
|
||||
EXEC sp_executesql @dropSql;
|
||||
ALTER TABLE dbo.AuditLog DROP CONSTRAINT PK_AuditLog;
|
||||
ALTER TABLE dbo.AuditLog ADD CONSTRAINT PK_AuditLog PRIMARY KEY CLUSTERED (OccurredAtUtc, EventId)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE UNIQUE NONCLUSTERED INDEX UX_AuditLog_EventId_Occurred ON dbo.AuditLog (EventId, OccurredAtUtc)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_OccurredAtUtc ON dbo.AuditLog (OccurredAtUtc DESC) INCLUDE (Status) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Site_Occurred ON dbo.AuditLog (SourceSiteId ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_CorrelationId ON dbo.AuditLog (CorrelationId) WHERE CorrelationId IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Channel_Status_Occurred ON dbo.AuditLog (Category ASC, Status ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Target_Occurred ON dbo.AuditLog (Target ASC, OccurredAtUtc DESC) WHERE Target IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Execution ON dbo.AuditLog (ExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ParentExecution ON dbo.AuditLog (ParentExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Node_Occurred ON dbo.AuditLog (SourceNode, OccurredAtUtc) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Kind_Occurred ON dbo.AuditLog (Kind ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Actor_Occurred ON dbo.AuditLog (Actor ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained ON dbo.AuditLog (OccurredAtUtc, EventId) WHERE ChainSeq IS NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog (ChainSeq) WHERE ChainSeq IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
END
|
||||
|
||||
INSERT INTO [__EFMigrationsHistory] ([MigrationId], [ProductVersion])
|
||||
VALUES (N'20261110033732_FlipAuditLogClusteredKey', N'10.0.13');
|
||||
|
||||
COMMIT;
|
||||
GO
|
||||
|
||||
@@ -116,7 +116,8 @@ row per lifecycle event across all channels.
|
||||
|
||||
**Indexes (first cut):**
|
||||
|
||||
- `IX_AuditLog_OccurredAtUtc` — primary time-range index for global scans.
|
||||
- Clustered `PK_AuditLog (OccurredAtUtc, EventId)` + unique `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` (#111) — the time-leading clustered key serves the Audit Log page's newest-first TOP-N and keyset paging as an ordered range; the EventId-leading unique index serves the idempotency probe.
|
||||
- `IX_AuditLog_OccurredAtUtc` — covering (`INCLUDE (Status)`) time-range index for the KPI window.
|
||||
- `IX_AuditLog_Site_Occurred (SourceSiteId, OccurredAtUtc)` — per-site filters.
|
||||
- `IX_AuditLog_Node_Occurred (SourceNode, OccurredAtUtc)` — per-node filters ("everything `central-a` did in window X", or pinning a misbehaving site node).
|
||||
- `IX_AuditLog_CorrelationId (CorrelationId)` — drilldown from a single operation.
|
||||
@@ -731,9 +732,13 @@ MS SQL for direct-write events). Unredacted secrets never persist.
|
||||
on a large or contended partition and leave an orphaned staging table for the next
|
||||
tick's CATCH branch to clean up.
|
||||
- **Partition-aligned uniqueness:** the switch no longer drops and rebuilds an index.
|
||||
`EventId` uniqueness rides the clustered `PK_AuditLog (EventId, OccurredAtUtc)`,
|
||||
which is aligned on `ps_AuditLog_Month(OccurredAtUtc)`, so `SWITCH PARTITION` has no
|
||||
non-aligned unique index to object to. The predecessor `UX_AuditLog_EventId` forced
|
||||
`EventId` uniqueness rides the clustered `PK_AuditLog (OccurredAtUtc, EventId)` and the
|
||||
unique `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` (migration
|
||||
`FlipAuditLogClusteredKey`, #111 — before it, the PK alone, keyed `(EventId, OccurredAtUtc)`),
|
||||
all aligned on `ps_AuditLog_Month(OccurredAtUtc)`, so `SWITCH PARTITION` has no
|
||||
non-aligned unique index to object to. The staging table takes the live table's clustered
|
||||
key order (read from `sys.index_columns` at purge time), so the purge works on either key
|
||||
shape; binaries older than the flip hard-code the old order and cannot switch a flipped table. The predecessor `UX_AuditLog_EventId` forced
|
||||
an offline whole-table index build inside the switch transaction — blocking every
|
||||
audit writer for its duration — and left a window in which the idempotency-supporting
|
||||
index did not exist at all. See migration `AlignAuditLogEventIdUniqueness`.
|
||||
|
||||
@@ -69,7 +69,7 @@ The configuration database stores all central system data, organized by domain a
|
||||
|
||||
- **New tables use UTC `DateTime` + the UTC value converter**, not `DateTimeOffset`. The `Notification` entity's `DateTimeOffset` timestamps are the **documented legacy exception** — on the SQLite test harness the `DateTimeOffset`→ISO-8601-string converter forced in-memory reductions in a few KPI paths (now pushed server-side where translatable). Prefer `datetime2` UTC columns for any new table.
|
||||
- **New repositories write MSSQL-only SQL with MSSQL-backed test fixtures** (`SkippableFact` gated on the live SQL Server container). The `NotificationOutboxRepository` dual SQLite/T-SQL dialect is the **documented legacy exception** (finding S8) — it predates the MSSQL-fixture convention; do not copy that pattern.
|
||||
- **AuditLog clustered key.** The `dbo.AuditLog` primary/clustered key is `(EventId, OccurredAtUtc)` — `OccurredAtUtc` is required by the monthly partition scheme (`ps_AuditLog_Month`), and `EventId` carries a separate unique index for `InsertIfNotExists` idempotency. **Tracked follow-up (P3):** benchmark whether flipping the clustered key to `(OccurredAtUtc, EventId)` improves range-scan locality once real volume exists. → #78
|
||||
- **AuditLog clustered key.** The `dbo.AuditLog` primary/clustered key is `(OccurredAtUtc, EventId)` since migration `FlipAuditLogClusteredKey` (#111; variant B2 of the #78 benchmark, `docs/plans/2026-11-09-auditlog-clustered-key-benchmark.md`) — time-leading so the Audit Log page's newest-first TOP-N and keyset cursor read an ordered range instead of the whole partitioned table (300k-row local run: 90,532 → 34 logical reads for page 1, 90,532 → 46 for a keyset page). `OccurredAtUtc` is also the partition column of `ps_AuditLog_Month`. The aligned unique `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` keeps the `InsertIfNotExists` idempotency probe a seek (14 reads, unchanged). Before the flip the key was `(EventId, OccurredAtUtc)`. Production script `docs/plans/sql/FlipAuditLogClusteredKey.sql` (+ `.rollback.sql`): an offline whole-table rebuild, run with both centrals stopped (old binaries' partition-switch staging table hard-codes the old key); DEV/QA sizing and the write window are tracked in #111.
|
||||
- **AuditLog reconciliation-pull paging (keyset, implemented).** `SiteAuditReconciliationActor` keeps a per-site `(Since, AfterId)` cursor and `GrpcPullAuditEventsClient` sends both `since_utc` and `after_id` on every `PullAuditEvents` call; the site (`ReadPendingSinceAsync`) serves rows strictly after the `(OccurredAtUtc, EventId)` pair when `after_id` is set and falls back to inclusive `>=` on the timestamp only for a first pull / empty `after_id`. Ingest stays idempotent on `EventId`, so the cursor is an exactness optimisation, not a correctness requirement. (The earlier note that central still sent a bare `sinceUtc` was stale.)
|
||||
- **KPI sampling cadence.** There is a single recorder cadence (`KpiHistory:SampleInterval`, default 60 s); per-site and per-node series are sampled in the same pass as the global ones, with no separate per-node cadence. See Component-KpiHistory.md. Benchmark record: `docs/plans/2026-11-09-auditlog-clustered-key-benchmark.md` (ScadaBridge#78).
|
||||
|
||||
@@ -98,11 +98,12 @@ The configuration database stores all central system data, organized by domain a
|
||||
- `ParentExecutionId` — `CAST(JSON_VALUE(DetailsJson,'$.parentExecutionId') AS uniqueidentifier)` PERSISTED — spawner's `ExecutionId`; null for top-level runs.
|
||||
- `IngestedAtUtc` — `CAST(SWITCHOFFSET(CAST(JSON_VALUE(DetailsJson,'$.ingestedAtUtc') AS datetimeoffset), 0) AS datetime2(7))` — central ingest timestamp; **not** persisted (SQL Server rejects PERSISTED on the non-deterministic `SWITCHOFFSET` expression).
|
||||
|
||||
*Clustered primary key:* `(EventId, OccurredAtUtc)` — composite so the key is partition-aligned, and the **sole** `EventId` uniqueness enforcement backing `InsertIfNotExistsAsync` idempotency. `EventId` is a GUID minted once at the emitting site in the same operation that stamps `OccurredAtUtc`, and neither field is ever re-stamped downstream, so a given `EventId` always arrives with the same `OccurredAtUtc` and can only map to one partition — pair-uniqueness is `EventId`-uniqueness for every row the system produces. The idempotency probe (`WHERE EventId = @id`) still seeks the clustered key's leading column, at the cost of one seek per partition rather than one seek overall.
|
||||
*Clustered primary key:* `(OccurredAtUtc, EventId)` (since `FlipAuditLogClusteredKey`, #111; previously `(EventId, OccurredAtUtc)`) — composite so the key is partition-aligned, time-leading so newest-first range reads seek. *EventId uniqueness:* `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)`, unique and partition-aligned, backing `InsertIfNotExistsAsync` idempotency (the PK enforces the same pair). `EventId` is a GUID minted once at the emitting site in the same operation that stamps `OccurredAtUtc`, and neither field is ever re-stamped downstream, so a given `EventId` always arrives with the same `OccurredAtUtc` and can only map to one partition — pair-uniqueness is `EventId`-uniqueness for every row the system produces. The idempotency probe (`WHERE EventId = @id`) seeks the unique index's leading column, at the cost of one seek per partition rather than one seek overall. The partition-switch purge reads the live clustered key order and builds its staging table to match, so it works on either shape.
|
||||
|
||||
A non-aligned single-column `UX_AuditLog_EventId` on `[PRIMARY]` used to carry that uniqueness; it was dropped by the `AlignAuditLogEventIdUniqueness` migration because a non-aligned index blocks `ALTER TABLE … SWITCH PARTITION`, forcing the retention purge to drop it, switch, and rebuild it **offline inside the switch transaction** on every run.
|
||||
|
||||
*Indexes* (all non-clustered, partition-aligned on `ps_AuditLog_Month(OccurredAtUtc)`):
|
||||
*Indexes* (all non-clustered, partition-aligned on `ps_AuditLog_Month(OccurredAtUtc)`; `FlipAuditLogClusteredKey` rebuilt all of them unchanged):
|
||||
- `UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)` UNIQUE (EventId uniqueness + idempotency probe, #111)
|
||||
- `IX_AuditLog_OccurredAtUtc` (primary time-range index for global scans)
|
||||
- `IX_AuditLog_Site_Occurred (SourceSiteId, OccurredAtUtc)` (per-site filters)
|
||||
- `IX_AuditLog_CorrelationId (CorrelationId) WHERE CorrelationId IS NOT NULL` (drilldown from a single operation)
|
||||
|
||||
@@ -118,11 +118,14 @@ public interface IAuditLogRepository
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Partition-aligned uniqueness — no index drop.</b> EventId uniqueness is
|
||||
/// enforced by the clustered <c>PK_AuditLog (EventId, OccurredAtUtc)</c>,
|
||||
/// which is aligned on <c>ps_AuditLog_Month(OccurredAtUtc)</c>, so
|
||||
/// enforced by the clustered <c>PK_AuditLog (OccurredAtUtc, EventId)</c> and the
|
||||
/// unique <c>UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)</c> (ScadaBridge#111;
|
||||
/// before that migration the PK alone, keyed <c>(EventId, OccurredAtUtc)</c>), all
|
||||
/// aligned on <c>ps_AuditLog_Month(OccurredAtUtc)</c>, so
|
||||
/// <c>ALTER TABLE … SWITCH PARTITION</c> has no non-aligned unique index to
|
||||
/// object to. The implementation creates a staging table with byte-identical
|
||||
/// schema, switches the partition's data into staging, and drops staging
|
||||
/// schema and the same clustered key order as the live table (read at run time, so
|
||||
/// either key shape purges), switches the partition's data into staging, and drops staging
|
||||
/// (discarding the rows). Nothing is dropped from the live table, so there is
|
||||
/// no window in which the idempotency enforcement is absent and no offline
|
||||
/// index rebuild inside the switch transaction.
|
||||
|
||||
+22
-12
@@ -182,18 +182,28 @@ public class AuditLogEntityTypeConfiguration : IEntityTypeConfiguration<AuditLog
|
||||
|
||||
// ── Keys + indexes ───────────────────────────────────────────────────
|
||||
|
||||
// Composite PK includes OccurredAtUtc — required by the monthly partition scheme
|
||||
// (ps_AuditLog_Month) so the clustered key is partition-aligned. It is ALSO the
|
||||
// only uniqueness enforcement the ingest path needs: EventId is a GUID minted
|
||||
// once at the emitting site and never re-stamped, so a given EventId always
|
||||
// arrives with the same OccurredAtUtc and can only ever land in one partition.
|
||||
// Uniqueness of the pair is therefore uniqueness of EventId in practice, and the
|
||||
// idempotency probe (WHERE EventId = @id) still seeks the clustered key's leading
|
||||
// column. The predecessor non-aligned UX_AuditLog_EventId on [PRIMARY] was
|
||||
// dropped by AlignAuditLogEventIdUniqueness — it existed only to give
|
||||
// single-column uniqueness and its non-alignment forced an offline drop/rebuild
|
||||
// around every partition-switch purge.
|
||||
builder.HasKey(e => new { e.EventId, e.OccurredAtUtc });
|
||||
// Clustered PK (OccurredAtUtc, EventId) — FlipAuditLogClusteredKey (ScadaBridge#111,
|
||||
// variant B2 of the 2026-11-09 benchmark). Time-leading so the partitioned table can
|
||||
// serve the Audit Log page's ORDER BY OccurredAtUtc DESC, EventId DESC TOP-N and its
|
||||
// keyset cursor as an ordered backward range seek instead of reading the whole table
|
||||
// (739k reads -> ~50 at 3M rows). OccurredAtUtc is also the partitioning column of
|
||||
// ps_AuditLog_Month, so the key stays partition-aligned. Before the flip the key was
|
||||
// (EventId, OccurredAtUtc); the pair - and therefore the uniqueness it enforces - is
|
||||
// unchanged, only the column order (and so the physical row order) moved.
|
||||
//
|
||||
// EF does not manage the physical clustering of dbo.AuditLog (raw-SQL migrations own
|
||||
// it); the key order here only keeps the model, snapshot and migration in agreement.
|
||||
builder.HasKey(e => new { e.OccurredAtUtc, e.EventId });
|
||||
|
||||
// EventId-leading unique index (B2): keeps the ingest idempotency probe
|
||||
// (WHERE EventId = @id) a seek - one shallow seek per partition, as before the flip -
|
||||
// and keeps a real uniqueness constraint led by EventId. Partition-aligned (raw SQL
|
||||
// ON ps_AuditLog_Month) so SWITCH PARTITION has nothing to object to. EventId is a GUID
|
||||
// minted once at the emitting site together with OccurredAtUtc and never re-stamped,
|
||||
// so pair-uniqueness is EventId-uniqueness in practice (see AlignAuditLogEventIdUniqueness).
|
||||
builder.HasIndex(e => new { e.EventId, e.OccurredAtUtc })
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UX_AuditLog_EventId_Occurred");
|
||||
|
||||
// Index names are locked for reconciliation/migration discoverability. The
|
||||
// column SETS migrate to the canonical/computed shape (alog.md §4 semantics
|
||||
|
||||
+2385
File diff suppressed because it is too large
Load Diff
+130
@@ -0,0 +1,130 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
{
|
||||
/// <summary>
|
||||
/// Flips <c>dbo.AuditLog</c>'s clustered key from <c>(EventId, OccurredAtUtc)</c> to
|
||||
/// <c>(OccurredAtUtc, EventId)</c> and adds the partition-aligned unique index
|
||||
/// <c>UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc)</c> — variant B2 of the
|
||||
/// 2026-11-09 clustered-key benchmark (ScadaBridge#78, #111;
|
||||
/// <c>docs/plans/2026-11-09-auditlog-clustered-key-benchmark.md</c>).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// <b>Why.</b> With random-GUID EventIds leading the clustered key, the Audit Log page's
|
||||
/// default query (<c>ORDER BY OccurredAtUtc DESC, EventId DESC</c>, newest 50) and its
|
||||
/// keyset cursor could not be served as an ordered seek on the partitioned table and read
|
||||
/// the whole table (739k logical reads at 3M rows); with the time-leading key the same
|
||||
/// shapes read ~50 pages. The EventId-leading unique index keeps the ingest idempotency
|
||||
/// probe (<c>WHERE EventId = @id</c>) a seek and keeps a uniqueness constraint led by
|
||||
/// EventId. Ingest cost is unchanged (benchmark: same rows/s, same fragmentation).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Shape of the rebuild.</b> Every non-clustered index carries the clustered key as its
|
||||
/// row locator, so they are dropped first and recreated last (each is built once, not twice).
|
||||
/// The clustered PK is dropped (the heap stays on <c>ps_AuditLog_Month</c>) and re-added in
|
||||
/// the new order on the same scheme. All twelve non-clustered indexes are recreated with
|
||||
/// their exact prior definitions, partition-aligned. This is an OFFLINE whole-table data
|
||||
/// move: plan a write window and size it from the real table first (#111). Site buffers
|
||||
/// retain unforwarded audit rows while central ingest is stopped.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Grants.</b> An index-only rebuild keeps every table- and column-level permission on
|
||||
/// <c>dbo.AuditLog</c> (the writer's DENY UPDATE/DELETE, the chainer's column UPDATE on
|
||||
/// <c>ChainSeq, RowHash</c>, the purger's SELECT/DELETE) — pinned by
|
||||
/// <c>FlipAuditLogClusteredKeyMigrationTests</c>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Mixed version.</b> Every query issued by old and new binaries names columns, never the
|
||||
/// key, so both read and write either shape. The ONE key-sensitive statement is the
|
||||
/// partition-switch purge's staging table, whose clustered key must match the table's: new
|
||||
/// binaries read the live key order from <c>sys.index_columns</c> and build staging to match
|
||||
/// (either shape), but OLD binaries hard-code <c>(EventId, OccurredAtUtc)</c>, so on a flipped
|
||||
/// table their monthly switch-out fails (msg 4913/4947, rolled back, nothing purged, reported
|
||||
/// as a purge failure and retried next tick). Ship this migration with the binaries.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Re-runnable and reversible.</b> Up is a no-op when the clustered key already leads with
|
||||
/// <c>OccurredAtUtc</c>; Down is the inverse rebuild (same cost) and a no-op when it already
|
||||
/// leads with <c>EventId</c>.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public partial class FlipAuditLogClusteredKey : Migration
|
||||
{
|
||||
/// <summary>
|
||||
/// The twelve non-clustered indexes, exactly as their creating migrations defined them
|
||||
/// (AddAuditLogTable / CollapseAuditLogToCanonical / AddAuditLogAndNotificationCoveringIndexes /
|
||||
/// ReviewRemediationIndexes / AddAuditLogHashChain), all on <c>ps_AuditLog_Month</c>.
|
||||
/// </summary>
|
||||
internal const string NonClusteredIndexDdl = @"
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_OccurredAtUtc ON dbo.AuditLog (OccurredAtUtc DESC) INCLUDE (Status) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Site_Occurred ON dbo.AuditLog (SourceSiteId ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_CorrelationId ON dbo.AuditLog (CorrelationId) WHERE CorrelationId IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Channel_Status_Occurred ON dbo.AuditLog (Category ASC, Status ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Target_Occurred ON dbo.AuditLog (Target ASC, OccurredAtUtc DESC) WHERE Target IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Execution ON dbo.AuditLog (ExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ParentExecution ON dbo.AuditLog (ParentExecutionId) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Node_Occurred ON dbo.AuditLog (SourceNode, OccurredAtUtc) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Kind_Occurred ON dbo.AuditLog (Kind ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Actor_Occurred ON dbo.AuditLog (Actor ASC, OccurredAtUtc DESC) ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_Unchained ON dbo.AuditLog (OccurredAtUtc, EventId) WHERE ChainSeq IS NULL ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE NONCLUSTERED INDEX IX_AuditLog_ChainSeq ON dbo.AuditLog (ChainSeq) WHERE ChainSeq IS NOT NULL ON ps_AuditLog_Month(OccurredAtUtc);";
|
||||
|
||||
/// <summary>Drops every non-clustered index on dbo.AuditLog (the twelve above plus the B2 unique index), each existence-guarded.</summary>
|
||||
internal const string DropNonClusteredIndexesSql = @"
|
||||
DECLARE @dropSql nvarchar(max) = N'';
|
||||
SELECT @dropSql = @dropSql + N'DROP INDEX ' + QUOTENAME(i.name) + N' ON dbo.AuditLog;' + CHAR(10)
|
||||
FROM sys.indexes i
|
||||
WHERE i.object_id = OBJECT_ID('dbo.AuditLog') AND i.type = 2 AND i.name IN (
|
||||
N'IX_AuditLog_OccurredAtUtc', N'IX_AuditLog_Site_Occurred', N'IX_AuditLog_CorrelationId',
|
||||
N'IX_AuditLog_Channel_Status_Occurred', N'IX_AuditLog_Target_Occurred', N'IX_AuditLog_Execution',
|
||||
N'IX_AuditLog_ParentExecution', N'IX_AuditLog_Node_Occurred', N'IX_AuditLog_Kind_Occurred',
|
||||
N'IX_AuditLog_Actor_Occurred', N'IX_AuditLog_Unchained', N'IX_AuditLog_ChainSeq',
|
||||
N'UX_AuditLog_EventId_Occurred');
|
||||
EXEC sp_executesql @dropSql;";
|
||||
|
||||
// Leading key column of the clustered index, or NULL when the table has none.
|
||||
private const string LeadingKeyColumnSql = @"
|
||||
(SELECT c.name FROM sys.index_columns ic
|
||||
JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = OBJECT_ID('dbo.AuditLog') AND ic.index_id = 1 AND ic.key_ordinal = 1)";
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
// Raw SQL, not the scaffolded DropPrimaryKey/AddPrimaryKey: EF's AddPrimaryKey cannot
|
||||
// place the key ON ps_AuditLog_Month, and the twelve non-clustered indexes must be
|
||||
// dropped first and rebuilt last. The whole body is one guarded batch so a re-run on
|
||||
// an already-flipped table changes nothing.
|
||||
migrationBuilder.Sql($@"
|
||||
IF {LeadingKeyColumnSql} = N'EventId'
|
||||
BEGIN
|
||||
{DropNonClusteredIndexesSql}
|
||||
ALTER TABLE dbo.AuditLog DROP CONSTRAINT PK_AuditLog;
|
||||
ALTER TABLE dbo.AuditLog ADD CONSTRAINT PK_AuditLog PRIMARY KEY CLUSTERED (OccurredAtUtc, EventId)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
CREATE UNIQUE NONCLUSTERED INDEX UX_AuditLog_EventId_Occurred ON dbo.AuditLog (EventId, OccurredAtUtc)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
{NonClusteredIndexDdl}
|
||||
END");
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
// The inverse rebuild: back to (EventId, OccurredAtUtc), the B2 unique index removed
|
||||
// (the clustered PK is again the only EventId-leading structure, as before).
|
||||
migrationBuilder.Sql($@"
|
||||
IF {LeadingKeyColumnSql} = N'OccurredAtUtc'
|
||||
BEGIN
|
||||
{DropNonClusteredIndexesSql}
|
||||
ALTER TABLE dbo.AuditLog DROP CONSTRAINT PK_AuditLog;
|
||||
ALTER TABLE dbo.AuditLog ADD CONSTRAINT PK_AuditLog PRIMARY KEY CLUSTERED (EventId, OccurredAtUtc)
|
||||
ON ps_AuditLog_Month(OccurredAtUtc);
|
||||
{NonClusteredIndexDdl}
|
||||
END");
|
||||
}
|
||||
}
|
||||
}
|
||||
+8
-4
@@ -1979,12 +1979,12 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
|
||||
modelBuilder.Entity("ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Entities.AuditLogRow", b =>
|
||||
{
|
||||
b.Property<Guid>("EventId")
|
||||
.HasColumnType("uniqueidentifier");
|
||||
|
||||
b.Property<DateTime>("OccurredAtUtc")
|
||||
.HasColumnType("datetime2");
|
||||
|
||||
b.Property<Guid>("EventId")
|
||||
.HasColumnType("uniqueidentifier");
|
||||
|
||||
b.Property<string>("Action")
|
||||
.IsRequired()
|
||||
.HasMaxLength(64)
|
||||
@@ -2069,7 +2069,7 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("nvarchar(256)");
|
||||
|
||||
b.HasKey("EventId", "OccurredAtUtc");
|
||||
b.HasKey("OccurredAtUtc", "EventId");
|
||||
|
||||
b.HasIndex("ChainSeq")
|
||||
.HasDatabaseName("IX_AuditLog_ChainSeq")
|
||||
@@ -2095,6 +2095,10 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations
|
||||
.IsDescending(false, true)
|
||||
.HasDatabaseName("IX_AuditLog_Actor_Occurred");
|
||||
|
||||
b.HasIndex("EventId", "OccurredAtUtc")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UX_AuditLog_EventId_Occurred");
|
||||
|
||||
b.HasIndex("Kind", "OccurredAtUtc")
|
||||
.IsDescending(false, true)
|
||||
.HasDatabaseName("IX_AuditLog_Kind_Occurred");
|
||||
|
||||
@@ -23,7 +23,8 @@ namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
|
||||
public class AuditLogRepository : IAuditLogRepository
|
||||
{
|
||||
// SQL Server error numbers for duplicate-key violations on the
|
||||
// partition-aligned clustered PK_AuditLog (EventId, OccurredAtUtc).
|
||||
// partition-aligned clustered PK_AuditLog (OccurredAtUtc, EventId) or the unique
|
||||
// UX_AuditLog_EventId_Occurred (EventId, OccurredAtUtc) — ScadaBridge#111.
|
||||
// 2601 is a unique-index violation; 2627 is a primary-key/unique-constraint
|
||||
// violation. The IF NOT EXISTS … INSERT pattern has a check-then-act race
|
||||
// window — two sessions can both pass the EXISTS check and then both attempt
|
||||
@@ -504,8 +505,9 @@ VALUES
|
||||
// guarded and the staging table is GUID-suffixed, so a re-run is idempotent.
|
||||
//
|
||||
// ALIGNED UNIQUENESS: there is no longer an index drop/rebuild around
|
||||
// the SWITCH. EventId uniqueness is now enforced solely by the clustered
|
||||
// PK_AuditLog (EventId, OccurredAtUtc), which is partition-aligned on
|
||||
// the SWITCH. EventId uniqueness is enforced by the clustered PK_AuditLog
|
||||
// (OccurredAtUtc, EventId since FlipAuditLogClusteredKey, ScadaBridge#111) and the
|
||||
// unique UX_AuditLog_EventId_Occurred, both partition-aligned on
|
||||
// ps_AuditLog_Month(OccurredAtUtc) — so ALTER TABLE … SWITCH PARTITION has no
|
||||
// non-aligned index to object to. The former dance dropped UX_AuditLog_EventId,
|
||||
// switched, then rebuilt it OFFLINE inside the same transaction: a whole-table
|
||||
@@ -595,10 +597,21 @@ VALUES
|
||||
ParentExecutionId AS CAST(JSON_VALUE(DetailsJson,'$.parentExecutionId') AS uniqueidentifier) PERSISTED,
|
||||
IngestedAtUtc AS CAST(SWITCHOFFSET(CAST(JSON_VALUE(DetailsJson,'$.ingestedAtUtc') AS datetimeoffset), 0) AS datetime2(7)),
|
||||
ChainSeq bigint NULL,
|
||||
RowHash binary(32) NULL,
|
||||
CONSTRAINT PK_{stagingTableName} PRIMARY KEY CLUSTERED (EventId, OccurredAtUtc)
|
||||
RowHash binary(32) NULL
|
||||
) ON [PRIMARY];
|
||||
|
||||
-- 2b. The staging clustered key must match dbo.AuditLog's or SWITCH refuses
|
||||
-- (msg 4913/4947). FlipAuditLogClusteredKey (ScadaBridge#111) changed it from
|
||||
-- (EventId, OccurredAtUtc) to (OccurredAtUtc, EventId), so read the live key
|
||||
-- order instead of assuming one: this binary purges either shape, before or
|
||||
-- after that migration (and after its Down).
|
||||
IF (SELECT c.name FROM sys.index_columns ic
|
||||
JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = OBJECT_ID('dbo.AuditLog') AND ic.index_id = 1 AND ic.key_ordinal = 1) = N'OccurredAtUtc'
|
||||
ALTER TABLE dbo.[{stagingTableName}] ADD CONSTRAINT PK_{stagingTableName} PRIMARY KEY CLUSTERED (OccurredAtUtc, EventId);
|
||||
ELSE
|
||||
ALTER TABLE dbo.[{stagingTableName}] ADD CONSTRAINT PK_{stagingTableName} PRIMARY KEY CLUSTERED (EventId, OccurredAtUtc);
|
||||
|
||||
-- 3. Switch the partition out. $partition.pf_AuditLog_Month returns
|
||||
-- the partition number that contains the supplied boundary value;
|
||||
-- SWITCH PARTITION N moves that partition's pages to the staging
|
||||
|
||||
+20
-11
@@ -39,9 +39,10 @@ public class AuditLogEntityTypeConfigurationTests : IDisposable
|
||||
[Fact]
|
||||
public void Configure_MapsToAuditLogTable_WithCompositePrimaryKey()
|
||||
{
|
||||
// Composite PK {EventId, OccurredAtUtc} is required by the partitioned
|
||||
// AuditLog table — the clustered key must include the partition column
|
||||
// (OccurredAtUtc) so each row can be located in its partition.
|
||||
// Composite PK {OccurredAtUtc, EventId} (FlipAuditLogClusteredKey, ScadaBridge#111):
|
||||
// the clustered key must include the partition column (OccurredAtUtc), and leading
|
||||
// with it lets the Audit Log page's newest-first TOP-N / keyset read an ordered range
|
||||
// instead of the whole partitioned table.
|
||||
var entity = _context.Model.FindEntityType(typeof(AuditLogRow));
|
||||
|
||||
Assert.NotNull(entity);
|
||||
@@ -51,12 +52,15 @@ public class AuditLogEntityTypeConfigurationTests : IDisposable
|
||||
Assert.NotNull(pk);
|
||||
|
||||
var pkPropertyNames = pk!.Properties.Select(p => p.Name).ToArray();
|
||||
Assert.Equal(new[] { nameof(AuditLogRow.EventId), nameof(AuditLogRow.OccurredAtUtc) }, pkPropertyNames);
|
||||
Assert.Equal(new[] { nameof(AuditLogRow.OccurredAtUtc), nameof(AuditLogRow.EventId) }, pkPropertyNames);
|
||||
}
|
||||
|
||||
/// <summary>Verifies that no non-aligned EventId index is declared and uniqueness rides the aligned clustered key.</summary>
|
||||
/// <summary>
|
||||
/// Verifies that no single-column (non-aligned) EventId index is declared, and that EventId
|
||||
/// uniqueness rides the aligned unique (EventId, OccurredAtUtc) index.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void Configure_DeclaresNoNonAlignedEventIdIndex_UniquenessRidesTheAlignedClusteredKey()
|
||||
public void Configure_DeclaresNoNonAlignedEventIdIndex_UniquenessRidesTheAlignedEventIdIndex()
|
||||
{
|
||||
// AlignAuditLogEventIdUniqueness: the standalone non-aligned
|
||||
// UX_AuditLog_EventId is gone. EventId uniqueness now rides the clustered
|
||||
@@ -69,6 +73,10 @@ public class AuditLogEntityTypeConfigurationTests : IDisposable
|
||||
//
|
||||
// Re-declaring a single-column unique index here would silently reinstate
|
||||
// the SWITCH incompatibility, so this test pins its absence.
|
||||
//
|
||||
// FlipAuditLogClusteredKey (ScadaBridge#111) moved OccurredAtUtc to the front of the
|
||||
// clustered key, so the EventId-leading seek now comes from UX_AuditLog_EventId_Occurred:
|
||||
// unique, and keyed (EventId, OccurredAtUtc) so it stays partition-aligned.
|
||||
var entity = _context.Model.FindEntityType(typeof(AuditLogRow));
|
||||
Assert.NotNull(entity);
|
||||
|
||||
@@ -76,11 +84,11 @@ public class AuditLogEntityTypeConfigurationTests : IDisposable
|
||||
entity!.GetIndexes(),
|
||||
i => i.GetDatabaseName() == "UX_AuditLog_EventId");
|
||||
|
||||
var pk = entity.FindPrimaryKey();
|
||||
Assert.NotNull(pk);
|
||||
var ux = Assert.Single(entity.GetIndexes(), i => i.GetDatabaseName() == "UX_AuditLog_EventId_Occurred");
|
||||
Assert.True(ux.IsUnique);
|
||||
Assert.Equal(
|
||||
new[] { nameof(AuditLogRow.EventId), nameof(AuditLogRow.OccurredAtUtc) },
|
||||
pk!.Properties.Select(p => p.Name).ToArray());
|
||||
ux.Properties.Select(p => p.Name).ToArray());
|
||||
}
|
||||
|
||||
/// <summary>Verifies that the configuration maps the expected number of properties.</summary>
|
||||
@@ -189,8 +197,9 @@ public class AuditLogEntityTypeConfigurationTests : IDisposable
|
||||
"IX_AuditLog_Site_Occurred",
|
||||
"IX_AuditLog_Target_Occurred",
|
||||
"IX_AuditLog_Unchained", // hash chain (ScadaBridge#41)
|
||||
// UX_AuditLog_EventId is intentionally absent — dropped by
|
||||
// AlignAuditLogEventIdUniqueness; the aligned clustered PK carries it.
|
||||
"UX_AuditLog_EventId_Occurred", // FlipAuditLogClusteredKey (ScadaBridge#111)
|
||||
// UX_AuditLog_EventId (single-column, non-aligned) is intentionally absent —
|
||||
// dropped by AlignAuditLogEventIdUniqueness.
|
||||
};
|
||||
|
||||
Assert.Equal(expected, indexNames);
|
||||
|
||||
+315
@@ -0,0 +1,315 @@
|
||||
using Microsoft.Data.SqlClient;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Migrations.Operations;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Types.Audit;
|
||||
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Migrations;
|
||||
using ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Repositories;
|
||||
using Xunit.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests.Migrations;
|
||||
|
||||
/// <summary>
|
||||
/// SQL Server tests for the <c>FlipAuditLogClusteredKey</c> migration (ScadaBridge#111,
|
||||
/// variant B2 of <c>docs/plans/2026-11-09-auditlog-clustered-key-benchmark.md</c>): the new
|
||||
/// clustered key and the aligned unique EventId index, every other index and grant preserved,
|
||||
/// a faithful Down, a re-runnable Up, a partition-switch purge that works on BOTH key shapes,
|
||||
/// and the read win itself — the Audit Log page's page-1 and keyset queries read a small
|
||||
/// fraction of the pages they read on the old key.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The tests in this class migrate the fixture database down and back up, so they own their
|
||||
/// fixture (one database per test class) and every test leaves the database fully migrated.
|
||||
/// </remarks>
|
||||
public sealed class FlipAuditLogClusteredKeyMigrationTests : IClassFixture<MsSqlMigrationFixture>
|
||||
{
|
||||
private const string FlipMigration = "20261110033732_FlipAuditLogClusteredKey";
|
||||
|
||||
// The migration immediately before the flip, so "Down" reverts exactly this one.
|
||||
private static readonly string PreviousMigration = MigrationBefore(FlipMigration);
|
||||
|
||||
private static readonly string[] NonClusteredIndexes =
|
||||
[
|
||||
"IX_AuditLog_OccurredAtUtc", "IX_AuditLog_Site_Occurred", "IX_AuditLog_CorrelationId",
|
||||
"IX_AuditLog_Channel_Status_Occurred", "IX_AuditLog_Target_Occurred", "IX_AuditLog_Execution",
|
||||
"IX_AuditLog_ParentExecution", "IX_AuditLog_Node_Occurred", "IX_AuditLog_Kind_Occurred",
|
||||
"IX_AuditLog_Actor_Occurred", "IX_AuditLog_Unchained", "IX_AuditLog_ChainSeq",
|
||||
];
|
||||
|
||||
private readonly MsSqlMigrationFixture _fixture;
|
||||
private readonly ITestOutputHelper _output;
|
||||
|
||||
/// <summary>Creates the test class with its own migrated SQL Server database.</summary>
|
||||
/// <param name="fixture">The per-class migration fixture.</param>
|
||||
/// <param name="output">Test output (records the measured logical reads).</param>
|
||||
public FlipAuditLogClusteredKeyMigrationTests(MsSqlMigrationFixture fixture, ITestOutputHelper output)
|
||||
{
|
||||
_fixture = fixture;
|
||||
_output = output;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// After the full chain: clustered PK (OccurredAtUtc, EventId), the unique aligned
|
||||
/// (EventId, OccurredAtUtc) index, all twelve non-clustered indexes with unchanged keys and
|
||||
/// filters, everything on ps_AuditLog_Month, and every append-only/chain grant intact.
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task Up_FlipsTheClusteredKey_AddsTheAlignedUniqueIndex_AndKeepsEveryIndexAndGrant()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
|
||||
var indexes = await ReadIndexesAsync();
|
||||
Assert.Equal("OccurredAtUtc, EventId", indexes["PK_AuditLog"].Keys);
|
||||
Assert.True(indexes["PK_AuditLog"].Clustered);
|
||||
Assert.Equal("EventId, OccurredAtUtc", indexes["UX_AuditLog_EventId_Occurred"].Keys);
|
||||
Assert.True(indexes["UX_AuditLog_EventId_Occurred"].Unique);
|
||||
Assert.Equal(NonClusteredIndexes.Length + 2, indexes.Count);
|
||||
Assert.All(indexes.Values, i => Assert.Equal("ps_AuditLog_Month", i.DataSpace));
|
||||
AssertTwelveNonClusteredIndexes(indexes);
|
||||
Assert.Equal(ExpectedGrants, await ReadGrantsAsync());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Down restores the old key and drops the B2 index; Up re-flips; re-running Up's SQL on an
|
||||
/// already-flipped table is a no-op; and a partition-switch purge succeeds on each shape
|
||||
/// (the staging table's key follows the live table's).
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task DownUp_RoundTrips_IsReRunnable_AndThePartitionSwitchPurgesEitherShape()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
|
||||
await MigrateToAsync(PreviousMigration);
|
||||
var down = await ReadIndexesAsync();
|
||||
Assert.Equal("EventId, OccurredAtUtc", down["PK_AuditLog"].Keys);
|
||||
Assert.False(down.ContainsKey("UX_AuditLog_EventId_Occurred"));
|
||||
AssertTwelveNonClusteredIndexes(down);
|
||||
Assert.Equal(ExpectedGrants, await ReadGrantsAsync());
|
||||
|
||||
// Old shape: the purge's staging table must take the old key order.
|
||||
await AssertSwitchOutPurgesAsync(new DateTime(2027, 3, 1, 0, 0, 0, DateTimeKind.Utc));
|
||||
|
||||
await MigrateToAsync(FlipMigration);
|
||||
var up = await ReadIndexesAsync();
|
||||
Assert.Equal("OccurredAtUtc, EventId", up["PK_AuditLog"].Keys);
|
||||
|
||||
// Re-run Up's SQL against the flipped table: guarded, so nothing changes.
|
||||
await using (var conn = _fixture.OpenConnection())
|
||||
{
|
||||
foreach (var op in new FlipAuditLogClusteredKey().UpOperations.OfType<SqlOperation>())
|
||||
{
|
||||
await using var cmd = new SqlCommand(op.Sql, conn) { CommandTimeout = 300 };
|
||||
await cmd.ExecuteNonQueryAsync();
|
||||
}
|
||||
}
|
||||
|
||||
Assert.Equal(up.Select(kv => (kv.Key, kv.Value)).OrderBy(x => x.Key),
|
||||
(await ReadIndexesAsync()).Select(kv => (kv.Key, kv.Value)).OrderBy(x => x.Key));
|
||||
|
||||
// New shape: same purge, now with the time-leading staging key.
|
||||
await AssertSwitchOutPurgesAsync(new DateTime(2027, 4, 1, 0, 0, 0, DateTimeKind.Utc));
|
||||
Assert.Equal(ExpectedGrants, await ReadGrantsAsync());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The read win, measured with the session's logical-read counter around the repository's real query: page 1
|
||||
/// (newest 50, no filter) and a keyset page read far fewer AuditLog pages on the flipped
|
||||
/// key than on the old one over the same rows.
|
||||
/// </summary>
|
||||
/// <returns>A task that represents the asynchronous operation.</returns>
|
||||
[SkippableFact]
|
||||
public async Task PageOneAndKeysetPage_ReadAFractionOfThePages_OnTheFlippedKey()
|
||||
{
|
||||
Skip.IfNot(_fixture.Available, _fixture.SkipReason);
|
||||
await SeedRowsAsync(new DateTime(2027, 6, 1, 0, 0, 0, DateTimeKind.Utc), rows: 20_000);
|
||||
|
||||
var (flippedPage1, flippedKeyset) = await MeasurePageReadsAsync();
|
||||
await MigrateToAsync(PreviousMigration);
|
||||
try
|
||||
{
|
||||
var (oldPage1, oldKeyset) = await MeasurePageReadsAsync();
|
||||
_output.WriteLine(
|
||||
$"logical reads, 20k rows - page 1: {flippedPage1} flipped vs {oldPage1} old; keyset: {flippedKeyset} flipped vs {oldKeyset} old");
|
||||
Assert.True(flippedPage1 * 10 < oldPage1,
|
||||
$"page 1: {flippedPage1} logical reads on (OccurredAtUtc, EventId) vs {oldPage1} on (EventId, OccurredAtUtc)");
|
||||
Assert.True(flippedKeyset * 10 < oldKeyset,
|
||||
$"keyset: {flippedKeyset} logical reads on (OccurredAtUtc, EventId) vs {oldKeyset} on (EventId, OccurredAtUtc)");
|
||||
}
|
||||
finally
|
||||
{
|
||||
await MigrateToAsync(FlipMigration);
|
||||
}
|
||||
}
|
||||
|
||||
private static readonly string[] ExpectedGrants =
|
||||
[
|
||||
"scadabridge_audit_chainer|SELECT|GRANT|",
|
||||
"scadabridge_audit_chainer|UPDATE|GRANT|ChainSeq",
|
||||
"scadabridge_audit_chainer|UPDATE|GRANT|RowHash",
|
||||
"scadabridge_audit_purger|DELETE|GRANT|",
|
||||
"scadabridge_audit_purger|SELECT|GRANT|",
|
||||
"scadabridge_audit_writer|DELETE|DENY|",
|
||||
"scadabridge_audit_writer|INSERT|GRANT|",
|
||||
"scadabridge_audit_writer|SELECT|GRANT|",
|
||||
"scadabridge_audit_writer|UPDATE|DENY|",
|
||||
];
|
||||
|
||||
private static void AssertTwelveNonClusteredIndexes(IReadOnlyDictionary<string, IndexShape> indexes)
|
||||
{
|
||||
Assert.Equal("OccurredAtUtc DESC", indexes["IX_AuditLog_OccurredAtUtc"].Keys);
|
||||
Assert.Equal("Status", indexes["IX_AuditLog_OccurredAtUtc"].Included);
|
||||
Assert.Equal("SourceSiteId, OccurredAtUtc DESC", indexes["IX_AuditLog_Site_Occurred"].Keys);
|
||||
Assert.Equal("CorrelationId", indexes["IX_AuditLog_CorrelationId"].Keys);
|
||||
Assert.Equal("([CorrelationId] IS NOT NULL)", indexes["IX_AuditLog_CorrelationId"].Filter);
|
||||
Assert.Equal("Category, Status, OccurredAtUtc DESC", indexes["IX_AuditLog_Channel_Status_Occurred"].Keys);
|
||||
Assert.Equal("Target, OccurredAtUtc DESC", indexes["IX_AuditLog_Target_Occurred"].Keys);
|
||||
Assert.Equal("([Target] IS NOT NULL)", indexes["IX_AuditLog_Target_Occurred"].Filter);
|
||||
Assert.Equal("ExecutionId", indexes["IX_AuditLog_Execution"].Keys);
|
||||
Assert.Equal("ParentExecutionId", indexes["IX_AuditLog_ParentExecution"].Keys);
|
||||
Assert.Equal("SourceNode, OccurredAtUtc", indexes["IX_AuditLog_Node_Occurred"].Keys);
|
||||
Assert.Equal("Kind, OccurredAtUtc DESC", indexes["IX_AuditLog_Kind_Occurred"].Keys);
|
||||
Assert.Equal("Actor, OccurredAtUtc DESC", indexes["IX_AuditLog_Actor_Occurred"].Keys);
|
||||
Assert.Equal("OccurredAtUtc, EventId", indexes["IX_AuditLog_Unchained"].Keys);
|
||||
Assert.Equal("([ChainSeq] IS NULL)", indexes["IX_AuditLog_Unchained"].Filter);
|
||||
Assert.Equal("ChainSeq", indexes["IX_AuditLog_ChainSeq"].Keys);
|
||||
Assert.Equal("([ChainSeq] IS NOT NULL)", indexes["IX_AuditLog_ChainSeq"].Filter);
|
||||
Assert.All(NonClusteredIndexes, n => Assert.False(indexes[n].Clustered, n));
|
||||
}
|
||||
|
||||
private async Task AssertSwitchOutPurgesAsync(DateTime month)
|
||||
{
|
||||
await SeedRowsAsync(month, rows: 50);
|
||||
await using var ctx = CreateContext();
|
||||
var purged = await new AuditLogRepository(ctx).SwitchOutPartitionAsync(month);
|
||||
Assert.Equal(50, purged);
|
||||
await using var conn = _fixture.OpenConnection();
|
||||
await using var cmd = new SqlCommand(
|
||||
"SELECT COUNT_BIG(*) FROM dbo.AuditLog WHERE OccurredAtUtc >= @s AND OccurredAtUtc < @e;", conn);
|
||||
cmd.Parameters.AddWithValue("@s", month);
|
||||
cmd.Parameters.AddWithValue("@e", month.AddMonths(1));
|
||||
Assert.Equal(0L, (long)(await cmd.ExecuteScalarAsync())!);
|
||||
}
|
||||
|
||||
private async Task<(long Page1, long Keyset)> MeasurePageReadsAsync()
|
||||
{
|
||||
await using var conn = new SqlConnection(_fixture.ConnectionString);
|
||||
await conn.OpenAsync();
|
||||
|
||||
// EF reuses the already-open connection, so the session's cumulative logical-read
|
||||
// counter (sys.dm_exec_sessions) brackets exactly the repository's query. The probe
|
||||
// itself reads no AuditLog pages.
|
||||
await using var ctx = new ScadaBridgeDbContext(
|
||||
new DbContextOptionsBuilder<ScadaBridgeDbContext>().UseSqlServer(conn).Options);
|
||||
var repo = new AuditLogRepository(ctx);
|
||||
|
||||
var before = await SessionLogicalReadsAsync(conn);
|
||||
var page1 = await repo.QueryAsync(new AuditLogQueryFilter(), new AuditLogPaging(50));
|
||||
var page1Reads = await SessionLogicalReadsAsync(conn) - before;
|
||||
Assert.Equal(50, page1.Count);
|
||||
|
||||
var cursor = page1[^1];
|
||||
before = await SessionLogicalReadsAsync(conn);
|
||||
var page2 = await repo.QueryAsync(
|
||||
new AuditLogQueryFilter(),
|
||||
new AuditLogPaging(50, cursor.OccurredAtUtc.UtcDateTime, cursor.EventId));
|
||||
var keysetReads = await SessionLogicalReadsAsync(conn) - before;
|
||||
Assert.Equal(50, page2.Count);
|
||||
Assert.True(page2[0].OccurredAtUtc <= cursor.OccurredAtUtc);
|
||||
return (page1Reads, keysetReads);
|
||||
}
|
||||
|
||||
private static async Task<long> SessionLogicalReadsAsync(SqlConnection conn)
|
||||
{
|
||||
await using var cmd = new SqlCommand("SELECT logical_reads FROM sys.dm_exec_sessions WHERE session_id = @@SPID;", conn);
|
||||
return Convert.ToInt64(await cmd.ExecuteScalarAsync());
|
||||
}
|
||||
|
||||
private async Task SeedRowsAsync(DateTime month, int rows)
|
||||
{
|
||||
// Set-based seed with random EventIds (as minted at the sites) spread over the month;
|
||||
// DetailsJson carries kind/status so the persisted computed columns materialise.
|
||||
await using var conn = _fixture.OpenConnection();
|
||||
await using var cmd = new SqlCommand(@"
|
||||
WITH n AS (SELECT TOP (@rows) ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS i
|
||||
FROM sys.all_objects a CROSS JOIN sys.all_objects b)
|
||||
INSERT INTO dbo.AuditLog (EventId, OccurredAtUtc, Actor, Action, Outcome, Category, Target, SourceNode, CorrelationId, DetailsJson)
|
||||
SELECT NEWID(), DATEADD(SECOND, CAST(i * (2419200.0 / @rows) AS int), @month), N'flip-test', 'ApiOutbound.ApiCall', 'Success',
|
||||
'ApiOutbound', N'erp', 'central-a', NULL,
|
||||
N'{""kind"":""ApiCall"",""status"":""Delivered"",""sourceSiteId"":""flip"",""pad"":""' + REPLICATE(N'x', 400) + N'""}'
|
||||
FROM n;", conn) { CommandTimeout = 300 };
|
||||
cmd.Parameters.AddWithValue("@rows", rows);
|
||||
cmd.Parameters.Add(new SqlParameter("@month", System.Data.SqlDbType.DateTime2) { Scale = 7, Value = month });
|
||||
await cmd.ExecuteNonQueryAsync();
|
||||
}
|
||||
|
||||
private async Task MigrateToAsync(string migration)
|
||||
{
|
||||
await using var ctx = new ScadaBridgeDbContext(new DbContextOptionsBuilder<ScadaBridgeDbContext>()
|
||||
.UseSqlServer(_fixture.ConnectionString, sql => sql.CommandTimeout(600)).Options);
|
||||
await ctx.GetService<IMigrator>().MigrateAsync(migration);
|
||||
}
|
||||
|
||||
private static string MigrationBefore(string migration)
|
||||
{
|
||||
using var ctx = new ScadaBridgeDbContext(new DbContextOptionsBuilder<ScadaBridgeDbContext>()
|
||||
.UseSqlServer("Server=unused;Database=unused").Options);
|
||||
var all = ctx.Database.GetMigrations().ToList();
|
||||
return all[all.IndexOf(migration) - 1];
|
||||
}
|
||||
|
||||
private ScadaBridgeDbContext CreateContext()
|
||||
=> new(new DbContextOptionsBuilder<ScadaBridgeDbContext>().UseSqlServer(_fixture.ConnectionString).Options);
|
||||
|
||||
private async Task<Dictionary<string, IndexShape>> ReadIndexesAsync()
|
||||
{
|
||||
await using var conn = _fixture.OpenConnection();
|
||||
await using var cmd = new SqlCommand(@"
|
||||
SELECT i.name, i.type, i.is_unique, ds.name, i.filter_definition,
|
||||
STUFF((SELECT ', ' + c.name + CASE WHEN ic.is_descending_key = 1 THEN ' DESC' ELSE '' END
|
||||
FROM sys.index_columns ic JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = i.object_id AND ic.index_id = i.index_id AND ic.key_ordinal > 0
|
||||
ORDER BY ic.key_ordinal FOR XML PATH('')), 1, 2, ''),
|
||||
STUFF((SELECT ', ' + c.name
|
||||
FROM sys.index_columns ic JOIN sys.columns c ON c.object_id = ic.object_id AND c.column_id = ic.column_id
|
||||
WHERE ic.object_id = i.object_id AND ic.index_id = i.index_id AND ic.is_included_column = 1
|
||||
ORDER BY ic.index_column_id FOR XML PATH('')), 1, 2, '')
|
||||
FROM sys.indexes i JOIN sys.data_spaces ds ON ds.data_space_id = i.data_space_id
|
||||
WHERE i.object_id = OBJECT_ID('dbo.AuditLog') AND i.type IN (1, 2);", conn);
|
||||
var result = new Dictionary<string, IndexShape>(StringComparer.Ordinal);
|
||||
await using var reader = await cmd.ExecuteReaderAsync();
|
||||
while (await reader.ReadAsync())
|
||||
{
|
||||
result[reader.GetString(0)] = new IndexShape(
|
||||
Clustered: reader.GetByte(1) == 1,
|
||||
Unique: reader.GetBoolean(2),
|
||||
DataSpace: reader.GetString(3),
|
||||
Filter: reader.IsDBNull(4) ? null : reader.GetString(4),
|
||||
Keys: reader.GetString(5),
|
||||
Included: reader.IsDBNull(6) ? null : reader.GetString(6));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private async Task<string[]> ReadGrantsAsync()
|
||||
{
|
||||
await using var conn = _fixture.OpenConnection();
|
||||
await using var cmd = new SqlCommand(@"
|
||||
SELECT pr.name, p.permission_name, p.state_desc, COL_NAME(p.major_id, p.minor_id)
|
||||
FROM sys.database_permissions p JOIN sys.database_principals pr ON pr.principal_id = p.grantee_principal_id
|
||||
WHERE p.major_id = OBJECT_ID('dbo.AuditLog') AND p.class = 1;", conn);
|
||||
var list = new List<string>();
|
||||
await using var reader = await cmd.ExecuteReaderAsync();
|
||||
while (await reader.ReadAsync())
|
||||
{
|
||||
list.Add($"{reader.GetString(0)}|{reader.GetString(1)}|{reader.GetString(2)}|{(reader.IsDBNull(3) ? "" : reader.GetString(3))}");
|
||||
}
|
||||
|
||||
return list.Order(StringComparer.Ordinal).ToArray();
|
||||
}
|
||||
|
||||
private sealed record IndexShape(bool Clustered, bool Unique, string DataSpace, string? Filter, string Keys, string? Included);
|
||||
}
|
||||
Reference in New Issue
Block a user