dohertj2/scadalink-design
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(configdb): add AuditLog migration with monthly partitioning and DB roles (#23)

Browse Source 
Bundle C of the #23 M1 foundation. Creates the centralized AuditLog table
with the partition function, partition scheme, partition-aligned
non-clustered indexes, and the two access-control roles documented in
alog.md §4.

Schema:
- pf_AuditLog_Month: RANGE RIGHT, 24 monthly boundaries (Jan 2026 – Dec 2027).
- ps_AuditLog_Month: ALL TO ([PRIMARY]) — dev/test parity.
- dbo.AuditLog: created via raw SQL ON ps_AuditLog_Month(OccurredAtUtc).
  Composite clustered PK {EventId, OccurredAtUtc} (partition column must be
  part of the clustered key). 22 columns matching the EF AuditEvent model.
- 5 reconciliation/query non-clustered indexes from alog.md §4
  (Channel_Status_Occurred, CorrelationId filtered, OccurredAtUtc,
  Site_Occurred, Target_Occurred filtered) — all partition-aligned.
- UX_AuditLog_EventId: non-aligned UNIQUE on EventId alone (preserves
  InsertIfNotExistsAsync idempotency from M1-T8). Non-aligned because
  partition-aligned unique indexes require the partition column in the key,
  which would weaken to composite uniqueness; the purge story (M2/M3)
  rebuilds this index around partition switches.

Access control:
- scadalink_audit_writer: GRANT INSERT + GRANT SELECT, DENY UPDATE + DENY DELETE
  on AuditLog. The explicit DENY guarantees later db_datawriter membership
  cannot quietly re-enable mutation.
- scadalink_audit_purger: GRANT SELECT on AuditLog, GRANT ALTER on SCHEMA::dbo
  (enables ALTER PARTITION FUNCTION SWITCH and SWITCH PARTITION).

Both role definitions are idempotent (IF DATABASE_PRINCIPAL_ID IS NULL).
Down() drops in reverse dependency order with IF EXISTS guards.

Integration tests (tests/ScadaLink.ConfigurationDatabase.Tests/Migrations/):
- MsSqlMigrationFixture: connects to the running infra/mssql container (or
  the SCADALINK_MSSQL_TEST_CONN override), creates a unique per-fixture
  database, applies the migrations, drops the DB on dispose. Marks itself
  Available=false when MSSQL is unreachable so tests early-return cleanly
  on CI without the dev container.
- AddAuditLogTableMigrationTests: 8 tests covering table existence,
  partition function/scheme, partition-aligned PK, the 5 named indexes,
  both roles' grants, and a smoke test that a writer-role user receives
  SqlException with "permission" on UPDATE AuditLog.

ConfigurationDatabase tests: 142 passing -> 150 passing (8 new integration
tests). Full solution builds clean.

Package: tests project locally overrides Microsoft.Data.SqlClient to 6.1.1
(EF SqlServer 10.0.7 needs >= 6.1.1; central package version is pinned at
6.0.2 for the production ExternalSystemGateway).
This commit is contained in:
Joseph Doherty
2026-05-20 10:25:25 -04:00
parent 7d9550f779
commit d9c99242a3
6 changed files with 2357 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1553
src/ScadaLink.ConfigurationDatabase/Migrations/20260520142214_AddAuditLogTable.Designer.cs generated Normal file
View File

File diff suppressed because it is too large Load Diff

201
src/ScadaLink.ConfigurationDatabase/Migrations/20260520142214_AddAuditLogTable.cs Normal file
View File

@@ -0,0 +1,201 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ScadaLink.ConfigurationDatabase.Migrations
{
/// <summary>
/// Bundle C (#23 M1): creates the centralized AuditLog table with monthly
/// partitioning and the two access-control roles documented in alog.md §4.
///
/// Structure:
/// 1. Partition function <c>pf_AuditLog_Month</c> (RANGE RIGHT) with 24
/// monthly boundaries covering 2026-01-01 through 2027-12-01 UTC.
/// 2. Partition scheme <c>ps_AuditLog_Month</c> mapping every partition to
/// [PRIMARY] (dev/test parity; production may relocate via filegroups).
/// 3. <c>AuditLog</c> table created via raw SQL so it is created directly
/// on the partition scheme. The clustered PK is composite
/// {EventId, OccurredAtUtc} — required because partition-aligned PKs
/// must include the partition column.
/// 4. Five reconciliation/query indexes from alog.md §4, plus the
/// UX_AuditLog_EventId unique index that preserves single-column
/// EventId uniqueness for InsertIfNotExistsAsync (M1-T8). All
/// non-clustered indexes are partition-aligned on
/// <c>ps_AuditLog_Month(OccurredAtUtc)</c>.
/// 5. Two database roles:
/// - <c>scadalink_audit_writer</c>: INSERT + SELECT on AuditLog, with
/// explicit DENY on UPDATE and DELETE so additive role membership
/// (e.g. later db_datawriter) cannot accidentally re-enable mutation.
/// - <c>scadalink_audit_purger</c>: SELECT on AuditLog and ALTER on
/// SCHEMA::dbo so the purger can run ALTER PARTITION FUNCTION SWITCH
/// and SWITCH PARTITION when sliding the retention window.
/// </summary>
public partial class AddAuditLogTable : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// 1) Partition function (monthly boundaries Jan 2026 – Dec 2027 UTC).
// RANGE RIGHT — the boundary value belongs to the right-hand partition,
// matching the convention used by SQL Server partition-switch tooling.
migrationBuilder.Sql(@"
CREATE PARTITION FUNCTION pf_AuditLog_Month (datetime2(7))
AS RANGE RIGHT FOR VALUES (
'2026-01-01T00:00:00', '2026-02-01T00:00:00', '2026-03-01T00:00:00', '2026-04-01T00:00:00',
'2026-05-01T00:00:00', '2026-06-01T00:00:00', '2026-07-01T00:00:00', '2026-08-01T00:00:00',
'2026-09-01T00:00:00', '2026-10-01T00:00:00', '2026-11-01T00:00:00', '2026-12-01T00:00:00',
'2027-01-01T00:00:00', '2027-02-01T00:00:00', '2027-03-01T00:00:00', '2027-04-01T00:00:00',
'2027-05-01T00:00:00', '2027-06-01T00:00:00', '2027-07-01T00:00:00', '2027-08-01T00:00:00',
'2027-09-01T00:00:00', '2027-10-01T00:00:00', '2027-11-01T00:00:00', '2027-12-01T00:00:00'
);");
// 2) Partition scheme mapping every partition to [PRIMARY].
migrationBuilder.Sql(@"
CREATE PARTITION SCHEME ps_AuditLog_Month
AS PARTITION pf_AuditLog_Month ALL TO ([PRIMARY]);");
// 3) Create the table directly on the partition scheme. Column shapes
// are copied from AuditLogEntityTypeConfiguration so the live schema
// matches the EF model exactly. The clustered PK is composite to
// satisfy SQL Server's rule that partition-aligned clustered indexes
// must include the partition column.
migrationBuilder.Sql(@"
CREATE TABLE dbo.AuditLog (
EventId uniqueidentifier NOT NULL,
OccurredAtUtc datetime2(7) NOT NULL,
IngestedAtUtc datetime2(7) NULL,
Channel varchar(32) NOT NULL,
Kind varchar(32) NOT NULL,
CorrelationId uniqueidentifier NULL,
SourceSiteId varchar(64) NULL,
SourceInstanceId varchar(128) NULL,
SourceScript varchar(128) NULL,
Actor varchar(128) NULL,
Target varchar(256) NULL,
Status varchar(32) NOT NULL,
HttpStatus int NULL,
DurationMs int NULL,
ErrorMessage nvarchar(1024) NULL,
ErrorDetail nvarchar(max) NULL,
RequestSummary nvarchar(max) NULL,
ResponseSummary nvarchar(max) NULL,
PayloadTruncated bit NOT NULL,
Extra nvarchar(max) NULL,
ForwardState varchar(32) NULL,
CONSTRAINT PK_AuditLog PRIMARY KEY CLUSTERED (EventId, OccurredAtUtc)
ON ps_AuditLog_Month(OccurredAtUtc)
) ON ps_AuditLog_Month(OccurredAtUtc);");
// 4) Reconciliation/query indexes from alog.md §4. All non-clustered
// indexes are partition-aligned on ps_AuditLog_Month(OccurredAtUtc)
// so partition-switch operations only touch a single partition. The
// filtered indexes carry their NOT NULL predicates as documented.
migrationBuilder.Sql(@"
CREATE NONCLUSTERED INDEX IX_AuditLog_OccurredAtUtc
ON dbo.AuditLog (OccurredAtUtc DESC)
ON ps_AuditLog_Month(OccurredAtUtc);");
migrationBuilder.Sql(@"
CREATE NONCLUSTERED INDEX IX_AuditLog_Site_Occurred
ON dbo.AuditLog (SourceSiteId ASC, OccurredAtUtc DESC)
ON ps_AuditLog_Month(OccurredAtUtc);");
migrationBuilder.Sql(@"
CREATE NONCLUSTERED INDEX IX_AuditLog_CorrelationId
ON dbo.AuditLog (CorrelationId)
WHERE CorrelationId IS NOT NULL
ON ps_AuditLog_Month(OccurredAtUtc);");
migrationBuilder.Sql(@"
CREATE NONCLUSTERED INDEX IX_AuditLog_Channel_Status_Occurred
ON dbo.AuditLog (Channel ASC, Status ASC, OccurredAtUtc DESC)
ON ps_AuditLog_Month(OccurredAtUtc);");
migrationBuilder.Sql(@"
CREATE NONCLUSTERED INDEX IX_AuditLog_Target_Occurred
ON dbo.AuditLog (Target ASC, OccurredAtUtc DESC)
WHERE Target IS NOT NULL
ON ps_AuditLog_Month(OccurredAtUtc);");
// The EventId uniqueness index supports InsertIfNotExistsAsync
// (M1-T8). It is INTENTIONALLY non-aligned (placed on [PRIMARY]
// rather than ps_AuditLog_Month).
//
// SQL Server's rule for unique partition-aligned indexes is that the
// partition column must be a SUBSET of the index key. Including
// OccurredAtUtc in the key would change the uniqueness semantics
// from "EventId is globally unique" to "(EventId, OccurredAtUtc)
// is unique", which is the same guarantee the composite PK already
// provides — it would not give us single-column EventId uniqueness.
//
// Trade-off: a non-aligned index disables ALTER TABLE … SWITCH
// PARTITION on AuditLog. The M1 purge story (M2/M3) uses an
// explicit rebuild path that drops and re-creates this index
// around the switch, so the aligned-indexes pattern is preserved
// for partition switching at purge time.
migrationBuilder.Sql(@"
CREATE UNIQUE NONCLUSTERED INDEX UX_AuditLog_EventId
ON dbo.AuditLog (EventId)
ON [PRIMARY];");
// 5) DB roles. Both definitions are idempotent so the migration is
// safe to re-apply against a database that already has the role.
// The DENY UPDATE / DENY DELETE on the writer role is deliberate —
// a future db_datawriter membership cannot quietly re-enable
// mutation because DENY outranks GRANT.
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadalink_audit_writer') IS NULL
EXEC sp_executesql N'CREATE ROLE scadalink_audit_writer';
GRANT INSERT ON dbo.AuditLog TO scadalink_audit_writer;
GRANT SELECT ON dbo.AuditLog TO scadalink_audit_writer;
DENY UPDATE ON dbo.AuditLog TO scadalink_audit_writer;
DENY DELETE ON dbo.AuditLog TO scadalink_audit_writer;");
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadalink_audit_purger') IS NULL
EXEC sp_executesql N'CREATE ROLE scadalink_audit_purger';
GRANT SELECT ON dbo.AuditLog TO scadalink_audit_purger;
GRANT ALTER ON SCHEMA::dbo TO scadalink_audit_purger;");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
// Drop in reverse dependency order so each statement's prerequisites
// still exist when it runs. Each DROP is guarded so a partial Up()
// (or a re-applied Down()) cannot fail on missing objects.
migrationBuilder.Sql(@"
IF DATABASE_PRINCIPAL_ID('scadalink_audit_purger') IS NOT NULL
EXEC sp_executesql N'DROP ROLE scadalink_audit_purger';
IF DATABASE_PRINCIPAL_ID('scadalink_audit_writer') IS NOT NULL
EXEC sp_executesql N'DROP ROLE scadalink_audit_writer';");
// Indexes are dropped implicitly when the table goes away, but
// dropping them explicitly first keeps the Down() statement self-
// describing and mirrors the Up() shape.
migrationBuilder.Sql(@"
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'UX_AuditLog_EventId' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX UX_AuditLog_EventId ON dbo.AuditLog;
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Target_Occurred' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX IX_AuditLog_Target_Occurred ON dbo.AuditLog;
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Channel_Status_Occurred' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX IX_AuditLog_Channel_Status_Occurred ON dbo.AuditLog;
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_CorrelationId' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX IX_AuditLog_CorrelationId ON dbo.AuditLog;
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_Site_Occurred' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX IX_AuditLog_Site_Occurred ON dbo.AuditLog;
IF EXISTS (SELECT 1 FROM sys.indexes WHERE name = 'IX_AuditLog_OccurredAtUtc' AND object_id = OBJECT_ID('dbo.AuditLog'))
DROP INDEX IX_AuditLog_OccurredAtUtc ON dbo.AuditLog;");
migrationBuilder.Sql(@"
IF OBJECT_ID('dbo.AuditLog', 'U') IS NOT NULL
DROP TABLE dbo.AuditLog;");
migrationBuilder.Sql(@"
IF EXISTS (SELECT 1 FROM sys.partition_schemes WHERE name = 'ps_AuditLog_Month')
DROP PARTITION SCHEME ps_AuditLog_Month;
IF EXISTS (SELECT 1 FROM sys.partition_functions WHERE name = 'pf_AuditLog_Month')
DROP PARTITION FUNCTION pf_AuditLog_Month;");
}
}
}

117
src/ScadaLink.ConfigurationDatabase/Migrations/ScadaLinkDbContextModelSnapshot.cs
View File

@@ -41,6 +41,123 @@ namespace ScadaLink.ConfigurationDatabase.Migrations
b.ToTable("DataProtectionKeys");
});
modelBuilder.Entity("ScadaLink.Commons.Entities.Audit.AuditEvent", b =>
{
b.Property<Guid>("EventId")
.HasColumnType("uniqueidentifier");
b.Property<DateTime>("OccurredAtUtc")
.HasColumnType("datetime2");
b.Property<string>("Actor")
.HasMaxLength(128)
.IsUnicode(false)
.HasColumnType("varchar(128)");
b.Property<string>("Channel")
.IsRequired()
.HasMaxLength(32)
.IsUnicode(false)
.HasColumnType("varchar(32)");
b.Property<Guid?>("CorrelationId")
.HasColumnType("uniqueidentifier");
b.Property<int?>("DurationMs")
.HasColumnType("int");
b.Property<string>("ErrorDetail")
.HasColumnType("nvarchar(max)");
b.Property<string>("ErrorMessage")
.HasMaxLength(1024)
.HasColumnType("nvarchar(1024)");
b.Property<string>("Extra")
.HasColumnType("nvarchar(max)");
b.Property<string>("ForwardState")
.HasMaxLength(32)
.IsUnicode(false)
.HasColumnType("varchar(32)");
b.Property<int?>("HttpStatus")
.HasColumnType("int");
b.Property<DateTime?>("IngestedAtUtc")
.HasColumnType("datetime2");
b.Property<string>("Kind")
.IsRequired()
.HasMaxLength(32)
.IsUnicode(false)
.HasColumnType("varchar(32)");
b.Property<bool>("PayloadTruncated")
.HasColumnType("bit");
b.Property<string>("RequestSummary")
.HasColumnType("nvarchar(max)");
b.Property<string>("ResponseSummary")
.HasColumnType("nvarchar(max)");
b.Property<string>("SourceInstanceId")
.HasMaxLength(128)
.IsUnicode(false)
.HasColumnType("varchar(128)");
b.Property<string>("SourceScript")
.HasMaxLength(128)
.IsUnicode(false)
.HasColumnType("varchar(128)");
b.Property<string>("SourceSiteId")
.HasMaxLength(64)
.IsUnicode(false)
.HasColumnType("varchar(64)");
b.Property<string>("Status")
.IsRequired()
.HasMaxLength(32)
.IsUnicode(false)
.HasColumnType("varchar(32)");
b.Property<string>("Target")
.HasMaxLength(256)
.IsUnicode(false)
.HasColumnType("varchar(256)");
b.HasKey("EventId", "OccurredAtUtc");
b.HasIndex("CorrelationId")
.HasDatabaseName("IX_AuditLog_CorrelationId")
.HasFilter("[CorrelationId] IS NOT NULL");
b.HasIndex("EventId")
.IsUnique()
.HasDatabaseName("UX_AuditLog_EventId");
b.HasIndex("OccurredAtUtc")
.IsDescending()
.HasDatabaseName("IX_AuditLog_OccurredAtUtc");
b.HasIndex("SourceSiteId", "OccurredAtUtc")
.IsDescending(false, true)
.HasDatabaseName("IX_AuditLog_Site_Occurred");
b.HasIndex("Target", "OccurredAtUtc")
.IsDescending(false, true)
.HasDatabaseName("IX_AuditLog_Target_Occurred")
.HasFilter("[Target] IS NOT NULL");
b.HasIndex("Channel", "Status", "OccurredAtUtc")
.IsDescending(false, false, true)
.HasDatabaseName("IX_AuditLog_Channel_Status_Occurred");
b.ToTable("AuditLog", (string)null);
});
modelBuilder.Entity("ScadaLink.Commons.Entities.Audit.AuditLogEntry", b =>
{
b.Property<int>("Id")