scadalink-design/docs/plans/2026-05-20-auditlog-m1-foundation.md

# Audit Log #23 — M1 Foundation Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:subagent-driven-development to implement this plan task-by-task (bundled cadence per `feedback_subagent_cadence`).

**Goal:** Land the `AuditLog` table (monthly-partitioned) plus DB roles in MS SQL, and add the Commons types + EF repo + new `ScadaLink.AuditLog` project skeleton that every later milestone depends on. After M1 the database is ready, the new project is wired into the solution, and `dotnet build && dotnet test` are both green.

**Architecture:** New `AuditEvent` record + audit enums + writer interfaces in Commons. New EF entity configuration + EF Core migration creating `AuditLog` table aligned to `ps_AuditLog_Month` partition scheme on `OccurredAtUtc`, plus `scadalink_audit_writer` and `scadalink_audit_purger` SQL roles. New `IAuditLogRepository` with append-only surface (no Update, no row-delete). New `src/ScadaLink.AuditLog/` project skeleton + `AuditLogOptions`.

**Tech Stack:** .NET 10 / EF Core 10.0.7 / Microsoft.Data.SqlClient 6.0.2 / xUnit 2.9.3 / running `infra/mssql` container for integration tests.

**Brainstorm decisions (locked):**
- **MSSQL test harness:** integration tests hit the existing `infra/mssql` container (require `cd infra && docker compose up -d`).
- **AuditEvent shape:** one record with nullable `IngestedAtUtc` (set centrally) and nullable `ForwardState` (set site-locally).
- **Filegroup:** PRIMARY, hard-coded.
- **Indexes:** five named explicitly via `.HasDatabaseName("IX_AuditLog_…")`.

**Pre-existing reality:**
- `Entities/Audit/AuditLogEntry.cs` (config-audit, 9 cols) **coexists with** new `AuditEvent` — no rename, no removal.
- `IAuditService` (config-audit) is distinct from new `IAuditWriter` / `ICentralAuditWriter`.
- `tests/ScadaLink.IntegrationTests/` uses EF in-memory — NOT usable for partition/role tests.
- Roadmap M1-T10 (project skeleton) must run before M1-T9 (options class). **Swapped in this plan.**

---

## Bundles (cadence-aligned)

Tasks 1–11 from the roadmap are grouped into 6 bundles. Each bundle = one implementer dispatch + one combined spec+quality reviewer. The final cross-bundle reviewer runs over the whole branch.

- **Bundle A — Commons types** (roadmap T1+T2+T3+T4): enums, AuditEvent record + ForwardState enum, IAuditWriter / ICentralAuditWriter, telemetry message DTOs.
- **Bundle B — EF entity mapping** (T5): DbSet + IEntityTypeConfiguration<AuditEvent> + indexes.
- **Bundle C — Migration with partitioning + DB roles** (T6+T7 merged — one migration file).
- **Bundle D — Repository** (T8): IAuditLogRepository + EF implementation + DI registration.
- **Bundle E — AuditLog project skeleton + options** (T10 then T9): new `src/ScadaLink.AuditLog/` project + `AuditLogOptions`.
- **Bundle F — Docs paper trail** (T11): controller-direct edit; no subagent needed for a 1–3 line update.

---

## Bundle A — Commons types

### Task 1: Add audit enums to Commons

**Files:**
- Create: `src/ScadaLink.Commons/Types/Enums/AuditChannel.cs`
- Create: `src/ScadaLink.Commons/Types/Enums/AuditKind.cs`
- Create: `src/ScadaLink.Commons/Types/Enums/AuditStatus.cs`
- Create: `src/ScadaLink.Commons/Types/Enums/AuditForwardState.cs`
- Create: `tests/ScadaLink.Commons.Tests/Types/Enums/AuditEnumTests.cs`

**AuditChannel members** (4): `ApiOutbound`, `DbOutbound`, `Notification`, `ApiInbound`.

**AuditKind members** (10, per alog.md §4): `ApiCall`, `ApiCallCached`, `DbWrite`, `DbWriteCached`, `NotifySend`, `NotifyDeliver`, `InboundRequest`, `InboundAuthFailure`, `CachedSubmit`, `CachedResolve`.

**AuditStatus members** (8, per alog.md §4): `Submitted`, `Forwarded`, `Attempted`, `Delivered`, `Failed`, `Parked`, `Discarded`, `Skipped`.

**AuditForwardState members** (3): `Pending`, `Forwarded`, `Reconciled`.

**Steps:**
1. Failing tests assert each enum's exact member set via `Enum.GetValues(typeof(T)).Cast<T>().Select(x => x.ToString())` against a string-array literal.
2. Run: fail (enums don't exist).
3. Implement the four enums (no `[Flags]`).
4. Run: pass.
5. Commit: `feat(commons): add Audit{Channel,Kind,Status,ForwardState} enums for #23`.

### Task 2: Add AuditEvent record

**Files:**
- Create: `src/ScadaLink.Commons/Entities/Audit/AuditEvent.cs` — `public sealed record AuditEvent` with the 20 central columns per alog.md §4, plus nullable `AuditForwardState? ForwardState` and nullable `DateTime? IngestedAtUtc`.
- Create: `tests/ScadaLink.Commons.Tests/Entities/Audit/AuditEventTests.cs`.

Properties (in alog.md §4 order):
`Guid EventId`, `DateTime OccurredAtUtc`, `DateTime? IngestedAtUtc`, `AuditChannel Channel`, `AuditKind Kind`, `Guid? CorrelationId`, `string? SourceSiteId`, `string? SourceInstanceId`, `string? SourceScript`, `string? Actor`, `string? Target`, `AuditStatus Status`, `int? HttpStatus`, `int? DurationMs`, `string? ErrorMessage`, `string? ErrorDetail`, `string? RequestSummary`, `string? ResponseSummary`, `bool PayloadTruncated`, `string? Extra`, `AuditForwardState? ForwardState`.

**Steps:**
1. Failing test constructs an `AuditEvent`, asserts each property reads back as set, asserts `with` expression produces a new instance with one field changed.
2. Run: fail.
3. Implement record with all properties as `init`-only.
4. Run: pass.
5. Commit: `feat(commons): add AuditEvent record (#23)`.

### Task 3: Add IAuditWriter and ICentralAuditWriter

**Files:**
- Create: `src/ScadaLink.Commons/Interfaces/Services/IAuditWriter.cs`
- Create: `src/ScadaLink.Commons/Interfaces/Services/ICentralAuditWriter.cs`
- Create: `tests/ScadaLink.Commons.Tests/Interfaces/Services/AuditWriterContractTests.cs`

Both interfaces expose `Task WriteAsync(AuditEvent evt, CancellationToken ct = default)`. XML doc comments name Audit Log #23 as the owner; `IAuditWriter` is the abstraction the boundary code calls, `ICentralAuditWriter` is the central-only flavor (used by direct-write paths in M2+).

**Steps:**
1. Failing reflection test: `typeof(IAuditWriter).GetMethod("WriteAsync")` returns a method whose parameters are `(AuditEvent, CancellationToken)` and return type is `Task`. Same for `ICentralAuditWriter`.
2. Run: fail.
3. Implement both interfaces with XML docs.
4. Run: pass.
5. Commit: `feat(commons): add IAuditWriter and ICentralAuditWriter (#23)`.

### Task 4: Add audit telemetry + pull message DTOs

**Files:**
- Create: `src/ScadaLink.Commons/Messages/Integration/AuditTelemetryEnvelope.cs` — `public sealed record AuditTelemetryEnvelope(Guid EnvelopeId, string SourceSiteId, IReadOnlyList<AuditEvent> Events)`.
- Create: `src/ScadaLink.Commons/Messages/Integration/PullAuditEventsRequest.cs` — `public sealed record PullAuditEventsRequest(string SourceSiteId, DateTime SinceUtc, int BatchSize)`.
- Create: `src/ScadaLink.Commons/Messages/Integration/PullAuditEventsResponse.cs` — `public sealed record PullAuditEventsResponse(IReadOnlyList<AuditEvent> Events, bool MoreAvailable)`.
- Create: `tests/ScadaLink.Commons.Tests/Messages/Integration/AuditTelemetryMessagesTests.cs`.

**Steps:**
1. Failing test constructs envelope with 3 events and asserts immutability and enumerability.
2. Failing test constructs `PullAuditEventsRequest` + `PullAuditEventsResponse` with `MoreAvailable=true`.
3. Run: fail.
4. Implement records.
5. Run: pass.
6. Commit: `feat(commons): add audit telemetry + pull message DTOs (#23)`.

**Bundle A acceptance:** Commons project compiles. Four enum tests, AuditEvent test, two interface contract tests, two telemetry-message tests all green. No existing tests regress.

---

## Bundle B — EF entity mapping

### Task 5: Extend ScadaLinkDbContext + add IEntityTypeConfiguration<AuditEvent>

**Files:**
- Modify: `src/ScadaLink.ConfigurationDatabase/ScadaLinkDbContext.cs` — add `public DbSet<AuditEvent> AuditLogs => Set<AuditEvent>();` in the existing `// Audit` section, **directly after** the existing `AuditLogEntries` DbSet. Do not remove or modify `AuditLogEntries`.
- Create: `src/ScadaLink.ConfigurationDatabase/Configurations/AuditLogEntityTypeConfiguration.cs` — `IEntityTypeConfiguration<AuditEvent>` mapping to table `AuditLog`, columns per alog.md §4 (with max lengths), PK on `EventId`, enum columns stored as `varchar(32)` via `HasConversion<string>().HasMaxLength(32)`. **No partition function declared here** — that goes in the migration's raw SQL.

Five indexes with explicit names:
- `IX_AuditLog_OccurredAtUtc` on (`OccurredAtUtc` desc)
- `IX_AuditLog_Site_Occurred` on (`SourceSiteId`, `OccurredAtUtc` desc)
- `IX_AuditLog_CorrelationId` on (`CorrelationId`) where `CorrelationId IS NOT NULL`
- `IX_AuditLog_Channel_Status_Occurred` on (`Channel`, `Status`, `OccurredAtUtc` desc)
- `IX_AuditLog_Target_Occurred` on (`Target`, `OccurredAtUtc` desc) where `Target IS NOT NULL`

- Modify: `OnModelCreating` — apply via `modelBuilder.ApplyConfiguration(new AuditLogEntityTypeConfiguration())`.
- Create: `tests/ScadaLink.ConfigurationDatabase.Tests/Configurations/AuditLogEntityTypeConfigurationTests.cs` — use a `ModelBuilder` directly (no DbContext required) and assert:
  - mapped table name is `AuditLog`,
  - PK is `EventId`,
  - exactly 21 properties are mapped (20 + ForwardState; IngestedAtUtc is one of the 20 per spec; but ForwardState is the +1),
  - the five indexes exist with the documented names.

**Steps:**
1. Failing test: model asserts on table name + PK + property count.
2. Implement config + apply in `OnModelCreating`; add the DbSet.
3. Failing test: model asserts five named indexes.
4. Add `HasIndex(...).HasDatabaseName(...)` for each.
5. Run: pass.
6. Commit: `feat(configdb): map AuditEvent to AuditLog table with PK and five named indexes (#23)`.

**Bundle B acceptance:** ConfigurationDatabase project compiles. Mapping test passes. No existing ConfigurationDatabase.Tests regress.

---

## Bundle C — Migration with partitioning + DB roles

### Task 6+7 (merged): Create migration with partition function/scheme/table + DB roles

**Files:**
- Generate: `src/ScadaLink.ConfigurationDatabase/Migrations/<yyyyMMddHHmmss>_AddAuditLogTable.cs` via:
  ```
  dotnet ef migrations add AddAuditLogTable --project src/ScadaLink.ConfigurationDatabase \
    --startup-project src/ScadaLink.Host --output-dir Migrations
  ```
- Customize the migration's `Up()`:
  1. Raw SQL: create partition function `pf_AuditLog_Month` (RANGE RIGHT FOR VALUES with month-boundaries from `2026-01-01` through `2027-12-01` UTC), and partition scheme `ps_AuditLog_Month` ALL TO ([PRIMARY]).
  2. Drop EF's auto-generated `CREATE TABLE` and replace with raw SQL that creates `AuditLog` ON `ps_AuditLog_Month(OccurredAtUtc)`. (Or: let EF generate the table, then `ALTER TABLE … ADD CONSTRAINT … PK … ON ps_AuditLog_Month(OccurredAtUtc)` — whichever EF 10 supports cleanly.)
  3. Create the five named indexes via `migrationBuilder.CreateIndex(...)`, partition-aligned on `ps_AuditLog_Month(OccurredAtUtc)` where appropriate.
  4. Raw SQL roles, idempotent (`IF NOT EXISTS … CREATE ROLE`):
     - `scadalink_audit_writer`: GRANT INSERT ON AuditLog; GRANT SELECT ON AuditLog. (No UPDATE, no DELETE.)
     - `scadalink_audit_purger`: GRANT ALTER ON SCHEMA::dbo; GRANT SELECT ON AuditLog. (Enables ALTER PARTITION FUNCTION SWITCH and SWITCH PARTITION.)
- `Down()` drops indexes, table, scheme, function, then both roles.
- Create: `tests/ScadaLink.ConfigurationDatabase.Tests/Migrations/AddAuditLogTableMigrationTests.cs` — uses a fixture connecting to the running `infra/mssql` container via the connection string in `infra/mssql/.env` (or skips with `Skip.If` when the env var `SCADALINK_MSSQL_TEST_CONN` is unset, so CI without the container still passes).

Integration test assertions:
- `sys.partition_functions` contains `pf_AuditLog_Month`.
- `sys.partition_schemes` contains `ps_AuditLog_Month`.
- `INFORMATION_SCHEMA.TABLES` contains `AuditLog` aligned to the partition scheme.
- `sys.indexes` contains the five expected named indexes.
- `sys.database_principals` contains both roles.
- Smoke test: log in as a user mapped to `scadalink_audit_writer`, attempt `UPDATE AuditLog …`, expect `SqlException` with permission error.

**Steps:**
1. Generate the migration; let EF auto-fill the body.
2. Failing integration test: assert partition function exists.
3. Edit migration to add the partition function + scheme + table alignment.
4. Re-run: pass.
5. Failing integration test: assert five indexes exist.
6. Add named indexes to migration.
7. Failing integration test: assert both roles exist with documented grants.
8. Add roles to migration.
9. Failing integration test: smoke `UPDATE AuditLog` as writer expects permission error.
10. Verify role grants exclude UPDATE.
11. Run: pass.
12. Commit: `feat(configdb): add AuditLog migration with monthly partitioning and DB roles (#23)`.

**Notes for the implementer:**
- Use `Microsoft.Data.SqlClient` directly in the test fixture (not EF) to issue raw SQL for grant assertions.
- `Skip.If(string.IsNullOrEmpty(Environment.GetEnvironmentVariable("SCADALINK_MSSQL_TEST_CONN")), "MSSQL not available")` — keeps tests CI-safe.
- Test database name: `ScadaLinkAuditMigrationTest_<guid>` (created per fixture, dropped on dispose).

**Bundle C acceptance:** Migration applied to a fresh test DB on the `infra/mssql` container creates the partition function/scheme/table/indexes/roles. Smoke test confirms UPDATE is denied for the writer role. All migration tests pass when `SCADALINK_MSSQL_TEST_CONN` is set; skip cleanly when unset.

---

## Bundle D — Repository

### Task 8: IAuditLogRepository + EF implementation + DI

**Files:**
- Create: `src/ScadaLink.Commons/Interfaces/Repositories/IAuditLogRepository.cs` — three methods:
  - `Task InsertIfNotExistsAsync(AuditEvent evt, CancellationToken ct = default);`
  - `Task<IReadOnlyList<AuditEvent>> QueryAsync(AuditLogQueryFilter filter, AuditLogPaging paging, CancellationToken ct = default);`
  - `Task SwitchOutPartitionAsync(DateTime monthBoundary, CancellationToken ct = default);`

  Plus two small DTOs in the same file (or co-located `Types/Audit/`):
  - `AuditLogQueryFilter` record: nullable `AuditChannel?`, `AuditKind?`, `AuditStatus?`, `string? SourceSiteId`, `string? Target`, `string? Actor`, `Guid? CorrelationId`, `DateTime? FromUtc`, `DateTime? ToUtc`.
  - `AuditLogPaging` record: `int PageSize`, `Guid? AfterEventId`, `DateTime? AfterOccurredAtUtc` (keyset).

- Create: `src/ScadaLink.ConfigurationDatabase/Repositories/AuditLogRepository.cs` — implements all three methods:
  - `InsertIfNotExistsAsync` uses raw SQL `IF NOT EXISTS (SELECT 1 FROM AuditLog WHERE EventId = @id) INSERT INTO AuditLog …` via `DbContext.Database.ExecuteSqlInterpolatedAsync` (bypasses change tracker).
  - `QueryAsync` builds an `IQueryable<AuditEvent>`, applies filters, projects, paged by keyset on `(OccurredAtUtc desc, EventId desc)`.
  - `SwitchOutPartitionAsync` builds a unique staging table name, runs `CREATE TABLE … <staging>` with identical schema and ON `[PRIMARY]`, runs `ALTER TABLE AuditLog SWITCH PARTITION <n> TO <staging>`, then `DROP TABLE <staging>`. All inside a single transaction. Computes partition number from `monthBoundary` via `$partition.pf_AuditLog_Month(@boundary)`.

- Modify: `src/ScadaLink.ConfigurationDatabase/ServiceCollectionExtensions.cs` — add `services.AddScoped<IAuditLogRepository, AuditLogRepository>();` after `INotificationOutboxRepository` line.

- Create: `tests/ScadaLink.ConfigurationDatabase.Tests/Repositories/AuditLogRepositoryTests.cs` — uses the same MSSQL fixture from Bundle C (skipped when env var unset) since `InsertIfNotExistsAsync` uses raw SQL that won't run on EF in-memory.

Tests:
- Insert for fresh `EventId` writes one row.
- Calling `InsertIfNotExistsAsync` again with the same `EventId` is a no-op (no exception, row count unchanged).
- `QueryAsync` returns rows in `(OccurredAtUtc desc, EventId desc)` order honoring all filter predicates.
- `QueryAsync` with non-null `AfterEventId`/`AfterOccurredAtUtc` keysets correctly to the next page.
- `SwitchOutPartitionAsync` for an old boundary removes the rows belonging to that partition from the live table.

**Steps:**
1. Failing test: insert + duplicate insert.
2. Implement using raw SQL.
3. Failing test: query order + filters.
4. Implement.
5. Failing test: keyset paging.
6. Implement.
7. Failing test: switch-out partition.
8. Implement.
9. Run all: pass.
10. Commit: `feat(configdb): IAuditLogRepository + EF implementation, append-only with partition-switch purge (#23)`.

**Bundle D acceptance:** Repository tests green. DI smoke test from existing ConfigurationDatabase.Tests still passes.

---

## Bundle E — AuditLog project skeleton + options

### Task 10 (first): Scaffold `src/ScadaLink.AuditLog/` project

**Files:**
- Create: `src/ScadaLink.AuditLog/ScadaLink.AuditLog.csproj` — TargetFramework `net10.0` (matches solution), references `ScadaLink.Commons` + `ScadaLink.ConfigurationDatabase`.
- Create: `src/ScadaLink.AuditLog/ServiceCollectionExtensions.cs` — `public static class ServiceCollectionExtensions { public static IServiceCollection AddAuditLog(this IServiceCollection services, IConfiguration config) { … } }` registering `AuditLogOptions` (from Task 9) and forwarding to `services.AddScoped<IAuditLogRepository, AuditLogRepository>()` (already registered by ConfigurationDatabase, so this is a no-op but documents the dependency).
- Create: `tests/ScadaLink.AuditLog.Tests/ScadaLink.AuditLog.Tests.csproj` with one smoke test.
- Create: `tests/ScadaLink.AuditLog.Tests/AddAuditLogTests.cs` — smoke test: `services.AddAuditLog(config); var p = services.BuildServiceProvider(); Assert.NotNull(p.GetService<IOptions<AuditLogOptions>>());`.
- Modify: `ScadaLink.slnx` — add both projects.

**Steps:**
1. `dotnet new classlib -n ScadaLink.AuditLog -o src/ScadaLink.AuditLog --framework net10.0` (then delete the default `Class1.cs`).
2. `dotnet new xunit -n ScadaLink.AuditLog.Tests -o tests/ScadaLink.AuditLog.Tests --framework net10.0`.
3. Add `<ProjectReference>` to Commons + ConfigurationDatabase in the src csproj; add reference to ScadaLink.AuditLog in the test csproj.
4. Add both projects to `ScadaLink.slnx` (inside the existing `/src/` and `/tests/` folders).
5. Add `<PackageReference Include="Microsoft.Extensions.Configuration.Binder" />` to the src csproj (already in `Directory.Packages.props`).
6. Create stub `ServiceCollectionExtensions.AddAuditLog` (just registers options; writer impl comes in M2).
7. Commit: `feat(auditlog): scaffold ScadaLink.AuditLog project (#23)`.

### Task 9: AuditLogOptions

**Files:**
- Create: `src/ScadaLink.AuditLog/Configuration/AuditLogOptions.cs` — class with:
  - `int DefaultCapBytes` (default 8192)
  - `int ErrorCapBytes` (default 65536)
  - `List<string> HeaderRedactList` (default: `[ "Authorization", "X-Api-Key", "Cookie", "Set-Cookie" ]`)
  - `List<string> GlobalBodyRedactors` (default: empty)
  - `Dictionary<string, PerTargetRedactionOverride> PerTargetOverrides` (default empty)
  - `int RetentionDays` (default 365; range [30, 3650])
- Create: `src/ScadaLink.AuditLog/Configuration/PerTargetRedactionOverride.cs` — minimal: `int? CapBytes`, `List<string>? AdditionalBodyRedactors`.
- Create: `src/ScadaLink.AuditLog/Configuration/AuditLogOptionsValidator.cs` — `IValidateOptions<AuditLogOptions>` checking `DefaultCapBytes > 0`, `ErrorCapBytes >= DefaultCapBytes`, `RetentionDays` in `[30, 3650]`.
- Modify: `src/ScadaLink.AuditLog/ServiceCollectionExtensions.AddAuditLog` to `services.AddOptions<AuditLogOptions>().Bind(config.GetSection("AuditLog")).ValidateOnStart(); services.AddSingleton<IValidateOptions<AuditLogOptions>, AuditLogOptionsValidator>();`.
- Add: `tests/ScadaLink.AuditLog.Tests/Configuration/AuditLogOptionsTests.cs`:
  - Bind valid section → values present.
  - Bind invalid `RetentionDays = 0` → validator rejects.
  - Bind invalid `ErrorCapBytes < DefaultCapBytes` → validator rejects.

**Steps:**
1. Failing test: valid bind round-trip.
2. Implement options class.
3. Failing test: invalid `RetentionDays`.
4. Implement validator.
5. Failing test: invalid `ErrorCapBytes`.
6. Validator covers it.
7. Run: pass.
8. Commit: `feat(auditlog): add AuditLogOptions + validator (#23)`.

**Bundle E acceptance:** New `src/ScadaLink.AuditLog/` project builds. Solution still builds. Smoke + options tests green. `ScadaLink.slnx` includes both new entries.

---

## Bundle F — Docs paper trail (controller-direct)

### Task 11: Register AuditLog project in Component-Host.md and confirm README

**Files:**
- Modify: `docs/requirements/Component-Host.md` — list `ScadaLink.AuditLog` in the central role's registration set.
- Modify: `README.md` — confirm row #23 link reflects the new project (no functional change unless missing).

This is a 1–3 line edit. Per the cadence memory, controller does it directly without a subagent.

**Commit:** `docs(audit): register ScadaLink.AuditLog project in Host role (#23)`.

---

## Final cross-bundle review

After all bundles ship:

- Dispatch a final code-reviewer subagent over the whole M1 branch.
- Acceptance gate (from goal prompt step E):
  - `dotnet test ScadaLink.slnx` green (full solution).
  - All M1 roadmap acceptance criteria met; each cited by name to the proving test.
- If green, merge to main `--no-ff` with summary message (step F).
- Update M2–M8 sections of the roadmap with realities learned (step G), commit.
- Status paragraph (step H).
- Proceed to M2 (step I).