# Component: Configuration Database

## Purpose

The Configuration Database component provides the centralized data access layer for all system configuration data stored in MS SQL. It owns the database schema, Entity Framework DbContext, repository implementations, unit-of-work support, migration management, and audit logging. All central components access configuration data through this component — no other component interacts with the configuration database directly.

## Location

Central cluster only. Site clusters do not access the configuration database — they receive all configuration via artifact deployment and instance deployment through the Communication Layer, and read it from local SQLite at runtime.

## Responsibilities

- Define and own the complete database schema for the configuration MS SQL database via EF Core Fluent API mappings.
- Provide the Entity Framework Core DbContext as the single point of access to the configuration database.
- **Implement** the per-component repository interfaces defined in Commons. The interfaces and POCO entity classes live in Commons (persistence-ignorant); this component provides the EF Core implementations.
- **Implement** the `IAuditService` interface defined in Commons. Handles JSON serialization of entity state and writes audit entries within the same unit-of-work transaction as the change being audited.
- Provide unit-of-work support via EF Core's DbContext for transactional multi-entity operations.
- Manage schema migrations via EF Core Migrations with support for generating SQL scripts for manual execution in production.
- Support seed data for initial system setup.
- Manage connection pooling and connection lifecycle for the configuration database.

**Note**: This component does **not** manage the Machine Data Database. The Machine Data Database is a separate concern with different access patterns (direct ADO.NET connections from scripts via `Database.Connection()`).

---

## Database Schema

The configuration database stores all central system data, organized by domain area:

### Template & Modeling
- **Templates**: Template definitions (name, parent template reference, description, nullable `FolderId` FK to `TemplateFolders` — null means the template lives at the tree root).
- **TemplateFolders**: Hierarchical organizational folders for templates (`Id`, `Name`, nullable `ParentFolderId` self-reference, `SortOrder`). Unique index on `(ParentFolderId, Name)` enforces case-insensitive sibling uniqueness. Folders are UI-only — they have no effect on template resolution or flattening.
- **Template Attributes**: Attribute definitions per template (name, value, data type, lock flag, description, data source reference).
- **Template Alarms**: Alarm definitions per template (name, description, priority, lock flag, trigger type, trigger configuration, on-trigger script reference).
- **Native Alarm Sources** (`NativeAlarmSources`): Native alarm source bindings per template — alarms produced by the underlying data source rather than evaluated by the Site Runtime. Columns: `Name`, `Description`, `ConnectionName` (the data connection that surfaces the native alarms), `SourceReference` (the source-side address/path the native alarms are read from), and `ConditionFilter` (optional filter narrowing which native alarm conditions are subscribed). FK `TemplateId` → `Templates` with cascade delete; unique index on `(TemplateId, Name)`.
- **Template Scripts**: Script definitions per template (name, lock flag, C# source code, trigger type, trigger configuration, minimum time between runs, parameter definitions, return value definitions).
- **Template Compositions**: Feature module composition relationships (composing template, composed template, module instance name).
- **Instances**: Instance definitions (template reference, site reference, area reference, enabled/disabled state).
- **Instance Attribute Overrides**: Per-instance attribute value overrides.
- **Instance Native Alarm Source Overrides** (`InstanceNativeAlarmSourceOverrides`): Per-instance overrides for native alarm sources, keyed by the source's path-qualified canonical name. Columns: `SourceCanonicalName` (required, sized to fit composed `[ModuleInstanceName].[SourceName]` paths) and the nullable override fields `ConnectionNameOverride`, `SourceReferenceOverride`, and `ConditionFilterOverride` (a null override leaves the template value in effect). FK `InstanceId` → `Instances` with cascade delete; unique index on `(InstanceId, SourceCanonicalName)`.
- **Instance Connection Bindings**: Per-attribute data connection binding for each instance.
- **Areas**: Hierarchical area definitions per site (name, parent area reference, site reference).

### Shared Scripts
- **Shared Scripts**: System-wide reusable script definitions (name, C# source code, parameter definitions, return value definitions).
- **Shared Schemas** (M9/T32): Named, reusable JSON Schema definitions (`SharedSchema` entity — `Id`, `Name` (unique), `Schema` `nvarchar(max)`). Referenced from parameter/return definitions using `{"$ref":"lib:Name"}`; the `$ref` resolver (cycle/depth-guarded) expands them at deploy-time, design-time validation, and inbound-API runtime. CRUD is exposed via `ISharedSchemaRepository` (Commons) and the Central UI Schema Library page.

### Sites & Data Connections
- **Sites**: Site definitions (name, identifier, description, NodeAAddress, NodeBAddress, GrpcNodeAAddress, GrpcNodeBAddress).
- **Data Connections**: Data connection definitions (name, protocol type, connection details) with site assignments.

### External Systems & Database Connections
- **External System Definitions**: External system contracts (name, connection details, retry settings).
- **External System Methods**: API method definitions per external system (method name, parameter definitions, return type definitions).
- **Database Connection Definitions**: Named database connections (name, connection details, retry settings).

### Notifications
- **Notification Lists**: List definitions (name, `Type` discriminator — `Email` / `Sms` — selecting the list's notification type and type-specific targets).
- **Notification Recipients**: Recipients per list (name, email address).
- **SMTP Configuration**: Email server settings.
- **Notifications**: The durable central notification queue owned by the Notification Outbox — one row per notification, the single source of audit truth. The schema is **type-agnostic** so it records any notification type the system supports (email and SMS today; the type-agnostic shape absorbs future types without a migration): a `Type` discriminator selects the type, and a `TypeData` JSON column (`nvarchar(max)`) carries any future per-type fields without a schema change. Columns: `NotificationId` (GUID, primary key — generated at the site, used as the idempotency key), `Type`, `ListName`, `Subject`, `Body`, `TypeData`, `Status`, `RetryCount`, `LastError`, `ResolvedTargets`, `SourceSiteId`, `SourceInstanceId`, `SourceScript`, `SiteEnqueuedAt`, `CreatedAt`, `LastAttemptAt`, `NextAttemptAt`, `DeliveredAt`. `Status` is a `NotificationStatus` enum stored with values `Pending`, `Retrying`, `Delivered`, `Parked`, `Discarded` (the site-local `Forwarding` state is never persisted centrally). Indexed on `Status` and `NextAttemptAt` for efficient dispatcher polling of due rows, and on `SourceSiteId` and `CreatedAt` for KPI computation and the Central UI query page. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-NotificationOutbox.md for the full lifecycle.

### Site Calls
- **SiteCalls**: The central audit table for cached site calls — `ExternalSystem.CachedCall()` and `Database.CachedWrite()` — owned by the Site Call Audit component and a sibling of the `Notifications` table. One row per cached operation. Columns: `TrackedOperationId` (`varchar(36)`, primary key — the GUID idempotency key stored in "D"-format string, matching the site SQLite and gRPC wire format), `Channel` (`varchar(32)` — `ApiOutbound` or `DbOutbound`; the trust-boundary channel that produced the call), `Target` (`varchar(256)` — human-readable target, e.g. `ERP.GetOrder` for an external call), `SourceSite` (`varchar(64)` — the site that submitted the cached call), `SourceNode` (`varchar(64)` NULL — cluster node that emitted the call; null for reconciled rows from retired nodes), `Status` (`varchar(32)` — `AuditStatus` enum name: `Submitted`, `Forwarded`, `Attempted`, `Delivered`, `Failed`, `Parked`, `Discarded`; lifecycle is monotonic), `RetryCount` (`int`), `LastError` (`nvarchar(1024)` NULL), `HttpStatus` (`int` NULL — last HTTP status code for API calls), `CreatedAtUtc` (`datetime2`), `UpdatedAtUtc` (`datetime2`), `TerminalAtUtc` (`datetime2` NULL), `IngestedAtUtc` (`datetime2` — central ingest timestamp). The table is populated **only** by Site Call Audit telemetry and reconciliation pulls — sites are the source of truth and the row is an eventually-consistent mirror, never written by a central dispatcher. Ingestion is **insert-if-not-exists** keyed on `TrackedOperationId`, then **upsert-on-newer-status**; the lifecycle is monotonic, so at-least-once and out-of-order telemetry are harmless. Indexes: `IX_SiteCalls_Source_Created (SourceSite, CreatedAtUtc DESC)` for per-site queries, `IX_SiteCalls_Status_Updated (Status, UpdatedAtUtc DESC)` for status-filtered views, and the filtered `IX_SiteCalls_NonTerminal` (on `CreatedAtUtc`, `WHERE TerminalAtUtc IS NULL`, INCLUDE `SourceSite, SourceNode, Status`) that backs the live-queue KPI predicates. Terminal rows are removed by a daily purge job — see Scheduled Maintenance below. See Component-SiteCallAudit.md for the full lifecycle.

### Data-layer conventions & tracked follow-ups (arch-review 04)

- **New tables use UTC `DateTime` + the UTC value converter**, not `DateTimeOffset`. The `Notification` entity's `DateTimeOffset` timestamps are the **documented legacy exception** — on the SQLite test harness the `DateTimeOffset`→ISO-8601-string converter forced in-memory reductions in a few KPI paths (now pushed server-side where translatable). Prefer `datetime2` UTC columns for any new table.
- **New repositories write MSSQL-only SQL with MSSQL-backed test fixtures** (`SkippableFact` gated on the live SQL Server container). The `NotificationOutboxRepository` dual SQLite/T-SQL dialect is the **documented legacy exception** (finding S8) — it predates the MSSQL-fixture convention; do not copy that pattern.
- **AuditLog clustered key.** The `dbo.AuditLog` primary/clustered key is `(EventId, OccurredAtUtc)` — `OccurredAtUtc` is required by the monthly partition scheme (`ps_AuditLog_Month`), and `EventId` carries a separate unique index for `InsertIfNotExists` idempotency. **Tracked follow-up (P3):** benchmark whether flipping the clustered key to `(OccurredAtUtc, EventId)` improves range-scan locality once real volume exists.
- **Tracked follow-ups (P4/P6):** the per-node KPI sampling cadence and the AuditLog reconciliation-pull keyset paging (the SiteCalls reconciliation pull already uses a composite keyset cursor; `PullAuditEvents` remains inclusive-`>=`, idempotent on `EventId`) are deferred, lower-urgency hardening items.

### Audit Log
- **AuditLog**: The central, append-only audit table owned by the Audit Log component — one row per script-trust-boundary lifecycle event across all channels (outbound API calls, outbound DB writes/reads, notifications, and inbound API requests). Sibling of the `Notifications` and `SiteCalls` tables but distinct: `AuditLog` is the immutable history that observes the other subsystems, not an operational state store.

  **Schema (post-C5 canonical collapse, migration `CollapseAuditLogToCanonical`):** The table stores the 10 canonical `ZB.MOM.WW.Audit.AuditEvent` fields directly as writable columns, plus six read-only server-side persisted computed columns derived from `DetailsJson` via `JSON_VALUE … PERSISTED`.

  *Writable columns (the 10 canonical `AuditEvent` fields):*
  - `EventId` (`uniqueidentifier NOT NULL`) — idempotency key; generated at the originator.
  - `OccurredAtUtc` (`datetime2(7) NOT NULL`) — UTC timestamp when the audited action occurred at its source.
  - `Actor` (`nvarchar(256)` NULL) — authenticated actor; null/empty for system/anon actions.
  - `Action` (`varchar(64) NOT NULL`) — canonical action verb, format `"{channel}.{kind}"` (e.g. `ApiOutbound.ApiCall`).
  - `Outcome` (`varchar(16) NOT NULL`) — normalized outcome: `Success`, `Failure`, or `Denied`.
  - `Category` (`varchar(32) NOT NULL`) — canonical `Category` column; for ScadaBridge this equals the channel name (`ApiOutbound`, `DbOutbound`, `Notification`, `ApiInbound`). Mapped to the `AuditChannel` enum in EF as `Channel`.
  - `Target` (`nvarchar(256)` NULL) — target of the action: external system name, DB connection name, notification list name, or inbound method name.
  - `SourceNode` (`varchar(64)` NULL) — cluster node that emitted the event (e.g. `node-a`, `central-a`); null for reconciled rows from a retired node.
  - `CorrelationId` (`uniqueidentifier` NULL) — links related audit rows (e.g. the cached-op lifecycle): `TrackedOperationId` for cached calls, `NotificationId` for notifications.
  - `DetailsJson` (`nvarchar(max)` NULL) — JSON extension bag carrying all ScadaBridge domain fields (channel, kind, status, executionId, parentExecutionId, sourceSiteId, sourceInstanceId, sourceScript, httpStatus, durationMs, errorMessage, errorDetail, requestSummary, responseSummary, payloadTruncated, extra, ingestedAtUtc, and others).

  *Persisted computed columns (read-only; derived from `DetailsJson` by SQL Server on INSERT):*
  - `Kind` — `JSON_VALUE(DetailsJson,'$.kind')` PERSISTED — channel-specific event kind (e.g. `ApiCall`, `CachedSubmit`).
  - `Status` — `JSON_VALUE(DetailsJson,'$.status')` PERSISTED — `AuditStatus` enum name for this event (e.g. `Submitted`, `Delivered`, `Parked`).
  - `SourceSiteId` — `JSON_VALUE(DetailsJson,'$.sourceSiteId')` PERSISTED — site of origin; null for central-direct events.
  - `ExecutionId` — `CAST(JSON_VALUE(DetailsJson,'$.executionId') AS uniqueidentifier)` PERSISTED — originating script execution / inbound request id.
  - `ParentExecutionId` — `CAST(JSON_VALUE(DetailsJson,'$.parentExecutionId') AS uniqueidentifier)` PERSISTED — spawner's `ExecutionId`; null for top-level runs.
  - `IngestedAtUtc` — `CAST(SWITCHOFFSET(CAST(JSON_VALUE(DetailsJson,'$.ingestedAtUtc') AS datetimeoffset), 0) AS datetime2(7))` — central ingest timestamp; **not** persisted (SQL Server rejects PERSISTED on the non-deterministic `SWITCHOFFSET` expression).

  *Clustered primary key:* `(EventId, OccurredAtUtc)` — composite so the key is partition-aligned. `UX_AuditLog_EventId` (unique, non-aligned on `[PRIMARY]`) enforces global `EventId` uniqueness for `InsertIfNotExistsAsync` idempotency.

  *Indexes* (all non-clustered, partition-aligned on `ps_AuditLog_Month(OccurredAtUtc)` except `UX_AuditLog_EventId`):
  - `IX_AuditLog_OccurredAtUtc` (primary time-range index for global scans)
  - `IX_AuditLog_Site_Occurred (SourceSiteId, OccurredAtUtc)` (per-site filters)
  - `IX_AuditLog_CorrelationId (CorrelationId) WHERE CorrelationId IS NOT NULL` (drilldown from a single operation)
  - `IX_AuditLog_Channel_Status_Occurred (Category, Status, OccurredAtUtc)` (KPI / dashboard tiles — note: column is `Category` in SQL, `Channel` in EF)
  - `IX_AuditLog_Target_Occurred (Target, OccurredAtUtc) WHERE Target IS NOT NULL` ("what did we send to system X")
  - `IX_AuditLog_Execution (ExecutionId)` — unfiltered (SQL Server forbids filtered indexes on computed columns)
  - `IX_AuditLog_ParentExecution (ParentExecutionId)` — unfiltered
  - `IX_AuditLog_Node_Occurred (SourceNode, OccurredAtUtc)` (per-node queries)

  **Monthly partitioning** on `OccurredAtUtc` via partition function `pf_AuditLog_Month` and partition scheme `ps_AuditLog_Month`; the primary retention purge is a partition switch (per-channel retention overrides add one sanctioned bounded row DELETE on the maintenance path — see AuditLog Table Purge). The table is populated only by Audit Log writers (site telemetry, central direct-write, reconciliation pulls); central ingest is **insert-if-not-exists** keyed on `EventId`. See Component-AuditLog.md for the full lifecycle, payload-capture policy, and ingestion paths.

### Inbound API
- **API Keys**: The `ApiKeys` SQL Server table was **retired** by migration `RetireInboundApiKeyStore`. Inbound API keys now live in the shared `ZB.MOM.WW.Auth.ApiKeys` SQLite store, not the configuration MS SQL database. The `IInboundApiRepository` no longer exposes any `ApiKey` operations; key management goes through `IInboundApiKeyAdmin` (Commons, `Interfaces/Security/`). The `ApprovedApiKeyIds` column on `ApiMethods` was removed at the same time — key-to-method scoping is now managed by the auth library.
- **API Methods**: Method definitions (name, parameter definitions, return value definitions, implementation script, timeout).

### Security
- **LDAP Group Mappings**: Mappings between LDAP group names and system roles (Admin, Design, Deployment).
- **Site Scoping Rules**: Per-mapping site scope restrictions for Deployment role.

### Deployment
- **Deployment Records**: Deployment history per instance (timestamp, user, status, deployed configuration snapshot).
- **System-Wide Artifact Deployment Records**: Deployment history for shared artifacts (timestamp, user, artifact type, status).

### Audit Logging
- **Audit Log Entries**: Append-only audit trail (timestamp, user, action, entity type, entity ID, entity name, state as JSON). Stores only the after-state — change history is reconstructed by comparing consecutive entries. Entries are never modified or deleted. No retention policy — retained indefinitely. Indexed on timestamp, user, entity type, entity ID, and action for efficient filtering.

---

## Data Access Architecture

### DbContext

A single `ScadaBridgeDbContext` (or a small number of bounded DbContexts if warranted) serves as the EF Core entry point. The DbContext:

- Maps the POCO entity classes defined in Commons to the database using **Fluent API only** — no data annotations on the entity classes.
- Configures relationships, indexes, constraints, and value conversions.
- Provides `SaveChangesAsync()` as the unit-of-work commit mechanism.

Each entity's Fluent mapping lives in its own `IEntityTypeConfiguration<T>` class under `Configurations/`, and `OnModelCreating` registers them all with `modelBuilder.ApplyConfigurationsFromAssembly(...)` — so a new mapping is picked up simply by adding its configuration class to the assembly.

#### Native Alarm Source Mappings

The native alarm source feature adds two EF-mapped entities (POCOs in Commons, Fluent mappings here), each exposed as a `DbSet` on `ScadaBridgeDbContext`:

- **`TemplateNativeAlarmSource`** → table `NativeAlarmSources` (`DbSet<TemplateNativeAlarmSource> TemplateNativeAlarmSources`). `Name` required (≤200), `Description` (≤2000), `ConnectionName` required (≤200), `SourceReference` required (≤1000), `ConditionFilter` (≤1000). Unique index `(TemplateId, Name)`. Owned by `Template` via `TemplateConfiguration` (`HasMany(t => t.NativeAlarmSources)` on FK `TemplateId`, `OnDelete: Cascade`) — deleting a template removes its native alarm sources.
- **`InstanceNativeAlarmSourceOverride`** → table `InstanceNativeAlarmSourceOverrides` (`DbSet<InstanceNativeAlarmSourceOverride> InstanceNativeAlarmSourceOverrides`). `SourceCanonicalName` required (≤400, wider than plain names so it can hold composed paths), `ConnectionNameOverride` (≤200), `SourceReferenceOverride` (≤1000), `ConditionFilterOverride` (≤1000). Unique index `(InstanceId, SourceCanonicalName)`. Owned by `Instance` via `InstanceConfiguration` (`HasMany(i => i.NativeAlarmSourceOverrides)` on FK `InstanceId`, `OnDelete: Cascade`).

Both mappings follow the same shape as the existing alarm definitions and alarm overrides: dedicated configuration classes that auto-register through `ApplyConfigurationsFromAssembly`, cascade-delete from their owning aggregate root, and a composite unique index that enforces name uniqueness within the owner.

### Per-Component Repository Implementations

Repository interfaces are defined in **Commons** alongside the POCO entity classes (see Component-Commons.md, REQ-COM-4). This component provides the **EF Core implementations** of those interfaces.

| Repository Interface (in Commons) | Consuming Component | Scope |
|---|---|---|
| `ITemplateEngineRepository` | Template Engine | Templates, attributes, alarms, native alarm sources, scripts, compositions, instances, overrides (including native alarm source overrides), connection bindings, areas |
| `IDeploymentManagerRepository` | Deployment Manager | Current deployment status per instance, deployed configuration snapshots, system-wide artifact deployment status per site (no deployment history — audit log provides historical traceability) |
| `ISecurityRepository` | Security & Auth | LDAP group mappings, site scoping rules |
| `IInboundApiRepository` | Inbound API | API keys, API method definitions |
| `IExternalSystemRepository` | External System Gateway | External system definitions, method definitions, database connection definitions |
| `INotificationRepository` | Notification Service | Notification lists (including the `Type` field), recipients, SMTP configuration |
| `INotificationOutboxRepository` | Notification Outbox | The `Notifications` table — insert-if-not-exists ingest, due-row polling, status transitions, KPI aggregate queries, and bulk delete of terminal rows used by the daily purge job |
| `ISiteCallAuditRepository` | Site Call Audit | The `SiteCalls` table — insert-if-not-exists ingest, upsert-on-newer-status, KPI aggregate queries, and bulk delete of terminal rows used by the daily purge job |
| `IHealthMonitoringRepository` | Health Monitoring | (Minimal — health data is in-memory; repository needed only if connectivity history is persisted in the future) |
| `ICentralUiRepository` | Central UI | Read-oriented queries spanning multiple domain areas for display purposes |
| `ISharedSchemaRepository` | Template Engine, Inbound API, Central UI | `SharedSchema` CRUD (M9/T32) — named, reusable JSON Schema definitions referenced via `{"$ref":"lib:Name"}` in parameter/return definitions |

Each implementation class uses the DbContext internally and works with the POCO entity classes from Commons. Consuming components depend only on Commons (for interfaces and entities) — they never reference this component or EF Core directly. The DI container in the Host wires the implementations to the interfaces.

#### Native Alarm Source Repository Methods

`ITemplateEngineRepository` (implemented by `TemplateEngineRepository`) gains CRUD for both native alarm source entities, mirroring the existing alarm-override methods one-for-one:

- **Template side**: `GetTemplateNativeAlarmSourceByIdAsync`, `GetNativeAlarmSourcesByTemplateIdAsync`, `AddTemplateNativeAlarmSourceAsync`, `UpdateTemplateNativeAlarmSourceAsync`, `DeleteTemplateNativeAlarmSourceAsync`.
- **Instance side**: `GetNativeAlarmSourceOverridesByInstanceIdAsync`, `GetNativeAlarmSourceOverrideAsync(instanceId, sourceCanonicalName)`, `AddInstanceNativeAlarmSourceOverrideAsync`, `UpdateInstanceNativeAlarmSourceOverrideAsync`, `DeleteInstanceNativeAlarmSourceOverrideAsync`.

The aggregate loaders are extended to eager-load the new children so a template or instance is returned fully populated: `GetTemplateWithChildrenAsync` (and the other template loaders) `.Include(t => t.NativeAlarmSources)`, and the instance-with-children loader `.Include(i => i.NativeAlarmSourceOverrides)` alongside its existing attribute, alarm, and connection-binding includes. Consistent with every other repository method here, the `Add`/`Update`/`Delete` operations only **stage** changes on the DbContext — the caller commits them by invoking `SaveChangesAsync()` (typically together with the matching `IAuditService.LogAsync()` call in one transaction).

### Unit of Work

EF Core's DbContext naturally provides unit-of-work semantics:

- Multiple entity modifications within a single request are tracked by the DbContext.
- `SaveChangesAsync()` commits all pending changes in a single database transaction.
- If any part fails, the entire transaction rolls back.
- **Optimistic concurrency** is used on deployment status records and instance lifecycle state via EF Core `rowversion` / concurrency tokens. This prevents stale deployment status transitions (e.g., two concurrent requests both trying to update the same instance's status). Template editing remains **last-write-wins** by design — optimistic concurrency is intentionally not applied to template content.
- For operations that span multiple repository calls (e.g., creating a template with attributes, alarms, and scripts), the consuming component uses a single DbContext instance (via DI scoping) to ensure atomicity.

### Example Transactional Flow

```mermaid
%%{init: {'theme':'base', 'themeVariables': {'textColor':'#111111','lineColor':'#555555','edgeLabelBackground':'#ffffff','fontSize':'15px'}}}%%
flowchart TD
    start(["Template Engine: Create Template"])
    add1["repository.AddTemplate(template)<br/>// template is a Commons POCO"]
    add2["repository.AddAttributes(attributes)<br/>// attributes are Commons POCOs"]
    add3["repository.AddAlarms(alarms)<br/>// alarms are Commons POCOs"]
    add4["repository.AddScripts(scripts)<br/>// scripts are Commons POCOs"]
    save["repository.SaveChangesAsync()<br/>// single transaction commits all"]
    db[("Configuration DB<br/>(MS SQL)")]

    start --> add1
    add1 --> add2
    add2 --> add3
    add3 --> add4
    add4 --> save
    save -. "single transaction" .-> db

    classDef start fill:#d5e8d4,stroke:#82b366,color:#111111;
    classDef proc fill:#dae8fc,stroke:#6c8ebf,color:#111111;
    classDef dec fill:#fff2cc,stroke:#d6b656,color:#111111;
    classDef muted fill:#f5f5f5,stroke:#999999,color:#666666;
    class start start
    class add1,add2,add3,add4 proc
    class save dec
    class db muted
```

---

## Audit Logging

The Configuration Database component implements the `IAuditService` interface (defined in Commons), providing audit logging as a built-in capability of the data access layer.

### IAuditService Implementation

Components call `IAuditService` after a successful operation:

```
IAuditService.LogAsync(user, action, entityType, entityId, entityName, afterState)
```

- **`user`**: The authenticated AD user who performed the action.
- **`action`**: The type of operation (`Create`, `Update`, `Delete`, `Deploy`, `Disable`, `Enable`).
- **`entityType`**: What was changed (`Template`, `Instance`, `SharedScript`, `Alarm`, `ExternalSystem`, `DatabaseConnection`, `NotificationList`, `ApiKey`, `ApiMethod`, `Area`, `Site`, `DataConnection`, `LdapGroupMapping`).
- **`entityId`**: Unique identifier of the specific entity.
- **`entityName`**: Human-readable name of the entity.
- **`afterState`**: The entity's state after the change, which the implementation serializes as JSON. Null for deletes.

### Transactional Guarantee

Audit entries are written **synchronously** within the same database transaction as the change. The `IAuditService` implementation adds an `AuditLogEntry` to the current DbContext. When the calling component calls `SaveChangesAsync()`, both the change and the audit entry commit together. This guarantees:

- If the change succeeds, the audit entry is always recorded.
- If the change fails and rolls back, the audit entry is also rolled back.
- No audit entries are lost due to process crashes between the change and the audit write.

### Integration Example

```mermaid
%%{init: {'theme':'base', 'themeVariables': {'textColor':'#111111','lineColor':'#555555','edgeLabelBackground':'#ffffff','fontSize':'15px'}}}%%
flowchart TD
    start(["Template Engine: Update Template"])
    upd["repository.UpdateTemplate(template)"]
    audit["auditService.LogAsync(user, &quot;Update&quot;, &quot;Template&quot;,<br/>template.Id, template.Name, template)"]
    save["repository.SaveChangesAsync()"]
    note["both the change and audit entry<br/>commit together"]

    start --> upd
    upd --> audit
    audit --> save
    save -.- note

    classDef start fill:#d5e8d4,stroke:#82b366,color:#111111;
    classDef proc fill:#dae8fc,stroke:#6c8ebf,color:#111111;
    classDef dec fill:#fff2cc,stroke:#d6b656,color:#111111;
    classDef warn fill:#ffe6cc,stroke:#d79b00,color:#111111;
    classDef alt fill:#e1d5e7,stroke:#9673a6,color:#111111;
    class start start
    class upd proc
    class audit alt
    class save dec
    class note warn
```

### Audit Entry Schema

| Field | Type | Description |
|-------|------|-------------|
| **Id** | int (identity) | Surrogate primary key for the audit entry. A 32-bit `int` identity is used deliberately: it matches the key type of every other entity in the schema (uniform repository and query code), and SQL Server identity values are not consumed by failed transactions in a way that materially accelerates exhaustion. At a sustained, unrealistically high rate of 100 audit rows per second the `int` range is not exhausted for roughly 680 years; the indefinite-retention policy does not change that horizon. If a future deployment genuinely approaches the limit, the column can be widened to `bigint` via a migration without a schema redesign. |
| **Timestamp** | DateTimeOffset | When the action occurred (UTC). |
| **User** | String | Authenticated AD username. |
| **Action** | String | The type of operation. |
| **EntityType** | String | What was changed. |
| **EntityId** | String | Unique identifier of the entity. |
| **EntityName** | String | Human-readable name (for display without deserializing state). |
| **State** | nvarchar(max) | Entity state after the change, serialized as JSON. Null for deletes. |
| **BundleImportId** | uniqueidentifier NULL | Correlation column threading every per-entity audit row written during a Transport bundle import back to the parent `BundleImported` event. Indexed via `IX_AuditLogEntries_BundleImportId`. Null for all rows not written as part of a bundle import. |

### State Serialization

- Entity state is serialized as **JSON** using the standard .NET JSON serializer.
- JSON is stored in `nvarchar(max)` and is queryable via SQL Server's `JSON_VALUE` and `OPENJSON` functions.
- For deletes, the state is null. The previous state can be found by querying the most recent prior entry for the same entity.

### Granularity

- **One audit entry per save operation**. When a user edits a template and changes multiple attributes in a single save, one entry is created with the full entity state after the save.

### Reconstructing Change History

Since only the after-state is stored, change history for an entity is reconstructed by querying all entries for that entity ordered by timestamp. Comparing consecutive entries reveals what changed at each step. This is a query-time concern handled by the Central UI.

### Audited Actions

| Category | Actions |
|----------|---------|
| Templates | Create, edit, delete templates |
| Scripts | Create, edit, delete template scripts and shared scripts |
| Alarms | Create, edit, delete alarm definitions |
| Instances | Create, override values, bind connections, area assignment, disable, enable, delete |
| Deployments | Deploy to instance (who, what, which instance, success/failure) |
| System-Wide Artifact Deployments | Deploy shared scripts / external system definitions / DB connections / data connections to site(s) (who, what, which site(s), result) |
| External Systems | Create, edit, delete definitions |
| Database Connections | Create, edit, delete definitions |
| Notification Lists | Create, edit, delete lists and recipients |
| Inbound API | API key create, enable/disable, delete. API method create, edit, delete |
| Areas | Create, edit, delete area definitions |
| Sites & Data Connections | Create, edit, delete sites. Define and assign data connections to sites |
| Security/Admin | Role mapping changes, site permission changes |

### Query Capabilities

The Central UI Configuration Audit Log Viewer (distinct from the operational Audit Log page in #23) can filter by:
- **User**: Who made the change.
- **Entity type**: What kind of entity was changed.
- **Action type**: What kind of operation was performed.
- **Time range**: When the change occurred.
- **Specific entity ID/name**: Changes to a particular entity.

Results are returned in reverse chronological order (most recent first) with pagination support.

---

## Database Roles

The configuration database defines dedicated SQL Server roles for the append-only `AuditLog` table. These roles are **optional DBA hardening**: the default deployment runs with a single connection principal for both writer and maintenance paths, so append-only is enforced application-side (CI grep guard + code review — see Component-AuditLog.md). A deployment that wants database-level enforcement provisions two logins mapped to these roles:

- **`scadabridge_audit_writer`** — the role for application code that ingests audit events (the `AuditLogIngestActor`, central direct-write paths, and the Notification Outbox dispatcher). Granted `INSERT` and `SELECT` on `AuditLog` only, with `DENY UPDATE` + `DENY DELETE` so a later `db_datawriter` membership cannot silently re-enable mutation (`DENY` > `GRANT`). Audit ingest is `INSERT … WHERE NOT EXISTS` keyed on `EventId`, which this grant set fully supports.
- **`scadabridge_audit_purger`** — the maintenance role for the `AuditLogPurgeActor`. Granted `SELECT` + `ALTER ON SCHEMA::dbo` for the monthly partition-switch dance, **plus** `CREATE TABLE` (the switch-out CREATEs a staging table — `ALTER ON SCHEMA::dbo` alone does not confer the database-level `CREATE TABLE` permission) and a scoped `DELETE ON dbo.AuditLog` (the per-channel retention override is a bounded row DELETE on the maintenance path). The `CREATE TABLE` + `DELETE` grants were added by the `FixAuditPurgerRoleGrants` migration (arch-review 04, S3); the earlier claim that row-level `DELETE` is "not granted even to the purge role" was stale once per-channel retention overrides shipped.

A CI grep guard (`AuditLogAppendOnlyGuardTests`) fails the build on any `UPDATE`/`DELETE` DML targeting `dbo.AuditLog` in the data-access layer source, save the two `// AUDIT-PURGE-ALLOWED`-marked maintenance-path mutations. In the default single-principal deployment this guard (with code review) IS the enforcement; the DB roles above harden it further when a DBA opts in. See Component-AuditLog.md (Security & Tamper-Evidence) for the full contract.

---

## Migration Management

### Entity Framework Core Migrations

- Schema changes are managed via EF Core Migrations (`dotnet ef migrations add`, `dotnet ef migrations script`).
- Each migration is a versioned, incremental schema change.
- New tables are introduced as their own migration — for example, the `Notifications` table for the Notification Outbox ships as a dedicated EF Core migration that creates the table, its `Type`/`Status` value conversions, and its dispatcher and KPI indexes.
- The native alarm source tables ship in a dedicated `AddNativeAlarmSources` migration, parallel in shape to the existing `AddInstanceAlarmOverrides` migration: it creates `NativeAlarmSources` and `InstanceNativeAlarmSourceOverrides` with their columns, the `TemplateId` → `Templates` and `InstanceId` → `Instances` cascade-delete foreign keys, and the `(TemplateId, Name)` / `(InstanceId, SourceCanonicalName)` unique indexes.
- The initial `AuditLog` migration creates the monthly partition function `pf_AuditLog_Month` and partition scheme `ps_AuditLog_Month`, then creates the `AuditLog` table aligned to that scheme on `OccurredAtUtc`, along with the indexes listed under Database Schema. The migration also creates the `scadabridge_audit_writer` and `scadabridge_audit_purger` DB roles with the grants described in Database Roles. The ongoing **partition-maintenance job** that rolls the scheme forward each month (creating the next month's partition ahead of time) and switches out expired partitions is owned by the **Audit Log component** (`AuditLogPurgeActor` and its monthly roll-forward step), not by the Configuration Database component — this component is responsible only for the initial schema, roles, and any EF migrations against the table going forward.

### Development Environment
- Migrations are **auto-applied** at application startup using `dbContext.Database.MigrateAsync()`.
- This allows rapid iteration without manual SQL execution.

### Production Environment
- Migrations are **never auto-applied**.
- SQL scripts are generated via `dotnet ef migrations script --idempotent` and reviewed by a DBA or engineer.
- Scripts are executed manually in SQL Server Management Studio (SSMS) or equivalent tooling.
- The Host startup in production validates that the database schema version matches the expected migration level and fails fast with a clear error if not.

### Migration Script Generation

```bash
# Generate idempotent SQL script for all pending migrations
dotnet ef migrations script --idempotent --output migration.sql --project <ConfigDbProject>

# Generate script from a specific migration to another
dotnet ef migrations script FromMigration ToMigration --output migration.sql
```

Generated scripts are idempotent — they can be safely re-run without causing errors or duplicate changes.

---

## Seed Data

The Configuration Database supports seeding initial data required for the system to be usable after a fresh installation. Seed data is applied as part of the migration pipeline.

### Seed Data Includes
- Default system configuration values.
- Any baseline reference data required by the application.

### Mechanism
- Seed data is defined using EF Core's `HasData()` in entity configurations or in dedicated seed migrations.
- Seed data is included in the generated SQL scripts, so it is applied alongside schema changes in both development and production.

---

## Scheduled Maintenance

### Notifications Table Purge

The `Notifications` table grows one row per notification and is never trimmed by normal operation — `Discarded` rows are deliberately retained for audit. To bound table growth while preserving a strong audit trail, a **daily purge job** deletes terminal rows (`Delivered`, `Parked`, `Discarded`) older than a configurable retention window (default 365 days). Non-terminal rows (`Pending`, `Retrying`) are never purged. The purge is a bulk `DELETE` against `INotificationOutboxRepository`; it is owned and scheduled by the Notification Outbox component (see Component-NotificationOutbox.md), which supplies the retention window from `NotificationOutboxOptions`. The Configuration Database component provides only the repository operation and the table.

### SiteCalls Table Purge

The `SiteCalls` table grows one row per cached site call and is never trimmed by normal operation. To bound table growth while preserving a strong audit trail, a **daily purge job** deletes terminal rows (`Delivered`, `Failed`, `Discarded`) older than a configurable retention window (default 365 days). Non-terminal rows (`Pending`, `Retrying`, `Parked`) are never purged. The purge is a bulk `DELETE`; it is owned and scheduled by the Site Call Audit component (see Component-SiteCallAudit.md), which supplies the retention window. The Configuration Database component provides only the repository operation and the table.

### AuditLog Table Purge

The `AuditLog` table is append-only and grows by every script-trust-boundary event across all channels. Unlike `Notifications` and `SiteCalls`, the primary purge is a **monthly partition switch** against the `ps_AuditLog_Month` scheme, not a row-level `DELETE`. A daily job switches out any partition whose latest `OccurredAtUtc` is older than the global retention window (default 365 days, configurable via `AuditLog:RetentionDays`) and drops the resulting staging table. On top of the global window, **per-channel retention overrides** (`AuditLog:PerChannelRetentionDays`, a dictionary keyed by canonical channel name with windows strictly shorter than the global one) expire a single channel's rows earlier via a bounded, batched `DELETE TOP` on the maintenance path — the one sanctioned row-level DELETE against `AuditLog` (see Component-AuditLog.md). The job is owned and scheduled by the Audit Log component (`AuditLogPurgeActor` — see Component-AuditLog.md), which is also the consumer of both options. The Configuration Database component contributes only the table, the partition function/scheme, the indexes, and the optional DB roles.

---

## Connection Management

- Connection strings are provided via the Host's `DatabaseConfiguration` options (bound from `appsettings.json`).
- EF Core manages connection pooling via the underlying ADO.NET SQL Server provider.
- The DbContext is registered as a **scoped** service in the DI container, ensuring each request/operation gets its own instance.
- No connection management for the Machine Data Database — that is handled separately by consumers (Inbound API scripts, external system gateway).

---

## Dependencies

- **Entity Framework Core**: ORM, DbContext, migrations, change tracking.
- **Microsoft.EntityFrameworkCore.SqlServer**: SQL Server database provider.
- **MS SQL Server**: The configuration database instance.
- **Commons**: POCO entity classes and repository interfaces that this component maps and implements.

## Interactions

- **Template Engine**: Uses `ITemplateEngineRepository` for all template, instance, and area data operations.
- **Deployment Manager**: Uses `IDeploymentManagerRepository` for deployment records and status tracking.
- **Security & Auth**: Uses `ISecurityRepository` for LDAP group mappings and site scoping.
- **Inbound API**: Uses `IInboundApiRepository` for API keys and method definitions.
- **External System Gateway**: Uses `IExternalSystemRepository` for external system and database connection definitions.
- **Notification Service**: Uses `INotificationRepository` for notification lists and SMTP configuration.
- **Notification Outbox**: Uses `INotificationOutboxRepository` for all access to the `Notifications` table — ingest, dispatch polling, status updates, KPI queries, and the daily purge of terminal rows.
- **Site Call Audit**: Uses `ISiteCallAuditRepository` for all access to the `SiteCalls` table — telemetry/reconciliation ingest, KPI queries, and the daily purge of terminal rows.
- **Central UI**: Uses `ICentralUiRepository` for read-oriented queries across domain areas, including config-audit queries for the Configuration Audit Log Viewer (the operational Audit Log page is owned by #23).
- **All central components that modify state**: Call `IAuditService.LogAsync()` after successful operations to record audit entries within the same transaction.
- **Host**: Provides database connection configuration. Registers DbContext, repository implementations, and `IAuditService` implementation in the DI container. Triggers auto-migration in development or validates schema version in production.
