# Component: Configuration Database

## Purpose

The Configuration Database component provides the centralized data access layer for all system configuration data stored in MS SQL. It owns the database schema, Entity Framework DbContext, repository implementations, unit-of-work support, migration management, and audit logging. All central components access configuration data through this component — no other component interacts with the configuration database directly.

## Location

Central cluster only. Site clusters do not access the configuration database — they receive all configuration via artifact deployment and instance deployment through the Communication Layer, and read it from local SQLite at runtime.

## Responsibilities

- Define and own the complete database schema for the configuration MS SQL database via EF Core Fluent API mappings.
- Provide the Entity Framework Core DbContext as the single point of access to the configuration database.
- **Implement** the per-component repository interfaces defined in Commons. The interfaces and POCO entity classes live in Commons (persistence-ignorant); this component provides the EF Core implementations.
- **Implement** the `IAuditService` interface defined in Commons. Handles JSON serialization of entity state and writes audit entries within the same unit-of-work transaction as the change being audited.
- Provide unit-of-work support via EF Core's DbContext for transactional multi-entity operations.
- Manage schema migrations via EF Core Migrations with support for generating SQL scripts for manual execution in production.
- Support seed data for initial system setup.
- Manage connection pooling and connection lifecycle for the configuration database.

**Note**: This component does **not** manage the Machine Data Database. The Machine Data Database is a separate concern with different access patterns (direct ADO.NET connections from scripts via `Database.Connection()`).

---

## Database Schema

The configuration database stores all central system data, organized by domain area:

### Template & Modeling
- **Templates**: Template definitions (name, parent template reference, description).
- **Template Attributes**: Attribute definitions per template (name, value, data type, lock flag, description, data source reference).
- **Template Alarms**: Alarm definitions per template (name, description, priority, lock flag, trigger type, trigger configuration, on-trigger script reference).
- **Template Scripts**: Script definitions per template (name, lock flag, C# source code, trigger type, trigger configuration, minimum time between runs, parameter definitions, return value definitions).
- **Template Compositions**: Feature module composition relationships (composing template, composed template, module instance name).
- **Instances**: Instance definitions (template reference, site reference, area reference, enabled/disabled state).
- **Instance Attribute Overrides**: Per-instance attribute value overrides.
- **Instance Connection Bindings**: Per-attribute data connection binding for each instance.
- **Areas**: Hierarchical area definitions per site (name, parent area reference, site reference).

### Shared Scripts
- **Shared Scripts**: System-wide reusable script definitions (name, C# source code, parameter definitions, return value definitions).

### Sites & Data Connections
- **Sites**: Site definitions (name, identifier, description, NodeAAddress, NodeBAddress, GrpcNodeAAddress, GrpcNodeBAddress).
- **Data Connections**: Data connection definitions (name, protocol type, connection details) with site assignments.

### External Systems & Database Connections
- **External System Definitions**: External system contracts (name, connection details, retry settings).
- **External System Methods**: API method definitions per external system (method name, parameter definitions, return type definitions).
- **Database Connection Definitions**: Named database connections (name, connection details, retry settings).

### Notifications
- **Notification Lists**: List definitions (name).
- **Notification Recipients**: Recipients per list (name, email address).
- **SMTP Configuration**: Email server settings.

### Inbound API
- **API Keys**: Key definitions (name/label, key value, enabled flag).
- **API Methods**: Method definitions (name, approved key references, parameter definitions, return value definitions, implementation script, timeout).

### Security
- **LDAP Group Mappings**: Mappings between LDAP group names and system roles (Admin, Design, Deployment).
- **Site Scoping Rules**: Per-mapping site scope restrictions for Deployment role.

### Deployment
- **Deployment Records**: Deployment history per instance (timestamp, user, status, deployed configuration snapshot).
- **System-Wide Artifact Deployment Records**: Deployment history for shared artifacts (timestamp, user, artifact type, status).

### Audit Logging
- **Audit Log Entries**: Append-only audit trail (timestamp, user, action, entity type, entity ID, entity name, state as JSON). Stores only the after-state — change history is reconstructed by comparing consecutive entries. Entries are never modified or deleted. No retention policy — retained indefinitely. Indexed on timestamp, user, entity type, entity ID, and action for efficient filtering.

---

## Data Access Architecture

### DbContext

A single `ScadaLinkDbContext` (or a small number of bounded DbContexts if warranted) serves as the EF Core entry point. The DbContext:

- Maps the POCO entity classes defined in Commons to the database using **Fluent API only** — no data annotations on the entity classes.
- Configures relationships, indexes, constraints, and value conversions.
- Provides `SaveChangesAsync()` as the unit-of-work commit mechanism.

### Per-Component Repository Implementations

Repository interfaces are defined in **Commons** alongside the POCO entity classes (see Component-Commons.md, REQ-COM-4). This component provides the **EF Core implementations** of those interfaces.

| Repository Interface (in Commons) | Consuming Component | Scope |
|---|---|---|
| `ITemplateEngineRepository` | Template Engine | Templates, attributes, alarms, scripts, compositions, instances, overrides, connection bindings, areas |
| `IDeploymentManagerRepository` | Deployment Manager | Current deployment status per instance, deployed configuration snapshots, system-wide artifact deployment status per site (no deployment history — audit log provides historical traceability) |
| `ISecurityRepository` | Security & Auth | LDAP group mappings, site scoping rules |
| `IInboundApiRepository` | Inbound API | API keys, API method definitions |
| `IExternalSystemRepository` | External System Gateway | External system definitions, method definitions, database connection definitions |
| `INotificationRepository` | Notification Service | Notification lists, recipients, SMTP configuration |
| `IHealthMonitoringRepository` | Health Monitoring | (Minimal — health data is in-memory; repository needed only if connectivity history is persisted in the future) |
| `ICentralUiRepository` | Central UI | Read-oriented queries spanning multiple domain areas for display purposes |

Each implementation class uses the DbContext internally and works with the POCO entity classes from Commons. Consuming components depend only on Commons (for interfaces and entities) — they never reference this component or EF Core directly. The DI container in the Host wires the implementations to the interfaces.

### Unit of Work

EF Core's DbContext naturally provides unit-of-work semantics:

- Multiple entity modifications within a single request are tracked by the DbContext.
- `SaveChangesAsync()` commits all pending changes in a single database transaction.
- If any part fails, the entire transaction rolls back.
- **Optimistic concurrency** is used on deployment status records and instance lifecycle state via EF Core `rowversion` / concurrency tokens. This prevents stale deployment status transitions (e.g., two concurrent requests both trying to update the same instance's status). Template editing remains **last-write-wins** by design — optimistic concurrency is intentionally not applied to template content.
- For operations that span multiple repository calls (e.g., creating a template with attributes, alarms, and scripts), the consuming component uses a single DbContext instance (via DI scoping) to ensure atomicity.

### Example Transactional Flow

```
Template Engine: Create Template
    │
    ├── repository.AddTemplate(template)      // template is a Commons POCO
    ├── repository.AddAttributes(attributes)  // attributes are Commons POCOs
    ├── repository.AddAlarms(alarms)          // alarms are Commons POCOs
    ├── repository.AddScripts(scripts)        // scripts are Commons POCOs
    └── repository.SaveChangesAsync()         // single transaction commits all
```

---

## Audit Logging

The Configuration Database component implements the `IAuditService` interface (defined in Commons), providing audit logging as a built-in capability of the data access layer.

### IAuditService Implementation

Components call `IAuditService` after a successful operation:

```
IAuditService.LogAsync(user, action, entityType, entityId, entityName, afterState)
```

- **`user`**: The authenticated AD user who performed the action.
- **`action`**: The type of operation (`Create`, `Update`, `Delete`, `Deploy`, `Disable`, `Enable`).
- **`entityType`**: What was changed (`Template`, `Instance`, `SharedScript`, `Alarm`, `ExternalSystem`, `DatabaseConnection`, `NotificationList`, `ApiKey`, `ApiMethod`, `Area`, `Site`, `DataConnection`, `LdapGroupMapping`).
- **`entityId`**: Unique identifier of the specific entity.
- **`entityName`**: Human-readable name of the entity.
- **`afterState`**: The entity's state after the change, which the implementation serializes as JSON. Null for deletes.

### Transactional Guarantee

Audit entries are written **synchronously** within the same database transaction as the change. The `IAuditService` implementation adds an `AuditLogEntry` to the current DbContext. When the calling component calls `SaveChangesAsync()`, both the change and the audit entry commit together. This guarantees:

- If the change succeeds, the audit entry is always recorded.
- If the change fails and rolls back, the audit entry is also rolled back.
- No audit entries are lost due to process crashes between the change and the audit write.

### Integration Example

```
Template Engine: Update Template
    │
    ├── repository.UpdateTemplate(template)
    ├── auditService.LogAsync(user, "Update", "Template", template.Id,
    │       template.Name, template)
    └── repository.SaveChangesAsync()  ← both the change and audit entry commit together
```

### Audit Entry Schema

| Field | Type | Description |
|-------|------|-------------|
| **Id** | Long / GUID | Unique identifier for the audit entry. |
| **Timestamp** | DateTimeOffset | When the action occurred (UTC). |
| **User** | String | Authenticated AD username. |
| **Action** | String | The type of operation. |
| **EntityType** | String | What was changed. |
| **EntityId** | String | Unique identifier of the entity. |
| **EntityName** | String | Human-readable name (for display without deserializing state). |
| **State** | nvarchar(max) | Entity state after the change, serialized as JSON. Null for deletes. |

### State Serialization

- Entity state is serialized as **JSON** using the standard .NET JSON serializer.
- JSON is stored in `nvarchar(max)` and is queryable via SQL Server's `JSON_VALUE` and `OPENJSON` functions.
- For deletes, the state is null. The previous state can be found by querying the most recent prior entry for the same entity.

### Granularity

- **One audit entry per save operation**. When a user edits a template and changes multiple attributes in a single save, one entry is created with the full entity state after the save.

### Reconstructing Change History

Since only the after-state is stored, change history for an entity is reconstructed by querying all entries for that entity ordered by timestamp. Comparing consecutive entries reveals what changed at each step. This is a query-time concern handled by the Central UI.

### Audited Actions

| Category | Actions |
|----------|---------|
| Templates | Create, edit, delete templates |
| Scripts | Create, edit, delete template scripts and shared scripts |
| Alarms | Create, edit, delete alarm definitions |
| Instances | Create, override values, bind connections, area assignment, disable, enable, delete |
| Deployments | Deploy to instance (who, what, which instance, success/failure) |
| System-Wide Artifact Deployments | Deploy shared scripts / external system definitions / DB connections / data connections / notification lists / SMTP config to site(s) (who, what, which site(s), result) |
| External Systems | Create, edit, delete definitions |
| Database Connections | Create, edit, delete definitions |
| Notification Lists | Create, edit, delete lists and recipients |
| Inbound API | API key create, enable/disable, delete. API method create, edit, delete |
| Areas | Create, edit, delete area definitions |
| Sites & Data Connections | Create, edit, delete sites. Define and assign data connections to sites |
| Security/Admin | Role mapping changes, site permission changes |

### Query Capabilities

The Central UI audit log viewer can filter by:
- **User**: Who made the change.
- **Entity type**: What kind of entity was changed.
- **Action type**: What kind of operation was performed.
- **Time range**: When the change occurred.
- **Specific entity ID/name**: Changes to a particular entity.

Results are returned in reverse chronological order (most recent first) with pagination support.

---

## Migration Management

### Entity Framework Core Migrations

- Schema changes are managed via EF Core Migrations (`dotnet ef migrations add`, `dotnet ef migrations script`).
- Each migration is a versioned, incremental schema change.

### Development Environment
- Migrations are **auto-applied** at application startup using `dbContext.Database.MigrateAsync()`.
- This allows rapid iteration without manual SQL execution.

### Production Environment
- Migrations are **never auto-applied**.
- SQL scripts are generated via `dotnet ef migrations script --idempotent` and reviewed by a DBA or engineer.
- Scripts are executed manually in SQL Server Management Studio (SSMS) or equivalent tooling.
- The Host startup in production validates that the database schema version matches the expected migration level and fails fast with a clear error if not.

### Migration Script Generation

```bash
# Generate idempotent SQL script for all pending migrations
dotnet ef migrations script --idempotent --output migration.sql --project <ConfigDbProject>

# Generate script from a specific migration to another
dotnet ef migrations script FromMigration ToMigration --output migration.sql
```

Generated scripts are idempotent — they can be safely re-run without causing errors or duplicate changes.

---

## Seed Data

The Configuration Database supports seeding initial data required for the system to be usable after a fresh installation. Seed data is applied as part of the migration pipeline.

### Seed Data Includes
- Default system configuration values.
- Any baseline reference data required by the application.

### Mechanism
- Seed data is defined using EF Core's `HasData()` in entity configurations or in dedicated seed migrations.
- Seed data is included in the generated SQL scripts, so it is applied alongside schema changes in both development and production.

---

## Connection Management

- Connection strings are provided via the Host's `DatabaseConfiguration` options (bound from `appsettings.json`).
- EF Core manages connection pooling via the underlying ADO.NET SQL Server provider.
- The DbContext is registered as a **scoped** service in the DI container, ensuring each request/operation gets its own instance.
- No connection management for the Machine Data Database — that is handled separately by consumers (Inbound API scripts, external system gateway).

---

## Dependencies

- **Entity Framework Core**: ORM, DbContext, migrations, change tracking.
- **Microsoft.EntityFrameworkCore.SqlServer**: SQL Server database provider.
- **MS SQL Server**: The configuration database instance.
- **Commons**: POCO entity classes and repository interfaces that this component maps and implements.

## Interactions

- **Template Engine**: Uses `ITemplateEngineRepository` for all template, instance, and area data operations.
- **Deployment Manager**: Uses `IDeploymentManagerRepository` for deployment records and status tracking.
- **Security & Auth**: Uses `ISecurityRepository` for LDAP group mappings and site scoping.
- **Inbound API**: Uses `IInboundApiRepository` for API keys and method definitions.
- **External System Gateway**: Uses `IExternalSystemRepository` for external system and database connection definitions.
- **Notification Service**: Uses `INotificationRepository` for notification lists and SMTP configuration.
- **Central UI**: Uses `ICentralUiRepository` for read-oriented queries across domain areas, including audit log queries for the audit log viewer.
- **All central components that modify state**: Call `IAuditService.LogAsync()` after successful operations to record audit entries within the same transaction.
- **Host**: Provides database connection configuration. Registers DbContext, repository implementations, and `IAuditService` implementation in the DI container. Triggers auto-migration in development or validates schema version in production.