jdescopingtool/openspec/changes/archive/2026-01-01-implement-data-access/specs/data-access/spec.md

# Data Access - Implementation Patterns

## ADDED Requirements

### Requirement: Connection factory pattern

The system SHALL implement `IDbConnectionFactory` to provide database connections via dependency injection.

#### Implementation Pattern

```csharp
public class DbConnectionFactory : IDbConnectionFactory
{
    private readonly IConfiguration _configuration;
    private readonly ILogger<DbConnectionFactory> _logger;

    public DbConnectionFactory(IConfiguration configuration, ILogger<DbConnectionFactory> logger)
    {
        _configuration = configuration;
        _logger = logger;
    }

    public async Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default)
    {
        var connectionString = _configuration.GetConnectionString("LotFinderDB")
            ?? throw new ConnectionException("LotFinderDB connection string not configured", "LotFinderDB");

        var connection = new SqlConnection(connectionString);
        try
        {
            await connection.OpenAsync(ct);
            return connection;
        }
        catch (SqlException ex)
        {
            _logger.LogError(ex, "Failed to connect to LotFinderDB");
            await connection.DisposeAsync();
            throw new ConnectionException("LotFinderDB: failed to open connection to database.", "LotFinderDB", ex);
        }
    }
}
```

#### Business Rules

- Connection factory SHALL be registered as singleton
- Connections SHALL be opened asynchronously before returning
- Callers SHALL dispose returned connections when finished
- `ConnectionException` SHALL be thrown on connection failure with inner exception preserved

#### Scenario: Successful connection creation

- **WHEN** valid connection string exists in configuration
- **AND** `CreateLotFinderConnectionAsync()` is called
- **THEN** a new `SqlConnection` is created, opened, and returned
- **AND** the caller is responsible for disposal

#### Scenario: Connection failure handling

- **WHEN** database is unreachable
- **AND** `CreateLotFinderConnectionAsync()` is called
- **THEN** error is logged with exception details
- **AND** `ConnectionException` is thrown with descriptive message
- **AND** inner exception is preserved for debugging

---

### Requirement: Async streaming pattern

The system SHALL use `IAsyncEnumerable<T>` with Dapper's `QueryUnbufferedAsync` for streaming large result sets.

#### Implementation Pattern

```csharp
public async IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(
    DateTime? lastUpdateDT = null,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateJdeConnectionAsync(ct);

    var sql = ApplySchemaPlaceholders(
        lastUpdateDT.HasValue
            ? JdeQueries.SQL_GET_WORKORDERS_FILTERED
            : JdeQueries.SQL_GET_WORKORDERS);

    var command = new CommandDefinition(
        sql,
        parameters: lastUpdateDT.HasValue
            ? new { dateUpdated = ToJdeDate(lastUpdateDT.Value), timeUpdated = ToJdeTime(lastUpdateDT.Value) }
            : null,
        commandTimeout: _options.Value.DefaultTimeoutSeconds,
        cancellationToken: ct);

    await foreach (var workOrder in connection.QueryUnbufferedAsync<WorkOrder>(command).WithCancellation(ct))
    {
        yield return workOrder;
    }
}
```

#### Business Rules

- All JDE/CMS collection queries SHALL return `IAsyncEnumerable<T>`
- Queries SHALL use `QueryUnbufferedAsync` to stream results
- Methods SHALL accept `CancellationToken` with `[EnumeratorCancellation]` attribute
- Cancellation SHALL be checked between row iterations via `WithCancellation(ct)`

#### Scenario: Stream large work order dataset

- **WHEN** `GetWorkOrdersAsync()` is called for 1 million work orders
- **THEN** results are streamed via `IAsyncEnumerable<WorkOrder>` one at a time
- **AND** memory usage remains constant regardless of result set size
- **AND** consumer can use `await foreach` syntax

#### Scenario: Cancel streaming operation

- **WHEN** cancellation is requested during `GetWorkOrdersAsync()` iteration
- **THEN** iteration stops after current row completes
- **AND** `OperationCanceledException` is thrown to consumer
- **AND** database connection is properly disposed

---

### Requirement: Schema placeholder replacement

The system SHALL replace schema placeholders in SQL queries from `DataAccessOptions`.

#### Implementation Pattern

```csharp
private string ApplySchemaPlaceholders(string sql)
{
    return sql
        .Replace("{ProductionSchema}", _options.Value.ProductionSchema)
        .Replace("{ArchiveSchema}", _options.Value.ArchiveSchema)
        .Replace("{StageSchema}", _options.Value.StageSchema);
}
```

#### Business Rules

- Schema names SHALL be configurable for environment-specific deployments
- Default values: PRODDTA (production), ARCDTAPD (archive), JDESTAGE (stage)
- Replacement SHALL occur at query execution time

#### Scenario: Query uses production schema

- **WHEN** SQL query contains `{ProductionSchema}.F4801`
- **AND** `DataAccessOptions.ProductionSchema` is "PRODDTA"
- **THEN** query is executed with `PRODDTA.F4801`

---

### Requirement: Table name whitelist validation

The system SHALL validate table names against an explicit whitelist to prevent SQL injection in `RebuildIndicesAsync`.

#### Implementation Pattern

```csharp
private static readonly HashSet<string> ValidTableNames = new(StringComparer.OrdinalIgnoreCase)
{
    "Branch", "DataUpdate", "FunctionCode", "Item", "JdeUser",
    "Lot", "LotLocation", "LotUsage_Curr", "LotUsage_Hist",
    "MisData", "OrgHierarchy", "ProfitCenter", "RouteMaster",
    "Search", "StatusCode", "WorkCenter",
    "WorkOrder_Curr", "WorkOrder_Hist",
    "WorkOrderComponent_Curr", "WorkOrderComponent_Hist",
    "WorkOrderRouting",
    "WorkOrderStep_Curr", "WorkOrderStep_Hist",
    "WorkOrderTime_Curr", "WorkOrderTime_Hist"
};

public async Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)
{
    if (!ValidTableNames.Contains(tableName))
    {
        throw new ArgumentException($"Invalid table name: {tableName}", nameof(tableName));
    }

    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
    var sql = $"ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95)";
    await connection.ExecuteAsync(sql, commandTimeout: _options.Value.RebuildIndexTimeoutSeconds);
}
```

#### Business Rules

- Table name MUST be validated against explicit whitelist before execution
- `ArgumentException` SHALL be thrown for invalid table names
- Comparison SHALL be case-insensitive

#### Scenario: Valid table name accepted

- **WHEN** `RebuildIndicesAsync("WorkOrder_Curr")` is called
- **THEN** table name passes whitelist validation
- **AND** index rebuild executes successfully

#### Scenario: SQL injection attempt blocked

- **WHEN** `RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--")` is called
- **THEN** table name fails whitelist validation
- **AND** `ArgumentException` is thrown with message "Invalid table name"
- **AND** no SQL is executed

---

### Requirement: Exception logging with scope context

The system SHALL log exceptions with structured scope context before throwing.

#### Implementation Pattern

```csharp
catch (OracleException ex)
{
    using (_logger.BeginScope(new Dictionary<string, object>
    {
        ["DataSource"] = "JDE",
        ["Operation"] = nameof(GetWorkOrdersAsync),
        ["QueryName"] = "SQL_GET_WORKORDERS"
    }))
    {
        _logger.LogError(ex, "Query execution failed");
    }
    throw new QueryException("Failed to execute work order query", "SQL_GET_WORKORDERS", ex);
}
```

#### Business Rules

- All exceptions SHALL be logged at throw site
- Log context SHALL include: DataSource, Operation, and QueryName where applicable
- Inner exceptions SHALL be preserved in thrown exceptions
- Structured logging SHALL enable log aggregation and analysis

#### Scenario: Query exception with full context

- **WHEN** JDE Oracle query fails with OracleException
- **THEN** error is logged with BeginScope containing DataSource, Operation, QueryName
- **AND** `QueryException` is thrown with descriptive message and inner exception

---

### Requirement: Table-valued parameter support

The system SHALL use DataTable for SQL Server table-valued parameters in lookup methods.

#### Implementation Pattern

```csharp
public async Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);

    var table = new DataTable();
    table.Columns.Add("ItemNumber", typeof(string));
    foreach (var itemNumber in itemNumbers)
    {
        table.Rows.Add(itemNumber);
    }

    var parameters = new DynamicParameters();
    parameters.Add("@itemNumbers", table.AsTableValuedParameter("dbo.ItemNumberFilterParameter"));

    return (await connection.QueryAsync<Item>(
        LotFinderQueries.SQL_LOOKUP_ITEMS,
        parameters,
        commandTimeout: _options.Value.DefaultTimeoutSeconds))
        .AsList();
}
```

#### Business Rules

- TVPs SHALL use `AsTableValuedParameter` extension with correct type name
- DataTable column names SHALL match TVP type column names
- TVPs enable efficient batch lookups with single database round-trip

#### Scenario: Batch lookup with TVP

- **WHEN** `LookupItemsAsync(["ITEM001", "ITEM002", "ITEM003"])` is called
- **THEN** DataTable is created with ItemNumber column
- **AND** single query executes with TVP parameter
- **AND** matching items are returned

---

### Requirement: Service registration extension method

The system SHALL provide `AddDataAccess` extension method for DI registration.

#### Implementation Pattern

```csharp
public static IServiceCollection AddDataAccess(
    this IServiceCollection services,
    IConfiguration configuration)
{
    services.Configure<DataAccessOptions>(
        configuration.GetSection(DataAccessOptions.SectionName));

    services.AddSingleton<IDbConnectionFactory, DbConnectionFactory>();
    services.AddScoped<ILotFinderRepository, LotFinderRepository>();
    services.AddScoped<IJdeRepository, JdeRepository>();
    services.AddScoped<ICmsRepository, CmsRepository>();

    return services;
}
```

#### Business Rules

- Extension method SHALL bind `DataAccessOptions` from "DataAccess" configuration section
- Connection factory SHALL be registered as singleton
- Repositories SHALL be registered as scoped services
- Method SHALL return `IServiceCollection` for chaining

#### Scenario: Register all data access services

- **WHEN** `services.AddDataAccess(configuration)` is called during startup
- **THEN** `DataAccessOptions` is bound from configuration
- **AND** `IDbConnectionFactory` is registered as singleton
- **AND** all repository interfaces are registered as scoped