jdescopingtool/openspec/changes/archive/2026-01-01-implement-data-access/specs/data-access/spec.md

Data Access - Implementation Patterns

ADDED Requirements

Requirement: Connection factory pattern

The system SHALL implement IDbConnectionFactory to provide database connections via dependency injection.

Implementation Pattern

public class DbConnectionFactory : IDbConnectionFactory
{
    private readonly IConfiguration _configuration;
    private readonly ILogger<DbConnectionFactory> _logger;

    public DbConnectionFactory(IConfiguration configuration, ILogger<DbConnectionFactory> logger)
    {
        _configuration = configuration;
        _logger = logger;
    }

    public async Task<SqlConnection> CreateLotFinderConnectionAsync(CancellationToken ct = default)
    {
        var connectionString = _configuration.GetConnectionString("LotFinderDB")
            ?? throw new ConnectionException("LotFinderDB connection string not configured", "LotFinderDB");

        var connection = new SqlConnection(connectionString);
        try
        {
            await connection.OpenAsync(ct);
            return connection;
        }
        catch (SqlException ex)
        {
            _logger.LogError(ex, "Failed to connect to LotFinderDB");
            await connection.DisposeAsync();
            throw new ConnectionException("LotFinderDB: failed to open connection to database.", "LotFinderDB", ex);
        }
    }
}

Business Rules

• Connection factory SHALL be registered as singleton
• Connections SHALL be opened asynchronously before returning
• Callers SHALL dispose returned connections when finished
• ConnectionException SHALL be thrown on connection failure with inner exception preserved

Scenario: Successful connection creation

• WHEN valid connection string exists in configuration
• AND CreateLotFinderConnectionAsync() is called
• THEN a new SqlConnection is created, opened, and returned
• AND the caller is responsible for disposal

Scenario: Connection failure handling

• WHEN database is unreachable
• AND CreateLotFinderConnectionAsync() is called
• THEN error is logged with exception details
• AND ConnectionException is thrown with descriptive message
• AND inner exception is preserved for debugging

---

Requirement: Async streaming pattern

The system SHALL use IAsyncEnumerable<T> with Dapper's QueryUnbufferedAsync for streaming large result sets.

Implementation Pattern

public async IAsyncEnumerable<WorkOrder> GetWorkOrdersAsync(
    DateTime? lastUpdateDT = null,
    [EnumeratorCancellation] CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateJdeConnectionAsync(ct);

    var sql = ApplySchemaPlaceholders(
        lastUpdateDT.HasValue
            ? JdeQueries.SQL_GET_WORKORDERS_FILTERED
            : JdeQueries.SQL_GET_WORKORDERS);

    var command = new CommandDefinition(
        sql,
        parameters: lastUpdateDT.HasValue
            ? new { dateUpdated = ToJdeDate(lastUpdateDT.Value), timeUpdated = ToJdeTime(lastUpdateDT.Value) }
            : null,
        commandTimeout: _options.Value.DefaultTimeoutSeconds,
        cancellationToken: ct);

    await foreach (var workOrder in connection.QueryUnbufferedAsync<WorkOrder>(command).WithCancellation(ct))
    {
        yield return workOrder;
    }
}

Business Rules

• All JDE/CMS collection queries SHALL return IAsyncEnumerable<T>
• Queries SHALL use QueryUnbufferedAsync to stream results
• Methods SHALL accept CancellationToken with [EnumeratorCancellation] attribute
• Cancellation SHALL be checked between row iterations via WithCancellation(ct)

Scenario: Stream large work order dataset

• WHEN GetWorkOrdersAsync() is called for 1 million work orders
• THEN results are streamed via IAsyncEnumerable<WorkOrder> one at a time
• AND memory usage remains constant regardless of result set size
• AND consumer can use await foreach syntax

Scenario: Cancel streaming operation

• WHEN cancellation is requested during GetWorkOrdersAsync() iteration
• THEN iteration stops after current row completes
• AND OperationCanceledException is thrown to consumer
• AND database connection is properly disposed

---

Requirement: Schema placeholder replacement

The system SHALL replace schema placeholders in SQL queries from DataAccessOptions.

Implementation Pattern

private string ApplySchemaPlaceholders(string sql)
{
    return sql
        .Replace("{ProductionSchema}", _options.Value.ProductionSchema)
        .Replace("{ArchiveSchema}", _options.Value.ArchiveSchema)
        .Replace("{StageSchema}", _options.Value.StageSchema);
}

Business Rules

• Schema names SHALL be configurable for environment-specific deployments
• Default values: PRODDTA (production), ARCDTAPD (archive), JDESTAGE (stage)
• Replacement SHALL occur at query execution time

Scenario: Query uses production schema

• WHEN SQL query contains {ProductionSchema}.F4801
• AND DataAccessOptions.ProductionSchema is "PRODDTA"
• THEN query is executed with PRODDTA.F4801

---

Requirement: Table name whitelist validation

The system SHALL validate table names against an explicit whitelist to prevent SQL injection in RebuildIndicesAsync.

Implementation Pattern

private static readonly HashSet<string> ValidTableNames = new(StringComparer.OrdinalIgnoreCase)
{
    "Branch", "DataUpdate", "FunctionCode", "Item", "JdeUser",
    "Lot", "LotLocation", "LotUsage_Curr", "LotUsage_Hist",
    "MisData", "OrgHierarchy", "ProfitCenter", "RouteMaster",
    "Search", "StatusCode", "WorkCenter",
    "WorkOrder_Curr", "WorkOrder_Hist",
    "WorkOrderComponent_Curr", "WorkOrderComponent_Hist",
    "WorkOrderRouting",
    "WorkOrderStep_Curr", "WorkOrderStep_Hist",
    "WorkOrderTime_Curr", "WorkOrderTime_Hist"
};

public async Task RebuildIndicesAsync(string tableName, CancellationToken ct = default)
{
    if (!ValidTableNames.Contains(tableName))
    {
        throw new ArgumentException($"Invalid table name: {tableName}", nameof(tableName));
    }

    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);
    var sql = $"ALTER INDEX ALL ON [{tableName}] REBUILD WITH (FILLFACTOR = 95)";
    await connection.ExecuteAsync(sql, commandTimeout: _options.Value.RebuildIndexTimeoutSeconds);
}

Business Rules

• Table name MUST be validated against explicit whitelist before execution
• ArgumentException SHALL be thrown for invalid table names
• Comparison SHALL be case-insensitive

Scenario: Valid table name accepted

• WHEN RebuildIndicesAsync("WorkOrder_Curr") is called
• THEN table name passes whitelist validation
• AND index rebuild executes successfully

Scenario: SQL injection attempt blocked

• WHEN RebuildIndicesAsync("WorkOrder]; DROP TABLE Search;--") is called
• THEN table name fails whitelist validation
• AND ArgumentException is thrown with message "Invalid table name"
• AND no SQL is executed

---

Requirement: Exception logging with scope context

The system SHALL log exceptions with structured scope context before throwing.

Implementation Pattern

catch (OracleException ex)
{
    using (_logger.BeginScope(new Dictionary<string, object>
    {
        ["DataSource"] = "JDE",
        ["Operation"] = nameof(GetWorkOrdersAsync),
        ["QueryName"] = "SQL_GET_WORKORDERS"
    }))
    {
        _logger.LogError(ex, "Query execution failed");
    }
    throw new QueryException("Failed to execute work order query", "SQL_GET_WORKORDERS", ex);
}

Business Rules

• All exceptions SHALL be logged at throw site
• Log context SHALL include: DataSource, Operation, and QueryName where applicable
• Inner exceptions SHALL be preserved in thrown exceptions
• Structured logging SHALL enable log aggregation and analysis

Scenario: Query exception with full context

• WHEN JDE Oracle query fails with OracleException
• THEN error is logged with BeginScope containing DataSource, Operation, QueryName
• AND QueryException is thrown with descriptive message and inner exception

---

Requirement: Table-valued parameter support

The system SHALL use DataTable for SQL Server table-valued parameters in lookup methods.

Implementation Pattern

public async Task<List<Item>> LookupItemsAsync(List<string> itemNumbers, CancellationToken ct = default)
{
    await using var connection = await _connectionFactory.CreateLotFinderConnectionAsync(ct);

    var table = new DataTable();
    table.Columns.Add("ItemNumber", typeof(string));
    foreach (var itemNumber in itemNumbers)
    {
        table.Rows.Add(itemNumber);
    }

    var parameters = new DynamicParameters();
    parameters.Add("@itemNumbers", table.AsTableValuedParameter("dbo.ItemNumberFilterParameter"));

    return (await connection.QueryAsync<Item>(
        LotFinderQueries.SQL_LOOKUP_ITEMS,
        parameters,
        commandTimeout: _options.Value.DefaultTimeoutSeconds))
        .AsList();
}

Business Rules

• TVPs SHALL use AsTableValuedParameter extension with correct type name
• DataTable column names SHALL match TVP type column names
• TVPs enable efficient batch lookups with single database round-trip

Scenario: Batch lookup with TVP

• WHEN LookupItemsAsync(["ITEM001", "ITEM002", "ITEM003"]) is called
• THEN DataTable is created with ItemNumber column
• AND single query executes with TVP parameter
• AND matching items are returned

---

Requirement: Service registration extension method

The system SHALL provide AddDataAccess extension method for DI registration.

Implementation Pattern

public static IServiceCollection AddDataAccess(
    this IServiceCollection services,
    IConfiguration configuration)
{
    services.Configure<DataAccessOptions>(
        configuration.GetSection(DataAccessOptions.SectionName));

    services.AddSingleton<IDbConnectionFactory, DbConnectionFactory>();
    services.AddScoped<ILotFinderRepository, LotFinderRepository>();
    services.AddScoped<IJdeRepository, JdeRepository>();
    services.AddScoped<ICmsRepository, CmsRepository>();

    return services;
}

Business Rules

• Extension method SHALL bind DataAccessOptions from "DataAccess" configuration section
• Connection factory SHALL be registered as singleton
• Repositories SHALL be registered as scoped services
• Method SHALL return IServiceCollection for chaining

Scenario: Register all data access services

• WHEN services.AddDataAccess(configuration) is called during startup
• THEN DataAccessOptions is bound from configuration
• AND IDbConnectionFactory is registered as singleton
• AND all repository interfaces are registered as scoped